Pipeline

constructor

elasticlunr.Pipeline()

elasticlunr.Pipelines maintain an ordered list of functions to be applied to
both documents tokens and query tokens.

An instance of elasticlunr.Index will contain a pipeline
with a trimmer, a stop word filter, an English stemmer. Extra
functions can be added before or after either of these functions or these
default functions can be removed.

When run the pipeline, it will call each function in turn.

The output of the functions in the pipeline will be passed to the next function
in the pipeline. To exclude a token from entering the index the function
should return undefined, the rest of the pipeline will not be called with
this token.

For serialisation of pipelines to work, all functions used in an instance of
a pipeline should be registered with elasticlunr.Pipeline. Registered functions can
then be loaded. If trying to load a serialised pipeline that uses functions
that are not registered an error will be thrown.

If not planning on serialising the pipeline then registering pipeline functions
is not necessary.

elasticlunr.Pipeline = function () {
  this._queue = [];
};

elasticlunr.Pipeline.registeredFunctions = {};

registerFunction

method

elasticlunr.Pipeline.registerFunction()

Option name	Type	Description
fn	Function	The function to register.
label	String	The label to register this function with

Functions that are used in the pipeline should be registered if the pipeline
needs to be serialised, or a serialised pipeline needs to be loaded.

Registering a function does not add it to a pipeline, functions must still be
added to instances of the pipeline for them to be used when running a pipeline.

elasticlunr.Pipeline.registerFunction = function (fn, label) {
  if (label in elasticlunr.Pipeline.registeredFunctions) {
    elasticlunr.utils.warn('Overwriting existing registered function: ' + label);
  }

  fn.label = label;
  elasticlunr.Pipeline.registeredFunctions[label] = fn;
};

getRegisteredFunction

method

elasticlunr.Pipeline.getRegisteredFunction()

Option name	Type	Description
label	String	The label of registered function.
return	Function

Get a registered function in the pipeline.

elasticlunr.Pipeline.getRegisteredFunction = function (label) {
  if ((label in elasticlunr.Pipeline.registeredFunctions) !== true) {
    return null;
  }

  return elasticlunr.Pipeline.registeredFunctions[label];
};

load

method

elasticlunr.Pipeline.load()

Option name	Type	Description
serialised	Object	The serialised pipeline to load.
return	elasticlunr.Pipeline

Loads a previously serialised pipeline.

All functions to be loaded must already be registered with elasticlunr.Pipeline.
If any function from the serialised data has not been registered then an
error will be thrown.

elasticlunr.Pipeline.load = function (serialised) {
  var pipeline = new elasticlunr.Pipeline;

  serialised.forEach(function (fnName) {
    var fn = elasticlunr.Pipeline.getRegisteredFunction(fnName);

    if (fn) {
      pipeline.add(fn);
    } else {
      throw new Error('Cannot load un-registered function: ' + fnName);
    }
  });

  return pipeline;
};

add

method

elasticlunr.Pipeline.prototype.add()

Option name	Type	Description
functions	Function	Any number of functions to add to the pipeline.

Adds new functions to the end of the pipeline.

Logs a warning if the function has not been registered.

elasticlunr.Pipeline.prototype.add = function () {
  var fns = Array.prototype.slice.call(arguments);

  fns.forEach(function (fn) {
    elasticlunr.Pipeline.warnIfFunctionNotRegistered(fn);
    this._queue.push(fn);
  }, this);
};

after

method

elasticlunr.Pipeline.prototype.after()

Option name	Type	Description
existingFn	Function	A function that already exists in the pipeline.
newFn	Function	The new function to add to the pipeline.

Adds a single function after a function that already exists in the
pipeline.

Logs a warning if the function has not been registered.
If existingFn is not found, throw an Exception.

elasticlunr.Pipeline.prototype.after = function (existingFn, newFn) {
  elasticlunr.Pipeline.warnIfFunctionNotRegistered(newFn);

  var pos = this._queue.indexOf(existingFn);
  if (pos === -1) {
    throw new Error('Cannot find existingFn');
  }

  this._queue.splice(pos + 1, 0, newFn);
};

before

method

elasticlunr.Pipeline.prototype.before()

Option name	Type	Description
existingFn	Function	A function that already exists in the pipeline.
newFn	Function	The new function to add to the pipeline.

Adds a single function before a function that already exists in the
pipeline.

Logs a warning if the function has not been registered.
If existingFn is not found, throw an Exception.

elasticlunr.Pipeline.prototype.before = function (existingFn, newFn) {
  elasticlunr.Pipeline.warnIfFunctionNotRegistered(newFn);

  var pos = this._queue.indexOf(existingFn);
  if (pos === -1) {
    throw new Error('Cannot find existingFn');
  }

  this._queue.splice(pos, 0, newFn);
};

remove

method

elasticlunr.Pipeline.prototype.remove()

Option name	Type	Description
fn	Function	The function to remove from the pipeline.

Removes a function from the pipeline.

elasticlunr.Pipeline.prototype.remove = function (fn) {
  var pos = this._queue.indexOf(fn);
  if (pos === -1) {
    return;
  }

  this._queue.splice(pos, 1);
};

run

method

elasticlunr.Pipeline.prototype.run()

Option name	Type	Description
tokens	Array	The tokens to run through the pipeline.
return	Array

Runs the current list of functions that registered in the pipeline against the
input tokens.

elasticlunr.Pipeline.prototype.run = function (tokens) {
  var out = [],
      tokenLength = tokens.length,
      pipelineLength = this._queue.length;

  for (var i = 0; i < tokenLength; i++) {
    var token = tokens[i];

    for (var j = 0; j < pipelineLength; j++) {
      token = this._queue[j](token, i, tokens);
      if (token === void 0 || token === null) break;
    };

    if (token !== void 0 && token !== null) out.push(token);
  };

  return out;
};

reset

method

elasticlunr.Pipeline.prototype.reset()

Resets the pipeline by removing any existing processors.

elasticlunr.Pipeline.prototype.reset = function () {
  this._queue = [];
};

get

method

elasticlunr.Pipeline.prototype.get()

Get the pipeline if user want to check the pipeline.

elasticlunr.Pipeline.prototype.get = function () {
  return this._queue;
};

toJSON

method

elasticlunr.Pipeline.prototype.toJSON()

Returns a representation of the pipeline ready for serialisation.
Only serialize pipeline function's name. Not storing function, so when
loading the archived JSON index file, corresponding pipeline function is
added by registered function of elasticlunr.Pipeline.registeredFunctions

Logs a warning if the function has not been registered.

elasticlunr.Pipeline.prototype.toJSON = function () {
  return this._queue.map(function (fn) {
    elasticlunr.Pipeline.warnIfFunctionNotRegistered(fn);
    return fn.label;
  });
};