elasticlunr

Pipeline

constructor
elasticlunr.Pipeline()

elasticlunr.Pipelines maintain an ordered list of functions to be applied to
both documents tokens and query tokens.

An instance of elasticlunr.Index will contain a pipeline
with a trimmer, a stop word filter, an English stemmer. Extra
functions can be added before or after either of these functions or these
default functions can be removed.

When run the pipeline, it will call each function in turn.

The output of the functions in the pipeline will be passed to the next function
in the pipeline. To exclude a token from entering the index the function
should return undefined, the rest of the pipeline will not be called with
this token.

For serialisation of pipelines to work, all functions used in an instance of
a pipeline should be registered with elasticlunr.Pipeline. Registered functions can
then be loaded. If trying to load a serialised pipeline that uses functions
that are not registered an error will be thrown.

If not planning on serialising the pipeline then registering pipeline functions
is not necessary.

elasticlunr.Pipeline = function () {
  this._queue = [];
};

elasticlunr.Pipeline.registeredFunctions = {};

registerFunction

method
elasticlunr.Pipeline.registerFunction()

Option name Type Description
fn Function

The function to register.

label String

The label to register this function with

Register a function in the pipeline.

Functions that are used in the pipeline should be registered if the pipeline
needs to be serialised, or a serialised pipeline needs to be loaded.

Registering a function does not add it to a pipeline, functions must still be
added to instances of the pipeline for them to be used when running a pipeline.

elasticlunr.Pipeline.registerFunction = function (fn, label) {
  if (label in elasticlunr.Pipeline.registeredFunctions) {
    elasticlunr.utils.warn('Overwriting existing registered function: ' + label);
  }

  fn.label = label;
  elasticlunr.Pipeline.registeredFunctions[label] = fn;
};

getRegisteredFunction

method
elasticlunr.Pipeline.getRegisteredFunction()

Option name Type Description
label String

The label of registered function.

return Function

Get a registered function in the pipeline.

elasticlunr.Pipeline.getRegisteredFunction = function (label) {
  if ((label in elasticlunr.Pipeline.registeredFunctions) !== true) {
    return null;
  }

  return elasticlunr.Pipeline.registeredFunctions[label];
};

load

method
elasticlunr.Pipeline.load()

Option name Type Description
serialised Object

The serialised pipeline to load.

return elasticlunr.Pipeline

Loads a previously serialised pipeline.

All functions to be loaded must already be registered with elasticlunr.Pipeline.
If any function from the serialised data has not been registered then an
error will be thrown.

elasticlunr.Pipeline.load = function (serialised) {
  var pipeline = new elasticlunr.Pipeline;

  serialised.forEach(function (fnName) {
    var fn = elasticlunr.Pipeline.getRegisteredFunction(fnName);

    if (fn) {
      pipeline.add(fn);
    } else {
      throw new Error('Cannot load un-registered function: ' + fnName);
    }
  });

  return pipeline;
};

add

method
elasticlunr.Pipeline.prototype.add()

Option name Type Description
functions Function

Any number of functions to add to the pipeline.

Adds new functions to the end of the pipeline.

Logs a warning if the function has not been registered.

elasticlunr.Pipeline.prototype.add = function () {
  var fns = Array.prototype.slice.call(arguments);

  fns.forEach(function (fn) {
    elasticlunr.Pipeline.warnIfFunctionNotRegistered(fn);
    this._queue.push(fn);
  }, this);
};

after

method
elasticlunr.Pipeline.prototype.after()

Option name Type Description
existingFn Function

A function that already exists in the pipeline.

newFn Function

The new function to add to the pipeline.

Adds a single function after a function that already exists in the
pipeline.

Logs a warning if the function has not been registered.
If existingFn is not found, throw an Exception.

elasticlunr.Pipeline.prototype.after = function (existingFn, newFn) {
  elasticlunr.Pipeline.warnIfFunctionNotRegistered(newFn);

  var pos = this._queue.indexOf(existingFn);
  if (pos === -1) {
    throw new Error('Cannot find existingFn');
  }

  this._queue.splice(pos + 1, 0, newFn);
};

before

method
elasticlunr.Pipeline.prototype.before()

Option name Type Description
existingFn Function

A function that already exists in the pipeline.

newFn Function

The new function to add to the pipeline.

Adds a single function before a function that already exists in the
pipeline.

Logs a warning if the function has not been registered.
If existingFn is not found, throw an Exception.

elasticlunr.Pipeline.prototype.before = function (existingFn, newFn) {
  elasticlunr.Pipeline.warnIfFunctionNotRegistered(newFn);

  var pos = this._queue.indexOf(existingFn);
  if (pos === -1) {
    throw new Error('Cannot find existingFn');
  }

  this._queue.splice(pos, 0, newFn);
};

remove

method
elasticlunr.Pipeline.prototype.remove()

Option name Type Description
fn Function

The function to remove from the pipeline.

Removes a function from the pipeline.

elasticlunr.Pipeline.prototype.remove = function (fn) {
  var pos = this._queue.indexOf(fn);
  if (pos === -1) {
    return;
  }

  this._queue.splice(pos, 1);
};

run

method
elasticlunr.Pipeline.prototype.run()

Option name Type Description
tokens Array

The tokens to run through the pipeline.

return Array

Runs the current list of functions that registered in the pipeline against the
input tokens.

elasticlunr.Pipeline.prototype.run = function (tokens) {
  var out = [],
      tokenLength = tokens.length,
      pipelineLength = this._queue.length;

  for (var i = 0; i < tokenLength; i++) {
    var token = tokens[i];

    for (var j = 0; j < pipelineLength; j++) {
      token = this._queue[j](token, i, tokens);
      if (token === void 0 || token === null) break;
    };

    if (token !== void 0 && token !== null) out.push(token);
  };

  return out;
};

reset

method
elasticlunr.Pipeline.prototype.reset()

Resets the pipeline by removing any existing processors.

elasticlunr.Pipeline.prototype.reset = function () {
  this._queue = [];
};

get

method
elasticlunr.Pipeline.prototype.get()

Get the pipeline if user want to check the pipeline.

elasticlunr.Pipeline.prototype.get = function () {
  return this._queue;
};

toJSON

method
elasticlunr.Pipeline.prototype.toJSON()

Returns a representation of the pipeline ready for serialisation.
Only serialize pipeline function's name. Not storing function, so when
loading the archived JSON index file, corresponding pipeline function is
added by registered function of elasticlunr.Pipeline.registeredFunctions

Logs a warning if the function has not been registered.

elasticlunr.Pipeline.prototype.toJSON = function () {
  return this._queue.map(function (fn) {
    elasticlunr.Pipeline.warnIfFunctionNotRegistered(fn);
    return fn.label;
  });
};