Lo-Dash Extensions

Extending Lo-Dash for fun and profit

If you’ve written any substantial ammount of JavaScript in the last year, chances are that you are using either Underscore.js or Lo-Dash. For us it’s been Lo-Dash.

One of the nicest things about Lo-Dash is that it is very easy to extend. Simply call:

_.mixin({
  myFunction: function() {
    // do something cool
  }
});

And voilá, you can now call _.myFunction anywhere in your code. What’s more, if your function takes a collection as its first argument and returns a collection, you get free chaining support (_.([1,2,3]).map(double).myFunction().value()).

If you do this you should probably call var _ = _.runInContext();, as this will ensure that you do not introduce incompatibilities with libraries that also depend on Lo-Dash.

With such convenience though comes a certain responsibility, as these functions become essentially global to your application. In our team the general rule is that these functions must be generally applicable (that is they solve a class of problems, rather than just a particular one) and must be well tested. In this blog post I would like to share some of our functions (feel free to copy paste them wherever you want) with some usecases. As a bonus I’ll also demonstrate how you can test them using the techniques I’ve shown in my post on quick_check.js.

Fill

JavaScript has many warts, but one of the more anoying ones is that the standard library implements some methods as mutating and some not. So one the one hand we have reverse, sort and splice which change the array inline. On the other we have concat and slice.

But sometimes we want the other behavior. In this example we implement an inline version of concat.

fill: function(destination, source) {
  _.each(source, function(value) {
    destination.push(value);
  });

  return destination;
},

Notice the return destination; at the end? This also allows this method to be used in chains. Now let’s test this. First let’s write a conventional test, just to make sure everything is OK:

it('fills the destination array', function () {
  var destination = [1];
  var source = [2, 3];
  var returnValue = _.fill(destination, source);
  expect(source).toEqual([2, 3]);
  expect(destination).toEqual([1, 2, 3]);
  expect(returnValue).toBe(destination);
});

So far this is pretty ordinary. But now we want to make sure this works for all arrays:

it('works the same as concat', function() {
  expect(function(destination, source) {
    var expected = destination.concat(source);
    _.fill(destination, source);
    return _.isEqual(destination, expected);
  }).forAll(qc.array, qc.array);
});

This is quite a common quick_check strategy. We can exploit the fact that one function behaves identically to another. Then we can assert that the relationship holds for every possible input (in this case a pair of arrays).

Batch

Sometimes you need to divide a problem into smaller chunks. Perhaps there is a limit to how many items an endpoint can process. For that we want a function that takes a list and a limit and returns a list of lists, where each of the child lists is shorter than the limit.

batch: function(items, batchSize) {
  var batched = [];

  while (items.length) {
    batched.push(items.splice(0, batchSize));
  }

  return batched;
},

And we proceed to write the test, verifying that the batch size is never smaller than the limit:

it('chunks are never greater than the limit', function () {
  expect(function(list, limit) {
    var batched = _.batch(list, limit);
    return _.all(batched, function(batch) {
      return batch.length <= limit;
    });
  }).forAll(qc.array, qc.real);
});

Right away when we run this test we know we have discovered a bug. quick_check doesn’t tell us, but this code will hang. We have an infinite loop. A bit of further investigation shows that the problem is if the limit is <= 0. An error in that condition would be fine, but we don’t want our code to hang accidentally. So we add a condition that adds an error and adjust our tests:

batch: function(items, batchSize) {
  if (batchSize <= 0) {
    throw new Exception('batch must be > 0 in _.batch');
  }
  var batched = [];

  while (items.length) {
    batched.push(items.splice(0, Math.ceil(batchSize)));
  }

  return batched;
}
it('chunks are never greater than the limit', function () {
  expect(function(list, limit) {
    if (limit > 0) {
      var batched = _.batch(list, limit);
      return _.all(batched, function(batch) {
        return batch.length <= limit;
      });
    }
  }).forAll(qc.array, qc.ureal.large);
});

Log

console.log is a blessing when debugging. It has three pitfalls:

  1. It's quite a lot to type. _.log is a lot shorter than console.log.

  2. I often want to inspect the return value of a function. Which means I have to transform

    function (a) {
      return doSomething(a);
    }

    Into:

    function(a) {
      var result = doSomething(a);
      console.log(result);
      return result;
    }

    With this helper I need to only do:

    function(a) {
      return _.log(doSomething(a));
    }
  3. I'm writing a complex processing pipeline using Lo-Dash chaining and I want to inspect the results in the middle of the chain.

    return _(data).map(process).filter(isCorrect).sortBy('sort').value();

    has to be changed to

    var processed =  _(data).map(process);
    console.log(processed.value());
    var onlyCorrect = processed.filter(isCorrect);
    console.log(processed.value());
    var sorted = onlyCorrect.sortBy('sort');
    console.log(sorted.value());
    return sorted.value();

    whereas using _.log we get:

    return _(data).map(process).log().filter(isCorrect).log().sortBy('sort').log().value();

Here’s the definition:

log: function (returnValue) {
  var console = window.console;
  console.log.apply(console, arguments);
  return returnValue;
},

I won’t bore you with the specs on this one, as they are very straightforward.

Not

This again is an example of a place where JavaScript has a lot of potential for functional programming, but is missing out because of missing library support. One such area is that the standard operators aren’t curried functions. One place where I find this especially annoying is with the ! operator. A common situation is when I write a boolean function that should match a particular type of data. Then, later, I want to use filter to get everything that doesn’t match that function. To do that I need to write:

_.filter(data, function(item) {
  return !isType(item);
});

With this method we can simplify it to a one liner _.filter(data, _.not(isType)).

not: function() {
  var cb = _.createCallback.apply(_, arguments);
  return function() {
    return !cb.apply(null, arguments);
  };
},

Note the _.createCallback. This is a method Lo-Dash exposes to create its fancy callbacks using strings for accessors and objects for conditions. This suits us well, since we can use _.not('property') to check for objects that have a falsey property (or undefined) and _.not({property: value}) which checks that an object doesn’t have property with value. This makes it more powerful than the version from Lo-Dash Contrib :)

Here’s the test:

describe('_.not', function () {
  it('negates itself', function() {
    expect(function(fn, value) {
      var doubleNegated = _.not(_.not(fn));
      return fn(value) === doubleNegated(value);
    }).forAll(qc.function(qc.bool), qc.any);
  });
});

Notice the use of a function generator to test over all boolean functions - the fn argument will be a new function every time.

You could use this as an example for creating functional operators yourself, when needed.

Clamp

Often we want a value, but only as long as it is in an acceptable range. If it overshoots or undershoots we want to take the highest or lowest acceptable value respectively. This can be done fairly easily. But since this is a shared function, we want it to be as robust and safe as possible:

clamp: function(value, minimum, maximum) {
  if (maximum == null) {
    maximum = Number.MAX_VALUE;
  }
  if (maximum < minimum) {
    var swap = maximum;
    maximum = minimum;
    minimum = swap;
  }
  return Math.max(minimum, Math.min(value, maximum));
}

Whoa! that’s a lot of code for something so simple. But in this case we wanted our function to make the maximum value optional and also not fall appart if the minimum happens to be larger than the maximum.

describe('_.clamp', function() {
  it('clamps values', function() {
    expect(function(a, b, c) {
      var value = _.clamp(a, b, c);
      return value >= Math.min(b, c) && value <= Math.max(b, c);
    }).forAll(qc.real, qc.real, qc.real);
  });
  it('doesn\'t clamp from the top if c isn\'t passed', function() {
    expect(function(a, b) {
      return _.clamp(a, b) === a || _.clamp(a, b) === b;
    }).forAll(qc.real, qc.real);
  });
});

Wrapping up

We have a few more functions, but this should get you started on writing your own. And if you’ve written a cool utility function, share it in the comments :) BTW, we’re looking for a front-end engineer to join our team if this is something that interests you.

Here’s the full source, docs and specs:

Update: Lo-Dash v3.0.0 is being released today. Among some amazing features like pipelining, it also delivers ‘native’ versions of some of the functions presented above. So instead of _.batch, you can use the native _.chunk and _.not can be replaced with _.negate, although it doesn’t support the string and object style callbacks.