Seeding a Sails.js Application's MongoDB Store Standalone Using Node.js and Waterline

As I was constructing the Sails.js application for the forthcoming Part 2 of my 3-part MEAN-stack series, I realized that I needed an easy way to seed the MongoDB instance on which my Sails app would run.  In this particular example, I wanted to seed three collections (Movie, Person, and MoviePerson) in MongoDB with data defined in .json files that matched the Models for these collections I'd created in Sails.

So, how do we go about doing this in a standalone fashion using Node.js and Sails' excellent Waterline ORM?

[NOTE: The approach below is confirmed to work in Sails v0.9.16, but is not functional for the forthcoming v0.10.  I will update with a post on that as soon as I've got a viable solution]

Reusing My Sails Configuration

One thing I wanted to do was to reuse the Models I had already created in Sails, as well as leverage the /config/adapters I'd already set.  Furthermore, I didn't want to have to rely on the full Sails stack to accomplish this: I wanted a simple Node.js script, using Waterline standalone, to perform this task.  Finally, I wanted this done in a "promise-y" way -- sequentially executed seeds.

First things first. Let's assume we have three JSON's: Movie.json, MoviePerson.json, and Person.json, all corresponding to collections/Sails models of the same name. Let's say they live on the root of your Sails app, on a /seeds directory. Let's create a seed.js file, intended to be a standalone .js file we can run using Node.js to seed our Mongo instance using Waterline.

We need to bring in Waterline, /config/adapters, the sails-mongo adapter, the Promise library, and lodash for some helper functions. Note that these libraries were pulled down to my Sails project using npm (i.e. sudo npm install lodash waterline sails-mongo promise --save )

var _ = require('lodash'),
    Waterline = require('waterline'),
    adaptersConfig = require('../config/adapters'),
    mongoAdapter = require('sails-mongo'),
    Promise = require('promise');

Now that we've got the prerequisite libraries, let's configure the Mongo adapter to reuse /config/adapters.js from my Sails project. This requires us to define all the properties needed by the sails-mongo library.

// Match our config
var mongoUrl = 'mongodb://' +
    adaptersConfig.adapters.mongo.user + ':' + adaptersConfig.adapters.mongo.password + '@' +
    adaptersConfig.adapters.mongo.host + ':' + adaptersConfig.adapters.mongo.port + '/' +
    adaptersConfig.adapters.mongo.database;

mongoAdapter.database = adaptersConfig.adapters.mongo.database;
mongoAdapter.host = adaptersConfig.adapters.mongo.host;
mongoAdapter.port = adaptersConfig.adapters.mongo.port;
mongoAdapter.user = adaptersConfig.adapters.mongo.user;
mongoAdapter.password = adaptersConfig.adapters.mongo.password;
mongoAdapter.config = { url: mongoUrl };

The mongoAdapter.config bit was an interesting find. Originally, I did not define it, and sails-mongo completely vomited. In tracing down the exception, it appears that sails-mongo (as of v0.9.8) requires that this be defined.

Reusing My Sails Models and Literally Defined JSON's

Now that we have our /config/adapters.js doing the work for us, what of the Models in Sails? Let's first define how we're going to seed the Movie collection. We will encapsulate this in a function, since the ultimate goal is to make this process "promise-y".

/**
 * Seeds movies in a promise-y way
 */
var seedMovie = function() {
    var MovieModel = require('../api/models/Movie'),
        MovieData = require('./Movie.json');

    return new Promise(
        function(fulfill, reject) {
            // Seed Movie Data from our JSON
            MovieModel.adapter = 'mongo';
            var Movie = Waterline.Collection.extend(MovieModel);

            // Invoke Waterline
            new Movie({ adapters: { mongo: mongoAdapter }}, function(err, collection) {
                if (err){
                    console.log('There was a problem initializing the Movie collection.');
                }
                else {
                    // First nuke all existing movies
                    collection.destroy().done(function(err, deletedMovies) {
                        console.log('Successfully deleted Movies!');
                        collection.createEach(MovieData, function(err, models) {
                            if (err || _.isUndefined(models)) {
                                console.log('Could not create Movie collection!');
                                fulfill();
                            }
                            else {
                                console.log('Successfully inserted Movies');
                                collection.find().done(function(err, existingMovies) {
                                    fulfill(existingMovies);
                                });
                            }
                        });
                    });
                }
            });
        }
    );
};

As you can see, by using the magic of Require, we are able to reuse our Movie model and the literal JSON of Movie documents we want in the Mongo collection. The only curveball is that we have to literally define the adapter for the Movie model before using Waterline to extend and use it.

From that point forward, it's out of the box Waterline. As you can see, we first destroy the collection to remove any documents in the collection, then invoke a createEach using our Movie.json, which is a literally defined array of Movie objects. The entire operation is encapsulated in a Promise that is the return type of this function.

Fulfill Our Promises!

Last and certainly not the least, how do we do this sequentially using the magic of Promises? Assuming we'd already created three functions in seed.js: seedMovie, seedPerson, and seedMoviePerson, all returning promises, we'd just chain them together like so:

// Node.js: fulfilling more promises than all politicians combined in history
seedMovie()
    .then(function(movies) {
        seedPerson().then(function(persons) {
            seedMoviePerson().then(function(moviePersons){
                console.log('Seeding process complete!');
                process.exit();
            });
        });
    });

Wrap-up

Once we are at this point, it's elementary -- simply run node seed.js and watch the magic happen. Check out the solution on GitHub if the step-by-step above didn't quite work out for you.

Hopefully, this post is useful for you in your Sails.js endeavors!