Code documentation - express-examples/sqlite

State of the art

Code documentation appears to be not so clear into JS world. I've been reading several pages and blogs about this, and finally I got 2 main solutions, jsdoc-toolkit and ScriptDoc.

At the beginning I leaned for ScriptDoc, because people say is very similar to Javadoc and some days ago I developed a mobile app using Titanium. But, I found JSDoc more mature, even when the project appears to be forgotten (last commit on June 2010).

Finally, I selected JSDoc. Here is a list of people using it. The project documentation here and a list of tags here.

An example

From /lib/rest/common.js

/** 
* @fileOverview Common JS for REST services (express-examples/sqlite)
*
* @author Rodolfo Campos <camposer at gmail dot com>
* @version 1.0
*/

/**
 * HTTP codes 
 * @see For all HTTP codes refer to: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html 
 */
var HTTP_CODE_OK = 200
  , HTTP_CODE_UNAUTHORIZED = 401
  , HTTP_CODE_FORBIDDEN = 403
  , HTTP_CODE_NOT_FOUND = 404
  , HTTP_CODE_METHOD_NOT_ALLOWED = 405
  , HTTP_CODE_PRECONDITION_FAILED = 412;

/** 
 * JSON Content-Type HTTP Header
 * @field
 */
var JSON_RESPONSE_HEADER = {'Content-Type': 'application/json; charset=utf-8'};

/** 
 * Writes HTTP code 200 and Content-Type for JSON in HTTP response headers
 * @function 
 * @param {HttpResponse} res HTTP response 
 * @param {Number} code HTTP code to be written in header
 */
var ok = function(res) {
  res.writeHead(HTTP_CODE_OK, this.JSON_RESPONSE_HEADER);
};

/** 
 * Writes HTTP code 4XX (error) and Content-Type for JSON in HTTP response headers
 * @function 
 * @param {HttpResponse} res HTTP response 
 * @param {Number} code HTTP code to be written into header
 */
var nok = function(res, code) {
  res.writeHead(code, JSON_RESPONSE_HEADER);
};

/**
 * Executes validate method using params, if true, calls execute method, if not, returns HTTP code 412
 * @param {HttpResponse} res HttpResponse object
 * @param {Mixed|Object} params Can be value or object (JSON)
 * @param {Function([params])} validate Function used for validation, if returns true everything is OK, 
 *   if not something went wrong and call returns HTTP_CODE_PRECONDITION_FAILED(412)
 * @param {Function(params, callback)|Function(callback)} execute Function executed if validate=true. 
 *   Parameters (params) can be value (e.g. String) or Object (JSON). Callback should receive 
 *   one parameter (result) 
 *   
 * @example 
 * common.call(
 *   res, 
 *   params.name, 
 *   function(name) { 
 *     var valid = true;
 *
 *     if (name == null) 
 *       valid = false; 
 * 
 *     return valid;
 *   },
 *   function(params, callback) {
 *     // ...
 *     // Get result
 *     callback(result);
 *   }
 * );
 */
this.call = function(res, params, validate, execute) {
  if (validate(params)) {
    if (params != null) {
      execute(params, function(result) { // TODO: Add extra parameter for error msg
        if (result) 
          ok(res); 
        else 
          nok(res, HTTP_CODE_FORBIDDEN); 

        res.end(JSON.stringify(result)); 
      });
    } else {
      execute(function(result) { // TODO: Add extra parameter for error msg
        if (result) 
          ok(res); 
        else 
          nok(res, HTTP_CODE_FORBIDDEN); 

        res.end(JSON.stringify(result)); 
      });
    }
  } else {
    nok(res, HTTP_CODE_PRECONDITION_FAILED); 
    res.end(); 
  }
};

Generating documentation

For generating the HTML documentation corresponding to the example above (common.js), you just have to download jsdoc-tool from here, unzip it, and run the following command:

$ java -jar jsrun.jar app/run.js -a -t=templates/jsdoc -d=common common.js

Where:

• jsrun.jar: Generator
• app/run.js: Bootstrap script
• -a: Include all functions, even undocumented ones
• templates/jsdoc: Directory of the templates to be used
• -d: Directory where output is going to be put. By default ./out folder
• common.js: File parsed

For JsDoc inline documentation use:

$ java -jar jsrun.jar app/run.js --help

You can use -r option for generating documentation recursively, specifying a directory to be explored. I like this option, but specifying directories, for example:

$ java -jar jsrun.jar app/run.js -a -t=templates/jsdoc -d=sqlite -r ~/express-examples/sqlite/lib/rest ~/express-examples/sqlite/lib/service

Note: If you specify the root directory as starting point, you're going to have a mess with all JS files in the project (including node_modules).

Updated code documentation can be found inside the project at /jsdoc folder.

REST - express-examples/sqlite

I've developed a very simple RESTful app, only considering CRUD operations over an user entity.

Naming convention

I followed some good recommendations mentioned in this article, shared by jjeronimo.

In general, the API is:

• getUsers. Retrieves all users (SELECT * FROM user).

curl -X GET http://localhost:3000/rest/users

• getUsersById. Retrieves an user by its id (SELECT * FROM user WHERE id = ?).

curl -X GET http://localhost:3000/rest/users/1

• addUser. Adds an user (INSERT INTO user(name) VALUES(?)).

curl -X PUT -H "Content-type:application/x-www-form-urlencoded" -d "name=rodolfo" http://localhost:3000/rest/users

• updateUser. Updates an user (UPDATE user SET name = ? WHERE id = ?).

curl -X POST -H "Content-type:application/x-www-form-urlencoded" -d "id=1&name=juan" http://localhost:3000/rest/users

• removeUser. Removes an user by its id (DELETE FROM user WHERE id = ?).

curl -X DELETE -H "Content-type:application/x-www-form-urlencoded" -d "id=1" http://localhost:3000/rest/users

As you can see, the URLs follow the pattern: rest/users. Some comments about this:

• URLs start with the word "rest" for easily identifying them as part of the REST API. Another part indicating the API version can be added, like: rest/v1/users (util for backwards compatibility issues).
• A REST API should be easy to discover, just adding and removing parameters. For example: if you want all users can use /users, but if you want an specific user can use /users/1. Maybe a better option, for more complex cases, could be: /users/id/1, because this way results "evident" something like: /users/name/rodolfo/status/0. However, I prefer to specify getXxxById following the entity name (written in plural) by its id.

Facade-Service

I think REST APIs should be coded using Facades. The idea is to decouple business logic into some classes, and treat REST, SOAP, and whatever interfaces aside.

Following an example extracted from /lib/rest/user.js:

var common = require('./common');

var userService = require('../service/user');

this.addUser = function(req, res) {
  var params = req.body;

  common.call(
    res, 
    params.name, 
    function(name) { 
      var valid = true;

      if (name == null) 
        valid = false; 
  
      return valid;
    },
    userService.newInstance().addUser
  );
};

// ...

As you can see in the code above, inside the REST addUser method there are only operations related with parameters filling and validations. For this specific implementation, I coded a generic method for calling services (common.call). But you could just take parameters, validate them, call the corresponding service method, and return the expected output; sort of controller in a MCV implementation.

Call method for integrating Service - REST Facade

I wrote a method call (lib/rest/common.js) for integrating Service methods and REST facades. Following the code main highlights:

// ...

this.call = function(res, params, validate, execute) {
  if (validate(params)) {
    if (params != null) {
      execute(params, function(result) { // TODO: Add extra parameter for error msg
        if (result) 
          ok(res); 
        else 
          nok(res, HTTP_CODE_FORBIDDEN); 

        res.end(JSON.stringify(result)); 
      });
    } else {
      execute(function(result) { // TODO: Add extra parameter for error msg
        if (result) 
          ok(res); 
        else 
          nok(res, HTTP_CODE_FORBIDDEN); 

        res.end(JSON.stringify(result)); 
      });
    }
  } else {
    nok(res, HTTP_CODE_PRECONDITION_FAILED); 
    res.end(); 
  }
};

// ...

The call method executes validate method using params, if true, calls execute method, if not, returns HTTP code 412 (HTTP_CODE_PRECONDITION_FAILED).

• res: HttpResponse object
• params: Can be value or object (JSON)
• validate: Function used for validation, if returns true everything is OK (HTTP_CODE_OK=200), if not, something went wrong and returns HTTP_CODE_PRECONDITION_FAILED(412)
• execute: Function executed if validate=true. Parameters (params) can be value (e.g. String) or Object (JSON). Callback should receive one parameter (result).

An example about how to use the common.call method can be found above (this.addUser).

Init script - express-example/sqlite

I've been reading in the Express Forum about deploying Node.js applications and found some interesting points of view about that. My proposal is to deploy using Capistrano. I hope to comment about it in this space soon.

The thing that takes me to this space now is related to init scripts... Some days ago I was "fighting" against Pentaho BI Server init scripts, and based on my previous work, I propose the following script (init.d/sqlite) for Node.js applications. For installing, you need to change the following variables:

SQLITEAPP_DIR="/home/rodolfo/workspace/express-examples/sqlite"
SQLITEAPP_LOG="$SQLITEAPP_DIR/log/sqlite.log"

After changing that variables to the root directory where you have the Express App, and where you want to leave logs, can just start or stop the server like this:

$ bash init.d/sqlite start
$ bash init.d/sqlite stop

If you want to include the script into system init.d scripts (for -maybe- adding them later to rcX.d scripts) can do the following:

$ chmod +x init.d/sqlite
$ sudo ln -s init.d/sqlite /etc/init.d/sqlite-express-example

Now for starting and stopping:

$ sudo /etc/init.d/sqlite-express-example start
$ sudo /etc/init.d/sqlite-express-example stop

NOTE: As you can see, my script has the name of the app, that's because I'm assuming one app per Node.js instance, but if you're going to have only one instance for all apps, can rename all variables and the script to something related to Node.js.

ERRATUM: Is obvious that the name sqlite for the script is a very bad choice. Why I put that name? Nevermind... You can easily change it.

DAO - express-examples/sqlite

According to Wikipedia, a DAO "is an object that provides an abstract interface to some type of database or persistence mechanism, providing some specific operations without exposing details of the database. It provides a mapping from application calls to the persistence layer".

This concept just introduces more abstraction and decoupling into your code. In my experience it results fundamental for maintaining painless an app.

Following, an implementation example of this pattern (lib/dao/user.js):

var sqlite3 = require('sqlite3').verbose();
var db = new sqlite3.Database('./db/db.sqlite');

this.newInstance = function() {
  return new impl();
};

function impl() {
  // ...

  this.getUsers = function(callback) {
    db.all("SELECT rowid AS id, name FROM user", function(err, rows) {
      if (err) {
        console.log(err);
        rows = null;
      }

      callback(rows);
    });
  };

  // ...
};

As you can see in the code above, we have a class with several methods (specially getUsers). If you want to query the database, you just have to call the necessary method. The rule is simple, group database operations in related classes (e.g. user) and never put a query outside that scope.

Another important thing is: DAO's shouldn't do anything else than database operations, no validation, no business logic! Validations (extra app forms) and "Business Logic" should be put in other classes... You can call them: logics, services, or whatever you want, but separated. Here is an example of a service class (lib/service/user.js):

var userDao = require('../dao/user');

this.newInstance = function() {
  return new impl();
};

function impl() {
  // ...

  this.getUsers = function(callback) {
    userDao.newInstance().getUsers(callback);
  };

  // ...
}

In this example, the service method getUsers doesn't include any business logic, but maybe this can change in the future and for that reason is better to keep it separated.

And finally, you can call the services from a controller or service facade (REST, SOAP, ...) like (lib/rest/user.js and lib/rest/common.js):

var userService = require('../service/user');

this.getUsers = function(req, res) {
  common.call(
    res, 
    null, // No params
    function(params) { return true },
    userService.newInstance().getUsers
  );
};

Classes and Factories - express-examples/sqlite

In JS there are several ways of defining and instantiating a class. A really good article commenting ways of doing that, courtesy of jjeronimo, available here.

Based on the article mentioned above, and my previous experience with factories implementations in Java (an example), I propose to implement classes this way (lib/dao/user.js):

var sqlite3 = require('sqlite3').verbose();
var db = new sqlite3.Database('./db/db.sqlite');

this.newInstance = function() {
  return new impl();
};

function impl() {
  // ...

  this.getUserById = function(id, callback) {
    db.all("SELECT rowid AS id, name FROM user WHERE id=?", [ id ], function(err, rows) {
      if (err) 
        console.log(err);

      callback((rows)?rows[0]:null);
    });
  };

  // ...
};

If you want to use the class defined above, you just have to do something like (lib/service/user.js):

var userDao = require('../dao/user');

  // ...

  this.getUserById = function(id, callback) {
    userDao.newInstance().getUserById(id, callback);
  };

  // ...

One of the main advantages of this approach, factories based, is that you can mockup very easy your classes for testing. For example:

this.newInstance = function(env) {
  if (env && env == 'test') 
    return new mock();
  else
    return new impl();
};

function impl() {
  // Normal implementation
}

function mock() {
  // Mockup implementation (for testing)
}

Then, if you want to test, you just have to specify the parameter test when instantiating the object, something like:

var service = require('service');
var serviceMock = service.newInstance('test');
// Test ...

var serviceReal = service.newInstance();
// Do something 'real'

SQLite - express-examples/sqlite

Installing SQLite3

SQLite3 is very simple, and perfect for examples "plug & play". For installing it:

$ sudo aptitude install sqlite3

Creating a Database

In my case I just entered:

$ sqlite3 db/db.sqlite

Once inside, you can enter SQL commands. Following sqlite example table:

sqlite> CREATE TABLE user (name VARCHAR(50)); 

In SQLite exists a default auto-increment field named `rowid`, that means you can make queries like:

sqlite> SELECT rowid AS id, name FROM user WHERE id=?

Installing SQLite plugin for Node.js

I'm using node-sqlite3. I found this plugin immature and not working properly under several scenarios, since I'm not really interested in developing something complex with sqlite, not going to invest more time on this, but maybe node-sqlite is a better option.

For installing it:

$ npm install sqlite3

No matter which of the 2 previously mentioned plugins you use, you have to use callback programming style.

var sqlite3 = require('sqlite3').verbose();
var db = new sqlite3.Database('./db/db.sqlite');

// Querying
db.all("SELECT rowid AS id, name FROM user WHERE id=?", [ id ], function(err, rows) {
  if (err) 
    console.log(err);

  callback((rows)?rows[0]:null);
});

// Adding
db.run("INSERT INTO user(name) VALUES(?)", [ name ]/*, function(error) {
  if (error) 
    console.log(error);

  callback((error)?false:true);
}*/);
callback(true); // TODO: Callback is not working!!

As you can see in the code above, callbacks are not working properly in `db.run` method. But, the really interesting thing I want to emphasize is the use of them (callbacks), this adds a-synchronicity to your code, which means things are not blocked until getting a response. Of course if you're experienced with thread capable languages (e.g. Java), this may not surprise you, but if you're PHP programmer you should be surprised. Another thing, the simplicity and let me say "harmony" of this approach is outstanding.

Routing - express-examples/sqlite

Changes for routing

When you install express it adds routes inside app.js, I'm not really comfortable with that because this file for me must be only for app configuration, and things like routes, which changes a lot, should be configured in other place. My changes were:

• Remove all routes mappings from app.js and add one call for global routes: routes.route(app);
• Leave calls for mappings inside routes/index.js separated in files. For example:

var views = require('./views');
var rest = require('./rest');

exports.route = function(app) {
   views.route(app);
   rest.route(app);
};

• Add route method for each group of routes (this can be splitted in more files for larger apps). For example (routes/rest.js):

var userRest = require('../lib/rest/user');
var BASE_URL = '/rest';

this.route = function(app) {
  this.users(app);
};

this.users = function(app) {
  app.get(BASE_URL + '/users', userRest.getUsers);
  app.get(BASE_URL + '/users/:id', userRest.getUserById);
  app.put(BASE_URL + '/users', userRest.addUser);
  app.delete(BASE_URL + '/users', userRest.removeUser); 
  app.post(BASE_URL + '/users', userRest.updateUser);
};

Note: I propose this files remain without any "business" logic, just used for routing purposes.

Some words about MVC

In this case I didn't use MVC, because my approach is for heavier clients, doing Ajax requests to RESTful applications. Something code-behind oriented. But... What if you don't want that? Well, here is a proposal (based on my actual code) for MVC:

• Change routes/views.js like this:

var userController = require('../lib/controller/user');

this.route = function(app) {
  app.get('/', userController.index);
};

Note: You can divide route function like in routes/rest.js

• Add /lib/controller/user.js like this:

var userService = require('../service/user');

this.index = function(req, res){
  var users = userService.getUsers();
  res.render('index', { title: 'List of users', session: req.session, users: users });
}

• Inside index.ejs render users:

// ...

<% for (var i=0; users.length; i++) { %>
    <tr>
      <td><%= users[i].id %></td>
      <td><%= users[i].name %></td>
    </tr>
<% } %>

// ...

HTML escaped using: http://www.htmlescape.net/htmlescape_tool.html