CSCI 421 - Lecture 5

Instructor's Class Notes

Part 2:6 - Writing a REST API: Exposing the MongoDB database to the application

Author's repo for chapter 6 is here
REST Overview:

REST - REpresentational State Transfer is an architectural style and set of standards (not a protocol) that define the mechanism for transferring structured data between web endpoints.
REST is stateless in that each transfer is stand-alone and has no pre-condition nor knowledge of history.
API - Application Programming Interface named such because your application will use the API to interact with data and/or to perform specific functions.
How it works:
Rules of a REST API:

URL based - The API is comprised of URLs that must be routed via Express. Each URL endpoint will be part of a REST API for creating, reading, updating and deleting data.
HTTP methods - The HTTP protocol has four different methods and each of your REST API URL endpoints will support at least one of them:

POST - Used to Create data
GET - Used to Read data
PUT - Used to Update data
DELETE - Used to Delete data

CRUD - Create, Read, Update, Delete and the URL methods go hand-in-hand and are a fundamental part of the REST standard.
Requests and Responses - A application sends a request to a REST API URL using one of the HTTP request method types and based on the method type a particular action and response is expected, as follows:
HTTP Response Codes - With each of the responses to a request an HTTP response code should be used to indicate success, failure, etc of the requested action:

Setting up an API in Express:

Creating Routes - Creating routes to controllers is the first part of a REST API:

app_api directory - In the application directory create a new directory for holding your API code, name it app_api. You should think of the app_api directory as holding all the code (models, routes, controllers) that drive the REST API part of your web application (every URL starting with '/api/...').
Routes - Just as in a static website, your web API requires routes to redirect URL requests to specific controllers of the API. For this, make sure you have an index.js file in the a routes folder with the app_api directory and wire it up in app.js as shown below:
```
var routes = require('./app_server/routes/index');
var routesApi = require('./app_api/routes/index');

...

app.use('/', routes);
app.use('/api', routesApi);
                            
```
Specifying the Request Method - It helps to understand the routes and controller variables in index.js of the app_api consider that the routes can specify the request method of GET, POST, etc., as shown below:
```
router.get('/locations', ctrlLocations.locationInfo);
router.post('/locations', ctrlLocations.locationsCreate);
                        
```
Passing Parameters to Requests URLs - For certain request APIs like those that lookup, update or delete specific entity, a parameter or parameters must be included that specify the entity. This specifier is most often the _id (aka identifier) as returned by MongoDB when the entity (a document) was created.

For example - Consider trying to load a specific review (abc) for a specific location (123), the URL would be as follows:


/api/locations/123/reviews/abc

With the routes in index.js for reviews like the following


var ctrlReviews = require('../controllers/reviews');

...

// reviews
router.post('/locations/:locationid/reviews', ctrlReviews.reviewsCreate);
router.get('/locations/:locationid/reviews/:reviewid', ctrlReviews.reviewsReadOne);
router.put('/locations/:locationid/reviews/:reviewid', ctrlReviews.reviewsUpdateOne);
router.delete('/locations/:locationid/reviews/:reviewid', ctrlReviews.reviewsDeleteOne);

Creating Controller Placeholders - Once the routes have been defined, you should create needed controllers and place them in the controllers folder of your app_api.

Example 1 - Consider a placeholder for locations.js that only has a controller that returns success for the call to create a location:


module.exports.locationsCreate = function (req, res) {
  res.status(200);
  res.json({"status" : "success"});
};

Example 2 - Since returning a status code and a JSON formatted response is something that all controllers will need to do, introducing a function to simplify the full response is useful, as shown below:


var sendJSONresponse = function(res, status, content) {
  res.status(status);
  res.json(content);
};

module.exports.locationsCreate = function (req, res) {
  sendJSONresponse(res, 200, {"status" : "success"});
};

Example 3 - Ultimately, each controller will have to interact directly with the database, so in order to do that, the MongoDB and appropriate model for the controller must added to the controller, as shown below:


var mongoose = require('mongoose');
var Loc = mongoose.model('Location');
                                                        
var sendJSONresponse = function(res, status, content) {
  res.status(status);
  res.json(content);
};

module.exports.locationsCreate = function (req, res) {
  sendJSONresponse(res, 200, {"status" : "success"});
};

Update app.js - Don't forget to update app.js so that the change in location of the database and its models are adjusted, as so:
```
require('./app_api/models/db');
                            
```
Testing the API - At this point you can test the API, just load the URL (as like the following): http://localhost:3000/api/locations/1234.

GET methods - Reading data - Writing a controller that gets data from the database is usually done in serveral ways including one document at a time given a specific identifer, by getting a list of documents, etc.

Mongoose query types - Moongoose makes it relatively easy to query a database in various ways. A few examples include:

find - General search based on a supplied query object
findById -Look for a specific ID
findOne -Get the first document to match the supplied query

Reading One Entity - Reading one entity is relatively straightforward, as shown by the locations.js controller code show below:


var mongoose = require('mongoose');
var Loc = mongoose.model('Location');
                                                        
var sendJSONresponse = function(res, status, content) {
  res.status(status);
  res.json(content);
};
                                        
module.exports.locationsReadOne = function(req, res) {
  Loc
    .findById(req.params.locationid)
    .exec(function(err, location) {
      sendJSONresponse(res, 200, location);
    });
};

Method chaining and Callback Function - The above shows method chaining, with the Loc model object's findById method returning a database query that is ready to be executed, and that chained to call the exec of the same model object in order to execute the query which returns the JSON of the document requested.
Note further that the exec method requires a callback function which is supplied in-line. The use of the callback function allows Mongoose to get the data asynchronously, without blocking the API and your web server, and return the response when the data is returned by the database engine.

A Complete Controller Method - A complete controller method that deals with possible database errors,a specified identifer that is not found, etc is below:


/* GET a location by the id */
module.exports.locationsReadOne = function(req, res) {
  console.log('Finding location details', req.params);
  if (req.params && req.params.locationid) {
    Loc
      .findById(req.params.locationid)
      .exec(function(err, location) {
        if (!location) {
          sendJSONresponse(res, 404, {
            "message": "locationid not found"
          });
          return;
        } else if (err) {
          console.log(err);
          sendJSONresponse(res, 404, err);
          return;
        }
        console.log(location);
        sendJSONresponse(res, 200, location);
      });
  } else {
    console.log('No locationid specified');
    sendJSONresponse(res, 404, {
      "message": "No locationid in request"
    });
  }
};

Note: Recognize the use of HTTP response codes (404, 200) above.

Controller Method that Returns a List - Following is a controller method that returns a list of locations
'


/* GET a list of all locations */
module.exports.locationsList = function(req, res) {
  console.log('Getting locations list');
  Loc
      .find()
      .exec(function(err, results) {
        if (!results) {
          sendJSONresponse(res, 404, {
            "message": "no locations found"
          });
          return;
        } else if (err) {
          console.log(err);
          sendJSONresponse(res, 404, err);
          return;
        }
        console.log(results);
        sendJSONresponse(res, 200, buildLocationList(req, res, results));
      }); 
};

var buildLocationList = function(req, res, results) {
  var locations = [];
  results.forEach(function(obj) {
    locations.push({
      name: obj.name,
      address: obj.address,
      rating: obj.rating,
      _id: obj._id
    });
  });
  return locations;
};

Node - See the helper function, buildLocationList, which uses the results array of objects to push JSON-formatted entities into a new array that is then returned.

POST methods - Adding data

POST Methods are used for adding a new data document to MongoDB.
db create() - The MongoDB command to use for creating a new document. The db commands are asynchronous, thus requiring a callback function so that they do not block website processing while executing.

Sample code:


/* POST a new location */
/* /api/locations */
module.exports.locationsCreate = function(req, res) {
  console.log(req.body);
  Loc
   .create({
      name: req.body.name,
      address: req.body.address,
      facilities: req.body.facilities.split(","),
      coords: [parseFloat(req.body.lng), parseFloat(req.body.lat)],
      openingTimes: [{
             days: req.body.days1,
             opening: req.body.opening1,
             closing: req.body.closing1,
             closed: req.body.closed1,
           }, {
             days: req.body.days2,
             op ening: req.body.opening2,
             closing: req.body.closing2,
             closed: req.body.closed2,
       }]
     }, function(err, location) {
       if (err) {
          console.log(err);
          sendJSONresponse(res, 400, err);
       } else {
          console.log(location);
          sendJSONresponse(res, 201, location);
       }
     }
   );
};

Note - The above example seems a little complex. but at its core it is the same pattern as other DB method calls. There is the create method call, which is passed in JSON for the document which is to be stored (though that document is complex in that includes several fields for the location and an openingTimes object), and then the in-line callback function for the error and/or response object from the execution of the method.

PUT methods - Updating data

PUT methods are used for updating a document.
db findOneAndUpdate() - The MongoDB command to use for updating a document previously written given its _id.

Sample Code:


/* Update one Book entry */
module.exports.bookUpdateOne = function(req, res) {
    console.log("Updating a book entry with id of " + req.params.id);
    console.log(req.body);
    bookModel
  	  .findOneAndUpdate(
	     { _id: req.params.id },
 	     { $set: {"bookTitle": req.body.bookTitle, "bookAuthor": req.body.bookAuthor}},
	     function(err, response) {
	         if (err) {
	  	         sendJSONresponse(res, 400, err);
	         } else {
		        sendJSONresponse(res, 201, response);
	        }
	    }
    );
};

Note You can specify only those things you wish to update in the entity by including the $set parameters (as many as you wish) before the callback function. The syntax of each $set is the field/object name followed by its new value, which could be a simple value or a JSON object per the schema being used.

DELETE method - Deleting data

DELETE method - Implementing this method is very much like implementing a PUT method in that an ID is required.
db findByIdAndRemove() - This db method takes and id as a parameter and if found deletes the entity.

Sample Code


/* Delete one Book */
module.exports.bookDeleteOne = function(req, res) {
    console.log("Deleting book entry with id of " + req.params.id);
    console.log(req.body);
    bookModel
        .findByIdAndRemove(req.params.id)
        .exec (
            function(err, response) {
                if (err) {
                            sendJSONresponse(res, 404, err);
                } else {
                            sendJSONresponse(res, 204, null);
                }
            }
        );
};

Note - The use of HTTP status code 204 (accepted) means that the deletion request was accepted and processed without error.

Part 2:7 - Consuming a REST API: Using an API from inside Express

Author's repo for chapter 7 is here
Calling an API from Express

The Architecture - Calling REST APIs within an Express application is done by the app_server controllers. Just as these controllers previously just rendered static information that was displayed through views, more sophisticated controllers can send requests to the REST API endpoints to get, post, put and delete entities, and in turn, send render responses to the views of your web app.
request object - For controllers to be able to send requests to the REST APIs, your application must have access to the request module. You can install that module and update the dependencies of your application by running the following within your application directory:
```
$ npm install --save request                    
                    
```
Controller Code - At the top of each controller .js file include code like the following below to create a request object and default apiOptions used whenever the request object is called:
```
var request = require('request');
var apiOptions = {
  server : "http://localhost:3000"  // Change as needed
};                    
                    
```
request syntax - The syntax of the request is as follows:
options - It is through the request options that you specify the needed parameters and information to the request, including:

Simple example - Below is a simple example that executes a GET and writes the response to the console:


var requestOptions = {
    url : "http://yourapi.com/api/path", method : "GET",
    json : {},
    qs: {}
};
request( requestOptions,
         function(err, response, body) {
              if (err) {
                console.log(err);
              } else if (response.statusCode === 200) {
                console.log(body);
              } else {
                console.log(response.statusCode);
              }
          }
        );

Getting Lists of Data from an API
Separation of Concerns - You have already seen code that uses the request module to get a JSON response from a REST API. The second thing that has to be done is a response has to be rendered that sends the information to a view.

An Example: - The following example gets a list of books and then renders the list via a helper function:


/* GET books lists */      
module.exports.list = function(req, res){
    var requestOptions, path;
    path = '/api/books';
    requestOptions = { 
        url : apiOptions.server + path,
        method : "GET",
        json : {},
        qs : {} 
        };
    request(
        requestOptions,
        function(err, response, body) {
            renderListPage(req, res, body);
        }
    );
};

/* Render the book list page */
var renderListPage = function(req, res, responseBody){
    res.render('book-list', {
        title: 'Book List',
        pageHeader: {
            title: 'Book List'
        },
        books: responseBody
    });
};

View - The above example would require a book-list view developed using whatever view engine standard you wish. The view would have to loop through each of the books returned and render the HTML required. The todo tutorial gives a good example of this written using .ejs.

Getting a Single Document from an API

Single documents are often needed in order to populate a view used to edit or to delete an entity. Again, getting the document and rendering the response are best done through a seperation of concerns, as shown below:


/* Book Show */
module.exports.edit = function(req, res) {
    var requestOptions, path;
    path = "/api/books/" + req.params.id;
    requestOptions = {
        url : apiOptions.server + path,
        method : "GET",
        json : {}
    }; 
    request(
        requestOptions,
        function(err, response, body) {
                renderShowPage(req, res, body);
	}
    );
};

/* Render the book show page */
var renderShowPage = function(req, res, responseBody){
    res.render('book-show', {
        title: 'Book Info',
        pageHeader: {
                title: 'Book Info'
        },
        book: responseBody
    });
};

Adding Data to the Database via the API

Adding a document to the database is rather straightforward. You need a controller that adds the data by calling the REST API and then renders the response which typically causes the loading of a listings page, as shown below


/* Book Add */
module.exports.add = function(req, res) {
    res.render('book-add', { title: 'Add Book' });
};    

/* Book Add Post */
module.exports.addPost = function(req, res){
    var requestOptions, path, postdata;
    path = '/api/books/';

    postdata = {
        bookTitle: req.body.bookTitle,
        bookAuthor: req.body.bookAuthor
    }; 

    requestOptions = {
      url : apiOptions.server + path,
      method : "POST",
      json : postdata
    };
    
    request(
      requestOptions,
      function(err, response, body) {
         if (response.statusCode === 201) {
              res.redirect('/book-list');
         } else {
              _showError(req, res, response.statusCode);
         } 
      }
    ); 
};

.add() and .addPost() methods - The controller requires a .add() method to be associated with the GET of the URL for the "book-add" page that is used for entering a new book. That method does not need to look up any data to send to the view so that method is simple. In response to the click event of adding a new book on that page the app posts its information via HTTP POST to that same URL and that is handled the ßßby the .addPost() method of the controller.
res.redirect - Note that a redirect response is sent when the book is successfully written to the database, causing the web application to go to the listings page next.
_showError - Refer to the book or the author's examples for code for the _showError method that displays an error on a webpage.

Updating Data in the Database via the API

Updating data requires an identifier and is usually performed through some type of an edit page in your website. Since the web page will need to display the current entity information for editing the conntroller must GET the current entities data and also provide a method for posting the update via a PUT.

Example:


/* Book Edit */
module.exports.edit = function(req, res) {
    var requestOptions, path;
    path = "/api/books/" + req.params.id;
    requestOptions = {
        url : apiOptions.server + path,
        method : "GET",
        json : {}
    }; 
    request(
        requestOptions,
        function(err, response, body) {
                renderEditPage(req, res, body);
        }
    );
};


/* Render the book edit page */
var renderEditPage = function(req, res, responseBody){
    res.render('book-edit', {
        title: 'Book Edit',
        pageHeader: {
            title: 'Book Edit'
        },
        book: responseBody
    });
};


/* Book Edit Post */
module.exports.editPost = function(req, res){
    var requestOptions, path, postdata;
    var id = req.params.id;
    path = '/api/books/' + id;

    postdata = {
        bookTitle: req.body.bookTitle,
        bookAuthor: req.body.bookAuthor
    };

    requestOptions = {
        url : apiOptions.server + path,
        method : "PUT",
        json : postdata
    };

    request(
	requestOptions,
        function(err, response, body) {
            if (response.statusCode === 201) {
                res.redirect('/book-list');
            } else {
                _showError(req, res, response.statusCode);
            }
        }
    );
};

Deleting Data in the Database via the API

Deleting data is done very much like editing. Typically a document to be deleted is presented to the user on a webpage (often with the fields shown as text or in read-only text boxes, and then the user is given the option through buttons to "delete" the entity or "cancel" the deletion. The "delete" button should be part of the form and the form action should be to POST (not get) the delete web page.

Example - Below is an example that shows the controller methods for rendering the get for the delete page (which GETs the entity informaton from the REST API for display on said web page), as well as the render for that page and a method for responding to the page POST (in response to the "delete:" button press) that in turn calls DELETE on the Rest API.


/* Book Delete */
module.exports.del = function(req, res) {
    var requestOptions, path;
    path = "/api/books/" + req.params.id;
    requestOptions = {
        url : apiOptions.server + path,
        method : "GET",
        json : {}
    };
    request(
	requestOptions,
        function(err, response, body) {
            renderDeletePage(req, res, body);
        }
    );
};

/* Render the blook delete page */
var renderDeletePage = function(req, res, responseBody){
        res.render('book-delete', {
        title: 'Book Delete',
        pageHeader: {
                title: 'Book Delete'
        },
        blog: responseBody
    });
};

/* Book Delete Post */
module.exports.deletePost = function(req, res){
    var requestOptions, path, postdata;
    var id = req.params.id;
    path = '/api/book/' + id;

    requestOptions = {
	url : apiOptions.server + path,
        method : "DELETE",
        json : {}
    };

    request(
        requestOptions,
        function(err, response, body) {
            if (response.statusCode === 204) {
                res.redirect('/book-list');
            } else {
                _showError(req, res, response.statusCode);
            }
        }
    );
};

Data Validation to Protect Data Integrity

Section 7.5 - Read this section in the book to understand how to protect data integrity at several levels of your application.