restoff v0.2.34

RESToff

Synchronize client/server data using your existing RESTful APIs.

• Under MIT license.
• Database agnostic and schema-less. Works against different database types (SQL, NoSQL, Graph, etc.) and service (MySQL, SQL Server, Oracle, Postgress, MongoDB, Redis, etc.). Supports multiple databases sources. * No need to propagate schema changes to a client.
• Reconciliation Automatic reconciliation of offline changes. In clientOnly mode, RESToff automatically provides a complete RESTful offline service for your app. Reconciliation works without soft deletes or last modified date columns. Note: reconciliation is much faster with soft delete and last modified date columns available. * Conflicts are resolved by default by saving both versions. Setting concurrency to 'overwrite' changes this behavior to optimistic, last-one-wins strategy where merge conflicts are overwritten
• Supports synchronous RESTful calls. * Simplifies source code and improves client responsiveness.
• Supports resources in the following formats: * Json
• Works in following frameworks
  • Standalone - RESToff can be used without a framework.
  • Angular - RESToff wrapped in an Angular 1 provider.
• Limitations and Expectations: Every resource must have one unique primaryKey field. Best to follow RESTful best known practices
• TODO Tests to verify order of pending changes edit record A, edit A again: 2nd edit also contains 1st edit
• Next features Support rolling back a pending change Replace concurrency option with a plugin model This also will allow support for custom merge conflict resolution support non-standard get/put/post. - Example: a request GET actually does a delete. support put/post updates where resource is changed on the server. - requires better mockable REST api backend for testing. support nested resources (example: /users/45/addresses). * support non-standard restful api: ability to map a user.

RestOff Usage

Examples:

// Create a restoff service instance.
var roff = restoff({
	rootUri : "http://api.example.com/" // Required Option
});

roff.get("users").then(function(user) {
	console.log(rest.dbRepo.read("users")); // Access repository directly
});

roff.delete("users/301378d5").then(function(deletedUserId) {
	console.log("User " + deletedUserId + " deleted");
});

var aUser = {
	"id" : "ffa454",
	"first_name": "Happy",
	"last_name": "User"
}

roff.post("users", aUser).then(function(user){
	console.log("Posted user %O.", user);
});

aUser.last_name = "put";

roff.put("users/ffa454", aUser).then(function(user){
	console.log("Put user %O.", user);
});

Let's synchronize subsets of resources on the backend service using the user id:

var roff = restoff({
	"rootUri" : "http://api.example.com/"
});

function synchronize(roff, userId) {
	roff.get("users/" + userId);
	roff.get("books?ownedby="+userId);
	roff.get("addresses?userid="+userId);
	...
	roff.forcedOffline = true; // Let's go offline or unplug from the internet

	roff.get("users/" + userId).then(function(user) {
		// Yep! We can work offline.
	});
}

Resoff Reconciliation in Detail

• A1, A2, B1, B2, C1, C2, C3: No potential merge conflicts
• A3: This is the case where the same primary key was generated on each client. In this case we will simply view it as a reconciliation of changes because: * You should be using UUIDs for keys and the chance of collisions is very low. If you are using incrementing IDs, then you will need an algorithm to generate these IDs uniquely on the client. See the generateId function.
• D3: The client/server deleted a record that was updated on the server/client. In this case, we honor the delete.
• B3: Using Brent reconciliation which means we assign a new resource id (primaryKey) to the resource and post it. This does not work with hierarchical data. The intent is to allow a user to take advantage of the existing User Interface of an Application to deal with the merge conflict. TODO: We will be providing a call back that lets the developer provide a custom merge conflict screen.

RESToff Options

rootUri Required

rootUri is appended to RESTful calls when an incomplete uri is provided. rootUri is also used, under the hood, to determine the name of the repository.

Examples:

var roff = restoff({
	"rootUri" : "http://api.example.com/",
});

// uri becomes http://api.example.com/users
// repository name is users
roff.get("users").then(function(result){
	// use the result here
}

// uri is http://another.example.com/users
// repository name is users
roff.get("http://another.example.com/users").then(function(result){
	// use the result here
}

persistenceDisabled Optional

When true, storage of RESTful results on the client are disabled.

• Effectivly bypass the core feature of RESToff causing it to act like a standard http library.
• Useful for debugging.

Example configuration:

var roff = restoff({
	rootUri : "http://api.example.com/",
	persistenceDisabled	: true
});

clientOnly Optional

Creates a backend RESTful service on your client.

• Allows your RESTful API to run 100% on the client.
• Note: By design, the pending repository runs in clientOnly mode meaning the pending RESTful endpoint never hits the backend service.

var rest = restoff({
	"rootUri" : "http://api.example.com/",
	"clientOnly" : true
});

rest.put("users", { Name : "Happy User"}).then(function(result) {
	rest.get("users").then(function(result)) {
		// never hit a backend RESTful service
	});
});

onCallPending(pendingAction, uri) Optional

onCallPending is the function called by RESToff before a pending call is executed. Provided is all information about the pending action and the uri record.

// TODO: Provide the json format for pendingAction and the uri record.

var rest = restoff({
	rootUri: "http://api.example.com/",
	onCallPending: function(pendingAction, uri) {
		console.log("Pending will execute %O %O", pendingAction, uri);
	}
});

dbService.primaryKeyName or options {primaryKeyName:"name"} Required: Default id

primaryKeyName is the name of the primary key field for a given repository.

• RESToff requires every repository to have a single primary key * the primary key can't be based on a composite key
• The default value is 'id'.

You can globally set the primaryKeyName or on a per RESTful call basis:

var roff = restlib.restoff({
	rootUri: "api.example.com",
	dbService: {
		primaryKeyName : "ID",
	},
});

roff.get("users").then(function(result) {
	console.log("Primary key name used was ID.");
});

roff.get("users", {primaryKeyName:"USER_ID"}).then(function(result) {
	console.log("Primary key name used was USER_ID.");
});

forcedOffline Optional

Forces the application to run in offline mode.

• Useful to see how an application behaves when it is offline. * Example: Reload all information, go offline, and see how the application behaves.
• isOnline will return false when forcedOffline is true.

var roff = restoff({
	rootUri: "http://api.example.com/"
});

synchronize(roff, "user456");
roff.forceOffline();
if (!roff.isOnline) {
	console.log ("We are offline!");
}

NOTE: There is a slight difference between clientOnly mode and forcedoffline mode.

• In forcedOffline mode, RESToff will store any put/post/delete actions in the pending repository.
• In clientOnly mode, Retsoff will not store any put/post/delete actions in the pending repository.

generateId Optional

generateId is the function called by RESToff to generate a primary key. Bey default, restOff uses RestOff.prototype._guidGenerate() but you can define your own.

var rest = restoff({
	rootUri: "http://api.example.com/",
	generateId: function() { return Math.floor((1 + Math.random()) * 0x10000); } // not a great unique key generator
});

onReconciliation(completedAction) Optional

onReconciliation is the function called by RESToff after reconciliation of a given resource is complete.

var rest = restoff({
	rootUri: "http://api.example.com/",
	onReconciliation: function(completedAction) {
	  console.log("The following completed action was reconciled and saved to the server %O.", completedAction);
	}
});

Note: Placed features will provide a more robust reconciliation process allowing the developer to provide their own custom reconciliation process. Currently, RESToff always applies Brent Reconciliation.

concurrency Optional

The concurrency option defaults to null and conflicts are resolved through Brent reconciliation (keep both versions) Setting concurrency = 'overwrite' will result in conflicts resolved by being overwritten by the local version

pendingUri and pendingRepoName

When offline, RESToff places any put/post/delete RESTful operations in a pending repository. RESToff uses it's own persistence engine meaning it calls itself using get/post/delete.

Use pendingUri and pendingRepoName to configure the URI and repository name of the pending repository.

• Note: In clientOnly mode, no changes are recorded in the pending repository.

RESToff Methods

autoQueryParamSet(name, value)

A parameter of name with value will be added/appended to EVERY RESTful api call. Useful for adding parameters such as an access token.

Example usage:

var roff = restoff({
	rootUri : "http://api.example.com/"
});
roff.autoQueryParamSet("access_token", "rj5aabcea");

// uri becomes http://api.example.com/users?access_token=rj5aabcea
roff.get("users").then(function(user)) {
	...
}

autoQueryParamGet(name)

Returns the value of the query parameter with the provided name.

Example usage:

var roff = restoff({
	rootUri : "http://api.example.com/"
});
roff.autoQueryParamSet("access_token", "rj5aabcea");
var paramValue = roff.autoQueryParamGet("access_token");

TODO: Add autoQueryParameters as an option.

autoHeaderParamSet(name, value)

A header of name with value will be added to the header of every RESTful api call. Useful for adding parameters such as an access token.

Example usage:

var roff = restoff({
	rootUri : "http://api.example.com/"
});
roff.autoHeaderParamSet("access_token", "rj5aabcea");

autoHeaderParamGet(name)

Returns the value of the header parameter with the provided name.

Example usage:

var roff = restoff({
	rootUri : "http://api.example.com/"
});
roff.autoHeaderParamSet("access_token", "rj5aabcea");
var headerValue = roff.autoHeaderParamGet("access_token");

TODO: Add autoHeaderParams as an option.

clear(repoName, forced)

Clears the cache of the given repository name if the repository exists.

• Does not clear the repository if there are pending changes. * Optional parameter forced - Pass a value of true to force clearing a repository even if it has pending changes.
• Does not create a repository named repoName to the repository if it doesn't exist.

Example:

var roff = restoff({
	rootUri : "http://api.example.com/"
});
roff.clear("users", true);
console.log("User repository, even if it had pending, was cleared ")

clearAll(forced)

Clears the cache of all repositories.

• Does not clear any repositories if even one of them have pending changes. * Optional parameter forced - Pass a value of true to force clearing all repositories even if there are any pending changes.

Example:

var roff = restoff({
	rootUri : "http://api.example.com/"
});
roff.clearAll(true);
console.log("User repository, even if it had pending, was cleared ");

delete(uri, options), deleteRepo(uri, options)

• delete(uri, [options]) asynchronously deletes a resource from a remote server and in the local repository.
• deleteRepo(uri, [options]) synchronously deletes a resource from in the local repository.

Note:

• A 404 (not found) is "ignored" and the resource is still considered removed from the local repository.
• If delete is called on a non-existent repository, an empty repository is created.

Example usage:

var roff = restoff();
return roff.delete("http://test.development.com:4050/users/553fdf")
.then(function(result){
	// user was deleted
});

roff.forcedOffline = true;
var result = roff.deleteRepo("http://test.development.com:4050/users/553fdf");

get(uri, options), getRepo(uri, options)

• get(uri, [options]) asynchronously retrieves a resource from a remote server. Uses the local repository when offline.
• getRepo(uri, [options]) synchronously retrieves a resource from the local repository.

Example usage:

var roff = restoff();
return roff.get("http://test.development.com:4050/testsweb/testdata/users")
.then(function(result){
	// use the result here
});

roff.forcedOffline = true;
var result = roff.getRepo("http://test.development.com:4050/testsweb/testdata/users");

post(uri, resource, options), postRepo(uri, resource, options)

• post(uri, resource, [options]) asynchronously posts a resource to a remote server and in the local repository adding the resource if it doesn't exist or overwriting the existing resource.

• postRepo(uri, resource, [options]) synchronously posts a resource in the local repository adding the resource if it doesn't exist or overwriting the existing resource.

• When online, post(...) calls will happen immediately.

• With postRepo(...), updates will happen when a get(...) is executed on that resource.

Example usage:

var roff = restoff();
var newUser = {
	"id" : "ffa454",
	"first_name": "Happy",
	"last_name": "User"
}

return roff.post("http://test.development.com:4050/users", newUser)
.then(function(result){
	// use the result here
});

roff.forcedOffline = true;
var result = roff.postRepo("http://test.development.com:4050/users", newUser)

put(uri, resource, options), putRepo(uri, resource, options)

• put(uri, resource, [options]) asynchronously puts a known resource on a remote server and in the local repository updating the resource id provided in the uri.
• putRepo(uri, resource, [options]) synchronously puts a known resource in the local repository updating the resource id provided in the uri.

Example usage:

var roff = restoff({
	"rootUri" : "http://test.development.com:4050/"
});
var existingUser = {
	"id" : "ffa454",
	"first_name": "Happy",
	"last_name": "User"
}

return roff.post("users/ffa454", existingUser)
.then(function(result){
	// use the result here
});

roff.forcedOffline = true;
var result = roff.postRepo("users/ffa454", existingUser)

uriFromClient(uri)

restOff may add additional query parameters when restOff.get(uri) is called. Use this method to see the final uri sent to the backend server.

Example usage:

var roff = restoff().autoQueryParamSet("access_token", "rj5aabcea");
var actualUri = roff.uriFromClient("http://test.development.com:4050/emailaddresses");
expect(actualUri)).to.equal("http://test.development.com:4050/emailaddresses?access_token=rj5aabcea");

FAQ

• What is the difference between offlineOnly and forcedOffline? - forcedOffline will line up any pending changes that are then synchronized when the client comes back online. - offlineOnly does not log any "pending changes" because the repository will never be synchronzied with a backend service.

Developoment Setup

Want to help out? Here is what you need to do to get started.

// TODO: Finish this part of documentation

Using In Your Projects

Change directory to your node project.

$ npm install --save restoff

Setup

$ npm install
$ npm install -g json-server   // for testing

Update Node Module Dependencies

$ npm outdated

Node Modules Used

• jsonplaceholder

Continuous Rebuild and Testing

See ./dist for files we build.

$ gulp

Test

$ gulp webtests

Test Server

Read documentation in gulpfile.js to see how to setup automated web testing.

$ gulp webserver

Publish and Push New Version

First time?

$ npm adduser           # Need to do one time

1) Verifty package.json version is newer than one in npm (visit https://www.npmjs.com/package/restoff)

2) Verify no pending changes

$ git status

3)

$ npm publish ./
$ git tag -a 0.1.4 -m "v0.1.4"  // 0.1.4 is an example
$ git push origin --tags

4) Increment version in package.json and update check-build test. Re-run tests.

Development Setup

• Install LiveReload Chrome extension.

Hosts File

Add to your /etc/hosts file:

127.0.0.1 test.development.com

Initial install

$ npm install
 (NPM install stuff ommitted)
$ mocha tests

  3 passing (or something like this)

Start the test suite

$ gulp
  (gulp stuff happens, look for this line...)
  Server started http://test.development.com:4050

... then open the test suite in your browser http://test.development.com:4050

RESToff Angular

RESToff is wraped in an angular provider.

Example Usage:

angular.module("fakeRoot", ["restoff"])
    .config(["restoffProvider", function (restoffProvider) {
        restoffProvider.setConfig({
            rootUri:"http://localhost/"
        });
    }]);

Note that we "hard code" the configuration, but you could also get the configuration from another service.

Development Issues

Live Reload Isn't Reloading

• In chrome, navigate to chrome://extensions/ then find the LiveReload extension and check Allow access to file URLs.
• Click on livereload icon in chrome browser: small circle in center should become solid.
• Is more than one instance of gulp running?

Hide HTTP Network Messages During Testing

• In chrome, right mouse click and inspect.
• From the console tab, click on filter (icon next to Top)
• Check "Hide network messages"