npm.io
1.1.0 • Published 9 years ago

qsapi

Licence
ISC
Version
1.1.0
Deps
4
Vulns
25
Weekly
0
SummaryDependencyVersions

QSAPI

Quasi-API - Hand sanitiser for your API

Why?

Sometimes API's are bad. Sometimes they fail, Sometimes they don't. Your application shouldn't have to deal with intermittent API issues, It shouldn't have to deal with mismatched property types, or properties missing altogether.

Usage

import {Qsapi, Schema} from 'qsapi'
const {type, transform, initial} = Schema 

var schema = {
    ip: {
        [type]: 'String',
        [initial]: '127.0.0.1',
        [transform]: (ip) => {
            return Number(ip.replace('.',''))
        }
    }
}

var initialData = {
    ip: '127.0.0.1'
}

var qsapi = Qsapi({ 
    options: { 
        url: 'https://whatsmyip.azurewebsites.net/json',
        timeout: 2000,
        retryCount: 5 
    },
    schema,
    intiialData
})

Fetch

QSAPI presumes that the API being called is unstable and often unavailable. It will by default attempt to fetch the resource data 3 times before rejecting the original promise. This default can be configured during initialisation of the QSAPI call.

Using fetch, in its most basic form, all you need to supply is a url, everything else is handled by the default values.

Schema modelling

A schema can be provided to QSAPI to transform the result of the API call to the expected object. This can be used to make sure the data coming back from the API is uniform and consistant to what the UI is expecting.

Examples

Fetch examples

Basic example

Make a GET request to google.com, timeout after 1 second, don't retry.

import {Fetch} from 'qsapi'
var opts = {
    url: 'http://www.google.com',

    // timeout after 1 second
    timeout: 1000,

    // don't retry
    retry: false
}

var instance = Fetch.req(opts)
instance.then((res) => {

    console.log(res) 
})

Advanced example:

import {Fetch} from 'qsapi'

var retryCount = 3
var opts = {
    url: 'http://httpstat.us/500',
    timeout: 2000,
    retry: (req) => {
        console.log(`retry attempt #${retryCount - req.retryCount + 1} ${req.url}`)
    },
    retryCount,
}

var instance = Fetch.req(opts)

// on successful response
instance.then((res) => {
    console.log('Success!', res)
})

// once retryCount reaches 0 and 
instance.catch((err) => {
    console.log(`${opts.url} could not be fetched: ${err.code}`)
})

Schema mapping example

Think for a moment that you were dealing with an API that returned a list of products, and price:

var data = {
    products: [
        {
            id: 'product1',
            name: 'product 1',
            description: 'the first product',
            price: 55
        }, 
        {
            id: 'product2',
            name: 'product 2',
            description: 'the second product',
            price: '66.50'
        },
        {
            id: 'product3',
            name: 'product 3',
            price: '$11.00'
        }
    ]
}

The API response above is not great, we have inconsitant fields which is common with NoSQL based data stores, we also have inconsistant typing of the price field across products.

If we were dealing with this API in the front end logic of our application, we would need to add a lot of bulk and complexity to be evaluated at runtime just to make sure the properties exist, and they are the type that we are expecting. Not only does this bulk the application out, it makes it generally harder to read and scale for any developers being on-boarded.

Using QSAPI schema mapping, we can define a schema for how we want our data to be structured, and typed:

import Schema from 'qsapi'
const {parse, type, initial, transform} = Schema

var schema = {
    products: {
        id: {
            [type]: 'string'
        },

        name: {
            [type]: 'string'
        },

        description: {
            [initial]: 'N/a'
        },

        price: {
            [transform]: (price) => {
                return parseFloat(price.toString().replace('

Using the schema defined above, we can parse our data source:


// ...(continued from above)...

var mappedData = parse(data, schema)

/*
    mappedData.products:

    [
        { 
            id: 'product1',
            name: 'product 1',
            description: 'the first product',
            price: 55 
        },
        { 
            id: 'product2',
            name: 'product 2',
            description: 'the second product',
            price: 66.5 
        },
        { 
            id: 'product3',
            name: 'product 3',
            price: 11,
            description: 'N/a' 
        } 
    ]
*/


After the mapping has been applied, each field is consistant in type, and also has the same fields.
description was added to product3, price was transformed from being mixed type in the data to a float in the mapped data




API


Qsapi(options, schema [, initialData])






Property
Description
Type
Default





options
Options for the fetch request
Object
{}




schema
The schema that the response fetch will be transformed to
Object
{}




initialData
If supplied, no request will be made, the initialData will be parsed through the schema
Object
{}





Methods:






Method
Description
Returns





fetch
See Fetch.req(options)
Promise







Fetch.req(options)


This is the main fetch function that returns a fetch instance (Promise)


QSAPI uses axios under the hood so any property supported by axios is also supported by QSAPI.


The options is an object that will accept the following:






Property
Description
Type
Default





url
The url to fetch
String
-




schema
The schema to use for the request
Object
-




method
The HTTP Method to use
String
'GET'




bailout
A function that gets evaluated, if the function returns true, the request will not run.
Function
() => { return false }




cache
Define if the response should be stored in the cache or not
Boolean
false




retry
A value to define if the request should retry on failure. If value is a function, it will get evaluated on retry
Function/Boolean
true




retryCount
A number to define how many times the request should retry
Number
3




headers
Define any headers that you many required
Object
{}




params
Define any URL parameters to be sent with a GET
Object
{}




data
Define any data to be sent with a POST
Object, FormData, File, Blob, Stream
{}




auth
Send HTTP Basic auth credentials. This will set a Authorization header
Object
{}




responseType
Indicate what type of data the response will carry
String
'json'




xsrfCookieName
The name of the cookie to use as a xsrf token
String
'XSRF-TOKEN'




xsrfHeaderName
The name of the header to use as a xsrf token
String
'X-XSRF-TOKEN'




onUploadProgress
A function that is called with the progressEvent of an upload
Function
() => {}




onDownloadProgress
A function that is called with the progressEvent of a download
Function
() => {}




maxContentLength
A number that defines the maximum length of the response content
Number
-




maxRedirects
A number that defines the maximum number of redirects (Node.js only)
Number
5





Example:


import {Fetch} from 'qsapi'

var opts = {
    url: 'http://whatismyip.azurewebsites.net/json',

    // cache the response
    cache: true,

    // called if request fails, the existance of this function causes retrying to be enabled.
    retry: (request) => {
        console.log(`Failed to load ${opts.url}, retrying`)
    },

    // the expected response type
    responseType: 'json'
}

// define an on error function that show when we give up.
var onError = (err) => {
    if (err.retryCount === 0) {
        console.log(`failed to load ${err.url}, giving up`)
    }
}

// setup the request instance
var instance = Fetch.req(opts)
instance.then((res) => {

    // when we have a response - output to the terminal
    console.log(`received response from ${opts.url}`)

    // then make the request again
    Fetch.req(opts).then((res) => {

        // when we have the response again, check if it was pulled from the cache
        if (res.cached) {
            console.log(`loaded response from cache for ${opts.url}`)
        }
        else {
            console.log(`received response from ${opts.url}`)
        }
    })
    .catch(onError)
})
.catch(onError)




Fetch.setup(config)


This method will set up the fetch instance with a cache. 


If you wish to use caching and want something a bit more elaborate than in-memory caching


Example:


import {Fetch} from 'qsapi'

var cacheStore = []

Fetch.setup({
    cache: {
        get: (key) => {
            // this will get called with the URL of the requested resources.
            // Must return a response.
            return cacheStore[key]
        },

        set: (key, value) => {
            // this will get called when the requested resource returns with a response.
            /*
                EG:
                key: 'http://www.google.com'
                value: {
                    data: {},
                    status: 200,
                    statusText: 'OK',
                    headers: {},
                    config: {}
                }
            */
            cacheStore[key] = value
        }
    }
})


Schema


Schema exports all of the symbols that we use to run specific logic on properties:






Property
Description
Type
Default





type
Used to indicate to the schema mapping what the output type should be
Symbol
-




initial
The value to be used if there is no data for this specific property
Symbol
-




transform
A function that gets evaluated, the first parameter is the data of the property being evaluated
Symbol
-




custom
Used to define properties that may not exist on the object. The parent of the property is passed as a property
Symbol
-




required
If this child property is not present, then the object will be removed from the result
Symbol
-




rename
Used to rename the field to a new field, the value of the symbol is the name of the new field
Symbol
-





Schema.parse(data, model)


This will parse the data using the model supplied, a new object will be returned.


Schema.type


Define that the object should include this property and it should be a JavaScript type (Not implemented yet)


Schema.initial


The default value to use for a property.


Example:


var data = {
    products: [{
        id: 1,
        name: 'car'
        SKU: '123'
    }]
}

var schema = {
    products: [{
        id: {
            [type]: 'number'
        }

        name: {
            [type]: 'string'
        },

        SKU: {
            [rename]: 'sku'
        },

        description: {
            [initial]: 'One of our products'
        }
    }]
}


Once parsed, the new data object will contain a product with an id, name, sku and a description of 'One of our products'


Schema.transform


Transform the object using the data object as a property.


Example:


var data = {
    products: [{
        id: 1,
        sku: 'someSku1'
    }]
}

var schema = {
    products: [{
        id: {
            [type]: 'number'
        },

        sku: {
            [transform]: (sku) => {
                return sku.toUpperCase()
            }
        }
    }]
}


This will return an object with the products array, any obj in the product array that has a sku will be transformed toUpperCase()


Schema.custom


If you there is no property by the name of what you want on the object, you can generate one by using the [custom] Symbol.


Example:


var data = {
    products: [{
        id: 1,
        sku: 'someSku1'
    }]
}
var schema = {
    products: [{
        id: {
            [type]: 'number'
        },

        [custom]: (product) => {
            return {
                name: product.sku.toLowerCase().replace(/\d+/gi, '')
            }
        }
    }]
}


This will add a name property to the objects in the array.


Schema.required


If specific data is required, and the object is pointless without it, you can use the required property.


Example:


var data = {
    products: [{
        id: 1,
        name: 'a plant',
        sku: 'someSku1'
    }, {
        id: 2,
        sku: 'someSku2'
    }]
}
var schema = {
    products: [{
        id: {
            [type]: 'number'
        },

        name: {
            [required]: true
        }
    }]
}


This will make sure that any objects in the products array will contain a name, in the above example, the products array will contain 1 object.




TODO


    

  •  Schema mapping
    • 

  •  Schema type transformation
    • 

  •  Fetch API
    • 

  •  Fetch setup to allow for retries, timeouts, bailouts
    • 

  •  Pre-fetch caching
    • 

  •  Post-fetch caching
    • 


, ''), 2).toFixed(2)
            }
        }
    }
}

Using the schema defined above, we can parse our data source:

__CODE_BLOCK_5__

After the mapping has been applied, each field is consistant in type, and also has the same fields. __INLINE_CODE_3__ was added to __INLINE_CODE_4__, __INLINE_CODE_5__ was transformed from being mixed type in the data to a __INLINE_CODE_6__ in the mapped data

API

__INLINE_CODE_7__

Property Description Type Default
__INLINE_CODE_8__ Options for the fetch request Object __INLINE_CODE_9__
__INLINE_CODE_10__ The schema that the response fetch will be transformed to Object __INLINE_CODE_11__
__INLINE_CODE_12__ If supplied, no request will be made, the initialData will be parsed through the schema Object __INLINE_CODE_13__

Methods:

Method Description Returns
__INLINE_CODE_14__ See __INLINE_CODE_15__ __INLINE_CODE_16__

__INLINE_CODE_17__

This is the main fetch function that returns a fetch instance (__INLINE_CODE_18__)

QSAPI uses axios under the hood so any property supported by axios is also supported by QSAPI.

The options is an object that will accept the following:

Property Description Type Default
__INLINE_CODE_19__ The url to fetch String -
__INLINE_CODE_20__ The schema to use for the request Object -
__INLINE_CODE_21__ The HTTP Method to use String __INLINE_CODE_22__
__INLINE_CODE_23__ A function that gets evaluated, if the function returns true, the request will not run. Function __INLINE_CODE_24__
__INLINE_CODE_25__ Define if the response should be stored in the cache or not Boolean __INLINE_CODE_26__
__INLINE_CODE_27__ A value to define if the request should retry on failure. If value is a function, it will get evaluated on retry Function/Boolean __INLINE_CODE_28__
__INLINE_CODE_29__ A number to define how many times the request should retry Number __INLINE_CODE_30__
__INLINE_CODE_31__ Define any headers that you many required Object __INLINE_CODE_32__
__INLINE_CODE_33__ Define any URL parameters to be sent with a GET Object __INLINE_CODE_34__
__INLINE_CODE_35__ Define any data to be sent with a POST Object, FormData, File, Blob, Stream __INLINE_CODE_36__
__INLINE_CODE_37__ Send HTTP Basic auth credentials. This will set a __INLINE_CODE_38__ header Object __INLINE_CODE_39__
__INLINE_CODE_40__ Indicate what type of data the response will carry String __INLINE_CODE_41__
__INLINE_CODE_42__ The name of the cookie to use as a xsrf token String __INLINE_CODE_43__
__INLINE_CODE_44__ The name of the header to use as a xsrf token String __INLINE_CODE_45__
__INLINE_CODE_46__ A function that is called with the progressEvent of an upload Function __INLINE_CODE_47__
__INLINE_CODE_48__ A function that is called with the progressEvent of a download Function __INLINE_CODE_49__
__INLINE_CODE_50__ A number that defines the maximum length of the response content Number -
__INLINE_CODE_51__ A number that defines the maximum number of redirects (Node.js only) Number __INLINE_CODE_52__

Example:

__CODE_BLOCK_6__

__INLINE_CODE_53__

This method will set up the fetch instance with a cache.

If you wish to use caching and want something a bit more elaborate than in-memory caching

Example:

__CODE_BLOCK_7__

__INLINE_CODE_54__

Schema exports all of the symbols that we use to run specific logic on properties:

Property Description Type Default
__INLINE_CODE_55__ Used to indicate to the schema mapping what the output __INLINE_CODE_56__ should be __INLINE_CODE_57__ -
__INLINE_CODE_58__ The value to be used if there is no data for this specific property __INLINE_CODE_59__ -
__INLINE_CODE_60__ A function that gets evaluated, the first parameter is the data of the property being evaluated __INLINE_CODE_61__ -
__INLINE_CODE_62__ Used to define properties that may not exist on the object. The parent of the property is passed as a property __INLINE_CODE_63__ -
__INLINE_CODE_64__ If this child property is not present, then the object will be removed from the result __INLINE_CODE_65__ -
__INLINE_CODE_66__ Used to rename the field to a new field, the value of the symbol is the name of the new field __INLINE_CODE_67__ -
__INLINE_CODE_68__

This will parse the data using the model supplied, a new object will be returned.

__INLINE_CODE_69__

Define that the object should include this property and it should be a JavaScript type (Not implemented yet)

__INLINE_CODE_70__

The default value to use for a property.

Example:

__CODE_BLOCK_8__

Once parsed, the new data object will contain a product with an id, name, sku and a description of 'One of our products'

__INLINE_CODE_71__

Transform the object using the data object as a property.

Example:

__CODE_BLOCK_9__

This will return an object with the products array, any obj in the product array that has a sku will be transformed toUpperCase()

__INLINE_CODE_72__

If you there is no property by the name of what you want on the object, you can generate one by using the __INLINE_CODE_73__ Symbol.

Example:

__CODE_BLOCK_10__

This will add a __INLINE_CODE_74__ property to the objects in the array.

__INLINE_CODE_75__

If specific data is required, and the object is pointless without it, you can use the required property.

Example:

__CODE_BLOCK_11__

This will make sure that any objects in the products array will contain a name, in the above example, the products array will contain 1 object.

TODO

  • Schema mapping
  • Schema type transformation
  • Fetch API
  • Fetch setup to allow for retries, timeouts, bailouts
  • Pre-fetch caching
  • Post-fetch caching