@civfund/fund-libraries v1.1.64

CIV-Fund Library

Updated to version 1.1.53

Introduction

An awesome npm module made by Civilization Fund.

Install · Usage

Initialize CCXT · Execute an Order · Define a Bucket · Ccxt Functions · Price Monitor · Watch Orders · Autosave · Mongoose · Test

Install

To use @civfund/fund-libraries in your code, first install it using npm:

npm install @civfund/fund-libraries

Usage

Then, require it in your code and use it like this:

const civfund = require('@civfund/fund-libraries')

civfund.doSomethingAwesome()

for a quick check that the installation was succesfull do this:

console.log(civfund.version)

the current version should be printed on the console.

Initialize CCXT

To initilize a ccxt instance simply require @civfund/fund-libraries in your file like this:

const civfund = require('@civfund/fund-libraries')

then it's possibile to open an exchange instance using the initializeCcxt or initializeProCcxt functions in different ways depending on which credentials you want to use.

It can be instantiated by using a subaccount name if registered or using Api keys as input.

N.B. Exposing API keys on a file can pose a significant security risk, as it can allow unauthorized access to the API and the resources it protects. Store API keys and passwords in a .env file.

1. Using a registered subaccount:

const exchange = civfund.initializeCcxt(cex, subaccount = 'Test')

2. Input API and secret:

const exchange = civfund.initializeCcxt(cex, subaccount = 'None', api = process.env... or 'Aak1e..', pwd = process.env... or 'Aak1e..')

Execute Order

Initialize a ccxt instance, if used multiple times in the same block just call it once and reuse it as many time as you want. This will avoid errors and will be more efficient.

const exchange = civfund.initializeCCxt(cex, subaccount = 'Test')

civfund.ccxtSyncroCreateOrder(cex, exchangeName, symbol, type, side, amount, price, clientId, stopPrice)

Based on the "type" this function will execute every kind of order available on Binance with a clientOrderId. For a market Order a stopPrice is not required.

Similarly to edit or cancel an order you can call the relevant methods:

civfund.ccxtSyncroCancelOrder(cex, pair, orderID)

civfund.ccxtSyncroEditOrder( cex, exchangeName, orderID, symbol, type, side, amount, price, clientId, stopPrice)

beign syncronous these functions can only return true.

Global Object

This class will instantiate an object with relative methods to update it keeping your code clean.

const mainListOfGrids = new civfund.Bucket(object)

The following methods will be avaialable:

mainListOfGids.property

to access one of the properties as usual;

mainListOfGids.replacePropertyValue(property, value)

this will edit a property, if not present it will be added;

mainListOfGids.replaceAllPropertyValues(values)

whatever is entered will be added or replaced to the object, the values can be an object, an array or a simple variable. If it is a key, value pair then it will be added by sorting the key to the right place (for example, if we have a grid and add a certain gridLevel, it will be sorted accordingly);

mainListOfGids.replaceObject(object)

it will copy another object, keep in mind that this method will still reference to the original object so both will be updated if they change in the future.

More methods will be added soon...

CcxtFunctions (Legacy)

As an example:

civfund.ccxt.ccxtFetchOrders(cex, pair)

Price Monitor

The price monitor is a class structure that allows you to instantiate all desired connections atomically, but for now binance "last" and "mark", and OKX prices are supported. You can use as many pairs as are supported from the exchange. It will come with a lot of functionality already.

The PriceMonitor class can be used to emit prices with custom functions or to trigger an action when a specified price is reached. When a list of prices is provided, the PriceMonitor class functions as a trigger. A list is a simple array of prices, at each index a certain information is assosiated, which can be a function as triggering an order, or a variable to update in some of the data structures (mainly objects).

The PriceMonitor class requires the exchange upon instantiation as this will specify from which exchange (es. Binance, OKX, etc...) you will receive updates.

The PriceMonitor class can be instantiated in one of five ways:

• When a list of prices is provided, the PriceMonitor class functions as a trigger, emitting an event when the specified price is reached. Example:

const monitor = new civfund.PriceMonitor('last', 'ETH/USDT', 'binanceusdm', list)

• When a list is not provided, the PriceMonitor class can be set to function as a trigger by providing a list at a later time using the 'setList' function. Example:

const monitor = new civfund.PriceMonitor('last', 'ETH/USDT', 'binanceusdm')

monitor.setList(list)

additionally it is possible to generate a priceList around the current price by providing the price, diff, and range values. Example:

const list = monitor.createPriceList(this.price, 0.01 , 80)

monitor.setList(list)

in this case a list of 161 numbers spaced by 0.01$ will be generated.

• When the emitter flag is set to true, the PriceMonitor class will function as a price emitter, emitting prices with custom functions on top. Example:

const monitor = new civfund.PriceMonitor('last', 'ETH/USDT', 'binanceusdm', null, true)

• The PriceMonitor class can be set to function as both a price emitter and a trigger by providing a list and setting the emitter flag to true. Example:

const monitor = new civfund.PriceMonitor('last', 'ETH/USDT', 'binanceusdm', list, true)

• Alternatively, the list can be provided at a later time:

const monitor = new civfund.PriceMonitor('last', 'ETH/USDT', 'binanceusdm', null, true)

monitor.setList(list)

Additionally:

• The PriceMonitor class is designed to allow users to define triggers based on a provided price list.
• The price list can be obtained from a file system, an external source, or by simply providing an array as input.
• The class has built-in event emitter and listener functionality that allows users to define and execute actions when specific price conditions are met.
• When a trigger is activated, the class will emit an event that includes the current price and its corresponding index in the price list.
• This allows users to easily relate their actions to specific price levels.
• The class also provides a simple method for retrieving the current price at any time.

The PriceMonitor class allows users to register listeners that will execute specific actions when a trigger is activated. Triggers are automatically defined based on the provided price list and are immediately available for monitoring and event triggering. Every time the price reaches or crosses a specified level, the registered listeners will execute their corresponding actions. The class will also automatically update its triggers as needed.

To use the event emitter functionality, users can register a listener like this:

ethusdtMonitor.on('trigger', (data) => { dosomething()})

When a trigger is activated, the listener will receive an object containing information about the trigger event. This object includes the index of the price in the price list that was crossed, the price received, and the side of the boundary that was hit (either 'upperbound' or 'lowerbound').

For example, if the trigger was activated when the price crossed the upper bound of a price in the list, the data object would contain the index of that price in the list, the current price, and the string 'upperbound'.

To retrieve the current price at any time, users can use the getPrice method like this:

ethusdtMonitor.getPrice()

When emitter mode is enabled, a 'price' event will be available for users to consume like this:

ethusdtMonitor.on('price', (price) => { dosomething()})

Workers Update

Starting from version 1.1.17 this class can be instantiated with tasks. This means that if a list of tasks in the rigth format is provided the price monitor instantiated will execute the relative tasks on a sperate thread when a certain price level is triggered. These tasks will be executed in parallel and when completed the worker will be automatically closed so that the core it was using is available again for other tasks.

Tasks Format

The task dataset consists of an array of objects, which can also be referred to as an "order list," since the main purpose of tasks within this module is to execute orders at a particular price level. This means that the number of elements of the array must be equal to the number of element in the price list in order to function properly. Also to each index of the price list corresponds a particular object of the order list or tasks array that will be executed on a separate thread.

An example of "order list" migth come as the following:

const orderList = [

{
    "name": "Test task",
    "run": "() => { console.log('Test1') }"
},

...

]

const monitor = new civfund.PriceMonitor('last', 'ETH/USDT', 'binanceusdm', priceList , true, orderList)

The Tasks field can also be set at a later stage by calling the 'setTasks' function. Example:

monitor.setTasks(orderList)

Start and Stop

In case the Price Monitor needs to be stopped it is possible to do so by calling the close() method. In this way the websocket settings will remain in memory and it will be possible at a later stage to restart the same price monitor by calling the open() method.

ex:

monitor.close()

monitor.open()

Events

Starting from v 1.1.28 the console.log will not be available anymore. You'll be able to get updates from the PricMonitor class by use of events listeners. Example:

monitor.on('opened', (message) => { console.log(message) }})

monitor.once('close', (message) => { console.log(message) }})

monitor.once('initialized', (message) => { console.log(message) }})

monitor.on('trigger', (message) => { console.log(message) }})

monitor.on('price', (message) => { console.log(message) }})

monitor.on('taskExecuted', (message) => { console.log(message)}})

Promised Orders

From version v1.1.58 the Price Monitor is extended with the ccxt creatr, edit and cancel order functions. These are a wrapped version of the ccxtSyncro but with the use of events it is will also make the code able to get the responses of each order execution which was not possible in the previous versions of the module.

monitor.on('success', (resObject) => { console.log(resObject) }})

monitor.on('error', (error) => { console.log(error) }})

In this way by defining two additional listeners it is possible to obtain 'success' objects, the object response from the exchange, or in case of error the error object will be returned via the 'error' event with data useful for monitoring or to make additional code based on the error. By default the priceMonitor will immeditaley attempt the order again with the same original parameters up to number of maxRetries. By default this value is 5, it can be adjusted by doing:

monitor.setMaxRetries(10)

or 0 to only return the error with no retries:

monitor.setMaxRetries(0).

In this way the execution of orders will be non-blocking and the main loop will process other code, so all promises by the order executions will be handled in parallel and managed by events.

Example

const cex = civfund.initializeCcxt('binanceusdm', 'sandbox')

priceMonitor.ccxtCreateOrder(cex, 'binanceusdm', 'ETH/USDT', 'market', 'buy', 1)

Watch Orders

The Watch Order Stream class provides an atomic structure for instantiating multiple authenticated connections. Currently, Binance and OKX are supported. The class allows you to use all available pairs and as many subaccounts as your .env file provides. The class comes with many built-in features, such as:

• The Watch Orders class allows users to instantiate multiple authenticated connections to Binance exchanges in an atomic fashion: it supports all available pairs and subaccounts.
• The class includes built-in features for efficient and effective stream management.
• The Watch Orders class has a built-in event emitter and listener module that is easy to use.
• When an order update is received from Binance, the class will emit an event that can be consumed to update strategy objects as needed.
• The class automatically updates three different types of variables with each order update received from Binance.
• These variables can be accessed and consumed at any time using a simple method.
• The class provides a simple method for retrieving the updated data at any time.
• The Watch Orders class will automatically reconnect in case of network outages and continue to monitor orders as usual once the connection is restored.

const civfund = require ('@civfund/fund-libraries')

const ethusdtWatcher = new civfund.WatchOrders(cex, subaccount = "None", symbol)

ethusdtWatcher.on('order', (order) => { dosomething()})

ethusdtWatcher.getOrders()

will print the list of the latest 40 orders. When an order is edited/closed/etc... it will be updated an transferred downstream;

ethusdtWatcher.getOrders('open')

will print the list of open orders;

ethusdtWatcher.getOrder()

will give the latest order sent from binance.

Workers Update

Starting from version 1.1.17 this class will run order updates on a separate thread. In this way the stream will not be blocked by any additional operation apart from receiving orders and sending those to the new thread so it will continuosly watch for new websocket updates. As soon as the thread is done with its task it is automatically closed and a the core used will be again free.

Start and Stop

Is it possible to stop the watching loop by runnng the stop() method. The instantiated watchOrder class will stay in memory so it will be possible at a later stage to run the loop again and continue to watch orders with the same settings.

Api and Secret

Is it now possible to instantiate a watchOrder class even if the subaccount is not already registered using Api keys as input.

N.B. Exposing API keys on a file can pose a significant security risk, as it can allow unauthorized access to the API and the resources it protects. Store API keys and passwords in a .env file.

const ethusdtWatcher = new civfund.WatchOrders(cex, subaccount = "None", symbol, api = process.env... or 'Aak1e..', pwd = process.env... or 'Aak1e..')

Autosave

The Autosave class allows you to automatically save and backup an entire MongoDB collection.

To use it, instantiate the class and specify an interval in milliseconds, the database name, the collection name, and a prefix for the variables to save:

const autoSave = new civfund.AutoSaveMongoDB('interval', 'databaseName', 'collectionName', 'prefix')

The Autosave class will look for all variables that start with the specified prefix and save their values at each interval. You can add or remove variables to autosave at any time by deleting them or adding new ones with keys that start with the prefix. For example:

• to add a variable:

autoSave.prefix_field1 = object1

autoSave.prefix_field2 = object2

in this case the prefix defined as an example is "prefix_".

• to delete:

delete this[key]

Note that if the value of one of the variable is dynamically changed within the server, the last value at each time interval will be replaced to MongoDB if an existent entry witht the same name and collection in the same database is found otherwise if not found it will be inserted.

Back Up

If the server goes down it is always possible to create a new instance of the AutoSaveMongoDB class and call the method loadData() to find all the documents in the collection. The returned variable will contain the array of documents returned by the loadData() method, and you can access the individual documents in the array by their index.

const docs = await autoSave.loadData()

For example, docs0 will be the first document in the array.

Start, Stop and Reset

To reset the class and delete all entries in the MongoDB collection, call stop() with the reset and resetDB parameters set to true. To load all documents in the collection, call the loadData() method on the class, which will return an array of the documents in the collection.

At any moment is possible to stop the process and clean up CPU call the stop() method on the class:

autoSave.stop()

the autosave process will be stopped but the class will remain instantiated and can be restarted at any time:

autoSave.start()

You can also change the interval by calling stop() with a new interval value:

autoSave.stop(20000)

the interval will be set at 20 seconds.

If it is needed to reset all the variable within the class object you can call the stop() method with the reset parameter set to true:

autoSave.stop('null', true)

To delete all entries on MongoDB for a particular collection and completely reset both the class and MongoDB collection call the stop() method with the "reset" and the "deleteAll" parameters set to true:

autoSave.stop('null', true, true)

Mongoose

The Mongoose module is a MongoDB object modeling tool designed to work in an asynchronous environment. We will use mongoose to define a schema and optimize the code when making request or uploading a document to MongoDB.

Usage

const { dbMongoose } = require('@civfund/fund-libraries')

Schema

A schema is a definition of the structure of a document, either completely or just a portion of it. Schemas are used to define models.

const schema = { name: String, age: Number, stories: { type: Schema.Types.ObjectId, ref: 'Story' } }

Concurrent Connections

The mongoose model of the Civfund libraries will establish a new connection and inherit the schema as soon as a new object is pushed to MongoDB with the insertOne function:

dbMongoose.insertOne('CIV-Fund', 'myCollection', 'myModel', myObject)

with myObject beign:

const myObject = { name: 'John Doe', email: 'john.doe@example.com' }

the inherited schema will be:

const schema = { name: String, email: String }

Each time a new schema is saved the connection will be stored with the key "dbName and collectionName and modelName" so to each collection of a given database there there could be multiple schema. In this way the number of collection is optimized.

The module will work with objects of all kind both with a defined schema and also without a schema already defined. Whne a schema is defined and subsequent objects follow the same schema it is advisable to reuse the model name to make the db request as efficient as possible by reusing both the connection and the schema. However functions like: getCollection and findOne can be called even without a defined schema, and can be used to fetch every kind of document from the collection.

So if the schema is not defined the connection will follow the basic mongoose query that doesn't require a schema allowing to more flexibility. If a model is already defined and the user is sure about the schema of the final documents to fetch using the moduleName parameter is strongly adviced to reduce the number of connection and keep the code as efficient as possible.

Instead instertOne and replaceOne must be called with a schema name, if the object kind is uploaded to the database for the first time simply provide a new model name to define a new schema for the connection.

SetURL

The url in the form:

mongodb://username:password@host:port/database

is required to authenticate to mongoDB with mongoose. As default the code will look for this url in the .env under the name "MONGODB_URL". In case this is not present it is possible to manually set the URL as an internal variable of the server by calling the function setURL. This function should be called before any other mongoose call in the code.

dbMongoose.setURL('mongodb://username:password@host:port/database')

Methods

InsertOne

• returns the same object if it is correctly uploaded to MongoDB. The code will check if any empty object is present and automatically assign a key value pair like: "default": "default" before uploading.
• the function will also check if the schema is already defined and if not it will create a new schema and a new connection to the database.

dbMongoose.insertOne('CIV-Fund', 'myCollection', 'myModel', myObject)

getCollection

• returns an array of all the documents in the collection restored to original (with empty objects instead of "default":"default").
• the function will also check if a connection to the database is already defined and if not it will create a new connection to the database.

dbMongoose.getCollection('CIV-Fund', 'myCollection')

findOne

• returns the first document in the collection that matches the query. The code will check if any empty object is present and automatically assign a key value pair like: "default": "default" before uploading.
• the function will also check if a connection to the database is already defined and if not it will create a new connection to the database.

dbMongoose.findOne('CIV-Fund', 'myCollection', 'filterFiled', 'filterValue', 'modelName')

if called like above and if the modelName already exist it will search only for object that follow the schema defined in the modelName thus optimizing the call. If not present or not specified like here:

dbMongoose.findOne('CIV-Fund', 'myCollection', 'filterFiled', 'filterValue')

the code will search for any kind of object in the collection.

replaceOne

• returns the replacement object that is replaced to MongoDB. The code will check if any empty object is present and automatically assign a key value pair like: "default": "default" before uploading.
• the function will also check if the schema is already defined and if not it will create a new schema and a new connection to the database.

dbMongoose.replaceOne('CIV-Fund', 'myCollection', 'myModel', myObject, replacementObject)

deleteAll

• returns the number of documents deleted in the collection.
• the function will also check if a connection to the database is already defined and if not it will create a new connection to the database.

Close Connections

When a mongoDb connection is not needed anymore it is possible to delete a single connection by calling:

await dbMongoose.closeConnection('CIV-Fund')

this async method will close the connection and remove all the models connected to the connection closed and return a true promise.

Otherwise it is possible to close all connections at once by calling:

await dbMongoose.closeAllConnections()

this async method will close all the connections and remove all the models connected to the connections closed.

Test

A full unit testing environment is avaiable in the test folder. First, make sure your .env file is properly set then simply run "npx jest testScriptName.test.js" to run a single suit matching the name provided or "npm test" to run all the suites and debug the module.