@sap/dwf-ndso-backend v2.3.3

Native DataStore Object (NDSO) Service Backend

This Node.js package contains service implementations (tasks) of the Native DataStore Object. The NDSO backend is part of the SAP HANA Data Warehousing Foundation product.

As such it is used, for example, by the DataStore Manage UI of the Database Explorer or the Data Warehouse Scheduler.

Change Log

Be aware that as this being a technical reuse package you are not supposed to consume it directly in a custom application. It is used by the aforementioned SAP products/tools though.

NDSO Task Reference

NDSO tasks are exposed as simple asynchronous functions

activate(tracer, client1, client2, schema, ndso, loadIds, callback) ⇒ Promise

Moves data from inbound queue(s) to active data and change log. Writing of active data and change log is done by the HANA core procedure SYS.DSO_ACTIVATE_CHANGES.

This operation takes care of

• verifying the provided load IDs
• setting status of earlier, failed activations to 'DELETED'
• compiling procedure options like aggregation behavior and before-image handling
• calling the procedure
• deleting load data from inbound queue(s)

Fulfills: ActivationResult

addSubscriber(tracer, client1, client2, schema, ndso, subscriberName, description, callback) ⇒ Promise

Adds a new subscriber entry to the subscribers entity

Fulfills: OperationResult

checkMetadataConsistency(tracer, client1, client2, schema, ndso, callback) ⇒ Promise

Performs consistency checks on the NDSO. The operation fails, if inconsistencies are found

Performed checks ensure:

• Inbound queues consistency
• Change log entries deleted for cleaned up activations
• Inbound queue entries deleted for finished activations
• Affected requests consistency

Fulfills: OperationResult

cleanupChangelog(tracer, client1, client2, schema, ndso, requestIds, callback) ⇒ Promise

Cleans up change log. The operation verifies, if the provided activation request IDs can be cleaned up and then deletes them from the change log

Fulfills: OperationResult

cleanupMetadata(tracer, client1, client2, schema, ndso, maxRequestId, maxTimestamp, callback) ⇒ Promise

Cleans up metadata up to either the provided maximal activation request ID or corresponding creation timestamp (whichever is higher).

Fulfills: OperationResult
Todo

• not implemented, i.e., it does not actually delete any data

deleteAll(tracer, client1, client2, schema, ndso, callback) ⇒ Promise

Deletes all NDSO data (main and metadata entities). The operation fails, if there are subscribers

Fulfills: OperationResult

deleteRequest(tracer, client1, client2, schema, ndso, requestIds, callback) ⇒ Promise

Deletes load requests. The operation verifies, if the provided request IDs can be deleted and then deletes them from the inbound queue(s)

Fulfills: OperationResult

deleteWithFilter(tracer, client1, client2, schema, ndso, queryOptions, propagateDeletion, callback) ⇒ Promise

Deletes active data by the provided query filter. Optionally writes an activation request into the change log which allows rollback

Fulfills: DeleteResult

getDataStores(tracer, client, schema, queryOptions, callback) ⇒ Promise

Provides a list of NDSOs in the schema

Fulfills: DataStores

getDataStoreFeature(tracer, client, schema, callback) ⇒ Promise

Provides information if the NDSO metamodel exists in the schema. The metamodel is a prerequisite to model NDSOs

Fulfills: DataStoreFeature

getLogForOperation(tracer, client, schema, ndso, operationId, callback) ⇒ Promise

Provides the message log for an operation

Fulfills: LogForOperation

getMetadata(tracer, client, schema, ndso, callback) ⇒ Promise

Provides NDSO metadata

Fulfills: Metadata

getMonitoringOverview(tracer, client, schema, queryOptions, callback) ⇒ Promise

Provides monitoring relevant information about all NDSOs in the schema.

Fulfills: MonitoringOverview

getOperationInfo(tracer, client, schema, ndso, queryOptions, callback) ⇒ Promise

Provides general request information

Fulfills: OperationInfo

getOperationsForRequest(tracer, client, schema, ndso, requestId, callback) ⇒ Promise

Provides the list of operations that affected a specific request

Fulfills: OperationsForRequest

getRequestInfo(tracer, client, schema, ndso, queryOptions, callback) ⇒ Promise

Provides general request information

Fulfills: RequestInfo

getRequestsForActivation(tracer, client, schema, ndso, maxRequestId, callback) ⇒ Promise

Provides requests that can be activated. Condition:

• finished load requests
• after latest activation
• not subject to a deleteRequest operation
• (optional) load request ID lower than provided maxRequestId

Fulfills: RequestsForOperation

getRequestsForCleanup(tracer, client, schema, ndso, maxRequestId, maxTimestamp, callback) ⇒ Promise

Provides requests that can be cleaned up. Condition:

• finished activation requests
• not subject to rollback/clean-up operation
• extracted by all subscribers
• (optional) activation request ID less than or equal to provided max ID or timestamp

Fulfills: RequestsForOperation

getRequestsForDeletion(tracer, client, schema, ndso, callback) ⇒ Promise

Provides requests that can be deleted. Condition:

• load request failed OR finished and after latest activation
• not subject to a deleteRequest operation

Fulfills: RequestsForOperation

getRequestsForRollback(tracer, client, schema, ndso, minRequestId, callback) ⇒ Promise

Provides requests that can be rolled back. Condition:

• finished activation requests
• after latest rollback/clean-up operation
• not yet extracted by any subscriber (as rollbacks are just deleted from change log, subscribers won't get a corresponding delta corrupting their data)
• (optional) load request ID greater than or equal to provided minRequestId

Fulfills: RequestsForOperation

getRowcountWithFilter(tracer, client, schema, ndso, queryOptions, callback) ⇒ Promise

Provides number of rows in active data filtered by the provided query filter

Fulfills: RowcountWithFilter

getSubscribers(tracer, client, schema, ndso, callback) ⇒ Promise

Provides a list of subscribers

Fulfills: Subscribers

removeSubscriber(tracer, client1, client2, schema, ndso, subscriberName, callback) ⇒ Promise

Removes a subscriber from the subscribers entity

Fulfills: OperationResult

repairRunningOperations(tracer, client1, client2, schema, ndso, callback) ⇒ Promise

Repairs running operations that are not really running anymore and sets their status to failed. This is done by attempting to retrieve a lock for the corresponding entry in the operation history

Fulfills: OperationResult

resetSubscriber(tracer, client1, client2, schema, ndso, subscriberName, callback) ⇒ Promise

Resets maxRequestId of a subscriber in the subscribers entity back to 0

Fulfills: OperationResult

rollback(tracer, client1, client2, schema, ndso, activationIds, callback) ⇒ Promise

Restores data from the change log back to active data. Writing of active data and change log is done by the HANA core procedure SYS.DSO_ROLLBACK_CHANGES.

This operation takes care of

• verifying the provided activation IDs
• compiling procedure options like aggregation behavior and before-image handling
• calling the procedure

Fulfills: OperationResult

storeCsv(tracer, client1, client2, schema, ndso, iqName, data, withHeader, callback) ⇒ Promise

Loads CSV data into the provided inbound queue.

If the data contain a header row, it is excluded from data but serves as field list for the INSERT statement.

Data is generally loaded by just providing the data unaltered to the prepared statement except for binary data types (VARBINARY, BLOB) where a hexadecimal string is expected instead

Fulfills: LoadResult

storeSql(tracer, client1, client2, schema, ndso, iqName, externalSql, callback) ⇒ Promise

Loads data into the provided inbound queue by the FROM clause provided via externalSql.

externalSql needs to provide technicalAttributes.recordMode and all semantical column names as the inbound queue (if the source table has different column names, they must be aliased accordingly).

Fulfills: LoadResult

Example

 SELECT
   ''    AS "technicalAttributes.recordMode",
   CUST  AS "CustomerName",
   PRICE AS "Amount"
 FROM CUST_PRICES

smokeTest(tracer, client1, client2, schema, ndso, callback) ⇒ Promise

Performs a smoke test on the provided NDSO, executing all tasks, beginning with operations related to subscribers, then deletes all of them and runs deleteAll to have a defined start-state

:warning: Only use this in test environments!

Fulfills: OperationResult

ActivationResult

Properties

DeleteResult

Properties

DataStores

Properties

DataStoreFeature

Properties

LogForOperation

Properties

MetadataMetaTable

Properties

MetadataActiveDataField

Properties

MetadataChangeLogField

Properties

MetadataInboundQueueField

Properties

MetadataProcedure

Properties

Metadata

Properties

MonitoringTableStats

Properties

MonitoringOverview

Properties

OperationInfo

Properties

OperationsForRequest

Properties

RequestInfo

Properties

RowcountWithFilter

Properties

Subscribers

Properties

OperationResult

Properties

LoadResult

Properties

QueryOptions

Some tasks offer a parameter queryOptions that allow for backend-side paging, sorting and filtering. The task expects it to be an object like this:

Example

{
  lim: 100,             // LIMIT in SQL: returns up to 100 rows
  off: 200,             // OFFSET in SQL: skips first 200 rows
  flt: {                // builds WHERE in SQL; multiple columns are connected
    COLUMN1: 'value1',  // by AND
    COLUMN2: 123,
    COLUMN3: [          // array elements for same column are connected by OR
      {
        op: 'BT',       // BT resolves to 'BETWEEN ? AND ?'
        val: [10, 20]
      },
      {
        op: 'EQ',       // EQ resolves to 'IN (?,?,?)'
        val: [95, 99, 101]
      },
      {
        op: 'LT',      // LT ('less than') resolves to '< ?'
        val: 50        // there's also LE ('less or equal')
      },
      {
        op: 'GT',      // GT ('greater than') resolves to '> ?'
        val: 'abc'     // there's also GE ('greater or equal')
      }
    ]
  },
  fltMode: 'PARTIAL',   // instead of 'WHERE = ?' this resolves to 'WHERE LIKE ?'
                        // and wildcards around value (e.g. '%value1%')
                        // default is 'EXACT'
  ord: [                // builds ORDER BY in SQL
    {
      col: 'COLUMN1'    // defaults to 'ASC'
    },
    {
      col: 'COLUMN3',
      dir: 'DESC'
    }
  ]
}

RequestsForOperation

Properties