@deeptrekker/api-domain NPM

DeepTrekker BRIDGE Web API

💡 Read the service disclaimer before using the API.

Intro

The BRIDGE web API is a programming interface we've developed that can be utilized by applications, either internal or external to Deep Trekker, for standardized communication with across devices within the Deep Trekker ecosystem. It is essentially a common language that our devices can understand and interpret, enabling them to carry out precise instructions, or send commands over to other connected devices for execution.

This documentation describes the core types, schemas, enumerations, and construct definitions used by client-side applications and server side devices to communicate with one another. Furthermore, it also describes the various supported connection methods and means of communicating data. Altogether, this forms the entirety of the DeepTrekker API.

💡 The API documentation and samples found here are generated directly from types and interface definitions from the actual domain code that some of our newer client-side applications make use of. This ensures that both the code and documentation remain up-to-date and stem from a single source of truth.

Communication

This API supports powerful bi-directional communication between servers and clients for relaying vehicle data. Under the hood, we are using the most versatile technologies and standards available in order to accommodate communication across a variety of different technological stacks.

Architecture

The following diagram illustrates a comprehensive view of various devices that exist and communicate with one another within the BRIDGE web system. Arrows are drawn in to show the flow of data while connection methods are differentiated by color.

postman-wss-1

Connection Methods

The JSON-based API data that flows through channels is inherently agnostic, so any communication protocol could be used to convey Deep Trekker API data. However, we have implemented two specific methods of connectivity that are officially supported and compatible with our vehicles, controllers and other peripheral devices.

WebRTC Connection

The WebRTC (Web Real-Time Communication) Connection API can be used for conveying JSON-based payloads in a web browser-based development environment. It provides several features that make it suitable for transmitting data over networks, with built-in encryption and optimization for variable network qualities. This is a modern web technology that powers peer-peer communication which can endure varying degrees of latency and interruptions throughout session and send data in ne. To start a session, clients initially have to negotiate via a https/websocket handshake with the signal server. Afterwards, they are free to communicate and exchange data in real time via numerous parallel data channels. In the temporary absence of the signalling server, existing client sessions can continue to live on.

Web Compatibility: The WebRTC Connection API is designed to work in web browsers, making it compatible with various platforms and operating systems. This allows developers to leverage its capabilities without worrying about platform-specific implementations.
Realtime communication: Realtime communication can be achieved via WebRTC. This is a modern web technology that powers peer-peer communication which can endure varying degrees of latency and interruptions throughout session and send data in ne. To start a session, clients initially have to negotiate via a https/websocket handshake with the signal server. Afterwards, they are free to communicate and exchange data in real time via numerous parallel data channels. In the temporary absence of the signalling server, existing client sessions can continue to live on.
Optimization for Variable Network Qualities: WebRTC is designed to adapt to different network conditions, allowing it to optimize data transmission for varying network qualities. It utilizes adaptive congestion control algorithms to adjust the transmission rate dynamically, optimizing the delivery of JSON payloads based on available bandwidth and network conditions.
Built-in Data Encryption: The WebRTC Connection API incorporates built-in encryption mechanisms to ensure secure transmission of data. It utilizes standardized encryption protocols such as Secure Real-Time Transport Protocol (SRTP) and Datagram Transport Layer Security (DTLS) to protect the confidentiality and integrity of the transmitted JSON payloads.
Support for Over-the-Internet Connections: The WebRTC Connection API is specifically designed for establishing real-time communication over the internet. It enables peer-to-peer connections between browsers, allowing data transmission directly between clients without the need for intermediate servers. This direct connectivity enables efficient transmission of JSON payloads across the internet, reducing latency and improving the overall user experience.

WebRTC Connection Architecture

Signal Server
To initialize a WebRTC communication room, a client would first need to contact the signalling server and negotiate a connection. This is done by performing a sequence of handshakes between a client and the signalling server. Afterwards, both the vehicle client and the controller clients can freely communicate with one another and sent/receive data.
💡 In the context of the DeepTrekker (WebRTC) API, we refer to the vehicle client of the Signal Server as the vehicle server.
SignalR
Clients can optionally integrate SignalR in client-side code to handle signalling negotiation and subsequent receive/send messages with a vehicle server. We use SignalR to encapsulate some of the complexity of WebRTC connections and use its protocols where applicable.
Documentation on how to integrate client-side SignalR code can be found here: https://docs.microsoft.com/en-us/aspnet/core/signalr/client-features?view=aspnetcore-3.1
💡 SignalR has "hubs" instead of "rooms" as a space for client communication. We will use the terms room and hub interchangeably throughout the documentation.
Manual Negotiation (No library)
For client side implementations that do not have access to a SignalR library, the following language agnostic handshake steps can be performed to establish a wss connection for communication with SignalR endpoints.
Please note that when populating the arguments field within websocket messages the value is a stringified json field matching the indicated object definitions (ie session_check, join_session, etc).
💡 Here is some documentation on websockets
Refer to the following steps that also include pictures illustrating how to perform the ceremony using Postman.
1. Send an http post request to signalling server (replace 'localhost' with the IP address of the BridgeBox)
  - URL: https://localhost:5001/sessionHub/negotiate?negotiateVersion=1
    - Query item: negotiateVersion=1 (included in URL above)
2. Get the "connectionToken" from the reply of the post request and use it to open a web socket to connect to signalling server (replace 'localhost' with the IP address of the BridgeBox)
  - URL: wss://localhost:5001/sessionHub?id= + connectionToken
  - Open web socket with the above URL (click "New" button beside "My workspace" bar in top left corner)
3. Send either of the following json objects as an initial payload via WSS protocol to connect to the HUB.
  - payload (in plain text)
    ⚠️ Ensure you copy this entire line including the invisible Record Separator "RS" ASCII character (\x1E) immediately following the "}" bracket:
```
{"protocol":"json", "version":1}
```
  - base64 encoded payload (includes the "RS" character)
```
eyJwcm90b2NvbCI6Impzb24iLCAidmVyc2lvbiI6MX0e
```
  💡 No other wss payloads will work until this one is sent first as the initial payload. You must include the invisible "RS" ascii character at the end of every payload thereafter as well. This is used to determine the end and start of different payloads. Read more about SignalR's use of the Record Separator.
4. After sending the initial json protocol payload, a payload response with an empty JSON object will be returned by the WSS endpoint.
  - The empty JSON message from server through web socket signals a successful connection to the signalling server
5. Send a session_check message. Be sure to include the "RS" ascii character at the end of any payload sent. After constructing your arguments json, stringify it and then construct the rest of the message as shown. This pattern will be followed in the rest of the sent messages.
  - Client requests list of sessions that are available to join
```
{
    "arguments": ["{\"client_id\": \"test\"}"],
    "invocationId": "0",
    "streamIds": [],
    "target": "session_check",
    "type": 1
}
```
6. Await a session_list message.
  - Client receives list of sessions that are available to them
```
{
    "type":1,
    "target":"session_info",
    "arguments":[{
        "session_id":"33555648",
        "connection_id":"ispG-PRx-G6BiFKrjGMn1A",
        "clients":[{}]
    }]
}
```
7. Send a join_session message.
  - Client requests to join an available session
```
{
    "arguments": ["{\"client_id\": \"test\", session_id:\"33555648\"}"],
    "invocationId": "0",
    "streamIds": [],
    "target": "join_session",
    "type": 1
}
```
8. WebRTC SDP negotiation will be initiated by the backend by sending an offer and ice candidates

WebRTC SDP Negotiation

The next part of the negotiation involves negotiating the SDP. A good video example can be found here.

💡 Ensure the JSON payloads are structured with correctly formatted values for their types. Even if it looks like a number but the interface has it as type string, it should still be wrapped in double quotes when sent in a payload to ensure error-free deserialization by strongly-typed languages.

Await a session_info message.

Client receives requested session information, should see themselves in the session_info

{
    "type":1,
    "target":"session_info",
    "arguments":[{
        "session_id":"33555648",
        "connection_id":"ispG-PRx-G6BiFKrjGMn1A",
        "clients":[{
            "client_id":"test",
            "connection_id":"odQS6Kiei5y0AaaoZdO5Pw"
        }]
    }]
}

Await an offer message.

Client receives the offer from the vehicle

{
    "type":1,
    "target":"offer",
    "arguments":[
        "{\n  \"target\": \"odQS6Kiei5y0AaaoZdO5Pw\",\n  \"caller\": \"ispG-PRx-G6BiFKrjGMn1A\",\n  \"sessionId\": \"33555648\",\n  \"sdp\": {\n    \"type\": \"offer\",\n    \"sdp\": \"v=0\\r\\no=- 272599448504607653 2 IN IP4 127.0.0.1\\r\\ns=-\\r\\nt=0 0\\r\\na=msid-semantic: WMS\\r\\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\\r\\nc=IN IP4 0.0.0.0\\r\\na=ice-ufrag:H9aZ\\r\\na=ice-pwd:aGo/p9pm1eUS1Aw2qzbgYuQy\\r\\na=ice-options:trickle\\r\\na=fingerprint:sha-256 2D:85:37:C8:CD:64:56:11:7E:A3:65:0D:E4:76:DB:70:AA:05:5B:D9:63:4C:04:24:F2:31:83:DB:BD:C6:6D:EF\\r\\na=setup:actpass\\r\\na=mid:0\\r\\na=sctp-port:5000\\r\\na=max-message-size:262144\\r\\n\"\n  }\n}"
    ]
}

Send an answer message.

Client sends their answer to the signaling server

{
    "type":1,
    "target":"answer",
    "invocationId":"3",
    "arguments":[
        "{\"caller\":\"odQS6Kiei5y0AaaoZdO5Pw\",\"sdp\":{\"sdp\":\"v=0\\r\\no=- 6398202607803557374 2 IN IP4 127.0.0.1\\r\\ns=-\\r\\nt=0 0\\r\\na=msid-semantic: WMS\\r\\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\\r\\nc=IN IP4 0.0.0.0\\r\\na=ice-ufrag:heL6\\r\\na=ice-pwd:VasvuVpgy6Alpi9q4np1OGWX\\r\\na=ice-options:trickle\\r\\na=fingerprint:sha-256 66:D8:E7:9F:35:2C:68:4E:D8:01:30:F2:49:A4:D4:81:55:AB:55:8E:15:69:C2:4D:71:48:78:52:22:57:12:98\\r\\na=setup:active\\r\\na=mid:0\\r\\na=sctp-port:5000\\r\\na=max-message-size:262144\\r\\n\",\"type\":\"answer\"},\"sessionId\":\"33555648\",\"target\":\"ispG-PRx-G6BiFKrjGMn1A\"}\n"
    ]
}

Send and Receive a ice_candidate messages.

Client and Vehicle sends their ice candidates to each other and responds with their ice-candidates

Receiving

{
    "type":1,
    "target":"ice_candidate",
    "arguments":[
        "{\n  \"target\": \"odQS6Kiei5y0AaaoZdO5Pw\",\n  \"caller\": \"ispG-PRx-G6BiFKrjGMn1A\",\n  \"sessionId\": \"33555648\",\n  \"candidate\": {\n    \"sdpMid\": \"0\",\n    \"sdpMlineIndex\": \"0\",\n    \"content\": \"candidate:557811690 1 udp 2122260223 10.77.100.1 34423 typ host generation 0 ufrag H9aZ network-id 1 network-cost 50\"\n  }\n}"
    ]
}

Sending

{
    "type":1,
    "target":"ice_candidate",
    "invocationId":"4",
    "arguments":[
        "{\"caller\":\"odQS6Kiei5y0AaaoZdO5Pw\",\"candidate\":{\"content\":\"candidate:1 1 UDP 2122317823 192.168.88.254 56130 typ host\",\"sdpMid\":\"0\",\"sdpMlineIndex\":\"0\"},\"sessionId\":\"33555648\",\"target\":\"ispG-PRx-G6BiFKrjGMn1A\"}\n"
    ]
}

Run application logic and utilize established Webrtc peer connection to communicate through the API (data channel DATA_CHANNEL_API).
Send a leave_session message when application wants to disconnect.
- Client requests to leave the session, other clients in the session and vehicle will receive updated session info
```
{
    "type":1,
    "target":"leave_session",
    "invocationId":"0",
    "arguments":["{\"client_id\":\"test\",    \"session_id\":\"33555648\"}\n"
    ]
}
```
A session_closed message will be broadcast to all clients if the vehicle server disconnects.

Video Channel
Video streams are transmitted via a dedicated WebRTC data channel between the vehicle server and client.
💡 We refer to this channel as the video channel.
- How to request a camera stream
  Locate the Camera section in the API and identify the camera you want to request the camera stream from. Additionally, there will be multiple streams under each camera. Each stream property has an active property. By setting the active property to true for a desired stream, a new video channel will be created with a stream ID matching the camera ID. This will request a camera stream from the vehicle. Set the active property of the previous camera stream to false to decrease bandwith consumption.
Data Channel
Another data channel exists which conveys our API messages between the vehicle server and client bidirectionally via a WebRTC connection. The payloads transmitted on this channel are structured based on the Deep Trekker API constructs shown in this document. The following sections describe these schemas and payloads in detail.

Direct Sockets Connection

The Direct Socket Connection API is a simpler method for conveying JSON-based payloads which is attractive in the sense that it does not require a handshake process to establish a connection. It provides a straightforward approach for transmitting data over a socket.

Socket Connection: The Direct Socket Connection API relies on socket connections, which are low-level communication channels between two endpoints. It allows direct and uninterrupted data transfer between the client and server without any intermediate layers or protocols.
Connection Establishment: Unlike protocols such as WebSocket, the Direct Socket Connection API does not involve a handshake process. In traditional socket communication, the client directly establishes a connection with the server using the server's IP address and port number. Once the connection is established, data can be sent and received without any additional negotiation steps.
Data Transmission: The Direct Socket Connection API provides a means to send and receive JSON payloads over the established socket connection. The data is typically serialized into a string representation, adhering to the JSON format. The sender converts the JSON payload into a string and transmits it through the socket, while the receiver parses the received string back into JSON for further processing.

It's important to note that the simplicity of the Direct Socket Connection API means that it lacks some of the higher-level features found in more complex protocols. For example, it does not include built-in encryption, reliability mechanisms, or support for handling network issues such as congestion or packet loss.

Expected Architecture

It is expected that a 1:1 relationship exists between a peer connection and a front-end client. Clients with multiple peer connections are not supported. It is expected that one or more data and video channels are to be used within a single peer connection. A data channel named "DATA_CHANNEL_API" will be opened on connection which facilitates api communications. Through the api the client may operate vehicle controls, receive live data, and request video streams.

Upon request of a video stream, the peer connection will be renegotiated to support the requested configuration. Depending on the hardware in use this has the potential to require too many resources from the backend and cause a crash. It is suggested to keep requested video to 2 1080p streams at a maximum (on currently released hardware as of Feb 2023). If more streams are required, the lower resolution substreams may be used at the cost of visual quality.

Payload

A payload is a chunk of data that can be transferred during a peer-peer WebRTC connection (vehicle server - client). This is conveyed via the data channel. For a payload to be valid, it must be a subset of the vehicle state and nested inside a TransferPayload object. We use JSON as our data-interchange format and adhere to our generated API JSON Schema for validating payloads.

JSON Schema

A JSON schema is a structure that defines how payloads should be created. It can also be used by clients and servers alike to validate incoming payloads. The schema for the DeepTrekker API can be accessed and imported from the api-domain package under dist/schemas/schema.json.

💡 We use the Json Schema Validator to help construct payloads. To use this tool, simply copy and paste the DeepTrekker API JSON Schema on the left hand side, then proceed to write a TransferPayload on the right hand side. You can use this documentation as a human readable reference to help aid you in constructing the payload. As you write, the validator will provide feedback on the correctness of the input based on the schema you provided.

Structure

A valid payload is essentially a tree structure with the TransferPayload as the wrapper. The TransferPayload contains various meta data that goes along with a payload such as information about the API version compatibility and the status of the transfer.

💡 If a payload is wrapped in the TransferPayload, we refer to it as a request or response.

There are 5 different types of transferred payloads:

Requests
Response
- ERROR

Sample GET request:

{@codeblock ~~/dist/samples/json/requests/transferGetRequest.json}

Errors are a type of response that gets returned, either as a result of a bad payload request sent or for reasons unprovoked depending on the error code supplied.

The full list of errors and their details can be seen here.

Refer to the individual sections in the table of contents for further examples on various different kinds of payloads.

💡 Samples in these sections do not contain any real mac addresses or IDs, and so, they should not be used to refer to any real device. These are only there to guide you on how to implement the correct structure. However, properties that represent enumerations or constants are an exception this. In these cases, we've explicitly linked them in the documentation (e.g. model > DeviceModel)

Parameters

Some parameters in payloads exist solely to convey information in one direction. These parameters will have one of the following attributes labelled in their documentation:

Readonly (Server -> Client)
Parameters labeled as readonly have values written by the server with the intention of being read by clients.
Writeonly (Client -> Server)
Parameters labeled as writeonly have values that are written by a client with the intention of being read by the server.

💡 The server will ignore any readonly parameters that exist in paylaods sent by clients (it will not throw away the entire payload though).

Lists

In most cases, lists of objects are organized as dictionaries in payloads and the API. This is because these objects are semi-immutable and have a unique ID associated with them. With this structure in place, accessing and merging payloads becomes a simpler process.

For example, vehicle devices do not typically change without a reboot a system. Therefore, it is reasonably safe for client to cache their IDs in memory and then utilize them throughout their code for subsequent access:

    // Typescript

    // using a previously cached device id during a session.

    state.device[cachedDeviceId]

Transfer Frequency

Payload transfers can occur very quickly. There is no limit on how fast payloads can be sent between clients and servers. It is the responsibility of the client to throttle any requests that are received depending on their use case. The vehicle server is able to handle any number of requests at any frequency by calling timeouts or throttling mechanisms accordingly without any noticeable effect to the client.

Rate Property

The rate property exists in certain payloads objects. It is a writeonly property used to continuously update a particular property in which there is no deterministic state. Each time this is sent, it will incrementally adjust the value associated nearby it in the payload.

Maintaining the Vehicle state

In some cases, decoupling the state from the UI can be a useful design pattern to employ when building an application. This is especially true when utilizing declarative style libraries such as ReactJS and Redux. Since we create subsets of a vehicle state when forming payloads, it naturally makes sense to maintain the entire vehicle state (in memory or cache) and just merge incoming payloads into it. Managing an isolated state of the vehicle that UI logic can be wired up to would encourage a clean separation of concerns and should facilitate the rapid development of a reliable application that can communicate with DeepTrekker systems.

The following examples show a snapshot of vehicle states that could hypothetically exist in memory at some point during the operation of a vehicle:

{@codeblock ~~/dist/samples/json/states/a200RandomState.json}

{@codeblock ~~/dist/samples/json/states/a150RandomState.json}

{@codeblock ~~/dist/samples/json/states/revolutionRandomState.json}

{@codeblock ~~/dist/samples/json/states/pivotRandomState.json}

Integration

Refer to the documentation on how to integrate the DeepTrekker API within a client-side solution for communicating with DeepTrekker related hardware.

NPM Packages

@deeptrekker/api-domain
To conform with our api standard, developers should incorporate this package in their client-side JavaScript/Typescript applications. This package contains a schema that defines the precise structure that payloads should be created in that will be understood when parsed by vehicle servers. It also contains well-defined types and constructs that can be directly imported when developing using the same language to streamline the development process.
💡 Since the documentation, and its associated samples, stem from this package, it should be referred to as the single source of truth for all clients that implement our API.
Contents
Set up
This package can be installed using the following methods
1. Install NPM package manager.
2. Navigate to the root of your client application directory.
3. In the terminal, run npm init to initialize.
4. Run npm install @deeptrekker/api-domain.
5. If you're developing a JS/TS application, you can import the necessary types into your project. Otherwise, you can reference the json schema also provided for validating payloads and conforming to our API.
@deeptrekker/sdk
👷‍♂️ Under construction

⚖️ Service Disclaimer

The Deep Trekker API ("our API library") is provided as is, without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. We make no representations or warranties that the API library will meet your requirements or that its operation will be uninterrupted or error-free. We will not be liable for any damages of any kind arising from the use of the API library, including but not limited to direct, indirect, incidental, punitive, and consequential damages.

We do not provide technical support for integrating our API library into your software application. While we strive to make our API library easy to use and provide documentation and sample code, it is your responsibility to ensure that your software is compatible with our API library and that the integration is implemented correctly. We will not be liable for any issues or errors that arise from the integration of our API library into your software application.

@js-temporal/polyfill ajv dotenv lodash mkdirp ts-pattern ts-toolbelt type-zoo

4 days ago

5 days ago

10 days ago

1 month ago

2 months ago