forever-chat-format v1.2.1
Forever Chat Format
Why
Because in the world where our communication is siloed into Google Hangouts, Facebook, iMessage, WhatsApp, etc etc it's getting hard to hold onto conversations for archival purposes. First thing we need is a universal format that we can export those proprietary chats into that's easy to understand, and that can be relied upon.
Basic Format
Output should be an array of hashes, and each object in the array is a JSON object defined as follows:
Chat Object Definition
| Name | Type | Description |
|---|---|---|
| sha | string | sha1 of sender, receiver, date, text, service |
| sender | string | The handle of the person sending the message |
| receiver | string | The handle of the person receiving the message |
| is_from_me | boolean | true if you sent the message |
| send_error | boolean | true if the message didn't send |
| date | ISO-8601 datetime | date message was sent/received by sender |
| service | string | Service message was sent using. All lowercase, dasherized. 'imessage', 'google-hangouts', 'facebook-messenger', 'sms' |
| date_read | ISO-8601 datetime | (optional) date message was sent/read by receiver |
| date_delivered | ISO-8601 datetime | (optional) date message was delivered by receiver |
| participants | array | (see definition below) An array of strings of handles involved in the conversations. Used for grouping. For a two person conversation, this has two entries in it. Sender and receiver are always here. |
| message_text | string | text representation of the message |
| message_segments | array | (see definition below) ordered segments of content as it appears in the message |
| attachments | array | (see definition below) array of attachments |
| associated_sha | string | sha of message this message is in response to. Generally used for reactions |
| _debug | object | (optional) debug information to help trace back to the source of the data |
Message Segment Definition
| Name | Type | Possible Values | Description |
|---|---|---|---|
| type | string | text, link, file | base segment type |
| file_type | string | null, or a mime_type | only valid when type == file |
| path | string | url or file path | |
| text | string | the rendered name of the link, or the text content for the text, or optionally |
Text Segment
{"type": "text", "text": "Why hello there"}Link Segment
{"type": "link", "text": "Whoa, unreal", "path": "http://latlmes.com/technology/why-we-need-a-new-chat-format-1"}File Segment
{"type": "file", "file_type": "image/png", "path": "/path/to/image.png"}Reaction Segment
{"type": "reaction", "reaction_type": "laughed"}Attachments
Array of attachments in the order they were received. Each attachment has the following properties. This information should also be made available in message_segments.
| Name | Type |
|---|---|
| path | string |
| type | string |
Usage
The included tests check whether some json is valid forever chat format.
var foreverJson = importer("test.json"); // done by an exporter
var {validate, runTests} = require('forever-chat-format-tests');
runTests(foreverJson); // runs mocha tests on dataThe last step of your exporter should be to pass your exported data to the prepare function. It will check that the data conforms to the forever-chat-format spec and then return a hash with two keys: messages, and validations. Messages is all the data your exporter returned, and validations are what you see below:
"validations": {
"version": 1.2 // forever chat version
"checkedCount": 100 // number of records checked
"errorCount": 0 // number of records with errors
"recordsWithErrors":
"ruleResults": { /* each rule keyed by name with errorCount and "erroredRecords" */
"unique-sha": {
"description": "Each record should have a unique sha",
"errorCount": 0,
"erroredRecords": []
}
...
}
}Testing
var foreverJson = importer("test.json"); // done by an exporter
var {validate, runTests} = require('forever-chat-format-tests');
runTests(foreverJson);