rdp v1.0.1

= 迁

== Reactive Data Processing (RDP)

https://github.com/harsha-mudi/rdp is more active.

== Install

npm install rdp

== What is Reactive Data Processing (RDP) ?

RDP is a Data Flow Framework modelled after Breadboard Programming. +

Data       =>   Electricity
Software   =>   Chip Design

This perspective fascillitates ideas from +

• Electronics Engineering
• Systems Engineering

Currently Electronics have better

• Testability
• Longevity
• Reliability
• Maintainability
• Quality

than Software Products, which are perpetually new, improved and broken. +

RDP is built on Low Level Ideas from Electronics, in hopes of replicating its merits.

• Are Low Level Implementations more Readable ? +
• Is Data Flow like Electricity ? +
• Can Low Level ideas Scale ? +

== What are the APIs provided by RDP ?

=== Reactive

Components and Cells for Reactive UI. + A Message Bus for signal flow programming. +

This part shares some similiarities with Backbone Marionette.

=== Data

This gives you the following Abstractions

• Scalar - Number|String|Array|Date|Boolean|Symbol
• Data - Immutable Flat Map
• Symbol
• NameSpace
• Entity
• Store

Store provides Application Level Snapshot and Restore. + Entities and Store can be used to implement Database like semantics in the Application itself.+

This part is similiar to Entitiy Systems Architecture. +

=== Processing

This gives you Abstractions like Board, System, Wire, Frame and Token. +

Systems are connected with Wires on a Board. +

Every System has +

• "feedback"
• "sysin"
• "sysout"
• "syserr"
• "debug"
• custom sockets

This part is similiar to Pure Data and NoFlo. +

A System inputs a Token and outputs a Tokens + Token has a Data and a Frame. + Frames enable Re-Entrant / Parallel Data Flow. + Frames are to Systems as stackframes are to functions. + The notion of frames is derived from MIT's Data Flow Architectures. +

== Where are the API docs ?

RDP is meant to be extended. + For the time being you can refer to the source code.

== What high level patterns does RDP use ?

• Message Bus
• Factory pattern
• http://entity-systems.wikidot.com/rdbms-with-code-in-systems[Entity Systems]
• https://github.com/tailrecursion/hoplon[Spreadsheets]
• Workflow Patterns
• https://github.com/substack/stream-handbook[Streams and Pipes]
• Reactive Components
• Promises
• XML and OSC

== What low level patterns does RDP use ?

• Scalars and Symbols from Lisp
• http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.124.1645[Monsoon Data Flow Processor Architecture]
• MIT's Tagged Token Architecture
• Motherboard
• Stack manipulation
• Call Frames
• Breadboard programming
• FSM's

== What can you do with it ?

In theory anything that is possible with a processor. +

== What are some unique things about RDP ?

RDP can implement the following per Application

• Tracing
• https://vimeo.com/102414660[Step Debugging]
• Profiling
• https://github.com/harsha-mudi/esti/blob/master/sim.xml[Simulation based testing]
• https://vimeo.com/102371548[Mirroring RPC]
• Flow Diagram Generation
• Application Snapshots (Save / Load)

== What is RDP useful for ?

• Data Analysis
• Games
• Simulation
• Systems Modelling
• Visual Applications
• Cross-Medium Applications
• Reactive Applications like Spreadsheet
• MVC framework

== Does RDP solve some problems with Data Flow ?

Yes

• Easy Step Debugging
• Modelling Complex Data
• Adaptability to modern UI
• NameSpace Abstraction for Memory management
• Flow Abstraction for Parallelism management

== Examples

• https://github.com/harsha-mudi/esti[Esti] ( A GUI task progress measuring app built with React.js, D3, and rdp )

== Roadmap

• Avoid Data Racing
• Avoid Memory Leaks
• Mirror Latency

== API

source,javascript

.A moving average calculator

// 0. imports // Literate Source generated by edde // The executable source order is different

var $b = require("../index.js"), S = $b.S, T = $b.T, D = $b.D, yx = require("yx"), // gives you extends keyword _ = require("lodash");

// 1. Board manages all instances in an Application var b = new $b.Board();

// D for Data b.add( S("gen1"), ArrayGenerator, D({from: 1, to: 50})); b.add( S("gen2"), ArrayGenerator, D({from: 100, to: 200}));

b.add( S("range-avg1"), RangeAverageFilter, D({width: 5}));

// 2. Wire up ! b.connect( "gen1", "range-avg1"); b.connect( "gen2", "range-avg1"); b.add( S("condump"), ConsoleDumper) b.connect( "range-avg1", "condump");

b.push("gen1", T().start()); b.push("gen2", T().start());

// 3. // extend the base System class // every token has data and state // both of which managed by the Board // start and end surround a data flow // the following is a "Generator" System yx.__extends(ArrayGenerator, $b.System);

function ArrayGenerator(b, conf) { ArrayGenerator.super.constructor.apply(this, arguments); }

ArrayGenerator.prototype.input = function (token) { // S is short for Symbol // start means start ! if (token.data.is(S("start"))) { var values = []; // conf is Data // Data has properties (props) var i = this.conf.prop("from"); do { values.push(i++); } while(i < this.conf.prop("to"));

    // send out a copy of start
    this.output(token.dup());

    _.map(values, function (e) {
        // dup means duplicate
        // stamp assigns data to token
        this.output(token.dup().stamp(e));
    }.bind(this));


    }
    this.output(token.dup().stop());

}

// 4. The following is a "Filter" System yx.__extends(RangeAverageFilter, $b.System);

function RangeAverageFilter (b, conf) { RangeAverageFilter.super.constructor.apply(this, arguments); }

RangeAverageFilter.prototype.input = function (token) {

if ($b.Scalar.type_of(token.data) === "Sym") {
    if (token.data.is(S("start"))) {
        var range = [];
        token.state.r.prop("range", []);
    } else if (token.data.is(S("stop"))) {
        // this.b.bus.trigger( .. tell the ui .. )
        // b is the board object
    }

} else { // access thread local state // did you notice that gen1 and gen2 are both connected to this ? var range = token.state.r.prop("range");

    if (range.length == this.conf.prop("width")) {
        var sum = 0;
        _.map(range, function (e) {
            sum += e;
        });

        var avg = sum / this.conf.prop("width");

        console.log(range);
        range.shift();
        this.output(token.dup().stamp(avg));
    } else {
        range.push(token.data);
    }
}

}

// 5. The following is a "Sink" System yx.__extends(ConsoleDumper, $b.System);

function ConsoleDumper(b, conf) { ConsoleDumper.super.constructor.apply(this, arguments); }

ConsoleDumper.prototype.input = function (token) { console.log(token.data); }