2.1.157 • Published 4 months ago

@thi.ng/transducers-binary v2.1.157

Weekly downloads
119
License
Apache-2.0
Repository
github
Last release
4 months ago

transducers-binary

npm version npm downloads Twitter Follow

This project is part of the @thi.ng/umbrella monorepo.

About

Binary data related transducers & reducers. This is a support package for @thi.ng/transducers.

Like the transducers and reducers defined in @thi.ng/transducers, all functions defined in this package too accept an optional input iterable for direct use.

Status

STABLE - used in production

Search or submit any issues for this package

Related packages

  • @thi.ng/binary - 100+ assorted binary / bitwise operations, conversions, utilities, lookup tables
  • @thi.ng/bitstream - ES6 iterator based read/write bit streams with support for variable word widths
  • @thi.ng/unionstruct - C-style struct, union and bitfield read/write views of ArrayBuffers

Installation

yarn add @thi.ng/transducers-binary

ES module import:

<script type="module" src="https://cdn.skypack.dev/@thi.ng/transducers-binary"></script>

Skypack documentation

For Node.js REPL:

# with flag only for < v16
node --experimental-repl-await

> const transducersBinary = await import("@thi.ng/transducers-binary");

Package sizes (gzipped, pre-treeshake): ESM: 2.70 KB

Dependencies

Usage examples

Several demos in this repo's /examples directory are using this package.

A selection:

ScreenshotDescriptionLive demoSource
Figlet-style bitmap font creation with transducersDemoSource
2D transducer based cellular automataDemoSource
Parser grammar livecoding editor/playground & codegenDemoSource
1D Wolfram automata with OBJ point cloud exportDemoSource

API

Generated API docs

import * as tx from "@thi.ng/transducers";
import * as txb from "@thi.ng/transducers-binary";

Random bits

// 10 samples with 50% probability of drawing a 1
[...txb.randomBits(0.5, 10)]
// [ 1, 0, 1, 1, 0, 1, 0, 1, 1, 0 ]

// infinite iterator without 2nd arg, so limit with `take()`
[...tx.take(10, txb.randomBits(0.1))]
// [ 0, 0, 0, 0, 0, 0, 1, 0, 0, 0 ]

import { Smush32 } from "@thi.ng/random";

// with seeded PRNG
[...txb.randomBits(0.5, 10, new Smush32(12345678))]
// [ 0, 0, 1, 1, 0, 0, 0, 0, 1, 0 ]

Streaming hexdump

This is a higher-order transducer, purely composed from other transducers. See code here.

src = [65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 33, 48, 49, 50, 51, 126, 122, 121, 120]

[...txb.hexDump({ cols: 8, address: 0x100 }, src)]
// [ '00000100 | 41 42 43 44 45 46 47 48 | ABCDEFGH',
//   '00000108 | 49 4a 21 30 31 32 33 7e | IJ!0123~',
//   '00000110 | 7a 79 78 00 00 00 00 00 | zyx.....' ]

Structured byte buffer construction

The bytes() reducer transforms a stream of declarative data definitions (optionally with Little-Endian encoding) into an Uint8Array.

const bytes = txb.bytes(
    // initial buffer capacity (grows on demand)
    32,
    // structured data
    [
        // default order is Big-Endian
        txb.u32(0xdecafbad),
        // force Little-endian (also works for floats)
        txb.u32(0x44332211, true),
        // all strings will be utf-8 encoded
        txb.str("vec4"),
        // use little-endian for each of these array vals
        txb.f32array([1, 2, 3, 4], true),
        txb.u8(0x2a)
    ]
);

console.log(tx.str("\n", txb.hexDump({}, bytes)));

// 00000000 | de ca fb ad 11 22 33 44 76 65 63 34 00 00 80 3f | ....."3Dvec4...?
// 00000010 | 00 00 00 40 00 00 40 40 00 00 80 40 2a 00 00 00 | ...@..@@...@*...

Bitstream

Decompose / transform a stream of fixed size words into their bits:

[...txb.bits(8, [0xf0, 0xaa])];
// [ 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0 ]

console.log(
    tx.transduce(
        tx.comp(
            txb.bits(8),
            tx.map((x) => (x ? "#" : ".")),
            tx.partition(8),
            tx.map((x) => x.join(""))
        ),
        tx.str("\n"),
        [0x00, 0x18, 0x3c, 0x66, 0x66, 0x7e, 0x66, 0x00]
    )
);
// ........
// ...##...
// ..####..
// .##..##.
// .##..##.
// .######.
// .##..##.
// ........

Extended to transform longer strings (taken from the bitmap-font example, live demo):

// font lookup table
const chars = {
    a: [0x00, 0x18, 0x3c, 0x66, 0x66, 0x7e, 0x66, 0x00],
    b: [0x00, 0x7c, 0x66, 0x7c, 0x66, 0x66, 0x7c, 0x00]
};

// re-usable transducer
const xfJoin = tx.map((x) => x.join(""));

// higher order transducer to transform single char from string
const xfChar = (i) =>
    tx.comp(
        tx.pluck(i),
        txb.bits(8),
        tx.map((x) => (x ? "#" : ".")),
        tx.partition(8),
        xfJoin
    );

// transform entire string
const banner = (src) =>
    tx.transduce(
        tx.comp(
            // dynamically create `xfChar` transducers for each char
            // and run them in parallel via `multiplex()`
            tx.multiplex(...tx.map((i) => xfChar(i), tx.range(src.length))),
            // then join the results for each line
            xfJoin
        ),
        // use `str()` reducer to build string result
        tx.str("\n"),
        // convert input string into stream of row-major bitmap font tuples
        tx.zip(...tx.map((x) => chars[x], src))
    );

console.log(banner("abba"));
// ................................
// ...##....#####...#####.....##...
// ..####...##..##..##..##...####..
// .##..##..#####...#####...##..##.
// .##..##..##..##..##..##..##..##.
// .######..##..##..##..##..######.
// .##..##..#####...#####...##..##.
// ................................

Base64 & UTF-8 en/decoding

Unlike JS default btoa() / atob() functions which operate on strings, these transducers stepwise convert byte values to base64 and back.

// here we first add an offset (0x80) to allow negative values to be encoded
// (URL safe results can be produced via opt arg to `base64Encode`)
enc = tx.transduce(
    tx.comp(tx.map((x) => x + 0x80), txb.base64Encode()),
    tx.str(),
    tx.range(-8, 8)
);
// "eHl6e3x9fn+AgYKDhIWGhw=="

// remove offset again during decoding, but (for example) only decode while val < 0
[
    ...tx.iterator(
        tx.comp(
            txb.base64Decode(),
            tx.map((x) => x - 0x80),
            tx.takeWhile((x) => x < 0)
        ),
        enc
    )
];
// [ -8, -7, -6, -5, -4, -3, -2, -1 ]

buf = tx.transduce(
    tx.comp(txb.utf8Encode(), txb.base64Encode()),
    tx.str(),
    "beer (🍺) or hot beverage (☕️)"
);
// "YmVlciAo8J+Nuikgb3IgaG90IGJldmVyYWdlICjimJXvuI4p"

tx.transduce(tx.comp(txb.base64Decode(), txb.utf8Decode()), tx.str(), buf);
// "beer (🍺) or hot beverage (☕️)"

Transducers

Reducers

Authors

Karsten Schmidt

If this project contributes to an academic publication, please cite it as:

@misc{thing-transducers-binary,
  title = "@thi.ng/transducers-binary",
  author = "Karsten Schmidt",
  note = "https://thi.ng/transducers-binary",
  year = 2018
}

License

© 2018 - 2021 Karsten Schmidt // Apache Software License 2.0

2.1.157

4 months ago

2.1.156

4 months ago

2.1.154

4 months ago

2.1.155

4 months ago

2.1.152

5 months ago

2.1.151

5 months ago

2.1.153

5 months ago

2.1.150

5 months ago

2.1.149

5 months ago

2.1.147

6 months ago

2.1.148

6 months ago

2.1.146

6 months ago

2.1.145

6 months ago

2.1.144

7 months ago

2.1.143

7 months ago

2.1.141

8 months ago

2.1.142

8 months ago

2.1.140

8 months ago

2.1.139

8 months ago

2.1.138

9 months ago

2.1.130

11 months ago

2.1.132

10 months ago

2.1.131

11 months ago

2.1.134

10 months ago

2.1.133

10 months ago

2.1.136

9 months ago

2.1.135

10 months ago

2.1.137

9 months ago

2.1.121

1 year ago

2.1.123

12 months ago

2.1.122

1 year ago

2.1.125

12 months ago

2.1.124

12 months ago

2.1.127

12 months ago

2.1.126

12 months ago

2.1.129

11 months ago

2.1.128

11 months ago

2.1.120

1 year ago

2.1.119

1 year ago

2.1.118

1 year ago

2.1.117

1 year ago

2.1.116

1 year ago

2.1.115

1 year ago

2.1.114

1 year ago

2.1.113

1 year ago

2.1.112

1 year ago

2.1.111

1 year ago

2.1.110

1 year ago

2.1.109

1 year ago

2.1.108

1 year ago

2.1.107

1 year ago

2.1.106

1 year ago

2.1.105

1 year ago

2.1.104

1 year ago

2.1.103

1 year ago

2.1.102

1 year ago

2.1.99

1 year ago

2.1.101

1 year ago

2.1.100

1 year ago

2.1.98

1 year ago

2.1.97

1 year ago

2.1.96

1 year ago

2.1.95

1 year ago

2.1.94

1 year ago

2.1.92

1 year ago

2.1.93

1 year ago

2.1.90

1 year ago

2.1.91

1 year ago

2.1.89

1 year ago

2.1.88

1 year ago

2.1.87

2 years ago

2.1.86

2 years ago

2.1.85

2 years ago

2.1.83

2 years ago

2.1.84

2 years ago

2.1.82

2 years ago

2.1.81

2 years ago

2.1.80

2 years ago

2.1.58

2 years ago

2.1.59

2 years ago

2.1.56

2 years ago

2.1.57

2 years ago

2.1.54

2 years ago

2.1.55

2 years ago

2.1.52

2 years ago

2.1.53

2 years ago

2.1.51

2 years ago

2.1.69

2 years ago

2.1.67

2 years ago

2.1.68

2 years ago

2.1.65

2 years ago

2.1.66

2 years ago

2.1.63

2 years ago

2.1.64

2 years ago

2.1.61

2 years ago

2.1.62

2 years ago

2.1.60

2 years ago

2.1.78

2 years ago

2.1.79

2 years ago

2.1.77

2 years ago

2.1.74

2 years ago

2.1.75

2 years ago

2.1.72

2 years ago

2.1.73

2 years ago

2.1.70

2 years ago

2.1.71

2 years ago

2.1.50

2 years ago

2.1.49

2 years ago

2.1.47

2 years ago

2.1.48

2 years ago

2.1.45

2 years ago

2.1.46

2 years ago

2.1.43

2 years ago

2.1.44

2 years ago

2.1.41

2 years ago

2.1.42

2 years ago

2.1.40

2 years ago

2.1.39

2 years ago

2.1.38

2 years ago

2.1.36

2 years ago

2.1.34

3 years ago

2.1.35

2 years ago

2.1.32

3 years ago

2.1.33

3 years ago

2.1.31

3 years ago

2.1.27

3 years ago

2.1.28

3 years ago

2.1.25

3 years ago

2.1.26

3 years ago

2.1.23

3 years ago

2.1.24

3 years ago

2.1.21

3 years ago

2.1.22

3 years ago

2.1.29

3 years ago

2.1.30

3 years ago

2.1.19

3 years ago

2.1.20

3 years ago

2.1.18

3 years ago

2.1.16

3 years ago

2.1.17

3 years ago

2.1.14

3 years ago

2.1.15

3 years ago

2.1.12

3 years ago

2.1.13

3 years ago

2.1.11

3 years ago

2.1.9

3 years ago

2.1.10

3 years ago

2.1.8

3 years ago

2.1.7

3 years ago

2.1.6

3 years ago

2.1.5

3 years ago

2.0.8

4 years ago

2.1.2

4 years ago

2.1.1

4 years ago

2.1.4

4 years ago

2.1.3

4 years ago

2.1.0

4 years ago

2.0.7

4 years ago

2.0.4

4 years ago

2.0.6

4 years ago

2.0.3

4 years ago

2.0.1

4 years ago

2.0.0

4 years ago

1.0.5

4 years ago

1.0.4

4 years ago

1.0.3

4 years ago

1.0.2

4 years ago

1.0.1

4 years ago

0.6.25

4 years ago

0.6.23

4 years ago

0.6.24

4 years ago

0.6.22

4 years ago

0.6.21

4 years ago

0.6.20

4 years ago

0.6.19

4 years ago

0.6.18

4 years ago

0.6.17

4 years ago

0.6.16

4 years ago

0.6.15

4 years ago

0.6.14

4 years ago

0.6.13

4 years ago

0.6.12

4 years ago

0.6.9

4 years ago

0.6.11

4 years ago

0.6.7

4 years ago

0.6.6

4 years ago

0.6.5

4 years ago

0.6.4

4 years ago

0.6.3

4 years ago

0.6.2

4 years ago

0.6.1

5 years ago

0.6.0

5 years ago

0.5.33

5 years ago

0.5.32

5 years ago

0.5.31

5 years ago

0.5.30

5 years ago

0.5.29

5 years ago

0.5.28

5 years ago

0.5.27

5 years ago

0.5.26

5 years ago

0.5.25

5 years ago

0.5.24

5 years ago

0.5.23

5 years ago

0.5.21

5 years ago

0.5.22

5 years ago

0.5.20

5 years ago

0.5.19

5 years ago

0.5.18

5 years ago

0.5.17

5 years ago

0.5.16

5 years ago

0.5.15

5 years ago

0.5.14

5 years ago

0.5.13

5 years ago

0.5.12

5 years ago

0.5.11

5 years ago

0.5.10

5 years ago

0.5.9

5 years ago

0.5.8

5 years ago

0.5.7

5 years ago

0.5.6

5 years ago

0.5.5

5 years ago

0.5.4

5 years ago

0.5.3

5 years ago

0.5.2

5 years ago

0.4.9

5 years ago

0.4.8

5 years ago

0.4.7

6 years ago

0.4.6

6 years ago

0.4.5

6 years ago

0.4.4

6 years ago

0.4.3

6 years ago

0.4.2

6 years ago

0.4.1

6 years ago

0.4.0

6 years ago

0.3.12

6 years ago

0.3.11

6 years ago

0.3.10

6 years ago

0.3.9

6 years ago

0.3.8

6 years ago

0.3.7

6 years ago

0.3.6

6 years ago

0.3.5

6 years ago

0.3.4

6 years ago

0.3.3

6 years ago

0.3.2

6 years ago

0.3.1

6 years ago

0.3.0

6 years ago

0.2.3

6 years ago

0.2.2

6 years ago

0.2.1

6 years ago

0.2.0

6 years ago

0.1.1

6 years ago

0.1.0

6 years ago