ksdumpjs v1.0.2
About
This is a tool to dump binary files into JSON using Kaitai Struct formats.
It instantiates instance values, populates enum name values and loads format imports.
It most likely does not cover many constructs that can be created using Kaitai Struct but it should work for most formats. It serves as a lightweight alternative to installing the Kaitai Struct Visualizer's ksdump.
Kaitai Struct
Kaitai Struct Visualizer
Install
npm install -g ksdumpjs
Usage
Concept
Without any args ksdumpjs will use ksy format files in ./formats to compile
parsers that are output to ./parsers and then parse binary files in
./binaries based on the format's meta section using ${id}.{file-extension}
to identify a matching binary for the format placing the JSON output in
./jsons.
Arguments
--format, -f: Path to a .ksy format file or directory of format files. Default is ./formats.
--binary, -b: Path to a binary file, directory, or glob pattern. Default is ./binaries.
--out, -o: Output path for JSON files. Default is ./jsons.
--parser, -p: Directory for compiled parsers. Default is ./parsers.
--spaces, -s: Number of spaces for formatted JSON output (use compact output if omitted).Details
| Format | Binary | Result | Comment |
|---|---|---|---|
| directory | directory | allow | Use .ksy meta {id}.{file-extension} to find matches in binary |
| file | directory | allow | Use .ksy meta {id}.{file-extension} to find match in binary |
| file | file | allow | |
| file | glob | allow | Use format on each binary result of glob |
| directory | file | deny | Ambiguous: cannot determine which .ksy file in the directory should match the binary file |
| directory | glob | deny | Ambiguous: cannot determine which .ksy file in the directory should match the binary files |
| glob | <any> | deny | Ambiguous: multiple .ksy files could match the binary files |
Note: use forwardslash for paths, backslash will escape any glob tokens.
Example
> node ksdump .\test\formats\zip.ksy .\test\samples\sample1.zip .\jsons -s
āŗ ksdump Initialized timer...
Processing: .\test\formats\zip.ksy
āļø Generating: Zip
-> Importing common/dos_datetime
Parsing common/dos_datetime
š Parsing binary: .\test\samples\sample1.zip
š¤ Transforming: .\test\samples\sample1.zip
š¤ Exporting: jsons\sample1.json
ā
Success jsons\sample1.json
[ā] ksdump Timer run for: 128msVerify correctness against Kaitai Struct Web-IDE exported json:
jq will sort the field order since the instantiated instance field orders
differs between ksdumpjs and the Kaitai Struct Web-IDE.
> ./jq -b -S . .\jsons\sample1.json > sorted_sample1.json
> ./jq -b -S . check_sample1.json > sorted_check_sample1.json
> git diff --no-index sorted_sample1.json sorted_check_sample1.json
> $?
TrueWhy
ksdumpjs allows you to parse structured binary data into JSON for easier integration with analysis tools, such as spreadsheets. Compared to the Kaitai Struct Web IDE, which struggles with large files due to browser memory limits, ksdumpjs uses json-stream-stringify to avoid memory constraints.
On a Ryzen 5700G, ksdumpjs can parse a 165MB binary file containing 200+ fields with nested types in ~3 seconds, and output 614MB of JSON in ~25 seconds (or 1GB formatted output in ~30 seconds).