rdup v1.1.2
RDUP (Rapid Deployment Update Packager)
RDUP is package management system designed to enable developers to rapidly deploy updates to users while maintaining file integrity and reducing bandwidth for sequential updates
Features
- Support random access
- Use JSON to store files' information
- Very easy to write a parser
- Supports Checksums for file verification
- Supports compression to shrink package contents
Supported Algorithms
Compression Algorithms
- BWTC Supports Level 1-9
- Bzip2 Supports Level 1-9
- Dmc
- Lzjb Supports Level 1-9
- LzjbR Supports Level 1-9
- Lzp3
- PPM
Hash Algorithms
- MD5
- SHA
- SHA224
- SHA256
- SHA384
- SHA512
- Whirlpool
Command line utility
Install
$ npm install rdup
Usage
$ rdup --help
Usage: rdup [options] [command]
Commands:
pack|p [options] <dir> <output>
create rdup archive
Options:
--ordering <file path> path to a text file for ordering contents
--exclude <expression> exclude files matching glob <expression>
--exclude-dir <expression> exclude dirs matching glob <expression> or starting with literal <expression>
--exclude-hidden exclude hidden files
--no-compress <expression> do not compress files matching <expression>
--hash <type> use hashing algorithm <type> for checksums
--algo <type> compress files using <type> algorithm
--level <level> compression level, Valid options: 1-9
list|l <archive>
list files of rdup archive
extract-file|ef <archive> <filename>
extract one file from archive
extract|e <archive> <dest>
extract archive
check|c <archive>
check archive for corrupt files
algorithms
Show supported hash and compression algorithms
Options:
-h, --help output usage information
-V, --version output the version number
Excluding multiple resources from being packed
Given:
app
(a) ├── x1
(b) ├── x2
(c) ├── y3
(d) │ ├── x1
(e) │ └── z1
(f) │ └── x2
(g) └── z4
(h) └── w1
Exclude: a, b
$ rdup pack app app.rup --exclude-dir "{x1,x2}"
Exclude: a, b, d, f
$ rdup pack app app.rup --exclude-dir "**/{x1,x2}"
Exclude: a, b, d, f, h
$ rdup pack app app.rup --exclude-dir "{**/x1,**/x2,z4/w1}"
Using programatically
Example
var rdup = require('rdup');
var src = 'some/path/';
var dest = 'name.rup';
rdup.createPackage(src, dest, function() {
console.log('done.');
})
Please note that there is currently no error handling provided!
Format
RDUP uses Pickle to safely serialize binary value to file, there is
also a node.js binding of Pickle
class.
The format of rup files is very flat:
| UInt32: header_size | String: header | Bytes: file1 | ... | Bytes: file42 |
The header_size
and header
are serialized with Pickle class, and
header_size
's Pickle object is 8 bytes.
The header
is a JSON string, and the header_size
is the size of header
's
Pickle
object.
Structure of header
is something like this:
{
"files": {
"tmp": {
"files": {}
},
"usr": {
"files": {
"bin": {
"files": {
"ls": {
"offset": "0",
"size": 100,
"executable": true,
"checksum": "<truncated>",
"compressed": true,
"csum": "<truncated>",
"csize": 74
},
"cd": {
"offset": "74",
"size": 100,
"executable": true,
"checksum": "<truncated>",
"compressed": true,
"csum": "<truncated>",
"csize": 74
}
}
}
}
},
"etc": {
"files": {
"hosts": {
"offset": "148",
"size": 32,
"checksum": "<truncated>",
"compressed": true,
"csum": "<truncated>",
"csize": 18
}
}
}
}
}
offset
, size
, and checksum
records the information to read the file from
archive, the offset
starts from 0 so you have to manually add the size of
header_size
and header
to the offset
to get the real offset of the file.
offset
is a UINT64 number represented in string, because there is no way to
precisely represent UINT64 in JavaScript Number
.
size
is a JavaScript Number
that is no larger than Number.MAX_SAFE_INTEGER
,
which has a value of 9007199254740991
and is about 8PB in size. We didn't
store size
in UINT64 because file size in Node.js is represented as Number
and it is not safe to convert Number
to UINT64.
checksum
is a SHA-256 hash represented in a hexadecimal string for the file
used for file verification. This is stored so that should a file become corrupt
a new copy can be extracted without extracting the entire package
When compression is enabled compressed
, csum
, csize
record information
for the compression.
compressed
is a JavaScript Boolean
representing if a file is compressed
using Bzip2. This is so that with future versions specific files can be
compressed in case certain files would be come larger than their uncompressed
counterpart.
csum
is a SHA-256 hash represented in a hexadecimal string for the compressed
file used for file verification. This is stored to make sure should a
file become corrupt that it will not cause an overflow or a corrupt extraction.
csize
is a JavaScript Number
used to represent the size of the compressed
file data. csize
is also used during the packaging process to increment offset
accordingly to prevent overreading the buffer.
Information
RDUP is a modified fork of Electron ASAR, and is maintained by Skriglitz of TriFractal Studios and is made available under the MIT License