0.6.0 • Published 7 years ago

chunksha v0.6.0

Weekly downloads
3
License
MIT
Repository
github
Last release
7 years ago

chunksha

Library to compute hash of a file in the browser

Why

There is a crypto api in the browser why not use it ? Few things:

  • digest methods of crypto api accepts only single argument which is a byteArray to compute hash from so there is no update method to compute hash in chunks
  • no progress report callback
  • no way to compute a hash of very large files > 3GB

Goal

This is just a toy project to implement sha1 and fulfill the above

Usage

<!DOCTYPE html>
<html>
  <head>
    <script src="src/5.js" charset="utf-8"></script>
    <script type="text/javascript" charset="utf-8">
      function handleFiles(files) {
        var worker = new Worker('src/5.js');
        worker.onmessage = function(e) {
          console.log(e.data);
        };
        worker.onerror = function (e) {
          console.log(e);
        };
        worker.postMessage(files[0]);
      }
    </script>
  </head>
  <body>
    <input type="file" id="input" onchange="handleFiles(this.files)">
  </body>  
</html>

Tests

npm install mocha
mocha test/index5.js 

Benchamarks

As a base to benchmark the library I've ised rusha.js implementation. Benchmark using sample files with different sizes. All file based tests are done in the Chrome browser The 100k iteration test is done in nodejs. All times are in seconds To run the test browser start a simple http server to serve index.html file the server is needed to overcome a webworker security error accessed from origin 'null'

python -m SimpleHTTPServer 8000
filesizebase1.js2.js4.js5.jssha1
medium.mp42.5 MB0.0671.190.790.0940.09dce6ae98c6a7187b3e08b389edbeff47e9d2e8a3
medium.zip24.3 MB0.35711.167.750.620.34b1ec91671e35882cd684db2b3b4c598db55a2544
good-big.zip492.1 MB6.557237.6156.412.126.3177d201055db8403e4079715bfccd8b442814cb34
wrong.zip840.7 MB11.102400.1275.022.1410.86f7aa6d9265c8ec5f52b07f69f98c2740c91c4ce1
mernis.tar.gz1.56 GB20.91540.0919.40aced1b7344c25b8c56fd330f66c19ae8cb9133a
snapshot.sav3.23 GBX37.56204676dcc0fdea3789e57da893353572f0316a72
ubuntu.vdi5.56 GBX71.868dbcad7decb8eae2577bd45854e3a8111232dcba
windows.vmdk22.58 GBX346.353ec5cc90676d136ba552b0a7d391932b81c2e155
100k iter55 bytes74.265.465.438.70

Mou icon

version 5 (current)

Few months passed since previous version New version of the Chrome browser support FileReaderSync Node version upgrade to 4.4.7 In the browser for files smaller than 1GB chunksha is slightly faster For files bigger that 1GB it is 1.1 times slower than rusha.js In my tests rusha crashed on 3GB file while chunksha continued to work.

version 4

In the browser 1.5 times slower than base implementation

version 2

In the browser 19 times slower than base implementation

verion 1

In the browser 28 times slower than base implementation

Authors and Contributors

Wriiten by (@szydan), based on a great implementation (Rusha) by (@srijs)

0.6.0

7 years ago

0.0.1

9 years ago