parse-wget

parse-wget

Parses complete or incomplete output file that generated by wget with --output-file option.

const parseWget = require('parse-wget');
var execSync = require('child_process').execSync;
try {
   execSync('wget -rHl1 --output-file=output https://nodejs.org/en/');
}
catch (e) {
   // Ignore HTTP errors
}
console.log(parseWget('output'));

The above example will be parsed as follow:

[
   // ...

   // Error
   {
      "timestamp": 1461159565,
      "remote": "http://fonts.gstatic.com/robots.txt",
      "error": {
         "code": 404,
         "message": "Not Found."
      }
   },

   // ...

   // Redirection
   {
      "timestamp": 1461159574,
      "remote": "http://www.google-analytics.com/",
      "redirect": "https://www.google.com/analytics/"
   },

   // Success
   {
      "timestamp": 1461159574,
      "remote": "https://nodejs.org/static/apple-touch-icon.png",
      "length": 5823,
      "type": "image/png",
      "local": "nodejs.org/static/apple-touch-icon.png",
      "read": 5823,
      "progress": 100,
      "complete": true,
      "estimate": 0,
      "elapsed": 0.002
   },

   // ...

Install

npm install --save parse-wget

parseWget(outputFile, [encoding [, dotStyle]])

Arguments

outputFile

Specify path of output file that generated by wget with --output-file option.

encoding (Optional)

Specify encoding. this argument will be passed to fs.fileReadSync. utf-8 is default.

dotStyle (Optional)

This is optional argument. Specify dotStyle when run wget with --progress=dot:style option. For example, when run wget with --progres=dot:giga, specify giga.

NOTE: Current version, output from wget with --progress=bar:force is not supported.

Return Value

Property	Type	Description
timestamp	Number	Start time in UNIX time
remote	String	Request URL
length	Number	Length of content in bytes
type	String	MIME type of content
local	String	Saved path
read	Number	Total read in bytes
progress	Number	Progress percentage
complete	Boolean	Whether download was complete
estimate	Number	Estimate for completion in seconds
elapse	Number	Elapse for completion in seconds
redirect	String	Redirect URL
error.code	Number	Error code
error.message	String	Error message

• length and type may be undefined or wrong because some server response no or wrong value.
• elapsed is undefined when incomplete state(!complete) and its value may be floating number.
• read may be inaccurate. This will be roughly estimated from number of dots, or progress and length. Check actual data for accuracy.

License

Distirubted under the MIT license.

Copyright (C) 2016 Retorillo