Skywalker NPM | npm.io

Skywalker

Walks a directory or a file and optionally applies transformations on the tree's members. There are other modules that do the same, I didn't like them, rolled my own.

Can't believe skywalker was not already in use on npmjs.

Install

	npm install skywalker

Features

Easy to use
Callbacks or evented style
Directory sizes
Supports limited depth, regex filtering, glob matching, mime-type filtering, extension filtering
Easy as hell to write plugins for

Usage

	var Tree = require('skywalker');

	Tree('~/some_dir')
		.ignoreDotFiles()
		.ignore(/(^|\/)_.*?$/g) //ignores files that begin with "_"
		.filter(/something/g,function(next,done){
			console.log('runs for each file or directory that match "something"');
			next()
		})
		.filter(/(^|\/)_.*?$/g,function(next,done){
			console.log('rejects all files or directories that begin with "_"',this._.path);
			done(null,false);
		})
		.extensionFilter('json',function(next,done){
			console.log('runs for each file that has a json extension');
			var file = this;
			require('fs').readFile(this._.path,{encoding:'utf8'},function(err,contents){
				if(err){
					file._.error = err;
					return next();
				}
				file._.contents = contents;
				try{
					file.data = JSON.parse(contents);
				}catch(err){
					file._.error = err;
				}
				next();
			})
		})
		.on('file',function(file){console.log('file event:',file._.path);})
		.on('directory',function(file){console.log('directory event:',file._.path);})
		.on('done',function(file){console.log('-----------------------');})
		.emitErrors(true)
		.on('error',function(err){console.log('ERROR',err);})
		.start(function(err,file){
			if(err){return console.log(err);}
			for(var n in file){
				console.log(file[n]);
			}
			/**or**/
			var children = file._.children;
			for(var i=0 ; i < children.length ; i++){
				console.log(children[i]._.name);
			}
		})
	;

If for some reason you want to set the root directory name later (not at instantiation), do that:

	Tree()
		.file('path_to_file')
		//other things
		.start(callback)

By default, skywalker does not emit errors, as it is expected that they will be handled in callbacks. However, if you prefer event-style error handling, do the following:

	Tree(dir)
	.emitError(true)
	.on('error',function(err){
		console.log('error',err);
	})
	.start(func);

Directories children are accessible in two manners:

	var t = Tree(dir).start(function(err,files){
		//either use:
		files._.children;
		for(var i = 0, l = files._.children.length;i < l;i++){
			console.log(files._.children[i]._.filename);
		}
		//or:
		files['child_name.jpg'];
		//or even:
		files['subdir']['child_name.jpg'];
		for(var n in files){
			console.log(n,files[n]._.path);
		}
	});

Children always exists on all file instances, even when they are not directories, so you can safely just loop, and the loop will not run when children.length is 0.
Any property other than children is not enumerable, so for...in loops are also safe to use without prior checking if the file is a directory.

However, if you have a filter running that disables files AND you are watching, then a file might be null, so you might want to do:

	var t = Tree(dir).start(function(err,files){
		for(var n in files){
			if(!files[n]){continue;}
			console.log(n,files[n]._.path);
		}
	});

Files Properties

All properties (name, path, etc) are stored on a property named "_". The following properties are to be found:

file._.path full path to the file
file._.dirname parent dir of the file
file._.filename filename of the file, including extension
file._.extension extension, without the dot, and in lowercase
file._.name filename without extension
file._.children only for directories: an array of children
file._.parents an array of parents (backref to the parents)
file._.contents empty, fill it with a string if your own callbacks
file._.mime mimetype, for example 'text/plain'
file._.mime.type for example 'text'
file._.mime.subType for example, 'plain'
file._.isDirectory true for directories, false for everything else
and all stats properties, which are: + dev + mode + nlink + uid + gid + rdev + blksize + ino + size works for directories too + blocks + atime converted to a unix timestamp + mtime converted to a unix timestamp + ctime converted to a unix timestamp

Plugins may add properties to this object (see below).

If you have, in your path, a file or folder named "_", then the properties of its parent will be overwritten. In that case, you have two options:
1 - Change the default namespace:

	Tree.propertiesPropertyKey('_somethingsafe_');
	// later...
	console.log(file._somethingsafe_.path)

2 - use the 'safe' namespace:

	console.log(file.__________.path);
	// Yes, that's 10 "_".
	// If you have a file named like that too,
	// then you are shit out of luck.

Note that both keys are usable at all times.

The default toString() function outputs the file's path, but if you set the contents property of the file...

	file._.contents = 'abcde';

...Then this is what toString() will output.

To detect mimetypes, skywalker uses node-mime. It is made available on the Tree.mime namespace

	//define a new mime-type:
	Tree.mime.define({
		'text/jade':['jade']
	})

Watch

Skywalker doesn't know how to watch, but it is "watch-ready". Thus, you are able to implement any watching system you like. swap start([callback]) with watch(watchFunction[,callback])

watchFunction receives two arguments: a "watchHelpers" object that contains helpers, and a callback function to run when ready.

	var t = Tree(dir)
		.on(/** something, function **/)
		/**...**/
		.watch(function(watchHelpers,done){
			var watcher = myWatchImplementation(watchHelpers.filename);
			watcher.on('new',function(filename){
				watchHelper.onCreated(filename);
			});
			watcher.on('systemError',function(err){
				done(err);
			});
			watcher.on('ready',function(){
				done(null,function(){watcher.stop();})
			});
		},callback)
	;
	//later:
	t.unwatch(); //calls watcher.stop()

Available helpers are:

watchHelpers.filename the root directory
watchHelpers.tree the skywalker instance
watchHelpers.onCreated(filepath)
watchHelpers.onChanged(filepath)
watchHelpers.onRemoved(filepath)
watchHelpers.onRenamed(filepathNew,filepathOld)
watchHelpers.onError(error)

Look in /watchers for an example implementation

As an alternative to implementing your own function, you may simply specify the implementation like so:

	var t = Tree(dir)
		.on(/** something, function **/)
		/**...**/
		.watch('gaze',callback)

Available implementations are gaze, watch, and chokidar. Note that they are not bundled with skywalker and that you will have to install them separately.

Events

the following events are emitted from Skywalker:

FILE: 'file': emitted when a file is processed
DIRECTORY: 'directory': emitted when a directory is processed
DONE: 'done': emitted when all files have been processed
ERROR: 'error': emitted when an error is encountered
CREATED: 'created': emitted when a file or directory is created (if watch()ing)
REMOVED: 'remove': emitted when a file or directory is deleted (if watch()ing)
CHANGED: 'change': emitted when a file or directory is modified (if watch()ing)
RENAMED: 'rename': emitted when a file or directory is renamed (if watch()ing)

The events strings live on Skywalker.Events, so instead of on('error'), you can use on(Tree.Events.ERROR).

Filters

There are several available filters, they all have the same signature: filterFunction(regex|glob|null,func)

regex or glob is either a regex or a string. If nothing is provided, the filter will match every file and directory
func is a function with signature callback(next,done). Next() processes the next file, and done() interrupts the current directory processing. You can call done(err) to output an error.

Available filters are:

filter(regex|glob,func): basic filter
directoryFilter(regex|glob,func): acts only on directories
fileFilter(regex|glob,func): acts only on files
extensionFilter(string,func): you can provide a space-separated string of extension (jpg jpeg bmp), will act only on those extensions
mimeFilter(regex|glob,func): will act only on matched mime type Careful! fileFilter, mimefilter and extensionFilter will not descend in sub-directories! Use a normal filter for that.

Additionally, you have some convenience filters to ignore things:

ignore(regex|glob): will ignore files that match the pattern
ignoreDirectories(regex|glob): you know what this does
ignoreFiles(regex|glob): that too
ignoreDotFiles(): ignores files and directories that begin with '.'

For an wide array of examples, check out skywalker-extended

Selectors

Selectors run after the tree has been parsed and allow for further filtering.

	var Tree = require('skywalker');
	var db = Tree.db;
	Tree(__dirname)
		.ignoreDotFiles()
		.selectors('F & size > 6100')
		.start(function(err,file){
			var c = file._.children;
			for(var n in c){console.log(c[n]._.path)}
		})

A selector presents itself as such: property operator value

property is any property of the file, found on the _ object. That is any 'native' property, or any property added by a plugin
operator is one of the operators below
value is the value compared to. some operators don't require a value

You can chain selectors with '&'. Available selectors are:

Selectors with properties: - GREATER_THAN : '>' example: size > 6100 - LOWER_THAN: '<' example: atime < 1416242596 - GREATER_OR_EQUAL: '>=' - LOWER_OR_EQUAL: '<=' - EQUAL: '==' - EQUAL_STRICT: '===' - MATCHES: '#' example: path # node_modules
Selectors that require a value only: - EXTENSION: '.' example . jpg (does not require a property) - PATH: '/' example / node_modules (similar to the matches example above, but globbing is allowed) - MIMETYPE: '@' example: @ text/javascript
Selectors that require an operator only: - ISDIR: 'D' - ISFILE:'F'

Plugins

Skywalker ships with a few examples plugins (not loaded, copy-paste them where you need them). They are:

checksum: outputs a checksum string in a property called checksum. Needs the checksum module.
images: outputs size (width,height), imageMode (landscape, portrait, square) and ratio (1.xxx) to the "_" property of images. Needs the image-size module.
json: parses json files. Sets the raw data on the "_.contents" and the parsed data on "_.data"
markdown: parses markdown files. Sets the raw data on "_.contents" and the rendered content on "_.rendered". Needs the markdown module.
size: adds a human readable size property called humanSize. Needs the filesize module.
websafe: turns file names ("my nice image.jpeg") to a string that can be used in a classname or as an id ("mynice_image"), and sets it on the ".safename" property