Phantomshot NPM

PhantomShot

Automated website screenshot comparsion with PhantomJS and Horseman.

Did you ever changed a template file or a piece of CSS and wanted to know if some pages of a huge web project are affected? PhantomShot was created to make such comparsion easier. It will take full screenshots of websites and compare them with older shots. You can write simple and powerful tests using Horseman. Don't worry about dynamic parts of a web page or pages that will change their contents often. You can simply hide or replace parts of a page before taking a screenshot. If you want you can launch a litte http server. So everybody can run your tests simply without command line knowledge.

Note: This tool is not made for cross browser website testing. Its made for comparing screenshots rendered with PhantomJS.

Setup PhantomShot

There are two ways of installing PhantomShot:

1. Install PhantomShot standalone command line tool via npm

You can use this option if you are a single developer with only one host system. However. Its recommended to use the second variant for better compatibility. If you recognize failed tests due to different text antialiasing or different transparency renderings you should definitely try the second variant.

npm install phantomshot --global

Phantamshot depends on NodeJS, PhantomJs and ImageMagick. Make sure you have them available in your path.

After installing you can try PhantomShot by running:

phantomshot --help

2. You can install PhantomShot stack by booting up a vagrant virtual machine:

If you do not want to install every dependency manually you should try the PhantomShot virtual machine setup via vagrant. This option will automatically install all the dependencies for you inside a virtual environment. You should use this option if you work across more machines with more than one developer. The great benefit of this installation method is that different machines will not longer render texts, antialiasing and transparency differently. To get started install vagrant and virtualbox (or also vmware) on your system. If you are using vagrant already in you project with a multi server setup you can just copy the PhantomShot section from the vagrant file into your own vagrant file.

You will find the PhantomShot vagrant setup in this separate git repository.

After your setup is complete run the following command to bring up your vagrant machine:

vagrant up phantomshot

Than ssh into your machine:

vagrant ssh phantomshot

Once you are logged in you can use the PhantomShot command:

phantomshot --help

Creating a new PhantomShot project

Create a new PhantomShot project by running

phantomshot init

This will create a .phantomshot folder in the current directory. This is the folder where all your tests and test results will live in. It also will create a example test file in that folder. If you are working with Git or Subversion you can add this folder to your version control. So every developer will have the latest tests and test results.

npm.io

Run your first test

After you have created the new PhantomShot project you can run the example test:

phantomshot test example

The first time you run the test it will create the original screenshot images in .phantomshot/example. All other tests will be compared with this images.

npm.io

Run the PhantomShot example test again

phantomshot test example

Now you will notice that the test will compare its results with the original images. All tests should be OK.

npm.io

Create your own tests

You can use the example test file as a boilerplate. Just create a copy of this file and name it after your test. You can use all methods provided by horseman inside your test.

Create more subtests

Within one testfile you can call the shot('subtest') method as often as you want. Make sure to use an unique name every time you use this method.

Simple workflow example

Run the test: Before you start implementing your changes run "phantomshot test". So you can make sure everything works fine.
Start Coding: Change your project files, fix some css errors or implement new features.
Run the test again
If your tests will succeed everything is fine.

Workflow for failed tests

If one or more tests fail you can take a look at the diff images. You will find the images in the "diff" subfolder inside your test and subtest directory.

npm.io

Than you should consider the following options:

The test fails but the diff is exactly what you have expected: That means everything is OK. Now you have to tell PhantomShot to confirm that test by running: "phantomshot confirm \<test_id> \<shot_name>". After running this command PhantomShot will drop the original image and will replace it with the last test image. If you run "phantomshot test test_ids..." again the test should be OK.
The test fails and you recognize some unexpected changes: That means something is wrong with your code. Repair your code and rerun the test.

Other reasons why a test can fail

The test itself needs to be updated: The test does not longer work because for example some elements cannot be found by Horseman or you changed some urls etc... In this case you can just update your testfile and run the test again.
The test fails because you have dynamic content on your page: Because of that every screenshot will be different. Use Horseman's evaluate function and the injected jQuery methods to replace or hide these contents.
The test fails because all texts are rendered diffrently: Try phantomshot with vagrant. On some machines the text rendering is different every time the page gets rendered.

Local configuration file

You can place a phantomshot.json file directly inside your project folder. This configuration will be available in all your test files. This is useful if you want to switch between development and live domains for example. There is no defined way how to use this file. You can use it in any way you want:

{
  "test_mode": "staging",
  "live_domain": "blacksmith-wuerzburg.de",
  "staging_domain": "test.blacksmith-wuerzburg.de"
}

Then you can access your config inside your tests like this:

var domain = config.live_domain;
if(config.test_mode=='staging'){
  domain = config.staging_domain;
}
//...
.open('http://'+domain+'/blog')
//...

Built in HTTP server

Start the build in HTTP Server by running:

phantomshot serve --port=3000

Than navigate to http://localhost:3000 using your favorite webbrowser. Make sure you have used "phantomshot init" before.

npm.io

Command overview

phantomshot --help - Displays the command overview
phantomshot init - Creates a new PhantomShot project
phantomshot test test_ids... - Executes all or the specified tests
phantomshot confirm \<test_id> \<shot_name> - Confirms the given shoot of the given test
phantomshot serve --port=3000 - Starts the build in HTTP Server.

Knows issues

At the moment there is an issue in Horseman or PhantomJs. Sometimes tests could break or hang because PhantomJs cannot open a URL for some reason. This error occours randomly and i am not able to catch those errors in PhantomShot programmatically at the moment.