@sosnoob/crawlit NPM

Crawlit - Dofus & Dofus-Touch encyclopedia parser - Build your own Dofus API

In order to respond to a need expressed by the community, I create an encyclopedia parser for Dofus available for :

Windows
OSX
Linux

Download : All versions are available here

:warning: Don't use the tool for more than 700 items per hour (for Dofus-Touch).

Crawlit was built to be very light, it can run on almost every configurations.

In fact Crawlit is a tool :

Developed by the dofus community, for the community
100% performance : Developed to be very light
Simple to use | Plug and use
Resume parsing feature after errors (connection off / 429 error / ...)
App packaged into executable, no need to install something
English & French encyclopedia supported (french item & english item)

App launched	App at the end

JSON files also provided in the repository

In case you wan't directly the dofus JSON files, I oftenly upload up-to-date encyclopedia JSON in the data/ folder.

JSON format

An item result example

{  
  "_id":"item ID",
  "name":"item name",
  "description":"item description.",
  "lvl":"item lvl",
  "type":"item type",
  "imgUrl":"image url of the item",
  "url":"Item's link",
  "stats":[ "many statistics line", "stat 2", ["..."], "stat n" ],
  "condition":[ "many conditions line", "condition 2", ["..."], "condition n" ],
  "set":{  
     "equipments":[itemId, itemId ...],
     "weapons":[itemId, itemId ...],
  }
}

Getting Started

With Npm Install globally crawlit :

npm install -g @sosnoob/crawlit
crawlit

With github Clone & access this repository locally :

git clone https://github.com/raczak/crawlit-dofus-encyclopedia-parser.git
cd crawlit-dofus-encyclopedia-parser

Install dependencies with npm :

npm install

:warning: There is an issue with npm start on some CLI, in this case prefere use node lib/app.js.

Optional To build packaged app

If you want to generate packaged app (executable) with pkg dependency, you MUST install zeit/pkg in npm global context.
Please follow Pkg documentation for troubleshooting.