tweet-analysis v1.0.5

Tweet analyser/querying

Usage

Install

You can install the command line tool with npm:

npm -g install tweet-analysis

All the dependencies will automatically be installed and a symbolic link will be created to add the command to your path. You can then directly type anywhere tweet-analyser in your terminal to use the tool.

Uninstall

You can uninstall the command line tool with npm:

npm -g uninstall tweet-analysis

How to

Export full text tweets in a structured format

To export data from tweets and display them in the terminal, you will need to type in this command:

 tweet-analyser inspect [dataPath] [data]

(the are only there for formatting purposes, do not add them in the commands)

Other parameters can be added to this command, separated by spaces :

• [dataPath] The path to a folder containing the files you want to use. Will be recursively searched in for csv files

• [data] (repeatable as much as you like). Datas may include datas such as user_name, lang, hashtags, created_at, retweet_count and * for everything

• [--table] to organize data in a structured table

• [--filter "data operator value"] to display the data meeting the requirement specified in the filter field. It can also be used to filter by date if used this way : [--filter "created_at > 23/03/2018 00:00:00"]. (If no hour is written midnight is implied.) You can use this option multiple times, allowing you to limit the time period of the tweets for exemple. Operators include =, !=, <, <=, >, >=. [--filter "data operator value"] can also be written as [--f "data operator value"]

• [--sort [data]] to sort the data extracted by the data specified in the field

• [--reverse] to reverse the order of the data

• [--top [number]] to only display an array of data from the first in position to [number]

• [--export [file_path]] to export the data gathered with the command into a new file, which path is specified in the field. Adding [--quiet] at the end will only export the data without displaying it in the terminal

• [--count] to only display the number of items found

Export tweets data into a graph

To export data from tweets and display them in a graph, you will need to type in this command:

 tweet-analyser graph [data_path] [columX_data] [columnY_data] [VegaLite_template_file_path] [export_graph_file_path]

(the are only there for formatting purposes, do not add them in the commands)

The parameter [--filter] can also be used here.

Exemples

Here are two exemples of graphs with their respective Vega-Lite templates:

• A bar chart giving the number of retweets according to the language of the Twitter user The command used : tweet-analyser graph ..\data lang retweet_count ..\vegaLiteTemplate.json' chart.svg. The content of the vegaLiteTemplate.json file:

{
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "title": "%yColumn%  according to %xColumn%",
    "width": 1280,
    "height": 720,
    "mark": "bar",
    "encoding": {
        "x": {
            "field": "%xColumn%",
            "type": "%xColumnType%"
        },
        "y": {
            "field": "%yColumn%",
            "type": "%yColumnType%",
            "aggregate": "sum"
        }
    }
}

• Using the same template, the following result can be obtained:
• You can also make time charts:

With the following template:

{
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "title": "%yColumn%  according to %xColumn%",
    "width": 1280,
    "height": 720,
    "mark": "bar",
    "encoding": {
        "x": {
            "timeUnit": "utcyearmonthdate",
            "field": "%xColumn%",
            "type": "%xColumnType%"
        },
        "y": {
            "field": "%yColumn%",
            "type": "%yColumnType%",
            "aggregate": "sum"
        }
    }
}

• Last exemple: And the vega-lite template:

{
    "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
    "title": "%yColumn%  according to %xColumn%",
    "width": 1280,
    "height": 720,
    "mark": "point",
    "encoding": {
        "x": {
            "field": "%xColumn%",
            "type": "%xColumnType%"
        },
        "y": {
            "field": "%yColumn%",
            "type": "%yColumnType%"
        }
    }
}

Build, test and maintain the source code

Download the repository

You can use simply SVN to download the trunk:

svn checkout svn+ssh://<username>@svn.code.sf.net/p/team-burton-gl02/svn/trunk team-burton-gl02-svn

Install the dependencies

You just have to go to the folder you just created with your terminal of choice and type:

npm install

NPM will automatically install all the dependencies needed.

Build the code

To take as little space as possible on the user system we use a build process that minifies the code in the src folder. To run it you can type in the following command in the folder:

npm run build

This will create a dist folder containing the minified code. A monitoring version of the script also exists: it will automatically watch the src folder and build on new changes. To start it type npm run buildmon.

Test the code

To test the code we use Jest to run tests located in the tests folder. To run the tests you can type the following command in the folder:

npm test

A monitoring version of the script also exists: it will automatically watch the src folder and test on new changes. To start it type npm run testmon.

Maintain the code

The code is stored in the src folder.

To run the code you don't need to type the name of the main file: node . will do the job just fine.

To test the code in a more real scenario you can choose to add the code to your PATH environment variable. To do that you just have to type the following on any OS in the project's folder:

npm link

You can now use the command everywhere on your computer by typing tweet-analyser in the terminal. Warning : make sure you do that when you have not installed the production version of the code like in the usage section. If that is the case uninstall that version before linking the local code.

Unlinking can be done simply with:

npm unlink

Help

The project gives an eslint configuration file and an editorconfig one: you are highly encouraged to follow theses. Most code editors have a plugin to take care of theses. You can also run manually run eslint with eslint <path> --fix to try to find and fix problems in your files. JsDoc is used to document most of the functions, so if you use an editor like Visual Studio Code you will have informations given by the intellisense.

Problems

Given that the user's location is set by the user, it is not possible to locate the tweets on a map. This specification has thus not been implemented.