projet-gl02 v1.0.4
Tweet analyser/querying
[members]
Usage
Install
You can install the command line tool with npm:
npm -g install tweet-analysis
All the dependencies will automatically be installed and a symbolic link will be created to add the command to your path. You can then directly type anywhere tweet-analyser
in your terminal to use the tool.
Uninstall
You can uninstall the command line tool with npm:
npm -g uninstall tweet-analysis
How to
Export full text tweets in a structured format
To export data from tweets and display them in the terminal, you will need to type in this command:
tweet-analyser inspect [dataPath] [data]
(the are only there for formatting purposes, do not add them in the commands)
Other parameters can be added to this command, separated by spaces :
[dataPath]
The path to a folder containing the files you want to use. Will be recursively searched in for csv files[data]
(repeatable as much as you like). Datas may include datas such as user_name, lang, hashtags, created_at, retweet_count and * for everything[--table]
to organize data in a structured table[--filter "data operator value"]
to display the data meeting the requirement specified in the filter field. It can also be used to filter by date if used this way :[--filter "created_at > 23/03/2018 00:00:00"]
. (If no hour is written midnight is implied.) You can use this option multiple times, allowing you to limit the time period of the tweets for exemple. Operators include =, !=, <, <=, >, >=.[--filter "data operator value"]
can also be written as[--f "data operator value"]
[--sort [data]]
to sort the data extracted by the data specified in the field[--reverse]
to reverse the order of the data[--top [number]]
to only display an array of data from the first in position to[number]
[--export [file_path]]
to export the data gathered with the command into a new file, which path is specified in the field. Adding[--quiet]
at the end will only export the data without displaying it in the terminal[--count]
to only display the number of items found
Export tweets data into a graph
To export data from tweets and display them in a graph, you will need to type in this command:
tweet-analyser graph [data_path] [columX_data] [columnY_data] [VegaLite_template_file_path] [export_graph_file_path]
(the are only there for formatting purposes, do not add them in the commands)
The parameter [--filter]
can also be used here.
Exemples
Here are two exemples of graphs with their respective Vega-Lite templates:
- A bar chart giving the number of retweets according to the language of the Twitter user
The command used :
tweet-analyser graph ..\data lang retweet_count ..\vegaLiteTemplate.json' chart.svg
. The content of thevegaLiteTemplate.json
file:
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"title": "%yColumn% according to %xColumn%",
"width": 1280,
"height": 720,
"mark": "bar",
"encoding": {
"x": {
"field": "%xColumn%",
"type": "%xColumnType%"
},
"y": {
"field": "%yColumn%",
"type": "%yColumnType%",
"aggregate": "sum"
}
}
}
- Using the same template, the following result can be obtained:
- You can also make time charts:
With the following template:
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"title": "%yColumn% according to %xColumn%",
"width": 1280,
"height": 720,
"mark": "bar",
"encoding": {
"x": {
"timeUnit": "utcyearmonthdate",
"field": "%xColumn%",
"type": "%xColumnType%"
},
"y": {
"field": "%yColumn%",
"type": "%yColumnType%",
"aggregate": "sum"
}
}
}
- Last exemple: And the vega-lite template:
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"title": "%yColumn% according to %xColumn%",
"width": 1280,
"height": 720,
"mark": "point",
"encoding": {
"x": {
"field": "%xColumn%",
"type": "%xColumnType%"
},
"y": {
"field": "%yColumn%",
"type": "%yColumnType%"
}
}
}
Build, test and maintain the source code
Download the repository
You can use simply SVN to download the trunk:
svn checkout svn+ssh://<username>@svn.code.sf.net/p/team-burton-gl02/svn/trunk team-burton-gl02-svn
Install the dependencies
You just have to go to the folder you just created with your terminal of choice and type:
npm install
NPM will automatically install all the dependencies needed.
Build the code
To take as little space as possible on the user system we use a build process that minifies the code in the src
folder. To run it you can type in the following command in the folder:
npm run build
This will create a dist
folder containing the minified code.
A monitoring version of the script also exists: it will automatically watch the src
folder and build on new changes. To start it type npm run buildmon
.
Test the code
To test the code we use Jest to run tests located in the tests
folder. To run the tests you can type the following command in the folder:
npm test
A monitoring version of the script also exists: it will automatically watch the src
folder and test on new changes. To start it type npm run testmon
.
Maintain the code
The code is stored in the src
folder.
To run the code you don't need to type the name of the main file: node .
will do the job just fine.
To test the code in a more real scenario you can choose to add the code to your PATH environment variable. To do that you just have to type the following on any OS in the project's folder:
npm link
You can now use the command everywhere on your computer by typing tweet-analyser
in the terminal.
Warning : make sure you do that when you have not installed the production version of the code like in the usage section. If that is the case uninstall that version before linking the local code.
Unlinking can be done simply with:
npm unlink
Help
The project gives an eslint configuration file and an editorconfig one: you are highly encouraged to follow theses. Most code editors have a plugin to take care of theses.
You can also run manually run eslint with eslint <path> --fix
to try to find and fix problems in your files.
JsDoc is used to document most of the functions, so if you use an editor like Visual Studio Code you will have informations given by the intellisense.
Problems
Given that the user's location is set by the user, it is not possible to locate the tweets on a map. This specification as thus not been implemented.