11.1.2 β€’ Published 6 months ago

punuy-datasets v11.1.2

Weekly downloads
-
License
ISC
Repository
github
Last release
6 months ago

punuy-datasets

Node.js CI

A collection of quantitative lexical semantic relations datasets, gathered from the academic literature, for the evaluation of semantic measures between concepts or entities. Currently focused on the English and Portuguese languages.

Datasets

English

#IDNameYearDomainSim/RelReferences
1rg65Rubenstein and Goodenough (1965)1965generalsimpaper πŸ“‘, website 🌐
2mervis210Mervis 2101975generalrelpaper πŸ“‘
3tversky42Tversky421979generalsimpaper πŸ“‘
4mc30Miller and Charles (1991)1991generalsimpaper πŸ“‘
5medin34Medin 341993generalsimpaper πŸ“‘
6resnik28Miller and Charles Replication1999generalsimpaper πŸ“‘, website 🌐
7rd27Resnik and Diab 20002000generalsimpaper πŸ“‘, website 🌐
8ws353WordSimilarity-3532002generalsimpaper πŸ“‘, website 🌐
9yp130YP-1302005generalsimpaper πŸ“‘
10bg100kPrinceton Evocation Data 0.42006generalevopaper πŸ“‘, website 🌐
11mesh2MeSH22006biomedicalsimpaper πŸ“‘, website 🌐
12zie55Ziegler et al. (2006)2006generalrelpaper πŸ“‘
13mayoSRSMayoSRS2007biomedicalrelpaper πŸ“‘, website 🌐
14miniMSRSMiniMayoSRS2007biomedicalrelpaper πŸ“‘, website 🌐
15gm30Gracia and Mena (2008)2008generalrelpaper πŸ“‘
16ps65Pirro and Seco (2008)2008generalsimpaper πŸ“‘
17ws353splitWordSim-353 Similarity and Relatedness Gold Standard2009generalrel/simpaper πŸ“‘, website 🌐
18umnsrsUMNSRS2010biomedicalrel/simpaper πŸ“‘, website 🌐
19mt287MT2872011generalrelpaper πŸ“‘, website 🌐
20atlasify240Atlasify2402012generalrelpaper πŸ“‘, website 🌐
21mturk771MTurk7712012generalrelpaper πŸ“‘, website 🌐
22reword26REWOrD2012generalrelpaper πŸ“‘, website 🌐
23scws2003SCWS2012generalsimpaper πŸ“‘, website 🌐
24ma28Martinez & Aldana2013generalsimpaper πŸ“‘, website 🌐
25rel122Rel-1222013generalrelpaper πŸ“‘, website 🌐
26srw2034Stanford Rare Word (RW) Similarity Dataset2013biomedicalsimpaper πŸ“‘, website 🌐
27wp300WP3002013generalsimpaper πŸ“‘, website 🌐
28baker143Baker 1432014generalsimpaper πŸ“‘, website 🌐
29geresid50GeReSiD2014geographicalrel/simpaper πŸ“‘, website 🌐
30men3000MEN2014generalrelpaper πŸ“‘, website 🌐
31sl7576SL75762014generalsimpaper πŸ“‘, website 🌐
32lm659Landrigan & Mirman (2015)2015generalrel/simpaper πŸ“‘, website 🌐
33simlex999SimLex-9992015generalsimpaper πŸ“‘, website 🌐
34tr9856TR98562015generalrelpaper πŸ“‘, website 🌐
35simverb3500SimVerb-35002016generalsimpaper πŸ“‘, website 🌐
36umnsrsModUMNSRS (modified)2016biomedicalrel/simpaper πŸ“‘, website 🌐
37semeval17SemEval-2017 Task 2.12017generalsimpaper πŸ“‘, website 🌐
38ali27Ali et al. 20182018generalsimpaper πŸ“‘
39card660Card-6602018generalsimpaper πŸ“‘, website 🌐
40gtrdGTRD2018geographicalrelpaper πŸ“‘, website 🌐
41word19kWikipedia Oriented Relatedness Dataset2018generalrelpaper πŸ“‘, website 🌐
42multisimlexMulti-SimLex2020generalsimpaper πŸ“‘, website 🌐
43emoconEmotion concepts 20222022generalsimpaper πŸ“‘, website 🌐
44sstars13Shooting-Stars2023generalrelpaper πŸ“‘, website 🌐

Portuguese

#IDNameYearDomainSim/RelReferences
1pt65PT652014generalrelpaper πŸ“‘, website 🌐
2lxrw2034LX-Rare Word Similarity Dataset2017generalsimpaper πŸ“‘, website 🌐
3lxsimlex999LX-SimLex-9992017generalsimpaper πŸ“‘, website 🌐
4lxws353LX-WordSim-3532017generalrelpaper πŸ“‘, website 🌐
5pap900PAP9002024generalrel/simwebsite 🌐

Use with Node.js

If you want to programatically use the datasets' data and metadata, install the Node.js wrapper code:

npm install

Then you can access the datasets like this:

import { mc30, rg65 } from "punuy-datasets";

for (const part of mc30.partitions) {
  console.log(part.id, part.data.length); // print partition id and size
}

Other uses

If you want to use another language or just want to access the data files directly, each dataset has its own folder inside profiles/.

Each dataset has a dataset.json file with its metadata and information about its partitions. Then, for each partition with id p_id, there is a p_id.part.json file containing the pairs and their ratings.

Adding a new dataset

If you can create the dataset profile yourself, PRs are welcome! You can find the dataset schema (in Typescript) here.

Or you can open an issue. Please include:

  1. a link to the dataset homepage (if it exists),
  2. a link to download the dataset, and
  3. a link to the paper describing it.

I'll get to it as soon as I have some free time ;)

Types and schema

TypeScript definitions can be found at src/lib/types.ts.

Automatically generated JSON schema definitions can be found at dataset-schema.json and partition-data-schema.json.

Automatically generated documentation can be found at docs/.

Datasets copyright and licenses

Did you find a problem with the licensing of a dataset? We did our best to find and include the licenses to all the datasets and to respect copyrights, but we might have missed something!

Please open an issue and we'll fix it ASAP.

See also

For cross-lingual datasets and datasets in other languages please check the following links:

Bugs and stuff

Open a GitHub issue or, preferably, send me a pull request.

License

The MIT License (MIT)

Copyright (c) 2025 AndrΓ© Santos andrefs@andrefs.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

11.1.2

6 months ago

11.0.7

6 months ago

11.0.6

6 months ago

11.0.5

6 months ago

11.0.3

6 months ago

11.0.2

6 months ago

11.0.1

6 months ago

10.2.0

6 months ago

10.1.1

6 months ago

10.1.0

7 months ago

10.0.3

7 months ago

10.0.2

7 months ago

10.0.1

7 months ago

10.0.0

7 months ago

9.2.2

8 months ago

9.2.1

8 months ago

8.1.1

8 months ago

8.1.0

8 months ago

8.0.3

9 months ago

8.0.1

9 months ago

7.1.0

9 months ago

7.0.5

9 months ago

7.0.4

9 months ago