leetfilter v0.1.1
leetfilter - Extensible Swear Filter
This is a library to help you filter out swear words and people who attempt to bypass filters using leets!
What are 'leets'?
Using leets is when you type in a symbol that looks like a letter.
If the original word is testswear
, a version with leets would be:
+35+5vv34r
testsw.ear
...or...
|e$7?uu&@12
testsw.ear.
These are a bit extreme, but you get the idea. This library can detect these.
Leetfilter also supports being used in discord bots, since it filters out regional indicators too.
How to Use
Install via npm -i leetfilter
or yarn add leetfilter
.
Leetfilter works with both CJS and ESM! Here are examples of how to initialize it:
CJS
const { LeetFilter } = require("leetfilter");
const swear = new LeetFilter({
debug: false,
filler: "*",
dict: 'small',
precompile: true,
customwords: ["testswear"]
});
ESM
import { LeetFilter } from "leetfilter";
const swear = new LeetFilter({
debug: false,
filler: "*",
dict: 'small',
precompile: true,
customwords: ["testswear"]
});
Leetfilter was created in TypeScript, so TypeScript is also supported!
Documentation
Initialization
Leetfilter takes in 5 optional parameters on initialization.
Dictionaries and Precompiling
There are currently 3 different dictionaries: tiny, small, and large.
Enum
There is an enum for dictionary type, which can be imported from leetfilter/other
. There are three values, Tiny
, Small
, and Large
. However, this enum isn't required to be used in the dict
parameter.
Tiny Dictionary
The tiny dictionary only contains one word: testswear. This is solely used for testing.
Value for dict
: Dictionary.Tiny
or 'tiny'
Small Dictionary
The small dictionary contains about 200 swear words. This is the reccomended dictionary to use since the large one takes a long time to use.
Timings for compiling: 0.1s Timings for filtering: 0.1-0.3s
Timings performed on an unboosted hacker repl on replit.com
Value for dict
: Dictionary.Small
or 'small'
Large Dictionary
The large dictionary contains about 2000 swear words. This dictionary is not reccomended because of it's large size. However, if you have a powerful host, it may be possible to use.
Timings for compiling: 1.2s Timings for filtering: 2-3s
Timings performed on an unboosted hacker repl on replit.com
Value for dict
: Dictionary.Large
or 'large'
Precompiling
Precompiling should usually be kept on so that if the dictionary ever gets changed or if custom words are added.
If custom words are added, leetfilter will give a warning if precompiling is turned off.
If you ever have issues with words being filtered when they are not meant to or vice versa, you should turn on precompiling.
Precompiling ONLY occurs when the LeetFilter
class is initialized. It will not affect the speed of your running program.
async LeetFilter.compile(): Promise<LeetFormat>
An asynchronous function that returns a Promise of type LeetFormat
([index: string]: string
). It compiles from the dictionary and the leet file.
LeetFilter.compileSync(): LeetFormat
A synchronous function that does the same as LeetFilter.compile()
, used for precompiling.
async LeetFilter.compileToFile(): Promise<LeetFormat>
Same as asynchronous LeetFilter.compile()
but it saves it to the buffer file to be used later.
LeetFilter.compileToFileSync(): LeetFormat
Same as before, but synchronous. Used for precompiling.
LeetFilter.setConfig(config: Config)
Same parameters as constructor. Calls constructor again with these parameters.
LeetFilter.censorText(word: string, amnt: number = 3): string
Takes in a word and a number, and uses this.config.filler
to change the last amnt
characters of word
and replace them with this.config.filler
.
get LeetFilter.getDict(): string
Returns file path to dictionary file.
async LeetFilter.filter(input: string): Promise<string | false>
Checks against compiled leet regex file to see whether input
contains swears, and if so returns a string with all the letters in the swear replaced with this.config.filler
. If there are no swears, it returns false.
Example code
This section is WIP.
Using an asynchronous function
// CJS
const { LeetFilter } = require("leetfilter");
// ESM
import { LeetFilter } from "leetfilter";
const swear = new LeetFilter({
debug: false,
filler: "*",
dict: 'small',
precompile: true,
customwords: ["testswear"] // added 'testswear' as a custom word
});
const main = async () => {
// Testing 'not a swear'
let res = await swear.filter("not a swear");
console.log(res); // result: false
// Testing 'testswear'
res = await swear.filter("testswear");
console.log(res); // result: '*********'
// Testing 'testsweara'
res = await swear.filter("testsweara");
console.log(res); // result: '*********a'
// Testing 'a testswear a'
res = await swear.filter("a testswear a");
console.log(res); // result: 'a ******** a'
// Testing '†3575vv34r'
res = await swear.filter("†3575vv34r");
console.log(res); // result: '*********'
}
main();
Contribution Information
./src
This directory contains all the source files. They should be .ts
files in camel case. All variable/methods should be camel case, but classes should be upper camel case.
./src/index.ts
The main file with the LeetFilter
class in it.
./src/other.ts
Where all the enums and types are kept.
./src/data
Directory that contains all the JSON data needed.
./src/data/compiled.leets.json
Where compiling to file puts the compiled data into. This shouldn't be touched.
./src/data/dictionary.*.json
Dictionaries files. Contains all the swear words. There is a tiny, small, and large dictionary.
./src/data/leets.json
A JSON file containing a regex for every possible leet for each letter. More leets can be added.
./dist
Where the compiled code sits. CJS files are .js
and ESM files are .mjs
. Compiled using tsup
.
./example
Directory containing example code.
./example/index.js
Example code, runnable using yarn example
.
./package.json
Scripts
yarn build
Builds CJS and ESM files in ./dist
and runs yarn data
. Uses tsup.
yarn example
Runs example code. This doesn't use ESM and also doesn't import the library. It just imports the ./dist/index.js
file. (File located in ./example/index.js
)
yarn data
Copies the data directory inside ./src
to ./dist
.
Credits
By Anvay Mathur (anvay.one@gmail.com) and Alex Shen (alex.from.sas@gmail.com)
Licensed under the MIT license, view more at LICENSE.txt.