# glsl-tokenizer

> r/w stream of glsl tokens

Latest version **2.1.5** (published 2018-10-06) · MIT license · 0 weekly downloads

## Install

```sh
npm install glsl-tokenizer
pnpm add glsl-tokenizer
yarn add glsl-tokenizer
bun add glsl-tokenizer
```

## Health

**Score 15/100 (F)** — status: abandoned.

Positive: no vulnerabilities.

Warnings: low downloads; no types; no esm support.

Negative: abandoned; low maintenance score.

## Facts

| | |
|---|---|
| Version | 2.1.5 |
| Published | 2018-10-06 |
| First published | 2012-11-18 |
| Weekly downloads | 0 |
| License | MIT |
| TypeScript types | none |
| Module format | CommonJS |
| Dependencies | 1 |
| Unpacked size | 19.9 KB |
| Known vulnerabilities | 0 |
| Install scripts | no |
| GitHub stars | 52 |
| Author | Chris Dickinson |
| Maintainers | bpostlethwaite, chrisdickinson, dfcreative, erkaman, gre, hughsk, mattdesl, mikkoh, mikolalysenko, rezaali, rreusser, substack, tatumcreative, thibauts, vorg, wwwtyro, yoshuawuyts |
| Keywords | glsl, tokenizer, stream |

## Links

- npm: https://www.npmjs.com/package/glsl-tokenizer
- Repository: https://github.com/gl-modules/glsl-tokenizer
- Homepage: https://github.com/gl-modules/glsl-tokenizer#readme
- Issues: https://github.com/gl-modules/glsl-tokenizer/issues
- npm.io page: https://npm.io/package/glsl-tokenizer

## Dependencies (1)

- [through2](https://npm.io/package/through2.md) ^0.6.3

## Alternatives

- [byte-size](https://npm.io/package/byte-size.md) — 2.1M weekly downloads
- [speed-limiter](https://npm.io/package/speed-limiter.md) — 16.0K weekly downloads
- [@powersync/node](https://npm.io/package/@powersync/node.md) — 10.9K weekly downloads
- [@ledgerhq/coin-cardano](https://npm.io/package/@ledgerhq/coin-cardano.md) — 1.0K weekly downloads
- [@jayesol/jayeson.lib.streamfinder](https://npm.io/package/@jayesol/jayeson.lib.streamfinder.md) — 1.0K weekly downloads

## Recent versions

- 2.1.5 (latest) — 2018-10-06
- 2.1.4 — 2018-08-27
- 2.1.3 — 2018-08-27
- 2.1.2 — 2016-05-26
- 2.1.1 — 2016-02-25
- 2.1.0 — 2016-02-25
- 2.0.2 — 2015-04-05
- 2.0.1 — 2015-03-29
- 2.0.0 — 2015-03-06
- 1.1.1 — 2014-07-08
- 1.1.0 — 2014-07-05
- 1.0.0 — 2014-07-05
- 0.0.9 — 2014-05-25
- 0.0.8 — 2012-11-25
- 0.0.7 — 2012-11-25
- … 7 more at https://npm.io/package/glsl-tokenizer/versions

## README

# glsl-tokenizer [![Build Status](https://travis-ci.org/glslify/glsl-tokenizer.svg?branch=master)](https://travis-ci.org/glslify/glsl-tokenizer)

Maps GLSL string data into GLSL tokens, either synchronously or using a
streaming API.

``` javascript
var tokenString = require('glsl-tokenizer/string')
var tokenStream = require('glsl-tokenizer/stream')
var fs = require('fs')

// Synchronously:
var tokens = tokenString(fs.readFileSync('some.glsl'))

// Streaming API:
fs.createReadStream('some.glsl')
  .pipe(tokenStream())
  .on('data', function(token) {
    console.log(token.data, token.position, token.type)
  })
```

# API

## tokens = require('glsl-tokenizer/string')(src, [opt])

Returns an array of `tokens` given the GLSL source string `src`

You can specify `opt.version` string to use different keywords/builtins, such as `'300 es'` for WebGL2. Otherwise, will assume GLSL 100 (WebGL1).

```js
var tokens = tokenizer(src, {
  version: '300 es'
})
```

## stream = require('glsl-tokenizer/stream')([opt])

Emits 'data' events whenever a token is parsed with a token object as output.

As above, you can specify `opt.version`.

# Tokens

```javascript
{ 'type': TOKEN_TYPE
, 'data': "string of constituent data"
, 'position': integer position within the GLSL source
, 'line': line number within the GLSL source
, 'column': column number within the GLSL source }
```

The available token types are:

* `block-comment`: `/* ... */`
* `line-comment`: `// ... \n`
* `preprocessor`: `# ... \n`
* `operator`: Any operator. If it looks like punctuation, it's an operator.
* `float`: Optionally suffixed with `f`
* `ident`: User defined identifier.
* `builtin`: Builtin function.
* `eof`: Emitted on `end`; data will === `'(eof)'`.
* `integer`
* `whitespace`
* `keyword`

# License

MIT, see [LICENSE.md](LICENSE.md) for further information.

---
_Source: https://npm.io/package/glsl-tokenizer · Machine-readable twin of the npm.io package page. Health data is recomputed on every publish._