modelzoo v0.0.4

modelzoo.js

The repo is based on gpt-tfjs and its TypeScript fork by lukemovement. A lot of models share common layers and preprocessing methods so it makes sense to continue the work in a single repo rather than copy-pasting the same transformer architecture over and over again or having to maintain multiple repos for each layer. Currently not a lot to see here.

🔴 Work in progress 🔴

Current models

• GPT
• Vision Transformer
• CLIP - in progress

Custom layers

• Logging
• Range
• Positional embedding
• Class embedding
• Slice
• GELU
• QuickGELU
• Multihead attention
• MLP
• Residual Attention block
• Transformer

Optimizers

• AdamW