@partnersync/flatten NPM

@partnersync/flatten

Simple, intelligent codebase flattening for LLMs. Creates a structured YAML representation of your repository and a single text file containing all your code - perfect for providing context to AI tools.

Features

🎯 Smart defaults - Just run flatten and it works
🔍 Git-aware - Includes commit hash in output filenames
📁 Intelligent filtering - Respects .gitignore and excludes lock files
🏷️ Version tracking - Timestamped outputs for easy chronological sorting
📦 Zero configuration - No config files needed
🚀 Lightweight - Minimal dependencies, fast execution
🚫 Smart exclusions - Use --exclude to filter out specific patterns
📊 File insights - Shows largest files when output exceeds 250KB
🖼️ Binary handling - Intelligently omits binary file contents
🧹 Built-in cleanup - Remove old flatten outputs with flatten cleanup

Installation

# Install globally
npm install -g @partnersync/flatten

# Or use directly with npx
npx @partnersync/flatten

Usage

Basic Flattening

# Basic usage - creates timestamped output files
flatten

# Process specific directories
flatten src/ lib/

# Include specific files (even if normally ignored)
flatten src/ README.md package.json

# Exclude patterns
flatten --exclude "*.png,*.jpg,infra/sql"

# Structure only (no file contents)
flatten --no-contents

# Allow non-git directories
flatten --no-git

# Override file limit
flatten --max-files 5000

# Show version
flatten --version

# Show help
flatten --help

Cleanup Old Files

# Show what would be removed (dry run)
flatten cleanup --dry-run

# Remove all flatten output files
flatten cleanup

# Remove files older than 7 days
flatten cleanup --days 7

# Get help on cleanup options
flatten cleanup --help

Output Files

Flatten creates two files with smart, timestamped naming:

flatten-YYYYMMDD-HHMMSS-<git-hash>-<status>.yaml - YAML structure of your repository
flatten-YYYYMMDD-HHMMSS-<git-hash>-<status>.txt - Flattened content of all files

Where:

YYYYMMDD-HHMMSS is the timestamp when flatten was run
<git-hash> is the short git commit hash (7 characters)
<status> is either 'clean' or 'dirty' based on git status

For non-git directories (with --no-git):

flatten-YYYYMMDD-HHMMSS.yaml - YAML structure
flatten-YYYYMMDD-HHMMSS.txt - Flattened content

Example Output

In a git repository:

flatten-20250610-121900-abc1234-clean.yaml    # Repository structure
flatten-20250610-121900-abc1234-clean.txt     # Flattened content

After making changes:

flatten-20250610-143200-abc1234-dirty.yaml    # Shows uncommitted changes exist
flatten-20250610-143200-abc1234-dirty.txt     # Includes uncommitted changes

With exclusions:

$ flatten --exclude "*.png,infra/sql"

✓ Created flatten-20250610-151230-abc1234-clean.yaml
✓ Created flatten-20250610-151230-abc1234-clean.txt (487.2 KB)
✓ Excluded 23 files matching patterns
✓ Omitted 3 binary files from content

Largest files:
  src/runtime/services/database/mssql-database.ts (31.4 KB)
  tools/generators/schema-generator.ts (28.7 KB)
  docs/architecture/component-model.md (24.3 KB)
  src/runtime/components/extractors/html-extractor.ts (19.8 KB)
  tests/integration/full-pipeline.test.ts (17.2 KB)

Ignoring Flatten Output Files

To prevent flatten output files from being committed to your repository, add these patterns to your .gitignore:

# Flatten output files
flatten-*.txt
flatten-*.yaml

This will ignore all flatten-generated files while keeping your repository clean.

Managing Output Files

The new timestamped naming makes it easy to manage multiple flatten outputs:

# List all flatten outputs (sorted by timestamp)
$ ls -l flatten-*.{yaml,txt}
-rw-r--r--  1 user  staff    869 Jun  8 12:29 flatten-20250608-122900-9e228ad-clean.yaml
-rw-r--r--  1 user  staff  52459 Jun  8 12:29 flatten-20250608-122900-9e228ad-clean.txt
-rw-r--r--  1 user  staff    318 Jun  8 14:54 flatten-20250608-145400-74706ff-dirty.yaml
-rw-r--r--  1 user  staff  15592 Jun  8 14:54 flatten-20250608-145400-74706ff-dirty.txt
-rw-r--r--  1 user  staff    869 Jun 10 12:19 flatten-20250610-121900-74706ff-clean.yaml
-rw-r--r--  1 user  staff  53297 Jun 10 12:19 flatten-20250610-121900-74706ff-clean.txt

# Clean up old files
$ flatten cleanup --dry-run
Would remove 4 files:
  - flatten-20250608-122900-9e228ad-clean.yaml (2 days old)
  - flatten-20250608-122900-9e228ad-clean.txt (2 days old)
  - flatten-20250608-145400-74706ff-dirty.yaml (2 days old)
  - flatten-20250608-145400-74706ff-dirty.txt (2 days old)

$ flatten cleanup --days 1
✓ Removed 4 files older than 1 days

What Gets Included?

Flatten includes all files except:

Git ignored files (respects .gitignore)
Lock files that don't add value for LLMs:
- package-lock.json, yarn.lock, pnpm-lock.yaml
- composer.lock, Gemfile.lock, poetry.lock, Cargo.lock
Minified files: *.min.js, *.min.css
Previous flatten outputs (files matching the output pattern)
Files matching your exclude patterns (when using --exclude)

Binary File Handling

Binary files (images, PDFs, executables, etc.) are:

Included in the YAML structure (so you can see what files exist)
Omitted from the text content (replaced with [Binary file omitted])

This keeps your output focused on readable code while maintaining awareness of all project files.

Exclude Patterns

The --exclude option accepts comma-separated patterns following gitignore-style rules:

# Exclude by extension
flatten --exclude "*.png,*.jpg"

# Exclude specific directories
flatten --exclude "dist,build"

# Exclude directory at any level
flatten --exclude "node_modules"

# Exclude from root only
flatten --exclude "/temp"

# Complex patterns
flatten --exclude "*.log,temp/,/cache,docs/*.pdf"

Smart Git Integration

When run inside a git repository, flatten:

Automatically switches to the repository root
Preserves your intended paths (relative to where you run it)
Includes the commit hash in output filenames
Detects uncommitted changes (dirty state)

# Example: Running from a subdirectory
cd ~/projects/MyApp/frontend
flatten src/

# Output:
# Switches to git root: ~/projects/MyApp
# Processes: frontend/src/
# Creates: flatten-20250610-152000-abc1234-clean.yaml and .txt

Safety Features

File count limit: Default 1000 files (override with --max-files)
Git repository boundary: Won't accidentally flatten your entire home directory
Clear error messages: Tells you exactly what to do when limits are hit
File size insights: Shows largest files when output exceeds 250KB

Use Cases

LLM Context: Provide complete codebase context to ChatGPT, Claude, or other AI assistants
Code Reviews: Share entire project structure and content easily
Documentation: Generate file inventories and snapshots
Archiving: Create readable snapshots tied to specific commits
Version Tracking: Timestamped outputs make it easy to track changes over time

Example Session

$ cd my-awesome-project
$ flatten

✓ Created flatten-20250610-152000-abc1234-clean.yaml
✓ Created flatten-20250610-152000-abc1234-clean.txt (2.3 MB)

$ # Make some changes...
$ echo "TODO: fix this" >> src/index.js
$ flatten

✓ Created flatten-20250610-153045-abc1234-dirty.yaml
✓ Created flatten-20250610-153045-abc1234-dirty.txt (2.3 MB)

$ # Exclude large SQL files
$ flatten --exclude "*.sql"

✓ Created flatten-20250610-154500-abc1234-dirty.yaml
✓ Created flatten-20250610-154500-abc1234-dirty.txt (1.8 MB)
✓ Excluded 15 files matching patterns
✓ Omitted 2 binary files from content

Largest files:
  src/services/data-processor.ts (89.2 KB)
  docs/images/architecture.png (45.3 KB) [binary]
  src/utils/validators.ts (38.1 KB)
  tests/integration/full-suite.test.ts (35.2 KB)
  src/models/complex-model.ts (31.7 KB)

$ # Check what we've created over time
$ ls -lt flatten-*.txt | head -5
-rw-r--r--  1 user  staff  1887232 Jun 10 15:45 flatten-20250610-154500-abc1234-dirty.txt
-rw-r--r--  1 user  staff  2412032 Jun 10 15:30 flatten-20250610-153045-abc1234-dirty.txt
-rw-r--r--  1 user  staff  2412001 Jun 10 15:20 flatten-20250610-152000-abc1234-clean.txt

$ # Clean up files older than 1 day
$ flatten cleanup --days 1
✓ Removed 6 files older than 1 days

$ # Too many files?
$ cd ~/large-monorepo
$ flatten

Error: Found 1,234 files (max: 1000)
Try: flatten --max-files 2000

Output Format Examples

YAML Structure (flatten-20250610-152000-abc1234-clean.yaml)

codebase:
  - path: .
    type: directory
    contents:
      - path: src
        type: directory
        contents:
          - path: src/index.js
            type: file
          - path: src/utils.js
            type: file
      - path: README.md
        type: file
      - path: package.json
        type: file

Flattened Content (flatten-20250610-152000-abc1234-clean.txt)

<src/index.js>
import { helper } from './utils';

console.log('Hello world');
console.log(helper());

</src/index.js>

<src/utils.js>
export function helper() {
  return 'helping!';
}

</src/utils.js>

<README.md>
# My Project

This is my awesome project.

</README.md>

<package.json>
{
  "name": "my-project",
  "version": "1.0.0"
}

</package.json>

Philosophy

This tool follows a minimalist philosophy:

Simple - No configuration files or complex options
Smart - Intelligent defaults that just work
Safe - Built-in protections against common mistakes
Useful - Solves real problems for developers working with LLMs

Contributing

See CONTRIBUTING.md for guidelines.

License

MIT - see LICENSE for details.

Acknowledgments

This project was inspired by the need to easily provide codebase context to LLMs. The original version was a simple bash script that evolved into this more robust npm package.

flatten codebase llm ai context

12 months ago

12 months ago

12 months ago

12 months ago

12 months ago