@lifeaitools/build-corpus

regen-mde

regen-mde is the Windows editor and conversion suite for Markdown, Word, PowerPoint, and PDF files. Its build-corpus CLI converts .docx, .pptx, .ppt, and .pdf files to Markdown while preserving the pieces that usually break in generic converters:

• Word OMML equations as KaTeX-readable TeX
• embedded images as local assets, base64 data URIs, or S3/R2-hosted URLs
• Markdown tables for simple Word tables
• HTML table fallback for complex tables
• a .wordstyle sidecar for Word styles, headers, footers, section properties, and table style package data used during Word roundtrips
• headings, lists, links, bold, italic, inline code, and code-style paragraphs
• PowerPoint slide extraction with slide title detection, table mapping, and repetitive footer suppression

Install

build-corpus ships through the npm package @lifeaitools/build-corpus. regen-mde is the Windows editor command and product name, not the npm package to publish or install. The package includes the Node command wrappers, Windows editor, and Python source used by the CLI.

OS	Recommended	Command	What you get
Ubuntu / Debian / Linux	npm	npm install -g @lifeaitools/build-corpus	build-corpus CLI; no editor
macOS	npm	npm install -g @lifeaitools/build-corpus	build-corpus CLI; no editor
Windows	npm (full kit)	npm install -g --allow-scripts=@lifeaitools/build-corpus @lifeaitools/build-corpus	build-corpus CLI plus the regen-mde editor and Explorer right-click menus

Ubuntu / Debian / Linux

The CLI uses Python internally; the editor is Windows-only, so on Linux/macOS the npm package installs just the converter wrapper and Python dependencies.

# prerequisites
sudo apt update
sudo apt install -y python3 python3-pip nodejs npm
sudo apt install -y libreoffice        # optional — only needed to convert legacy .ppt

# install the CLI
npm install -g @lifeaitools/build-corpus
build-corpus --help

On Ubuntu 24.04+ the postinstall installs Python dependencies into your user site; if that is blocked it prints the manual dependency command and still completes. The regen-mde editor is not built on Linux/macOS.

Windows

The npm package ships the Windows installer plus the conversion CLI:

npm pack @lifeaitools/build-corpus

Extract the package and run dist\release\regen-mde-<version>-win-x64-setup.exe for a normal Windows install. The installer creates Start Menu entries for regen-mde and Uninstall regen-mde, registers right-click Explorer verbs for .docx and .md, and removes those entries during uninstall.

The legacy global npm command path is still supported for automation:

npm install -g --allow-scripts=@lifeaitools/build-corpus @lifeaitools/build-corpus

--allow-scripts permits this package's postinstall only, which installs the Windows editor integration and Explorer menus. Current npm installations can otherwise suppress lifecycle scripts by policy; the CLI still installs, but the editor integration does not.

The editor launches maximized with Output Health collapsed, leaving the document canvas as wide as possible. Use its right-side hamburger button to open Output Health for the current session.

On Windows, the installer and supported automation paths add right-click Explorer menus for .docx, .pptx, .ppt, .md, and folders:

• Life AI -> Open in regen-mde
• opens .md directly and opens .docx by converting it into editable Markdown first
• Life AI -> Convert to Markdown
• runs build-corpus "%1" --out-same-dir for .docx, .pptx, and .ppt
• writes .md, assets, and reports beside the source document
• Life AI -> Convert to Word
• runs build-corpus "%1" --to word --out-same-dir
• writes .docx and export report beside the source document
• Life AI -> Inline Markdown Images
• runs build-corpus "%1" --inline-images
• writes <name>.inline.md with local or HTTP image references embedded as data URIs
• folder Convert Documents to Markdown
• runs build-corpus "%V" --out-same-dir
• converts all .docx, .pptx, and .ppt files in the selected folder tree

The installer also registers .md under Explorer's New menu so you can create a blank Markdown document directly from New.

Set BUILD_CORPUS_SKIP_WINDOWS_MENU=1 before a global npm install if you do not want the Explorer menu. Set BUILD_CORPUS_SKIP_EDITOR=1 before a global npm install if you want the CLI conversion verbs but not the editor open verbs.

To remove the Windows Explorer menus without uninstalling the package:

build-corpus --uninstall-windows

If you uninstall the global npm package, build-corpus now removes those Explorer menu entries automatically during uninstall.

For a project-local install, use npx:

npm install @lifeaitools/build-corpus
npx build-corpus --help

On Windows, if build-corpus launches a Python executable and fails with ModuleNotFoundError, a stale legacy pip install may be shadowing the npm command. Remove it with:

py -3 -m pip uninstall build-corpus

For S3/R2 image upload support:

py -3 -m pip install -r requirements.txt

Basic Usage

build-corpus input.docx --out out
build-corpus deck.pptx --out out
build-corpus input.md --to word --out out
build-corpus input.md --to word --word-template C:\path\custom.dotx --out out
regen-mde input.md
regen-mdeditor input.md
regen-mdeditor input.md
build-corpus editor input.md
build-corpus editor input.docx

PDF Ingest

PDF conversion is an evidence-preserving, one-way ingest into Markdown. Native text and tables use pdf-inspector; image-only pages can be recovered with Tesseract or the local clauth agent endpoint.

build-corpus brief.pdf --out out --ocr auto
build-corpus scanned-tables.pdf --out out --ocr claude-agent

--ocr auto processes only pages without a usable text layer. --ocr force requires Tesseract and processes every page. --ocr claude-agent sends only textless page images to the local clauth POST /call-agent interface for table- and number-sensitive transcription; this is an explicit costed choice and those page images leave the local process.

Every output carries source SHA-256 evidence, original page markers, extractor and OCR provenance, and a promotion gate. A PDF with no recoverable text is written with status: quarantine and promotable: false rather than silently entering the corpus. Page markers and source evidence survive the PDF -> Markdown -> Word -> Markdown round-trip.

regen-mde

regen-mde is a Windows WebView2 desktop app bundled with the package. It uses the same local Build Corpus conversion engine as the CLI:

• Markdown opens directly.
• Word and PowerPoint files open by converting into Markdown.
• Save writes Markdown.
• Save As writes a new Markdown file.
• Export DOCX writes Word output through the Markdown-to-Word route.

Build the Windows executable locally:

npm run editor:windows

The executable is written to:

dist\windows-editor\BuildCorpusEditor.exe

Convert every .docx in a folder:

build-corpus ./word-files --out ./markdown

Convert every supported file type in a folder (.docx, .pptx, .ppt):

build-corpus ./source-files --out ./markdown

Convert specific selected files or folders from automation:

build-corpus .\a.docx .\deck.pptx .\folder --out-same-dir

Move successfully processed source .docx, .pptx, and .ppt files into sources beside each file:

build-corpus ./source-files --out-same-dir --move-sources

Write Markdown beside each source document:

build-corpus ./word-files --out-same-dir

Image Modes

Local asset files, the default:

build-corpus input.docx --images assets

Single-file Markdown with base64 image data URIs:

build-corpus input.docx --images base64

Re-merge an existing Markdown file that references local or HTTP-hosted images into a single Markdown file with inline image data:

build-corpus input.md --inline-images

Upload images to S3-compatible storage and write public URLs:

build-corpus input.docx --images s3 --config examples\build-corpus.config.example.json

Cloudflare R2 uses the same s3 mode. Set endpoint_url to:

https://ACCOUNT_ID.r2.cloudflarestorage.com

Image layout directive (md → word)

Attach a {…} directive to any Markdown image to control its width, alignment, and wrap in the exported Word document. The directive is plain Markdown text, round-trips through the converter, and is ignored by other Markdown renderers.

![grain elevator](grain-elevator.png){ w=1/2 align=right wrap=square }

Axis	Key	Values
Width	w= / width=	50% (of text column) · 1/2 1/3 3/4 (fraction) · half third quarter three-quarter full (keyword) · 3in 2.5" 8cm 40mm (exact) · bleed (full page width)
Alignment	align=	left · center · right
Wrap	wrap=	inline (in the text run) · standalone (own line, no wrap) · square / tight (text-wrapping float)

Rules:

• Width fractions are relative to the text column (page width minus left/right margins), resolved from the live section — so full fills the column and never runs under the margins. bleed is the one value that spans the physical page.
• wrap=square or wrap=tight with align=left/align=right produce a floating (anchored) image that body text wraps around, on the side opposite the image. tight hugs the image outline; square wraps to its bounding box.
• align=center (or any wrap without a left/right side) keeps the image inline and positions it with paragraph justification — you cannot wrap text beside a centered image in Word.
• Bare shorthand also works: {half right square} == {w=half align=right wrap=square}.
• No directive → the legacy default (inline, 5.8in) is preserved unchanged.

The hardest real case — a half-column, right-aligned figure with square text wrap around it — is a single directive: { w=1/2 align=right wrap=square }.

DOCX figure patching without Markdown export

Patch images into an existing Word package directly when the source .docx styles, settings, theme, fonts, and document properties must remain unchanged. This mode rewrites only the document body, document relationships, new media parts, and [Content_Types].xml when a new image extension requires it.

build-corpus input.docx --patch-figures figures.json --out patched.docx --open-word

{
  "figures": [
    {
      "marker": "FIGURE C1",
      "image": "output/model/datacenter/images/c1.png",
      "placement": "insert-before-marker",
      "width": "bleed",
      "align": "center",
      "alt": "Figure C1 cover working document"
    }
  ]
}

placement accepts insert-before-marker, insert-after-marker, and replace-existing-image (before-marker / after-marker are accepted as aliases). Width uses the same language as Markdown image directives: full, 3/4, 1/2, bleed, percentages, and exact inches such as 3in. Add wrap=square or wrap=tight with align=left|right to create a floating wrapped figure.

The command writes <output>.figure-report.json with every applied figure, changed package part hashes, protected-part preservation status, and fidelity_ok. It refuses to overwrite the source .docx; omit --out to write <name>.patched.docx beside the input.

Blockquotes with embedded items (sidebar)

A blockquote that contains only prose stays a single styled quote paragraph. A blockquote that carries block-level content — a table, a list, a fenced code block, a heading, a standalone image, a nested quote, or multiple paragraphs — is rendered into a Word sidebar: a single-cell container (left accent border

• light fill, using the template's Sidebar paragraph style when a template is in play) with the inner Markdown rendered inside it. This is the only way Word can hold a table or list inside a callout — a plain quote paragraph cannot.

> **Assumptions**
>
> | Driver | Value |
> | --- | --- |
> | Discount rate | 8% |
>
> - Stabilized by year 3
> - Exit at a 6.5% cap

The table, the list, and the heading all render as real Word objects within the sidebar box, and it round-trips. Nested blockquotes produce nested sidebars.

Config

Copy examples/build-corpus.config.example.json and edit it for your environment.

{
  "conversion": {
    "equations": "tex",
    "images": "s3"
  },
  "output": {
    "out": "out",
    "out_same_dir": false
  },
  "s3": {
    "bucket": "build-corpus-assets",
    "public_base_url": "https://assets.example.com",
    "prefix": "knowledge-base",
    "endpoint_url": "https://ACCOUNT_ID.r2.cloudflarestorage.com",
    "region_name": "auto",
    "access_key_id": "%R2_ACCESS_KEY_ID%",
    "secret_access_key": "%R2_SECRET_ACCESS_KEY%"
  }
}

Build Corpus expands environment variables in JSON string values, so credentials do not need to be committed.

Output Placement

There are two output modes.

Write all converted Markdown into one output tree:

{
  "output": {
    "out": "./markdown",
    "out_same_dir": false
  }
}

Write each .md, asset folder, and report beside the source .docx:

{
  "output": {
    "out_same_dir": true
  }
}

The same-dir mode is equivalent to:

build-corpus ./word-files --out-same-dir

Markdown to Word Templates

Markdown -> Word conversion uses this template precedence:

1. --word-template <path>
2. word.template in the JSON config
3. an adjacent generated .wordstyle/style-package.docx from a prior DOCX -> Markdown run
4. the bundled installed package template
5. built-in fallback styles if no template can be found

Template files are treated as style sources. Build Corpus creates a fresh output document body, then applies the template's Word styles, numbering, theme, fonts, and settings. It does not reuse the template body content as the exported document.

Word roundtrip style coverage

DOCX -> Markdown emits a <name>.wordstyle sidecar beside the generated Markdown. That sidecar is part of the output, not a cache. Keep it with the Markdown when testing or editing a Word roundtrip.

The sidecar preserves Word package/style data that Markdown cannot express directly:

• styles, table styles, numbering, fonts, theme, settings, and content types
• headers and footers, including their relationships and related media
• section properties such as margins, page size, columns, and header/footer references
• original table property XML when present

Markdown -> Word automatically reuses the adjacent sidecar if no explicit --word-template is supplied:

build-corpus "C:\path\source.docx" --out-same-dir
build-corpus "C:\path\source.md" --to word --out-same-dir

Important limits:

• The sidecar preserves many style elements, but the Markdown editor does not yet expose all of them as editable Markdown semantics.
• Headers, footers, section/page setup, and table geometry can be preserved and reapplied, but they need dedicated editor UI before users can edit them safely.
• Floating objects, text boxes, charts, SmartArt, live Word fields, comments, tracked changes, and exact run/paragraph typography are not fully converted today.

For the full 100% style rubric, known non-converted issues, and Claude/MCP usage contract, read docs/word-roundtrip-style-coverage.md. For the prioritized parser, frontmatter, table, XML, and save-time fixups spec, read docs/word-markdown-fixups-spec.md.

Table dressing (house_style)

build-corpus can apply a declarative "house style" to the tables and page chrome of a Word export. It is off by default — the dressing layer activates only when a house_style: key is present (in frontmatter or config). A plain, no-frontmatter Markdown file exports exactly as before.

Turning it on

Add a house_style to the document frontmatter:

---
house_style: prt          # shipped: prt, futurville-navy, futurville-asp, oliizoi
style_vars:
  version: v3.0
  date: July 2026
---

or supply it (and any overrides) from build-corpus.config.json so a repo can theme every document without touching each file:

{
  "house_style_values": {
    "house_style": "prt",
    "table.header_fill": "1B3A5C",
    "table.band_fill": "F5F4EF"
  }
}

What it does

For every unstyled table (one with no explicit build-corpus:table meta):

• coloured header row, hairline borders, zebra banding, and content-aware column widths;
• header-cell text colour is set at the paragraph level so text typed into a header cell later in Word inherits the header colour, not near-black;
• content-aware justification — full-cell currency right-aligns, a bare number / percent / range centers, a short equation centers, and prose (even prose containing a $ mid-string) stays left. No cell is left full-justified (jc="both");
• matrix header rows repeat across page breaks (<w:tblHeader/>);
• a WCAG contrast guard warns when header text/fill fall below AA (4.5:1); opt into automatic inversion with table.auto_contrast: true.

Tables that carry author-supplied explicit meta (a non-default table style, explicit column widths, or a cell fill) are left untouched — author intent wins.

table.* override keys

Key	Meaning	Default
table.header_fill	header row fill (hex, no #)	profile
table.header_text	header text colour	profile
table.band_fill	zebra band fill	profile
table.label_color	label-column text colour	profile
table.border_color / table.border_sz	border colour / size (⅛ pt)	profile
table.band	zebra banding on/off	true
table.auto_justify	content-aware justification	true
table.respect_explicit_meta	skip author-styled tables	true
table.repeat_header	repeat matrix header across pages	true
table.auto_contrast	auto-invert low-contrast header text	false
header_left / header_right / footer_center / page_numbers	running chrome	—
page.top_in / page.bottom_in / page.left_in / page.right_in	page margins (inches)	template / default

Precedence chain

For every house-style value, highest wins:

CLI / frontmatter  >  config house_style_values  >  shipped profile  >  default (off)

For a single table cell's alignment:

author cnfStyle  >  Markdown column separator (|:--:|)  >  tblLook  >  position (row-0 header, col-0 label)  >  content class

Page margins honor the --word-template <w:pgMar> when a template is supplied; page.*_in frontmatter/config overrides it per side.

Round-trip carrier

Table design survives docx → md → docx via an HTML comment emitted above each table:

<!-- build-corpus:table id="q1-metrics" meta={"cols":[{"w":2000}],"tblLook":[["firstColumn","0"]],"cells":{"3":{"0":{"cnfStyle":[["firstRow","1"]]}}}} -->

meta carries style, cols, borders, inner_borders, outer_borders, width, align, wrap, shade, pad, tblLook (table role flags) and cells (per-cell cnfStyle role bits). It is emitted only when a table has properties worth preserving, and re-consumed on the way back to Word. The contract lives in src/build_corpus/table_meta.py.

For AI-authored table formatting, the useful border and placement knobs are:

<!-- build-corpus:table id="callout" meta={"inner_borders":"single,4,808080","outer_borders":"none","width":"50%","align":"right","wrap":"around"} -->

borders is the legacy all-edges shortcut. Use inner_borders for insideH / insideV, outer_borders for top / left / bottom / right. Border strings are "<style>,<size>,<color>"; size is OOXML eighths of a point, and color is six-digit RGB. width accepts percent ("50%"), fractions ("1/2"), or absolute lengths ("3in"); align accepts left, center, right; wrap accepts around or none.

Observability

When a house style ran, the export report (*.export-report.json) gains a house_style_report block: profile, tables_dressed, tables_skipped_explicit_meta, header, footer, and any warnings (e.g. contrast). The key is absent when no house style was applied.

Full field grammar and the field-observations rationale: work-items/build-corpus-0.9-field-observations-plane.md.

Equations

Equation handling is real in both directions:

DOCX → Markdown — Word OMML equations are converted to KaTeX-readable TeX (via omml2latex). The default mode is parseable TeX:

build-corpus input.docx --equations tex

Equation images are only for visual debugging:

build-corpus input.docx --equations image

Markdown → Word — inline $...$ and display $...$ LaTeX are converted to native Office Math (OMML) that Word renders as real equations — not raw text in a math font. The pipeline is latex2mathml → our own MathML→OMML converter (no external mathml2omml dependency), so commands like \sum, \int, \frac, \Delta, \rightarrow, and \leq render correctly:

build-corpus notes.md --to word --out out

If a fragment cannot be parsed as LaTeX, it falls back to the literal text in Cambria Math and is flagged in the export report's warnings. Fence display equations with $ on their own lines and no blank lines inside the fence.

Fidelity report (md → word)

Every md→word export writes export-report.json (and a build-corpus-batch-report.json across a batch) so you can confirm nothing was silently dropped or altered. Beyond the raw output stats, the report carries:

• fidelity_ok — top-level ship gate. true only when every reconciliation row matches (and zero equations fell back). The batch summary prints all_fidelity_ok plus the list of fidelity_failures.
• reconciliation — input vs output per element type:

  "reconciliation": {
    "tables":    { "in": 1, "out": 1, "ok": true },
    "equations": { "in": 3, "out_omml": 2, "fell_back": 1, "ok": false },
    "images":    { "in": 2, "out": 0, "failed": 2, "ok": false },
    "code_blocks": { "in": 0, "out": 0, "ok": true },
    "headings":  { "in": 1, "out": 1, "ok": true },
    "links":     { "in": 1, "out": 1, "ok": true }
  }

• issues — one entry per problem with the source line: { "type", "line", "source"|"target", "reason" }.
• text_fixups — markdown escapes the engine resolved on your content, e.g. { "total": 2, "currency_unescaped": 2 }. Escaped currency like \$252.3B is kept as literal text ($252.3B), never mistaken for inline math.
• A one-line stdout digest for a quick CLI glance:

  [OK] tables 1/1  [!!] equations 2/3 (1 fell back)  [!!] images 0/2 (2 failed)  …  -> fidelity_ok=false

Image failures carry a specific reason so you know how to react:

reason	meaning	fix
missing-file	target path not found	correct the path
unsupported-on-platform	EMF/WMF that needs metafile→PNG conversion	install LibreOffice / run on Windows
unsupported-format	.html/.jsx/.svg etc. — cannot be embedded	pre-render to PNG via a render pipeline
skipped-remote	http(s)/data: target	localize the asset first

build-corpus does not rasterize HTML/JSX — that belongs to a separate render step (e.g. a headless-browser screenshot). It flags them and moves on.

PowerPoint Notes

• .pptx is processed directly.
• .ppt is converted to .pptx first using LibreOffice (soffice --headless --convert-to pptx).
• Repeated boilerplate blocks that appear on most slides are removed from the emitted Markdown.
• Slide images are exported from the original package binaries (ppt/media/*), not screen-captured display rasters.
• Markdown output uses size-aware HTML image tags (<img ... width= height=>) based on OOXML display extents (a:xfrm/a:ext).
• The export report includes low_dpi_images to flag images whose effective on-slide DPI is under 150.

Validation

The package includes a KaTeX validator for emitted Markdown math:

build-corpus-katex out

Release Gates

regen.mde releases are npm-only. Before publishing, run the required gate:

npm run test:required
npm run pack:dry-run
npm run release:e2e

test:required runs the unit, Word compatibility, and fixture patch suites. release:e2e uses explicit dry-run mode by default so it can validate the npm pack file list and write .tmp/release-e2e-report.json without triggering the Windows editor prepack build. To verify the installed package, create the real .tgz artifact and run:

node scripts/release-e2e.mjs --pack .\regen.mde-<version>.tgz

Every release must include Word compatibility evidence from npm run test:compat with REQUIRE_WORD_COMPAT=1, or an explicit skip reason when Word/COM is not available. The complete bump, tag, publish, and install-verification flow lives in docs/release.md.

Repeatable Test Wrappers

Run a single known DOCX through conversion plus validators:

.\scripts\run-smoke.ps1 -Docx ".\fixtures\sample.docx" -Out ".tmp\smoke" -Images assets

Run a whole folder corpus:

.\scripts\run-corpus.ps1 -Source ".\fixtures\wordtest" -Out ".tmp\wordtest" -Images base64

Build a public online DOCX corpus for regression testing:

python .\tools\collect_online_docx_corpus.py --out ".tmp\online-docx\source-docx" --target 50
.\scripts\run-corpus.ps1 -Source ".tmp\online-docx\source-docx" -Out ".tmp\online-docx\markdown"

Build a public online PPTX corpus and compare input/output extraction:

python .\tools\collect_online_pptx_corpus.py --out ".tmp\online-pptx\source-pptx" --target 20
.\scripts\run-corpus.ps1 -Source ".tmp\online-pptx\source-pptx" -Out ".tmp\online-pptx\markdown"
python .\tools\compare_pptx_inputs_outputs.py --manifest ".tmp\online-pptx\source-pptx\online-pptx-manifest.json" --out ".tmp\online-pptx\markdown" --report ".tmp\online-pptx\markdown\pptx-io-compare.json"

Failed Documents

If a document does not convert correctly, open an issue with:

• the .docx file if it is safe to share
• the generated .md
• the export-report.json
• the command and config used
• a screenshot of the expected Word output if layout is the issue

For confidential files, strip or replace sensitive content before sharing. The useful part is the broken DOCX structure, not the private text.