paper-parser

paper-parser

Parse scientific PDF papers into structured, selectable HTML and Markdown — real text with reading order, not scanned pages. Two-column layouts are linearized column by column, section headings are detected, tables are rebuilt, sub/superscripts are preserved, and figures plus display equations are rendered as cropped images.

import { parsePdf } from 'paper-parser'

const paper = await parsePdf('/papers/2608.12457.pdf', {
  outputDir: '/tmp/paper-out',   // figure & equation images land here
})

console.log(paper.title)         // "Climates of Gl 514 b"
console.log(paper.abstract)      // "The continuous discovery of exoplanets, ..."
console.log(paper.sections)      // [{ level: 3, title: '1. INTRODUCTION', page: 1 }, ...]
console.log(paper.figures)       // [{ file: 'fig-004.png', page: 3, kind: 'raster', ... }, ...]
console.log(paper.html)          // selectable HTML with image references
console.log(paper.markdown)      // Markdown rendering of the same document

Prerequisites

paper-parser shells out to poppler — the binaries must be on PATH:

• pdftotext (layout extraction with -bbox)
• pdfimages (embedded raster figures)
• pdftoppm (page crops for vector figures and display equations)

macOS: brew install poppler · Debian/Ubuntu: apt install poppler-utils

Install

npm install paper-parser

Requires Node.js ≥ 22 (ESM only).

API

parsePdf(pdfPath, options?) → Promise<ParsedPaper>

Parses a PDF file. All work happens in-process via poppler subprocesses; nothing is written outside options.outputDir.

option	type	default	meaning
outputDir	string	required	Directory that receives extracted raster figures (fig-*.png) and page crops (crop-<page>-<n>.png). Serve this directory to the browser to display images.
imageBase	string	'/paper-review/fig/'	URL prefix used for <img src> in paper.html.
pagesBase	string	'/paper-review/pages/'	URL prefix used for full-page images when a scanned PDF triggers fallback mode.
cropScale	number	150	Render resolution (dpi) for vector-figure and display-equation page crops.

Throws a descriptive Error when outputDir is missing, when the PDF cannot be read, or when a poppler binary is missing (Command failed: pdftotext ...).

ParsedPaper

field	type	description
title	string | null	Detected paper title (multi-line titles are joined).
abstract	string | null	The abstract paragraph, detected with or without an "ABSTRACT" label.
pages	number	Number of pages in the PDF.
fallback	boolean	true when the PDF has no extractable text (a scan); html is then full-page images.
html	string	The converted paper as HTML. Body text is real selectable text; figures and equations are <img> tags; page breaks are marked.
markdown	string	The same document as Markdown (heading levels, tables, figure references).
sections	Section[]	Detected section headings in reading order.
figures	Figure[]	Detected figures in reading order, with the file name to serve.
equations	Equation[]	Display equations in reading order.
version	string	Parser version — useful as a cache key for stored conversions.

interface Section {
  level: 1 | 2 | 3      // heading level in the html
  title: string         // heading text
  page: number          // 1-based page
}

interface Figure {
  file: string | null   // file name inside outputDir (null when no image was extracted)
  page: number          // 1-based page
  kind: 'raster'        // embedded bitmap image from pdfimages
      | 'crop'          // vector figure rendered as a page crop
  width: number         // natural width in px (raster) or pt (crop)
  height: number
}

interface Equation {
  plain: string         // extracted text of the equation
  file: string | null   // crop image file name; null when rendered as text
  page: number
  rendered: boolean     // true when the equation is an image (fractions/cases stay intact)
}

convertPdf(pdfPath, dir) — legacy alias

Equivalent to parsePdf(pdfPath, { outputDir: dir }) with the original /paper-review/* image prefixes. Kept so existing callers keep working.

VERSION

The parser version string, exported for cache invalidation.

Serving the output

paper.html references images by file name under imageBase. A minimal standalone reader:

import { createServer } from 'node:http'
import { readFile, stat } from 'node:fs/promises'
import { join } from 'node:path'
import { parsePdf } from 'paper-parser'

const OUT = '/tmp/paper-out'
const paper = await parsePdf('paper.pdf', { outputDir: OUT, imageBase: '/fig/' })

createServer(async (req, res) => {
  try {
    if (req.url === '/') return send(res, paper.html, 'text/html')
    if (req.url.startsWith('/fig/')) {
      const file = join(OUT, decodeURIComponent(req.url.slice(5)))
      return send(res, await readFile(file), 'image/png')
    }
    res.writeHead(404).end()
  } catch { res.writeHead(404).end() }
}).listen(3000)

function send(res, body, type) {
  res.writeHead(200, { 'content-type': type })
  res.end(body)
}

How it works

1. pdftotext -bbox extracts every word with its bounding box.
2. Column gutters are found by straddle analysis and validated across the majority of pages; each page's words are split at the gutter and re-linearized as full-width flow → left column → right column, with spanning lines re-merged.
3. The body font size is the mode of line heights; headings are detected by size, ALL-CAPS lines, and numbered/Roman-numeral section patterns — with prose guards so sentences and journal names are not headings.
4. Title, authors/affiliations, and abstract are classified as front matter (the abstract works even when the PDF has no "ABSTRACT" label).
5. Tables are rebuilt from column-aligned lines; sub/superscripts are tagged by size and position relative to their line.
6. Embedded raster figures come from pdfimages (smask companions and sub-40pt icons are filtered); vector figures and display equations are located from text density/scatter and rendered with pdftoppm at cropScale dpi.
7. Scanned PDFs with no text fall back to full-page images.

Errors

symptom	cause
Command failed: pdftotext	poppler is not installed or not on PATH
options.outputDir is required	parsePdf needs a directory for images
blank figures	smask/alpha-companion images are filtered automatically; report a reproducer if one slips through

License

MIT