gscdump

gscdump

Google Search Console API wrapper with typed query builder, streaming pagination, and SEO analysis functions.

Features

• Typed Query Builder - Drizzle-style API with filter constraints narrowing result types
• Streaming Pagination - Memory-efficient iteration over large datasets (>25k rows)
• SEO Analysis - Pure functions for cannibalization, striking distance, movers & shakers, decay detection
• Edge-Compatible - Works in Cloudflare Workers, Deno, and other edge runtimes
• Full API Coverage - Sites, sitemaps, indexing, and analytics

Install

npm install gscdump

This package is the edge-safe surface: REST client + query builder + cross-package contracts. For the storage engine (Parquet/DuckDB, planner, adapters), install @gscdump/engine. For analyzers, install @gscdump/analysis.

Usage

import { fetchKeywordsWithComparison, fetchPagesWithComparison } from 'gscdump'
import { daysAgo, today } from 'gscdump/query'

// Auth accepts token string or object
const auth = 'ya29.xxx...'
// or: { accessToken: 'ya29.xxx...' }

const range = {
  period: { start: daysAgo(28), end: today() },
}

// Pages with top keyword per page
const pages = await fetchPagesWithComparison(auth, site, range)

// Keywords with change percentages
const keywords = await fetchKeywordsWithComparison(auth, site, range)

Streaming Large Datasets

For memory-efficient pagination of large datasets (>25k rows):

import { queryRecursiveStream } from 'gscdump'

// Stream keyword+page combinations - yields batches as they're fetched
for await (const batch of queryRecursiveStream(client, site, {
  dimensions: ['query', 'page'] as const, // as const required for type inference
  startDate: '2024-01-01',
  endDate: '2024-01-31',
})) {
  // batch: { keyword: string, page: string, clicks, impressions, ctr, position }[]
  await db.insert(batch)
}

Typed Query Builder

Drizzle-style query builder with full type safety. Filter constraints flow through to result types.

import { and, between, contains, country, Country, date, device, Device, eq, gsc, inArray, page } from 'gscdump/query'

const body = gsc
  .select('page', 'query', 'device', 'country')
  .where(and(
    eq(device, Device.MOBILE),
    inArray(country, [Country.USA, Country.GBR]),
    contains(page, '/blog/'),
    between(date, '2024-01-01', '2024-01-31')
  ))
  .toBody()

// Use with client.searchAnalytics.query(siteUrl, body)

With date helpers:

import { and, between, date, daysAgo, gsc, query, regex, today } from 'gscdump/query'

const q = gsc
  .select('query', 'page')
  .where(and(
    between(date, daysAgo(28), today()),
    regex(query, /how to/)
  ))
  .limit(100)

Operators:

Operator	Narrows Type?	Description
eq(col, val)	✓	Exact match
ne(col, val)	✗	Not equal
inArray(col, [a, b])	✓	Value in array (becomes a | b)
contains(col, str)	✗	String contains
like(col, '%pattern%')	✗	SQL LIKE pattern
regex(col, /pattern/)	✗	Regex match
notRegex(col, /pattern/)	✗	Regex exclusion
between(col, start, end)	✗	Inclusive range (primarily for date)
gte(col, val)	✗	Greater than or equal
lte(col, val)	✗	Less than or equal
gt(col, val)	✗	Greater than
lt(col, val)	✗	Less than
and(...filters)	✓	Merge constraints
or(...filters)	✗	Any match
not(filter)	✗	Invert filter

Analysis Functions

Analysis functions are pure - they operate on typed data arrays and return typed results.

import {
  analyzeCannibalization,
  analyzeDecay,
  analyzeMovers,
} from '@gscdump/analysis'
import { fetchKeywordsWithComparison } from 'gscdump'

// Fetch data first
const { current, previous } = await fetchKeywordsWithComparison(auth, site, range)

// Run pure analysis on the data
const movers = analyzeMovers(current, previous)
const decay = analyzeDecay(current, previous)
const cannibalization = analyzeCannibalization(keywordPageData)

Exports

Sites: fetchSites, fetchSitesWithSitemaps, fetchSitemaps, getSitemap, submitSitemap, deleteSitemap, inspectUrl, batchInspectUrls

Indexing: requestIndexing, getIndexingMetadata, batchRequestIndexing

Analytics: fetchAnalyticsWithComparison, fetchPagesWithComparison, fetchKeywordsWithComparison, fetchDevicesWithComparison, fetchCountriesWithComparison, fetchSearchAppearanceWithComparison, fetchDates, fetchDatesWithComparison, fetchPages, fetchPage, fetchKeyword

Analysis (Pure) — these live in @gscdump/analysis: analyzeOpportunity, analyzeBrandSegmentation, analyzeConcentration, analyzeDecay, analyzeMovers, analyzeCannibalization, analyzeZeroClick, analyzeSeasonality, analyzeClustering

Low-level: gscClient, queryRecursive, queryRecursiveStream, createQueryBody, withPropertyAggregation, withSearchAppearance, withDataType, withFreshData, withFinalData

Query Builder (gscdump/query): gsc, eq, ne, and, or, inArray, contains, like, regex, notRegex, not, between, gte, gt, lte, lt, page, query, device, country, date, searchAppearance, Device, Country, today, daysAgo

Error Utilities: isQuotaError, isRateLimitError, isAuthError, getErrorCode, getErrorMessage, getRetryAfter, analyzeGscError, formatGscErrorForCli

Utils: formatDateGsc, percentDifference

Related

• @gscdump/cli — CLI: sync, query, dump, analyze, mcp, store admin.
• @gscdump/engine — Append-only Parquet/DuckDB storage engine.
• @gscdump/analysis — SEO analyzers (row-based + DuckDB-native + D1-ready).
• @gscdump/nuxt — Nuxt layer wrapping the full stack.

License

MIT