@x12i/web-queries

@x12i/web-queries

Question-driven web scoping: plan search queries from a question (+ optional facts), execute via @x12i/search-adapter, return normalized web context.

Current version: 1.3.0

Install

npm install @x12i/web-queries @x12i/search-adapter

Requires Node 18+ and TAVILY_API_KEY for live search (via search-adapter).

Quick start (core)

import { createSearchAdapter } from "@x12i/search-adapter";
import { createWebScoper } from "@x12i/web-queries";

const scoper = createWebScoper({
  searchAdapter: createSearchAdapter(),
});

const result = await scoper.search({
  question: "What is the current patch status for CVE-2021-44228?",
  record: { cveId: "CVE-2021-44228" },
});

if (result.ok) {
  console.log(result.context!.summary);
  console.log(result.context!.findings);
  console.log(result.context!.sources);
  console.log(result.context!.meta.queriesUsed);
}

API overview

Method	Role
search({ question, record?, options? })	Core — one question → one WebContext
searchMany(pack)	Appendix A — multi-question pack with shared URL fetch, snapshot short-circuit, optional cache
plan({ question, record?, options? })	Preview planned queries without executing search

---

Core — search()

Input

{
  requestId?: string;                     // optional correlation id; generated when absent
  question: string;                        // required
  record?: Record<string, unknown>;        // optional known facts
  options?: {
    maxQueries?, includeDomains?, excludeDomains?, preferredDomains?,
    freshnessDays?, topic?, gapFill?, gapType?,
    hints?, queryTemplates?, fetchPages?, fetchTopK?
  };
}

Success output

{
  requestId: string,
  ok: true,
  context: {
    summary?: string,
    summaryOrigin?: "provider" | "derived",
    findings: WebFinding[],   // sourceIds[] → lookup in sources
    sources: WebSource[],
    meta: {
      requestId,
      queriesUsed, retrievedAt, sourceCount, evidenceCount, discoveredCount,
      intent, strategy, shape, timingMs
    }
  }
}

Failure output

{
  requestId: string,
  ok: false,
  error: {
    reason: "no_queries" | "no_adapter" | "search_failed" | "not_eligible" | "error",
    message?: string
  }
}

---

Appendix A — searchMany() pack

Batch several questions with optimized shared fetching. Requires searchAdapter.searchMany and searchAdapter.fetchUrlContent.

Input

{
  requestId?: string,                      // optional correlation id shared by all questions
  questions: [
    {
      id: string,                           // stable key for results
      question: string,
      purpose?: string,
      record?: Record<string, unknown>,     // overrides pack record
      options?: WebScopeOptions,
      mappedProperty?: string,              // dot-path into snapshot/record
      queryTemplates?: string[],
    }
  ],
  record?: Record<string, unknown>,         // shared facts
  snapshot?: Record<string, unknown>,       // for mappedProperty short-circuit
  options?: WebScopeOptions,                // shared per-call defaults
  forceWeb?: boolean,                       // skip snapshot/cache short-circuits
  primaryQuestionId?: string,               // pack.primary selection
  concurrency?: number,                     // default 3
  dedupe?: "exact" | "normalized" | "off", // default "normalized"
}

Example

const pack = await scoper.searchMany({
  record: { cveId: "CVE-2021-44228" },
  snapshot: { patchStatus: "patched in 2.17.1" },
  questions: [
    {
      id: "patch-status",
      question: "What is the patch status for CVE-2021-44228?",
      mappedProperty: "patchStatus",
    },
    { id: "timeline", question: "When was CVE-2021-44228 first exploited?" },
  ],
  concurrency: 3,
});

Pack output

{
  requestId: string,
  ok: true,
  results: {
    "patch-status": {
      ok: true,
      status: "resolved_from_snapshot",  // | resolved_from_cache | web | miss
      context?: WebContext,
      resolvedFrom?: { mappedProperty, value },
      error?: { reason, message },
    },
    "timeline": { ok: true, status: "web", context: { /* WebContext */ } },
  },
  pack: {
    primaryQuestionId: "patch-status",
    primary?: WebContext,                // convenience aggregate
    scopes: Record<string, WebScopePackScopeEntry>,
  }
}

Pack execution pipeline

1. Dedupe questions within the pack (duplicate ids share the first outcome)
2. Per unique question: snapshot short-circuit → optional cache → discovery-only searchMany
3. Collect discovered URLs across all questions; dedupe by normalized URL
4. fetchUrlContent once per unique URL; merge evidence into each question
5. Map to WebContext; optional cache save

Question status values:

Status	Meaning
resolved_from_snapshot	mappedProperty resolved from snapshot or pack record
resolved_from_cache	Returned by config.cache.get
web	Live web search + fetch completed
miss	Failed (no_adapter, no_queries, search_failed, etc.)

---

Configuration

createWebScoper({
  searchAdapter: createSearchAdapter(),   // required for search() and searchMany()

  defaults: {
    maxQueries: 3,
    maxFindings: 10,
    maxSources: 10,
    maxResultsPerQuery: 5,
  },

  snippets: {
    include: false,        // include source snippets in output
    maxCharsPerSource: 512,
    fetchPages: false,     // core search(): fetch top URLs via adapter
    fetchTopK: 3,          // pack mode: URLs to fetch per question after discovery
  },

  rawPassthrough: false,   // attach adapter SearchManyResult at context.raw

  isEligible: (input) => true,

  onEvent: ({ phase }) => { /* "plan" | "search" | "map" */ },

  cache: {                 // optional; used by searchMany pack
    get: async ({ requestId, question, record }) => null,
    save: async ({ requestId, question, record, context }) => {},
  },
});

Per-call overrides via options on each search() or pack question.

requestId is returned on every search() and searchMany() result, including failures. When omitted, @x12i/web-queries generates one and stamps it into context.meta.requestId for returned contexts.

---

Development

From monorepo root:

npm install
npm run build --workspace=@x12i/web-queries
npm run test --workspace=@x12i/web-queries
npm run test:integration --workspace=@x12i/web-queries

Integration tests require TAVILY_API_KEY in web-queries/.env.

---

Memorix persistence (remember prior research)

For clients that should reuse prior scoped research across calls, use the companion package @x12i/web-scoper-memorix. It is a drop-in wrapper around @x12i/web-queries with the same search(), searchMany(), and plan() API plus automatic Memorix knowledge caching.

import { createMemorixWebScoperFromEnv } from "@x12i/web-scoper-memorix";

const scoper = await createMemorixWebScoperFromEnv();

const first = await scoper.search({
  question: "{{cveId}} patch status",
  record: { cveId: "CVE-2021-44228" },
});

// Second call with the same template + facts returns from Memorix when TTL has not expired.
const second = await scoper.search({
  question: "{{cveId}} patch status",
  record: { cveId: "CVE-2021-44228" },
});

await scoper.close();

Requires MONGO_URI. TTL defaults and overrides are documented in the memorix package README.

Related packages

• @x12i/search-adapter — Tavily-backed search execution layer
• @x12i/web-scoper-memorix — Memorix knowledge-backed TTL cache for scoped research
• Monorepo spec: see docs/SPEC.md (includes Appendix A)