@headwall/trusted-network-providers

trusted-network-providers

A lightweight Node.js library for identifying IP addresses that belong to trusted network providers. Prevent your firewalls from blocking legitimate services like payment processors, search engine crawlers, and monitoring tools.

v2.0.0 Breaking Changes: This version migrates from CommonJS to ES modules. Use import instead of require(). See the migration guide for details.

Quick Start

npm install @headwall/trusted-network-providers

import trustedProviders from '@headwall/trusted-network-providers';

// Load built-in providers (Googlebot, Stripe, PayPal, Cloudflare, etc.)
trustedProviders.loadDefaultProviders();

// Initialize dynamic providers
await trustedProviders.reloadAll();

// Check an IP address
const provider = trustedProviders.getTrustedProvider('66.249.66.87');
console.log(provider); // "Googlebot"

const unknown = trustedProviders.getTrustedProvider('123.123.123.123');
console.log(unknown); // null

Use Cases

• Firewall Management: Whitelist IPs for WordPress hosting networks
• Rate Limiting: Bypass rate limits for trusted crawlers
• Access Control: Allow trusted services through security layers
• Traffic Analysis: Identify and categorize legitimate traffic sources

Built-in Providers

Includes 20+ trusted providers out of the box:

• Search Engines: Googlebot, Bingbot, AhrefsBot, SemrushBot
• Advertising: Google AdsBot, AdSense (Google Special Crawlers)
• User-Triggered Fetchers: Google (Gmail image proxy, Chrome prefetch proxy, Feedfetcher, etc.)
• Payment Processors: Stripe, PayPal, Opayo
• Email Services: Outlook, Brevo, Mailgun
• CDN/Infrastructure: Cloudflare, BunnyNet
• Development Tools: GTmetrix, GetTerms, Labrika
• Social Media: FacebookBot
• E-commerce: ShipHero
• Networks: Private/Internal (RFC 1918)

Key Features

• Fast synchronous IP lookups (< 1ms typical with caching)
• Support for IPv4 and IPv6
• CIDR range matching
• Dynamic provider updates from external sources
• Provider lifecycle events (reload, error, stale detection)
• Configurable result caching with TTL
• Easy custom provider integration
• Zero configuration with sensible defaults

API

Core Functions

// Load default providers
loadDefaultProviders();

// Update all providers with dynamic data
await reloadAll();

// Check if IP is trusted (returns provider name or null)
getTrustedProvider(ipAddress);

// Check if IP is trusted (returns boolean)
isTrusted(ipAddress);

// Provider management
addProvider(provider);
deleteProvider(providerName);
hasProvider(providerName);
getAllProviders();

// Testing
await runTests();

Lifecycle & Monitoring

// Get provider status and metadata
const status = getProviderStatus('Googlebot');
// Returns: { name, state, lastUpdated, lastError }
// State: 'ready' | 'loading' | 'error' | 'stale'

// Listen to provider lifecycle events
trustedProviders.on('reload:success', ({ provider, timestamp }) => {
  console.log(`${provider} reloaded at ${new Date(timestamp)}`);
});

trustedProviders.on('reload:error', ({ provider, error }) => {
  console.error(`${provider} failed to reload:`, error);
});

trustedProviders.on('stale', ({ provider, lastUpdated, staleDuration }) => {
  console.warn(`${provider} is stale (last updated ${staleDuration}ms ago)`);
});

// Configure staleness detection (default: 24 hours)
trustedProviders.setStalenessThreshold(12 * 60 * 60 * 1000); // 12 hours

Caching & Performance

// Configure IP lookup result cache TTL (default: 1 hour)
trustedProviders.setResultCacheTTL(30 * 60 * 1000); // 30 minutes
const currentTtl = trustedProviders.getResultCacheTTL();

Logging

// Configure log level: 'silent' | 'error' | 'warn' | 'info' | 'debug'
trustedProviders.setLogLevel('info');
const level = trustedProviders.getLogLevel();

Provider State Constants

The library exports constants for checking provider states programmatically:

import trustedProviders, {
  PROVIDER_STATE_READY,
  PROVIDER_STATE_LOADING,
  PROVIDER_STATE_ERROR,
  PROVIDER_STATE_STALE,
} from '@headwall/trusted-network-providers';

const status = trustedProviders.getProviderStatus('Stripe API');
if (status.state === PROVIDER_STATE_ERROR) {
  // Handle provider failure
}

CLI

A trusted-lookup command is included for ad-hoc IP checks from the terminal.

Run it without installing, via npx:

# Check a single IP
npx -p @headwall/trusted-network-providers trusted-lookup 66.249.66.87

# Check several at once
npx -p @headwall/trusted-network-providers trusted-lookup 66.249.66.87 1.2.3.4

Or install globally to get the bare command:

npm install -g @headwall/trusted-network-providers
trusted-lookup 66.249.90.77

It prints the matching provider for each IP (or "not trusted"), and exits 0 when every IP is trusted or 1 if any are untrusted — handy for scripts and CI:

if npx -p @headwall/trusted-network-providers trusted-lookup "$ip"; then
  echo "$ip is trusted"
fi

Documentation

• Providers - Built-in provider reference and custom provider guide
• Security - Security features and production recommendations
• DNS Security Guide - DNS/SPF provider security considerations
• Migration Guide - Upgrading from v1.x to v2.x

Example: Custom Provider

import trustedProviders from '@headwall/trusted-network-providers';

trustedProviders.addProvider({
  name: 'My Custom Network',
  testAddresses: ['10.0.0.1'],
  ipv4: {
    addresses: ['10.0.0.1'],
    ranges: ['10.0.0.0/24'],
  },
  ipv6: {
    addresses: [],
    ranges: [],
  },
});

Example: Express.js Middleware

import express from 'express';
import trustedProviders from '@headwall/trusted-network-providers';

const app = express();

app.use((req, res, next) => {
  const provider = trustedProviders.getTrustedProvider(req.ip);
  if (provider) {
    req.trustedProvider = provider;
    // Skip rate limiting, logging, etc.
  }
  next();
});

Example: Long-Running Service with Monitoring

import trustedProviders from '@headwall/trusted-network-providers';

// Configure logging for production
trustedProviders.setLogLevel('warn');

// Load providers
trustedProviders.loadDefaultProviders();
await trustedProviders.reloadAll();

// Monitor provider health
trustedProviders.on('stale', ({ provider, lastUpdated }) => {
  console.warn(`Provider ${provider} is stale, triggering reload...`);
  trustedProviders.reloadAll();
});

trustedProviders.on('reload:error', ({ provider, error }) => {
  // Alert your monitoring system
  console.error(`Critical: ${provider} reload failed:`, error.message);
});

// Check provider health periodically
setInterval(
  () => {
    const googlebot = trustedProviders.getProviderStatus('Googlebot');
    if (googlebot.state === 'error' || googlebot.state === 'stale') {
      console.warn(`Googlebot health check failed: ${googlebot.state}`);
    }
  },
  60 * 60 * 1000
); // Every hour

Maintenance

Update bundled IP assets before releases:

./scripts/update-assets.sh

License

MIT Paul Faulkner

Contributing

Issues and pull requests welcome on GitHub.