@tokenring-ai/serper

Overview

The @tokenring-ai/serper package provides seamless integration with Serper.dev's Google Search and News APIs, enabling Token Ring agents and applications to perform real-time web searches, fetch news articles, and extract web page content without direct Google API integration. This package extends the @tokenring-ai/websearch module to provide a complete web search provider implementation.

Key Features

• Google Search Integration: Perform organic web searches with knowledge graphs, related searches, and "people also ask" results
• Google News Search: Access real-time news articles with source, date, and snippet information (last hour by default)
• Web Page Fetching: Extract markdown content and metadata from web pages using Serper's scraping service
• Location-Based Search: Support for geographic targeting through gl and location parameters
• Language Support: Multi-language search capabilities through hl parameter
• Retry Logic: Built-in retry mechanism with exponential backoff via doFetchWithRetry for all search and fetch operations
• Type Safety: Full TypeScript support with Zod schema validation
• Comprehensive Error Handling: Detailed error messages with hints for common issues
• Plugin Architecture: Automatic registration with Token Ring applications via websearch service
• Timeout Support: Configurable timeout for web page fetching via AbortController

Installation

bun install @tokenring-ai/serper

Chat Commands

This package does not include chat commands. Searches should be performed through the websearch service provided by @tokenring-ai/websearch or by using the SerperWebSearchProvider directly.

Core Components

SerperWebSearchProvider

The main provider class that extends WebSearchProvider from @tokenring-ai/websearch.

Constructor

constructor(config: SerperWebSearchProviderOptions)

Parameters:

• apiKey (required): Your Serper.dev API key
• defaults (optional): Default search parameters
  • gl: Geographic region code (e.g., 'us', 'uk')
  • hl: Language code (e.g., 'en', 'de')
  • location: Specific location for localized results
  • num: Number of results to return
  • page: Page number for pagination

Throws: Error if apiKey is not provided

Public Methods

searchWeb

async searchWeb(query: string, options?: WebSearchProviderOptions): Promise<WebSearchResult>

Performs a Google web search and returns organic results, knowledge graphs, related searches, and "people also ask" results.

Parameters:

• query (required): Search query string
• options (optional): Search options
  • countryCode: Geographic region code (e.g., 'us', 'uk')
  • language: Language code (e.g., 'en', 'de')
  • location: Specific location for localized results
  • num: Number of results to return
  • page: Page number for pagination

Returns: Promise<WebSearchResult> containing:

• organic: Array of organic search results
• knowledgeGraph: Knowledge graph information if available
• peopleAlsoAsk: Array of related questions if available
• relatedSearches: Array of related search queries if available

Note: This method uses doFetchWithRetry for automatic retry with exponential backoff.

Example:

import SerperWebSearchProvider from '@tokenring-ai/serper';

const provider = new SerperWebSearchProvider({
  apiKey: process.env.SERPER_API_KEY!,
  defaults: {
    gl: 'us',
    hl: 'en',
    num: 10
  }
});

const results = await provider.searchWeb('Token Ring AI framework');
console.log(results.organic); // Array of organic search results
console.log(results.knowledgeGraph); // Knowledge graph if available
console.log(results.peopleAlsoAsk); // Array of related questions if available
console.log(results.relatedSearches); // Array of related search queries if available

searchNews

async searchNews(query: string, options?: WebSearchProviderOptions): Promise<NewsSearchResult>

Performs a Google News search and returns recent news articles.

Parameters:

• query (required): News search query string
• options (optional): Search options
  • countryCode: Geographic region code
  • language: Language code
  • location: Specific location for localized results
  • num: Number of results to return
  • page: Page number for pagination

Returns: Promise<NewsSearchResult> containing:

• news: Array of news articles with title, link, snippet, date, source, and position

Note: The news search includes a hardcoded date filter for the last hour (tbs: "qdr:h"). This is a known limitation (see source code TODO for potential future enhancement). For older news, use regular web search with date-specific queries - see Best Practices for workarounds.

Note: This method uses doFetchWithRetry for automatic retry with exponential backoff.

Example:

const news = await provider.searchNews('artificial intelligence breakthroughs', {
  countryCode: 'us',
  num: 5,
  page: 1
});

news.news.forEach(article => {
  console.log(`Title: ${article.title}`);
  console.log(`Source: ${article.source}`);
  console.log(`Date: ${article.date}`);
  console.log(`Snippet: ${article.snippet}`);
  console.log(`Link: ${article.link}`);
});

fetchPage

async fetchPage(url: string, options?: WebPageOptions): Promise<WebPageResult>

Fetches and extracts content from a web page using Serper's scraping service.

Parameters:

• url (required): URL of the webpage to fetch
• options (optional): Fetch options
  • timeout: Request timeout in milliseconds

Endpoint: POST https://scrape.serper.dev

Returns: Promise<WebPageResult> containing:

• markdown: Extracted markdown content
• metadata: Page metadata including title, description, OpenGraph properties

Note: This method uses doFetchWithRetry for automatic retry with exponential backoff. It includes timeout support via AbortController - if the timeout is reached, the entire operation (including retries) will be aborted.

Example:

const page = await provider.fetchPage('https://example.com', {
  timeout: 10000
});

console.log(page.markdown); // Extracted markdown content
console.log(page.metadata); // Page metadata including title, description, OpenGraph properties

Private Methods

googleSearch

private async googleSearch(query: string, opts?: SerperSearchOptions): Promise<SerperSearchResponse>

Internal method for performing Google searches via the Serper API.

Endpoint: POST https://google.serper.dev/search

Parameters:

• query: Search query string
• opts: Search options including gl, hl, location, num, page, autocorrect, type, and extraParams

googleNews

private async googleNews(query: string, opts?: SerperNewsOptions): Promise<SerperNewsResponse>

Internal method for performing Google News searches via the Serper API.

Endpoint: POST https://google.serper.dev/news

Parameters:

• query: News search query string
• opts: Search options including gl, hl, location, num, page, type, and extraParams

Note: Uses hardcoded tbs: "qdr:h" for last hour results.

buildPayload

private buildPayload(query: string, opts?: Record<string, unknown>): Record<string, unknown>

Builds the request payload by merging query, defaults, and options.

Parameters:

• query: Search query string (required)
• opts: Additional search options

Returns: Request payload object with undefined/null values removed

Throws: Error if query is empty

parseJsonOrThrow

private async parseJsonOrThrow<T>(res: Response, context: string): Promise<T>

Parses JSON response or throws an error with detailed information.

Parameters:

• res: Response object
• context: Context string for error messages

Returns: Parsed JSON object

Throws: Error with status, hint, and details properties if response is not OK

Services

WebSearchService Integration

The SerperWebSearchProvider implements the WebSearchProvider interface and integrates with the WebSearchService from @tokenring-ai/websearch.

Service Registration:

When using the plugin, the provider is automatically registered with the websearch service:

import TokenRingApp from '@tokenring-ai/app';
import { WebSearchService } from '@tokenring-ai/websearch';

const app = new TokenRingApp({
  websearch: {
    providers: {
      serper: {
        type: 'serper',
        apiKey: process.env.SERPER_API_KEY!,
        defaults: {
          gl: 'us',
          hl: 'en',
          num: 10
        }
      }
    }
  }
});

// Access the provider through the websearch service
const websearchService = app.requireService(WebSearchService);
const results = await websearchService.search('your query', 'serper');

Provider Documentation

SerperWebSearchProvider

This package provides a web search provider implementation for the Serper API.

Provider Interface

The provider implements the following interface methods from WebSearchProvider:

• searchWeb(query: string, options?: WebSearchProviderOptions): Promise<WebSearchResult>
• searchNews(query: string, options?: WebSearchProviderOptions): Promise<NewsSearchResult>
• fetchPage(url: string, options?: WebPageOptions): Promise<WebPageResult>

Provider Configuration

type SerperWebSearchProviderOptions = {
  apiKey: string;
  defaults?: {
    gl?: string;
    hl?: string;
    location?: string;
    num?: number;
    page?: number;
  };
};

Registration Patterns

Plugin-based Registration:

import TokenRingApp from '@tokenring-ai/app';

const app = new TokenRingApp({
  websearch: {
    providers: {
      serper: {
        type: 'serper',
        apiKey: process.env.SERPER_API_KEY!,
        defaults: {
          gl: 'us',
          hl: 'en'
        }
      }
    }
  }
});

Programmatic Registration:

import { WebSearchService } from '@tokenring-ai/websearch';
import SerperWebSearchProvider from '@tokenring-ai/serper';

const provider = new SerperWebSearchProvider({
  apiKey: process.env.SERPER_API_KEY!,
  defaults: {
    gl: 'us',
    hl: 'en'
  }
});

// Register with websearch service
websearchService.registerProvider('serper', provider);

RPC Endpoints

This package does not define RPC endpoints.

Configuration

Configuration Schema

import { z } from 'zod';

export const SerperDefaultsSchema = z.object({
  gl: z.string().optional(),
  hl: z.string().optional(),
  location: z.string().optional(),
  num: z.number().optional(),
  page: z.number().optional(),
});

export const SerperWebSearchProviderOptionsSchema = z.object({
  apiKey: z.string(),
  defaults: SerperDefaultsSchema.optional(),
});

export const packageConfigSchema = z.object({
  websearch: z.object({
    providers: z.record(z.object({
      type: z.literal('serper'),
      apiKey: z.string(),
      defaults: SerperDefaultsSchema.optional(),
    })).optional(),
  }).optional(),
});

Configuration Example

import TokenRingApp from '@tokenring-ai/app';

const app = new TokenRingApp({
  websearch: {
    providers: {
      serper: {
        type: 'serper',
        apiKey: process.env.SERPER_API_KEY!,
        defaults: {
          gl: 'us',
          hl: 'en',
          num: 10
        }
      }
    }
  }
});

Configuration Options

Option	Type	Required	Description
apiKey	string	Yes	Serper.dev API key
defaults.gl	string	No	Geographic region code
defaults.hl	string	No	Language code
defaults.location	string	No	Specific location
defaults.num	number	No	Number of results
defaults.page	number	No	Page number

Environment Variables

• SERPER_API_KEY: Your Serper.dev API key

Integration

Plugin Registration

This package integrates with Token Ring applications through the plugin system. When installed, it automatically registers the Serper provider with the websearch service.

Plugin Installation

import TokenRingApp from '@tokenring-ai/app';
import serperPlugin from '@tokenring-ai/serper/plugin';

const app = new TokenRingApp();

await app.installPlugin(serperPlugin, {
  websearch: {
    providers: {
      serper: {
        type: 'serper',
        apiKey: process.env.SERPER_API_KEY!,
        defaults: {
          gl: 'us',
          hl: 'en'
        }
      }
    }
  }
});

Agent Integration

The provider can be used as a tool for agents:

import { Agent } from '@tokenring-ai/agent';
import SerperWebSearchProvider from '@tokenring-ai/serper';

const provider = new SerperWebSearchProvider({
  apiKey: process.env.SERPER_API_KEY!
});

const agent = new Agent({
  name: 'Researcher',
  tools: [
    {
      name: 'searchWeb',
      description: 'Search the web using Google',
      handler: async (query: string) => {
        return await provider.searchWeb(query);
      }
    },
    {
      name: 'searchNews',
      description: 'Search for news articles',
      handler: async (query: string) => {
        return await provider.searchNews(query);
      }
    }
  ]
});

Usage Examples

Basic Web Search

import SerperWebSearchProvider from '@tokenring-ai/serper';

const provider = new SerperWebSearchProvider({
  apiKey: process.env.SERPER_API_KEY!,
  defaults: {
    gl: 'us',
    hl: 'en',
    num: 10
  }
});

// Perform a web search
const results = await provider.searchWeb('Token Ring AI framework');
console.log('Organic results:', results.organic.length);

// Access different result types
if (results.knowledgeGraph) {
  console.log('Knowledge Graph:', results.knowledgeGraph.title);
}

if (results.peopleAlsoAsk) {
  console.log('People also ask:', results.peopleAlsoAsk.length);
}

if (results.relatedSearches) {
  console.log('Related searches:', results.relatedSearches.map(r => r.query));
}

News Search

// Search for recent news (last hour by default)
const news = await provider.searchNews('artificial intelligence breakthroughs', {
  countryCode: 'us',
  num: 5,
  page: 1
});

news.news.forEach(article => {
  console.log(`Title: ${article.title}`);
  console.log(`Source: ${article.source}`);
  console.log(`Date: ${article.date}`);
  console.log(`Snippet: ${article.snippet}`);
  console.log(`Link: ${article.link}`);
});

Web Page Fetching

// Fetch and extract content from a webpage
const page = await provider.fetchPage('https://tokenring.ai', {
  timeout: 10000
});

console.log('Page title:', page.metadata.title);
console.log('Description:', page.metadata.description);
console.log('Markdown content:', page.markdown.substring(0, 200) + '...');

Integration with Token Ring Agents

import TokenRingApp from '@tokenring-ai/app';
import { WebSearchService } from '@tokenring-ai/websearch';

const app = new TokenRingApp({
  websearch: {
    providers: {
      serper: {
        type: 'serper',
        apiKey: process.env.SERPER_API_KEY!,
        defaults: {
          gl: 'us',
          hl: 'en'
        }
      }
    }
  }
});

// Access the provider through the websearch service
const websearchService = app.requireService(WebSearchService);
const results = await websearchService.search('your query', 'serper');

Best Practices

1. API Key Security: Store your Serper API key in environment variables and never commit it to version control
2. Rate Limiting: Implement appropriate delays between requests to avoid rate limiting
3. Caching: Consider caching repeated search queries to reduce API usage
4. Error Handling: Always handle potential errors from search operations
5. Configuration Defaults: Set reasonable default values for search parameters to ensure consistent behavior
6. Timeout Management: Configure appropriate timeouts for page fetching operations. Note that fetchPage has retry logic via doFetchWithRetry, but timeouts will abort the entire operation (including retries) via AbortController
7. Query Validation: Validate search queries before sending to the API
8. Result Processing: Handle cases where results may be empty or incomplete
9. News Search Limitation: News search is hardcoded to return results from the last hour only (tbs: "qdr:h"). For older news, use regular web search with date-specific queries. A future enhancement may make the date range configurable (see source code TODO)

Error Handling

The package provides comprehensive error handling with helpful hints:

try {
  const results = await provider.searchWeb('query');
} catch (error) {
  if (error instanceof Error) {
    if (error.status === 401) {
      console.log('Invalid API key - check SERPER_API_KEY');
    } else if (error.status === 429) {
      console.log('Rate limit exceeded - reduce request frequency');
    } else {
      console.log('Search failed:', error.message);
      if (error.hint) {
        console.log('Hint:', error.hint);
      }
      if (error.details) {
        console.log('Details:', error.details.slice(0, 500));
      }
    }
  }
}

Error Object Structure

Error responses include:

• status: HTTP status code (400, 401, 429, etc.)
• message: Human-readable error message
• hint: Suggestion for resolving the error (optional)
• details: Raw response snippet (optional)

Common error responses:

• 401: Invalid API key - check SERPER_API_KEY
• 429: Rate limit exceeded - reduce request frequency
• 400: Invalid request parameters (missing required fields)

Rate Limits and Credits

• Credit Tracking: The raw Serper API response includes a credits field, but this is filtered out in the returned results using the pick utility. The searchWeb, searchNews, and fetchPage methods only return the relevant fields for each operation.
• Rate Limiting: Automatic retries with exponential backoff for 429 and 5xx errors via doFetchWithRetry.
• Error Hints: Clear messages for common issues such as invalid API key (401) or rate limits (429).

State Management

This package does not implement state management. The provider is stateless and does not persist any data between requests.

Testing

Testing Setup

The package uses vitest for unit testing:

bun run test
bun run test:coverage

Testing Examples

import { describe, it, expect, beforeEach, vi } from 'vitest';
import SerperWebSearchProvider from '@tokenring-ai/serper';

describe('SerperWebSearchProvider', () => {
  let provider: SerperWebSearchProvider;

  beforeEach(() => {
    provider = new SerperWebSearchProvider({
      apiKey: 'test-api-key',
    });
  });

  it('should be created with valid apiKey', () => {
    expect(provider).toBeDefined();
  });

  it('should throw error without apiKey', () => {
    expect(() => {
      new SerperWebSearchProvider({ apiKey: '' });
    }).toThrow('SerperWebSearchProvider requires apiKey');
  });
});

Package Exports

The package exports the following:

• SerperWebSearchProvider: The main provider class for Serper API integration
• SerperDefaultsSchema: Zod schema for default search parameters
• SerperWebSearchProviderOptionsSchema: Zod schema for provider options
• plugin: Token Ring plugin for automatic service registration

Build and Development

# Install dependencies
bun install

# Run tests
bun run test

# Run tests in watch mode
bun run test:watch

# Generate coverage report
bun run test:coverage

# Type check
bun run build

Dependencies

Production Dependencies

Package	Version	Description
@tokenring-ai/app	0.2.0	Application framework for plugin integration
@tokenring-ai/agent	0.2.0	Agent framework
@tokenring-ai/websearch	0.2.0	Web search provider base class
@tokenring-ai/utility	0.2.0	Utility functions (pick, doFetchWithRetry)
zod	^4.3.6	Runtime type validation

Development Dependencies

Package	Version	Description
vitest	^4.1.1	Testing framework
@vitest/coverage-v8	^4.1.1	Coverage tooling
typescript	^6.0.2	TypeScript compiler

Related Components

Related Packages

• @tokenring-ai/websearch: Web search provider interface and service
• @tokenring-ai/app: Token Ring application framework
• @tokenring-ai/agent: Agent orchestration system
• @tokenring-ai/utility: Shared utilities and helpers

Type Definitions

Request Types

SerperSearchRequest

Request payload structure for Google search

{
  q: string;
  gl?: string;
  hl?: string;
  location?: string;
  num?: number;
  page?: number;
  autocorrect?: boolean;
  type?: "search";
}

SerperNewsRequest

Request payload structure for Google News search

{
  q: string;
  gl?: string;
  location?: string;
  num?: number;
  page?: number;
  type?: "news";
}

SerperSearchParameters

Search parameters in response

{
  q: string;
  gl?: string;
  hl?: string;
  autocorrect?: boolean;
  page?: number;
  type?: string;
}

Response Types

SerperSearchResponse

Response structure for Google search

{
  searchParameters: SerperSearchParameters;
  knowledgeGraph?: SerperKnowledgeGraph;
  organic: SerperOrganicResult[];
  peopleAlsoAsk?: SerperPeopleAlsoAsk[];
  relatedSearches?: SerperRelatedSearch[];
}

SerperNewsResponse

Response structure for news search

{
  searchParameters: SerperSearchParameters;
  news: SerperNewsResult[];
  credits?: number;
}

SerperPageResponse

Response structure for page fetch

{
  text: string;
  markdown: string;
  metadata: {
    title?: string;
    description?: string;
    "og:title"?: string;
    "og:description"?: string;
    "og:url"?: string;
    "og:image"?: string;
    "og:type"?: string;
    "og:site_name"?: string;
    [key: string]: any;
  };
  credits?: number;
}

Result Types

SerperKnowledgeGraph

{
  title: string;
  type: string;
  website?: string;
  imageUrl?: string;
  description?: string;
  descriptionSource?: string;
  descriptionLink?: string;
  attributes?: Record<string, string>;
}

SerperSitelink

{
  title: string;
  link: string;
}

SerperOrganicResult

{
  title: string;
  link: string;
  snippet: string;
  date?: string;
  position: number;
  attributes?: Record<string, string>;
  sitelinks?: SerperSitelink[];
}

SerperPeopleAlsoAsk

{
  question: string;
  snippet: string;
  title: string;
  link: string;
}

SerperRelatedSearch

{
  query: string;
}

SerperNewsResult

{
  title: string;
  link: string;
  snippet: string;
  date: string;
  source: string;
  position: number;
}

Limitations

• Relies on Serper.dev quotas and rate limits
• Only supports text/HTML content (no binary content)
• Web page fetching limited to Serper's scraping capabilities
• News search includes automatic date filtering (tbs="qdr:h")

License

MIT License - Copyright (c) 2025 Mark Dierolf

See LICENSE for details.