@tokenring-ai/wikipedia
The @tokenring-ai/wikipedia package provides seamless integration with Wikipedia's API, enabling Token Ring agents and applications to search for articles, retrieve content, and access a global knowledge base. This plugin wraps Wikipedia's REST API, providing a clean interface for article searches and page content retrieval.
Overview and Purpose
The @tokenring-ai/wikipedia package enables Token Ring AI agents to interact with Wikipedia programmatically. It provides a service-based architecture with pre-built tools for searching articles and retrieving raw wiki markup content. The package is designed for integration within the Token Ring ecosystem, allowing agents to leverage Wikipedia's knowledge base for research, fact-checking, and content generation tasks.
Key Features
- API Integration: Access Wikipedia's search and raw content APIs
- Flexible Search: Search Wikipedia articles with configurable limits and pagination
- Content Retrieval: Fetch raw wiki markup content for any Wikipedia page
- Multi-Language Support: Works with Wikipedia content in multiple languages via baseUrl configuration
- Type Safety: Full TypeScript support with Zod schema validation
- Agent Tools: Two tools automatically registered for agent interaction
- Configurable: Optional baseUrl for different Wikipedia language editions
- Error Handling: Comprehensive error handling with status codes and clear error messages
- Retry Logic: Built-in retry logic via
doFetchWithRetryfor transient network failures - Service Architecture: Extends
HttpServicefor consistent HTTP handling
Core Components
WikipediaService
The main service class that handles all Wikipedia API interactions. Implements TokenRingService interface and extends HttpService from @tokenring-ai/utility.
Location: pkg/wikipedia/WikipediaService.ts
Constructor
constructor(config: ParsedWikipediaConfig)
Parameters:
config(ParsedWikipediaConfig): Configuration objectbaseUrl(optional): Base URL for Wikipedia API, defaults tohttps://en.wikipedia.org
Properties:
name: "WikipediaService"description: "Service for searching Wikipedia articles"options: Service configuration (ParsedWikipediaConfig)baseUrl: Protected property storing the base URLdefaultHeaders: Protected property with User-Agent header ("TokenRing-Writer/1.0 (https://github.com/tokenring/writer)")
Service Methods
search(query: string, opts?: WikipediaSearchOptions): Promise<any>
Performs a Wikipedia search and returns structured search results with articles matching the query.
Parameters:
query(required): Search query stringopts(optional): Search optionslimit(optional): Number of results to return (1-500, default: 10)namespace(optional): Article namespace (default: 0 = article namespace)offset(optional): Pagination offset for large result sets (default: 0)
Returns: Promise resolving to Wikipedia API search response:
{
query: {
searchinfo: {
totalresults: number;
[key: string]: any;
};
search: Array<{
title: string;
pageid: number;
namespace: number;
snippet: string;
[key: string]: any;
}>;
[key: string]: any;
};
}
Throws: Error if query is empty
Example:
import WikipediaService, {WikipediaConfigSchema} from "@tokenring-ai/wikipedia";
const wikipedia = new WikipediaService(WikipediaConfigSchema.parse({}));
// Basic search
const results = await wikipedia.search("artificial intelligence");
console.log(`Found ${results.query.search.length} results`);
// Search with custom parameters
const specializedResults = await wikipedia.search("Token Ring AI framework", {
limit: 20,
namespace: 0,
offset: 0
});
getPage(title: string): Promise<string>
Retrieves raw wiki markup content for a specific Wikipedia page title. Uses doFetchWithRetry from @tokenring-ai/utility for reliable network requests.
Parameters:
title(required): Page title (URL-encoded in practice, but the service handles this)
Returns:
- Raw wiki markup content as a text string
Throws: Error if title is empty or page retrieval fails (includes status code in error object)
Example:
// Get raw wiki markup for a page
const content = await wikipedia.getPage("Token_Ring");
console.log(content.substring(0, 200) + "...");
// Returns: Raw wiki markup content
Services
WikipediaService
The WikipediaService is a TokenRingService that can be required by agents using the requireServiceByType method.
Provider Type:
import WikipediaService from "@tokenring-ai/wikipedia";
// In an agent context
const wikipedia = agent.requireServiceByType(WikipediaService);
Service Registration:
The service is automatically registered when the plugin is installed:
import TokenRingApp from "@tokenring-ai/app";
import wikipediaPlugin from "@tokenring-ai/wikipedia";
const app = new TokenRingApp();
app.install(wikipediaPlugin, {
wikipedia: {
baseUrl: "https://en.wikipedia.org"
}
});
Providers
This package does not use a provider registry pattern. The WikipediaService is a standalone service class that implements TokenRingService and is registered directly with the application.
RPC Endpoints
This package does not define RPC endpoints.
Chat Commands
This package does not define chat commands. The functionality is exposed through agent tools instead.
Configuration
Service Options
Wikipedia configuration is optional and uses Zod schema validation:
import {WikipediaConfigSchema} from "@tokenring-ai/wikipedia";
interface WikipediaConfig {
baseUrl?: string; // Defaults to "https://en.wikipedia.org"
}
Schema Definition:
export const WikipediaConfigSchema = z.object({
baseUrl: z.string().default("https://en.wikipedia.org")
});
export type ParsedWikipediaConfig = z.output<typeof WikipediaConfigSchema>;
Configuration Examples
Basic Configuration
// In your application configuration
const config = {
wikipedia: {
baseUrl: "https://en.wikipedia.org"
}
};
Multi-Language Configuration
// For German Wikipedia
const germanConfig = {
wikipedia: {
baseUrl: "https://de.wikipedia.org"
}
};
// For French Wikipedia
const frenchConfig = {
wikipedia: {
baseUrl: "https://fr.wikipedia.org"
}
};
// For Spanish Wikipedia
const spanishConfig = {
wikipedia: {
baseUrl: "https://es.wikipedia.org"
}
};
// For Japanese Wikipedia
const japaneseConfig = {
wikipedia: {
baseUrl: "https://ja.wikipedia.org"
}
};
Plugin Installation with Configuration
import TokenRingApp from "@tokenring-ai/app";
import wikipediaPlugin from "@tokenring-ai/wikipedia";
const app = new TokenRingApp();
app.install(wikipediaPlugin, {
wikipedia: {
baseUrl: "https://en.wikipedia.org"
}
});
Note: The baseUrl should be the base URL without the API path. The service constructs the full API endpoint internally.
Wikipedia Tools
The package provides two tools for agent integration, automatically registered with the TokenRing chat service when the plugin is installed.
wikipedia_search
Search Wikipedia articles with structured results.
Tool Definition:
{
name: "wikipedia_search",
displayName: "Wikipedia/search",
description: "Search Wikipedia articles. Returns structured JSON with search results.",
inputSchema: z.object({
query: z.string().min(1).describe("Search query"),
limit: z.number().int().positive().max(500).optional().describe("Number of results (1-500, default: 10)"),
offset: z.number().int().min(0).optional().describe("Offset for pagination (default: 0)"),
}),
execute: async (input, agent) => {
const wikipedia = agent.requireServiceByType(WikipediaService);
agent.infoMessage(`[wikipediaSearch] Searching: ${input.query}`);
const results = await wikipedia.search(input.query, {limit: input.limit, offset: input.offset});
return { type: 'json' as const, data: results };
}
}
Input Schema:
z.object({
query: z.string().min(1).describe("Search query"),
limit: z.number().int().positive().max(500).optional().describe("Number of results (1-500, default: 10)"),
offset: z.number().int().min(0).optional().describe("Offset for pagination (default: 0)"),
})
Returns: TokenRingToolJSONResult with search results
Usage:
// In agent task execution, tool is auto-available
const agent = new Agent();
const results = await agent.executeTool("wikipedia_search", {
query: "artificial intelligence",
limit: 10
});
console.log(results.data.query.search);
// Output: Array of search results with title, snippet, pageid, etc.
wikipedia_getPage
Retrieve raw wiki markup content for a specific Wikipedia page.
Tool Definition:
{
name: "wikipedia_getPage",
displayName: "Wikipedia/getPage",
description: "Retrieve a Wikipedia page's raw wiki markup content by title.",
inputSchema: z.object({
title: z.string().min(1).describe("Wikipedia page title"),
}),
execute: async (input, agent) => {
const wikipedia = agent.requireServiceByType(WikipediaService);
try {
agent.infoMessage(`[wikipediaGetPage] Retrieving: ${input.title}`);
return await wikipedia.getPage(input.title);
} catch (e: any) {
const message = e?.message || String(e);
throw new Error(`[${name}] ${message}`);
}
}
}
Input Schema:
z.object({
title: z.string().min(1).describe("Wikipedia page title"),
})
Returns: TokenRingToolTextResult with raw wiki markup content
Usage:
// In agent task execution, tool is auto-available
const agent = new Agent();
const content = await agent.executeTool("wikipedia_getPage", {
title: "Machine learning"
});
console.log(content.data);
// Output: Raw wiki markup content of the page
Package Structure
pkg/wikipedia/
├── index.ts # Main exports (WikipediaService)
├── WikipediaService.ts # Core service class extending HttpService
├── plugin.ts # TokenRingPlugin registration
├── tools.ts # Barrel export for tools/
├── tools/
│ ├── search.ts # Wikipedia search tool
│ └── getPage.ts # Wikipedia page retrieval tool
├── package.json # Package metadata and dependencies
├── vitest.config.ts # Test configuration
└── test/
└── WikipediaService.integration.test.ts # Integration tests
Usage Examples
Service Integration
import WikipediaService, {WikipediaConfigSchema} from "@tokenring-ai/wikipedia";
// Basic usage
const wikipedia = new WikipediaService(WikipediaConfigSchema.parse({}));
// Search Wikipedia articles
const results = await wikipedia.search("Token Ring AI framework");
console.log(`Total results: ${results.query.search.length}`);
results.query.search.forEach(result => {
console.log(`${result.title} - ${result.snippet}`);
});
// Search with custom parameters
const techResults = await wikipedia.search("artificial intelligence", {
limit: 20,
offset: 0
});
// Get raw wiki markup for a page
const content = await wikipedia.getPage("Token_Ring");
console.log(content.substring(0, 100) + "...");
// Multi-language Wikipedia
const germanWikipedia = new WikipediaService({
baseUrl: "https://de.wikipedia.org"
});
const germanResults = await germanWikipedia.search("KI-Forschung");
const germanContent = await germanWikipedia.getPage("KI");
Agent Tool Usage
import Agent from "@tokenring-ai/agent/Agent";
// Tools are auto-registered with agent after package installation
const agent = new Agent();
// Search Wikipedia
const searchResults = await agent.executeTool("wikipedia_search", {
query: "machine learning",
limit: 5
});
console.log(searchResults.data.query.search);
// Pagination example
const firstPage = await agent.executeTool("wikipedia_search", {
query: "typescript",
limit: 10,
offset: 0
});
const secondPage = await agent.executeTool("wikipedia_search", {
query: "typescript",
limit: 10,
offset: 10
});
// Get page content
const pageContent = await agent.executeTool("wikipedia_getPage", {
title: "TypeScript"
});
console.log(pageContent.data.substring(0, 500));
Integration with Token Ring Applications
import TokenRingApp from '@tokenring-ai/app';
import wikipediaPlugin from '@tokenring-ai/wikipedia';
const app = new TokenRingApp();
// Register Wikipedia plugin
app.install(wikipediaPlugin, {
wikipedia: {
baseUrl: "https://en.wikipedia.org"
}
});
await app.start();
// Tools will be auto-registered and accessible via agent.executeTool()
Complete Workflow Example
import Agent from "@tokenring-ai/agent/Agent";
import WikipediaService from "@tokenring-ai/wikipedia";
async function researchTopic(agent: Agent, query: string) {
// Search for relevant articles
const searchResult = await agent.executeTool("wikipedia_search", {
query,
limit: 5
});
// Process results
if (searchResult.data.query.search.length === 0) {
throw new Error("No relevant articles found");
}
// Get content from the most relevant article
const topArticle = searchResult.data.query.search[0];
const pageContent = await agent.executeTool("wikipedia_getPage", {
title: topArticle.title
});
return {
title: topArticle.title,
snippet: topArticle.snippet,
content: pageContent.data,
pageId: topArticle.pageid
};
}
// Usage
const agent = new Agent();
const research = await researchTopic(agent, "quantum computing");
console.log(`Research on: ${research.title}`);
console.log(`Content length: ${research.content.length} characters`);
Response Types
WikipediaSearchResponse
The search method returns a structured response with comprehensive metadata:
{
query: {
searchinfo: {
totalresults: number; // Total number of matching articles
[key: string]: any;
};
search: Array<{
title: string; // Article title
pageid: number; // Page ID (unique identifier)
namespace: number; // Article namespace (0=main, etc.)
snippet: string; // Search snippet (HTML-formatted)
size?: number; // Page size in bytes
wordcount?: number; // Word count
[key: string]: any;
}>;
[key: string]: any; // Additional API fields
};
}
Tool Response Types
Search Tool Response
{
type: 'json';
data: WikipediaSearchResponse;
}
GetPage Tool Response
{
type: 'text';
data: string; // Raw wiki markup
}
Error Handling
The service implements comprehensive error handling:
- Input Validation: Required
queryandtitleparameters validated before API calls - HTTP Errors: Non-2xx responses include status code
- Error Properties: Errors include
messageand optionalstatusproperties - Tool-level Error Wrapping: The
wikipedia_getPagetool wraps errors with tool name prefix - Retry Logic:
getPage()usesdoFetchWithRetryfor transient network failures
Error Examples
import WikipediaService, {WikipediaConfigSchema} from "@tokenring-ai/wikipedia";
const wikipedia = new WikipediaService(WikipediaConfigSchema.parse({}));
// Empty query error
try {
const results = await wikipedia.search("");
} catch (error) {
console.log(error.message); // "query is required"
}
// Empty title error
try {
const results = await wikipedia.getPage("");
} catch (error) {
console.log(error.message); // "title is required"
}
// HTTP error (page not found)
try {
const results = await wikipedia.getPage("NonExistentPage12345");
} catch (error) {
console.log(error.message); // "Wikipedia page retrieval failed (404)"
console.log(error.status); // HTTP status code: 404
}
// Tool-level error wrapping
try {
await agent.executeTool("wikipedia_getPage", {title: ""});
} catch (error) {
console.log(error.message); // "[wikipedia_getPage] title is required"
}
Common Error Scenarios
- Missing Query:
Error: query is required - Missing Title:
Error: title is required - HTTP Error:
Error: Wikipedia page retrieval failed (404)withstatusproperty - Network Error: Service-level errors from fetch implementation (retry logic applied)
- Rate Limiting: HTTP 429 error with rate limit information
API Reference
Wikipedia API Endpoints
Search Endpoint
Location: /w/api.php?{params}
Parameters:
action=query- Action type (fixed)list=search- List search resultssrsearch={query}- Search query stringformat=json- Return format (fixed)srlimit={number}- Number of results (1-500, default: 10)srnamespace={number}- Article namespace (default: 0)sroffset={number}- Pagination offset (default: 0)
Example Request:
GET https://en.wikipedia.org/w/api.php?action=query&list=search&srsearch=AI&format=json&srlimit=10
Response Structure:
{
"batchcomplete": true,
"query": {
"searchinfo": {
"totalresults": 12345
},
"search": [
{
"title": "Artificial intelligence",
"pageid": 123456,
"namespace": 0,
"snippet": "...",
"size": 123456,
"wordcount": 5000
}
]
}
}
Raw Content Endpoint
Location: {baseUrl}/w/index.php?{params}
Parameters:
title={page}- Page titleaction=raw- Return raw wiki markup (fixed)
Example Request:
GET https://en.wikipedia.org/w/index.php?title=Artificial_intelligence&action=raw
Response:
- Raw wiki markup text (plain text, not JSON)
URLSearchParams Construction
The service uses URLSearchParams for parameter building:
// Search endpoint
const params = new URLSearchParams({
action: "query",
list: "search",
srsearch: query,
format: "json",
srlimit: String(opts.limit || 10),
srnamespace: String(opts.namespace || 0),
sroffset: String(opts.offset || 0),
});
// Raw content endpoint
const params = new URLSearchParams({
title: title,
action: "raw",
});
HTTP Headers
The service sets a custom User-Agent header for all requests:
defaultHeaders = {
"User-Agent": "TokenRing-Writer/1.0 (https://github.com/tokenring/writer)"
}
This is required by Wikipedia's API policy for proper identification of API clients.
Best Practices
-
Rate Limiting: Wikipedia API has rate limits; implement appropriate delays if performing many searches
// Example: Add delay between requests
const delay = (ms: number) => new Promise(resolve => setTimeout(resolve, ms));
for (const query of queries) {
await wikipedia.search(query);
await delay(1000); // 1 second delay
} -
Query Optimization:
- Use specific, descriptive queries for better results
- Consider using Boolean operators (e.g., "machine learning AND neural networks")
- Adjust
limitparameter based on result quantity needs
-
Pagination: For large result sets, use
offsetparameter to paginate through resultsconst pageSize = 50;
let offset = 0;
const allResults = [];
do {
const results = await wikipedia.search(query, { limit: pageSize, offset });
if (results.query.search.length === 0) break;
allResults.push(...results.query.search);
offset += pageSize;
} while (offset < results.query.searchinfo.totalresults); -
Page Titles: Use underscores for spaces in page titles (
token_ringnottoken ring)- Consider URL-encoding when dealing with special characters programmatically
- Wikipedia API expects underscores in page titles
-
Namespace Filtering: Use namespace parameter in service calls to target specific content types
- 0: Main article namespace (default)
- 14: Category namespace
- 108: Template namespace
- Other namespaces available per Wikipedia documentation
- Note: The
wikipedia_searchtool does not expose the namespace parameter; use the service directly for namespace filtering
-
Error Handling: Always wrap API calls in try-catch blocks
try {
const results = await wikipedia.search("your query");
// Process results
} catch (error) {
if (error.status === 429) {
// Rate limited - implement backoff
await delay(5000);
return wikipedia.search("your query"); // Retry
}
throw error;
} -
Caching: Cache frequently accessed search results and page content
const cache = new Map<string, any>();
async function cachedSearch(query: string) {
if (cache.has(query)) return cache.get(query);
const results = await wikipedia.search(query);
cache.set(query, results);
return results;
}
async function cachedGetPage(title: string) {
if (cache.has(title)) return cache.get(title);
const content = await wikipedia.getPage(title);
cache.set(title, content);
return content;
} -
Multi-Language: Configure appropriate
baseUrlfor different language editions- English:
https://en.wikipedia.org - German:
https://de.wikipedia.org - French:
https://fr.wikipedia.org - Spanish:
https://es.wikipedia.org - Japanese:
https://ja.wikipedia.org
- English:
-
Content Processing: Raw wiki markup requires parsing for human-readable content
- Consider using a wiki markup parser library
- Strip templates and references for clean text
- Handle special characters and formatting
-
Retry Logic: The
getPage()method usesdoFetchWithRetryfrom@tokenring-ai/utilityfor automatic retry on transient failures. Thesearch()method uses the baseHttpService.fetchJson()which does not include retry logic.
Integration
Agent Integration
The package integrates with the Token Ring agent system through:
- Service Registration:
WikipediaServiceis registered as aTokenRingService - Tool Registration: Two tools are automatically registered with
ChatService - Service Access: Agents can access the service via
requireServiceByType
import Agent from "@tokenring-ai/agent/Agent";
import WikipediaService from "@tokenring-ai/wikipedia";
// In an agent context
const wikipedia = agent.requireServiceByType(WikipediaService);
const results = await wikipedia.search("machine learning");
Plugin Installation
import TokenRingApp from "@tokenring-ai/app";
import wikipediaPlugin from "@tokenring-ai/wikipedia";
const app = new TokenRingApp();
// Install with default configuration (English Wikipedia)
app.install(wikipediaPlugin, {
wikipedia: {
baseUrl: "https://en.wikipedia.org"
}
});
// Or with custom configuration
app.install(wikipediaPlugin, {
wikipedia: {
baseUrl: "https://de.wikipedia.org" // German Wikipedia
}
});
await app.start();
Tool Registration
Tools are automatically registered when the plugin is installed:
// In plugin.ts
export default {
name: packageJSON.name,
version: packageJSON.version,
description: packageJSON.description,
install(app, config) {
app.waitForService(ChatService, chatService =>
chatService.addTools(tools) // Tools auto-registered
);
app.addServices(new WikipediaService(config.wikipedia));
},
config: packageConfigSchema
} satisfies TokenRingPlugin<typeof packageConfigSchema>;
Testing
Running Tests
cd pkg/wikipedia
bun run test
Test Configuration
# Run tests in watch mode
bun run test:watch
# Run tests with coverage
bun run test:coverage
# Verify build compiles without errors
bun run build
Integration Tests
The package includes integration tests that verify:
- Wikipedia search functionality with various parameters
- Page content retrieval
- Error handling for invalid inputs
- Support for different language editions
- Pagination with offset support
Test file: pkg/wikipedia/test/WikipediaService.integration.test.ts
Example Test
import {describe, it, expect} from "vitest";
import WikipediaService, {WikipediaConfigSchema} from "../WikipediaService.ts";
describe("WikipediaService", () => {
const wikipedia = new WikipediaService(WikipediaConfigSchema.parse({}));
it("should search for articles", async () => {
const results = await wikipedia.search("artificial intelligence", {limit: 5});
expect(results.query.search).toHaveLength(5);
expect(results.query.search[0]).toHaveProperty("title");
expect(results.query.search[0]).toHaveProperty("snippet");
});
it("should retrieve page content", async () => {
const content = await wikipedia.getPage("Artificial_intelligence");
expect(typeof content).toBe("string");
expect(content.length).toBeGreaterThan(0);
});
it("should throw error for empty query", async () => {
await expect(wikipedia.search("")).rejects.toThrow("query is required");
});
it("should throw error for empty title", async () => {
await expect(wikipedia.getPage("")).rejects.toThrow("title is required");
});
});
Dependencies
Production Dependencies
@tokenring-ai/app(0.2.0): Application framework for service registration@tokenring-ai/chat(0.2.0): Chat functionality for tool registration@tokenring-ai/agent(0.2.0): Agent framework for tool execution@tokenring-ai/utility(0.2.0): HTTP service base class and general utilities (doFetchWithRetry,HttpService)zod(^4.3.6): Runtime type validation for configuration and inputs
Development Dependencies
vitest(^4.0.18): Testing framework@vitest/coverage-v8(^4.0.18): Code coveragetypescript(^5.9.3): TypeScript support
The service extends HttpService from @tokenring-ai/utility for base HTTP functionality and uses doFetchWithRetry for reliable network requests on page retrieval.
Limitations
- API Rate Limits: Wikipedia API has rate limits; high-volume usage requires implementation of rate limiting
- No News Integration: This package provides Wikipedia article search only, not news updates
- No Article Editing: Read-only API access; cannot create or modify articles
- Network Dependency: Requires network connectivity to Wikipedia API
- Content Type: Returns raw wiki markup; does not render HTML or formatted content
- Namespace Support: Service supports standard Wikipedia namespaces (0, 14, 108, etc.), but the
wikipedia_searchtool does not expose the namespace parameter - No Image Handling: Does not provide direct access to images or media files
- No Category Browsing: Search only, no category tree traversal
- Retry Logic: Only
getPage()uses retry logic viadoFetchWithRetry;search()does not
Related Components
@tokenring-ai/research- Research service that may integrate Wikipedia functionality@tokenring-ai/websearch- General web search integration@tokenring-ai/browser- Browser-based content retrievalHttpService- Base class for HTTP service implementations in@tokenring-ai/utilityTokenRingAgent- Agent framework for tool execution and service accessChatService- Chat service for tool registration in@tokenring-ai/chatdoFetchWithRetry- Retry-enabled fetch utility in@tokenring-ai/utility
Notes
- Wikipedia API uses underscores for spaces in titles (
Token_RingnotToken Ring) - The
getPage()method usesdoFetchWithRetryfrom utility package for reliability - The
search()method uses baseHttpService.fetchJson()without retry logic - User-Agent header defaults to
"TokenRing-Writer/1.0 (https://github.com/tokenring/writer)" - Search results include HTML-formatted snippets for display purposes
- Page content returns raw wiki markup
- Maximum search results per request: 500
- Namespace parameter values match Wikipedia's namespace ID system (available in service, not in tool)
- All API calls are asynchronous and return Promises
- Configuration uses Zod schema validation for type safety
- Tool-level error wrapping adds tool name prefix to error messages
License
MIT License - see the root LICENSE file for details.
Part of the Token Ring AI monorepo - building the future of AI-powered development tools.