S3 Plugin

AWS S3 integrations for filesystem operations and CDN functionality with bucket management.

Overview

The @tokenring-ai/s3 package provides AWS S3 integrations for the Token Ring AI ecosystem. It implements two main abstractions: FileSystemProvider for treating S3 buckets as a filesystem, and CDNResource for content delivery operations.

Key Features

• S3 Filesystem: Treat S3 buckets as a filesystem with read, write, delete, and list operations
• CDN Interface: Upload, delete, and manage resources for content delivery
• Directory Simulation: Uses S3 prefixes to simulate directory structures
• Path Normalization: Handles relative and absolute paths with key normalization
• Error Handling: Graceful handling of non-existent objects and S3 errors

Core Components

S3FileSystemProvider

Extends FileSystemProvider to provide S3-backed file operations.

Constructor:

new S3FileSystemProvider({
  bucketName: string,           // required
  clientConfig?: S3ClientConfigType  // optional AWS config
})

Key Methods:

• writeFile(fsPath, content): Promise<boolean>: Write file to S3
• readFile(fsPath, encoding?): Promise<any>: Read file (utf8 or buffer)
• deleteFile(fsPath): Promise<boolean>: Delete object
• exists(fsPath): Promise<boolean>: Check if path exists
• stat(fsPath): Promise<StatLike>: Get file/directory stats
• getDirectoryTree(fsPath, options?): AsyncGenerator<string>: List directory contents
• createDirectory(fsPath, options?): Promise<boolean>: Create directory prefix
• copy(source, dest, options?): Promise<boolean>: Copy object within bucket
• rename(oldPath, newPath): Promise<boolean>: Move object (copy + delete)

Unsupported Operations:

• chmod, watch, executeCommand, glob, grep (throws errors)

S3CDNResource

Extends CDNResource for S3-based content delivery.

Constructor:

new S3CDNResource({
  bucket: string,               // required
  region: string,               // required
  accessKeyId: string,          // required
  secretAccessKey: string,      // required
  baseUrl?: string              // optional, defaults to S3 URL
})

Key Methods:

• upload(data: Buffer, options?): Promise<UploadResult>
  • Uploads buffer with optional filename, contentType, metadata
  • Returns: { url, id, metadata }
• delete(url: string): Promise<DeleteResult>
  • Deletes object from extracted key
  • Returns: { success, message }
• exists(url: string): Promise<boolean>
  • Checks object existence via HEAD request

Usage Examples

Using S3FileSystemProvider

import { S3FileSystemProvider } from '@tokenring-ai/s3';

const provider = new S3FileSystemProvider({
  bucketName: 'my-bucket',
  clientConfig: { region: 'us-east-1' }
});

// Write a file
await provider.writeFile('hello.txt', 'Hello, S3!');

// Read it back
const content = await provider.readFile('hello.txt', 'utf8');
console.log(content); // "Hello, S3!"

// List directory
for await (const path of provider.getDirectoryTree('.')) {
  console.log(path);
}

// Create directory
await provider.createDirectory('docs');

// Copy file
await provider.copy('hello.txt', 'backup/hello.txt');

Using S3CDNResource

import { S3CDNResource } from '@tokenring-ai/s3';

const cdn = new S3CDNResource({
  bucket: 'my-cdn-bucket',
  region: 'us-east-1',
  accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
  baseUrl: 'https://my-cdn.example.com'
});

// Upload
const buffer = Buffer.from('Image data...');
const result = await cdn.upload(buffer, {
  filename: 'image.png',
  contentType: 'image/png',
  metadata: { author: 'User' }
});
console.log(result.url);

// Delete
await cdn.delete(result.url);

// Check existence
const exists = await cdn.exists(result.url);

Configuration Options

S3FileSystemProviderOptions

• bucketName: Required S3 bucket name
• clientConfig: Optional AWS SDK config (region, credentials, endpoint)

S3CDNResourceOptions

• bucket: Required S3 bucket name
• region: Required AWS region
• accessKeyId: Required AWS access key
• secretAccessKey: Required AWS secret key
• baseUrl: Optional custom URL base (defaults to https://{bucket}.s3.amazonaws.com)

Environment Variables (recommended):

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_REGION

Dependencies

• @tokenring-ai/agent@0.1.0: Core agent framework
• @tokenring-ai/cdn@0.1.0: CDN abstraction
• @tokenring-ai/filesystem@0.1.0: Filesystem abstraction
• @aws-sdk/client-s3@^3.0.0: AWS S3 client

Notes

• Directories are simulated using S3 prefixes (trailing /)
• No real-time watching or shell execution (S3 is object storage)
• Path normalization prevents traversal above bucket root
• Binary files supported via Buffer
• Use AWS IAM for least-privilege access