ircstore

Persistent IRC log storage with S2 compression and real-time WebSocket streaming.

What is ircstore?

ircstore is a high-performance IRC logging system that:

• Stores IRC messages using S2 compression for efficient disk usage
• Provides REST API for querying historical logs
• Streams real-time messages via WebSocket
• Maintains separate streams for each IRC channel

REST API

curl https://ircstore.rotko.net/api/channels

["#rust", "#linux", "#networking"]

curl "https://ircstore.rotko.net/api/logs/%23rust?limit=50&offset=0"

[
  {
    "timestamp": 1698765432,
    "channel": "#rust",
    "nick": "alice",
    "message": "anyone familiar with async traits?"
  }
]

WebSocket API

const ws = new WebSocket('wss://ircstore.rotko.net/ws');

ws.onmessage = (event) => {
    const msg = JSON.parse(event.data);
    console.log(`[${msg.channel}] ${msg.nick}: ${msg.message}`);
};

{
  "timestamp": 1698765432,
  "channel": "#rust",
  "nick": "alice",
  "message": "anyone familiar with async traits?"
}

Example: Python Client

import requests
import websocket
import json

# Get available channels
channels = requests.get('https://ircstore.rotko.net/api/channels').json()
print(f"Available channels: {channels}")

# Get recent logs
logs = requests.get('https://ircstore.rotko.net/api/logs/%23rust?limit=10').json()
for msg in logs:
    print(f"[{msg['nick']}] {msg['message']}")

# Subscribe to real-time updates
def on_message(ws, message):
    msg = json.loads(message)
    print(f"[{msg['channel']}] {msg['nick']}: {msg['message']}")

ws = websocket.WebSocketApp('wss://ircstore.rotko.net/ws', on_message=on_message)
ws.run_forever()

Example: JavaScript Client

// Fetch historical logs
async function getLogs(channel, limit = 100) {
    const res = await fetch(`/api/logs/${encodeURIComponent(channel)}?limit=${limit}`);
    return await res.json();
}

// Real-time subscription
const ws = new WebSocket('wss://ircstore.rotko.net/ws');
const messageHandlers = new Map();

ws.onmessage = (event) => {
    const msg = JSON.parse(event.data);
    const handler = messageHandlers.get(msg.channel);
    if (handler) handler(msg);
};

// Subscribe to specific channel
function subscribeToChannel(channel, callback) {
    messageHandlers.set(channel, callback);
}

subscribeToChannel('#rust', (msg) => {
    console.log(`${msg.nick}: ${msg.message}`);
});

Data Format

IrcEvent Structure

interface IrcEvent {
    timestamp: number;  // Unix timestamp
    channel: string;    // IRC channel name
    nick: string;       // Sender nickname
    message: string;    // Message content
}

Implementation Details

• Built with Rust using Axum web framework
• S2 compression for efficient storage
• Append-only log structure for each channel
• Tokio async runtime for high concurrency
• CORS enabled for browser-based clients

Source Code

The project is open source and available at github.com/rotkonetworks/ircstore