Last updated: 2026-07-19
API Server
Http
Minimal HTTP/1.1 server for the OpenAI-compatible inference API.
Provides connection-level request parsing, JSON and SSE response helpers, and a TCP listener that hands off accepted connections to the route dispatcher.
3 exports shown
struct
Connection
pub const Connection = structActive client connection with request/response capabilities.
Wraps a TCP stream and provides helpers for reading HTTP requests and writing JSON, error, and SSE responses.
Methods
9method
Connection.readRequest
pub fn readRequest(self: *Connection) !RequestRead and parse an HTTP/1.1 request from the connection.
Reads headers until `\r\n\r\n`, extracts method/path/Content-Length, then reads the remaining body bytes.
method
Connection.sendJson
pub fn sendJson(self: *Connection, status: u16, body: []const u8) !voidSend a JSON response with the given HTTP status code.
method
Connection.sendError
pub fn sendError(self: *Connection, status: u16, err_type: []const u8, message: []const u8) !voidSend an OpenAI-format JSON error response.
method
Connection.sendSseStart
pub fn sendSseStart(self: *Connection) !voidSend SSE streaming response headers with chunked transfer encoding.
After this call, use writeSseEvent to send individual events.
method
Connection.writeSseEvent
pub fn writeSseEvent(self: *Connection, data: []const u8) !voidWrite a single SSE event as a chunked transfer-encoding frame.
method
Connection.writeSseComment
pub fn writeSseComment(self: *Connection, text: []const u8) !voidWrite a chunked SSE comment line (`: {text}\n\n`) to keep the connection alive.
Useful when a client or intermediate proxy would time out on a quiet stream before the next model token is ready.
method
Connection.writeSseDone
pub fn writeSseDone(self: *Connection) !voidWrite the final SSE `[DONE]` event and send the chunked transfer terminator.
method
Connection.isPeerClosed
pub fn isPeerClosed(self: *Connection) boolCheck whether the remote connection is definitively dead, to let a streaming decode loop bail out before the next token instead of discovering the same thing on the next write.
This only reports `true` on an unambiguous signal: a TCP reset, refusal, or an already-disconnected socket. A zero-byte peek (the peer's write side reached EOF) is deliberately treated as "not closed": HTTP/1.1 clients commonly half-close the upload side right after sending the request body while still reading the response, so that signal cannot distinguish "client hung up" from "compliant client, still receiving the SSE stream". Bailing out on it would truncate valid streams for such clients — worse than the wasted decode time this function is meant to save. Only a write failure (`catch return` at each write-path call site) proves that case.
method
Connection.close
pub fn close(self: *Connection) voidClose the underlying TCP stream and free any heap-allocated overflow body from `readRequest`.
enum
Method
pub const Method = enum GET, POST, OPTIONS, UNKNOWN }HTTP request method parsed from the request line.
`UNKNOWN` is the fallback for any method not explicitly handled by the server.
struct
Server
pub const Server = structHTTP server that binds and listens on a TCP port.
Accepts connections and wraps them in Connection structs for request handling.
Methods
3method
Server.init
pub fn init(allocator: std.mem.Allocator, port: u16) !ServerBind to all interfaces on the given port and start listening.
method
Server.accept
pub fn accept(self: *Server) !ConnectionBlock until a client connects, then return a Connection for that client.
method
Server.deinit
pub fn deinit(self: *Server) voidStop listening and release the server socket.