Open WebUI Saturn Pipe

Goal and Purpose

The owui_saturn.py file implements a dynamic service discovery pipe for Open WebUI that automatically discovers and connects to AI model services advertised on the local network. This pipe enables Open WebUI to seamlessly integrate with multiple LLM providers without manual configuration, leveraging zero-configuration networking (Zeroconf/mDNS) for automatic service discovery.

The primary goal is to eliminate manual endpoint configuration by allowing AI services to advertise themselves via DNS-SD under the _saturn._tcp service type. Open WebUI can then automatically discover these services, enumerate their available models, and route requests accordingly.

The implementation consists of three main components:

SaturnDiscovery: Uses the dns-sd command-line tool to browse for services advertising the _saturn._tcp service type. It performs service lookup to resolve hostnames, ports, and priority values for discovered services.

Pipe Class: Implements the Open WebUI pipe interface with configurable valves for discovery timeout, failover behavior, cache TTL, and request timeout. It maintains a cached list of discovered services and their available models, periodically refreshing based on the cache TTL.

Model Routing and Failover: When a request arrives, the pipe parses the model string to determine the target service and model ID. It supports automatic failover to alternative services hosting the same model if the primary service fails, with services prioritized by their advertised priority values.

The pipe exposes discovered models through the pipes() method, which returns a list of available models prefixed with service names. Requests are routed to the appropriate service's /v1/chat/completions endpoint, with support for both streaming and non-streaming responses.