@userx v1.1.2 unload_models_from_vram.py
Python · 123 lines32 class Valves(BaseModel):33 OLLAMA_ENDPOINT: str = Field(34 default="http://host.docker.internal:11434",35 description="Base URL of the Ollama REST API",36 )37 REQUEST_TIMEOUT: int = Field(38 default=3,39 description="HTTP timeout in seconds",40 )41 VERIFY_SSL: bool = Field(42 default=True,43 description="Verify HTTPS certificates",Unload Models from VRAM
Welcome to the Unload Models from VRAM function for Open‑WebUI!
This streamlined tool effortlessly frees GPU memory by unloading models through Ollama’s REST API. If your VRAM feels a bit crowded, clear it out with a single click—think of it as spring‑cleaning for your digital workspace. 😉
Overview
This function:
- Retrieves Loaded Models: Calls
/api/psto list models currently in VRAM. - Unloads Each Model: Sends a POST to
/api/generate(prompt:"",keep_alive:0,stream:false) for every model found. - One‑Click Action: Just hit the “Unload Models from VRAM” button—no extra confirmation screens.
- Configurable Settings: Endpoint URL, time‑outs, SSL verification, delay, and logging level are all adjustable.
- Robust Error Handling: Clear emitter messages and detailed logs help you troubleshoot misconfigurations, time‑outs, or connection issues.
- Progress Feedback: Real‑time status updates (with a progress bar) keep you informed while each model unloads.
Features
| Feature | Description |
|---|---|
| REST API Integration | Seamlessly interacts with Ollama’s API. |
| Configurable Endpoint & Time‑out | Defaults: http://host.docker.internal:11434, 3 s. |
| SSL Verification Toggle | VERIFY_SSL valve lets you enable or disable TLS cert checks. |
| Adjustable Logging Level | DEBUG, INFO, WARNING, ERROR, CRITICAL. |
| Selective Unload | Pass a models list in the request body to unload just one or two heavyweights. |
| Delay Between Unloads | Optional (default 200 ms) delay ensures the UI renders progress nicely. |
| Graceful Error Handling | Concise user messages + verbose logs when you need them. |
| Asynchronous Operation | Non‑blocking design keeps the chat UI responsive. |
Configuration
| Valve | Default | Purpose |
|---|---|---|
OLLAMA_ENDPOINT | http://host.docker.internal:11434 | Base URL of the Ollama REST API. |
REQUEST_TIMEOUT | 3 seconds | HTTP request time‑out. |
UNLOAD_DELAY_MS | 200 | Wait time between model unloads (set 0 for maximum speed). |
VERIFY_SSL | true | Toggle TLS certificate verification. |
LOG_LEVEL | INFO | Logging verbosity. |
- Version:
1.1.2 - Required Open‑WebUI Version:
0.6.0
0
0 
Unload Models from VRAM
Welcome to the Unload Models from VRAM function for Open‑WebUI!
This streamlined tool effortlessly frees GPU memory by unloading models through Ollama’s REST API. If your VRAM feels a bit crowded, clear it out with a single click—think of it as spring‑cleaning for your digital workspace. 😉
Overview
This function:
- Retrieves Loaded Models: Calls
/api/psto list models currently in VRAM. - Unloads Each Model: Sends a POST to
/api/generate(prompt:"",keep_alive:0,stream:false) for every model found. - One‑Click Action: Just hit the “Unload Models from VRAM” button—no extra confirmation screens.
- Configurable Settings: Endpoint URL, time‑outs, SSL verification, delay, and logging level are all adjustable.
- Robust Error Handling: Clear emitter messages and detailed logs help you troubleshoot misconfigurations, time‑outs, or connection issues.
- Progress Feedback: Real‑time status updates (with a progress bar) keep you informed while each model unloads.
Features
| Feature | Description |
|---|---|
| REST API Integration | Seamlessly interacts with Ollama’s API. |
| Configurable Endpoint & Time‑out | Defaults: http://host.docker.internal:11434, 3 s. |
| SSL Verification Toggle | VERIFY_SSL valve lets you enable or disable TLS cert checks. |
| Adjustable Logging Level | DEBUG, INFO, WARNING, ERROR, CRITICAL. |
| Selective Unload | Pass a models list in the request body to unload just one or two heavyweights. |
| Delay Between Unloads | Optional (default 200 ms) delay ensures the UI renders progress nicely. |
| Graceful Error Handling | Concise user messages + verbose logs when you need them. |
| Asynchronous Operation | Non‑blocking design keeps the chat UI responsive. |
Configuration
| Valve | Default | Purpose |
|---|---|---|
OLLAMA_ENDPOINT | http://host.docker.internal:11434 | Base URL of the Ollama REST API. |
REQUEST_TIMEOUT | 3 seconds | HTTP request time‑out. |
UNLOAD_DELAY_MS | 200 | Wait time between model unloads (set 0 for maximum speed). |
VERIFY_SSL | true | Toggle TLS certificate verification. |
LOG_LEVEL | INFO | Logging verbosity. |
- Version:
1.1.2 - Required Open‑WebUI Version:
0.6.0
0
0