Ollama Model Unload

Creates a button to instantly unload Ollama model(s) that are running in vRAM.

@pkeffect

🇺🇸

v0.0.5

2 years ago

@pkeffect

🇺🇸

v0.0.5

2 years ago

ollama_model_unload.py

Python · 364 lines

197    class Valves(BaseModel):198        OLLAMA_HOSTS: List[str] = Field(199            default=DEFAULT_OLLAMA_HOSTS,200            description="List of Ollama host IPs or **hostnames** to connect to.  Supports both local addresses (localhost, 127.0.0.1, ollama) and remote addresses (e.g., 192.168.1.100, my-ollama-server.com).",  # Updated description for remote hosts201        )202        OLLAMA_PORT: int = Field(203            default=DEFAULT_OLLAMA_PORT,204            description="Port number for Ollama API (default: 11434)",205        )206        WAIT_BETWEEN_UNLOADS: int = Field(207            default=0, description="Seconds to wait between model unloads (default: 0)"208        )

🚀Ollama Model Unloader

A powerful Open WebUI extension that helps you free up system resources by unloading Ollama models when they're not in use.

📋Overview

Ollama Model Unloader is a simple but effective tool that allows you to unload AI models from memory with just a click. This is particularly useful when:

You need to free up RAM or VRAM on your system
You want to switch between different models without restarting Ollama
You're experiencing system slowdowns due to multiple loaded models

✨Features

🖥️ Works with both local and remote Ollama hosts
🔍 Automatically searches multiple locations for Ollama servers
🔌 Supports custom port configurations
📊 Displays real-time progress and results
🔄 Automatically collapses output panel when finished (configurable)
🛡️ Robust error handling for a smooth experience

⚙️Configuration Options

You can customize the behavior of the Ollama Model Unloader through the following settings:

Setting	Description	Default Value
`OLLAMA_HOSTS`	List of Ollama host IPs or hostnames to connect to	`["localhost", "127.0.0.1", "ollama", "host.docker.internal"]`
`OLLAMA_PORT`	Port number for Ollama API	`11434`
`WAIT_BETWEEN_UNLOADS`	Seconds to wait between model unloads	`0`
`AUTO_CLOSE_OUTPUT`	Whether to automatically close/collapse output when finished	`True`
`AUTO_CLOSE_DELAY`	Seconds to wait before automatically closing output	`3`

⚠️Important Notes

Unloading models might not instantly release system RAM if models have spilled over from VRAM. Monitor your system resources.
High CPU usage might persist temporarily after unloading if system RAM is heavily utilized.
Avoid unloading models while they are actively generating text to prevent potential issues.
If issues persist after unloading, consider restarting your Ollama service manually.
For remote Ollama instances, ensure the Ollama API is accessible over your network and that any necessary firewalls are configured to allow connections.

📦Installation

In Open WebUI, navigate to the Extensions panel
Find "Ollama Model Unloader" in the community extensions
Click "Install"
The tool will now appear in your toolbar for easy access

📝Changelog

0.0.5 - Added more error handling and support for remote Ollama servers
0.0.4 - Refactor
0.0.3 - Added error reporting and better checks
0.0.2 - Added status and progress
0.0.1 - Initial upload to openwebui community

🔧How It Works

When activated, the tool:

Searches for Ollama servers on the configured hosts and port
Gets a list of all currently running models
Sends an API request to unload each model by setting keep_alive to 0
Verifies that models are properly unloaded
Provides status updates throughout the process

💻Technical Details

The unloader works by making HTTP requests to the Ollama API. Specifically, it:

Uses GET /api/ps to find running models
Uses POST /api/generate with keep_alive: 0 to unload models
Includes comprehensive error handling for network issues, API errors, and unexpected scenarios

👨‍💻Author

Developed by pkeffect

🔗Links

📄License

This project is licensed under the MIT License

Required Open WebUI Version: 0.6.0 or higher