logo
Sign In

LLM Web Search

tool

Enables LLMs to search the web. Now with proxy support! Github: https://github.com/mamei16/LLM_Web_search_OWUI

mamei16
@mamei16
·
6 months ago
15K
34
36K
Updated 6 months ago
Created Nov 12, 2024
llm_web_search.py
Python · 1886 lines
"""
LLM Web Search
version: 0.8.4

Copyright (C) 2024 mamei16

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,

NEW in v0.8.1: Add settings to customize search backends and region . Tool no longer supports webui versions < v0.6.41
NEW in v0.7.0: Allow model to restrict search to specific website/domain
NEW in v0.6.0: Use DuckDuckGo as default search backend
NEW in v0.5.0: Added new neural chunking method

This tool is LLM_Web_search ported to Open WebUI, retaining as much functionality as possible given the different environments. This means that the documentation on search types, keyword retrievers and chunking methods from the original project still apply to this tool.

Table of Contents

Initial Setup

Configure Embedding Model Save Location

This tool processes search results locally on device. For this purpose, three embedding models need to be downloaded automatically when the first web search is executed. Before using this tool for the first time, you'll need to configure the "Embedding Model Save Path" setting (or "Valve"), which controls where the embedding models will be downloaded to.

Settings (Valves)

Most settings are already explained briefly in the UI, and are expanded on in the readme of the original project. In the following subsection(s), some extra info that would be too verbose to put into the UI will be added.

Chunking method

If you run the tool on CPU only, choose 'character-based'. Otherwise, the 'neural' chunking method should produce the best results. The 'semantic' option is a middle ground between 'character-based' and 'neural' in terms of quality and speed.

Keyword Retriever

If you run the tool on CPU only, choose 'bm25', otherwise choose 'splade'.

Ensemble Weighting

Set this value to 0 to use keyword retrieval only, or 1 to use purely dense retrieval.

Proxy Support

This tool adheres to the proxy settings of the main web UI, as described in the open webui documentation. Optionally, both requests fetching search results from Duckduckgo and requests downloading result webpages can be routed through a proxy.

Search Backends

Choose between DuckDuckGo, DDGS and Searxng.

Possible Issue with AMD ROCm on Linux

If you're running both the LLM server (such as Oobabooga's textgen webUI or Ollama) and Open WebUI on the same linux machine, have an AMD GPU and use ROCm, you may encounter freezes/hanging in either the web search itself (being stuck at either the "Downloading and chunking webpages..." or "Retrieving relevant results..." stage) or the LLM response(s) following the web search.

This is currently an open issue and has so far only been confirmed to occur when using Oobabooga's textgen webUI as the LLM server. You can avoid any issues by using the llama.ccp server directly.

3