logo
Sign In

Brained Search Xng OR

tool

Smart SearXNG Search, OpenRouter version

cernunnos
@cernunnos
·
a year ago
41
0
1.6K
Updated a year ago
Created Nov 9, 2025
brained_search_xng_or.py
Python · 756 lines
"""
title: BDS Smart Search Xng with BeautifulSoup (don't run too many BeautifulSoup scrapers at same time, an other version of this tool use Jina scraper)
author: Lug, scraper part from Web search v0.2.1 by siwadon
author_url: https://brainedai.netlify.app
description: An advanced search tool that uses 2 LLMs to perform URL selection, and content extraction. Returns only the most relevant information.
required_open_webui_version: 0.6.0
requirements: aiohttp, asyncio, requests, beautifulsoup4
version: 0.9.2
license: MIT
"""

import json

https://brainedai.netlify.app/en/tools/web-search/brained-search/

Version 0.9.2, now it can scrape arxiv paper (if a HTML version exist, not the PDF), and other improvements.

A tool specially designed for BDS1 agent, but work well for any deepsearch agent.

This tool works in four steps:
Step 1: It searches the internet using a few keywords (query)
Step 2: LLM 1 selects the URLs to scrape based on a query related to the cycle (query_2)
Step 3: It scrapes the URLs selected by LLM 1
Step 4: LLM 2 selects interesting information from scraped pages based on a query related to the cycle (query_2)

Important:
You need to have a local SearXNG server.
The minimum parameters to enter are the API keys for OpenRouter.
If you put a LangSearch API key (free) you will have a solid fallback.

If you don’t want to install a local SearXNG server, use Brained Search DDGS (soon) or Brained Search LangSearch (soon).

This one uses Beautifulsoup scraper (CPU). So if you put 1 main (cloud) and two “secondary” (this version that use CPU), you don’t use too many resources.

The function of this tool is: xng_smart_search

A special version for OpenRouter that allows you to configure fallback models and prioritize providers using a list (provider IDs are separated by commas without spaces) or by enabling “Llm 2 Enable Throughput Sort.” If you enable this option, you disable the list of providers to prioritize and enable prioritization based on throughput, and you also enable the ability to disable providers that are too expensive (fast but expensive).
https://openrouter.ai/docs/features/provider-routing

0