Skip to content

Commit

Permalink
feat(tools): searXNG web search tool (#286)
Browse files Browse the repository at this point in the history
  • Loading branch information
michael-desmond authored Jan 22, 2025
1 parent 72bfab6 commit 99334ff
Show file tree
Hide file tree
Showing 4 changed files with 194 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .env.template
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ BEE_FRAMEWORK_LOG_SINGLE_LINE="false"
# GOOGLE_API_KEY="your-google-api-key"
# GOOGLE_CSE_ID="your-custom-search-engine-id"

# For SearXNG Search Tool
# SEARXNG_BASE_URL="http://127.0.0.1:8888"

# For Elasticsearch Tool
# ELASTICSEARCH_NODE=""
# ELASTICSEARCH_API_KEY=""
Expand Down
69 changes: 69 additions & 0 deletions docs/searxng-tool.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# 🔍 SearXNGTool

## Description

This tool allows an agent to run web searches using SearXNG. SearXNG is a metasearch engine that aggregates results from multiple search engines. SearXNG does not require an API key, you can run SearXNG directly on your laptop to faciliate easy access to web search functionality for your agent.

## Setup

Follow the steps to create a private SearXNG instance. For more advanced usage see the [SearXNG project documentation](https://github.com/searxng/searxng).

### 1. Create a local folder for the SearXNG configuration files.

The files will be automatically written to this location, but you will need to make a minor modification.

```
mkdir ~/searxng
```

### 2. Run the SearXNG docker container.

```
docker run -d --name searxng -p 8888:8080 -v ./searxng:/etc/searxng --restart always searxng/searxng:latest
```

### 3. Edit the configuration files and restart the container.

When you first run a SearXNG docker container, it will write configuration files to the `~/searxng` folder.

```
settings.yml
uwsgi.ini
```

Open `~/searxng/settings.yml`, find the `formats:` list and add `json`.

```yaml
search:
formats:
- html
- json
```
Stop and restart the container.
### 4. Check installation
Navigate to `http://localhost:8888` and you should see an SearXNG interface.

## Usage

Configure the SEARXNG_BASE_URL environment variable:

```
SEARXNG_BASE_URL="http://127.0.0.1:8888"
```

Run the tool:

```js
import "dotenv/config";
import { SearXNGTool } from "bee-agent-framework/tools/search/searXNGSearch";
const tool = new SearXNGTool();
const result = await tool.run({
query: "Worlds longest lived vertebrate",
});
console.log(result.getTextContent());
```
1 change: 1 addition & 0 deletions docs/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ These tools extend the agent's abilities, allowing it to interact with external
| `WikipediaTool` | Search for data on Wikipedia. |
| `GoogleSearchTool` | Search for data on Google using Custom Search Engine. |
| `DuckDuckGoTool` | Search for data on DuckDuckGo. |
| [`SearXNGTool`](./searxng-tool.md) | Privacy-respecting, hackable metasearch engine. |
| [`SQLTool`](./sql-tool.md) | Execute SQL queries against relational databases. |
| `ElasticSearchTool` | Perform search or aggregation queries against an ElasticSearch database. |
| `CustomTool` | Run your own Python function in the remote environment. |
Expand Down
121 changes: 121 additions & 0 deletions src/tools/search/searXNGSearch.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
/**
* Copyright 2025 IBM Corp.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import { ToolEmitter, Tool, ToolInput } from "@/tools/base.js";
import { z } from "zod";
import { Emitter } from "@/emitter/emitter.js";
import { createURLParams } from "@/internals/fetcher.js";
import { RunContext } from "@/context.js";
import {
SearchToolOptions,
SearchToolOutput,
SearchToolResult,
SearchToolRunOptions,
} from "@/tools/search/base.js";
import { ValidationError } from "ajv";
import { parseEnv } from "@/internals/env.js";

export interface SearXNGToolOptions extends SearchToolOptions {
baseUrl?: string;
maxResults: number;
}

type SearXNGToolRunOptions = SearchToolRunOptions;

export interface SearXNGToolResult extends SearchToolResult {}

export class SearXNGToolOutput extends SearchToolOutput<SearXNGToolResult> {
static {
this.register();
}

createSnapshot() {
return {
results: this.results,
};
}

loadSnapshot(snapshot: ReturnType<typeof this.createSnapshot>) {
Object.assign(this, snapshot);
}
}

export class SearXNGTool extends Tool<
SearXNGToolOutput,
SearXNGToolOptions,
SearXNGToolRunOptions
> {
name = "Web Search";
description = `Search for online trends, news, current events, real-time information, or research topics.`;

public readonly emitter: ToolEmitter<ToolInput<this>, SearchToolOutput<SearXNGToolResult>> =
Emitter.root.child({
namespace: ["tool", "search", "searXNG"],
creator: this,
});

inputSchema() {
return z.object({
query: z.string().min(1).describe(`Web search query.`),
});
}

public constructor(options: SearXNGToolOptions = { maxResults: 10 }) {
super(options);

if (options.maxResults < 1 || options.maxResults > 100) {
throw new ValidationError([
{
message: "Property 'maxResults' must be between 1 and 100",
propertyName: "options.maxResults",
},
]);
}
}

protected async _run(
input: ToolInput<this>,
_options: Partial<SearchToolRunOptions>,
run: RunContext<this>,
) {
const params = createURLParams({
q: input.query,
format: "json",
});

const baseUrl = this.options.baseUrl || parseEnv("SEARXNG_BASE_URL", z.string());
const url = `${baseUrl}?${decodeURIComponent(params.toString())}`;
const response = await fetch(url, {
signal: run.signal,
});

if (!response.ok) {
throw new Error(await response.text());
}

const data = await response.json();

return new SearXNGToolOutput(
data.results
.slice(0, this.options.maxResults)
.map((result: { url: string; title: string; content: string }) => ({
url: result.url || "",
title: result.title || "",
description: result.content || "",
})),
);
}
}

0 comments on commit 99334ff

Please sign in to comment.