Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tavily Doesn't Handle Multiple Queries #223

Open
mvanniasingheTT opened this issue Mar 4, 2025 · 0 comments
Open

Tavily Doesn't Handle Multiple Queries #223

mvanniasingheTT opened this issue Mar 4, 2025 · 0 comments

Comments

@mvanniasingheTT
Copy link
Contributor

Describe the bug
This bug appears to be an issue with the Tavily search engine used by the search agent.

To Reproduce
Steps to reproduce the behavior:

  1. Deploy the search agent
  2. Ask any prompt that involves two searches

Image

Notice how the search doesn't work unless the questions are asked individually.

What team did Carlos Sainz join for the 2025 F1 season? What team did Marc Marquez join for the 2025 MotoGP season?
--
Starting tool: _Exception with inputs: {}
Done tool: _Exception
Tool output was: Invalid or incomplete response
--
INFO:     172.18.0.3:41498 - "POST /poll_requests HTTP/1.1" 200 OK
/usr/local/lib/python3.11/dist-packages/langchain/memory/chat_memory.py:55: UserWarning: 'ConversationBufferMemory' got multiple output keys: dict_keys(['output', 'intermediate_steps', 'messages']). The default 'output' key is being used. If this is not desired, please manually set 'output_key'.
  warnings.warn(
What team did Carlos Sainz join for the 2025 F1 season?
--
Starting tool: tavily_search_results_json with inputs: {}
Done tool: tavily_search_results_json
Tool output was: [{'url': 'https://www.williamsf1.com/posts/22c46812-ee00-494b-b96d-80f19303ec95/williams-racing-welcomes-carlos-sainz?srsltid=AfmBOoq0of8JvENNwupoEu50QuLQq7P88nPu8B21oR5eR1tHBU-XYVET', 'content': "Williams Racing today announces that Carlos Sainz will complete the team's driver line-up for the 2025 FIA Formula 1 World Championship season and beyond."}, {'url': 'https://www.youtube.com/watch?v=KsKf_Mscuvg', 'content': 'Carlos Sainz set the fastest lap of the day on day two of pre ... Ranking the 2025 F1 teams after pre-season testing. THE RACE New 379K'}]
--
INFO:     172.18.0.3:37866 - "POST /poll_requests HTTP/1.1" 200 OK

I further tested the search engine on its own without the LLM and it appears it can only do 1 search query at a time.

>>> tavily_client.search("What team did carlos sainz join in f1 2021 and what team did marc marquez join in 2025")
{'query': 'What team did carlos sainz join in f1 2021 and what team did marc marquez join in 2025', 'follow_up_questions': None, 'answer': None, 'images': [], 'results': [{'title': 'Carlos Sainz Races, Wins and Teams | F1 Driver | F1 History', 'url': 'https://www.formulaonehistory.com/drivers/carlos-sainz/', 'content': 'Carlos Sainz is a Spanish Formula One driver who has raced for Ferrari, Red Bull, Toro Rosso, Williams, and McLaren Formula One teams. ... which had caught the eye of the Ferrari team. In 2021, Sainz made his highly anticipated move to Ferrari, ... 2025 F1 Pre-Season Testing - Bahrain: Full Report & Highlights', 'score': 0.82186085, 'raw_content': None}, {'title': 'Williams Racing welcomes Carlos Sainz for 2025, 2026 and beyond', 'url': 'https://www.williamsf1.com/posts/22c46812-ee00-494b-b96d-80f19303ec95/williams-racing-welcomes-carlos-sainz', 'content': "Williams Racing today announces that Carlos Sainz will complete the team's driver line-up for the 2025 FIA Formula 1 World Championship season and beyond. With a decade of F1 competition under his belt, Sainz will join Williams from Scuderia Ferrari on a two-year agreement with options to extend.", 'score': 0.8106142, 'raw_content': None}, {'title': 'Carlos Sainz to remain in F1, undecided on 2025 team - ESPN', 'url': 'https://www.espn.com/f1/story/_/id/40639472/carlos-sainz-remain-f1-undecided-2025-team', 'content': 'Sainz joining Swiss-based Sauber, who will become the factory Audi team in 2026, had looked increasingly unlikely given their form this season -- the only team yet to score a point -- and the', 'score': 0.7982658, 'raw_content': None}, {'title': 'Carlos Sainz to Ferrari for 2021: 5 reasons to be excited about the new ...', 'url': 'https://www.formula1.com/en/latest/article/5-reasons-to-be-excited-about-the-new-leclerc-sainz-partnership-at-ferrari.5hLqMNajnQMtK2pTBFhgOd', 'content': "F1 STORE: High speed and high style - F1's 2025 team kits have arrived Discover more news Following the sensational confirmation that Carlos Sainz would decamp from McLaren to become Ferrari's newest driver when he replaces Sebastian Vettel for 2021, we look at five reasons to get excited about the Spaniard going up against Charles Leclerc.", 'score': 0.7931224, 'raw_content': None}, {'title': "Carlos Sainz has joined Ferrari's F1 team for 2021… - Top Gear", 'url': 'https://www.topgear.com/car-news/formula-one/carlos-sainz-has-joined-ferraris-f1-team-2021', 'content': "Sainz - a supremely talented racer with 102 GPs under his belt, and who last year finished 6th in the Drivers' Championship - will join Charles Leclerc for the 2021 F1 season, and", 'score': 0.7476348, 'raw_content': None}], 'response_time': 2.52}

It fails to handle multiple orthogonal search queries. I think once adding the LLM reasoning on top of this, since the search can't find the needed results the LLM responds the way it does.

Expected behavior
Ideally, we want to use a search engine for the agent that can handle multiple queries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant