Skip to content

Commit 0731b2a

Browse files
committed
feat(browser/captcha): support hints in captcha detection
1 parent f0321b6 commit 0731b2a

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed

npiai/core/tool/_browser.py

+7-2
Original file line numberDiff line numberDiff line change
@@ -304,7 +304,9 @@ async def back_to_top(self):
304304

305305
return f"Successfully scrolled to top"
306306

307-
async def detect_captcha(self, ctx: Context, return_to: str | None = None):
307+
async def detect_captcha(
308+
self, ctx: Context, return_to: str | None = None, hints: str | None = None
309+
):
308310
url = await self.get_page_url()
309311
screenshot = await self.get_screenshot(full_page=True, max_size=(1280, 720))
310312

@@ -350,13 +352,16 @@ async def handle_captcha(captcha_type: Literal["none", "captcha", "login"]):
350352
ChatCompletionSystemMessageParam(
351353
role="system",
352354
content=dedent(
353-
"""
355+
f"""
354356
You are given a screenshot of a webpage. Determine if a captcha or login form with input fields is present in the screenshot. If a captcha is present, call the tool with the argument "captcha". If a login form is present, call the tool with the argument "login". If neither is present, call the tool with the argument "none".
355357
356358
NOTE:
357359
- Popups like cookie consent banners should not be considered as login forms.
358360
- The login action can be ignored if the main content is visible in the screenshot.
359361
- Login button that triggers a login form should not be considered as login forms. Only visible login forms with input fields should be considered.
362+
363+
HINTS:
364+
{hints}
360365
"""
361366
),
362367
),

0 commit comments

Comments
 (0)