Skip to content

Commit

Permalink
V4.2 (#166)
Browse files Browse the repository at this point in the history
1. AsyncDDGS performance improved: await resp.acontent(),
2. _get_vqd improved: using regex pattern to extract vqd from html,
3. Removed deprecated functions: ddg, ddg_answers, ddg_images, ddg_maps, ddg_news, ddg_suggestions, ddg_translate, ddg_videos,
4. README: updated async example,
5. Ruff: added lint rules to pyproject.toml,
6. CLI: support functions have been changed to private,
7. Docstrings updated,
8. Small code improvements.
  • Loading branch information
deedy5 authored Jan 12, 2024
1 parent 25e7059 commit cb97f81
Show file tree
Hide file tree
Showing 15 changed files with 310 additions and 398 deletions.
1 change: 1 addition & 0 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:
python -m pip install .[dev]
- name: Ruff
run: |
ruff check .
ruff format . --check --target-version py38
- name: Pytest
run: |
Expand Down
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -157,4 +157,6 @@ cython_debug/
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
#.idea/

.vscode/
7 changes: 0 additions & 7 deletions .vscode/settings.json

This file was deleted.

30 changes: 25 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,23 +183,43 @@ with DDGS() as ddgs:
Here is an example of initializing the AsyncDDGS class:
```python3
import asyncio
import logging
import sys
from itertools import chain
from random import shuffle

import requests
from duckduckgo_search import AsyncDDGS

# bypass curl-cffi NotImplementedError in windows https://curl-cffi.readthedocs.io/en/latest/faq/
if sys.platform.lower().startswith("win"):
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

async def get_results():
async with AsyncDDGS() as ddgs:
results = [r async for r in ddgs.text("cat", max_results=5)]
def get_words():
word_site = "https://www.mit.edu/~ecprice/wordlist.10000"
resp = requests.get(word_site)
words = resp.text.splitlines()
return words

async def aget_results(word):
async with AsyncDDGS(proxies=proxies) as ddgs:
results = [r async for r in ddgs.text(word, max_results=None)]
return results

async def main():
ddgs_results = await get_results()
print(ddgs_results)
words = get_words()
shuffle(words)
tasks = []
for word in words[:10]:
tasks.append(aget_results(word))
results = await asyncio.gather(*tasks)
print(f"Done")
for r in chain.from_iterable(results):
print(r)


if __name__ == "__main__":
logging.basicConfig(level=logging.DEBUG)
asyncio.run(main())
```
It is important to note that the DDGS and AsyncDDGS classes should always be used as a context manager (with statement).
Expand Down
21 changes: 5 additions & 16 deletions duckduckgo_search/__init__.py
Original file line number Diff line number Diff line change
@@ -1,29 +1,18 @@
"""Duckduckgo_search
~~~~~~~~~~~~~~
Search for words, documents, images, videos, news, maps and text translation
"""Duckduckgo_search.
Search for words, documents, images, videos, news, maps and text translation
using the DuckDuckGo.com search engine.
"""

import logging

# ruff: noqa: F401
# isort: off
from .compat import (
ddg,
ddg_answers,
ddg_images,
ddg_maps,
ddg_news,
ddg_suggestions,
ddg_translate,
ddg_videos,
)

# isort: on
from .duckduckgo_search import DDGS
from .duckduckgo_search_async import AsyncDDGS
from .version import __version__

__all__ = ["DDGS", "AsyncDDGS", "__version__", "cli"]

# A do-nothing logging handler
# https://docs.python.org/3.3/howto/logging.html#configuring-logging-for-a-library
logging.getLogger("duckduckgo_search").addHandler(logging.NullHandler())
2 changes: 1 addition & 1 deletion duckduckgo_search/__main__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""for using as 'python3 -m duckduckgo_search'"""
"""For using as 'python3 -m duckduckgo_search'."""
from .cli import cli

if __name__ == "__main__":
Expand Down
Loading

0 comments on commit cb97f81

Please sign in to comment.