Skip to content

Latest commit

 

History

History
14 lines (10 loc) · 429 Bytes

README.md

File metadata and controls

14 lines (10 loc) · 429 Bytes

ImgScraper

Simple image scraper for a simple site.

TODO

  • Use one function for the multiprocess
  • Removed the use of multiple files splitted in bash
  • Gets number of total processes and uses the 80% of the total processes for the downloader
  • Duplicate name control
  • Img Sub folder tree with sub-tree for bodyparts
  • Add support for different url's
  • Hash image comparison for finding dupes