Multithreaded webcrawler written in C++. It requires url of page to start crawling, regular expression of strings to search for, depth of recursion in which to look for strings defined by regular expression and number of threads to use.
libcurl
C++ in version at least 14
cd MultithreadedWebcrawler
mkdir build
cd build
cmake ..
make
url_regex=(http|ftp|https)://([\w_-]+(?:(?:\.[\w_-]+)+))([\w.,@?^=%&:/~+#-]*[\w@?^=%&/~+#-])?
regex=[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+
start_url=https://panoramafirm.pl/szukaj?k=lekarz&l=warszawa
max_threads=4
max_depth=2
./MultithreadedWebcrawler [path_to_config_file]
[path_to_config_file] - relative path to config file
./MultithreadedWebcrawler ../config.cfg
jziolkow@wp.pl
malgosia.pietowska@wp.pl
kontak@koronyporcelanowe.pl
kami.lecznica@poczta.internetdsl.pl
radoslaw1983@gmail.com
kalinalap@tlen.pl
cmp@hipokrates.tychy.pl
recepcja@przychodniaeumedica.pl
sanus.kalisz@gmail.com
katarzyna.pietrzak57@wp.pl
jola_piotrowicz@autograf.pl
rojecka@wp.pl
medjana@medjana.pl
kasiaflak@o2.pl
gabinet@drwaclawik.pl
kancelaria@achler.pl
odl@fundacja.walbrzych.pl
achtamalka@o2.pl
krysia@cmeureka.pl
michalak.k12@gmail.com
angrzel@gazeta.pl
Bartosz Pieszko
bartosz.pieszko@gmail.com