Baidu can not crawl CloudFlare CDN

[huan]

This problem caused the Baidu.com can not index the pre-angel.com website at all. (Zero Index on 5 Nov 2019)

From: https://ziyuan.baidu.com/dashboard/index?site=http://www.pre-angel.com/

Crawled Failure

HTTP Error 040 (???)

HTTP Error 403

[Yang2001-created]

Hi Huan, i've encountered the exact issue like yours, i was wondering if you have solved crawler issue yet？Thanks！

[huan]

Hi, Yang,

Unfortunately, I did not solved this problem.

Now I'm not using the cloud flare cdn anymore by switching to a singapore server with nginx.

Please let me know if you found any solution for this issue in the future, thanks!

[Yang2001-created]

Hi Huan,

We've added user agent to allow Baidu spider and we did see on the Cloudflare firewall that its allowed attempts made by Baidu crawler.

But still didn't work... HTTP/1.1 403 Forbidden, Crawled Failure.

I ask Cloudflare customer service about this, seems like they couldn't find what is the issue either...

Thanks anyway.