Openlibrary.org zim file is incomplete

[RavanJAltaie]

The openlibrary.org zim file in dev library is not complete.
It doesn't open anything beyond the main page.
The file link: https://dev.library.kiwix.org/viewer#openlibrary.org_2023-08/A/openlibrary.org/
The recipe is: https://farm.openzim.org/recipes/openlibrary.org

#openzim/zim-requests#326

[rgaudin]

I see two issues:

• Source website fetches images from covers.openlibrary.org, so that must be included as well.
• Source website apparently uses absolute path to resources <img alt="Internet Archive logo" src="/static/images/ia-logo.svg" width="160">. You can open a ticket at browsertrix-crawler about this.

[kelson42]

@rgaudin Few remarks:

• I think we should strongly consider to start a FAQ for each of our scraper (on the wiki of the repo for example), so each time we get a question with a non-obvious answer, we can document it. What do you think? Maybe that way we can reduce the amount of low-quality exchanges betwee content-managers and developers?
• I think this is not the most efficient approach to request @RavanJAltaie to open ticket on upstream repository. We see how challenging it can be already with opening tickets for our repos. To me this is the role of the developers.

[rgaudin]

I agree about us writing tickets for upstream. As for the rest, it deserves a collective discussion which is actually awaiting your come back. Glad to see there's momentum

[kelson42]

See openzim/cms#97 as well

[rgaudin]

Taking a second look at the issue, it's actually a recipe problem:

• ZIM file is 1.4MB so that's a hint that not much have been captured
• The problem in the recipe is that it uses custom as scopeType. In this mode, you need to specify includes and excludes completely. The only auto include in this mode is the passed URL.