Zimit1 - (Some?) Youtube videos fails to play on kiwix-serve

[benoit74]

Maybe related to #144, we now have a situation where a Youtube video also fails to play on desktop.

Page with the video: https://dev.library.kiwix.org/content/welder.app_en_2024-02/A/mp_/https://www.welder.app/ni.fi/now.i.see.you.-.nisy/master/tree

Recipe: https://farm.openzim.org/recipes/welder.app_en

Last task for the moment: https://farm.openzim.org/pipeline/d767b02b-2388-4763-88a1-c2f7f64fabf6/debug

I'm doing a bit of investigation to better understand the "blast radius".

[benoit74]

Edit: I only tested kiwix-serve nightly for now (dev.library.kiwix.org)

[benoit74]

I think that the problem is that the video is not present in the ZIM but is present inside the WARC (video plays well on replayweb.page).

I confirm the ultimate issue (video not playing) is present with zimit 1.5.1, 1.6.0 and 1.6.3.

I tested the WARC only with zimit 1.6.0 and 1.6.3.

Commands used to build the ZIM:

docker run -d --rm -v $PWD:/output ghcr.io/openzim/zimit:1.6.3 zimit --url https://www.welder.app/ni.fi/now.i.see.you.-.nisy/master/tree --limit 1 --name "careables_one_page_1.6.3"

Command used to keep WARC.gz:

docker run -it --rm -v $PWD:/output ghcr.io/openzim/zimit:1.6.3 crawl --url https://www.welder.app/ni.fi/now.i.see.you.-.nisy/master/tree --limit 1 --keep --cwd /output/careables_1.6.3 --combineWARC

I confirm another website is working well (mesquartierschinois, our favorite ^^).

I've uploaded useful files to https://tmp.kiwix.org/ci/issues-files/warc2zim-191/

[benoit74]

This is probably not worth investigating in zimit1 since zimit2 is already on the roadmap, I would suggest to add the underlying recipe (Careables) as a test case for zimit2.

The online downside of this will be that we will have to wait for zimit2 to publish the corresponding ZIM, if zimit1 issue is quick to fix it is not the best approach (but I do not expect zimit1 issue to be easy to fix).

WDYT?

[Jaifroid]

@benoit74 If the issue is in the warc2zim code, it is worth finding out the underlying cause so that both codebases can be fixed. In my opinion, we should try to keep warc2zim in step with warc2zim2 if the same issue is affecting both, at least until we are sure we can reliably replicate all the functions of zimit1 in zimit2.

I'll try to test on KJS, but I'm running out of disk space and need to clear some.....!

[Jaifroid]

I confirm the video also fails to play on the PWA.

There are five videos in the ZIM (actually, 10 MP4s, because each video appears to be stored twice). However, extracting them directly and playing them, I can't find the video that is supposed to play in the embedded page for that article.

This confirms that the video isn't in the ZIM in this instance, so it's not a mirror version of #144. In the latter case, the video is confirmed to be in the ZIM, but the browser appears to be calling for a subtly different URL when run in a browser on Android, causing the fuzzy substitutions to fail. (That is a conjecture.)

[benoit74]

This will be fixed by openzim/zimit#291 in zimit2 / warc2zim2