Handle updated Kolibri channels #137
Conversation
Copy the Kolibri server content seeding code common to vanilla Kolibri and Endless Key to a separate module so that any changes can be shared between the hooks. https://phabricator.endlessm.com/T34697
A simple wrapper around `eibkolibri.seed_remote_channels`. https://phabricator.endlessm.com/T34697
This Kolibri option no longer exists in 0.16, which Endless Key is using. https://phabricator.endlessm.com/T34697
The FIXME has been addressed by not caching the Kolibri homedir on the image builder and instead using the content server as the cache for Kolibri content. There's no need to use a temporary homedir and copy it into place.
This simplifies both configurations as the content seeding and installation can be done in a single shell hook. https://phabricator.endlessm.com/T34697
When a channel is updated is updated on Kolibri Studio, importing the channel and content again does not fetch new content nodes. Instead, a channel update process is needed, which also requires generating the diff between the old and new versions of the channel. Fortunately, there are APIs for those tasks, so run them if the channel already exists on the server to ensure the it has seeded all content nodes from the (potentially) updated channel. https://phabricator.endlessm.com/T34697
lib/eibkolibri.py
Outdated
| host = urlparse(base_url).netloc | ||
| creds = netrc_creds.authenticators(host) | ||
| if not creds: | ||
| logger.info(f'No credentials for {host}') |
There was a problem hiding this comment.
logger.info(f'No credentials for {host} in {netrc_path}')perhaps?
| """Wait for remote Kolibri job to complete""" | ||
| logger.debug(f'Waiting for job {job_id} to complete') | ||
| last_marker = None | ||
| while True: |
There was a problem hiding this comment.
Do we want to rate-limit this to check every 1 second rather than spamming the server as fast as we can go?
There was a problem hiding this comment.
That's fair. I might split the difference to 0.5 seconds so there's less artificial slowdown.
lib/eibkolibri.py
Outdated
| f'Job {job_id} failed: {data["exception"]}\n{data["traceback"]}' | ||
| ) | ||
| raise Exception(f'Job {job_id} failed') | ||
| elif status == 'CANCELED': |
There was a problem hiding this comment.
I went to check the spelling here and it really is CANCELED for tasks, though for Morango sync it's CANCELLED.
I wasn't aware of the US spelling for this word (though I guess it's consistent with other Commonwealth vs US variations) but I'm really happy to see that both spellings are used in Kolibri!
There was a problem hiding this comment.
I hadn't even thought about that! I just copied the strings directly out of the module.
If a Kolibri channel has been updated on Studio, the content server needs to run a channel update procedure or it won't import new content nodes.
While here, I couldn't resist the temptation to refactor all of this so that all the dispersed hooks can go away. I ran the Endless Key process locally and it seemed to work.
https://phabricator.endlessm.com/T34697