[BUG] Clearing the metadata dir will delete required .json files.

[fraimondo]

Currently, the backend cleans the data dir when the jobs finish, but does not clear the metadata. This is on purpose, to keep track of completed tasks/batches.

However, after several runs, one can end with several "trees". The idea of the clear command is to clear this "finished" trees, in which the level 0 json files do not have a matching data dir, and the non-level 0 json files that do not have their correspoding parent json.