Latent embeddings

[shekshaa]

Hello, thanks for sharing the code of this great benchmark!

I have a question about latent embeddings. For neighborhood similarity and information imbalance metrics, the latents obtained by different methods are needed to have a consistent comparison. I think one set of such embeddings are provided for IgG-1D dataset as in the file conf-het-1_wrangled_latents.npz. I can't find corresponding files for other datasets (e.g. to run compute_information_imbalance.py one needs such an embedding file as input). Are you planning to release them as well?

[michal-g]

Hi, in general we expect you to generate such latent embeddings on your own with the methods you are evaluating using CryoBench; the embeddings at conf-het-1_wrangled_latents.npz are meant as a useful example for comparison and testing. We may at some point generate toy examples of such embeddings to serve a similar purpose for the other metrics as well, but this is tricky as embeddings files tend to be rather bulky to be placed in a git repo.

We have also provided updated documentation on how to run each of our example methods to generate results; these docs are still a work-in-progress so please feel free to let us know if you have any questions or suggestions!