Adding Functionality to Distributed Communication

[scotty110]

Is there any want/need to flesh out the Distributed Communication package? I was thinking of possible parity with PyTorch (https://pytorch.org/docs/stable/distributed.html).

I am happy to contribute to MLX, just would like some feedback if this is something that would be beneficial to the project before I go ahead.

[angeloskath]

Hi scotty! Some of these we definitely want to add but we are taking it a bit slowly as they are not very widely used and they would increase the footprint of what we need to maintain for now.

Is there a specific operation you want to use or had in mind?