v0.2.1

🚀 Streaming v0.2.1

Streaming v0.2.1 is released! Install via pip:

pip install --upgrade mosaicml-streaming==0.2.1

Bug Fixes

• Make StreamingDataset smarter about when to init dist itself, fixing env var rendezvous problem (#94).
• Shorten shared memory names for Mac OSX (#95).
• Reduce memory usage in StreamingDataset, alleviating inscrutable worker OOMs with large datasets (#96).
• Better exception handling in downloading (#98).
• Hard require fork for dataloader multiprocessing in Mac OSX due to unpickleable objects (#101).

What's Changed

• Also check if dist env vars are set. If not set, don't init dist. by @knighton in #94
• Shorten the names of shared memory objects to make OSX happy. by @knighton in #95
• Just do the partitioning/shuffling in the local leader worker. by @knighton in #96
• propagate the actual exception and raise by @karan6181 in #98
• Set multiprocessing method as fork for Mac OS by @karan6181 in #101
• Bump version to 0.2.1 by @karan6181 in #102

Full Changelog: v0.2.0...v0.2.1