Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

About batch processing for main architecture RNN #4

Open
murataryusei opened this issue Jul 5, 2023 · 2 comments
Open

About batch processing for main architecture RNN #4

murataryusei opened this issue Jul 5, 2023 · 2 comments

Comments

@murataryusei
Copy link

Hello,
Thank you for open-sourcing this repository!
If the main architecture is RNN, how should I implement batch processing?
@JJGO
Copy link
Owner

JJGO commented Jul 7, 2023

Could you elaborate about what do you mean by batch processing? By default, in hyperlight, a batch of data uses the same hypernetwork input and thus the same weights. To use multiple sets of weights within the same batch, the recommended way to do it is using gradient accumulation.

@murataryusei
Copy link
Author

Thank you for your reply! Ultimately, I have resolved my question regarding batch processing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JJGO @murataryusei