Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

关于pretrainedmodel问题 #6

Open
xce1123 opened this issue Mar 18, 2022 · 1 comment
Open

关于pretrainedmodel问题 #6

xce1123 opened this issue Mar 18, 2022 · 1 comment

Comments

@xce1123
Copy link

xce1123 commented Mar 18, 2022

您好,我在您代码的基础上在bertSequenceLabeling的基础上init初始化的时候添加了层,发现即使forward中不使用效果也会变差,请问这个load原始模型的过程是哪里需要修改吗?添加新的层是会影响其它层load的参数吗?不是太理解,期待回复,谢谢

@ArrowLuo
Copy link
Owner

Hi @xce1123, sorry for the delayed reply. I am not sure whether your weights are initialized. Maybe the log is useful if you can upload it. I do not think the new layers will affect other layers but you should be careful with the learning rate, bigger one for new layers and a smaller one for loaded weights. Best.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants