How to load the finetuned model?

[code-iter]

Hey mate. I've been having problems trying to load the finetuned model.safetensors. I've finetuned the model using the provided notebook script, and in the end after saving the model i got the following files:

• config.json;
• generation_config.json;
• model.safetensors.

Now, i've tried to load the model using transformers but i keep getting errors. the last one was:

AttributeError: module 'transformers_modules.vikhyatk.moondream2.fb2293ab7450beb1dae536b069f5966becf58e5c.moondream' has no attribute 'Moondream'

i've tried many scripts to try and load the finetuned model.safetensors, like:

model = AutoModelForCausalLM.from_pretrained(
    pretrained_model_name_or_path=model_root_dir,
    use_safetensors=True,
    state_dict=load_file(model_path),
    config=model_config,  # Provide a configuration file for the model
    trust_remote_code=True,
)

But nothing seems to work. Could you help me on this?

[forensicmike]

Hi, I had problems with this too, specifically it starting giving me pyvips errors again (I installed the pip package but there's a missing DLL, I presume this is why they are telling you to use revision = "2024-08-26" with transformers.)

I got around it by saving the text encoder only (an instance of PhiForCasualLM)

So after fine tuning, you can do:
model.text_model.save_pretrained('./models/phifortext-ft')
or even
torch.save(model.text_model.state_dict(), './models/textmodel-finetune.pt')

I was then able to load the model normally and call
model.text_model.load_state_dict(torch.load('g:/ai/_vdataset/models/phitext.pt', weights_only=False))