Error in loading IIL pretrained models

[ariutti]

hi,
i'm moving my first steps in using nn.ar in SuperCollider.
I'm experiencing some problem: while I'm able to correctly load all the 10 "official" models from RAVE, I'm not able to do the same for any of the 18 models from IIL.

If for example I load na "official" RAVE pretrained model this way:

NN.load(\test, "/home/nicola/.config/ACIDS/RAVE/darbouka_onnx.ts", action:_.describe);

I get this (which is expected):

-> NNModel(nil, nil)
NNModelDesc: loading /home/nicola/.config/ACIDS/RAVE/darbouka_onnx.ts
NNModelDesc: loaded /home/nicola/.config/ACIDS/RAVE/darbouka_onnx.ts
WARNING: NN: overriding cached info for '/home/nicola/.config/ACIDS/RAVE/darbouka_onnx.ts'

*** NNModel(test)
path: /home/nicola/.config/ACIDS/RAVE/darbouka_onnx.ts
minBufferSize: 128
- method forward: 1 ins, 1 outs
- method decode: 4 ins, 1 outs
- method encode: 1 ins, 4 outs

While, if I try to load an IIL model like this:

NN.load(\test, "/home/nicola/Desktop/rave-models/birds_dawnchorus_b2048_r48000_z8.ts", action:_.describe);

I get this error:

-> NNModel(nil, nil)
NNModelDesc: loading /home/nicola/Desktop/rave-models/birds_dawnchorus_b2048_r48000_z8.ts
ERROR: NNModelDesc can't load model /home/nicola/Desktop/rave-models/birds_dawnchorus_b2048_r48000_z8.ts
ERROR: NNModelInfo: can't load info file '/tmp/nn-sc-1154.yaml'

PROTECTED CALL STACK:
	a FunctionDef	0x55d4bbd8c580
		sourceCode = "<an open Function>"
	Meta_NNModelInfo:fromFile	0x55d4bbd8c1c0
		arg this = NNModelInfo
		arg infoFile = /tmp/nn-sc-1154.yaml
	NNModel:initFromFile	0x55d4bc181200
		arg this = NNModel(nil, nil)
		arg infoFile = /tmp/nn-sc-1154.yaml
		var info = nil
	a FunctionDef	0x55d4bc180080
		sourceCode = "<an open Function>"
	Function:prTry	0x55d4bc43f0c0
		arg this = a Function
		var result = nil
		var thread = a Routine
		var next = nil
		var wasInProtectedFunc = false
	
CALL STACK:
	Exception:reportError
		arg this = <instance of Error>
	Nil:handleError
		arg this = nil
		arg error = <instance of Error>
	Thread:handleError
		arg this = <instance of Thread>
		arg error = <instance of Error>
	Thread:handleError
		arg this = <instance of Routine>
		arg error = <instance of Error>
	Object:throw
		arg this = <instance of Error>
	Function:protect
		arg this = <instance of Function>
		arg handler = <instance of Function>
		var result = <instance of Error>
	Routine:prStart
		arg this = <instance of Routine>
		arg inval = 671.388961083
^^ The preceding error dump is for ERROR: NNModelInfo: can't load info file '/tmp/nn-sc-1154.yaml'

What am I doing wrong?
I'm on Ubuntu Studio 22.04
SuperCollider 3.14.0-dev

[elgiano]

Hi @ariutti
thanks for the message! I've being trying to load the same exact models you load, and it works on my machine (arch linux, supercollider 3.14-dev, latest nn.ar).

NN.load(\iil, "~/rave/birds_dawnchorus_b2048_r48000_z8.ts", action:_.describe)

NNModelDesc: loading ~/rave/birds_dawnchorus_b2048_r48000_z8.ts
NNModelDesc: loaded ~/rave/birds_dawnchorus_b2048_r48000_z8.ts
NNModelDesc: couldn't read attribute 'none'

*** NNModel(iil)
path: ~/rave/birds_dawnchorus_b2048_r48000_z8.ts
minBufferSize: 512
- method encode: 1 ins, 8 outs
- method decode: 8 ins, 1 outs
- method forward: 1 ins, 1 outs

Are you also running the latest nn.ar? I haven't tried with older versions. It seems like what fails on your machine is the backend.load function, which is the same as nn_tilde. Perhaps running the updated torch version that comes with latest nn.ar would fix it? I hope so!

[ariutti]

Hello @elgiano and thank you for your help,
I've just downloaded the latest version of nn.ar, and tried again loading the model, same error unfortunately.

Perhaps important to know is that the IIL models themselves also conflict in some way if, instead of loading them via nn.ar into SuperCollider, I try to load them into RAVE VST inside a DAW (Reaper in my case).

When I do this, the plugin stops working with non-standard behaviour ranging from the DAW crashing, to the inability to run the plugin again unless the model is deleted from the model folder where it had been imported.

Could it be something wrong with my environment setting that affects both nn.ar and RAVE VST for some reason?
Thank you so much
nicola

[elgiano]

Sorry to hear, that's something strange indeed. I've been working with people at IIL recently, and they can load their models in nn.ar (and nn_tilde) correctly. My guess at this point is that something strange is happening with libtorch. Do you have it installed system-wise? And if you do, which version?

…

-------- Original Message --------

On 9/22/24 15:19, Nicola Ariutti wrote: Hello ***@***.***(https://github.com/elgiano) and thank you for your help, I've just downloaded the latest version of nn.ar, and tried again loading the model, same error unfortunately. Perhaps important to know is that the IIL models themselves also conflict in some way if, instead of loading them via nn.ar into SuperCollider, I try to load them into RAVE VST inside a DAW (Reaper in my case). When I do this, the plugin stops working with non-standard behaviour ranging from the DAW crashing, to the inability to run the plugin again unless the model is deleted from the model folder where it had been imported. Could it be something wrong with my environment setting that affects both nn.ar and RAVE VST for some reason? Thank you so much nicola — Reply to this email directly, [view it on GitHub](#11 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AACNMWRO3CJ7I57Q2XOF3PTZX27WRAVCNFSM6AAAAABORYRAP2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRWG44DKOBWGA). You are receiving this because you were mentioned.Message ID: ***@***.***>

[ariutti]

Thank you @elgiano ,
I'm also thinking the problem is related to libtorch.

From my investigation (I put some put some prints here and there) it seems that everything works fine without any problems on the sclang side. Things start to get murkier to me as soon as the ball gets passed to the NNModelDesc class.
In particular when the backend calls torch::jit::load(path); even if I'm not shure...
Maybe this is the point where the code breaks.

I don't have any of the libtorch installed system-wise (at least if I open my packet manager I don't see the "installed flag" on those). How can I be shure of not having libtorch installed system wise?

But I see that, from inside the nn.ar extension there's a folder named ignore containin:

• libc10.so
• libgomp-98b21ff3.so.1
• libtorch_cpu.so
• and libtorch.so

Does it make sense to you? Are those libraries used in any way from the extension?

[elgiano]

In particular when the backend calls torch::jit::load(path); even if I'm not shure...

I think it happens there as well. Then it's inside libtorch. Since you're already modifying cpp code, could you print libtorch's version? You could use this line:

Print("Torch version %s\n", TORCH_VERSION);

I would put it in PluginLoad, it's in NNUGens.cpp, line 393. Then you would see the message when you boot the server. Latest version on Linux should say Torch version 2.3.0

[ariutti]

hi @elgiano ,

I built the nn.ar plugin adding the line Print("Torch version %s\n", TORCH_VERSION); in line 393 inside NNUGens.cpp in order for this block of code looks like this:

PluginLoad(NNUGens) {
  Print("Torch version %s\n", TORCH_VERSION);
  // Plugin magic
  ft = inTable;

  registerUnit<NN::NNUGen>(ft, "NNUGen", false);
  NN::Cmd::definePlugInCmds();
}

Fact is that now, when I try to boot the server I get a segmentation fault error and the server is not able to boot up.
Commenting out the line and building again works as expected instead.
Does it make sense to you?

[elgiano]

Very strange, my guess is that you compile and link two different versions of libtorch? I'm shooting in the dark at this point. I would try with a clean build: 1. ensure you download libtorch 2.3.1, I'm not sure it works with latest (=2.4.1): https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-2.3.1%2Bcpu.zip 2. remove your build folder, make it again 3. Perhaps also remove your nn.ar folder in SuperCollider/Extensions 4. build and install again (instructions here: https://github.com/elgiano/nn.ar?tab=readme-ov-file#configure-and-build) Hope it works!

…

-------- Original Message -------- On 9/24/24 11:09, Nicola Ariutti wrote: hi @elgiano , I built the nn.ar plugin adding the line Print("Torch version %s\n", TORCH_VERSION); in line 393 inside NNUGens.cpp in order for this block of code looks like this: PluginLoad(NNUGens) { Print("Torch version %s\n", TORCH_VERSION); // Plugin magic ft = inTable; registerUnit<NN::NNUGen>(ft, "NNUGen", false); NN::Cmd::definePlugInCmds(); } Fact is that now, when I try to boot the server I get a segmentation fault error and the server is not able to boot up. Commenting out the line and building again works as expected instead. Does it make sense to you? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[ariutti]

I @elgiano,

I return to you a little embarrassed because I have just made a discovery that has made it possible for me to solve my problem. All these errors I am experiencing are due, of course, as is often the case, to a mistake on my part :)

I had downloaded all the IIL-models using git, but had not installed the git large file storage beforehand.
The download had not given me any errors and I had not noticed anything. Then, a minute ago, I saw the weight of the files (very small) and realised that something was wrong.

I installed lfs with my GNU/Linux package manager and did a git clone again (now the whole folder containing all the models weights about 10 GB). I tried to import the models into SuperCollider with NN.ar() and no longer encounter the problems.

I apologise for this error due to my misunderstanding of how hugging face cloning mechanisms works.

Thank you very much for your support, I would never have got there without your help.
Thanks again and sorry for wasting your time