livecodable dsp

[ahihi]

this is an implementation of livecodable DSP by interpreting sclang code embedded in the OSC messages sent from Tidal. the code is extensively commented, so please see files changed for the details!

this is currently implemented as a "hack", but imho it is kind of a game changer and i propose that it might be worth considering as a core feature (but disabled by default, given the security implications). thoughts?

[jwaldmann]

amazing. next (on the Haskell side): make an SC backend for https://hackage.haskell.org/package/csound-expression to produce programmatically the strings that describe the DSP ...

[ahihi]

there are certainly interesting possibilities for DSLs, but lets keep discussion here focused on the SuperDirt implementation :)

i am currently working on extending this idea to per-orbit global effects, so we can write our own delays, reverbs, etc!

tidal.gdsp.mp4

[telephon]

I really like it. Simple and appropriately dirty. Let's take some time to think about how to best integrate it into core – of course it would need to be switched on explicitly.

Moreover, we should test the address the OSC message was sent from and only accept if it is 127.0.0.1 (just as a minimum protection).

We used to have a message called avoidTheWorst in our Republic system, that filtered all kinds of dangerous stuff, but of course it was exactly only a protection against unserious attempts.

[telephon]

It could also be an OSC responder with a separate OSC message that builds the synth def. Then tidalcycles could do this separation into another message. Or at least a separate method.

Also, we need to rebuild only when something in the code changes. The code could be kept ins a set and you check for changes?

[ahihi]

here are my motivations for some of the decisions in the current implementation, and some thoughts on integration.

the module:

• synthdef is rebuilt on every event
  • this allows using sclang-side randomness and other control structures to determine the synth graph, a very inspiring form of dynamism that was natural in SC2
• receives the input signal as variable in and the output (and input) bus as out
  • enables both standalone sound source and effect unit use cases with minimal typing
• output is implicit
  • less typing
  • explicit output can still be done with Out.ar and the out variable
  • maybe needs some more sophisticated logic, like GraphBuilder.wrapOut used by Function.play?
• goes first in the module chain (after dirt_sound)
  • can further process the sound with existing SuperDirt modules
• reads event data via environment variables
  • ~freq (etc) is quick to type, and automatically available
  • this is also only possible if we rebuild on each event
  • a downside is that these values get baked into the synthdef and cannot be modulated via control buses
    • i am considering also adding support for using NamedControls (\freq.kr), similarly to what i did for the global effect

the global effect:

• synthdef (and synth) is rebuilt when the code (~gdsp) changes
  • this is really the only way that makes sense for long-lived effects like delay or reverb
• receives dry signal as variable in and the two buses dryBus and effectBus
  • consistent with the module
• output is implicit and replaces the content of dryBus
  • the idea was to easily support both effects that replace the input signal (RLPF.ar(in, …)) and ones that add to it (in + CombN.ar(in, …))
  • but explicit output still possible with dryBus and effectBus
• goes first in the effect chain
  • again, enables usage as both effect and sound source for further processing by existing effects
• reads event data via NamedControls
  • environment variables obviously will not work, given the persistent nature of the effect

with the above in mind, i am not sure using separate OSC messages would be an improvement, while it would certainly increase complexity. the current design requires no additional support from frontends, which i consider a big advantage.

as for integration into core, i think something like the structure i ended up with in the latest commit could work: we would introduce a new type of event diversion which runs in a Routine and can signal "i need to run some code after a server sync" by returning a post-sync callback. for hackability, it would also be nice if this was a list of functions rather than a single function (like the current diversions are), so that new diversions could easily be added while keeping existing ones in place.

with the above mechanism, enabling the livecodable dsp functionality would be just a matter of adding functions to the new "syncable diversions" list. it seems like there could be other use cases for this, though i cant think of any right now. it does raise the question of whether such a mechanism should be even more general, perhaps having the ability to wait for arbitrary things such as Conditions. but then the api design becomes a lot more complicated…

finally, there is some room for optimization:

• the silence synth should probably just free itself immediately
  • or we could have special handling for this and not spawn a synth at all
• rather than spawning the "default" global effect synths when there is no ~gdsp, it would be better to not spawn any synth.

[telephon]

Sorry, for some reason I haven't seen your reply, it wasn't in my notifications. I think we can make a version that does not require hacking the diversion.

But it'll have to wait till later this week.

[ahihi]

no worries, interested to hear your ideas whenever you have time!