Realtime Spherical Harmonics ambient lighting testing

[Frooxius]

Hello everyone!

I've got another thing I'd like some eyes on before I release it to the main build - the new realtime ambient light computation with Compute Shaders!

Build Version

2025.1.15.1419

What is being tested

As part of migrating more pieces of Unity to FrooxEngine for the eventual separation, I've reworked how the ambient lighting works - exposing the spherical harmonics coefficients and adding components that allow them to be computed from a Reflection Probe using compute shaders.

This not only gives you much better control over the ambient lighting, but also improves existing content - the Unity's computation is pretty heavy and CPU based, so it needs to be throttled, whereas this implementation can be updated every frame.

You can see this video for comparison: https://bsky.app/profile/frooxius.bsky.social/post/3lfq6ctt7622e

General testing notes

• There's "AmbientLightSH2" component. This works similarly to the "Skybox" component where it determines the ambient light in the scene
• There's also "ReflectionProbeAmbientSH2" component - this will compute SH2 values from the probe's current texture (Realtime/OnChanges probes work best, for Baked the texture currently needs to be uncompressed)
• ReflectionProbe has a new "OnChanges" mode and "ChangesSource" field - this works similarly to Realtime mode, except it only re-renders when the ChangesSource (or the probe itself) is changed. This is typically linked to the skybox material, so the probe re-renders only when the skybox material changes
• The global reflection probe and ambient light is no longer provided globally by Unity, but it needs to be setup in each world
• Each world is auto-upgraded to use this new setup, adding the above components and linking them together to match the legacy behavior

Avoiding data corruption

• DO NOT save any converted worlds/items for any purpose other than testing - they might break or have conversion issues!

How to test

Currently I'm keeping the build still on the prerelease branch.

• On Steam, switch to the prerelease branch and make sure the build updates before you run it
• Make sure your build version matches the one at the top of this message

What to test

• Open a bunch of different worlds and check the ambient lighting
• Try variety of worlds with lots of different types of lights and skyboxes
• Try worlds with more dynamic skyboxes and lighting changes
• Open multiple worlds and switch between them
• It can be useful to disable the directional light source in the world so you see the contribution of the ambient light better
• (Optional) Mess with the components themselves, take them apart and try to build your own ambient lighting systems - blending, lerping and so on - the auto-converted system isn't how things always need to be done - you have more control, so if you want to mess with it, please do! It can help discover some more bugs maybe, though it's not the main focus right now.

What to watch out for

• Ambient lighting is incorrect or doesn't update when it should
• Compare how the worlds look relative to the main build (might need to run builds side by side or take screenshots for this)
  -- The goal isn't for the ambient light to look identical, but close
• Check performance, especially when the skybox is being changed and the values are recalculated - is there noticeable drop compared to the main build?

Tuning ambient light to match the original

• The ambient light can be tuned on the ReflectionProbeAmbientSH2 component with the Order0...2 sliders - this allows you to change the influence of the different orders, in the increasing rate of frequency
• Play with those sliders to change how the ambient lighting works - typically boosting lower orders and decreasing higher orders smooths out the ambient light (which can be more sharp/drastic compared to the default Unity)
• If you find values that look better overall for large number of worlds and match the old look, please share them! We can change the defaults for converted content to look closer

What NOT to test & report

• If the AmbientLIghtSH2 component is missing in the world, the behavior is currently undefined - it'll just be stuck with previous one
• Generally things that do not involve this system - any existing unrelated bugs with components and other systems.
  -- If you're unsure, try to replicate the issue on the main build and see if it also occurs - if it also happens there, just report it as normal bug

If you find something that got broken

• Create a GitHub issue for a pre-release bug: https://github.com/Yellow-Dog-Man/Resonite-Issues/issues/new?assignees=shiftyscales&labels=prerelease&projects=&template=prerelease.yml
• Make sure to include the log
• Please include description of what happened. Be succinct, but exact.

If you find that stuff works okay

Please report it too and include details on what you generally tested, so we know things are okay. You don't need to make a new issue for this - just a brief comment in this Discussion is fine.

This lets us know that a bunch of stuff was tested and get better confidence that stuff won't break with release.

How long will testing go for?

At very least until sometime next week since I'll be at FC and traveling. If there are major issues, we'll delay the release, but otherwise this will probably get released sometime next week (from this post).

[epicEaston197]

Hey it seems like you didn't actually push the pre-release I've restarted steam multiple times and I'm still not receiving the build

[Frooxius]

It's there on Steam when I check it. I'm not sure why it's not going through, I'll re-push it soon.

[troyBORG]

I switched mine to pre-release. Told it to update and I'm on 2025.1.10.214

[Frooxius]

Okay, it should serve the latest version now! I don't know what was wrong with it, the build was in SteamPipe and the hashes matched the new version, but it wasn't serving it.

Try updating/restarting Steam, it should give you the proper version now!

[NukiRaccoon]

Testing Notes
Opened Resonite, version 2025.1.15.1419, Desktop mode, looking over instructions again...

Open a bunch of different worlds and check the ambient lighting
Try variety of worlds with lots of different types of lights and skyboxes
Try worlds with more dynamic skyboxes and lighting changes
Open multiple worlds and switch between them

Mess with the components themselves, take them apart and try to build your own ambient lighting systems - blending, lerping and so on - the auto-converted system isn't how things always need to be done - you have more control, so if you want to mess with it, please do! It can help discover some more bugs maybe, though it's not the main focus right now.

What to watch out for
Ambient lighting is incorrect or doesn't update when it should
Compare how the worlds look relative to the main build (might need to run builds side by side or take screenshots for this)
-- The goal isn't for the ambient light to look identical, but close
Check performance, especially when the skybox is being changed and the values are recalculated - is there noticeable drop compared to the main build?

World 0 - LocalHost (little building)
Loaded fine, added the "ReflectionProbeAmbientSH2" component to the one reflection probe, re-baked... Already seeing something that looks different: the ceiling is much darker.

Everything else seems fine...

World 1 - Support Group Meeting Center
This one has lots of probes, and uses the Cloud Home's sun system.
Loads fine, but once again, things look a lot darker, even when lit from elsewhere...

Baking new maps under what should be the same lighting conditions seems to create lighter probe maps...?

Shadows have also changed. They seem more accurate now, to be fair, but they've changed. There's also a circle around me where they are not showing...?

Ok, I'm under the impression there's something I'm not understanding, but from what I can see, the new system is not working correctly in every case, and the results of baking a probe has been significantly changed. It updates faster for sure, seems like you're going in the right direction, but I'm not really sure what to even report for this other than "Baked Probes just don't work right anymore".

I have to go do something else, so I'm going to stop for now, and check back later. If you want a report on this, I can get you one.

Edit: Photos from Current Build
I notice the shadows also behaving weird in this one, so maybe it's that world, but if you look, the flag isn't inexplicably dark in this one.

Same with the LocalHome house: ceiling is much lighter...

[VirgilCore]

I've also noticed that shadows are blue-er now?

[shiftyscales]

Could you create a prerelease issue and show samples of this, @VirgilCore?

[troyBORG]

I turned off the "Skybox only" which makes it so it takes the ground into consideration which makes the lighting look better. But I noticed that if you have something emitting colors near it, it will read those for the colors for a few seconds before stopping when you change the skybox lighting..

https://troyborg.us-southeast-1.linodeobjects.com/2025-01/2025-01-18%2014-18-59.mp4

If I don't have the thing sitting right on top of the Reflection Probe it doesn't do it.
So I guess just another reason to not place stuff at spawn directly....

https://troyborg.us-southeast-1.linodeobjects.com/2025-01/2025-01-18%2014-40-35.mp4

[Frooxius]

That's normal. The probe will render everything in its vicinity whenever change is triggered if you have it rendering everything and not just the skybox.

You could just place it elsewhere where there's no other stuff. Once we have more filtering options for reflection probes it'll help.