vulkan: Start on dynamic_state_commands system

[magcius]

Description
The eventual goal here is to cache PSOs in a format that they can be reused; e.g. if the user creates two PSOs that only differ with depth-test, we could reuse the same PSO, and use dynamic state to flip things around. Even on D3D12, we could keep the same HLSL/DXC binary cached and only create a new PSO.

This is a very early version of this that still creates a new PSO, but switches to using dynamic state in Vulkan when available.

Checklist

• Run cargo fmt.
• Run taplo format.
• Run cargo clippy. If applicable, add:
  • --target wasm32-unknown-unknown
• Run cargo xtask test to run tests.
• If this contains user-facing changes, add a CHANGELOG.md entry.

[cwfitzgerald]

Sorry for taking so long to get back to you on this one.

I generally don't hate the code, my only real apprehension is that it adds a good amount of complexity and opportunities to subtly mess the code up. Maybe if the dynamic state handling could be encapsulated in some way, I would feel better about this.

Are there performance ramifications to aggressively going to dynamic state? We might have more redundant state changes.

[magcius]

Maybe if the dynamic state handling could be encapsulated in some way, I would feel better about this.

Yeah, that's how I felt writing a lot of this too, and why I filed a draft PR for this before working on it too much. Do you have any suggestions here? I could also limit the choices for dynamic state to only respect Vulkan 1.3 and stop caring about the older extensions. This would thin out the dynamic state code; and perhaps devices without Vulkan 1.3 are where this code would be the most helpful.

The other issue right now is that create_render_pipeline and create_mesh_pipeline are 90% the same, and a lot of the same code exists in both; this PR just duplicates the same changes in both, which makes it more tedious to update and review. I tried to refactor these out, but the nature of on-the-stack variables means it's difficult to do without a bigger refactor, though probably one that's worth doing. Not sure I know enough Rust to attempt it in earnest though.

Are there performance ramifications to aggressively going to dynamic state? We might have more redundant state changes.

We could cache the state at a command buffer level and omit calls if they're the same (assuming it's cheap to compare), but in my experience, drivers already do a fair job at this, and all the dynamic state is often pretty cheap to set on the hardware side anyway.

The big piece that's missing here to actually make it efficient is:

1. Cache SPIR-V shader module by dynamic inputs (pipeline overrides, stage linkage, vertex inputs? Anything that affects SPIR-V output basically). This means that if you create two pipelines with the same shader module but different state, we won't bother re-parsing and re-translating the module from scratch.
2. Cache render pipelines in a way that "zeroes" dynamic state from the cache key. This means that creating two different render pipelines that differ only in dynamic state can share the underlying Vulkan PSO.

[cwfitzgerald]

Yeah, that's how I felt writing a lot of this too, and why I filed a draft PR for this before working on it too much. Do you have any suggestions here? I could also limit the choices for dynamic state to only respect Vulkan 1.3 and stop caring about the older extensions. This would thin out the dynamic state code; and perhaps devices without Vulkan 1.3 are where this code would be the most helpful.

I don't have amazingly actionable ideas here besides "throw it in a struct with methods". Maybe something that amounts to a builder (so that you keep mutating the builder as you find what state to include) which you can then get an array to add to the vulkan descriptor and a state objet which contains the list of dynamic state changes the pipeline needs. I hope that makes any kind of sense.