How do I intergrate/submit my own upscaler to be apart of gamescope?

[KingOwnageXV]

Hello, I was wondering if their is an official process to get your own upscaler intergrated into gamescope - I am a co-creator of the program "Lossless Scaling" which you can find here: https://store.steampowered.com/app/993090

We have been working on our own lightweight spatial upscaler to compete with FSR 1 and in many ways it exceeds it in quality to some degree. It's called "LS1" and I am willing to share the source code with gamescope if they are willing to at least look at it for potential integration. It works great on APUs and provides excellent upscaling quality and theirs even a more light performance version of it too.

If I'm not in the right place to request this please direct me where or give me some contact information, as this is currently the best spatial upscaler to date. I am also willing to provide a free license key of the software with any gamescope dev if they are interested, thank you for your time

[misyltoad]

You can just pull request it, see how FSR and NIS are implemented in Gamescope. Please also provide comparisons.

[KingOwnageXV]

You can just pull request it, see how FSR and NIS are implemented in Gamescope. Please also provide comparisons.

Comparisons between native and LS1 or LS1 to other upscalers such as FSR 1? I can do both

[dgmvecuador]

Will LS1 need a license to use?? Maybe implement Anime4K??

[KingOwnageXV]

Will LS1 need a license to use?? Maybe implement Anime4K??

Anime4k would not be a great fit for Steam Deck / Gamescope, due the purpose of the upscaler and the purpose of this service.

LS1 will not need a license to use, if it is added it will be free and avalible to everyone to benefit from. LS1 already has better results than FSR 1 in a lot of areas and unlike FSR 1 which is abandoned since FSR 2 released LS1 is continuously being updated and improved, and that vision will remain true until we hit the physical limitations of what a spatial upscaler can achieve and there is no more room to improve on it, so we really hope it is considered being added, as I'd be willing to give the source code.

[JohnnyJosda]

Will LS1 need a license to use?? Maybe implement Anime4K??

Anime4k would not be a great fit for Steam Deck / Gamescope, due the purpose of the upscaler and the purpose of this service.

LS1 will not need a license to use, if it is added it will be free and avalible to everyone to benefit from. LS1 already has better results than FSR 1 in a lot of areas and unlike FSR 1 which is abandoned since FSR 2 released LS1 is continuously being updated and improved, and that vision will remain true until we hit the physical limitations of what a spatial upscaler can achieve and there is no more room to improve on it, so we really hope it is considered being added, as I'd be willing to give the source code.

Can LS1 outperform FSR in terms of performance?

[campo-clams]

Replying to #788 (comment)

I have used ls1 on windows and I find I can lower the res a bit more while keeping quality with it compared to FSR 1 I would love to be able to use the upscaler on my steam deck and I hope you consider it, if it’s possible to integrate it

[Slvrtnge]

@KingOwnageXV apologies for the ping, but has there been any update on adding your upscaler to gamescope? Having this option (and perhaps someone making it configurable via a Decky Plugin afterward) would be a HUGE benefit to the deck community. Personally I’d even pay for your regular software for even newer versions of your excellent upscaler. I can only afford a steam deck so getting even more performance out of it would mean the world to me and others. Any info appreciated :) keep up the great work!

[kangurek-kao]

You can just pull request it, see how FSR and NIS are implemented in Gamescope. Please also provide comparisons.

Half-Life 2 (Windows 11, Intel UHD Graphics 10210U, DX9)

1080p, 8xMSAA

768p, 8xMSAA, FSR1 (Sharpness 5/10)

768p, 8xMSAA, NIS (Sharpness 5/10)

768p, 8xMSAA, LS1 (Sharpness 1/4)

Notice how LS1 has less aggressive edge smoothing, finer contrast and better subpixel detail preservation. There are oversharpening artifacts in some places for sure, but they're nowhere near as bad as the ones seen in NIS. Not to mention that LS1 seems to be the most efficient algorithm when GPU bound (while NIS seems to be the least efficient).

[sharkautarch]

Hmm
Does anyone know if LS1 uses only compute shaders, or if it also makes sure of stuff that'd involve the graphics pipeline?
Reason why I ask is that gamescope's vulkan backend almostly entirely uses compute-only parts of the vulkan pipeline, with the only place making use of graphics pipeline parts being the reshade integration...

So it would theoretically be possible to use LS1 w/ gamescope if LS1 was ported to using the reshade .fx language...
Except that currently gamescope runs reshade shaders on each frame first, and then, if you're upscaling with gamescope, upscales the frame.
So that may yield lower quality results...

@kangurek-kao does LS1 do upscaling, or does it only do sharpening?
Because if LS1 normally does both upscaling and sharpening, it may be possible for gamescope to create a custom extension to the reshade .fx shader language that provides direct support for upscaling w/ reshade shaders
The only catch is that currently gamescope uses really restrictive vulkan syncronization barriers, which works fine for gamescope compute-based compositing shaders, but may the hinder performance of very complex reshade shaders (which may be the case for LS1). The only practical solution would be for gamescope to integrate some form of rendergraph...

[kangurek-kao]

@sharkautarch It does both upscaling (custom scaling algorithm) and sharpening (five levels, from 0 to 4). It should use compute shaders, as the software of origin (Lossless Scaling) works via hooking up to Windows APIs (DXGI, WGC, GDI) and injecting itself at the very end of window rendering pipeline (thus the requirement of running software in windowed mode).

Based on machine learning, it is a small and fast neural network for general use, optimized for scaling factors in the 1x - 2x range. LS1 has similar performance and image quality to FSR, while surpassing it in many cases.

There is nothing special about model training. It learns to scale LR (low-res) to HR (hi-res) sharpened images. First, this is a very small model. Second, it has a custom layers configuration to help the shader version run faster, and finally, there are all possible shader optimizations.

Almost every filter will have an inaccuracy for a line a few pixels wide on all edges. This is due to the fact that a matrix is ​​used to process the image, so the pixels around are also used to process each pixel. At the edge of the image, some pixels do not exist, so zeros are used instead

It seems to me that the sharpness is sufficient, since level 4 roughly corresponds to 10 for the FSR.

I've tried this [adding more sharpness levels] before and failed to properly train the network with smaller sharpness intervals.

[LS1's colorspace is] YCbCr

There's also a LS1 Performance model, which is 40% more efficient than LS1, but at cost of being less precise and more prone to ringing and oversharpening. This is noticeable especially at higher resolutions (e.g. 1440p -> 2160p)

Edit: Typos

[sharkautarch]

@kangurek-kao

It should use compute shaders
Well that's good since that should make it easier to integrate into gamescope
the software of origin (Lossless Scaling) works via hooking up to Windows APIs (DXGI, WGC, GDI) and injecting itself at the very end of window rendering pipeline

Hmmm
Well for integrating into gamescope, there's some vulkan pipeline & vulkan compute dispatch command stuffs which at least someone will have to deal with in order to connect any new shader (in general).
But in terms of the shader itself, if you're using HLSL, it looks like there's a HLSL -> glsl/vulkan compiler that may make it easier to port your shaders to ones that'll be compatible with gamescope: https://github.com/microsoft/ShaderConductor

Tho I think you may still need to do some editing yourself, because you'd ideally have your shaders take a sampler2d texture that acts as the shader's input image, and an image2D texture that acts as the output
For reference, see the descriptor set that gamescope's shaders use (the lines that start with layout(binding = <num>):
https://github.com/ValveSoftware/gamescope/blob/master/src/shaders/descriptor_set.h

The binding number that's specified in the shader is also the same numbering used when binding resources and/or images to the vulkan pipeline:

gamescope/src/rendervulkan.cpp

Line 742 in a215a8d

std::array<VkDescriptorSetLayoutBinding, 7 > layoutBindings = {

gamescope/src/rendervulkan.cpp

Line 1436 in a215a8d

std::array<VkWriteDescriptorSet, 7> writeDescriptorSets;

(Specifically the number assigned to .dstBinding for VkWriteDescriptorSet, and the number assigned to .binding for VkDescriptorSetLayoutBinding)
Yes, it is pretty tedious and complicated...
Feel free to ask for help if you have any questions

[misyltoad]

You can just compile HLSL with glslangValidator or any of the other things with minor alterations fwiw

[Boulotaur2024]

Eagerly waiting to have this implemented in Gamescope, as this seems to be the only way to have Lossless Scaling in Linux

[sharkautarch]

Only @kangurek-kao @KingOwnageXV can really work on integration of it into gamescope, since no one else has access to the source code of Lossless Scaling

If they have any questions, they can always ask here
I can also dm them a link to a discord server where they can ask questions about gamescope there
EDIT: Oops, I should have said KingOwnageXV has access to the source code, not kangurek

[kangurek-kao]

Only @kangurek-kao can really work on integration of it into gamescope, since no one else has access to the source code of Lossless Scaling

FYI I'm only responsible for PL translations via Crowdin. I have no access to the source code and I guess that @KingOwnageXV might be the one that would be responsible for shader prep.

[sharkautarch]

@kangurek-kao My bad on that point

[Slvrtnge]

Anyone know of any progress on this? Would love another upscaler on my deck! A decky plug-in would be incredible but even a less convenient way I’d still try.

[omfgnuts]

@KingOwnageXV Any luck with this?

[meatz0ne]

@KingOwnageXV Would love to see this implemented for steam deck, we need frame generation.