Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to struct interop for DOTS #70

Open
wants to merge 39 commits into
base: master
Choose a base branch
from

Conversation

limoka
Copy link
Contributor

@limoka limoka commented Jan 22, 2023

This PR includes three improvements, which make working with DOTS based games much easier:

  • Allow to inject custom structs into il2cpp runtime.
  • A fix to how methods with struct pointers are generated.
  • Allow Generic structs to be marked as blittable. This change could be breaking as some structs previously marked as classes now are structs. I'm performing checks if the struct is indeed blittable and making it's generic parameters constrained to unmanaged.

@limoka limoka marked this pull request as draft January 23, 2023 12:28
@limoka limoka marked this pull request as ready for review February 2, 2023 18:44
@limoka
Copy link
Contributor Author

limoka commented Feb 2, 2023

With the last commit, I think I fixed the generation issue Kasuromi found earlier. Now I added another kind of TypeSpecifics: GenericBlittableStruct, meaning blittability depends on used type arguments. For such structs I generate both boxed and unboxed versions, and switch them depending on the generic arguments.

@limoka limoka requested a review from js6pak May 23, 2023 08:46
mosamadeeb added a commit to mosamadeeb/MMXD-Mods that referenced this pull request Sep 11, 2023
@ds5678 ds5678 added enhancement New feature or request generation Related to assembly generation labels Jul 15, 2024
@ds5678
Copy link
Collaborator

ds5678 commented Sep 9, 2024

Hello! I am open to reviewing this pull request if you rebase it.

@limoka
Copy link
Contributor Author

limoka commented Sep 9, 2024

I can do that sometime later, however recently I'm rather busy with Core Keeper's 1.0 release.
On top of that I no longer have a test game. Core Keeper no longer uses il2cpp, and the only other game I know of that uses this PR is VRising.

@limoka
Copy link
Contributor Author

limoka commented Sep 9, 2024

Also if I had to guess there is a lot of changes that would be needed because of #124 for this to work properly.

@ds5678
Copy link
Collaborator

ds5678 commented Sep 9, 2024

Also if I had to guess there is a lot of changes that would be needed because of #124 for this to work properly.

That is correct. The switch to AsmResolver required many changes. In some ways, it was a rewrite of the generator.

@limoka
Copy link
Contributor Author

limoka commented Sep 9, 2024

Do you have any Cecil to AsmResolver migration guides perhaps? I'm not familiar with the latter.

@ds5678
Copy link
Collaborator

ds5678 commented Sep 9, 2024

They are very similar, though I don't have any guides.

@limoka limoka force-pushed the dots-improvements branch from f38c606 to 6663045 Compare January 23, 2025 13:06
@limoka
Copy link
Contributor Author

limoka commented Jan 23, 2025

@ds5678 Hello, I have rebased and updated the PR in order to work with AsmResolver as you asked. (Previous version of the PR is available here)

I have checked that the output of this version of PR is consistent or very close to original PR output. To verify this I used your tool mentioned in your #124 PR.
For these checks I have used VRising game, which is one of other known to me il2cpp + DOTS games (Core Keeper has switched to mono, so I'm unable to test on it)
On VRising's ProjectM.dll all 7k types match, only ~3k methods differ from ~55k, only ~3k fields differ from ~54k. Of the differences I found, most are either fixed issues or irrelevant differences.

Additionally I have installed a few mods from VRising Thunderstore (Bloodstone, VCF). From my quick test, all tested mods are working.

@@ -6,7 +6,7 @@ namespace Il2CppInterop.Runtime.Attributes;
/// This attribute indicates that the target should not be exposed to IL2CPP in injected classes
/// </summary>
[AttributeUsage(AttributeTargets.Method | AttributeTargets.Constructor | AttributeTargets.Property |
AttributeTargets.Event)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not injecting all the instance fields of a struct sounds like a bad idea.

Copy link
Contributor Author

@limoka limoka Jan 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, you are definitely correct that this is a bad idea.

The reason I did this was that you cannot inject a generic type. And I needed to have my job structs contain a handle type. Like this:

public struct ExampleJob
{
    //These fields can be used as is
    public float inputData;
    public NativeArray<Entity> entities;
    public EntityCommandBuffer ecb;

    // I can't inject this type, nor can I use original type (Because it doesn't have code compiled for CustomComponentCD)
    // So I hide the field
    [HideFromIl2Cpp] 
    public ModComponentDataFromEntity<CustomComponentCD> inputFromEntity;
}

In order not to have crazy issues I had to hack Job Scheduler code, access internal structs and manually adjust the struct size for the code to work.
Or simply it's a giant hack, but it did work. (Here's the code: link)

As such I see two solutions:

  1. Remove this ability and implement generic injection or some way to use generic types with types not used by the game code (This one would probably be a huge thing, and would simplify a lot of things I had to do)
  2. Keep this for now, but perhaps warn if encountered to inform the modder about consequences of doing this

Comment on lines -201 to -202
if (baseClassPointer.ValueType || baseClassPointer.EnumType)
throw new ArgumentException($"Base class {baseType} is value type and can't be inherited from");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does baseClassPointer.ValueType return true for System.ValueType? If not, this shouldn't be changed.

Copy link
Contributor Author

@limoka limoka Jan 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure it does return true. This check being removed is exactly why structs can be injected. This is big part of this PR.

Specifically structs are needed because Unity's ECS (Entity Component System) heavily relies on structs to hold game state. By injecting custom ones we can add our own game state.


namespace Il2CppInterop.Generator.Utils;

public struct GenericParameterContext
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's wrong with AsmResolver.DotNet.Signatures.GenericContext?

Copy link
Contributor Author

@limoka limoka Jan 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that It handles a different problem. Specifically GenericContext allows to resolve a type from a GenericParamterSignature.
But what I need here is to resolve GenericParameter (Which has all the information about the parameter including constraints and attributes) from GenericParamterSignature (Which is a weird type that doesn't even know who it belongs to).

This is all caused by the weird design of AsmResolver. I have read some messages on their discord describing why GenericParamterSignature is as it is. However the fact metadata stores it this way, doesn't really justify complicating handling of GenericParamterSignature by making them glorified ints. I found Cecil's approach of GenericParameters having all context at all times much more convenient, and the code was written around that.

So GenericParameterContext was needed to implement the logic with AsmResolver. Perhaps maybe Washi can take a look at this code and say how they would do it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request generation Related to assembly generation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants