Future API changes

[monkey0506]

FromDelegate

Currently, this project does not support any ref-qualified parameters. In large part, this was because System.Action and System.Func cannot support them. However, C# 10 added natural function types, which allows passing in a lambda or method name to a parameter of type System.Delegate (under certain rules).

We are already restricting the parameters describing marshalling behaviors to values that can be statically analyzed for source generation. Restricting the first parameter of a FromDelegate method under these same rules would not seem strange from a development perspective. We have to be able to analyze the delegate that we are receiving in order for the source generator to produce the necessary code.

The first practical upshot of FromDelegate would be that it would allow ref, in, and out parameters to be defined! However, this does come with some challenges. In order to support IntelliSense (and avoid false errors being reported), the definition of Invoke must be static. This can be overcome using a ref struct for each parameter. When using FromDelegate, the ref kind of Invoke's arguments can be inferred. However, when using FromFunctionPointer, we would not have a managed method to infer this information from, and ref structs cannot be used as type parameters. A more robust solution would be required to support both scenarios (managed to unmanaged and unmanaged to managed).

Another benefit of FromDelegate is that the marshalling behaviors can also be directly inferred from the lambda or method passed as the first argument. This means that we could completely remove the marshalAs arguments currently in place for FromAction and FromFunc. Discovering the IMethodSymbol for the delegate argument might look something like:

static (context, cancellationToken) =>
{
    var invocationExpressionSyntax = (InvocationExpressionSyntax)context.Node;
    if (invocationExpressionSyntax.ArgumentList.Arguments.Count < 1)
    {
        return null;
    }
    var GetOperation = context.SemanticModel.GetOperation;
    return (GetOperation(invocationExpressionSyntax, cancellationToken) as IInvocationOperation)?.Arguments
        .Take(1)
        .Select(static x => x?.Value as IConversionOperation) // conversion to System.Delegate
        .Select(static x => x?.ChildOperations.FirstOrDefault() as IDelegateCreationOperation) // creation of delegate from natural type
        .Select(static x => x?.Target switch
        {
            IAnonymousFunctionOperation y => y.Symbol, // method is a lambda
            IMethodReferenceOperation y => y.Method, // method is a named method
            _ => null // method is stored variable (or other unknown construct?) - generate a diagnostic here!
        }).First();
}

From that point, we can inspect the ref kind, the scoped kind, and the attributes of each parameter. The MarshalAsAttribute attributes are ready for use in the generated source code using only the ToString() method, no additional parsing is needed.

When will it arrive?

I have conducted proof-of-concept tests on these ideas, but I am working on this project in my very limited free time. I cannot make any promises about when these feature changes might get worked on, but I can say that I'm excited about the potential of seeing them implemented. If anyone sees this, thank you for reading!

[monkey0506]

Runtime fallbacks - Partially Implemented by #27

The current interceptors branch requires the MarshalAsAttribute parameters and the CallingConvention to be a compile-time constant expression that the generator can make deterministic decisions about. Namely, this means that these parameters must be supplied with inline new expressions, or a reference to a static readonly field that is initialized inline (currently, any values assigned in a static constructor are ignored completely).

Being able to parse these parameters as compile-time constant expressions is the optimal case, but previous work on the incremental-generators branch has shown that handling values that the generator can't make decisions about can be parsed at runtime with a simple conditional branch (if or switch). This would incur a (very) small performance hit when the end-user invokes the FromAction, FromFunc, or FromFunctionPointer methods versus the optimal case, but the object returned would be otherwise identical. Only creating the object would be impacted.

This would remove all of the diagnostics that the generator currently produces as well, which is also ideal.

Update: The CallingConvention can be parsed at runtime, but the marshalling behaviors still have to produce the static MarshalAsAttributes in the generated source. An update to add runtime handling of the calling convention will be made available.

[monkey0506]

Marshal map - Implemented by #25

Although the marshalling behavior can be greatly simplified in the managed to unmanaged case with FromDelegate, the marshalling behavior cannot be inferred in the unmanaged to managed case. To simplify defining the marshalling behaviors and to allow for better use with generic parameters (e.g., using these methods inside a generic class), it is possible to conceive of a dictionary-style type that would map a parameter type to a marshalling behavior (read as: MarshalAsAttribute), rather than having to define this by the parameter's ordinal position.

This map would fall under the same stringent static analysis rules currently applied to marshalling behaviors. Something such as (see [Proposal]: Dictionary expressions #7822):

private static readonly MarshalMap map =
[
    typeof(string): new MarshalAsAttribute(UnmanagedType.LPUTF8Str), // marshal all `string` arguments as UTF-8
    typeof(bool): new MarshalAsAttribute(UnmanagedType.I4) // marshal all `bool` arguments as 32-bit
];

This map could then be applied such that every string parameter and bool parameter are marshalled. In the case that an unmanaged function uses different marshalling for parameters that are represented by the same managed type, the FromFunctionPointer method would still require the explicit ordinal parameter arguments. Or, the MarshalMap could possibly support adding marshalling behaviors by ordinal index in addition to by parameter type.

The ordinal marshalling descriptor would be given the highest precedence, followed by the MarshalMap, and finally the default .NET marshalling behavior for a given type.

[monkey0506]

MarshallingDescriptor generic type argument - Implemented by #28

The current marshalMap, marshalParamsAs, and marshalReturnAs parameters were an attempt to solve a problem: how do we allow for these marshalling behaviors to be defined at the call-site while still presenting a compile-time constant expression that we can statically parse in order to generate real source code MarshalAsAttribute attributes? You can't attach attributes to a method invocation (e.g., FromAction or FromFunc), so these parameters were added instead.

This approach worked as a proof-of-concept, but it strongly restricted the usage, even when it was discovered that these parameters could be stored as a static readonly field. The parameter value had to be an inline object creation syntax (new) or it had to be a direct reference to that field - namely, you can not chain these parameters through other method calls.

static readonly MarshalAsAttribute marshalReturnAs = new MarshalAsAttribute(UnmanagedType.LPUTF8Str);
// ...
var func = INativeFunc<string>.FromFunctionPointer(pNative, CallingConvention.Cdecl, marshalReturnAs); // OK, param is `static readonly` field
// ...
INativeFunc<string> GetMethod(nint ptr, CallingConvention callingConvention, MarshalAsAttribute? marshalling)
{
    return INativeFunc<string>.FromFunctionPointer(ptr, callingConvention, marshalling); // ERROR - marshalling can't be parsed
}

In the case of GetMethod, the callingConvention can still be checked at runtime, but the attributes that need to be added to the source can't be parsed.

Instead of this approach, it is possible to pass an additional generic type argument that can carry this information. This can be implemented through an interface:

internal interface MarshallingDescriptor<TDescriptor> where TDescriptor : MarshallingDescriptor<TDescriptor>
{
    protected static abstract MarshalMap? MarshalMap => get;
    protected static abstract MarshalAsAttribute?[]? MarshalParamsAs => get;
    protected static abstract MarshalAsAttribute? MarshalReturnAs => get;
}

Then an additional generic type parameter can be added to the interfaces to carry this information through to the call-site:

internal sealed class MarshallingDescriptor : MarshallingDescriptor<MarshallingDescriptor> // this naming is fine
{
    private static readonly MarshalAsAttribute marshalReturnAs = new MarshalAsAttribute(UnmanagedType.LPUTF8Str);
    
    static MarshalMap? MarshallingDescriptor<MarshallingDescriptor>.MarshalMap => null;
    static MarshalAsAttribute?[]? MarshallingDescriptor<MarshallingDescriptor>.MarshalParamsAs => null;
    static MarshalAsAttribute? MarshallingDescriptor<MarshallingDescriptor>.MarshalReturnAs => marshalReturnAs;
}

// ...

INativeFunc<string, TMarshalling> GetMethod<TMarshallingDescriptor>(nint ptr, CallingConvention callingConvention)
    where TMarshallingDescriptor : MarshallingDescriptor<TMarshallingDescriptor>
{
    return INativeFunc<string, TMarshallingDescriptor>.FromFunctionPointer(ptr, callingConvention);
}

// ...

var func = GetMethod<MarshallingDescriptor>(pNative, CallingConvention.Cdecl);
var str = func.Invoke(); // get result from native

This proposal is already lengthy, but the calling convention can also be included in the descriptor to avoid runtime handling if a compile-time constant expression can be supplied there. For example, GetMethod is invoked with CallingConvention.Cdecl, but because FromFunctionPointer is forwarded that argument, we can't see the constant expression and have to fall back to checking the calling convention at runtime. Including this in the descriptor would allow for compile-time analysis and improved runtime performance.

[monkey0506]

Facilitate unsafe API - Implemented by #30

Exposing direct access to function pointers would yield performance benefits when invoking the method, but the API to do so is somewhat challenging, particularly when the unmanaged function pointer's type arguments don't match the managed object's type arguments.

Function pointers also require that the /unsafe compiler option is supplied, and there is no built-in mechanism to check for this (e.g., no pre-defined UNSAFE constant for use with a #if directive).

IUnmanagedAction and IUnmanagedFunc

It should be possible to include definitions such as IUnmanagedAction and IUnmanagedFunc which take both managed and unmanaged type argument lists (as previously permitted by the generic FromFunctionPointer in v1.0). For example:

#if UNSAFE
internal unsafe interface IUnmanagedFunc<T1, T2, TResult, U1, U2, UResult> : INativeFunc<T1, T2, TResult>
{
    // ...
}
#endif // UNSAFE

These interfaces would require the project to define UNSAFE to indicate that the unsafe keyword can be used, otherwise the new interfaces will be omitted from the compilation entirely (as they cannot add any function or utility in that case).

When defined, these interfaces would implement the corresponding INativeAction/INativeFunc interface (the "base interface"), so that is always an available fallback for the user, and provides type-erasure of the unmanaged type arguments when that is desired.

The T-prefixed type arguments would directly correspond to the base interface's type arguments. The U-prefixed type arguments would correspond to the unmanaged function pointer type arguments of the same ordinality (U1 being the function pointer's first type argument, UResult being the return type of the function pointer; for IUnmanagedAction, the function pointer return type would always be specified as void).

Existing factory methods

These interfaces would include their own static factory methods. FromAction and FromFunc would simply forward their arguments to the base interface methods:

public IUnmanagedFunc<T1, T2, TResult, U1, U2, UResult> FromFunc(Func<T1, T2, TResult> func, CallingConvention callingConvention)
{
    return (IUnmanagedFunc<T1, T2, TResult, U1, U2, UResult>)INativeFunc<T1, T2, TResult>.FromFunc(func, callingConvention);
}

The method FromFunctionPointer(nint, CallingConvention) (with or without the IMarshaller generic type argument) would similarly forward its parameters.

New factory methods

The benefit of the new interfaces would come in additional overloads of FromFunctionPointer, which given the additional generic type arguments, could then take unmanaged function pointer arguments directly:

public IUnmanagedFunc<T1, T2, TResult, U1, U2, UResult> FromFunctionPointer(delegate* unmanaged[Cdecl]<U1, U2, UResult> functionPtr)
{
    return (IUnmanagedFunc<T1, T2, TResult, U1, U2, UResult>)INativeFunc<T1, T2, TResult>.FromFunctionPointer((nint)functionPtr, CallingConvention.Cdecl);
}

The CallingConvention argument is removed, because it is instead specified by the function pointer itself. Cdecl, Stdcall, and Thiscall unmanaged function pointers would be supported. The IMarshaller generic type argument would also be supported for these method overloads, without conflict in the CallingConvention, as an explicit value is always preferred over the marshaller.

New properties

The new interfaces could also expose new properties that correspond to the unmanaged function pointers that are supported:

public delegate* unmanaged[Cdecl]<U1, U2, UResult> AsCdeclPtr { get; }
public delegate* unmanaged[Stdcall]<U1, U2, UResult> AsStdCallPtr { get; }
public delegate* unmanaged[Thiscall]<U1, U2, UResult> AsThisCallPtr { get; }

Because the calling convention is baked into the signature of function pointers, there's no way to omit or include only the properties that match the actual specified CallingConvention of the object. However, the current code generation always produces implementation classes that represent a single calling convention. When the calling convention cannot be determined at compile-time, this generates four distinct implementations for each of CallingConvention.Cdecl, CallingConvention.StdCall, CallingConvention.ThisCall, and CallingConvention.Winapi. Except for Winapi, each of these could implement the properties without any runtime checking:

// given that this implementation class is for Cdecl calling convention
public delegate* unmanaged[Cdecl]<U1, U2, UResult> AsCdeclPtr => (delegate* unmanaged[Cdecl]<U1, U2, UResult>)functionPtr;
public delegate* unmanaged[Stdcall]<U1, U2, UResult> AsStdCallPtr => null;
public delegate* unmanaged[Thiscall]<U1, U2, UResult> AsThisCallPtr => null;

Note that functionPtr is already being stored as a field in the implementation class.

For implementation classes representing CallingConvention.Winapi, this would require a runtime check of the OS to appropriately choose between AsCdeclPtr and AsStdCallPtr to return non-null; AsThisCallPtr would always produce null in this case. However, this could be optimized by the implementation class to only perform that runtime check once (in the constructor).

Using the new interfaces

Finally, this would enable the user to directly utilize unmanaged function pointers in their code:

[UnmanagedCallersOnly(CallConvs = [typeof(CallConvCdecl)])]
public static void UnmanagedPrintLine(nint message)
{
    if (message != nint.Zero)
    {
        var str = Marshal.PtrToStringUni(message);
        Console.WriteLine(str);
    }
}

// ...

var action = IUnmanagedAction<string, nint>.FromFunctionPointer(&UnmanagedPrintLine);
action.Invoke("Hello World!"); // managed invocation through delegate
var str = Marshal.StringToCoTaskMemUni("Goodbye, Galaxy!");
action.AsCdeclPtr(str); // invoke through function pointer
Marshal.FreeCoTaskMem(str);

This enables the user to take full advantage of the performance of unmanaged function pointers without having to explicitly type out the function pointer type, which is error-prone. Being able to use the & address-of operator on the UnmanagedCallersOnly-attributed method and invoke the pointer through AsCdeclPtr is the entire appeal of this feature.

Note that the call to Invoke goes through a managed delegate which will invoke the built-in marshaller. Because we haven't supplied any custom marshalling, this will use default string marshalling. If we supplied, for example, a MarshalMap that maps all string arguments to a UnmanagedType.LPUTF8Str based MarshalAsAttribute, then Invoke would respect that marshalling behavior, and the method would need to un-marshal the message as UTF-8. The AsCdeclPtr invocation does not invoke any marshalling, which is why we must allocate and deallocate the memory for the message ourselves.

[monkey0506]

Facilitating unsafe APIs is conceptually sound, but this specific approach hasn't been tested. This would require checking the invocation site of the factory methods to determine whether the implementation class should implement the base interface or the unmanaged interface. This would require expanding the MethodReference by checking whether the factory method invocation is nested inside of one of the unmanaged interfaces. This could be simplified down to a check of whether the invocation is in the declarations file, as no existing methods in that file invoke the factory methods.

If the invocation site indicates that we need to implement the IUnmanagedAction/IUnmanagedFunc interface, then we would need to check the GenericSymbolReferenceTree for that interface to get the unmanaged (U-prefixed) generic type arguments.

Finally, implementation classes that implement the new interface would need to implement the new interface properties (as detailed above).