Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making use of LibraryImport to reduce marshalling, smaller optimizations #194

Open
kirides opened this issue Jun 11, 2024 · 3 comments
Open
Labels
good first issue Good for newcomers

Comments

@kirides
Copy link

kirides commented Jun 11, 2024

This project may benefit a lot from using LibraryImport-Attribute instead of DllImport (or combined with? idk if and how fallback would work)

As DuckDb is in Process, reducing the marshalling overhead by introducing LibraryImport may further increase throughput.

Side note:
Things like StringVectorDataReader may also benefit from using string.Create(....) to reduce allocations to just a single "string", removing the additional new char[...] allocation.

Also DuckDB.NET.Data.Extensions.GuidConverter may benefit a lot from using Span<char> buffer = stackalloc char[36] to build the guid-representation in memory without allocating at all.

private static readonly char[] HexDigits = "0123456789abcdef".ToCharArray();
public static Guid ConvertToGuid_Optimized(DuckDB.NET.Native.DuckDBHugeInt input)
{
    Span<char> buffer = stackalloc char[32];
    long num = input.Upper ^ long.MinValue;
    int position = 0;
    ByteToHex(buffer, ref position, (ulong)((num >> 56) & 0xFF));
    ByteToHex(buffer, ref position, (ulong)((num >> 48) & 0xFF));
    ByteToHex(buffer, ref position, (ulong)((num >> 40) & 0xFF));
    ByteToHex(buffer, ref position, (ulong)((num >> 32) & 0xFF));
    //buffer[position++] = '-';
    ByteToHex(buffer, ref position, (ulong)((num >> 24) & 0xFF));
    ByteToHex(buffer, ref position, (ulong)((num >> 16) & 0xFF));
    //buffer[position++] = '-';
    ByteToHex(buffer, ref position, (ulong)((num >> 8) & 0xFF));
    ByteToHex(buffer, ref position, (ulong)(num & 0xFF));
    //buffer[position++] = '-';
    ByteToHex(buffer, ref position, (input.Lower >> 56) & 0xFF);
    ByteToHex(buffer, ref position, (input.Lower >> 48) & 0xFF);
    //buffer[position++] = '-';
    ByteToHex(buffer, ref position, (input.Lower >> 40) & 0xFF);
    ByteToHex(buffer, ref position, (input.Lower >> 32) & 0xFF);
    ByteToHex(buffer, ref position, (input.Lower >> 24) & 0xFF);
    ByteToHex(buffer, ref position, (input.Lower >> 16) & 0xFF);
    ByteToHex(buffer, ref position, (input.Lower >> 8) & 0xFF);
    ByteToHex(buffer, ref position, input.Lower & 0xFF);
    return Guid.ParseExact(buffer, "n");

    static void ByteToHex(Span<char> buffer, ref int position, ulong value)
    {
        buffer[position++] = HexDigits[(value >> 4) & 0xF];
        buffer[position++] = HexDigits[value & 0xF];
    }
}
@Giorgi
Copy link
Owner

Giorgi commented Jun 11, 2024

I have been planning to look into LibraryImport for quite some time but didn't get to it. Hopefully I will find time for it in the next few weeks.

Other suggestions look interesting too. If you want to feel free to send a PR and I will be happy to review it.

@Giorgi
Copy link
Owner

Giorgi commented Jun 13, 2024

@kirides Implemented Guid and BitString suggestions in c83fafa

@Giorgi
Copy link
Owner

Giorgi commented Jun 14, 2024

LibraryImport requires #ifdef-s to check for .net 7 as it is unavailable in the previous versions. @kirides Want to send a PR?

@Giorgi Giorgi added the good first issue Good for newcomers label Jun 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants