Destination is returning 404 when proxied

[brunocasarotti]

Hi, I created this very basic proxy that for some reason it is returning 404 for proxied calls. If I call the destination directly it works as expected. Can someone give me some light of what may be happening here?

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddReverseProxy()
    .LoadFromMemory(new[]
        {
            new RouteConfig
            {
                RouteId = "proxy-route",
                ClusterId = "proxy-cluster",
                Match = new RouteMatch
                {
                    Path = "{**catch-all}"
                }
            }
        },
        new[]
        {
            new ClusterConfig
            {
                ClusterId = "proxy-cluster",
                Destinations = new Dictionary<string, DestinationConfig>
                {
                    { "destination1", new DestinationConfig { Address = "https://rpc.edu-chain.raas.gelato.cloud/API KEY GOES HERE" } }
                }
            }
        });

var app = builder.Build();

app.MapReverseProxy();

app.Run();

This destination has a free endpoint where you can just omit the api key and it works. When omitted, the proxied call works but it doesn't when it is specified. Really confusing.

Here is an example of an http call, which should be a POST with the following body:

{
  "jsonrpc": "2.0",
  "method": "eth_getBlockReceipts",
  "params": [
    "0xf3c2a555d10781c6394433b1e287c02af6e3a04cfc6a3cb7b32ad2830fc20bfa"
  ],
  "id": 1
}

I have also noticed that if you specify a wrong api key in postman it returns unauthorized but if proxied it also returns 404.

I have this code running for other destinations and it works just fine. Could it be something related with this specific host?
Thanks in advance!

[MihaZupan]

First guess would be that the backend server is picky about differences in headers.
E.g. YARP includes X-Forwarded headers by default, you could try adding .WithTransformXForwarded(xDefault: ForwardedTransformActions.Off) on the route config.

If that doesn't work, try doing a network capture to see if/how requests differ.

[MihaZupan]

It's because of the trailing slash - #2757 (reply in thread). Your transform is trimming it.

https://rpc.edu-chain.raas.gelato.cloud/123   => 405
https://rpc.edu-chain.raas.gelato.cloud/123/  => 404

[brunocasarotti]

Did you run the project? I don't get this result if I run it. I removed the trailing slash at the end in my route and I am calling my host without any path. No matter what, I always get a 404.

using Microsoft.AspNetCore.HttpLogging;
using Yarp.ReverseProxy.Configuration;
using Yarp.ReverseProxy.Transforms;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddReverseProxy()
    .LoadFromMemory(new[]
        {
            new RouteConfig
            {
                RouteId = "proxy-route",
                ClusterId = "proxy-cluster",
                Match = new RouteMatch
                {
                    Path = "{**catch-all}"
                }
            }
        },
        new[]
        {
            new ClusterConfig
            {
                ClusterId = "proxy-cluster",
                Destinations = new Dictionary<string, DestinationConfig>
                {
                    { "destination1", new DestinationConfig { Address = "https://rpc.edu-chain.raas.gelato.cloud/123" } }
                }
            }
        })
    .AddTransforms(context =>
    {
        context.AddRequestTransform(transformContext =>
        {
            // transformContext.ProxyRequest.RequestUri = new Uri(transformContext.DestinationPrefix);
            return new ValueTask(Task.CompletedTask);
        });
    });
builder.Services.AddHttpLogging(opts => { opts.LoggingFields = HttpLoggingFields.All; });
var app = builder.Build();
app.UseHttpLogging();

app.MapReverseProxy();


app.Run();

[brunocasarotti]

From the logs I can see it is adding the trailing slash at the end but I don't have any transforms anymore:

info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[8]
      Duration: 236.3964ms
info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[1]
      Request:
      Protocol: HTTP/1.1
      Method: POST
      Scheme: http
      PathBase: 
      Path: /
      Accept: */*
      Connection: keep-alive
      Host: localhost:5016
      User-Agent: PostmanRuntime/7.43.0
      Accept-Encoding: gzip, deflate, br
      Content-Type: application/json
      Content-Length: 161
      Postman-Token: [Redacted]
info: Yarp.ReverseProxy.Forwarder.HttpForwarder[9]
      Proxying to https://rpc.edu-chain.raas.gelato.cloud/123/ HTTP/2 RequestVersionOrLower 
info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[3]
      RequestBody: {
        "jsonrpc": "2.0",
        "method": "eth_getBlockReceipts",
        "params": [
          "0x8096991e911a3559bd40a5e7d4b7b18e4240d71d1fbdd8e2bde4f98b896d61b7"
        ],
        "id": 1
      }
info: Yarp.ReverseProxy.Forwarder.HttpForwarder[56]
      Received HTTP/2.0 response 404.
info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[2]
      Response:
      StatusCode: 404
      Content-Type: text/plain; charset=utf-8
      Date: Thu, 20 Feb 2025 17:14:18 GMT
      Vary: [Redacted]
      Via: [Redacted]
      Content-Length: 19
      X-Content-Type-Options: [Redacted]
info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[4]
      ResponseBody: 404 page not found
      
info: Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware[8]
      Duration: 630.4625ms

[MihaZupan]

There is no such thing as an "empty path" here, it'll always be at least a single /.

And that one slash will be combined with your prefix, resulting in the trailing slash.
Your transform ignored the incoming request and just uses the prefix, hence no slash.

[brunocasarotti]

Ok got what you are saying now. Since it is set as a prefix, it will always add a / at the end. How can I achieve what I need then? I don't want to call yarp passing the api key in the path, instead, I want it to already know what api keys to use, from the route. That is the why I have specified it there. I didn't know about this. I assume I am configuring it wrongly then. Any suggestions on how to do it the proper way? Maybe storing it in the metadata of the route and apply the transform if it contains any value?

[brunocasarotti]

Maybe this would be a more idiomatic way of working with yarp? Am I over complicating this?

builder.Services.AddReverseProxy()
    .LoadFromMemory(new[]
        {
            new RouteConfig
            {
                RouteId = "proxy-route",
                ClusterId = "proxy-cluster",
                Match = new RouteMatch
                {
                    Path = "{**catch-all}"
                }
            }
        },
        new[]
        {
            new ClusterConfig
            {
                ClusterId = "proxy-cluster",
                Destinations = new Dictionary<string, DestinationConfig>
                {
                    {
                        "destination1", new DestinationConfig
                        {
                            Address = "https://rpc.edu-chain.raas.gelato.cloud",
                            Metadata = new Dictionary<string, string>
                            {
                                { "custom-path", "abc" }
                            }
                        }
                    }
                }
            }
        })
    .AddTransforms(context =>
    {
        context.AddRequestTransform(transformContext =>
        {
            if(context.Route.Metadata != null && context.Route.Metadata.TryGetValue("custom-path", out var customPath))
            {
                context.AddPathPrefix(customPath);
            }
            return new ValueTask(Task.CompletedTask);
        });
    });

[MihaZupan]

Am I over complicating this?

Yes :)
Your config was fine.

If your intention is to proxy everything to the same url with no path besides the API key, then your current solution of setting the RequestUri in a request transform is fine.

Note that an "empty" path is the special case here.
If your API expected host.com/apiKey/someAction, then a YARP request to /someAction would work.
It's only if it expects host.com/apiKey that there's a problem, because / is the minimum.

So if you envision having APIs like host.com/apiKey, host.com/apiKey/someAction, ..., change your transform to check the path as well, e.g.

if (transformContext.Path == "/")
{
    transformContext.ProxyRequest.RequestUri = new Uri(transformContext.DestinationPrefix);
}

[MihaZupan]

Or alternatively override just the Path:

if (transformContext.Path == "/")
{
    transformContext.Path = PathString.Empty;
}

and let YARP create the Uri. This would also let potential query strings work.