Memory leak for Express requests with large attached objects

[john-matroid]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/node

SDK Version

9.2.0

Framework Version

Node 20.16.3

Link to Sentry event

No response

Reproduction Example/SDK Setup

We're seeing a Sentry-related memory leak in an Express web server. I suspect the leak has been present for quite some time, but it became more apparent recently when we started attaching large objects (a couple hundred KB to a few MB) to some of our Express requests.

The issue feels similar to #14965, but the retention stack is slightly different, and the issue was not resolved by upgrading to the latest version of the SDK. Feel free to mark as duplicate and reopen there.

Versions:

• Node: 20.16.3
• Sentry SDK: 9.2.0
• Express: 4.21.2

Sentry usage -- fairly vanilla initialization:

Sentry.init({
  dsn: process.env.SENTRY_DSN,
  release: process.env.VERSION,
  environment: process.env.ENV_NAME,
  integrations: [
    nodeProfilingIntegration(),
  ],
});

const app = express();
Sentry.setupExpressErrorHandler(app);

/* Other express app setup */

Adding a touch of context in some other error handling middleware, leak is still present if I remove this:

if (user) {
    Sentry.setUser({
      id: user.id ? user.id : user.toString(),
      username: user.name && `${user.name.first} ${user.name.last}`,
      email: user.email,
    });
}

Sample retention stack:

Unfortunately I can't provide a full heapdump or source code from our app to reproduce.

Steps to Reproduce

Sorry, I know how frustrating it is to not get perfect reproduction steps, but I can't share source code. I can take a stab at a minimal reproducing setup if I have some spare time. I'd start by:

• Creating an Express app that attaches some large objects to requests
• Occasionally throwing errors to be caught by Sentry
• Running some non-trivial request load

Expected Result

Flat memory usage

Actual Result

Rising memory usage. Sentry 7.12x.x version is 7.120.1:

[AbhiPrasad]

Thanks for writing in @john-matroid

that attaches some large objects to requests

What would help is to get an idea of what these large objects are. Are they holding references to buffers or similar?

The timeouts in the stack are pointing me to our request session logic, but considering all of this is new logic in v9, it feels strange that v7 also has this problem:

sentry-javascript/packages/node/src/integrations/http/SentryHttpInstrumentation.ts

Line 470 in 4a7aa21

export function recordRequestSession({

Can you confirm that 9.2.0 actually leaks? Perhaps the memory usage just goes up but then becomes flat. The increased memory usage is not ideal, but feels better than an OOM.

[john-matroid]

Appreciate the quick reply here. The objects are ABAC-related permissions objects created by CASL. A lot of the memory use boils down to arrays of objects with a handful of short strings as values.

Happy to run 9.2.0 longer to see if the leak is persistent. In terms of the retention pointers, here's what it looked like for 7.120:

[AbhiPrasad]

We might end up having to delete the request manually off the sdk processing metadata once the request is over.

@timfish does that seem reasonable to you?

[timfish]

@timfish does that seem reasonable to you?

Yep that seems reasonable!