Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dynamic-sampling): organization settings #11791

Merged
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
714bbef
feat(dynamic-sampling): first draft of organization settings
constantinius Nov 12, 2024
4092b56
fix(dynamic-sampling): improved overview and description of dynamic s…
constantinius Nov 13, 2024
cf6ed25
fix(dynamic-sampling): use suggestions
constantinius Nov 14, 2024
c9eab8e
fix(dynamic-sampling): use suggestions
constantinius Nov 14, 2024
0559a31
fix(dynamic-sampling): use suggestions
constantinius Nov 14, 2024
778beb6
fix(dynamic-sampling): add screenshot and elaborate on SDK sampling d…
constantinius Nov 14, 2024
c73f242
docs(dynamic-sampling): move retention priorities under dynamic-sampling
constantinius Nov 14, 2024
3a07833
fix(dynamic-sampling): renaming retention policies to (dynamic) sampl…
constantinius Nov 14, 2024
a872516
docs(dynamic-sampling): better distinction between SDK and dynamic sa…
constantinius Nov 15, 2024
aa9d35c
fix(dynamic-sampling): working in feedback on distinction of sampling…
constantinius Nov 18, 2024
0405349
fix(dynamic-sampling): more feedback taken into account
constantinius Nov 18, 2024
a1a33fb
feat(dynamic-sampling): work in feedback
constantinius Nov 20, 2024
4795538
Update docs/organization/dynamic-sampling/index.mdx
constantinius Nov 22, 2024
797efb9
Update docs/organization/dynamic-sampling/index.mdx
constantinius Nov 22, 2024
d5cae09
Update docs/organization/dynamic-sampling/index.mdx
constantinius Nov 22, 2024
1b212fc
fix(dynamic-sampling): disabling including graphic, as it will be rep…
constantinius Nov 22, 2024
70990f8
liza mock updates
lizokm Nov 25, 2024
11ba516
docs(dynamic-sampling): several updates
constantinius Jan 28, 2025
be109c7
Format
jan-auer Jan 28, 2025
6e2c87e
Merge branch 'master' into constantinius/feat/dynamic-sampling/organi…
jan-auer Jan 28, 2025
aba7b8b
Replace <Note> with <Alert>
jan-auer Jan 28, 2025
f334b04
docs(dynamic-sampling): set focus on target sample rate
constantinius Jan 28, 2025
3c7a5c6
docs(dynamic-sampling): update screenshots from actual UI
constantinius Jan 30, 2025
df915d9
fix(dynamic-sampling): adding image for distributed traces
constantinius Feb 4, 2025
8307844
fix(dynamic-sampling): updating image for distributed spans
constantinius Feb 4, 2025
81e1f08
fix(dynamic-sampling): fix wording around AM2/AM3 availabilitiy
constantinius Feb 4, 2025
8aa5890
fix: Wording
jan-auer Feb 5, 2025
60ac005
feat: Better images
jan-auer Feb 5, 2025
0dc606b
ref: Rearrange modes
jan-auer Feb 5, 2025
45ad919
fix: Replace transactions and restructure biases section
jan-auer Feb 5, 2025
5550bb0
fix: Title case
jan-auer Feb 6, 2025
704ab13
Merge branch 'master' into constantinius/feat/dynamic-sampling/organi…
jan-auer Feb 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/concepts/key-terms/tracing/trace-view.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ Also, in these cases you can click "Open In Discover" to see all the events in t

Broken subtraces may be caused by:

- SDK sampling. Setting a sample rate that's too low may prevent the SDK from sending a transaction. We recommend [sending us all of your transaction data](/product/performance/retention-priorities/#deciding-on-your-sdk-sample-rate).
- SDK sampling. Setting a sample rate that's too low may prevent the SDK from sending a transaction. We recommend [sending us all of your transaction data](/organization/dynamic-sampling/#deciding-on-your-sdk-sample-rate).
- [Ad blockers](/platforms/javascript/troubleshooting/#dealing-with-ad-blockers) may prevent transactions in browsers being sent, but HTTP requests to backend projects will still create child transactions
- [Rate-limiting](/product/accounts/quotas/#limiting-events) on a project may cause only some events to be sent to Sentry
- [Project permissions](/product/accounts/membership/#restricting-access) may mean you do not have access to transactions in another project
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,18 +1,54 @@
---
title: Retention Priorities
sidebar_order: 130
description: "Learn how Sentry determines retention priorities and how to update them."
title: Dynamic Sampling
sidebar_order: 30
description: "Learn how to make your billing quota go farther with Sentry's Dynamic Sampling."
---

## Overview

High-volume projects that generate many events can quickly consume a large portion of your billing quota, leaving less room for other projects and decreasing visibility across them.

With high-volume projects, you'll often see many similar events, which can be redundant and less helpful for debugging. But with Dynamic Sampling, you can maximize your quota by adjusting sample rates based on each project’s event volume. The sample rate is a percentage from 0% (no events passed) to 100% (all events passed), that can be tailored to each project’s needs.

When you use Dynamic Sampling, projects in your organization are sampled based on their recent event volumes. High-volume projects are sampled at lower rates while lower-volume projects are sampled at higher rates or not sampled at all, based on your recent event data. This makes it possible for each project to retain a meaningful set of key events, improving visibility into all projects regardless of their size.

![The dynamic sampling mode and target sample rate setting.](./img/mode-and-target-sample-rate.png)

## Modes

You can choose between two Dynamic Sampling modes for your organization: manual or automatic.

* **`automatic`** (default): In automatic mode, the sample rate for each project is set automatically based on relative project volume, so events fit within your organization’s billing quota. This boosts visibility into lower-volume projects. To account for any changes in event volume, sample rates are recalibrated on a regular basis. Automatic mode overrides any manually-set project sample rates.

* **`manual`**: If you want to set the exact sample rate for each project in your organization yourself, choose manual mode. Note, that in this mode, there is no automatic recalibration.


[]: # (TODO: Rename AM3 with the actual user-facing name.)

<Note>

Retention priorities are only available on the latest version of our plans, starting from 1M reserved transactions. Anyone in your org with owner or billing-level permissions can update your plan by going to Settings > Subscription, and clicking the "Manage Subscription" button.
Please note that these Dynamic Sampling modes are only available for customers on the AM3 platform.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call out that on AM2 there's also DS, the sample rate is chosen by the system based on transaction volume (1M+), and you still have controls for the sampling priorities?


</Note>

While storing all your data makes sense at relatively low volumes, as your application scales, storing a raw copy of all your data has diminishing returns. When data reaches high volumes, Sentry begins to automatically prioritize retaining certain transactions over others with server-side data retention strategies called retention priorities.

It's important to note that even when we begin to only store the most valuable data based on retention priorities, the [performance metrics](/product/performance/metrics/) you see are still based on **all** the events you send to Sentry. This ensures that you get a full and complete view of your application's health.
## Differences to Sampling in the Sentry SDKs

Dynamic Sampling is different from the sampling that you can configure in the Sentry SDKs. The SDK sampling rate is applied before events are sent to Sentry, while Dynamic Sampling is applied after events are received by Sentry. This means that Dynamic Sampling can be used to adjust the sample rate for events that have already been sent to Sentry and only take into account the volume of events that have been received by Sentry.

Also in contrast to the Dynamic Sampling, the SDK sample rate is never dynamically adjusted.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly speaking, "never" isn't true :) traces_sampler allows customers to write their own code for dynamically adjusting sample rates. I think the true differences can rather be summarized as:

  • Sampling in SDKs affects how much is captured and sent to Sentry. For high throughput scenarios, this may be necessary to reduce instrumentation overheads.
  • To change sample rates in SDKs, one has to update the traces_sample_rate or traces_sampler callback, which usually requires a new deployment or release.
  • With dynamic sampling, one can change sample rates without touching the application or SDK. This affects the amount of data stored at Sentry, however, it will not have an effect on how much is sent from the SDKs.
  • Dynamic sampling continuously monitors incoming traffic patterns and adjusts fine-grained sample rates within the projects to prioritize more important data. See the sampling priorities for examples.


## Dynamic Sampling Priorities

<Note>

Dynamic Sampling priorities are only available on the latest version of our plans, starting from 1M reserved transactions. Anyone in your org with owner or billing-level permissions can update your plan by going to Settings > Subscription, and clicking the "Manage Subscription" button.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be updated. DS priorities are available on AM2 and AM3 (right now this is still correct, as AM2 is latest).


</Note>

While storing all your data makes sense at relatively low volumes, as your application scales, storing a raw copy of all your data has diminishing returns. When data reaches high volumes, Sentry begins to automatically prioritize retaining certain transactions over others with server-side data sampling strategies called dynamic sampling priorities.

It's important to note that even when we begin to only store the most valuable data based on dynamic sampling priorities, the [performance metrics](/product/performance/metrics/) you see are still based on **all** the events you send to Sentry. This ensures that you get a full and complete view of your application's health.

## Prerequisites

Expand All @@ -32,8 +68,6 @@ It's important to note that even when we begin to only store the most valuable d
- .NET: 3.22.0 or later
- Go: 0.16.0 or later

## Retention Priorities

Below is a list of the transactions we prioritize and deprioritize. They are enabled by default, but can be updated on a per-project basis to better fit your organization's needs.

### Prioritized Transactions
Expand All @@ -42,7 +76,7 @@ Sentry automatically prioritizes and retains the following types of transactions

#### Latest Release

Latest releases are a retention priority because when you create a new release, we assume you'll want to have more visibility during the early adoption phase, which improves your ability to catch new issues as your release is gaining traction.
Latest releases are a dynamic sampling priority because when you create a new release, we assume you'll want to have more visibility during the early adoption phase, which improves your ability to catch new issues as your release is gaining traction.
Because you may want to deploy the same release in different environments, we'll also take your environment into account and prioritize those transactions.

#### Dev Environments
Expand All @@ -59,7 +93,7 @@ Since dev environments usually generate a small number of transactions as compar

<Note>

You won't be able to disable the Low Volume Projects retention priority because it's set on an organization-wide level, not on a per-project basis.
You won't be able to disable the Low Volume Projects dynamic sampling priority because it's set on an organization-wide level, not on a per-project basis.

</Note>

Expand All @@ -72,7 +106,7 @@ We prioritize low-volume transactions for the same reason we prioritize low-volu

### Investigation Mode

Our automated retention priorities work well in a generic manner to collect a baseline of samples because they prioritize retaining data that is valuable for any customer at any given time. However, there are certain scenarios, such as investigating a particular issue, when very specific data becomes temporarily more important. In such cases, sometimes the automated sampling priorities do not provide enough samples, so you may want to create investigation rules.
Our automated dynamic sampling priorities work well in a generic manner to collect a baseline of samples because they prioritize retaining data that is valuable for any customer at any given time. However, there are certain scenarios, such as investigating a particular issue, when very specific data becomes temporarily more important. In such cases, sometimes the automated sampling priorities do not provide enough samples, so you may want to create investigation rules.

Investigation rules are sampling rules that can be created through the Sentry UI to temporarily boost the number of transactions collected for a particular scenario (for example, transactions with specific tag values). These rules can be created to retain more samples when too few (less than 5) are available. Once created, a rule will be active for up to 48 hours or until 100 samples are collected. You'll be notified via email when enough transactions have been collected.

Expand Down Expand Up @@ -120,9 +154,9 @@ We deprioritize health check type transactions because while they're important f
- `*/readyz`
- `*/ping`

### How to Change Retention Priorities
### How to Change Dynamic Sampling Priorities

To make changes to a project's retention priorities in Sentry, go to **Settings > Projects**, select the project you're interested in, then scroll down to "Performance" in the "PROCESSING" section and make any updates under "RETENTION PRIORITIES" using the toggles. Each of your projects can have an individual set of retention priorities.
To make changes to a project's dynamic sampling priorities in Sentry, go to **Settings > Projects**, select the project you're interested in, then scroll down to "Performance" in the "PROCESSING" section and make any updates under "DYNAMIC SAMPLING PRIORITIES" using the toggles. Each of your projects can have an individual set of dynamic sampling priorities.

<Arcade src="https://demo.arcade.software/bEv0SO2UEmqbVJVok99o?embed" />

Expand Down
2 changes: 1 addition & 1 deletion docs/organization/early-adopter-features/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ Limitations:
- [Issue Views](/product/issues/issue-views/)
- [Issue Status](/product/issues/states-triage/) tags
- [Span Summary](/product/performance/transaction-summary/#span-summary)
- [Investigation Mode](/product/performance/retention-priorities/#investigation-mode) for retention priorities in Tracing
- [Investigation Mode](/organization/dynamic-sampling/#investigation-mode) for sampling priorities in Tracing
- [Uptime Monitoring](/product/alerts/uptime-monitoring/)
- [Dynamic Alerts](/product/alerts/create-alerts/metric-alert-config/#dynamic-alerts)
2 changes: 1 addition & 1 deletion docs/product/explore/discover-queries/query-builder.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Click on any of these sections to further refine your search. For example, click

When performing searches on transaction events, based on your search criteria
and sample rate, the events available may be limited because Discover uses
sampled data only. Learn more by reading about [our retention priorities](/product/performance/retention-priorities/).
sampled data only. Learn more by reading about [our sampling priorities](/organization/dynamic-sampling/).

</Note>

Expand Down
2 changes: 1 addition & 1 deletion docs/product/explore/traces/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ For a list of built-in key fields used in Sentry, check out our [Searchable Prop

<Note>

Trace Explorer uses sampled data. Your search results may be limited based on sampling rates and retention policies. Learn more about [retention priorities](/product/performance/retention-priorities/).
Trace Explorer uses sampled data. Your search results may be limited based on sampling rates and retention policies. Learn more about [sampling priorities](/organization/dynamic-sampling/).

</Note>

Expand Down
Loading