Understanding Trace Sampling and Span Retention in Sentry.io
Trace sampling and span retention are critical components of Sentry.io's performance monitoring capabilities. This article explains the mechanisms behind trace sampling, long-term span retention, and how users can customize sampling logic to meet their needs.
Overview of Sampling Mechanisms
Sentry.io employs two primary sampling mechanisms:
Ingest/Trace Sampling: This is head-based and determined per trace at the root level. Child spans inherit the sampling decision of their parent transaction. If the root span is dropped, all child spans are also dropped.
Long-term Retention Downsampling: This applies to spans stored in the long-term sampled store (up to 13 months). Unlike ingest sampling, spans in the same trace can be retained or dropped independently, which may result in incomplete traces for older data.
Retention Policies and Sampling Percentages
Retention Windows: Business and Enterprise plans provide a 30-day full-fidelity retention period and up to 13 months of sampled retention. This does not apply to Team or Business plans that use transaction-based billing, where transaction retention is 90 days and sampled retention is not available. However, there is no fixed sampling percentage for the 13-month period, and customers cannot modify this rate.
Sampling Percentages: Under the current implementation, the highest-fidelity long-term tier retains approximately 1/8 (12.5%) of spans that successfully reach Sentry’s storage. This percentage is not a contractual guarantee and applies after any SDK sampling or ingestion drops.
Querying Older Data: Queries involving data older than ~30 days are routed to the long-term sampled store, starting at the 1/8 tier. The tier used depends on the volume of spans ingested for the queried projects during the queried time range, with thresholds for deeper tiers being very high (by default, 1/64 above ~10 billion ingested spans and 1/512 above ~100 billion, and the thresholds can be overridden per organization).
Custom Sampling Logic with tracesSampler
To implement custom sampling logic, use the tracesSampler callback. This allows you to make per-transaction sampling decisions based on specific criteria. For example:
To sample 100% of critical journeys while sampling 1% of other transactions, return
1.0for critical journeys and0.01for others in thetracesSamplerlogic.If both
tracesSampleRateandtracesSamplerare set, the result fromtracesSamplertakes precedence. Avoid dynamically adjusting the globaltracesSampleRateas it affects unrelated transactions and only applies when a transaction starts.
FAQs on Trace Sampling and Span Visibility
Can child spans be retained if the root span is dropped?
No, child spans inherit the sampling decision of the root span. If the root span is dropped, all child spans are also dropped. The one exception is a distributed trace where a downstream service's tracesSampler explicitly overrides the incoming parent decision; Sentry advises against this because it breaks distributed traces.
Will automatic spans be sampled at 1% while custom critical journeys are at 100%?
No, sampling decisions are made per transaction at its start. Child spans inherit the parent transaction’s sampling decision. To ensure a transaction is sampled at 100%, the identifying signal must be available to tracesSampler when the transaction is created.
Conclusion
Understanding and configuring trace sampling and span retention in Sentry.io allows users to optimize data visibility and storage. By leveraging mechanisms like tracesSampler, users can implement tailored sampling strategies to meet their specific needs.