Distributed tracing enables you to track the flow of work that is performed by an application as it processes a request. This visibility can help you find errors and performance problems in your code. Tanzu Observability by Wavefront supports OpenTracing and OpenTelemetry for tracing.
In an application that consists of multiple services, an incoming request typically starts a chain of requests that are propagated from one service to the next. Distributed tracing gives you end-to-end visibility into that chain across services, even when those services are running in different environments.
You can use our tracing dashboards and browsers to:
- Monitor your application to make sure its response times are as expected.
- Troubleshoot and analyze reported errors.
- Pinpoint the specific operations that bottlenecks occur in.
Distributed Tracing Videos
Watch these videos to listen to our Co-founder Clement Pang introduce distributed tracing, and give updates including improved ingestion options and intelligent sampling:
Instrument Your Application
If your application uses an OpenTelemetry SDK, you can configure the application to send trace data to Tanzu Observability using any of the following options:
- Directly send OpenTelemetry data to the Wavefront proxy - [Recommended]
- Or use the OpenTelemetry Collector and the Wavefront proxy
An application must be instrumented for tracing before it can send trace data to Tanzu Observability. We support several options. Here’s the big picture:
Configure Your Already Instrumented Applications
If you have already instrumented your code with Jaeger or Zipkin, you can forward the trace data to the Wavefront proxy using a tracing integration. The proxy sends the data to the Wavefront service.
Configure Applications That are Not Instrumented
The Wavefront OpenTracing SDKs let you to choose how to send trace data to Wavefront – through a Wavefront proxy or directly to the Wavefront service. Using a Wavefront proxy is generally recommended.
If you have not yet instrumented your code, you can add instrumentation by using Wavefront OpenTracing SDKs or the Wavefront Java Tracing Agent.
Visualize Distributed Tracing Data
You can visualize the trace data that you collect from your instrumented application with several dashboards and browsers. We show information on your applications and services, and you can navigate from one dashboard to another to see more detail:
Examine Applications Using Application Status
Get an overview of how the applications and services are linked, understand the health of each service, and troubleshoot when your applications or services run into issues. You can get an overview and see the overall health of each application using the application map, table view and grid view. See Application Status.
Application Map view gives you an overview of how the applications and services are linked. You can focus on a specific service, view Request, Error, and Duration (RED) metrics for each service and see the tracing traffic, including the traffic direction.
Table view lists the applications and services in the form of a table. You can see the Request, Error, and Duration (RED) metrics at a glance and sort the data.
Grid view lists the application and services in a grid. You can see the RED metrics for each of the application’s services.
Examine Application RED Metrics Using Service Dashboard
The default, read-only dashboard for a service lets you explore trace data sent by each service in your application.
Examine Operation RED Metrics Using Operation Dashboard
The Operation Dashboard shows the RED metrics for each operation. You can view data for each operations using the filters and drill down to the traces browser.
Examine Traces Using Traces Browser
The Traces Browser supports a streamlined task flow for examining traces. You can perform trace queries, view query results, expand traces to see their member spans, and expand individual spans to see their details without having to navigate between pages and pop-ups.
Examine Downloaded Traces Using Offline Traces
You can export traces you view in the Traces Browser, save them locally as JSON files, and view them later using Offline Traces.
Create a Sampling policy
Our intelligent sampling algorithm reduces the volume of ingested traces. The goals of intelligent sampling are to retain traces that are likely to be informative. But sometimes intelligent sampling discards traces that you want to keep. You can create a sampling policy to fine-tune intelligent sampling and let the Wavefront service know that you want to keep certain spans.
Configure Applications Using Application Configuration
The Apdex score helps you compare the response time of a service based on the response time threshold that you define. Define the response time threshold for each service using the Application Configuration page.
- Familiarize yourself with the tracing concepts. See Tracing Concepts in Wavefront for details.
- A large-scale web application can produce a high volume of traces. Many traces might be reported every minute, and each trace might consist of many spans, each with many tags. We limit the volume of trace data by performing intelligent sampling.