How to send cross-service OpenTelemetry traces from Python to Jaeger: end-to-end setup with Docker Compose

In this tutorial, you will

• Create a local OpenTelemetry collector and Jaeger setup using Docker Compose

• Build a Python Flask Server Application and instrument it with OpenTelemetry

• Build a Python HTTP Client Application and instrument it with OpenTelemetry

• Learn how to use context propagation to traces cross-service requests

• Trace your cross-service requests in Jaeger

Tools we will use

Jaeger is an open-source distributed tracing platform. In this tutorial, we will use a version that contains all components in a single Docker image. It will enable a fast and convenient way to display distributed traces on a local machine.

Docker Compose is a tool for defining and running multi-container applications. It allows us to describe all Docker images in a single configuration file, making it handy for local testing.

We use OpenTelemetry Collector to forward traces from the Python applications to Jaeger. In this case, we could manage without it, however, the OpenTelemetry Collector is useful, for example, if you want to enrich data, or collect not only traces but also metrics, or experiment with different vendors.

Pre-requisites

We will use the Unix command interface. If you use Windows, you can use WSL.

Install the following programs if you don’t have them

1. Docker Compose

2. Python3 and pip

Set up Jaeger and OpenTelemetry Collector using Docker Compose

1. Create Collector config file

    cat > otel-collector-config.yaml << 'EOF'
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
    processors:
    extensions:
      health_check: {}
    exporters:
      otlp:
        endpoint: jaeger:4317
        tls:
          insecure: true
    service:
      pipelines:
        traces:
          receivers: [otlp]
          exporters: [otlp]
    EOF
   

2. Create a Docker compose file

    cat > docker-compose.yaml << 'EOF'
    services:
      otel-collector:
        image: otel/opentelemetry-collector-contrib:latest
        command: ["--config=/etc/otel-collector-config.yaml"]
        volumes:
          - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
        ports:
          - "4317:4317" # OTLP gRPC receiver
      jaeger:
        image: jaegertracing/all-in-one:latest
        ports:
          - "6831:6831/udp" # UDP port for Jaeger agent
          - "16686:16686" # Web UI
          - "14268:14268" # HTTP port for spans(venv)
    EOF
   

3. Start Docker Compose:

    docker compose -f docker-compose.yaml up
   

4. Verify that the local Jaeger instance works: open http://localhost:16686/ in your browser.

Create a virtual environment and install Python libraries

1. Create and activate a virtual environment

    python3 -m venv venv
    source ./venv/bin/activate
   

2. Install python packages

    pip install flask
    pip install urllib3
   

3. Install OpenTelemetry instrumentation

    pip install opentelemetry-distro
    opentelemetry-bootstrap -a install
    pip install opentelemetry-exporter-otlp-proto-grpc # send traces over OTLP
    pip install opentelemetry-instrumentation-urllib3 # instrumentation for urllib3 library
   

   Instrument a cross-server request

   1. Create a simple Flask server application. It will serve HTTP requests.

      ```bash mkdir server cat > server/app.py << 'EOF' from flask import Flask, jsonify

      app = Flask(name)

      @app.route('/example1/') def trace(arg): return jsonify({"trace": f"Trace argument is {arg}"})

if name == "main": app.run(host="0.0.0.0", port=8080, debug=True) EOF

    2. Start the Flask server application with OpenTelemetry instrumentation

        ```bash
        cd server && \
        opentelemetry-instrument \
          --service_name demo-server \
          --metrics_exporter none \
          --logs_exporter none \
          flask run -p 8080

3. Verify that your Flask application works: open http://localhost:8080/example1/test in your browser

4. Create a simple client application. We will use urllib3 to make HTTP requests and URLLib3Instrumentor for OpenTelemetry instrumentation. Create a file simple-client.py with following content

    import urllib3
    from opentelemetry.instrumentation.urllib3 import URLLib3Instrumentor
   
    def strip_query_params(url: str) -> str:
        return url.split("?")[0]
   
    URLLib3Instrumentor().instrument(
        # Remove all query params from the URL attribute on the span.
        url_filter=strip_query_params,
    )
   
    http = urllib3.PoolManager()
    response = http.request("GET", "http://localhost:8080/example1/test")
   
    if response.status == 200:
        print("Response:", response.json())
    else:
        print("Error:", response.status_code)
   

5. Run the client application

    opentelemetry-instrument \
      --service_name demo-client \
      --metrics_exporter none \
      --logs_exporter none \
    python simple-client.py
   

6. Verify the results in Jaeger: open http://localhost:16686/, choose “demo-client” in the field Service in the left panel, and click “Find Traces

7. 

   Now, you should see cross-service traces in the search results. When you open a trace, you can see how much time it took at each stage:

Congratulations! You’ve just implemented your first cross-service trace in OpenTelemetry.

Clean up

It is the optional step if you want to clean up the environment on your machine.

1. Clean up the Python virtual environment

    deactivate
    rm -rf venv/
   

2. Shut down Docker Compose components

    docker compose -f docker-compose.yaml down
   

Tip

Prefer instrumented Python libraries to generate telemetry data. For example, we used the instrumented library urllib3 to produce the traces rather than manually instrumenting with OpenTelemetry in this tutorial. You can find the full list of instrumented libraries here.