Monitoring Nginx traces with ClickStack

The easiest way to add tracing to Nginx is using the official Nginx image with OpenTelemetry support built-in.

Using the nginx:otel image

Replace your current Nginx image with the OpenTelemetry-enabled version:

# In your docker-compose.yml or Dockerfile
image: nginx:1.27-otel

This image includes the ngx_otel_module.so pre-installed and ready to use.

Add OpenTelemetry configuration to your nginx.conf file. The configuration loads the module and directs traces to ClickStack’s OTLP endpoint.

First, get your API key:

1. Open HyperDX at your ClickStack URL
2. Navigate to Settings → API Keys
3. Copy your Ingestion API Key
4. Set it as an environment variable: export CLICKSTACK_API_KEY=your-api-key-here

Add this to your nginx.conf:

load_module modules/ngx_otel_module.so;

events {
    worker_connections 1024;
}

http {
    # OpenTelemetry exporter configuration
    otel_exporter {
        endpoint <clickstack-host>:4317;
        header authorization ${CLICKSTACK_API_KEY};
    }
    
    # Service name for identifying this nginx instance
    otel_service_name "nginx-proxy";
    
    # Enable tracing
    otel_trace on;
    
    server {
        listen 80;
        
        location / {
            # Enable tracing for this location
            otel_trace_context propagate;
            otel_span_name "$request_method $uri";
            
            # Add request details to traces
            otel_span_attr http.status_code $status;
            otel_span_attr http.request.method $request_method;
            otel_span_attr http.route $uri;
            
            # Your existing proxy or application configuration
            proxy_pass http://your-backend;
        }
    }
}

If running Nginx in Docker, pass the environment variable to the container:

services:
  nginx:
    image: nginx:1.27-otel
    environment:
      - CLICKSTACK_API_KEY=${CLICKSTACK_API_KEY}
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro

Replace <clickstack-host> with your ClickStack instance hostname or IP address.

Understanding the configuration

What gets traced: Each request to Nginx creates a trace span showing:

• Request method and path
• HTTP status code
• Request duration
• Timestamp

Span attributes: The otel_span_attr directives add metadata to each trace, allowing you to filter and analyze requests in HyperDX by status code, method, route, etc.

After making these changes, test your Nginx configuration:

nginx -t

If the test passes, reload Nginx:

# For Docker
docker-compose restart nginx

# For systemd
sudo systemctl reload nginx

Once configured, log into HyperDX and verify traces are flowing, you should see something like this, if you don’t see traces, try adjusting your time range:

If you don’t have ClickStack running yet, start it with:

docker run --name clickstack-demo \
  -p 8080:8080 -p 4317:4317 -p 4318:4318 \
  clickhouse/clickstack-all-in-one:latest

Wait about 30 seconds for ClickStack to fully initialize before proceeding.

• Port 8080: HyperDX web interface
• Port 4317: OTLP gRPC endpoint (used by nginx module)
• Port 4318: OTLP HTTP endpoint (used for demo traces)

Download the sample traces file and update timestamps to the current time:

# Download the traces
curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/nginx-traces-sample.json

The dataset includes:

• 1,000 trace spans with realistic timing
• 9 different endpoints with varied traffic patterns
• ~93% success rate (200), ~3% client errors (404), ~4% server errors (500)
• Latencies ranging from 10ms to 800ms
• Original traffic patterns preserved, shifted to current time

Set your API key as an environment variable (if not already set):

export CLICKSTACK_API_KEY=your-api-key-here

Get your API key:

1. Open HyperDX at your ClickStack URL
2. Navigate to Settings → API Keys
3. Copy your Ingestion API Key

Then send the traces to ClickStack:

curl -X POST http://localhost:4318/v1/traces \
  -H "Content-Type: application/json" \
  -H "Authorization: $CLICKSTACK_API_KEY" \
  -d @nginx-traces-sample.json

You should see a response like {"partialSuccess":{}} indicating the traces were successfully sent. All 1,000 traces will be ingested into ClickStack.

1. Open HyperDX and log in to your account (you may need to create an account first)
2. Navigate to the Search view and set the source to Traces
3. Set the time range to 2025-10-25 13:00:00 - 2025-10-28 13:00:00

Here’s what you should see in your search view:

Download the dashboard configuration file.

1. Open HyperDX and navigate to the Dashboards section.
2. Click “Import Dashboard” in the upper right corner under the ellipses.

3. Upload the nginx-trace-dashboard.json file and click finish import.