docs: improve OTel monitoring doc with Quick Start guide and VS Code settings examples (#4243)

zhichli · web-flow · commit 53602fcd69d5 · 2026-03-06T18:15:53.000Z
- Replace env var Quick Start with step-by-step Aspire Dashboard guide
- Add concise intro explaining what the Aspire Dashboard is
- Convert all example configurations from env vars to VS Code settings JSON
- Keep env var reference table for official documentation
- Note where env vars are still required (e.g. auth headers)
diff --git a/docs/monitoring/agent_monitoring.md b/docs/monitoring/agent_monitoring.md
@@ -6,24 +6,54 @@ All signal names and attributes follow the [OTel GenAI Semantic Conventions](htt
 
 ## Quick Start
 
-Set these environment variables before launching VS Code:
+The fastest way to see Copilot Chat traces locally — no cloud account required. This guide uses the [Aspire Dashboard](https://aspire.dev/dashboard/standalone/), a lightweight container image from Microsoft that provides a trace viewer with a built-in OTLP endpoint. It can be used standalone, without the rest of .NET Aspire.
+
+### Prerequisites
+
+- **Docker** installed
+- **VS Code** with the GitHub Copilot Chat extension
+
+### 1. Start the Aspire Dashboard
 
 ```bash
-# Enable OTel and point to a local collector
-export COPILOT_OTEL_ENABLED=true
-export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
+docker run --rm -d \
+  -p 18888:18888 \
+  -p 4317:18889 \
+  --name aspire-dashboard \
+  mcr.microsoft.com/dotnet/aspire-dashboard:latest
+```
+
+This exposes the dashboard UI on port `18888` and an OTLP (gRPC) endpoint on port `4317`.
 
-# Launch VS Code
-code .
+### 2. Configure VS Code
+
+Open **Settings** (`Ctrl+,`) and add:
+
+```json
+{
+  "github.copilot.chat.otel.enabled": true,
+  "github.copilot.chat.otel.exporterType": "otlp-grpc",
+  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4317"
+}
 ```
 
-That's it. Traces, metrics, and events start flowing to your collector.
+> **Note:** You can also use environment variables instead of VS Code settings (see [Configuration](#configuration)). Environment variables always take precedence.
+
+### 3. Generate Telemetry
+
+Open Copilot Chat and send any message — for example, ask a question in Agent mode.
+
+### 4. View Traces
+
+Open http://localhost:18888 → **Traces**. You'll see `invoke_agent` spans with nested `chat` and `execute_tool` children.
 
-> **Tip:** To get started quickly with a local trace viewer, run [Jaeger](https://www.jaegertracing.io/) in Docker:
-> ```bash
-> docker run -d --name jaeger -p 16686:16686 -p 4318:4318 jaegertracing/jaeger:latest
-> ```
-> Then open http://localhost:16686 and look for service `copilot-chat`.
+### Teardown
+
+```bash
+docker stop aspire-dashboard
+```
+
+> **Tip:** See the [Aspire Dashboard standalone docs](https://aspire.dev/dashboard/standalone/) for more configuration options.
 
 ---
 
@@ -278,10 +308,12 @@ These custom attributes are included in all traces, metrics, and events, allowin
 
 By default, **no prompt content, responses, or tool arguments are captured** — only metadata like model names, token counts, and durations.
 
-To capture full content:
+To capture full content, add to your VS Code settings:
 
-```bash
-export COPILOT_OTEL_CAPTURE_CONTENT=true
+```json
+{
+  "github.copilot.chat.otel.captureContent": true
+}
 ```
 
 This populates these span attributes:
@@ -305,25 +337,33 @@ Content is captured in full with no truncation.
 
 **OTLP/gRPC:**
 
-```bash
-export COPILOT_OTEL_ENABLED=true
-export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
-export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
+```json
+{
+  "github.copilot.chat.otel.enabled": true,
+  "github.copilot.chat.otel.exporterType": "otlp-grpc",
+  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4317"
+}
 ```
 
 **Remote collector with authentication:**
 
-```bash
-export COPILOT_OTEL_ENABLED=true
-export OTEL_EXPORTER_OTLP_ENDPOINT=https://collector.example.com:4318
-export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer your-token"
+```json
+{
+  "github.copilot.chat.otel.enabled": true,
+  "github.copilot.chat.otel.otlpEndpoint": "https://collector.example.com:4318"
+}
 ```
 
+> **Note:** Authentication headers are only configurable via the `OTEL_EXPORTER_OTLP_HEADERS` environment variable (e.g., `Authorization=Bearer your-token`). See [Environment Variables](#environment-variables).
+
 **File-based output (offline / CI):**
 
-```bash
-export COPILOT_OTEL_ENABLED=true
-export COPILOT_OTEL_FILE_EXPORTER_PATH=/tmp/copilot-otel.jsonl
+```json
+{
+  "github.copilot.chat.otel.enabled": true,
+  "github.copilot.chat.otel.exporterType": "file",
+  "github.copilot.chat.otel.outfile": "/tmp/copilot-otel.jsonl"
+}
 ```
 
 **Console output (quick debugging):**
@@ -376,66 +416,89 @@ First successful span export is logged to the console (`[OTel] First span batch
 
 ---
 
-## Backend Setup & Verification
+## Backend Setup Guides
+
+Copilot Chat's OTel data works with any OTLP-compatible backend. This section provides step-by-step setup guides for recommended backends.
 
-Copilot Chat's OTel data works with any OTLP-compatible backend. This section covers setup and verification for recommended backends.
+### Aspire Dashboard
+
+See [Quick Start](#quick-start) above for setup. The [Aspire Dashboard](https://aspire.dev/dashboard/standalone/) is the simplest option — a single Docker container with a built-in OTLP endpoint and trace viewer. No cloud account or collector needed.
 
 ### OTel Collector + Azure Application Insights
 
 [Azure Application Insights](https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview) ingests OTel traces, metrics, and logs through an [OTel Collector](https://opentelemetry.io/docs/collector/) with the `azuremonitor` exporter. This repo includes a ready-to-use collector setup in `docs/monitoring/`.
 
-**1. Start the collector:**
+**1. Create an Application Insights resource:**
+
+1. Go to the [Azure Portal](https://portal.azure.com/).
+2. Click **Create a resource** → search **Application Insights** → **Create**.
+3. Choose your subscription, resource group, name, and region → **Review + Create** → **Create**.
+4. Once deployed, go to the resource → **Overview** → copy the **Connection String**.
+
+**2. Start the OTel Collector:**
 
 ```bash
-# Set your App Insights connection string (from Azure Portal → App Insights → Overview)
 export APPLICATIONINSIGHTS_CONNECTION_STRING="InstrumentationKey=...;IngestionEndpoint=..."
 
-# Start the OTel Collector
 cd docs/monitoring
 docker compose up -d
 ```
 
-**2. Verify the collector is healthy:**
+Verify the collector is healthy:
 
 ```bash
 # Should return 200
 curl -s -o /dev/null -w "%{http_code}" http://localhost:4328/v1/traces \
   -X POST -H "Content-Type: application/json" -d '{"resourceSpans":[]}'
 ```
 
-**3. Launch VS Code:**
+**3. Configure VS Code:**
 
-```bash
-COPILOT_OTEL_ENABLED=true \
-COPILOT_OTEL_CAPTURE_CONTENT=true \
-OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4328 \
-code .
+Open **Settings** (`Ctrl+,`) and add:
+
+```json
+{
+  "github.copilot.chat.otel.enabled": true,
+  "github.copilot.chat.otel.exporterType": "otlp-http",
+  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4328"
+}
+```
+
+Optionally, to capture full prompt/response content:
+
+```json
+{
+  "github.copilot.chat.otel.captureContent": true
+}
 ```
 
-**4. Generate telemetry** — Send a chat message in Copilot Chat (e.g., "explain this file" in agent mode).
+> **Warning:** Content capture includes prompts, code, and file contents. Only enable in trusted environments.
 
-**5. Verify in App Insights:**
+**4. Generate telemetry** — Open Copilot Chat and send any message (e.g., use Agent mode).
 
-- **Traces:** Application Insights → Transaction search → filter by "Trace" or "Request".
+**5. Verify data:**
 
-- **Logs:** Application Insights → Logs:
-  ```kql
-  traces
-  | where timestamp > ago(1h)
-  | where message contains "GenAI" or message contains "copilot_chat"
-  | project timestamp, message, customDimensions
-  | order by timestamp desc
-  ```
+- **Jaeger (local):** Open http://localhost:16687, select service `copilot-chat`, click **Find Traces**.
+- **App Insights (Azure):** Go to your Application Insights resource → **Transaction search** → filter by "Trace" or "Request".
 
-- **Metrics:** Application Insights → Metrics → "Custom" namespace, or via Logs:
-  ```kql
-  customMetrics
-  | where timestamp > ago(1h)
-  | where name startswith "gen_ai" or name startswith "copilot_chat"
-  | summarize avg(value), count() by name
-  ```
+Run this query in **Application Insights → Logs** to confirm:
 
-> **Note:** Traces typically appear within 1-2 minutes. Metrics may take 5-10 minutes.
+```kql
+traces
+| where timestamp > ago(1h)
+| where message contains "GenAI" or message contains "copilot_chat"
+| project timestamp, message, customDimensions
+| order by timestamp desc
+```
+
+For metrics (may take 5–10 minutes to appear):
+
+```kql
+customMetrics
+| where timestamp > ago(1h)
+| where name startswith "gen_ai" or name startswith "copilot_chat"
+| summarize avg(value), count() by name
+```
 
 **Collector config** (`docs/monitoring/otel-collector-config.yaml`):
 
@@ -466,18 +529,45 @@ service:
 
 > **Note:** The docker-compose maps ports to `4328`/`4327` on the host to avoid conflicts. Adjust in `docker-compose.yaml` if needed. Add additional exporters (e.g., `otlphttp/jaeger`) to fan out to multiple backends. See `docs/monitoring/otel-collector-config.yaml` for the full config including `batch` processor and `logs` pipeline.
 
+### Jaeger
+
+[Jaeger](https://www.jaegertracing.io/) is an open-source distributed tracing platform. It accepts OTLP directly — no collector needed.
+
+**1. Start Jaeger:**
+
+```bash
+docker run -d --name jaeger -p 16686:16686 -p 4318:4318 jaegertracing/jaeger:latest
+```
+
+**2. Configure VS Code:**
+
+```json
+{
+  "github.copilot.chat.otel.enabled": true,
+  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4318"
+}
+```
+
+**3. Verify:** Open http://localhost:16686, select service `copilot-chat`, and click **Find Traces**.
 
 ### Langfuse
 
 [Langfuse](https://langfuse.com/) is an open-source LLM observability platform with native OTLP ingestion and support for [OTel GenAI Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-agent-spans/). See the [Langfuse docs](https://langfuse.com/docs/opentelemetry/introduction) for full details on capabilities and limitations.
 
 **Setup:**
 
+```json
+{
+  "github.copilot.chat.otel.enabled": true,
+  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:3000/api/public/otel",
+  "github.copilot.chat.otel.captureContent": true
+}
+```
+
+Then set the auth header via environment variable (required — no VS Code setting for headers):
+
 ```bash
-export COPILOT_OTEL_ENABLED=true
-export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:3000/api/public/otel
 export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic $(echo -n '<public-key>:<secret-key>' | base64)"
-export COPILOT_OTEL_CAPTURE_CONTENT=true
 ```
 
 Replace `<public-key>` and `<secret-key>` with your Langfuse API keys from **Settings → API Keys**.
