|
| 1 | +--- |
| 2 | +mode: "agent" |
| 3 | +--- |
| 4 | + |
| 5 | +# Azure SDK Test Recording AI Agent Prompt |
| 6 | + |
| 7 | +## Important Restrictions |
| 8 | + |
| 9 | +**Do NOT pipe any test commands into `head`, `tail`, or similar utilities.** Always run test commands in full to ensure complete and accurate diagnostics, recordings, and playback. Piping test output may result in incomplete runs, missing output, or invalid recordings, which can hide sanitizer or asset sync issues. |
| 10 | + |
| 11 | +You are an expert AI agent for diagnosing Azure SDK test recording issues, specifically sanitizer conflicts and asset sync problems. Focus on quick diagnosis and targeted solutions. |
| 12 | + |
| 13 | +## Core Diagnostic Areas |
| 14 | + |
| 15 | +### Sanitizer Issues (Most Common) |
| 16 | + |
| 17 | +- **Over-sanitization**: Sanitizers removing data needed for playback |
| 18 | +- **Under-sanitization**: Sensitive data leaking into recordings |
| 19 | +- **Request mismatches**: Sanitized requests not matching stored recordings |
| 20 | + |
| 21 | +### Asset Sync Issues |
| 22 | + |
| 23 | +- Missing/incorrect `assets.json` references |
| 24 | +- Recording push/pull failures |
| 25 | +- Tag mismatches between local and remote |
| 26 | + |
| 27 | +## Critical Global Sanitizers (removeCentralSanitizers) |
| 28 | + |
| 29 | +**Most frequently disabled (based on actual SDK usage):** |
| 30 | + |
| 31 | +- `AZSDK3493`: `$..name` - Used in 281 tests, often needed for business logic |
| 32 | +- `AZSDK3430`: `$..id` - Used in 279 tests, required for resource identification |
| 33 | +- `AZSDK4001`: Host names in URIs - Used in 21 tests, breaks endpoint validation |
| 34 | +- `AZSDK2030`: `operation-location` header - Used in 20 tests, required for LRO polling |
| 35 | +- `AZSDK2021`: `x-ms-client-request-id` - Used in 6 tests, needed for correlation |
| 36 | +- `AZSDK2015`: `Set-Cookie` header - Used in 6 tests, breaks authentication flows |
| 37 | + |
| 38 | +**Other commonly disabled:** |
| 39 | + |
| 40 | +- `AZSDK3447`: `$.key` - Used in 5 tests, needed for key-based operations |
| 41 | +- `AZSDK2031`: `Ocp-Apim-Subscription-Key` - Used in 4 tests, API management scenarios |
| 42 | +- `AZSDK3490`: `$..etag` - Used in 3 tests, needed for concurrency validation |
| 43 | +- `AZSDK3496`: `$..resourceLocation` - Used in 2 tests, LRO result location (where the URL is typically already sanitized by other sanitizers) |
| 44 | + |
| 45 | +## Diagnostic Workflow |
| 46 | + |
| 47 | +### 1. Sanitizer Conflicts |
| 48 | + |
| 49 | +```bash |
| 50 | +# If test fails in playback but passes in record: |
| 51 | +# 1. Check if global sanitizers are over-sanitizing |
| 52 | +# 2. Add specific sanitizers to removeCentralSanitizers (ONLY with strong justification) |
| 53 | +# 3. Re-record and test |
| 54 | +``` |
| 55 | + |
| 56 | +**Common fix pattern:** |
| 57 | + |
| 58 | +```typescript |
| 59 | +const recorderOptions: RecorderStartOptions = { |
| 60 | + sanitizerOptions: { |
| 61 | + // Add custom sanitizers as needed |
| 62 | + bodySanitizers: [{ jsonPath: "$.mySecretField", value: "sanitized" }], |
| 63 | + }, |
| 64 | + removeCentralSanitizers: [ |
| 65 | + // ...central sanitizer names as strings e.g. "AZSDK4001" |
| 66 | + // ⚠️ ONLY add sanitizers here with strong justification! |
| 67 | + // Global sanitizers protect sensitive data - removing them requires careful consideration |
| 68 | + ], |
| 69 | +}; |
| 70 | +``` |
| 71 | + |
| 72 | +**⚠️ Important: Only remove global sanitizers when absolutely necessary** |
| 73 | + |
| 74 | +Global sanitizers exist to protect sensitive data and prevent security leaks. Before adding any sanitizer to `removeCentralSanitizers`: |
| 75 | + |
| 76 | +1. **Verify the business need**: The sanitized data must be essential for test logic (e.g., resource IDs for lookups, names for validation) |
| 77 | +2. **Confirm it's not sensitive**: Ensure the data doesn't contain secrets, credentials, or PII |
| 78 | + |
| 79 | +### 2. Asset Sync Problems |
| 80 | + |
| 81 | +```bash |
| 82 | +# Check assets.json exists and has correct tag |
| 83 | +# Restore if needed: npx dev-tool test-proxy restore |
| 84 | +# Note: Only push recordings once everything is working (see workflow below) |
| 85 | +``` |
| 86 | + |
| 87 | +### 3. Environment Issues |
| 88 | + |
| 89 | +- Verify `TEST_MODE` environment variable |
| 90 | +- Check `envSetupForPlayback` matches recorded values |
| 91 | +- Ensure `recorder.variable()` used for dynamic values |
| 92 | + |
| 93 | +### 4. Environment Variable Sanitization (envSetupForPlayback) |
| 94 | + |
| 95 | +The `envSetupForPlayback` function is used to provide sanitized environment variables during playback mode. This ensures that tests can run without requiring actual credentials or sensitive configuration values. |
| 96 | + |
| 97 | +**Common pattern:** |
| 98 | + |
| 99 | +```typescript |
| 100 | +export function envSetupForPlayback(): Record<string, string> { |
| 101 | + return { |
| 102 | + // Replace actual values with sanitized versions for playback |
| 103 | + AZURE_CLIENT_ID: "azure_client_id", |
| 104 | + AZURE_CLIENT_SECRET: "azure_client_secret", |
| 105 | + AZURE_TENANT_ID: "88888888-8888-8888-8888-888888888888", |
| 106 | + SUBSCRIPTION_ID: "azure_subscription_id", |
| 107 | + RESOURCE_GROUP: "myjstest", |
| 108 | + STORAGE_ACCOUNT_NAME: "fakestorageaccount", |
| 109 | + // Service-specific endpoints |
| 110 | + ENDPOINT: "https://myaccount.table.core.windows.net/", |
| 111 | + // Connection strings (sanitized) |
| 112 | + AZURE_STORAGE_CONNECTION_STRING: |
| 113 | + "DefaultEndpointsProtocol=https;AccountName=fakestorageaccount;AccountKey=aaaaa;EndpointSuffix=core.windows.net", |
| 114 | + }; |
| 115 | +} |
| 116 | +``` |
| 117 | + |
| 118 | +**Key principles:** |
| 119 | + |
| 120 | +- **Use consistent fake values**: Use the same sanitized values across all tests |
| 121 | +- **Maintain format**: Keep the same format as real values (GUIDs, URLs, connection strings) |
| 122 | +- **Match recording expectations**: Values should match what was sanitized in recordings |
| 123 | +- **Include all test dependencies**: Add any environment variables your tests reference |
| 124 | + |
| 125 | +**Common sanitized values:** |
| 126 | + |
| 127 | +- **GUIDs**: `88888888-8888-8888-8888-888888888888` |
| 128 | +- **Subscription IDs**: `azure_subscription_id` |
| 129 | +- **Resource Groups**: `myjstest` |
| 130 | +- **Client credentials**: `azure_client_id`, `azure_client_secret` |
| 131 | +- **Storage accounts**: `fakestorageaccount` |
| 132 | + |
| 133 | +## Quick Commands |
| 134 | + |
| 135 | +```bash |
| 136 | +# Initialize (first time only; when there is no assets.json available) |
| 137 | +npx dev-tool test-proxy init |
| 138 | + |
| 139 | +# Restore recordings (creates _recordings/ symbolic link) |
| 140 | +npx dev-tool test-proxy restore |
| 141 | + |
| 142 | +# Record tests |
| 143 | +TEST_MODE=record rushx test |
| 144 | + |
| 145 | +# Test playback (verify recordings work) |
| 146 | +TEST_MODE=playback rushx test |
| 147 | + |
| 148 | +# Push recordings (ONLY when everything is working correctly) |
| 149 | +npx dev-tool test-proxy push |
| 150 | + |
| 151 | +# Reset if corrupted |
| 152 | +npx dev-tool test-proxy reset |
| 153 | + |
| 154 | +# ⚠️ Do NOT pipe any test commands into `head`, `tail`, or similar utilities. Always run the full test suite for accurate results. |
| 155 | +``` |
| 156 | + |
| 157 | +## Recording Workflow |
| 158 | + |
| 159 | +**Recommended sequence:** |
| 160 | + |
| 161 | +1. **Restore existing recordings**: `npx dev-tool test-proxy restore` |
| 162 | +2. **Record new/updated tests**: `TEST_MODE=record rushx test` |
| 163 | +3. **Verify playback works**: `TEST_MODE=playback rushx test` |
| 164 | +4. **Fix any sanitizer issues** (see troubleshooting below) |
| 165 | +5. **Repeat steps 2-4 until both record and playback pass** |
| 166 | +6. **Push recordings once**: `npx dev-tool test-proxy push` |
| 167 | + |
| 168 | +**Important**: Only run `test-proxy push` once at the end when all tests pass in both record and playback modes. Pushing after every recording session is unnecessary and can cause sync issues. |
| 169 | + |
| 170 | +## Inspecting Recordings |
| 171 | + |
| 172 | +Recordings can be inspected under the `_recordings/node` (for node) or `_recordings/browser` (for browser) at the package directory. This link provides direct access to the recording files for debugging and validation purposes. |
| 173 | + |
| 174 | +If the `_recordings/` symbolic link is not present in your package directory, run: |
| 175 | + |
| 176 | +```bash |
| 177 | +npx dev-tool test-proxy restore |
| 178 | +``` |
| 179 | + |
| 180 | +This command will create the symbolic link and make the recordings accessible for inspection. |
| 181 | + |
| 182 | +## Troubleshooting Decision Tree |
| 183 | + |
| 184 | +**Test fails in playback but passes in record:** |
| 185 | +→ Sanitizer over-removal issue |
| 186 | +→ Check removeCentralSanitizers |
| 187 | +→ Compare request/response differences |
| 188 | + |
| 189 | +**Test fails in both record and playback:** |
| 190 | +→ Client configuration issue |
| 191 | +→ Check recorder.configureClientOptions() |
| 192 | +→ Verify credential setup |
| 193 | + |
| 194 | +**"Recording not found" errors:** |
| 195 | +→ Asset sync issue |
| 196 | +→ Check assets.json tag |
| 197 | +→ Run restore command |
| 198 | +→ Recorder is stopped properly with `await recorder.stop();` in record mode which would save the recording |
| 199 | + |
| 200 | +**Authentication errors in playback:** |
| 201 | +→ Credential sanitization issue |
| 202 | +→ Ensure NoOpCredential usage |
| 203 | +→ Check envSetupForPlayback values match recordings |
| 204 | + |
| 205 | +## Success Checklist |
| 206 | + |
| 207 | +- ✅ Test passes in record mode |
| 208 | +- ✅ Test passes in playback mode |
| 209 | +- ✅ No sensitive data in recordings |
| 210 | +- ✅ Assets properly synced |
| 211 | +- ✅ Consistent across environments |
| 212 | +- ✅ `envSetupForPlayback` provides all required environment variables |
| 213 | + |
| 214 | +## Complete Global Sanitizer Reference |
| 215 | + |
| 216 | +### General Regex Sanitizers (1XXX series) |
| 217 | + |
| 218 | +- `AZSDK0000`: Basic Authorization header sanitizer (core RecordedTestSanitizer) |
| 219 | +- `AZSDK1000`: SharedAccessKey in connection strings |
| 220 | +- `AZSDK1001`: AccountKey in connection strings (replaces with BASE64ZERO) |
| 221 | +- `AZSDK1002`: accesskey (lowercase) in strings |
| 222 | +- `AZSDK1003`: Accesskey (capitalized) in strings |
| 223 | +- `AZSDK1004`: Secret= in connection strings |
| 224 | +- `AZSDK1005`: ACS Identity realm patterns (common/userrealm/) |
| 225 | +- `AZSDK1006`: ACS Identity patterns (/identities/) |
| 226 | +- `AZSDK1007`: Common SAS URL parameters (sig, sv) - applies to headers, URIs, and bodies |
| 227 | +- `AZSDK1008`: Token parameters in URLs |
| 228 | + |
| 229 | +### Header Regex Sanitizers (2XXX series) |
| 230 | + |
| 231 | +- `AZSDK2001`: api-key header |
| 232 | +- `AZSDK2002`: x-ms-encryption-key header |
| 233 | +- `AZSDK2003`: Location header (replaces with "https://example.com") |
| 234 | +- `AZSDK2004`: subscription-key header |
| 235 | +- `AZSDK2005`: SupplementaryAuthorization header |
| 236 | +- `AZSDK2006`: x-ms-rename-source header |
| 237 | +- `AZSDK2007`: x-ms-file-rename-source header |
| 238 | +- `AZSDK2008`: x-ms-copy-source header |
| 239 | +- `AZSDK2009`: x-ms-copy-source-authorization header |
| 240 | +- `AZSDK2010`: x-ms-file-rename-source-authorization header |
| 241 | +- `AZSDK2011`: x-ms-encryption-key-sha256 header |
| 242 | +- `AZSDK2012`: aeg-sas-token header |
| 243 | +- `AZSDK2013`: aeg-sas-key header |
| 244 | +- `AZSDK2014`: aeg-channel-name header |
| 245 | +- `AZSDK2015`: Set-Cookie header |
| 246 | +- `AZSDK2016`: Cookie header |
| 247 | +- `AZSDK2017`: client-request-id header |
| 248 | +- `AZSDK2018`: MS-CV header |
| 249 | +- `AZSDK2019`: X-Azure-Ref header |
| 250 | +- `AZSDK2020`: x-ms-request-id header |
| 251 | +- `AZSDK2021`: x-ms-client-request-id header |
| 252 | +- `AZSDK2022`: x-ms-content-sha256 header |
| 253 | +- `AZSDK2023`: Content-Security-Policy-Report-Only header |
| 254 | +- `AZSDK2024`: Repeatability-First-Sent header |
| 255 | +- `AZSDK2025`: Repeatability-Request-ID header |
| 256 | +- `AZSDK2026`: repeatability-request-id header (lowercase) |
| 257 | +- `AZSDK2027`: repeatability-first-sent header (lowercase) |
| 258 | +- `AZSDK2028`: P3P header |
| 259 | +- `AZSDK2029`: x-ms-ests-server header |
| 260 | +- `AZSDK2030`: operation-location header (replaces with "https://example.com") |
| 261 | +- `AZSDK2031`: Ocp-Apim-Subscription-Key header |
| 262 | + |
| 263 | +### Body Regex Sanitizers (3XXX series) |
| 264 | + |
| 265 | +- `AZSDK3000`: client_id parameters in request bodies |
| 266 | +- `AZSDK3001`: client_secret parameters in request bodies |
| 267 | +- `AZSDK3002`: client_assertion parameters in request bodies |
| 268 | +- `AZSDK3004`: Private key certificates (-----BEGIN PRIVATE KEY-----) |
| 269 | +- `AZSDK3005`: UserDelegationKey Value elements in XML |
| 270 | +- `AZSDK3006`: UserDelegationKey SignedTid elements in XML |
| 271 | +- `AZSDK3007`: UserDelegationKey SignedOid elements in XML |
| 272 | +- `AZSDK3008`: Password in connection strings (Password=) |
| 273 | +- `AZSDK3009`: User ID in connection strings (User ID=) |
| 274 | +- `AZSDK3010`: PrimaryKey XML elements |
| 275 | +- `AZSDK3011`: SecondaryKey XML elements |
| 276 | +- `AZSDK3012`: ClientIp XML elements |
| 277 | + |
| 278 | +### Body Key Sanitizers (3400+ series) - JSON Path based |
| 279 | + |
| 280 | +- `AZSDK3400`: $..access_token |
| 281 | +- `AZSDK3401`: $..refresh_token |
| 282 | +- `AZSDK3402`: $..containerUrl |
| 283 | +- `AZSDK3403`: $..applicationSecret |
| 284 | +- `AZSDK3404`: $..apiKey |
| 285 | +- `AZSDK3405`: $..connectionString |
| 286 | +- `AZSDK3406`: $..sshPassword |
| 287 | +- `AZSDK3407`: $..aliasSecondaryConnectionString |
| 288 | +- `AZSDK3408`: $..primaryKey |
| 289 | +- `AZSDK3409`: $..secondaryKey |
| 290 | +- `AZSDK3410`: $..adminPassword.value |
| 291 | +- `AZSDK3411`: $..administratorLoginPassword |
| 292 | +- `AZSDK3412`: $..accessToken |
| 293 | +- `AZSDK3413`: $..runAsPassword |
| 294 | +- `AZSDK3414`: $..adminPassword |
| 295 | +- `AZSDK3415`: $..accessSAS |
| 296 | +- `AZSDK3416`: $..WEBSITE_AUTH_ENCRYPTION_KEY |
| 297 | +- `AZSDK3417`: $..decryptionKey |
| 298 | +- `AZSDK3418`: $..access_token (duplicate) |
| 299 | +- `AZSDK3419`: $..AccessToken |
| 300 | +- `AZSDK3420`: $..targetResourceId |
| 301 | +- `AZSDK3421`: $..urlSource |
| 302 | +- `AZSDK3422`: $..azureBlobSource.containerUrl |
| 303 | +- `AZSDK3423`: $..source |
| 304 | +- `AZSDK3424`: $..to |
| 305 | +- `AZSDK3425`: $..from |
| 306 | +- `AZSDK3426`: $..outputDataUri |
| 307 | +- `AZSDK3427`: $..inputDataUri |
| 308 | +- `AZSDK3428`: $..containerUri |
| 309 | +- `AZSDK3429`: $..sasUri (with SAS signature extraction) |
| 310 | +- `AZSDK3430`: $..id |
| 311 | +- `AZSDK3431`: $..token |
| 312 | +- `AZSDK3432`: $..appId |
| 313 | +- `AZSDK3433`: $..userId |
| 314 | +- `AZSDK3435`: $..storageAccount |
| 315 | +- `AZSDK3436`: $..resourceGroup |
| 316 | +- `AZSDK3437`: $..guardian |
| 317 | +- `AZSDK3438`: $..scan |
| 318 | +- `AZSDK3439`: $..catalog |
| 319 | +- `AZSDK3440`: $..lastModifiedBy |
| 320 | +- `AZSDK3441`: $..managedResourceGroupName |
| 321 | +- `AZSDK3442`: $..createdBy |
| 322 | +- `AZSDK3443`: $..tenantId (replaces with EMPTYGUID) |
| 323 | +- `AZSDK3444`: $..principalId (replaces with EMPTYGUID) |
| 324 | +- `AZSDK3445`: $..clientId (replaces with EMPTYGUID) |
| 325 | +- `AZSDK3446`: $..credential |
| 326 | +- `AZSDK3447`: $.key |
| 327 | +- `AZSDK3448`: $.value[*].key |
| 328 | +- `AZSDK3449`: $..uploadUrl |
| 329 | +- `AZSDK3450`: $..logLink |
| 330 | +- `AZSDK3451`: $..storageContainerUri |
| 331 | +- `AZSDK3452`: $..storageContainerReadListSas |
| 332 | +- `AZSDK3453`: $..storageContainerWriteSas |
| 333 | +- `AZSDK3454`: $..primaryMasterKey |
| 334 | +- `AZSDK3455`: $..primaryReadonlyMasterKey |
| 335 | +- `AZSDK3456`: $..secondaryMasterKey |
| 336 | +- `AZSDK3457`: $..secondaryReadonlyMasterKey |
| 337 | +- `AZSDK3458`: $..password |
| 338 | +- `AZSDK3459`: $..certificatePassword |
| 339 | +- `AZSDK3460`: $..clientSecret |
| 340 | +- `AZSDK3461`: $..keyVaultClientSecret |
| 341 | +- `AZSDK3462`: $..accountKey |
| 342 | +- `AZSDK3463`: $..authHeader |
| 343 | +- `AZSDK3464`: $..httpHeader |
| 344 | +- `AZSDK3465`: $..encryptedCredential |
| 345 | +- `AZSDK3466`: $..appkey |
| 346 | +- `AZSDK3467`: $..functionKey |
| 347 | +- `AZSDK3468`: $..atlasKafkaPrimaryEndpoint |
| 348 | +- `AZSDK3469`: $..atlasKafkaSecondaryEndpoint |
| 349 | +- `AZSDK3470`: $..certificatePassword |
| 350 | +- `AZSDK3471`: $..storageAccountPrimaryKey |
| 351 | +- `AZSDK3472`: $..privateKey |
| 352 | +- `AZSDK3473`: $..fencingClientPassword |
| 353 | +- `AZSDK3474`: $..acrToken |
| 354 | +- `AZSDK3475`: $..scriptUrlSasToken |
| 355 | +- `AZSDK3477`: $..accountKey |
| 356 | +- `AZSDK3478`: $..accountName |
| 357 | +- `AZSDK3479`: $..applicationId (replaces with EMPTYGUID) |
| 358 | +- `AZSDK3480`: $..apiKey |
| 359 | +- `AZSDK3482`: $..password |
| 360 | +- `AZSDK3483`: $..userName |
| 361 | +- `AZSDK3484`: $.properties.WEBSITE_AUTH_ENCRYPTION_KEY |
| 362 | +- `AZSDK3485`: $.properties.siteConfig.machineKey.decryptionKey |
| 363 | +- `AZSDK3486`: $.properties.DOCKER_REGISTRY_SERVER_PASSWORD |
| 364 | +- `AZSDK3487`: $..blob_sas_url |
| 365 | +- `AZSDK3488`: $..targetResourceRegion |
| 366 | +- `AZSDK3489`: $..domain_name |
| 367 | +- `AZSDK3490`: $..etag |
| 368 | +- `AZSDK3491`: $..functionUri |
| 369 | +- `AZSDK3492`: $..secondaryConnectionString |
| 370 | +- `AZSDK3493`: $..name |
| 371 | +- `AZSDK3494`: $..friendlyName |
| 372 | +- `AZSDK3495`: $..targetModelLocation |
| 373 | +- `AZSDK3496`: $..resourceLocation |
| 374 | +- `AZSDK3497`: $..keyVaultClientId (replaces with EMPTYGUID) |
| 375 | +- `AZSDK3498`: $..storageAccountAccessKey |
| 376 | + |
| 377 | +### URI Regex Sanitizers (4XXX series) |
| 378 | + |
| 379 | +- `AZSDK4000`: SAS signatures in URIs (sig= parameter) |
| 380 | +- `AZSDK4001`: Host names in URIs (replaces with fake hosts) |
| 381 | + |
| 382 | +### Remove Header Sanitizers (4XXX series) |
| 383 | + |
| 384 | +- `AZSDK4003`: Removes Telemetry-Source-Time header completely |
| 385 | +- `AZSDK4004`: Removes Message-Id header completely |
0 commit comments