feat(online-eval): add --output-config and --tags, --description on update - #2266

Fourth and last of the stack. Now based directly on refactor (#2265 and #2294 have merged).

--output-config on create and update

Same passthrough contract as batch evaluation's flag of the same name: inline JSON, file://<path>, or -, parsed and forwarded with field names and values untouched, rejected before any side effect if malformed.

A separate module and a separate generated type, deliberately. The online-evaluation OutputConfig is a plain object with no logStreamName; the batch one is a tagged union that has one. An online evaluation runs continuously rather than as one job with one stream, so sharing the type would be wrong even though the flag name matches.

--tags on create, --description on update

--tags reuses TagsSchema + parseJsonFlagWithSchema, matching project add memory, so {"team":42} is rejected by the CLI rather than by the API. --description on update was simply missing — create had it.

The execution role, which is what makes the destination usable

At create, executionPolicy granted result writes only to /aws/bedrock-agentcore/evaluations/*. A CLI-managed role plus a custom log group would therefore have created a config whose results could not be written anywhere — the flag would have looked like it worked. So the create-time policy now also grants:

• the customer-named group, for DEDICATED_LOG_GROUP
• the sampled source groups, for SOURCE_LOG_GROUP

update never provisions or re-scopes the execution role — it forwards a caller's --role-arn as-is and leaves the policy to them, matching harness update and the pattern #2294 established for online-eval. So the destination-aware widening applies at create only.

Two invariants worth reviewing closely

A single ARN stays a bare string, not a one-element array. IAM treats those identically, but this document's exact text is hashed to name the inline policy (scopePolicyName). Wrapping it would rename the policy attached to every existing config with no custom destination, orphaning the grant it is currently running on. My first attempt did exactly that, and the fixture suite caught it — onlineEvalExecutionRole.test.ts now pins it.

A destination already inside the reserved /aws/bedrock-agentcore/evaluations/ namespace contributes nothing, since the wildcard covers it. That keeps the document stable for a config whose stored outputConfig is the service-managed default the API echoes back — otherwise the same orphaning problem would hit every config ever created through the CLI.

One shared SourceResolver per command

create and update each build a single SourceResolver and thread it through every stdin-capable flag (--tags, --filters, --data-source-config, --output-config). The resolver's one-stdin guard then fires across options: --filters - --output-config - is reported as a conflict instead of the second flag silently reading an empty string after the first drains stdin.

Live verification — an exploratory account, us-west-2

Create with a custom dedicated group and tags → the config came back with logGroupName: /company/pr4-online-eval-results, and the CLI-provisioned role's policy granted exactly the intended scope:

{
  "Sid": "WriteEvaluationResults",
  "Action": ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:DescribeLogStreams", "logs:PutLogEvents"],
  "Resource": [
    "arn:aws:logs:us-west-2:123456789012:log-group:/aws/bedrock-agentcore/evaluations/*",
    "arn:aws:logs:us-west-2:123456789012:log-group:/company/pr4-online-eval-results*"
  ]
}

Tags landed: {"purpose": "pr4-verify", "team": "agentcore-cli"}.

Update with a new --description and --output-config → both reached the service and a follow-up get confirmed them; the execution role was left untouched, as intended.

Cleanup: the config, the provisioned IAM role and its inline policy, and the /company/pr4-online-eval-results log group were all deleted (verified absent).

Tests

• online-eval.flags.test.tsx (new, 13 tests) — output config from inline / stdin reaching Core unchanged on both commands; tags parsed to a map; a non-string tag value rejected; omitted flags staying undefined; malformed --output-config or --tags failing with an empty Core call log, which is the assertion that no IAM role was provisioned for a config that was never created; --description reaching Core; and, on each command, a second option reading from stdin rejected before any SDK call.
• onlineEvalExecutionRole.test.ts — the bare-string invariant, the custom dedicated group, SOURCE_LOG_GROUP scoping to the source prefix, a reserved-namespace destination adding nothing, and a destination change producing a different policy name.

Verification

• bun test — 3242 pass, 0 fail (228 files)
• bun run typecheck, bun run lint:check, bun run format:check — clean
• Existing online-eval fixture suite passes unchanged, which is the backward-compatibility evidence for the policy document