Codestin Search App

ndycode · 2026-06-10T18:36:46Z

Summary

Companion to the direct-coverage wave (#559–#573), in the repo's existing test/property/ tradition. resolveUnsupportedCodexFallbackModel steers which model the proxy retries with when ChatGPT accounts reject a Codex model — a wrong fallback means silent infinite retry loops or model downgrades the user never asked for. The existing example-based coverage (via the fetch-helpers suites) pins specific cases; this adds test/property/model-fallback.property.test.ts (6 fast-check properties) pinning the invariants across the whole input space.

The generator produces every default-chain model under provider prefixes (openai/, models/), reasoning-effort suffixes (-low, -xhigh, …), and arbitrary casing — exactly the spellings canonicalizeModelName must normalize.

Invariants pinned

The feature toggle off, or an error body that is not an unsupported-model error, always yields undefined.
Any returned fallback is a member of the canonical chain for the requested model, is never the current model, never an already-attempted model, and respects the gpt-5.3-codex → gpt-5.2-codex legacy-edge toggle.
With nothing attempted, the first chain target wins (deterministic retry order); with every chain target attempted, the resolver gives up rather than looping.
Attempted-model spellings canonicalize the same way as requested models — an attempt recorded as OPENAI/GPT-5.3-CODEX-HIGH still skips gpt-5.3-codex.

Validation

vitest run test/property/model-fallback.property.test.ts — 6/6 passing (default fast-check run counts)
npm run typecheck — clean
npx eslint test/property/model-fallback.property.test.ts --max-warnings=0 — clean

https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB

Generated by Claude Code

note: greptile review for oc-chatgpt-multi-auth. cite files like `lib/foo.ts:123`. confirm regression tests + windows concurrency/token redaction coverage.

Greptile Summary

adds test/property/model-fallback.property.test.ts — 6 fast-check properties that pin the fallback invariants of resolveUnsupportedCodexFallbackModel across the full input space of provider prefixes, effort suffixes, and arbitrary casing. the local canonicalize helper now passes chain values through normalization before comparison, closing the coupling issue flagged in the previous wave.

properties 1–2 check the toggle-off and wrong-error-body guard paths; properties 3–6 cover chain membership, ordering, exhaustion, and attempted-model spelling normalization.
two coverage gaps: property 6 only generates lowercase attempted-model spellings (missing the uppercase path the PR description claims to cover), and property 4 hardcodes the legacy-edge toggle to true, leaving the gpt-5.3-codex + toggle-off = undefined branch untested.

Confidence Score: 5/5

test-only addition; no production code changes, no concurrency or token-safety surface introduced.

all changes are confined to a new property test file; the implementation under test is unchanged. the two gaps noted leave a small slice of the stated invariants unexercised, but neither gap conceals a defect in the resolver itself.

test/property/model-fallback.property.test.ts — properties 4 and 6 have the coverage gaps described above.

Important Files Changed

Filename	Overview
test/property/model-fallback.property.test.ts	adds 6 fast-check properties for resolveUnsupportedCodexFallbackModel; local canonicalize helper now correctly mirrors the implementation; two minor coverage gaps: property 6 never generates uppercase attempted-model spellings, and property 4 hardcodes the legacy-edge toggle to true

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["resolveUnsupportedCodexFallbackModel(options)"] --> B{fallbackOnUnsupportedCodexModel?}
    B -- false --> Z1[return undefined]
    B -- true --> C{errorBody is unsupported-model error?}
    C -- no --> Z2[return undefined]
    C -- yes --> D["canonicalize requestedModel → currentModel"]
    D --> E{currentModel in chain?}
    E -- no --> Z3[return undefined]
    E -- yes --> F["iterate chain targets"]
    F --> G{legacyEdge=false AND target=gpt-5.2-codex AND current=gpt-5.3-codex?}
    G -- yes --> H[skip target]
    H --> F
    G -- no --> I{target === currentModel?}
    I -- yes --> H
    I -- no --> J{target in attempted set?}
    J -- yes --> H
    J -- no --> K[return target]
    F -- exhausted --> Z4[return undefined]

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
test/property/model-fallback.property.test.ts:156-181
**property 6 never generates uppercase attempted-model spellings**

`prefix` and `suffix` are both drawn from lowercase constants, and `firstTarget` is already canonical (lowercase). the composed `attemptedModels` entry is therefore always lowercase — e.g. `openai/gpt-5.2-codex-high` — so the uppercase case the PR description calls out (`OPENAI/GPT-5.3-CODEX-HIGH`) is never exercised. adding `fc.boolean()` to produce an upper-cased variant (mirroring what `arbSpelledModel` does) would close the gap.

### Issue 2 of 2
test/property/model-fallback.property.test.ts:124-138
**property 4 hardcodes `fallbackToGpt52OnUnsupportedGpt53: true`**

`gpt-5.3-codex` has only one chain target (`gpt-5.2-codex`). with the toggle forced `true`, `firstTarget` is always `gpt-5.2-codex` and the assertion passes. with `false`, the resolver skips that target and returns `undefined`, so `expect(result).toBe(firstTarget)` would fail — but this path is never reached. parameterising the toggle and asserting `result === undefined` when `canonical === "gpt-5.3-codex" && !legacyEdge` would give this property real coverage of the toggle interaction.

_{Reviews (2): Last reviewed commit: "test: canonicalize expected chain target..." | Re-trigger Greptile}

fast-check properties over resolveUnsupportedCodexFallbackModel (via the fetch-helpers facade), generating every default-chain model under provider prefixes, reasoning-effort suffixes, and arbitrary casing: - the feature toggle off, or a non-unsupported error body, always yields undefined - any returned fallback is a member of the canonical chain for the requested model, is never the current model, never an already-attempted model, and respects the gpt-5.3 -> gpt-5.2 legacy edge toggle - with nothing attempted the first chain target wins; with every target attempted the resolver gives up - attempted-model spellings canonicalize the same way as requested models, so a differently-spelled attempt still skips its target https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB

chatgpt-codex-connector · 2026-06-10T18:36:51Z

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

coderabbitai · 2026-06-10T18:36:54Z

Warning

Review limit reached

@ndycode, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 6 minutes and 3 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6ef54776-df2f-4844-a39f-35133b96db45

📥 Commits

Reviewing files that changed from the base of the PR and between b566656 and 3c60683.

📒 Files selected for processing (1)

test/property/model-fallback.property.test.ts

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch claude/audit-55-model-fallback-property

✨ Simplify code

Create PR with simplified code
Commit simplified code in branch claude/audit-55-model-fallback-property

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

The resolver normalizes chain values through canonicalizeModelName; the test helper now mirrors that transform so the expectations stay valid even if a future chain entry is added in a non-canonical spelling. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB

…racle (#593) * test: property-check the auth rate limiter against a sliding-window oracle Three fast-check properties over the real module under fake timers (maxAttempts 3 / 1s window so sequences cross the expiry boundary): - model-based: for any record/reset/advance interleaving across three accounts with decorated id spellings (case/whitespace variants), getAttemptsRemaining and canAttemptAuth match a trivial timestamps-in-window oracle keyed by canonical id - pinning both the sliding window and the trim+lowercase bucket mapping at once - checkAuthRateLimit throws AuthRateLimitError exactly when blocked, carrying the canonical accountId, zero attemptsRemaining, and a resetAfterMs that agrees with the live getTimeUntilReset - over-recording past maxAttempts can never wedge a bucket: any burst unblocks fully after one quiet window, with getTimeUntilReset bounded by windowMs throughout Config and registry are module state, so each property restores the documented defaults and clears buckets, and ids are namespaced per iteration. Companion to #574/#575/#579/#592; same conventions. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: exhaustive event narrowing and constant-derived gaps length Greptile flagged the bare else branches (a future event kind would silently model as a reset) and the literal 11 coupled to MAX_ATTEMPTS * 4 - 1. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

…contracts (#594) * test: property-check SessionAffinityStore TTL, eviction, and reindex contracts Five fast-check properties over the real store using its injectable now parameters (1s TTL floor so sequences cross expiry often): - model-based TTL/upsert equivalence: for any remember/update/forget/ advance interleaving through whitespace-decorated key spellings, getPreferredAccountIndex and getLastResponseId match a trivial TTL map (remember preserves the continuation id, response-id writes refresh expiry and never create entries) - capacity: size() never exceeds maxEntries, and LRU eviction can never evict the entry just written - write-version conflicts: a stale version loses to a live entry on both the index and response-id channels, but may rebind once the entry expires - forgetAccount + reindexAfterRemoval mirror an account-array splice, with both return counts pinned against the model - prune removes exactly the expired entries; lazily-reaped sessions (touched while expired) correctly do not count as prunable The prune model initially missed that updateLastResponseId deletes an expired entry outright; fast-check found the 8-event counterexample and the model now mirrors the lazy reap. Companion to #574/#575/#579/#592/#593; same conventions. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: fix remember model after lazy reap; cover clearAll; decorate prune keys Greptile P1: the model carried a responseId across expiry, but the assertion block's reads lazily reap expired entries from the store, so a remember after expiry finds no existing entry and the id is gone - the model now inherits the id only from a live entry (verified at FAST_CHECK_NUM_RUNS=1000, where the original 4-event counterexample sequence reproduces without the fix). P2s: a sixth property pins clearAll (#474 invalidation) - size drops to zero, every read goes null, and the store stays usable - and the prune property now routes remember/forget/updateLastResponseId through decorated key spellings like the model property does. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

…595) * test: property-check table formatter alignment under unicode content Four fast-check properties over buildTable/buildTableRow with cell content mixing ASCII, CJK, and emoji so widths are exercised in display columns rather than UTF-16 units (ui-02): - every line of any table (header, separator, all rows) has exactly the layout's display width, for any column set, alignment mix, missing cells, or extra cells beyond the column count - content that fits is preserved verbatim with padding on the declared side - overflowing content truncates to a prefix of the original plus an ellipsis, never exceeding the column width even when a wide glyph cannot fill the final column - zero-width columns render empty and never leak an ellipsis that would desync the row from the header layout Companion to the property suites in #574/#575/#592-#594. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: cover right-aligned overflow truncation too https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: validate overflow prefix on the left channel; generate default align Greptile P1: stripping leading spaces on a right-aligned row conflated alignment padding with spaces that belong to the truncated content (latent spurious failure under a different fast-check seed - e.g. ' a<CJK>' at width 3 right-aligned). Prefix fidelity now checks the left-aligned rendering of the same value, where the cell starts at column 0, while the right-aligned row keeps its width and trailing- ellipsis assertions. Verified at FAST_CHECK_NUM_RUNS=1000. Greptile P2: arbColumn and the fit property now generate undefined align too, exercising formatCell's default-left branch. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

) * test: property-check plugin health classification and report shape Three fast-check properties over getAccountHealth/formatHealthReport: - counts and status derive exactly from the per-account classification for any pool (timestamps straddling now, undefined fallbacks, health 0..100): healthy iff not rate-limited, not cooling, health >= 50, circuit closed; status partitions empty->healthy, none->unhealthy, some->degraded, all->healthy - an open circuit disqualifies an otherwise perfect account: tripping the account:<index> fallback breaker for any subset is reflected in circuitState, subtracted from healthyAccountCount, and drives the status partition - the formatted report names every account with its health percentage and exactly the flags its classification implies, and the summary lines appear iff their counts are nonzero Companion to the property suites in #574/#575/#592-#595. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: exercise half-open disqualification and email-aliased circuits Greptile flagged two coverage gaps: the half-open state was unreachable without advancing the clock, and accounts sharing an email silently shared a breaker without an explicit contract. Property 2 now probes open -> half-open under fake timers (both states disqualify, and the report renders the precise circuit flag), pool emails are unique by construction, and a dedicated property pins shared-email accounts to one breaker while distinct emails stay isolated - derived through the same getAccountIdentityKey the SUT uses. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

* test: property-check model-map resolution invariants Six fast-check properties over the model resolution pipeline (resolveNormalizedModel / getModelProfile / getNormalizedModel), generating known aliases, synthesized GPT-5 spellings (minors 0-9, mini/nano/pro/codex variants, -/./space separators, suffixes), provider prefixes, random casing, and raw garbage: - closed world: every resolution lands on a MODEL_PROFILES key, so getModelProfile's DEFAULT_MODEL fallback is pure defence - idempotence: normalized outputs are fixpoints - provider prefixes and casing never change the resolution - codex dominance: unmapped ids mentioning codex resolve to CURRENT_CODEX_MODEL, never a general model - the inverse: unmapped general GPT-5 spellings stay codex-free and in the gpt-5 family (no silent codex routing for general requests) - every explicit MODEL_MAP alias resolves to its mapped target under any prefix/casing spelling Companion to the property suites in #574/#575/#592-#596. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: combine prefix and casing mutation in the invariance property https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

…598) * test: property-check display-width clustering and parseKey totality Six fast-check properties over the ui-02 measurement layer and the stdin key parser: - displayWidth is total, integer, non-negative, and bounded by two columns per code point over an adversarial alphabet (ZWJ, VS-16, keycap, combining marks, skin tones, regional indicators, CJK, emoji) - plain alphabets (no joiners/modifiers) sum per-character widths exactly and concatenate additively - the table formatter's standing assumption - truncateToWidth returns a self-consistent prefix (its reported width IS displayWidth of the kept text), stays in budget, stops only when the remaining gap is 0 or 1 columns (clusters max out at 2), and is idempotent - truncation prefixes grow monotonically with the width budget - parseKey is total over arbitrary byte buffers and only returns known KeyAction values - the full recognized-sequence table is pinned, and everything else maps to null Companion to the property suites in #574/#575/#592-#597. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: make the 2-column cluster granularity self-enforcing One extra budget column can admit at most two more columns of content, which is externally equivalent to 'no cluster wider than 2' - the assumption the truncation maximality bound relies on. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

) * test: property-check forecast recommendation and summary contracts Seven fast-check properties over recommendForecastAccount, summarizeForecast, and buildForecastExplanation with generated result pools (0-8 accounts, full availability/risk/wait/flag space): - a recommendation always points at a recommendable result (not disabled, hard-failed, exhausted, or unavailable); null only when no such candidate exists - a ready candidate always wins with the minimal risk score among ready candidates, and the reason says so - with no ready candidate, the shortest delayed wait wins - an empty candidate pool names the actual blocker class: 'blocked or exhausted' guidance iff some account is blocked/exhausted rather than disabled/hard-failed (the #exhausted-flag regression guard) - the recommendation is invariant under input order - the summary partitions availability exactly (ready + delayed + unavailable === total) and counts high-risk rows - the explanation mirrors inputs in order and marks selected on exactly the recommended index Companion to the property suites in #574/#575/#592-#598. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: keep forecast generator in the reachable domain; shuffle permutations Greptile flagged that availability and exhausted were drawn independently (evaluateForecastAccount only emits exhausted accounts as delayed) and that order-invariance only probed reversal. The generator now derives the pairing, and the invariance property checks an arbitrary generated permutation plus the reversal. Validated at FAST_CHECK_NUM_RUNS=1000. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

#600) * test: property-check context-overflow classification and synthetic SSE Four fast-check properties over the prompt-too-long recovery path: - the seven documented overflow phrases classify at any casing and position, but only behind the 400 status gate (phrase list pinned verbatim so dropping one fails here) - noise-only and empty bodies never classify at any status - the synthetic response round-trips: 200 with the synthetic headers, parseable Responses-API SSE (created first, completed last, model echoed, /compact notice in the terminal output payload), and is never re-classified as overflow - the recovery path cannot recurse on its own output (recovery-01 dialect guard) - handleContextOverflow intercepts exactly the classifier-positive 400s, and a declined response's body stays readable (clone-read) Companion to the property suites in #574/#575/#592-#599. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: await the async round-trip property so its assertions gate the test Greptile P1: the it callback was synchronous, so the asyncProperty's promise was dropped and vitest greened the test before any assertion ran. Same pattern as the fourth test now. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

…#601) * test: property-check env boolean parsing and policy key/tag contracts Four fast-check properties over the shared env parser and the account policy identity helpers: - parseBooleanEnv: the six documented literals parse under any casing and whitespace padding; every other string (and undefined) returns undefined, never a boolean - the contract that unified three divergent local copies - getAccountPolicyKey: always a sha256: handle that never leaks the raw identity into the policy file key; accountId wins over email; email matches case-insensitively; no identity degrades to the shared unknown bucket - normalizeAccountPolicyTag: outputs are idempotent fixpoints in the [a-z0-9._-]{1,64} language, and null occurs exactly for inputs that trim to nothing (disallowed runs are dashed, not dropped) Validated at FAST_CHECK_NUM_RUNS=1000. Companion to the property suites in #574/#575/#592-#600. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: pin both directions of the whitespace-only-iff-null tag contract https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: generate the whitespace-only accountId fall-through A defined but whitespace-only accountId defers to the email identity (the accountId?.trim() || chain); the existing branch assertions cover it once generated. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

* test: property-check the circuit breaker's availability contract Five fast-check properties over the real CircuitBreaker under fake timers (small windows so sequences cross every boundary): - isAvailable/getTimeUntilAvailable agree with the very next canExecute outcome after ANY event sequence - the polling view callers route on is a faithful prediction, never stale - an open circuit rejects strictly before resetTimeoutMs and admits the probe exactly at it, from any prior history (fresh forced open) - opens exactly at failureThreshold inside one window, never below - failures spaced beyond failureWindowMs never accumulate - the half-open slot admits exactly one probe; success closes with a clean failure count, failure reopens for a full timeout Companion to the property suites in #574/#575/#579; same conventions. https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB * test: pin getTimeUntilReset alongside the injectable-now twin https://claude.ai/code/session_01XNtnkLbBiXZxfQQYLMpucB --------- Co-authored-by: Claude <[email protected]>

This was referenced Jun 10, 2026

test: property-check write-queue serialization and clamp invariants #575

Merged

docs: record the direct-coverage wave in the audit delivery log #568

Merged

greptile-apps Bot reviewed Jun 10, 2026

View reviewed changes

Comment thread test/property/model-fallback.property.test.ts

coderabbitai Bot approved these changes Jun 10, 2026

View reviewed changes

This was referenced Jun 10, 2026

fix: deduplicateAccounts fixpoint loop and identity property tests #579

Merged

chore: drop unused @fast-check/vitest devDependency #581

Merged

ndycode merged commit f0a20e0 into main Jun 11, 2026
2 checks passed

This was referenced Jun 11, 2026

test: property-check the circuit breaker's availability contract #592

Merged

test: property-check model-map resolution invariants #597

Merged

Navigation Menu

test: property-check the unsupported-model fallback invariants - #574

test: property-check the unsupported-model fallback invariants#574
ndycode merged 2 commits into
mainfrom
claude/audit-55-model-fallback-property

ndycode commented Jun 10, 2026 •

edited by greptile-apps Bot

Loading

Uh oh!

chatgpt-codex-connector Bot commented Jun 10, 2026

Uh oh!

coderabbitai Bot commented Jun 10, 2026 •

edited

Loading

Review limit reached

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ndycode commented Jun 10, 2026 • edited by greptile-apps Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Invariants pinned

Validation

note: greptile review for oc-chatgpt-multi-auth. cite files like lib/foo.ts:123. confirm regression tests + windows concurrency/token redaction coverage.

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

chatgpt-codex-connector Bot commented Jun 10, 2026

Uh oh!

coderabbitai Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ndycode commented Jun 10, 2026 •

edited by greptile-apps Bot

Loading

note: greptile review for oc-chatgpt-multi-auth. cite files like `lib/foo.ts:123`. confirm regression tests + windows concurrency/token redaction coverage.

coderabbitai Bot commented Jun 10, 2026 •

edited

Loading