Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: PR comment watcher — auto-monitor and respond to GitHub PR comments - #2

Open
CodeWithBryan wants to merge 19 commits into
mainfrom
feat/pr-comment-watcher
Open

CodeWithBryan wants to merge 19 commits into
mainfrom
feat/pr-comment-watcher

Conversation

@CodeWithBryan

Copy link
Copy Markdown
Owner

Summary

  • PR Comment Watcher: Standalone polling service (orchestration/PrCommentWatcher) that monitors GitHub PRs for new comments (review inline, review summary, conversation) on 60s intervals
  • PR Response Reactor: Listens for AI turn completions, parses structured replies, posts individual responses back to GitHub, and resolves review threads
  • UI Toggle: PrWatchToggle component in BranchToolbar showing detected PR # with enable/disable, plus eye icon in sidebar for watched threads
  • Full plumbing: WebSocket RPC handlers, projection table for persistence, GitHubCli API extensions (6 new methods), WsRpcClient extensions

Architecture

  • Per-thread polling fibers via SynchronizedRef<Map<string, ActivePoller>>
  • Synthetic thread.turn.start dispatch with structured comment template
  • ID-based comment diffing with coalescing (skip dispatch while turn in progress)
  • Auto-disable on PR close/merge, rehydration from DB on restart

New files (9)

  • packages/contracts/src/prCommentWatcher.ts
  • apps/server/src/persistence/Migrations/024_ProjectionPrWatchers.ts
  • apps/server/src/persistence/Services/ProjectionPrWatchers.ts
  • apps/server/src/persistence/Layers/ProjectionPrWatchers.ts
  • apps/server/src/orchestration/Services/PrCommentWatcher.ts
  • apps/server/src/orchestration/Layers/PrCommentWatcher.ts
  • apps/server/src/orchestration/Services/PrResponseReactor.ts
  • apps/server/src/orchestration/Layers/PrResponseReactor.ts
  • apps/web/src/components/PrWatchToggle.tsx

Modified files (12)

  • Contract types, RPC definitions, GitHubCli service/layer, ProviderRuntimeIngestion, OrchestrationReactor, server.ts, ws.ts, BranchToolbar.tsx, Sidebar.tsx, WsRpcClient, test harnesses

Test plan

  • 20 unit tests for buildSyntheticMessage and filterNewComments
  • 7 unit tests for parseAiResponse
  • 1 test for OrchestrationReactor startup ordering
  • TypeScript type check passes (0 errors)
  • Lint passes (0 errors)
  • Manual: enable watcher on branch with open PR, verify polling detects new comments
  • Manual: verify AI responds with structured replies posted back to GitHub
  • Manual: verify thread resolution on addressed review comments

Standalone service that monitors GitHub PR comments for threads with
open PRs, auto-dispatches AI turns to address reviewer feedback, and
posts replies back to the PR with thread resolution.
Fix session start flow (use existing ensureSessionForThread), add
CommandId/MessageId minting, correct owner/repo resolution via
GitCore + parseGitHubRepositoryNameWithOwnerFromRemoteUrl, fix turn
completion detection via provider runtime stream, clarify gh api
usage, INTEGER columns for IDs, coalescing mechanism, UI placement,
workspace precondition, server-side Map for turn tracking, correct
WS_METHODS location, move service to orchestration namespace, expose
ProviderRuntimeIngestion stream for PrResponseReactor.
13 tasks covering contracts, persistence, GitHubCli methods,
ProviderRuntimeIngestion stream exposure, PrCommentWatcher service,
PrResponseReactor, service wiring, WS RPC handlers, UI toggle,
integration tests, sidebar indicator, and final verification.
Fix makeDrainableWorker API, OrchestrationEngineService naming,
PrResponseReactor event correlation via threadId + read model,
WS handler cwd resolution via project.workspaceRoot, turn context
map keyed by threadId, complete mock in reactor test.
Map comment node IDs to thread node IDs via GraphQL before resolving.
Move processCompletedTurn definition before makeDrainableWorker call.
Adds migration 024, service interface, and SQL layer for PR watcher
projection persistence, enabling the PR comment watcher feature to
track watched PRs per thread with last-seen comment/review tracking.
Adds 6 new methods to GitHubCliShape and GitHubCliLive — listPrReviewComments,
listPrReviews, listPrIssueComments, replyToReviewComment, createPrComment, and
resolveReviewThread — all using gh api subcommands for GitHub REST/GraphQL access.
…erRuntimeIngestion

Add streamRuntimeEvents (Stream<ProviderRuntimeEvent>) to the service interface and
implement it via an unbounded PubSub that publishes every event entering processRuntimeEvent.
Update the OrchestrationReactor test mock to satisfy the extended interface.
Wire prWatchToggle and onPrWatchStatus RPC methods into WsRpcClient,
create a PrWatchToggle component that shows PR watch status with
toggle capability, and integrate it into BranchToolbar next to the
branch selector.
…ilterNewComments

Extract buildSyntheticMessage and filterNewComments as exported pure
functions from PrCommentWatcher so they can be tested independently
without an Effect runtime. Adds 20 tests covering message template
generation (header, PR URL, validation warning, comment-id tags,
section grouping, empty-body filtering) and comment ID diffing logic.
… and server layer

Add mock layers for PrCommentWatcher and PrResponseReactor to
server.test.ts and OrchestrationEngineHarness.integration.ts so
layer composition type-checks. Fix server.ts layer wiring to
properly provide dependencies to PrResponseReactorLive and
ProjectionPrWatcherRepositoryLive.
@coderabbitai

coderabbitai Bot commented Apr 14, 2026

Copy link
Copy Markdown

Walkthrough

A comprehensive PR Comment Watcher feature is added to the orchestration system. The system polls GitHub PRs for new comments and reviews, dispatches orchestration turns to address feedback, and posts AI-generated replies back to GitHub via new services, persistence layers, and RPC endpoints.

Changes

Cohort / File(s) Summary
GitHub CLI Methods & Tests
apps/server/src/git/Layers/GitHubCli.ts, apps/server/src/git/Layers/GitHubCli.test.ts, apps/server/src/git/Layers/GitManager.test.ts
Added GH CLI-backed methods for listing PR review comments/reviews/issue comments, replying to review comments, creating PR comments, and resolving review threads via GraphQL; tests and fake CLI helpers updated to exercise these paths.
Contracts & RPC
packages/contracts/src/git.ts, packages/contracts/src/prCommentWatcher.ts, packages/contracts/src/rpc.ts, packages/contracts/src/index.ts
Added typed schemas for PR review/comment entities and watcher status/toggle types; added WebSocket RPCs prWatchToggle and subscribePrWatchStatus and re-exported new contract types.
PR Comment Watcher (service, layer, tests)
apps/server/src/orchestration/Services/PrCommentWatcher.ts, apps/server/src/orchestration/Layers/PrCommentWatcher.ts, apps/server/src/orchestration/Layers/PrCommentWatcher.test.ts
New PrCommentWatcher service and live layer: per-thread polling fibers, synthetic message builder, comment diffing/filtering, turn-context management, status PubSub stream, watch/unwatch lifecycle, and associated tests.
PR Response Reactor (service, layer, tests)
apps/server/src/orchestration/Services/PrResponseReactor.ts, apps/server/src/orchestration/Layers/PrResponseReactor.ts, apps/server/src/orchestration/Layers/PrResponseReactor.test.ts
New PrResponseReactor service and live layer: consumes provider runtime turn.completed events, parses assistant responses into structured replies, posts replies to GitHub (review replies or issue comments), resolves review threads when possible, and tests for parsing logic.
Orchestration Reactor Wiring & Tests
apps/server/src/orchestration/Layers/OrchestrationReactor.ts, apps/server/src/orchestration/Layers/OrchestrationReactor.test.ts, apps/server/integration/OrchestrationEngineHarness.integration.ts
Wired PrCommentWatcher and PrResponseReactor into orchestration reactor startup; tests and integration harness updated/extended with stub/no-op layers for PR services.
Provider Runtime Event Streaming
apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts, apps/server/src/orchestration/Services/ProviderRuntimeIngestion.ts
Added an internal PubSub for ProviderRuntimeEvent and exposed streamRuntimeEvents so downstream consumers (e.g., PrResponseReactor) can subscribe to runtime events.
Persistence: Projection & Migration
apps/server/src/persistence/Services/ProjectionPrWatchers.ts, apps/server/src/persistence/Layers/ProjectionPrWatchers.ts, apps/server/src/persistence/Migrations/024_ProjectionPrWatchers.ts, apps/server/src/persistence/Migrations.ts
Added projection_pr_watchers schema, repository interface and SQL-backed layer (upsert, get, list enabled, delete, update last-seen, enable/disable), and registered migration entry.
Server Layer & WS Integration
apps/server/src/server.ts, apps/server/src/server.test.ts, apps/server/src/ws.ts
Adjusted server layer composition to provide PrCommentWatcher and projection repository; added WS RPC handlers prWatchToggle and subscribePrWatchStatus; tests wired with mock/no-op PrCommentWatcher.
Web UI & RPC Client
apps/web/src/components/PrWatchToggle.tsx, apps/web/src/components/BranchToolbar.tsx, apps/web/src/components/Sidebar.tsx, apps/web/src/rpc/wsRpcClient.ts
Added PrWatchToggle component, integrated watch toggle into BranchToolbar and Sidebar UI (watch indicator), implemented client RPC methods for toggle and status subscription, and a hook/context to surface watched-thread IDs.
Docs & Design
docs/superpowers/plans/2026-04-13-pr-comment-watcher.md, docs/superpowers/specs/2026-04-13-pr-comment-watcher-design.md
Added implementation plan and detailed design spec describing architecture, flows, data models, and UI/WS integration.

Sequence Diagram(s)

sequenceDiagram
    participant Web as Web Client
    participant Server as Server (WS)
    participant Watcher as PrCommentWatcher
    participant GitHub as GitHub API
    participant Engine as Orchestration Engine
    participant Reactor as PrResponseReactor
    participant Provider as Provider Runtime

    Web->>Server: prWatchToggle(threadId, enabled=true)
    Server->>Watcher: watchThread(threadId, prNumber, cwd)
    Watcher->>GitHub: gh api /repos/.../pulls/{prNumber}/comments
    Watcher->>Watcher: Start polling fiber, track lastSeenIds

    Note over GitHub,Watcher: Polling loop running...

    GitHub-->>Watcher: New review comments detected
    Watcher->>Watcher: filterNewComments(comments, lastSeenId)
    Watcher->>Watcher: buildSyntheticMessage(reviews, comments)
    Watcher->>Engine: dispatch thread.turn.start with synthetic message
    Watcher->>Watcher: storeTurnContext(threadId)

    Engine->>Provider: Process turn (AI analysis)
    Provider-->>Engine: AI response generated

    Engine-->>Reactor: Emit turn.completed event
    Reactor->>Engine: getReadModel().getThread(threadId)
    Reactor->>Reactor: parseAiResponse(assistantMessage)

    loop For each parsed reply
        Reactor->>GitHub: replyToReviewComment or createPrComment
        GitHub-->>Reactor: Success
    end

    Reactor->>GitHub: resolveReviewThread (GraphQL mutation) 
    GitHub-->>Reactor: Thread resolved

    Reactor->>Watcher: clearTurnContext(threadId)
    Reactor->>Watcher: markTurnComplete(threadId)

    Watcher->>Web: streamStatus update (last polled at)
Loading
🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: a PR comment watcher feature that auto-monitors and responds to GitHub PR comments, matching the primary objective.
Description check ✅ Passed The PR description comprehensively covers what changed, why, architecture details, lists all new/modified files, includes test coverage summary, and follows the repository's expected structure for feature PRs despite deviating from the template format.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

Children must go inside the render button element, not as
TooltipTrigger children. Matches base-ui pattern used in Sidebar.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 17


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9d425e5f-c717-4cca-82d6-f53576afdfa0

📥 Commits

Reviewing files that changed from the base of the PR and between f7fa62a and 9ad528f.

📒 Files selected for processing (32)
  • apps/server/integration/OrchestrationEngineHarness.integration.ts
  • apps/server/src/git/Layers/GitHubCli.test.ts
  • apps/server/src/git/Layers/GitHubCli.ts
  • apps/server/src/git/Layers/GitManager.test.ts
  • apps/server/src/git/Services/GitHubCli.ts
  • apps/server/src/orchestration/Layers/OrchestrationReactor.test.ts
  • apps/server/src/orchestration/Layers/OrchestrationReactor.ts
  • apps/server/src/orchestration/Layers/PrCommentWatcher.test.ts
  • apps/server/src/orchestration/Layers/PrCommentWatcher.ts
  • apps/server/src/orchestration/Layers/PrResponseReactor.test.ts
  • apps/server/src/orchestration/Layers/PrResponseReactor.ts
  • apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
  • apps/server/src/orchestration/Services/PrCommentWatcher.ts
  • apps/server/src/orchestration/Services/PrResponseReactor.ts
  • apps/server/src/orchestration/Services/ProviderRuntimeIngestion.ts
  • apps/server/src/persistence/Layers/ProjectionPrWatchers.ts
  • apps/server/src/persistence/Migrations.ts
  • apps/server/src/persistence/Migrations/024_ProjectionPrWatchers.ts
  • apps/server/src/persistence/Services/ProjectionPrWatchers.ts
  • apps/server/src/server.test.ts
  • apps/server/src/server.ts
  • apps/server/src/ws.ts
  • apps/web/src/components/BranchToolbar.tsx
  • apps/web/src/components/PrWatchToggle.tsx
  • apps/web/src/components/Sidebar.tsx
  • apps/web/src/rpc/wsRpcClient.ts
  • docs/superpowers/plans/2026-04-13-pr-comment-watcher.md
  • docs/superpowers/specs/2026-04-13-pr-comment-watcher-design.md
  • packages/contracts/src/git.ts
  • packages/contracts/src/index.ts
  • packages/contracts/src/prCommentWatcher.ts
  • packages/contracts/src/rpc.ts

Comment on lines +240 to +272
listPrReviewComments: (input) =>
execute({
cwd: input.cwd,
args: [
"api",
`/repos/${input.owner}/${input.repo}/pulls/${input.prNumber}/comments`,
"--paginate",
],
}).pipe(
Effect.map((result) => JSON.parse(result.stdout) as ReadonlyArray<PrReviewComment>),
),
listPrReviews: (input) =>
execute({
cwd: input.cwd,
args: [
"api",
`/repos/${input.owner}/${input.repo}/pulls/${input.prNumber}/reviews`,
"--paginate",
],
}).pipe(
Effect.map((result) => JSON.parse(result.stdout) as ReadonlyArray<PrReview>),
),
listPrIssueComments: (input) =>
execute({
cwd: input.cwd,
args: [
"api",
`/repos/${input.owner}/${input.repo}/issues/${input.prNumber}/comments`,
"--paginate",
],
}).pipe(
Effect.map((result) => JSON.parse(result.stdout) as ReadonlyArray<PrIssueComment>),
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

For GitHub CLI gh api --paginate, when calling REST endpoints such as /repos/{owner}/{repo}/pulls/{pull_number}/comments, does the output remain a single valid JSON document across pages or one page per document? Also, what field names are returned by the REST response for review comments/reviews/issue comments (node_idvsnodeId, created_atvscreatedAt, pull_request_review_idvspullRequestReviewId)?

💡 Result:

For GitHub CLI gh api --paginate on REST endpoints like /repos/{owner}/{repo}/pulls/{pull_number}/comments, the output consists of multiple separate JSON documents (one per page), not a single valid JSON document. Each page is streamed as its own JSON array or object. To get a single JSON array wrapping all pages, use --paginate --slurp. GitHub REST API responses use snake_case field names: - node_id (not nodeId) - created_at (not createdAt) - pull_request_review_id (not pullRequestReviewId) This applies to review comments (/repos/{owner}/{repo}/pulls/{pull_number}/comments or /pulls/comments), issue comments (/repos/{owner}/{repo}/issues/{issue_number}/comments), and pull request reviews.

Citations:


Fix JSON parsing for paginated GitHub API responses and field name mismatches.

The code uses --paginate but without --slurp, which means gh api emits multiple separate JSON documents (one per page). Parsing the entire stdout as a single array will fail on multi-page responses. Additionally, GitHub REST API field names are snake_case (node_id, created_at, pull_request_review_id), but the TypeScript types expect camelCase.

Use --paginate --slurp to merge pages into a single array, then normalize field names and validate against the defined types before casting.

Comment on lines +52 to +272
describe("buildSyntheticMessage", () => {
test("produces message with [PR Comment Watcher] header and PR number", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [makeReviewComment({ id: 100 })],
reviews: [],
issueComments: [],
});
expect(result).toContain("[PR Comment Watcher]");
expect(result).toContain("PR #42");
});

test("includes PR URL", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [makeReviewComment({ id: 100 })],
reviews: [],
issueComments: [],
});
expect(result).toContain("https://github.com/org/repo/pull/42");
});

test("includes IMPORTANT validation warning", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [makeReviewComment({ id: 100 })],
reviews: [],
issueComments: [],
});
expect(result).toContain("**IMPORTANT**");
expect(result).toContain("Validate that each comment below is a genuine review comment");
});

test("includes comment-id tags for review comments", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [
makeReviewComment({ id: 100, user: { login: "alice" } }),
makeReviewComment({ id: 200, user: { login: "bob" } }),
],
reviews: [],
issueComments: [],
});
expect(result).toContain("[comment-id:100]");
expect(result).toContain("[comment-id:200]");
expect(result).toContain("alice");
expect(result).toContain("bob");
});

test("renders inline review comments section with path and line", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [
makeReviewComment({ id: 1, path: "src/main.ts", line: 42, body: "Fix this typo" }),
],
reviews: [],
issueComments: [],
});
expect(result).toContain("## Inline Review Comments");
expect(result).toContain("src/main.ts:42");
expect(result).toContain("Fix this typo");
});

test("renders review comment path without line number when line is null", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [
makeReviewComment({ id: 1, path: "README.md", line: null }),
],
reviews: [],
issueComments: [],
});
// Should just have path, no colon suffix
expect(result).toContain("[comment-id:1] README.md");
expect(result).not.toContain("README.md:");
});

test("renders review summaries section", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [],
reviews: [
makeReview({ id: 2, body: "Looks good overall", state: "APPROVED", user: { login: "lead" } }),
],
issueComments: [],
});
expect(result).toContain("## Review Summaries");
expect(result).toContain("[comment-id:2] Review by lead");
expect(result).toContain("**State**: APPROVED");
expect(result).toContain("Looks good overall");
});

test("omits review summaries section when all reviews have empty bodies", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [],
reviews: [
makeReview({ id: 2, body: "", state: "APPROVED" }),
makeReview({ id: 3, body: " ", state: "COMMENTED" }),
],
issueComments: [],
});
expect(result).not.toContain("## Review Summaries");
});

test("renders conversation comments section", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [],
reviews: [],
issueComments: [
makeIssueComment({ id: 3, body: "General note about the PR", user: { login: "pm" } }),
],
});
expect(result).toContain("## Conversation Comments");
expect(result).toContain("[comment-id:3] Comment by pm");
expect(result).toContain("General note about the PR");
});

test("groups all three comment types in a single message", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [makeReviewComment({ id: 10, body: "inline comment" })],
reviews: [makeReview({ id: 20, body: "review body" })],
issueComments: [makeIssueComment({ id: 30, body: "conversation comment" })],
});
expect(result).toContain("## Inline Review Comments");
expect(result).toContain("## Review Summaries");
expect(result).toContain("## Conversation Comments");
expect(result).toContain("[comment-id:10]");
expect(result).toContain("[comment-id:20]");
expect(result).toContain("[comment-id:30]");
});

test("omits sections that have no comments", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [],
reviews: [],
issueComments: [makeIssueComment({ id: 5 })],
});
expect(result).not.toContain("## Inline Review Comments");
expect(result).not.toContain("## Review Summaries");
expect(result).toContain("## Conversation Comments");
});

test("includes createdAt for review comments", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [
makeReviewComment({ id: 1, createdAt: "2026-04-10T10:00:00Z" }),
],
reviews: [],
issueComments: [],
});
expect(result).toContain("**Created**: 2026-04-10T10:00:00Z");
});

test("includes submittedAt for review summaries", () => {
const result = buildSyntheticMessage({
...basePrInput,
reviewComments: [],
reviews: [
makeReview({ id: 1, body: "LGTM", submittedAt: "2026-04-11T09:00:00Z" }),
],
issueComments: [],
});
expect(result).toContain("**Submitted**: 2026-04-11T09:00:00Z");
});
});

// ---------------------------------------------------------------------------
// filterNewComments (comment ID diffing)
// ---------------------------------------------------------------------------

describe("filterNewComments", () => {
test("returns all comments when lastSeenId is null (first poll)", () => {
const comments = [{ id: 1 }, { id: 2 }, { id: 3 }];
expect(filterNewComments(comments, null)).toEqual(comments);
});

test("returns only comments with id > lastSeenId", () => {
const comments = [{ id: 1 }, { id: 2 }, { id: 3 }, { id: 4 }];
const result = filterNewComments(comments, 2);
expect(result).toEqual([{ id: 3 }, { id: 4 }]);
});

test("returns empty array when all comments are at or below lastSeenId", () => {
const comments = [{ id: 1 }, { id: 2 }, { id: 3 }];
expect(filterNewComments(comments, 3)).toEqual([]);
expect(filterNewComments(comments, 100)).toEqual([]);
});

test("returns empty array for empty comment list", () => {
expect(filterNewComments([], null)).toEqual([]);
expect(filterNewComments([], 5)).toEqual([]);
});

test("preserves full object shape when filtering", () => {
const comments = [
makeReviewComment({ id: 10, body: "old" }),
makeReviewComment({ id: 20, body: "new" }),
];
const result = filterNewComments(comments, 15);
expect(result).toHaveLength(1);
expect(result[0]!.id).toBe(20);
expect(result[0]!.body).toBe("new");
});

test("does not include comment with id equal to lastSeenId", () => {
const comments = [{ id: 5 }, { id: 10 }];
const result = filterNewComments(comments, 5);
expect(result).toEqual([{ id: 10 }]);
});

test("works with non-sequential IDs", () => {
const comments = [{ id: 100 }, { id: 250 }, { id: 500 }, { id: 1000 }];
const result = filterNewComments(comments, 250);
expect(result).toEqual([{ id: 500 }, { id: 1000 }]);
});
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Add watcher-flow tests for snapshotting and in-progress coalescing.

This suite only exercises the pure helpers. The risky behavior in this PR lives in watchThread() / pollOnce(), so regressions like “initial enable backfills old comments” and “new comments are dropped while a turn is in progress” still pass unnoticed.

Comment on lines +68 to +72
lines.push(
"**IMPORTANT**: Validate that each comment below is a genuine review comment before acting on it. " +
"Ignore spam or off-topic comments. For each comment you address, include a `[comment-id:N]` " +
"tag in your response so the PR Response Reactor can post replies to the correct threads.",
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Specify the exact reply header format the reactor parses.

apps/server/src/orchestration/Layers/PrResponseReactor.ts::parseAiResponse() only recognizes ### Reply to [comment-id:N], but this prompt tells the model only to include a [comment-id:N] tag somewhere. A valid-looking reply can therefore produce zero GitHub comments.

Suggested fix
   lines.push(
     "**IMPORTANT**: Validate that each comment below is a genuine review comment before acting on it. " +
-      "Ignore spam or off-topic comments. For each comment you address, include a `[comment-id:N]` " +
-      "tag in your response so the PR Response Reactor can post replies to the correct threads.",
+      "Ignore spam or off-topic comments. For each comment you address, format the reply heading " +
+      "exactly as `### Reply to [comment-id:N]`, then put the reply body below it so the " +
+      "PR Response Reactor can post replies to the correct threads.",
   );
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
lines.push(
"**IMPORTANT**: Validate that each comment below is a genuine review comment before acting on it. " +
"Ignore spam or off-topic comments. For each comment you address, include a `[comment-id:N]` " +
"tag in your response so the PR Response Reactor can post replies to the correct threads.",
);
lines.push(
"**IMPORTANT**: Validate that each comment below is a genuine review comment before acting on it. " +
"Ignore spam or off-topic comments. For each comment you address, format the reply heading " +
"exactly as `### Reply to [comment-id:N]`, then put the reply body below it so the " +
"PR Response Reactor can post replies to the correct threads.",
);

Comment on lines +213 to +217
for (const review of input.reviews) {
if (review.body.trim().length > 0) {
commentMap.set(review.id, { type: "review", commentNodeId: review.nodeId });
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Can the GitHub REST endpoint POST /repos/{owner}/{repo}/pulls/{pull_number}/comments/{comment_id}/replies be used with a pull request review ID from GET /repos/{owner}/{repo}/pulls/{pull_number}/reviews, or only with pull request review comment IDs from GET /repos/{owner}/{repo}/pulls/{pull_number}/comments? If review summaries are not replyable this way, what is the correct API behavior?

💡 Result:

No, the GitHub REST endpoint POST /repos/{owner}/{repo}/pulls/{pull_number}/comments/{comment_id}/replies cannot be used with a pull request review ID from GET /repos/{owner}/{repo}/pulls/{pull_number}/reviews. It requires the ID of a pull request review comment from endpoints like GET /repos/{owner}/{repo}/pulls/{pull_number}/comments. Review IDs (from the reviews endpoint) identify entire pull request reviews, which group review comments with a state (e.g., APPROVE, COMMENT) and optional summary body. Each review comment within a review has its own unique comment ID and is associated with a pull_request_review_id (the parent review ID). The replies endpoint specifically creates replies to individual review comments: "For the comment_id, provide the ID of the review comment you are replying to. This must be the ID of a top-level review comment, not a reply to that comment. Replies to replies are not supported." Using a review ID as comment_id will result in a 404 Resource not found error, as it does not match a valid review comment resource. Review summaries (the body field in a review object) are not replyable via this endpoint. There is no API to directly reply to a review summary; replies are threaded under specific review comments. To comment on the review as a whole, use issue comments (since PRs are issues) or create a new review comment.

Citations:


Review summaries (review.id / review.nodeId) cannot be replied to using the reviews endpoint—this will cause 404 errors.

According to GitHub's REST API, the replies endpoint (POST /repos/{owner}/{repo}/pulls/{pull_number}/comments/{comment_id}/replies) requires a pull request review comment ID (from GET /repos/{owner}/{repo}/pulls/{pull_number}/comments), not a review ID (from GET /repos/{owner}/{repo}/pulls/{pull_number}/reviews). Review IDs identify entire review objects (which include a summary body and state like APPROVE or COMMENT). Review summaries themselves are not replyable through this endpoint. Passing a review ID as comment_id will result in a 404 Resource not found error.

If the intent is to respond to a review summary, use issue comments or create a new review comment instead.

Comment on lines +222 to +269
const turnContext: PrWatcherTurnContext = {
threadId: input.threadId,
prNumber: input.prNumber,
prUrl: input.prUrl,
owner: input.owner,
repo: input.repo,
cwd: input.cwd,
commentMap,
};
turnContextMap.set(input.threadId, turnContext);

// TTL cleanup for stale turn context
setTimeout(() => {
if (turnContextMap.get(input.threadId) === turnContext) {
turnContextMap.delete(input.threadId);
}
}, TURN_CONTEXT_TTL_MS);

// Mark turn in progress
yield* Ref.update(turnsInProgressRef, (set) => {
const next = new Set(set);
next.add(input.threadId);
return next;
});

const messageText = buildSyntheticMessage({
reviewComments: input.reviewComments,
reviews: input.reviews,
issueComments: input.issueComments,
prUrl: input.prUrl,
prNumber: input.prNumber,
});

const now = new Date().toISOString();
yield* orchestrationEngine.dispatch({
type: "thread.turn.start",
commandId,
threadId: input.threadId as any,
message: {
messageId,
role: "user",
text: messageText,
attachments: [],
},
runtimeMode: "full-access",
interactionMode: "default",
createdAt: now,
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Roll back turn state if dispatching the synthetic turn fails.

turnContextMap and turnsInProgressRef are updated before orchestrationEngine.dispatch(). If dispatch throws, this thread stays permanently marked in progress and future polls will never dispatch again.

Suggested fix
     const turnContext: PrWatcherTurnContext = {
       threadId: input.threadId,
       prNumber: input.prNumber,
       prUrl: input.prUrl,
       owner: input.owner,
       repo: input.repo,
       cwd: input.cwd,
       commentMap,
     };
     turnContextMap.set(input.threadId, turnContext);

     // TTL cleanup for stale turn context
     setTimeout(() => {
       if (turnContextMap.get(input.threadId) === turnContext) {
         turnContextMap.delete(input.threadId);
       }
     }, TURN_CONTEXT_TTL_MS);

     // Mark turn in progress
     yield* Ref.update(turnsInProgressRef, (set) => {
       const next = new Set(set);
       next.add(input.threadId);
       return next;
     });

     const messageText = buildSyntheticMessage({
       reviewComments: input.reviewComments,
       reviews: input.reviews,
       issueComments: input.issueComments,
       prUrl: input.prUrl,
       prNumber: input.prNumber,
     });

     const now = new Date().toISOString();
-    yield* orchestrationEngine.dispatch({
-      type: "thread.turn.start",
-      commandId,
-      threadId: input.threadId as any,
-      message: {
-        messageId,
-        role: "user",
-        text: messageText,
-        attachments: [],
-      },
-      runtimeMode: "full-access",
-      interactionMode: "default",
-      createdAt: now,
-    });
+    yield* orchestrationEngine.dispatch({
+      type: "thread.turn.start",
+      commandId,
+      threadId: input.threadId as any,
+      message: {
+        messageId,
+        role: "user",
+        text: messageText,
+        attachments: [],
+      },
+      runtimeMode: "full-access",
+      interactionMode: "default",
+      createdAt: now,
+    }).pipe(
+      Effect.onError(() =>
+        Effect.sync(() => {
+          turnContextMap.delete(input.threadId);
+        }).pipe(
+          Effect.andThen(
+            Ref.update(turnsInProgressRef, (set) => {
+              const next = new Set(set);
+              next.delete(input.threadId);
+              return next;
+            }),
+          ),
+        ),
+      ),
+    );
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const turnContext: PrWatcherTurnContext = {
threadId: input.threadId,
prNumber: input.prNumber,
prUrl: input.prUrl,
owner: input.owner,
repo: input.repo,
cwd: input.cwd,
commentMap,
};
turnContextMap.set(input.threadId, turnContext);
// TTL cleanup for stale turn context
setTimeout(() => {
if (turnContextMap.get(input.threadId) === turnContext) {
turnContextMap.delete(input.threadId);
}
}, TURN_CONTEXT_TTL_MS);
// Mark turn in progress
yield* Ref.update(turnsInProgressRef, (set) => {
const next = new Set(set);
next.add(input.threadId);
return next;
});
const messageText = buildSyntheticMessage({
reviewComments: input.reviewComments,
reviews: input.reviews,
issueComments: input.issueComments,
prUrl: input.prUrl,
prNumber: input.prNumber,
});
const now = new Date().toISOString();
yield* orchestrationEngine.dispatch({
type: "thread.turn.start",
commandId,
threadId: input.threadId as any,
message: {
messageId,
role: "user",
text: messageText,
attachments: [],
},
runtimeMode: "full-access",
interactionMode: "default",
createdAt: now,
});
const turnContext: PrWatcherTurnContext = {
threadId: input.threadId,
prNumber: input.prNumber,
prUrl: input.prUrl,
owner: input.owner,
repo: input.repo,
cwd: input.cwd,
commentMap,
};
turnContextMap.set(input.threadId, turnContext);
// TTL cleanup for stale turn context
setTimeout(() => {
if (turnContextMap.get(input.threadId) === turnContext) {
turnContextMap.delete(input.threadId);
}
}, TURN_CONTEXT_TTL_MS);
// Mark turn in progress
yield* Ref.update(turnsInProgressRef, (set) => {
const next = new Set(set);
next.add(input.threadId);
return next;
});
const messageText = buildSyntheticMessage({
reviewComments: input.reviewComments,
reviews: input.reviews,
issueComments: input.issueComments,
prUrl: input.prUrl,
prNumber: input.prNumber,
});
const now = new Date().toISOString();
yield* orchestrationEngine.dispatch({
type: "thread.turn.start",
commandId,
threadId: input.threadId as any,
message: {
messageId,
role: "user",
text: messageText,
attachments: [],
},
runtimeMode: "full-access",
interactionMode: "default",
createdAt: now,
}).pipe(
Effect.onError(() =>
Effect.sync(() => {
turnContextMap.delete(input.threadId);
}).pipe(
Effect.andThen(
Ref.update(turnsInProgressRef, (set) => {
const next = new Set(set);
next.delete(input.threadId);
return next;
}),
),
),
),
);

Comment on lines +26 to +30
<button
type="button"
onClick={handleToggle}
className={`inline-flex items-center gap-1 rounded-md px-2 py-1 text-xs cursor-pointer outline-hidden hover:bg-accent focus-visible:ring-1 focus-visible:ring-ring ${isWatching ? "text-green-400" : "text-muted-foreground opacity-60"}`}
/>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Expose the toggle state to assistive tech.

This is a two-state button, but the rendered <button> does not publish whether monitoring is currently on or off. Add aria-pressed={isWatching} (and ideally a stateful aria-label) so screen readers can announce the current state.

♿ Suggested fix
           <button
             type="button"
             onClick={handleToggle}
+            aria-pressed={isWatching}
+            aria-label={
+              isWatching
+                ? `Disable PR comment monitoring for PR #${prNumber}`
+                : `Enable PR comment monitoring for PR #${prNumber}`
+            }
             className={`inline-flex items-center gap-1 rounded-md px-2 py-1 text-xs cursor-pointer outline-hidden hover:bg-accent focus-visible:ring-1 focus-visible:ring-ring ${isWatching ? "text-green-400" : "text-muted-foreground opacity-60"}`}
           />
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
<button
type="button"
onClick={handleToggle}
className={`inline-flex items-center gap-1 rounded-md px-2 py-1 text-xs cursor-pointer outline-hidden hover:bg-accent focus-visible:ring-1 focus-visible:ring-ring ${isWatching ? "text-green-400" : "text-muted-foreground opacity-60"}`}
/>
<button
type="button"
onClick={handleToggle}
aria-pressed={isWatching}
aria-label={
isWatching
? `Disable PR comment monitoring for PR #${prNumber}`
: `Enable PR comment monitoring for PR #${prNumber}`
}
className={`inline-flex items-center gap-1 rounded-md px-2 py-1 text-xs cursor-pointer outline-hidden hover:bg-accent focus-visible:ring-1 focus-visible:ring-ring ${isWatching ? "text-green-400" : "text-muted-foreground opacity-60"}`}
/>

Comment on lines +176 to +223
useEffect(() => {
if (environmentIds.length === 0) return;

// Maintain a mutable working set and a per-environment snapshot so we can
// correctly rebuild on snapshot/update/remove events without losing state
// from other environments.
const perEnv = new Map<string, Set<string>>();
const unsubscribes: Array<() => void> = [];

for (const envId of environmentIds) {
const connection = readEnvironmentConnection(envId as EnvironmentId);
if (!connection) continue;

perEnv.set(envId, new Set());

const unsub = connection.client.git.onPrWatchStatus((event) => {
if (event._tag === "snapshot") {
const envSet = new Set<string>();
for (const w of event.watchers) {
if (w.enabled) envSet.add(w.threadId);
}
perEnv.set(envId, envSet);
} else if (event._tag === "updated") {
const envSet = perEnv.get(envId) ?? new Set();
if (event.watcher.enabled) {
envSet.add(event.watcher.threadId);
} else {
envSet.delete(event.watcher.threadId);
}
perEnv.set(envId, envSet);
} else if (event._tag === "removed") {
perEnv.get(envId)?.delete(event.threadId);
}

// Rebuild the merged set
const merged = new Set<string>();
for (const envSet of perEnv.values()) {
for (const id of envSet) merged.add(id);
}
setWatchedIds(merged);
});
unsubscribes.push(unsub);
}

return () => {
for (const unsub of unsubscribes) unsub();
};
}, [environmentIds.join(",")]); // eslint-disable-line react-hooks/exhaustive-deps

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Clear watched state when there are no subscribed environments.

When environmentIds becomes empty, this effect returns without resetting watchedIds, so stale eye indicators can remain visible after the last environment disconnects.

Suggested fix
  useEffect(() => {
-    if (environmentIds.length === 0) return;
+    if (environmentIds.length === 0) {
+      setWatchedIds(EMPTY_WATCHED_SET);
+      return;
+    }
+
+    setWatchedIds(EMPTY_WATCHED_SET);

     // Maintain a mutable working set and a per-environment snapshot so we can

Comment on lines +428 to +446
```typescript
it.effect("listPrReviewComments fetches inline review comments via gh api", () =>
Effect.gen(function* () {
mockedRunProcess.mockResolvedValueOnce({
stdout: JSON.stringify([
{ id: 100, node_id: "MDI0", body: "fix this", path: "src/index.ts", line: 42, original_line: 42, side: "RIGHT", in_reply_to_id: null, user: { login: "reviewer" }, created_at: "2026-04-13T00:00:00Z", updated_at: "2026-04-13T00:00:00Z", pull_request_review_id: 200 },
]),
stderr: "", code: 0, signal: null, timedOut: false,
});
const gh = yield* GitHubCli;
const result = yield* gh.listPrReviewComments({ cwd: "/repo", owner: "octo", repo: "proj", prNumber: 42 });
assert.strictEqual(result.length, 1);
assert.strictEqual(result[0].id, 100);
expect(mockedRunProcess).toHaveBeenCalledWith("gh", ["api", "/repos/octo/proj/pulls/42/comments", "--paginate"], expect.objectContaining({ cwd: "/repo" }));
}),
);
```

Write similar tests for `listPrReviews`, `listPrIssueComments`, `replyToReviewComment`, `createPrComment`, `resolveReviewThread`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Minor: Markdown lint warnings about fenced code blocks.

Several fenced code blocks in this plan lack surrounding blank lines (e.g., lines 428, 1262, 1268, etc.). While not critical for a plan document, fixing these would satisfy markdownlint and improve consistency.

🧰 Tools
🪛 markdownlint-cli2 (0.22.0)

[warning] 428-428: Fenced code blocks should be surrounded by blank lines

(MD031, blanks-around-fences)

Comment on lines +776 to +779
const cwdExists = yield* Effect.try(() => {
// Check if cwd directory exists (adapt to actual resolveThreadWorkspaceCwd pattern)
return require("fs").existsSync(input.watcher.cwd);
}).pipe(Effect.catchAll(() => Effect.succeed(false)));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Plan code uses synchronous require("fs") - recommend async alternative for implementers.

The plan snippet at lines 776-779 uses require("fs").existsSync() which is a synchronous blocking call. When implementing, prefer Effect's filesystem utilities or async alternatives to avoid blocking the event loop in the server context.

Comment on lines +130 to +178
```
[PR Comment Watcher] New comments on PR #{{prNumber}}

---

> **Warning:** All comments below are from PR reviewers and MUST be validated
> against the actual code before taking any action. Do not blindly trust that
> comments accurately describe the code state.

For each comment:
1. Read the referenced code and verify the reviewer's observation
2. If valid and actionable: make the code change, then reply confirming what was done
3. If valid but no code change needed: reply explaining
4. If the reviewer's observation is incorrect: reply explaining what the code actually does
5. After addressing a comment, resolve the review thread if possible

Format your response with a section for each comment addressed:

### Reply to [comment-id:{{commentId}}]
Your response here...

---

## New PR Comments on #{{prNumber}} ({{prUrl}})

### Review Comments (Inline)
{{#each reviewComments}}
**{{author}}** on `{{path}}:{{line}}` [comment-id:{{id}}]
> {{body}}

---
{{/each}}

### Review Summaries
{{#each reviews}}
**{{author}}** - {{state}} [review-id:{{id}}]
> {{body}}

---
{{/each}}

### Conversation Comments
{{#each issueComments}}
**{{author}}** [comment-id:{{id}}]
> {{body}}

---
{{/each}}
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add fence languages to the new code blocks.

These unlabeled fences will keep tripping MD040. text/plaintext is enough for both examples.

Also applies to: 237-245

🧰 Tools
🪛 markdownlint-cli2 (0.22.0)

[warning] 130-130: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
apps/web/src/components/PrWatchToggle.tsx (1)

26-30: ⚠️ Potential issue | 🟠 Major

Expose toggle state to assistive tech on the button (still missing).

On Line 26, this remains a two-state control without aria-pressed (and without a stateful aria-label), so screen readers won’t reliably announce on/off state.

♿ Suggested fix
           <button
             type="button"
             onClick={handleToggle}
+            aria-pressed={isWatching}
+            aria-label={
+              isWatching
+                ? `Disable PR comment monitoring for PR #${prNumber}`
+                : `Enable PR comment monitoring for PR #${prNumber}`
+            }
             className={`inline-flex items-center gap-1 rounded-md px-2 py-1 text-xs cursor-pointer outline-hidden hover:bg-accent focus-visible:ring-1 focus-visible:ring-ring ${isWatching ? "text-green-400" : "text-muted-foreground opacity-60"}`}
           >

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: ab5ab6f1-e2c3-4b51-9bfe-221acfb3fc61

📥 Commits

Reviewing files that changed from the base of the PR and between 9ad528f and 34222f1.

📒 Files selected for processing (1)
  • apps/web/src/components/PrWatchToggle.tsx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant