-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Implement Upstash Workflows for better background job handling #2865
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughIntroduces an Upstash Workflow-based “partner-approved” pipeline. Adds a new API workflow route with three steps (create-default-links, send-email, send-webhook). Replaces in-place side-effects in partner approval flows with triggerWorkflows. Adds workflow client, logger, and script; updates dependencies; minor webhook TODO. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Admin as Admin/Server Action
participant App as App Server
participant QW as Upstash Workflow Client
participant API as POST /api/workflows/partner-approved
participant DB as Database
participant Email as Email Service
participant WH as Webhook Publisher
Admin->>App: Approve partner(s)
App->>QW: triggerWorkflows({ workflowId:"partner-approved", body:{programId, partnerId, userId} })
QW-->>API: Invoke workflow HTTP endpoint (batched/retried)
API->>DB: Step 1: derive/create default links
API->>Email: Step 2: send approval emails (idempotent)
API->>WH: Step 3: send partner.enrolled webhook
API-->>QW: Log outcomes
QW-->>App: Trigger result
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
apps/web/lib/partners/approve-partner-enrollment.ts (1)
46-54: Avoid mutating createdAt during approval.Same concern as bulk flow; don’t reset
createdAt. Prefer anapprovedAtfield if available.Apply this diff:
data: { status: "approved", - createdAt: new Date(), + // approvedAt: new Date(), // if schema supportsapps/web/lib/actions/partners/bulk-approve-partners.ts (1)
47-64: Don't overwrite ProgramEnrollment.createdAt — add approvedAt and set that on approval.createdAt is the creation timestamp in packages/prisma/schema/program.prisma; remove the createdAt update in apps/web/lib/actions/partners/bulk-approve-partners.ts, add an optional approvedAt DateTime? to the ProgramEnrollment model, set approvedAt: new Date() in the update, and run a migration.
Files:
- packages/prisma/schema/program.prisma (model ProgramEnrollment)
- apps/web/lib/actions/partners/bulk-approve-partners.ts (updateMany)
Suggested change to TS (remove createdAt; set approvedAt after adding schema field):
data: { status: "approved", - createdAt: new Date(), + // approvedAt: new Date(), // add approvedAt to schema and set here
🧹 Nitpick comments (8)
apps/web/package.json (1)
50-50: Add a guardrail for reproducible builds.Consider pinning
@upstash/workflowto an exact version or using a Renovate/Dependabot rule to auto‑PR updates. This reduces surprise upgrades in critical infra code.apps/web/lib/cron/qstash-workflow-logger.ts (1)
12-36: Include error stacks and structured fields in logs.Capture
error.stackand addworkflowIdinto the object payload for easier querying.Apply this diff:
export function createWorkflowLogger({ workflowId, workflowRunId, }: { workflowId: string; workflowRunId: string; }) { return { info: ({ message, data }: LogData) => { - console.info(`[Upstash Workflow:${workflowId}] ${message}`, { + console.info(`[Upstash Workflow:${workflowId}] ${message}`, { workflowRunId, + workflowId, ...data, }); }, error: ({ message, error, data }: ErrorData) => { - console.error(`[Upstash Workflow:${workflowId}] ${message}`, { + console.error(`[Upstash Workflow:${workflowId}] ${message}`, { workflowRunId, - error: error?.message || error, + workflowId, + error: error?.message || error, + stack: error?.stack, ...data, }); }, }; }apps/web/lib/cron/qstash-workflow.ts (1)
31-33: Flow control key is too coarse; may throttle unrelated runs and allow same‑entity concurrency.Key only uses
workflowId. Use a composite key to serialize per partner/program and prevent duplicates racing.Apply this diff:
- flowControl: { - key: workflow.workflowId, - parallelism: WORKFLOW_PARALLELISM, - }, + flowControl: { + key: [ + workflow.workflowId, + (workflow.body as any)?.programId, + (workflow.body as any)?.partnerId, + ] + .filter(Boolean) + .join(":"), + parallelism: WORKFLOW_PARALLELISM, + },apps/web/lib/partners/approve-partner-enrollment.ts (1)
91-99: Idempotency: safe, but consider per‑entity flowControl.Trigger body is good; with the refactor to composite
flowControl.key, concurrent approvals for the same partner will serialize.apps/web/app/(ee)/api/workflows/partner-approved/route.ts (4)
92-99: Minor: avoid O(n*m) filtering when skipping existing default links.Use a Set of existing
partnerGroupDefaultLinkIds for O(n) filtering.Apply this diff:
- // Skip existing default links - for (const link of links) { - if (link.partnerGroupDefaultLinkId) { - partnerGroupDefaultLinks = partnerGroupDefaultLinks.filter( - (defaultLink) => defaultLink.id !== link.partnerGroupDefaultLinkId, - ); - } - } + // Skip existing default links + const existing = new Set( + links + .map((l) => l.partnerGroupDefaultLinkId) + .filter(Boolean) as string[], + ); + partnerGroupDefaultLinks = partnerGroupDefaultLinks.filter( + (d) => !existing.has(d.id), + );
203-206: Log only non‑PII identifiers.Current log prints recipient emails. Prefer ids/counts to avoid PII in logs.
Apply this diff:
- logger.info({ - message: `Sending email notification to ${partnerUsers.length} partner users.`, - data: partnerUsers, - }); + logger.info({ + message: `Sending email notification to ${partnerUsers.length} partner users.`, + data: { userIds: partnerUsers.map(({ user }) => user.id) }, + });
208-238: Idempotency per recipient and better error logging for resend batch.Use a per‑recipient key to avoid dedup across different users, and include provider error in logs before throwing.
Apply this diff:
- const { data, error } = await resend.batch.send( - partnerUsers.map(({ user }) => ({ + const { data, error } = await resend.batch.send( + partnerUsers.map(({ user }) => ({ subject: `Your application to join ${program.name} partner program has been approved!`, from: VARIANT_TO_FROM_MAP.notifications, to: user.email!, react: PartnerApplicationApproved({ @@ }), headers: { - "Idempotency-Key": `application-approved-${programEnrollment.id}`, + "Idempotency-Key": `application-approved-${programEnrollment.id}-${user.id}`, }, - })), + })), ); if (data) { logger.info({ message: `Sent emails to ${partnerUsers.length} partner users.`, data: data, }); } if (error) { - throw new Error(`Failed to send email notification to partner users.`); + logger.error({ + message: "Failed to send email notification to partner users.", + error, + data: { programEnrollmentId: programEnrollment.id }, + }); + throw new Error("Resend batch.send error"); }
289-292: Harden payload parsing.Guard against empty/non‑JSON payloads to return a 400 instead of throwing.
Apply this diff:
{ - initialPayloadParser: (requestPayload) => { - return payloadSchema.parse(JSON.parse(requestPayload)); - }, + initialPayloadParser: (requestPayload) => { + if (!requestPayload) { + throw new Error("Missing request payload"); + } + return payloadSchema.parse(JSON.parse(requestPayload)); + }, },
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (8)
apps/web/app/(ee)/api/workflows/partner-approved/route.ts(1 hunks)apps/web/lib/actions/partners/bulk-approve-partners.ts(3 hunks)apps/web/lib/cron/qstash-workflow-logger.ts(1 hunks)apps/web/lib/cron/qstash-workflow.ts(1 hunks)apps/web/lib/partners/approve-partner-enrollment.ts(3 hunks)apps/web/lib/webhook/qstash.ts(1 hunks)apps/web/package.json(1 hunks)apps/web/scripts/workflow.ts(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-09-17T17:44:03.921Z
Learnt from: TWilson023
PR: dubinc/dub#2857
File: apps/web/lib/actions/partners/update-program.ts:96-0
Timestamp: 2025-09-17T17:44:03.921Z
Learning: In apps/web/lib/actions/partners/update-program.ts, the team prefers to keep the messagingEnabledAt update logic simple by allowing client-provided timestamps rather than implementing server-controlled timestamp logic to avoid added complexity.
Applied to files:
apps/web/lib/partners/approve-partner-enrollment.tsapps/web/lib/actions/partners/bulk-approve-partners.ts
📚 Learning: 2025-08-26T15:38:48.173Z
Learnt from: TWilson023
PR: dubinc/dub#2736
File: apps/web/lib/api/bounties/get-bounty-or-throw.ts:53-63
Timestamp: 2025-08-26T15:38:48.173Z
Learning: In bounty-related code, getBountyOrThrow returns group objects with { id } field (transformed from BountyGroup.groupId), while other routes working directly with BountyGroup Prisma records use the actual groupId field. This is intentional - getBountyOrThrow abstracts the join table details.
Applied to files:
apps/web/lib/partners/approve-partner-enrollment.ts
🧬 Code graph analysis (4)
apps/web/lib/cron/qstash-workflow.ts (1)
packages/utils/src/constants/main.ts (1)
APP_DOMAIN_WITH_NGROK(20-25)
apps/web/app/(ee)/api/workflows/partner-approved/route.ts (8)
apps/web/lib/cron/qstash-workflow-logger.ts (1)
createWorkflowLogger(13-36)apps/web/lib/api/programs/get-program-enrollment-or-throw.ts (1)
getProgramEnrollmentOrThrow(6-97)apps/web/lib/api/partners/create-partner-default-links.ts (1)
createPartnerDefaultLinks(32-86)apps/web/lib/types.ts (2)
PlanProps(179-179)RewardProps(500-500)apps/web/lib/api/groups/get-group-or-throw.ts (1)
getGroupOrThrow(4-51)packages/email/src/resend/client.ts (1)
resend(3-5)apps/web/ui/partners/program-reward-description.tsx (1)
ProgramRewardDescription(6-97)apps/web/lib/zod/schemas/partners.ts (1)
EnrolledPartnerSchema(306-370)
apps/web/lib/partners/approve-partner-enrollment.ts (2)
apps/web/lib/types.ts (1)
WorkspaceProps(185-201)apps/web/lib/cron/qstash-workflow.ts (1)
triggerWorkflows(19-62)
apps/web/lib/actions/partners/bulk-approve-partners.ts (1)
apps/web/lib/cron/qstash-workflow.ts (1)
triggerWorkflows(19-62)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build
🔇 Additional comments (2)
apps/web/lib/actions/partners/bulk-approve-partners.ts (2)
6-8: Good move to centralize side‑effects via workflows.
66-108: Batch audit logging call likely incorrect; ensure API supports arrays.
recordAuditLogis used with a single object elsewhere. If it doesn’t accept arrays, map to multiple calls.Apply this diff if bulk isn’t supported:
- await Promise.allSettled([ - recordAuditLog( - updatedEnrollments.map(({ partner }) => ({ - workspaceId: workspace.id, - programId: program.id, - action: "partner_application.approved", - description: `Partner application approved for ${partner.id}`, - actor: user, - targets: [ - { - type: "partner", - id: partner.id, - metadata: partner, - }, - ], - })), - ), + await Promise.allSettled([ + ...updatedEnrollments.map(({ partner }) => + recordAuditLog({ + workspaceId: workspace.id, + programId: program.id, + action: "partner_application.approved", + description: `Partner application approved for ${partner.id}`, + actor: user, + targets: [ + { + type: "partner", + id: partner.id, + metadata: partner, + }, + ], + }), + ),
Summary by CodeRabbit
New Features
Refactor
Chores