-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Add retry mechanism for failed requests #3213
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughSwaps many cron routes to a new cron-specific error responder, changes the core API error helper to accept an options object (and detect transient Prisma errors), and adds a QStash retry-queue utility for retrying transient request failures. Changes
Estimated code review effortπ― 4 (Complex) | β±οΈ ~45 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touchesβ Failed checks (1 warning)
β Passed checks (2 passed)
β¨ Finishing touches
π§ͺ Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 4
Caution
Some comments are outside the diff and canβt be posted inline due to platform limitations.
β οΈ Outside diff range comments (3)
apps/web/app/api/analytics/dashboard/route.ts (1)
162-171: FixwaitUntilusage; error handler change is fineThe switch to
handleAndReturnErrorResponse({ error })aligns with the function's optionsβobject signature. However, this line:waitUntil(await redis.set(cacheKey, response, { ex: 60 }));awaits the Redis write before passing it to
waitUntil, sowaitUntilreceives the resolved value ("OK") rather than the Promise. You likely want:waitUntil(redis.set(cacheKey, response, { ex: 60 }));so the cache write runs as deferred work as intended.
apps/web/app/(ee)/api/cron/network/calculate-program-similarities/route.ts (1)
202-211: QStash URL is missing the/networksegmentThe route is located at
api/cron/network/calculate-program-similarities, but thepublishJSONcall at line 203 uses/api/cron/calculate-program-similarities, which is a non-existent route. This will cause scheduled follow-up batches to fail.Update to:
- url: `${APP_DOMAIN_WITH_NGROK}/api/cron/calculate-program-similarities`, + url: `${APP_DOMAIN_WITH_NGROK}/api/cron/network/calculate-program-similarities`,apps/web/app/(ee)/api/cron/utils.ts (1)
1-33:β οΈ Missing retry mechanism contradicts PR objectives.The PR is titled "Add retry mechanism for failed requests" and the AI summary claims this introduces "Prisma-based transient error detection for retry decisions" and "a QStash-aware retry queue utility for failed requests." However,
handleCronErrorResponseonly formats error responsesβit doesn't detect transient errors, queue retries, or implement any retry logic.If the retry mechanism exists in other files not included in this review, please clarify. Otherwise, this appears to be a straightforward error handling refactoring rather than a retry implementation.
β»οΈ Duplicate comments (4)
apps/web/app/(ee)/api/cron/workspaces/delete/route.ts (1)
109-109: Same QStash retry logic concern as other routes.This route also uses
verifyQstashSignature(line 19) but switches tohandleCronErrorResponse. Please ensure the QStash retry mechanism is preserved in the new error handler.apps/web/app/(ee)/api/cron/cleanup/expired-tokens/route.ts (1)
67-67: Same QStash retry logic concern as other routes.This route uses
verifyQstashSignature(line 19) but switches tohandleCronErrorResponse. Ensure QStash retry logic is preserved.apps/web/app/(ee)/api/cron/payouts/process/route.ts (1)
83-83: Same QStash retry logic concern as other routes.This route uses
verifyQstashSignature(line 31) but switches tohandleCronErrorResponse. Ensure QStash retry logic is preserved.apps/web/app/(ee)/api/cron/groups/remap-default-links/route.ts (1)
10-10: Cron utils integration is consistent with other routesUsing
{ handleCronErrorResponse, logAndRespond }here matches the pattern in other cron routes; same note as inimport/csvabout confirming whether the simpler cron error handler should influence QStash retry behavior.Also applies to: 230-238
π§Ή Nitpick comments (19)
apps/web/app/app.dub.co/(dashboard)/[slug]/(ee)/program/payouts/payout-stats.tsx (1)
140-141: Recommended refactor: Remove redundant " USD" suffix.The
currencyFormatterfunction already formats amounts with a currency symbol (e.g.,$1,234.56). Appending" USD"results in redundant currency indication (e.g.,$1,234.56 USD). Consider passing the currency option explicitly if needed, or removing the manual suffix.As per the currencyFormatter documentation: the function uses
Intl.NumberFormatwith the currency style and defaults to USD when no explicit currency is provided.Example refactor:
- {currencyFormatter(eligiblePendingPayouts?.amount ?? 0, {}) + - " USD"} + {currencyFormatter(eligiblePendingPayouts?.amount ?? 0, { currency: "USD" })}You may defer this cleanup to a future PR if prioritizing other changes.
Also applies to: 198-198
apps/web/lib/auth/partner.ts (1)
171-174: Consider passing requestHeaders for QStash retry support.The error handling update is correct and properly passes
responseHeaders. However, since this function has access toreq, consider also passingrequestHeadersto enable QStash-aware retry logic when partner routes are called via QStash callbacks.Apply this diff to add requestHeaders support:
} catch (error) { return handleAndReturnErrorResponse({ error, responseHeaders, + requestHeaders: req.headers, }); }apps/web/ui/modals/domain-auto-renewal-modal.tsx (1)
96-98: Minor formatting change unrelated to PR objective.The text reflow here appears to be an unrelated formatting change. While it has no functional impact, it's worth noting that this cosmetic change doesn't align with the PR's retry mechanism objective.
apps/web/app/api/oauth/userinfo/route.ts (1)
73-77: Good improvement with consistent error naming.The change correctly updates error handling to use the new object-based signature and improves consistency by renaming
etoerror. The CORS headers are properly passed asresponseHeaders.For completeness with the retry mechanism, consider also passing
requestHeaders:} catch (error) { return handleAndReturnErrorResponse({ error, responseHeaders: CORS_HEADERS, + requestHeaders: req.headers, }); }apps/web/ui/customers/customer-sales-table.tsx (1)
61-61: Cosmetic simplification - unrelated to PR objective.The cell renderer has been simplified to a concise single-line form. While this improves readability, it's unrelated to the retry mechanism that is the focus of this PR.
apps/web/ui/customers/customer-partner-earnings-table.tsx (1)
36-36: Cosmetic simplification - unrelated to PR objective.Both the "Sale Amount" and "Commission" cell renderers have been simplified to concise single-line forms. While this improves code consistency with
customer-sales-table.tsx, these formatting changes are unrelated to the retry mechanism objective of this PR.Also applies to: 41-41
apps/web/lib/api/queue-failed-request-for-retry.ts (2)
44-53: Consider wrapping QStash publish in try-catch.If
qstash.publishJSONfails (network issues, QStash unavailable), the error will propagate and could mask the original error or cause unexpected behavior in the caller. Since this is a best-effort retry mechanism, failures should be logged but not block the error response.- const response = await qstash.publishJSON({ - url, - method, - body: await parseRequestBody(errorReq), - headers: { - ...Object.fromEntries(errorReq.headers.entries()), - }, - delay: "10s", - retries: 5, - }); - - if (response.messageId) { + try { + const response = await qstash.publishJSON({ + url, + method, + body: await parseRequestBody(errorReq), + headers: { + ...Object.fromEntries(errorReq.headers.entries()), + }, + delay: "10s", + retries: 5, + }); + + if (response.messageId) { + console.log("Request queued for retry", { + method, + url, + messageId: response.messageId, + }); + + logger.info("request.retry.queued", { + url, + method, + messageId: response.messageId, + }); + + await logger.flush(); + } + } catch (e) { + console.error("Failed to queue request for retry", { method, url, error: e }); + logger.error("request.retry.queue_failed", { url, method, error: e }); + await logger.flush(); + }
48-50: Consider filtering sensitive headers before forwarding.Forwarding all headers to QStash may inadvertently include sensitive information beyond the API key (e.g., cookies, internal tracing headers). Consider explicitly allowlisting headers that are necessary for retry.
+const ALLOWED_RETRY_HEADERS = [ + "authorization", + "content-type", + "x-api-version", + // Add other necessary headers +]; + +function filterHeaders(headers: Headers): Record<string, string> { + const filtered: Record<string, string> = {}; + for (const key of ALLOWED_RETRY_HEADERS) { + const value = headers.get(key); + if (value) filtered[key] = value; + } + return filtered; +}Then use
headers: filterHeaders(errorReq.headers)instead of spreading all entries.apps/web/lib/api/errors.ts (1)
59-63: Consider exportingErrorHandlerOptionsfor caller type safety.The interface is used in the public
handleAndReturnErrorResponsesignature but isn't exported. Callers may need to import this type to properly type their error handling code.-interface ErrorHandlerOptions { +export interface ErrorHandlerOptions { error: unknown; responseHeaders?: Headers; requestHeaders?: Headers; }apps/web/app/(ee)/api/cron/import/csv/route.ts (1)
24-24: Confirm cron retry behavior with new error handlerThis route now uses
handleCronErrorResponse({ error }), which (perapps/web/app/(ee)/api/cron/utils.ts) just maps the error to{ error, status }and doesnβt look at the request headers or setUpstash-NonRetryable-ErrorlikehandleAndReturnErrorResponsedoes. If this endpoint is invoked by QStash Cron, that may change how permanent vs transient errors are retried. Worth confirming whether cron failures should always be retried, or if you also want the nonβretryable header logic here.Also applies to: 124-132
apps/web/app/api/links/metatags/route.ts (1)
34-45: Preserve CORS headers on error responsesWrapping the error in
{ error }matches the newhandleAndReturnErrorResponsesignature. To keep CORS behavior consistent, consider:} catch (error) { return handleAndReturnErrorResponse({ error, responseHeaders: corsHeaders, }); }so callers receive the same CORS headers on failures as on success.
apps/web/app/(ee)/api/cron/payouts/charge-succeeded/route.ts (1)
66-73: Standardized cron error response; consider tightening log typingSwitching the catch path to
handleCronErrorResponse({ error })keeps this route aligned with the new cron-wide error handling and retry semantics. To make the log call a bit safer, consider guardingerror.message(as you already do in other routes) with aninstanceof Errorcheck orString(error).apps/web/app/(ee)/api/cron/bounties/notify-partners/route.ts (1)
211-218: Error response now aligned with cron-wide behaviorDelegating to
handleCronErrorResponse({ error })after logging ensures a uniform JSON+status shape for cron failures, which is important for the new retry flow. As a minor improvement, you could mirror theinstanceof Errorguard used in other routes to avoid relying onerror.messagewhenerrormight not be anError.apps/web/app/(ee)/api/cron/import/tolt/route.ts (1)
47-49: Consistent cron error response; consider adding a logReturning
handleCronErrorResponse({ error })standardizes the failure shape for this import job. If you want parity with other cron routes, consider logging the error (and perhapspayload.action) before returning, to make diagnosing failed Tolt imports easier.apps/web/app/(ee)/api/cron/groups/sync-utm/route.ts (1)
145-151: Unified cron error response; optional hardening of logThe switch to
handleCronErrorResponse({ error })after logging ensures failures from this sync job surface through the standardized cron error pipeline. As a minor improvement, you may want to mirror theinstanceof Errorguard pattern from other routes soerror.messageaccess is always safe.apps/web/app/(ee)/api/cron/import/short/route.ts (1)
61-68: Cron error response integration looks good; minor logging nitThe outer catch now logs and then returns
handleCronErrorResponse({ error }), which is aligned with the new cron error model and retry behavior. If you want to be defensive, you can guarderror.messagein the log (and in the inner catch when building theDubApiErrormessage) in case a non-Error slips through.apps/web/app/(ee)/api/cron/payouts/balance-available/route.ts (1)
166-173: Standardized cron error handling; optional log robustnessUsing
handleCronErrorResponse({ error })after logging ensures payout failures are reported through the same cron error pipeline that drives retries and monitoring. As a small hardening step, you might guarderror.messagein the log with aninstanceof Errorcheck, to be resilient to non-Error throws.apps/web/app/(ee)/api/cron/fx-rates/route.ts (1)
5-5: Cron error handler integration is correct; consider minor type-safety tweakThe import path and
handleCronErrorResponse({ error })usage look correct and align this route with the new cron error handling.If your TS config uses
useUnknownInCatchVariables, you may eventually want to narrowerrorbefore interpolatingerror.messagein the log (e.g.,error instanceof Error ? error.message : String(error)), but thatβs an optional clean-up and not introduced by this PR.Also applies to: 52-52
apps/web/app/(ee)/api/cron/payouts/process/updates/route.ts (1)
9-9: Error routing wired correctly; log message could be clearerThe import path and
handleCronErrorResponse({ error })usage look correct and hook this updates worker into the shared cron error / retry handling.Optionally, you might later tweak the log message (
Error sending Stripe payout) to more clearly reference thepayouts/process/updatesjob so logs are easier to grep by route, but thatβs not blocking.Also applies to: 180-180
apps/web/app/(ee)/api/cron/cleanup/demo-embed-partners/route.ts
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
π§Ή Nitpick comments (1)
apps/web/app/(ee)/api/cron/import/rebrandly/route.ts (1)
87-94: Cron error handling correctly delegates tohandleCronErrorResponse(optional hardening)Routing the outer catch through
handleCronErrorResponse({ error })standardizes cron responses and aligns with the cron-specific error pattern described in the learnings (no QStash callback detection or Upstash nonβretry headers in cron routes). Based on learnings, this is the intended behavior.If your
tsconfigusesuseUnknownInCatchVariables, you might optionally normalize the error before accessing.message:- } catch (error) { - await log({ - message: `Error importing Rebrandly links: ${error.message}`, - type: "cron", - }); - - return handleCronErrorResponse({ error }); - } + } catch (error) { + const err = error instanceof Error ? error : new Error(String(error)); + + await log({ + message: `Error importing Rebrandly links: ${err.message}`, + type: "cron", + }); + + return handleCronErrorResponse({ error: err }); + }Purely a safety/clarity tweak; behavior is otherwise sound.
π Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
π Files selected for processing (2)
apps/web/app/(ee)/api/cron/cleanup/demo-embed-partners/route.ts(2 hunks)apps/web/app/(ee)/api/cron/import/rebrandly/route.ts(3 hunks)
π§ Files skipped from review as they are similar to previous changes (1)
- apps/web/app/(ee)/api/cron/cleanup/demo-embed-partners/route.ts
π§° Additional context used
π§ Learnings (5)
π Common learnings
Learnt from: devkiran
Repo: dubinc/dub PR: 3213
File: apps/web/app/(ee)/api/cron/auto-approve-partner/route.ts:122-122
Timestamp: 2025-12-15T16:45:51.667Z
Learning: In the Dub codebase, cron endpoints under apps/web/app/(ee)/api/cron/ use handleCronErrorResponse for error handling, which intentionally does NOT detect QStash callbacks or set Upstash-NonRetryable-Error headers. This allows QStash to retry all cron job errors using its native retry mechanism. The selective retry logic (queueFailedRequestForRetry) is only used for specific user-facing API endpoints like /api/track/lead, /api/track/sale, and /api/links to retry only transient Prisma database errors.
π Learning: 2025-12-09T12:54:41.818Z
Learnt from: devkiran
Repo: dubinc/dub PR: 3207
File: apps/web/lib/cron/with-cron.ts:27-56
Timestamp: 2025-12-09T12:54:41.818Z
Learning: In `apps/web/lib/cron/with-cron.ts`, the `withCron` wrapper extracts the request body once and provides it to handlers via the `rawBody` parameter. Handlers should use this `rawBody` string parameter (e.g., `JSON.parse(rawBody)`) rather than reading from the Request object via `req.json()` or `req.text()`.
Applied to files:
apps/web/app/(ee)/api/cron/import/rebrandly/route.ts
π Learning: 2025-12-15T16:45:51.667Z
Learnt from: devkiran
Repo: dubinc/dub PR: 3213
File: apps/web/app/(ee)/api/cron/auto-approve-partner/route.ts:122-122
Timestamp: 2025-12-15T16:45:51.667Z
Learning: In cron endpoints under apps/web/app/(ee)/api/cron, continue using handleCronErrorResponse for error handling. Do not detect QStash callbacks or set Upstash-NonRetryable-Error headers in these cron routes, so QStash can retry cron errors via its native retry mechanism. The existing queueFailedRequestForRetry logic should remain limited to specific user-facing API endpoints (e.g., /api/track/lead, /api/track/sale, /api/links) to retry only transient Prisma/database errors. This pattern should apply to all cron endpoints under the cron directory in this codebase.
Applied to files:
apps/web/app/(ee)/api/cron/import/rebrandly/route.ts
π Learning: 2025-05-29T09:49:19.604Z
Learnt from: devkiran
Repo: dubinc/dub PR: 2433
File: apps/web/ui/modals/add-payment-method-modal.tsx:60-62
Timestamp: 2025-05-29T09:49:19.604Z
Learning: The `/api/workspaces/${slug}/billing/payment-methods` POST endpoint in the billing API returns either an error (handled by response.ok check) or a response object containing a `url` property for successful requests.
Applied to files:
apps/web/app/(ee)/api/cron/import/rebrandly/route.ts
π Learning: 2025-06-18T20:26:25.177Z
Learnt from: TWilson023
Repo: dubinc/dub PR: 2538
File: apps/web/ui/partners/overview/blocks/commissions-block.tsx:16-27
Timestamp: 2025-06-18T20:26:25.177Z
Learning: In the Dub codebase, components that use workspace data (workspaceId, defaultProgramId) are wrapped in `WorkspaceAuth` which ensures these values are always available, making non-null assertions safe. This is acknowledged as a common pattern in their codebase, though not ideal.
Applied to files:
apps/web/app/(ee)/api/cron/import/rebrandly/route.ts
𧬠Code graph analysis (1)
apps/web/app/(ee)/api/cron/import/rebrandly/route.ts (1)
apps/web/app/(ee)/api/cron/utils.ts (1)
handleCronErrorResponse(22-33)
β° Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build
π Additional comments (2)
apps/web/app/(ee)/api/cron/import/rebrandly/route.ts (2)
1-7: Error-handling imports are consistent and necessaryKeeping
DubApiErrorfor explicitbad_requestthrows and addinghandleCronErrorResponsefor the top-level catch cleanly separates domain errors from cron response formatting. This matches the shared cron-utils pattern; no issues here.
82-85: Workspace error message formatting fix looks goodThe updated message string:
message: `Workspace: ${workspace?.slug || workspaceId}. Error: ${error.message}`,removes the stray
$and reads clearly while preserving the workspace context and underlying error. All good here.
Summary by CodeRabbit
Bug Fixes
New Features
Refactor
βοΈ Tip: You can customize this high-level summary in your review settings.