-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Optimize /cron/payouts #2939
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize /cron/payouts #2939
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughRemoved the standalone createPayout implementation and moved payout grouping, filtering (banned partners, holding period), creation/reuse, assignment, and aggregation into the route handler which now processes pending commissions grouped by programId and partnerId. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Cron as CRON
participant Route as Payouts Route (route.ts)
participant DB as Database
Cron->>Route: Trigger payouts cron
Route->>DB: Query pending commissions grouped by programId, partnerId
Route->>DB: Query program enrollments / banned partners
alt Partner banned
Route->>DB: Update matching pending commissions -> canceled
Route-->>Cron: Log cancellations
else Partner eligible
loop per (programId, partnerId)
Route->>DB: Apply holding-period filter & select commissions/clawbacks
Route->>DB: Find existing pending payout (program, partner)
alt Existing pending payout
Route->>DB: Assign commissions -> set payoutId, mark processed
Route->>DB: Re-aggregate and update payout.amount and periodEnd
else No existing payout
Route->>DB: Create payout (id=createId, periodStart, periodEnd=endOfMonth)
Route->>DB: Assign commissions -> set payoutId, mark processed
Route->>DB: Aggregate and set payout.amount
end
Route-->>Cron: Log created/updated payout
end
end
note over Route,DB #D8EEF8: PeriodStart = earliest commission createdAt<br/>PeriodEnd = end-of-month of latest commission
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai do a full review please – do you see any discrepancies from our old approach? |
🧩 Analysis chain🏁 Script executed: Length of output: 13800 @steven-tey: I'll do a full review comparing the old and new approaches. I've identified several significant discrepancies: Critical Issues1. Commission Status Not Updated
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
apps/web/app/(ee)/api/cron/payouts/create-payout.ts(0 hunks)apps/web/app/(ee)/api/cron/payouts/route.ts(2 hunks)
💤 Files with no reviewable changes (1)
- apps/web/app/(ee)/api/cron/payouts/create-payout.ts
🧰 Additional context used
🧬 Code graph analysis (1)
apps/web/app/(ee)/api/cron/payouts/route.ts (1)
apps/web/lib/api/create-id.ts (1)
createId(66-71)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 4
♻️ Duplicate comments (2)
apps/web/app/(ee)/api/cron/payouts/route.ts (2)
36-43: Fix partner aggregation map.Current reducer overwrites
acc[programId]with a single string, so programs with multiple partners either blow up (partnerId.inexpects an array) or silently drop partners. Revert to collecting arrays of partnerIds (deduping as needed) before issuing the Prisma query.
70-83: Map banned partners bypartnerId.
bannedPartners.map((partner) => partner.id)grabs the join row id, leaving the actual commissions for banned partners untouched. Usepartner.partnerIdso the update cancels the intended records.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
apps/web/app/(ee)/api/cron/payouts/create-payout.ts(0 hunks)apps/web/app/(ee)/api/cron/payouts/route.ts(2 hunks)
💤 Files with no reviewable changes (1)
- apps/web/app/(ee)/api/cron/payouts/create-payout.ts
|
@coderabbitai I've addressed all of your feedback, thank you! can you double check now? |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 13941 @steven-tey: Great progress! I can confirm that you've addressed 3 out of 4 critical issues: ✅ Status Update - Fixed (Line 195) However, there's still one critical issue remaining: ❌ Missing Transaction WrapperThe operations are still not wrapped in a transaction. This means if any step fails partway through, you'll have inconsistent data. Specifically, the risk is: Lines 173-214: Creating/reusing payout → updating commissions → updating payout amount are three separate database operations without atomicity. What could go wrong:
Recommended fix:Wrap the per-partner logic in a transaction: for (const { partnerId, commissions } of partnerIdsToCommissionsArray) {
await prisma.$transaction(async (tx) => {
// All your existing logic here, but replace `prisma` with `tx`
const sortedCommissions = commissions.sort(/* ... */);
const totalEarnings = sortedCommissions.reduce(/* ... */);
const periodStart = sortedCommissions[0].createdAt;
const periodEnd = endOfMonth(/* ... */);
let payoutToUse = existingPendingPayouts.find(/* ... */);
if (!payoutToUse) {
payoutToUse = await tx.payout.create({/* ... */});
}
await tx.commission.updateMany({/* ... */});
if (existingPendingPayouts.find((p) => p.id === payoutToUse.id)) {
await tx.payout.update({/* ... */});
}
});
}This ensures all operations succeed or fail together. Would you like me to provide the complete updated code block? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
♻️ Duplicate comments (2)
apps/web/app/(ee)/api/cron/payouts/route.ts (2)
221-237: Fix period boundary logic when reusing payouts.When appending commissions to an existing payout, the code updates
periodEndto the new value but doesn't account for the existing period boundaries. This can cause reports to show incorrect date ranges:
periodStartis never updated, so if the new batch has earlier commissions, the payout won't reflect themperiodEndshould useMAX(existing, new)logic rather than blindly overwriting, otherwise a new batch from an earlier month could move it backwardsUpdate the payout logic to properly expand period boundaries:
// if we're reusing a pending payout, we need to update the amount if (existingPendingPayouts.find((p) => p.id === payoutToUse.id)) { const updatedPayout = await prisma.payout.update({ where: { id: payoutToUse.id, }, data: { amount: { increment: totalEarnings, }, - periodEnd, + periodStart: { + set: periodStart < payoutToUse.periodStart ? periodStart : payoutToUse.periodStart, + }, + periodEnd: { + set: periodEnd > payoutToUse.periodEnd ? periodEnd : payoutToUse.periodEnd, + }, }, }); console.log( `Since we're reusing payout ${payoutToUse.id}, add the new earnings of ${totalEarnings} to the payout amount, making it ${updatedPayout.amount}`, ); }
191-237: Wrap payout operations in a transaction.The payout creation/update and commission updates are separate database calls without transaction protection. If any operation fails mid-sequence, you'll end up with orphaned payouts (no commissions assigned), commissions pointing to payouts with incorrect totals, or inconsistent state from concurrent cron runs.
Wrap the per-partner block in
prisma.$transaction:for (const { partnerId, commissions } of partnerIdsToCommissionsArray) { - // sort the commissions by createdAt - const sortedCommissions = commissions.sort( - (a, b) => a.createdAt.getTime() - b.createdAt.getTime(), - ); - - // sum the earnings of the commissions - const totalEarnings = sortedCommissions.reduce( - (total, commission) => total + commission.earnings, - 0, - ); - - // earliest commission date - const periodStart = sortedCommissions[0].createdAt; - - // end of the month of the latest commission date - // e.g. if the latest sale is 2024-12-16, the periodEnd should be 2024-12-31 - const periodEnd = endOfMonth( - sortedCommissions[sortedCommissions.length - 1].createdAt, - ); - - let payoutToUse = existingPendingPayouts.find( - (p) => p.partnerId === partnerId, - ); - - if (!payoutToUse) { - payoutToUse = await prisma.payout.create({ - data: { - id: createId({ prefix: "po_" }), - programId, - partnerId, - periodStart, - periodEnd, - amount: totalEarnings, - description: `Dub Partners payout (${program.name})`, - }, - }); - console.log( - `No existing payout found, created new one ${payoutToUse.id} for partner ${partnerId}`, - ); - } - - const updatedCommissions = await prisma.commission.updateMany({ - where: { - id: { in: commissions.map((c) => c.id) }, - }, - data: { - status: "processed", - payoutId: payoutToUse.id, - }, - }); - console.log( - `Updated ${updatedCommissions.count} commissions to have payoutId ${payoutToUse.id}`, - ); - - // if we're reusing a pending payout, we need to update the amount - if (existingPendingPayouts.find((p) => p.id === payoutToUse.id)) { - const updatedPayout = await prisma.payout.update({ - where: { - id: payoutToUse.id, - }, - data: { - amount: { - increment: totalEarnings, - }, - periodEnd, - }, - }); - console.log( - `Since we're reusing payout ${payoutToUse.id}, add the new earnings of ${totalEarnings} to the payout amount, making it ${updatedPayout.amount}`, - ); - } + await prisma.$transaction(async (tx) => { + // sort the commissions by createdAt + const sortedCommissions = commissions.sort( + (a, b) => a.createdAt.getTime() - b.createdAt.getTime(), + ); + + // sum the earnings of the commissions + const totalEarnings = sortedCommissions.reduce( + (total, commission) => total + commission.earnings, + 0, + ); + + // earliest commission date + const periodStart = sortedCommissions[0].createdAt; + + // end of the month of the latest commission date + // e.g. if the latest sale is 2024-12-16, the periodEnd should be 2024-12-31 + const periodEnd = endOfMonth( + sortedCommissions[sortedCommissions.length - 1].createdAt, + ); + + let payoutToUse = existingPendingPayouts.find( + (p) => p.partnerId === partnerId, + ); + + if (!payoutToUse) { + payoutToUse = await tx.payout.create({ + data: { + id: createId({ prefix: "po_" }), + programId, + partnerId, + periodStart, + periodEnd, + amount: totalEarnings, + description: `Dub Partners payout (${program.name})`, + }, + }); + console.log( + `No existing payout found, created new one ${payoutToUse.id} for partner ${partnerId}`, + ); + } + + await tx.commission.updateMany({ + where: { + id: { in: commissions.map((c) => c.id) }, + }, + data: { + status: "processed", + payoutId: payoutToUse.id, + }, + }); + console.log( + `Updated commissions to have payoutId ${payoutToUse.id}`, + ); + + // if we're reusing a pending payout, we need to update the amount + if (existingPendingPayouts.find((p) => p.id === payoutToUse.id)) { + const updatedPayout = await tx.payout.update({ + where: { + id: payoutToUse.id, + }, + data: { + amount: { + increment: totalEarnings, + }, + periodEnd, + }, + }); + console.log( + `Since we're reusing payout ${payoutToUse.id}, add the new earnings of ${totalEarnings} to the payout amount, making it ${updatedPayout.amount}`, + ); + } + }); }
🧹 Nitpick comments (1)
apps/web/app/(ee)/api/cron/payouts/route.ts (1)
168-170: Remove redundant sorting.The commissions are already ordered by
createdAtascending from the database query (lines 130-132), so sorting them again in memory is unnecessary.- // sort the commissions by createdAt - const sortedCommissions = commissions.sort( - (a, b) => a.createdAt.getTime() - b.createdAt.getTime(), - ); - // sum the earnings of the commissions - const totalEarnings = sortedCommissions.reduce( + const totalEarnings = commissions.reduce( (total, commission) => total + commission.earnings, 0, ); // earliest commission date - const periodStart = sortedCommissions[0].createdAt; + const periodStart = commissions[0].createdAt; // end of the month of the latest commission date // e.g. if the latest sale is 2024-12-16, the periodEnd should be 2024-12-31 const periodEnd = endOfMonth( - sortedCommissions[sortedCommissions.length - 1].createdAt, + commissions[commissions.length - 1].createdAt, );
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
apps/web/app/(ee)/api/cron/payouts/route.ts(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
apps/web/app/(ee)/api/cron/payouts/route.ts (1)
apps/web/lib/api/create-id.ts (1)
createId(66-71)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build
Summary by CodeRabbit
Refactor
Bug Fixes