[Fix]: Add better and more informative error handling to email-rendering engines #1124

nams1570 · 2026-01-20T06:51:59Z

Summary of Changes

We ran into an error on sentry when the vercel fallback and freestyle improvements were pushed to prod. Spiking into the error handling revealed that the errors could be more informative to enable easier debugging.

We improve the error handling and add extra test coverage to cover the error pathways through the code. Note that we do not test vercel sandbox itself nor the fallback mechanism-this is because a) these will be logged and tested in prod with the sanity test code, and b) creating a mock vercel sandbox instance the way we have a mock freestyle server would just slow down any tests that pass through the email rendering pipeline, all for something thats meant to just be a fallback. However, locally, we tested with scripts and real vercel sandbox test project to success. Note that we also tried running the existing email-rendering test suite with fake freestyle credentials and real vercel-sandbox credentials (to mimic the fallback) and they passed.

Summary by CodeRabbit

New Features
- Email preview now defaults to user/project when not provided and merges preview variables from templates.
Bug Fixes
- Error responses are consistently JSON-formatted and include stack traces plus richer execution context for clearer troubleshooting.
- Batch processing: a single failing template now fails the entire batch and surfaces the specific template error.
Tests
- Expanded tests covering bundling/runtime errors, validation failures, JSON error formatting, preview behavior, and batch-failure scenarios.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

vercel · 2026-01-20T06:52:05Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
stack-backend	Ready	Preview, Comment	Jan 20, 2026 8:21pm
stack-dashboard	Ready	Preview, Comment	Jan 20, 2026 8:21pm
stack-demo	Ready	Preview, Comment	Jan 20, 2026 8:21pm
stack-docs	Ready	Preview, Comment	Jan 20, 2026 8:21pm

coderabbitai · 2026-01-20T06:52:14Z

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

📝 Walkthrough

Walkthrough

Adds renderEmailWithTemplate; widens ExecuteResult error type to allow unknown errors; enriches error payloads with message/stack/cause and innerCode/innerOptions; JSON-stringifies runtime errors across freestyle and vercel-sandbox flows; adds executeJavascript helper and updates tests for JSON error/assertion behavior.

Changes

Cohort / File(s)	Summary
Email rendering tests `apps/backend/src/lib/email-rendering.test.tsx`	Reworks tests to expect JSON-stringified error payloads and stack traces; renames/repurposes bundling/runtime error tests; adds batch-failure and `renderEmailWithTemplate` preview/preview-variables tests.
Email rendering logic `apps/backend/src/lib/email-rendering.tsx`	Adds exported `renderEmailWithTemplate`; widens `ExecuteResult` error type from `string` → `unknown`; JSON-stringifies runtime errors in dev/test and sandbox paths; returns richer error objects (message, stack, cause); attaches inner metadata on sandbox fallback paths.
JS execution engine `apps/backend/src/lib/js-execution.tsx`	Attaches `innerCode` and `innerOptions` to `StackAssertionError` payloads across Freestyle and Vercel Sandbox flows; improves sandbox result handling (buffer read, guarded parsing); exports `executeJavascript(code, options)` helper; extends sanity/fallback reporting with inner context.
Manifests / deps `apps/backend/package.json`, `package.json`	Bumps `@vercel/sandbox` dependency to `^1.2.0`; minor manifest line adjustments.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant EmailRenderer
  participant FreestyleEngine
  participant VercelSandbox
  participant ResultStore

  Caller->>EmailRenderer: renderEmailWithTemplate(template, vars, opts)
  EmailRenderer->>FreestyleEngine: execute template (freestyle)
  alt Freestyle succeeds
    FreestyleEngine-->>EmailRenderer: { status: "ok", data }
    EmailRenderer->>ResultStore: persist/return OK result
    ResultStore-->>Caller: OK response
  else Freestyle fails
    FreestyleEngine-->>EmailRenderer: { status: "error", error } (includes stack)
    EmailRenderer->>VercelSandbox: fallback run (provide innerCode, innerOptions)
    alt Sandbox succeeds
      VercelSandbox-->>EmailRenderer: { status: "ok", data }
      EmailRenderer->>ResultStore: persist/return OK result
      ResultStore-->>Caller: OK response
    else Sandbox fails
      VercelSandbox-->>EmailRenderer: { status: "error", error } (attached innerCode/innerOptions)
      EmailRenderer-->>Caller: return error (JSON-stringified payload with message, stack, cause, inner metadata)
    end
  end

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

#1091: Modifies the same email-rendering and js-execution paths; introduces executeJavascript and freestyle/vercel-sandbox fallback—directly related.
#969: Adds freestyle execution guards and richer error/context propagation in email-rendering.tsx.
#875: Changes batched rendering behavior and relies on the same ExecuteResult/error shapes adjusted here.

Suggested reviewers

BilalG1
N2D4

Poem

🐰 I hopped through code at dawn's first light,

Wrapped errors tidy, gave stacks a sight.
Inner bits snug in their little den,
One bad template trips the whole batch then.
🥕 Hooray — the rabbit danced again!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change—improving error handling in email-rendering engines—which is clearly reflected in the changeset across multiple files.
Description check	✅ Passed	The description provides a clear summary of changes, motivates the fix with a Sentry error, explains the testing approach, and explicitly notes local manual testing success.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

On Sentry, we saw an error with vercel sandbox. This was thrown because the runner script itself had no safeguards. Also, the error raised was uninformative. So we improve the error handling in the file.

greptile-apps · 2026-01-20T06:54:41Z

Greptile Overview

Greptile Summary

This PR improves error handling in the email rendering pipeline by adding structured error objects with stack traces and comprehensive debugging context.

Key Changes:

Wrapped Vercel Sandbox runner script in try-catch to prevent crashes and ensure all errors are properly captured and serialized
Changed error format from strings to structured objects containing message, stack, and cause fields for better debugging
Added innerCode and innerOptions to all error captures throughout the execution pipeline to provide complete context for Sentry debugging
Enhanced test coverage with new tests for bundling errors, runtime errors, arktype validation failures, and batch failure behavior
Fixed potential issue where Vercel Sandbox runner could crash without returning error information

Impact:
The changes significantly improve observability when email rendering fails in production, making it easier to diagnose issues from Sentry logs without needing to reproduce errors locally.

Confidence Score: 5/5

Safe to merge with minimal risk - defensive error handling improvements with comprehensive test coverage
All changes are defensive improvements to error handling with no breaking changes to existing functionality, backed by thorough test coverage for the new error paths
No files require special attention

Important Files Changed

Filename	Overview
apps/backend/src/lib/js-execution.tsx	Added comprehensive try-catch wrapper to Vercel Sandbox runner script and enhanced error context across all error captures with innerCode and innerOptions for better debugging
apps/backend/src/lib/email-rendering.tsx	Changed error handling to return structured error objects (message, stack, cause) instead of strings, and added innerCode/innerOptions to error captures

Sequence Diagram

sequenceDiagram
    participant Client
    participant EmailRendering
    participant BundleAndExecute
    participant JSExecution
    participant FreestyleEngine
    participant VercelSandbox
    participant RunnerScript

    Client->>EmailRendering: renderEmailsForTenancyBatched(requests)
    EmailRendering->>BundleAndExecute: bundleAndExecute(files)
    BundleAndExecute->>JSExecution: executeJavascript(code, options)
    
    alt Freestyle execution
        JSExecution->>FreestyleEngine: execute(code, options)
        alt Success
            FreestyleEngine-->>JSExecution: {status: "ok", data: result}
            JSExecution-->>BundleAndExecute: result
            BundleAndExecute-->>EmailRendering: Result.ok(data)
        else Error
            FreestyleEngine-->>JSExecution: {status: "error", error: {message, stack, cause}}
            Note over JSExecution: Fallback to Vercel Sandbox
            JSExecution->>VercelSandbox: execute(code, options)
        end
    end
    
    alt Vercel Sandbox execution
        VercelSandbox->>VercelSandbox: Create sandbox
        VercelSandbox->>VercelSandbox: Install node modules
        VercelSandbox->>RunnerScript: Write code.mjs and runner.mjs
        VercelSandbox->>RunnerScript: Run node runner.mjs
        
        alt Runner try-catch success
            RunnerScript->>RunnerScript: import('./code.mjs')
            RunnerScript->>RunnerScript: Execute function
            alt Function success
                RunnerScript->>RunnerScript: {status: "ok", data: result}
            else Function error
                RunnerScript->>RunnerScript: {status: "error", error: {message, stack, cause}}
            end
            RunnerScript->>RunnerScript: writeFileSync(result.json)
            RunnerScript-->>VercelSandbox: exitCode: 0
        else Runner crash
            RunnerScript-->>VercelSandbox: exitCode: non-zero
            Note over VercelSandbox: Unexpected failure (OOM, timeout, etc.)
        end
        
        VercelSandbox->>VercelSandbox: Read result.json
        VercelSandbox->>VercelSandbox: JSON.parse(resultJson)
        VercelSandbox-->>JSExecution: result or error
        
        alt Vercel success
            JSExecution-->>BundleAndExecute: result
            BundleAndExecute->>BundleAndExecute: JSON.stringify(error)
            BundleAndExecute-->>EmailRendering: Result.ok(data)
        else Vercel error
            JSExecution-->>BundleAndExecute: error
            BundleAndExecute->>BundleAndExecute: JSON.stringify(error)
            BundleAndExecute-->>EmailRendering: Result.error(string)
        end
    end
    
    EmailRendering-->>Client: Result<EmailRenderResult[], string>

greptile-apps

_{No files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

Copilot

Pull request overview

This pull request enhances error handling across the email rendering pipeline by adding more informative error context and improving safeguards in the JavaScript execution engines. The changes were prompted by production errors in Sentry after deploying Vercel fallback and Freestyle improvements.

Changes:

Added innerCode and innerOptions context to all error logs throughout the execution pipeline for better debugging
Implemented try-catch wrapper in Vercel Sandbox runner script to handle errors gracefully instead of failing with exit codes
Updated error structure from simple strings to objects containing message, stack, and cause properties
Added comprehensive test coverage for error scenarios including bundling errors, runtime errors, and batch failure behavior

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 11 comments.

File	Description
apps/backend/src/lib/js-execution.tsx	Added error context (`innerCode`, `innerOptions`) to all StackAssertionError instances; implemented try-catch wrapper in Vercel Sandbox runner script to capture execution errors with stack traces
apps/backend/src/lib/email-rendering.tsx	Changed error structure from string to object with message/stack/cause; updated error serialization to use JSON.stringify for structured error objects
apps/backend/src/lib/email-rendering.test.tsx	Added new test cases for bundling errors, runtime errors (component throws, arktype validation), and batch failure scenarios; updated test descriptions for clarity

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

apps/backend/src/lib/js-execution.tsx

apps/backend/src/lib/email-rendering.tsx

apps/backend/src/lib/js-execution.tsx

apps/backend/src/lib/email-rendering.tsx

apps/backend/src/lib/js-execution.tsx

We had a small bug where when the vercel fallback failed, we still returned the freestyle failure. We also make errors more informative.

There is sufficient nuance/ complexity to dissuade merging the two renderEmailxxx functions. While the render-email.test.tsx covers the main path through renderEmailWithTemplate, preview mode is never explicitly tested.

apps/backend/src/lib/js-execution.tsx

We now read from a file buffer rather than cat. Cat returns the same result for an empty file and a nonexistent file. readFile makes it more explicit when the file wasn't created. We simplify runner to better match freestyle behavior

N2D4

as discussed

apps/backend/src/lib/email-rendering.tsx

Copilot AI review requested due to automatic review settings January 20, 2026 06:52

github-actions bot assigned nams1570 Jan 20, 2026

fix: Vercel Sandbox runner error handling

538cced

On Sentry, we saw an error with vercel sandbox. This was thrown because the runner script itself had no safeguards. Also, the error raised was uninformative. So we improve the error handling in the file.

Copilot started reviewing on behalf of nams1570 January 20, 2026 06:52 View session

nams1570 force-pushed the update-freestyle-vercel-sandbox-error-handling branch from 2ffb861 to a4dc1a0 Compare January 20, 2026 06:52

greptile-apps bot reviewed Jan 20, 2026

View reviewed changes

Copilot AI reviewed Jan 20, 2026

View reviewed changes

nams1570 added 3 commits January 19, 2026 23:09

fix: better error handling for email-rendering

24816c1

We had a small bug where when the vercel fallback failed, we still returned the freestyle failure. We also make errors more informative.

chore: add test coverage for error paths in email-rendering

9231e2f

chore: add preview mode tests for renderEmailWithTemplate

cc82e50

There is sufficient nuance/ complexity to dissuade merging the two renderEmailxxx functions. While the render-email.test.tsx covers the main path through renderEmailWithTemplate, preview mode is never explicitly tested.

nams1570 force-pushed the update-freestyle-vercel-sandbox-error-handling branch from a4dc1a0 to cc82e50 Compare January 20, 2026 07:09

vercel bot reviewed Jan 20, 2026

View reviewed changes

apps/backend/src/lib/js-execution.tsx Outdated Show resolved Hide resolved

nams1570 requested a review from N2D4 January 20, 2026 08:09

fix: simplify runner script and parsing

96f7762

We now read from a file buffer rather than cat. Cat returns the same result for an empty file and a nonexistent file. readFile makes it more explicit when the file wasn't created. We simplify runner to better match freestyle behavior

N2D4 requested changes Jan 20, 2026

View reviewed changes

nams1570 requested a review from N2D4 January 20, 2026 19:45

N2D4 reviewed Jan 20, 2026

View reviewed changes

apps/backend/src/lib/email-rendering.tsx Show resolved Hide resolved

N2D4 approved these changes Jan 20, 2026

View reviewed changes

Merge branch 'dev' into update-freestyle-vercel-sandbox-error-handling

200fe07

nams1570 merged commit bea1de4 into dev Jan 20, 2026
24 checks passed

nams1570 deleted the update-freestyle-vercel-sandbox-error-handling branch January 20, 2026 20:38

coderabbitai bot mentioned this pull request Jan 28, 2026

[Refactor] [Fix] Email Rendering Pipeline Refactor, Error Handling, and Bug Fixes #1140

Open

[Fix]: Add better and more informative error handling to email-rendering engines #1124

[Fix]: Add better and more informative error handling to email-rendering engines #1124

Conversation

nams1570 commented Jan 20, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of Changes

Summary by CodeRabbit

Uh oh!

vercel bot commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai bot commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Other AI code review bot(s) detected

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Possibly related PRs

Suggested reviewers

Poem

Uh oh!

greptile-apps bot commented Jan 20, 2026

Greptile Overview

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

N2D4 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nams1570 commented Jan 20, 2026 •

edited by coderabbitai bot

Loading

vercel bot commented Jan 20, 2026 •

edited

Loading

coderabbitai bot commented Jan 20, 2026 •

edited

Loading