ent/mutation.go (2)

53985-53987: ClearedFields includes scheduled_job_id.
Duplicate of earlier nillable concern for this field.

---

54032-54034: ClearField handles scheduled_job_id.
Duplicate of earlier nillable concern for this field.

internal/integration/stripe/subscription.go (6)

181-185: Return NotFound (not Internal) when mapping is missing; optionally import-on-miss

Treating a missing mapping as Internal masks a real “not found” case and hampers recovery paths.

Apply this diff to reclassify:

-    return ierr.NewError("no existing subscription mapping found").
-        WithHint("Existing subscription mapping not found").
-        Mark(ierr.ErrInternal)
+    return ierr.NewError("no existing subscription mapping found").
+        WithHint("No FlexPrice mapping found for the given Stripe subscription ID").
+        WithReportableDetails(map[string]interface{}{"stripe_subscription_id": stripeSubscriptionID}).
+        Mark(ierr.ErrNotFound)

Optionally, you can choose to auto-import when missing:

• Call s.CreateSubscription(txCtx, stripeSubscriptionID, services) and return nil on success.
• Guard against duplicates with a unique constraint on the mapping. I can draft this change if desired.

---

206-211: Fix log key: it's a Stripe subscription ID

Use a precise field name to avoid confusion in logs.

-    s.logger.Errorw("failed to update subscription",
-        "error", err,
-        "subscription_id", stripeSubscriptionID)
+    s.logger.Errorw("failed to update subscription",
+        "error", err,
+        "stripe_subscription_id", stripeSubscriptionID)

---

415-462: Apply the same billing-anchor fix in the transactional create path

The same epoch-anchor issue exists in createFlexPriceSubscription. Mirror the change to avoid 1970-01-01 defaults.

Suggested change (illustrative):

-    billingAnchor := time.Unix(stripeSub.BillingCycleAnchor, 0).UTC()
+    var billingAnchor *time.Time
+    if stripeSub.BillingCycleAnchor != 0 {
+        t := time.Unix(stripeSub.BillingCycleAnchor, 0).UTC()
+        billingAnchor = &t
+    }
@@
-        BillingAnchor:      &billingAnchor,
+        BillingAnchor:      billingAnchor,

Also applies to: 505-520

---

464-551: Clarify naming and dedupe request-building logic

• The helper name “createFlexPriceSubscriptionWithoutTx” is misleading: it still uses the passed context, which is a txCtx when called from UpdateSubscription. Consider renaming to “createFlexPriceSubscriptionDirect” and document “no additional transaction wrapper.”
• Extract a small builder to remove duplication between the two create helpers.

Example extraction:

func (s *stripeSubscriptionService) buildCreateReq(stripeSub *stripe.Subscription, customerID, planID string) (dto.CreateSubscriptionRequest, error) {
    // compute billingPeriod, dates, trial, optional billingAnchor (as per the anchor fix)
    // return dto.CreateSubscriptionRequest{...}, nil
}

Then both helpers call buildCreateReq(...) and pass to SubscriptionService.CreateSubscription(...).

---

372-385: Guard against missing/partial price info in billing-period calc

If Items or Price.Recurring is absent, this can panic. Add simple guards and default gracefully.

Example:

 func (s *stripeSubscriptionService) calculateBillingPeriod(stripeSub *stripe.Subscription) types.BillingPeriod {
-    switch stripeSub.Items.Data[0].Price.Recurring.Interval {
+    if stripeSub.Items == nil || len(stripeSub.Items.Data) == 0 || stripeSub.Items.Data[0].Price == nil || stripeSub.Items.Data[0].Price.Recurring == nil {
+        return types.BILLING_PERIOD_MONTHLY
+    }
+    switch stripeSub.Items.Data[0].Price.Recurring.Interval {

---

189-194: Minor: fix typos in comments

“exisitng” → “existing”; “subcription” → “subscription”.

internal/integration/stripe/customer.go (1)

204-220: Consider restructuring the externalID assignment for clarity.

The externalID variable is assigned twice:

• Line 204: Set to stripeCustomer.ID
• Line 219: Overwritten with lookupKey if it exists

While functionally correct, this pattern can be confusing. Consider moving the initial assignment closer to where it's used in Step 3 (line 254) or using a more explicit conditional assignment.

Apply this diff to improve clarity:

 func (s *CustomerService) CreateCustomerFromStripe(ctx context.Context, stripeCustomer *stripe.Customer, environmentID string, customerService interfaces.CustomerService) error {
-	externalID := stripeCustomer.ID
-
 	// Step 1: Check by flexprice_customer_id, if exists just return
 	if flexpriceID, exists := stripeCustomer.Metadata["flexprice_customer_id"]; exists && flexpriceID != "" {
 		existing, err := customerService.GetCustomer(ctx, flexpriceID)
 		if err == nil && existing != nil {
 			s.logger.Infow("FlexPrice customer already exists, skipping creation",
 				"flexprice_customer_id", flexpriceID,
 				"stripe_customer_id", stripeCustomer.ID)
 			return nil
 		}
 	}
 
+	// Determine external ID: prefer lookup key, fallback to Stripe customer ID
+	externalID := stripeCustomer.ID
 	// Step 2: Check by flexprice_lookup_key
 	if lookupKey, exists := stripeCustomer.Metadata["flexprice_lookup_key"]; exists && lookupKey != "" {
 		externalID = lookupKey

internal/repository/clickhouse/feature_usage.go (1)

1393-1422: Consider cursor-based pagination for better performance.

Using OFFSET for pagination in ClickHouse, especially with ORDER BY timestamp DESC, can be expensive for large datasets. Each query must scan and skip all preceding rows. For exports that may process millions of records, consider refactoring to use cursor-based pagination with a WHERE timestamp < ? clause instead.

Example cursor-based approach:

// First batch: no cursor
query := `SELECT ... FROM feature_usage FINAL
WHERE tenant_id = ? AND environment_id = ?
  AND timestamp >= ? AND timestamp < ?
  AND sign = 1
ORDER BY timestamp DESC
LIMIT ?`

// Subsequent batches: add cursor
query := `SELECT ... FROM feature_usage FINAL
WHERE tenant_id = ? AND environment_id = ?
  AND timestamp >= ? AND timestamp < ?
  AND timestamp < ? -- cursor from last batch
  AND sign = 1
ORDER BY timestamp DESC
LIMIT ?`

This eliminates the need to skip rows and improves query performance, especially for large offsets.

ent/migrate/schema.go (3)

1828-1830: Confirm file_url default/non-nullability won’t break existing data

file_url is reintroduced with Default: "" and not marked Nullable. If previous rows had NULLs, ALTER might fail or coerce unexpectedly.

If nulls existed, consider making file_url Nullable (and enforce non-empty at write-time) to avoid migration issues.

---

1374-1427: Consider FK from scheduled_jobs.connection_id to connections.id

Enforcing referential integrity prevents pointing to deleted/missing connections.

Do this in ent/schema/scheduledjob.go via an edge to Connection and regenerate. Prefer ON DELETE RESTRICT (or SET NULL if business rules allow).

---

1417-1425: Unique index scope check

Unique(connection_id, entity_type) is fine if connection IDs are globally unique. If not guaranteed globally, include tenant/environment in the unique index.

internal/integration/s3/config.go (1)

26-31: Prefer returning (Config, error) with validation of required fields

Returning nil hides cause; bucket/region/creds should be validated early.

Example:

• Signature: NewConfigFromConnection(...) (*Config, error)
• Validate Bucket, Region, AWS creds; normalize Compression/Encryption to allowed set.

Also applies to: 50-63

ent/schema/scheduledjob.go (3)

24-80: Constrain interval/entity_type using enums to prevent invalid values

Using free-form strings risks invalid data. Define ent.Enum with allowed values (daily/weekly/monthly, valid entity types), or central constants mirror.

---

24-80: Add edge to Connection to generate FK

Define:

// In ScheduledJob schema
edge.To("connection", Connection.Type).
    Field("connection_id").
    Unique()

Then regenerate to emit FK and improve integrity.

---

83-97: Consider tenant/environment-scoped unique constraint

If connection IDs aren’t globally unique, make the unique index include tenant_id/environment_id.

internal/api/router.go (1)

508-513: Gate /v1/test endpoints behind feature flags in production

They’re private but still sensitive. Recommend env/flag guard or build tags to disable in prod.

internal/domain/scheduledjob/repository.go (2)

28-30: Clarify “enabled-only” semantics for GetByEntityType

Comment says “enabled scheduled jobs,” but signature doesn’t convey that. Either:

• enforce enabled=true in implementation and update comment to be explicit, or
• accept an Enabled filter parameter for parity with List.

---

38-48: Document/validate pagination and filters

Please specify behavior for Limit/Offset when zero/negative, and enforce sane caps (e.g., max page size) in implementations to avoid unbounded scans. Also note whether TenantID/EnvironmentID are required.

internal/service/sync/export/base.go (2)

14-21: Avoid []byte for large exports; prefer streaming

Returning raw []byte encourages loading entire datasets in memory. Consider:

• Replace PrepareData with a streaming API (e.g., io.Reader, csv.Writer/JSON encoder) or
• Have Export handle streaming directly without an intermediate []byte.

---

63-71: Validate request upfront

Add lightweight checks:

• EntityType not empty/supported,
• ConnectionID/TenantID/EnvID non-empty,
• StartTime <= EndTime.
  Fail fast with ErrValidation + hint.

internal/temporal/activities/export/export_activity.go (1)

16-21: Deduplicate exporter selection; delegate to ExportService

Avoid drift by wiring ExportService into the activity and delegating ExportData to it.

Apply this refactor:

 type ExportActivity struct {
   featureUsageRepo events.FeatureUsageRepository
   s3Client         *s3Integration.Client
   logger           *logger.Logger
+  exportService    *syncExport.ExportService
 }
 
 func NewExportActivity(
   featureUsageRepo events.FeatureUsageRepository,
   s3Client *s3Integration.Client,
   logger *logger.Logger,
 ) *ExportActivity {
   return &ExportActivity{
     featureUsageRepo: featureUsageRepo,
     s3Client:         s3Client,
     logger:           logger,
+    exportService:    syncExport.NewExportService(featureUsageRepo, s3Client, logger),
   }
 }
 
@@
- // Get the appropriate exporter based on entity type
- var response *syncExport.ExportResponse
- var err error
-
- switch input.EntityType {
- case types.ExportEntityTypeFeatureUsage:
-   exporter := syncExport.NewUsageExporter(a.featureUsageRepo, a.s3Client, a.logger)
-   response, err = exporter.Export(ctx, request)
- // Add more entity types as needed
- // case types.ExportEntityTypeCustomer:
- //     exporter := syncExport.NewCustomerExporter(...)
- //     response, err = exporter.Export(ctx, request)
- default:
-   return nil, ierr.NewError("unsupported entity type").
-     WithHintf("Entity type '%s' is not supported for export", input.EntityType).
-     Mark(ierr.ErrValidation)
- }
+ // Delegate to ExportService (handles unknown entity validation)
+ response, err := a.exportService.Export(ctx, request)
+ if err != nil {
+   a.logger.Errorw("export failed", "error", err, "entity_type", input.EntityType)
+   return nil, err
+ }

Also applies to: 23-33, 77-89

internal/service/scheduled_job_orchestrator.go (3)

87-88: Honor job.Enabled when creating schedule

Start paused if the job is disabled.

-  Paused: false, // Start immediately
+  Paused: !job.Enabled, // Respect job enabled state

---

68-71: Timezone/overlap policy

Consider setting an explicit timezone (e.g., UTC or tenant-specific) and an overlap policy to prevent concurrent runs if one overruns. Verify available fields in your Temporal SDK version (e.g., ScheduleSpec Timezone and Overlap options).

---

222-239: Manual window alignment

Current logic subtracts a fixed duration from now, which may split periods. Consider aligning to period boundaries (start of hour/day/week/month) or using LastRunAt/NextRunAt when available to avoid gaps/overlaps.

internal/temporal/workflows/export/execute_export_workflow.go (2)

43-51: Use per-activity timeouts; export likely needs a longer window

Exporting can exceed 10m. Keep short timeouts for task CRUD; extend only the export step.

Apply this diff to add a longer timeout context for the export step:

   // Activity options
   activityOptions := workflow.ActivityOptions{
-    StartToCloseTimeout: 10 * time.Minute, // Max time for each activity
+    StartToCloseTimeout: 2 * time.Minute, // Task create/update/complete are quick
     RetryPolicy: &temporal.RetryPolicy{
       MaximumAttempts: 3,
     },
   }
   ctx = workflow.WithActivityOptions(ctx, activityOptions)
+
+  // Longer timeout only for the export step
+  exportActivityOptions := workflow.ActivityOptions{
+    StartToCloseTimeout: 30 * time.Minute,
+    RetryPolicy: &temporal.RetryPolicy{
+      MaximumAttempts:    3,
+      BackoffCoefficient: 2.0,
+    },
+  }
+  ctxExport := workflow.WithActivityOptions(ctx, exportActivityOptions)

And update the export call to use ctxExport:

// change at Line 101
err = workflow.ExecuteActivity(ctxExport, exportActivity.ExportData, exportInput).Get(ctx, &exportOutput)

---

132-136: Completion failure can leave task stuck in PROCESSING

If CompleteTask fails, the task may remain PROCESSING despite a successful export.

Proposed approach:

• Retry CompleteTask with backoff.
• If it still fails, attempt a best-effort UpdateTaskStatus to a terminal status (e.g., COMPLETED) with minimal metadata.

Please confirm the existence of a Completed/Success status in internal/types/task.go and wire a fallback update if available. If desired, I can generate a patch once the status constant name is confirmed.

internal/service/sync/export/usage_export.go (3)

193-199: Validate and default JobConfig before use

Call SetDefaults + Validate on S3 job config to catch early misconfiguration.

-  // Validate job config is provided
-  if request.JobConfig == nil {
+  // Validate job config is provided and correct
+  if request.JobConfig == nil {
     return nil, ierr.NewError("job configuration is required").
       WithHint("S3 job configuration must be provided in the export request").
       Mark(ierr.ErrValidation)
   }
+  request.JobConfig.SetDefaults()
+  if err := request.JobConfig.Validate(); err != nil {
+    return nil, ierr.WithError(err).
+      WithHint("Invalid S3 job configuration").
+      Mark(ierr.ErrValidation)
+  }

---

223-231: Filename should include extension and reflect compression; honor key prefix

Add .csv (or .csv.gz) for better content-type/UX and include optional KeyPrefix.

-  // Generate filename with timestamp
-  filename := fmt.Sprintf("feature_usage_%s", time.Now().Format("20060102_150405"))
+  // Generate filename with timestamp and extension
+  ts := time.Now().Format("20060102_150405")
+  compressed := request.JobConfig != nil && request.JobConfig.Compression == "gzip"
+  base := fmt.Sprintf("feature_usage_%s.csv", ts)
+  if compressed {
+    base += ".gz"
+  }
+  // Prepend optional key prefix (ensure single slash)
+  filename := base
+  if kp := request.JobConfig.KeyPrefix; kp != "" {
+    // avoid double slashes
+    if strings.HasSuffix(kp, "/") {
+      filename = kp + base
+    } else {
+      filename = kp + "/" + base
+    }
+  }

Note: Add strings to imports.

---

191-244: Large export memory use; consider streaming or multipart upload

Buffering full CSV in memory can OOM on large ranges. Prefer:

• Stream rows via io.Pipe to S3 multipart upload.
• Or chunk by MaxFileSizeMB (config) and upload multiple parts/files.

I can draft a streaming UploadCSV variant with io.Pipe and S3 multipart if your S3 client supports it.

internal/domain/scheduledjob/model.go (2)

31-34: Avoid magic string for status "published"

Use a typed constant (e.g., types.StatusPublished) to prevent drift with DB/Ent values.

---

68-106: Simplify and harden S3 config parsing using JSON round-trip; handle int/float types

Manual map type assertions are brittle. Marshal the map and unmarshal into S3JobConfig to normalize types (e.g., float64 vs int).

Add import:

import "encoding/json"

Apply this diff within GetS3JobConfig:

-  config := &types.S3JobConfig{}
-
-  // Parse from map
-  if bucket, ok := j.JobConfig["bucket"].(string); ok {
-    config.Bucket = bucket
-  }
-  if region, ok := j.JobConfig["region"].(string); ok {
-    config.Region = region
-  }
-  if keyPrefix, ok := j.JobConfig["key_prefix"].(string); ok {
-    config.KeyPrefix = keyPrefix
-  }
-  if compression, ok := j.JobConfig["compression"].(string); ok {
-    config.Compression = compression
-  }
-  if encryption, ok := j.JobConfig["encryption"].(string); ok {
-    config.Encryption = encryption
-  }
-  if endpointURL, ok := j.JobConfig["endpoint_url"].(string); ok {
-    config.EndpointURL = endpointURL
-  }
-  if virtualHostStyle, ok := j.JobConfig["virtual_host_style"].(bool); ok {
-    config.VirtualHostStyle = virtualHostStyle
-  }
-  if maxFileSizeMB, ok := j.JobConfig["max_file_size_mb"].(float64); ok {
-    config.MaxFileSizeMB = int(maxFileSizeMB)
-  }
+  config := &types.S3JobConfig{}
+  raw, err := json.Marshal(j.JobConfig)
+  if err != nil {
+    return nil, fmt.Errorf("marshal job_config: %w", err)
+  }
+  if err := json.Unmarshal(raw, config); err != nil {
+    return nil, fmt.Errorf("unmarshal job_config: %w", err)
+  }

Keep defaults + validate as you already do.

internal/service/scheduled_job_test.go (3)

15-51: Remove or use MockScheduledJobRepo

It’s unused and can confuse future readers. Drop it or add tests that leverage it.

---

227-230: Unused helper

boolPtr is unused; remove to satisfy linters.

---

114-141: Expand test coverage

Add tests for:

• ScheduledJob.GetS3JobConfig (defaults, validation errors, int/float handling).
• IsEnabled/IsDue behavior (status + enabled, NextRunAt cases).

I can scaffold table-driven tests for these quickly if you’d like.

Also applies to: 169-190, 192-225

internal/integration/s3/client.go (2)

79-82: Preserve root cause when wrapping GetDecryptedS3Config error.

NewError discards the original error stack/context.

-	if err != nil {
-		return nil, nil, ierr.NewError("failed to get S3 configuration").
-			WithHint("Invalid S3 configuration").
-			Mark(ierr.ErrValidation)
-	}
+	if err != nil {
+		return nil, nil, ierr.WithError(err).
+			WithHint("failed to get S3 configuration").
+			Mark(ierr.ErrValidation)
+	}

---

59-75: Differentiate NotFound vs other repository errors when fetching connection.

All errors are marked ErrNotFound, potentially masking DB/system failures. Prefer preserving the cause and marking appropriately when the repo indicates not-found.

• If the repository exposes a sentinel/not-found check, use it to decide ErrNotFound vs ErrDatabase/ErrInternal.
• Otherwise, wrap with WithError(err) and add a hint.

Would you like a targeted patch after confirming the repository error contract?

internal/temporal/activities/export/task_activity.go (3)

65-65: Don't set StartedAt when status is PENDING

StartedAt should be set on first transition to PROCESSING (UpdateTaskStatus already handles it). Setting it at creation skews timing and metrics.

-		StartedAt: &now,
+		StartedAt: nil,

---

51-51: Normalize to UTC and keep UpdatedAt current on writes

• Use time.Now().UTC() for timestamps and store/format times in UTC.
• Set BaseModel.UpdatedAt on updates to preserve auditability.

-	now := time.Now()
+	now := time.Now().UTC()
@@
-	now := time.Now()
+	now := time.Now().UTC()
+	existingTask.BaseModel.UpdatedAt = now
@@
-	now := time.Now()
+	now := time.Now().UTC()
+	existingTask.BaseModel.UpdatedAt = now
@@
-			"start_time": input.StartTime.Format(time.RFC3339),
-			"end_time":   input.EndTime.Format(time.RFC3339),
+			"start_time": input.StartTime.UTC().Format(time.RFC3339),
+			"end_time":   input.EndTime.UTC().Format(time.RFC3339),
@@
-	existingTask.Metadata["completed_at"] = now.Format(time.RFC3339)
+	existingTask.Metadata["completed_at"] = now.Format(time.RFC3339)

Please confirm if repository hooks already maintain UpdatedAt; if so, we can skip manual assignment.

Also applies to: 130-131, 186-187, 62-64, 193-195

---

122-127: Avoid taking a pointer to an input field directly

Take a local copy before assigning to a pointer field to avoid holding references into the input struct.

-		existingTask.TotalRecords = &input.RecordInfo.TotalRecords
+		total := input.RecordInfo.TotalRecords
+		existingTask.TotalRecords = &total

internal/service/scheduled_job.go (1)

80-81: Use UTC for persisted timestamps.

Consistency across services; avoids TZ drift.

-now := time.Now()
+now := time.Now().UTC()
@@
-	now := time.Now()
+	now := time.Now().UTC()

Also applies to: 195-197

ent/scheduledjob/where.go (2)

272-275: Case‑insensitive contains may be slow in Postgres

ContainsFold typically lowers both sides, defeating indexes. If these filters are on hot paths, consider CITEXT columns, trigram/GIN indexes, or dedicated searchable columns.

Also applies to: 707-710

---

37-55: Caution using GT/LT on opaque string IDs

IDGT/GTE/LT/LTE compare lexicographically. Unless IDs are ULIDs (time‑sortable), prefer CreatedAt/UpdatedAt for pagination/range queries.

internal/integration/stripe/subscription.go (1)

424-424: Avoid epoch billing anchor when BillingCycleAnchor is 0 (duplicate issue).

Both createFlexPriceSubscription (line 424) and the new createFlexPriceSubscriptionWithoutTx (lines 485, 517) unconditionally set billingAnchor = time.Unix(stripeSub.BillingCycleAnchor, 0).UTC(). When BillingCycleAnchor == 0, this produces 1970-01-01, which is likely unintended. Only set the anchor when provided; otherwise leave it nil.

Apply this diff to createFlexPriceSubscription:

-    billingAnchor := time.Unix(stripeSub.BillingCycleAnchor, 0).UTC()
+    // Only set billing anchor when present
+    var billingAnchor *time.Time
+    if stripeSub.BillingCycleAnchor != 0 {
+        t := time.Unix(stripeSub.BillingCycleAnchor, 0).UTC()
+        billingAnchor = &t
+    }
 
     // Set trial dates if applicable
     ...
     createReq := dto.CreateSubscriptionRequest{
         ...
-        BillingAnchor:      &billingAnchor,
+        BillingAnchor:      billingAnchor,
         ...
     }

Apply the same fix to createFlexPriceSubscriptionWithoutTx:

-    billingAnchor := time.Unix(stripeSub.BillingCycleAnchor, 0).UTC()
+    // Only set billing anchor when present
+    var billingAnchor *time.Time
+    if stripeSub.BillingCycleAnchor != 0 {
+        t := time.Unix(stripeSub.BillingCycleAnchor, 0).UTC()
+        billingAnchor = &t
+    }
 
     // Set trial dates if applicable
     ...
     createReq := dto.CreateSubscriptionRequest{
         ...
-        BillingAnchor:      &billingAnchor,
+        BillingAnchor:      billingAnchor,
         ...
     }

Also applies to: 485-485, 517-517

internal/api/v1/test_usage_export.go (2)

98-115: Retain the selected job to use its ConnectionID.

Capture the enabled job you choose so you can pass ConnectionID to the export request.

@@
-	// Use the first enabled job
-	var jobConfig *types.S3JobConfig
+	// Use the first enabled job
+	var (
+		selectedJob *scheduledjob.ScheduledJob
+		jobConfig   *types.S3JobConfig
+	)
 	var configErr error
 	for _, job := range scheduledJobs {
 		if job.Enabled {
 			jobConfig, configErr = job.GetS3JobConfig()
 			if configErr != nil {
@@
-			break
+			selectedJob = job
+			break
 		}
 	}
@@
-	if jobConfig == nil {
+	if jobConfig == nil || selectedJob == nil {
 		h.logger.Warnw("no enabled scheduled job found for feature usage export")
 		c.JSON(http.StatusNotFound, gin.H{
 			"error": "No enabled scheduled job found for feature usage export. Please enable a scheduled job.",
 		})
 		return
 	}

---

128-135: Include ConnectionID in ExportRequest (required for S3 credentials).

ConnectionID is needed to resolve/decrypt S3 creds. Use the selected job’s ConnectionID.

 request := &export.ExportRequest{
-		EntityType: types.ExportEntityTypeEvents,
+		EntityType:   types.ExportEntityTypeEvents,
+		ConnectionID: selectedJob.ConnectionID,
 		TenantID:   tenantID,
 		EnvID:      envID,
 		StartTime:  startTime,
 		EndTime:    endTime,
 		JobConfig:  jobConfig,
 }

internal/temporal/activities/export/task_activity.go (2)

27-37: Add input validation for CreateTaskInput.

Validate required fields and time window to prevent unusable tasks.

Add:

// Validate checks required fields and time window.
func (in CreateTaskInput) Validate() error {
	if in.TaskID == "" {
		return fmt.Errorf("task_id is required")
	}
	if in.WorkflowID == "" {
		return fmt.Errorf("workflow_id is required")
	}
	if in.ScheduledJobID == "" || in.TenantID == "" || in.EnvID == "" || in.EntityType == "" {
		return fmt.Errorf("scheduled_job_id, tenant_id, env_id, and entity_type are required")
	}
	if in.StartTime.IsZero() || in.EndTime.IsZero() {
		return fmt.Errorf("start_time and end_time are required")
	}
	if in.EndTime.Before(in.StartTime) {
		return fmt.Errorf("end_time must be >= start_time")
	}
	return nil
}

Then call it at the start of CreateTask.

---

45-92: Make CreateTask idempotent and normalize timestamps.

• Idempotency: check for existing task by TaskID and return it (Temporal retries).
• Time: use UTC; metadata times in UTC; don’t set StartedAt at creation (set when processing begins).
• Context: set tenant/env on ctx for repo access.
• WorkflowID: store nil when empty.

 func (a *TaskActivity) CreateTask(ctx context.Context, input CreateTaskInput) (*CreateTaskOutput, error) {
+	// Validate input
+	if err := input.Validate(); err != nil {
+		return nil, ierr.NewError(err.Error()).Mark(ierr.ErrValidation)
+	}
 	a.logger.Infow("creating export task",
@@
-	// Create task with pre-generated ID
-	now := time.Now()
+	// Bind tenant/env to context for repository operations
+	ctx = types.SetTenantID(ctx, input.TenantID)
+	ctx = types.SetEnvironmentID(ctx, input.EnvID)
+
+	// Idempotency: if task already exists, return it
+	if existing, err := a.taskRepo.Get(ctx, input.TaskID); err == nil && existing != nil {
+		a.logger.Infow("export task already exists (idempotent)", "task_id", input.TaskID)
+		return &CreateTaskOutput{TaskID: existing.ID}, nil
+	}
+
+	now := time.Now().UTC()
+	var wfID *string
+	if input.WorkflowID != "" {
+		wfID = &input.WorkflowID
+	}
 	newTask := &task.Task{
-		ID:             input.TaskID,      // Use pre-generated ID
-		WorkflowID:     &input.WorkflowID, // Store Temporal workflow ID
+		ID:             input.TaskID, // Use pre-generated ID
+		WorkflowID:     wfID,         // Store Temporal workflow ID if present
 		EnvironmentID:  input.EnvID,
 		TaskType:       types.TaskTypeExport,
 		EntityType:     types.EntityType(input.EntityType),
 		ScheduledJobID: input.ScheduledJobID,
 		FileURL:        "", // Will be set after upload
 		FileType:       types.FileTypeCSV,
 		TaskStatus:     types.TaskStatusPending,
 		Metadata: map[string]interface{}{
-			"start_time": input.StartTime.Format(time.RFC3339),
-			"end_time":   input.EndTime.Format(time.RFC3339),
+			"start_time": input.StartTime.UTC().Format(time.RFC3339),
+			"end_time":   input.EndTime.UTC().Format(time.RFC3339),
 		},
-		StartedAt: &now,
+		StartedAt: nil,
 		BaseModel: types.BaseModel{
 			TenantID:  input.TenantID,
 			Status:    types.StatusPublished,
 			CreatedAt: now,
 			UpdatedAt: now,
 		},
 	}

ent/mutation.go (2)

39649-39666: Clear methods for non-nillable fields (see past review comments)*

The issue with ClearCreatedBy(), ClearUpdatedBy(), and ClearEnvironmentID() methods for non-nillable fields has already been identified in past review comments with verification scripts and fix suggestions. These methods will cause NOT NULL violations at runtime.

Also applies to: 39698-39715, 39747-39764

---

52824-52871: Clear methods for non-nillable scheduled_job_id field (see past review comments)*

The issue with ClearScheduledJobID() method for the non-nillable scheduled_job_id field has already been identified in past review comments with verification scripts and fix suggestions. This method will cause NOT NULL violations at runtime.

internal/integration/s3/export.go (2)

82-86: Use Content-Encoding: gzip; keep original Content-Type

Set ContentType based on the uncompressed format and add ContentEncoding="gzip" when compressing. This avoids mislabeling payloads as application/gzip.

Apply this diff:

-	// Determine content type
-	contentType := request.ContentType
-	if contentType == ""
-	{
-		contentType = c.getContentType(request.Format, request.Compress)
-	}
+	// Determine content type (based on uncompressed format)
+	contentType := request.ContentType
+	if contentType == "" {
+		contentType = c.getContentType(request.Format, false)
+	}

@@
-	uploadInput := &s3.PutObjectInput{
+	uploadInput := &s3.PutObjectInput{
 		Bucket:      aws.String(c.config.Bucket),
 		Key:         aws.String(key),
 		Body:        bytes.NewReader(data),
 		ContentType: aws.String(contentType),
+		// Optional: set ContentLength for clarity
+		ContentLength: aws.Int64(int64(len(data))),
 	}
+	// Add content encoding when compressed
+	if request.Compress && c.config.Compression == "gzip" {
+		uploadInput.ContentEncoding = aws.String("gzip")
+	}

Also applies to: 88-95

---

176-183: MaxFileSizeMB=0 currently blocks all uploads; treat 0 as “no limit”

Only enforce the size check when the configured limit is > 0.

Apply this diff:

-	// Check max file size
-	maxSizeBytes := int64(c.config.MaxFileSizeMB * 1024 * 1024)
-	if int64(len(request.Data)) > maxSizeBytes {
-		return ierr.NewErrorf("file size exceeds maximum allowed size of %d MB", c.config.MaxFileSizeMB).
-			WithHintf("Reduce the file size or increase the max_file_size_mb limit").
-			Mark(ierr.ErrValidation)
-	}
+	// Check max file size (0 or negative => no limit)
+	if c.config.MaxFileSizeMB > 0 {
+		maxSizeBytes := int64(c.config.MaxFileSizeMB) * 1024 * 1024
+		if int64(len(request.Data)) > maxSizeBytes {
+			return ierr.NewErrorf("file size exceeds maximum allowed size of %d MB", c.config.MaxFileSizeMB).
+				WithHintf("Reduce the file size or increase the max_file_size_mb limit").
+				Mark(ierr.ErrValidation)
+		}
+	}

internal/service/scheduled_job.go (4)

135-145: Authorize Get by tenant/environment.

Add a post-fetch scope check to avoid ID enumeration across tenants (as flagged in previous review).

 func (s *scheduledJobService) GetScheduledJob(ctx context.Context, id string) (*dto.ScheduledJobResponse, error) {
 	job, err := s.repo.Get(ctx, id)
 	if err != nil {
 		s.logger.Errorw("failed to get scheduled job", "id", id, "error", err)
 		return nil, ierr.WithError(err).
 			WithHint("Scheduled job not found").
 			Mark(ierr.ErrNotFound)
 	}
+
+	// Enforce tenant/environment scoping
+	if job.TenantID != types.GetTenantID(ctx) || job.EnvironmentID != types.GetEnvironmentID(ctx) {
+		return nil, ierr.ErrNotFound
+	}
 
 	return dto.ToScheduledJobResponse(job), nil
 }

---

174-258: Authorize Update by tenant/environment; recalc next run when enabling.

Missing authorization check and NextRunAt recalculation when enabling (as flagged in previous review).

 	// Get existing job
 	job, err := s.repo.Get(ctx, id)
 	if err != nil {
 		s.logger.Errorw("failed to get scheduled job for update", "id", id, "error", err)
 		return nil, ierr.WithError(err).
 			WithHint("Scheduled job not found").
 			Mark(ierr.ErrNotFound)
 	}
 
+	// Enforce tenant/environment scoping
+	if job.TenantID != types.GetTenantID(ctx) || job.EnvironmentID != types.GetEnvironmentID(ctx) {
+		return nil, ierr.ErrNotFound
+	}
 
 	// Track if enabled status changed
 	wasEnabled := job.Enabled
@@
 	if req.Enabled != nil {
 		job.Enabled = *req.Enabled
+		// Recalc NextRunAt if just enabled
+		if !wasEnabled && job.Enabled {
+			now := time.Now()
+			nextRun := job.CalculateNextRunTime(now)
+			job.NextRunAt = &nextRun
+		}
 	}

---

260-272: Stop Temporal schedule and authorize before delete.

Currently deletes DB record only; schedule may keep running. Also no scope check (as flagged in previous review).

 func (s *scheduledJobService) DeleteScheduledJob(ctx context.Context, id string) error {
-	err := s.repo.Delete(ctx, id)
+	// Fetch for scoping and to stop schedule
+	job, getErr := s.repo.Get(ctx, id)
+	if getErr != nil {
+		s.logger.Errorw("failed to get scheduled job for delete", "id", id, "error", getErr)
+		return ierr.WithError(getErr).
+			WithHint("Failed to delete scheduled job").
+			Mark(ierr.ErrDatabase)
+	}
+	// Enforce tenant/environment scoping
+	if job.TenantID != types.GetTenantID(ctx) || job.EnvironmentID != types.GetEnvironmentID(ctx) {
+		return ierr.ErrNotFound
+	}
+	// Stop schedule first
+	if s.orchestrator != nil {
+		if err := s.orchestrator.StopScheduledJob(ctx, job.ID); err != nil {
+			s.logger.Errorw("failed to stop temporal schedule before delete", "id", job.ID, "error", err)
+		}
+	}
+	err := s.repo.Delete(ctx, id)
 	if err != nil {
 		s.logger.Errorw("failed to delete scheduled job", "id", id, "error", err)
 		return ierr.WithError(err).
 			WithHint("Failed to delete scheduled job").
 			Mark(ierr.ErrDatabase)
 	}
 
 	s.logger.Infow("scheduled job deleted successfully", "id", id)
 	return nil
 }

---

293-308: Scope GetByEntityType by tenant/environment; prefer List(filters).

Avoid unscoped reads; reuse ListFilters to include tenant/environment (as flagged in previous review).

-func (s *scheduledJobService) GetScheduledJobsByEntityType(ctx context.Context, entityType types.ScheduledJobEntityType) ([]*dto.ScheduledJobResponse, error) {
-	jobs, err := s.repo.GetByEntityType(ctx, string(entityType))
+func (s *scheduledJobService) GetScheduledJobsByEntityType(ctx context.Context, entityType types.ScheduledJobEntityType) ([]*dto.ScheduledJobResponse, error) {
+	filters := &scheduledjob.ListFilters{
+		TenantID:      types.GetTenantID(ctx),
+		EnvironmentID: types.GetEnvironmentID(ctx),
+		EntityType:    string(entityType),
+		Limit:         1000,
+		Offset:        0,
+	}
+	jobs, err := s.repo.List(ctx, filters)
 	if err != nil {
 		s.logger.Errorw("failed to get scheduled jobs by entity type", "entity_type", entityType, "error", err)
 		return nil, ierr.WithError(err).
 			WithHint("Failed to retrieve scheduled jobs").
 			Mark(ierr.ErrDatabase)
 	}
 
 	responses := make([]*dto.ScheduledJobResponse, 0, len(jobs))
 	for _, job := range jobs {
 		responses = append(responses, dto.ToScheduledJobResponse(job))
 	}
 
 	return responses, nil
 }

ent/migrate/schema.go (1)

1828-1830: Add FK and indexes for tasks.scheduled_job_id and workflow_id

• Missing FK: tasks.scheduled_job_id should reference scheduled_jobs.id (ON DELETE SET NULL) to prevent orphans.
• Missing indexes: queries by scheduled_job_id/workflow_id will scan without indexes.

Implement in Ent schema (not this generated file), then re-generate:

// ent/schema/task.go
field.String("scheduled_job_id").Optional().Nillable().
  SchemaType(map[string]string{"postgres":"varchar(50)"}),

edge.From("scheduled_job", ScheduledJob.Type).
  Ref("tasks").
  Field("scheduled_job_id").
  Unique().
  Nillable().
  Annotations(entsql.OnDelete(entsql.SetNull)),

field.String("workflow_id").Optional().Nillable().
  SchemaType(map[string]string{"postgres":"varchar(255)"}),

index.Fields("tenant_id", "environment_id", "scheduled_job_id"),
index.Fields("tenant_id", "environment_id", "workflow_id"),

Also applies to: 1845-1866

internal/integration/s3/config.go (1)

21-24: Still missing AWSSessionToken propagation.

The configuration struct and constructor continue to drop AWSSessionToken, so STS/assumed-role credentials remain unusable. Please add the session token field and copy the value from S3ConnectionMetadata.

Apply:

 type Config struct {
@@
-	AWSAccessKeyID     string
-	AWSSecretAccessKey string
+	AWSAccessKeyID     string
+	AWSSecretAccessKey string
+	AWSSessionToken    string
 }
@@
 		// From encrypted_secret_data
 		AWSAccessKeyID:     secretData.AWSAccessKeyID,
 		AWSSecretAccessKey: secretData.AWSSecretAccessKey,
+		AWSSessionToken:    secretData.AWSSessionToken,
 }

Also applies to: 46-48

internal/repository/clickhouse/feature_usage.go (1)

1395-1422: Add FINAL when reading from ReplacingMergeTree.

feature_usage is backed by ReplacingMergeTree; without FINAL, exports can include obsolete rows. Please update the query to FROM feature_usage FINAL … to ensure merged results.

Apply:

-	FROM feature_usage
+	FROM feature_usage FINAL

internal/service/feature_usage_tracking.go (1)

1252-1254: Re-allow expired prices tied to active subscriptions.

Setting priceFilter.AllowExpiredPrices = false drops sunsetted-but-still-billable prices, so meters referencing those prices vanish from data.Prices and downstream cost math reports zero. This regresses any customer still on grandfathered pricing. Please drop the assignment (or explicitly allow expired prices) so we still load the referenced price records.

 		priceFilter := types.NewNoLimitPriceFilter()
 		priceFilter.PriceIDs = priceIDs
 		priceFilter.WithStatus(types.StatusPublished)
-		priceFilter.AllowExpiredPrices = false

internal/api/v1/scheduled_job.go (1)

50-53: Respect explicit enabled=false on create.

Because CreateScheduledJobRequest.Enabled is a plain bool, this block turns an explicit false from clients into true, so you can’t create a disabled job. Please drop the forced default (or switch the DTO field to *bool and only default when nil) so client intent is preserved.

-	// Default enabled to true if not provided
-	if !req.Enabled {
-		req.Enabled = true
-	}

internal/integration/s3/README.md (1)

158-203: Keep Parquet in one section only.

Parquet still shows up under “Supported Formats” even though it’s also listed as a future enhancement. Please remove it from the supported list (or the future list if it’s actually implemented) so the doc is consistent.

internal/temporal/workflows/export/execute_export_workflow.go (1)

56-79: Verify that CreateTask is idempotent with pre-generated TaskID.

The workflow uses a pre-generated TaskID from input (line 58), which should help with idempotency. However, ensure the repository layer enforces unique constraint on TaskID to prevent duplicate task creation on activity retries.

Run this script to verify the task repository's handling of duplicate TaskIDs:

#!/bin/bash
# Check for unique constraints or idempotency handling in task repository
rg -nP '(UNIQUE|unique|ON CONFLICT|OnConflict|Upsert|CheckExists)' --type=go -g '**/task/**' -C3

internal/api/dto/scheduled_job.go (1)

11-17: Align JobConfig and Enabled defaults with schema.

There's a mismatch between the DTO and the ent schema:

1. JobConfig: Marked binding:"required" in the DTO but the schema defines it as .Optional() with default {}. Either remove binding:"required" from the DTO (treating omitted as empty map) or remove .Optional() from the schema.

2. Enabled: As a non-pointer bool, it defaults to false when omitted in the request, but the schema has Default(true). Change Enabled to *bool in the DTO and apply the schema default (true) when nil, or remove the schema default if false is the intended default.

Apply this diff to align with schema defaults:

 type CreateScheduledJobRequest struct {
 	ConnectionID string                 `json:"connection_id" binding:"required"`
 	EntityType   string                 `json:"entity_type" binding:"required"`
 	Interval     string                 `json:"interval" binding:"required"`
-	Enabled      bool                   `json:"enabled"`
-	JobConfig    map[string]interface{} `json:"job_config" binding:"required"`
+	Enabled      *bool                  `json:"enabled"`
+	JobConfig    map[string]interface{} `json:"job_config"`
 }

Then update ToCreateInput() to apply defaults:

 func (r *CreateScheduledJobRequest) ToCreateInput() *types.CreateScheduledJobInput {
-	enabled := r.Enabled // default will be handled by service if needed
+	enabled := true // schema default
+	if r.Enabled != nil {
+		enabled = *r.Enabled
+	}
+	jobConfig := r.JobConfig
+	if jobConfig == nil {
+		jobConfig = make(map[string]interface{})
+	}
 	return &types.CreateScheduledJobInput{
 		ConnectionID: r.ConnectionID,
 		EntityType:   types.ScheduledJobEntityType(r.EntityType),
 		Interval:     types.ScheduledJobInterval(r.Interval),
 		Enabled:      enabled,
-		JobConfig:    r.JobConfig,
+		JobConfig:    jobConfig,
 	}
 }

internal/service/sync/export/usage_export.go (1)

118-156: Mitigate CSV injection risk (formula injection).

Cells starting with =, +, -, or @ can be interpreted as formulas by spreadsheet applications, creating a security risk (CSV injection).

Apply this diff to add sanitization:

+// sanitizeCSVValue prevents CSV injection by escaping values that start with formula indicators
+func sanitizeCSVValue(s string) string {
+	if len(s) == 0 {
+		return s
+	}
+	switch s[0] {
+	case '=', '+', '-', '@':
+		return "'" + s // Prefix with single quote to escape
+	default:
+		return s
+	}
+}
+
 // Write batch data to CSV
 for _, usage := range usageData {
 	// Convert properties map to JSON string
@@
 	row := []string{
-		usage.ID,
-		usage.TenantID,
-		usage.EnvironmentID,
-		usage.ExternalCustomerID,
-		usage.CustomerID,
-		usage.SubscriptionID,
-		usage.SubLineItemID,
-		usage.PriceID,
-		usage.MeterID,
-		usage.FeatureID,
-		usage.EventName,
-		usage.Source,
+		sanitizeCSVValue(usage.ID),
+		sanitizeCSVValue(usage.TenantID),
+		sanitizeCSVValue(usage.EnvironmentID),
+		sanitizeCSVValue(usage.ExternalCustomerID),
+		sanitizeCSVValue(usage.CustomerID),
+		sanitizeCSVValue(usage.SubscriptionID),
+		sanitizeCSVValue(usage.SubLineItemID),
+		sanitizeCSVValue(usage.PriceID),
+		sanitizeCSVValue(usage.MeterID),
+		sanitizeCSVValue(usage.FeatureID),
+		sanitizeCSVValue(usage.EventName),
+		sanitizeCSVValue(usage.Source),
 		usage.Timestamp.Format(time.RFC3339),
 		usage.IngestedAt.Format(time.RFC3339),
 		fmt.Sprintf("%d", usage.PeriodID),
 		usage.QtyTotal.String(),
 		string(propertiesJSON),
-		usage.UniqueHash,
+		sanitizeCSVValue(usage.UniqueHash),
 	}

internal/types/sync_config.go (1)

81-131: Block empty entities & negative file size in S3 config

MaxFileSizeMB accepts negatives, and EntityTypes lets empty strings slip through because ExportEntityType.Validate() treats "" as optional. A negative size will break chunking downstream, and an empty entity will blow up lookups later. Please reject MaxFileSizeMB < 0 and short-circuit when an entity string is blank before calling Validate().

internal/repository/ent/scheduledjob.go (5)

106-150: Scope Update by tenant/environment

UpdateOneID bypasses tenant/environment predicates, so any caller with an ID can modify another tenant’s job. Use Update().Where(...ID..., TenantID, EnvironmentID...) before applying setters.

---

152-168: Scope Delete by tenant/environment

DeleteOneID ignores tenant/env, allowing cross-tenant deletes. Switch to Delete().Where(scheduledjob.ID(id), scheduledjob.TenantID(...), scheduledjob.EnvironmentID(...)).

---

174-215: Guard nil filters before dereferencing

filters.TenantID etc. panic when filters is nil. Add a nil guard (e.g., default to an empty ListFilters) before dereferencing.

---

295-323: Scope UpdateLastRun by tenant/environment

UpdateOneID again skips tenant/env enforcement for last-run updates. Use Update().Where(...TenantID..., ...EnvironmentID...) to avoid cross-tenant writes.

---

331-349: Propagate TemporalScheduleID to domain model

The mapper drops TemporalScheduleID, so callers never see stored schedule IDs. Include TemporalScheduleID: entJob.TemporalScheduleID.

internal/integration/s3/client.go (2)

168-175: Stop leaking decrypted keys (and fix undefined min)

Logging decrypted lengths/prefixes both exposes secrets and calls an undefined min(), breaking the build. Replace the log with a non-sensitive message (e.g., just connection ID + session token flag).

---

176-205: Validate bucket/region before building S3 config

jobConfig only checked for nil; missing bucket/region slips through and fails later. Hard-fail when either is empty (trim whitespace) right after the nil check.

ent/scheduledjob/where.go (1)

412-420: Align field nullability with predicates (schema vs generated code)

IsNil/NotNil predicates exist for fields that are non-pointer strings in the model (CreatedBy, UpdatedBy, EnvironmentID, LastRunStatus, LastRunError, TemporalScheduleID). Update schema to mark fields Nillable() (making them *string) or make them NOT NULL and drop IsNil/NotNil, then re-generate.

Also applies to: 487-495, 562-570, 952-960, 1027-1035, 1102-1110

internal/types/scheduled_job.go (1)

118-141: Add validation: disallow negative MaxFileSizeMB

Validate should reject values < 0 before setting defaults. Otherwise downstream exporters can panic/stall.

Apply this diff:

 func (s *S3JobConfig) Validate() error {
@@
   if s.Encryption != "" && s.Encryption != "AES256" && s.Encryption != "aws:kms" {
     return ierr.NewError("invalid encryption type").
       WithHint("Encryption must be one of: AES256, aws:kms").
       Mark(ierr.ErrValidation)
   }
 
+  // Validate max file size
+  if s.MaxFileSizeMB < 0 {
+    return ierr.NewError("invalid max file size").
+      WithHint("MaxFileSizeMB must be >= 0; use 0 to apply the default").
+      Mark(ierr.ErrValidation)
+  }
+
   // Set defaults
   if s.Compression == "" {
     s.Compression = "gzip"
   }
   if s.Encryption == "" {
     s.Encryption = "AES256"
   }
   if s.MaxFileSizeMB == 0 {
     s.MaxFileSizeMB = 100
   }
 
   return nil
 }

internal/integration/stripe/customer.go (2)

204-205: Consider deferring externalID initialization.

The externalID is assigned to stripeCustomer.ID upfront but may be overridden by flexprice_lookup_key at line 219. While functionally correct due to early returns, initializing it closer to where it's actually used (Step 3, line 254) would improve readability.

---

427-463: Suggest parameterizing the helper to eliminate code duplication.

While the new helper is well-structured, there's code duplication between this helper (lines 427-463) and the inline mapping creation in CreateCustomerInStripe (lines 172-197). The only meaningful difference is the created_via metadata field:

• This helper uses "provider_to_flexprice" (webhook direction)
• CreateCustomerInStripe uses "flexprice_to_provider" (API direction)

Consider adding a direction parameter to make the helper reusable for both cases, eliminating ~20 lines of duplicate code.

Example refactor:

-func (s *CustomerService) createEntityIntegrationMapping(ctx context.Context, customerID string, stripeCustomer *stripe.Customer) error {
+func (s *CustomerService) createEntityIntegrationMapping(ctx context.Context, customerID string, stripeCustomer *stripe.Customer, direction string) error {
 	if s.entityIntegrationMappingRepo == nil {
 		return nil
 	}

 	mapping := &entityintegrationmapping.EntityIntegrationMapping{
 		ID:               types.GenerateUUIDWithPrefix(types.UUID_PREFIX_ENTITY_INTEGRATION_MAPPING),
 		EntityID:         customerID,
 		EntityType:       types.IntegrationEntityTypeCustomer,
 		ProviderType:     string(types.SecretProviderStripe),
 		ProviderEntityID: stripeCustomer.ID,
 		Metadata: map[string]interface{}{
-			"created_via":           "provider_to_flexprice",
+			"created_via":           direction,
 			"stripe_customer_email": stripeCustomer.Email,
 			"stripe_customer_name":  stripeCustomer.Name,
 			"synced_at":             time.Now().UTC().Format(time.RFC3339),
 		},
 		EnvironmentID: types.GetEnvironmentID(ctx),
 		BaseModel:     types.GetDefaultBaseModel(ctx),
 	}

Then update callers:

• Line 240: s.createEntityIntegrationMapping(ctx, existing.Customer.ID, stripeCustomer, "provider_to_flexprice")
• Line 278: s.createEntityIntegrationMapping(ctx, customerResp.ID, stripeCustomer, "provider_to_flexprice")
• Lines 172-197 in CreateCustomerInStripe: Replace inline code with s.createEntityIntegrationMapping(ctx, ourCustomer.ID, stripeCustomer, "flexprice_to_provider")

internal/api/v1/test_usage_export.go (1)

125-127: Use UTC timestamps for export windows.

Avoid timezone ambiguity in file names/filters.

-	endTime := time.Now()
+	endTime := time.Now().UTC()
 	startTime := endTime.Add(-24 * time.Hour)

internal/temporal/activities/export/task_activity.go (2)

141-154: Use UTC for status timestamps.

Normalize timestamps to UTC.

-	now := time.Now()
+	now := time.Now().UTC()

---

203-212: Use UTC for completion timestamps and metadata.

Normalize CompletedAt and metadata timestamps.

-	now := time.Now()
+	now := time.Now().UTC()
@@
-	existingTask.Metadata["completed_at"] = now.Format(time.RFC3339)
+	existingTask.Metadata["completed_at"] = now.UTC().Format(time.RFC3339)

internal/domain/task/model.go (1)

82-85: Consider implementing basic Task validation.

Currently Validate() returns nil; add minimal checks (e.g., ID non-empty, TaskType/EntityType set) if used by services.

internal/types/secret.go (1)

34-38: Providers added correctly; consider hoisting allowed set to avoid per-call allocations

Change is good. To avoid rebuilding the slice on every Validate(), use a package-level set.

Apply within Validate:

-allowedSecretProviders := []SecretProvider{
-  SecretProviderFlexPrice,
-  SecretProviderStripe,
-  SecretProviderRazorpay,
-  SecretProviderS3,
-  SecretProviderAthena,
-  SecretProviderBigQuery,
-  SecretProviderSnowflake,
-  SecretProviderRedshift,
-}
-if !lo.Contains(allowedSecretProviders, p) {
+if _, ok := allowedSecretProvidersSet[p]; !ok {
   ...
 }

Add once at package level:

var allowedSecretProvidersSet = map[SecretProvider]struct{}{
  SecretProviderFlexPrice: {},
  SecretProviderStripe:    {},
  SecretProviderRazorpay:  {},
  SecretProviderS3:        {},
  SecretProviderAthena:    {},
  SecretProviderBigQuery:  {},
  SecretProviderSnowflake: {},
  SecretProviderRedshift:  {},
}

Also applies to: 42-51

internal/repository/ent/task.go (1)

361-415: Query looks correct; add a supporting index for performance

The lookup filters by tenant_id, environment_id, scheduled_job_id, task_type, task_status, status and orders by completed_at DESC. Add a composite index (including completed_at) to avoid full scans at scale.

Example (ent schema Task.Indexes):

index.Fields("tenant_id", "environment_id", "scheduled_job_id", "task_type", "task_status", "status", "completed_at").
  StorageKey("idx_tasks_export_lookup")

internal/testutil/inmemory_task_store.go (1)

100-125: Make CompletedAt comparison nil-safe

Edge-case: if one of the CompletedAt pointers is nil, a later valid timestamp may be skipped.

- if lastTask == nil || (t.CompletedAt != nil && lastTask.CompletedAt != nil && t.CompletedAt.After(*lastTask.CompletedAt)) {
+ if lastTask == nil ||
+   (t.CompletedAt != nil && (lastTask.CompletedAt == nil || t.CompletedAt.After(*lastTask.CompletedAt))) {
     lastTask = t
   }

ent/schema/task.go (1)

42-55: Add composite index to support “last successful export” query

To match repository query filters and order, add an index including scheduled_job_id and completed_at.

 func (Task) Indexes() []ent.Index {
   return []ent.Index{
     index.Fields("tenant_id", "environment_id", "task_type", "entity_type", "status").
       StorageKey("idx_tasks_tenant_env_type_status"),
     index.Fields("tenant_id", "environment_id", "created_by", "status").
       StorageKey("idx_tasks_tenant_env_user"),
     index.Fields("tenant_id", "environment_id", "task_status", "status").
       StorageKey("idx_tasks_tenant_env_task_status"),
+    index.Fields("tenant_id", "environment_id", "scheduled_job_id", "task_type", "task_status", "status", "completed_at").
+      StorageKey("idx_tasks_export_lookup"),
   }
 }

Optional: If “unset scheduled_job_id” should be NULL (not ""), mark the field Nillable() and adjust domain to use *string. Otherwise keep as Optional.

Also applies to: 110-119

ent/task/where.go (1)

112-121: Predicate additions are consistent; consider indexing for query patterns

The new predicate helpers for scheduled_job_id and workflow_id follow ent conventions and look correct.

Recommendation: if you frequently query tasks by scheduled_job_id/workflow_id (e.g., to fetch last successful export), add DB indexes on these columns (plus ordering column like completed_at) in ent/schema to keep queries efficient at scale.

Also applies to: 747-895

internal/temporal/activities/export/scheduledjob_activity.go (3)

55-105: Time handling and config parsing are good; make UTC explicit and accept RFC3339Nano

• Use time.Now().UTC() to be explicit and avoid env-dependent behavior.
• When parsing last export end_time elsewhere, also accept time.RFC3339Nano to tolerate higher-precision timestamps.

Small improvements; logic otherwise reads well.

---

109-174: Incremental window: improve robustness (parse tolerance, range clamp, max lookback)

• Accept RFC3339 or RFC3339Nano for end_time parsing.
• If lastEndTime > endTime, clamp start to endTime.Add(-interval) to avoid negative duration windows.
• Optionally cap lookback (e.g., 30/90 days) to prevent huge first-run exports.

These guardrails reduce surprises and heavy loads.

---

186-226: Update flow is fine; consider partial update and capturing failure details

• To reduce read-modify-write races, a repository partial update (fields: last_run_at, last_run_status, next_run_at) can be used.
• If status indicates failure, optionally persist last_run_error (if available) alongside status.

Nice-to-have; current code is functional.

cmd/server/main.go (1)

343-346: Test endpoints are always registered; gate behind config/flag in production

Recommend exposing TestExport/TestUsageExport only in non-production modes or behind a feature flag to avoid unintended access.

internal/integration/s3/export.go (7)

209-213: Guard against empty key when trimming leading slash

Indexing key[0] risks panic if the key is empty. Use a length check.

Apply this diff:

-	// Remove leading slash if present
-	if key[0] == '/' {
-		key = key[1:]
-	}
+	// Remove leading slash if present
+	if len(key) > 0 && key[0] == '/' {
+		key = key[1:]
+	}

---

159-165: Validation vs. fallback mismatch for FileName

You error when FileName is empty but also implement a fallback name later. Align behavior by allowing empty FileName and relying on the fallback.

Apply this diff to relax validation:

-	if request.FileName == "" {
-		return ierr.NewError("file name is required").
-			WithHint("File name must be provided").
-			Mark(ierr.ErrValidation)
-	}

This keeps the fallback at Lines 196-201 meaningful.

Also applies to: 196-201

---

96-101: Provide KMS key ID when using aws:kms

If configured, set SSEKMSKeyId; otherwise, KMS encryption may fail or use an unintended key.

Apply this diff:

 	} else if c.config.Encryption == "aws:kms" {
 		uploadInput.ServerSideEncryption = types.ServerSideEncryptionAwsKms
+		if c.config.KMSKeyID != "" {
+			uploadInput.SSEKMSKeyId = aws.String(c.config.KMSKeyID)
+		}
 	}

Please confirm c.config.KMSKeyID (or equivalent) exists in your config.

---

170-174: Validate supported formats (csv, json, parquet)

Fail fast on unknown formats to avoid bad Content-Type/extension.

Apply this diff:

 	if request.EntityType == "" {
 		return ierr.NewError("entity type is required").
 			WithHint("Entity type must be provided").
 			Mark(ierr.ErrValidation)
 	}
+
+	// Validate format
+	switch request.Format {
+	case ExportFormatCSV, ExportFormatJSON, ExportFormatParquet:
+	default:
+		return ierr.NewErrorf("unsupported export format: %s", request.Format).
+			Mark(ierr.ErrValidation)
+	}

---

10-14: Optional: alias AWS S3 import to avoid self-package name confusion

Using awss3 alias improves clarity in a package also named s3. Update usages accordingly.

Apply this diff:

-	"github.com/aws/aws-sdk-go-v2/service/s3"
+	awss3 "github.com/aws/aws-sdk-go-v2/service/s3"

And replace occurrences of s3.PutObjectInput with awss3.PutObjectInput in this file.

---

246-262: Optional: Parquet Content-Type

Consider "application/vnd.apache.parquet" for parquet payloads; current octet-stream works but is less specific.

Apply this diff:

 	case ExportFormatParquet:
-		return "application/octet-stream"
+		return "application/vnd.apache.parquet"

---

62-80: Optional: stream compression and multipart uploads for large files

Current approach double-buffers data (compress then upload). For big exports, consider:

• io.Pipe to gzip while streaming to S3
• aws-sdk-go-v2/feature/s3/manager Uploader for multipart concurrency and resilience

This reduces memory and speeds uploads.

Also applies to: 225-244

internal/integration/factory.go (1)

216-229: Unused parameter in GetStorageProvider.

The connectionID parameter is not used in the method body. If this is intentional for future extensibility (e.g., supporting per-connection S3 clients), document this in the method comment. Otherwise, consider removing it to avoid confusion.

If the parameter is intended for future use, update the comment:

 // GetStorageProvider returns an S3 storage client for the given connection
+// Note: connectionID parameter is reserved for future use when supporting
+// connection-specific client configurations.
 // Currently only S3 is supported. In the future, Azure Blob Storage, Google Cloud Storage,
 // and other providers can be added by checking the connection's provider type.
 func (f *Factory) GetStorageProvider(ctx context.Context, connectionID string) (*s3.Client, error) {

Alternatively, if not needed, remove the parameter:

-func (f *Factory) GetStorageProvider(ctx context.Context, connectionID string) (*s3.Client, error) {
+func (f *Factory) GetStorageProvider(ctx context.Context) (*s3.Client, error) {

ent/task.go (1)

39-42: Align ScheduledJobID nullability with DB to avoid ambiguity

DB column is nullable, but Go field is non-pointer string. This collapses NULL to "", prevents clearing to NULL, and complicates predicates.

Prefer making it nillable in the Ent schema so the generated field is *string (like WorkflowID):

// ent/schema/task.go
field.String("scheduled_job_id").
  Optional().
  Nillable().
  SchemaType(map[string]string{"postgres":"varchar(50)"})

If you intentionally want non-null with empty default, change the schema to Default("") and drop Nillable so DB is NOT NULL with default "", then re-generate. Re-run ent generate after updating.

Also applies to: 160-172

internal/types/scheduled_job.go (1)

86-97: Replace magic strings with typed enums for compression/encryption

Avoid stringly-typed values for Compression/Encryption. Introduce typed constants to reduce errors.

Example:

+type CompressionType string
+const (
+  CompressionGzip CompressionType = "gzip"
+  CompressionNone CompressionType = "none"
+)
+type EncryptionType string
+const (
+  EncryptionAES256 EncryptionType = "AES256"
+  EncryptionKMS    EncryptionType = "aws:kms"
+)
 type S3JobConfig struct {
   Bucket           string `json:"bucket"`
   Region           string `json:"region"`
   KeyPrefix        string `json:"key_prefix,omitempty"`
-  Compression      string `json:"compression,omitempty"`
-  Encryption       string `json:"encryption,omitempty"`
+  Compression      CompressionType `json:"compression,omitempty"`
+  Encryption       EncryptionType  `json:"encryption,omitempty"`
   EndpointURL      string `json:"endpoint_url,omitempty"`
   VirtualHostStyle bool   `json:"virtual_host_style,omitempty"`
   MaxFileSizeMB    int    `json:"max_file_size_mb,omitempty"`
 }

Also applies to: 118-130