[web-console] Add Scarf Pixel to track OSS usage #5486

Karakatiza666 · 2026-01-21T17:25:29Z

Add Scarf.sh Tracking for OSS Deployments

This PR adds lightweight usage tracking to help us understand how Feldera OSS is being deployed in the wild.

What Gets Tracked by Scarf

When users access the web-console in an Open Source deployment, a tracking pixel loads with two metadata fields:

Version: The Feldera version number (e.g., 0.227.0)
Build source: Whether the deployment came from:
- ci - Official GitHub release Docker images
- source - Built locally from source code
- Geographic location: Country/region derived from IP address
- Timestamp: When the deployment was accessed

What Doesn't Get Tracked

❌ Enterprise deployments - Automatically excluded via edition check
❌ Personal identifiers - No user IDs, emails, or account information
❌ Usage patterns - No behavioral tracking or feature usage monitoring
❌ Session data - No tracking across pages or time

Why This Matters

This helps us answer important questions about the OSS community:

How many users deploy from official releases vs building from source?
Which versions are most widely deployed?
Where in the world is Feldera being used?
When should we sunset old versions?

Privacy

The tracking pixel is visible in browser network requests and can be blocked by browser extensions or network policies. Geographic data is aggregated at the country/region level. We only see deployment patterns, not individual users or detailed usage behavior.

Copilot

Pull request overview

This PR adds Scarf.sh tracking to monitor Open Source deployments of the Feldera web console, helping understand deployment patterns and version adoption in the wild.

Changes:

Adds build source tracking (ci vs source) through environment variables and API configuration
Implements Scarf tracking pixel in web console that loads only for Open Source edition deployments
Updates OpenAPI schema and TypeScript types to include build_source field in Configuration

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
openapi.json	Adds `build_source` field to Configuration schema with description
js-packages/web-console/src/routes/+layout.svelte	Implements Scarf tracking pixel with conditional rendering for OSS deployments
js-packages/web-console/src/lib/services/manager/types.gen.ts	Generated TypeScript types including `build_source` and other API changes
js-packages/web-console/src/lib/services/manager/sdk.gen.ts	Generated SDK code for new API endpoints
crates/pipeline-manager/src/api/endpoints/config.rs	Adds `build_source` field to Configuration struct
crates/pipeline-manager/build.rs	Captures `FELDERA_BUILD_SOURCE` environment variable during build
.github/workflows/build-rust.yml	Sets `FELDERA_BUILD_SOURCE=ci` for GitHub Actions builds

Copilot · 2026-01-23T23:43:27Z

js-packages/web-console/src/routes/+layout.svelte

+  const shouldTrack = $derived(
+    browser && page.data.feldera?.config?.edition === 'Open source'
+  )
+  const scarfTrackingUrl = $derived(
+    shouldTrack
+      ? `https://static.scarf.sh/a.png?x-pxid=c5e8a21a-5f5a-424d-81c4-5ded97174900&version=${encodeURIComponent(page.data.feldera!.config!.version)}&build_source=${encodeURIComponent(page.data.feldera!.config!.build_source)}`


The hardcoded pixel ID and URL construction makes this difficult to maintain or test. Consider extracting the pixel ID and base URL as named constants at the module level.

Suggested change

const shouldTrack = $derived(

browser && page.data.feldera?.config?.edition === 'Open source'

)

const scarfTrackingUrl = $derived(

shouldTrack

? `https://static.scarf.sh/a.png?x-pxid=c5e8a21a-5f5a-424d-81c4-5ded97174900&version=${encodeURIComponent(page.data.feldera!.config!.version)}&build_source=${encodeURIComponent(page.data.feldera!.config!.build_source)}`

const SCARF_BASE_URL = 'https://static.scarf.sh/a.png'

const SCARF_PIXEL_ID = 'c5e8a21a-5f5a-424d-81c4-5ded97174900'

const shouldTrack = $derived(

browser && page.data.feldera?.config?.edition === 'Open source'

)

const scarfTrackingUrl = $derived(

shouldTrack

? `${SCARF_BASE_URL}?x-pxid=${SCARF_PIXEL_ID}&version=${encodeURIComponent(page.data.feldera!.config!.version)}&build_source=${encodeURIComponent(page.data.feldera!.config!.build_source)}`

if you do extract them, maybe create a function that generates the URL too
Is there a spec of the shape of the URL?

Past the HTTP origin and the x-pxid query parameter there is no fixed spec. The $derived syntax already defines an expression that get's re-evaluated as needed, so I'll just separate the scarfPixelId as a constant

Copilot · 2026-01-23T23:43:27Z

js-packages/web-console/src/routes/+layout.svelte

+  const scarfTrackingUrl = $derived(
+    shouldTrack
+      ? `https://static.scarf.sh/a.png?x-pxid=c5e8a21a-5f5a-424d-81c4-5ded97174900&version=${encodeURIComponent(page.data.feldera!.config!.version)}&build_source=${encodeURIComponent(page.data.feldera!.config!.build_source)}`
+      : ''
+  )
+


Non-null assertions (!) used on nested optional properties could cause runtime errors if the data structure is incomplete. While shouldTrack checks for the parent object, it doesn't verify that version and build_source exist before accessing them.

Suggested change

const scarfTrackingUrl = $derived(

shouldTrack

? `https://static.scarf.sh/a.png?x-pxid=c5e8a21a-5f5a-424d-81c4-5ded97174900&version=${encodeURIComponent(page.data.feldera!.config!.version)}&build_source=${encodeURIComponent(page.data.feldera!.config!.build_source)}`

: ''

)

let scarfTrackingUrl = ''

$: {

const version = page.data.feldera?.config?.version

const buildSource = page.data.feldera?.config?.build_source

if (shouldTrack && version && buildSource) {

scarfTrackingUrl = `https://static.scarf.sh/a.png?x-pxid=c5e8a21a-5f5a-424d-81c4-5ded97174900&version=${encodeURIComponent(

version

)}&build_source=${encodeURIComponent(buildSource)}`

} else {

scarfTrackingUrl = ''

}

}

mihaibudiu · 2026-01-24T01:36:29Z

.github/workflows/build-rust.yml

  CARGO_FLAGS: "--release --locked --all-targets"
  CARGO_FEATURES_BASE: "pubsub-emulator-test,iceberg-tests-fs,iceberg-tests-glue"
  FELDERA_PLATFORM_VERSION_SUFFIX: ${{ github.sha }}
+  FELDERA_BUILD_SOURCE: ci


I interpreted this name as an imperative to build the sources
But maybe you want to use a name like "BUILD_ORIGIN"

mihaibudiu · 2026-01-24T01:37:29Z

crates/pipeline-manager/build.rs

    println!("cargo:rerun-if-env-changed=FELDERA_PLATFORM_VERSION_SUFFIX");
    println!("cargo:rustc-env=FELDERA_PLATFORM_VERSION_SUFFIX={platform_version_suffix}");

+    // Capture build source (ci vs source)


what does the second "source" mean?

As in, built locally from source

mihaibudiu · 2026-01-24T01:38:45Z

js-packages/web-console/src/lib/services/manager/sdk.gen.ts

    ...options
  })

+/**


I understand that this is generated code which does not need to be reviewed

Yes, from the OpenAPI spec

mihaibudiu · 2026-01-24T01:39:42Z

openapi.json

          },
+          "build_source": {
+            "type": "string",
+            "description": "Build source: \"ci\" for GitHub Actions builds, \"source\" for local builds"


what does "local" really mean?
can people fake this?

Yes, easily, but I don't think that's a concern for us. Since the tracking pixel has all public info and is distributed in the app sources everything about it can be manipulated anyway

mihaibudiu · 2026-01-24T01:41:37Z

js-packages/web-console/src/routes/+layout.svelte


+  // Scarf.sh tracking for OSS deployments
+  // Only track Open source edition (excludes Enterprise builds)
+  const shouldTrack = $derived(


people can easily change this. So I think it should be documented somewhere; you may need consent in places like EU perhaps anyway. Maybe you can document it in the README on github at least, but we should find out whether we need to do more to disclose this.

Lalith said the privacy notice on the main website is enough

I suggest keeping an opt out environment variable.

mihaibudiu · 2026-01-24T01:43:29Z

js-packages/web-console/src/routes/+layout.svelte

+  const shouldTrack = $derived(
+    browser && page.data.feldera?.config?.edition === 'Open source'
+  )
+  const scarfTrackingUrl = $derived(
+    shouldTrack
+      ? `https://static.scarf.sh/a.png?x-pxid=c5e8a21a-5f5a-424d-81c4-5ded97174900&version=${encodeURIComponent(page.data.feldera!.config!.version)}&build_source=${encodeURIComponent(page.data.feldera!.config!.build_source)}`


if you do extract them, maybe create a function that generates the URL too
Is there a spec of the shape of the URL?

Signed-off-by: Karakatiza666 <[email protected]>

Karakatiza666 force-pushed the scarf branch from 3555210 to 795ae80 Compare January 23, 2026 23:41

Karakatiza666 marked this pull request as ready for review January 23, 2026 23:41

Karakatiza666 requested review from Copilot and mihaibudiu January 23, 2026 23:41

Copilot AI reviewed Jan 23, 2026

View reviewed changes

mihaibudiu approved these changes Jan 24, 2026

View reviewed changes

Karakatiza666 force-pushed the scarf branch 3 times, most recently from 3e23218 to 514a837 Compare January 26, 2026 15:58

[web-console] Add Scarf Pixel to track OSS usage

97b17e1

Signed-off-by: Karakatiza666 <[email protected]>

Karakatiza666 force-pushed the scarf branch from 514a837 to 97b17e1 Compare January 26, 2026 15:59

Karakatiza666 enabled auto-merge January 26, 2026 15:59

Karakatiza666 added this pull request to the merge queue Jan 26, 2026

Merged via the queue into main with commit 8bfa95c Jan 26, 2026
1 check passed

Karakatiza666 deleted the scarf branch January 26, 2026 18:05

-  const scarfTrackingUrl = $derived(
-    shouldTrack
-      ? `https://static.scarf.sh/a.png?x-pxid=c5e8a21a-5f5a-424d-81c4-5ded97174900&version=${encodeURIComponent(page.data.feldera!.config!.version)}&build_source=${encodeURIComponent(page.data.feldera!.config!.build_source)}`
-      : ''
-  )
+  let scarfTrackingUrl = ''
+  $: {
+    const version = page.data.feldera?.config?.version
+    const buildSource = page.data.feldera?.config?.build_source
+    if (shouldTrack && version && buildSource) {
+      scarfTrackingUrl = `https://static.scarf.sh/a.png?x-pxid=c5e8a21a-5f5a-424d-81c4-5ded97174900&version=${encodeURIComponent(
+        version
+      )}&build_source=${encodeURIComponent(buildSource)}`
+    } else {
+      scarfTrackingUrl = ''
+    }
+  }

[web-console] Add Scarf Pixel to track OSS usage #5486

[web-console] Add Scarf Pixel to track OSS usage #5486

Uh oh!

Conversation

Karakatiza666 commented Jan 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants