Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions crates/feldera-types/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -763,6 +763,8 @@ pub struct RuntimeConfig {
///
/// The worker threads are evenly divided among the hosts. For single-host
/// deployments, this should be 1 (the default).
///
/// Multihost pipelines are an enterprise-only preview feature.
pub hosts: usize,

/// Storage configuration.
Expand Down
2 changes: 2 additions & 0 deletions docs.feldera.com/docs/architecture/enterprise.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Feldera Enterprise brings all the power of our incremental compute platform into
### One Pipeline = One Pod
Each Pipeline runs in its own Kubernetes Pod, resource- and fault-isolated from other pipelines and the control plane. Each Pipeline can use a **Persistent Volume** or **S3 bucket** for its storage.

Pipelines that run in multiple pods, to scale out resource-hungry computations across multiple machines, are a preview feature.

### Fine-Grained Resource Management
The control plane and each Pipeline can be configured with resource reservations and limits for CPU, memory, and storage. When using Persistent Volumes for storage, Pipelines can also specify the specific storage class to use. These controls help enforce SLAs and deal with noisy neighbours in shared clusters.

Expand Down
166 changes: 145 additions & 21 deletions docs.feldera.com/docs/get-started/enterprise/https.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,31 @@
This document describes how to configure the endpoints of Feldera components (API server, compiler,
runner, pipelines) to serve HTTPS in the Enterprise edition.

- The same certificate (and thus private key) are used by all components
- Authentication (e.g., mTLS) is currently not yet supported (besides for the
[existing authentication methods of the API server](authentication/index.mdx))
For Feldera to support HTTPS with single-host pipelines, the
administrator must configure Feldera with a [wildcard
certificate](#wildcard-certificate-generation) and a private key for
its main DNS domain, e.g. for `*.feldera.svc.cluster.local`. The
Feldera API server, compiler runner, pipeline runner, pipeline
stateful sets, and other top-level components share this certificate
and key.

For Feldera to also support HTTPS with multihost pipelines, the
administrator must additionally configure Feldera with a [private CA
certificate chain](#private-ca-certificate-chain-generation) and
private key. This allows the Feldera pipeline runner to generate keys
for multihost pipeline pods in their nested DNS domains,
e.g. `<pipeline>-<ordinal>.<pipeline>.feldera.svc.cluster.local`,
which a single wildcard certificate cannot cover. The generated
certificates are only used for connections between Feldera components,
such as between the pipeline pods and the pipeline manager, not
external software.

:::note

Feldera does not support mTLS authentication. The API server supports
[API authentication](authentication/index.mdx).

:::

This document does not apply to the connection to the Postgres database used by the control plane,
which can be [configured separately](helm-guide.md).
Expand All @@ -19,23 +41,28 @@ as it expects HTTPS. The pipelines will need to be recompiled.

:::

## Certificate
## Wildcard Certificate Generation

A certificate and associated private key need to be generated for the relevant hostnames.
To support HTTPS, the administrator must generate and install a
wildcard certificate for `*.<namespace>.svc.cluster.local`, where
`<namespace>` is the Kubernetes namespace in use, such as `feldera`.

### Certificate structure
:::info

- The Feldera components communicate with each other in Kubernetes using the
`SERVICE.NAMESPACE.svc.cluster.local` hostnames.
- As such, a single certificate should be generated for the wildcard hostname
`*.NAMESPACE.svc.cluster.local`
- (Optional) When using `kubectl` port forwarding, it is also useful to add
hostname `localhost` and/or `127.0.0.1` to the certificate
When using `kubectl` port forwarding, it is also useful to add
hostname `localhost` and/or `127.0.0.1` to the certificate.

:::

### Example (self-signed)
Any method for generating a signed certificate is acceptable. We
assume later that the certificate file is named `tls.crt` and the
private key file `tls.key`. The following sections describe two ways
to generate these files.

For example, suppose that we install Feldera in namespace `feldera` and will
be using `kubectl` port forwarding.
### Generating a self-signed certificate with OpenSSL

Suppose that we install Feldera in namespace `feldera` and will be
using `kubectl` port forwarding.

1. Create `x509_v3.ext` to specify the SANs (Subject Alternative Names):
```
Expand All @@ -57,25 +84,105 @@ be using `kubectl` port forwarding.

3. This will create the `tls.key` and `tls.crt` files.

### Not self-signed
### Using a certificate signed by an external CA

If the certificate is not self-signed, but instead is signed by a root CA or an intermediate CA,
the `tls.crt` should contain the complete bundle of the entire chain up until the root certificate
authority. As such, it should contain multiple `-----BEGIN CERTIFICATE----- (...) -----END CERTIFICATE-----`
sections, starting with the leaf certificate and ending with the root certificate. The `tls.key`
should only contain the single private key of the leaf certificate.

## Private CA Certificate Chain Generation

To support HTTPS for multihost pipelines, the administrator must
additionally configure Feldera with a private CA certificate chain and
private key. This allows the Feldera pipeline runner to generate keys
for multihost pipeline pods in their nested DNS domains,
e.g. `<pipeline>-<ordinal>.<pipeline>.feldera.svc.cluster.local`,
which a single wildcard certificate cannot cover.

The private CA certificate chain can be and should be separate from
any other PKI, because it is used only for connections between Feldera
components, such as between the pipeline pods and the pipeline
manager, not external software.

Any method for generating a private CA certificate chain is
acceptable. We assume later that the CA certificate chain file is
named `private_ca.crt` and the private key file `private_ca.key`. The
section below describes one way to generate these files.

### Generating a private CA with OpenSSL

1. Create `root_x509_v3.ext` to specify options for the root private CA:

```
[x509_v3]
basicConstraints = CA:TRUE,pathlen:1
```

2. Create `intermediate_x509_v3.ext` to specify options for the
intermediate private CA:

```
[x509_v3]
basicConstraints = CA:TRUE,pathlen:0
```

3. Generate the root CA with `openssl`:

```
openssl req -x509 -newkey rsa:4096 -nodes \
-keyout private_root_tls.key -out private_root_tls.crt -days 365 \
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The root and intermediate CA validity periods are hard-coded (365/360 days) without explanation. Consider either aligning them (common practice is a longer-lived root) or briefly documenting why these values were chosen, so readers don’t cargo-cult short-lived CA lifetimes into production.

Copilot uses AI. Check for mistakes.
-subj '/CN=FelderaTest Private Root CA' \
-config root_x509_v3.ext -extensions x509_v3
```

4. Generate and sign the intermediate CA with `openssl`:

```
openssl req -new -newkey rsa:4096 -nodes \
-keyout private_intermediate_tls.key -out private_intermediate_tls.csr \
-subj '/CN=FelderaTest Intermediate CA'
openssl x509 -req \
-in private_intermediate_tls.csr -CA private_root_tls.crt -CAkey private_root_tls.key \
-CAcreateserial -out private_intermediate_tls.crt \
-days 360 -sha256 \
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The root and intermediate CA validity periods are hard-coded (365/360 days) without explanation. Consider either aligning them (common practice is a longer-lived root) or briefly documenting why these values were chosen, so readers don’t cargo-cult short-lived CA lifetimes into production.

Copilot uses AI. Check for mistakes.
-extfile intermediate_x509_v3.ext -extensions x509_v3
```

5. Produce the secret to install into Kubernetes:

```
cat private_intermediate_tls.crt private_root_tls.crt > private_ca.crt
cat private_intermediate_tls.key > private_ca.key
```

## Configure Kubernetes

1. Create the Kubernetes TLS Secret (e.g., in namespace `feldera`, and named `feldera-https-config`)
with the local `tls.key` and `tls.crt` files:
1. Create the Kubernetes TLS secret for `tls.key` and `tls.crt` files
from [wildcard certificate
generation](#wildcard-certificate-generation):

```
kubectl create -n feldera secret tls feldera-https-config --key tls.key --cert tls.crt
kubectl create -n <namespace> secret tls feldera-https-config --key tls.key --cert tls.crt
```

The secret (i.e., certificate and key) will be mounted as a directory with corresponding files
by each of the Feldera component (API server, compiler, runner, pipelines) pods.

2. Provide in the Helm installation the reference for the `httpsSecretRef` value.
2. To additionally support multihost pipelines, create a Kubernetes
TLS secret for `private_ca.key` and `private_ca.crt` from [private
CA certificate chain
generation](#private-ca-certificate-chain-generation):

```
kubectl create -n <namespace> secret tls feldera-ca-config --key private_ca.key --cert private_ca.crt
```

3. Provide in the Helm installation the reference for the
`httpsSecretRef` and, for multihost support, `caSecretRef`, value.
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor grammar issue: “caSecretRef, value” should be plural (e.g., “values”) or rephrased (e.g., “set httpsSecretRef and (for multihost) caSecretRef”).

Copilot uses AI. Check for mistakes.

If you did not create the private CA for multihost pipeline support:

- **Via file `values.yaml`:**
```
Expand All @@ -89,7 +196,24 @@ should only contain the single private key of the leaf certificate.
--set httpsSecretRef="feldera-https-config"
```

3. (Optional) Port forward:
If you did create the private CA for multihost pipeline support:


- **Via file `values.yaml`:**
```
httpsSecretRef: "feldera-https-config"
caSecretRef: "feldera-ca-config"
```

- **In the command:**
```
helm upgrade --install feldera \
... \
--set httpsSecretRef="feldera-https-config" \
--set caSecretRef="feldera-ca-config"
```

4. (Optional) Port forward:
```
kubectl port-forward -n feldera svc/feldera-api-server 8080:8080
```
Expand Down
4 changes: 2 additions & 2 deletions openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -9258,7 +9258,7 @@
},
"hosts": {
"type": "integer",
"description": "Number of DBSP hosts.\n\nThe worker threads are evenly divided among the hosts. For single-host\ndeployments, this should be 1 (the default).",
"description": "Number of DBSP hosts.\n\nThe worker threads are evenly divided among the hosts. For single-host\ndeployments, this should be 1 (the default).\n\nMultihost pipelines are an enterprise-only preview feature.",
"default": 1,
"minimum": 0
},
Expand Down Expand Up @@ -10812,7 +10812,7 @@
},
"hosts": {
"type": "integer",
"description": "Number of DBSP hosts.\n\nThe worker threads are evenly divided among the hosts. For single-host\ndeployments, this should be 1 (the default).",
"description": "Number of DBSP hosts.\n\nThe worker threads are evenly divided among the hosts. For single-host\ndeployments, this should be 1 (the default).\n\nMultihost pipelines are an enterprise-only preview feature.",
"default": 1,
"minimum": 0
},
Expand Down