Thank you for visiting my page.

EdgeCDN-X is an open source CDN solution built purely on top of k8s and CNCF projects.

My plan with this project is to document the progress and create a project which will be easily understandable and deployable by community members. For updates feel free to sign up to the newsletter

Official Documentation

EdgeCDN-X is consists of the following components:

• Control-plane - Hosting the UI endpoint and responsible for rolling out the services and configuration via ArgoCD Cluster Generators
• Routing - Routing engine hosts CoreDNS with custom plugins with additional features
• Caching - Caching engine handled via Nginx Ingress controller with customized annotations

Control plane is using ArgoCD and custom CRDs and operators to describe the topology, services. This component is using ArgoCD cluster generators to roll out the desired configuration to the individual locations

Routing component supports 3 different routing engines:

• DNS Routing - In Alpha ✅
• 302 Redirection - Planned 🔜
• URL Rewriting API - Planned 🔜

Routing component routes the individual requests via the following steps:

• Prefix static routing to individual location (sourced from static prefix list ✅ or BGP 🔜) ✅
• GeoLookup to locations if static routing returns no destination ✅
• Consistent hashing in location to maximize cache-hit ratio. ✅
• Active healthchecks to make sure destinations are healthy and available ✅
• Fallback routing to different location if location has no active nodes ✅
• Caching with multi cache type support. E.g. SSD caching RAMDISK caching ✅

Routing engine is rolled out to each location with edgecdnx.com/routing label in the cluster metadata

edgecdnx-prefixlist CoreDNS module gathers all the prefixes for the individual locations. These prefixes must be normalized and must be non overlapping. There's a helper operator which helps to achieve Prefix Consolidation and Supernet subnetting, to make sure there are no overlaps in the Prefixes. The Module is using CoreDNS's Metadata interface to find the desired destination for a given prefix.

Features:

• Routing to location based on Client IP address
• Prefixes are stored in a fast balanced AVL Tree to ensure speedy lookups
• EDNS0 Subnet extension support
• IPv4 and IPv6 Supported

Status: ✅

edgecdnx-geolookup CoreDNS module finds the most suitable locatio based on MMDB2 DB. This module uses geoip metadata to enrich the necessary fields. Geolookup module assigns weights and score for each request and the location is based on this score. If multiple locations are found with the same score, the requests are balanced based on associated weight. (e.g. eu-west and eu-east routing to Germany in ratio of 40:60)

Geolookup configurations are coming from CRDs with an example configuration shown here:

---
apiVersion: infrastructure.edgecdnx.com/v1alpha1
kind: Location
metadata:
  name: eu-west-1
spec:
  fallbackLocations:
  - us-west-1
  nodeGroups:
    - name: ssd
      nodes:
        - name: n1
          ipv4: 74.220.31.183
      nodeSelector:
        kubernetes.civo.com/civo-node-pool: fra1-c1
      cacheConfig:
        name: "ssd"
        path: "/var/cache/ssd"
        keysZone: "100m"
        inactive: "10080m"
        maxSize: "4096m"
  geoLookup:
    weight: 50
    attributes:
      geoip/continent/code:
        weight: 1000
        values:
          - value: "EU"
            weight: 10
          - value: "AF"

Status: ✅

edgecdnx-services. This module is responsible for building the SOA and NS records and also enriches metadata with customer specific information for better routing decitions down the line. All thes services are auto loaded via the k8s client, so it is not required to reload the Configuration when a new service is configured.

Example configuration:

        edgecdnxservices {
            namespace edgecdnx-routing
            soa ns1
            email noc.edgecdnx.com
            ns ns1 189.167.203.182
            ns ns2 190.167.203.183
        }

Status: ✅

edgecdnx-cache manifest rolls out an NGINX Deamonset for each location specified. Each location is specified as a k8s cluster and cluster definition must be labeled with edgecdnx.com/caching label in the cluster metadata

The Ingress controller is rolled out as a daemonset. Multiple instances of the caching engine can be rolled out for different caching tiers (e.g., ram, ssd). The underlying storage for caching must be prepared beforehand. To ensure maximum performance the ingress controller is attached to the hostNetwork and mounts the Caches as HostPath Volume. These settings can be modified editing the base manifest.

We definitely want to support multiple origin types such as, S3 and Static Origins

• Static origins - HTTP, HTTPS based origins - ✅
• S3 Origins - ✅

Sometimes query params modify the requested content. By default the EdgeCDN-X Platform ignores all query params when building the cache key for the returned content. In order to whitelist these query params we have to specity the queryParams in the cacheKey parameter of the Service Spec:

apiVersion: infrastructure.edgecdnx.com/v1alpha1
kind: Service
metadata:
  name: akldjgsofheiwu.cdn.edgecdnx.com
spec:
  name: akldjgsofheiwu.cdn.edgecdnx.com
  domain: akldjgsofheiwu.cdn.edgecdnx.com
  originType: static
  cacheKey:
    queryParams:
      - "v"
      - "ver"
      - "version"

This snippet whill ensure, that the requests coming to akldjgsofheiwu.cdn.edgecdnx.com/content.js?ver=1.0.0 and akldjgsofheiwu.cdn.edgecdnx.com/content.js?ver=1.0.1 will be cached individually. If a query param is not defined, it is stripped of when sent to upstream. If no query Param is defined in the cacheKey the query params are passed to the upstream but they're not considered when building the cache key to store the response, which means the requests from the previous response would return the same content.

This is a bit tricky use case. As per ACME, DNS based certificate issuance can be created for the owned domain (currently edgecdnx.com), we have to solve the challenges of distributing that certificate to the individual endpoints. For customer domains we do not have access to their registrar and NS so we have to fallback to HTTP based challenge. The problem is, that since this is a CDN, the challenge can end up on any of the nodes due to DNS redirection. For this purpose, we will start the challenge on the control plane and build a small helper reverse proxy, which will direct those requests from the individual endpoints to the control plane endpoint where the cert issuance is in progress. Once issued, we have to distribute the Certificates to the individual Endpoints. ✅

To avoid access to certain objects publicly it is possible to use URL signatures to prevent unauthorized access to the resources. These signatures are often used for signing Stream (HLS or MPEG) playlists. Further down the line, once the signature is verified a session cookie is issued which the client can use to access the stream without having to Sign each segment's request. The session is only valid for a specific stream. Further [reading(https://github.com/EdgeCDN-X/secure-urls)] - ✅

• Multi cache support - ✅ - Supported, Multiple Nginx definitions have to be defined
• S3 upstream connector - ✅ - Forked from Nginx and adapted to our needs.
• DNS routing - ✅
• 302 redirection routing - 🔜 - Will be supported by attaching to the CoreDNS gRPC endpoint
• Active Healthchecks - In Progress - ✅
• Control plane based on CRDs - v1alpha1 version ✅
• Controller UI 🔜
• WAF - Web Application Firewall with mod-security ✅

• Tiered caching - Edge, local, regional caches to further lower latency and reduce load on origin servers
• Edge caching - Caching on offshore locations, such as boats, trains, remote locations by deploying a cache as an edge computing enpoint
• Large File Support - support for ranges requests and caching to support effective caching of large files