Codestin Search App

❗️
This is a legacy Apache Ignite documentation
The new documentation is hosted here: https://ignite.apache.org/docs/latest/

If Ignite will be used in the pure in-memory mode or as a caching layer on top of a 3rd party database (RDBMS, NoSQL) then it can be deployed as a stateless solution.

Prerequisities

Makes sure that you've done the following:

Deployed a Kubernetes cluster in the desired environment.
Configured RBAC Authorization
Deployed Ignite Service

Kubernetes IP Finder

To enable Apache Ignite nodes auto-discovery in Kubernetes, you need to enable TcpDiscoveryKubernetesIpFinder in IgniteConfiguration. Let's create an example configuration file called example-kube-rbac.xml and define the IP finder configuration as follows:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  <property name="discoverySpi">
    <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
      <property name="ipFinder">
         <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
           <property name="namespace" value="ignite"/>
         </bean>
      </property>
    </bean>
  </property>
</bean>
</beans>

📘
Kubernetes IP finder
To learn more about Kubernetes IP finder and Apache Ignite nodes auto-discovery in Kuberentes environment, refer to this documentation page.

Now, it's time to prepare a Kubernetes deployment configuration for Ignite pods and deploy them

Ignite Pods Deployment

Finally, let's define a YAML configuration for Ignite pods:

# An example of a Kubernetes configuration for Ignite pods deployment.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  # Custom Ignite cluster's name.
  name: ignite-cluster
spec:
  # A number of Ignite pods to be started by Kubernetes initially.
  replicas: 2
  template:
    metadata:
      labels:
        app: ignite
    spec:
      serviceAccountName: ignite
      containers:
        # Custom Ignite pod name.
      - name: ignite-node
        image: apacheignite/ignite:2.5.0
        env:
        - name: OPTION_LIBS
          value: ignite-kubernetes
        - name: CONFIG_URI
          value: https://raw.githubusercontent.com/apache/ignite/master/modules/kubernetes/config/example-kube-rbac.xml
        ports:
        # Ports to open.
        # Might be optional depending on your Kubernetes environment.
        - containerPort: 11211 # REST port number.
        - containerPort: 47100 # communication SPI port number.
        - containerPort: 47500 # discovery SPI port number.
        - containerPort: 49112 # JMX port number.
        - containerPort: 10800 # SQL port number.
        - containerPort: 10900 # Thin clients port number.

As you can see, the configuration defines a couple of environment variables (OPTION_LIBS and CONFIG_URIL) that will be processed by a special shell script used by Ignite's docker image. The full list of docker image's configuration parameters is available on Docker Deployment page.

📘
Ignite Docker Image Version
Kubernetes is supported in Apache Ignite 1.9 and later versions. Makes sure to use a docker image with a valid version. You can find a full list of tags here.

Next, go ahead and deploy Ignite pods in Kubernetes using the configuration above:

kubectl create -f ignite-deployment.yaml

Check that Ignite pods are up and running:

kubectl get pods

Pick a name of one of the pods available

NAME                              READY     STATUS    RESTARTS   AGE
ignite-cluster-3454482164-d4m6g   1/1       Running   0          25m
ignite-cluster-3454482164-w0xtx   1/1       Running   0          25m

and get the logs from it making sure that both Ignite pods were able to discover each other and from the cluster:

kubectl logs ignite-cluster-3454482164-d4m6g

Adjusting Ignite Cluster Size

You can adjust Apache Ignite cluster size on the fly using the standard Kubernetes API. For instance, if you want to scale out the cluster from 2 to 5 nodes then the command below can be used:

kubectl scale --replicas=5 -f ignite-deployment.yaml

Double check the cluster was scaled out successfully:

kubectl get pods

The output should show that you now have 5 Ignite pods up and running:

NAME                              READY     STATUS    RESTARTS   AGE
ignite-cluster-3454482164-d4m6g   1/1       Running   0          34m
ignite-cluster-3454482164-ktkrr   1/1       Running   0          58s
ignite-cluster-3454482164-r20f8   1/1       Running   0          58s
ignite-cluster-3454482164-vf8kh   1/1       Running   0          58s
ignite-cluster-3454482164-w0xtx   1/1       Running   0          34m

Deploying on Microsoft Azure

Deploying Apache Ignite in Kubernetes on Microsoft Azure article provides a step-by-step guide on how to run the cluster on Microsoft Azure.

Deploying on Amazon AWS

Kubernetes and Apache Ignite Deployment on AWS article provides a step-by-step guide on how to run the cluster on Amazon AWS.

Deploying on OpenShift

OpenShift is a supported distribution of Kubernetes using Docker containers. However, it has own RBAC (Role-based Access Control) features which are not fully compatible with those provided by Kubernetes out-of-the-box. That is why some commands can lead to "access forbidden" errors. In this case, it is required to use some additional settings, as follows:

Create a service account with view role using OpenShift CLI:

$ oc create sa ignite
$ oc policy add-role-to-user view system:serviceaccount:`<project>`:ignite

Note, <project> is an OpenShift project name.

Specify a namespace parameter of TcpDiscoveryKubernetesIpFinder:

<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
                    <!-- Set a project name as the namespace. -->
                    <property name="namespace" value="<project>"/>
                </bean>
            </property>
        </bean>
    </property>
</bean>

Add serviceAccountName to deployment configuration:

apiVersion: v1
kind: DeploymentConfig
metadata:
  name: ignite-cluster
spec:
  # Start two Ignite nodes by default.
  replicas: 2
  template:
    metadata:
      labels:
        app: ignite
    spec:
      serviceAccountName: ignite
    ...