The OpenShift platform is a “batteries included” distribution of Kubernetes. It comes with EVERYTHING you need to run a Kubernetes platform from a developer and sysadmin-friendly UI, to monitoring, alerting, platform configuration, and ingress networking. OpenShift was one of the first Kubernetes distributions to realize that having a Kubernetes platform that solved how to load-balance incoming requests for applications was important. OpenShift achieved this through the use of “Routes”. Upstream in Kubernetes this need has been implemented through the use of Ingress, and more recently Gateway API.
While OpenShift comes with a built-in Ingress controller, it is also possible to add additional Ingress controllers or even replace the existing Ingress controller with a third-party controller. These third-party ingress controllers don’t have to be software-based either. Companies like Citrix and F5 have created ingress controllers that allow for automated configuration of external load balancing appliances such as Citrix ADC (formerly Netscaler) and F5 BIG-IP.
In this post, we will look at the Citrix ADC Ingress Controller and configure it against a virtual instance of Citrix ADC called Citrix ADC VPX.
A Word of Warning: The following post shows how to configure OpenShift with Citrix ADC integration, but it is NOT intended as a production solution. Be sure to work with your Red Hat and Citrix support personnel to create a fully supported configuration for your needs.
To try this out you will need an existing Citrix ADC that you wish to configure, as well as an existing OpenShift Cluster. Your Citrix ADC needs to have a Subnet IP (SIP) on the same subnet that your OpenShift cluster is deployed on. This allows the Citrix ADC appliance to directly connect to your OpenShift cluster without passing through a router. Your OpenShift cluster must also be using OpenShiftSDN as the SDN network in your cluster. The current release of the Citrix Operator (v1.18.5 as of this post) does NOT support OVNKubernetes. Based on release notes, Citrix ADC Ingress controller version 1.21.9 and later will support OVN-Kubernetes.
Configure Citrix ADC
We will NOT be covering the initial configuration of a Citrix ADC. If you need assistance in deploying a virtual Citrix ADC see Deploying VPX for instructions on getting a virtual instance deployed. Then come back here to continue.
We will use ssh to connect to the ADC and we will create a new user login with the proper permissions that will be used by the Citrix Ingress controller to configure our ADC. The following commands are from the Official ADC Ingress Controller install docs.
WARNING: Be sure to update the password as listed below to something more secure.
$ ssh nsroot@<citrix lb ip>
> add system user ocpint ocpint1234
> add cmdpolicy ocpint-policy ALLOW '^(\?!shell)(\?!sftp)(\?!scp)(\?!batch)(\?!source)(\?!.*superuser)(\?!.*nsroot)(\?!install)(\?!show\s+system\s+(user|cmdPolicy|file))(\?!(set|add|rm|create|export|kill)\s+system)(\?!(unbind|bind)\s+system\s+(user|group))(\?!diff\s+ns\s+config)(\?!(set|unset|add|rm|bind|unbind|switch)\s+ns\s+partition).*|(^install\s*(wi|wf))|(^\S+\s+system\s+file)^(\?!shell)(\?!sftp)(\?!scp)(\?!batch)(\?!source)(\?!.*superuser)(\?!.*nsroot)(\?!install)(\?!show\s+system\s+(user|cmdPolicy|file))(\?!(set|add|rm|create|export|kill)\s+system)(\?!(unbind|bind)\s+system\s+(user|group))(\?!diff\s+ns\s+config)(\?!(set|unset|add|rm|bind|unbind|switch)\s+ns\s+partition).*|(^install\s*(wi|wf))|(^\S+\s+system\s+file)'
> bind system user ocpint ocpint-policy 0
You are leaving without saving changes. Save changes now (Y/N)? [N]:y
Install and Configure Citrix Ingress Operator Controller
With the Citrix ADC configured with our “ocpint” user, we can now install and configure OpenShift and the Citrix Ingress Controller. We will start by installing the Citrix Ingress Controller Operator.
- Log in to the OpenShift Console
- Select Operators->OperatorHub
- Search for “Citrix Ingress”
- Select “Citrix Ingress Controller” and click Install
- Select “A specific namespace..”
- Using the drop-down, select “Create Project” and call it “citrix-ingress”
- Leave all remaining options at default and click Install
With the operator installed, we can create a Citrix Ingress controller. We will do the remainder of this install from the command line.
Install and Configure Citrix Ingress Controller
We will start by creating a secret that contains the username and password you created in the Configure Citrix ADC section above.
$ oc login
$ oc project citrix-ingress
$ oc create secret generic nslogin --from-literal=username='ocpint' --from-literal=password='ocpint1234'
We will need a TLS secret that holds the TLS certificate that will be used by the Citrix ADC. For this post, we will re-use the existing certificate that is in use by the OpenShift router, however, you can also create a new certificate if you wish by following the steps documented in Replacing Default Ingress to create a new SSL certificate. We will start by getting a copy of the existing certificate:
$ oc get secret/router-certs-default -n openshift-ingress -o yaml > citrix-default-cert.yaml
citrix-default-cert.yaml and update the file, removing the excess data that is not required. Update the fields to match the YAML below:
tls.crt: <cert data here>
tls.key: <key data here>
With the citrix-default-cert.yaml updated, we can apply this to our cluster in the citrix-ingress Project.
$ oc create -f citrix-default-cert.yaml -n citrix-ingress
We can now create our Citrix ADC ingress controller. You will need the following information:
- NetscalerManagement IP - this is the IP or hostname used to manage the Netscaler
- Subnet IP - This is an IP address that is configured on the Netscaler and assigned to the subnet that your OpenShift cluster is on. You can not use this ingress controller without a SNIP
- VIP for the Ingress - This is the IP that will be used as your new load-balanced Ingress IP
- Cluster Name - This is the name of your OpenShift cluster. You must configure this if you will be connecting multiple clusters to the ADC.
Using the YAML below, create a new file called
citrix-ingress-controller.yml and update lines 13, 16, 19, 22, and 23 with the appropriate information for your environment.
4 name: cic
5 namespace: citrix-ingress
7 adcCredentialSecret: 'nslogin'
8 cncPbr: false
10 install: false
11 retainOnDelete: false
12 defaultSSLCertSecret: 'citrix-ingress/citrix-default-cert'
13 entityPrefix: '<cluster Name>'
14 ignoreNodeExternalIP: false
16 accept: 'no'
17 logLevel: INFO
18 nodeWatch: true
19 nsIP: <ns Managment IP>
20 nsPort: 443
21 nsProtocol: HTTPS
22 nsSNIPS: '<subnet ip>'
23 nsVIP: '<virtual ip>'
24 openshift: true
25 pullPolicy: Always
27 create: true
28 setAsDefaultIngressClass: false
29 updateIngressStatus: false
With your CitrixIngressController defined in YAML, its time to apply it to your cluster:
$ oc create -f citrix-ingress-controller.yml
At this point, you can update your DNS entry for your Ingress wildcard to point to the VIP that you defined in the YAML in the previous section. The process for updating your DNS record will depend on your specific DNS infrastructure. Once your DNS records have updated, refresh your OpenShift console and you will be sending all ingress traffic through the Citrix Netscaler.
How does this Work?
The Citrix Ingress Controller connects to the Citrix ADC and configures Virtual Servers for each OpenShift Route that exists in the cluster. It will also create new Virtual Servers for any new Routes that are created in the cluster automatically. If you scale your OpenShift cluster (adding or removing worker nodes), the Citrix Ingress Controller will actively add or remove the new nodes to the static routes configured on the ADC.
One thing to keep in mind, the architure of the Citrix Ingress Controller requires that if you are using multiple OpenShift clusters ensure that your ClusterNetwork CIDRs do not overlap, or you will have issues with connectivity to your clusters. Also, ensure that you define a different entityPrefix for each cluster to eliminate any cross talk or cross configuration.
OpenShift has multiple options for Ingress and Routes. If you are looking for ways to integrate with an upstream Citrix Netscaler Citrix supplies an easy-to-use Operator that can be integrated with your existing OpenShift cluster. This integration brings the familiarity of a hardware-based load balancer to the OpenShift and Kubernetes world.