K8s Ingress with NSX Advanced Load Balancer

Abbreviations used in this article:

  • NSX Advanced Load Balancer = NSX-ALB
  • K8s = Kubernetes (8 letters between the K and s in Kubernetes)
  • SSL = Secure Sockets Layer
  • AKO = Avi Kubernetes Operator (AVI now a VMware product called NSX Advanced Load Balancer)

In one of my previous posts (https://yikes.guzware.net/2020/10/08/ako-with-antrea-on-native-k8s-cluster/) I wrote about how to install and configure AKO (Avi Kubernetes Operator) to use as Service type LoadBalancer.

This post will try to cover the basics of how to use NSX Advanced LoadBalancer (from now on NSX-ALB) as Ingress.
For more information on Ingress in Kubernetes: https://kubernetes.io/docs/concepts/services-networking/ingress/

An API object that manages external access to the services in a cluster, typically HTTP.
Ingress may provide load balancing, SSL termination and name-based virtual hosting.

My use case for needing an Ingress was to host my own WordPress page in my K8s cluster with SSL termination and to expose my WordPress deployment as a HTTP service (not exposing the WordPress pod ports 80/443 through Service type Loadbalancer). As I am completely new to the world of Ingress the whole process of getting this up and running was something new I had to learn.

It started out with just getting WordPress up and running by using the helm chart bitnami/wordpress which worked out fine (I will write a post about the additional requirements I had to implement in my K8s environment to get this up and running as this is only focusing on Ingress with NSX-ALB).
But I wanted to expose my wordpress installation to the Internet, and before doing that I wanted it to be signed by my LetsEncrypt certificate. Here the first challenge started. Thinking as with regular vms its only the matter of importing the certs to my ssl folder in Linux and point Apache to it. Turns out its not the way its meant to be done. We use Ingress and “inject” the SSL certificate there instead and let our NSX-ALB handle the SSL certificate (SSL Termination).

Diagram depicts SSL Termination being performed at the secure socket layer (ssl) via a load blancer between application users and application web servers.



My first challenge was to my certificate “into” K8s. What I ended up doing was like this (its probably not the only or even the “most” correct way to do it, but it worked out the way I wanted):

Importing the certificate
I wanted to utilize a default secret or certificate to use for the Ingresses in my environment as I have a wildcard certificate for my domain. This made it also a bit simpler for me when deploying additional services with Ingress. Some digging around until I found a way to do it.

Import the certs with kubectl command
To import the certs into K8s I ran this command:
kubectl create secret generic nameofsecret -n avi-system --from-file=key.pem --from-file=cert.pem

Then I need to make NSX-ALB aware of my certificate and make it the default secret, so I grabbed an example from avinetworks.com documentation https://avinetworks.com/docs/ako/1.4/default-secret-for-tls-ingresses/:


apiVersion: v1
kind: Secret
metadata:
  name: router-certs-default
  namespace: avi-system
type: kubernetes.io/tls
data:
  tls.crt: 
    -----BEGIN CERTIFICATE-----
    [...]
    -----END CERTIFICATE-----
  tls.key:
    -----BEGIN PRIVATE KEY-----
    [...]
    -----END PRIVATE KEY-----

The only reason I imported my cert earlier with the command kubectl create secret was to grab it in a “better” format so I could paste into the example above with this command:
kubectl get secrets -n avi-system nameofsecret -o yaml

I then copied the output of this command into the example above and applied it with
kubectl apply -f nameofexamplewithcert.yaml

and done.
The next thing was to just apply an empty/dummy Ingress service to see if NSX-ALB created it for me. I had to add the pathType: ImplementationSpecific to the example for make it to work.


apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress1
  annotations:
    ako.vmware.com/enable-tls: "true"
spec:
  ingressClassName: avi-lb
  rules:
  - host: "ingress.guzware.net"
    http:
      paths:
      - path: /foo
        pathType: ImplementationSpecific
        backend:
          service:
            name: avisvc1
            port:
              number: 80

As soon as I applied this one NSX-ALB automatically created the Virtual Service for me and used the correct fqdn (hostname) with the correct certificate and I could resolve it (because NSX-ALB is configured as a DNS provider). Plug and play, fully automatic. Lovely. Now I could test it out on my WordPress deployment.

Leave a Reply

Your email address will not be published. Required fields are marked *