Today I wanted to get nginx set up on my 3 node microk8s cluster with cert-bot. If you plan on exposing anything to the public, you'll usually reach for a webserver like apache, nginx. There are more modern webservers such as Caddy or Traefik, but I chose to go with nginx.

If you plan on exposing anything to the public, this is a must. I'm on the latest stable channel of microk8s, version 1.21.

Before we get started, there are a couple things we need to do first.

NAT Port Forward Rule

You need to open up port 443 and 80 and forard them to the IP of one of your K8 nodes.

Example: ip.of.node:80 - ip.of.node:443

I use Ubiquiti networking gear. If you use the same, then you can go into Settings > Routing & Firewall > Port Forwarding and set this up.

DNS Setup

I use cloudflare to manage all my DNS. I’ve set up an A record that points to my public IPv4 address. Make sure to disable Cloudflare proxy (at least for now) so that we can verify that cert-manager did in fact create the certificates.

Now that thats out of the way, we can start to get our hands dirty with some Kubernetes deployments!

Nginx deployment

First, enable DNS and ingress for your cluster and then create the deployment yaml file

sudo microk8s enable dns ingress

nginx-service.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: webserver-depl
spec:
  selector:
    matchLabels:
      app: webserver-app
  template:
    metadata:
      labels:
        app: webserver-app
    spec:
      containers:
        - name: webserver-app
          image: nginx:1.8
---
apiVersion: v1
kind: Service
metadata:
  name: webserver-svc
spec:
  selector:
    app: webserver-app
  ports:
  - name: webserver-app
    protocol: TCP
    port: 80
    targetPort: 80

Then apply it with:

sudo microk8s kubectl apply -f

ingress deployment

An ingress in k8s exposes both HTTP and HTTPS routes from outside the cluster, to services within. Be sure to read more about kubernetes ingress objects.

ingress.yaml

apiVersion: networking.k8s.io/v1 
kind: Ingress
metadata:
  name: ingress
spec:
  rules:
  # replace with your domain
  - host: "example.com"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: webserver-svc
            port:
              number: 80

Apply it

sudo microk8s kubectl apply -f ingress.yaml

You should be able to see the default "welcome to nginx" splash screen when you visit your domain now. You'll get an SSL warning when visiting, but don't worry, we'll get to that!

cert-manager setup

Now that nginx is routing traffic, we can set up cert-manager.

sudo microk8s kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.3.1/cert-manager.yaml

We need to create two deployments for our certificates; one for staging, and one for production.

letsencrypt-staging.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
#change to your email
    email: [email protected]
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-staging
    solvers:
    - http01:
        ingress:
          class: public

letsencrypt-staging.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
#change to your email
    email: [email protected]
    privateKeySecretRef:
       name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: public

Apply both:
```bash
sudo microk8s kubectl apply -f letsencrypt-staging.yaml

sudo microk8s kubectl apply -f letsencrypt-prod.yaml

Once thats done, you need to update the ingress deployment to use the staging certificate.

ingress.yaml

apiVersion: networking.k8s.io/v1 
kind: Ingress
metadata:
  name: ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-staging"
spec:
  tls:
  - hosts:
    - example.com
    secretName: tls-secret
  rules:
  - host: "example.com"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: webserver-svc
            port:
              number: 80

Apply it:

sudo microk8s kubectl apply -f ingress.yaml

We can check on the status of our certificate. You'll know it worked if Ready=True. This will take a couple minutes so give it time.

If it never shows "True", then get more info on it by running: sudo microk8s kubectl describe certificate tls-secret

Finally, swap out the staging certificate for the production certficiate.

ingress.yaml

apiVersion: networking.k8s.io/v1 
kind: Ingress
metadata:
  name: ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - example.com
    secretName: tls-secret
  rules:
  - host: "example.com"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: webserver-svc
            port:
              number: 80

Apply the updates:

sudo microk8s kubectl apply -f ingress.yaml`

Time to cross our fingers, we want Ready=True just like our staging certificate.

sudo microk8s kubectl get certificate

If all is well, you should now have a valid certificate when you visit your site!