Traefik + Azure Kubernetes

Just a random note or two …

At work we moved to use Azure for most of our hosting, for ‘reasons’. We run much of our workload through kubernetes.

The Azure portal has a nice integration to easily deploy a project from a github repo into Kubernetes, and when it does, it puts each project in it’s own namespace.

In order to deploy some new functionality, I finally bit the bullet and tried to get some sort of Ingress router in place. I chose to use Traefik.

Some random notes ….

1. You need to configure/run Traefik with –providers.kubernetescrd.allowCrossNamespace=true, without this it’s not possible for e.g. Traefik (in the ‘traefik’ namespace) to use MyCoolApi in the ‘api’ namespace. The IngressRoute HAS to be in the same namespace as traefik is running in …. and the IngressRoute needs to reference a service in a different namespace…
2. While you’re poking around, you probably want to load traefik with –log.level=DEBUG
3. Use cert-manager for LetsEncrypt certificates (see https://www.andyroberts.nz/posts/aks-traefik-https/ for some details)
4. You need to make sure you’re using a fairly recent Kubernetes variant – ours was on 1.19.something, which helpfully just silently”didn’t work” when trying to get the cross namespace stuff working.
5. Use k9s as a quick way to view logs/pods within the cluster.

Example Ingress Route

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  namespace: traefik
  name: projectx-ingressroute
  annotations:
    kubernetes.io/ingress.class: traefik
    cert-manager.io/cluster-issuer: my-ssl-cert

spec:
  entryPoints:
    - websecure    
  routes:
    - kind: Rule
      match: Host(`mydomain.com`) && PathPrefix(`/foo`) 
      services:
        - name: foo-api-service
          namespace: foo-namespace
          port: 80
  tls:
    secretName: my-ssl-cert-tls
    domains:
    - main: mydomain.com

Initially I tried to use traefik’s inbuilt LetsEncrypt provider support; and wanted to have a shared filesystem (azure storage, cifs etc) so multiple Traefik replicas could both share the same certificate store…. unfortunately this just won’t work, as the CIFS share gets mounted with 777 perms, which Traefik refuses to put up with.