Pre-requisites

Local setup

Make sure your kubernetes setup connects to the cluster.

Install Helm - on MacOS as easy with brew as:

brew install helm

Add local path storage provisioner

Permanent storage for etcd is needed as we want to set up APISIX in traditional mode (no need to separate planes as all development and also, we do want to take the advantage of Admin API) and in this mode, APISIX needs running etcd which in turn requires resilient storage. It kind of makes no real difference as storage is on local volume and we will rely on resilience in the number of components rather than storage i.e. running 3 instances of etcd. Source.

kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.32/deploy/local-path-storage.yaml

As volumes are created, you can see them under the following path on each relevant node:

ls -la /opt/local-path-provisioner/

Installing APISIX

Get a version from https://github.com/apache/apisix-helm-chart/blob/master/charts/apisix/values.yaml, check into your repository and modify as needed. See modifications in rest of this section and then, install APISIX as here:

helm repo add apisix https://charts.apiseven.com
helm repo update
envsubst '$APISIX_ADMIN_TOKEN $APISIX_VIEWER_TOKEN' < kubernetes/apisix/values.yaml | helm install apisix apisix/apisix --create-namespace  --namespace apisix -f -

But wait - read through customisations - that will explain why envsubst is needed.

While this will run reasonably quickly, it will take while for etcd pods to bring up and even more to APISIX. Couple of minutes.

Selected modifications to values.yaml

The full file, as I customised it, is here.

Take out APISIX_ADMIN_TOKEN & APISIX_VIEWER_TOKEN

Default values.yaml contain admin and viewer default tokens and one should replace by this:

    credentials:
      # -- Apache APISIX admin API admin role credentials
      admin: $APISIX_ADMIN_TOKEN
      # -- Apache APISIX admin API viewer role credentials
      viewer: $APISIX_VIEWER_TOKEN

The tokens can be set on command line and use envsubst to substitute into values.yaml when deploying (like in this post).

export APISIX_ADMIN_TOKEN=`openssl rand -hex 16`
export APISIX_VIEWER_TOKEN=`openssl rand -hex 16`

This way, at least the tokens won’t be committed into git repository storing rest of the files.

Note: with envsubst is also necessary to specify which variables to replace as otherwise it will treat any occurrence of $ followed by string as variable and replace by nothing.

Use local storage for etcds

See above.

etcd:
### needs this to use loco file system to simulate persistent volumes for storage - we don't have them so we use boot volume.
    persistence:
      enabled: true
      storageClass: local-path
      size: 1Gi

Server tokens

The most basic security measure - don’t tell others exactly what software you’re running… or at least hide away version. Note that this is not perfect and some default pages do leak product and version number (or at least did in the past).

enableServerTokens: false

Configure APISIX to pass requests to ramblings

Add an upstream and route as below:

curl http://127.0.0.1:9180/apisix/admin/upstreams/1 -H "X-API-KEY: $APISIX_ADMIN_TOKEN" -X PUT -i -d '{
    "hash_on": "vars",
    "name": "DeckAPI",
    "nodes": 
      {
        "ramblings-app.ramblings.svc.cluster.local:8080": 1
      }
    ,
    "desc": "DeckAPI ",
    "type": "roundrobin",
    "pass_host": "rewrite",
    "upstream_host": "ramblings-app.ramblings.svc.cluster.local",
    "tls": {
      "verify": false
    },
    "scheme": "http"
}'

Routes

curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $APISIX_ADMIN_TOKEN" -X PUT -i -d '{
        "name": "1",
        "status": 1,
        "id": "1",
        "enable_websocket": false,
        "priority": 0,
        "uri": "/deck-api/*",
        "methods": [
          "GET",
          "POST",
          "PUT",
          "DELETE"
        ],
        "upstream_id": "1"
}'

Both should be returing HTTP 201 with upstream and route.

Note: This is all very straightforward but with upstreams, we use kubernetes service to indicate the destination for reequests and kubernetes service expects that requests sent to it match what the service name is. This is not likely to match to external DNS so in a way, one must use combination of rewrite and upstream_host to make work. We will change this to use APISIX kubernetes discovery to bypass the service and let APISIX control it.

Goodies

APISIX UI

If you forward admin port locally, you should be able to reach the ‘ui’ for APISIX via http://localhost:53959/ui URL (provided admin port is mapped as 53959).

UI application is simple but useful to avoid manually firing APIs at APISIX but it also uses native APISIX APIs so it is useful for prototyping and testing.

It can be disabled in values.yaml file and probably best to disable in production.

Deploying updated values.yaml to the cluster

envsubst '$APISIX_ADMIN_TOKEN $APISIX_VIEWER_TOKEN' < kubernetes/apisix/values.yaml | helm upgrade --install apisix apisix/apisix --create-namespace  --namespace apisix -f -

Get upstreams

curl http://127.0.0.1:9180/apisix/admin/upstreams -H "X-API-KEY: $APISIX_ADMIN_TOKEN" -X GET

Get routes

curl http://127.0.0.1:9180/apisix/admin/routes -H "X-API-KEY: $APISIX_ADMIN_TOKEN" -X GET