Kubernetes Quickstart
minikube
quickly sets up a local Kubernetes cluster on macOS, Linux, or Windows (WSL). This quickstart is a great way to explore running your own OpenZiti Controller, Router, and Console.
Tools of the Trade
We'll use the preferred minikube
Docker driver for this quickstart. You can run minikube
in WSL with Docker Engine or Docker Desktop, but keep an eye out for one extra step to run minikube tunnel
at the necessary point in the process.
- Install Docker
- Install
kubectl
- Install Helm
- Install
minikube
- Install
ziti
CLI - Install an OpenZiti Tunneler app
- Optional: Install
curl
andjq
for testing an OpenZiti Service in the terminal.
Make sure these command-line tools are available in your executable search PATH
.
Configure DNS
Your computer running minikube
needs to resolve these three domain names. They will all resolve to the same IP address where minikube exposes ingresses on your OS.
- miniziti-controller.miniziti.internal
- miniziti-console.miniziti.internal
- miniziti-router.miniziti.internal
- Windows (WSL2)
- macOS
- Linux
This step allows you to connect to the minikube
ingresses from Windows or WSL. We'll need that connection to the ingresses later when we run the ziti
CLI in WSL.
Enable localhost binding in WSL.
%USERPROFILE%\.wslconfig
Add
localhostforwarding=true
to the[wsl2]
section like this.[wsl2]
# Turn off default connection to bind WSL 2 localhost to Windows localhost
localhostforwarding=trueAdd the DNS names to the system hosts file. This allows us to resolve the DNS names from both Windows and WSL.
Edit the system hosts file.
%SYSTEMROOT%\system32\drivers\etc\hosts
Add this line to the system hosts file.
127.0.0.1 miniziti-controller.miniziti.internal miniziti-router.miniziti.internal miniziti-console.miniziti.internal
Restart WSL.
wsl --shutdown
In WSL, verify that Docker restarted successfully.
docker ps
If Docker has restarted then you'll see your running containers, if any. If you don't have any running containers then you'll only see the headings, like this. This is a healthy result and you can skip to the next numbered step.
If you see an error then you need to restart Docker. If you have Docker Desktop it may be necessary to "Quick Docker Desktop" in the system tray menu (right-click tray icon).
You can verify that the
docker-desktop-data
instance is running with this PowerShell command.wsl -l -v
Example output:
If Docker Desktop is not running then it may be necessary to start it in WSL, depending on how you have installed Docker Desktop.
In WSL, verify that localhost is bound. You know it's working if you see the *.miniziti.internal DNS names in
/etc/hosts
duplicated from Windows.grep ziti /etc/hosts
It should look like this.
In WSL, Run
minikube tunnel
.Keep a separate terminal window open so you can make sure the tunnel is still running and provide your password if prompted. Create a "miniziti" profile if you haven't, and run the tunnel.
infoYou will likely be prompted by
minikube tunnel
for your WSL user's password, but this prompt may not occur immediately. This grants permission for the tunnel to add a route to the minikube node IP.minikube --profile miniziti start
minikube --profile miniziti tunnel
In macOS, minikube
sets up localhost port forwarding to the IP of the Docker container of the minikube
node. Docker will forward localhost:80,443/tcp to support the HTTP ingresses to your miniziti cluster.
Add the DNS names to your system hosts file.
sudo tee -a /etc/hosts <<< "127.0.0.1 miniziti-controller.miniziti.internal miniziti-router.miniziti.internal miniziti-console.miniziti.internal"
Run
minikube tunnel
.Keep a separate terminal window open so you can make sure the tunnel is still running and provide your password if prompted. Create a "miniziti" profile if you haven't, and run the tunnel.
infoYou will likely be prompted by
minikube tunnel
for your macOS user's password, but this prompt may not occur immediately. This grants permission for the tunnel to add a route to the minikube node IP.minikube --profile miniziti start
minikube --profile miniziti tunnel
On Linux, the IP of the minikube
node running in Docker is routeable via the bridge interface created by minikube
. Create a "miniziti" profile if you haven't, and obtain the node's external IP for your hosts file.
minikube --profile miniziti start
sudo tee -a /etc/hosts <<< "$(minikube --profile miniziti ip) miniziti-controller.miniziti.internal miniziti-router.miniziti.internal miniziti-console.miniziti.internal"
- miniziti script
- Manual Steps
miniziti Script
You can run the miniziti shell script or perform the steps manually yourself. Click the "Manual Steps" tab directly above to switch your view.
It's recommended that you read the script before you run it. It's safe to re-run the script if it encounters a temporary problem.
Be sure to complete the DNS setup steps above before running this command.
bash ./miniziti.bash
Manual Steps to Learn by Doing
This section explains the actions performed by the miniziti.bash
script.You need a BASH or ZSH terminal for pasting commands.
Create the miniziti
Cluster
First, let's create a brand new minikube
profile named "miniziti".
minikube --profile miniziti start
minikube
will try to configure the default context of your KUBECONFIG. Let's test the connection to the new cluster.
You can always restore the KUBECONFIG context from this Minikube quickstart like this:
minikube --profile miniziti update-context
Let's do a quick test of the current KUBECONFIG context.
kubectl cluster-info
A good result looks like this (no errors).
Install Ingress Addons
You will need two Minikube addons:
ingress
: installs the Nginx ingress controller. Ingresses provide access into the cluster and are the only things exposed to networks outside the cluster. This is required by the miniziti script, but the Helm charts can be configured to use other ingress controllers.ingress-dns
: provides a DNS server that can answer queries about the cluster's ingresses, e.g. "miniziti-controller.miniziti.internal" which will be created when you install the OpenZiti Controller Helm chart.
minikube --profile miniziti addons enable ingress
minikube --profile miniziti addons enable ingress-dns
OpenZiti will need SSL passthrough, so let's patch the ingress-nginx
deployment to enable that feature.
kubectl patch deployment "ingress-nginx-controller" \
--namespace ingress-nginx \
--type='json' \
--patch='[{"op": "add",
"path": "/spec/template/spec/containers/0/args/-",
"value":"--enable-ssl-passthrough"
}]'
# wait for ingress-nginx
kubectl wait jobs "ingress-nginx-admission-patch" \
--namespace ingress-nginx \
--for condition=complete \
--timeout=120s
kubectl wait pods \
--namespace ingress-nginx \
--for=condition=ready \
--selector=app.kubernetes.io/component=controller \
--timeout=120s
Now your miniziti cluster is ready for some OpenZiti!
Install the OpenZiti Controller
Allow Kubernetes to Manage Certificates
You need to install the required Custom Resource Definitions (CRD) for the OpenZiti Controller.
kubectl apply \
--filename https://github.com/cert-manager/cert-manager/releases/latest/download/cert-manager.crds.yaml
kubectl apply \
--filename https://raw.githubusercontent.com/cert-manager/trust-manager/v0.4.0/deploy/crds/trust.cert-manager.io_bundles.yaml
Add the OpenZiti Helm Repository
Add the OpenZiti Helm Repo
helm repo add "openziti" https://openziti.io/helm-charts/
Install the Controller
Let's create a Helm release named "miniziti-controller" for the OpenZiti Controller. This will also install sub-charts cert-manager
and trust-manager
in the same Kubernetes namespace "ziti-controller."
Install the Controller chart
helm upgrade "ziti-controller" openziti/ziti-controller \
--namespace miniziti --create-namespace --install \
--set clientApi.advertisedHost="miniziti-controller.miniziti.internal" \
--set trust-manager.enabled=true \
--set cert-manager.enabled=true \
--set trust-manager.app.trust.namespace="miniziti" \
--values https://openziti.io/helm-charts/charts/ziti-controller/values-ingress-nginx.yamlThis may take a few minutes. Wait the controller's pod status progress to "Running." You can get started on the DNS set up in the next section, but you need the controller up and running to install the router.
kubectl wait deployments "ziti-controller" \
--namespace miniziti \
--for condition=Available=True \
--timeout=240s
Cluster DNS
Configure CoreDNS in the miniziti cluster. This is necessary no matter which host DNS resolver method you used above.
Add the DNS query forwarder for names like *.miniziti.internal to the end of the value of
Corefile
in CoreDNS's configmap. Don't forget to substitute the real IP fromminikube --profile miniziti ip
if you get something different than "192.168.49.2", and be mindful to keep the indentation the same as the default.:53
handler.Edit as shown, save file, then exit the editorkubectl edit configmap "coredns" \
--namespace kube-systemminiziti.internal:53 {
errors
cache 30
forward . 192.168.49.2
}Delete the running CoreDNS pod so a new one will pick up the Corefile change you just made.
kubectl delete pods \
--namespace kube-system \
--selector k8s-app=kube-dnsVerify that the DNS names for the ingress zone, e.g. *.miniziti.internal, are resolvable from inside your cluster. This is required for your pods to communicate with the OpenZiti Controller's advertised address.You will know it's working because you see the same IP address in the response as when you run
minikube --profile miniziti ip
.kubectl run "dnstest" --rm --tty --stdin --image=busybox --restart=Never -- \
nslookup miniziti-controller.miniziti.internal
Install the Router
Log in to OpenZiti.
kubectl get secrets "ziti-controller-admin-secret" \
--namespace miniziti \
--output go-template='{{index .data "admin-password" | base64decode }}' \
| xargs -rl ziti edge login miniziti-controller.miniziti.internal:443 \
--yes --username "admin" \
--passwordCreate a Router with role "public-routers" and save the enrollment one-time-token as a temporary file.
ziti edge create edge-router "miniziti-router" \
--role-attributes "public-routers" \
--tunneler-enabled \
--jwt-output-file /tmp/miniziti-router.jwtInstall the Router Chart.
helm upgrade "ziti-router" openziti/ziti-router \
--namespace miniziti --create-namespace --install \
--set-file enrollmentJwt=/tmp/miniziti-router.jwt \
--set edge.advertisedHost=miniziti-router.miniziti.internal \
--set ctrl.endpoint=ziti-controller-ctrl.miniziti.svc:443 \
--values https://openziti.io/helm-charts/charts/ziti-router/values-ingress-nginx.yaml
kubectl wait deployments "ziti-router" \
--namespace miniziti \
--for condition=Available=TrueThese Helm chart values configure the router to use the controller's cluster-internal service that provides the router control plane, i.e., the "ctrl" endpoint.
Verify the new router is "online=true"
ziti edge list edge-routers
Install the Console
Install the chart
helm upgrade "ziti-console" openziti/ziti-console \
--namespace miniziti --create-namespace --install \
--set ingress.advertisedHost="miniziti-console.miniziti.internal" \
--set "settings.edgeControllers[0].url=https://ziti-controller-client.miniziti.svc:443" \
--values https://openziti.io/helm-charts/charts/ziti-console/values-ingress-nginx.yamlWait for deployment.
You'll see an Nginx 503 error while the console is deploying.
kubectl wait deployments "ziti-console" \
--namespace miniziti \
--for condition=Available=True \
--timeout=240sGet the admin password on your clipboard.
kubectl get secrets "ziti-controller-admin-secret" \
--namespace miniziti \
--output go-template='{{"\nINFO: Your OpenZiti Console http://miniziti-console.miniziti.internal password for \"admin\" is: "}}{{index .data "admin-password" | base64decode }}{{"\n\n"}}'Open https://miniziti-console.miniziti.internal in your web browser and login with username "admin" and the password from your clipboard.
Create OpenZiti Identities and Services
Here's a BASH script that runs several ziti
CLI commands to illustrate a minimal set of identities, services, and policies.
ziti edge create identity device "miniziti-client" \
--jwt-output-file /tmp/miniziti-client.jwt --role-attributes httpbin-clients
ziti edge create identity device "httpbin-host" \
--jwt-output-file /tmp/httpbin-host.jwt --role-attributes httpbin-hosts
ziti edge create config "httpbin-intercept-config" intercept.v1 \
'{"protocols":["tcp"],"addresses":["httpbin.miniziti.private"], "portRanges":[{"low":80, "high":80}]}'
ziti edge create config "httpbin-host-config" host.v1 \
'{"protocol":"tcp", "address":"httpbin","port":8080}'
ziti edge create service "httpbin-service" \
--configs httpbin-intercept-config,httpbin-host-config
ziti edge create service-policy "httpbin-bind-policy" Bind \
--service-roles '@httpbin-service' --identity-roles '#httpbin-hosts'
ziti edge create service-policy "httpbin-dial-policy" Dial \
--service-roles '@httpbin-service' --identity-roles '#httpbin-clients'
ziti edge create edge-router-policy "public-routers" \
--edge-router-roles '#public-routers' --identity-roles '#all'
ziti edge create service-edge-router-policy "public-routers" \
--edge-router-roles '#public-routers' --service-roles '#all'
ziti edge enroll /tmp/httpbin-host.jwt
Install the httpbin
Demo API Server Chart
This Helm chart installs an OpenZiti fork of go-httpbin
, so it doesn't need to be accompanied by an OpenZiti Tunneler. We'll use it as a demo API to test the OpenZiti Service you just created named "httpbin-service".
helm install "miniziti-httpbin" openziti/httpbin \
--set-file zitiIdentity=/tmp/httpbin-host.json \
--set zitiServiceName=httpbin-service
Add the Client Identity
Add the client identity you created to your OpenZiti Tunneler.
Follow the instructions for your tunneler OS version to add the OpenZiti Identity that was saved as filename /tmp/miniziti-client.jwt
(or WSL's "tmp" directory, e.g., \\wsl$\Ubuntu\tmp
in Desktop Edge for Windows).
As soon as identity enrollment completes you should have a new OpenZiti DNS name available to this device. Let's test that with a DNS query.
nslookup httpbin.miniziti.private
Test the Demo API Service
curl -sSf -XPOST -d ziti=awesome http://httpbin.miniziti.private/post | jq .data
Visit http://httpbin.miniziti.private/get in your web browser in macOS, Linux, or Windows to see a JSON test response from the demo server.
Explore the OpenZiti Console
Now that you've successfully tested the OpenZiti Service, check out the various entities in your that were created by the script in http://miniziti-console.miniziti.internal/.
Next Steps
In the OpenZiti Console, try to revoke then restore your permission to acess the demo services.
Deploy a non-Ziti demo application to Kubernetes and securely share it with a Ziti proxy pod
Add a configs, service, and policies to access the Kubernetes apiserver with OpenZiti.
Hint: the apiserver's address is "kubernetes.default.svc:443" inside the cluster.
Hint: After you create the configs, service, and policies, grant "Bind" permission for the service to "miniziti-router" by adding a role.
Connect to the K8s apiserver from another computer with
kubeztl
, the OpenZiti fork ofkubectl
.kubeztl
works by itself without an OpenZiti Tunneler.
Cleanup
- Windows (WSL2)
- macOS
- Linux
Remove the relevant
*.miniziti.internal
DNS names from the system hosts file.Edit the system hosts file.
%SYSTEMROOT%\system32\drivers\etc\hosts
Remove this line.
127.0.0.1 miniziti-controller.miniziti.internal miniziti-router.miniziti.internal miniziti-console.miniziti.internal
Remove the relevant
*.miniziti.internal
names from/etc/hosts
.sudo sed -iE '/miniziti.*\.internal/d' /etc/hosts
Remove the relevant
*.miniziti.internal
names from/etc/hosts
.sudo sed -iE '/miniziti.*\.internal/d' /etc/hosts
Delete the cluster.
minikube --profile miniziti delete
In your OpenZiti Tunneler, "Forget" your Identity.