Deploy Minikube on remote Ubuntu

Never install the Minikube in the same machine which you are coding!

I worked with Kubernetes for around one year, and the number of services grows from 0 to about 20. All services have been deployed on the local machine when developing new features but the Minikube ate all the memory and CPU resource. It’s awful that I even couldn’t open one new tab on Chrome. To solve this problem, I decided to buy a new machine (such as Intel NUC), and deploy the cluster on the new machine. It could also be feasible that you deploy the services on idle PCs if you can install Ubuntu on them.

Minikube ate all my Mac resource

Install Minikube on Ubuntu

We choose to use KVM as the virtual machine on Ubuntu because it does not use tons of resources but did nothing compared with Virtualbox.

• Install kubectl

$ curl -Lo kubectl https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/bin/.
$ kubectl version

• Install Minikube (amd64 / x86_64), see more details on the official website.

$ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
$ sudo dpkg -i minikube_latest_amd64.deb

• Install KVM on Ubuntu

$ sudo apt install cpu-checker && sudo kvm-ok
# Pre-installation check
$ sudo kvm-ok
# You should see the following response
INFO: /dev/kvm exists
KVM acceleration can be used
# Install official dependencies
$ sudo apt-get install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils

# Install kvm2 driver
$ curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2
$ sudo install docker-machine-driver-kvm2 /usr/local/bin/

• Run Minikube

# Change memory, cpus and disk-size value based on your situation. Be careful that you cannot change it after the cluster is created unless recreate it.
# Find you host IP, 192.168.1.10 is an example!
$ minikube start --vm-driver kvm2 --memory 12046 --cpus 6 --apiserver-ips=192.168.1.10 --disk-size='100000mb'

Port forward

I spent hours configuring port forwarding with iptables and ssh, but none of them works well. Then I tried it with Nginx, everything works as expected : ) I will show how to forward port for Kubectl and you can do a similar thing for the specific service you have in Minikube.

Port forwarding

• Install Nginx

$ sudo apt update
$ sudo apt install nginx

• Config Nginx

# First check the minikube ip, for example 192.168.39.50
$ minikube ip
# Config nginx.conf file, make it transform traffic without dealing the SSL and certs
$ sudo vim /etc/nginx/nginx.conf
# Add the following config and save the file
stream {
  server {
      listen 192.168.1.10:52000;
      #TCP traffic will be forwarded to the specified server
      proxy_pass 192.168.39.50:8443;       
  }
}
# Check Nginx config
$ sudo nginx -t
# Restart the Nginx
$ sudo systemctl restart nginx

• Open the port on the firewall

# Add port
$ sudo ufw allow 52000
# Check ufw status
$ sudo ufw status

Remote access

You can access the Minikube cluster by adding the Minikube context on the remote machine, such as the Mac you used for coding.

• Check Minikube config (on the server machine)

# Show the config information
$ cat ~/.kube/config
# Example
apiVersion: v1
clusters:
- cluster:
  certificate-authority: /home/ubuntu/.minikube/ca.crt
...
user:
   client-certificate:
      /home/ubuntu/.minikube/profiles/minikube/client.crt
   client-key: 
      /home/ubuntu/.minikube/profiles/minikube/client.key

• Config the Minikube (on the remote machine!)

1. Download the ca.crt,client.crt, clent.key files
2. Edit remote machine config file, add cluster, context, and user separately.

$ sudo vim ~/.kube/config
apiVersion: v1
clusters:
- cluster:
    certificate-authority: ./FILE_PATH/ca.crt # <-- new
    server: https://192.168.1.10:52000  # <-- server host IP
  name: minikube
contexts:
- context:
    cluster: minikube
    user: minikube
  name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
  user:
    client-certificate: ./FILE_PATH/client.crt # <-- new
    client-key: ./FILE_PATH/client.key # <-- new

Now you can access the Minikube by kubectl command on the remote machine and it even works well port-forward from services.