clusterctl for Developers

This document describes how to use clusterctl during the development workflow.

Prerequisites

  • A Cluster API development setup (go, git, etc.)
  • A local clone of the Cluster API GitHub repository
  • A local clone of the GitHub repositories for the providers you want to install

Getting started

Build clustertl

From the root of the local copy of Cluster API, you can build the clusterctl binary by running:

make clusterctl

The output of the build is saved in the bin/ folder; In order to use it you have to specify the full path, create an alias or copy it into a folder under your $PATH.

Create a clusterctl-settings.json file

Next, create a clusterctl-settings.json file and place it in your local copy of Cluster API. Here is an example:

{
  "providers": ["cluster-api","bootstrap-kubeadm","control-plane-kubeadm", "infrastructure-aws"],
  "provider_repos": ["../cluster-api-provider-aws"]
}

providers (Array[]String, default=[]): A list of the providers to enable. See available providers for more details.

provider_repos (Array[]String, default=[]): A list of paths to all the providers you want to use. Each provider must have a clusterctl-settings.json file describing how to build the provider assets.

Run the local-overrides hack!

You can now run the local-overrides hack from the root of the local copy of Cluster API:

cmd/clusterctl/hack/local-overrides.py

The script reads from the local repositories of the providers you want to install, builds the providers’ assets, and places them in a local override folder located under $HOME/.cluster-api/overrides/. Additionally, the command output provides you the clusterctl init command with all the necessary flags.

clusterctl local overrides generated from local repositories for the cluster-api, bootstrap-kubeadm, control-plane-kubeadm, infrastrcuture-aws providers.
in order to use them, please run:

clusterctl init  --core cluster-api:v0.3.0 --bootstrap kubeadm:v0.3.0 --infrastructure aws:v0.5.0

See Overrides Layer for more information on the purpose of overrides.

Available providers

The following providers are currently defined in the script:

  • cluster-api
  • bootstrap-kubeadm
  • control-plane-kubeadm
  • infrastructure-docker

More providers can be added by editing the clusterctl-settings.json in your local copy of Cluster API; please note that each provider_repo should have its own clusterctl-settings.json describing how to build the provider assets, e.g.

{
  "name": "infrastructure-aws",
  "config": {
    "componentsFile": "infrastructure-components.yaml",
    "nextVersion": "v0.5.0",
  }
}

Additional steps in order to use the docker provider

Before running the local-overrides hack:

  • Run make -C test/infrastructure/docker docker-build REGISTRY=gcr.io/k8s-staging-capi-docker to build the docker provider image using a specific REGISTRY (you can choose your own).

  • Run make -C test/infrastructure/docker generate-manifests REGISTRY=gcr.io/k8s-staging-capi-docker to generate the docker provider manifest using the above registry/Image.

Run the local-overrides hack and save the clusterctl init command provided in the command output to be used later.

Edit the clusterctl config file located at ~/.cluster-api/clusterctl.yaml and configure the docker provider by adding the following lines (replace $HOME with your home path):

providers:
  - name: docker
    url: $HOME/.cluster-api/overrides/infrastructure-docker/latest/infrastructure-components.yaml
    type: InfrastructureProvider

If you are using Kind for creating the management cluster, you should:

  • run the following command to create a kind config file for allowing the Docker provider to access Docker on the host:
cat > kind-cluster-with-extramounts.yaml <<EOF
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
nodes:
  - role: control-plane
    extraMounts:
      - hostPath: /var/run/docker.sock
        containerPath: /var/run/docker.sock
EOF
  kind create cluster --config ./kind-cluster-with-extramounts.yaml --name clusterapi
  kubectl cluster-info --context kind-clusterapi
  • Run kind create cluster --config ./kind-cluster-with-extramounts.yaml to create the management cluster using the above file

  • Run kind load docker-image gcr.io/k8s-staging-capi-docker/capd-manager-amd64:dev to make the docker provider image available for the kubelet in the management cluster.

Run clusterctl init command provided as output of the local-overrides hack.

Connecting to a workload cluster on docker

The command for getting the kubeconfig file for connecting to a workload cluster is the following:

kubectl --namespace=default get secret/capi-quickstart-kubeconfig -o jsonpath={.data.value} \
  | base64 --decode \
  > ./capi-quickstart.kubeconfig

When using docker-for-mac MacOS, you will need to do a couple of additional steps to get the correct kubeconfig for a workload cluster created with the docker provider:

# Point the kubeconfig to the exposed port of the load balancer, rather than the inaccessible container IP.
sed -i -e "s/server:.*/server: https:\/\/$(docker port capi-quickstart-lb 6443/tcp | sed "s/0.0.0.0/127.0.0.1/")/g" ./capi-quickstart.kubeconfig

# Ignore the CA, because it is not signed for 127.0.0.1
sed -i -e "s/certificate-authority-data:.*/insecure-skip-tls-verify: true/g" ./capi-quickstart.kubeconfig

Known issues

A known issue affects Calico with the Docker provider v0.2.0. After you deploy Calico, apply this patch to work around the issue:

kubectl --kubeconfig=./capi-quickstart.kubeconfig \
  -n kube-system patch daemonset calico-node \
  --type=strategic --patch='
spec:
  template:
    spec:
      containers:
      - name: calico-node
        env:
        - name: FELIX_IGNORELOOSERPF
          value: "true"
'