Unit tests run very quickly. They don’t require any additional services to execute. They focus on individual pieces of logic and allow integration bugs to slip through. They are fast and great for getting the initial implementation worked out.
envtest is a testing environment that is provided by
kubebuilder. This environment spins up a local instance of
etcd and the kube-apiserver. This allows reconcilers to do all the things they need to do to function very similarly
to how they would in a real environment. These tests provide integration testing between reconcilers and kubernetes.
test target through
make will run all of the unit and
Integration tests use a real cluster and real dependencies to run tests. The dependencies are managed manually and are not meant to be run locally. This is used during CI to ensure basic functionality works as expected.
scripts/ci-integration.sh for more details.
The end-to-end tests are similar to the integration tests except they are designed to manage dependencies for you and have a more complete test using a docker provider to test the Cluster API mechanisms. The end-to-end tests can run locally without modifying the host machine or on a CI platform.
The test run can be controlled to some degree by environment variables.
SKIP_RESOURCE_CLEANUP: Will prevent the end-to-end tests from cleaning up the local resources they create. This is useful when debugging an error that requires cluster inspection. After the error occurs and the tests fail, the cluster remains. You can then
docker execinto a running container to get information to assist in debugging. This parameter is also useful if you simply want a local cluster to experiment with. After the tests run and succeed you are left with a cluster in a known working state for experimentation.
FOCUSvariable allows running of certain tests. The default on CI is
FOCUS='Basic'. In order to run all the tests, set
FOCUS='Basic|Full'. This will run the basic test and then run all other tests.
make test-capd-e2e-full will build all manifests and provider images then use those manifests and images in the test
run. This is great if you’re modifying API types in any of the providers.
make test-capd-e2e-images will only build the images for all providers then use those images during the e2e tests and
use whatever manifests exist on disk. This is good if you’re only updating controller code.
make test-capd-e2e will only build the docker provider image and use whatever provider images already exist on disk.
This is good if you’re working on the test framework itself. You’ll likely want to build the images at least once
and then use this for a faster test run.
make test-capd-e2e-full SKIP_RESOURCE_CLEANUP=true
make test-capd-e2e FOCUS='Basic|Full'