Test Selectors

As a project grows in scope its important that unit tests are quick to run. Test that take a noticeable time to run diminish the motivation to run tests frequently.

Dividing tests into categories means selective tests can be run, continuing to provide fast feedback. Longer running tests or very large test suites can be run less often without loosing quality in the feedback from tests.

Test selectors can be included, so only that category of tests run. Or exclude test selectors so all tests except that category runs.

lambdaisland/kaocha focus and skipping

kaocha can be used to group tests into categories in the tests.edn configuration, making it easy to focus or exclude different types of tests (e.g. :unit and :spec)

Defining test catagories with metadata

Add metadata to deftest functions to provide categories of tests

(deftest ^:integration register-customer
  (is ,,,))

Example from Banking on Clojure

(deftest ^:database new-customer-test
  (testing "New customer generative testing")
  (is (spec/valid?
        :customer/id
        (:customer/id (SUT/new-customer
                        (spec-gen/generate (spec/gen :customer/unregistered)))))))

Running selective tests

Start a test running by specifying test selectors to include and/or exclude.

kaocha supports meta data on deftest expressions and has its own metadata tag for skipping tests, ^:koacha/skip

Examples of tests with and without test selectors

(deftest simple-test
  (is (= 1 1)))

(deftest ^:integration system-update-test
  (is (spec/valid? :system/update (long-running-function))))

(deftest ^:kaocha/skip under-development-test
  (is (= 3 21/7)))

Tests with test selector metadata can be skipped using a tests.edn configuration

#kaocha/v1
{:tests [{:kaocha.filter/skip-meta [:integration]}]}

Running kaocha will only run the simple-test, skipping the other two tests.

Specifying --skip-meta on the command line gives the same results

bin/kaocha --skip-meta :metadata-name

Running tests with the universal argument will prompt for test selector filters and only Run those tests that match the selector inclusions/exclusions.

SPC u , t a runs all tests, prompting for tests selector names to include (space separated)

Spacemacs Clojure Cider Test Runner test selector prompt - include

Then prompting for the test selectors to exclude. A warning displays if CIDER does not find the test selector name.

Spacemacs Clojure Cider Test Runner test selector prompt - include

Invoke the CIDER test runner commands with the universal argument and CIDER will prompt for test selector filters, running only those tests that match the selector inclusions/exclusions.

C-c C-t p runs all the tests in a project.

C-u C-c C-t p prompts for test selectors and runs the matching tests in a project.

C-c C-t l runs all tests currently evaluated in the REPL.

C-u C-c C-t l prompts for test selectors and runs the matching tests currently evaluated in the REPL.

CIDER first prompts for the test selectors to include:

Spacemacs Clojure Cider Test Runner test selector prompt - include

Then prompts for the test selectors to exclude. A warning displays if CIDER does not find the test selector name.

Spacemacs Clojure Cider Test Runner test selector prompt - include

The Cognitect Labs test runner uses command line options to specify test selectors, --include and --exclude.

practicalli/clojure-deps-edn configuration provides the :test/congnitect alias.

clojure -M:test/cognitect --include :database only runs tests with the ^:database test selector

clojure -M:test/cognitect --exclude :integration runs all tests except those with the ^:integration test selector

References

results matching ""

    No results matching ""