REPL Driven Development

Always be REPL'ing

Coding without a REPL feels limiting. The REPL provides fast feedback from code as its crafted, testing assumptions and design choices every step of the journey to a solution - John Stevenson, Practical.li

Clojure is a powerful, fun and highly productive language for developing applications and services. The clear language design is supported by a powerful development environment known as the REPL (read, evaluate, print, loop). The REPL gives you instant feedback on what your code does and enables you to test either a single expression or run the whole application (including tests).

REPL driven development is the foundation of working with Clojure effectively

An effective Clojure workflow begins by running a REPL process. Clojure expressions are written and evaluated immediately to provide instant feedback. The REPL feedback helps test the assumptions that are driving the design choices.

• Read - code is read by the Clojure reader, passing any macros to the macro reader which converts those macros into Clojure code.
• Evaluate - code is compiled into the host language (e.g. Java bytecode) and executed
• Print - results of the code are displayed, either in the REPL or as part of the application.
• Loop - the REPL is a continuous process that evaluates code, either a single expression or the whole application.

Design decisions and valuable data from REPL experiments can be codified as specifications and unit tests

Practicalli REPL Reloaded Workflow

The principles of REPL driven development are implemented in practice using the Practicalli REPL Reloaded Workflow and supporting tooling. This workflow uses Portal to inspect all evaluation results and log events, hot-load libraries into the running REPL process and reloads namespaces to support major refactor changes.

Evaluating source code

A REPL connected editor is the primary tool for evaluating Clojure code from source code files, displaying the results inline.

Source code is automatically evaluated in its respective namespace, removing the need to change namespaces in the REPL with (in-ns) or use fully qualified names to call functions.

Evaluate Clojure in a Terminal UI REPL

Entering expressions at the REPL prompt evaluates the expression immediately, returning the result directly underneath

Rich Comment blocks - living documentation

The (comment ,,,) function wraps code that is only run directly by the developer using a Clojure aware editor.

Expressions in rich comment blocks can represent how to use the functions that make up the namespace API. For example, starting/restarting the system, updating the database, etc. Expressions provide examples of calling functions with typical arguments and make a project more accessible and easier to work with.

Clojure Rich Comment to manage a service

(ns practicalli.gameboard.service)

(defn app-server-start [port] ,,,)
(defn app-server-start [] ,,,)
(defn app-server-restart [] ,,,)

(defn -main
  "Start the service using system components"
  [& options] ,,,)

(comment
  (-main)
  (app-server-start 8888)
  (app-server-stop)
  (app-server-restart 8888)

  (System/getenv "PORT")
  (def environment (System/getenv))
  (def system-properties (System/getProperties))
  ) ; End of rich comment block

Rich comment blocks are very useful for rapidly iterating over different design decisions by including the same function but with different implementations. Hide clj-kondo linter warnings for redefined vars (def, defn) when using this approach.

;; Rich comment block with redefined vars ignored
#_{:clj-kondo/ignore [:redefined-var]}
(comment
  (defn value-added-tax []
    ;; algorithm design - first idea)

  (defn value-added-tax []
    ;; algorithm design - second idea)

  ) ;; End of rich comment block

The "Rich" in the name is an honourary mention to Rich Hickey, the author and benevolent dictator of Clojure design.

Design Journal

A journal of design decisions makes the code easier to understand and maintain. Code examples of design decisions and alternative design discussions are captured, reducing the time spent revisiting those discussions.

Journals simplify the developer on-boarding processes as the journey through design decisions are already documented.

A Design Journal is usually created in a separate namespace, although it may start as a rich comment at the bottom of a namespace.

A journal should cover the following aspects

• Relevant expressions use to test assumptions about design options.
• Examples of design choices not taken and discussions why (saves repeating the same design discussions)
• Expressions that can be evaluated to explain how a function or parts of a function work

The design journal can be used to create meaningful documentation for the project very easily and should prevent time spent on repeating the same conversations.

Example design journal

Design journal for TicTacToe game using Reagent, ClojureScript and Scalable Vector Graphics

Viewing data structures

Pretty print shows the structure of results from function calls in a human-friendly form, making it easier for a developer to parse and more likely to notice incorrect results.

Tools to view and navigate code

• Cider inspector is an effective way to navigate nested data and page through large data sets.
• Portal Inspector to visualise many kinds of data in many different forms.

Code Style and idiomatic Clojure

Clojure aware editors should automatically apply formatting that follows the Clojure Style guide.

Live linting with clj-kondo suggests common idioms and highlights a wide range of syntax errors as code is written, minimizing bugs and therefore speeding up the development process.

Clojure LSP is build on top of clj-kondo

Clojure LSP uses clj-kondo static analysis to provide a standard set of development tools (format, refactor, auto-complete, syntax highlighting, syntax & idiom warnings, code navigation, etc).

Clojure LSP can be used with any Clojure aware editor that provides an LSP client, e.g. Spacemacs, Doom Emacs, Neovim, VSCode.

Clojure Style Guide

The Clojure Style guide provides examples of common formatting approaches, although the development team should decide which of these to adopt. Emacs clojure-mode will automatically format code and so will Clojure LSP (via cljfmt). These tools are configurable and should be tailored to the teams standard.

Data and Function specifications

Clojure spec is used to define a contract on incoming and outgoing data, to ensure it is of the correct form.

As data structures are identified in REPL experiments, create data specification to validate the keys and value types of that data.

;; ---------------------------------------------------
;; Address specifications
(spec/def ::house-number string?)
(spec/def ::street string?)
(spec/def ::postal-code string?)
(spec/def ::city string?)
(spec/def ::country string?)
(spec/def ::additional string?)

(spec/def ::address   ; Composite data specification
  (spec/keys
   :req-un [::street ::postal-code ::city ::country]
   :opt-un [::house-number ::additional]))
;; ---------------------------------------------------

As the public API is designed, specifications for each functions arguments are added to validate the correct data is used when calling those functions.

Generative testing provides a far greater scope of test values used incorporated into unit tests. Data uses clojure.spec to randomly generate data for testing on each test run.

Test Driven Development and REPL Driven Development

Test Driven Development (TDD) and REPL Driven Development (RDD) complement each other as they both encourage incremental changes and continuous feedback.

Test Driven Development fits well with Hammock Time, as good design comes from deep thought

• RDD enables rapid design experiments so different approaches can easily and quickly be evaluated .
• TDD focuses the results of the REPL experiments into design decisions, codified as unit tests. These tests guide the correctness of specific implementations and provide critical feedback when changes break that design.

Unit tests should support the public API of each namespace in a project to help prevent regressions in the code. Its far more efficient in terms of thinking time to define unit tests as the design starts to stabilize than as an after thought.

clojure.test library is part of the Clojure standard library that provides a simple way to start writing unit tests.

Clojure spec can also be used for generative testing, providing far greater scope in values used when running unit tests. Specifications can be defined for values and functions.

Clojure has a number of test runners available. Kaocha is a test runner that will run unit tests and function specification checks.

Automate local test runner

Use kaocha test runner in watch mode to run tests and specification check automatically (when changes are saved)

clojure -X:test/watch

Continuous Integration and Deployment

Add a continuous integration service to run tests and builds code on every shared commit. Spin up testable review deployments when commits pushed to a pull request branch, before pushing commits to the main deployment branch, creating an effective pipeline to gain further feedback.

• CircleCI provides a simple to use service that supports Clojure projects.
• GitHub Workflows and GitHub actions marketplace to quickly build a tailored continuous integration service, e.g. Setup Clojure GitHub Action.
• GitLab CI

Live Coding with Data - Stuart Halloway

There are few novel features of programming languages, but each combination has different properties. The combination of dynamic, hosted, functional and extended Lisp in Clojure gives developers the tools for making effective programs. The ways in which Clojure's unique combination of features can yield a highly effective development process.

Over more than a decade we have developed an effective approach to writing code in Clojure whose power comes from composing many of its key features. As different as Clojure programs are from e.g. Java programs, so to can and should be the development experience. You are not in Kansas anymore!

This talk presents a demonstration of the leverage you can get when writing programs in Clojure, with examples, based on my experiences as a core developer of Clojure and Datomic.