Skip to content

Kafka and Clojure - Immutable event streams applicationsλ︎

Kafka is Kafka is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, extremely fast and runs in production in thousands of companies (Braintree).

Clojure is a dynamic, general-purpose programming language, combining the approachability and interactive development of a scripting language with an efficient and robust infrastructure for multithreaded programming. Clojure is a compiled language, yet remains completely dynamic – every feature supported by Clojure is supported at runtime. Clojure provides easy access to the Java frameworks, with optional type hints and type inference, to ensure that calls to Java can avoid reflection.

Why Kafka and Clojureλ︎

  • Easy interaction with and well-maintained libraries for other JVM-based systems (e.g. Kafka and ElasticSearch)
  • Concurrency and IPC capabilities
  • Clojure's sequence abstractions simplified the code and made testing easy

Kafka

Currently under development

This book is very early alpha stage and a long way from being complete

Tutorialsλ︎

Libraries and projectsλ︎

  • franzy - suite of Clojure libraries for Apache Kafka. It includes libraries for Kafka consumers, producers, partitioners, callbacks, serializers, and deserializers.
  • Kafka Streams Clojure - Clojure transducers interface to Kafka Streams. This combo provides the best of both worlds for building streaming applications on Kafka with Clojure
  • milena - This Kafka client library allows the user to exchange records while speaking clojure
  • fundingcircle/jackdaw - Clojure library for Apache Kafka

Presentations on Kafkaλ︎