Kafka and Clojure - Immutable event streams applicationsλ︎
Kafka is Kafka is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, extremely fast and runs in production in thousands of companies (Braintree).
Clojure is a dynamic, general-purpose programming language, combining the approachability and interactive development of a scripting language with an efficient and robust infrastructure for multithreaded programming. Clojure is a compiled language, yet remains completely dynamic – every feature supported by Clojure is supported at runtime. Clojure provides easy access to the Java frameworks, with optional type hints and type inference, to ensure that calls to Java can avoid reflection.
Why Kafka and Clojureλ︎
- Easy interaction with and well-maintained libraries for other JVM-based systems (e.g. Kafka and ElasticSearch)
- Concurrency and IPC capabilities
- Clojure's sequence abstractions simplified the code and made testing easy
Currently under development
This book is very early alpha stage and a long way from being complete
Tutorialsλ︎
- Apache Kafka - quickstart
- Part 1: Apache Kafka for beginners - What is Apache Kafka - CloudKarafka
- Cloudurable Kafka Tutorial - current tutorial for Meet-a-mentor study group
- Apache Kafka Tutorial - TutorialsPoint
- Kafka in Clojure
- Writing a Kafka Producer and High Level Consumer in Clojure
- Hello World Kafka
Libraries and projectsλ︎
- franzy - suite of Clojure libraries for Apache Kafka. It includes libraries for Kafka consumers, producers, partitioners, callbacks, serializers, and deserializers.
- Kafka Streams Clojure - Clojure transducers interface to Kafka Streams. This combo provides the best of both worlds for building streaming applications on Kafka with Clojure
- milena - This Kafka client library allows the user to exchange records while speaking clojure
- fundingcircle/jackdaw - Clojure library for Apache Kafka
Presentations on Kafkaλ︎
- Introduction to Apache Kafka by James Ward
- Building a data pipeline with Clojure and Kafka
- Lisp in the Machine - Clojure at Braintree
- Ployconf 15 - The LISP in the Machine A Clojure experience report from Braintree / Joe Nash
- One Million Clicks per Minute with Kafka and Clojure - Devon Peticolas
- From REST to CQRS with Clojure, Kafka, & Datomic - Bobby Calderwood
- David Pick - Building a Data Pipeline with Clojure and Kafka
- Lambda Days 2018 - Andrea Crotti - Tame Kafka streams with Clojure
- "Commander: Better Distributed Applications through CQRS and Event Sourcing" by Bobby Calderwood
- Streaming Data Platforms & Clojure with Derek Troy-West
- Clojure at 4,000 msg/s-What We Learned, Loved, and Loathed - Nathan Barnett
- Managing One of the World's Largest Clojure Code Bases - Donevan Dolby
- Reducing Microservice Complexity with Kafka and Reactive Streams - by Jim Riecken
- PolyConf 15: Distributed systems the easy way with Clojure and Mesos / Pierre-Yves Ritschard