Understanding Write-Ahead Logs: Durability Beyond the Flush
Discover how modern databases use Write-Ahead Logs to protect your data during failures.
Series
Building Blocks of Modern Data Systems
Posts in this series
From Modulo to Consistent Hashing: Optimizing Distributed Storage
High-Performance Hashing Solutions for Scalable Storage Systems
Bloom Filter: Definitely No, Probably Yes
In large-scale distributed systems, knowing what you don’t have is often more valuable than knowing what you do. Let’s understand with a practical example. Imagine you are building a recommendation engine for a blogging site like Hashnode or Medium, ...
When Bloom Filters Fail: False Positives, Memory Trade-offs, Production Lessons
In my previous article, Bloom Filter: Definitely No, Probably Yes, we saw that a Bloom filter acts like a ‘magic’ toolbox to perform quick operations on large datasets to determine whether a value is certainly not in the set. However, this 'definitel...
High-Watermark in Distributed Systems: A Deep Dive with Apache Kafka
Let me start with a simple question. When a producer sends a message to Kafka, the leader broker writes it down to its local disk and sends the acknowledgment back. Is this considered done and availab