Reading tracker active
0 / 8 read
0%
← Home
Architecture Notes

Technical Writing

Notes on distributed systems, AI infrastructure, databases, and observability. Written for engineers who want depth, not surface-level takes.

SEVEN DATABASES · SIX STORAGE ENGINES · ONE COST EACHAPP DATAMongoDBtrades joinsRELATIONALPostgreSQLtrades scale-outSEARCHElasticsearchtrades freshnessCACHERedistrades durabilityMETRICSPrometheustrades detailANALYTICSClickHousetrades updatesWRITESCassandratrades readsENGINEB-Treesorted · balancedpays on writeENGINEInverted Idxword → docspays on mergeENGINEHash Slots16,384 · in-mempays on durabilityENGINETime Seriesscrape · compresspays on cardinalityENGINEColumn StoreMergeTree · sparsepays on updatesENGINELSM-Treememtable → SSTablepays on read+ DynamoDBEVERY ENGINE PAYS SOMEWHERE — AT WRITE TIME, AT READ TIME, OR IN HOW BYTES SIT ON DISKTHE THREE COSTSwrite · read · layoutTHE ACTUAL QUESTIONwhat is it refusing to be good atFAILURE MODEShot keys · cardinalityaryan.devdatabases · distributed-systems
NEW
databasesdistributed-systems

Why Every Backend Engineer Eventually Ends Up Learning 7 Different Databases

Every database is choosing whether it pays its cost during writes, during reads, or in how bytes sit on disk. Once I started reading them that way, picking one stopped being a comparison and started being a question about which cost I could absorb.

Jul 30, 2026·8 min read
KAFKA · APPEND-ONLY LOG, TWO CLOCKSPRODUCERappendWRITES ONLY EVER LAND AT THE TAILPARTITION 0 · SEGMENT FILES ON DISKSEGMENTS0off 0+SEGMENTS1off 100+SEGMENTS2off 200+committedSEGMENTS3off 300+SEGMENTS4off 400+SEGMENTS5off 500+SEGMENTS6off 600+SEGMENTS7off 700+log endCONSUMERreadcommits manuallyHEARTBEAT CLOCKsession.timeout.ms · 45sPROCESSING CLOCKmax.poll.interval.ms · 5mA CONSUMER CAN HEARTBEAT PERFECTLY AND STILL BE EVICTED FOR SLOW PROCESSINGaryan.devdistributed-systems · databases
distributed-systemsdatabases

You Don't Need Kafka Internals. You Need Four of Them.

Every Kafka guide hands you the same nine-row config table. In practice almost every failure I've debugged came down to four things: the log's shape, two timeouts people conflate, the rebalance tax, and the fact that auto-commit is a correctness bug with a config flag.

May 22, 2026·5 min read