LinkedIn
Messaging
21 min
LinkedIn Needed a Message Queue. They Built the One the Entire Internet Runs On.
In 2010, LinkedIn was drowning in data it couldn't move. Every ML model, every recommendation engine, every real-time feature was starving because there was no reliable way to get activity data from the website into the systems that needed it. Jay Kreps, Jun Rao, and Neha Narkhede spent a year building a fix. They named it after Franz Kafka. The rest of the internet adopted it.
1B events/day at launch (2011)
1T messages/day by 2015
7T messages/day by 2019
+1
80%+ of Fortune 100 run it today