Ziheng Wang
Jul 23, 2022

--

This is a very interesting rebuttal as I ponder replacing Kafka with just an S3 bucket in my application. In particular I am running into a problem where I need high throughput batch inserts (at network limit, not Kafka's limit, alternatively I can just append large objects but Kafka doesn't work well with those) and fast indexed lookups into the Kafka topic, both of which run into problems with Kafka. Just uploading those large objects into a S3 bucket and notifying consumers solves the first problem but not the second problem. I wonder if there are cloud-hosted strongly RAW consistent databases that support inserts at network IO limits?

--

--