Designing a detection engine that costs $0.0004 per event

Detection engines look simple from the product side: events come in, alerts come out. The hard part is doing it for $0.0004 per event when your customers are sending 30 million events a day.

Setting a per-event budget

Before any code, we set a budget: under half a millicent per event, end-to-end. If we couldn't hit that, the customer pricing model didn't work.

Architecture in three sentences

Events land in a Kafka topic, partitioned by tenant. A pool of stateless workers consumes the topic and writes to ClickHouse. Continuous queries run hourly against the same ClickHouse cluster, materialising into a per-tenant evidence table.