Apache Storm

Distributed real-time computation system for processing unbounded streams of data. Does for real-time what Hadoop did for batch, offering fault-tolerant, scalable processing with guaranteed delivery. Simple to set up and operate, language-agnostic, and integrates with the queueing and database technologies teams already use.

Trusted by enterprises such as Twitter, Spotify, Yahoo, Alibaba, Groupon, and Flipboard for production workloads. Storm pioneered real-time stream processing and remains a solid choice for low-latency tuple-at-a-time processing. Compared to Apache Flink or Kafka Streams, Storm offers a lower-level topology model with straightforward deployment and integration into existing infrastructure.

Key features:

• Topology-based stream processing with spouts and bolts, repartitioning streams between computation stages
• Fault-tolerant design that guarantees data will be processed
• Integrates with Kafka, RabbitMQ, Cassandra, HBase, and other messaging and storage systems
• Multi-language support via Thrift, including Java, Python, and Clojure
• Distributed RPC for low-latency query patterns

Common applications include real-time analytics and personalization, online machine learning, continuous computation, ETL pipelines, and log processing. Developers use Storm for content feeds, search indexing, security monitoring, ad auction analysis, and IoT event streams. Topologies consume data from queues, transform it through custom logic, and write results to databases or downstream services.