Ingest & Search JSON events in Real Time (III): Flume Architecture & BenchMarking

Manuel Lamelas Big Data Architecture Leave a Comment

To end this series of articles in Ingestion & Searching we are going to see the Flume Architecture for High Availability and see some benchmark tests. Flume Architecture To achieve high availability we have two flume characteristics to play with: 1. File Channel vs Memory Channel This is a decision on 100% delivery vs fast ingestion. With file channel the …

Ingest & Search JSON events in Real Time (I): Collecting Data

Manuel Lamelas Big Data Architecture, Technology 2 Comments

Ingesting & searching data in real time is one of the most important needs companies have nowadays. If we are talking IoT or we want to collect data from a huge quantity of sources, this is where this post may come in handy. We searched a simple way of achieving the ingestion & storage of JSON events. The idea behind this …