Data Logistics with Apache Nifi

Manuel Lamelas BigData Architecture, Technology Leave a Comment

As announced in a previous post we’re now going to introduce you to Apache Nifi, the latest trend in ingestion tools. A new project from the Apache Software Foundation that allows you to manage data flows with a cool graphical interface. If we didn’t catch your attention yet, wait until you hear this: NSA created it!!! Nifi – the UPS of data …

Kerberos & Hadoop: Securing Big Data (part I)

Celeste Duran BigData Architecture, Technology Leave a Comment

When I began to use Hadoop with Kerberos I felt as I was in the middle of the ocean. I found a lot of information about Kerberos technology but it was very difficult for me to find something about how to use it on Hadoop, why to use it and how to configure it for working with Hadoop. This trilogy of posts is going to …

Morphlines – Hadoop ETL by Cloudera

Manuel Lamelas BigData Architecture Leave a Comment

Today we are going to talk about Morphlines,  an open source framework developed by Cloudera, that provides a new way to do ETL on Hadoop. What are these morphlines? Morphlines are simple configurations files that defines how to transform data on the fly. It consists on a file that describes the steps a data flow has to pass in order to …

NoSQL vs Relational: Which database to use

Iván Alejandro Marugán BigData Architecture Leave a Comment

Nowadays information collection has changed a lot. Everybody wants to save more data and allow our users to consume that information in real time and in an easy way. This means that performance, scalability and availability are three key factors for database implementations. For this reason NoSQL databases have made their appearance. What’s a NoSQL database? A NoSQL database (“non SQL”, …

Random Forest – Modeling The Titanic Voyage with R

David Carrasco Data Science Leave a Comment

What’s a Random Forest? Random Forest is a machine learning algorithm used normally for classification and regression tasks in supervised learning which consists on an ensemble or group of simple decision tree models to predict the value of a target variable based on a bunch of input variables. The main advantage regarding a simple decision trees is that reduces the …

Random Forest – The magic behind the algorithm

David Carrasco Data Science Leave a Comment

Random Forest is an ensemble model based on decision trees which is built through the Bagging technique used for classification and regression tasks in supervised learning (Although, it can be used in unsupervised learning too). Yes, it’s a bit more technical than the definition in the previous post, isn’t it? Well, let’s explain the main concept behind the Random Forest …

Ingest & Search JSON events in Real Time (III): Flume Architecture & BenchMarking

Manuel Lamelas BigData Architecture Leave a Comment

To end this series of articles in Ingestion & Searching we are going to see the Flume Architecture for High Availability and see some benchmark tests. Flume Architecture To achieve high availability we have two flume characteristics to play with: 1. File Channel vs Memory Channel This is a decision on 100% delivery vs fast ingestion. With file channel the …

Meetup Big Data Madrid: 30 de marzo a las 19:00

Datatons BigData Architecture, Sin categoría Leave a Comment

El concepto Big Data está adquiriendo una gran importancia en los últimos tiempos y esta tendencia va en aumento. Los datos son muy valiosos, pero mucho más si sabemos cómo extraer la información importante, la que realmente necesitamos. Si aún no lo tienes muy claro, te aclaramos las ideas en nuestro Meetup Big Data Madrid ¿Quieres venir? Pues, ¡apunta en …

H2O.ai y Data Science Madrid: como agua de mayo

Datatons Data Science Leave a Comment

Parece que la primavera quiere aparecer: días más largos, temperaturas más suaves durante el día… todo empieza a florecer y nosotros no podíamos ser menos. Nos quitamos la bufanda y los guantes y nos lanzamos de lleno a la primavera. Así que, para celebrar el resurgir de la vida, ¿qué mejor manera que traeros el agua de mayo de los …

Data Science Madrid Meetup Enero 2017

Datatons BigData Architecture, Business, Data Science, Technology Leave a Comment

El pasado 21 de de Enero tuvo lugar el meetup mensual de Data Science Madrid, en esta ocasión además pudimos colaborar con nuestros colegas de Madrid 4 OpenStack para celebrar en el evento en el Auditorio de Medialab Prado. La agenda de las charlas fué la siguiente: • “Open Stack y Big Data” por Daniel Mellado y Carlos Camacho Introducción …