Tugdual Grall

Batch processing has been, until recently, the standard model for big data. Largely, this is due to the very large influence of the original processing MapReduce implementation in Hadoop and the difficulty in replacing MapReduce in the original Hadoop framework.

Batch processing has been, until recently, the standard model for big data. Largely, this is due to the very large influence of the original processing MapReduce implementation in Hadoop and the difficulty in replacing MapReduce in the original Hadoop framework.

More recently, there has been a shift to streaming architectures using tools such as Apache Spark and Kafka. These architectures offer surprisingly large benefits in terms of simplicity and robustness, but they are also surprisingly different from previous message queuing designs. The changes in these new systems allow enormously higher scalability and make fault tolerance relatively simple to achieve while maintaining good latency.

In this talk I will describe the key design tools and best practice techniques used in modern systems including percolators, the big-data oscilloscope, replayable queues, state-point queuing and universal micro-architectures. The benefits that I will highlight include

• a decrease in total system complexity

• flexible throughput/latency trade-offs

• fault tolerance without the difficulties of the lamdba architecture and

• easy debuggability

I will detail several Apache projects that attempt to support flow computing.

In this session you will learn more about Couchbase Server capabilities and the benefits of a distributed JSON document database.

Couchbase Server is an Apache 2.0 licensed open source project that provides easy scalability, consistent high-performance and always on capabilities for interactive applications. In this session you will learn more about Couchbase Server capabilities and the benefits of a distributed JSON document database. It will include demonstration of applications interacting with Couchbase from different programming languages.

You will also learn about Couchbase developer community, that not only develop Couchbase server but also the different client SDKs to ease the creation of scalable applications in Java, Ruby, PHP, Javascript/Node, Python, and even Go and C#.