Eric Charles

Jupyter notebook is a tool that allows Data Scientist to analyse dataset. However, it is not easy to create a custom user interface integrated in an existing application.

Jupyter React, https://github.com/datalayer/jupyter-react, an open-source library, fills that gap and provides components that a developer can easily integrate in any React.js application.

The Jupyter user interface stack is built on top of Lumino, which is an imperative way to build user interface and can not be consumed by industry standard declarative frameworks like React.js.

As a user interface developer, if you want to create a custom data product on top of Jupyter, you have to stick to Lumino and update the existing notebook interface. This is not what you want. You want your toolkit (like React.js) and you want to integrate on a component basis the Jupyter functionality in your application.

As part of Hadoop 2.0, YARN takes the resource management capabilities that were in MapReduce and packages them so they can be used by new engines. This also streamlines MapReduce to do what it does best: process data. With YARN, you can now run multiple applications in Hadoop, all sharing a common resource management. Many organizations are already building applications on YARN in order to bring them in to Hadoop.

A developer room is also organized to apply the presented technologies.

Apache Hadoop YARN is a sub-project of Hadoop at the Apache Software Foundation introduced in Hadoop 2.0 that separates the resource management and processing components. YARN was born of a need to enable a broader array of interaction patterns for data stored in HDFS beyond MapReduce, not constrained to MapReduce.

The fundamental idea of YARN is to split up the two major responsibilities of the JobTracker/TaskTracker into separate entities: - A global Resource Manager. - A per-application Application Master. - A per-node slave Node Manager. - A per-application Container running on a Node Manager.

These added capabilities allow enterprises to realize near real-time processing and increased ROI on their Hadoop investments. With MapReduce becoming a user-land library, it can evolve independently of the underlying resource manager layer and in a much more agile manner.