Martin Junghanns

Graphs are used in many different applications as they are an intuitive way for representing complex relationships between entities, as for example in social, communication, financial or geographical networks. Graphs in these domains can be very large, potentially spanning multiple millions and even billions of nodes and edges. In order to get analytical insights out of these structures, scalable implementations of graph algorithms are necessary. Rust is the ideal language for implementing such algorithms, due to its well-known aspects, such as "fearless concurrency" and memory safety as well as its great out-of-the-box performance and its expressive type system.

In our presentation, we will talk about the "graph" project, a collection of open source crates that we are working on. The project includes an in-memory graph representation, APIs for building in-memory graphs from various data sources, and a small collection of high-performance graph algorithms. In addition to these building blocks, we started developing a Python wrapper called graph-mate for a NetworkX-like experience and an Apache Arrow endpoint for integrating the project in distributed applications. The presentation will include a project overview, a walk through the Rust API, and a demo for using the project via Python.

Abstract

Graph pattern matching is one of the most interesting and challenging operations in graph analytics. Query languages like openCypher, implemented in systems like Neo4j, SAP HANA Graph and Redis Graph, allow the intuitive definition of graph patterns including structural and semantic predicates.

For now, graph query languages are most prominent in graph database systems such as Neo4j. However, we think that many systems can benefit from having such a language in their toolbox. One of these systems is Apache Spark, which is one of the most popular open source frameworks in the context of distributed processing of large data volumes within complex analytical workloads. To bring the benefits of Cypher from the graph database realm into the world of Big Data, we at Neo4j started developing Cypher for Apache Spark (CAPS). CAPS is primarily focused on graph-powered data integration and graph analytical query workloads within the Spark ecosystem. In addition, CAPS is our testbed for Cypher language extensions as specified in the openCypher project; for example, multiple graphs, graph transformations and construction, and query composition.

In our talk, we want to motivate use-cases for CAPS and give an overview of new querying capabilities which we demonstrate using Apache Spark and Apache Zeppelin. Furthermore, we briefly present the internal architecture highlighting the main differences between Neo4j and CAPS.

Intended audience and goal of the talk

Developers and analysts, trying to express graph queries on distributed graphs

Links to background information on the given talk for the hungry and impatient

• OpenCypher
• Neo4j
• Apache Spark

Links to previous talks, code snippets or repositories

• CAPS

Abstract

Graph pattern matching is one of the most interesting and challenging operations in graph analytics. However, it is primarily supported by graph database systems such as Neo4j but, besides research prototypes, not generally available for distributed (not-only graph) processing frameworks like Apache Flink or Apache Spark.

In our talk, we want to give an overview of our current implementation of Cypher on Apache Flink. Cypher is the Neo4j graph query language and enables the intuitive definition of graph patterns including structural and semantic predicates. As the Neo4j graph data model is not supported out-of-the box by Apache Flink, we leverage Gradoop, a Flink-based graph analytics framework based on Apache Flink that already provides an abstraction of schema-free property graphs.

We will give a brief overview about the technologies used to implement Cypher, explain our query engine and give a demonstration of the available language features. Finally, we will discuss open challenges and missing features hopefully motivating people to contribute.

The project is a cooperation between the University of Leipzig and Neo Technology.

Intended audience and goal of the talk

Developers and analysts, trying to express graph queries on distributed graphs

Links to background information on the given talk for the hungry and impatient

• OpenCypher
• Neo4j
• Apache Flink
• Gradoop

Links to previous talks, code snippets or repositories

• GitHub
• Extended Property Graph Model
• FlinkForward 2015
• FOSDEM 2016

At the Leipzig University, we develop Gradoop [1], a framework for distributed, declarative graph analytics on top of Apache Flink [2]. Gradoop is designed around the so-called Extended Property Graph Model (EPGM) and supports semantically rich, schema-free graph data. In this model, a database consists of multiple property graphs, which we call logical graphs. These graphs are application specific subsets from shared vertex and edge sets. The EPGM provides operators for both single graphs as well as collections of graphs. Operators may also return single graphs or graph collections thus enabling the definition of analytical workflows in a declarative way.

In my talk, I would like to give an overview of Gradoop, the EPGM and its operators and show how Apache Flink helps us by presenting a subset of our operator implementations. Furthermore, I will sketch the usefulness of Gradoop by presenting an analytical use case from the business intelligence domain.

Gradoop is open-source and licenced under GPLv3. The Gradoop source code and a short documentation can be found on GitHub [3], a more detailed explanation of the data model and our operators can be found in a technical report [4].

[1] http://www.gradoop.com [2] http://flink.apache.org/ [3] https://github.com/dbs-leipzig/gradoop [4] http://arxiv.org/pdf/1506.00548.pdf

Intended audience and goals of the talk

The targeted audience are analysts and/or developers that already have some knowledge about graph systems, e.g. semantically expressive graph database systems like Neo4j or generic graph processing systems like Apache Giraph. The goal is to present our concept of combining those two approaches to allow powerful graph analytics in a distributed way.

Previous talks about Gradoop

Flink Forward, October 2015: http://flink-forward.org/?session=gradoop-scalable-graph-analytics-with-apache-flink