Claude Warren

Cross cultural teams bring a special set of pitfalls that are not present in mono cultural teams; from hidden misunderstandings to perceived hostility or indifference. This talk introduces the anthropological concept of cultural relativism, explores how it can be used to uncover potential conflicts that arise out of cultural differences, and presents culturally sensitive strategies for addressing them.

You will gain an understanding of the interplay between culture and perception, and will understand how to apply the prism of cultural relativism to detect, understand, and disarm cultural differences that lead to failure within cross cultural communities.

Claude Warren will present an overview of his recent work to implementing multidimensional Bloom filters. This talk will focus on the implementation of the multidimensional Bloom filters, their current use in modern software, and how they are applicable as a multi-column index, an index into large document collections, or an index to encrypted data. The talk will briefly describe Bloom filters and their construction before discussing strategies for the management of multiple Bloom filters; also known as multidimensional Bloom filters. Data from a comparative analysis of several multidimensional Bloom filter strategies will be presented along with discussion of when to select one implementation over another.

All source code referenced is under the Apache 2 or similar open source license and is available on Github.

In​ the analysis of b​ig data there are often problem queries that don’t scale because they require huge compute resources to generate exact results, or don’t parallelize well. Examples include c​ount-distinct, ​quantiles, most frequent items, joins, matrix computations, and graph analysis. Algorithms that can produce accuracy guaranteed approximate answers for these problem queries are a required toolkit for modern analysis systems that need to process massive amounts of data​ quickly. For interactive queries there may not be other viable alternatives, and in the case of real­-time streams, these specialized algorithms, called stochastic, s​treaming, sublinear algorithms,​ or 's​ketches',​ are the only known solution. This technology has helped Yahoo successfully reduce data processing times from days to hours or minutes on a number of its internal platforms and has enabled subsecond queries on real-time platforms that would have been infeasible without sketches. This talk provides a short introduction to sketching and to Apache DataSketches, an open source library of these algorithms designed for large production analysis systems.

Fast: Sketches are fast. The sketch algorithms in this library process data in a single pass and are suitable for both real-time and batch. Sketches enable streaming computation of set expression cardinalities, quantiles, frequency estimation and more. This allows simplification of system's architecture and fast queries of heretofore difficult computational tasks.

Big Data Platforms: This library has been specifically designed for big data platforms. Included are adaptors for Hadoop Pig, Hive, Spark, Druid, and Postgresql, which also can be used as examples for other systems, and many other capabilities typically required in big data analysis systems. For example, a Memory package for managing large off-heap memory data structures. Our sketch library is implemented in Java, C++ and Python and provides binary compatibility across languages and platforms. Some of our sketches provide off-Java-heap capability which dramatically improves performance in large systems. Our APIs provide a rich set of options to enable fine tuning performance parameters that are particularly important for large systems.

Analysis: Built-in Theta Sketch set operators (Union, Intersection, Difference) produce sketches as a result (and not just a number) enabling full set expressions of cardinality, such as ((A ∪ B) ∩ (C ∪ D)) \ (E ∪ F). This capability along with predictable and superior accuracy (compared with Include/Exclude approaches) enable unprecedented analysis capabilities for fast queries.

Bloom filters are a probabilistic data structure that tell us where things are not. They also utilize one way hash functions to build a probabilistic representation of an object. This talk will address how this structure can be used to provide an index into encrypted data that can be made publicly available with minimal risk.

Talk will cover how bloom filters are constructed, the Flat Bloofi indexing implementation and how to take the properties to be indexed and create Bloom filters, and then how to associate the bloom filter with the encrypted object in the index.

The result is an extremely fast index that can retrieve data items containing partial keys.

After this talk participants will be able to provide search capabilities across a collection of encrypted objects.

Code examples will be provided.

Contract testing provides a means for developers to provide tests to implementers of Java SPI and API frameworks to ensure that the expected functionality is correctly implemented. It is an open source junit extension that tests implementations of java interfaces. This talk will cover why and how to execute tests.

Contract testing provides a means for developers to provide tests to implementers of SPI and API frameworks to ensure that the expected functionality is implemented. Junit-contracts is a Junit runner and associated annotations that allows developers to develop test for correct implementation of an interface that are then reused by any implementation of the interface. The junit-contract tests suite does this automatically discovers the tests to be run and executes them. This presentation is an overview of the design principles behind the junit-contracts tool, how to develop tests that it can run, and how they can make development easier for project developers, QA teams and the system integrator while reducing support requests and problem tickets.

This presentation is intended for development teams that provide an SPI or user extensible API and who wish to provide tests that ensure the correct implementation of interfaces. This talk should also be of interest to QA testers that need to validate that a number of implementations of a single interface are correct and for systems integrators to wish to ensure that implementations of SPI/APIs in glue code function correctly.