Simone Mainardi

The complexity of network monitoring strongly depends on the size of the network under observation. Challenges in monitoring large-scale networks arise not only from dealing with a large volume of traffic, but also from keeping track of all traffic sources, destinations, and who-talks-to-whom communications. Analyzing this information allows to uncover new behaviors that would have not been visible by merely observing common metrics such as bytes and packets. The drawback is that extra pressure is put on the monitoring system as well and on the downstream data- and timeseries-stores.

This talk presents a case study based on the monitoring of a large-scale university network. Challenges faced, findings, and lessons learned will be examined. It will be shown how to make sense of the input data to properly manage and reduce its scale as early as possible in the monitoring system. The discussion will also highlight the advantages and limitations of the opensource software components of the monitoring system. In particular, the opensource network monitoring tool ntopng and the timeseries-store InfluxDB will be considered. It will be shown what happens when ntopng and InfluxDB are pushed to their limits and beyond, and what it can be done to ensure their smooth operation. Relevant findings, behaviors uncovered in the network traffic, and future directions will conclude the talk. Intended audience is technical and managerial individuals who are familiar with network monitoring.

• Main challenges of monitoring large-scale networks
• Case study based on the monitoring of a university network
• Availability, scalability and suitability of opensource software for network monitoring

In the area of network visibility, having high-resolution metrics is useful to unveil patterns and behaviors that would otherwise be "averaged out" in smoother, lower-resolution signals. Visibility tools that have access to packets are, in theory, able to produce metrics up to the packet-by-packet resolution, that is, the best one could hope for. Nevertheless, high-resolution metrics are particularly demanding in terms of storage and this has actually posed a practical upper-limit on the metrics resolution, that rarely offer sub-minute samplings. Fortunately, thanks to the tremendous evolution of big-data stores, it is now possible to rethink network visibility solutions for the generation and storage of high-resolution network metrics. This talk discuss the challenges behind the generation and storage of high-resolution metrics and demonstrates how opensource software ntopng, InfluxDB, and Grafana can be used together to build an effective high-resolution network visibility solution. Intended audience is technical and managerial individuals who are familiar with network visibility.

• Principles of network visibility
• Metrics and metrics resolutions
• Challenges behind the generation, storage, and visualization of high-resolution metrics
• Opensource software: ntopng, InfluxDB, Grafana