The IoT data surge – how can operators manage the flood of new data streams from the IoT?

IoT Growth Statistics

The “Internet of Things” (IoT) has become one of the market’s hottest trends. And it’s not surprising with the amount of growth that has been predicted:

  • There will be nearly 26 billion IoT devices worldwide by 2020.
  • 75% of global business leaders are exploring the economic opportunities of IoT.
  • The UK government recently approved 45 million pounds in research funding for IoT technologies.
  • Cities are expected to spend $41 trillion in the next 20 years on infrastructure upgrades for IoT.

It is clear to see that the IoT represents a huge opportunity, with an incredibly broad range of applications. But with all of this growth, what does it mean for mobile network operators?


New data streams from the IoT

The internet of things covers a huge range of industries and scales, and it is expected to stimulate growth in multiple markets, such as traffic monitoring in a smart city, or structural sensing and industrial monitoring in factories and farms.

applications of the IoT data streams

The three key data types expected to play a larger role as the IoT becomes a reality are:

  • Sensor data
  • Machine to machine data
  • Network data

Many of the end nodes of the IoT would have been previously unconnected objects. Innovative technology is allowing us to install new, or enhance existing, wireless sensor nodes. These are able to provide small amounts of data on an intermittent or regular basis. Improved access to this more frequent, varied, and numerous sensor data will provide better insight into system and network status, improving decision making, and thereby enhance the day-to-day experiences and resources of an operator.

However, these data types present significant challenges related to data volume and variety, as well as predictive modelling. Therefore operators need to take a proactive approach to managing this surge of complex data.


How to manage IoT data

With billions of devices already connected, and even more on the way, organizations should prepare for a real time data flood that cannot be ignored. The number of sensors and volumes of gathered data can range wildly depending on the application. Data will also come into the network at varying frequencies, and in multiple formats, adding to its complexity.

To find value in these new data streams, operators need a network management tool with which to access the new data sets in real-time, and quickly and easily analyse and act upon the new insights.

Scalability – the application must be easily scalable, with proven performance handling big data.

Heterogeneous data sources – The framework must be built to handle various heterogeneous data sources, blending data from the different domains and communication layers (sensor and machine data with data from data warehouses, enterprise applications, social media, and more)

Multiple data types – Semi-structured and unstructured data sets must be supported, from any data source, with any kind of translation and enrichment. The IoT data must also provide cohesion with an E2E analysis.

Automation – Collected data may need to be filtered, parsed, or analysed before being delivered to its intended source. Before the data is stored, there needs to be some automation to facilitate this and make the data manageable and relevant.


Visualising IoT data

Of course, this is of little use without a way of discovering meaningful patterns in equipment and device data. Operators require advanced analytics which can present data in compelling visualizations, interactive reporting (guided analysis), and customised dashboards.

Once a pattern is recognized, metrics can be embedded into the data stream to drive automatic adjustments in connected systems, or initiate alerts for immediate actions and better decisions.

With these advanced analytics techniques in place, operators can move beyond monitoring existing conditions, troubleshooting and evaluating thresholds, to forecasting and understanding complex scenarios and network behaviour. This will be a key differentiator in the IoT world and Sysmech is prepared to be a part of it, with Zen Service Assurance Software.