Extracting Value from Next General Digital Infrastructure (Big Data, IoT, Edge, 5G, HPC, Cloud, AI)
The future competitive position of Europe will be based in the capability of regions, public administrations, research organisations, and Large and Small Industry (and in particular Start-ups) to extract data insight from next-generation digital infrastructure. In 2018 we are at the beginning of a great wave of enabling technologies from IoT, 5G, HPC, Edge Computing to Big Data and AI across all domains. The focus beyond 2020 will need to be on the knowledge and fusion technologies necessary to extract valid and accurate insight that can be used to make useful and meaningful decisions for business and society. With many of these decisions taken in near to real time.
IoT represents an excellent opportunity to leverage big-data analytics in a wide range of domains, including smart cities, smart manufacturing, etc. IoT enables to enrich data-sets as the number and variety of data sources increase significantly. IoT challenges current big-data solutions in which data analytics is performed in a centralized computing solution, typically located on the cloud:
On the one side, IoT pressures communication/computation resources by sending tons of data to be processed on by a centralized point. This approach is not scalable.
On the other side, there is a need to implement real-time data-in-motion stream analytics to extract valuable knowledge. This cannot be provided with centralized solutions.
There is therefore the need to develop decentralized solutions in which the computation of data analytics is distributed across multiple computing elements (from the edge to the cloud) and to efficiently coordinate edge/cloud computing resources to provide a holistic view of the system.
Current parallel processor architectures enables to significantly increase the computation capabilities on the edge side, reducing the pressure on the cloud. This solution will enable the development of combined/coordinated data-in-motion/data-at-rest analytics to face systems with data sources geographically distributed.
On the other hand big-data developers cannot deal with the complexity of programming in such an heterogeneous environment. The technology must be unified so programmers only focus on the description of the analytics, leaving to the underlying software architecture the distribution of the computation across the compute continuum.
The Big Data Value Strategic Research and Innovation Agenda (BDV SRIA) already identified some of these challenges and as a result there are some ongoing projects tackling these challenges, e.g the CLASS project that develops a novel software architecture for distributed computing environments and applying to some Smart City use cases.
The H2020 LEIT Work Prorgramme offers upcoming opportunities to further develop solutions to these challenges through large scale pilots deployed in different applications domains.
There is a need of cooperation in between ongoing research and innovation projects from different communities (Big Data and IoT) to jointly tackle this challenge towards the future.
The session will explore how the above-described challenges are being tackled in the different ongoing Big Data and IoT initiatives and in different sectors, and what cooperation actions are needed in between the IoT and Big Data Community to join efforts towards future scalable solutions.