IV.a Context, policy and trends - BDV Big Data Value Association

The EuroHPC Joint Undertaking (JU) was established in 2018 with the mission of “developing, deploying, extending and maintaining in the EU a world-leading federated, secure and hyper-connected supercomputing, quantum computing, service and data infrastructure ecosystem”^[1]. The JU is comprised of public members (EU and member states) and private members (European Technology Platform for High-Performance Computing, BDVA, and European Quantum Industry Consortium). The development of EuroHPC objectives materialises through its Multi-Annual Strategic Programme and the corresponding multi-annual financial framework, which, for the period 2021-2027, has planned a contribution from the EU of 3 B€. So far, EuroHPC has funded 46 R&I projects^[2], many of which are working on the convergence between HPC, big data, and AI ^[3] ^[4] ^[5] ^[6].

In relation to this, on the 24^th January 2024, the EC launched the “AI Innovation Package”[7] to support European startups and SMEs in developing trustworthy AI that respects EU values and rules. This package includes an amendment of the EuroHPC Regulation to set up AI Factories, that implies (i) acquire, upgrade and operate AI-dedicated supercomputers to enable fast ML and training of large General Purpose AI (GPAI) models, (ii) access to the AI dedicated supercomputers, widening of the use of AI to a large number of public and private users, offering a one-stop shop for startups and innovators, and (iii) enable the development of emerging AI applications based on General Purpose AI models.

Linked to the EuroHPC JU, the European Quantum Industry Consortium (QuIC^[8]), founded in 2021, is dedicated to boosting the European quantum-technology industry competitiveness and economic growth and fostering value creation. Additionally, the Quantum Flagship^[9] is a large-scale initiative created in 2018 and funded by the EC with an investment of 1 B€ for 10 years. It has 4 application areas (computing, simulation, communication, and sensing & metrology), with 24 funded projects until 2022 working on these areas^[10].

Bringing together data, edge and cloud, the “European Alliance for Industrial Data, Edge and Cloud” was created in 2020. By considering that “cloud and edge technologies are strategic innovation enablers for the uptake of emerging technologies, such as AI, the Internet of Things, and 5G”, the alliance brings together businesses, member states representatives and relevant experts to discuss how to strengthen the position of EU industry on cloud and edge technologies^[11]. The Alliance has published the cloud-edge strategic industrial roadmap[12] to identify investment areas for the joint development and deployment of the next generation of European cloud and edge technologies. Additionally, the Alliance is working on different thematic roadmaps to address specific aspects.

Additionally, as part of its “Connectivity Package”[13] published in 2024, the EC aims to push towards the connectivity goals of the Digital Decade. As part of these efforts, the EC published the white paper “How to master Europe’s digital infrastructure needs?”[14], where the term “Connected, collaborative, computing networks (3C) is coined: “a future ecosystem that spans over the entire computing continuum, from semiconductors and radio technologies to connectivity infrastructure, data management, and applications.”

Bringing different technology expertise together, the TransContinuum Initiative (TCI)[15], hosted under the activities of the European Technology Platform for High-Performance Computing (ETP4HPC)[16], clustered 6 associations to address complex multi-technologies challenges and produce a set of white paper, e.g., on the Real-Time DTs and Extremes weather predictions. TCI in 2024 has been extended to include 15 associations into the recently created “Computing Continuum”[17], where those associations work together to pursue alignment, identify synergies, and potential collaborations.

Finally, the European Cloud, Edge and IoT Continuum Initiative[18] aims to realise a pathway for the understanding and development of the Cloud, Edge and IoT (CEI) Continuum by promoting cooperation between a wide range of research projects, developers and suppliers, business users and potential adopters.

About specific projects addressing the development of data pipelines and AI applications in the computing continuum, it is worth mentioning the EC H2020 ICT-40 call^[19] “Cloud Computing: towards a smart cloud computing continuum”, and more specifically in some of the projects funded therein ^[20] ^[21] ^[22] ^[23]. More recently, as part of Horizon Europe Programme, the EC has funded 5 more projects under the topic “Cognitive Computing Continuum: Intelligence and automation for more efficient data processing” (started in January 2024) ^[24] ^[25] ^[26] ^[27] ^[28]. Finally, some projects working under the topic Efficient trustworthy AI – making the best of data[29] are covering this specifically for the computing continuum[30] or addressing data and AI pipelines[31].

Additionally, the R&I projects under the EU Cloud, Edge and IoT Continuum initiative[32] include cognitive cloud, swarm computing, cognitive computing continuum, open source, and SW technologies. The CSA projects[33] bring all the ecosystem together and define roadmaps to achieve their objectives in this line. Finally, the Spectrum Project[34] aims to deliver a Strategic Research, Innovation, and Deployment Agenda (SRIDA) and a Technical Blueprint for a European computing and data continuum.

Among all the previously mentioned actions, the following high-level trends that will drive the activities in the following years have been identified:

Proliferation of IoT devices. In 2023, there were an estimated 41.8 billion active IoT-connected devices worldwide. IDC estimates there will be 55.7 billion connected IoT devices by 2025, generating almost 80B zettabytes (ZB) of data. Gartner predicted more than 64 B IoT devices in 2020. There are more conservative predictions, but in general, the proliferation of IoT devices is a fact.
Increased integration of edge computing with IoT implies that data collected from IoT devices are increasingly processed locally (either on the device or on an edge node) instead of being sent to a cloud server for processing.
Adoption of AI at the edge / embedded ML (TinyML). Considering the two previous trends, the potential of edge AI (integration of AI in smart devices) has been unlocked, enabling those devices to learn from existing data and use this knowledge to make assessments, predictions, and decisions.
Improved capabilities of wireless connectivity. Widespread development of 5G networks, faster and more reliable connectivity for IoT devices, ultra-low latency and ultra-high bandwidth communications (e.g., Innovative Optical and Wireless Network[35]) and high data throughput can be leveraged to improve the efficiency of IoT networks and their connection with other infrastructures, fostering a more responsive and interconnected ecosystem.
Integration of AI and Data with HPC. More and more applications from data (data-intensive workflows, simulation and modelling, optimisations, prediction, …) and AI (parallel computing, training of large models) will likely benefit from the power of the HPC infrastructure.
Related to the previous point, the recently launched AI Innovation package identifies AI Factories as ecosystems to connect AI practitioners (to train, tune, and run large AI models) to AI-dedicated HPC supercomputers, also guaranteeing access to the huge amount of data required for these models.
Quantum computing. Some aspects of quantum computing (like focusing on error correction, quantity to quality, specific applications, …) are paving the way for this technology to consolidate in the following years.

New paradigms for computing: neuromorphic computation/chips. Neuromorphic computation is suited to AI on the edge and is the way to advance HPC capabilities due to its potential to do large-scale calculations with high energy efficiency. It is also an example of computing applicable to heterogeneous architectures.
AI is applied to manage complex problems, like orchestration, allocation, load balance distribution, … that affect the effective management, seamless integration, and use of technologies and infrastructures.

[1] https://eurohpc-ju.europa.eu/about/discover-eurohpc-ju_en

[2] https://eurohpc-ju.europa.eu/research-innovation/our-projects_en

[3] https://www.acrossproject.eu/

[4] https://deep-projects.eu/project/deep-sea/

[5] https://eflows4hpc.eu/

[6] https://www.hidalgo2.eu/

[7] https://ec.europa.eu/commission/presscorner/detail/en/ip_24_383

[8] https://www.euroquic.org/

[9] https://qt.eu/

[10] https://qt.eu/projects/

[11] https://digital-strategy.ec.europa.eu/en/policies/cloud-alliance

[12] https://ec.europa.eu/newsroom/dae/redirection/document/102590