1c. Media and Entertainment

Scope and Introduction

Over the past decade, the European media sector has undergone a major transformation, marked by the breakdown of traditional boundaries between content formats, markets, and distribution channels. Accelerated by the Covid-19 crisis, this shift is driven by five interconnected trends: the rise of new digital technologies, the emergence of digital-native creators and platforms, changing consumption habits, evolving monetisation models, and shifting societal expectations of media's role.

A second wave of digital disruption is now unfolding, led by generative AI, Large Language Models (LLMs), Virtual Worlds (VW), immersive media, and virtual production technologies. These advances are reshaping how content is created, distributed, and experienced, blurring the lines between entertainment, gaming, shopping, and social interaction in what is increasingly referred to as "liquid media."

However, this innovation surge occurs within a global media landscape dominated by U.S.-based platforms that control streaming, video sharing, and social networks. Despite efforts by European broadcasters to adapt, structural challenges—such as fragmented infrastructure, limited access to investment, and a still-incomplete Digital Single Market—hinder Europe's ability to compete at scale. European start-ups, while numerous, struggle to grow in a capital environment that remains less supportive than in the U.S. or China.

These changes present both existential threats and powerful opportunities. Europe's media ecosystem is under pressure from declining revenues, eroding public trust, and the loss of influence among digital-native audiences. To regain strategic autonomy and remain relevant, the EU must invest boldly in innovation, cross-sector collaboration, and European-owned infrastructures.

VW, in this context, offer a transformative opportunity—both as a creative frontier and as a testbed for inclusive, rights-based digital futures. The European Commission views VW as a cornerstone for economic dynamism and digital autonomy but insists they must be shaped by European values: inclusion, accessibility, cultural diversity, and public interest.

Use Cases

This section outlines key research areas within the media and entertainment domain, detailing their significance, current challenges, and proposed objectives for future innovation.

Media Production

1c.1 Artificial Intelligence-driven virtual production and rendering

AI-driven tools and real-time rendering engines are set to revolutionise media production by automating and streamlining content creation. These technologies enable instant generation of 3D environments from natural language descriptions, provide intelligent assistance for tasks like colour grading and scene layout, and offer creative suggestions. This approach aims to reduce reliance on expensive hardware, shorten production timelines, and accelerate feedback cycles, ultimately democratising cinematic-quality virtual scene production for smaller teams and independent creators.

Challenges and opportunities: Current production processes are often characterised by physical set building, extensive post-production for visual effects, and fragmented, manually coordinated workflows. Existing AI applications are typically narrow and not integrated across the full production pipeline. Key challenges include long iteration cycles, high costs associated with skilled labour and hardware, a lack of automation in critical areas like colour grading and lighting, complexity in coordinating remote or hybrid teams, limited real-time feedback for creative decisions, and issues with AI explainability and interoperability.

Research and Innovation Objectives: AI-Driven Virtual Production and Rendering integrates foundational technologies from across 2a–2f, leveraging advanced visualisation and real-time rendering systems (2a) to produce high-fidelity virtual scenes. It draws on interaction models (2b) for intuitive content control using voice, gesture, and predictive inputs, while authoring tools (2c) support collaborative, AI-assisted scene creation and real-time co-design workflows. Interoperability standards (2d) are critical to ensure compatibility across distributed creative pipelines and platforms. Digital assets and virtual environments (2e) form the backbone of scalable scene elements and enable adaptive reusability in diverse production settings. Applied AI (2f) plays a central role, powering generative content creation, lighting and scene layout suggestions, version control, and embedded explainability to preserve creative intent. On the socio-technical side (3), key intersections include transparency, explainability, and human oversight (3e)—essential for maintaining trust in AI-assisted creative processes—and innovation, IP, and creativity (3c), which are central to defining rights and ownership in AI-generated media.

1c.2 Volumetric media and avatar presence

The ability to capture and reconstruct people volumetrically in real-time allows for their realistic representation as avatars in immersive environments. This facilitates rich human presence, interaction, and emotional expressiveness across entertainment, sports, training, and social XR applications. Furthermore, real-time volumetric capture enables the seamless integration of remote individuals into live virtual productions or broadcasts, supporting interactive interviews, performances, and collaborative segments.

Challenges and opportunities: Currently, volumetric capture is largely confined to specialised studios, requiring expensive and complex multi-camera systems. Most avatars rely on hand-modelled rigs or low-fidelity motion capture. Challenges include high costs, limited scalability and portability, significant hardware requirements, restricted emotional realism, reliance on marker-based facial tracking, and technical barriers for small teams or live applications. While interest is growing in various sectors, widespread adoption is hindered by these technical and cost barriers.

Research and Innovation Objectives: Volumetric Media and Avatar Presence spans all core technology areas (2a–2f), integrating advanced visualisation and sensing (2a) for lifelike avatars and real-time volumetric capture. It leverages real-time interaction models (2b) for social presence and embodiment, and authoring tools (2c) for designing, animating, and customizing avatars and experiences. Standardisation (2d) ensures interoperability of avatars and volumetric formats across platforms, while digital humans' frameworks (2e) enable identity, behaviour, and emotional realism. Applied AI (2f) supports avatar animation, emotion recognition, data compression, and ensures ethical use and explainability. In addition to these technical layers, the use case intersects with key Section 3 socio-technical areas: privacy and data governance (3b) are critical in managing captured biometric and behavioural data; identity, autonomy, and agency (3a) relate to how users control their digital representations; and inclusion and accessibility (3a) are essential to ensure that avatar-based interactions are usable and representative for diverse populations.

1c.3 Camera tracking and smart Extended Reality stages

Precise and automated tracking of physical cameras and actors within modular XR stages ensures perfect synchronisation between physical and digital environments. This allows for seamless blending of real and virtual scenes, crucial for immersive storytelling and live production. Leveraging visual analysis methods also enhances the overall coherence between physical displays and virtual environments, addressing alignment, geometry, and visual consistency, and harmonising lighting and colour for believable integration.

Challenges and opportunities: Current camera tracking often involves manual configuration and custom solutions. XR stage setups typically rely on proprietary integrations of displays, sensors, and game engines, often requiring recalibration for each production, which increases setup time and limits portability. Key challenges include a lack of standardisation, hardware fragmentation, tracking drift or latency, high calibration costs, reliance on specialist crews, and difficulty adapting to changing set configurations. While high-end productions utilise these technologies, broader adoption is constrained by their technical complexity and cost.

Research and Innovation Objectives: Camera Tracking and Smart XR Stages integrates key technologies across areas 2a–2f, combining advanced sensing and visualisation (2a) for precise real-time alignment of physical and digital environments, with interaction models (2b) to support seamless spatial coherence during live production. Authoring tools (2c) facilitate efficient setup and calibration of XR stages, while interoperability standards (2d) ensure compatibility across diverse hardware systems. Digital Twins and spatial models (2e) contribute by enabling dynamic, responsive representations of physical stage elements, allowing for real-time adaptation and accurate synchronisation between physical and virtual components. Applied AI (2f) enhances camera tracking, lighting consistency, and automated scene adjustments. From the socio-technical perspective in Section 3, key intersections include safety, security, and consumer protection (3b), which are essential for maintaining reliability and system integrity in high-stakes production environments, and inclusion and accessibility (3a), to ensure XR stages support a wide range of user needs and collaborative workflows. Domain specific research topics are the development of open XR tracking protocols to promote interoperability and the establishment of plug-and-play calibration standards for easier setup and reduced reliance on specialist crews.

Immersive Experiences

1c.4 Immersive interactive events

Immersive interactive events allow digital-native participants to experience real-time, shared entertainment in virtual environments, such as virtual concerts or film festivals. This enables social engagement, interaction with content, and even co-creation of performances, moving beyond passive viewing towards active participation and connection. These events are persistent, globally accessible, and layered with live social interactions, fostering meaningful fan-to-fan and fan-to-creator connections.

Challenges and opportunities: Current practices largely involve passive livestreaming or in-game events with limited interactivity. Existing VR social platforms host niche community events. Key challenges include predominantly passive audience experiences, limited scalability and social richness, poor integration of user-generated content, fragmented tools for cross-platform access and creator monetisation, and high technical barriers combined with a lack of standards for real-time shared XR events. While experimentation is growing, particularly among younger generations, widespread use is limited by technical constraints, platform fragmentation, and XR device adoption.

Research and Innovation Objectives: Standardisation on infrastructure, system and runtime interoperability (2d). Sensing devices for audience input (2a). Real-time co-creation frameworks, need for real-time user interaction (2b) and interactive experience design tools (2c). The need for new (AI-assisted) storytelling models (2f) and novel human-AI interaction paradigms (2f).

1c.5 Immersive educational and health applications

Immersive educational and health applications enable students and mental health service users to engage with content in interactive formats, promoting more effective learning and emotionally resonant support. These experiences are designed to be informative, enjoyable, and motivating, leveraging immersive environments and game-based interactions to deliver personalised and effective services.

Challenges and opportunities: Current educational games are often two-dimensional with limited interactivity, and virtual classrooms rely on basic video conferencing. Mental health tools typically include basic telehealth services and chatbot therapy, lacking deep immersion or personalisation. While some simulation-based medical training exists in VR or AR contexts, it often lacks gamification and adaptive feedback. Challenges include low emotional engagement, limited personalisation, a lack of embodiment and social presence in remote services, generic one-size-fits-all designs, difficulty sustaining motivation, insufficient integration of biometric or behavioural data, and ethical and accessibility concerns within XR settings.

Research and Innovation Objectives: The requirement on accessibility in games and game mechanics (3a) and the technological realisation of this (2f). The need for personalisation frameworks and adaptation to user contexts (2c).

1c.6 Innovative and adaptive game design

Game developers and creative studios seek to design and deploy games that dynamically adapt to diverse user contexts, including cultural, linguistic, physical, and technological variations. This is achieved through modular tools, AI-driven personalisation, and accessible motion capture. This approach aims to broaden audience reach, facilitate experimentation with inclusive mechanics, and reduce production costs while maintaining creative flexibility and quality.

Challenges and opportunities: Most games are currently designed with a limited number of languages and cultural contexts, with localisation often added as a post-development step. Integrating advanced personalisation and accessibility features is costly and complex. Motion capture remains resource-intensive, requiring specialised studios or significant budgets. Co-located multiplayer experiences are rare and technically fragmented across platforms. Challenges include a lack of integrated frameworks for dynamic cultural or linguistic adaptation, limited availability of low-cost motion capture tools for independent developers and educators, fragmentation in tools supporting accessibility, multiplayer, and content reuse, and barriers to creative experimentation due to steep learning curves in XR and AI integration.

Research and Innovation Objectives: The mitigation of cultural bias and working on diversity in content and representation (3a.1). The requirement on accessibility in games and game mechanics (3a.1, 3a.2). The need for personalisation frameworks and adaptation to user contexts (2c.9). The need for standards and codes for transparent and ethical XR design (3e.4).

1c.7 Secure, scalable and strategic media ecosystems

Ensuring that digital media production and distribution are secure, interoperable, and aligned with European values is crucial for protecting creators' rights, enabling cross-platform innovation, and building resilient infrastructures. This supports long-term cultural and economic sustainability by adopting open standards and decentralised infrastructures within VW. Blockchain-based content authentication and licensing ensure transparency and traceability, while AI governance frameworks align algorithms with ethical and legal norms. Interoperable metadata schemas enhance portability and discoverability across platforms, and decentralised identifiers and verifiable credentials protect identity and consent.

Challenges and opportunities: Current content rights and metadata are often managed in proprietary, siloed systems, with licensing and access handled manually. Platform ecosystems frequently lack transparency and portability. Key challenges include rights fragmentation, data silos, a lack of cross-border interoperability, piracy, platform dependency, weak monetisation channels, and the absence of comprehensive AI governance frameworks and ethical safeguards. While some media platforms are exploring blockchain-based licensing or interoperable metadata, widespread and standardised adoption remains limited.

Research and Innovation Objectives: Secure, Scalable, and Strategic Media Ecosystems draws on all key technology areas 2a–2f to establish resilient and ethically grounded infrastructure for media production and distribution. It integrates immersive content systems and device-level safeguards (2a) to support trusted media environments and applies interaction models (2b) to maintain secure, user-centric experiences across platforms. Authoring and distribution frameworks (2c) enable creators to manage content securely and flexibly, while interoperability standards (2d) are essential for cross-platform metadata, rights management, and secure licensing mechanisms. Digital assets and infrastructure models (2e) support persistent identity, verifiable credentials, and scalable content portability. Applied AI (2f) enhances metadata generation, content moderation, and algorithmic governance, while embedding ethics-by-design principles to ensure accountability and fairness. From the socio-technical perspective (Section 3), this use case intersects most strongly with privacy, data protection, and data governance (3b) (ensuring user data and digital rights are protected across distributed infrastructures) and democracy and fundamental rights (3a), which underpin the development of media ecosystems that respect freedom of expression, transparency, and European digital autonomy.

Recommendations

To cultivate a robust, innovative, and ethically sound media and entertainment ecosystem within VW, a strategic framework encompassing technological advancement, infrastructure development, and regulatory alignment is essential.

AI-Driven Production: A concerted effort is needed to advance AI-driven virtual production and rendering. This involves developing sophisticated tools for voice- and gesture-based scene control, predictive scene layout, and lighting design, which can significantly reduce production timelines and democratise access to cinematic-quality virtual content. Integration of embedded explainability into AI models is crucial to preserve creative intent and foster trust among creators.

Volumetric Technologies: Significant investment should be directed towards volumetric media and avatar presence technologies. This includes developing efficient compression algorithms for volumetric data to enable seamless streaming and establishing clear privacy and ethical guidelines for the use of avatars and captured volumetric information. The creation of scalable real-time pipelines for volumetric capture and rendering will enhance human presence and emotional expressiveness in immersive environments.

XR Stage Standards: The development of open standards and plug-and-play calibration for camera tracking and smart XR stages is paramount. This will facilitate seamless blending of real and virtual scenes, crucial for immersive storytelling and live production, by ensuring perfect synchronisation between physical and digital environments. Integrating dynamic Digital Twin technology for XR stages will further enhance real-time synchronisation and adaptability.

Interactive Events: Fostering immersive interactive events requires the development of adaptive, multi-user storytelling models that respond to audience input and behaviour. Establishing interoperable standards for XR events and environments will facilitate cross-platform access, while creating real-time audience engagement and co-creation frameworks will empower participants to influence performances, moving beyond passive viewing to active participation.

Educational Applications: In the realm of immersive educational and health applications, a focus on personalisation frameworks and the investigation of game mechanics' impact on mental health and cognitive development is crucial. Establishing XR design principles tailored for neurodiverse and vulnerable populations will ensure these applications are effective, informative, and emotionally resonant for all users.

Adaptive Game Design: For innovative and adaptive game design, efforts should concentrate on mitigating cultural narrative bias to ensure broader appeal and inclusivity. Developing adaptive game mechanics that cater to diverse physical and cognitive abilities and establishing ethical frameworks for AI personalisation in interactive media will promote responsible and inclusive game development.

Strategic Ecosystems: To build secure, scalable, and strategic media ecosystems, it is vital to develop ethics-by-design AI approaches, research the sustainability of distributed ledgers, and align regulatory frameworks for interoperable standards. This will ensure transparency, traceability, and protection of creators' rights, fostering long-term cultural and economic sustainability within VW.