3e Trust and human oversight

Foundations for transparency, explainability and human oversight

The emergence of immersive Virtual Worlds (VW) presents unprecedented opportunities but also heightens concerns regarding transparency, explainability, and human oversight. These environments integrate AI-driven algorithms, pervasive data collection, and complex socio-technical systems that profoundly shape user experiences. Algorithmic opacity and a lack of accountability in immersive spaces can erode user trust, undermine privacy and autonomy, and potentially threaten democratic values. Transparency and responsibility are recognised as dominant principles for addressing XR’s ethical challenges, and international bodies emphasise transparency and human control as prerequisites for protecting human rights in AI-infused virtual environments. This chapter examines key challenges of algorithmic opacity, content moderation governance, and data practices in VW, linking each to specific research gaps and innovation requirements, with an emphasis on societal and governance dimensions alongside technical advances in explainability.

Research topics for transparency, explainability and human oversight

3e.1 Design for human oversight of Extended Reality

VW platforms increasingly rely on algorithms and AI agents to mediate user experiences, from content recommendations to non-playable character behaviours and adaptive learning scenarios. However, these algorithmic operations often function as "black boxes," making their decisions uninterpretable and unpredictable. In immersive settings, AI-driven agents or recommendation engines can subtly shape users' perceptions and actions without their awareness, leading to an asymmetry of information that can be exploited. This opacity undermines user trust and complicates accountability, as it becomes unclear who or what is responsible for automated decisions. Ensuring human oversight is critical for safeguarding fundamental rights and promoting user empowerment in these environments.

Problem Definition and Research Gap: A significant problem is the "black box" nature of algorithmic operations and AI agents in VW, making their decisions opaque to users and even developers. This lack of transparency leads to trust erosion and accountability issues, as users are rarely informed about automated tasks or algorithmic decision-making processes. There is a notable absence of research on Explainable AI (XAI) techniques specifically tailored to immersive XR contexts, as traditional explainability tools do not translate well to 3D/AR interfaces. Open questions remain on how to convey algorithmic workings in real-time within an embodied experience without disrupting immersion, and few studies examine user comprehension of AI explanations in VR. From a policy perspective, regulatory mandates for transparency, such as the EU AI Act, lack defined enforcement mechanisms and standards for "appropriate transparency" in XR. There is insufficient research on audit frameworks for complex, proprietary XR algorithms, and a need for multi-disciplinary research to link technical transparency solutions with ethical design, avoiding information overload and aligning explanations with human values.

Research and Innovation Objectives:

• Create new techniques for explainable AI in immersive environments, enabling VW systems (recommender engines, AI agents, moderation bots) to provide clear, contextually relevant explanations to users in real-time.
• Research visualisation and interaction methods for conveying algorithmic logic within VR/AR settings, as well as methods to change these settings.

3e.2 Establish auditing frameworks

Establishing robust auditing frameworks for VW is essential for increasing transparency and accountability in their operations. When algorithmic operations and content moderation are opaque, user trust is eroded, and it becomes difficult to hold platforms accountable for potentially biased or arbitrary decisions. Implementing transparent logging mechanisms and independent audit tools allows regulators and third parties to inspect and verify platform behaviour. This provides a necessary mechanism for scrutiny, ensuring that VW meet explainability and risk management criteria, thereby fostering a culture of algorithmic accountability within the XR industry.

Problem Definition and Research Gap: A significant gap exists in establishing standardised frameworks for auditing XR platforms. There is a lack of defined processes for independent auditors or oversight bodies to technically inspect proprietary XR algorithms for bias or risks, given their complexity. Current regulations, such as the EU AI Act, may mandate certain transparency, but enforcement mechanisms and standards for "appropriate transparency" in XR are still undefined. Additionally, while the EU plans centres for algorithmic transparency for very large online platforms, extending such oversight to VW platforms requires further definition and implementation. This absence of clear, verifiable audit procedures hinders accountability and limits the ability of external bodies to ensure algorithms meet necessary explainability and risk management criteria.

Research and Innovation Objectives:

• Design and pilot standardised frameworks for auditing XR platforms, including transparent logging mechanisms, independent audit tools, and procedures for regulators or third parties to verify platform behaviour.
• Make VW operations inspectable and verifiable, laying the groundwork for accountability.

3e.3 Implement AI content labelling and moderation explainability

Developing robust systems to label AI-generated or manipulated content and ensuring explainable content moderation decisions are critical for maintaining user trust and protecting fundamental rights in VW. The immersive, real-time nature of these environments amplifies challenges around content moderation, where harassment, hate speech, and abusive behaviours can take novel forms demanding swift intervention. However, current internal moderation processes often lack transparency, leading to user distrust and perceptions of bias or arbitrariness. By providing clear explanations and appeals processes for moderation decisions, and by effectively labelling AI-generated content, platforms can enhance legitimacy, uphold free expression, and ensure due process rights for users.

Problem Definition and Research Gap: A key problem is the lack of transparent and accountable content moderation processes in VW, where platforms often handle moderation internally with little disclosure to users or outsiders. Users frequently do not understand why decisions were made, leading to distrust and assumptions of bias. There are minimal avenues for appeal or external oversight, raising concerns about due process rights and the ability of regulators and civil society to scrutinise platform practices effectively. Furthermore, content moderation in XR is a nascent field with significant gaps in developing tools suited to 3D/VR environments, particularly for automated detection of visual or behavioural violations and the interpretation of complex spatial behaviour or voice chat. There is also a gap in understanding user expectations and optimal ways to convey moderation processes. While regulations like the EU Digital Services Act (DSA) mandate transparency, their extension and enforcement mechanisms for immersive platforms require further development.

Research and Innovation Objectives:

• Develop robust systems to label AI-generated or manipulated content in VW (avatars, objects, media) so users can distinguish synthetic from real.
• Ensure content moderation decisions—whether by human or AI—are accompanied by meaningful explanations and appeals processes in XR, building on DSA’s transparency requirements.
• Bring existing and new mechanisms for combating disinformation to XR environments.

3e.4 Develop standards and codes for transparent and ethical design

Collaborating across industry, academia, and civil society to produce European standards or codes of conduct for transparent and ethical XR system design is crucial. Such standards define best practices for transparency, explainability, and user oversight, ensuring that VW are developed with human values and rights at their core. These guidelines are essential for avoiding dark patterns, requiring algorithmic fairness assessments, and ensuring human-in-the-loop control for high-risk applications, thereby bolstering user trust and accountability. This collaborative approach helps define norms in a rapidly evolving technological landscape, informing future regulation and voluntary certification schemes.

Problem Definition and Research Gap: A significant gap exists in establishing widely adopted standards and codes of conduct specifically tailored to transparency and ethical design in XR systems. While there are emerging efforts, there is no comprehensive set of European-level guidelines that define best practices for explainability and user oversight. This includes a lack of clear guidance on how to avoid dark patterns, establish requirements for algorithmic fairness assessments in VW, and implement provisions for human-in-the-loop control for high-risk applications effectively. The absence of such unified standards means that transparency and ethical considerations are often implemented inconsistently or insufficiently across different XR platforms, leaving users vulnerable and impeding the development of a trusted ecosystem.

Research and Innovation Objectives:

• Collaborate across industry, academia, and civil society to produce European standards or codes of conduct that define best practices for transparency, explainability, and user oversight in XR system design.
• Include guidelines on avoiding dark patterns, requirements for algorithmic fairness assessments in VW, and provisions for human-in-the-loop control for high-risk applications.
• Inform both future regulation and voluntary certification schemes for XR platforms.

3e.5 Monitor and mitigate algorithmic harms

Continuously monitoring the societal impact of VW algorithms and developing intervention strategies to mitigate harms is crucial for safeguarding fundamental rights and societal trust. This approach ensures that transparency and oversight are living commitments, allowing for the collection of evidence and the iterative, evidence-based adaptation of governance. By identifying early warning indicators for issues like algorithmic bias, addictive design, or harmful manipulation in VR, this objective enables proactive measures to ensure VW flourish as environments for creativity, connection, and commerce, grounded in respect for digital rights and EU laws and values.

Problem Definition and Research Gap: A significant research gap exists in the continuous monitoring of the societal impact of VW algorithms, particularly their effects on mental health, equality, or democratic discourse. There is a need to develop robust early warning indicators for emerging issues such as algorithmic bias, addictive design patterns, or harmful manipulation within VR environments. Furthermore, research is required to innovate effective intervention strategies, such as configurable user settings or regulatory “circuit breakers,” that can mitigate these identified harms. This signifies a challenge in establishing an iterative, evidence-based approach to governance that can adapt as new algorithmic harms manifest in immersive technologies.

Research and Innovation Objectives:

• Launch research initiatives to continuously monitor the societal impact of VW algorithms, studying effects on mental health, equality, or democratic discourse.
• Develop early warning indicators for issues like algorithmic bias, addictive design, or harmful manipulation in VR.
• Innovate intervention strategies to mitigate these harms, such as configurable user settings or regulatory “circuit breakers” for certain features.
• Treat transparency and oversight as living commitments by collecting evidence and adapting governance in an iterative, evidence-based manner.