On behalf of RTR-GmbH, A-SIT Plus has carried out the study “Security Risks for Trust Service Providers from AI-based Threats”. It provides a comprehensive analysis of how the use and misuse of artificial intelligence affect the security of qualified trust services, and highlights the technical, organisational and procedural measures that can be taken to address these risks.
The study makes it clear that AI is fundamentally changing the security landscape of digital trust services. On the one hand, it opens up new opportunities for increasing efficiency, supporting operational processes and detecting anomalies. On the other hand, it creates new avenues for attack and exacerbates existing threats, such as through deepfakes, synthetic identities, manipulated documents or AI-assisted social engineering attacks.
The study focuses on a phase-based approach covering the entire lifecycle of a trust service – from implementation through authorisation and operational use to the termination of the service. This allows AI-based threats to be precisely assigned to the respective process steps and suitable countermeasures to be systematically derived.
Across all phases, the study highlights a wide range of risks: during implementation, these range from tampered hardware and compromised supply chains to vulnerabilities introduced by AI-powered development tools; during ongoing operations, the focus is on deepfakes, phishing, compromised biometrics, manipulated signature data and attacks on surveillance and machine learning systems; and even in the decommissioning phase, disinformation campaigns, fake communications or manipulated handover data can significantly compromise the security and reliability of trust services.
The study identifies key countermeasures including the use of certified hardware, cryptographic protection mechanisms, security-by-design and zero-trust principles, as well as clear governance rules for the use of AI. In addition, training in the detection of AI-based deception, standardised verification and approval processes, tamper-proof logging and the dual-control principle in security-critical processes are recommended.
Secure identity verification and the authenticity of digital interactions are of particular importance. The study emphasises that combining various verification mechanisms – such as register queries, chip data reading and biometric liveness checks – can significantly reduce vulnerability to deepfakes and identity fraud. Equally central remain the principle of “What You See Is What You Sign”, as well as the use of secure hardware components and phishing-proof authentication methods.
Furthermore, the study emphasises that AI must be understood not only as a risk but also as a tool for defence. It can thus contribute to the detection of anomalies, the analysis of attack patterns and the support of quality assurance. However, this requires clear governance, the use of validated models, as well as traceable documentation and ongoing review of the systems in use.
The study was published at the same time as the event. The slides are available on the event website.
The costs incurred by Rundfunk und Telekom Regulierungs-GmbH (RTR-GmbH) in producing the study “Security risks for trust service providers posed by AI-based threats” are announced as €16,755.00 (net) in accordance with Article 20(5) of the Federal Constitutional Act.