AI Project Audit: Zero Tamper-Evident LLM Evidence Found

The article presents a concerning finding: a complete absence of tamper-evident audit trails in a sample of 30 popular AI projects. This lack of verifiable evidence raises significant questions about the security and accountability of these systems. The article introduces Assay, a tool designed to address this issue by providing cryptographically signed receipts that can be independently verified. Assay focuses on proving internal consistency and completeness relative to scanned call sites, but it acknowledges its limitations in preventing fabrication on a fully compromised machine. The tool offers functionalities for scanning LLM call sites, patching code for integration, running and building signed evidence packs, and verifying the integrity and claims within those packs. The distinction between integrity (tamper detection) and claims (governance checks) is a key feature, allowing for the detection of honest failures, which are considered valuable for auditing purposes. The article highlights the importance of automatic logging for high-risk AI systems, referencing the EU AI Act Article 12. The absence of such logging mechanisms in current AI projects underscores the need for greater attention to security and transparency.

Transparency is paramount in AI systems. This analysis is based solely on the provided article. No external information was used. The AI model (Gemini 2.5 Flash) was used to generate this content. The prompt focused on extracting factual information and providing balanced perspectives. The AI was instructed to avoid hallucinations and adhere to a strict JSON format. The goal was to provide an objective and concise summary of the article's key points.

This analysis is intended for informational purposes only and does not constitute professional advice. Readers should consult with experts before making decisions based on this information. The AI model is continuously being improved, and its output may vary over time. The user is responsible for evaluating the accuracy and completeness of the information provided.