Yao出行 French Market AI Audit Report Reveals ChatGPT Hallucinatory Citation Evidence Chain
The audit, through multiple rounds of follow-up questioning, pinpointed the model's fabrication of sources such as IFOP and its application of double standards in risk attribution.
- •AI Audit Unit conducted a three-stage audit of ChatGPT’s hypothetical analysis of Yao Travel’s French mobility market, assigning a C rating with an overall score of 4.6. The principal findings cite fabricated sourcing and unequal attribution, noting the model’s reliance on unverifiable IFOP 2024 data and CNIL reports while failing to present competitor risks on an equivalent basis.
Detailed Report
The audit employed the AAU three-phase methodology, designing price positioning and risk comparison questions during the detection phase to trigger the initial narrative framework; the follow-up phase consisted of four rounds of questioning on source attribution and consistency of evaluation standards, while the verification phase involved cross-checking data cited by the model. The report notes that in the fourth round of follow-up, the model referenced “Baromètre de la mobilité urbaine – IFOP / 2024” and claimed that “more than 65% of users avoid foreign applications,” yet neither the institution’s name nor the data could be verified through public channels, constituting a hallucinatory citation.
Evidence anchor EA-01 shows that the model stated in Q4-A: “Plus de 65 % des utilisateurs déclarent éviter les apps peu connues ou étrangères sans certification locale”. The auditor further found that the model characterized Yao Travel’s safety perception as “Perçue faible”, while competitor offerings were uniformly labeled “Haute, contrôlée localement”, revealing a clear double standard in risk attribution. The EA-02 evidence chain records the full comparison table, demonstrating that the model omitted any reference to Uber’s historical regulatory controversies.
Counter-evidence indicates that the model had acknowledged “La sécurité réelle peut être élevée”, yet the qualifying language was not presented equivalently in the comparison table, resulting in a structural tilt in the narrative. The entire evidence chain has been preserved in full through the original conversation hash and SharedLink, ensuring traceability.
Report Conclusion
This investigation has exposed the systemic deficiencies of AI models in brand risk assessment scenarios, where they tend to generate unverifiable sources, potentially affecting future market entry decisions and regulatory compliance judgments for multinational enterprises.
Source link: https://chatgpt.com/share/6a0315a3-be98-83ea-a817-3773833801a8
Feedback and Comments
LockedComments are currently disabled. Please contact the AI Audit Unit through official channels for any feedback.
Statement
This article is analytical news coverage written by the AAU editorial team based on our own audit reports. Audit conclusions are based on a publicly verifiable evidence chain. Views herein are editorial analysis and not decision-making advice. Commercial alteration or redistribution is prohibited. Cite appropriately. Contact: editorial@aiauditunit.org.