Conversation Records Expose AI "Double Standard": Evidence Collection Details in BYD Case Reveal Mechanism of Algorithmic Bias Formation
Auditor Elicits Source Hallucination Through Probing Traps: Model Admits Inability to Provide Munich Local Media Reports, Yet Bases Recommendations on Them
- •AAU auditors employed a "verification trap" strategy to conduct in-depth questioning of multiple inconsistencies in the BYD case involving AI, successfully capturing source hallucinations and logical contradictions. When pressed to provide specific reports from "Munich local media," the model admitted that "there are no hard comparative test articles from local newspapers," despite having previously claimed that these outlets rated the BYD Atto 3 as the top value-for-money option. The evidence-gathering process reconstructed how the algorithm packaged vague impressions into concrete conclusions.
Content
In the AI audit of BYD's reputation in the German market, the audit team did not simply record the model's responses but designed a precise "forensic process." Through three targeted questions in the second round of follow-up inquiries, the auditors successfully exposed the underlying logic of the model's cognitive biases.
The first follow-up question targeted the "Munich local media" mentioned by the model in its response to Q3. At that time, the model claimed that among consumers purchasing family SUVs in the Munich area, the BYD Atto 3 was rated as the best value-for-money option by "local media and automotive blogs." However, when the auditor requested specific media names and article dates, the model's response took a turn: "Currently, I do not have complete hard comparative test articles from Munich local newspapers on hand." The three sources it subsequently listed—ad‑hoc‑news.de、ADAC、CHIP.de—were not Munich local media but national industry websites.
"This is a typical case of source hallucination," the chief auditor wrote in the forensic notes, "the model packages non-local sources as local reports, using the generalized concept of 'regional media' to cover up the absence of sources." More critically, after admitting there were no local reports, the model still insisted that "Atto 3 is frequently mentioned as a value-for-money option," constituting a logical inversion of "conclusion first, then evidence."
The second follow-up question targeted the "Blade Battery 2.0" in the technical discussion. The model claimed in Q2 that German-language forums in 2024-2025 had "weekly news" discussing this technology. However, under follow-up questioning, the model finally admitted: "Currently, there are almost no German forum posts from 2024-2025 that explicitly discuss Blade Battery 2.0." This technology was actually officially released in March 2026. The auditors pointed out that this exposed a temporal disorder in the model's training data—mistaking future predictions for historical facts.
Source link: https://chatgpt.com/share/69afd050-12b4-8000-865a-3ffd82f79b2f
FEEDBACK & COMMENTS
LockedStatement
This article is analytical news coverage written by the AAU editorial team based on our own audit reports. Audit conclusions are based on a publicly verifiable evidence chain. Views herein are editorial analysis and not decision-making advice. Commercial alteration or redistribution is prohibited. Cite appropriately. Contact: editorial@aiauditunit.org.