TEC-MET-2026

AAU Algorithm Perception Measurement System White Paper

AAU Prompt Matrix 5.0 · AAU Data Architecture and Technical Laboratory

1. Overview

As the penetration rate of large language models in business and social decision-making rises exponentially, traditional code auditing can no longer assess the risks of generative AI. AAU adopts a "black-box behaviorism" methodology that does not rely on white-box access to internal model parameters, but rather quantifies cognitive biases, factual hallucinations, and security boundaries of AI models in specific vertical domains through high-concurrency, adversarial external stress testing. This system aims to establish a quantifiable, reproducible third-party evaluation standard independent of model developers.

2. Core Testing Architecture: Prompt Matrix

The core of auditing lies in how to ask questions. AAU abandons random single questioning methods and has developed the structured AAU Prompt Matrix dynamic testing framework.

2.1 Adversarial Red Team Testing

We do not test AI's performance under compliant instructions, but rather test its defensive capabilities under extreme inducement. ● Inductive Attacks: Implant false premises and observe whether AI will comply with fallacies. ● Role-Playing Jailbreaks: Require AI to play specific roles with extreme positions to test the robustness of its safety guardrails. ● Multi-Round Context Contamination: Inject interference information in long conversation histories to test the model's memory decay and logical consistency regarding core facts.

2.2 Cross-Language Parallel Corpus

To eliminate cultural biases from single-language training data, all standard tests are executed synchronously in seven major language environments. ● Semantic Alignment: Ensure that prompts in different languages such as Chinese, English, and German have completely consistent semantic instructions. ● Culture-Specific Probes: Implant localized contexts for sensitive topics in specific regions to detect whether the model exhibits "double standard" outputs.

2.3 Dynamic Temperature Control

The audit process covers both deterministic and creative output modes. ● Precision Mode: Set system temperature parameters to 0.1 to 0.3 to test the model's ability to accurately reproduce factual data. ● Divergent Mode: Set system temperature parameters to 0.7 to 1.0 to test whether the model produces uncontrollable hallucinations in open-ended questions.

3. Global Node Sampling Network

To circumvent large models' "geo-fencing" strategies and compliance filtering mechanisms in different regions, AAU has deployed a distributed physical sampling network.

3.1 Physical Node Distribution

Audit requests are not sent from a single server, but are routed through physical nodes or residential IP proxy networks deployed by AAU in the following hubs: ● Asia-Pacific Nodes: Singapore, Tokyo, Hong Kong ● European and American Nodes: Frankfurt, London, California ● Emerging Markets: São Paulo, Dubai

3.2 Latency and Concurrency Control

The system records the first-token generation time and total generation time for each prompt to assess whether there are discriminatory differences in the model's reasoning power allocation across different regions.

4. Scoring Algorithm and Indicator System

AAU adopts a multi-dimensional vector scoring model, ultimately generating a single Perception Health Index (PHI).

4.1 Hallucination Rate (HR)

Calculate the proportion of factual errors in model outputs. ● Calculation Logic: Compare entities, data, and times in model outputs with AAU's preset "ground truth knowledge graph." ● Determination Threshold: If key factual errors exceed one instance, the entry is marked as "hallucination."

4.2 Sentiment Bias (SB)

Use natural language processing technology to analyze the emotional polarity of adjectives the model uses for specific brands or entities. ● Semantic Distance: Calculate the cosine similarity between brand words and negative word vectors such as "expensive," "dangerous," and "backward." ● Baseline Comparison: Compare the target brand's sentiment score with industry baseline (such as the average of the top 3 in the industry).

4.3 Commercial Visibility Weight (CVW)

In recommendation scenarios, evaluate the ranking weight of target brands in AI-generated lists. ● Top-N Appearance Rate: Frequency of brand appearance in "top five recommendations." ● First-Place Recommendation Rate: Probability of the brand being listed by AI as "preferred" or "best."

5. Fides Evidence Locking Protocol

To ensure the judicial-level evidentiary validity of audit results, AAU has independently developed the Fides Protocol to solidify full-process data.

5.1 Original Session Fingerprint

The system automatically captures the original conversation link or complete JSON log generated by the model, including Request ID, token consumption, and generation timestamp.

5.2 Hash Blockchain

Perform SHA-256 hash operations on each piece of key evidence (especially negative evidence judged as D-level). The generated hash value will be written to a distributed ledger, generating a unique Trace ID. Once on-chain, any tampering with the original evidence will cause hash verification to fail.

5.3 Evidence Snapshot Archiving

In addition to text logs, the system also takes full-screen screenshots of the conversation interface and applies digital watermarks as supplementary backup for visual evidence.

6. Risk Rating Definition

Based on the above quantitative data, the system automatically generates risk ratings from A to D. ● A-Level (Verified): Algorithm performance is neutral and accurate. Hallucination rate is below 1%, and sentiment bias is within industry standard deviation. ● B-Level (Neutral): Minor occasional errors exist but do not constitute systemic risk. Regular monitoring is recommended. ● C-Level (Skewed): Obvious tendency bias or outdated data detected. May mislead brand reputation; data intervention is recommended. ● D-Level (Critical): Serious malicious hallucinations, defamatory outputs, or content violating safety ethics exist. Immediate crisis PR and legal procedures are required.

Technical Disclaimer

This methodology aims to approximate the true performance of algorithms as closely as possible, but due to the probabilistic nature of large language models, audit results only represent the system state under specific time windows and specific sampling parameters. AAU reserves the right to periodically iterate on testing models and algorithm weights.