I am Vivek Kumar, an AI governance architect and privacy and cybersecurity professional pursuing graduate study at Indiana Wesleyan University. My professional focus is helping organizations connect AI adoption with access controls, evidence, accountability, and human oversight. This coursework develops my ability to evaluate model performance and communicate AI limitations clearly.
Professional direction: responsible, auditable enterprise AI adoption. A key strength is translating technical AI risks into practical governance decisions. This summary is adapted from my existing professional portfolio.
Explainability helps people inspect an answer. Validation tests whether a system works. Metrics make the successes and failures visible.
Understand
Explainability makes the factors behind an AI output understandable to people.
Test
Validation checks whether the system works on realistic, unseen tasks.
Measure
Performance metrics show how often it succeeds, fails, or treats groups differently.
Understand + test + measure → informed human decisions. None alone guarantees trust.
The validation loop
01 →Define The Task
Specify intended users, acceptable errors, sensitive data, and situations requiring human review.
02 →Test Unseen Cases
Use held-out examples; prevent test leakage. Add realistic, adversarial, multilingual, and rare cases.
03 →Check The Explanation
Perturb relevant inputs and test whether outputs change as expected. Check evidence citations separately.
04 →Compare + Review
Compare against a baseline and examine subgroup errors with domain reviewers.
05 →Monitor + Repeat
Record model version and settings. Re-test after changes, investigate incidents, and update safeguards.
Monitoring feeds new cases back into testing. Metrics inform release and review decisions; they do not replace human accountability.
Read the presentation as accessible text
Why transparency matters
A plausible answer can still be wrong
HEALTHCARE
A clinician needs supporting evidence and limitations before using an AI-generated summary.
FINANCE
A reviewer needs consistent criteria, fairness checks, and a way to challenge an outcome.
LAW
A lawyer must verify cited authorities and protect confidential information.
For GPT, Claude, Gemini, and Llama: ask what evidence supports the output and when a person must review it. [5, 6]
Why explanation is difficult
Five obstacles → five practical responses
COMPLEXITY + OPACITY
Billions of interacting parameters obscure causes → inspect features and test interventions.
PLAUSIBLE STORIES
Generated explanations may omit real influences → test explanation faithfulness.
DATA + BIAS
Uneven representation can produce unequal errors → evaluate relevant user groups.
CONTEXT SENSITIVITY
Small wording changes can change an answer → run paraphrase and stress tests.
REGULATION + PRIVACY
Accountability needs vary by use and jurisdiction → document decisions and protect sensitive data.
Attention maps and readable rationales offer clues; neither alone proves the cause of an output. [1–6]
What industry is doing
Different tools reveal different kinds of evidence
OPENAI / GPT
Sparse autoencoders extract interpretable features from GPT-4 activations. Features are partial insights, not a full explanation. [1]
ANTHROPIC / CLAUDE
Circuit tracing investigates internal pathways in a studied Claude model. Findings remain incomplete and model-specific. [2]
GOOGLE / GEMINI + GEMMA
Gemma Scope provides tools for inspecting open Gemma models. Gemma findings do not directly explain proprietary Gemini decisions. [3]
META / LLAMA
Open weights and model cards support inspection and disclose evaluation results and limitations. Openness alone does not explain a particular answer. [4]
Classify the evidence: internal inspection (OpenAI, Anthropic, Google Gemma) versus documentation and access (Meta).
Validate before relying
Define → test → compare → review → monitor
1 / DEFINE THE TASK
Specify intended users, acceptable errors, sensitive data, and situations requiring human review.
2 / TEST UNSEEN CASES
Use held-out examples; prevent test leakage. Add realistic, adversarial, multilingual, and rare cases.
3 / CHECK THE EXPLANATION
Perturb relevant inputs and test whether outputs change as expected. Check evidence citations separately.
4 / COMPARE + REVIEW
Compare against a baseline and examine subgroup errors with domain reviewers.
5 / MONITOR + REPEAT
Record model version and settings. Re-test after changes, investigate incidents, and update safeguards.
Feedback loop: monitoring → revised test cases → revalidation. A strong benchmark score cannot certify every real-world use. [4–6]
Measure what matters
Match each metric to a decision
CORRECTNESS / TASK ACCURACY
Correct responses ÷ evaluated responses. Use expert scoring for open-ended answers.
GROUNDING / UNSUPPORTED CLAIM RATE
Unsupported factual claims ÷ checked factual claims. Check sources for support and correctness.
ROBUSTNESS / RETAINED SUCCESS
Success rate on perturbed cases versus original cases. Report both rates and the gap.
FAIRNESS / SUBGROUP ERROR GAP
Compare error rates across relevant groups; report sample sizes and uncertainty.
EXPLANATION / FAITHFULNESS + CLARITY
Test whether cited factors affect outputs, then ask users if the explanation is understandable. These are separate tests.
Choose thresholds for the use case before testing; report uncertainty. Fluency, self-reported confidence, and user preference are insufficient alone. [5, 6]
Connect explanation to governance
NIST principles + EU AI Act duties
NIST / EXPLANATION QUALITY
Provide evidence; make it understandable to the intended reader; check that explanations reflect the process; respect knowledge limits. These are four distinct principles. [7]
NIST / GOVERNANCE PRACTICE
Use Govern, Map, Measure, and Manage to assign owners, assess the use case, test evidence, and address risk. These are ongoing functions, not a fixed sequence. [6]
EU / TRANSPARENCY + OVERSIGHT
For high-risk systems, Article 13 addresses information for deployers; Article 14 addresses effective human oversight. Governance evidence includes instructions, limitations, and review controls. [8]
EU / EXPLANATION RIGHTS
Article 86 covers specified significantly adverse decisions based on Annex III high-risk systems, except point 2, subject to its conditions and exceptions. It is not a right to every chatbot answer. [8]
Governance application: record permitted sources, access limits, test results, and review ownership. NIST is voluntary guidance; EU duties depend on scope and applicability. [6–8]
Sources & portfolio context
Evidence, limits, and professional relevance
RESEARCH SOURCES
[1] Gao et al. (2024), Scaling and evaluating sparse autoencoders.
[2] Anthropic (2025), On the biology of a large language model.
[3] Google, Gemma Scope documentation.
VALIDATION + GOVERNANCE
[4] Meta (2024), Llama 3.1 model card.
[5] Turpin et al. (2023), Unfaithful explanations in chain-of-thought prompting.
[6] NIST (2023), AI Risk Management Framework 1.0.
PORTFOLIO SKILL
Communicating AI reliability to a non-technical audience. Extends Assignment 4.2’s model metrics and Assignment 4.3’s safety discussion into an accessible visual decision framework.
Vivek Kumar | AIML-501: Model Development | Indiana Wesleyan University | Assignment 4.4 | Full source links accompany this artifact.
EXPLANATORY DOCUMENT
How the information is organized
This visual artifact explains why a convincing AI answer is not enough to establish trust. I organized the content into purpose, obstacles, industry responses, validation, metrics, and governance so readers can move from understanding a problem to identifying the evidence needed for a decision. The industry comparison separates internal inspection from documentation and access. The governance section uses NIST’s four explainability principles: explanation, meaningfulness, explanation accuracy, and knowledge limits [7]. These categories distinguish whether supporting evidence exists, whether the audience understands it, whether it reflects the system’s process, and where the system should stop. The consistent layout and plain language help non-technical readers compare these ideas. Validation and metrics then test reliability rather than treating a readable explanation as proof of correctness [5, 6].
For my AI governance work, the main takeaway is that an explanation needs to support a reviewable decision. For a connected enterprise assistant, I would ask which sources it was permitted to retrieve, what evidence supports its answer, what access restrictions applied, and who owns the decision to accept or escalate the result. Retrieval records provide useful evidence without fully explaining the model’s internal computation. NIST’s Govern, Map, Measure, and Manage functions provide a voluntary framework for organizing this work [6]. The EU AI Act adds a legal perspective: Articles 13 and 14 address transparency and human oversight for high-risk systems, while Article 86 provides an explanation right for specified affected decisions, subject to its conditions and exceptions [8]. The difficult distinction is between explaining an output and demonstrating that it deserves reliance. I would address it through source checks, held-out testing, subgroup error analysis, and documented human review, building on the evaluation and safety themes in Assignments 4.2 and 4.3.
RESEARCH & EVIDENCE
Sources and limitations
This coursework synthesis distinguishes voluntary NIST guidance from EU AI Act requirements. Legal applicability depends on the system, role, use, and applicable provisions; this artifact does not certify compliance.
[1] Gao, L., Dupré la Tour, T., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., & Wu, J. (2024). Scaling and evaluating sparse autoencoders. Read source
[2] Anthropic. (2025). On the biology of a large language model. Read source
[4] Meta. (2024). Llama 3.1 model card. Read source
[5] Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023). Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. Read source
[7] Phillips, P. J., Hahn, C., Fontana, P., Yates, A., Greene, K. K., Broniatowski, D. A., & Przybocki, M. A. (2021). Four principles of explainable artificial intelligence (NISTIR 8312). Read source
[8] European Parliament & Council of the European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Articles 13, 14, and 86. Consolidated text consulted November 15, 2026. Read source