One Prompt Change That Forces Claude to Be Honest

As AI models become more intelligent, they are less likely to admit when they don't know an answer, leading to confident guessing. This creates an 'honesty gap'. Compounding this is 'automation bias', where users trust more confident AI outputs less, leading to unchecked errors. This challenge is critical in tasks requiring AI to extract information from documents, where it might infer rather than stick to the source.

Instead of asking AI for confidence scores (which it can lie about), force it to give blank values when uncertain or when it doesn't know. Crucially, require it to explain *why* it left a field blank. This allows quick identification of uncertain data and helps users understand where errors might originate, making corrections easier. The prompt specifies extracting only explicitly stated values and giving a one-sentence explanation for blank fields.

The AI currently treats wrong answers with the same value as blank ones. This rule aims to alter the AI's incentive by explicitly stating that a wrong answer is three times worse than a blank answer. This encourages the AI to provide more blank answers when unsure, similar to how a new employee would avoid costly mistakes if penalized.

Even with previous rules, AI might still infer on complex tasks, deviating from strict extraction. This rule creates a safety net by requiring the AI to add a 'source' column for each extracted field. The source should explicitly state 'extracted' (with evidence from the document) or 'inferred' (with an explanation of what was inferred and from where). This helps users quickly validate outputs by focusing only on inferred fields, increasing trust in the AI's results.

The three rules—forcing blank answers with explanations, penalizing wrong answers more than blank ones, and requiring the AI to show its source—reduce the burden on users and increase trust in AI outputs. Instead of checking everything, users only need to review blank fields and inferred information, streamlining the validation process for complex tasks.