Trust hasn't been deserved.
Why people don't trust AI to read their documents. In their own words.
Walk into any methods seminar. Bring up AI extraction. You'll hear some version of these.
- “It hallucinates.”
- “It just makes stuff up.”
- “AI doesn't really understand.”
- “The validation studies have been mixed.”
- “Same input, different answer every time.”
- “It made up a citation.”
- “The numbers don't match the table.”
- “It's confidently wrong.”
- “It's a black box. I can't audit what it did.”
- “Worked on the demo. Broke on my data.”
- “The model version changed and my pipeline broke.”
- “I don't trust it for anything that matters.”
Each one is real. Each one is solvable. But the biggest problem hasn't been discussed enough. It isn't a technology problem. It's a scientific one.
@karlrohe