Ask HN: Is anyone using LLM based document processing in production?
I'm wondering if anyone is actually using LLMs to process documents reliably in production. One hallucination could lead to a host of issues. For example, if someone is using LLMs to process documents and enter data into an ERP, if even one number is off it could cause accounting issues, inventory issues etc. Human in the loop doesn't help because the human would just have to read the document themselves to ensure accuracy, defeating the point of the automation.
Every extracted field comes with a precise citation back to the source document (page + snippet + bounding box + confidence score) so reviewers can verify where each value came from.
Hallucinations get flagged automatically because there's no supporting text in the source.
The goal is to make HITL fast and not have reviewers read through the whole document.
I have a project with them, processing auto insurance claims. Mostly extracting details from police reports like license plate numbers, extracting details of the incident.
"Human in the loop doesn't help because the human would just have to read the document themselves to ensure accuracy, defeating the point of the automation."
They're doing it manually without it. Semi-auto beats manual readily. There's still checks like submission of the number to grab the details of the individuals involved, and if the names, vehicle type, etc don't match, that automatically flags that something's off.
Every extracted field comes with a precise citation back to the source document (page + snippet + bounding box + confidence score) so reviewers can verify where each value came from.
Hallucinations get flagged automatically because there's no supporting text in the source.
The goal is to make HITL fast and not have reviewers read through the whole document.
"Human in the loop doesn't help because the human would just have to read the document themselves to ensure accuracy, defeating the point of the automation."
They're doing it manually without it. Semi-auto beats manual readily. There's still checks like submission of the number to grab the details of the individuals involved, and if the names, vehicle type, etc don't match, that automatically flags that something's off.