LLM’s suck at complex OCR, and probably will for a while. LLMs are excellent for many text-generation or summarization tasks, but they falter at the precise, detail-oriented job of OCR—especially when dealing with complicated layouts, unusual fonts, or tables. These models get lazy, often not following prompt instructions across hundreds of pages, failing to parse information, and “thinking” too much.

LLMs process images through high-dimensional embeddings, essentially creating abstract representations that prioritize semantic understanding over precise character recognition

Consider a simple table cell containing "1,234.56". The LLM might understand this represents a number in the thousands, but lose critical information about:

Exact decimal placement
Whether commas or periods are used as separators
Font characteristics indicating special meaning
Alignment within the cell (right-aligned for numbers, etc.)

https://news.ycombinator.com/item?id=42966958