Clients often ask which AI approach they should use. The usual answer is "it depends". Each session in this series sets out, for one task, what the answer depends on and which options to consider.
Consider a common question: what is the best way to get the contents out of a few thousand PDF contracts?
There is no single answer. Small scripts, classic OCR, cloud document services, specialist vendors, open models and the big general models like Gemini, GPT and Claude can all do it. Each is the right choice in some situations. The choice depends on factors such as:
The options also change every few months: an approach that was too expensive last year may now be the cheapest adequate option. Each session takes one task and sets out the current options, how they work, what they cost, and which to try first in common situations.
Each dot is one way of doing the task. Up means better results on a fair test. Right means more expensive per unit of work.
An option is on the frontier if nothing else is both better and cheaper. Only these options merit detailed evaluation.
Every option below the line is beaten on both quality and cost by an option on it. This is usually most of the market.
Constraints such as "must run in our Azure tenant" remove options. This moves the frontier and may change the best choice.
Illustrative data. Real charts are in each session.
A short history, explaining why several approaches remain in use.
How each approach works internally, illustrated with real examples.
Vendors, open models and cloud services, including which can run on Azure, Google Cloud or AWS.
The main public benchmarks, who runs them, and their limitations.
Quality against cost, from benchmarks other people have run, with sources and dates.
Dated default picks for common client situations, and what to test them against.
Getting text, tables and fields out of PDFs and scans. Covers PDF libraries, OCR, cloud services, specialist models and vision language models.
Open the session →Search APIs, "deep research" tools and agents that browse. Their accuracy and cost per answer.
Agents that run experiments in a loop to improve a model, a prompt, a piece of code or a business metric.
Code completion tools and agents that implement changes and open pull requests, and their uses in data work.
Turning messy text and tables into clean, typed data. Schemas, validation, and matching records.
Chunking, embeddings, rerankers, and when a long context window is simpler than retrieval.
Speech to text, telling speakers apart, and turning calls and meetings into usable data.
Labelling millions of records with small models, large models, or a mix of the two.