Making LLMs work on millions of rows

A model with a 100 million token context window still cannot read a 10 million row table. The numbers do not work out. DataTune is what we built so the model does not have to.

Why this is a real limit

Picture a mid-sized enterprise table. Ten million rows. Twenty columns. Fifty characters of text per cell on average. That is around 2.5 billion tokens of data. Even the largest context windows are roughly twenty-five times smaller. The model never gets to see the whole thing, no matter how generous the context budget gets.

So teams compromise. They feed the model a sample. Or a summary. Or just the schema. Each one of these is a different kind of lie about what is actually in the data. The model gives an answer based on a rough abstraction, and the team has no way to know whether that answer would have held up against the real rows.

What DataTune does instead

DataTune does not try to fit the whole table into one prompt. It splits the data into batches, sends those batches to the model in parallel, and lets the model do its work one batch at a time. The orchestration runs on Dask, so it scales the same way the data scales.

You write the transformation in plain English. DataTune does the rest. The model sees enough of the data, in small enough pieces, to give a real answer on every row.

The four primitives

Everything in DataTune is one of four operations.

• MAP. Look at each row and add or transform fields. Extract a category from a product description. Pull a date out of free text. Turn a paragraph into a structured record.
• FILTER. Keep or drop rows based on a plain-English rule. Keep only negative reviews. Drop rows that look like test data.
• EXPAND. Add new rows or new context to existing ones. Split a multi-item record into one row per item. Enrich a customer record with attributes inferred from the data already on hand.
• REDUCE. Aggregate across rows. Summarise. Count. Roll up.

Most real-world pipelines chain these together. Extract sentiment, then filter for negative reviews, then reduce into a list of top complaints. Every step runs across the full table.

What it looks like in code

import datatune as dt
from datatune.llm.llm import OpenAI
import dask.dataframe as dd

llm = OpenAI(model_name="gpt-4o-mini")
df = dd.read_csv("products.csv")

categorised = dt.map(
    prompt="Extract Category and Subcategory from the name and description.",
    output_fields=["Category", "Subcategory"],
    input_fields=["Name", "Description"],
)(llm, df)

electronics = dt.filter(
    prompt="Keep only electronics products.",
    input_fields=["Name"],
)(llm, categorised)

result = dt.finalize(electronics).compute()
result.to_csv("electronics.csv")

Twelve lines, a real LLM call on every row, runs on a dataset of any size. No sampling. No summarising. No giving up on parts of the data because they did not fit.

What this changes for data teams

Teams stop reaching for hand-written rules to clean their data. They stop hiring a tagging vendor when their catalog needs new categories. They stop hand-engineering features that an LLM could pick out if it had the access. The work that used to take weeks of fragile ETL becomes a short script.

And because the pipeline is just code, every step is versioned, reviewable, and re-runnable. A new column from product takes a sentence, not a sprint.