A PostgreSQL query can be logically correct, indexed, and still spend most of its life doing the wrong work. That is the narrow problem QORL attacks. In a September 16 experiment, a 4B open-weights model learned to search for better physical plans than PostgreSQL's default planner on a join-heavy benchmark.

The headline result is 1.81x geometric-mean speedup. It sounds like a new database engine. It is not. The useful reading is more specific: a small model can search for workload-specific hints when someone can afford to measure candidate plans, and when the same queries run often enough to repay that search.

QORL speedup grows with more candidate selection

The project is called QORL, short for query optimization via reinforcement learning. Rohan Bansal trained a distilled 4B Qwen model inside a harness that exposes PostgreSQL metadata, planner output, candidate evaluation, and a way to keep the default plan. The model does not rewrite SQL semantics. It proposes hints that alter join order, scan type, join method, parallelism, or selected planner settings. PostgreSQL still executes the query and returns the only feedback that matters here: measured runtime.

That setup matters because query planning is a nasty search problem. Join ordering is NP-hard, and PostgreSQL has to make a quick choice using estimates that can be badly wrong on correlated, non-uniform data. The older Join Order Benchmark, or JOB, was designed to expose exactly that weakness. It contains 113 queries across 33 structural families over an IMDb database. The original JOB paper found that cardinality errors can push cost-based optimizers toward poor plans even when the database engine itself is sound.

QORL turns that weakness into a training signal. The model inspects relations, column statistics, and the default plan, then submits candidate actions. The harness converts those structured actions into pg_hint_plan hints, runs the query, and measures the result. The final policy was built in two stages: supervised distillation from larger-model trajectories, followed by agentic reinforcement learning that rewarded plans beating PostgreSQL while penalizing noisy or duplicate attempts.

What the 1.81x number actually means

The number is real within the experiment, but it is easy to misread. QORL's final evaluation gave the model three trajectories per JOB query. Each trajectory could submit up to five candidates. The best feedback across those three trajectories produced a 1.81x geometric-mean speedup and a 1.81x total workload speedup. That is a best-of-15 search budget, not one model response dropped into a production connection pool.

The author's own model selection was weaker: 1.40x when the model chose the result at the end of each trajectory. Picking the best measured feedback within each trajectory reached 1.44x. The gap tells us where much of the result comes from. QORL learned useful search behavior, but repeated candidate generation and an explicit selection rule did a lot of work too.

The broader workload result is still impressive. Across the 113 queries, the final setup cut summed latency by 44.7%, with 68 wins and zero regressions in the best-feedback-across-three evaluation. Earlier checkpoints show why the training details matter. The initial 4B model produced no valid candidate on 81 queries. After supervised fine-tuning, it understood the harness but reached only 1.16x geometric-mean speedup. The 1,200-update RL checkpoint raised that to 1.41x before the extra rollout selection step.

This is a benchmark interpretation, not a victory lap. The evaluation deliberately rewards searching a known IMDb workload. Bansal argues that this is appropriate for a company that repeatedly runs its own analytical queries, but it also means the result does not prove transfer to another schema, another PostgreSQL version, or another workload shape. A model that has learned useful plan motifs for one database can still make awful choices on yours.

There is also measurement noise. The project had to tune the database before training was trustworthy. Moving PostgreSQL's shared_buffers from 128 MB to 2 GB reduced the no-op error rate by about four times. The p90 query error fell from 13% to roughly 1% or zero in the calibration runs, while changing work_mem between the 4 MB and 32 MB settings did not fix the noise. If you do not stabilize cache behavior, reinforcement learning can reward a lucky timing result instead of a better plan.

The best gains came from familiar controls rather than magical reasoning. QORL frequently used Leading hints to alter join order, scan changes, and Parallel hints. The repository exposes four experiment modes: calibration, evaluation, supervised fine-tuning, and reinforcement learning. That is a useful design choice. It lets you measure whether your test rig is noisy before you spend compute teaching a model to exploit it.

Where this belongs in production

Do not put a 4B model in the latency path of every SQL request. The writeup makes the opposite case. Inline inference would add work to one-off queries, while the model's search may need multiple executions before it finds a good plan. The sane target is a repeated analytical workload with an identifiable query set, stable data shape, and enough executions to amortize exploration.

A cautious deployment would look like this. First, collect slow or repeatedly executed queries and save their SQL, schema snapshot, indexes, PostgreSQL version, and representative statistics. Second, calibrate the measurement rig with repeated default-plan runs. Use a warm cache policy and isolate the workers; the QORL results show that 2 GB of shared_buffers mattered more than work_mem for this experiment. Third, run candidate search offline with a hard timeout and keep PostgreSQL's default plan as an explicit fallback. Fourth, replay winners against a held-out sample and reject anything that regresses beyond a small threshold. Only then attach a hint to the production query through a controlled store.

That workflow also gives teams a simpler baseline. PostgreSQL statistics maintenance, better indexes, and carefully chosen pg_hint_plan hints cost less operational complexity. The pg_hint_plan documentation supports scan, join-method, join-order, parallelism, row-correction, and GUC hints, which covers most of the action space QORL explores. If one bad join order is hurting a handful of queries, a human with EXPLAIN ANALYZE may fix it faster than building an RL system.

QORL becomes interesting when the manual loop keeps recurring. A warehouse team may have thousands of runs of the same expensive reports, enough historical executions to measure alternatives, and a schema whose correlations defeat the default estimates. In that case, the model is not replacing the database planner. It is an offline search assistant that packages repeated tuning into a reproducible experiment.

The price is also easier to understand than most AI training stories. The reported total was about $1,200: roughly $800 for 95 hours on a rented 2x H100 node and $400 in API fees for larger-model demonstration trajectories. The author also used home GPUs and paid about $9 per day in electricity when they were fully loaded. That is cheap for a research prototype, but it is not free, and it excludes the engineering cost of building a safe rollout and validation system.

The practical verdict is simple. Keep PostgreSQL's planner for ordinary traffic. Try QORL-style search only for repeated, join-heavy analytics where a 44.7% workload reduction would pay for measurement and maintenance. Start with calibration and a small offline evaluation. If the default plan is already stable, the model adds machinery without adding much value. If your workload contains a few stubborn query families that keep wasting time, a 4B model may be enough to find the missing plan choices, provided you measure every one.

Sources