A one-line idea goes in. Three hours later, Spark-to-Paper says it has a compiled research paper, complete with citations, experiments, figures, and a trail of logs. The price claimed by its authors is $8.1 per manuscript. That sounds like the end of the research assistant problem until you read what the gates actually measure.

Spark-to-Paper's artifact pipeline from idea to compiled paper

Spark-to-Paper is an open repository and an August 12 preprint built as 13 composable skills inside Claude Code. It is closer to a checked build system than to an autonomous scientist. Claude handles judgment-heavy work such as framing a question, selecting literature, writing prose, and arguing with the draft. Python scripts handle tasks with explicit pass or fail conditions: citation records, numeric claims, SVG structure, and LaTeX compilation. That split is the useful idea. The marketing line is "one sentence in, one paper out." The engineering line is that every stage leaves an artifact another stage can inspect.

The project reports 99.5% citation validity across eight controlled research topics, 96.4% figure editability, and 74% precision for issues raised by its adversarial review. Its controlled ablation injected 36 unsupported claims. A single-pass draft detected 14% of them. The full integrity and review stack detected 92%. Those numbers are worth reading, but they are not a peer-review guarantee. They describe a test designed by the same project, on a bounded set of probes, with a specific definition of detection.

What the install actually requires

The setup is refreshingly ordinary, but it is not zero setup. The repository expects Claude Code, Python 3.10 or newer, and a LaTeX installation with latexmk for compilation. The practical prerequisite line is Python 3.10+ and latexmk. The documented path is to clone the repository into Claude's skills directory, start a session, and run ts-paper on an idea or proposal. The pipeline writes its working state to disk instead of hiding it inside an orchestration service.

That file-backed design changes how you should use it. The planning stage writes a blueprint.json. Citation retrieval produces refs.bib and a claim-to-citation map. The writing stage creates section-level TeX files. Figures land under figures/, experiment code and outputs under experiments/, and each stage writes an input, decisions, and output trace under logs/. The final gate checks for main.pdf, resolved bibliography entries, and a clean LaTeX run.

There are two modes, and choosing the wrong one is the fastest way to get a misleading paper. Proposal Mode is for an idea without measured results. It can design experiments and reserve result tables, but unknown values must stay blank. Data-Aware Mode is for a proposal that arrives with real results or for a later run after experiments have produced them. Quantitative prose is then checked against results.facts.json. If a number is measured but absent from that file, the workflow should write TBD rather than inventing a value.

For a first run, use a small proposal with a dataset and a cheap baseline. Do not begin with a claim that needs a cluster, a proprietary corpus, or a difficult environment. Let the pipeline produce its blueprint, bibliography, and empty result tables first. Inspect those files before allowing an experiment stage to run. The point is not to save eight dollars. The point is to make the handoff between a model's plausible plan and actual evidence visible.

The local requirements also expose the main operational catch. The paper can run without a GPU for ordinary plots, but optional raster figure tooling and external model endpoints add their own dependencies and keys. LaTeX failures are not cosmetic. A missing font, a broken package, or an unresolved citation can stop the build. Treat the generated project like code that needs a reproducible environment, not like a chat transcript you can copy into a submission system.

Where the evidence gates stop

The strongest result in the paper is the 14% to 92% fabrication-detection jump. It shows why deterministic checks belong in an agent workflow. A script can catch a result number that has no entry in a facts file. It can catch a citation key with no bibliography record. It can reject a PDF whose figure is a flattened image when the project requires editable vectors. Those are real failures, and a language model is bad at reliably policing its own output after a long run.

The gate cannot prove that a supported claim is a good claim. A number can exist in results.facts.json and still come from a flawed experiment. A citation can resolve to a real paper and still fail to support the sentence that cites it. The project keeps semantic judgment with Claude, including whether evidence supports, weakens, contradicts, or merely leaves a claim open. That is a sensible division of labor, but it means the red-green build status is not a scientific validity score.

The review stage has a similar boundary. Reviewers must quote the paper verbatim before raising an issue, then separate reviewers try to refute those findings. That anti-skim rule is a good defense against generic feedback. The reported 74% precision means 42 of 57 raised issues were verified in the authors' evaluation. It does not tell you recall, and it does not replace a domain expert who knows whether the baseline is fair or the data split leaks information.

The system also names a failure mode that many agent loops hide: the Self-Refutation Loop. If repeated experiments keep rejecting the original objective, Spark-to-Paper bounds the experiment, critique, and revision cycle at seven attempts. It writes a failure report instead of forcing the paper to pretend the hypothesis worked, then starts a new direction. That is a better default than endless prompt-based optimism. In research, a clean negative result is an output. A polished unsupported conclusion is not.

The cost claim needs the same caution. The authors report 11.9 million tokens, $8.1 per manuscript, and 3.2 hours on average. In shorthand, that is $8.1 and 3.2 hours for one manuscript. That makes the workflow cheap enough for exploratory drafts, not necessarily cheap enough for unattended research at scale. The number covers the authors' instrumented runs and model pricing assumptions. Your bill will move with context size, retries, experiment duration, figure calls, and the model behind Claude Code. More importantly, $8.1 does not include the human hour required to inspect the proposal, audit the data, and decide whether the final claim survived its own evidence.

My recommendation is narrow. Install Spark-to-Paper if you want a repeatable proposal-to-draft workflow and are willing to keep the human at the evidence checkpoint. Use Proposal Mode for ideas, Data-Aware Mode only when you have actual result files, and make run_gates.py all a required build step. Before submission, manually sample citations, rerun the headline experiment, inspect the data split, and read the limitations section against the raw outputs.

That is where the project becomes useful. It does not remove judgment from research. It moves some of the easy-to-forget checks into code and leaves a trace when the model changes its mind. For an $8 draft, that is a good bargain. For an $8 paper submission with no expert review, it is still a trap.

Sources