A staged workflow hands you a folder of Markdown files and one essay. A single prompt hands you one essay. Before you adopt the longer pipeline, find out whether the extra files bought a better essay or just a fuller folder.
This guide shows how to test that yourself. It uses JP Essay, an open-source Claude Code skill, only as an example of the pattern. The project write-up describes a sequence of research, outline, draft, review by a separate AI reader, rewrite, and fact-check, with intermediate Markdown files kept for inspection. JP was not run for this guide, so nothing here is a verdict on it. The scorecard works on any pipeline with the same shape.
Why you test argument structure first
Test in this order: structure, factual accuracy, readability, then editing time saved. This order is an inference from how staged workflows work, not a published benchmark.
Structure comes first because the staged steps (outline, fresh-reader review, reorganization) exist mainly to fix the argument. If the argument is weak, every later step polishes the wrong thing. Facts come second because rewriting changes which claims survive, so you want a stable claim list before you verify anything.
Readability comes third because sentence-level problems are cheap to fix. Editing time comes last because it summarizes the other three: you spend time fixing whatever failed.
Break the order only if your essays are heavy on data. In that case, test factual accuracy first, because one wrong number can invalidate the whole piece.
Step 1: Set up a fair side-by-side run
A fair comparison changes one variable: the process. Hold everything else constant.
- Write one question and a target length. Choose a topic you can judge, ideally one where you already know the main facts.
- Write the direct prompt before you see any workflow output, so you cannot tune it to lose. Keep it honest: the question, the length, the audience.
- Use the same model, the same tool access (web search on or off for both), and the same audience brief.
- Run the direct prompt once and the workflow once. If budget allows, repeat both on a second question, because a single run is noisy.
- Strip file names and formatting hints, then label the finals A and B. If you can, have someone who did not run them score the finals.
- Record wall-clock time and token cost for each run. The workflow will cost more, so the test asks whether it earns that cost back.
The JP write-up notes that you need your own Claude Code access and that model usage is not included. Budget for the extra calls.
Step 2: Verify the artifacts exist
Before you judge quality, confirm the pipeline produced its intermediate work. The JP write-up mentions a checking script that validates required steps. You can reproduce the idea for any workflow.
The file names below are illustrative. Rename them to match your pipeline.
UNEXECUTED: nobody has run this script. Run it yourself before you trust it.
## check_artifacts.py (UNEXECUTED)
## Python 3.9+, standard library only, no dependencies.
import sys
from pathlib import Path
REQUIRED = [
'research.md', 'outline.md', 'draft.md',
'review.md', 'revision.md', 'factcheck.md', 'final.md',
]
def main(run_dir: str) -> int:
root = Path(run_dir)
problems = []
for name in REQUIRED:
path = root / name
if not path.is_file():
problems.append(f'MISSING: {name}')
elif not path.read_text(encoding='utf-8').strip():
problems.append(f'EMPTY: {name}')
if problems:
print('\n'.join(problems))
return 1
print(f'OK: {len(REQUIRED)} artifacts present and non-empty')
return 0
if __name__ == '__main__':
if len(sys.argv) != 2:
print('usage: python check_artifacts.py <run_dir>')
sys.exit(2)
sys.exit(main(sys.argv[1]))
Expected output for a complete run:
OK: 7 artifacts present and non-empty
Verification step: run python check_artifacts.py runs/test1 against a real run folder, then run echo $? (or echo %ERRORLEVEL% on Windows). You should see 0. We cover related ground in how to move from quickstart to production (checklist).
Failure case: delete or empty review.md, then run the script again. Expect MISSING: review.md (or EMPTY: review.md) and exit code 1.
This script proves only that files exist. A file containing one line passes. Treat the script as a gate, not a score.
Step 3: Score the four tests
Score each final essay from 1 to 5 on each test, then apply the pass threshold. These thresholds are starting points. Adjust them to your standards, but set them before you read the outputs.
| Test | 5 means | 3 means | 1 means | Pass | |---|---|---|---|---| | 1. Argument structure | A fresh reader restates the thesis and every main step correctly | Thesis is clear, one step is missed or misordered | Reader cannot state the thesis | 4 or higher | | 2. Factual accuracy | Every critical claim verified by a human, none wrong | Minor claims unverified or imprecise | A critical claim is wrong or unsourced | 4 or higher, and zero wrong critical claims | | 3. Readability | No rewrite needed for clarity | A few paragraphs need rework | Hard to follow throughout | 3 or higher | | 4. Editing time saved | Less time to publishable than direct output | About equal | More time than direct output | Workflow edit time below direct edit time |
The workflow passes overall if it passes tests 1 and 2 and beats direct prompting on at least one of tests 3 and 4. A workflow that wins only on readability is not worth its cost.
Test 1: argument structure
Open a new session with no context. Give it only the final essay and ask it to write the thesis and main points from memory. Then do the same yourself. This mirrors the fresh-reader step in the JP write-up, where a separate agent reconstructs the argument from memory.
Compare both reconstructions to your intended argument. A pass means the thesis and key steps match. If the reader invents a different thesis, the structure failed, however smooth the prose.
Test 2: factual accuracy
Build a claim log. Extract every number, name, date, quote, and any claim the conclusion depends on. Use one row per claim:
| Claim | Where it appears | Source checked | Verified by human (Y/N) | Result | |---|---|---|---|---|
Verify against primary sources yourself. Do not accept the workflow's own fact-check file as verification. Use it as a map of what to check first.
Test 3: readability
Read each final aloud without knowing which is which. Mark every sentence you stumble on and every paragraph where you lose the thread. Score by how much rewriting the text needs, not by how the prose sounds.
Test 4: editing time saved
Start a timer and edit each final until you would publish it. Include the time you spend verifying claims from the log, because a polished essay with unchecked facts is not done. Record the minutes for each and compare.
Step 4: Trace weak spots through the artifacts
This is where the extra files pay off, if they pay off at all. When a final essay fails a test, walk backward through the intermediate work.
Also read: s3 gateway endpoint vs interface endpoint: which to pick in depth
The JP write-up names three possible origins of an unclear conclusion: the outline, a missing source, or a revision that weakened the argument. Check each in order:
- Does the outline contain the conclusion's logic?
- Does the research file hold a source for the claim?
- Does the draft state the point more clearly than the revision does?
Diff the draft against the revision to find where clarity dropped.
A direct prompt leaves no such trail. If the workflow's failures are traceable and its fixes are one-step edits, that is a real advantage even when the scores tie.
What AI review and AI fact-checking can still miss
The JP write-up says an AI review can still miss problems. It also says the fact-checking stage does not remove the need to verify important claims yourself. Take both warnings seriously.
The gaps below are inferences from how such systems work, not findings from the write-up:
- A fresh AI reader may share blind spots with the model that wrote the draft.
- It may fill in missing steps from its own background knowledge, which makes a gappy argument look coherent.
- An AI fact-check may consult the same sources the research step gathered, so a bad source can pass twice.
- Neither stage can tell you whether a claim matters to your readers or whether a citation is stale.
That is why Test 2 requires a human check on critical claims, no matter how good the fact-check file looks.
What to expect next
Your first side-by-side run may teach you more about your own rubric than about the tools. Your thresholds will probably shift after you read both finals. That is fine if you write down the change and rerun.
After two or three questions, you should see whether the workflow's gains show up in structure and traceability, or only in the file count.
