Job Hunter: An Autonomous Job-Application Pipeline That Knows Where It Stops

An end-to-end agent that searches job boards, scores fit against a master resume, tailors a resume and cover letter per posting with Claude, and drives real ATS forms — graded weekly against live boards on whether it fabricates anything and on exactly how far each board's automation actually reaches, rather than claiming a uniform "it works everywhere."

The idea, in one sentence

Job searching is a pipeline of small, repetitive judgment calls — does this posting fit, which two resume bullets actually matter for it, what does this specific ATS form want — and this project automates that pipeline end to end, from discovering a posting to a filled-out application form, while stopping deliberately short of the one step that shouldn't be automated: clicking submit. Every generated package is staged for the owner's own review first, by design.

Does the tailored resume actually tell the truth?

A resume-tailoring pipeline's biggest failure mode isn't a bad match — it's a good-looking match that quietly invents an employer, a date, a skill, or a metric the candidate never had. The eval harness runs 16 fixture persona/posting pairs through the real tailoring prompt with live Claude calls (not mocked), including 6 pairs built specifically to be adversarial: a seniority mismatch, a skills-adjacent tool swap, an employment gap, a cross-domain pivot, a regulated-industry credential overclaim, and a quantified-metric fabrication trap. Each output is graded on multiple axes, one of which is load-bearing: no_fabrication — did the tailored resume invent anything not present in the source. That axis averages 100/100 across all 16 pairs. The overall weighted score, 0.906, is pulled down mostly by keyword_coverage averaging 68.6/100 on the adversarial set — intentionally lower, because the honest resume for those cases must decline to claim vocabulary the candidate doesn't actually have, and a high keyword score there would mean the eval caught the pipeline lying.

What "integrates with an ATS" actually means, board by board

Six distinct applicant-tracking platforms are tested, two live boards each: Greenhouse, Lever, Ashby, Workable, Jobvite, and SmartRecruiters. "Integrates with" is not the same claim on all six, and collapsing it into one number would hide the real picture:

Five of the six platforms genuinely populate a real application form with matched fields and a landed résumé attachment before hitting a deliberate stopping point. The sixth, SmartRecruiters, is a different situation entirely: a bot-detection layer blocks the browser session before a form even renders, so it hands off to manual apply with zero automation — a real, permanent characteristic of that board rather than a flaky one-off, confirmed on every re-check since this monitoring began.

Catching drift before it goes silent

ATS vendors change their forms without announcing it, and a job-application pipeline that silently starts filling the wrong field is worse than one that visibly fails. Two independent suites re-check the same 12 live boards on a recurring schedule: one re-fetches each posting and diffs its schema against a frozen baseline (catching a board that renamed or reordered fields), the other runs a real (non-submitting) Playwright fill against the same boards and diffs the result against a frozen per-board expectation (catching a board where the form still renders but stopped accepting the same inputs). Both suites are clean, 12/12, as of the most recent run (2026-08-29) — but "clean now" understates the point. The real value is in the history: both suites have genuinely dipped below 100% multiple times as boards changed underneath them, and recovered within days each time, because a dip is a real signal to go fix the affected board's driver, not a flake to retry away.

Reducing the review bottleneck without automating past it

Because every package still stops for the owner's own review before anything is submitted, the pipeline's other job is making that review fast rather than repetitive. The onboarding flow collects role preferences and a resume once per search profile, not per application:

Across a batch of staged applications, most of the ATS forms above ask overlapping boilerplate questions — work authorization, sponsorship, address, years of experience, degree, salary range — that don't vary by posting. Rather than surfacing each one per application, the pipeline consolidates them into a single digest so the owner answers each recurring question once, and every staged package that needed it gets the same answer applied automatically:

What we'd do differently

The honest limits are worth naming plainly. SmartRecruiters' WAF has no workaround that doesn't cross into the kind of bot-detection evasion this project deliberately avoids, so that board stays manual-only rather than being "fixed" the wrong way. The keyword-coverage axis of the eval harness is a heuristic proxy for resume quality, not a guarantee the highest-scoring bullets are truly the best ones for a given reader. And the drift checks catch a board that changed underneath the pipeline, but only on the same re-check cadence the suites run on — a board that changes and reverts between two scheduled runs would pass through unnoticed.

The actual point

The interesting part of this project isn't that an LLM can write a passable cover letter — it's building the harness that keeps that capability honest at scale: grading every tailored resume for fabrication instead of trusting the model's own confidence, treating "this ATS integration works" as a per-board claim that needs its own evidence rather than one number for all six platforms, and re-checking that evidence on a schedule instead of assuming a working integration stays working. Automating the tedious 90% of a job search only pays off if the system is honest about the 10% it can't do.

Reproducibility

Every number on this page is pulled directly from the project's own metrics.json and its eval/drift report files, generated by scripts/eval_tailoring_quality.py, scripts/verify_ats_drift.py, and scripts/verify_ats_fill_drift.py against live Claude calls and live ATS boards — not mocked, not re-derived here.