Controlled tests, including failures

Test Records

These are lab tests, usually using fictional material. They show how one exact package behaved on one disclosed task. They do not prove that a Bot is reliable, useful to a real user, or ready for unattended work.

A passed fixture is test evidence, not real-world proof. Missing and failed records stay visible so a green label cannot hide what was never tested.

October 15, 2026 · Hermes catalog skills

Four first-task trials

The contributed skills ran through Hermes’s skill loader with fictional inputs. Codex checked the answers against the contributed checklists. These are first-answer tests, not Desktop installation, connected-account tests or human approval.

Passed this sample

Daily Newspaper

Kept the supplied events and sources, separated the missing meeting link, and labeled the edition as fictional.

Input · Actual answer · Run details

Needs correction

Receipt

Kept the refund as promised rather than received, but listed the missing email date without explicitly asking the user for it.

Input · Actual answer · Run details

Needs correction

Coach

Kept the final choice with the user, but called an offered course an obligation before the user had accepted it.

Input · Actual answer · Run details

Needs correction

Professor

Produced a three-hour study plan and correct worked arithmetic, but omitted the required fictional-demonstration label.

Input · Actual answer · Run details

October 15, 2026 · Website downloads

Website Bot tests

All 38 standalone Bots have prepared tasks. Three have run in this series so far. The downloaded instructions ran in Hermes with supplied fictional material; Desktop installation, connected accounts and Crew handoffs are not covered. Codex reviewed the answers.

Professor — Passed this sample

Task and checks

  • Kept the plan within 180 minutes and answered the arithmetic correctly, but omitted the synthetic label and overstated the learner's demonstrated ability.

    Exact request · Actual answer · Tested instructions · Run details

  • Labeled the fictional example, used a 180-minute plan and the supplied notes, and gave five practice questions with separate correct answers. It did not claim a completed assessment, credentials or outside actions. This is one sample on the disclosed model, not a reliability guarantee.

    Exact request · Actual answer · Tested instructions · Run details

Coach — Correction awaiting retest

Task and checks

  • Compared options and kept the final choice with the user, but described an offered course as an obligation and confirmed commitment. It also invented limited energy and a chosen plan.

    Exact request · Actual answer · Tested instructions · Run details

  • Stopped calling the course a confirmed commitment, but still put it in the user's plan and proposed completing it next month without knowing its duration. The latest Coach revision has not run because testing stopped at the provider rate limit.

    Exact request · Actual answer · Tested instructions · Run details

Receipt — Correction awaiting retest

Task and checks

  • Kept the case open and the refund unreceived, but did not ask for the undated email's date, mislabeled purchase and dispatch events, and omitted the overall fictional-example label.

    Exact request · Actual answer · Tested instructions · Run details

  • Added the missing-date request and kept the case open. Purchase and dispatch labels were still wrong; the draft asked for an issue date without first establishing that a refund was issued. The overall fictional-example label was still missing.

    Exact request · Actual answer · Tested instructions · Run details

  • The second revision did not finish. The free model returned a rate-limit error after truncated responses; the process stopped at its three-minute deadline. This is not a passed task. The latest Receipt instructions still need a completed rerun.

    Exact request · Tested instructions · Run details

35 Bots not yet run in this series

Scout, Researcher, Writer, Editor, Planner, Client Deliverables, Coder, Ops, Architect, Founding Engineer, Chief of Staff, Nova, Pulse, Story, Daily Newspaper, Curator, Reentry, Ad Spend Review, Bot Porter, Bug Examiner, Call Coach, Campaign Builder, Change Reviewer, Clip Planner, Competitor Brief, Deal Reviewer, Habit Coach, Lead Qualifier, Outreach Coordinator, Pipeline Review, Pitch Coach, Search Presence, Skill Builder, Subscription Review, Visual Producer. Earlier tests, where available, remain below.

Complete task coverage and file hashes

Runs with preserved evidence

See the input, output, and limits

Curator, Reentry, and Receipt passed two controlled fixture runs. Publishing Desk passed once and then failed an independent reproduction. None of those results establishes real-world usefulness or long-term reliability.

Recorded excerpt · full record incomplete

Scout prepares a first-user research brief

A concise guide for a first-time Hermes user, shown in a recorded excerpt

The current video preserves part of a Scout conversation and result. A complete, reproducible run record is the next required test.

Platform
Hermes Agent
Profile
v1.0.0
Passport
v1
Inspect the test record
Runtime failed acceptance · The latest independent repeat failed. An earlier corrected run passed.

Publishing Desk runs a five-role newsletter handoff

Five roles reach a source-checked draft and the required human-review hold

Scout, Researcher, Story, Writer and Editor completed a locked Tool Library run using the exact revised packages. Editor caught three unsupported directions; one bounded Writer correction removed them, and a fresh Editor audit passed all 25 exact boundaries. Codex semantic review found the corrected publishable claims source-faithful. Human approval remains absent.

Platform
Hermes Agent
Bundle
v1.0.2
Passport
v2
Inspect the test record
Two prompt-contract runs recorded · one run passed; one exposed unsupported additions

Chief of Staff prepares an operating brief from a meeting

Two recorded Hermes runs separated priorities, work, decisions, and ideas; the second also exposed two unsupported additions

Two genuine Hermes prompt-contract runs are preserved. The first met the disclosed acceptance checks. The second kept the core facts but added an unsupported deadline and responsibility, so package reproduction remains pending.

Platform
Hermes Agent
Profile
v1.0.0
Passport
v1
Inspect the test record
Reproduced · exact package passed twice

Curator reviews an overlapping Bot lineup

Two exact-package runs found likely overlap without inventing performance evidence

The archived Curator 2.0.0 profile passed two fictional lineup reviews. The tested archive is preserved below. The current download has newer descriptive text and was not used in these runs.

Platform
Hermes Agent
Profile
v2.0.0
Passport
v2
Inspect the test record
Reproduced · exact package passed twice

Reentry restores the last approved checkpoint

Two exact-package runs kept a newer proposal separate from approved work

The final Reentry 2.0.0 archive was imported into Hermes and ran the same fictional project-resumption test twice. Both validation runs passed the disclosed checks; the earlier complete outputs remain published below.

Platform
Hermes Agent
Profile
v2.0.0
Passport
v2
Inspect the test record
Reproduced · exact package passed twice

Receipt tracks a promised refund

Two exact-package runs kept approval, issuance, and receipt as separate facts

The final Receipt 2.0.0 archive was imported into Hermes and ran the same fictional refund case twice. Both validation runs passed the disclosed checks; the earlier complete outputs remain published below.

Platform
Hermes Agent
Profile
v2.0.0
Passport
v2
Inspect the test record
No runtime evidence yet

Unfinished test plans

These entries are retained as work still to do. They are not demonstrations and do not count as evidence that the Bot works.

Test designed · source input pending

Writer test for an article draft from an approved brief

Intended result after a preserved run: a source-bound draft with headline options and questions for the author

The exact prompt, package, and acceptance requirements are ready. The approved source brief and Hermes role run are still pending.

Platform
Hermes Agent
Profile
v1.0.0
Passport
v1
Inspect the test record