What is the campaign INSTRUCTION.md?

This is the INSTRUCTION.md the campaign was launched with, stored by the API in the create request and returned unchanged whenever the campaign is read. It is the same text that was in the composer box when Run campaign was pressed, and it is what every setter trial read in its sandbox. Because a campaign is immutable, this text can never drift from what actually ran, which makes it the anchor for every task in the pack.

Reading it next to the tasks is the best way to judge the setter. If the tasks match the prompt's intent, the run did its job. If they wander, the prompt was probably too loose, and the fix is to write a tighter prompt and launch again. A restart from this campaign's certified pack keeps the tasks and only measures the solver afresh, so the prompt matters most on the very first run.

What are the task files?

Task files are the working materials the setter placed in the task's environment: the code, data, fixtures, and configuration a solver finds when its sandbox starts. The page lists them from the task's stored evidence, and each file can be opened and read on its own. The API checks every file's size and checksum against the record written when the evidence was saved, and refuses to serve a file that does not match, so nothing here has been altered since the run. Files over a fixed size are truncated for display, and very large trees are capped so a single task cannot overwhelm a reader.

Only files a solver was allowed to see are listed. The grading module, the setter's oracle solution, launch configuration, environment secrets, and result records are private and never appear, which is what keeps a task honest when it is shared. A certified task pack also carries a few files of its own, such as the grader module and the validator notes, which are shown under the pack rather than under a single task. Reading the files alongside the instruction is the quickest way to judge whether a task is fair.

What is the stored evidence?

Stored evidence is everything the worker saved about a task once its trial tree finished: the files each trial produced, the logs it wrote, and the trajectory that records what the model said and did at every step. The worker persists it with a length and a checksum for every file, and the API verifies both before serving anything, refusing a file that does not match rather than showing an altered one. The page reads the evidence tree rooted at the task, so a setter, its validator, every solver attempt, and any judge are all one read away.

The evidence is what turns a number into an explanation. Open a solver's trajectory to see why an attempt failed, or a validator's log to see why a task was rejected. Private material never appears: the task's configuration, its grading module, its oracle solution, launch details, and any secrets are withheld, which is what keeps a task honest when it is shared. Large files are truncated for reading, and the whole tree is bounded by size and count so one task cannot overwhelm the page.

What is the Atlas?

The Atlas is the view of every task you can see, gathered across all your campaigns and the example campaigns into one place. The page builds it in your browser: it asks the API for your campaign list and the example list, then requests the task rows of each campaign in turn, a few campaigns at a time, your own first and then the examples. Every task row keeps its campaign, its outcome, its solve rate, and its identifier, so nothing in the Atlas is a copy or a summary. It is the same data the campaign pages read, laid side by side.

Use it to compare rather than to inspect. The summary counts what was gathered, and the matrix draws every task as a cell colored by outcome and solve rate, so you can see at a glance which campaigns found the frontier and which did not. Selecting a task opens it in its own campaign with its instruction and evidence. Because it reads many campaigns, the Atlas takes a moment to fill, and it fills in the order the reads finish.

What does Compute timeout bound?

Compute timeout gives the campaign an overall time budget in seconds. The worker treats it as the budget for the full attempt, including startup and terminal work. It reserves part of that time for saving evidence and cleaning up, so generation cannot spend every second. When the work reaches its deadline, the worker cancels unfinished compute. Completed evidence remains the basis for the results that can be read.

I would size this budget alongside the task count and rollouts per task. Giving each agent a long limit does not extend the campaign deadline around it. If the campaign stops with work unfinished, inspect the completed outcomes and where time was spent. A new campaign can use a more suitable budget, while the finished record keeps the settings that actually governed it.

What does Concurrent trials control?

Concurrent trials caps how many trials this campaign can execute at the same time. It controls overlap across the work rather than the size of an individual sandbox. The worker reserves room for child trials such as validation and solving when it admits generation work. Each account also has a limit on the concurrency requested across active campaigns. The API checks that capacity before accepting another campaign.

More concurrency can reduce waiting when enough independent work is ready. It cannot remove a dependency that must finish before later work can start. It also does not increase the processors or memory inside one trial. I would raise it when the work is ready to overlap and the account has capacity, then compare the elapsed time with the resulting evidence.

How do Iterations organize the work?

Iterations divides the requested task samples into numbered batches. Campaign assigns samples to those batches by their position in the requested set. The worker finishes a batch before starting the next, while trials within a batch can overlap within the concurrency limit. Mulberry carries the completed history into the next batch. The number of iterations cannot exceed the number of tasks.

This choice changes how much work can happen together and how often later work receives a completed history. More iterations make smaller batches when the task count stays fixed. Fewer iterations allow larger batches, but the concurrency limit still bounds active trials. I would compare the generation plan and the resulting tasks when choosing between more overlap and more opportunities to use earlier evidence.

How does Method guide a campaign?

Method chooses the rule Campaign follows when it generates tasks. Naive self play asks setters for independent candidates using the certified task pack. Mulberry first builds a taxonomy, which groups the kinds of work your campaign should cover. It assigns categories with coverage in mind and gives later iterations a history of earlier tasks and measurements. Both methods send their candidates through validation and solver attempts.

I would choose the method according to how you want the search to learn. Naive self play gives you independent proposals within the task setting. Mulberry can use earlier evidence to guide later proposals and avoid repeating the same work. Compare the tasks and their measured outcomes, because choosing a method does not guarantee useful difficulty.

What do Rollouts per task measure?

Rollouts per task is the number of solver attempts requested for each valid task. Each attempt runs in its own sandbox under the solver settings. The verifier scores the work produced by that attempt. Several attempts help show whether success is repeatable or uncertain. Only attempts with comparable measurements contribute to the reported solve rate.

A solve rate based on very few attempts can move sharply after one different outcome. More rollouts give you more evidence about that same task, while more tasks broaden the set of cases. Both increase the work the campaign requests. I would read the successful and measured attempt counts together before deciding that a small difference in solve rate matters.

How does Runtime image shape the environment?

The runtime image supplies the starting software environment for generated tasks. It can include an operating system layer, an interpreter, and installed tools. Campaign uses this image when preparing the environment in which generated tasks run. The task setting adds the files and dependencies needed for its work, while pack authoring uses its own environment. A missing tool in that environment can stop a task before the solver gets a fair attempt.

I would choose an image that suits the language and tooling in your INSTRUCTION.md. Its tag or digest identifies the image you request. An image digest identifies fixed image contents more precisely than a tag that can be updated. Keep the environment consistent when comparing solver results, and use the trial evidence to check whether failures came from the task or its software setup.

What does Sandbox backend provide?

The sandbox backend is the service that starts the computers used by trials. Campaign uses Modal for this job. Each trial receives a sandbox with the resources and network policy assigned to its role. The runtime image and task files provide the software inside that sandbox. The backend supplies the place where those commands run.

This field is fixed because the Campaign request currently accepts only Modal. You can still adjust the resources and limits provided to setter and solver trials. Those choices affect their working conditions within the supported backend. I would inspect a trial failure before changing its limits, because a provisioning failure and an agent running out of time describe different problems.

What does Setter network access allow?

Setter network access controls general internet access for task generation and repair sandboxes. The taxonomy trial also uses that policy. When access is available, those agents can fetch dependencies or consult external material from their sandbox. Model requests use a separate Campaign proxy and remain available when general internet access is disabled. Validators and agentic judges have their own network policy.

Choose the policy to match the work you want your setter to do. Research across external sources needs a different environment from work on a supplied repository alone. Pack authoring runs online and certification runs offline under their own rules. I would check which kind of trial failed before blaming this switch for a network error.

What does Solver network access allow?

Solver network access controls general internet access during an attempt at a task. An enabled solver can reach external services and fetch material from its sandbox. A disabled solver must work with what its environment already provides. Campaign still supplies model access through its separate proxy. This setting belongs to solver attempts, while validation and judging follow their own policies.

I would match this choice to the ability you want to measure. Looking up information is part of some tasks, while others should be solved from supplied evidence. Keep that choice consistent across attempts you intend to compare. If an offline attempt fails while installing dependencies, inspect the task environment before treating the failure as a reasoning result.

What does Task pack source choose?

Task pack source tells Campaign where its task setting comes from. Authoring starts with your INSTRUCTION.md and creates the code used to generate and grade tasks. That code must pass certification before generation proceeds. An existing source starts from a certified task pack that your account owns. It creates a new campaign using that pack rather than sending a new authoring prompt.

I would author a pack when you want to change the kind of work the campaign produces. Reusing a certified pack keeps its task setting while you try another campaign configuration. The setting can generate new task samples, so reusing a pack does not promise identical tasks. Check the source before launch, because it determines which instructions and generation rules the new campaign uses.

How does Tasks change the campaign?

Tasks is the number of task samples you ask Campaign to produce. Each sample starts with generation and can proceed through validation and solver attempts. A requested sample can finish measured, rejected, or unmeasured. Increasing this number therefore asks for more work without guaranteeing the same number of usable tasks. The final outcomes tell you how much of that request became evidence.

More task samples can expose a broader range of cases in your domain. Rollouts per task controls repeated attempts at each case, which answers a different question. I would start with enough tasks to inspect the generation quality before increasing the scale. Read the measured count alongside the requested count so unfinished or rejected work cannot look like coverage.

What does Action timeout limit for the setter?

Action timeout limits one command run by the setter, measured in seconds. A build, a test invocation, or a command that waits on a service can spend that allowance. Campaign passes this setting to the code that runs the agent's commands. It gives a stalled command a boundary instead of letting it wait indefinitely. The whole agent run still has its separate time limit.

A setter may need a long command to install dependencies or validate generated files. I would check whether a timed-out command was doing useful work or waiting on something unavailable. Raising this limit helps only when the command needs more time to finish. The setter also needs enough agent time left to read the result and complete the task.

What does Agent timeout limit for the setter?

Agent timeout is the time allowed for the setter's whole agent run, in seconds. That run includes model replies and the commands the agent performs while authoring and checking a task. The worker supplies this value to the agent execution policy. It differs from action timeout, which applies to one command. The campaign's overall deadline can also end work before this individual limit is reached.

I would allow enough time for the setter to inspect the environment and check its task. Reaching the limit tells you that the work did not finish under the chosen conditions. It does not establish whether the proposed task would have been useful with more time. Read the final commands and validation evidence before deciding whether to change the allowance or the instructions.

What does Context token budget bound for the setter?

Context token budget bounds token use over a setter run. Campaign checks both the growing conversation and cumulative usage reported across model responses. Reading earlier material again contributes to later requests, and reported usage includes generated output. This is different from counting only the unique words in a file or setting a cap on one reply. The runner checks this budget before requesting another response.

A setter can spend this budget while reading code, inspecting command output, and revisiting its own changes. Long histories can make later turns expensive even when each new reply is short. I would inspect repeated reads and noisy output before increasing the allowance. A larger budget permits more model work, while the campaign and agent deadlines still apply.

What do CPUs provide for the setter?

The CPUs setting gives each setter sandbox its allocation of processor cores. Those processors run commands such as tests, builds, and data preparation. Work that can use several cores may finish sooner with a larger allocation. A command that uses only one core may see little benefit. Campaign passes this resource request separately from the model settings.

For a setter, that compute supports building the task and checking that its environment works. Extra cores can help a parallel build without improving the ideas the model proposes. I would look for time spent in commands before increasing this value. Keep the machine resources in the campaign record when comparing the cost of producing tasks.

What does Max output tokens limit for the setter?

Max output tokens caps the length of one model response from the setter. Tokens are the pieces of text the model uses for its input and output. The worker sends this limit with the model's generation settings. It applies again to each response rather than granting one total allowance for the whole trial. The context token budget places a separate boundary on usage across the run.

A setter may need room for a detailed command or a complete piece of code in one response. A small output cap can cut that response short before it is useful. I would look for incomplete output in the evidence before increasing it. More room permits a longer response, while the agent still has to use that room to produce valid work.

What does Memory provide for the setter?

Memory is the working space available to a setter sandbox, measured in megabytes. Programs use it for data they need while running. A compiler or test process may need much more of it than a short shell command. Running out can make those programs fail before their work is complete. This allocation is separate from the model's context token budget.

For a setter, memory must support the environment and checks used while constructing a task. A failure during compilation or data loading may point to this resource rather than the task instructions. I would inspect the failing process before increasing the allocation. Giving every setter more memory changes the resources requested for each of those sandboxes.

What does Step limit count for the setter?

Step limit bounds the number of model turns in a setter run. A turn lets the model choose what to do next after reading the current conversation. That reply can contain commands whose results become part of the following input. Campaign counts the calls that ask the model for its next turn. A turn count and elapsed time measure different parts of the work.

A setter uses turns to inspect, construct, test, and revise a task. Too few can end that process while important checks remain. More turns allow further work, but they do not guarantee a better task. I would inspect whether the extra turns produce useful corrections and keep the other time and token limits in view.

What does Storage provide for the setter?

Storage is the disk resource requested for each setter sandbox, in megabytes. It holds files such as the task workspace, installed packages, and generated output. Files remain available to commands during that trial even when a process has finished. Working memory serves running programs, so it is a separate resource. Campaign passes the storage request into the sandbox configuration.

For a setter, the workspace can grow while it builds fixtures, installs tools, and checks the task. I would include those temporary files when considering the disk requirement. A larger storage request does not mean every file will appear in the evidence viewer. Campaign publishes only the files its evidence rules permit after the trial ends.

How does Temperature affect the setter?

Temperature changes how the model samples possible next tokens for the setter. Lower values concentrate sampling on choices the model already rates highly. Higher values spread that sampling across more alternatives. Campaign passes this setting with the model request. A low value can reduce variation without promising identical replies on every attempt.

For a setter, variation can change the task ideas and implementations it proposes. Higher temperature may broaden that search, but it does not guarantee useful diversity or valid tasks. I would read the generated examples and their validation outcomes together. Change the setting because it improves the work you observe, rather than treating randomness itself as quality.

What does Verifier timeout limit for the setter?

Verifier timeout bounds the grading work attached to a setter trial, in seconds. The worker passes it separately from the time allowed for the agent itself. A verifier may run checks after the agent has produced its work. Those checks need time to complete before the trial can yield trustworthy evidence. An unfinished check is not proof that the work passed.

For a setter, this limit belongs to the checks attached to generated work and the related trial policies. It is not an extra allowance for the setter to keep writing. I would inspect which check timed out and whether the environment was ready for it. Pack certification has its own checking limits, so increasing this field does not extend every check in the campaign.

What does Action timeout limit for the solver?

Action timeout limits one command run by the solver, measured in seconds. A build, a test invocation, or a command that waits on a service can spend that allowance. Campaign passes this setting to the code that runs the agent's commands. It gives a stalled command a boundary instead of letting it wait indefinitely. The whole agent run still has its separate time limit.

A solver may choose the right test command and still run into a limit that is too short for that environment. I would read the command and its output before changing the allowance. A longer action limit lets one command wait longer within the attempt's budget. Keep it consistent across comparisons so one solver is not given more time for the same operation.

What does Agent timeout limit for the solver?

Agent timeout is the time allowed for the solver's whole agent run, in seconds. That run includes model replies and the commands the agent performs while attempting a task. The worker supplies this value to the agent execution policy. It differs from action timeout, which applies to one command. The campaign's overall deadline can also end work before this individual limit is reached.

I would choose this limit according to the kind of solver performance you want to measure. A strict deadline makes completion speed part of the challenge. A longer allowance lets an attempt spend more time exploring and checking its work. Compare results under the same limit, and read timeout evidence before making a claim about the solver's capability.

What does Context token budget bound for the solver?

Context token budget bounds token use over a solver run. Campaign checks both the growing conversation and cumulative usage reported across model responses. Reading earlier material again contributes to later requests, and reported usage includes generated output. This is different from counting only the unique words in a file or setting a cap on one reply. The runner checks this budget before requesting another response.

A solver can spend this budget by reading the workspace and repeatedly carrying its conversation into later turns. A long attempt may therefore reach the cumulative limit before filling the model's maximum context window. I would compare token usage alongside elapsed time and step count. Keep the budget fixed when you want solver results to represent the same allowance for model work.

What do CPUs provide for the solver?

The CPUs setting gives each solver sandbox its allocation of processor cores. Those processors run commands such as tests, builds, and data preparation. Work that can use several cores may finish sooner with a larger allocation. A command that uses only one core may see little benefit. Campaign passes this resource request separately from the model settings.

For a solver, the machine is part of the conditions under which an attempt is measured. Extra cores may help a parallel test run without changing the model that chose the commands. I would keep this allocation consistent across solver comparisons. Inspect the command output and timings before treating a slow build as a reasoning failure.

What does Max output tokens limit for the solver?

Max output tokens caps the length of one model response from the solver. Tokens are the pieces of text the model uses for its input and output. The worker sends this limit with the model's generation settings. It applies again to each response rather than granting one total allowance for the whole trial. The context token budget places a separate boundary on usage across the run.

A solver may need a long response to express a command, a patch, or its final work. A response cut at the output limit can leave that step incomplete. I would check the recorded output before treating the result as a failure to understand the task. Use the same cap for attempts you want to compare under equal generation limits.

What does Memory provide for the solver?

Memory is the working space available to a solver sandbox, measured in megabytes. Programs use it for data they need while running. A compiler or test process may need much more of it than a short shell command. Running out can make those programs fail before their work is complete. This allocation is separate from the model's context token budget.

For a solver, memory is part of the environment in which the attempt is judged. A resource failure can prevent useful work even when the chosen approach was sound. I would inspect the evidence and compare allocations before interpreting the outcome. Keep the same memory allowance when you want attempts to represent the same working conditions.

What does Step limit count for the solver?

Step limit bounds the number of model turns in a solver run. A turn lets the model choose what to do next after reading the current conversation. That reply can contain commands whose results become part of the following input. Campaign counts the calls that ask the model for its next turn. A turn count and elapsed time measure different parts of the work.

A solver uses turns to inspect the task, choose an approach, and respond to command results. A low limit makes the number of decisions part of the challenge. More turns permit further exploration but still fit within the time and token budgets. I would compare the recorded turns as well as the final result when deciding whether the allowance suits your evaluation.

What does Storage provide for the solver?

Storage is the disk resource requested for each solver sandbox, in megabytes. It holds files such as the task workspace, installed packages, and generated output. Files remain available to commands during that trial even when a process has finished. Working memory serves running programs, so it is a separate resource. Campaign passes the storage request into the sandbox configuration.

For a solver, storage must accommodate the supplied workspace and the files produced during the attempt. Build outputs and downloaded dependencies can add to that footprint. I would inspect file-related failures before changing the task or model settings. The storage request concerns the trial workspace, while the evidence viewer follows its own rules about which files are retained and shown.

How does Temperature affect the solver?

Temperature changes how the model samples possible next tokens for the solver. Lower values concentrate sampling on choices the model already rates highly. Higher values spread that sampling across more alternatives. Campaign passes this setting with the model request. A low value can reduce variation without promising identical replies on every attempt.

For a solver, sampling can lead different attempts through different approaches to the same task. Repeated rollouts let you observe that spread under a chosen temperature. I would keep the setting fixed when comparing solve rates across tasks. A higher or lower value changes the conditions of the evaluation, so neither is automatically the more capable result.

What does Verifier timeout limit for the solver?

Verifier timeout bounds the grading work attached to a solver trial, in seconds. The worker passes it separately from the time allowed for the agent itself. A verifier may run checks after the agent has produced its work. Those checks need time to complete before the trial can yield trustworthy evidence. An unfinished check is not proof that the work passed.

For a solver, this gives grading time to inspect the attempt after its work is available. The worker also derives relevant judging limits from the solver's verifier allowance. I would read grading failures separately from evidence that the solver produced an incorrect answer. Give the checks enough time to finish, and keep that allowance consistent across the attempts you compare.

What are the campaign settings?

These settings shape the campaign as a whole. They describe the work you request and the conditions under which it should run. Campaign saves that configuration when you launch. The record then lets you see what governed the work you are reading.

I would choose these settings alongside your INSTRUCTION.md. Your account remembers choices to make the next campaign easier to prepare. You can hover a setting name here to learn what it controls. Once a campaign has started, a different configuration belongs to a new campaign.

What is the generated task difficulty mix?

The bar under this caption shows how the campaign's tasks divided by outcome, and it is built in your browser from the task rows the API returned. Measured tasks were attempted and graded, so each carries a solve rate. Rejected tasks were refused by the validator or by certification, so no solver was scored against them. Unmeasured tasks were cut short when the run stopped or timed out. The difficulty itself lives one level down, in each measured task's solve rate, which is the fraction of graded attempts that passed.

Difficulty is a reading of the pack, not a setting you choose, and the setter shapes it through the campaign's method. In a naive self-play run each iteration is written independently from the prompt, so the spread reflects the prompt directly. In a Mulberry run the setter tracks which categories are covered and lowers the number of solver successes it aims for as the run progresses, so later tasks lean harder on purpose. The valuable tasks sit in the frontier band, where the solver sometimes passes and sometimes fails, and the task matrix in the Atlas is where that spread is drawn task by task.

What are the example campaigns?

Example campaigns are finished runs that the team has marked as worth showing to everyone. The API keeps a separate list for them and only admits a campaign that ended normally, produced a certified pack, and measured at least one task, so an example is always a complete story with real results rather than a demonstration that stopped halfway. They are read with the same request as your own campaigns, which means every task, trial, and piece of evidence you can see on your own runs is open to you on these.

Read them the way you would read a worked solution before an exam. Each shows a real INSTRUCTION.md, the tasks a setter wrote from it, and how a solver fared, so you can calibrate how much detail a prompt needs and what a healthy spread of solve rates looks like. Nothing in an example can be edited or restarted from this page, but a good one is the fastest way to learn what to write before spending a campaign of your own.

What are the setter settings?

These settings govern the agents that generate and repair tasks. The worker also uses setter trial settings for related validation and taxonomy work. Those agents need room to inspect, construct, and check what they produce. Pack authoring and certification have additional policies of their own.

I would judge these choices by the quality of the generated tasks and their validation evidence. A resource limit can stop useful work before it is complete. More allowance gives the agent room to work without promising a better task. Hover a setting name to understand its role before deciding what to change.

What are the setter and solver trials?

Every task is built and tested by a small tree of trials, and the tabs on this page let you read each one. The setter trial is the model run that wrote the task: its instruction, its files, its verifier, and a solution that proves it can be passed. The validator trial then opens that task in a clean sandbox and checks that it is well formed, that the verifier accepts the oracle and rejects a corrupt or empty answer, and that nothing leaks. If the validator finds a problem the setter gets up to two chances to repair it, with full revalidation each time. Only a validated task moves on.

The solver trials are the independent attempts at the finished task, as many as you asked for in rollouts per task, run in parallel in fresh sandboxes. Each one is graded by the task's own verifier, and an optional judge trial can read an attempt and give a verdict when the task calls for it. The page reads these from the same evidence tree it uses for everything else. The setter and solver tabs share a shape, a log, a file list, and a transcript, so the two halves of self-play can be read side by side.

What are the solver settings?

These settings define the working conditions for solver attempts. Each attempt receives the configuration stored with the campaign. That makes the allowance part of what the resulting solve rate measures. The worker also uses solver settings for related agentic judging work.

I would keep these conditions consistent when comparing solver results. A time or resource limit can affect whether an otherwise useful approach finishes. Read the attempt and grading evidence alongside the final outcome. Hover a setting name to see what that particular allowance changes.

What is your campaign task history?

The task history lists every task this campaign has produced, one row per sample, read from the API in order. A task row is written only once its whole trial tree has finished and its evidence has been saved, so a row that appears here is complete and will not change. Each task carries a stable identifier derived from the campaign and its sample number, its outcome, its solve rate when it was measured, and the instruction and files the setter authored for it. The page loads rows in pages and keeps loading until the API says there are no more.

Selecting a row opens that task: its INSTRUCTION.md, its files, its stored evidence, and the trials that authored, validated, and attempted it. Read the task history as the campaign's report card. Early rows come from the first iteration and later rows from later ones, so you can see whether the setter's tasks grew sharper as the run went on. A task that was rejected still appears, with the reason in its evidence, because knowing what the validator refused is part of understanding what the setter learned to write.

What is your campaign history?

Your campaign history is the list of every campaign you have launched, newest first, read from the API each time the page opens. One request returns up to fifty of your most recent campaigns with their status, the counts of tasks measured so far, and the time stamps that show when each one started and ended. Every row is yours alone: the API looks up your identity from your key and only ever returns campaigns created under it. The title of each row is the first meaningful line of its INSTRUCTION.md.

Selecting a row opens that campaign in the main view, where the page reads its events, tasks, and evidence in turn. A running campaign keeps updating as new events arrive, and a finished one reads exactly as it did on the day it ended, because campaigns never change after launch. If you have more than fifty campaigns, the oldest fall off this list but still exist and still open from their own links. The example campaigns below your history are finished runs anyone can read, published to show what a good run looks like.

What are campaign trials?

Campaign trials are the trials that no task owns yet, read from the same live trial stream that feeds the rest of the page. They appear when a campaign does work before a task exists or while one is still being written: the certification trials that prove the task pack, the taxonomy trial of a Mulberry run, a setter trial whose task has not landed as a row yet, and any trial whose parent the page cannot place. The worker streams each trial's commands and their output as they happen, so a rollout here updates in place while a command runs, and the page keeps up to four thousand log lines per trial before it marks the rest as truncated.

Read them when a campaign is quiet or stalls before producing tasks, because that is where the story is. A certification that failed shows which gate refused the pack, a taxonomy trial shows the categories the model proposed, and a setter trial shows a task being authored in real time. Once a task's row is written, its trials move under that task and leave this list, so on a finished campaign only the campaign-level trials remain. Nothing here carries a solve rate, only the record of the work that made the tasks possible.

What is this task's evidence?

A task's evidence is the complete record of how it came to be and how it was attempted, saved by the worker when the task's trial tree finished. The page asks the API for the tree rooted at the task, which walks every trial from the setter that wrote it, through the validator that checked it, to each solver attempt and any judge that graded one. For every trial you can open its log stream, the list of files it produced, and its trajectory, which is the step by step transcript of what the model said and did. The API verifies the length and checksum of every stored file before serving it, so what you read is exactly what was saved.

Read the evidence when a number surprises you. A solve rate of zero with a reasonable instruction often turns out to be a broken verifier, and the solver trajectory shows the moment it went wrong. Private files never appear here: the task's own configuration, its grading module, its oracle solution, and any secret keys are held back so a reader cannot cheat the task or leak a credential. Very long text files are shown with their head and tail and a note about what was cut, and the page stops after four thousand log lines per trial with a marker that says more exist.

What is the campaign data?

Campaign data is a set of files you can download, each one assembled in your browser from the same API responses that draw the page. The set holds the campaign's immutable record, the exact create request it was launched from, which you can import back into the settings panel or replay, the task rows as JSON and as a spreadsheet table, every worker event with its payload, the trial snapshots and log rows this browser has retained, the Mulberry taxonomy when there is one, an inventory of the trials whose evidence is stored, and a small shell script that reads the whole campaign again with your own key.

Nothing is fetched specially for a download, so a file reflects exactly what you can see, and a finished campaign exports the same files every time. Use them to take results somewhere else, a notebook, a spreadsheet, or a report, or to launch a sibling campaign from the request file. The files contain the readable parts of the campaign and not the private ones, so grading modules and oracle solutions stay behind just as they do on the page. If you want another person to explore the campaign live rather than from a snapshot, share it instead.

What is the task matrix?

The task matrix lays out every task the Atlas has gathered as a grid, one cell per task, colored by its outcome and by its solve rate when it was measured. The Atlas builds it in your browser by asking the API for the tasks of each campaign in turn, your own campaigns first and then the examples, a few campaigns at a time so the page stays responsive. No new numbers are invented for the matrix. Each cell shows exactly the outcome the worker recorded for that task.

Read it as a map of difficulty. A band of mixed cells is a campaign that found the frontier, where the solver sometimes passes and sometimes fails. A row of uniform cells, all pass or all fail, is a campaign that missed it, either because the prompt asked for something too easy or because the tasks were too hard or broken. Hovering a cell names its task and campaign, and selecting one opens the task so you can read its instruction and evidence.

What is the workload analysis?

The workload analysis is the bar that shows how the campaign's tasks divided into their three outcomes. The page counts the task rows the API has returned and shows how many were measured, how many the validator or certification rejected, and how many were left unmeasured because the run stopped or timed out first. The bar grows as task rows arrive, so on a running campaign it fills from left to right while the worker finishes each sample.

The shape tells you where the run's effort went. A bar that is mostly measured means the setter wrote sound tasks and the solver was scored against nearly all of them. A large rejected share means the setter struggled to write tasks that passed validation, which usually points back to the prompt. An unmeasured share means compute ran out before the pack was finished, so consider a longer timeout or fewer tasks next time. It is a budget view, not a quality view: for quality, read the solve rates in the summary and the matrix.

What is reproducibility?

Reproducibility means another person could run the same campaign and expect the same kind of result. The API makes that possible by freezing everything that shaped the run: the INSTRUCTION.md, the settings, the runtime image, and the runtime pins that name the exact model versions and worker code the campaign used. The random seed is derived from the campaign's own identifier, so any randomness in the run is tied to it. This section shows the two records that produced the workload, the trial configuration every sandbox ran under and the activity the campaign agent logged along the way.

Two things are honest limits. Language models are not perfectly deterministic, so a rerun writes similar tasks rather than identical ones. And a campaign cannot be resumed after it stops, because the worker only ever claims a queued run with no work pinned to it. What you can do is restart from the certified task pack: the API accepts the pack's identifier, checks it byte for byte against what was certified, and measures a solver against the very same tasks. That is the reproducible half, and it is the half that makes a benchmark trustworthy.

What is sharing a campaign?

Sharing a campaign creates a link that lets another Vmax user read it. Only a finished campaign can be shared, and the API refuses to share one that is still running. When you share, the API mints a share token, stores only a hash of its secret, and hands the full token to you once. A campaign has one live share at a time, and you may set an expiry of seven, thirty, or ninety days, or none. Rotate replaces the token with a new one and Revoke ends sharing, and a link that has been rotated or revoked stops working at once.

The reader still signs in with their own Vmax key, because the API checks identity before it even looks up the token, and their reads are rate limited per person and per share so a link cannot be hammered. What they see is the campaign's public projection: the tasks and their outcomes, the evidence a solver was allowed to see, and the counts and time stamps, with the title reduced to the method's name and your identity left out. Grading modules, oracle solutions, and secrets never cross the link.

What is Create paper?

Create paper starts a new post on Vmax and opens it in the editor. Pressing it makes one request to the Vmax backend to create a post in the world markdown format, saved as markdown, then takes you straight to the editor with that post open. The post belongs to you from the first moment, so you can leave and come back to it from your profile.

A paper is the document form of the site: text, figures, and equations rendered on a page, with a three dimensional scene that reacts to what you write. The scene is chosen from the content, so a paper with many images grows grass and a paper with equations raises a castle. Write it the way you would write for a reader, and the world takes shape around the words.

What is the certified task pack?

A certified task pack is the set of tasks a campaign produced, proven by the worker to be sound before any solver was measured against them. Certification runs in an offline sandbox and passes each task through a series of gates: the code imports, the task has the right shape and contract, the environment starts, the oracle solution passes the verifier, a corrupt or empty answer fails it, and malformed or adversarial inputs are refused. The pack carries a few files of its own, such as the grader module, calibration notes, and the validator's report, which you can read here.

The pack is what makes a campaign reusable. When you launch a new campaign from an existing pack, the API checks the pack belongs to you and the worker certifies it again, byte for byte against what was recorded, before measuring a solver on the same tasks. That is how the same benchmark can score two solvers fairly. Only a campaign that ended normally with a certified pack can be restarted from, which is also the rule for becoming an example campaign.

What does a complete campaign mean?

A campaign is complete when the worker has taken every sample to a terminal state, saved all of its evidence, and written the final event that closes the run. At that point the API marks the campaign done and records the time it ended. Every task row exists, every solve rate is final, and the certified task pack, if the run produced one, is ready to be restarted from. Nothing will change again, so the page reads a completed campaign the same way forever.

Complete does not mean every task was measured. A task can end rejected when validation refused it, and that still counts as finished work. Read the summary to see how the outcomes divided and the matrix to see where the solve rates landed. A complete campaign with a healthy frontier band is the product working as designed: a domain you described, turned into a pack of real tasks, each with a measured level of difficulty.

What is a shared campaign?

A shared campaign is a finished campaign someone opened to you with a share link. The page reads it through the API's public routes, which look up the token from the link after confirming your own key, and return a public projection of the campaign: its tasks with their outcomes and solve rates, the evidence a solver was allowed to see, the summary counts, and the time stamps. The owner's name and key are absent, and the title is reduced to the method that ran, so what you see is the work rather than the person.

You can read it the way the owner does, opening tasks, instructions, files, and solver trajectories, with the same private material held back. Files come with a header that says they were served through the redaction rules, so nothing here could reveal a grader or a secret. If the link stops working, the owner rotated or revoked it, or it expired. Ask them for a new one. Your reads are rate limited so a shared campaign stays available to everyone it was opened to.

What is a smoke trial?

A smoke trial is a quick, deliberately small run used to check that a task setting works end to end before a real campaign spends compute on it. It builds the environment, runs a short attempt, and grades it, the way you might switch a machine on and watch for smoke before trusting it with a full day's work. The tasks and trials it produces are marked as smoke so they are never mistaken for measured results.

On the live campaign page you will not usually see one. The rows the worker writes for a real run are never marked smoke, and the smoke marking exists for fixtures and internal checks of the console itself. If a row does carry the mark, read it as a rehearsal: useful for confirming the plumbing, and not a solve rate to build on.

What is the task instruction?

A task's instruction is the INSTRUCTION.md the setter wrote for that one task, the text a solver reads at the start of every attempt. The page fetches it from the task's stored evidence, where the worker saved it beside the task's files once the task was complete. It describes the situation, what the solver must produce, and how it will be judged, in the setter's own words. Because the same file is given to every solver attempt, it is the fairest place to look when you wonder whether a solve rate reflects the solver or the task.

Think of it as the exam question, separate from the exam materials and the answer key. The files beside it are the materials. The verifier that grades an attempt, and the oracle solution the setter used to prove the task could be solved, are kept private so a reader cannot game the task. A good instruction is specific about the goal and quiet about the method, which leaves the solver room to think while still being gradeable. If an instruction reads as vague, that is usually a sign the campaign prompt was vague too.

What happens after Run campaign?

When you press Run campaign, the client sends one request to the API with your INSTRUCTION.md and settings. Inside a single database transaction the API checks the request against its schema, confirms you have room for another run, and writes the campaign as queued with its first event. The capacity rule is simple: two active campaigns per person, and no more than sixty four requested concurrent trials across them, so a third launch is refused with a clear message rather than silently waiting. If a campaign restarts from an existing certified pack, the API also checks that the pack is real and belongs to you.

Only after that record is safely committed does the API try to wake the compute worker, with a short timeout and one retry. That start is best effort on purpose: even if the worker does not answer right away, you already own a queued campaign, and the worker claims queued work when it comes online. The client never resends a create request, so a slow first launch cannot become two campaigns. Once the API answers, the page moves you straight into the live view, where the first events arrive as the worker picks the run up.

What is the refined prompt?

Refine sends your draft to the API, which makes one call to a language model with a fixed set of instructions and hands back a cleaner INSTRUCTION.md. The model is asked to keep everything you meant and to organize it under five headings: Objective, Task contract, Allowed inputs, Grading and acceptance, and Exclusions. Where your draft is silent, it writes Not specified rather than inventing a requirement, and it strips out anything that belongs to the settings panel, such as task counts or difficulty targets. The call has a budget of a couple of minutes and a bounded reply length, so it finishes in one round.

Nothing is created or changed by Refine. The API stores no draft, starts no campaign, and the refined text simply replaces what is in your box for you to read and edit. If the model service is not configured on the deployment you are using, or the provider fails, you get a plain error and your draft stays where it was. Treat the refined prompt as a good first edit, not a final answer: the setter will read exactly what you launch, so the last look is always yours.

What are the Atlas errors?

The Errors view gathers every error the Atlas can see into one table, from two places. The first is the campaigns themselves: a campaign the worker closed as failed, with the reason it wrote into the record, and every task the validator or certification rejected, with the reason stored on its row. The second is the trip between this page and the API: every answer the client did not expect, recorded in the same ledger each campaign page keeps, plus any campaign whose tasks the Atlas could not read. Each row names when it happened, which campaign it belongs to, what the page was doing, the route when there was one, what came back, and what the contract promised instead.

Read the source column first. A campaign error is the run reporting on itself, and the fix lives in the prompt, the settings, or the task the row opens. An API error is about the machinery, and a repeated one on the same route is worth copying to whoever runs the deployment. The list refreshes as the Atlas reads more campaigns and as new answers arrive, and a repeated answer bumps a count rather than adding a row, so the table stays short enough to scan. An empty table means every campaign read cleanly and every answer matched what the client expected.

What is the Atlas summary?

The Atlas summary counts what the Atlas has gathered so far: how many campaigns it has read, how many tasks they contain, and how those tasks divide into measured, rejected, and unmeasured outcomes. The counts come straight from the task rows the API returned for each campaign, and they grow while the Atlas is still reading, so the numbers settle once every campaign has answered.

Read it as a headline for the matrix beneath. A large measured share means most of the tasks across your campaigns carry real solve rates. A large rejected or unmeasured share means many runs were cut short or fought their validators, which is a reason to revisit prompts and timeouts. Counts here are of tasks, not attempts, so a task with eight rollouts counts once.

What is the campaign agent activity?

The activity feed is the campaign's own diary, written by the worker as it runs and read back by the page as a stream of events. Each event has a sequence number, so the page asks the API only for events after the last one it has seen, in pages of up to five hundred, and nothing is ever missed or shown twice. The worker records when the campaign was created, when it was claimed, when a task was written and validated, when each solver attempt began and ended, and when the run reached its terminal state. Pressing Stop adds a stop requested event to that same stream.

Reading the feed tells you what the campaign is doing right now and why. A quiet stretch usually means sandboxes are still starting or a long attempt is in progress, and the page says so when the first read takes a while on an idle deployment. The feed is the most honest surface on the page, because it is the same log the worker itself relies on: if something went wrong, the event that describes it is here, in order, with its time.

What are campaign client errors?

Client errors are the problems this page ran into while talking to the API, recorded in your browser and listed here so they are not lost. Each entry names the request that failed, the route and status that came back, the message, what the client expected instead, when it first and last happened, and how many times, with repeats of the same error folded into one row. Common causes are a proxy that timed out while the Campaign deployment was waking up, a response the client refused to decode because it did not match the contract, or a request refused for capacity. The client fails closed on purpose: when a response does not match what it expects, it reports the error rather than guessing.

Nothing in this list came from the worker or from your tasks. Those problems live in the activity feed and the evidence. Read a client error as a note about the trip between the page and the API. The first read on an idle deployment can take tens of seconds, and the page says so, but a repeated failure on the same route is worth reporting with the entry copied here, because it points at exactly the request that did not work.

What is your campaign summary?

The summary is a small set of numbers the page computes from the tasks the API has stored for this campaign. Each task ends in one of three outcomes. Measured means the solver attempted it and the attempts were graded, so it carries a real solve rate. Rejected means the validator or certification decided the task was not sound, so no solver was ever scored against it. Unmeasured means the campaign stopped or timed out before the task reached a final evaluation. The summary counts those outcomes and shows how far the run has come against the task count you asked for.

A solve rate is the fraction of graded solver attempts that passed, with infrastructure failures left out of the denominator so a crashed sandbox never counts as a wrong answer. That is why the number of attempts matters: more rollouts per task make each solve rate a steadier estimate. The most useful campaigns collect their tasks in the frontier band, where the solver passes some attempts and fails others, because that is where a task teaches something. A pack full of perfect scores was too easy, and a pack full of zeros was too hard or too broken to learn from.

What is the Debug Paper?

The Debug Paper is a sample post rendered inside the tools shell, so the document system can be looked at without visiting a real paper. It uses the same markdown pipeline, the same components, and the same scene selection as a published post, which means what you see here is exactly what a reader would see, only with the tool's controls around it.

It is useful when a paper renders oddly. Because the content is fixed and known, a difference between this page and a real post points at the post, and a difference between this page and yesterday points at the code. It is also a gentle way to learn how headings, images, equations, and quotes shape the world around a paper before writing one of your own.

What are the Debug Vmax Tools?

The Debug Vmax Tools are the internal pages of the tools shell, listed from a single registry in the code so every page appears in the sidebar and at its own address. Each row names a page and the surface it exercises, from the campaign console to the island catalog to this help codex. They are called debug tools because they exist to let the team look at one part of the product at a time, with the same components the product itself uses.

Nothing here is hidden from you on purpose. It is simply the workshop rather than the showroom. If a page in the product misbehaves, the matching tool is where it can be reproduced with less around it. Browse them to understand how Vmax is put together, and use the help card on each to learn what it shows.

What is the campaign table of contents?

The table of contents is the list of sections on this campaign page, in the order they appear, each one a link that scrolls you to it. The page watches its own sections as they render and rebuilds the list whenever one appears, changes, or goes away, so the outline is a faithful map of this campaign rather than a fixed menu. A section that has something to count, such as the task history or the workload, shows its count beside its name.

Use it to jump rather than scroll. A campaign page can be long once every task row has loaded, and the outline is the quickest way back to the summary or forward to the evidence of a single task. Each heading matches the section it opens, and every section carries its own help card, so the outline is also a map of what the page can explain.

What is the task setting implementation?

The task setting implementation is the code that turns a task from a description into something a sandbox can run and grade. For every task the setter writes a task module that builds the environment, a grader module that scores an attempt, and a small set of support files, and the worker certifies that code before any solver sees it. Certification runs a series of gates in an offline sandbox: the modules import cleanly, the task has the expected shape and contract, the environment starts, the oracle solution passes, a corrupt or empty answer fails, and adversarial and malformed inputs are refused. The worker allows a few attempts, with repair feedback between them.

You can read the pack's grader module and validator notes under the certified task pack, while each task's own private grading files stay hidden so the task cannot be gamed. This is the layer that makes a solve rate mean something. A verifier that accepts nonsense would make every task look easy, and one that rejects the right answer would make every task look impossible, so proving the verifier before measuring anyone is what keeps the numbers honest.

What is the taxonomy?

In a Mulberry campaign the setter does not write tasks straight from the prompt. It first runs a taxonomy trial that reads your INSTRUCTION.md and proposes between six and twelve categories, each with a key, a label, and a short description, and the worker accepts the plan only when that trial resolves cleanly, allowing it two attempts. The taxonomy is saved with the campaign and shown here before any task lands, so you can see how the setter understood your domain before it spent compute on it.

From then on the run is sequential. For each sample the setter picks the category with the fewest tasks so far, aims for a number of solver successes that falls as the run progresses, and consults a history of what it has already written, so later tasks lean harder and cover new ground. Read the taxonomy as the setter's outline of your subject. If a category looks wrong, the prompt is the place to fix it, and a naive self-play run is the alternative when you would rather have every iteration written independently.

What is the trial configuration?

The trial configuration table shows the resource limits every trial in this campaign ran under, exactly as they were stored with the campaign when it was created. There are two rows of settings, one for the setter and one for the solver, and each names the sandbox's processors, memory, disk, time limits, step limit, reply length, temperature, and context budget. The worker reads these values when it starts each sandbox, so the table is a record of what actually ran rather than a suggestion.

The setter's settings do more than author tasks: the same limits also govern the validator, the repair trials, the taxonomy trial, and the certification sandbox, because all of those are setup work rather than attempts. The solver's settings govern every attempt and any agentic judge. Reading the table beside a surprising result often explains it: a short timeout measures speed rather than skill, and a small step limit can stop a solver that was on the right track. Change a limit by launching a new campaign, since a running one is fixed.

What are the World Examples?

The World Examples are the catalog of scenes the Vmax engine can build, each one shown on its own island so you can study a single feature in isolation. Every entry comes from a preset in the code that names the island's type, its structures, and any behaviour running on it, and the page renders each preset with the same stage the rest of the site uses. The home page's full archipelago is listed here too, rendered through its own world stage exactly as it appears on the front page.

Read the catalog to learn what a paper can summon. A scene's bracketed label names its content and, where there is one, its mode: a castle, a forest, an erupting volcano, a fleet of raiding lumberjacks. Some scenes are triggered by what a paper contains and others exist only as presets for testing. Either way, what you see here is the real engine, so a scene that looks right in the catalog looks right on a paper.