Find the hardest tasks. Train, then find harder tasks.

Self-play at your fingertips, repeated. Campaign does the finding with our proprietary technology, building verifiable tasks from your data, measuring them, and marking the ones worth training on, over and over again.

Read the overview →

One campaign, from the first page to the frontier

  1. Describe the work once.

    Every campaign starts from one INSTRUCTION.md: the domain, what a good task looks like, the deliverable a solver must produce, how it is checked, and what to reject. It is the only input the run has, and every task that follows descends from it. This one asks for consensus bugs in a pinned Raft library, each one reproduced by a seeded simulation of crashes, partitions, and reordered messages.

  2. The rules come before the game.

    Before a single task exists, the campaign agent turns that page into a task setting: a small package of code that tells the setter how to build a task, what each environment contains, and how a verifier decides a reward.

  3. Proven before it scores anything.

    The task setting is checked against known answers before it is trusted to grade anything. Once it passes, the campaign records the certified task pack and a checksum of its contents, so every reward in the run traces back to code that was proven first.

  4. Coverage is planned, not left to chance.

    This campaign runs the Mulberry method, so a taxonomy trial first agrees a plan of seven categories for the domain, from leader election to storage recovery. Each new task is sampled against the least-covered category, so a long run spreads across the whole subject instead of circling one corner of it.

  5. Every task arrives with its own test.

    Self-play has two sides. On the first, a setter model writes one concrete task: the instruction a solver is handed, an environment with the repository and its dependencies, and a verifier that reads what an attempt produced and returns pass or fail.

  6. Unfair tasks never reach the scoreboard.

    A validator checks every authored task against the campaign’s acceptance rules before it can count. The second task’s hidden test fails on some seeds even with a correct fix, so it cannot tell right from wrong, and it is rejected. A rejected task is never measured, never counted, and never spends a single attempt.

  7. Eight attempts, each in its own sandbox.

    On the other side, a solver model attempts the task eight times, each rollout in its own sandbox under the same limits. The task’s own verifier scores every attempt pass or fail, and an attempt that could not be measured gets no reward rather than a guessed one, so the solve rate counts only what was checked.

  8. Mixed results are the signal.

    This task passed three of its eight attempts, so it lands in the frontier band, the tasks a model solves 12% to 50% of the time, and it lights up. Its attempts hold both successes and failures, the contrast reinforcement learning learns from. A task passed every time, or never, gives training nothing to compare.

  9. Then it keeps going, unattended.

    A long-horizon campaign runs iteration after iteration inside the time and attempt budget you set; here, ten batches of ten tasks. Each batch finishes before the next begins, trials inside a batch run side by side up to the concurrency you chose, and nothing needs you while it works.

  10. Later batches lean harder on purpose.

    Mulberry hands each new batch the history of the work before it and lowers its target solve rate as the run goes on. Watch the passes thin out: the solver cleared 54% of its attempts in the first batch and 18% in the last, while the frontier kept filling in.

  11. A day of work, and all of it evidence.

    After 22 hours and 37 minutes of self-play, inside the 23.5-hour compute budget that is the most one campaign allows, the run measured 94 tasks through 752 graded attempts, rejected six it could not grade fairly, and marked 81 in the frontier band. Every task keeps its instruction, attempts, files, and scores, so any result traces back to what produced it.

  12. Train on it. Raise the bar. Run again.

    Export the frontier tasks with their full instructions and measured solve rates, and train your model on them in your own pipeline. When your model outgrows them, raise the bar in the INSTRUCTION.md and launch the next campaign to find harder tasks, round after round.

Read the whole day on one chart.

The same campaign in the Campaign Conclusion Timeseries: tasks started and concluded, the solve rate falling as later batches lean harder, and the share of measured tasks in the frontier band across nearly a day. Drag across the chart to zoom into any hour.

100 tasks / 955 trials
Interval conclusion22h 38m
0%25%50%75%100%0s5h 39m11h 19m16h 58m22h 38mCampaign execution startedTasksday-task-0001 concluded at 2h 27m: 3 of 8 rollouts passed, a frontier taskday-task-0003 concluded at 2h 42m: 5 of 8 rollouts passedday-task-0004 concluded at 2h 49m: 4 of 8 rollouts passed, a frontier taskday-task-0005 concluded at 2h 54m: 2 of 8 rollouts passed, a frontier taskday-task-0006 concluded at 2h 53m: 6 of 8 rollouts passedday-task-0007 concluded at 2h 59m: 4 of 8 rollouts passed, a frontier taskday-task-0008 concluded at 2h 58m: 3 of 8 rollouts passed, a frontier taskday-task-0009 concluded at 3h 3m: 8 of 8 rollouts passedday-task-0010 concluded at 3h 10m: 4 of 8 rollouts passed, a frontier taskday-task-0011 concluded at 4h 43m: 4 of 8 rollouts passed, a frontier taskday-task-0012 concluded at 4h 49m: 3 of 8 rollouts passed, a frontier taskday-task-0013 concluded at 4h 55m: 5 of 8 rollouts passedday-task-0014 concluded at 4h 54m: 2 of 8 rollouts passed, a frontier taskday-task-0015 concluded at 5h 3m: 4 of 8 rollouts passed, a frontier taskday-task-0016 concluded at 5h 6m: 3 of 8 rollouts passed, a frontier taskday-task-0017 concluded at 5h 4m: 7 of 8 rollouts passedday-task-0018 concluded at 5h 14m: 4 of 8 rollouts passed, a frontier taskday-task-0019 concluded at 5h 16m: 2 of 8 rollouts passed, a frontier taskday-task-0020 concluded at 5h 15m: 3 of 8 rollouts passed, a frontier taskday-task-0021 concluded at 6h 59m: 3 of 8 rollouts passed, a frontier taskday-task-0022 concluded at 6h 55m: 4 of 8 rollouts passed, a frontier taskday-task-0023 concluded at 7h 0m: 2 of 8 rollouts passed, a frontier taskday-task-0025 concluded at 7h 5m: 4 of 8 rollouts passed, a frontier taskday-task-0026 concluded at 7h 22m: 3 of 8 rollouts passed, a frontier taskday-task-0027 concluded at 7h 20m: 5 of 8 rollouts passedday-task-0028 concluded at 7h 16m: 2 of 8 rollouts passed, a frontier taskday-task-0029 concluded at 7h 25m: 4 of 8 rollouts passed, a frontier taskday-task-0030 concluded at 7h 31m: 3 of 8 rollouts passed, a frontier taskday-task-0031 concluded at 9h 1m: 2 of 8 rollouts passed, a frontier taskday-task-0032 concluded at 9h 11m: 3 of 8 rollouts passed, a frontier taskday-task-0033 concluded at 9h 16m: 4 of 8 rollouts passed, a frontier taskday-task-0034 concluded at 9h 12m: 3 of 8 rollouts passed, a frontier taskday-task-0035 concluded at 9h 21m: 2 of 8 rollouts passed, a frontier taskday-task-0036 concluded at 9h 20m: 6 of 8 rollouts passedday-task-0037 concluded at 9h 33m: 3 of 8 rollouts passed, a frontier taskday-task-0038 concluded at 9h 32m: 1 of 8 rollouts passed, a frontier taskday-task-0039 concluded at 9h 30m: 4 of 8 rollouts passed, a frontier taskday-task-0040 concluded at 9h 44m: 2 of 8 rollouts passed, a frontier taskday-task-0041 concluded at 11h 17m: 3 of 8 rollouts passed, a frontier taskday-task-0042 concluded at 11h 16m: 2 of 8 rollouts passed, a frontier taskday-task-0044 concluded at 11h 28m: 3 of 8 rollouts passed, a frontier taskday-task-0045 concluded at 11h 26m: 4 of 8 rollouts passed, a frontier taskday-task-0046 concluded at 11h 33m: 2 of 8 rollouts passed, a frontier taskday-task-0047 concluded at 11h 38m: 1 of 8 rollouts passed, a frontier taskday-task-0048 concluded at 11h 48m: 3 of 8 rollouts passed, a frontier taskday-task-0049 concluded at 11h 43m: 5 of 8 rollouts passedday-task-0050 concluded at 11h 42m: 2 of 8 rollouts passed, a frontier taskday-task-0051 concluded at 13h 33m: 2 of 8 rollouts passed, a frontier taskday-task-0052 concluded at 13h 29m: 3 of 8 rollouts passed, a frontier taskday-task-0053 concluded at 13h 27m: 1 of 8 rollouts passed, a frontier taskday-task-0054 concluded at 13h 44m: 2 of 8 rollouts passed, a frontier taskday-task-0055 concluded at 13h 39m: 4 of 8 rollouts passed, a frontier taskday-task-0056 concluded at 13h 38m: 2 of 8 rollouts passed, a frontier taskday-task-0057 concluded at 13h 47m: 0 of 8 rollouts passedday-task-0058 concluded at 13h 50m: 3 of 8 rollouts passed, a frontier taskday-task-0059 concluded at 13h 59m: 2 of 8 rollouts passed, a frontier taskday-task-0060 concluded at 13h 58m: 1 of 8 rollouts passed, a frontier taskday-task-0061 concluded at 15h 35m: 2 of 8 rollouts passed, a frontier taskday-task-0062 concluded at 15h 45m: 1 of 8 rollouts passed, a frontier taskday-task-0063 concluded at 15h 43m: 3 of 8 rollouts passed, a frontier taskday-task-0064 concluded at 15h 39m: 2 of 8 rollouts passed, a frontier taskday-task-0066 concluded at 15h 54m: 1 of 8 rollouts passed, a frontier taskday-task-0067 concluded at 15h 49m: 2 of 8 rollouts passed, a frontier taskday-task-0068 concluded at 16h 6m: 3 of 8 rollouts passed, a frontier taskday-task-0069 concluded at 16h 4m: 0 of 8 rollouts passedday-task-0070 concluded at 16h 11m: 2 of 8 rollouts passed, a frontier taskday-task-0071 concluded at 17h 44m: 1 of 8 rollouts passed, a frontier taskday-task-0072 concluded at 17h 50m: 2 of 8 rollouts passed, a frontier taskday-task-0073 concluded at 17h 56m: 3 of 8 rollouts passed, a frontier taskday-task-0074 concluded at 17h 55m: 1 of 8 rollouts passed, a frontier taskday-task-0075 concluded at 18h 0m: 2 of 8 rollouts passed, a frontier taskday-task-0076 concluded at 18h 7m: 0 of 8 rollouts passedday-task-0077 concluded at 18h 5m: 2 of 8 rollouts passed, a frontier taskday-task-0078 concluded at 18h 4m: 1 of 8 rollouts passed, a frontier taskday-task-0079 concluded at 18h 17m: 3 of 8 rollouts passed, a frontier taskday-task-0080 concluded at 18h 16m: 2 of 8 rollouts passed, a frontier taskday-task-0081 concluded at 20h 0m: 2 of 8 rollouts passed, a frontier taskday-task-0082 concluded at 20h 3m: 1 of 8 rollouts passed, a frontier taskday-task-0084 concluded at 20h 11m: 2 of 8 rollouts passed, a frontier taskday-task-0085 concluded at 20h 6m: 1 of 8 rollouts passed, a frontier taskday-task-0086 concluded at 20h 12m: 3 of 8 rollouts passed, a frontier taskday-task-0087 concluded at 20h 21m: 1 of 8 rollouts passed, a frontier taskday-task-0088 concluded at 20h 17m: 0 of 8 rollouts passedday-task-0089 concluded at 20h 22m: 2 of 8 rollouts passed, a frontier taskday-task-0090 concluded at 20h 32m: 1 of 8 rollouts passed, a frontier taskday-task-0091 concluded at 22h 2m: 1 of 8 rollouts passed, a frontier taskday-task-0092 concluded at 22h 12m: 2 of 8 rollouts passed, a frontier taskday-task-0093 concluded at 22h 17m: 1 of 8 rollouts passed, a frontier taskday-task-0094 concluded at 22h 13m: 0 of 8 rollouts passedday-task-0095 concluded at 22h 22m: 2 of 8 rollouts passed, a frontier taskday-task-0096 concluded at 22h 28m: 1 of 8 rollouts passed, a frontier taskday-task-0098 concluded at 22h 33m: 3 of 8 rollouts passed, a frontier taskday-task-0099 concluded at 22h 31m: 1 of 8 rollouts passed, a frontier taskday-task-0100 concluded at 22h 34m: 2 of 8 rollouts passed, a frontier task
No recorded work at this time
1–40 / 100
  1. Iteration 1, measuredday-task-000159m 54s3 of 8 passedFrontier
  2. Iteration 1, rejectedday-task-00026m 54sNot measured
  3. Iteration 1, measuredday-task-00031h 4m5 of 8 passed
  4. Iteration 1, measuredday-task-00041h 9m4 of 8 passedFrontier
  5. Iteration 1, measuredday-task-00051h 6m2 of 8 passedFrontier
  6. Iteration 1, measuredday-task-00061h 3m6 of 8 passed
  7. Iteration 1, measuredday-task-00071h 8m4 of 8 passedFrontier
  8. Iteration 1, measuredday-task-00081h 5m3 of 8 passedFrontier
  9. Iteration 1, measuredday-task-00091h 2m8 of 8 passed
  10. Iteration 1, measuredday-task-00101h 7m4 of 8 passedFrontier
  11. Iteration 2, measuredday-task-00111h 4m4 of 8 passedFrontier
  12. Iteration 2, measuredday-task-00121h 1m3 of 8 passedFrontier
  13. Iteration 2, measuredday-task-00131h 6m5 of 8 passed
  14. Iteration 2, measuredday-task-00141h 3m2 of 8 passedFrontier
  15. Iteration 2, measuredday-task-00151h 11m4 of 8 passedFrontier
  16. Iteration 2, measuredday-task-00161h 5m3 of 8 passedFrontier
  17. Iteration 2, measuredday-task-00171h 2m7 of 8 passed
  18. Iteration 2, measuredday-task-00181h 10m4 of 8 passedFrontier
  19. Iteration 2, measuredday-task-00191h 4m2 of 8 passedFrontier
  20. Iteration 2, measuredday-task-00201h 1m3 of 8 passedFrontier
  21. Iteration 3, measuredday-task-00211h 9m3 of 8 passedFrontier
  22. Iteration 3, measuredday-task-00221h 3m4 of 8 passedFrontier
  23. Iteration 3, measuredday-task-00231h 0m2 of 8 passedFrontier
  24. Iteration 3, rejectedday-task-00247m 54sNot measured
  25. Iteration 3, measuredday-task-00251h 2m4 of 8 passedFrontier
  26. Iteration 3, measuredday-task-00261h 10m3 of 8 passedFrontier
  27. Iteration 3, measuredday-task-00271h 7m5 of 8 passed
  28. Iteration 3, measuredday-task-00281h 1m2 of 8 passedFrontier
  29. Iteration 3, measuredday-task-00291h 9m4 of 8 passedFrontier
  30. Iteration 3, measuredday-task-00301h 6m3 of 8 passedFrontier
  31. Iteration 4, measuredday-task-00311h 0m2 of 8 passedFrontier
  32. Iteration 4, measuredday-task-00321h 8m3 of 8 passedFrontier
  33. Iteration 4, measuredday-task-00331h 5m4 of 8 passedFrontier
  34. Iteration 4, measuredday-task-003459m 54s3 of 8 passedFrontier
  35. Iteration 4, measuredday-task-00351h 7m2 of 8 passedFrontier
  36. Iteration 4, measuredday-task-00361h 4m6 of 8 passed
  37. Iteration 4, measuredday-task-00371h 9m3 of 8 passedFrontier
  38. Iteration 4, measuredday-task-00381h 6m1 of 8 passedFrontier
  39. Iteration 4, measuredday-task-00391h 3m4 of 8 passedFrontier
  40. Iteration 4, measuredday-task-00401h 8m2 of 8 passedFrontier
cmp_example_day_long_run

Start the loop with one page of instructions.

Find the hard tasks, train on them, then find harder ones. Campaign is open to approved Vmax accounts: join the waitlist for early access, or sign in if you already have one.

Read the overview →