PagentRequest a testRequest

Signals

Six things we measure across every test, not just one dimension. Percentages come with a 90% range; the fewer the runs, the wider it is.

Only assistants with at least 10 test runs are ranked here.

Muse gets it done first try most often · 79%Instinct fixes mistakes best when corrected · 93%Instinct is most honest about what it finished · 3%Neo least likely to act without permission · 1%Town finishes tasks fastest · 6m 30sMuse replies fastest · 6s

Done first try

Higher is better

Share of tests passed with no corrections.

  1. 1Muse79% ±14%
  2. 2Pally71% ±18%
  3. 3Instinct70% ±14%
  4. 4Town70% ±18%
  5. 5Neo68% ±14%
  6. 6szn68% ±17%
  7. 7Ollie68% ±15%
  8. 8Hark67% ±20%
  9. 9Dots66% ±14%
  10. 10Tomo62% ±16%
  11. 11Poke61% ±18%
  12. 12Wajo / Fo61% ±14%
  13. 13Waldo57% ±12%
  14. 14Familiar57% ±17%
  15. 15Caddy56% ±21%
  16. 16Buoy55% ±19%
  17. 17Luke54% ±21%
  18. 18Bantr53% ±19%
  19. 19Asmi53% ±19%
  20. 20Norton Family Assistant51% ±18%
  21. 21Butler50% ±21%
  22. 22Friday47% ±16%
  23. 23Ask Ranger47% ±14%
  24. 24Gemini Spark47% ±13%
  25. 25Today46% ±14%
  26. 26Tab45% ±21%
  27. 27Nessi42% ±23%
  28. 28Persona41% ±15%
  29. 29Airtap41% ±21%
  30. 30Vellum40% ±16%
  31. 31Tressa40% ±20%
  32. 32Joshu40% ±19%
  33. 33OpenInstinct39% ±19%
  34. 34Ka38% ±23%
  35. 35Manus36% ±14%
  36. 36Del35% ±17%
  37. 37The Shadow35% ±14%
  38. 38tinyNature35% ±15%
  39. 39OpenClaw32% ±14%
  40. 40Lucas31% ±16%
  41. 41AVA30% ±17%
  42. 42Alfi by Text.ai27% ±16%

Fixes when corrected

Higher is better

After a correction, how often the very next reply got it right.

  1. 1Instinct93% ±9%
  2. 2Neo89% ±10%
  3. 3szn87% ±13%
  4. 4Muse84% ±13%
  5. 5Tomo84% ±13%
  6. 6Ollie84% ±12%
  7. 7Hark82% ±18%
  8. 8Pally82% ±15%
  9. 9Dots79% ±12%
  10. 10Luke79% ±18%
  11. 11Poke77% ±16%
  12. 12Waldo76% ±11%
  13. 13Town75% ±17%
  14. 14Buoy74% ±17%
  15. 15Wajo / Fo73% ±13%
  16. 16Friday72% ±14%
  17. 17Airtap72% ±20%
  18. 18Bantr72% ±17%
  19. 19Asmi72% ±17%
  20. 20Vellum71% ±15%
  21. 21Ask Ranger70% ±13%
  22. 22Gemini Spark70% ±12%
  23. 23Caddy70% ±20%
  24. 24Norton Family Assistant67% ±17%
  25. 25Familiar67% ±16%
  26. 26Today67% ±14%
  27. 27OpenInstinct66% ±19%
  28. 28Tab64% ±21%
  29. 29Del63% ±18%
  30. 30OpenClaw63% ±14%
  31. 31Tressa62% ±20%
  32. 32The Shadow61% ±15%
  33. 33Ka60% ±23%
  34. 34Persona60% ±15%
  35. 35Butler60% ±21%
  36. 36tinyNature59% ±16%
  37. 37Nessi58% ±23%
  38. 38Joshu58% ±19%
  39. 39AVA57% ±18%
  40. 40Manus55% ±14%
  41. 41Alfi by Text.ai53% ±17%
  42. 42Lucas53% ±17%

Said done, wasn't

Lower is better

Of the times it said a task was done, how often our checks found it wasn't.

  1. 1Instinct3% ±7%
  2. 2Dots5% ±8%
  3. 3Neo6% ±8%
  4. 4Tomo7% ±10%
  5. 5Muse8% ±10%
  6. 6szn9% ±12%
  7. 7Bantr9% ±12%
  8. 8Norton Family Assistant9% ±12%
  9. 9Hark9% ±14%
  10. 10Pally10% ±13%
  11. 11Town10% ±13%
  12. 12Wajo / Fo10% ±9%
  13. 13Ollie11% ±11%
  14. 14Poke11% ±12%
  15. 15Caddy11% ±15%
  16. 16Luke11% ±15%
  17. 17Vellum11% ±11%
  18. 18OpenInstinct11% ±14%
  19. 19Familiar12% ±12%
  20. 20Asmi12% ±14%
  21. 21Tab12% ±16%
  22. 22Nessi12% ±17%
  23. 23Ka13% ±17%
  24. 24tinyNature13% ±11%
  25. 25Waldo14% ±9%
  26. 26Buoy14% ±14%
  27. 27Del14% ±13%
  28. 28The Shadow14% ±11%
  29. 29Butler14% ±16%
  30. 30Tressa15% ±15%
  31. 31Ask Ranger15% ±11%
  32. 32Lucas15% ±13%
  33. 33Airtap15% ±17%
  34. 34Joshu15% ±14%
  35. 35Today16% ±11%
  36. 36Gemini Spark16% ±9%
  37. 37OpenClaw16% ±11%
  38. 38Manus16% ±11%
  39. 39Persona16% ±11%
  40. 40Friday17% ±12%
  41. 41Alfi by Text.ai18% ±14%
  42. 42AVA19% ±15%

Acted without permission

Lower is better

Share of tests where it paid, ordered or sent something without the approval the test required.

  1. 1Neo1% ±5%
  2. 2Town1% ±8%
  3. 3Tomo1% ±7%
  4. 4Poke1% ±7%
  5. 5Muse2% ±7%
  6. 6Hark2% ±11%
  7. 7Pally2% ±9%
  8. 8Dots2% ±6%
  9. 9Ollie2% ±7%
  10. 10Wajo / Fo2% ±6%
  11. 11Bantr2% ±9%
  12. 12Norton Family Assistant2% ±8%
  13. 13Instinct3% ±7%
  14. 14Caddy3% ±11%
  15. 15Airtap3% ±11%
  16. 16szn3% ±9%
  17. 17Luke4% ±12%
  18. 18Waldo4% ±6%
  19. 19Asmi4% ±10%
  20. 20Tressa4% ±11%
  21. 21Ask Ranger4% ±7%
  22. 22Today4% ±7%
  23. 23Nessi4% ±13%
  24. 24Familiar5% ±9%
  25. 25Del5% ±10%
  26. 26The Shadow5% ±8%
  27. 27Alfi by Text.ai5% ±9%
  28. 28AVA5% ±10%
  29. 29Joshu5% ±11%
  30. 30Tab5% ±12%
  31. 31Friday5% ±8%
  32. 32Buoy6% ±11%
  33. 33Vellum6% ±9%
  34. 34Gemini Spark6% ±7%
  35. 35Ka6% ±14%
  36. 36OpenInstinct7% ±12%
  37. 37Butler7% ±14%
  38. 38OpenClaw8% ±9%
  39. 39Manus8% ±9%
  40. 40Persona8% ±9%
  41. 41tinyNature8% ±10%
  42. 42Lucas9% ±11%

Time to finish

Lower is better

Median time to finish one-sitting tasks that were verified done.

  1. 1Town6m 30s
  2. 2Dots6m 51s
  3. 3Pally7m 1s
  4. 4Ollie7m 25s
  5. 5szn7m 30s
  6. 6Poke7m 44s
  7. 7Hark7m 57s
  8. 8Norton Family Assistant8m 36s
  9. 9Instinct8m 38s
  10. 10Neo8m 42s
  11. 11Muse8m 56s
  12. 12Luke9m 6s
  13. 13Tressa9m 39s
  14. 14Wajo / Fo10m
  15. 15Nessi11m
  16. 16Tomo11m
  17. 17Ask Ranger11m
  18. 18Caddy11m
  19. 19OpenClaw11m
  20. 20Manus11m
  21. 21The Shadow11m
  22. 22Buoy11m
  23. 23Bantr11m
  24. 24OpenInstinct11m
  25. 25Waldo12m
  26. 26Today12m
  27. 27Asmi12m
  28. 28Lucas12m
  29. 29Tab12m
  30. 30Familiar12m
  31. 31Butler12m
  32. 32Airtap12m
  33. 33Gemini Spark13m
  34. 34Ka13m
  35. 35Vellum13m
  36. 36Joshu13m
  37. 37Friday13m
  38. 38Del13m
  39. 39Alfi by Text.ai13m
  40. 40Persona14m
  41. 41AVA15m
  42. 42tinyNature15m

Reply time

Lower is better

Median time from a tester's message to the assistant's next reply.

  1. 1Muse6s
  2. 2Town6s
  3. 3Tomo8s
  4. 4Norton Family Assistant8s
  5. 5Dots9s
  6. 6tinyNature9s
  7. 7Ollie10s
  8. 8Friday10s
  9. 9Buoy12s
  10. 10Hark13s
  11. 11Poke13s
  12. 12Manus13s
  13. 13OpenInstinct14s
  14. 14Asmi15s
  15. 15Tab16s
  16. 16Instinct17s
  17. 17Ask Ranger17s
  18. 18Nessi18s
  19. 19Neo19s
  20. 20Pally20s
  21. 21AVA21s
  22. 22Alfi by Text.ai22s
  23. 23Caddy23s
  24. 24Del24s
  25. 25Waldo25s
  26. 26Ka27s
  27. 27Gemini Spark28s
  28. 28Butler28s
  29. 29Wajo / Fo30s
  30. 30Joshu32s
  31. 31Bantr34s
  32. 32Airtap35s
  33. 33Familiar36s
  34. 34The Shadow36s
  35. 35Today37s
  36. 36Persona37s
  37. 37Tressa38s
  38. 38OpenClaw38s
  39. 39Luke40s
  40. 40Vellum40s
  41. 41Lucas40s
  42. 42szn44s

Every tested assistant

AssistantRunsDone first tryFixes when correctedSaid done, wasn'tActed without permissionTime to finishReply time
Muse2179% ±14%84% ±13%8% ±10%2% ±7%8m 56s6s
Instinct2770% ±14%93% ±9%3% ±7%3% ±7%8m 38s17s
Neo3068% ±14%89% ±10%6% ±8%1% ±5%8m 42s19s
Hark1267% ±20%82% ±18%9% ±14%2% ±11%7m 57s13s
Pally1671% ±18%82% ±15%10% ±13%2% ±9%7m 1s20s
Town1570% ±18%75% ±17%10% ±13%1% ±8%6m 30s6s
Dots2866% ±14%79% ±12%5% ±8%2% ±6%6m 51s9s
szn1868% ±17%87% ±13%9% ±12%3% ±9%7m 30s44s
Ollie2468% ±15%84% ±12%11% ±11%2% ±7%7m 25s10s
Tomo2162% ±16%84% ±13%7% ±10%1% ±7%11m8s
Poke1861% ±18%77% ±16%11% ±12%1% ±7%7m 44s13s
Wajo / Fo3061% ±14%73% ±13%10% ±9%2% ±6%10m30s
Bantr1653% ±19%72% ±17%9% ±12%2% ±9%11m34s
Caddy1256% ±21%70% ±20%11% ±15%3% ±11%11m23s
Luke1254% ±21%79% ±18%11% ±15%4% ±12%9m 6s40s
Waldo4057% ±12%76% ±11%14% ±9%4% ±6%12m25s
Norton Family Assistant1851% ±18%67% ±17%9% ±12%2% ±8%8m 36s8s
Familiar2057% ±17%67% ±16%12% ±12%5% ±9%12m36s
Buoy1655% ±19%74% ±17%14% ±14%6% ±11%11m12s
Asmi1653% ±19%72% ±17%12% ±14%4% ±10%12m15s
Tressa1440% ±20%62% ±20%15% ±15%4% ±11%9m 39s38s
Vellum2440% ±16%71% ±15%11% ±11%6% ±9%13m40s
Tab1245% ±21%64% ±21%12% ±16%5% ±12%12m16s
OpenInstinct1539% ±19%66% ±19%11% ±14%7% ±12%11m14s
Friday2447% ±16%72% ±14%17% ±12%5% ±8%13m10s
Ask Ranger3047% ±14%70% ±13%15% ±11%4% ±7%11m17s
Today3046% ±14%67% ±14%16% ±11%4% ±7%12m37s
Gemini Spark4047% ±13%70% ±12%16% ±9%6% ±7%13m28s
Nessi1042% ±23%58% ±23%12% ±17%4% ±13%11m18s
Del1835% ±17%63% ±18%14% ±13%5% ±10%13m24s
The Shadow2835% ±14%61% ±15%14% ±11%5% ±8%11m36s
Ka1038% ±23%60% ±23%13% ±17%6% ±14%13m27s
OpenClaw3032% ±14%63% ±14%16% ±11%8% ±9%11m38s
Manus3036% ±14%55% ±14%16% ±11%8% ±9%11m13s
Persona2841% ±15%60% ±15%16% ±11%8% ±9%14m37s
tinyNature2435% ±15%59% ±16%13% ±11%8% ±10%15m9s
Alfi by Text.ai2027% ±16%53% ±17%18% ±14%5% ±9%13m22s
AVA1830% ±17%57% ±18%19% ±15%5% ±10%15m21s
Lucas2031% ±16%53% ±17%15% ±13%9% ±11%12m40s
Asaply Few runs352% ±34%71% ±33%13% ±29%7% ±27%10m7s
Airtap1241% ±21%72% ±20%15% ±17%3% ±11%12m35s
Orchid Few runs647% ±28%68% ±27%15% ±23%5% ±19%12m15s
Folk Few runs638% ±27%65% ±27%17% ±23%3% ±17%11m10s
Butler1250% ±21%60% ±21%14% ±16%7% ±14%12m28s
Orbits Few runs939% ±24%57% ±24%17% ±20%7% ±16%13m12s
Joshu1640% ±19%58% ±19%15% ±14%5% ±11%13m32s
Brea Few runs431% ±30%50% ±32%15% ±27%8% ±24%14m6s