Agents is the core domain because it measures not what a model knows but whether it can finish something on its own: read the environment, choose tools, execute across many steps, recover after failure. Its link to AGI is the most direct of any domain here - one operational definition of AGI is completing economically valuable tasks without a human steering each step. Judging this domain requires two rulers, and using only one misleads you. The first is task completion: the Arena Agent board (2026-09-22) measures net improvement, led by Claude Fable 5.1 at 13.71 and GPT-6 Astra at 11.54, and it reports praise alongside confirmed_success (31.83 / 19.83 for Fable 5.1); confirmed_success matters more than praise because it separates "looked useful" from "actually got the thing done". The second ruler is the scaffold: the same model behind a different harness produces very different results, since the top of SWE-bench Verified is Sonar Foundation Agent + Claude 4.5 Opus at 79.2%, live-SWE-agent with that same model is also 79.2%, and TRAE + Doubao-Seed-Code reaches 78.8% - which says "which combination is best" is an engineering question, not a model question. Agents and harness therefore have to be treated as two separate domains: this one collects things that finish tasks, the harness domain collects what agents run on.
AlphaEvolve: pointing a model at problems that come with an objective scoring function
Google DeepMind's coding-agent-driven evolutionary search: Gemini Flash proposes candidates in volume and Gemini Pro in quality, automatic evaluators score them, and high scorers stay in the population as next-round context, i.e. evolutionary search with an LLM as the mutation operator; it applies only where an automatic evaluator exists, hence the math and agents filing. An evolved scheduling heuristic has run in production in Google data centres (Borg) for over a year, recovering 0.7% of Google's global compute (Google's compute, not all compute on earth), the card reading and the only result validated by long-running production. An evolved matmul kernel is 23% faster at specific sizes, and about 20% of 50+ open maths problems improved, including 4x4 complex matrix multiplication in 48 multiplications against Strassen's 1969 record of 49. Verification differs from the prover line: a Lean check is mathematical correctness, "23% faster" is an empirical reading on specific hardware. No public weights, Early Access is a waitlist, and the precondition is an evaluator you write yourself. Graded B (contested): a year of production behind the Borg result, nothing reproduced.
0.7%Google global compute recovered (production-verified)Contested · 2025-05
Gamma: the orchestration tool that turns a body of source material into a presentable deck
What Gamma does is not "prompt for one slide" but reorganise existing material into something presentable: it ingests long documents, PDFs, pasted text or a URL, and returns a full deck with outline, pagination, imagery and layout, in a card-based structure where each page can be reordered, rewritten or re-illustrated on its own. It represents the kind of AI that is genuinely useful in the documents vertical - the value is in orchestration and editorial selection, not in generation quality. Its ceiling is equally clear: the output lives in Gamma's own layout system, and exporting to pptx loses a layer of fidelity - which happens to be a hard requirement for most organisations.
Web / pptx / pdfOutput formatsVendor Claim · 2025-01
WorkBuddy: Tencent's all-scenario AI office workbench
Tencent's office agent: state the requirement and it plans, calls tools and generates files, leaving both process and result for review — connected to Tencent's IM, docs, mail, meetings and knowledge base.
Muse: Meta's personal agent for long-running tasks
Meta's personal AI agent, released in the US on 2026-09-08 across iOS, Android and web: runs on a cloud VM, takes a goal in chat, and lets you watch it drive a browser.
Doubao Work: ByteDance's productivity agent, from decks to computer control
ByteDance's August 2026 productivity agent: PPT, documents and sheets, deep research, scheduled tasks — plus reading and writing local files, filling forms across pages, and phone-based task dispatch.
Hermes Agent: the open-source agent that turns self-improvement into a closed learning loop
The self-improving agent from Nous Research (248k+ stars). Its learning loop has five mechanisms: periodic memory nudges, autonomous skill creation after tasks, skills that self-improve in use, FTS5 session search with LLM summarization, and Honcho user modeling. One TUI, seven terminal backends (local/Docker/SSH/Singularity/Modal/Daytona/Vercel Sandbox), six chat platforms from a single gateway, no model lock-in. It doubles as a trajectory data-production apparatus. Self-improvement is falsifiable and no controlled experiment has been run, so this is graded as needing reproduction.
ByteDance's open-source super agent harness (82k+ stars). 2.0 is a ground-up rewrite sharing no code with 1.x: sub-agents, extensible skills, sandbox and file system, long-term memory, session goals, plus Manual Context Compaction that hands context management back to the operator. Positioning lifted from a deep-research framework to a runtime for any task. Not independently benchmarked; graded needs-reproduction.
Open-source orchestration for teams of AI agents: a Node.js server and React UI where you bring your own agents, set goals, and track work and spend from one dashboard. It looks like a task manager; underneath sit org charts, budgets, governance, goal alignment and agent coordination. It manages business goals, not pull requests. MIT, self-hostable.
Orca: the ADE that grounds parallel agents in git worktrees
Orca calls itself an ADE (Agentic Development Environment), the AI orchestrator for 100x builders (75k+ stars, MIT/TypeScript, YC-backed). Central mechanism: fan one prompt across Codex, Claude Code, OpenCode and Pi, each isolated in its own git worktree, then compare results and merge the winner. Also WebGL terminals, Design Mode (click a UI element to send its HTML/CSS/screenshot into the prompt), native GitHub and Linear integration, diff annotations routed back to the agent, SSH worktrees and mobile notifications. Not independently benchmarked; graded needs-reproduction.
PPT Master: AI generates native PowerPoint decks from any document
Hand a PDF, DOCX, web page or just a topic to any agent-capable tool and get a natively editable PPTX on your own machine — real slide masters, native shapes, data-backed charts and editable formulas.
Multica: AI coding agents that show up on the board
An open-source, self-hostable workspace where you assign issues to AI coding agents the way you would to a teammate: they pick up the work, run on a runtime you control, comment as they go, and hand results back for review. Works with 26 agent CLIs and keeps intent, runs, decisions and diffs on one issue, with code never leaving your machine.
LangGraph: agents as long-running processes that must persist, interrupt and resume
A low-level stateful-graph orchestration framework from LangChain, MIT licensed, with Python and JS/TS implementations, and Klarna, Replit and Elastic named as production users in the README. Its lineage is stated in the acknowledgements - inspired by Google Pregel and Apache Beam, interface borrowing from NetworkX - which means the BSP graph-computation model applied to agents: nodes are model calls or tool executions, edges are control flow, state is an explicit object. The value is not in the loop-the-model-with-tools layer but in four runtime capabilities: durable execution (resume exactly where it stopped after a crash), human-in-the-loop interrupts (approval can stay suspended for hours or days), separate short-term and long-term memory, and LangSmith execution-path tracing. Its actual claim is that agent reliability is a runtime problem, not a model problem. Usable without LangChain; any CompiledStateGraph can be handed to Deep Agents above it as a sub-agent. 42.1k stars. Not run yet, and checkpoint-resume consistency is unverified, so this is graded as pending reproduction.
DeepTutor is HKUDS's open-source "lifelong personalized tutoring" platform (Apache-2.0, Python 3.11+ / Next.js 16, arXiv 2604.26962, 40.6k stars): chat, multi-engine RAG knowledge bases, living books, writing and practice unified in one learning workspace. Highlights: (1) Subagents & Partners — consult real agent harnesses on your machine (Claude Code, Codex, Grok CLI, Kimi, opencode, Hermes, DeepSeek and more) from inside a chat turn, streaming their work into the conversation, plus persistent IM companions; (2) multi-engine knowledge — versioned RAG libraries across LlamaIndex, PageIndex, GraphRAG, LightRAG (incl. remote server), self-hosted WeKnora, Tencent IMA, MarginNote 4, Kiwix ZIM archives and Obsidian vaults, with pluggable parsing (MinerU/Docling/Tika/markitdown/PyMuPDF4LLM/LiteParse) and KBs that track GitHub repos and doc sites with 24h hash-diffed sync; (3) Book compiles sources into editable typed blocks (text/quizzes/flashcards/timelines/concept graphs/interactive HTML), exports Markdown, and long compiles pause-and-resume; (4) personalized learning (Mastery Path, immersive reading with verified citations, read-aloud/quiz/translation) and a Co-Writer split-view Markdown workspace with KaTeX live preview; (5) an office-skills code sandbox running model-generated code over docx/pdf/pptx/xlsx; (6) an agent-native CLI — the open SKILL.md format is picked up automatically by Claude Code/Codex, and deeptutor run wraps into LangChain/AutoGen loops. Led by Bingxi Zhao, fully open source with no paid online products.
Failures in this domain are almost all silent, so they need to be named one by one. One, completion rate decays exponentially with steps: at 95% per step, twenty steps leave 36%, and the Terminal-Bench leader's 58.18% accuracy corresponds to a pass@5 of 0.7121, meaning a substantial share of success is bought by retrying; real long-horizon tasks are not "a bit harder", they are a different order of magnitude. Two, the confirmation signal is unreliable: the large gap between praise and confirmed_success (31.83 versus 19.83 for Fable 5.1) shows that user satisfaction and task completion are different things, and an agent will hand back something well-formatted that did not solve the problem. Three, side effects are irreversible: code can be rolled back with git, but sending a message, placing an order, deleting data or changing production configuration cannot; the vast majority of agents today have no transaction boundary and no rehearse-then-commit mechanism, and that is their biggest production risk, ahead of accuracy. Four, environment and tool errors propagate: when an API returns HTML instead of JSON, or times out, or rate-limits, the agent often treats that as fact and keeps reasoning, freezing the error into every later step. Five, permissions and audit gaps: an agent operates with a person's credentials, so an overreaching action looks identical to a human action in the logs, and attribution cannot be separated at the system level. Cost and latency: the true cost of a long task is tokens times retries times tool-call latency, and the mini-SWE-agent + Claude 4.5 Opus row on the SWE-bench board is the only publicly comparable per-task price anchor at $376.95 for 76.8%; budget production agents on that order, not on the cost of one conversation.
As an Amazon Associate, we earn from qualifying purchases.
