Keep pulling the thread on Lilian Weng.
In the near-term, harness engineering is predicted to evolve towards meta-methodology, focusing on improving the process of finding answers rather than just the answers themselves.
The Self-Taught Optimizer (STOP) framework improved performance when using GPT-4 as the base model, but degraded performance when using weaker models like GPT-3.5 and Mixtral.
A harness evolved with the Agentic Harness Engineering (AHE) framework on Terminal-Bench-2 successfully transferred its capabilities to the SWE-bench-verified benchmark without additional modifications.
Agents discovered by the Darwin Gödel Machine (DGM) using Claude 3.5 Sonnet as a base model achieved performance comparable to or better than handcrafted agents on SWE-bench Verified (50% vs 20%) and Polyglot (30.7% vs 14.2%).
Autonomous research agents exhibit six recurring failure modes: bias toward training-data defaults, implementation drift, memory degradation, over-optimism, insufficient domain intelligence, and weak scientific taste.
Many harness improvements may eventually be internalized into core model behavior, but the interface with external context and tools will likely remain.
The object of optimization in harness systems is progressing from instruction prompts to structured context, then to workflow, harness code, and finally optimizer code.
The Agentic Context Engineering (ACE) framework treats context as an evolving playbook instead of a continuously lengthening prompt.
The Meta Context Engineering (MCE) framework separates the mechanism of how to manage context from the actual content within the context.
A range of models, from Qwen3.5-9B to Claude Opus 4.6, demonstrated similar capabilities for updating harnesses in experiments by Lin et al. (2026).
The performance benefit a model gains from a harness is non-monotonic with respect to model size, with middle-tier models showing the most improvement.
The Agentic Harness Engineering (AHE) framework outperformed human-designed harnesses such as OpenCode, Terminus-2, and Codex on the Terminal-Bench-2 benchmark, with the exception of the 'Hard' difficulty tier.