The Decoder· Jonathan Kemper·· 3 hours agoAI Score64
Study finds coding agent harnesses add cost but not pass rate on SWE-bench Verified
Fancy harnesses make coding agents more expensive but not better, study finds
AI summary
A study by Liu and Han found that swapping coding agent harnesses shifts SWE-bench Verified results about as much as simply rerunning the same setup.
Source: The Decoder · the-decoder.com