AI模型和桩式如何添加代码？我创建了一个开源benchmark来测量代码复杂度
How much code do AI models and harnesses add? I made an open-source benchmark to measure code complexity.
0 r/LocalLLaMA 2026-11-09 阅读原文 ↗
原文链接：https://www.reddit.com/r/LocalLLaMA/comments/1xrathc/how_much_code_do_ai_models_and_harnesses_add_i/
中文导读
现有的benchmark衡量解决率和token输出/成本，但不惩罚代码过度或过度工程化，这会在长期内增加技术债务。parsimony衡量语法单位（语句、调用、名称）在其他benchmark发布的补丁中。它不衡量行数、注释、测试。每个模型或桩式都需要花费时间和学习成本。
导读由机器生成，结论与数字请以原文为准。
原文正文（未翻译）
 | Existing benchmarks measure the solve rate and token output/cost, but don't punish excess code or overengineering, which increases technical debt in the long term. Parsimony counts syntax units (statements, calls, names) in the patches other benchmarks publish. It doesn't count lines, comments, tests. Each model or harness is compared with the human fix for the same issues it solved. - Smarter models surprisingly write slightly more code. - Models: the median model adds 1.9× the human fix on SWE-bench Verified (older models), but about 1× on DeepSWE. - Harnesses: with the same two open-weight models, the harness alone moves it from 0.80× (Codex) to 0.99× (mini-SWE-agent, OpenClaw). The harness data comes from Joel Niklaus's Harness Leaderboard. https://github.com/La5u/Parsimony Looking for: public patches for newer models, any ideas or contributors for self run models. * some newer models are self run in codex and claude code, shown as faded (unverified) in DeepSWE [link][comments] |
原文约 1065 字，读原文 ↗
正在加载讨论……