AI agent chip design benchmark: the gap between working designs and target performance

Can AI agents take on semiconductor design as well? Build-A-Chip-Bench, released by ChipAgents with technical advice from NVIDIA, is a benchmark intended to measure this question. The announcement date is Oct 14th 2026.

1. How is it different from existing benchmarks?

Existing benchmarks mostly examine one small step. They receive a specification and write RTL (register-transfer-level code), or verify a completed design. ChipAgents points out that RTL can work correctly in simulation yet miss timing, exceed the area budget, or fail routing.

Build-A-Chip-Bench evaluates the entire flow from workload specifications to GDS (chip layout files). It includes architecture, RTL, verification, synthesis, placement, routing, and signoff.

2. Two published figures

Even figures released by the same company should not be combined into a single line because the measurement methods differ.

- Press release: The agents tested passed functional verification on an average of about 40% of tasks. They met physical performance, power, and area (PPA) targets on only about 10%.

- Research page: The best-performing agent passed the hard gates (functional correctness, routability, hold timing, and DRC) on 65% of tasks, with an overall score of 16.6%.

The averages and top values, as well as the task configurations, differ, so it is difficult to compare the two figures directly. Still, the conclusion is the same. There is a large gap between creating a functionally correct design and creating a physically usable chip.

3. Three practical takeaways

First, passing simulation is only an intermediate point. In practice, problems emerge in later stages, and their causes may lie in decisions made much earlier.

Second, the key issue is whether an agent can go back and fix earlier decisions. The research page explains that the ability to optimize while preserving parts that already work, and to plan while accounting for slow tool execution, remain weak.

Third, evaluation tools should also be considered when looking at benchmark scores. This benchmark uses open-source tools and an open-source PDK. The same results cannot be assumed for commercial processes or commercial EDA tools.

4. What has not yet been confirmed

- The number of evaluation tasks, the names of the models tested, and the execution costs have not been confirmed.

- Both figures were announced by the company, and reproducibility results from an independent institution have not been confirmed.

- NVIDIA stated that it provided technical advice on the benchmark configuration and scoring criteria. This does not mean that NVIDIA independently reproduced the results.

5. Conclusion

The ability to write RTL is already an area where benchmark scores have risen substantially. The research page states that ChipBench has reached 90.6%. The next question is whether the design passes timing, area, and routing all together. When evaluating AI for chip design, the question should be whether it can be physically manufactured, rather than whether the code runs.

Related article: NVIDIA Nemotron 3.5 Super VL and Bolt released: what to look at first in local agents ↗

Sources

- ChipAgents press release, Oct 14th 2026: https://chipagents.ai/newsroom/chipagents-nvidia-build-a-chip-bench ↗

- ChipAgents research page, Build-A-Chip-Bench: https://chipagents.ai/research/build-a-chip-bench ↗

- NVIDIA Newsroom (mentions collaboration with ChipAgents): https://nvidianews.nvidia.com/news/nvidia-and-semiconductor-leaders-advance-ai-across-chip-design-and-manufacturing-for-ai-defined-labs ↗

This English edition was written and translated with AI assistance, with sources provided. For errors or additional information, please request a correction. Editorial policy