METR found maintainers would reject roughly half of AI PRs that pass SWE-bench automated grading, with a 24-point gap that suggests benchmark scores substantially overstate production readiness.
Want to check another podcast?
Enter the RSS feed of a podcast, and see all of their public statistics.
