That AI code reviewer my coworker swore by nearly shipped garbage

My coworker Dave pushed this AI code review tool on our team for like 2 months, said it catches stuff humans miss every time. I finally let it run on a big pull request last Thursday, the thing flagged 14 issues but 11 of them were false alarms about style stuff. The two real catches were things I probably would have caught anyway, but here's the kicker. It completely ignored a race condition in our database connection pool that crashed the staging server 3 times in 4 hours. Dave swore the training data was solid for our stack, but it was mostly trained on Python projects and we're in Go. Has anyone else had an AI reviewer that gives you confidence then whiffs on the actual logic bugs? I'm thinking of going back to just having a human pair review for anything past 500 lines.

1 comments

1 Comment

jenniferh8321d ago

Sounds like it saved you two catches, so maybe not all garbage.