Brooke Hopkins: Evals are the new crash tests (SF Tech Week 2025)

As part of SF Tech Week, I attended a meetup after work in SoMa today. 

What's happening: Titled Evals Are the New Crash Tests: Safely Scaling Conversational AI, the meetup was held at Coval's cozy HQ in San Francisco's SoMa district.

Miki Hardisty (CEO, Olelo Intelligence) gave me the scoops on the event.

See also: Keep building, no matter what: coffee w/ Eric Nakagawa & Miki Hardisty

My takeaways: 

• I liked how Coval kept the food and drink budget to a minimum - water, pretzels, popcorn, and La Croix. The strength of the content was what mattered.
• You don't want to automate all evals
• Krew uses Coval for performance benchmarking
• Latency is an issue with voice AI

The bottom line: We're in the commercial break of the second inning for AI. It's still early and it's definitely time to level up.

Enjoy the video and pics!

"What we do is we take simulated conversations, or real-time synthetic data, so you don't have to go back and forth with your agent. And then we run analysis on top of that." Brooke Hopkins, CEO, Coval

Full house tonight at Coval HQ in SoMa