“The new PRD are the evals,” Xavi Amatriain, Expedia Group’s first chief AI and data officer, told the VB Transform 2026 audience last week in Menlo Park. “So basically, you encode what you want the product to do through your evals, which might include red teaming evals and all kinds of other things, which already have a bunch of security requirements. So, you already embed that into the PRD and the product design document before you even start coding.”
He pushed it further. “With AI-assisted or AI-generated code, that’s gonna be the future. It’s like all your thinking is gonna go into the evals.”
Amatriain served as VP of AI and Compute Enablement at Google across the platforms powering Gemini and Google Search before his December 2025 appointment at Expedia. He's mentored talent who went on to found Perplexity and Scale AI.
VentureBeat’s VB Pulse research on the evaluation gap reinforced the stakes. Sixty-six percent of the 157 enterprises surveyed already permit some production deployment without human review or are building toward it within the next 12 months, yet only 5% fully trust the automated evaluations that would make that decision. Half have shipped an agent that passed internal evals but then failed with a real customer.
“The more guardrails and artificial business rules and sort of rules that you put into the system, the worse off,” Amatriain said. “Not only because they’re brittle, but also because they actually mess up with the feedback loop. You are actually biasing the user and the feedback you get from the user, and then you’re learning that in the wrong way.” He called guardrails “a necessary evil” and said the goal is to minimize their impact over time.
Not everyone at Transform agreed. Other speakers argued during the event that the highest-risk actions still demand very firm guardrails.
Expedia governs AI through three layers instead. Principles come first, communicated broadly. “I like to encode at a very high level how I expect decisions to be made, because in a large organization you’re gonna have a lot of distributed decision making,” Amatriain said. “And sometimes, if you’re lucky enough, those principles might be embedded in your culture. But most of the time, my experience has been they’re not.” The processes and tools that enforce them follow. “Principles look really nice on a picture on some wall, but you need to then give them teeth,” he said. Automation sits on top of both.