It’s an increasingly common tale within corporations today: The AI project performs admirably in testing during the pilot phase, gets the green light for a broader rollout…and then stops working properly; Or it fails to deliver the expected business results.

Finger pointing, recriminations, and embarrassment ensue.

The problem is not always the technology. In fact, the fault is often in the planning, processes, and expectations that companies have established—or not established—around their AI projects, according to business leaders who spoke at a roundtable discussion at Fortune Brainstorm Tech this month.

For starters, not every AI project deserves to be rolled out widely, said Amgen Chief Technology Officer Sean Bruich.

“It’s so easy with a pilot to let a thousand flowers bloom,” he said. That’s not a bad thing, since it encourages experimentation. But, he said, “the key to making pilots scale successfully is actually having a wide number of ideas, but a very tight governance on which pilots are actually greenlit.”

A key criteria before taking the next step, said Salesforce Chief Customer and Commercial Officer Lashonda Anderson-Williams, is understanding the intended outcome of the project. Too many companies are focused on the successful implementation of AI features—the technological bells of whistles—instead of the business outcome, she says.

That mentality is a recipe for disappointment: The AI features work great, but the new technology isn’t driving meaningful business results.

Agents needs a map

When it comes to agentic AI, Anderson-Williams noted, a detailed understanding of the workflow—which individuals, groups, or touch points are necessary to complete a task— is critical. What a lot of companies are finding, she said, is that documentation of the workflow either doesn’t exist or is poorly documented: “When you put AI on top of that, the expectation is you’re going to see some magic, and there’s no magic there.”

Access to data is a particularly common stumbling block that AI projects encounter in the transition from the pilot phase to full deployment. With data often scattered in different silos throughout an organization, and with all that data governed by different access privileges and by varying privacy and security considerations, things can get complicate fast. It’s important to map out the contours of the AI project and all the potential data that will be required ahead of time, the panelists stressed. “The earlier we can uncover that in discovery, the better we’ll be set up for success,” Thomson Reuters Chief Data Officer Caitlin Halferty said.

That also means getting buy-in from the right groups and stakeholders within the organization. “Is there some element of PII (personally identifiable information) or confidential data that’s going to trigger privacy?” Halfery said. If the answer is yes, then the right people need to be part of the project. “Is there a cyber element? Let’s get security on board,” she said.

Amgen’s Bruich echoed the importance of broad buy-in, noting that an AI project that is transformational to the company will by necessity involve leaders in finance, technology, HR, and other groups across the organization. A truly impactful AI project, he said, needs to do more than just make work processes more efficient for a small group of employees. It needs to deliver “an outcome that matters to the enterprise.”