Artificial intelligence is advancing rapidly toward creating agents that can perform tasks at nearly human level, according to industry leaders. However, most organizations struggle to move these systems from concept to real-world deployment. A recent report from a major data platform found that only 19% of organizations have deployed AI agents, and even then, only to a limited extent. The obstacles are not technical but strategic: how to control what agents access, how to measure if they are working correctly, and how to keep costs in check. Addressing these three concerns before implementation gives agents the best chance of success.
Control it: Governance is the foundation
The first challenge is governance: ensuring that agents only access the data and tools they are permitted to use. An AI agent is far more powerful than a simple chatbot. It can connect to databases, run code, send emails, and execute multi-step workflows. That power introduces risks, especially when dealing with sensitive information. For example, a women's health app with millions of users must guarantee that one user never sees another user's personal data. Similarly, an asset manager sending portfolio reports to clients cannot afford to mix up confidential information.
To address this, organizations need a robust governance system that enforces data segmentation. A data catalog serves as a single view of all data, tools, and identities. It tracks which agent accesses what information and which user is interacting with the agent. Permissions must be deterministically enforced, not just suggested in a prompt. Companies that treat governance as a first-class design principle are far more likely to get agents into production than those that freewheel the process.
Another aspect of governance is connecting the right data to the right question. Instead of a transactional chatbot that requires the user to ask multiple follow-ups, an agent should automatically pull together related pieces of information. For instance, an online car marketplace built an internal agent that, given a question about a convertible, can also retrieve traffic data, demographic trends, and pricing history. This reduces the burden on the user and delivers richer insights.
Evaluate for correctness: Know if it's any good
The second best practice is rigorous evaluation. It is not enough to just deploy an agent and hope for the best. Every step of the agent's reasoning must be auditable. In practice, evaluation often involves domain experts, not just software engineers. For the women's health app, physicians evaluated whether the agent's responses were medically accurate and appropriately nuanced. They provided feedback that led to improvements in the orchestration system.
Evaluation should be ongoing and occur at multiple levels. It should check not only the final answer but also each intermediate step the agent takes. If an error is detected, the agent can be rolled back to the evaluation stage, redeployed, and retrained. This continuous loop is essential for building automated learning into the agent. Companies that rigorously evaluate their agents are six times more likely to successfully deploy them in production.
Accuracy also differentiates the user experience. A financial services firm used evaluation to ensure that its agent could identify new product opportunities by analyzing portfolios. This led to identifying over $15 million in potential new business. Without evaluation, such opportunities might have been missed or accompanied by errors that damage trust.
Start small: Maximize efficiency and payoff
The third concern is cost, but it is largely an outcome of doing the first two things well. Once governance and evaluation are in place, cost becomes an implementation detail. However, it must be considered from the start. Leaders recommend starting small with focused use cases rather than trying to replace entire workflows. For example, a convenience store chain deployed an agent to help service technicians access repair manuals and past issues. This simple application led to a 25% increase in first-time fix rates and a 40% reduction in repair time.
A university used agents to analyze recordings of calls with prospective students, extracting factors that influence their decisions. Humans taking the calls could not take comprehensive notes, but the agent could process hundreds of calls and reveal patterns the institution had never seen before. Such small, measurable victories build confidence and momentum.
Trying to replace a complex enterprise resource planning (ERP) system with a single agent prompt would be foolish. Instead, leaders advise attacking the problem component by component. By building atomic pieces that can be tested and verified, organizations can later combine them into a larger system. This approach also keeps costs predictable and allows for early wins.
Prepare your data for velocity
A common thread across all three best practices is the quality of underlying data. Clean, well-organized data accelerates every stage of agent development. As one expert put it, if your data is in good shape, you could build and deploy an agent this afternoon. If your data is in rough shape, most of the time will be spent cleaning it up. Investing in data organization upfront pays dividends in faster iterations, better evaluations, and lower overall costs.
The industry is still in the early stages of agentic AI, comparable to the early days of the web when companies were building web pages without fully understanding the return on investment. Yet early adopters are already seeing key performance indicators move in the right direction. The path forward is clear: start with governance, evaluate rigorously, and start small. These practices give agents the head start they need to become truly human-level tools.
Source: ZDNET News