One of the biggest challenges companies face when implementing AI is the gap between a successful demonstration and a system that can operate safely in real-world conditions. An AI agent might perform perfectly during testing, but what happens when users ask unexpected questions, information changes, or the agent is given access to sensitive data and business operations?
Oracle’s workshop explored these challenges through a practical demonstration of an AI-powered travel assistant. The agent was tasked with helping two travellers book flights and accommodation. Its responsibilities included understanding customer preferences, searching for available options, comparing prices, and preparing offers. However, certain actions, such as confirming bookings or making payments, needed to remain outside its authority.
The demonstration illustrated how AI behaviour depends not only on the language model being used but also on the system surrounding it, often called the harness. This includes the tools the AI can access, the information it receives, the permissions it has, and the rules controlling its actions.
By comparing different configurations and modifying components of the system, the workshop demonstrated how the same underlying model could behave differently depending on its environment.
Why instructions alone are not enough
A central message of the workshop was that simply telling an AI what it should or should not do is insufficient.
For example, developers might instruct an agent never to reveal private customer information, invent discounts, or complete bookings without permission. While these instructions might work during demonstrations, they do not guarantee that the agent will follow them consistently in every situation.
Real users express their intentions differently, and unexpected combinations of requests can expose weaknesses that were not apparent during initial testing. An agent might misunderstand who is authorised to access information, rely on outdated prices, or attempt actions beyond its intended responsibilities.
The workshop demonstrated how these risks can be reduced by moving important safeguards out of the AI’s instructions and into the surrounding system.
Better architecture improves both safety and efficiency
The workshop also highlighted that better system design can improve performance and reduce unnecessary resource consumption.
In the demonstrated comparison, the initial version of the agent relied heavily on instructions and repeatedly called external tools to complete tasks. The improved version introduced more structured tools, controlled access to information, clear approval procedures, and mechanisms for checking current prices and availability.
The results showed that the improved system could complete comparable tasks with substantially fewer tool calls and fewer AI processing tokens. This demonstrated that stronger safeguards do not necessarily make AI systems slower or more complicated to operate. Well-designed architecture can make them more efficient.
Another important aspect was testing. Rather than evaluating an AI agent only through ideal scenarios, companies need to test situations involving unexpected user behaviour, changing information, conflicting requests, and potential security risks.
Continuous monitoring and clearly defined responsibilities are also necessary once an AI system is deployed.
The main takeaway
Taking AI agents into production is a system design challenge, not simply a prompting challenge.
A successful prototype proves that an idea can work under certain conditions. It does not prove that the solution is ready for real customers, sensitive information, or business-critical operations.
For companies adopting AI, the priority should therefore shift from making impressive demonstrations to building systems that are secure, reliable, efficient, and accountable.
The future of AI in business is not just about choosing increasingly powerful models. It is about designing the right environment around those models, defining what they can and cannot do, keeping humans in control of important decisions, and continuously testing how they behave in the real world.