An artificial intelligence (AI) pilot can impress in a controlled demonstration. Then employees try to use it with real customer records, restricted data, and systems that were never part of the test. For enterprises, that is where the harder work begins: making AI reliable enough for daily operations and valuable enough to justify its cost.
In Deloitte’s 2026 State of AI in the Enterprise survey, only 25% of respondents said they had moved at least 40% of their AI pilots into production. A pilot may avoid the fragmented data, access restrictions, and approval processes that a production system must handle. Those requirements can expose problems the sandbox never tested.
Costs can undermine a pilot’s business case as usage grows, particularly when model or application programming interface (API) usage is difficult to predict.
A chatbot may also perform well in a demonstration but slow employees down if they must transfer information manually to customer relationship management (CRM) or enterprise resource planning (ERP) systems. Time saved, work completed, and cost per task offer a clearer test of business value.
Paul Lunow, CTO of digital solutions at software development firm Vention, sees the biggest blind spot as confusing a prototype built from scratch with a system ready to work inside an existing organization.
“Greenfield AI projects look deceptively easy because they avoid enterprise complexity. The moment the solution needs to operate inside an existing organization, all the traditional engineering constraints reappear,” he told TechNewsWorld.
The obstacles often lie beyond the engineering team. “Even when engineering teams can move quickly, the organization may lack the governance, workflows, and decision-making structures needed to approve and operationalize what they build,” Lunow explained.
AI’s variable outputs add to the challenge. Teams must test how a system performs across different requests and conditions, as well as whether it meets the organization’s requirements for access, reliability, and cost.
Production demands also change how teams build AI-assisted software. At Vention, engineers define what the software must do, use AI to help generate code, and test the result against those requirements. Failures help them refine both the specifications and the code.
That process also needs visibility into what AI systems do and what they cost. Teams must be able to detect changes in performance or spending when a model or another component is updated.
“Cost predictability matters alongside technical reliability. A system may be capable of producing production-quality software but still be impractical if its model usage or token consumption is unpredictable,” Lunow said.
Give AI Access in Stages
Before deploying AI across business units, companies need to identify the systems it will use, the data it can access, and the actions it can take. That work starts with mapping existing data sources, permissions, integrations, and workflows. It also requires clean enough data for the intended task.
A narrow first use case lets teams test access controls against a real task. If the controls prevent useful work, employees may return to their old workflows. If permissions are too broad, the system may expose data or take unwanted actions.
“Transparency and observability should be foundational. Enterprises need to know what agents are doing, which data they are accessing, what actions they are attempting, and why they are being allowed or blocked,” Lunow said.
Teams can begin with limited permissions, monitor what the system does, and expand access as its behavior becomes better understood.
From AI Coding Tools to Team Workflows
Generating code is only one part of delivering software. Teams still have to test it, integrate it with existing systems, and deploy it.
Vention has been working toward a model in which engineers define requirements and supervise more AI-generated work. Moving from individual use of coding tools to a shared team process has proved harder than giving employees access to the tools.
“Standardizing tools, sharing knowledge, integrating AI into workflows, and maintaining engineering quality are much harder than simply giving developers access to AI tools,” Lunow said.
That distinction matters when leaders expect AI tools to make development teams 10 times faster, sometimes with fewer people. Code output alone cannot establish such a gain. A more useful measure is the time and resources required to move an idea into production.
“Faster code generation has limited value if testing, integration, customer validation, and deployment remain bottlenecks,” Lunow said.
When to Build and When to Buy
The same questions apply when a company decides to build an AI tool or buy one. A ready-made product may be faster to deploy, but the company still needs to assess how it connects to existing systems, protects data, and is priced as usage grows. Building in-house brings its own maintenance, security, and staffing demands.
“AI can generate a working interface quickly, but organizations still need to consider maintenance, security, operations, and long-term ownership,” Lunow cautioned.
Before choosing, companies should establish who controls their data, who will maintain the system, and how difficult it would be to change providers. A tool that is easy to launch may prove costly to operate or replace.
Read the full article here
