AI tools are transforming how we approach software development – summarizing code, generating snippets, answering questions like “What does this do?” and generating code. This is especially true in environments like IBM i, where systems evolve over decades and carry complex interdependencies.

However, in the case of business-critical IBM i applications, the need for the right AI tool is more crucial than ever. These systems hold decades of business rules, exceptions, and implicit logic – often hidden in dormant programs, unreachable branches, and outdated rules. Simply applying AI to generate code or analyze dependencies, without understanding these layers deeply and separating signal from noise, can lead to false results that propagate into compounding mistakes, wasted resources, and unreliable outcomes.

The Real Failure Mode: False Results That Quietly Snowball

AI tools, especially those focused on analysis or code generation, tend to treat all code they process as equally relevant. This becomes problematic when dealing with long-running systems with significant legacy codebases.

When AI tools absorb outdated or inactive logic, they generate false or misleading results. These outputs may appear to be correct but are not grounded in the real behavior of the system today. Here’s where the real risks lie – these errors don’t manifest immediately, but instead create progressive mistakes:

• Incorrect assumptions infiltrate design decisions and migration plans
• Impact analysis misses real dependencies or flags irrelevant ones
• Teams migrate or refactor logic that no longer runs
• Verification efforts grow because the AI’s recommendations lack trust
• AI usage costs rise as more context and code are fed into the system

This is why AI speed often turns into AI cleanup, especially when dealing with legacy systems.

Why This Matters Most Before and During Migration Programs

Modernization failures typically aren’t due to engineering teams lacking technical skills – they arise from missing system understanding. Hidden dependencies, undocumented rules, and institutional knowledge that has faded over time are the culprits. A Forrester Consulting study (commissioned by Rocket Software, 2024) found that 90% of rewrite projects fail on the first try due to missing or misunderstood context.

If you don’t understand what your system does end-to-end, a rewrite becomes repeated rework. AI tools – especially those used for code generation – can certainly help with modernization efforts, but they can only be effective if they are grounded in a reliable model of your system.

The Hidden Cost Isn’t Tokens – It’s Verification and Rework

Even when AI-generated code is useful, it introduces a major verification gap. SonarSource’s State of Code findings (Jan 2026) highlight that 96% of developers don’t fully trust AI-generated code, and 48% don’t always verify it before committing.

In critical systems, this lack of trust results in additional verification work, increasing the overall cost of using AI tools. Moreover, security concerns are compounded by these tools: Veracode’s 2025 GenAI Code Security Report showed that 45% of AI-generated code failed security tests and introduced vulnerabilities.

This doesn’t mean “don’t use AI”; it means that tools designed for AI code generation and analysis must be equipped with system-level understanding to reduce the risk of false outputs and avoid the costly rework associated with verifying unreliable code.

Why File-Level AI Tools Break Down on IBM i

Many AI tools are optimized for local context: focusing on individual files, functions, or snippets. In contrast, IBM i systems operate like interconnected networks: they include cross-program calls, shared DB2 for i files, batch jobs, timing assumptions, data propagation, and implicit workflows that span multiple areas of the system.

To manage this complexity, the key questions teams must ask are rarely “What does this function do?” Instead, they should ask:

• Where is this business rule actually enforced?
• Which programs read/write this file—directly and indirectly?
• What happens if a field changes?
• Which paths still execute in production—and which don’t?

Shallow tools, like simple “chat over code” assistants, may answer confidently but fail to reflect the true execution reality of the system. Without system-level clarity, AI tools can miss crucial dependencies or generate misleading code that doesn’t function as intended in the live system.

What the Right AI Tool Looks Like

The right approach combines AI code generation with System Clarity: a structured understanding of what runs, what depends on what, and where business logic resides. Before relying on AI to generate code or perform analysis, your tool should:

1. Separate active logic from dead codeIdentify what truly executes (vs. what merely exists).
2. Map dependencies as relationshipsShow how programs, data, jobs, and processes interact – not as a file-by-file analysis but as a navigable network of dependencies.
3. Anchor AI’s outputs in evidenceAny AI tool should point back to the source locations and dependencies that validate the results it generates.
4. Feed AI the right contextTo ensure AI focuses on what matters, reduce ambiguity by giving it only relevant context, ensuring that the AI processes only signal, not noise.

This doesn’t eliminate AI; it makes AI dependable by providing the system context it needs to produce trustworthy, reliable code generation or analysis.

Why Structured Context (Graphs + Retrieval) Improves Reliability

Recent findings, such as those from Thoughtworks (Apr 2024), support the idea that AI tools are more reliable when paired with structured representations of the system. Specifically, retrieval-augmented generation over knowledge graphs of the codebase has shown to significantly improve reliability. It preserves system structure that raw AI models alone cannot derive from text.

The message here is simple: AI tools should not be asked to infer the system from raw code. Instead, they should reason within a map of your system, where dependencies, relationships, and critical processes are already outlined and understood.

From Files to Systems

A file-level view helps you edit. A system view helps you decide.

When AI tools are given a system-level representation – modeling your application as interconnected flows (program calls + data + rules + jobs) while distinguishing active and inactive paths – AI stops improvising. Instead, it can begin generating or analyzing with evidence.

This reduces false results, prevents cascading mistakes, and ensures that modernization scopes are defensible. You can see impact paths, data propagation, and rule enforcement clearly before any code change happens.

The Takeaway: Embrace the Right AI Tool for Smarter Change

AI is an invaluable tool for transforming and modernizing business-critical systems. However, to fully leverage AI’s potential, system clarity must come first. AI tools that don’t deeply understand your system can lead to costly mistakes, unnecessary rework, and unreliable code generation.

The key takeaway is: AI should be used in combination with tools that provide comprehensive, system-level clarity. These tools should not only separate active business logic from obsolete code but also properly map dependencies and deliver evidence-backed answers.

With the right foundation, AI becomes a reliable assistant for change, allowing teams to confidently move forward, reduce risks, and avoid escalating costs. Before applying AI to your legacy systems, ensure you have a clear, curated view of what your system actually does. Only then can you leverage AI to its full potential, driving change without the disruptive surprises that can derail your project.