The cloud-native ceiling

For the past decade, the cloud-native paradigm — defined by containers, microservices and DevOps agility — served as the undisputed architecture of speed. As CIOs, you successfully used it to decouple monoliths, accelerate release cycles and scale applications on demand.

But today, we face a new inflection point. The major cloud providers are no longer just offering compute and storage; they are transforming their platforms to be AI-native, embedding intelligence directly into the core infrastructure and services. This is not just a feature upgrade; it is a fundamental shift that determines who wins the next decade of digital competition. If you continue to treat AI as a mere application add-on, your foundation will become an impediment. The strategic imperative for every CIO is to recognize AI as the new foundational layer of the modern cloud stack.

This transition from an agility-focused cloud-native approach to an intelligence-focused AI-native one requires a complete architectural and organizational rebuild. It is the CIO’s journey to the new digital transformation in the AI era. According to McKinsey’s “The state of AI in 2025: Agents, innovation and transformation,” while 80 percent of respondents set efficiency as an objective of their AI initiatives, the leaders of the AI era are those who view intelligence as a growth engine, often setting innovation and market expansion as additional, higher-value objectives.

The new architecture: Intelligence by design

The AI lifecycle — data ingestion, model training, inference and MLOps — imposes demands that conventional, CPU-centric cloud-native stacks simply cannot meet efficiently. Rebuilding your infrastructure for intelligence focuses on three non-negotiable architectural pillars:

1. GPU-optimization: The engine of modern compute

The single most significant architectural difference is the shift in compute gravity from the CPU to the GPU. AI models, particularly large language models (LLMs), rely on massive parallel processing for training and inference. GPUs, with their thousands of cores, are the only cost-effective way to handle this.

• Prioritize acceleration: Establish a strategic layer to accelerate AI vector search and handle data-intensive operations. This ensures that every dollar spent on high-cost hardware is maximized, rather than wasted on idle or underutilized compute cycles.
• A containerized fabric: Since GPU resources are expensive and scarce, they must be managed with surgical precision. This is where the Kubernetes ecosystem becomes indispensable, orchestrating not just containers, but high-cost specialized hardware.

2. Vector databases: The new data layer

Traditional relational databases are not built to understand the semantic meaning of unstructured data (text, images, audio). The rise of generative AI and retrieval augmented generation (RAG) demands a new data architecture built on vector databases.

• Vector embeddings — the mathematical representations of data — are the core language of AI. Vector databases store and index these embeddings, allowing your AI applications to perform instant, semantic lookups. This capability is critical for enterprise-grade LLM applications, as it provides the model with up-to-date, relevant and factual company data, drastically reducing “hallucinations.”
• This is the critical element that vector databases provide — a specialized way to store and query vector embeddings, bridging the gap between your proprietary knowledge and the generalized power of a foundation model.

3. The orchestration layer: Accelerating MLOps with Kubernetes

Cloud-native made DevOps possible; AI-native requires MLOps (machine learning operations). MLOps is the discipline of managing the entire AI lifecycle, which is exponentially more complex than traditional software due to the moving parts: data, models, code and infrastructure.

Kubernetes (K8s) has become the de facto standard for this transition. Its core capabilities — dynamic resource allocation, auto-scaling and container orchestration — are perfectly suited for the volatile and resource-hungry nature of AI workloads.

By leveraging Kubernetes for running AI/ML workloads, you achieve:

• Efficient GPU orchestration: K8s ensures that expensive GPU resources are dynamically allocated based on demand, enabling fractional GPU usage (time-slicing or MIG) and multi-tenancy. This eliminates long wait times for data scientists and prevents costly hardware underutilization.
• MLOps automation: K8s and its ecosystem (like Kubeflow) automate model training, testing, deployment and monitoring. This enables a continuous delivery pipeline for models, ensuring that as your data changes, your models are retrained and deployed without manual intervention. This MLOps layer is the engine of vertical integration, ensuring that the underlying GPU-optimized infrastructure is seamlessly exposed and consumed as high-level PaaS and SaaS AI services. This tight coupling ensures maximum utilization of expensive hardware while embedding intelligence directly into your business applications, from data ingestion to final user-facing features.

Competitive advantage: IT as the AI driver

The payoff for prioritizing this infrastructure transition is significant: a decisive competitive advantage. When your platform is AI-native, your IT organization shifts from a cost center focused on maintenance to a strategic business driver.

Key takeaways for your roadmap:

1. Velocity: By automating MLOps on a GPU-optimized, Kubernetes-driven platform, you accelerate the time-to-value for every AI idea, allowing teams to iterate on models in weeks, not quarters.
2. Performance: Infrastructure investments in vector databases and dedicated AI accelerators ensure your models are always running with optimal performance and cost-efficiency.
3. Strategic alignment: By building the foundational layer, you are empowering the business, not limiting it. You are executing the vision outlined in “A CIO’s guide to leveraging AI in cloud-native applications,” positioning IT to be the primary enabler of the company’s AI vision, rather than an impedance.

Conclusion: The future is built on intelligence

The move from cloud-native to AI-Native is not an option; it is a market-driven necessity. The architecture of the future is defined by GPU-optimization, vector databases and Kubernetes-orchestrated MLOps.

As CIO, your mandate is clear: lead the organizational and architectural charge to install this intelligent foundation. By doing so, you move beyond merely supporting applications to actively governing intelligence that spans and connects the entire enterprise stack. This intelligent foundation requires a modern, integrated approach. AI observability must provide end-to-end lineage and automated detection of model drift, bias and security risks, enabling AI governance to enforce ethical policies and maintain regulatory compliance across the entire intelligent stack. By making the right infrastructure investments now, you ensure your enterprise has the scalable, resilient and intelligent backbone required to truly harness the transformative power of AI. Your new role is to be the Chief Orchestration Officer, governing the engine of future growth.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Artificial intelligence’s first great application is in the belly of the beast that birthed it. Computer systems are filled with the hard-coded numbers that make them perfect for applying data-driven machine learning algorithms. Autonomous cars need to fret over fog, wayward pedestrians, and rain. The machines themselves, however, are filled with precise values that lead to crisp decisions. They may not always be simple, but they’re easier than guiding a car through a snowstorm.

Nowhere is the opportunity for AI more evident than in the world of DevOps, a data-rich, back-office practice that presents a perfect sandbox for exploring the power of artificial intelligence. The teams in charge of operations now have a burgeoning collection of labor-saving and efficiency-boosting tools and platforms on offer under the acronym AIOps, all of which promise to apply the best artificial intelligence algorithms to the work of maintaining IT infrastructure.

What AIOps platforms do

Some of the simplest tasks for AIOps involve speeding up the way software is deployed to cloud instances. All the work that DevOps teams do can be enhanced with smarter automation capable of watching loads, predicting demand, and even starting up new instances when requests spike.

Clever AIOps tools generate predictions about machine loads and watch to see whether anything deviates from their estimates. Anomalies might be turned into alerts that generate emails, Slack messages, or, if the deviation is large enough, pager calls. A good part of the AIOps stack is devoted to managing alerts and ensuring that only the most significant problems turn into something that interrupts a meeting or a good night’s sleep.

These methods for watching for unusual levels or activity are sometimes deployed to bolster security, a more challenging task, making some AIOps tools the purview of both security staff and the DevOps team.

Sophisticated AIOps tools also offer “root cause analysis,” which creates flowcharts to track how problems ripple through the various machines in a modern enterprise application. A database that’s overloaded will slow down an API gateway that, in turn, freezes a web service. These automated catalogs of the workflow can help teams spot the underlying problem faster by documenting and tracking the chains of troublemaking. 
Lately there’s more talk of “self-healing” systems that run autonomously. Some managers find it unnerving to give AIOps systems too much leeway. Others are captivated that the machines can clear more IT tickets by themselves. 


Gen AI: The AIOps interface evolves

Some AIOps platforms are integrating more generative AI tools that allow human staff to interact more conversationally with the tools using natural language. The discussion still involves very technical details about the underlying stack, but the conversation happens in a human language, not something like SQL.

There are also mixed feelings about this evolution. Some AIOps tool users believe it will democratize the work to enable people who may not have as much training to oversee the IT estate. Others feel that if the discussion is all about the nuts and bolts of deployment, it won’t make much difference if it’s a bit easier to interface with AIOps platforms in natural language. The conversation will still be very technical at its heart. But even if some aren’t so sure about the need for generative AI, the conversational interface is hard to resist.

What to look for in an AIOps platform

Many of the tools in this survey are built on top of monitoring systems with a long history. They began as tools that tracked events in complex enterprise stacks and have now been extended with artificial intelligence. A few of the tools began in AI labs and grew outwards. In either case, anyone evaluating these platforms will want to look at the range of connectors that gather data.

Some AIOps platforms will better integrate with your stack than others. All offer a basic set of pathways to collect raw data, but some connectors are better than others. Anyone considering adopting an AIOps platform will want to evaluate how well each AIOps offering integrates with your particular databases and services.

Top AIOps platforms available today

Here are 14 of the leading AIOps tools simplifying the job of keeping enterprise IT infrastructure humming.

BigPanda

BigPanda focuses on detecting strange behavior and orchestrating the teams assigned to solve it. Its eponymous platform offers root cause analysis and proactive event detection that integrates with the major cloud providers. Its L1 Automation takes over more of the workload that comes after a problem appears, allowing AI-driven automation to speed smarter decisions. BigPanda simplifies IT’s workflow by creating tickets for systems such as Jira or ServiceNow, sending out alerts, and providing workflow plans with rollback strategies that target root causes. The goal is to create a smart knowledge graph that knows the burgeoning enterprise stack and to provide intelligent plans for keeping it humming.

BMC Helix

IT service management (ITSM) professionals often turn to the BMC Helix platform for managing problems and stack evolution. BMC’s AI-powered solution focuses on both root cause analysis and providing a conversational interface that helps all levels of the team diagnose and fix problems. The BMC Helix platform doesn’t just focus on AIOps and backend workflows; there are also well-integrated products for customer service management and SecOps for supporting outward-facing action.

Datadog

Datadog has been adding AI tools such as Watchdog or Bits to its performance management suite so that DevOps teams get smarter warnings when performance begins to fail. The tools include a collection of ML-based options for building performance forecasts based on historical records adjusted for season and time of day. Changes in metrics such as latency, RAM consumption, or network bandwidth can trigger alerts if they depart from norms. Datadog is adding more agentic services so the tools can act autonomously, reducing the need for human intervention. The company is also offering preview access for options that can analyze code and even rewrite it to eliminate an error. The tool is integrated with Datadog’s security detection system, and it can work with virtual machines, cloud instances, and serverless functions.

Digitate ignio

The ignio AIOps platform from Digitate focuses on closed-loop automation, delivering agility and resiliency to IT and business operations. The focus is monitoring the inward- and outward-facing business health while also optimizing costs, especially in clouds. The company estimates its autonomous collection of tools can handle 40% of issues proactively and reduce manual effort by 60% in typical configurations. There are hundreds of integrations and a low-code tool for adding others. The company’s other products include similar efforts for managing workloads and tracking and solving issues in ERPOps and procurement.

Dynatrace

The three major strategic technologies at the core of Dynatrace are Analytics, AI, and Automation. The machine learning and LLMs are part of a broad, full-featured monitoring tool for tracking cloud-based VMs, containers, and other serverless solutions. In go log files, event reports, and other triggers, and out come what the company calls “precise, AI-powered answers.” The core includes a collection of agents that can be programmed to watch for specific events or collections of events. The AI at the center is called Davis, a deterministic AI that constructs flowcharts and trees so that it can pinpoint the root cause of any anomaly or failure. Davis works in concert with Grail, a data lakehouse filled with telemetry; SmartScape, a tool for mapping the topology of the enterprise; and AutomationEngine, a tool for integrating the gathered intelligence. Properly configured, it can run autonomously by triggering changes, such as rebooting an instance, that should fix the cause without waiting for a human to get in the loop.

GitHub Copilot

Most AIOps tools are designed to help software that’s already up and running. GitHub Copilot starts earlier in the process, helping when code is written. As the company’s ad copy says, “Make your editor your most powerful accelerator.” The tool watches what a programmer types, making completion suggestions. Trained on a gazillion lines of open-source code, Copilot’s ideas are grounded in some form of reality. There are still questions about who is the ultimate author of the new code, whether the AI can be trusted, and whether the millions of open-source coders deserve some credit or hat tip for assistance. The answer may be “perhaps.” A bigger question? How much better does Copilot understand your code, and does it really do much better than autocomplete? That answer: Most of the time Copilot knows.

IBM Watson Cloud Pak for AIOps

IBM created the Watson Cloud Pak for AIOps by integrating its general Watson brand AI with its larger cloud presence. The tool brings automated root cause analysis to data collected from cloud monitoring software. They like to say AI can turn incident response from a crazed search for blame into a unified, information-driven solution-fest. Watson watches constantly over the stream of events until they reach a configurable level of severity. Then Watson responds with a programmable collection of basic alerts or automated responses. IBM has integrated the results with its other Cloud Paks, including Network, Business, and Robotic Process Automation.

LogicMonitor

LogicMonitor is a hybrid extensible platform that gathers telemetry from all corners of an enterprise stack, from the databases and data lakes to the networks and virtual machines. It reaches across cloud services and deep into the on-prem machines. All this data from 3,000-plus integrated collectors is sorted, analyzed, and monitored for anomalies using standard rules and a collection of agentic AIs. The platform bundles a root cause detector with an alert system based on dynamic thresholds adjusted from historical data. Its early warning system depends on a forecasting module that extends this historical data to compute thresholds on latency, bandwidth, and other metrics. LogicMonitor prioritizes reducing “alert fatigue” to avoid the overwhelming “alert storms” to help teams focus their efforts on truly anomalous behavior.

Moogsoft

Moogsoft, now part of Dell Technologies, is a specialized AIOps solution that integrates with major performance monitoring tools such as New Relic, Datadog, AWS Cloudwatch, and AppDynamics. The product moves the data through a pipeline that deduplicates events, enriches them with contextual data from other sources, and correlates the data before raising an alarm. The AI engine deploys generative AI for explanation and various statistical and clustering algorithms to place new alarms in the context of historical behavior. The goal is “noise reduction” to reduce challenges humans face in making sense of the alarms.

New Relic

When problems appear, New Relic uses an AI engine to analyze performance data collected from a range of cloud tracking tools such as Splunk, Grafana, and AWS’s CloudWatch. The tool can be configured with flexible levels of sensitivity for a variety of events of potential severity. You can tell New Relic that, for instance, a low-priority error should raise an alarm only if it occurs several times over 15 minutes. But a high-priority event like a crashed server will generate a pager alert immediately. The issue log tracks all events and includes a Correlation Decision report that lays out the logical steps taken by the AI en route to raising an alarm. Customers have a wide range of ways to customize how the historical data is stored for analysis and retrieval. The goal is to minimize the metrics that measure the mean time to detection (MTTD) and then support the human enough to reduce the mean time to investigate (MTTI) and mean time to resolve (MTTR).

PagerDuty

The name suggests PagerDuty is all about waking up a human to resolve an IT issue. That’s in the past. PagerDuty today proclaims it’s “powered by AI” to make some of the decisions before calling a human. The system focuses heavily on automating much of the incident response whether it’s an internal problem or one that’s raised by customers through its customer support portal. 

ServiceNow

The platform built by ServiceNow is devoted to delivering an army of AI agents to handle any enterprise chore, some of which fall under the same umbrella as AIOps. The IT Operations Management (ITOM) suite, for example, combines machine learning with workflow automations to watch carefully and respond quickly based on past knowledge. The AI Control Tower connects all the agents to a central hub that can answer basic questions about cloud stability and more complex questions about governance and management. ServiceNow’s goal is all encompassing control over practically every corner of the enterprise stack.

ScienceLogic

The Skylar One platform from ScienceLogic aims to deliver a collection of smart observers that watch over and perhaps intercede on behalf of the enterprise cloud. The product is aimed at complex, hybrid environments by building a complete model to give any AI and supervising humans the necessary context for understanding what’s working and, when needed, what’s not. Notable tools inside the tent include a low-code tool for automating workflows the old-fashioned way, and Skylar Advisor, an AI-driven tool that offers advice on how to fix issues. A real-time dashboard using Skylar Analytics gives humans fast visual cues to what’s happening.

Splunk AppDynamics

The Splunk Observability portfolio is designed to watch an enterprise stack, grade its performance, and analyze how that performance affects various business metrics.  AppDynamics, a division of Cisco that has been folded into the Splunk portfolio, can watch over complex stacks, ferret out root causes, and make suggestions for fixing the most crucial parts as quickly as possible. It works with all types of custom and licensed software, on premises, in the cloud, or both. The Splunk AI Assistant offers a conversational interface that uses machine learning to track metrics that diverge from historical baselines gathered from data such as behavior analytics. The system can build a flowchart and learn how events cascade until system failure, thereby helping identify root causes. Agentic architectures built with custom machine learning can be linked with open standards such as Model Control Protocol (MCP). AppDynamics pushes correlating these metrics with hard “business outcomes” such as sales numbers and a “self-healing mentality” for its platform by providing links that can automate the resolution of common failures with a mixture of open standards.