Enterprise AI

Best Cloud Infrastructure for AI Agents in 2026

Compare AWS, Azure, and Google Cloud for agent hosting, model access, security, scalability, and the costs that matter in production.

Edward Yang17 min read

The best cloud infrastructure for an AI agent depends on where its data lives, what its tools execute, and which services the team can operate. Start with the cloud already supporting those systems. Move to another provider when a specific runtime, network, model, or cost requirement justifies the integration work.

AWS AgentCore, Microsoft Foundry Agent Service, and Google's Agent Runtime provide managed agent hosting. Specialist platforms can add a place to run generated code or a way to resume interrupted work. You can adopt those capabilities without moving the whole application.

An agent calling a model API has different requirements from a coding agent running builds or a service hosting model weights. This guide separates those workloads, compares platform capabilities, and explains the billing units. The platform recommendations below are based on documentation, not a cross-cloud performance test.

Figure 1. AWS, Azure, and Google Cloud can all support AI agents. Compare compute, model access, data, security, and operations together when choosing a platform.
Figure 1. AWS, Azure, and Google Cloud can all support AI agents. Compare compute, model access, data, security, and operations together when choosing a platform.
Get Started with Theresa for FREE

Which cloud should you shortlist?

The shortlist below separates business integrations, code execution, and work that pauses for approval.

Your situationStart withWhat would justify a different choice?
Your agent works with existing AWS applicationsAgentCore or your current AWS application hostingA required environment or operating capability that the existing setup cannot provide economically
Employees use Microsoft identity and need an agent in TeamsMicrosoft Foundry Agent ServiceCustom code, networking, or tool requirements that do not fit the selected service
Company data and applications already run on Google CloudGoogle Agent Runtime or existing Google Cloud hostingA demonstrated capability or cost benefit large enough to justify new connections and support work
You already operate reliable application serversYour current container or virtual-machine platformManaged hosting removes enough maintenance work to cover its cost and migration effort
The agent executes generated code or controls a browserA sandbox alongside the applicationRequired packages, persistence, isolation, or hardware are missing from your current execution tools

A new project can start with ordinary application hosting and a model API. Add a managed runtime when its execution and operating features solve a concrete problem. Hosting model weights is a separate decision; an agent using a hosted model does not need its own inference GPUs.

Get Started with Theresa for FREE

What is AI agent cloud infrastructure?

AI agent cloud infrastructure is the computing, storage, networking, and monitoring used to run an agent. It keeps the application reachable, gives its tools somewhere to execute, and stores the information needed to continue its work.

Figure 2. The agent runtime coordinates requests, model calls, and saved state. Controlled tools connect it to business systems, while identity checks, queues, and monitoring support execution.
Figure 2. The agent runtime coordinates requests, model calls, and saved state. Controlled tools connect it to business systems, while identity checks, queues, and monitoring support execution.

The diagram separates the purchases. Application hosting runs the agent; model access supplies its responses; saved progress lets work continue after a pause. Controlled tools may call existing business APIs or use a sandbox to execute generated code. These pieces can share one provider or use separate services. A sandbox's files and a workflow's record of completed actions serve different purposes.

Agent runtime and model runtime

The agent runtime runs the code that coordinates model requests and tool calls. When it calls a hosted model API, ordinary CPU and memory may be sufficient for that application work. The provider operates the model's inference hardware separately.

The model runtime loads the model files and generates responses, a process called inference. Hosting it yourself adds memory, accelerator, startup, and serving-capacity requirements. A deployment can therefore use one provider for the agent and another for inference, subject to its data and network requirements.

Model charges cover the input and output processed by the AI. Hosting charges cover the computing resources used to run the agent's application. If model calls make up most of the bill, cheaper hosting may barely reduce the total cost.

Runtime sessions and durable tasks

A runtime session is a period of work in a running environment. A task can last longer, for example while waiting overnight for approval. Durable execution saves enough progress for a worker to continue after that pause or a failure. It can be part of the hosting platform or supplied by a separate workflow service.

Saving progress does not automatically make repeating an external action safe. A payment or ticket update still needs a way to recognize that it already happened.

Infrastructure and business authority

Cloud identity establishes which service is making a request. The application must also enforce which customer record or business operation that request is allowed to affect. Hosting the agent inside a private network does not replace those rules.

Get Started with Theresa for FREE

How does AI agent infrastructure work?

For an agent investigating an account issue, the infrastructure supports the following steps.

Requests enter an authenticated application

An employee submits a request through a chat interface or business application. The entry point identifies the user, assigns a task ID, and starts a run. Event-triggered tasks use a service identity. Longer work can enter a queue and continue in a background worker after the employee closes the page.

The runtime coordinates models and tools

The runtime sends context to the model and checks proposed tool actions before executing them. It might query account history, search documents, or prepare a ticket update. Results return to the agent for its next decision. Each additional model call or retry affects completion time and cost.

State and records survive interruptions

Save progress outside temporary workers. If an update loses its confirmation, check whether the receiving system saved it before trying again. Our enterprise deployment guide explains recovery and duplicate prevention.

Monitoring follows the complete task

A trace connects a request to its model calls, tool results, and outcome. It shows whether a slow task waited on inference, a database, or an approver. Check the resulting business record too: a successful HTTP response cannot establish that the correct account received the intended update.

Get Started with Theresa for FREE

How to choose the right cloud platform

Choose one representative workflow before comparing features. An account investigation that must read a private database and wait for approval will expose requirements that a standalone chat demo never exercises.

Start with data location and existing systems

List the applications and records the agent needs, including which system holds the current value for each fact. Keeping execution near those systems can simplify access and avoid unnecessary transfers.

For one client deployment, Theresa chose Google Cloud because all of the client's applications already ran there.

Match hosting to the workload

These three illustrative deployments lead to different infrastructure choices:

Work to completeA practical starting setupThe requirement that changes the choice
An internal data agent answers questions using approved database queriesExisting application hosting near the database, a hosted model, and controlled query toolsA private connection or model quota may matter more than a new agent runtime; predefined tools may need no code sandbox
A coding agent changes a repository and runs its testsAn agent application plus a sandbox with build tools and working filesDependencies, memory, and keeping files between sessions determine whether the environment can finish the job
An agent prepares a purchase request, waits for approval, then submits itA worker with saved task state and an approval event that resumes itThe system must remember the approved request and check whether submission already succeeded before retrying

A browser task may need signed-in sessions and downloaded files. A large software build may need far more memory than the application coordinating it. Size the tool environment separately, and choose whether to preserve it or recreate it from saved files.

Test scalability across the entire path

Model quotas, databases, and business APIs can limit throughput before application compute does. For example, 60 tasks per minute requiring five model calls each produce 300 calls per minute before retries. If the model quota is lower, more workers create more throttling; queue work, reduce calls, or obtain more quota.

Concurrency is the number of runs handled at once. Each may hold documents, tool responses, and database connections, so measure memory during busy periods. For interactive work, record time to the first useful response and time to completion separately. A background job may tolerate queuing while still needing to finish before a business deadline.

Examine security at the service level

Check who can invoke the agent, which identity calls each tool, and which records it can access. Keep credentials out of model context and grant tools only the access they need.

A private entry point does not make outgoing model requests or log storage private. Check each selected service's region, network path, and encryption support. For generated code, examine the sandbox's isolation boundary and restrict its network access; isolating the machine does not stop it misusing credentials you deliberately provide.

Decide what must stay available

Set the acceptable interruption for the business task. An internal assistant may fall back to an existing manual process, while an overnight operation may need automatic recovery before its deadline.

A second location helps only if its dependencies remain available. Two agent copies can both fail when they rely on one unavailable database. Test replacement workers and dependency failures, including whether recovery could submit the same action twice. Add multi-region operation when the required recovery time justifies its cost and complexity.

Compare the complete bill

Separate model usage from application hosting, then add data access and operating costs. A cheap CPU rate says little about a task that makes many expensive model calls.

Here is a calculated example using Google's published standard USD rates. Assume 10,000 attempted tasks per month, five Gemini 2.5 Flash calls per task, and 2,000 uncached input tokens plus 400 billable output tokens per call. Output includes any reasoning tokens. Separately assume 100 billable runtime vCPU-hours and 200 GiB-hours of RAM. These are planning inputs, not measured performance.

ComponentUsage and published rateCalculated charge
Gemini 2.5 Flash input100 million text input tokens at $0.30 per million$30.00
Gemini 2.5 Flash output20 million output tokens at $2.50 per million$50.00
Google Agent Runtime CPU100 vCPU-hours at $0.085$8.50
Google Agent Runtime RAM200 GiB-hours at $0.009$1.80
Model and runtime subtotalStandard usage rates before discounts, credits, or tax$90.30

This subtotal excludes storage, database queries, network transfers, logs, paid tools, and engineering or review time. Estimate these costs from the configuration you plan to run. The example uses standard model requests, without batch or caching discounts.

At 8,000 correct completions, the subtotal is about 1.13 cents per correct task. The complete figure must include the omitted costs and any retries beyond the assumed calls. Failed tasks still cost money, even when they produce nothing useful.

Check operational control and portability

Have the operating team deploy a version, inspect a failed run, export task records, and disable a tool during the pilot. Record the staff effort alongside service charges.

Moving an agent means moving its saved progress, identities, and business connections as well as code. Keep business rules and tool interfaces documented outside a provider's console so the team can estimate that work.

Get Started with Theresa for FREE

Best cloud platforms for AI agents

These are reasons to shortlist each platform, not exclusive capabilities or performance rankings.

PlatformReason to consider itTrade-off to examine
AWS AgentCore RuntimeA choice between isolated, on-demand sessions and managed machines for longer work or GPU toolsChoose the compute type around the job. Session duration, resource needs, and billing differ between the two
Microsoft Foundry Agent ServiceA path from configured prompt agents to hosted custom code, with Microsoft identity and publishing to TeamsDecide how much behavior to configure versus code. Private incoming access and outgoing tool connections need separate setup
Google Agent RuntimeManaged agent execution alongside Google's session, memory, evaluation, and monitoring servicesChoose which services the application actually needs. Resource allocation affects cost, and regional and security support varies by service

All three support agent hosting, private business connections, and retained context. Compare what they preserve: conversation history, working files, and completed business actions are different records.

Private access requires the applicable AWS VPC, Azure networking, or Google Private Service Connect configuration. Record permissions remain the application's responsibility.

Specialist platforms can fill a gap in the selected cloud service. Their roles overlap: workflow platforms may host agent code, while compute platforms may also offer sandboxes. Compare the capability you need and the work left to your team.

OptionWhere it fitsWhat remains your responsibility
Control PlaneApplication hosting across cloud infrastructure through a common operating interface; sandboxes are also availableBusiness integrations, task recovery design, and the configuration and cost of the locations you select
E2BSDK-controlled sandboxes for code execution and tools, with filesystem and memory preserved through pause/resumeThe surrounding application, business access, and the record of what the agent completed
ModalSandboxes alongside serverless compute, including GPU-backed execution when tools need itSetup and recovery beyond the 24-hour sandbox limit; GPU sandboxes use gVisor, which filters system calls through a separate software kernel
LangSmith DeploymentHosted agent execution with durable progress, streaming, and scaling; also offers customer-hosted deployment choicesTool integrations and matching hosting and license requirements to your environment; separate code-execution needs
Trigger.devRunning TypeScript background tasks with retries, queues, and waitsBusiness-action duplicate prevention and services outside those tasks, such as databases or code sandboxes

For example, an application can keep its customer data on its existing cloud and call E2B for one code-execution step. Only the information needed for that step should enter the sandbox. That adds another service and data boundary, but does not require migrating the database. Compare this arrangement with the cloud's own execution tools before adding a supplier.

AWS: flexible compute and longer-running sessions

Amazon Bedrock AgentCore Runtime runs your agent code with a choice of frameworks and model providers. Choose isolated virtual-machine sessions (microVMs), lasting up to eight hours, or AWS-managed EC2 Instances for sessions up to 14 days, GPU work, or agents sharing a machine.

With supported stop/resume cycles, a coding agent can resume a stopped session with its files, installed packages, and build outputs intact.

Your application still needs to record completed business actions and decide where to continue. Preserving a repository does not tell the agent whether it already opened a pull request. Tool permissions and connections to business systems also need configuration.

Azure: configured agents or custom code

Microsoft Foundry offers prompt agents configured through instructions and tools, or hosted agents running your code in managed containers. Hosted agents give you control over the framework and request handling.

Designated session files survive idle periods. The Responses interface also manages conversation history; custom request handling leaves that to your code. Sessions expire after 30 days without activity, so they are not permanent document storage.

Recovery after a process interruption requires explicit setup. Foundry can restart the work, while your code saves completed steps and prevents repeated actions. You also configure business-system access through the user's permissions or the agent's own identity.

Google Cloud: managed execution with session and memory services

Google Agent Runtime deploys agent applications from source or containers. You configure the compute resources and number of simultaneous requests; Google operates the hosting.

Sessions stores conversation events and state so an agent can resume an interaction. Google's Agent Development Kit manages sessions automatically when deployed; other implementations can use the API. Memory Bank separately retains information across conversations. Saving a conversation does not preserve the worker's installed software or local files.

Your team connects the required data and tools, grants access, and stores working files where they can survive worker replacement. It also defines how saved conversation state translates into the next unfinished business action.

Pricing comparison: published rates and what they exclude

These USD rates price individual components, so the rows do not represent equivalent deployments. Region, runtime version, allowances, and contract terms can change the applicable bill.

ServicePublished pricing exampleAdditional costs to consider
AWS AgentCore Runtime microVMsV1: $0.0895 per vCPU-hour and $0.00945 per GB-hour. V2 consumption: $0.1276 per vCPU-hour and $0.0169 per GB-hour.Model inference, storage, network traffic, and separately metered capabilities.
Azure Foundry Agent ServiceNo additional agent-service charge for Foundry-native prompts and workflows; hosted agents incur container-compute charges. Obtain the applicable regional rates in Azure's pricing tools.Model tokens, tools, knowledge connections, and relevant licenses.
Google Agent RuntimeStandard rates: $0.085 per vCPU-hour and $0.009 per GiB-hour. Confirm applicable discounts and account-specific credits separately.Model usage and separately billed data, gateway, memory, or other services.

AWS microVM billing distinguishes resource consumption and runtime versions. Google charges for allocated runtime CPU and memory, excluding idle time between turns. A lower hourly rate may therefore produce a higher bill if more resources are allocated than the work uses. GB and GiB are different memory units, so normalize those as well.

Use the same workload to estimate each candidate's bill. The $90.30 example above prices one assumed Google configuration; it does not establish that Google is cheaper or faster than AWS or Azure. Different resource allocation and execution time can outweigh a lower published rate.

When custom hosting is the better choice

Managed agent services are not mandatory. A team with an established container platform may prefer to host its orchestration code there and call model APIs separately. This can suit a simple workflow or code that needs libraries and system tools unsupported by a managed agent service.

Custom hosting is attractive when your team already handles patching, capacity, task recovery, identity integration, and monitoring. Otherwise, budget the engineering work to establish those capabilities alongside compute charges.

Google's architecture guidance treats the agent runtime and model runtime as separate choices and discusses managed agent hosting, Cloud Run, and GKE. The same design question applies elsewhere: which services do you need the provider to operate, and which can your team already run?

Run a pilot that can settle the decision

Compare two plausible configurations with the same tasks, model, and input sizes. Record unavoidable differences, such as provider-specific search, so their effects are not mistaken for better hosting.

Record correct completions, failures, model and hosting costs, and staff effort. Measure typical completion time and p95: the time within which 95% of completed tasks finish. Count failures separately so dropping difficult work cannot make a service look faster.

If neither candidate reaches the required systems reliably, address the connection or data problem before comparing speed. Forward-deployed engineering support can help with integration work that changing hosts would leave unresolved.

Get Started with Theresa for FREE

What to record before choosing

Keep the chosen services, region, model, resource settings, measured costs, and recovery results with the decision. Record why the other candidate was rejected so a later price or requirement change can be assessed without repeating the whole investigation.

Use the results to identify what needs changing: application hosting, tool execution, or task recovery. A problem with one component may not require replacing the others.

The best cloud infrastructure for AI agents is the setup that completes the business's work at an acceptable total cost. Include migration and ongoing support in that decision.

More from the blog

All posts →
Enterprise AI

Best Solutions for AI-Driven Debugging

Seven AI debugging tools compared by the bugs they fit, the evidence they show for a root cause, and what they cost to start.

Joena Zhang13 min read
Enterprise AI

AI Agent Debugging: The Ultimate Guide

How to find out why an AI agent failed: traces show what happened in a run, and the agent's code shows why the system allowed it.

Edward Yang16 min read

See what AI can bring to your business.

Contact us