Artificial intelligence news.
Straight from the original announcements.

Airbnb Uses Cost per Task to Tune Coding Agents on Amazon Bedrock

November 18, 2026, 22:48By the AI News Bites Editorial TeamSource: AWS
Airbnb Uses Cost per Task to Tune Coding Agents on Amazon Bedrock
Image: AWS

Airbnb is using cost per task, alongside model evaluations and production monitoring, to select models and manage coding agents running through Amazon Bedrock. The approach is designed to track the full cost of completing an engineering task rather than pricing individual model requests in isolation.

Key points
  • Airbnb runs AirChat coding agents through an LLM gateway on Amazon Bedrock.
  • The company evaluates models using its own engineering tasks before rollout.
  • Cost per task includes all token costs across every request needed to finish a task.
  • Airbnb uses CloudWatch metrics to monitor latency, throttling, quota use and spending.
  • Production data can lead Airbnb to change model defaults and effort ceilings.

Airbnb runs coding agents through Bedrock

Thousands of engineers and engineering-adjacent people at Airbnb use AirChat, the company’s internal tool for running coding agents, every week. The agents send model requests through Airbnb’s large language model gateway, which runs on Amazon Bedrock.

Engineers can run AirChat on AirDev workspaces, which are personal development machines in the cloud. They can also hand a task to AirChat Remote, which runs cloud agents in separate cloud workspaces while the engineer works on something else. Both routes pass through the gateway to Amazon Bedrock.

AWS says Bedrock provides access to foundation models from multiple providers through a common API, while AWS manages the model infrastructure. It also integrates with AWS access controls and monitoring, and processes Airbnb’s code, prompts and model responses within AWS.

The pace of model releases makes choosing defaults more difficult. According to Airbnb, eight new models came out in about five weeks. By mid-October, about two thirds of requests sent by its engineers through coding agents went to those new models. Each model release can affect output tokens, cost, latency and capacity at the same time.

Cost per task includes every request

Airbnb evaluates models before rollout using its own engineering tasks. The company measures both a score and the cost per task for every model and every permitted reasoning effort. Reasoning effort controls how much a model reasons before responding.

Airbnb defines cost per task as the list-price cost of all tokens an agent uses to finish one task. Because a coding agent can make many requests and reread the context on each request, Airbnb adds together the costs of the complete task rather than looking at a single request.

The calculation includes uncached input tokens, cache reads, cache writes and output tokens for every request. Each type is priced at its own rate. Airbnb records the average cost beside the score for each model, reasoning effort and agent client, including low-scoring attempts.

The company uses these results to choose a default model and effort for its coding tools, as well as an effort ceiling, which is the highest effort allowed for a model. In one model test described by Airbnb, moving from low to high reasoning effort raised the score by 0.8 points, while cost per task was 2.5 times as high.

Production metrics show when choices need to change

Airbnb’s evaluation system is paired with production monitoring. The gateway records the model, client, team and user for every request, and Airbnb uses those records in aggregate for cost and capacity planning. CloudWatch supplies token counts, latency, errors and throttles.

AWS identifies several key signals in the AWS/Bedrock namespace. InvocationLatency, TimeToFirstToken, OutputTokenCount and derived output tokens per second help investigate latency. InvocationThrottles and request token reservations help track throttling. EstimatedTPMQuotaUsage measures estimated tokens per minute quota consumption across Bedrock inference APIs and can support proactive alarms before usage reaches a quota limit.

For spending, Airbnb monitors InputTokenCount, CacheReadInputTokenCount, CacheWriteInputTokenCount and OutputTokenCount. AWS says cache reads cost less than uncached input, while cache writes can cost more depending on the model, so a lower cache hit rate can increase cost per request.

Airbnb compares these production signals with evaluation scores and cost per task. When real usage does not match evaluation results, the company investigates the cause and can revisit model defaults or effort ceilings. The setup gives engineering teams a way to connect model quality decisions with the operational effects of latency, capacity, throttling and token consumption.

Related news

Kimi Code CLI adds native Amazon Bedrock provider, no separate API key needed
Products and toolsNovember 20, 2026, 01:33

Kimi Code CLI adds native Amazon Bedrock provider, no separate API key needed

Moonshot AI has added a native bedrock provider to Kimi Code CLI v2.7.0, letting the open source coding agent reach models on Amazon Bedrock with standard AWS…

AWS lets AI agents buy x402 products through Marketplace billing
Products and toolsNovember 20, 2026, 00:02

AWS lets AI agents buy x402 products through Marketplace billing

AWS has added AWS Billing to Amazon Bedrock AgentCore payments, allowing AI agents to discover and purchase x402 enabled software, APIs, data and content through AWS…

AWS SageMaker AI Adds Monitoring for Inference Endpoint Drift
Products and toolsNovember 19, 2026, 22:28

AWS SageMaker AI Adds Monitoring for Inference Endpoint Drift

Amazon SageMaker AI can now identify latency drift, underused GPUs and missed serving optimizations on eligible generative AI inference endpoints. The monitoring is…