Artificial intelligence news.
Straight from the original announcements.

Modal launches Sticky Sessions to keep users on the same container

October 20, 2026By the AI News Bites Editorial TeamSource: Modal
Modal launches Sticky Sessions to keep users on the same container
Image: Modal

Modal has launched Sticky Sessions, a feature that sends every request in a session to the same container while keeping serverless autoscaling. It is built on the company's low-latency Modal Servers and targets stateful workloads such as voice agents, chat rooms and multiplayer games.

Key points
  • Modal launched Sticky Sessions, which route all requests in a session to the same container.
  • The feature is built on Modal Servers and enabled with the @modal.sessioned() decorator.
  • Signed session tokens carry routing metadata, so the proxy stores no session mapping.
  • Autoscaling uses active sessions as the unit of concurrency, with queuing when capacity runs out.
  • Target uses include voice agents, shared chat rooms and multiplayer games such as Curvytron.

What Sticky Sessions do

According to Modal, a growing number of workloads need something between autoscaling serverless compute and sandboxes, where users control container lifecycles. Examples the company gives include a speech-to-text model with decoder state for an audio stream, a voice agent that holds a conversation, and a multiplayer game. In these cases, users must return to the same container on every reconnect for the whole session.

To use the feature, developers create a Modal Server with the @modal.sessioned() decorator and start sessions through the SDK or the HTTP API. In Modal's example, a call to Server.sessions.start(idle_timeout=600) returns a session whose token is passed in a Modal-Authorization header.

Modal says earlier options fell short. A WebSocket connection through Web Functions is tied to a Function input, not a container, so a reconnect can land elsewhere. Setting max_containers=1 guarantees stickiness but gives up scaling. The existing Modal-Affinity-Routing-Key header offers only best-effort routing, and the container can be scaled down or replaced at any time.

How Modal built it

Sticky Sessions reuse the infrastructure behind Modal Servers, including the company's in-house stateless proxy, called fprs. Modal lists three things the request-level routing did not provide: fixed routing, a session lifecycle tracked by the container's host, and autoscaling based on sessions instead of requests.

  • Authorization: Each sessions.start() call returns a signed session token. Modal encodes routing metadata directly into the token, so the proxy does not have to store a session-to-container mapping across restarts and replicas. Sharing the token lets several participants reach the same state.
  • Load balancing: Session state lives only on the owning host, so load balancers periodically probe containers for a snapshot of available concurrency. They combine that with recent routing decisions to spread sessions and avoid wasted roundtrips to full containers.
  • Autoscaling: Because session traffic cannot be moved, sessions become the unit of concurrency. A container can run one or more sessions at once. When capacity runs out, new sessions queue until a session ends or a new container is ready. The autoscaler watches active sessions and queued demand.

Use cases and what it means

Modal names three application types the feature supports. The Modal Phonon Voice Bot runs a Pipecat agent with Gradium Phonon text-to-speech over WebRTC, with each call pinned to a container so signaling reaches the peer connection. A sessioned chat room example backs each room with a session, letting clients drop and reconnect to the same room. Gaming applications can keep state in memory to maintain continuity and reduce lag, and the feature supports both preserving progress and hosting concurrent rooms.

Modal also runs its own us-east deployment of the multiplayer game Curvytron on Modal Servers with Sticky Sessions. Players create a room, share the link and play together, and Modal scales based on the active room count.

For developers, the change means stateful, session-bound apps can run on Modal without fixing the fleet at one container. The source does not give pricing or availability details.

Related news

AWS details four ways to speed up Amazon Bedrock AgentCore sessions
For developersNovember 13, 2026, 23:02

AWS details four ways to speed up Amazon Bedrock AgentCore sessions

AWS has outlined four architecture patterns for reducing session startup delays in Amazon Bedrock AgentCore runtime, including snapshot-based initialization, build hooks…

Dynatrace and Microsoft bring observability context to Azure AI workflows
Products and toolsNovember 13, 2026, 18:36

Dynatrace and Microsoft bring observability context to Azure AI workflows

Dynatrace will showcase integrations with Microsoft tools that bring observability data into AI-assisted development, Azure AI applications and cloud operations. The…

Modal introduces burst-based pricing for Sandbox workloads
For developersNovember 12, 2026

Modal introduces burst-based pricing for Sandbox workloads

Modal is changing how its Sandboxes use and are billed for computing resources: they burst beyond a small default request when workloads spike, then release those…