Confluent Brings Managed AI Inference Directly to Real-Time Streams

Confluent has made Managed Model Inference generally available in Confluent Cloud for Apache Flink, allowing users to call supported AI models directly from real-time data streams through SQL. The company says the service removes the need to manage provider credentials, endpoints and serving infrastructure for those models.
- Managed Model Inference is generally available in Confluent Cloud for Apache Flink.
- Supported models can be called from Flink SQL without provider credentials or endpoint configuration.
- LLMs and embedding models are generally available across Amazon Bedrock, Azure OpenAI and Google Vertex AI.
- Forecasting, anomaly detection and text analysis are in early access.
- LLM and embedding usage is billed at the cloud provider's token price with no Confluent markup.
Inference moves closer to streaming data
Confluent says Managed Model Inference is designed to reduce the delay between an event arriving and an AI-powered decision being made. Instead of sending data through a separate processing pipeline, users can run inference in the stream, so the model's output arrives with the event.
The company points to use cases where timing affects the value of a decision. A suspicious transaction may be stopped while a payment is in progress, while a manufacturing issue can be detected before it causes downtime. Similar timing-sensitive examples include restocking a retail shelf before it empties and identifying unusual energy demand as it happens.
Managed Model Inference runs from Flink SQL against supported models hosted by cloud service providers. Users can run SHOW MODELS to see the models available in their region, then call one by name, such as AI_COMPLETE(`system.anthropic.claude-sonnet-4-6`, 'your prompt').
LLMs and embeddings are generally available
Managed Model Inference for LLMs and embedding models is generally available in Confluent Cloud. Forecasting, anomaly detection and text analysis are in early access.
AI_COMPLETE lets users prompt an LLM for tasks such as classification, summarization, extraction and judgment. AI_EMBEDDING converts text into vectors for semantic search and retrieval-augmented generation.
The supported models come from three cloud platforms: Claude, Amazon Nova and Titan embeddings on Amazon Bedrock, GPT-5.4 and text-embedding-3 on Azure OpenAI, and Gemini and Gemini Embedding on Google Vertex AI. Confluent says the SQL remains the same across clouds, with only the model name changing.
For forecasting and anomaly detection, the service adds AI_FORECAST and AI_DETECT_ANOMALIES. The initial time-series models include IBM Granite TTM, IBM Granite FlowState, IBM Granite PatchTST-FM, IBM Granite TSPulse and Google TimesFM. Text analysis models include Piiranha for personally identifiable information detection and DeBERTa for sentiment analysis.
Managed and external model access
Managed Model Inference sits alongside Confluent's External Model Inference, which has been generally available since March 2025. External Model Inference is intended for models outside the managed list or for a user's own endpoint. It requires users to create the connection, provide their own provider credentials and API keys, and pay the provider directly.
With Managed Model Inference, users call supported models by name without setting up a provider account, storing an API key or operating a connection, endpoint or serving infrastructure. Usage appears on the Confluent bill. For LLMs and embeddings, Confluent says customers pay the cloud provider's token price with no Confluent markup, while usage is metered in input and output tokens and shown separately from Flink compute.
Organization administrators can set a monthly budget in US dollars through the Confluent CLI. Alerts are sent at 50%, 90% and 100% by email, Slack, Microsoft Teams and webhook. Confluent says these budgets send alerts but do not cap spending, so inference continues after 100%.
For users, the announcement means supported AI models can be added to streaming applications with less integration and operational work. The early access categories also extend the same SQL-based approach beyond language and embedding models, while the separate external option remains available for custom endpoints and models outside the managed list.
Confluent: Managed Model Inference Brings AI into Real-Time Data Streams


