noun
Model serving is the deployment of a trained AI model—such as a language or image generator—so users can interact with it. This involves hosting the model on servers, optimizing it for speed, and ensuring it can handle multiple requests.
It typically works by sending user inputs (like prompts) to the model, processing them, and returning outputs (like text or images). Cloud platforms like AWS or Google Cloud often handle this, but smaller models may run locally for privacy or cost reasons.
For developers, model serving changes workflows by requiring infrastructure management, cost monitoring, and sometimes trade-offs between speed and quality. For example, a company using a $0.0001-per-request AI model for customer support might spend $100 daily at 1 million requests, forcing budget adjustments.
A startup serves its image-generation model on a cloud platform, allowing users to create custom artwork for $0.50 per image.
In other languages
See also: Inférence