by Author: Prashant Sharma, Gaurav Bhandari
Co-Authors: Vivek Kanna

Purpose 

As machine learning models grow in complexity—particularly those involving deep learning and GPU-based inference—organizations are looking for ways to operationalize these models without the burden of managing container infrastructure. Traditionally, deploying such models required building Docker containers, pushing them to registries, and orchestrating them in external environments—introducing friction, delays, and infrastructure overhead. With the advent of Model Serving in Snowflake via Snowpark Container Services (SPCS), teams can now deploy and scale externally trained models without ever writing Dockerfiles or managing containers manually. Snowflake handles the underlying container orchestration, letting data scientists focus solely on their models and inference logic. This blog walks through how Model Serving, in combination with the Model Registry, enables streamlined, GPU-accelerated inference entirely within Snowflake—eliminating data movement, reducing operational complexity, and accelerating time to insight.

In this blog, we demonstrate how to deploy and scale machine learning inference in Snowflake using externally trained ML models. We will explore how Snowflake Model Registry and Model Serving enable easy deployment of custom inference solutions on Snowpark Container Services (SPCS), abstracting away the complexity of container management.

Use Cases

With Model Serving in SPCS, we can now deploy use cases with models requiring GPU compute or high inferencing parallelism directly within Snowflake without having to worry about the underlying infrastructure. Some of such use cases are listed below:

• Deep Learning Models
  • Image Processing: Models for tasks such as image classification, and object detection.
  • Natural Language Processing: Transformer-based models like BERT, GPT, and their variants for tasks like text summarization, sentiment, analysis, or translations.
• Advanced Computer Vision
  • Surveillance and Security: Real-time analysis of images for threat detection or facial recognition.
• Large-Scale Data Applications
  • Big Data Analytics: Classification, pattern recognition, and clustering in large datasets.
  • Predictive Maintenance: Forecast equipment failures by analyzing sensor data from industrial systems.

Introducing Model Serving in Snowpark Container Services (SPCS)

Inference is a big part of any Data Science / Machine Learning workflow. 

Snowflake now offers two powerful options for running ML inference: traditional Warehouse-based inferencing and the new Model Serving via SPCS. Let’s explore the scenarios where to use each approach based on model complexity, performance needs, and integration requirements:

• Warehouse Inferencing
  • Models are deployed as UDFs (User-Defined Functions) Stored Procedures inside Snowflake.
  • Inference runs within Snowflake’s virtual warehouses
  • Best for:
    • Lightweight models (e.g., linear regression, basic classifiers)
    • Business intelligence & dashboarding scenarios
      

• Model Serving in SPCS
  • Models run as containerized applications, providing specialized compute resources like Nvidia GPUs.
  • Allows serving models via REST APIs, enabling external systems to interact with predictions in real-time.
  • Best for:
    • Large-scale, complex models (e.g., deep learning, NLP, time-series forecasting) requiring GPUs or are bigger than 5GB
    • Near Real-time, low-latency inference.
    • Cases where models require additional dependencies beyond standard SQL/Python UDFs.
    • Need integration with external applications

Key benefits of Model Serving via SPCS

Model Serving via SPCS enables seamless model deployment, eliminating the need for external tools like VSCode or PyCharm for Python-stored procedures. Developers can build, test, and deploy models directly in Snowflake without manually building and pushing Docker containers, reducing development cycles and operational overhead.

Other key benefits include:

• In-Snowflake Inference – Run models directly within Snowflake with no data movement.
• GPU-Powered Execution – Ideal for deep learning and NLP workloads.
• Custom Environments – Utilize PyPI packages and external dependencies effortlessly.
• Real-Time Inference – API-based model serving with low latency.
• Simplified Infrastructure – No need for separate model hosting or external compute resources.

SPCS also supports both batch and real-time inferencing, ensuring efficient scaling and minimal inference latency. By integrating model deployment within Snowflake, machine learning workloads become more efficient, streamlined, and scalable.

For customers, this results in faster and more reliable access to model-driven insights, as inferencing is directly integrated with their data in Snowflake. There is no need to move data between platforms, reducing security risks and operational overhead. Additionally, this simplifies compliance and governance, as all processing remains within Snowflake’s environment.

From a competitive perspective, SPCS provides an advantage over traditional cloud-based model deployment solutions that require additional infrastructure and complex integrations. By consolidating model deployment and inferencing within Snowflake, organizations can reduce costs and complexity while improving the performance and accessibility of AI-driven applications.

Let’s explore the end-to-end flow of Model Deployment and Serving via SPCS:

1. Snowflake Notebooks for Development and Deployment: Snowflake Notebooks on Container Runtime provides an integrated environment for building, testing, and deploying models without requiring external development tools. This ensures consistency and efficiency throughout the development lifecycle.
2. Image Registry: Before logging the model, the associated environment (including custom dependencies and packages) must be pushed to Snowflake’s Image Registry. This ensures that the model runs in a consistent and reproducible environment when deployed via SPCS. The registered image is referenced during model logging, linking the model to its runtime context.
3. Logging the Model into the Model Registry: Once the model is trained and validated, it is logged into Snowflake’s Model Registry with a single command. This ensures proper version control and easy access for future use.
4. Deployment on SPCS: Deploying the model for inferencing in Snowpark Container Services requires a single command, significantly simplifying the process.
5. Inference Service Attachment to Model Versions: Each logged model version can have an inference service attached, enabling real-time and batch inferencing capabilities within Snowflake. After deployment, batch inferencing can be conducted within Snowflake, leveraging its optimized infrastructure for large-scale processing.
6. Model Registry UI: Snowflake’s UI provides a structured interface to manage, monitor, and analyze model performance, ensuring ease of access and operational efficiency.

Lets glance through the Model Registry UI

Navigate to the Models tab in the snowsight UI:

• Displays a list of registered models.
• Shows metadata like Model name, inference services status, type, owner & created date
• Below, we have a SENTENCE_TRANSFORMER model registered in Model Registry, installed from snowflake-ml-python package. Refer an example notebook here

Inference Services tab in the snowsight UI:

• Displays a list of inference services created.
• Shows metadata like Service name, Status, served model & updated date
• On select shows logs and other details like model being served, endpoint, compute pool etc
• Below, we see an inference service KIPI_EMBEDDING_SERVICE  created for the SENTENCE_TRANSFORMER model which uses a compute pool with name GPU_NV_S.

Below shows the detailed view of a selected model from the registry.

• Displays metadata such as model name, description, and tags.
• Lists all available versions of the model with status (e.g., active, archived).
• Allows users to create a new version or navigate to version-specific details.
• Useful for managing multiple iterations of the same model.
• Each versioned model has a Lineage tab that shows the full data flow lineage information for the model, including any datasets that were used to train the model, any feature views from Feature Store, and the source data tables.

Next, let’s check out the details of the inference service associated with a model version.

• Displays data like endpoint, compute pool, number of instances, functions
• Shows the current status of the inference service and logs in case service is running

Finally, let’s take a look at the files associated with a model version.

• Displays artifacts such as serialized model files (e.g., .pkl, .pt, .onnx), config files, or supporting scripts.
• Allows users to upload, download, or delete files.
• Ensures version traceability by associating specific files with each model version.
• Below we see the files present in the SENTENCE_TRANSFORMER model we registered via snowflake_ml_python package.

Model Registry UI is a friendly and one stop shop for all things related to Model management and associated metadata.

Conclusion

By combining Snowpark Container Services and Model Registry, Snowflake ML enables seamless execution of GPU-based inference for externally trained ML models, regardless of where they were built. This approach:

• Simplifies code deployment.
• Reduces infrastructure management(docker dependency) overhead.
• Ensures scalability for enterprise ML applications.

Leverage Snowflake’s capabilities to unlock the full potential of your machine learning workflows.

Have questions or want to explore how this solution can work for your organization?

Feel free to reach out to us at appsupport@kipi.ai — we’re here to help!

Appendix

Snowflake Notebooks code snippets for Model Serving

^#1 Deploy a Hugging Face sentence transformer for GPU-powered inference using Snowflake Notebook:

1. Install dependencies using “pip” – requires External Access Integration for pypi & external model ^{#2 #3} 

2. Import libraries – snowflake snowpark, snowflake ml and sentence transformer 

3. Define name for Model, Image repository, Compute pool & service. Other than these, Compute pool configurations needs to be defined i.e. # Nodes, Instance family.

4. Establish a snowflake connection & create a Model registry using the session.

5. Download sentence transformer – hugging face model

6. Prepare input data and test embeddings

7. Log the model to the registry. 

Required parameters are: 

1. model
2. mode_name
3. version[Optional]
4. sample_input_data
5. pip_requirements
6. other parameters
7. Get all available model versions

9. Use the latest version of the logged Model

10. Validate compute pool – if no compute pool exists create one

11. Create Image Repository for storing the images built by the service

12. Create service using the selected Model version.

Required parameters are:

1. service_name
2. service_compute_pool
3. image_repo
4. ingress_enabled
5. gpu_requests
6. max_instances
7. Build_external_access_integration
8. other parameters
9. To list all services running in a model

14. To get the service status

15. To use the SQL based Service function

16. Run model using the service and model version – batch inferencing.

Required parameters: 

1. input_data
2. service_name
3. function_name
4. other parameters^#4

References

1. Deploy a Hugging Face sentence transformer for GPU-powered inference
2. External access for Snowflake Notebooks – Create an external access integration for PyPI
3. External access for Snowflake Notebooks – Create an external access integration for HuggingFace

About kipi.ai

Kipi.ai, a WNS company, is a leading analytics and AI services provider, specializing in transforming data into actionable insights through advanced analytics, AI, and machine learning. As an Elite Snowflake Partner, we are committed to helping organizations optimize their data strategies, migrate to the cloud, and unlock the full potential of their data. Our deep expertise in the Snowflake AI Data Cloud enables us to drive seamless data migration, enhanced data governance, and scalable analytics solutions tailored to your business needs. At kipi.ai, we empower clients across industries to accelerate their data-driven transformation and achieve unprecedented business outcomes.

For more information, visit www.kipi.ai and www.wns.com.