Author: Mridul Vij
What is Snowflake Marketplace?
Snowflake marketplace is a feature provided by Snowflake where you can discover and access third-party data and services, as well as market your own data products across the Snowflake Data Cloud.
The Snowflake Marketplace is available globally to all non-VPS Snowflake accounts hosted on Amazon Web Services, Google Cloud Platform, and Microsoft Azure, with the exception of Microsoft Azure Government.
As a consumer, you can access and research a wide variety of data available on the Snowflake Marketplace. This data can be used in machine learning, historical data research, and forecasting (such as traffic and weather reports).
As a data provider, rather than maintaining connections with each individual consumer, you can use listings on the Snowflake Marketplace to share selected data offerings with numerous consumers at once. We will go over some of the various options for listing your data in the marketplace.
Benefits of Snowflake Marketplace
- It enables seamless connectivity to third-party data providers, reducing integration costs, delays, and risks by eliminating the requirement for traditional file transfer techniques.
- It is a feature that gives customers access to a wide variety of data from around the globe.
- It is a feature that allows service providers to list their data and make it widely accessible to everyone.
- It provides providers access to all of the inside information about their data, including who has used it, who is interested in data, etc.
What is a listing?
A listing is an enhanced method of Secure Data Sharing and uses the same provider and consumer model as in Snowflake marketplace.
Instead of establishing sharing relationships with every consumer individually, listing enables you to share your data with multiple customers at once.
The following Listing’s data-sharing features:
- Listing publishes data publicly to all users on the Snowflake Marketplace platform.
- It can be used to charge consumers for access to the data in the share.
- It helps monitor the interest of consumers in your listing and also tell about the usage of the data by consumers in the share.
- It helps consumers by providing metadata about the share, such as a title, description, sample SQL queries, and information about the data provider.
Types of Listings
In total there are three types of listings:
- Private Listing
- Free Listing
- Paid Listings
- Personalized listings
Additionally, listings provide you the choice of making a listing available either publicly on the Snowflake Marketplace, privately to selected customers in the same region, or privately to selected consumers in other regions using replication.
Free Listings
A free listing is a type of private listing. In this listing, Consumers can instantly access a published dataset by clicking on the “Get Data” option.
Free Listing by the name itself indicates that it’s a free-of-cost listing in which the consumer is not required to pay for data, while in some circumstances the supplier may request a small fee and share the data via private share.
This type of listing is mostly used for advertising data, or
for providing non-customer-specific data, sample data for testing or onboarding new clients for full data.
For data sharing, free listing supports Auto-Fulfillment, which enables global access to your data without the need for regional data replication manually. Auto-fulfillment will be covered in more detail later.
Paid Listings
A paid listing is a type of private listing. It is similar to paid listings, however as the name indicates, the data in this listing is paid, and customers are charged for the data they access or consume and get instant access by paying price.
As a provider, you can create paid listings to charge consumers to access or use your data.
Additionally, the provider has the choice to offer a free trial alongside complete data so that customers can review the free trial data before purchasing full data.
In Paid listings, Snowflake will take care of billing and payment collecting for Paid listings and once consumers cancel their data purchase, will no longer have access to the paid listing data.
The pricing model is covered in more detail below.
There are different pricing models in paid listing which are discussed below:
- Usage-Based: In usage-based, the consumer will be charged a monthly data fee plus a per query charge but in months where usage occurs.
i.e consumer will be charged on the month in which they have used or queried the data irrespective of the fact if they are listing data in their database or not. The per-query fee is optional.
There are many options for setting up a free trial of a paid listing, which is discussed below:
- Limited Time: In this, the consumer will only have access to the data during the free trial period for the predetermined amount of time.
- Limited functionality: In this consumers have access to a separate dataset that has limited functionality or has been restricted.
- Limited functionality and Time: It is the combination of 1st and 2nd options.
2. Subscription-based: In the subscription-based, Consumers are charged upfront at the start of a billing or access period.
Depending on the pricing plan, consumers might get billed once or multiple times for a specified billing period.
Note: Consumers cannot repurchase listings that bill one time.
There are two subscription-based plans available:
- Recurring subscription-based plan: In this, consumers will be charged repeatedly for a specified billing period.
- Non-recurring subscription-based plan: In this consumers might get charged one time upfront.
This type of listing is mostly used for selling full data to consumers.
For data sharing, paid listing supports Auto-Fulfillment, which enables access to your data in most of the regions without the need for regional data replication manually.
IMPORTANT: The following function must be used in the paid data secure view to prevent users who have not purchased from seeing the paid data during the free trial.
WHERE SYSTEM$IS_LISTING_PURCHASED() = TRUE
Note:
- If your listing includes a secure user-defined function (UDF), you cannot limit access to the UDF. Both paying customers and trial customers of your listing will be able to access the secure UDF.
- Be careful while attaching the data in the paid listing and use the above where clause to restrict the paid listing from being accessible to free trial users.
- Paid listings are only available in specific regions in a total 32 regions.
Personalized Listings
A personalized listing is quite different from the private listing we discussed earlier. As its name indicates, the data in this listing can be personalized for each user where consumers request specific or custom datasets from providers which, following an agreement, are shared with consumers via direct share.
A personalized listing can contain full data or data that is specific to each consumer.
It is just like a banner in a marketplace that informs customers about the data the supplier has to offer and can be tailored to meet their specific needs.
In personalized listing, pricing needs to be discussed directly with the provider before entering into an arrangement.
In the personalized listing, providers will have to take care of billing and payment collecting for data and also have to handle the access if consumers no longer use the data.
This type of listing is mostly used for selling customized or full data to consumers.
For data sharing, the personalized listing does not support Auto-Fulfillment, providers have to manually create new accounts and replicate data after agreeing on data in the region where the consumer resides.
How to Monitor Usage of your Listing
- The simplest method is to visit the provider studio tab, where you can get aggregated and detailed analytics of the listings under the Analytics tab and overview analytics under the Home tab.
2. To obtain more comprehensive information, you may also use the views that Snowflake provides under the Snowflake Database.
The below-listed Snowflake Database’s Schema can be used for monitoring listings
DATA_SHARING_USAGE and ORGANIZATION_USAGE
Data Dictionary for Listing
Listings Data Dictionary is only supported in private (paid and free) listings. It allows providers to display the contents (table, view and functions) of a listing directly in the marketplace without requiring the customer to install it in their account.
As a consumer, you have two options for viewing a listing’s contents:
- Featured objects: these are the objects that the provider has chosen to appear first.
- All objects: consumers can also view all objects within the listing by clicking View all objects under the Data dictionary tab.
Following are the steps to setup data dictionary in listings:
- Open a new listing by clicking Create Listing.
2. Now, the data dictionary option will be displayed at the top of the listing. If it isn’t, scroll down to the Data Product section to find the data dictionary option and click on the edit button as shown in the picture below.
3. After clicking on the data dictionary option you will be able to see all tables, views and functions present in the listings. From there, you can select the view or object you wish to highlight by clicking on it and adding it to featured.
4. Note We must add comments/ Description to each and every column of the object before connecting a view for a data dictionary so that the consumer gets a better understanding of the data they are getting.
5. We are now ready to connect the object. Once the object is connected, all users of the Snowflake Marketplace will be able to view all objects’ data dictionaries, which is a description of every column under the data dictionary section of listing.
Auto-Fulfillment
It assists the provider in transferring data to different areas where customers are located. Providers do not need to open an account in a new region and replicate data there for data sharing to customers since this is an automatic activity.
This can be enabled in both paid and free listings but not in personalized listings.
Auto-Fulfillment for paid listings is available in only certain region (32 regions) but for free listings it is available on all region which are mentioned below.
Refresh settings for Auto-Fulfillment can be configured up to an 8-day interval, meaning that data will be updated or replicated to accounts created by Auto-Fulfillment every eighth day.
What to do if a view shared in the listing gets recreated and the consumer is unable to see shared data?
When you recreate a view attached to a listing, the view will break its connection to the listing as shown in the picture below and consumers won’t be able to access that listing through their Snowflake account.
Solution: To fix this problem, go to the secure share associated with the listing and use the edit button to reassign the view. By doing this, the view will be reconnected to the listing, and the consumer will be able to access it from the same database they created earlier on their Snowflake account.
IMPORTANT:
- Avoid attaching views other than the one that is necessary because those will also be shared with the consumer.
- Always use a secure share instead of using a table for sharing.
Common error faced while setting up Listing
- Error when enabling Auto-Fulfillment:
Sometimes when a provider tries to enable Auto-Fulfillment in the private listing, It shows an “Auto-fulfillment has failed” error, which does not allow the provider to enable auto replication on that listing.
This error occurs because Snowflake does not allow the user to enable Auto-replication on the database that has already been manually shared or replicated to another account in a different region.
Solution: This can be solved by the following method
- Creating a different database that has all of the objects required for the share, and has not been previously replicated.
- Removing existing replication the current database
- Convert the secondary and primary databases to use replication groups and set up a manual replication group if desired.
2. Error when adding view to secure share:
Sometimes when a provider tries to change the view or re-attach the recreated view to the same secure share, It shows “Failed to update Secure Share”, which prevents users from editing Secure Share or adding view.
This error occurs because a secure view which is getting attached to secure share is referring to a table in another database i.e a secure view created in one database is referring to a table in another database.
Solution: The problem can be fixed by changing the secure view’s reference to a different table that is present in the same database as the view.
This can be done either by creating a view in the same database as the table or by generating a copy or clone of the table in the database where the view already exists.
Conclusion
The Snowflake Marketplace provides a simple way to access third-party data, lowering the costs, risks, and delays associated with integration. It also makes it easier for customers and providers to transfer data internationally. Along with enabling data exchange, Marketplace gives suppliers a variety of tools for monitoring data usage and enhancing data security.
This blog contains a detailed discussion of sharing your data publicly using listings on the Snowflake Marketplace. This blog also covers a variety of problems individuals have while sharing data, along with solutions to those problems.
