Delta Sharing: A Practical End-to-End Implementation Guide with Databricks, Python, and Spark
Data sharing between organizations is often more complicated than it initially appears.
A typical requirement sounds simple:
“We have a Delta table in Databricks. How can we securely share it with another organization without giving them access to our storage account?”
This is where Delta Sharing becomes useful.
Delta Sharing provides an open protocol for securely sharing data across organizational and platform boundaries. A provider can share selected tables or views with a recipient without giving the recipient direct access to the provider’s underlying cloud storage.
In this article, we’ll build a complete example covering:
- Delta Sharing architecture
- Provider and recipient concepts
- Creating a share
- Adding tables and views
- Creating recipients
- Granting permissions
- Understanding credentials
- Using the credential/profile from Python
- Reading shared data with Spark
- Databricks-to-Databricks sharing
- Sharing filtered views
- Security best practices
- Troubleshooting
- Production recommendations
1. What Is Delta Sharing?
At a high level, Delta Sharing allows one organization to expose selected data to another organization through a controlled sharing mechanism.
Without Delta Sharing, teams often resort to approaches like:
Provider
|
| Export files
v
Cloud Storage
|
| Grant storage access
v
RecipientThis creates several problems:
- Storage permissions must be managed.
- Credentials must be exchanged.
- Data may need to be copied.
- Revoking access can become complicated.
- The recipient may receive more access than necessary.
With Delta Sharing, the architecture becomes:
DATA PROVIDER
|
Delta Table
|
v
Delta Share
|
| Secure sharing
v
Recipient
|
+-------+-------+
| |
Databricks Python/SparkThe provider remains the owner of the underlying data.
2. The Four Important Concepts
Before writing any SQL, it is important to understand four concepts.
Provider
The provider owns the data.
For example:
Company A
Databricks
Unity CatalogShare
A share is a logical collection of tables or views that the provider wants to expose.
Example:
sales_shareRecipient
A recipient represents the external consumer.
Example:
company_bCredential
For open Delta Sharing scenarios, the recipient needs authentication information to connect to the sharing endpoint.
The important distinction is:
Authorization
|
| Who can access what?
v
Share + Recipient + Permissions
Authentication
|
| How does the consumer prove its identity?
v
Credential / Sharing ProfileThis distinction is one of the most important things to understand when implementing Delta Sharing.
3. Reference Architecture
Let’s use a simple real-world scenario.
Company A owns:
production.sales.ordersCompany B needs access to order information.
The architecture is:
COMPANY A
┌──────────────────┐
│ Databricks │
│ │
│ Unity Catalog │
│ │
│ production │
│ └── sales │
│ └── orders │
└────────┬─────────┘
|
v
┌──────────────┐
│ sales_share │
└──────┬───────┘
|
| Delta Sharing
|
v
┌──────────────┐
│ Company B │
│ │
│ Python/Spark │
│ Databricks │
└──────────────┘4. Example Data
Suppose our source table contains:
production.sales.orderswith the following columns:
ColumnTypeorder_idBIGINTcustomer_idBIGINTcustomer_nameSTRINGemailSTRINGcountrySTRINGamountDECIMALorder_dateDATE
Example data:
order_idcustomer_idcustomer_namecountryamount1001501JohnCanada250.001002502AliceUSA450.001003503BobCanada125.00
Now imagine that Company B should not receive email addresses.
Instead of sharing the raw table, we’ll create a controlled view.
5. Create a Governed View
Create a view containing only the columns Company B needs:
CREATE VIEW production.sales.orders_external AS
SELECT
order_id,
customer_id,
customer_name,
country,
amount,
order_date
FROM production.sales.orders;Notice that:
emailis not included.
This is an important design pattern:
Raw Table
|
v
Governed View
|
v
Delta Share
|
v
External ConsumerInstead of exposing your raw data directly, expose a deliberately designed external interface.
6. Create the Delta Share
Now create a share:
CREATE SHARE sales_share;Verify the share:
SHOW SHARES;You should see something similar to:
sales_shareAt this point, the share exists but does not necessarily contain any data objects yet.
7. Add the Table or View to the Share
Add the external view:
ALTER SHARE sales_share
ADD VIEW production.sales.orders_external;You can inspect the share:
DESCRIBE SHARE sales_share;Conceptually:
sales_share
|
+-- production.sales.orders_externalYou can also share a table directly when appropriate:
ALTER SHARE sales_share
ADD TABLE production.sales.orders;However, for external consumers, a governed view is often preferable when you need to control columns or rows.
8. Create a Recipient
Now create a recipient representing Company B:
CREATE RECIPIENT company_b;Check the recipient:
DESCRIBE RECIPIENT company_b;Think about the objects this way:
Share
|
|-- What data?
|
+-- orders_external
Recipient
|
|-- Who?
|
+-- company_b9. Grant Access to the Recipient
Now authorize Company B to consume the share:
GRANT SELECT ON SHARE sales_share
TO RECIPIENT company_b;The resulting relationship is:
production.sales.orders_external
|
v
sales_share
|
| SELECT
v
company_bAt this point, Company B has been authorized to access the share.
But there is an important question:
How does Company B authenticate when it connects?
This brings us to credentials.
10. Where Does the Credential Come In?
This is a common point of confusion.
The following SQL:
CREATE SHARE sales_share;CREATE RECIPIENT company_b;GRANT SELECT ON SHARE sales_share
TO RECIPIENT company_b;
is primarily provider-side configuration.
It establishes authorization.
For an open Delta Sharing implementation, the recipient also needs the appropriate sharing profile/credential.
Conceptually, it contains information similar to:
{
"shareCredentialsVersion": 1,
"endpoint": "https://<sharing-endpoint>/delta-sharing/",
"bearerToken": "<SECRET_TOKEN>"
}The exact credential contents and generation flow depend on the Delta Sharing deployment and Databricks configuration.
The important part is understanding how the credential is consumed.
11. How the Credential Is Used
Suppose Company B receives a profile:
company_a.shareThe recipient stores it securely:
/config/delta-sharing/company_a.shareThe Python application then uses that profile:
import delta_sharingprofile = "/config/delta-sharing/company_a.share"client = delta_sharing.SharingClient(profile)
This is where the credential becomes part of the consumer connection.
The application doesn’t normally need to manually write:
headers = {
"Authorization": "Bearer <token>"
}The Delta Sharing client handles the protocol interaction using the profile.
Conceptually:
Recipient
|
|
company_a.share
|
+-------+-------+
| |
endpoint credential
| |
+-------+-------+
|
v
Delta Sharing Client
|
v
Delta Sharing Endpoint
|
v
Shared DataThis distinction is important:
Provider:
"Company B is allowed to access this."Recipient:
"Here is the credential I use to authenticate."12. Store Credentials Securely
Never put the credential directly into application source code.
Avoid:
TOKEN = "eyJhbGciOi..."Also avoid committing:
company_a.shareto Git.
For example, add:
*.shareto .gitignore.
In production, use a secret-management system such as:
- Azure Key Vault
- AWS Secrets Manager
- Google Secret Manager
- Databricks Secrets
- Kubernetes Secrets
- Your enterprise secret-management platform
A production architecture might look like:
Secret Manager
|
v
Application
|
v
Delta Sharing Client
|
v
Delta Sharing Endpoint13. Install the Python Client
On the consumer side:
pip install delta-sharingThen:
import delta_sharing14. Discover Available Shares
Create the client:
import delta_sharingprofile = "/config/delta-sharing/company_a.share"client = delta_sharing.SharingClient(profile)
List shares:
shares = client.list_shares()for share in shares:
print(share)Example:
sales_share15. Discover Available Tables
You can inspect the available objects through the Delta Sharing client.
Conceptually:
tables = client.list_tables("company_a.sales_share")for table in tables:
print(table)You might see:
sales.orders_external16. Read the Shared Data with Python
Now load the shared data into Pandas:
import delta_sharingprofile = "/config/delta-sharing/company_a.share"table_url = (
profile
+ "#company_a.sales_share.sales.orders_external"
)df = delta_sharing.load_as_pandas(table_url)print(df.head())
Example output:
order_id customer_id customer_name country amount order_date
0 1001 501 John Canada 250.0 2026-09-01
1 1002 502 Alice USA 450.0 2026-09-02
2 1003 503 Bob Canada 125.0 2026-09-03Notice that email is not present.
Get Prabhanshu Kumar Singh’s stories in your inbox
Join Medium for free to get updates from this writer.
The provider controls the exposed schema.
17. Process the Data
Once loaded, the recipient can process the data normally.
For example:
result = (
df.groupby("country")["amount"]
.sum()
)print(result)Output:
country
Canada 375.0
USA 450.0The recipient can now use the data for analytics without needing direct access to the provider’s underlying storage.
18. Using Spark
For Spark-based workloads, the Delta Sharing connector can be used according to the connector/runtime supported by your environment.
Conceptually:
df = (
spark.read
.format("deltaSharing")
.load(
"/config/delta-sharing/company_a.share"
"#company_a.sales_share.sales.orders_external"
)
)Then:
df.show()You can perform normal Spark operations:
from pyspark.sql.functions import sumresult = (
df.groupBy("country")
.agg(sum("amount").alias("total_amount"))
)result.show()
Example:
+-------+------------+
|country|total_amount|
+-------+------------+
|Canada |375.00 |
|USA |450.00 |
+-------+------------+19. Row-Level Filtering
Column filtering isn’t the only useful technique.
Suppose Company B should only receive Canadian orders.
Create:
CREATE VIEW production.sales.canada_orders_external AS
SELECT
order_id,
customer_id,
customer_name,
country,
amount,
order_date
FROM production.sales.orders
WHERE country = 'Canada';Add the view:
ALTER SHARE sales_share
ADD VIEW production.sales.canada_orders_external;Now Company B only receives:
Canada
Canada
CanadaThis pattern is useful for:
- Regional data sharing
- Data residency
- Customer-specific datasets
- Business-unit isolation
- Regulatory requirements
20. Multiple Recipients
A provider can have multiple recipients.
For example:
sales_share
/ | \
/ | \
v v v
Company B Company C Company DCreate another recipient:
CREATE RECIPIENT company_c;Grant access:
GRANT SELECT ON SHARE sales_share
TO RECIPIENT company_c;Different shares can also be used for different consumers:
finance_share
|
+-- invoices
+-- payments
marketing_share
|
+-- customer_segments
sales_share
|
+-- orders
+-- customers
This makes governance much easier.
21. Databricks-to-Databricks Sharing
The experience is different when both organizations use Databricks.
The architecture can look like:
Company A
Databricks
|
v
Delta Share
|
v
Company B
Databricks
|
v
Unity Catalog
|
v
SQL / SparkInstead of manually handling an open-sharing profile in application code, the Databricks-to-Databricks sharing workflow integrates with the Databricks/Unity Catalog environment.
The recipient can then work with the shared data through its Databricks environment.
For example:
SELECT *
FROM shared_catalog.sales.orders;The exact catalog and object names depend on how the recipient configures the shared catalog.
22. Open Delta Sharing vs Databricks-to-Databricks
AreaOpen Delta SharingDatabricks-to-DatabricksConsumerExternal platformsDatabricksPythonYesYesSparkYesYesAuthenticationSharing profile/credentialDatabricks identity/sharing configurationUnity Catalog experienceDepends on setupStrongExternal organizationsExcellentExcellentBest use caseCross-platform sharingDatabricks ecosystem
The important point is:
Don’t assume every Delta Sharing implementation uses a
.sharefile.
The credential/profile workflow is particularly relevant to open Delta Sharing consumers.
23. Security Best Practices
Least Privilege
Only expose what the recipient needs.
Prefer:
Raw Table
|
v
External View
|
v
Delta Shareinstead of:
Raw Table
|
v
Delta Sharewhen sensitive information is involved.
Never Share Storage Credentials
The recipient should not receive:
AWS Access Key
Azure Storage Key
SAS Token
GCP Service Account Keyjust to consume shared Delta data.
Delta Sharing is designed to avoid this type of direct storage access.
Protect the Sharing Profile
Treat the profile as sensitive.
Do not:
Commit it to Git
Send it through public chat
Put it in application source
Store it in a public bucket
Log itRotate Credentials
A good production process is:
Generate new credential
|
v
Deploy new credential
|
v
Test consumer
|
v
Revoke old credentialDon’t revoke the old credential before verifying that the consumer has successfully migrated.
24. Network Considerations
The consumer needs network connectivity to the Delta Sharing endpoint.
A simplified architecture is:
Consumer
|
| HTTPS
v
Firewall / Proxy
|
v
Delta Sharing Endpoint
|
v
ProviderValidate:
- DNS
- HTTPS/TLS
- Firewall rules
- Proxy configuration
- Egress rules
- Private connectivity requirements
- Cloud-region restrictions
- Enterprise network policies
A common production failure is not authentication but simply network connectivity.
25. Troubleshooting
“Share Not Found”
Verify the share exists:
SHOW SHARES;Then inspect it:
DESCRIBE SHARE sales_share;“Table Not Found”
Verify the table/view is part of the share:
DESCRIBE SHARE sales_share;Check that you added the object:
ALTER SHARE sales_share
ADD VIEW production.sales.orders_external;“Authentication Failed”
Check:
Profile
|
+-- endpoint
|
+-- credentialVerify:
- Profile path
- Credential validity
- Credential expiration
- Endpoint
- Network access
- Recipient configuration
“I Can See Data I Shouldn’t”
Review the object being shared.
If you are sharing the raw table:
ALTER SHARE sales_share
ADD TABLE production.sales.orders;consider replacing it with a controlled view.
For example:
CREATE VIEW production.sales.orders_external AS
SELECT
order_id,
customer_id,
customer_name,
country,
amount,
order_date
FROM production.sales.orders;26. Production Architecture
A strong enterprise architecture looks like this:
DATA PROVIDER Unity Catalog
|
v
+---------------+
| Raw Delta Data|
+-------+-------+
|
v
+------------------+
| Governed Views |
| |
| Column filtering |
| Row filtering |
+--------+---------+
|
v
+-------------+
| Delta Share |
+------+------+
|
+----------+----------+
| |
v v
Recipient A Recipient B
| |
v v
Secure Auth Secure Auth
| |
v v
Python Databricks
/Spark |
| |
+----------+----------+
|
v
AnalyticsThis provides a clean separation between:
- Raw data
- Governance
- Sharing
- Authentication
- Consumption
27. Recommended Project Structure
For a real implementation, you can organize the consumer application like this:
delta-sharing-consumer/
│
├── src/
│ ├── main.py
│ ├── delta_client.py
│ └── transformations.py
│
├── config/
│ └── README.md
│
├── tests/
│ └── test_delta_client.py
│
├── requirements.txt
│
├── .gitignore
│
└── README.mdDo not place the actual credential file into the repository.
For example:
.gitignore*.share
.env28. End-to-End Implementation Summary
The entire process can be reduced to these steps.
Provider
CREATE SHARE sales_share;Then:
ALTER SHARE sales_share
ADD VIEW production.sales.orders_external;Create the recipient:
CREATE RECIPIENT company_b;Grant access:
GRANT SELECT ON SHARE sales_share
TO RECIPIENT company_b;Recipient
Securely configure the appropriate Delta Sharing profile/credential.
Then:
import delta_sharingprofile = "/config/delta-sharing/company_a.share"client = delta_sharing.SharingClient(profile)
Load data:
table_url = (
profile
+ "#company_a.sales_share.sales.orders_external"
)df = delta_sharing.load_as_pandas(table_url)And finally:
print(df.head())That’s the complete conceptual flow:
Provider
|
| Create Share
|
| Add Table/View
|
| Create Recipient
|
| Grant Access
|
v
Delta Sharing
|
| Authentication
|
v
Recipient
|
| Python / Spark / Databricks
|
v
Shared Data29. Final Takeaways
The most important concepts to remember are:
1. A Share defines what data is exposed
sales_share
|
+-- orders_external2. A Recipient defines who can consume it
company_b3. Permissions define authorization
GRANT SELECT ON SHARE sales_share
TO RECIPIENT company_b;4. Credentials handle authentication
For open Delta Sharing, the recipient uses the appropriate sharing profile/credential to connect to the Delta Sharing endpoint.
5. The credential is used by the client
For example:
client = delta_sharing.SharingClient(profile)6. Don’t expose raw data unnecessarily
Prefer:
Raw Table
|
v
Governed View
|
v
Delta Share7. Don’t share storage credentials
Delta Sharing should provide controlled access without giving the consumer direct access to the provider’s underlying cloud storage.
Conclusion
Delta Sharing provides a powerful way to implement cross-organization data sharing without creating unnecessary copies of data or exposing the provider’s underlying storage credentials.
The core architecture is straightforward:
PROVIDER
|
Delta / Unity Catalog
|
v
Governed View
|
v
Delta Share
|
v
Recipient
|
Secure Authentication
|
v
Python / Spark / DatabricksFor production implementations, the most important principles are least privilege, governed views, secure credential management, network security, auditing, and credential rotation.
