Environment Variables in Databricks Serverless

Environment variables are now available in Serverless through both the UI and DABs.

What are environment variables?

Environment variables are settings you give to your code, like which environment to use or how detailed the logs should be. You can change these settings without editing the code.

You can access the article on the SunnyData blog for free.

Classic vs serverless

You can use environment variables with both classic and serverless compute, but you set them differently.

| Classic compute                    | Serverless jobs                    |
| ---------------------------------- | ---------------------------------- |
| Configure variables on the compute | Define named entries on the job    |
| through spark_env_vars.            | and select them per task.          |
| Supports init scripts.             | Does not support init scripts.     |

Python reads both the same way, through os.environ or os.getenv.

And job parameters

• Job parameters are useful for inputs that change between runs, like a date or country to process.
• Environment variables are useful for application settings, like…