Working in a notebook python in interactive mode is easy to install new packages and restarting the kernel. However, when we deal with large datasets to generate embeddings, for instance, running a notebook interactively is prohibitive.

Last week I was working on a project that I need to create embedding using USE (Universal Sentence Embedding) for 500 thousand sentences (each one with around 6 words) and for that I need to create a job to run my notebook.

In Watson Studio every time that you run a job is necessary to install all the packages that are not pre-installed and after that TensorFlow packages needed to restart the kernel.

After my crazy search over the internet, the only answer that I found was to insert some code in the notebook cell to restart the kernel but this didn't work.

The solution was almost hidden in the Watson Studio documentation and turns out that it is pretty simple (you can find the issue here). You need to customize your environment so when you create your python notebook the necessary packages are already there.

So in the following I will show how you can customize your environment in Watson Studio.

The first thing to do is go in environments under management and create a new template.

After that create a new environment and give a name, a description and choose the configuration as language and hardware.

With the new environment set choose your new environment in the list and edit it.

In the edit mode go to Costomization and it will open a space for you to add your packages.

In the customization part you show first add nodefaults with defaults in conda packages since it you need more than on repository. You can add URLs and connect with other repositories in case you are using more complex packages. In my case since packages are from TensorFlow these will be enough.

Since for my package I need last version of TensorFlow I add it bellow pip dependencies packages to get an update.

In the last part I add the names of the packages that I need to be added.

After add your packages you just need to create you notebook and choose the new environment.

In the new notebook with customized environment just import your packages and will be loaded without you need to restart the kernel!

In case that you have a more complex package to add I point you for the documentation of customize environment here.

Hope this post will help you!!!