Preface

Many websites will offer their recommendation engines but they don’t always deliver what you would expect. I.e. after listening to particular songs on Spotify it will continue playing “similar” songs. But in my experience, they are never the ones I want to listen to. In other cases you may want something custom which the recommendation engine doesn’t provide. Perhaps I would like to combine several selections or I wish to filter results based on particular criteria.

In 2023, large language models (LLMs) became a thing. As I’m getting used to OpenAI’s ChatGPT and Google’s Bard I noticed from different examples that they are very good at providing personalized results. In this blog post, I chose to use Bard over ChatGPT mainly because:

1. the free version of ChatGPT is limited to information up to year 2021
2. Bard can provide pictures with links.

To use Bard all you need to is to visit this link and login with your Google account. Now we’re good to dive into several different areas and analyze recommendations.

Movies

Q1: Which recent movies (from 2017) had great reception and are similar to Woody Allen’s style?

Google Bard returned a list of 5 movies with links to IMDB. I can say that I’m pretty impressed with this result because I’ve seen and liked all of them.

Even though I liked suggestions, I don’t want to rewatch them. I asked Bard to provide some more movies and received some others like Licorice Pizza, Frances Ha, Bergman Island. After the second attempt I added some movies to my watch list but noticed that Bard forgot the requirement to provide recent movies.

Music

Q2: Make some hip hop recommendations based on the following albums:
1. INI — Center of Attention
2. Guru — Jazzmatazz, Vol. 2: The New Reality
3. A Tribe Called Quest — The Low End Theory
Also, what’s common between these albums?

This time I received interesting results which seem to be related to my inputs. I’m familiar with some of these albums and will check out other ones as well.

The second part of this question asks to describe what’s common between the albums I listed. And the answer is rich. Bard correctly identifies jazzyness as the common denominator and even describes how these artists relate to jazz.

A2: “These albums all share some common elements, including:
- Jazzy samples: Many of the songs on these albums use jazz samples, which gives them a smooth, laid-back sound.
- Conscious lyrics: The lyrics on these albums often deal with social and political issues, and they often have a positive message.
- Eclectic mix of styles: These albums often mix different styles of hip hop, such as jazz, funk, and soul.
- Innovative production: The producers of these albums often use innovative techniques, such as sampling and scratching, to create a unique sound.
- Strong musicianship: The artists on these albums are all talented musicians, and they play their instruments well.
In addition to these common elements, the albums you mentioned also all have a strong jazz influence. Guru was a founding member of the Gang Starr duo, and he was known for his love of jazz. He even released a series of albums called Jazzmatazz, which blended hip hop with jazz. INI was also influenced by jazz, and their album Center of Attention features samples from jazz artists such as Miles Davis and John Coltrane. A Tribe Called Quest was also known for their jazzy sound, and their album The Low End Theory is considered one of the greatest jazz-hop albums of all time.”

Q3: I like Data Science podcasts like Data Skeptic and Linear Digressions, what would your recommend next?

Again, the recommendations seem to be on point. I.e., “Not So Standard Deviations” seems to be akin to the name “Linear Digressions”. Another suggestion was “Machine Learning Mastery” by Jason Brownlee. I used to read his blog posts and was surprised that he also has a podcast. Except that I checked. And he doesn’t. So Bard was creative on this one.

Q4: I enjoyed the following two books
1. Scale: The Universal Laws of Growth, Innovation, Sustainability, and the Pace of Life in Organisms, Cities, Economies, and Companies written by Geoffrey West 2. The Black Swan: The Impact of the Highly Improbable by Nassim Nicholas Taleb.
What should I read next? Return suggestions and provide their Amazon and Goodreads ratings for context. Limit results only to the ones with Goodreads rating of 4 or higher.

This returns suggestions which I’m somewhat familiar with. I’ve read one of the suggestions already and I’ve seen some of the others in Amazon, Goodreads among similar books. So it’s safe to say that recommendations are useful.

I’ve also requested ratings. For some reason Bard provides ratings only separately, and I’ve checked that ratings are not necessarily accurate. This is something I expect to see improved in future model iterations.

Recipes

Finally, all this experimentation made me hungry! What if you want to make something new based on the products you have at home? I wanted a bowl and asked for additional ingredients:

Q5: I want to make a bowl of buckwheat, tuna and shallots. What other vegetables should I cook together?

As with previous recommendations, I received several common suggestions:

The ingredients suggestion is also packaged with instructions how to cook:

I didn’t use all of the ingredients but I picked up the ones I had in the fridge, such as bell peppers, peas, mushrooms. I enjoyed this meal and will surely repeat. And this picture is for the skeptics who are in disbelief that I actually used Bard’s recipe recommendation:

Conclusions

I’m writing this in September 2023, and these are just several use cases of what an LLM can recommend. You can also use it to generate ideas for attractions, restaurants, gifts, and for many more things. But there’s ample room for LLMs to improve. In some conversations, Bard would forget previous requirements, provide inaccurate results, become overly creative, or limit how it presents information. Nevertheless, you can already begin experimenting with personalized recommendations. Each individual case can be custom-tailored, resulting in outcomes enriched with additional context.

Thanks for reading. If you like this content, consider subscribing me on Medium and leaving your thoughts in the comments. Also, I am happy to connect on LinkedIn!

Introduction

Model interpretability methods are gaining popularity in the field of machine learning. More and more libraries emerged like SHAP, Lime, InterpretML which explain many different models and enable both local and global interpretability (see my previous post). Understanding how features relate to the output is especially important in deep learning where data and models are very complex.

Captum is a new open source library (currently in beta) which is built on PyTorch and is meant to aid in model comprehension. It can be useful to design transparent and better performing models, troubleshoot models in production, show explanations to users and so on.

• Multi-modal. It is suited for a wide range of problems, from tabular data to NLP and computer vision.
• Easy to Use. It produces results with few lines of code and most PyTorch models should be supported.
• Extensible. It is open source and can be extended with new algorithms and features (for example you can define your own feature attribution class) [1].

There is also a bunch of tutorials already to help with getting started. Let’s jump into overview of functionality with some usage examples.

Captum Algorithms

Captum offers a range of algorithms which are grouped into three categories: primary attribution, layer attribution and neuron attribution. Each of these uses methods like DeepLIFT, SHAP or others to calculate attributions.

For experimenting let’s grab some data — scikit-learn’s California housing data set. It consists of 8 features to predict Price (regression task). A 10k sample with some outliers removed indicates that the relationships with target variable are mostly non-linear and looks something like this:

Then, let’s define a basic neural network which is slightly more complex than a linear regression model:

HousingModel(
  (lin1): Linear(in_features=8, out_features=100, bias=True)
  (relu1): ReLU()
  (lin2): Linear(in_features=100, out_features=50, bias=True)
  (relu2): ReLU()
  (lin3): Linear(in_features=50, out_features=10, bias=True)
  (relu3): ReLU()
  (lin4): Linear(in_features=10, out_features=1, bias=True)
)

Primary Attribution

When we are building a new model, one of interesting questions we can ask is which features are important in making predictions? Primary attribution can be regarded as global feature explanations because it evaluates feature contributions to the output of the model. Below are definitions of some of the main algorithms used for primary attribution:

Integrated Gradients. Approximates the integral of gradients along the path (straight line from baseline to input) and multiplies with (input — baseline). Integrated Gradients rely on integrating (summing) the gradients (of the output prediction with respect to the input) over a series of carefully chosen variants of the input. [2].

DeepLift. Explains differences in the non-linear activations’ outputs in terms of the differences of the input from its corresponding reference.

GradientSHAP. Approximates SHAP values based on the expected gradients. It adds gaussian noise to each input example #samples times, selects a random point between each sample and randomly drawn baseline from baselines’ distribution, computes the gradient for it and multiples it with (input — baseline). Final SHAP values represent the expected values of gradients * (input — baseline) for each input example.

NoiseTunnel. Adds gaussian noise to each input example #samples times, calls any above mentioned attribution algorithm for all #samples per example and aggregates / smoothens them based on different techniques for each input example [3].

Without further ado, let’s compute feature attributions with different algorithms (stick to default parameters like baselines and number of steps). Feature attributions are averaged across the entire test set which may lead to some side effects (like positive and negative attributions canceling out each other). Model weights are also included in the comparison:

Main takeaways:

• Based on selected algorithm, results can return entirely different attributions (this can be adjusted by experimenting with algorithm’s parameters).
• Most algorithms are aligned that feature MedInc is important (this one is linearly related to Price).
• Both Latitude and Longitude seem to be important as well but there is an apparent disagreement between algorithms, for example, GradientSHAP does not recognize them as significant. This misalignment may be due to more complex relationships.
• Smoothing Integrated Gradients with SmoothGrad (one of noise tunnel types) did not produce significantly different primary attributions.
• Interestingly, Integrated Gradients, Deeplift and InputXGradient demonstrated practically identical results in all cases. Since the model is quite basic and non-linearities are not so strong DeepLift is close to gradient methods.
• Model weights tells us the learnt correlation with Price —zero weight means no correlation, high means positive and low means negative correlation. Oddly, weights are in the opposite direction of what is expected (higher income should be associated with higher price). Weights may become indirectly correlated with Price as there is more than one layer in the neural network.

Layer Attribution

Next, layer attribution explains each neuron’s contribution to the output of particular layer. In this experiment we will test Layer Conductance, which is an extension of Integrated Gradients applied to hidden neurons.

Layer Conductance. Decomposes integrated gradients via chain rule. It approximates the integral of gradients defined by a chain rule, described as the gradients of the output w.r.t. to the neurons multiplied by the gradients of the neurons w.r.t. the inputs, along the path from baseline to inputs. Finally, the latter is multiplied by (input — baseline).

In a GIF above, each of the 4 linear layers displays Layer Conductance and weights. We can see that for this model in each of the layers only few neurons are responsible for the most of attributions. 4th linear layer contains 10 neurons and the 2nd neuron takes nearly all attributions, while weights are distributed more or less evenly. Also, in the last layer attributions are aligned with weights in terms of direction (this is not valid for other layers).

Neuron Attribution

Finally, the most granular neuron attribution group evaluates contribution of each neuron. Here as well we use Conductance (same definition applies as before) which will help understand contribution of each input feature for particular neurons.

Let’s use the last (4th) layer and pick three neurons for inspection:

• Neuron 0 — attributions and weights are zero;
• Neuron 2 — highest attributions (positive);
• Neuron 8 — small attributions and weights (also positive).

Neuron 0 is not even visible while neuron 2 has the highest variability as expected. Latitude and Longitude are very important (more important than MedInc). What’s interesting is that their contributions are in opposite directions. As mentioned previously, this may cause attribution effects to cancel out when summing their contributions. In turn, this can distort the global picture of feature importance. One way to mitigate this is to look at absolute values for primary attributions (but then information about direction is lost).

Final Notes

This was a brief introduction on Captum, a new model interpretation framework for PyTorch. At its early stage, Captum already provides a lot of algorithms and features which can be used to understand deep learning models better.

One thing to note while running these attribution algorithms on GPU, is some of them can be memory hungry. With 4 GB of GPU memory I did run into errors with DeepLIFT SHAP algorithm which tried to allocate 15 GB: RuntimeError: CUDA out of memory. Since these algorithms have different appetite for resources, one or another may be preferred depending on context. Luckily, documentation provides algorithm space complexity among other information.

There is a lot more functionality with respect to computer vision, NLP or multi-modal models which was not covered, for example:

To be fair, some features overlap with libraries like SHAP and Lime, but Captum is specifically designed for the audience which uses PyTorch and I expect this library to become popular.

Whether it’s used to understand model features, their interactions or to prevent unexpected model behavior, model interpretability is becoming more and more relevant. Maybe eventually model interpretation will become a standard practice for building ML models? It will be interesting to observe what other features this library will bring to the table to really stand out.

References

1. Captum. Model Interpretability for Pytorch
2. How Important Is a Neuron?
3. Captum Algorithm Comparison Matrix
4. Notebook on GitHub

Preface

Statistics and machine learning

Model interpretation (or explainability) recently gained attention in the field of machine learning because ML models are very accurate, but often times it’s relevant to also know what is happening behind the scenes when model makes predictions. This topic actually involves both statistics and machine learning, so lets start with a theoretical discussion and then jump to a practical application.

Classical statistics can be categorized into descriptive and inferential. Descriptive statistics deals with exploring the observed sample data while statistical inference is concerned with making propositions about a population using data drawn from the population with some form of sampling.

Machine learning is sometimes called applied or glorified statistics and there are many more opinions. There is some overlap between statistics and machine learning, but distinctions between these fields are mostly in application. In 2001 paper Statistical Modeling: The Two Cultures Leo Breiman separated the field of statistical modeling into two communities:

1. Data modeling. This community relies on assumptions that there is a stochastic data model and typically is focused on estimating parameters from data and using the model for information. Evaluation is based on goodness-of-fit tests and residual analysis.
2. Algorithmic modeling. This community treats the data mechanism to be unknown. Input x variables are used to predict response y variable. Evaluation is based on predictive accuracy [1].

Machine learning cares about model performance, but explainability falls in learning information category (first community — statistics). Then maybe there is something to be learned from statisticians? R. A. Fisher specified three main aspects to consider for a valid inference:

1. Model specification. George E. P. Box noted that “all models are wrong, but some are useful.” This step involves setting candidate models and selecting one or more models. Picking candidate models is not straightforward, typically it’s a subjective process which comes from experience.
2. Estimation of model parameters. Model parameters are estimated or learned from the data (internal configuration) and are required to make predictions (for ex. linear model estimates coefficients, neural network estimates weights)[2]. In practical terms, estimating model parameters in Python will usually require to call .fit() method of a model.
3. Estimation of precision. Here it is important to set aside some data for model evaluation and select error metrics to measure model’s accuracy. ML algorithms may require hyperparameter tuning. Hyperparameters are usually specified by the user (external configuration) and they help estimate model parameters. Modern ML algorithms contain from few to hundreds of hyperparameters. A standard approach for selecting hyperparameters is to run k-fold cross validation and optimize a desired metric [3].

In statistics the goal of modeling is approximating and understanding the data-generating process. It’s clear that if different ML algorithms (SVM, k-NN, XGBoost, etc.) would be fitted on the same data set there would also be differences in how they explain the data. None of them are regarded as the “true model” which generated the data, but rather an abstraction of empirical data, which can help answer certain questions. Moreover:

“The model does not represent a belief about or a commitment to the data generation process. Its purpose is purely functional. No ML practitioner would be prepared to testify to the “validity” of a model; this has no meaning in Machine Learning, since the model is really only instrumental to its performance.”[4]

To sum up, machine learning cares about performance and many times feature relationships with the target are considered to be a blackbox. But as soon as model explainability becomes important, then more care should be taken (put your statistician’s hat). For instance, some statistical models assume feature independence, while a Data Scientist may deem this to be unimportant. If there are 10 important X predictors which have a high degree of multicollinearity (lets’ say Pearson correlation ρ>0.9), then what may happen is that they will divide the importance of the same underlying driver and it will be more complicated to discern inner workings of such model than of the one having less features.

InterpretML

InterpretML is a new framework backed by Microsoft with a simple idea to bring existing model interpretation frameworks in one place and package them in a practical way. It works with both types of models: glassbox (interpretable — linear models, decision trees, etc.) and blackbox (non interpretable — gradient boosting, SVM, etc.).

Another interesting thing which InterpretML brings is an implementation of a glassbox model — Explainable Boosting Machine (EBM). Authors claim that it is designed not only to be interpretable, but also to have a comparable accuracy to such popular algorithms like Random Forest, XGBoost, etc. You can learn more about its mechanics in InterpretML repository and their research paper [5, 6].

EDA

Data

For the following experiment Predict Ad Clicks data set from hackerearth was selected. To download the data visit the link (login required) and clone this repository to access the code.

For the sake of experiment, a smaller sample was selected — 100,000. This data set is convenient to experiment on because it has only a handful (5 categorical and 4 numerical) predictors to perform a binary classification. More details:

• ID is completely unique and object type, it won't be used;
• datetime contains detailed time and can be preprocessed in different ways to maximize its utility;
• siteid, offerid, category, merchant will be used as numeric columns (even though they actually represent high cardinality categories);
• countrycode, browserid, devid are categorical and therefore will be preprocessed and converted to numeric.
• click is the target variable which will be classified. 1 refers to instances where it was clicked.

Data sample is randomly split into training (70%) and holdout (30%) sets. The random split maintains an approximately the same proportion of target variable in both sets (96% — 0 and 4% — 1).

Numerical features

In this step lets just briefly check feature variation against target variable. Popular visualization libraries like matplotlib and seaborn could be used but that would be too mainstream for a blog post. Lets try something different. For example, Altair visualization library has somewhat intimidating syntax, but produces good-looking plots.

For numerical features (again, they are actually just high cardinality categorical features) density plots should do the trick. With visualization libraries it’s sometimes simpler to write a loop than to make a facet plot:

Altair density plots produce quite choppy distributions (other libraries smooth them out more by default). Distributions are more or less aligned across numerical features, but there is a notable bump in category between 35,000 and 50,000 values:

Categorical features

Categorical features with low cardinality can be conveyed with a heatmap. Since the target click is imbalanced (there are significantly less clicks than no clicks) the key here is to display relative frequency by group (0 and 1 click categories sum to 1 respectively if there are no missing values):

In this case, categorical features seem to be really helpful in separating clicks vs no clicks. For instance, all these factors appear to contribute to more clicks:

• c and d countrycode;
• Google Chrome and InternetExplorer browserid;
• and Desktop devid.

Note that this data set is most likely not fully cleaned because same browsers are named differently as separate categories (IE, Internet Explorer, etc.), but here this will not be accounted for.

Model pipeline

Typically model pipeline is more complex than just calling .fit() and .predict() methods. Therefore, lets construct a simple, but yet realistic pipeline. It’s important to look ahead while doing that. If any methods in model interpretation framework do not support missing values or categorical features, then this should be addressed before the model is built. For instance, InterpretML’s ExplainableBoostingClassifier will raise the following error if any of the features contain missing values:

ValueError: Missing values are currently not supported.

Pipeline will consist of the following steps:

1. Transform datetime feature to numeric (day, hour, etc.) so that a model can make the full use of the date variable. For that, a custom DateTransformer() will be used (inspired by this blog post and StackOverflow answer).
2. Categorical features are required to be encoded in a numerical representation and there are various ways to do that. Instead of creating many dummy variables, lets see if boosting algorithms are able to extract useful patterns themselves. Here OrdinalEncoder() was selected from category_encoders library because scikit-learn version doesn’t support missing values or out of sample encoding.
3. In the data set there are 3 features which contain missing values and often times an ML algorithm will not support them. One option is to impute missing values. Since in this data set there is a mixture of categorical and numerical features, a single solution to impute with mean, median or mode may have unwanted side effects. For this experiment, SimpleImputer(fill_value=0) should do the trick by replacing missing values with 0 (again, let the algorithm do all the thinking).
4. After doing data transformations, the final step in the pipeline is the predictive algorithm (also called an estimator). For comparison reasons two pipelines will be built, one will contain ExplainableBoostingClassifier and the other LGBMClassifier.

Model validation

After building two pipelines it’s important to check whether the models are useful at prediction. The goal is to build explainable models which are not fitted to noise (overfit data) and are able to generalize out of sample. Models which approximate data well, should also provide relevant insight into underlying structure of data.

Without getting into accuracy metric details, one handy method can be used to validate model — scikit-learn’s classification_report() . In this case, it was used for both models on training (70k) and holdout (30k) sets and compressed into a single data frame:

Training set metrics are displayed here just to see how well did the model learn training data. LightGBM can easily overfit smaller data sets and here it shows higher metric values on the training set than Explainable Boosting Machine.

On the other hand, both models show very similar accuracy on the test set. This is an imbalanced classification task and the majority class (0) are nearly perfectly predicted while minority class (1) is predicted reasonably well.

Model interpretation

In this section InterpretML and SHAP Python libraries will be tested on previously created pipelines. The unfortunate part is that model interpretation frameworks refuse to be fully compatible with pipelines and require workarounds.

Lets start with InterpretML, it has a useful ClassHistogram() , which enables doing some EDA on data. There is a caveat. In order to use this, the data set can’t contain missing values, which means it has to undergo pipeline steps. Therefore, lets create a training set which is preprocessed and then visualize it.

from interpret import show
from interpret.data import ClassHistogram

X_t_prep = pd.DataFrame(data=pipeline_ebm[0:3].transform(X_t), columns=feature_names)
hist = ClassHistogram().explain_data(X_t_prep, y_t, name = 'Train Data')
show(hist)

This creates a dashboard which displays Plotly interactive histograms colored by click counts. The supplied preprocessed data set contains only numeric features, for examplebrowserid now is in a numeric representation from 1 to 12. Two of browserid categories show a noticeably high percentage of clicks (but with this encoding it’s unclear which ones):

Next thing to check is global explanations with both InterpretML and SHAP. Since pipeline is not supported directly, the estimator has to be extracted in each case:

# InterpretML
ebm_global = pipeline_ebm['model'].explain_global()
show(ebm_global)
# SHAP
explainer = shap.TreeExplainer(pipeline_lgb['model'])
shap_values = explainer.shap_values(X_t_prep)
shap.summary_plot(shap_values, X_t_prep, plot_type="bar", plot_size=(10,5))

Feature importance summary shows that two categorical features countrycode and browserid are very important. Their ability to separate clicks was already seen in EDA section. There is definitely a slight disagreement between both estimators in feature importance. What they both agree is that month is the worst performing feature without any importance. There is a good reason for that — it has no variance (all clicks happened in same month).

Now lets look at single feature influence on the target variable. Typically clear relationships can be seen for the most important features. To view this, InterpretML requires to call the global dashboard again, while SHAP has dependence_plot method:

#InterpretML
show(ebm_global)
#SHAP
shap.dependence_plot(ind="countrycode", shap_values=shap_values[0], features=X_t_prep, interaction_index=None)

In this particular case, InterpretML has a more appealing visualization, but they both tell the same story (just in opposite directions). SHAP dependence plot tells that 1 and 5 country codes strongly influence the click prediction downwards to 0. InterpretML also shows that the same country codes have the strongest influence.

And finally, for local explanation comparison, one random observation was selected.

ind = [69950]
# InterpretML
ebm_local = pipeline_ebm['model'].explain_local(X_t_prep.iloc[ind], y_t.iloc[ind], name='Local')
show(ebm_local)
# SHAP
shap.initjs()
shap.force_plot(explainer.expected_value[0], shap_values[0][ind,:], X_t_prep.iloc[ind])

EBM predicted 0.35 and LightGBM 0.88, while true value was 1 — clicked. Both SHAP and InterpertML plots display that countrycode was the main driver in their respective explanations. day = 17 has some effect in the decision, but this data sample covers only one month and in a more realistic application constructing a variable something likedayofweek should be more useful.

The downside of SHAP’s so called “force plot” is that feature names which had the smallest impact are not visible.

End notes

In this blog post it was briefly discussed what it takes to build a pipeline which is explainable. To summarize:

• In the beginning there is data. Any preprocessing steps will affect model pipeline, predictions and explanations. When building an explainable model it’s up to the analyst to consider things like incosistent categorical feature labels, determining meaning and usefulness of a feature, missing values treatment, etc. Because this will be seen in the end result.
• An ordinal encoder doesn’t require creating many columns in the model pipeline but it will make the interpretability afterwards more difficult. There would be more clarity with dummy variable encoding.
• A feature with no variance monthreally didn’t make any decisions in LightGBM or Explaining Boosting Machine (EBM) models. In this particular case this feature could be dropped but it would probably become an interesting feature in the real world once there are more months of data.
• This library introduces EBM algorithm which has a comparable accuracy to LightGBM and is designed to be interpretable with InterpretML framework. EBM has a slower training time but it makes predictions fast which can be a crucial factor for a model in production.
• InterpretML library also provides tools like ClassHistogram, which can be useful to conduct EDA, but you may need to sample data. Since it's interactive and runs under Plotly, it may quickly become something that takes long to load as data grows.

Currently InterpretML is in Alpha release and already shows some compelling features. Some functionality is missing, but this shouldn’t discourage to start testing it. In the roadmap there is planned support for missing values, improvements for categorical encoding, R language interface and more.

Thanks for reading, do you have examples/suggestions for improvements (code can be accessed here)? Share them in the comments section, thanks! Also, I am happy to connect on LinkedIn!

References

1. Statistical Modeling: The Two Cultures
2. What is the Difference Between a Parameter and a Hyperparameter?
3. Model Selection and Multimodel Inference: A Practical Information Theoretic Approach, Second Edition (Ch. 1)
4. Machine Learning vs. Statistics
5. InterpretML repository
6. InterpretML: A Unified Framework for Machine Learning Interpretability