In this series of posts, we are exploring the challenges of clinical trials and how historical data and computational models can be used to make those trials more efficient. Patient recruitment is the biggest bottleneck for many trials [1]. Using carefully cultivated historical data in place of real patients in the control arm of a clinical trial can reduce recruitment time and time to market [2]. This post will describe how computational models can improve the utility of historical data in clinical trials.

Consider a simple example. Suppose we want to run a trial for a new treatment on patients between the ages of 65 and 75, and we have historical control data from an earlier trial on patients between the ages of 60 and 70. Although the control populations are similar, they are clearly not the same. Can we still use the control arm data from the first trial to inform the new trial?

Clearly, borrowing data from a historical trial is good practice if both study populations are identical. Requiring the historical study population to exactly mirror our current trial’s design and population is impractical, but — with the help of a model — we can use similar historical data to predict the endpoint for the current control arm without sacrificing statistical rigor [3].

To use the historical control arm data in our example, we first need to understand what effect the age covariate might have on our trial endpoint. Once we can measure the effect, we can control for any differences between the historical and trial patient populations. For the sake of simplicity, let’s use a linear model to parameterize the effect of age on our trial endpoint:

endpoint = c + β ∙ age

The parameter β tells us the degree to which a patient’s age influences the endpoint. After fitting this model to the historical data, we can predict the endpoint value and uncertainty in the current control arm by extrapolating the historical data.

Another common extrapolation problem involves translating between clinical trials with different durations. Suppose that the past trial in our example was run for 12 months, while the current trial will run for 18 months. How can we use the historical data now? The historical data covers most of the trial, but we might reasonably expect the endpoint to continue evolving with another 6 months of observations.

As in the previous example, we can use a statistical model to extrapolate to the longer study — again, a linear model:

endpoint = a + b ∙ t

The parameter b controls the variation of the endpoint value with time. As before, by fitting this model on the historical data we can extrapolate the endpoint value and uncertainty from 12 months to predict the values at 18 months in the current control arm.

For many diseases, a patient steadily progresses or has isolated flare states of increased severity. These patterns make it possible to predict a patient’s future state given their current and past disease trajectory. Furthermore, since historical data will generally encompass patients at differing stages of a disease, the historical data captures a wider range of disease progression than the trial duration would indicate. Therefore, historical data covering a shorter period than a current trial may still be useful for predicting the current trial’s endpoint.

When we use data from past clinical trials, we have the opportunity to benefit from the diversity of data drawn from many different studies. Having a mixture of patients at early, intermediate, and late stages of a disease allows us to build a complete model of disease trajectory. Similarly, we can apply a diverse set of historical data to many different scenarios — not just trials that match the design of a previous study.

Models are the key to unlocking the full potential of historical data to inform current trials with a broad spectrum of trial designs and patient covariates. Combining models with historical data even allows us to make predictions about a trial’s outcome given a particular trial design and particular patient population. But before we can rely on these predictions to make critical decisions in the drug development process, we must address the concern that using historical data will increase uncertainty and lead to a loss in power. In our next blog post, we’ll show how comprehensive models can be used to make robust, precise predictions about disease progression in clinical trial control arms.

The Advantages of Modeling Clinical Data for Control Arms was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Those who cannot remember the past are condemned to repeat it.
 — George Santayana

In our last blog post, we talked about why clinical trials are such a bottleneck in the drug development process. Clinical trials take a long time, eating into the patent lifetime of a therapeutic. Why do they take a long time? One big reason is that enrollment is a real challenge. Finding prospective patients and enrolling them is laborious and time consuming. For a major drug, the cost of running a phase III trial can exceed $250 million [1], but the cost of lost time on patent and foregone sales revenue can easily reach billions of dollars a year.

How can this situation be improved? The easiest way to shorten the enrollment period is to enroll fewer patients, but that alone would decrease the statistical power of the trial. Instead, if we reduce the number of patients in just the control arm and replace those patients with historical control data, we can maintain power and shorten enrollment time. This innovative trial design helps useful drugs reach the market faster and allows a greater percentage of patients participating in clinical trials to get what they want and receive potentially life saving experimental treatments. Next, we will examine why this innovation is a good idea in general, and how it can be safely put into practice.

Control arms are a sticking point for patients involved in clinical trials. When a patient enrolls in a trial, they want to be part of the treatment arm so that they can benefit from a potentially transformative therapy. The risk of ending up in the control arm is a significant deterrent for participation; approximately 30% of patients declining to participate in trials express concern over receiving a placebo [2,3,4]. In disease areas without a lot of viable therapies, these control arms are effectively run over and over, in trial after trial. To test whether a particular drug should be put on the market, we frequently don’t need any more control arm data. Historical data is enough.

The fact that historical data can serve as a valid stand-in for a control arm in a trial is well known, and natural history models are regularly used in trials. In disease areas such as oncology with low life expectancies, running control arms can be unethical if they deny patients potential life-saving experimental therapies during a clinical trial [5,6]. In rare diseases, patient enrollment may be so challenging that historical data is a necessity [7,8]. Finally, in disease areas such as neuroscience where few new therapies have been approved, historical data can be ample and well suited to current trials and save time and resources.

Accurately deploying historical data in a clinical trial requires care. The primary concern is that errors can be inflated due to bias in the historical dataset relative to the current trial. Bias between current and historical data can come from many sources: from a covariate, from different trial designs, and from the evolution of the standard of care over time.

A recent paper [9] presents a nice example highlighting how historical data must be properly used. Data from two past metastatic colorectal cancer (mCRC) trials were compared to data from a more recent mCRC trial. Comparing the historical controls to the treatment arm of the newer trial, an apparently significant effect is observed. However, no effect is seen when comparing the treatment and control arms of the newer trial. Taken on its face, the historical data would erroneously lead to the conclusion that the treatment is effective.

Comparing the trials more closely, the distribution of a covariate (a patient’s duration on the chemotherapeutic agent oxaliplatin) differs between the historical controls and the recent trial. Accounting for this covariate corrects the discrepancy, and the effects are consistent across trials. The overarching lesson is that historical data must be carefully matched to the trial data and design, which is a focus of statisticians developing innovative trial designs [10,11].

Innovative trial designs are not only better for patients and trial sponsors, they are also encouraged by regulators. Beyond extensive statements of interest in this area, the FDA has recently begun active support of natural history studies, made budget requests for funding to further invest in natural history models, and launched a pilot program in innovative trial design [12,13,14]. The increasing challenges of drug development combined with the openness and evolution of regulators suggest that it is inevitable that natural history models will play a critical role in future clinical trials.

As we’ve seen, natural history models offer many advantages in situations where enrolling patients in control arms is inadvisable. But we must also recognize that historical data is not a panacea and has its own limitations. Using historical data requires understanding the differences between the populations of trials and their designs. Biases can develop as the standard of care or general health of the patient population evolves. Often, modeling or augmentation of the historical data is required to make that data comparable to a particular trial.

More extensive modeling is one way to improve the utility of natural history models. By using available data to try and predict more about patients, including secondary endpoints or additional covariates, we can build more powerful, realistic, and flexible models of clinical trials and better understand their limitations. In the next post in this series, we’ll discuss comprehensive modeling of patients in more detail.

Historical Data for Control Arms of Clinical Trials was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

New drugs must pass a series of clinical trials before they can be marketed and sold to the public. These trials are meant to prove that new treatments are safe and effective. It is widely known that clinical trials are extremely expensive — the cost of bringing a new drug to market today is estimated to be more than $2 billion.¹ What is less well known is that astronomical cost ranks a distant second to the biggest problem with clinical trials.

The biggest problem with clinical trials is they take too long. The longer a drug spends in development, the longer it takes for that drug to reach the market and benefit patients. Delays to clinical trials cost billions of dollars and affect millions of patients every year.

What’s holding up clinical trials? Regulatory red tape? Scientific rigor? Corporate bureaucracy? If we want to speed up clinical trials, it makes sense to begin with the biggest bottleneck.

Finding and enrolling patients is the most time consuming part of clinical trials. It is not unusual to see trials take years to enroll, and many trials never reach their enrollment goals. 86% of clinical trials fail to complete patient enrollment on time.² 19% of trials fail because they never enroll enough patients.³ To understand why patient recruitment is so difficult and time consuming, it is helpful to review how trial sponsors (e.g. BioPharma companies) go about it.

Trial sponsors don’t just need to recruit patients, they must also send those patients somewhere for treatment and examination. The front line work of clinical trials takes place at sites, which range in size from a single doctor’s office to a large medical center.

Before a trial can begin, the sponsor must identify, contract with, and activate whatever sites it feels are necessary to enroll and treat a sufficient number of patients. First, the sponsor engages with attractive sites that 1) reach a large number of patients that meet trial eligibility requirements, and 2) have the capacity to administer the trial protocol. Next, the sponsor and candidate sites negotiate a financial agreement compensating sites for their participation in the trial. Finally, contracted sites need to be activated. Activation involves everything from securing study approval from an Institutional Review Board to training site staff and providing required materials. On average, it takes 100 days to activate a single site.⁴

While sponsors are identifying and activating sites, they must also find suitable patients. Trial eligibility requirements have grown more strict as study protocols have increased in complexity.⁵ Stricter eligibility requirements reduce the number of potential patients a trial can enroll and increase the time and expense of screening patients. More complex screening discourages patients from enrolling while increasing the number of patients that must be screened in order to meet a trial’s enrollment goals.

Complex study protocols tend to impose a greater burden on participating patients and increase trial drop-out rates.⁶ The higher the drop-out rate, the more patients must be enrolled to say — with statistical certainty — that an observed effect is the result of the experimental treatment and not a result of chance.

Finally, because patients can only be enrolled in one trial at a time, they are a precious resource to trial sponsors. For many difficult to treat diseases, clinical trials must enroll large numbers of patients to achieve enough statistical power to confidently detect a modest effect on disease progression. Multiple treatment arms — a common feature in complex trial designs — further increase enrollment requirements.

Patient recruitment and enrollment present clinical trial sponsors with many challenges. Surprisingly, the most straightforward solution — reducing the number of patients needed for a given trial — has been largely overlooked.

In an upcoming series of posts, we will show how the patient recruitment needs for many trials can by dramatically reduced without increasing the chance of Type I and Type II experimental errors. Furthermore, we will demonstrate how smaller trials greatly reduce the time required to bring new treatments to market.

What’s wrong with clinical trials? was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Someday in the near future, a primary care physician will sit down in front of a computer to pull up all of the medical information about the patient who just walked into her office. The information won’t just be dumped to the screen as a big table of numbers. It will be thoughtfully curated and presented by artificial intelligence to help guide the physician towards the best diagnosis and treatment for her patient.

One way to make A.I. more palatable is to use systems that can explain why they make a particular prediction. The idea is that the physician can take the reasoning into account. She will be more likely to follow a recommended treatment if the evidence used by the algorithm came from a recently published systematic review than if it came from a horoscope. That makes sense.

But, restricting technologies to those that are easily explainable may be putting our algorithms in handcuffs. Human biology is incredibly complex and it is likely that complex algorithms will be required to make sense of it. If that is true, then the physician may end up disregarding all recommendations of the explainable A.I. because it simply doesn’t work.

This vision for A.I. enabled personalized medicine is quite alluring. Patients receive better care at lower cost. Doctors caring for patients instead of combing through charts. Everybody wins — unless they lose.

Machine learning systems make mistakes. They pass along any biases from the data they were trained on to the problems to which they are applied. It is inevitable that A.I. systems will make mistakes in health care, and these mistakes will have real — potentially life-threatening — costs. As a result, many people are concerned about the role that “black box” machine learning models will play in the future of medicine.

Uncertainty is an alternative to explainability that places no constraints on the complexity of the underlying algorithm. That is, algorithms should provide more than just a prediction — they need to describe how confident they are in that prediction. After all, isn’t that all we wanted in the first place? For the physician to know how much she should trust the algorithm?

A.I. and machine learning are going to play an important role in the future of medicine. Luckily, we already have the tools to build models that can describe their degree of confidence, even if they can’t explain why.

Uncertainty is more important to AI than explainability was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Paysage is a new PyTorch-powered python library for machine learning with Restricted Boltzmann Machines. We built Paysage from scratch at Unlearn.AI in order to bring the power of GPU acceleration, recent developments in machine learning, and our own new ideas to bear on the training of this model class.

We are excited to release this toolkit to the community as an open-source software library. This first release is version 0.2.0. We will be pushing regular updates from here on out, with many new features in the pipeline. 😉

The next release (0.3.0) is scheduled for July 10, 2018.

Contents:

• Paysage Capabilities
• Theory
• Annotated Example

Paysage is a powerful library for training RBMs — and more-generally, energy-based neural network models.

Paysage allows one to construct models

• composed of several different kinds of layer types,
• of any depth.

Paysage implements a number of training methods such as,

• layerwise training for deep models,
• (persistent) contrastive divergence with MCMC,
• TAP-based, sampling-free extended mean field methods.

Paysage is built on top of two backends: numpy and PyTorch. The latter unlocks the power of the GPU for greatly optimized training and sampling.

We at Unlearn are excited to release this project and let the community take it to new heights.

What is an energy-based model?

Energy-based models (EBMs) are machine learning models which arise from the specification of an energy function E(X; θ). This function yields a scalar value for each configuration of some specified variables X, and depends on some model parameters, θ, which can be adjusted. Therefore the energy function forms a parameterized family of surfaces over the variables X. Training such a model involves searching through the parameters to find the energy surface which closely matches some desired configuration. A probabilistic perspective on such models is available by virtue of the Boltzmann distribution,

So an EBM also represents a probability distribution over X for each choice of θ. From this perspective, training can then be thought of as fitting the probability distribution p(X; θ) to some desired distribution — perhaps one which is measured by some given dataset.

Visible and hidden variables

It is often the case that only some subset of the configuration variables X are observed in some training set. In this case the observed variables are called visible and the rest hidden or latent variables of the model. So X splits into X = (V, H). In this case the family of probability distributions defined by the model which are relevant to training are the marginals over the hidden variables,

EBMs are natural tools in unsupervised learning

Suppose one observes a bunch of samples of variables V. These samples can be thought of as random draws from some hypothetical data distribution p_d(V). An EBM can be trained to approximate the hypothetical data distribution by fitting p(V; θ) to p_d(V). The resulting model allows one to,

• draw new representative samples from the model distribution, thus simulating the processes which generate the training data,
• answer counter-factual questions about the data variables,
• make discriminative predictions between training data variables by conditioning the model distribution,
• impute missing data from the model distribution.

Furthermore, the fact that an EBM describes an explicit, parametrized probability distribution is a benefit compared to some of the other kinds of generative models.

Energy-based neural networks

An energy-based neural network is an EBM in which the energy function E(v,h;θ) arises from the architecture of an artificial neural network. It is probably best to understand this by way of example. Consider a two-layer Gaussian-Bernoulli Restricted Boltzmann Machine (GBRBM). Such a model has, say, n visible real-valued units, and say m hidden binary-valued units. So a configuration of visible and hidden variables constitutes a pair of vectors (v,h) with an n-dimensional real vector v, and an m-dimensional binary vector h. The visible and hidden units are separated into two layers, the visible called a Gaussian layer and the hidden a Bernoulli layer.

The energy function for this model takes the form,

This energy function has a contribution from each layer (a quadratic potential from the Gaussian layer and a linear potential from the Bernoulli layer) along with a quadratic term which connects the layers mediated by the weight matrix W.

From this one specification we can in principle write down the model probability distribution function p(v; m,s,b,W). The most popular training scheme is to maximize the log-likelihood of the data given the model. That is, we want to compute,

The goal of this optimization problem is to find the parameters θ which maximize the likelihood of having observed the training data from the model distribution. There is more details on training further down in this blog post.

Restricted Boltzmann Machines

Restricted Boltzmann Machines (RBMs) are an important special class of energy-based neural networks. By definition an RBM has at least one hidden layer and carries the systematic restriction that the weight matrices only connect units in adjacent layers. This restriction allows a means of sampling the model distribution (block Gibbs sampling) which is vastly more efficient than the alternatives in the unrestricted case. Paysage was primarily designed to facilitate training of RBMs.

What follows is some more in-depth commentary about a couple of the features of Paysage and an annotated example of training a 2-layer Gaussian-Bernoulli Boltzmann Machine on MNIST.

Generalities on model construction in Paysage

Paysage allows you to construct any model whose energy function takes the form:

in which H_i() is the layer i’s energy function, r_i() it’s rescale function, W_i is the weight matrix connecting layers i and i-1. (Here we have conveniently set h_0 = v to compactify the formula)

In Paysage training and sampling such models relies on block Gibbs sampling; so the layer energy functions must lead to conditional probabilities p(v|rest), p(h_i|rest) that are relatively efficient to compute.

Paysage provides three built-in layer types in the layers module, GaussianLayer, BernoulliLayer, and OneHotLayer. Building a model is as simple as stacking such layers together:

from paysage.models.dbm import BoltzmannMachine
from paysage.layers import GaussianLayer, BernoulliLayer, OneHotLayer

# Instantiate a deep model with four layers:
dbm = BoltzmannMachine([BernoulliLayer(300), GaussianLayer(100), BernoulliLayer(30), OneHotLayer(10)])

How training works:

Maximizing the log likelihood over the parameters can be attempted via stochastic gradient descent. The expectation of the gradient of the log likelihood is,

which involves evaluating expectations over the model distribution. The primary difficulty in training these kinds of EBMs is the ‘intractability’ of the probability distributions p(v,h; θ). In particular, the denominator, Z, in the definition is often impossible to compute analytically. Fortunately there are a number of numerical schemes for apporoximating expectations over distributions of this exponential type.

Paysage implements Markov-chain Monte Carlo via block-Gibbs sampling to evaluate the gradient above (see [1] for the classic presentation of ‘Boltzmann learning’). Alternately, Paysage implements a sampling-free training scheme for RBMs arising from extended mean-field approximations to the free energy of the model (adapted from [2]).

[1] Ackley, David H; Hinton Geoffrey E; Sejnowski, Terrence J (1985), “A learning algorithm for Boltzmann machines”, Cognitive science, Elsevier, 9 (1): 147–169

[2] Tramel, Gabrie, Manoel, Caltagirone, Krzakala
A Deterministic and Generalized Framework for Unsupervised Learning with Restricted Boltzmann Machines.

An annotated example

Let’s now look in detail at the code required to build and train a basic RBM on some real dataset. We can take the binary-valued MNIST example as a reference. A utility to download this dataset is contained in paysage’s mnist example folder. The real-valued data consists of 28x28 images of digits with pixel values in {0,1}.

Since we are going to train with minibatches we need to instantiate a data batcher. The ‘in_memory_batch’ object puts the whole dataset in memory. While using the GPU backend, this means all the data will be loaded into VRAM. So you can only use this for relatively small datasets like MNIST. There are 6000 samples in the training set. We will use a batch size of 100 in this example.

import pandas

from paysage import backends as be
from paysage import batch
from paysage import preprocess as pre

# We have to convert the data to {0,1}:
transform = pre.Transformation(pre.binarize_color)

# fraction in train/validate split
train_fraction=.95

batch_size = 100
samples = be.float_tensor(pandas.read_hdf('path_to_mnist_data'.h5, key='train/images').as_matrix())
data = batch.in_memory_batch(samples, batch_size, train_fraction, transform=transform)

In Paysage a model is specified by a sequence of layer types. Each layer is a standalone object in the ‘layers’ module. The RBM model above is instantiated as a BoltzmannMachine object like so:

from paysage.models.dbm import BoltzmannMachine
from paysage.layers import BernoulliLayer

num_hidden_units=200

vis_layer = BernoulliLayer(data.ncols, center=False)
hid_layer = BernoulliLayer(num_hidden_units, center=True)

# fix gauge by setting hidden params constant
hid_layer.set_fixed_params(hid_layer.get_param_names())

rbm = BoltzmannMachine([vis_layer, hid_layer])

The next step is to initialize the model.

rbm.initialize(data, 'pca', epochs = 20, verbose=False)

Here we select PCA initialization. This scheme initializes the weight columns to the PCA components of the data — iteratively so as to work on arbitrarily large datasets.

Now we select the fitting method. We will train the model on MNIST with the most standard optimization algorithm, stochastic gradient descent.

from paysage import fit

sgd = fit.SGD(rbm, data, fantasy_steps=10)

During training there are a number of generative metrics which are evaluated on the validation data after each epoch. We add an extra that is not part of the default set. The ‘PerformanceMonitor’ is a member of sgd:

from paysage import metrics as M

sgd.monitor.generator_metrics.append(M.JensenShannonDivergence())

Next we select a learning rate schedule and an optimizer.

from paysage import schedules
from paysage import optimizers

learning_rate = schedules.PowerLawDecay(initial=1e-3, coefficient=5)
opt = optimizers.ADAM(stepsize=learning_rate)

ADAM is the default. The PowerLawDecay is a decaying learning rate schedule. Finally we train with the most standard version of Boltzmann-learning, persistent contrastive divergence, PCD.

num_epochs = 200
mc_steps = 10
beta_std = 0.95

sgd.train(opt, num_epochs, method=fit.pcd, mcsteps=mc_steps, beta_std=beta_std)

Here we have selected PCD with 10 iterations of Markov-chain Monte Carlo. The model is sampled with a driven sampler, an advanced sampling method which accelerates training speed.

Once training completes we’ll want to see the metrics progression and fantasy particles.

from . import mnist_util as util

util.show_metrics(rbm, cd.monitor)

valid = data.get('validate')
util.show_reconstructions(rbm, valid, show_plot, n_recon=10, vertical=False)
util.show_fantasy_particles(rbm, valid, show_plot, n_fantasy=5)
util.show_weights(rbm, show_plot, n_weights=100)

mnist_util is an MNIST-specific utility which provides plotting functions to evaluate the performance of models. Here we show the model metrics, some reconstructions from the model, some random samples drawn from the model distribution, and then the columns of the learned weight matrix presented as images.

Here are some fantasy particles:

And some weights chosen at random plotted as images:

There you have it! Hopefully this little introduction-tutorial will get you started experimenting with Paysage yourself.

There are more examples of models trained with Paysage in the recent ML survey, A High-bias, Low-variance Introduction to Machine Learning for Physicists.

We hope that this open-source repository will help the community engage with the power of RBMs. Contributions welcome!

Introducing Paysage was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Recent tech headlines have been dominated by breathless stories about AI bursting into new domains. Brushing aside the hype, there have in fact been astounding achievements in the application of ML techniques to autonomous systems, computer vision, and natural language processing.

Much of these gains have been made in the field of supervised learning — the task of learning a mapping from some input variables to some output variables given a collection of input-output pairs. Indeed, there is a growing viewpoint that the new techniques have rendered several supervised learning domains ‘solved’. It’s hard to argue with such an attitude when a computer model can recognize traffic signs or convert images to text at a rate far superior to humans.

But simpler problems always give way to harder ones. Increasingly, the task of machine learning will be to explore the formidable frontier of unsupervised learning. Unsupervised learning is the task of learning a representation inherent in some data — without anything but the data itself. This definition is broad; it includes summarization, clustering, and density estimation. We can restrict the definition to one closer to what Yann LeCun prefers to call predictive learning — the task of learning a model that can predict the values of any subset of variables in a dataset conditioned on the rest. This confines his notion of unsupervised learning roughly to that of joint probability density estimation. Namely, given a dataset, learn a model which represents either explicitly or implicitly a joint probability distribution over all the variables in the dataset. From such a model one can then perform any prediction task by conditioning. Even under this more restrictive definition, LeCun made the following characterization:

‘If [artificial] intelligence was a cake, unsupervised learning would be the cake, supervised learning would be the icing on the cake, and reinforcement learning would be the cherry on the cake. We know how to make the icing and the cherry, but we don’t know how to make the cake.’ — Yann LeCun NIPS 2016

Most models that estimate joint probability functions are generative. This means that the model affords a means of generating samples drawn from the modeled distribution. Pursuing this exceedingly useful property follows from what we dub the ‘generative thesis’ — that a good way to understand the structure of a dataset is to simulate processes that generate the data. The problem of modeling a dataset becomes the problem of building a machine that can generate new data points as if they had come from the original dataset.

The most often employed generative model types in deep unsupervised learning are Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Restricted Boltzmann Machines (RBMs).

Comparing generative models

The quality of a generative model must be judged by how well its distribution matches the distribution of the data. There are a few metrics that are commonly used for comparing different types of generative models.

Visual Inspection: The most common method to assess the performance of a model is to look at the samples it has generated. Many people object to this because it is qualitative. However, it is also inappropriate for non-image or textual data. For example, one probably cannot form a useful opinion about a generative model for protein sequences just by looking at the sequences it spits out.

Reconstruction Error: VAEs and RBMs can be viewed as methods that can take a data point, embed it into a lower dimensional space, then reverse the embedding to reconstruct the original data point. The error of the reconstruction tends to decrease during training, but one can easily obtain models with low reconstruction error but terrible samples. Moreover, reconstruction error isn’t an appropriate metric for GANs (which don’t do reconstruction).

Inception Scores: Inception is a neural network trained on ImageNet to perform object recognition. A number of methods for evaluating generative models of images compute scores based on the output of the inception network. These scores may correlate with human perception of image quality (so this is a type of automated visual inspection). However, inception scores obviously don’t apply for anything that isn’t an image because the inception network is a model trained on images.

Log Likelihood: Estimating the log likelihood of the data under the model using Parzen window estimates or Annealed Importance Sampling is another common approach for quantifying the performance of generative model. The log likelihood is an appropriate metric for all models and data types but can be difficult to estimate. Moreover, log likelihoods can be difficult to interpret in high dimensions [1].

Measuring the distance between the data and the model

Let p_d(x) be the data distribution density and let p_m(x) denote the model distribution density. What is a good measure of the distance between these distributions?

There are a number of means of quantifying the distance between these distributions. A particularly common set is the f-divergences, in particular the Kullback-Leibler Divergence (KLD), Reverse Kullback-Leibler Divergence (RKLD), and the Jensen-Shannon Divergence (JSD).

In our recent paper on Boltzmann Encoded Adversarial Machines, we implemented an approximation to these metrics to help guide real-time insight into training dynamics. Although the technique was presented in [2] back in 2009, to our knowledge no other ML paper uses online estimates of the KL, RKL, and JS divergences.

The Kullback-Leibler Divergence (KLD) ,

is a positive-valued metric valued on the extended line (it can be +infinity), which is zero if and only if p(x)=q(x) for all x.

Importantly, this divergence is not symmetric. As a result it is useful to study both the KL(p_d || p_m) and the Reverse-KL(p_d || p_m) :

Look, for instance, at the following example of a GAN in which the model is learning to represent the mixture of Gaussians shown in orange (Figure 1).

Figure 1. Three f-divergences plotting during GAN training. On the bottom row, the orange density is the ground truth mixture of gaussians distribution, whereas the blue densities are the model distributions at epochs 15 and 65.

At the start of training the model learns to cover the whole domain with a smoothed distribution, thus achieving a fairly low KLD. However, the central hole in the data contributes to a rather large RKLD until the model learns to push that area down. After epoch 55 or so the model actually starts to forget some of the modes — a phenomenon called mode collapse — which is known to afflict some types of GANs.

In general KLD is sensitive to the model’s failure to cover all of the modes of the training dataset. Complimentarily, the RKLD is sensitive to spurious modes in the model distribution.

One problem with these divergences is that they are overly sensitive — they can explode when comparing distributions with disjoint support. A more stable comparison, the Jensen-Shannon Divergence (JSD),

is symmetric and finite. There are theoretical reasons why this should fall during training, and we can see that in Figure 1.

These f-divergences are useful in quantifying the quality of a model’s fit. But there is much more to be said in this regard. Many recent advancements in generative ML (esp. with models trained via back propagation) recommend estimating courser, more-stable distributional metrics in order to define a training loss. See WGANs, Cramer-GANs, Sinkhorn Audodiff, OT-GANs and more.

Moving beyond benchmarks

‘All models are wrong, but some are useful.’ — George Box, Robustness in Statistics, 1979

There is a trend in machine learning research to set up a “battle royale” in which the performance of different algorithms are compared on a number of benchmarks. Example projects include MLPerf and DAWNbench. In doing so, however, the community needs to be careful not to put our own biases into these tests.

First, current benchmark initiatives coming from the machine learning community are strongly biased towards computer vision and natural language processing. There are many applications of machine learning outside of these areas that require different focus: e.g., handling multimodal data, missing observations, and time series. Most models cannot be applied to all types of problems. Is a new technique better if it enables state-of-the-art performance on a specialized task, or general performance improvements on many tasks?

Second, models can be wrong in different ways. Therefore, different metrics will rank models differently. For example, the KLD punishes generative models that drop modes from the data distribution whereas the RKLD punishes models with spurious modes. Is it worse for a generative model to drop a mode or to have a spurious mode?

Finally, there are many factors that affect the decision about which technique to apply for a real problem. Obviously, we care about the performance of the model. We also usually care about the computational cost in training the model (which is incorporated in recent benchmarks like MLPerf and DAWNbench). However, applied researchers also care about development time, which is hard to measure. After all, many people write code in Python even though it runs much slower than Fortran on benchmarks. Similarly, some types of machine learning models are finicky. Is it better to have a model that performs pretty well after a small amount of hyperparameter tuning, or a model that performs really well but requires a lot of hyperparameter tuning?

We may have to accept that the search for universal principles of intelligence may not lead to a universal algorithm for artificial intelligence. Instead, we may end up with an ecosystem of algorithms — each one filling its niche.

— — — — — —

[1] A Note on the Evaluation of Generative Models, Lucas Theis, Aaron van der Oord, and Mathias Bethege. ICLR 2016.

[2] Divergence Estimation for Multidimensional Densities via k-Nearest-Neighbor Distances, Qing Wang, Sanjeev R. Kulkarni, and Sergio Verdú, IEEE Transactions of Information Theory, vol. 55, no. 5, May 20, 2009.

The travails of comparing generative models was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

The first step in understanding something is to identify the right level of description for the problem [1]. For example, imagine an engineer who is working on a prototype for a new car. The engineer needs to draw up a set of blueprints describing the design. Should she write down the molecular structure of the car to illustrate all of the bonds between its constituent molecules? Of course not — that would be the wrong level of description. Instead, she should write down the structure of the car in terms of its natural components like its wheels, engine, and windshield.

Now imagine a computational biologist who wants to build a model to predict the prognosis of a patient with Alzheimer’s disease. What variables should he choose? The patient’s DNA sequence, transcriptome, or proteome? Maybe it’s not individual genes that matter but biochemical pathways and cellular processes?

Unfortunately, despite all of our progress in biology we don’t know the right level of description for the field. And this ignorance is holding back progress in medicine. Before getting in to the details, however, it may be helpful to step back and provide some background on molecular biology. This section gives a small glimpse into the complexity of biological systems. Feel free to skip ahead if you don’t need an intro to molecular biology.

The most basic introduction to molecular biology

A person’s genetic code is a sequence of nucleotides (the chemicals adenine, thymine, cytosine, and guanine) that are strung together like letters in a document. A document probably contains multiple sentences; and each sentence should contain multiple words.

A genetic word is a sequence of three nucleotides called a codon. A genetic sentence is a sequence of codons called a gene. Each gene provides instructions on how to make a molecular machine called a protein. Humans have approximately 20,000 genes providing instructions on how to make approximately 20,000 proteins.

The process of manufacturing one of these molecular machines begins in the nucleus of each cell. The DNA in a cell is tightly wound around proteins called histones, folded up like a crumpled piece of paper. To read a sentence, the cell has to stretch out that portion of the DNA by unwinding it from the histones.

Once a stretch of DNA is unwound, the cell points to a particular sentence it wants to read. A protein called a transcription factor binds to the DNA at the start of this gene. This initiates a process called transcription where the cell copies the instructions in the gene into a new sequence called a messenger RNA.

The cell copies the message for two reasons. The first reason is that the assembly line that makes the molecular machines is located in a different part of the cell. Therefore, the sentence has to be copied and taken to the assembly line for manufacturing. The second reason is that each copy of the message will be used to produce a fixed number of copies of the desired machine. Therefore, the cell can control how many of each machine it makes by controlling how many copies it makes of each message.

Once the messenger RNA reaches the assembly line, an assembly line worker called a ribosome reads the instructions and creates a protein. A protein is a molecule constructed from 20 chemicals called amino acids. Each of the codons in the messenger RNA represents an amino acid. The ribosome reads each of these words in the sentence and strings the corresponding amino acids together to create a protein.

Proteins are the molecular machines of the cell and are responsible for most cellular activities. They transmit signals by changing shape when they bind to specific molecules; and they facilitate chemical reactions that produce materials the cell needs.

What do we measure?

Instead of being driven by what we should measure, most models in biology are driven by what we can measure. For example, some common types of genomic data include:

Single Nucleotide Polymorphisms (SNPs): If each genetic word is a sequence of three nucleotides that tells the ribosome which amino acid to use next when building a protein, then changing one of the letters in a word could make the ribosome use a different amino acid. Therefore, one level of description for biology could be to catalog all of these one letter changes in the entire DNA sequence. That would be a really long catalog, so biologists often restrict the analysis to single letter changes that occur in at least 1% of the population. These are called Single Nucleotide Polymorphisms (or SNPs).

It’s pretty easy to make a catalog of the SNPs in an individual person. SNP based tests are now used identify ancestry and predict the relative risk of some diseases. While there are statistical associations between some SNPs and diseases, it is very rare that a SNP is highly predictive.

Transcriptomics: All of the messenger RNAs created by a collection of cells is called a transcriptome. Unlike a person’s DNA sequence, which is effectively the same throughout life, a person’s transcriptome is constantly changing in response to his/her environment. A cell controls the number of copies of its molecular machines by controlling the number of copies of each machine-encoding message. Therefore we may be able to learn a lot about a cell (or collection of cells) by counting all of these messages. Today transcriptomics experiments are often done by sequencing all of the messenger RNA molecules with a technology known as RNAseq.

Proteomics: Since proteins are the molecular machines of the cell, it may make sense to directly measure the amounts of each protein in a collection of cells. Experiments that measure the amounts of different proteins are collectively called proteomics; there are several different techniques for making these measurements. In many cases only a small subset of all of the 20,000 possible proteins can be measured in any given experiment.

Metabolomics: Although proteins are the molecular machines of the cell, they are usually busy creating other chemicals needed by the cell. The levels of these chemicals called metabolites can also be measured. These types of experiments are collectively known as metabolomics.

What should we measure?

Does one of the types of measurements listed above stick out to you as the right level of description for biology?

My opinion is that we don’t know — and in all likelihood, none of these experiments are likely to capture the right level of description. SNP arrays, RNAseq, proteomics, and metabolomics are simply types of experiments that address what we can measure — not what we should measure. Why should nature be so simple that what we should measure also happens to be one of the things that is easy to measure?

Although the experiments we have today are certainly very valuable, they haven’t brought about the revolution in precision medicine that we were hoping for. To get there I think we need to pursue two parallel lines of research. In the short term, we need to develop machine learning based approaches that can integrate data from many types of experiments in a way that is robust to our ignorance. Below, I’ll argue that techniques from deep learning have the potential to solve this problem by automatically constructing features. In the long term, we need to invest in basic research that facilitates collaborations between experimental and theoretical biologists so that we can answer the question — “what is the right level of description for biology?”.

Discovering Abstractions with Machine Learning

Machine learning is the use of computer algorithms that learn to make predictions without any human guidance. Progress towards this goal has advanced rapidly in recent years due to a class of algorithms known as Deep Learning.

Teaching a computer to make accurate predictions is not that difficult if you know the right way to describe the problem. In machine learning, people say that you have to know the right features. In the past, human experts would draw up rules for combining raw data (e.g., gene expression data) into higher level features that reflected prior knowledge (e.g., the activation of biological pathway). However, this process of feature engineering requires a lot of effort and only works if you have a lot of accurate prior knowledge.

Deep Learning algorithms provide a technological leap forward because they learn to construct the right features all by themselves. To do this, a Deep Learning algorithm takes in the raw data and transforms it. Then it takes the transformed data and transforms it again, and so on. This process of repeated transformations allows the model to discover the right level of abstraction for the data.

The figure above shows an example of this process for a dataset consisting of images of faces. The first layer of transformations identifies edges and blobs of color in the images. The second layer of transformations combines the edges and blobs into facial features like the eyes, nose, and mouth. Finally, a third layer of transformations combines the facial features into representations of faces.

The same process can be used to discover the right level of description for problems in biology. Though, there are barriers to the application of Deep Learning in biology that are less of an issue in areas (such as image analysis) where Deep Learning has been most useful. In particular, datasets in biology are much smaller than in other areas where machine learning is applied (for example, clinical trials often have only a few hundred people in them) and systematic errors make it difficult to generalize results across datasets. As a result, new approaches to Deep Learning that can mitigate these problems will be required before we will see widespread adoption in biology.

The Past and Future of Theory in Biology

Biology has a reputation as a reductionist observational science where scientists catalog all of the small differences they’ve observed during experiments without necessarily tying it all together. That is, biologists have trouble seeing the forest for the trees. This reputation is really quite unfair. In fact, there is a rich history of theory and top-down thinking in biology.

The idea of a gene — a discrete unit of heredity — is one of the most powerful theories in biology. The discovery of the gene is a remarkable story (well told in The Gene: An Intimate History by Siddhartha Mukherjee). In the mid-1800’s, Gregor Mendel performed a vast series of experiments in which he crossed pea plants with different characteristics and observed how their traits were passed on. He discovered simple mathematical laws that suggested each parent had a particle of information describing a physical trait. The two alleles of the parents combined to determine the trait of their offspring. Nobody knew the mechanism — that genes are encoded in DNA — for nearly 100 years, but the theory of the gene and the mathematical rules on inheritance stood on their own.

The search for unifying concepts, expressed in mathematical rules, should be a hallmark of all sciences. As with the discovery of the gene, the quest for unifying concepts requires a tight coupling between theory and experiment and usually proceeds from the discovery of macroscopic laws to the uncovering of microscopic mechanisms.

We need to change the incentives in biology to favor research that will answer these basic questions. We need to encourage collaborations between theorists and experimentalists that answer simple questions using well-designed experiments with large sample sizes. Journals will have to accept papers with equations in the main text. The focus should be on making predictions, not ad hoc analyses that quantify changes with p-values. Funding agencies will have shift their priorities to basic science.

There are a lot of social barriers to the development of theoretical biology. But, if we don’t make the effort, we’ll drown in the details like an engineer trying to build a car out of individual atoms.

Don’t Model Humans with SNPs was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Here’s the abstract from out our white paper on the bioRxiv:

Transcriptional regulation is extremely complicated. Unfortunately, so is working with transcriptional data. Genes can be referred to using a multitude of different identifiers and are assigned to an ever increasing number of categories. Gene expression data may be available in a variety of units (e.g, counts, RPKMs, TPMs). Batch effects dominate signal, but metadata may not be available. Most of the tools are written in R. Here, we introduce a library, genemunge, that makes it easier to work with transcriptional data in python. This includes translating between various types of gene names, accessing Gene Ontology (GO) information, obtaining expression levels of genes in healthy tissue, correcting for batch effects, and using prior knowledge to select sets of genes for further analysis.

Code for genemunge is freely available on Github.

We released genemunge under the MIT license to engage with the computational biology community and to spur further development of computational biology tools in python. If you find the library useful, drop us a line. We would love to hear how you integrated genemunge into your project, and we welcome feedback on how the library could be improved.

Unlearn.AI releases genemunge! was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Machine learning methods have dramatically improved in recent years. However, most of that improvement has been for problems in image processing such as recognizing objects in images, or drawing new images of faces. Images are nice because they regular arrays of numbers and because there are lots of them. Most problems in science are not computer vision problems, so thinking about ways that machine learning be applied in science requires a different set of background reading.

With that in mind, here is our company’s “Awesome List” of recommended background reading on the intersection of unsupervised learning and generative models, statistics, statistical physics, and biology.

Unsupervised Learning and Generative Models

Ackley, David H., Geoffrey E. Hinton, and Terrence J. Sejnowski. “A learning algorithm for Boltzmann machines.” Cognitive science 9.1 (1985): 147–169.

Abstract: The computational power of massively parallel networks of simple processing elements resides in the communication bandwidth provided by the hardware connections between elements. These connections can allow a significant fraction of the knowledge of the system to be applied to an instance of a problem in a very short time. One kind of computation for which massively parallel networks appear to be well suited is large constraint satisfaction searches, but to use the connections efficiently two conditions must be met: First, a search technique that is suitable for parallel networks must be found. Second, there must be some way of choosing internal representations which allow the preexisting hardware connections to be used efficiently for encoding the constraints in the domain being searched. We describe a general parallel search method, based on statistical mechanics, and we show how it leads to a general learning rule for modifying the connection strengths so as to incorporate knowledge about a task domain in an efficient way. We describe some simple examples in which the learning algorithm creates internal representations that are demonstrably the most efficient way of using the preexisting connectivity structure.

**Hinton, Geoffrey E., and Ruslan R. Salakhutdinov. “Reducing the dimensionality of data with neural networks.” science 313.5786 (2006): 504–507.

Abstract: High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such “autoencoder” networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.

Comment: The paper that kicked-off the deep learning revolution.

Salakhutdinov, Ruslan, and Hinton, Geoffrey E. “Deep Boltzmann Machines.” Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics. 2009.

Abstract: We present a new learning algorithm for Boltzmann machines that contain many layers of hidden variables. Data-dependent expectations are estimated using a variational approximation that tends to focus on a single mode, and data-independent expectations are approximated using persistent Markov chains. The use of two quite different techniques for estimating the two types of expectation that enter into the gradient of the log-likelihood makes it practical to learn Boltzmann machines with multiple hidden layers and millions of parameters. The learning can be made more efficient by using a layer-by-layer “pre-training” phase that allows variational inference to be initialized with a single bottom-up pass. We present results on the MNIST and NORB datasets showing that deep Boltzmann machines learn good generative models and perform well on handwritten digit and visual object recognition tasks.

*Hinton, Geoffrey E. “A practical guide to training restricted Boltzmann machines.” Neural networks: Tricks of the trade. Springer, Berlin, Heidelberg, 2012. 599–619.

Abstract: Restricted Boltzmann machines (RBMs) have been used as generative models of many different types of data. RBMs are usually trained using the contrastive divergence learning procedure. This requires a certain amount of practical experience to decide how to set the values of numerical meta-parameters. Over the last few years, the machine learning group at the University of Toronto has acquired considerable expertise at training RBMs and this guide is an attempt to share this expertise with other machine learning researchers.

Comment: Step-by-step recipes for old-new school machine learning.

**Goodfellow, Ian, et al. “Generative adversarial nets.” Advances in neural information processing systems. 2014.

Abstract: We propose a new framework for estimating generative models via adversarial nets, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to 1/2 everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.

Comment: Sometimes, maximizing the likelihood of the data is a bad way to train generative models. The original GAN paper is a brilliant idea on how to avoid it.

**Mehta, Pankaj, et al. “A high-bias, low-variance introduction to Machine Learning for physicists”

Abstract: Machine Learning (ML) is one of the most exciting and dynamic areas of modern research and application. The purpose of this review is to provide an introduction to the core concepts and tools of machine learning in a manner easily understood and intuitive to physicists. The review begins by covering fundamental concepts in ML and modern statistics such as the bias-variance tradeoff, overfitting, regularization, and generalization before moving on to more advanced topics in both supervised and unsupervised learning. Topics covered in the review include ensemble models, deep learning and neural networks, clustering and data visualization, energy-based models (including MaxEnt models and Restricted Boltzmann Machines), and variational methods. Throughout, we emphasize the many natural connections between ML and statistical physics. A notable aspect of the review is the use of Python notebooks to introduce modern ML/statistical packages to readers using physics-inspired datasets (the Ising Model and Monte-Carlo simulations of supersymmetric decays of proton-proton collisions). We conclude with an extended outlook discussing possible uses of machine learning for furthering our understanding of the physical world as well as open problems in ML where physicists maybe able to contribute. (Notebooks are available at this https URL).

Comment: A comprehensive review of all areas of machine learning with example code. Disclosures: 1) I am one of the coauthors, 2) it is very long.

Statistics

Geyer, Charles J. “Markov chain Monte Carlo maximum likelihood.” (1991).

Abstract: Markov chain Monte Carlo (e. g., the Metropolis algorithm and Gibbs sampler) is a general tool for simulation of complex stochastic processes useful in many types of statistical inference. The basics of Markov chain Monte Carlo are reviewed, including choice of algorithms and variance estimation, and some new methods are introduced. The use of Markov chain Monte Carlo for maximum likelihood estimation is explained, and its performance is compared with maximum pseudo likelihood estimation.

*Biau, David Jean, Brigitte M. Jolles, and Raphaël Porcher. “P value and the theory of hypothesis testing: An explanation for new researchers.” Clinical Orthopaedics and Related Research® 468.3 (2010): 885–892.

Abstract: In the 1920s, Ronald Fisher developed the theory behind the p value and Jerzy Neyman and Egon Pearson developed the theory of hypothesis testing. These distinct theories have provided researchers important quantitative tools to confirm or refute their hypotheses. The p value is the probability to obtain an effect equal to or more extreme than the one observed presuming the null hypothesis of no effect is true; it gives researchers a measure of the strength of evidence against the null hypothesis. As commonly used, investigators will select a threshold p value below which they will reject the null hypothesis. The theory of hypothesis testing allows researchers to reject a null hypothesis in favor of an alternative hypothesis of some effect. As commonly used, investigators choose Type I error (rejecting the null hypothesis when it is true) and Type II error (accepting the null hypothesis when it is false) levels and determine some critical region. If the test statistic falls into that critical region, the null hypothesis is rejected in favor of the alternative hypothesis. Despite similarities between the two, the p value and the theory of hypothesis testing are different theories that often are misunderstood and confused, leading researchers to improper conclusions. Perhaps the most common misconception is to consider the p value as the probability that the null hypothesis is true rather than the probability of obtaining the difference observed, or one that is more extreme, considering the null is true. Another concern is the risk that an important proportion of statistically significant results are falsely significant. Researchers should have a minimum understanding of these two theories so that they are better able to plan, conduct, interpret, and report scientific experiments.

Comment: Classical hypothesis testing remains the dominant framework for analysis in many areas of science. It’s important to understand how it is supposed to work.

Gelman, Andrew, Jennifer Hill, and Masanao Yajima. “Why we (usually) don’t have to worry about multiple comparisons.” Journal of Research on Educational Effectiveness 5.2 (2012): 189–211.

Abstract: Applied researchers often find themselves making statistical inferences in settings that would seem to require multiple comparisons adjustments. We challenge the Type I error paradigm that underlies these corrections. Moreover we posit that the problem of multiple comparisons can disappear entirely when viewed from a hierarchical Bayesian perspective. We propose building multilevel models in the settings where multiple comparisons arise. Multilevel models perform partial pooling (shifting estimates toward each other), whereas classical procedures typically keep the centers of intervals stationary, adjusting for multiple comparisons by making the intervals wider (or, equivalently, adjusting the p values corresponding to intervals of fixed width). Thus, multilevel models address the multiple comparisons problem and also yield more efficient estimates, especially in settings with low group-level variation, which is where multiple comparisons are a particular concern.

*Gelman, Andrew, and John Carlin. “Beyond power calculations: assessing type S (sign) and type M (magnitude) errors.” Perspectives on Psychological Science 9.6 (2014): 641–651.

Abstract: Statistical power analysis provides the conventional approach to assess error rates when designing a research study. However, power analysis is flawed in that a narrow emphasis on statistical significance is placed as the primary focus of study design. In noisy, small-sample settings, statistically significant results can often be misleading. To help researchers address this problem in the context of their own studies, we recommend design calculations in which (a) the probability of an estimate being in the wrong direction (Type S [sign] error) and (b) the factor by which the magnitude of an effect might be overestimated (Type M [magnitude] error or exaggeration ratio) are estimated. We illustrate with examples from recent published research and discuss the largest challenge in a design calculation: coming up with reasonable estimates of plausible effect sizes based on external information.

Comment: Classical hypothesis testing remains the dominant framework for analysis in many areas of science. It’s important to understand why it doesn’t work very well.

Statistical Physics

Bryngelson, Joseph D., and Peter G. Wolynes. “Spin glasses and the statistical mechanics of protein folding.” Proceedings of the National Academy of Sciences 84.21 (1987): 7524–7528.

Abstract: The theory of spin glasses was used to study a simple model of protein folding. The phase diagram of the model was calculated, and the results of dynamics calculations are briefly reported. The relation of these results to folding experiments, the relation of these hypotheses to previous protein folding theories, and the implication of these hypotheses for protein folding prediction schemes are discussed.

*Roudi, Yasser, Erik Aurell, and John A. Hertz. “Statistical physics of pairwise probability models.” Frontiers in computational neuroscience 3 (2009): 22.

Abstract: Statistical models for describing the probability distribution over the states of biological systems are commonly used for dimensional reduction. Among these models, pairwise models are very attractive in part because they can be fit using a reasonable amount of data: knowledge of the mean values and correlations between pairs of elements in the system is sufficient. Not surprisingly, then, using pairwise models for studying neural data has been the focus of many studies in recent years. In this paper, we describe how tools from statistical physics can be employed for studying and using pairwise models. We build on our previous work on the subject and study the relation between different methods for fitting these models and evaluating their quality. In particular, using data from simulated cortical networks we study how the quality of various approximate methods for inferring the parameters in a pairwise model depends on the time bin chosen for binning the data. We also study the effect of the size of the time bin on the model quality itself, again using simulated data. We show that using finer time bins increases the quality of the pairwise model. We offer new ways of deriving the expressions reported in our previous work for assessing the quality of pairwise models.

Comment: This is a great overview of some models at the intersection of physics and machine learning.

Biology

Oshlack, Alicia, Mark D. Robinson, and Matthew D. Young. “From RNA-seq reads to differential expression results.” Genome biology 11.12 (2010): 220.

Abstract: Many methods and tools are available for preprocessing high-throughput RNA sequencing data and detecting differential expression.

Leek, Jeffrey T., et al. “Tackling the widespread and critical impact of batch effects in high-throughput data.” Nature Reviews Genetics 11.10 (2010): 733.

Abstract: High-throughput technologies are widely used, for example to assay genetic variants, gene and protein expression, and epigenetic modifications. One often overlooked complication with such studies is batch effects, which occur because measurements are affected by laboratory conditions, reagent lots and personnel differences. This becomes a major problem when batch effects are correlated with an outcome of interest and lead to incorrect conclusions. Using both published studies and our own analyses, we argue that batch effects (as well as other technical and biological artefacts) are widespread and critical to address. We review experimental and computational approaches for doing so.

Tkačik, Gašper, and Aleksandra M. Walczak. “Information transmission in genetic regulatory networks: a review.” Journal of Physics: Condensed Matter 23.15 (2011): 153102.

Abstract: Genetic regulatory networks enable cells to respond to changes in internal and external conditions by dynamically coordinating their gene expression profiles. Our ability to make quantitative measurements in these biochemical circuits has deepened our understanding of what kinds of computations genetic regulatory networks can perform, and with what reliability. These advances have motivated researchers to look for connections between the architecture and function of genetic regulatory networks. Transmitting information between a network’s inputs and outputs has been proposed as one such possible measure of function, relevant in certain biological contexts. Here we summarize recent developments in the application of information theory to gene regulatory networks. We first review basic concepts in information theory necessary for understanding recent work. We then discuss the functional complexity of gene regulation, which arises from the molecular nature of the regulatory interactions. We end by reviewing some experiments that support the view that genetic networks responsible for early development of multicellular organisms might be maximizing transmitted ‘positional information’.

*Lang, Alex H., et al. “Epigenetic landscapes explain partially reprogrammed cells and identify key reprogramming genes.” PLoS computational biology 10.8 (2014): e1003734.

Abstract: A common metaphor for describing development is a rugged “epigenetic landscape” where cell fates are represented as attracting valleys resulting from a complex regulatory network. Here, we introduce a framework for explicitly constructing epigenetic landscapes that combines genomic data with techniques from spin-glass physics. Each cell fate is a dynamic attractor, yet cells can change fate in response to external signals. Our model suggests that partially reprogrammed cells are a natural consequence of high-dimensional landscapes, and predicts that partially reprogrammed cells should be hybrids that co-express genes from multiple cell fates. We verify this prediction by reanalyzing existing datasets. Our model reproduces known reprogramming protocols and identifies candidate transcription factors for reprogramming to novel cell fates, suggesting epigenetic landscapes are a powerful paradigm for understanding cellular identity.

Comment: Illustrates how concepts from statistical physics and machine learning can be used to understand genetic regulation.

*Lovell, David, et al. “Proportionality: a valid alternative to correlation for relative data.” PLoS computational biology 11.3 (2015): e1004075.

Abstract: In the life sciences, many measurement methods yield only the relative abundances of different components in a sample. With such relative — or compositional — data, differential expression needs careful interpretation, and correlation — a statistical workhorse for analyzing pairwise relationships — is an inappropriate measure of association. Using yeast gene expression data we show how correlation can be misleading and present proportionality as a valid alternative for relative data. We show how the strength of proportionality between two variables can be meaningfully and interpretably described by a new statistic ϕ which can be used instead of correlation as the basis of familiar analyses and visualisation methods, including co-expression networks and clustered heatmaps. While the main aim of this study is to present proportionality as a means to analyse relative data, it also raises intriguing questions about the molecular mechanisms underlying the proportional regulation of a range of yeast genes.

Comment: Lots of experimental data in biology are actually relative fractions (e.g., relative species abundances from microbiota studies, relative transcript abundances from RNA sequencing). The implications are not widely appreciated.

Awesome Papers on Machine Learning, Physics, and Biology was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

In the last few years, computers have made tremendous strides in their ability to learn and think. With a bit of effort, and lots of data, a computer can be taught to recognize objects in images, to understand analogies, or even to play games like Go at a world class level of skill. It is no surprise that these improvements to artificial intelligence have spawned a new industry of companies applying machine learning to, well, everything.

The explosion of interest in artificial intelligence has also led to an explosion of confusion. Just what exactly is machine learning, anyway?

Let’s start with a few of the common buzzwords:

Artificial Intelligence (AI): the ability of machines to perform tasks that usually require human intelligence.

Machine Learning: algorithms that enable computers to learn about the world without being explicitly programmed for individual tasks.

Deep Learning: an approach to machine learning using models that build representations of objects from combinations of pieces that can be re-used.[¹]

Machine learning is a prerequisite to artificial intelligence — intelligent computers need to be able to learn from new observations as they interact with the world. Deep Learning is just one approach to this problem that is loosely inspired by the way the human brain processes information. There are many approaches to machine learning and AI that do not use deep learning, but deep learning has proven to be an especially powerful approach in recent years.

How Machine Learning Works

At the most basic level, machine learning is about memory. To illustrate this, let’s go through an example of what may be the simplest possible machine learning algorithm — a nearest neighbor classifier.

We would like to teach a computer to recognize when an image is a picture of a car. In this case, a nearest neighbor classifier would look like this:

1. We collect a training set of labeled images; maybe 1 million pictures of cars and 1 million pictures that do not have cars in them.

2. We decide on a way to measure the similarity between two pictures. The simplest way is just to compare the values of all of the pixels.

3. We tell the computer how to make a guess about images it has never seen before. Find the most similar image in memory (i.e., from the training set).

- If this nearest neighbor image in the training set is a picture of a car, then guess that the new image is also a picture of a car.

- Otherwise, guess that the new image is not a car.

That’s it! It is a very simple algorithm.

Basically, we told the computer the answer for 2 million examples and had it memorize them. Although it works, this is not a very efficient way of learning something. What if you wanted the computer to also be able to recognize the make and model of the car in the image, or the face of the person driving it? Looking back through every image of every thing we have ever encountered each time we need to recognize something would basically take forever (not to mention a huge amount of space). And what if we gave the computer an image unlike anything it has seen before, such as a picture of a tree when there were no training images of trees? This algorithm would not fare well.

It is much more efficient to teach the computer to recognize pieces of objects because we can build many types of things using various combinations of the pieces. This is the basic idea behind deep learning.

Deep Neural Networks

A deep neural network is a mathematical model capable of learning to build representations of objects from their component pieces. For example, a neural network designed to recognize images processes the data through a series of layers. The first layer learns to look for simple features like lines or blobs of color. Deeper layers combine these low-level features to form shapes, and the final layers combine these shapes into objects.

This is an extremely efficient way to store information. Instead of memorizing the label of every single image, the computer learns how to transform the input image into an abstract representation and only has to memorize the labels of a few of these abstract classes. For example, the network can learn to recognize a car as a sum of low level features such as the patterns and textures that make up wheels, windows, and body shapes. Moreover, there are efficient ways to train these types of models, so much so that it has become pretty easy to train a good deep neural network as long you have a really big dataset with reliable class labels.

The Problems with Memorization

Deep learning works well for classification because similar observations get transformed into the same abstract representations. However, these models (and, in fact, most machine learning models) can fail spectacularly when they are presented with an observation that is very different from anything they have seen before.

In theory, it is possible to design a machine learning algorithm that is capable of continually learning from new observations. The algorithm needs to remember all of the things it has already learned that allowed it to perform well on previous tasks, find common components of different tasks so that it can re-use its knowledge, and learn any new skills it needs as it goes along. Unfortunately, current approaches to machine learning do not do this well — they specialize on a single task at a time and forget how to do this task when taught a new one.

The Future of Machine Learning

Most of the successes in machine learning during the last few years have been on problems with lots of data (e.g., millions of images) where it is possible to provide the computer with feedback about the correct answer — these types of tasks belong to a field known as _supervised learning_. There are many tasks where datasets are small (e.g., hundreds of patients in a clinical trial) or where the correct answer is unknown — these types of tasks belong to a field known as _unsupervised or semi-supervised learning_. We can think of the following analogy:

- Supervised learning techniques are like school. The computer is presented with many question/answer examples, and it will eventually be tested on some new questions it hasn’t seen before but that are similar to the ones it studied.

- Unsupervised learning techniques try to replicate the intuitive learning that we do everyday as we observe the world. For example, infants learn from experience that dropped objects will fall to the ground.

In order to send a computer to school, we need to provide a teacher. We need a lot of data in the form of questions and answers. But creating these data can be very costly — both in terms of time and money. Therefore, developing computers that have the ability to learn on their own using unsupervised techniques is a crucial step on the road to more general artificial intelligence.

[¹]: I used a pretty imprecise definition of deep learning for the sake of accessibility. To be more precise, deep learning refers to the use of neural networks that process the data through many layers of transformations. This allows the model to learn an appropriate feature space, instead of requiring human designed features. In practice, the shallow layers learn local features, whereas the deeper layers combine these local features into global features.

What is machine learning? was originally published in Unlearn.AI on Medium, where people are continuing the conversation by highlighting and responding to this story.