<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dan Saunders on Medium]]></title>
        <description><![CDATA[Stories by Dan Saunders on Medium]]></description>
        <link>https://medium.com/@danjsaund?source=rss-4565a4947e1b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*XPwrvStW9AeT6FF4Qh9TxQ.jpeg</url>
            <title>Stories by Dan Saunders on Medium</title>
            <link>https://medium.com/@danjsaund?source=rss-4565a4947e1b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:47:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@danjsaund/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Book Review: “The Feeling of Life Itself”]]></title>
            <link>https://medium.com/@danjsaund/book-review-the-feeling-of-life-itself-25078b3c27a9?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/25078b3c27a9</guid>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Thu, 05 Mar 2020 13:41:14 GMT</pubDate>
            <atom:updated>2020-03-05T13:41:14.772Z</atom:updated>
            <content:encoded><![CDATA[<p>Author: <a href="https://alleninstitute.org/what-we-do/brain-science/about/team/staff-profiles/christof-koch/">Christof Koch</a></p><p>Published: September 2019</p><p>When I heard that Christof Koch was writing a new book, I immediately put it on my wish list, and purchased it shortly after it came out. I had read <a href="https://www.goodreads.com/en/book/show/13557108-consciousness"><em>Consciousness: Confessions of a Romantic Reductionist</em></a> a few years prior, and thought it was stellar, a unique blend of anecdotes from living a life while practicing neuroscience and thinking about consciousness. It was raw; there were emotions and personal details that are typically absent or, at best, stilted when encountered in popular science books. There’s a lot of speculation about the nature of consciousness, how much we can learn about it with existing experimental methods, and to what extent it can be engineered or augmented.</p><p><em>The Feeling of Life Itself</em> reminds me of that book in several ways. Koch’s writing is approachable as ever, and sprinkled with anecdotes from life and scientific practice alike. Speculation about melding minds, engineering consciousness, neuromorphic computation, pure consciousness, etc. abound, all rooted in Koch’s pet theory (first articulated by <a href="https://en.wikipedia.org/wiki/Giulio_Tononi">Giulio Tononi</a>), <a href="http://www.scholarpedia.org/article/Integrated_information_theory"><em>integrated information theory</em></a><em> </em>(IIT), a mathematical theory of consciousness. Koch argues that, in order for a system to be conscious, it needs to have high integrated information; i.e., its components needed to be tightly integrated and recurrently connected; it needs to have “irreducible cause-effect power”.</p><p>He argues that consciousness is proportional to the degree of integrated information of a system, and thus, humans are more conscious than simpler mammals, which are more conscious than bugs, which are more conscious than single-celled organisms, because each have larger and more integrated brains / nervous systems than the next. Computers are therefore minimally conscious (no matter the programming), argues Koch, since they can be reduced to their individual components, which can easily be decoupled from their neighbors without losing their individual functionality.</p><p>As much as I liked the premise of the book, I found it hard to read. Although I <em>bought</em> the book in September, I only <em>finished</em> it just a few weeks ago. I think this is due to my uneasiness with the theory that integrated information theory is a good predictor of levels of consciousness, and, more fundamentally, whether I can accept Koch’s definition of consciousness to begin with.</p><p>If you are willing to suspend disbelief here and there, the book is an entertaining, speculative journey through what separates humans and machines, and provides a pleasant argument for the sanctity of life in all its forms. I’m not saying Koch is wrong; I just find <em>some</em> of the implications of IIT difficult to believe.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25078b3c27a9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Book Review — Reinforcement Learning: An Introduction]]></title>
            <link>https://medium.com/@danjsaund/book-review-reinforcement-learning-an-introduction-7d08cf15d263?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d08cf15d263</guid>
            <category><![CDATA[academia]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[textbooks]]></category>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Tue, 23 Apr 2019 15:03:10 GMT</pubDate>
            <atom:updated>2019-04-23T15:03:10.854Z</atom:updated>
            <content:encoded><![CDATA[<h3>Book Review — Reinforcement Learning: An Introduction</h3><h3>Metadata</h3><p><em>Authors</em>: <a href="http://incompleteideas.net/">Richard S. Sutton</a> and <a href="http://www-all.cs.umass.edu/~barto/">Andrew G. Barto</a> (go UMass!)</p><p><em>Edition</em>: 2nd</p><p><em>Publication date</em>: November 13, 2018</p><p><em>Pages</em>: 552</p><h3>My RL Background</h3><p>Reinforcement learning (RL) was on the periphery of my university studies for quite some time. We did some exploratory work on framing supervised learning as an RL problem in spiking neural networks (SNNs), and ending up writing <a href="https://arxiv.org/abs/1903.11012">a paper on converting neural network-parameterized RL policies to SNNs</a> in the <a href="https://binds.cs.umass.edu/">BINDS lab</a>.</p><p>SNNs seem especially well-suited to reward-based learning, as there are plenty of ideas to draw on from theoretical and experimental neuroscience that suggest that neural circuits learn in the presence of global neuromodulatory signals. These signals can be thought of as <em>reward</em> or <em>reinforcement</em>. There’s some good work on doing RL with SNNs (see e.g., <a href="http://www.jneurosci.org/content/30/40/13326">here</a> and <a href="https://link.springer.com/referenceworkentry/10.1007%2F978-1-4419-1428-6_1713">here</a>), but much work remains to be done before they can be used for RL in complex environments.</p><h3>Book review</h3><p>I can whole-heartedly recommend <em>Reinforcement Learning: An Introduction.</em> I started on the 1st edition in late 2018, realized a 2nd edition had been released, and switched over. Consequently, I missed some of the new material in the early parts of the new version, but I’m planning to read up on those sometime soon.</p><p>The writing and pacing of the book is extremely well done. I was especially impressed by the authors’ ability to tie in concepts from every part of the book throughout, which made for a great learning experience. Not only was I able to grasp the individual concepts as they were introduced, but felt I understood how they were related to the bigger picture. There are enough details in the book to implement many of the standard algorithms in RL, and the ideas behind them are explained succinctly and carefully related to each other as they’re introduced. This allows the reader to visualize a “space” of reinforcement learning algorithms / approaches, and to see where the gaps in our knowledge lie.</p><p>Part I describes the RL problem: an <em>agent</em> seeks to maximize the cumulative (discounted) reward from an environment over time. At each <em>timestep </em>(a parameter of the agent-environment formulation), the agent receives an <em>observation</em> of the environment, selects an <em>action</em> conditioned on it (and on the history of their interaction), and obtains a <em>reward</em>. This agent-environment interaction is often formulated as <a href="https://en.wikipedia.org/wiki/Markov_decision_process">Markov Decision Process (MDP)</a>, “…a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker.” There are other such formalisms, like <a href="https://en.wikipedia.org/wiki/Partially_observable_Markov_decision_process">partially observed MDPs</a>, but the book focuses on MDPs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*kt3V9mZj3MSL4bxT.jpg" /><figcaption>Process diagram of the agent-environment interaction (<a href="https://www.kdnuggets.com/2018/03/5-things-reinforcement-learning.html">source</a>).</figcaption></figure><p>The authors describe the dimensions along which RL algorithms vary and how they affect learning: <em>n</em>-step (temporal difference) vs. “infinite-step” (Monte Carlo) returns, exploration vs. exploitation, value iteration vs. policy optimization, eligibility traces and how they generalize <em>n</em>-step returns, model-based vs. model-free approaches, on-policy vs. off-policy training, prediction vs. control, etc.</p><p>There is a brief discussion on dynamic programming, which requires complete knowledge of environment dynamics. The authors use this as a basis for the discussion of <em>value approximation</em> and <em>policy gradient </em>methods, which have no such requirement of known dynamics.</p><p><em>Model-based</em> methods build a model of the environment and uses it for planning many timesteps into the future, or simulates it to avoid querying a computationally demanding or dangerous real-life environment. <em>Model-free</em> methods learn directly from the environment, improving a value function to mirror the true value of states of the environment, or directly optimizing a policy to maximize cumulative discounted reward.</p><p><em>Temporal difference</em> (TD) learning is a central component of reinforcement learning, and many parallels are drawn between RL algorithms (<em>engineering</em>), learning in animals (<em>behavioral psychology</em>), and learning in neural circuits (<em>neuroscience</em>). TD learning “<a href="https://en.wikipedia.org/wiki/Temporal_difference_learning">…refers to a class of model-free reinforcement learning methods which learn by bootstrapping from the current estimate of the value function.</a>” TD learning stipulates that error signals, or, more accurately, <a href="https://www.cell.com/current-biology/pdf/S0960-9822(17)30266-X.pdf">reward prediction error signals</a>, conveys a mismatch in expectation vs. reality, thus enabling a learning algorithm to make adjustments to reduce this mismatch.</p><p><em>On-policy</em> algorithms collects experience with the policy being evaluated, while <em>off-policy</em> algorithms learn the value of a “target policy” independently of the “behavioral policy”. The behavioral policy is often similar enough to the target policy such that importance weighting can be used to apply experience gathered by the former to make parameter updates to the latter.</p><p>Machine learning methods are employed in the “Approximate Solution Methods” section, where features of observations from an environment are computed by a ML model or constructed by a human expert. These are used in cases where the environment’s observation space is combinatorially large, from which it’s hopeless to find an optimal policy, and our goal instead is to find a good approximation. For example, the recent success of RL on Atari game-playing relies on down-sampling, stacking, and processing in-game video frames with a convolutional neural network before actions can be selected.</p><p><em>Policy gradient</em> methods optimize parametric policies to maximize a scalar performance measure (often the cumulative discounted reward) by following the gradient of the performance measure with respect to policy parameters. <em>Actor-critic</em> methods learn both a policy (actor) and an approximation to the value function (critic), the latter of which is used as a baseline to reduce variance in policy gradient updates.</p><p>The <em>exploration vs. exploitation</em> dilemma refers to the trade-off between exploring one’s environment in order to collect information and reduce uncertainty, and exploiting one’s knowledge of the environment to maximize reward. Simply put, a RL agent’s knowledge of its environment could first be maximized, and then exploited for garnering maximum reward. This process can be iterated until a good (approximately optimal) policy is found. Exploration can be accomplished by following a policy with high entropy (e.g., actions sampled from a uniformly random distribution), or by following an <a href="https://arxiv.org/abs/1810.12894">intrinsic “curiosity-driven” reward</a>; there are many possible methods. Exploitation consists of choosing actions that lead to the highest expected reward.</p><p>Coming from a bit of a neuroscience background, I particularly enjoyed the chapters on psychology and neuroscience, although the latter is much more compelling than the former. The <em>Classical</em> and <em>operant conditioning</em> learning paradigms of behavioral psychology are related to <em>prediction</em> and <em>control</em> in reinforcement learning, respectively. It is hypothesized that dopaminergic neurons are responsible for deliverable a far-ranging reward prediction error signal, suggesting a biological implementation of temporal difference learning.</p><p>The book is comprehensive, and I can’t hope to cover everything here. In sum, I’m glad to have had this as a resource when jump-starting my foray into research on and application of reinforcement learning. Give it a read and let me know what you think!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d08cf15d263" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Book Review: The Book of Why]]></title>
            <link>https://medium.com/@danjsaund/book-review-the-book-of-why-24aee7b71697?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/24aee7b71697</guid>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[book-review]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[books]]></category>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Mon, 31 Dec 2018 14:39:36 GMT</pubDate>
            <atom:updated>2018-12-31T14:39:36.647Z</atom:updated>
            <content:encoded><![CDATA[<p>Author: <a href="http://bayes.cs.ucla.edu/jp_home.html">Judea Pearl</a></p><p>Published: May 2018</p><p><strong>Note</strong>: I wrote most of this blog post in the summer of 2018, and then forgot about it. I recently remembered it, so it’s time to finish it!</p><p><a href="https://www.basicbooks.com/titles/judea-pearl/the-book-of-why/9780465097609/"><em>The Book of Why</em></a> is perhaps my favorite book I’ve read so far in 2018. It’s a popular science book, but it seems geared towards those with at least some understanding of probability and statistics. The book is about <a href="https://en.wikipedia.org/wiki/Causal_inference"><strong>causal inference</strong></a>; what is it, when / why it was developed, why it’s “necessary”. The main concept throughout the book is the “<em>ladder of causation</em>”, which has three rungs:</p><ol><li><em>Association</em>: Having to do with seeing / observing events, and questions of the form “What if I see X?”, or “How are the variables X and Y related?”</li><li><em>Interventions</em>: Doing, intervening; with questions like “What would Y be if I do X?”, or “What needs to happen in order for Y to occur?”</li><li><em>Counterfactuals</em>: Imagining, retrospection; with questions like “What if X had not occureed? Did X cause Y?”</li></ol><p>A central argument in the book is that classical statistics only deal with questions on the first rung of the ladder of causation; i.e., causation is all but forbidden in the discipline. In particular, statistics is mostly concerned with the reduction or summarization of data, while causal inference is further concerned with discovering the strength and directionality of relationships between variables. Statistics has embraced the <em>randomized controlled trial</em> (RCT), an important tool for untangling causal effects, say, in studies of drug effectiveness, but much more sophisticated methods have been developed since. Such causal inference methods are intended for cases where RCTs are not applicable or unethical; e.g., when randomly assigning a potentially harmful treatment / lifestyle modification (say, smoking or non-smoking).</p><p>An important tool of practitioners of causal inference is the <em>structural causal model</em>, from which a <em>causal graph</em> can be drawn. There’s a one-to-one relationship between the two. However, it’s typically easier for humans to think about graphs (and for computers to work with equations!), so I’ll focus on those here. A causal graph is given by a set of nodes representing random variables (<em>endogenous</em> and <em>exogenous</em>; the former are considered to be deterministic functions of some number of random variables, and the latter, sampled from some distribution) and directed edges between (representing causal links). An important point is that the functional relationships between variables is not restricted to any particular class (but many be assumed to be linear, parametric, etc.). Causal graphs are typically acyclic,</p><p>Causal graphs are a useful tool for incorporating scientific knowledge into a model of a particular real-world process. For example, it is fairly obvious that changing atmospheric pressure affects the reading on a barometer, and not the other way around. Supposedly, there are ways to discover causal relationships from raw data, a process called <em>causal discovery</em>, but the book does not go into these methods. Once we have posited a graph, we can use collected data on the nodes in it to provide evidence for independencies implied by the graph. A graph criterion called <a href="https://www.andrew.cmu.edu/user/scheines/tutor/d-sep.html"><em>d</em>-separation</a> tells us when two sets of variables are independent, or conditionally independent given some other set of variables.</p><p>Pearl is something of a zealot for the nascent field, and certainly has a personal stake in its success, so the excitement of the text should be taken with a grain of salt. Nevertheless, this excitement is part of why I liked the book so much. Many of the ideas are statistical, but re-cast in the light of causality. So, these aren’t necessarily new ideas, but perhaps better thought-out and, importantly, theoretically justified with causal terminology.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*tHMYW-NNrDWuRiZg.png" /></figure><p>I think it would be interesting to see how concepts from causal inference might inform reinforcement learning (RL). In particular, should RL agents build causal models of their environments? The RL “perception-action” cycle (pictured to the left) contains directed edges, and can be made into a DAG via time-ordering; i.e., the environment at time <em>t</em> affects the interpreter at time <em>t </em>through an observation, which affects the agent at time <em>t </em>through the reward signal and state (processed observation), whose action(s) affect the state of the environment at time <em>t+1</em>.</p><p>Building a detailed causal model of action -&gt; environment state -&gt; reward (where any node could be expanded into a set of random variables) may be a way to improve RL methods. Indeed, model-based RL builds a model of the environments transition dynamics and reward signal. Perhaps theoretical ideas from causal inference could spur progress in this endeavor.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=24aee7b71697" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Notes on “Supervised learning in spiking neural networks with FORCE training”]]></title>
            <link>https://medium.com/@danjsaund/notes-on-supervised-learning-in-spiking-neural-networks-with-force-training-f5d6c9fbb4d?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/f5d6c9fbb4d</guid>
            <category><![CDATA[neuroscience]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Sun, 23 Sep 2018 17:10:40 GMT</pubDate>
            <atom:updated>2018-09-23T17:10:40.558Z</atom:updated>
            <content:encoded><![CDATA[<p>Paper available <a href="https://www.nature.com/articles/s41467-017-01827-3">here</a>.</p><h3>Abstract</h3><p>This paper demonstrates the applicability of the <a href="https://www.cell.com/action/showPdf?pii=S0896-6273%2809%2900547-9">FORCE</a><em> </em>(not to be confused with the <a href="http://www.scholarpedia.org/article/Policy_gradient_methods#Likelihood_Ratio_Methods_and_REINFORCE">REINFORCE</a> algorithm in the reinforcement learning literature) supervised learning method to <em>spiking neural networks</em> (SNNs). It’s used to train SNNs to mimic dynamical systems, classify inputs, and store discrete sequences, as well as to model two model circuits. FORCE-trained SNNs reproduce comparable complex behaviors to those of their inspired circuits and yield information not readily available from pharmacological manipulations / spike timing statistics.</p><h3>Introduction</h3><p>Claim: humans don’t use control theory / machine learning (ML)/ etc. to solve problems, they use SNNs (This is self-evident. But, that’s not to say concepts from control theory / ML aren’t implemented in biological SNNs.).</p><p>A broad class of methods have been presented that allow one to enforce certain behavior / dynamics onto SNNs. These top-down techniques begin with an intended task for an SNN, and determine what synapse strengths should be to achieve it. Dominant approaches include the FORCE method, spike-based / predictive coding networks, and the neural engineering framework (NEF).</p><p>While NEF and spike-based coding approaches creating functional SNNs, they aren’t agnostic toward the underlying network of neurons. Additionally, in order to apply either approach, the task must be specified in closed-form differential equations. Despite these constraints, both methods have led to a resurgence in top-down analysis of network function.</p><p>FORCE training is agnostic toward both the SNN and the task the network is trained to solve. FORCE training takes any high-dimensional dynamical system (a <em>reservoir</em>) and uses its dynamics to compute. The target behavior doesn’t need to be specified in differential equations; all that’s needed is a supervising error signal. FORCE is therefore applicable to many more types of systems and tasks. Unfortunately, FORCE training has only been implemented in networks of firing rate-based neurons, with little work done on SNN implementation.</p><p>It is shown that FORCE training can be applied to SNNs, and is robust against different implementations, neuron models, and supervising signals.</p><h3>Results</h3><h4>FORCE training weight matrices</h4><p>The authors explore applying FORCE to train SNNs to perform arbitrary tasks. The synaptic weight matrix in these networks is a sum of a set of static weights (set to initialize the network into chaotic spiking) and a set of learned weights (determined online using the Recursive Least Square (RSL) supervised learning method). The goal of RLS is to minimize the squared error between the network dynamics and the target dynamics, considered successful if the network dynamics mimic the target dynamics post-training.</p><h4>FORCE trained rate networks learn using chaos</h4><p>To demonstrate the method and compare with SNNs implementations, FORCE is applied to a network of rate-based neurons (rate equations). The static weight matrix initializes high-dimensional chaotic dynamics, which form a suitable<em> </em>reservoir to allow the network to learn from a target signal quickly. RLS is activated after a short initialization period, and, post-training, the networks is able to reproduce a 5Hz sinusoidal oscillator with slight frequency / amplitude error.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*SAPoi7PDV8tAmyhVScUPTg.png" /><figcaption>Pre-learning, firing rates for 5 neurons (randomly chosen from a network of 1,000) exhibit chaotic trajectories. During and after learning, the chaos is “controlled” and converted into steady oscillations.</figcaption></figure><h4>FORCE trained spiking network learn using chaos</h4><p>FORCE training is implementing in SNNs with different spiking neurons to compare the robustness of the method across neuron models (namely, theta, leaky integrate-and-fire (LIF), and Izhikevich models).</p><p>First, to demonstrate the chaotic nature of the networks, single spikes are deleted. After deletion, spike trains immediately diverge, indicating chaotic dynamics. All neuron models exhibit bimodal <em>interspike-interval (ISI) distributions</em> indicative of possible transitions to rate chaos for different proportions of static and learned weights.</p><p>FORCE training in rate-based neurons works by quickly stabilizing firing rates, and subsequent changes are made to “stabilize” it. For fast learning, RLS has to be applied on a faster time scale in SNNs than in rate-based networks. The learning rate was made fast enough to stabilize the spiking basis during the first supervising signal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/513/1*SczEdETe6t4ur9vctrrNLQ.png" /><figcaption>Target dynamics and FORCE-trained network output. In many cases, decoded network output matches the target dynamics well; i.e., with little error.</figcaption></figure><p>With these modifications, FORCE is successfully applied to trains SNNs to mimic various oscillator types (sinusoids at different frequencies, sawtooth, Van der Pol, and with noisy teaching signals). All neuron types were able to learn to reproduce all oscillator types (with minor modifications to the theta neuron model). FORCE training was robust to initial chaotic network states. Oscillators with higher (lower) frequencies are learned over larger parameter regions in networks with faster (slower) synaptic decay time constants. For SNNs, the authors observe that, in some cases, systems with dominant eigenvalues outperform systems without, and in other cases, the opposite is true.</p><p>For the considered target dynamics, the Izhikevich model had the greatest accuracy and fastest training times, apparently due to its spike frequency adaptation, which operates on a long time scale. The long time scale affords the SNN reservoir a greater memory capacity, allowing the learning of longer signals.</p><p>Since oscillators are simple dynamical systems, the authors considered two more complicated target dynamics: those of a low-dimensional chaotic system, and statistically classifying inputs applied to a network of neurons. A SNN of theta neurons was able to reproduce the butterfly attractor and Lorenz-like trajectories post-training. Since the supervising dynamics were more complex, training took longer and required more neurons (5,000 neurons and 45 seconds of training). The SNN achieved comparable performance to a network of rate-based neurons, and both networks were able to reproduce the stereotypical Lorenz tent map. However, in general, the rate-based network performed somewhat better. The authors show that populations of neurons can be FORCE-trained to classify inputs, similar to a feed-forward network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/742/1*MGItqv64IkRp5VWM7BN9wQ.png" /><figcaption>Lorenz system and network attractor dynamics plotted together, and network state trajectories plotted at different viewing angles. The system’s dynamics strongly resemble those of the canonical butterfly-shaped attractor post-training.</figcaption></figure><h4>FORCE training spiking networks to reproduce complex signals</h4><p>The authors wondered whether FORCE-trained networks could encode signals similar to those from naturally-occurring spatio-temporal sequences (e.g., songbird singing). This is formulated as very long oscillations that are repeatedly presented to the network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*YaWYss0n0EsPZnJpeAlpPw.png" /><figcaption>The first four bars of Ode to Joy along with the 5-dimensional decoded output of the FORCE-trained network. Decoded notes match with the target notes in terms of note onset and length.</figcaption></figure><p>The first pattern considered is a sequence of pulses in a 5-dimensional supervising signal, corresponding to the notes in the first four bars of Ode to Joy. A network of Izhikevich neurons was able to reproduce the bars. Networks of all three neuron types were able to reproduce the bars when using larger synaptic decay time constants.</p><p>The networks displayed some <em>stereotypical</em> (as opposed to <em>random</em>) errors, occurring in places where sub-sequences of notes were not unique.</p><h4>FORCE trained networks can reproduce songbird singing</h4><p>The authors construct a circuit that reproduces a birdsong (in the form of a spectrogram) recorded from an adult zebra finch. The learned singing behavior of these birds is owed to two primary nuclei: the HVC and the Robust nucleus of the Arcopallium (RA). The RA-projecting neurons in HVC form a chain of spiking activity and each fires only once at a specific time in the song. This chain of firing is transmitted to the RA circuit, wherein each neuron bursts at multiple times during the song.</p><p>The authors focus on a single network of RA neurons, and model the HVC chain of inputs as a series of successive pulses (not FORCE-trained for simplicity). The pulses are fed into a population of Izhikevich neurons that are successfully FORCE-trained to reproduce the spectrogram of the recorded birdsong. With certain parameter settings, the spiking statistics of RA neurons are reproduced both qualitatively and quantitatively.</p><p>Manipulations are made to the excitatory synapses which alter the network’s excitatory-inhibitory balance. The network is robust to down-scaling excitatory weights, still reproducing the song at a lower intensity. Upscaling excitatory weights by 15% or more drastically reduced song performance, and by 20% or more, replaced the singing with high-intensity, seizure-like activity. A similar result is observed through injection of bicuculline in RA.</p><h4>High-dimensional temporal signals improve FORCE training</h4><p>The authors hypothesize the performance of the songbird network was associated with the precise, clock-like inputs from the HVC, and that similar inputs could aid in encoding / replay of other types of signals. To test this, they removed the HVC input pattern and found the replay of the learned song was destroyed, similar to experimental lesioning of this area in adult canaries. These types of signals are then referred to as high-dimensional temporal signals (HDTS).</p><p>A network of Izhikevich neurons is FORCE-trained to internally generate its own HDTS while simultaneously being trained to reproduce the first bar of Ode to Joy. The network is able to learn both signals at once, with less training time and greater accuracy than without the HDTS. Another network is FORCE-trained to learn the <em>first</em> <em>four bars </em>in additional to its own 64-dimensional HDTS, again learning both signals without any error in the sequence of notes. Thus, internally generated HDTS can make FORCE-training faster / more accurate / more robust to longer signals.</p><h4>FORCE-trained encoding and replay of an episodic memory</h4><p>The authors wanted to know whether HDTS input signals could help neurons to learn natural high-dimensional signals. They trained a network of Izhikevich neurons to learned a 1,920-dimensional supervisor that corresponded to the pixels of an 8 second movie clip. The HDTS’s were either generated by a separate network or fed directly as input into an encoding / replay network. In the former case, an HDTS can be easily learned by a network, and in the latter case, we can freely manipulate the HDTS. The HDTS could also be learned jointly with the movie scene, constituting a 1,920 + 64 dimensional supervisor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*ucBM8W5GXuZwEpN30jWurg.png" /><figcaption>HDTS input signals were (left) generated by a separate SNN, or (right) fed directly as input into the replay network.</figcaption></figure><p>The networks was successfully trained to replay the movie clip in both cases. The HDTS inputs were necessary for both training and replay: networks could still replay individual frames from the clip without the HDTS, but the order of frames was incorrect and apparently chaotic.</p><p>Replay network performance decreased approximately linearly with the proportion of neurons removed from the network, with the amplitude of mean activity decreasing as well. The HDTS network was much more sensitive: randomly lesioning 10% of HDTS neurons stopped the network’s output.</p><p>It is speculated that compressed or reversed replay of an event might be important for memory consolidation. The authors wonder if networks trained with HDTS could replay the movie clip in accelerated time by compressing the HDTS in time post-training. The replay network is able to replay the movie in compressed time <em>up to a compression factor of 16x</em> with accuracy dropping sharply for further compression. The effect of the time compression on the mean activity was to introduce <em>high-frequency oscillations</em>, whose frequency <em>scaled linearly with the degree of compression</em>. With increasing compression, <em>large waves of synchronized activity</em> emerged in the mean population activity. Reverse replay was also successful, obtained by <em>reversing the order of the HDTS components</em>.</p><p>Compression of task-dependent sequence of spikes has been observed experimentally; e.g., a recorded sequence of neuronal cross correlations in rats elicited during a spatial sequence task reappeared in compressed time during sleep, with compression ratios between 5.4–8.1. This is similar to the compression ratios achieved using the movie-replay networks without incurring significant error in replay.</p><h3>Discussion</h3><p>Main take-away: FORCE-training can be used to train initially chaotic SNNs to mimic the functions / dynamics of populations of neurons. This is agnostic to neuron model and supervising signal.</p><p>High-dimensional temporal signals (HDTS) are used to aid networks to learn complicated dynamics by separating neurons into time-dependent assemblies. These signals made FORCE-training faster and more accurate, and by manipulating them, replay networks could compress and even reverse replay.</p><p>The HDTS conferred a slow oscillation in the mean population activity reminiscent of slow theta oscillations in the hippocampus, which is associated with memory (proposed to serve as a <em>clock</em> <em>for memory formation</em>).</p><p>The authors argues that they’ve shown an example of how natural stimuli (serving as proxies for memories) can be bound to underlying oscillations in a population of neurons, which forces the neurons to fire in distinct temporal assemblies. This mirrors experimental results which show that theta power was predictive of correct replay. Blocking hippocampal theta oscillations has been found to disrupt learning, similar to how block the HDTS prevents learning and accurate replay with networks trained with HDTS present.</p><p>FORCE-trained networks could be used to elucidate hippocampal function , and can also be constructed to represent different components of the hippocampal circuit.</p><p>FORCE training allows one to use any sufficiently complicated dynamical system for universal computation. In SNNs, this is difficult because of the balance of the fixed, chaos-inducing weights, and the learned feedback weights. If the former is too strong, the feedback weights cannot control the behavior of the network, and if it is too weak, the system’s dynamics can no longer form a suitable reservoir.</p><p>At present, all top-down procedures for constructing functional SNNs (including FORCE training) need more work in order to be considered biologically plausible. However, synapse parameters should not be considered biologically implausible simply because the methods that generated them are.</p><p>FORCE-trained rate-based networks have been successful in accounting for / predicting experimental data. Therefore, FORCE-trained SNNs may be useful for generating predictions involving voltage traces, spike times, and neuronal parameters.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f5d6c9fbb4d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Book review: “Reinventing Discovery: The New Era of Networked Science”]]></title>
            <link>https://medium.com/@danjsaund/book-review-reinventing-discovery-the-new-era-of-networked-science-7ceb6ba3c8da?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/7ceb6ba3c8da</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[science]]></category>
            <category><![CDATA[open-science]]></category>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Wed, 22 Aug 2018 19:21:31 GMT</pubDate>
            <atom:updated>2018-08-23T14:07:34.529Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://press.princeton.edu/titles/9517.html">Reinventing Discovery</a> by <a href="http://michaelnielsen.org/">Michael Nielsen</a> (an excellent writer across the board, IMHO) is a book about the impending revolution in the sciences towards networked and “open” science, and the promise and limitations of collective intelligence.</p><p>When I bought this book, I thought it had just been released, and didn’t realize until I cracked it open that it was from <em>seven years ago</em>. This is not a big problem, as the vast majority of the book is still relevant, but some examples from the past few years would have nicely supported some of Nielsen’s points.</p><p>These point include (but are not limited to):</p><ol><li>The internet is changing the way science is done, from individualistic and competitive to collaborative.</li><li>Non-scientists can play a part in scientific discovery.</li><li>New rewards (instead of just publications) are needed to incentivize scientists to work “out in the open”; i.e., share their discoveries freely.</li></ol><p>In the 17th century, the invention of the scientific journal was a major revolution in the way science was done at the time: instead of keeping research results secret, scientists could now publish them in order to claim the prestige associated with their intellectual property. Other scientists could then subscribe to the journal, for a fee, to stay up to date on findings in their field. In recent times, the pay-to-read scientific journal model has shown some strain. Since much of science is publicly funded, why aren’t the results publicly available? On the flip side, the pay-to-publish, free-to-read model represents a step forward, but still requires (oftentimes large) funds in the possession of the publishing author(s).</p><p>Point #1 is exemplified by such services as the <a href="https://arxiv.org/">arXiv</a>, where <a href="https://en.wikipedia.org/wiki/Preprint">pre-prints</a> can be uploaded with zero fees. Such a service allows an author to claim the origination of their ideas, while removing the monetary barriers described above. The only expense associated with the arXiv is in operating its servers, which is secured by “<a href="https://confluence.cornell.edu/pages/viewpage.action?pageId=340900096">a global collective of institutional members</a>”. For many, this is an improvement over paying for-profit academic publishers for the right to read or publish publicly funded research. However, papers published in prestigious journals often carry much more prestige than only freely available on pre-print servers.</p><p>Point #2 is made best by examples about the <a href="http://zoo1.galaxyzoo.org/">Galaxy Zoo</a> project and the <a href="https://fold.it/portal/">Foldit</a> game. The former asks amateur astronomers to classify images of galaxies into specific categories (on a volunteer basis); e.g., whether depicted galaxies are elliptical or spiral. The latter frames the question of protein folding as a game, in which contestants compete to find the lowest-energy configuration of a protein, which ultimately help scientists understand the dynamics of protein folding in reality.</p><p>In regards to point #3, some new rewards have already been developed since the book’s publication. The Journal of Open Source Software (<a href="http://joss.theoj.org/">JOSS</a>), for example, is an academic journal that reviews open source research software and publishes short papers about them, which can be used a form of academic credit when filling out a curriculum vitae or applying for jobs. <em>Reinventing Discovery</em> does an excellent job of listing other such incentives or even existing agreements between scientists to do their work out in the open. An exciting example is <a href="https://www.ncbi.nlm.nih.gov/genbank/">GenBank</a>, a publicly available DNA sequence database, and the accompanying commitment of geneticists to freely share all data on the human genome.</p><p>However, there are still strong incentives for keeping data, code, and ideas in general secret. These typically boil down to producing publications and patents, in order to further one’s career or deepen one’s pockets.</p><p>I liked the author’s writing style for much of the book, but certain chapters felt like they were a simple re-hashing of earlier chapters. The book succeeds best when relaying ideas through anecdotes (about open / networked science), and fails mostly when restating the book’s or the current chapter’s thesis again and again, in slightly different wordings.</p><p>Some of my favorite anecdotes were about the <a href="http://zoo1.galaxyzoo.org/">Galaxy Zoo</a>, the <a href="http://michaelnielsen.org/polymath1/index.php?title=Main_Page">Polymath Project</a>, <a href="https://www.google.org/flutrends/about/">Google Flu Trends</a>, and <a href="https://en.wikipedia.org/wiki/Kasparov_versus_the_World">Kasparov vs. the world</a>, mostly because I’d never heard about these before, and also because they neatly illustrated where <em>collective intelligence</em> succeeds or fails. The chapters that revolve around an example or two and relate these back to the book’s thesis were the most interesting to read.</p><p>Another big idea in the book is the concept of an <em>open data web</em><strong><em>, </em></strong>the idea of a network of data residing in / alongside the internet, the aim of which is facilitate efficient access to data which can be shared freely. The structure of this web should increase the prevalence of interdisciplinary science, in that seemingly disparate lines of work can be linked or even merged together (e.g., concepts from <a href="https://en.wikipedia.org/wiki/Control_theory">control theory</a> and <a href="https://en.wikipedia.org/wiki/Reinforcement_learning">reinforcement learning</a>), simply by having scientists spend the time to develop it. This is a task which must be incentivized in order for scientists to spend their valuable time on it. However, it seems the data web would add enough value (in terms of scientific productivity) to warrant the creation of this new incentive.</p><p>I would happily recommend this book, as well as Nielsen’s writings on <a href="http://michaelnielsen.org/blog/">his blog</a>, or his excellent <a href="https://www.amazon.com/Quantum-Computation-Information-10th-Anniversary/dp/1107002176">textbook on quantum computing and information theory</a> (which I haven’t finished yet). His <a href="http://michaelnielsen.org/">main website</a> is a good place to go to find out about all the projects he has in the works.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7ceb6ba3c8da" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Book Review: “Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems”]]></title>
            <link>https://medium.com/@danjsaund/book-review-theoretical-neuroscience-computational-and-mathematical-modeling-of-neural-systems-d794466b6559?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/d794466b6559</guid>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[mathematical-modeling]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neuroscience]]></category>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Mon, 13 Aug 2018 15:52:08 GMT</pubDate>
            <atom:updated>2018-08-13T15:52:08.988Z</atom:updated>
            <content:encoded><![CDATA[<p>This <a href="http://www.gatsby.ucl.ac.uk/~dayan/book/">book</a>, by <a href="http://www.gatsby.ucl.ac.uk/~dayan/">Peter Dayan</a> and <a href="http://neuroscience.columbia.edu/profile/larryabbott">LF Abbott</a>, is an exciting exploration of brains and their components. Basic differential equations, probability and statistics, and linear algebra are needed to understand it. It was published in 2001 by MIT Press.</p><p>The book is comprised of three parts:</p><ol><li><strong>Neural encoding and decoding</strong>: The study of how stimuli are converted into neural responses, in particular, action potentials or “spikes”. Chapters include “Neural encoding I: Firing rates and spike statistics”, “Neural encoding II: Reverse correlation and receptive fields”, “Neural decoding”, and “Information theory”. The authors alternate between describing neurons and their physiology / circuitry, providing analogous mathematical models, and comparing said models with real neural data. The chapter on information theory is especially useful for understanding how much is possible, in principle, to encode with neurons.</li><li><strong>Neurons and neural circuits</strong>: Single compartment and multi-compartment models of individual neurons are presented in terms of electrical circuit theory, and their properties compared. Firing rate models of networks of neurons are presented and analyzed with respect to real neural data. Chapters include “Model neurons I: Neuroelectronics”, “Model neurons II: Conductances and morphology”, and “Network models”.</li><li><strong>Adaptation and Learning</strong>: Both simple and realistic notions of synaptic plasticity are used to describe learning in pairs of interconnected neurons. Statistical machine learning and reinforcement learning concepts are related to learning functions in neural circuits and to the classical and instrumental conditioning of animals. Chapters include “Plasticity and learning”, “Classical conditioning and reinforcement learning”, and “Representational learning”. Learned representations from machine learning algorithms are compared with primate visual system receptive fields.</li></ol><p>Coming from the study of machine learning (and a bit of neuroscience along the way), this book was at the right level of reading difficulty for me to learn quite a bit while reading relatively fast. Indeed, the last few chapters of the book has a good discussion of machine learning models that I was mostly familiar with, but were related to neuroscience concepts in useful ways that I hadn’t seen before.</p><p>Overall, the book was a solid read, and I would recommend it. On the other hand, there are some parts of the book that I could have done without. I won’t go into specifics here (this would require a careful traversal of the book again), but I will say that certain sections seemed long-winded and not apparently useful. Some of the book’s plots were simple and illuminating; others were extremely difficult to understand.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d794466b6559" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Notes on “ Task-Driven Convolutional Recurrent Models of the Visual System”]]></title>
            <link>https://medium.com/@danjsaund/notes-on-task-driven-convolutional-recurrent-models-of-the-visual-system-d259ad319ac6?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/d259ad319ac6</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neuroscience]]></category>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Mon, 30 Jul 2018 15:11:57 GMT</pubDate>
            <atom:updated>2018-07-30T15:11:57.591Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://arxiv.org/pdf/1807.00053.pdf">This recent paper</a> (it’s a pre-print / work in progress) explores how to incorporate useful recurrence in <a href="http://cs231n.github.io/convolutional-networks/">convolutional neural networks</a> (CNNs) in order to boost accuracy on <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3306444/">object recognition</a> on natural images.</p><p>The authors argue that CNNs, although “<a href="http://www.pnas.org/content/111/23/8619">quantitatively accurate models of temporally-averaged responses of neurons in the primate brain’s visual system</a>”, do not exhibit the following features found in biological systems:</p><ol><li><a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3437251/">Local recurrent connections within cortical areas</a></li><li><a href="https://www.ncbi.nlm.nih.gov/pubmed/19397867">Long-range feedback from downstream to upstream areas</a></li></ol><p>The <a href="http://www.image-net.org/">ImageNet dataset</a> is used to test the methods in the paper. The argument is that it “… contains many images with properties that might make use of recurrent processing … (e.g. heavy occlusion, the presence of multiple foreground objects, etc).” Furthermore, “… some of the most effective recent solutions to ImageNet … repeat the same structural motif across many layers, which suggests that they might be approximable by the temporal unrolling of shallower recurrent networks …”.</p><p>The authors’ attempt at incorporating <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">standard forms of recurrence</a> (e.g., RNNs or LSTM cells) into CNNs didn’t show a significant performance improvement relative to the typical strictly <a href="http://www.deeplearningbook.org/contents/mlp.html">feed-forward</a> structure. This diverges from previous work, in which simple forms of recurrence are able to boost performance on simpler object recognition tasks (e.g., the <a href="https://www.cs.toronto.edu/~kriz/cifar.html">CIFAR-10</a> dataset). To overcome this shortcoming, new recurrent cell types (e.g., the “Reciprocal Gated Cell”) were hand-designed to integrate into a baseline CNN architecture. Architecture search was thenused to choose between thousands of local recurrent cells and long-range feedback configurations. The resulting “ConvRNN” network architecture schematic is shown in the figure below (taken from their paper).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bvm1ZeMWqn5EahYlVZ8SwQ.png" /></figure><p>Ultimately, these efforts resulted in a recurrent CNN model that matched the performance of the much deeper <a href="https://arxiv.org/pdf/1512.03385.pdf">ResNet-34 network</a> while using only 75% as many parameters. The ImageNet-driven ConvRNN provides “… a quantitatively accurate model of neural dynamics at a 10 millisecond resolution across intermediate and higher visual cortical areas.”</p><p>I really enjoyed this paper because it’s able to unify mature ideas from both machine learning and neuroscience and develops some truly great ideas. And, the writing is extremely clear and understandable! I’m excited by the hybridization of convolution and recurrence; although it’s been done before in different capacities, the authors hypothesize good reasons for why <em>their </em>method seems to work so well. One argument is that standard recurrent cells appear to lack the combination of the <strong>gating</strong> (“… in which the value of a hidden state determines how much of the bottom-up input is passed through, retained, or discarded at the next time step …”) and <strong>bypassing</strong> (“… where a zero-initialized hidden state allows feedforward input to pass on to the next layer unaltered, as in the identity shortcuts of ResNet-class architectures …”) properties hypothesized of recurrent connections in cortical areas. And, both features are thought to alleviate the problem of <a href="https://en.wikipedia.org/wiki/Vanishing_gradient_problem">vanishing gradients</a> as they are back-propagated to earlier layers.</p><p>Clever baseline results are established in order to convince the reader that the work advances the state of the art, since the addition of recurrent layers adds parameters to model, which “… could improve task performance for reasons unrelated to recurrent computation”:</p><ol><li>“ Feedforward models with more convolution filters (“wider”) or more layers (“deeper”) to approximately match the number of parameters in a recurrent model …”</li><li>“Replicas of each ConvRNN model unrolled for a minimal number of time steps, defined as the number that allows all model parameters to be used at least once.”</li></ol><p>An enormous amount of computation is needed for the architecture search: the authors use “… hundreds of second generation Google Cloud Tensor Processing Units (TPUv2s) …” to search over both learning hyperparameters and network architecture choices. However, the search may be worth the effort: it demonstrated that the aforementioned Reciprocal Gated Cell was better to integrate into a ImageNet-classifying CNN than the LSTM cell, increasing classification accuracy and reducing the number of learned parameters.</p><p>The following figure from the paper is my favorite:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w1IJOlExZBSLaVBGN1yFVw.png" /></figure><p>“… most units from V4 are best fit by layer 6 features; pIT by layer 7; and cIT/aIT by layers 8/9.” The analysis of the artificial network’s dynamics in conjuction with observed neural responses is genius, and I’d really like to see more of this. Given that these models can accurately model real neural dynamics, it should be possible to use them as prescriptive models of said dynamics, making for a cheap way to test hypotheses about the primate visual system.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d259ad319ac6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Book Review: Computer Age Statistical Inference]]></title>
            <link>https://medium.com/@danjsaund/book-review-computer-age-statistical-inference-195a71ee78bb?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/195a71ee78bb</guid>
            <category><![CDATA[inferential-statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[computer-science]]></category>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Fri, 22 Jun 2018 13:27:15 GMT</pubDate>
            <atom:updated>2018-06-25T13:11:31.801Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s time for another book review!</p><h3>Background</h3><p><a href="https://web.stanford.edu/~hastie/CASI/">Computer Age Statistical Inference</a>, by <a href="http://statweb.stanford.edu/~ckirby/brad/">Bradley Efron</a> and <a href="http://web.stanford.edu/~hastie/">Trevor Hastie</a>, is an effort to explain the development of statistics, in theory and practice, beginning at the end of the 19th century until today. It was published in 2016 by Cambridge University Press. Both Efron and Hastie are professors of statistics and biostatistics at Stanford University, and are extremely prolific writers on their subjects.</p><h3>Summary</h3><p>The authors make a strong distinction between <em>algorithmic</em> and <em>inferential </em>aspects of statistical analysis. The former refers to how data is processed; i.e., what procedures we apply to data to produce estimates of statistics in question. The latter is concerned with assessing the “goodness” of the aforementioned statistical procedures. For example, <em>averaging</em> is an example of a statistical algorithm for estimating the mean of data, while the <em>standard error</em> (square root of variance) is a typical way to assess its accuracy. This hints at an important theme throughout the book: “…the same data that supplies an estimate can also assess its accuracy.” However, the computation of the standard error is an algorithm itself, which is subject to inferential analysis concerning <em>its</em> accuracy!</p><p>The algorithmic aspect of analysis is unreliable without strong inferential justification. Averaging seems intuitively correct, but without the standard error, it would be difficult to know precisely how much data to collect in order to get an accurate estimate of the mean. Mathematics is required to understand the properties of estimators, such as <em>efficiency</em>, <em>biasedness</em>, or <em>variance</em>. For example, it is easy to show to show the <a href="http://mathworld.wolfram.com/SampleVariance.html">sample variance</a> is biased, so the naive algorithm for computing sample variance must be corrected for unbiasedness.</p><p>In recent years, there has been an unbalanced development of the two aspects in favor of algorithmic progress. There has been a proliferation of interesting datasets and computing power, making it easy to apply simple compute-intensive methods instead of the complicated and restrictive ideas from classical statistics.</p><p>For example, the <em>bootstrap</em> algorithm (and others like it) <em>resample</em> a dataset many times in order to get more precise estimates of a statistic. <em>Resampling </em>here means that many “fake” datasets are created by sampling <em>with replacement</em> from the original, “real” dataset. The statistic in consideration is estimated from each “fake” dataset, and the estimates are averaged together to provide a less variable estimate overall. The catch is, hundreds or thousands of bootstrap resampled datasets may be needed to create these accurate estimates; this is only recently possible thanks to the advancement of computing power.</p><p>There is good inferential justification for the bootstrap, but for many prediction-oriented methods in <em>machine learning</em>, this is lacking. For example, it’s widely accepted that <a href="https://ai.stackexchange.com/questions/1479/do-scientists-know-what-is-happening-inside-artificial-neural-networks"><em>we don’t understand why deep neural networks work</em></a>, and a mature inferential theory of such methods doesn’t seem likely to materialize any time soon. The book discusses neural networks in Chapter 18 (without a single mention of inference), giving a high-level description of their construction, training, and relationship to simpler prediction methods.</p><h3>Book structure</h3><p>The book is split into three parts: “Classical Statistical Inference”, “Early Computer-Age Methods”, and “Twenty-First-Century Topics”.</p><p>In the first part (roughly 1900–1950), a distinction between <em>frequentist</em>, <em>Bayesian</em>, and <em>Fisherian</em> inference is made, and their properties are described and compared. There’s also material on parametric models, important across all approaches to statistical inference, culminating in a discussion of the general construction of exponential families.</p><p>In part two (roughly 1950–1995), statisticians were free to develop algorithms that could be implemented out by early computers (rather than by mechanical calculator or hand!). This led to the methods such as the jacknife, the bootstrap, ridge regression, cross-validation, and more, all potentially infeasible before computers were widely available.</p><p>In part three (roughly 1995 — present), inference is largely set aside as powerful prediction algorithms take center stage. The book makes an effort to showcase recent inferential efforts towards justifying these methods, but concedes that several aren’t yet well understood. The last two chapters are on the advanced topics of “Inference After Model Selection” (combining discrete model selection and continuous regression analysis) and “Empirical Bayes Estimation Strategies” (using indirect evidence in practice, and “ learning the equivalent of a Bayesian prior distribution from ongoing statistical observations”).</p><h3>Notes</h3><p>On a personal note, this book was tough. It’s recommended for graduate students in statistics, and I’m a student of computer science. However, I think it’s crucial to study in order to do machine learning research. Really, it’s an important subject for anyone who does quantitative work! Some general notes about the text:</p><ul><li>The notation in the book is strange. This could simple be statisticians’ notation, but it took some getting used to.</li><li>Too much background knowledge is sometimes assumed. I frequently had to read other material to understand certain parts.</li><li>Some of the more advanced topics went completely over my head! I struggled especially with the final two chapters. This is a strong motivator to learn more.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=195a71ee78bb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Some thoughts on writing]]></title>
            <link>https://medium.com/@danjsaund/some-thoughts-on-writing-5d836b219a14?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d836b219a14</guid>
            <category><![CDATA[writing]]></category>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Tue, 19 Jun 2018 12:52:53 GMT</pubDate>
            <atom:updated>2018-06-19T21:31:18.472Z</atom:updated>
            <content:encoded><![CDATA[<p>I keep a journal for a number of reasons, chief among them to practice my writing. Some days, I don’t know what it is that I want to put down, but I convince myself to fill a page anyways. I think of it as a sort of <em>eat your vegetables </em>ritual which I (try to) do every morning.</p><p>Thanks to a recommendation from a friend, I’ve been listening here and there to the <a href="https://www.alieward.com/ologies/">Ologies</a> podcast, in which a different -ologist is interviewed every week or so. In the episode on mythology, <a href="http://www.tellingabetterstory.com/">John Bucher</a> convincingly argues that, in order to be a great writer, one first has to be a faithful reader. This is a fairly obvious practice to recommend, but it resonated especially well with me: improvements in my writing skills seem to stem from what I’m reading!</p><p>This is not to say I think I’m a <em>good</em> writer. I’ve had only the typical training, and the rest of my skills are self-taught. In order to challenge myself, I’m focusing on more free-form, public writing, starting with this blog.</p><p>Writing becomes more enjoyable through practice. “Practice” here connotes consistency and repetition. At first, it may be forced, and in my case, it was a self-development effort born out of frustration with my own mediocrity. I tried to convince my future self that my words were an accurate depictions of my thoughts and experiences from that time. It’s an excellent way of tracking progress at work, in relationships, and in the development of your own worldview. In time, some nuance developed, and I’ve begun to feel <em>proud</em> of the “voice” that is coming out of the pages. A page’s worth of journaling can take no more than a few minutes, but it feels best when careful thought is dedicated to each sentence.</p><p>I have a tendency towards technical writing, which makes sense in light of my perpetual studentship and typical reading material. This is not (necessarily) a bad thing; such writing can be beautiful or inspiring in the right hands (to name a few: Jane Goodall, Carl Sagan, Roger Penrose, …), and various layperson-friendly mediums exist for the dissemination of scientific findings. Several prominent scientists dedicate a chunk of their time to communicating their work, on Twitter or in journalistic settings.</p><p>However, I don’t want to constrain myself to purely technical subjects, and neither should the average “technical” person. This is born out of another frustration: the difficulty of <em>communicating what I read </em>and <em>how it modifies my understanding of the world</em>. A goal of science, if I understand it correctly, is to nudge public opinion in the direction of “truth” as suggested by scientific experiments, and it’s desirable to take part in this effort.</p><p>I’ve had some success in writing up short summaries of books I’ve read, and sharing these with my friends. I want to do this better; namely, technical jargon and terminology should be discarded in favor of sharing useful information clearly. This isn’t always easy to do; it’s can be difficult to communicate simple heuristics for understanding parts of the world from, say, math-laden textbooks (somewhat paradoxically).</p><p>But, an effort can always be made! I like the <a href="https://distill.pub/">distill.pub</a> venue for exactly this purpose, although, quality-wise, it seems a bit out of reach for people like me. So, that is one purpose for this blog, and why I will continue to make an effort to write.</p><p>Another thing that motivates me is the notion that <em>good writing is beautiful</em>. The topic doesn’t matter much as long as the story is told well. We are inclined to only consume media that support our philosophy of life, politics, or life’s work, but any good story is often so because of its structure and nuance and choice of words. You needn’t be convinced by, or agree with, everything you read; it’s often more useful to read something disagreeable and think about whether or not your beliefs need revision.</p><p>So, I’ll try to write for myself and for potential readers to appreciate, not for content alone, but also for form, elegance, and <em>convincing</em>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d836b219a14" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Book Review: Deep Learning textbook]]></title>
            <link>https://medium.com/@danjsaund/book-review-deep-learning-textbook-8160308ceee6?source=rss-4565a4947e1b------2</link>
            <guid isPermaLink="false">https://medium.com/p/8160308ceee6</guid>
            <category><![CDATA[textbooks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[reading]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Dan Saunders]]></dc:creator>
            <pubDate>Wed, 06 Jun 2018 21:21:15 GMT</pubDate>
            <atom:updated>2018-06-06T21:23:37.513Z</atom:updated>
            <content:encoded><![CDATA[<p>I’ve finished reading the <a href="http://www.deeplearningbook.org/">Deep Learning textbook</a> (by Ian Goodfellow, Yoshua Bengio, and Aaron Courville) after owning it for about 1.5 years. Having 710 pages of text, this amounts to my reading roughly 1.3 pages per day… of course, most days I didn’t read it at all, especially at particularly busy times of the school year. It’s somewhat surprising that there’s already a text on the subject, since the field “began” around 2006, and gained a significant following only around 2012. Since the field has garnered so much interest so quickly, however, there’s plenty to learn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2uaBxfJbd-zWokCgyo6iPg.jpeg" /></figure><p>Overall, I’m very happy with the book. It features a nice looking cover (presumably preduced with the nifty <a href="https://en.wikipedia.org/wiki/DeepDream">DeepDream</a> technique), clear and simple formatting, and generally excellent writing. It’s broken down into 3 sections and 20 chapters. The first section gives a good mathematical background and machine learning fundamentals; I particularly enjoyed the chapter on probability and information theory, and the machine learning reference is excellent. The second section is the most important for students or new practicioners: feedforward networks, regularization, optimization, convolutional networks, recurrent networks, methodology, and applications are discussed in detail, all in the context of deep learning. Finally, the third section covers less-developed research areas such as autoencoder networks, representation learning, probabilistic graphical models, approximate inference, and deep generative modeling. This chapter is more speculative, since many of the ideas are only a few years old at the time of writing!</p><p>Let’s break down some (personal) pros and cons:</p><h3>Pros</h3><ul><li><em>Conversational writing style</em>: Although occasionally ambiguous, I prefer this style to the terse, headache-inducing theorems and lemmas of math textbooks. This can be a con at times, when I want more equations, but fortunately these exist in the references.</li><li><em>Emphasis on progression of ideas</em>: So, textbooks typically do this, but this one does it well. I’m a sucker for the history of science, and though deep learning has a short history itself, the work which it relies on is extensive. I had a couple of “aha!” moments reading the historical notes and references sprinkled throughout.</li><li><em>Coverage</em>: I can’t think of any topic that I’d consider a glaring omission from the book, but this is likely because I’m not old enough to know the breadth of the field! Anyways, there are some great and somewhat obscure pointers to papers in Section 3 (Deep Learning Research) which I may not have come across otherwise. Tying in the work at the intersection of deep learning and Bayesian networks, probabilistic modeling, approximate inference, etc. is very helpful to see what avenues of research might be feasible in the future.</li></ul><h3>Cons</h3><ul><li><em>Egregious self-citation</em>: There were citations of Goodfellow’s and Bengio’s papers nearly every other page. Okay, maybe they deserve them because of their foundational contributions to the very young field, but it was a bit grating at times.</li><li><em>Sequencing chapters</em>: The sequence of chapters 16–20 were all building off of one another, which is a once a pro and a con. It felt something like a chore because certain things didn’t make total sense until after I had read the next chapter(s). However, by the end, confusing points were mostly resolved, which was actually very satisfying!</li></ul><p>Again, I think this book is well worth the read. Having a working knowledge of linear algebra, probability + statistics, and basic machine learning concepts will allow you to grasp the first two portions of the book without a problem. However, the third part can be challenging: taking a break to read up on specific topics came in handy for me when I was nearly finished with the book.</p><p>What should I read next? Here are some options (feel free to make suggestions!):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MMF6r_9nuAm9qlBCl-beIg.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8160308ceee6" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>