<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ikram Chraibi Kaadoud on Medium]]></title>
        <description><![CDATA[Stories by Ikram Chraibi Kaadoud on Medium]]></description>
        <link>https://medium.com/@ichraibik?source=rss-c59511f426bf------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by Ikram Chraibi Kaadoud on Medium</title>
            <link>https://medium.com/@ichraibik?source=rss-c59511f426bf------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 15:09:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ichraibik/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How are life episodes stored in the brain?]]></title>
            <link>https://medium.com/@ichraibik/how-are-life-episodes-stored-in-the-brain-31895eda97f3?source=rss-c59511f426bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/31895eda97f3</guid>
            <category><![CDATA[cognitive-modeling]]></category>
            <category><![CDATA[brain]]></category>
            <category><![CDATA[life-episode]]></category>
            <category><![CDATA[scientific-popularization]]></category>
            <category><![CDATA[memories]]></category>
            <dc:creator><![CDATA[Ikram Chraibi Kaadoud]]></dc:creator>
            <pubDate>Mon, 27 Feb 2023 14:48:18 GMT</pubDate>
            <atom:updated>2023-02-27T14:49:15.822Z</atom:updated>
            <content:encoded><![CDATA[<p>Here is the adapted version of the answer I posted on <a href="https://fr.quora.com/De-quelle-mani%C3%A8re-les-souvenirs-sont-ils-stock%C3%A9s-dans-le-cerveau">Quora (French version)</a> to the question “How are memories stored in the brain?” 4 years ago.<br>As this question has come up again recently in my work in the cognitive modeling field, I thought it would be interesting to share it here with all the scientific references.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rqFIL6YlJb-WJhgH1T5gsw.jpeg" /><figcaption>Photo de <a href="https://unsplash.com/fr/@averey?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Robina Weermeijer</a> sur <a href="https://unsplash.com/fr/photos/3KGF9R_0oHs?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h3>First, let’s give some context!</h3><p><strong>Cognitive modeling</strong> is a field of computer science concerned with modeling the cognitive functions that enable problem-solving and mental processing of information. This can concern modeling memory, reasoning, or attention.<br>Cognitive modeling is a field <strong>very close to artificial intelligence</strong> since a part of people in this field aims at creating AIs with the ability to adapt and understand the context, in order to have a successful interaction with the human involved. Cognitive modeling, despite recent works, does not concern only machine and deep learning. There are also cognitive models using Symbolic AI algorithms.<br>Working thus on the <strong>design of AI inspired by humans implies questioning how humans do it at the cognitive level</strong>, and thus asking questions such as the one we are addressing here</p><p>Cognitive modeling can also be very closely related to <strong>computational neuroscience</strong>, a field of research whose objective is to discover, understand and model the principles that govern neural development, the structure and physiology of the nervous system, and the emergence and functioning of associated cognitive abilities. This field may involve mathematical models and neural networks to propose artificial “models” of cerebral structures.</p><h3>Life episodes in the brain</h3><p>The brain is a set of cerebral structures that work in synergy to allow, among other things, each of us to use our own cognitive functions (memory, attention, language, executive functions, and visual-spatial functions) and to be who we are!</p><p>Memories are <strong>episodes</strong> in an individual’s life. It is therefore the episodic memory that comes into play.</p><p>Defined as the memory of personally experienced events in a specific spatiotemporal context (Tulving, 1983), episodic memory is a sub-part of declarative memory and is essentially “biographical”. Indeed, it allows us to link the memories of an individual’s life over time in order to form his or her identity. Consisting of information specific to individual experience, dated, localized, and sometimes emotionally connoted, it is opposed to semantic memory, the memory of facts and concepts (Tulving, 1972; Tulving, 1995).</p><p>With a specific brain substrate (internal temporal structures, hippocampus, and frontal cortex) (Insigrini, 2008), this memory system is characterized by information encoding, storage, and retrieval processes, which are essential for the formation, maintenance, and retrieval of representations of information perceived in this system (Ergis, 2008).</p><p>A key structure in the study of episodic memory, the <strong>hippocampus</strong> is closely related to the cortex and is present symmetrically in each hemisphere. In addition to memory, two other main functions characterize this brain structure: spatial navigation (O’Keefe, 1971) and behavioral inhibition (Gray, 2000)</p><p>The involvement of the hippocampus in episodic memory (Eichenbaum, 1993; Scoville, 1957; Squire, 2002) was initially demonstrated by the case of Henry Gustav Molaison (Scoville, 1957) (called the HM patient): the removal of the hippocampus to treat the patient’s epilepsy highlighted its importance in the process of acquiring new memories.Other patients with hippocampal lesions (observations of Scoville and Brenda Milner) have confirmed the importance of the hippocampus in episodic memory (Draganski, 2006).</p><p>The degeneration of the hippocampus is one of the signs of Alzheimer’s disease which manifests itself in the difficulty and then the impotence to acquire new memories for an individual.</p><h3>Encoding</h3><p>Encoding is the first step in the memorization process and is a process of information processing whose role is to process the characteristics of events (at the sensory-motor and emotional levels) and to transform them into a representation or memory trace. The latter, on a biological level, corresponds to a new configuration of neuronal activation.</p><p>It is notably during the encoding phase that individuals can put in place strategies for acquiring memories, such as the organization of information or mnemonics.</p><p>There are<strong> two types of encoding: the “superficial” and the “deep”. </strong>The first refers to rote learning, i.e. the exact reproduction of the original information. The second, more complex, consists of looking for associations with information already stored in order to relate common elements and distinguish those that are different.</p><p>This last point explains why we tend to model the hippocampus (or one of its substructures) by <strong>associative memories</strong> (such as the Steinbuch model or the Hopfield model) which have the characteristic of being addressable by their content (if a part of the model is altered, it remains possible to retrieve a part of the information despite everything) and to learn the association between two elements (the word “Red” and the color red) (Beati, 2013)</p><h3>Storage and consolidation</h3><p>Occurring after encoding, storage is a process that allows the maintenance of encoded information in the appropriate brain structures. It is biologically translated by changes in brain activity and it impacts the cellular level (signal transduction between neurons and modification of brain plasticity) and systemic level (formation and maintenance of memory circuitry)(McGaugh, 2000). This second stage involves consolidation. This process, which is responsible for the reinforcement of memories, allows for the automatic unconscious repetition of information involving the cortex, to anchor the information sufficiently so that it can be retained for a long time. Active especially during sleep, consolidation participates in the restructuring of memories into long-term memory (Inostroza, 2013)</p><h3>Retrieval</h3><p>Once the information has been stored, it is the so-called retrieval process, which is active, controlled, and requires resources, that brings it back to consciousness. This process, also known as “recall” is defined as the access, selection, and reactivation or reconstruction of stored internal representations from cues (Buchanan, 2007; Tulving, 1976). Retrieval is considered optimized, according to the “principle of specificity of encoding” (Plancher, 2013) when the memory trace integrates both the content of the episode and its context. The latter can thus facilitate access to the representation later on.</p><p>Another element that facilitates recovery is repetition. Indeed, the more memory is recalled (brought back to consciousness), the more its memory trace is reinforced. As a result, it is retained longer.</p><h3>In summary and in simplified form</h3><p>It is the strengthening of neuronal connections in the brain that allows us to keep a memory! The hippocampus records during the day the episodes of life and during the sleep phases, it interacts with the amygdala (involved in the cognitive process of decision-making) to make the cortex learn the important episodes by repeating them several times.</p><h3>References</h3><ul><li>Beati, T. &amp;. Alexandre F. (2013). How to combine learning of regularities and special cases? A bio-inspired model. At CAP’13, French Conference on Machine Learning.</li><li>Buchanan, T. W. (2007). Retrieval of emotional memories. Psychology Bulletin, 133(5), 761</li><li>Draganski, B, Gaser C, Kempermann G, Kuhn HG, Winkler J, et al. (2006). Temporal and Spatial Dynamics of Brain Structure Changes during Extensive Learning. The Journal of Neuroscience.</li><li>Eichenbaum, H. (1993). Memory, Amnesia and the hippocampal system. MIT Press</li><li>Ergis, AM, Eusop-Roussel E. (2008). Early episodic memory impairments in Alzheimer’s disease. Rev Neurol (Paris), 164 Suppl 3, , S96-S101.</li><li>Gray, J. A., &amp; McNaughton, N. (2000). The Neuropsychology of Anxiety: An Enquiry into the Functions of Septo-Hippocampal System. Oxford University Press.</li><li>Inostroza, M., &amp; Born, J. (2013). Sleep for preserving and transforming episodic memory. Annu Rev Neurosci.</li><li>Insigrini, M, Taconnat L (2008). Episodic memory, frontal functioning, and aging. Rev Neurol (Paris), 164 Suppl 3, 91–95.</li><li>McGaugh, J. L. (2000). Memory, a century of consolidation. Science, 287(5451), 248–251.</li><li>O’Keefe, J., &amp; Dostrovsky, J. (1971). The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain research, 34(1), 171 -175.</li><li>Plancher, G, Barra J, Orriols E, Piolino P. (2013). The influence of action on episodic memory, A virtual reality study. Q J Exp Pshychol (Hove), 895–909.</li><li>Scoville, W. B., &amp; Milner, B. (1957). Lost of Recent Memory after bilateral hippocampal lesions. Journal of neurology, neurosurgery and psychiatry, 11.</li><li>Squire, L.R. &amp; Schacter, D.L. (Eds.) (2002). The Neuropsychology of Memory (3rd Edition). New York: Guilford Press.</li><li>Tulving, E. (1972). Episodic and semantic memory. In Organization of Memory. London: Academic, 381, e402.</li><li>Tulving, E. (1976). The role of semantic memory in the storage and retrieval of episodic information. Psychology Bulletin, 19–25.</li><li>Tulving, E. (1983). Element of episodic memory. Oxford, Oxford University Press.</li><li>Tulving, E. (1995). Memory systems: Quo Vadis. (M. Press, Ed.) Gazzaniga MS (Ed.), The Cognitive Neurosciences, 839–847.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31895eda97f3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Developmental robotics, the study of knowledge development in Artificial Intelligence (AI)]]></title>
            <link>https://medium.com/@ichraibik/developmental-robotics-the-study-of-knowledge-development-in-artificial-intelligence-ai-b0173d2b2ff3?source=rss-c59511f426bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/b0173d2b2ff3</guid>
            <category><![CDATA[cognitive-modeling]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[scientific-popularization]]></category>
            <category><![CDATA[children]]></category>
            <category><![CDATA[developmental-robotics]]></category>
            <dc:creator><![CDATA[Ikram Chraibi Kaadoud]]></dc:creator>
            <pubDate>Tue, 18 Oct 2022 08:01:45 GMT</pubDate>
            <atom:updated>2022-10-18T08:10:35.626Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Last update: 18/10/2022</em></p><p>This article for the general public is also co-published in <a href="https://www.lemonde.fr/blog/binaire/2022/10/18/robotique-developpementale-ou-letude-du-developpement-des-connaissances-dans-une-intelligence-artificielle-ia/"><strong>French</strong> on the “Blog Binaire”</a>, a scientific mediation blog associated with <a href="https://www.lemonde.fr/">the newspaper “lemonde.fr”.</a> Thanks to <a href="https://www.linkedin.com/in/pascal-guitton-6a03834/"><strong>Pascal Guitton</strong></a> and <a href="http://www-sop.inria.fr/members/Thierry.Vieville/index.fr.html"><strong>Thierry Vieville</strong></a> for reviewing the French version of this article and to <a href="https://www.lemonde.fr/blog/binaire/a-propos/">the Binaire team</a> for sharing it. This article is also available on: <a href="https://ikramchraibik.com/2022/10/18/developmental-robotics-the-study-of-knowledge-development-in-artificial-intelligence-ai/">https://ikramchraibik.com/blog/</a></p><h3>Learning to learn, an AI issue but not only</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/363/0*x7s0PwF7-Wgy7hZt" /><figcaption>Photo de<a href="https://unsplash.com/@santesson89?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText"> Andrea De Santis</a> de<a href="https://unsplash.com/?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText"> Unsplash</a></figcaption></figure><p>It is often said and admitted that many algorithms are inspired by living organisms and that, conversely, the artificial can advance the understanding of the living<strong>*</strong>. As part of this approach, the development of knowledge in humans is an area that has been widely studied, for example, by computational methods using machine learning approaches or robotic approaches (Cangelosi, 2018). The goal: to realize flexible and efficient algorithms or robots capable of interacting effectively with humans and their environment.<br>One of the main challenges that exist during a human-machine interaction is to take into account the variability of the environment. In other words, the changing context of the interaction. A response from a machine that is relevant at a given moment may not be the right one a few moments later. This can be due to the environment (change of place, change of time) or to the individual with whom the machine interacts.</p><p><strong><em>*</em></strong><em>Attention dear reader, to be inspired by the living does not imply creating an artificial double; for example to mimic or duplicate: </em><a href="https://www.usinenouvelle.com/editorial/video-airbus-deploie-un-peu-plus-les-ailes-de-son-avion-experimental-albatrossone.N1013844"><em>the design of planes can be inspired by birds</em></a><em> but it is not for as much that the planes flap their wings</em></p><blockquote>For example, if I tell you “My train is in 10 minutes” and I am at the other end of town (the station being opposite my location), it is obvious that it will leave without me. If I am in front of the station, then it would be appropriate to hurry! Finally, if I am in the train when I say these words, then everything is good for me!</blockquote><blockquote>Thanks to <a href="https://www.imt-atlantique.fr/fr/personne/yannis-haralambous">Dr. Yannis Haralambous</a>, researcher in Natural Language Processing, Text Mining and Grapho-Linguistics at IMT Atlantic and the DECIDE team at LAB-STICC, CNRS for sharing this example</blockquote><p>Contextual understanding and assimilation is a subject in its own right between humans and, by extension, between humans and machines.</p><p>So how can we make a machine learn on its own to interact with a changing environment? In other words, how do we make this agent learn to reason: analyze the situation, deduce or infer a behavior, execute it, analyze the results and learn from its own interaction?<br>There is already a lot of work in the field of <strong>reinforcement learning</strong> in AI, which is interested in the development of artificial agents, or in <strong>cobotics</strong>, where the robotic system must take into account the location of human operators in order not to risk injuring them. But there are also approaches that tend to be inspired by cognition, and particularly that of children: <strong>developmental robotics approaches.</strong></p><blockquote><em>“One of the most recent paradigms, developmental robotics, proposes not to focus on the “adult” intelligence of an individual capable of solving a priori a large class of problems, but rather to study the way this intelligence is constituted during the cognitive and sensory-motor development of the individual. We are not trying to reproduce an immediately intelligent robot, but a robot that will be able to learn, starting with a reduced amount of innate knowledge. The robot learns to model its environment, the objects that surround it, its own body, it learns elements of language from lexicon to grammar, all this in strong interaction with the physical world that surrounds it but also through social interactions with humans or even other robots. The model that preoccupies the artificial intelligence researcher is no longer the chess player, but simply the baby and young child, capable of learning and developing cognitively.</em>”</blockquote><blockquote><strong>Extract</strong> from the presentation page (in french) of a series of lectures on this topic in 2013: <a href="https://x-recherche.polytechnique.org/post/Conf%C3%A9rence-Robotique-D%C3%A9veloppementale">https://x-recherche.polytechnique.org/post/Conf%C3%A9rence-Robotique-D%C3%A9veloppementale</a></blockquote><h3>A few definitions before going further!</h3><p>Let’s first go over a few definitions before tackling this fascinating subject of artificial cognition.</p><p>So what is Reinforcement Learning? It is a field of Machine Learning that focuses on how artificial agents take actions in an environment by balancing exploration (e.g., of uncharted territory) and exploitation (e.g., of current knowledge of reward sources)(Chraibi Kaadoud et al, 2022). This area addresses the issue of designing autonomous agents that can evolve through experience and interaction (Sutton, Barto, et al., 1998).</p><p>The second concept to be clarified is that of <strong>Robotics</strong>:<br>Robotics is a scientific and industrial field whose object of study is the robot itself. This includes its performance, energy, electronic, mechanical and even automatic characteristics.</p><p><strong>Cobotics</strong> is a scientific field that studies collaborative human-robot systems. A cobot is defined as a collaborative robot working in the same workspace as a human. For example, a toy robot or a host robot. Cobotics is very close to robotics, however, it does not encompass all the problems of robotics. Indeed, in cobotics, the heart of the matter is the perception of the cobot of its environment, its interaction with the human, and conversely, the perception, the interaction, and the acceptability of the human of its cobot. Cobotics is distinguished by an ergonomic and cognitive engineering component, absent from robotics.<br>Cobotics is therefore multidisciplinary in nature and lies at the intersection of three fields: robotics, ergonomics, and cognitive sciences (Salotti et al, 2018). Note that cobotics is not directly related to the topic of developmental robotics, but it is essential to distinguish between these two topics, hence this small aside.</p><p>Finally, this brings us to <strong>developmental robotics</strong>. This field is also known by other synonyms: cognitive developmental robotics, autonomous mental development, and epigenetic robotics.<br>This field of research is dedicated to <strong>the study of the design of behavioral and cognitive capacities of artificial agents in an autonomous way</strong>. In other words, this field is concerned with the development of robot behaviors and their representation of the world with which they interact and of everything related to their knowledge.<br>Intrinsically interdisciplinary, this field is directly inspired by the principles and mechanisms of development observed in the natural cognitive systems of children.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*hqBlN6C_jIK9W_O7" /><figcaption>Photo de<a href="https://unsplash.com/@rfieldss?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText"> Ryan Fields</a> de<a href="https://unsplash.com/s/photos/toddler-playing?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText"> Unsplash</a></figcaption></figure><p><strong>Indeed, what could be more curious and autonomous than a child in the discovery of his world?</strong></p><p>This field thus tends to draw on the development of children’s cognitive processes to design artificial agents that learn to explore and interact with the world as children do (Lungarella, 2003; Cangelosi, 2018).<br>How? The traditional approach is to start from theories of human and animal development belonging to the fields of developmental psychology, neuroscience, development, evolutionary biology, and linguistics and then formalize and implement them in robots or artificial agents.<br>Please note that developmental robotics is not the same as evolutionary robotics, which uses populations of robots that interact with each other and evolve over time.</p><h3>Why is developmental robotics interesting?</h3><p>In order to have artificial agents that evolve and adapt as they experience, researchers have been observing children at different stages of their learning and the development of this learning ability. Infants in particular actively create and select their learning experience by being guided by their curiosity. Work has therefore focused on modeling curiosity in AI to determine the impact of curiosity on the evolution of the learning abilities of artificial agents (Oudeyer and Smith, 2016). The application domains are numerous and can allow, for example, the design of robots capable of learning things over the long term and evolving in their learning or efficient algorithms capable of generating context-appropriate explanations in natural language for example.<br>Beyond the design of intelligent agents, the experimentation of these artificial models in robots allows researchers to confront their theory on developmental psychology with reality and thus confirm or refute their hypothesis on the development of children for example. Developmental robotics can thus be a tool for scientific experimentation and exploration of new hypotheses on human and animal development theories. A tool at the service of the child, inspired by the child.<br>A better understanding of human and animal cognitive development can then allow the design of machines (robots, artificial agents) adapted to interact with children as they grow and their context evolves. This also allows the creation of more child-friendly applications in educational technologies as shown in the diagram “What is developmental robotics?”.</p><p>Beyond human-machine interaction, this exciting field also raises questions about artificial curiosity, artificial creativity, and even the question of the motivation of an artificial agent or robot! In other words, developmental robotics also allows scientific discoveries to be made in the service of understanding the cognitive development of children and the design of artificial agents or machines that learn to learn throughout their experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/0*Lt8rW_-9TBU-BFE1" /><figcaption>Diagram “What is developmental robotics? The combination of cognitive science and artificial intelligence models in the service of understanding cognitive development and autonomous lifelong learning”. Images: <a href="https://pixabay.com/">@Pixabay</a></figcaption></figure><h3>Take away message</h3><p>The design of successful human-machine interaction is a quest in itself for which different approaches are possible: that of reinforcement learning which focuses on the artificial agent as an object of study in a given context, that of robotics which focuses on the robot as a subject of study from a mechanical and software point of view, and finally that of developmental robotics which is inspired by the cognitive development of children in order to create flexible, adaptable and evolving artificial machines/agents. This epic is in its infancy and many discoveries are still to come. However, let’s remember one thing: “how to learn to learn?” is as much a question of humans as of robots!</p><h3>References &amp; to learn more</h3><ul><li>Cangelosi, A., Schlesinger, M., 2018. From babies to robots: the contribution of developmental robotics to developmental psychology. Child Development Perspectives 12, 183–188.</li><li>Chraibi Kaadoud, I., Bennetot, A., Mawhin, B., Charisi, V. &amp; Díaz-Rodríguez, N. (2022). “Explaining Aha! moments in artificial agents through IKE-XAI: Implicit Knowledge Extraction for eXplainable AI”. <em>Neural Networks</em>,<em> 155</em>, p.95–118. <a href="https://doi.org/10.1016/j.neunet.2022.08.002">10.1016/j.neunet.2022.08.002</a></li><li>Droniou, A. (2015). <em>Apprentissage de représentations et robotique développementale: quelques apports de l’apprentissage profond pour la robotique autonome</em> (Doctoral dissertation, Université Pierre et Marie Curie-Paris VI).</li><li>Lungarella, M., Metta, G., Pfeifer, R., Sandini, G., 2003. Developmental robotics: a survey. Connection science 15,151–190.</li><li>Oudeyer, P. Y., &amp; Smith, L. B. (2016). How evolution may work through curiosity‐driven developmental process. <em>Topics in Cognitive Science</em>, 8(2), 492–502.</li><li>Padois Vincent (2011) Dossier « iCub et les robots de services » pour le site <a href="http://www.futura-sciences.com/">www.Futura-sciences.com</a> URL :<a href="https://www.futura-sciences.com/tech/dossiers/robotique-icub-robots-service-1143/"> https://www.futura-sciences.com/tech/dossiers/robotique-icub-robots-service-1143/</a></li><li>Salotti, J. M., Ferreri, E., Ly, O., &amp; Daney, D. (2018). Classification des systèmes cobotiques. <em>Ingénierie cognitique</em>, <em>1</em>(1).<a href="https://hal.archives-ouvertes.fr/hal-01943946"> https://hal.archives-ouvertes.fr/hal-01943946</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b0173d2b2ff3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hierarchical representations in deep neural networks]]></title>
            <link>https://medium.com/@ichraibik/hierarchical-representations-in-deep-neural-networks-3ef4d091f4ac?source=rss-c59511f426bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ef4d091f4ac</guid>
            <category><![CDATA[latent-space]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[interpretability]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Ikram Chraibi Kaadoud]]></dc:creator>
            <pubDate>Wed, 07 Sep 2022 07:12:18 GMT</pubDate>
            <atom:updated>2022-09-07T07:12:36.807Z</atom:updated>
            <content:encoded><![CDATA[<p>This article is also available on my personal website: <a href="https://ikramchraibik.com/blog/">https://ikramchraibik.com/blog/</a></p><blockquote><strong>Short introduction:</strong> Explainability in AI and interpretability in Machine learning are very active areas. However, one of the key concepts in these domains, namely the <strong>hierarchical latent representations of deep neural networks and their characteristics</strong>, is far from being simple to understand. I propose here a brief definition, extracted from a scientific article. You can find <a href="https://hal.archives-ouvertes.fr/hal-03343687/document">the <strong>original article</strong> at this link and cited at the end,</a></blockquote><p>One key property of the<a href="https://hal.archives-ouvertes.fr/hal-03343687/document"> Representation learning</a> domain in deep neural networks is the ability to provide both<strong> high-level features</strong> and<strong> low-level features</strong> for the same learned data. Recall that a deep neural network will encode a latent representation at each hidden layer. Since the layer n units can be all or partially connected to the layer n + 1 units, each layer uses the previous layer as input. If the previous layer is a hidden layer, then the input is already a<strong> latent representation, i.e. an abstract feature representation that characterizes the data.</strong> Thus, each layer extracts an abstract feature representation of the previous layer. As a result, a deep neural network learns multiple levels of abstraction and implicitly encodes a hierarchy of latent and abstract representations that are built progressively, layer by layer. The layers that are close to the input layer will encode a<strong> low-level feature representation</strong>, whereas those deeper inside the architecture will encode a <strong>high-level feature representation.</strong> In other words, the closer the considered layer is to the output layer, the more the representation is abstract [Bengio et al., 2013; Zhong et al., 2016; Lesort et al., 2018], as represented in the figure below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/0*xEftRu6ZF16ptssR" /><figcaption>Illustrative and schematic representation of the position of a low level representation and a high level representation in a deep neural network. hx refers to the x th hidden layer in the network.</figcaption></figure><p>It has also been shown that, in deep learning algorithms, hidden representations tend to keep dominant information and propagate them across hidden layers, regardless of the width or depth increase of the deep neural networks [Nguyen et al.,2021]. This characteristic of RL is also a key one for XAI: by extracting and comparing the low-level and the high-level representations of a deep architecture, we consider that it is possible to explicit the inner mechanism of the architecture by observing the differences between the representations</p><p><strong>To cite this article, please cite the original scientific paper:</strong></p><p>Ikram Chraibi Kaadoud, Lina Fahed, Philippe Lenca. Explainable AI: a narrative review at the crossroad of Knowledge Discovery, Knowledge Representation and Representation Learning. <em>Twelfth International Workshop Modelling and Reasoning in Context (MRC) @IJCAI 2021</em>, Aug 2021, Montréal (virtual), Canada. pp.28–40. <a href="https://hal.archives-ouvertes.fr/hal-03343687">⟨hal-03343687⟩</a></p><h3>Reference</h3><ul><li>[Bengio et al., 2013] Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. <em>IEEE transactions on pattern analysis and machine intelligence</em>, 35(8):1798–1828, 2013.</li><li>[Lesort et al., 2018] Timothee Lesort, Natalia DiazRodriguez, Jean-Franc¸ois Goudou, and David Filliat. State Representation Learning for Control: An Overview. Neural Networks, 108:379–392, December 2018.</li><li>[Nguyen et al.,2021] Thao Nguyen, Maithra Raghu, and Simon Kornblith. Do wide and deep networks learn the same things? Uncovering how neural network representations vary with width and depth. In <em>9th International Conference on Learning Representations</em>,<em> Virtual Event, Austria, </em>May 3–7, 2021. OpenReview.net, 2021.</li><li>[Zhong et al., 2016] Guoqiang Zhong, Li-Na Wang, Xiao Ling, and Junyu Dong. An overview on data representation learning: From traditional feature learning to recent deep learning. <em>The Journal of Finance and Data Science, </em>2(4):265–278, 2016.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ef4d091f4ac" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interpretable Machine learning: some needed definitions]]></title>
            <link>https://medium.com/@ichraibik/interpretable-machine-learning-some-needed-definitions-7c314c2a3263?source=rss-c59511f426bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c314c2a3263</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[interpretable-ml]]></category>
            <category><![CDATA[latent-space]]></category>
            <dc:creator><![CDATA[Ikram Chraibi Kaadoud]]></dc:creator>
            <pubDate>Wed, 24 Aug 2022 07:03:05 GMT</pubDate>
            <atom:updated>2022-08-24T07:03:05.211Z</atom:updated>
            <content:encoded><![CDATA[<p>This article is also available on my personal website: <a href="https://ikramchraibik.com/blog/">https://ikramchraibik.com/blog/</a></p><blockquote><strong>Short introduction:</strong> <strong>Interpretability in Machine learning</strong> is a very active area. The number of publications is exploding, the number of projects on Github too, and many popularization articles deal with these topics. However, paradoxically, it is difficult to find clear definitions of commonly used terms. I centralize here definitions from a scientific article (reviewed and validated by researchers). I extracted in this article parts of <a href="https://hal.archives-ouvertes.fr/hal-03343687/document">the original article cited at the end,</a> published in August 2021</blockquote><p>In the literature of neural networks, concepts like <strong>latent space</strong> and <strong>latent representation</strong> have been developed and widely used. However, to the best of our knowledge, no complete definitions have been clearly proposed for such concepts. Due to the importance of both concepts, we choose to formulate their definition next. We start by defining basics concepts like data, information, features, and knowledge :</p><p><strong>Definition 2.1 </strong>The raw material that represents the input of an algorithm is called <strong>data</strong>. Data can be noisy, partial/complete, un/structured and of different types [Grazzini and Pantisano, 2015; Malhotra and Nair, 2015].</p><p><strong>Definition 2.2</strong> A data set is a collection of data that describes real-word <strong>objects</strong> (such as cars, documents, animal, etc.) through multiple properties called <strong>features</strong> [Bishop, 2006].</p><p><strong>Definition 2.3</strong> Once data is analyzed and correlated, it represents <strong>information</strong>. Information can be reproduced from data and its importance depends on the context it is generated from/for [Grazzini and Pantisano, 2015; Malhotra and Nair, 2015].</p><p><strong>Definition 2.4</strong> <strong>Knowledge</strong> is a set of information that is assessed by a human, i.e. human adds a value and semantics according to his/her own background and context [Grazzini and Pantisano, 2015; Malhotra and Nair, 2015].</p><p><strong>Definition 2.9 Neural networks </strong>are machine learning models with several architectures, that are usually structured by one or several layers (input, hidden and output). Each layer is composed of one or several computational units called artificial neurons — conceptually derived from biological neurons [McCulloch and Pitts, 1943; Abraham, 2005]. Computational units can also be a Long Short Term Memory (well known also as LSTM) [Hochreiter and Schmidhuber, 1997] or Gated recurrent units [Cho et al., 2014]. A <strong>deep neural network </strong>has many hidden layers, units, and edges with weights. Units of layer n can be all or partially connected to units of layer n+ 1. Due to this inner complexity, deep neural networks are a typical example of black-boxes.</p><p><strong>Definition 2.10</strong> In neural networks, an<strong> activation pattern</strong> refers to units activation values of one of the layers. An activation pattern is a numerical vector of the size of the layer it is associated with. A hidden pattern refers to the activation pattern of a hidden layer.</p><p><strong>Definition 2.11 Latent space</strong> refers to the abstract multidimensional space associated to each layer of a neural network where the representation of the learned data is implicitly built. Latent space contains the meaningful internal features (definition 2.2) representations of learned data, which makes it not directly interpretable. In a deep neural network (definition 2.9), each hidden layer, whether it has the same number of units or not, has its own latent space. It is thus possible<br>to extract several implicit representations from this network.<br>The<strong> latent space</strong> can be used to achieve a data dimensionality reduction when the hidden layer is smaller than the input layer. This is the case for example with autoencoders and variational autoencoders [Kingma and Welling, 2014], models that can reduce high-dimensional inputs into efficient and representative low-dimensional representations [Roberts et al., 2018b].</p><p><strong>Definition 2.12</strong> <strong>Latent or hidden representation</strong> refers to the data representation implicitly encoded by a neural network during the learning task and thus is hidden-layer dependent [Bengio et al., 2013]. It is a machine-readable data representation that contains features of the original data that have been learned by the associated hidden layer. One key property of latent space (definition 2.11) is that real-world objects (definition 2.2) that are semantically close (e.g. cars of different brands), will end up grouped together in one latent space: their respective hidden representation in the corresponding layer will be close to each other compared to other objects that are not semantically close (e.g. cats) [Roberts et al., 2018a]. Thus, a latent representation is useful for pattern analysis (definition 2.5) and for similarity detection between objects (definition 2.2) using clustering methods.</p><p><strong>To cite these definitions, please cite the original article :</strong></p><p>Ikram Chraibi Kaadoud, Lina Fahed, Philippe Lenca. Explainable AI: a narrative review at the crossroad of Knowledge Discovery, Knowledge Representation and Representation Learning. <em>Twelfth International Workshop Modelling and Reasoning in Context (MRC) @IJCAI 2021</em>, Aug 2021, Montréal (virtual), Canada. pp.28–40. <a href="https://hal.archives-ouvertes.fr/hal-03343687">⟨hal-03343687⟩</a></p><blockquote>A little note: I hope these definitions help you better understand all these concepts related to interpretable ML</blockquote><h3>Reference</h3><ul><li>[Abraham, 2005] Ajith Abraham. Artificial neural networks. <em>Handbook of measuring system design</em>, 2005.</li><li>[Bengio et al., 2013] Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. <em>IEEE transactions on pattern analysis and machine intelligence</em>, 35(8):1798–1828, 2013.</li><li>[Bishop, 2006] Christopher M Bishop. <em>Pattern recognition and machine learning</em>. springer, 2006</li><li>[Cho et al., 2014] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In <em>The 18th Empirical Methods in Natural Language Processing</em>, pages 1724–1734, Doha, Qatar, October 2014. Association for Computational Linguistics.</li><li>[Grazzini and Pantisano, 2015] Jacopo Grazzini and Francesco Pantisano. Collaborative research-grade software for crowd-sourced data exploration: from context to practice — part I: Guidelines for scientific evidence provision for policy support based on big data and open technologies. EUR 27094. <em>Luxembourg: Publications Office of the European Union</em>, (JRC94504), 2015.</li><li>[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. ¨ <em>Neural computation</em>, 9(8):1735–1780, 1997.</li><li>[Kingma and Welling, 2014] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In <em>The 2nd International Conference on Learning Representations</em>, 2014.</li><li>[Malhotra and Nair, 2015] Meenakshi Malhotra and TR Gopalakrishnan Nair. Evolution of knowledge representation and retrieval techniques.<em> International Journal of Intelligent Systems and Applications</em>, 7(7):18–28, 2015.</li><li>[McCulloch and Pitts, 1943] Warren S McCulloch and Walter Pitts. A logical calculus of the ideas immanent in nervous activity. <em>The bulletin of mathematical biophysics</em>, 5(4):115–133, 1943.</li><li>[Roberts et al., 2018a] Adam Roberts, Jesse Engel, Colin</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c314c2a3263" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Data to Knowledge]]></title>
            <link>https://medium.com/@ichraibik/from-data-to-knowledge-52c42a13973c?source=rss-c59511f426bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/52c42a13973c</guid>
            <dc:creator><![CDATA[Ikram Chraibi Kaadoud]]></dc:creator>
            <pubDate>Wed, 10 Aug 2022 07:02:30 GMT</pubDate>
            <atom:updated>2022-08-10T07:02:30.752Z</atom:updated>
            <content:encoded><![CDATA[<p>This article is also available on my personal website: <a href="https://ikramchraibik.com/blog/">https://ikramchraibik.com/blog/</a></p><blockquote><strong>Short introduction</strong>: Data is an indispensable element for the development of neural networks in Machine learning and more globally in AI. Their characteristics (e.g. quality, quantity, velocity) can impact the performance of the neural network and beyond, the success of an AI project. Why ? Because data contains information (correlated data) that becomes knowledge when a human interprets it. This phenomenon of moving from data to knowledge has been discussed in an article that I co-authored (see citation at the end of this article). <strong>I extracted in this article the section of </strong><a href="https://hal.archives-ouvertes.fr/hal-03352727/document"><strong>the original article</strong> cited at the end,</a> that discuss this matter.</blockquote><p><strong>Target audience: </strong>Computer sciences audience, project managers</p><p><strong>Keywords</strong>: <strong>Data, information, Knowledge</strong></p><h3>Definitions: Data, Information, and Knowledge</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/0*HqEVB9c7AOhLLudX" /><figcaption>FIGURE 4 in (Boulos et al, 2021)| Data transformation into information (from 1 to 5) then knowledge through Knowledge data discovery steps: <strong>(a)</strong> raw data can be collected to become target data, which can also be acquired or generated by an external process: it is the data generation, acquisition, or collection step. <strong>(b)</strong> Target data are submitted to a data preprocessing step according to the data mining techniques targeted: target data can thus be cleaned, filtered, completed, and anonymized if necessary. This second step allows to obtain preprocessed data. <strong>©</strong> Preprocessed data are submitted to data mining techniques to detect, identify, and extract patterns and relevant features. <strong>(d) </strong>Discovered patterns and features can thus be rearranged (e.g., with visual tools) for or within the phase of interpretation during the postprocessing step. The result is the generation of information. <strong>(e) </strong>Information evaluated by a human becomes knowledge: this step is an external one (i.e., not technical) and takes into account knowledge of the situation (context, issues, stakeholders, etc.).</figcaption></figure><p>(<strong>Grazzini and Pantisano, 2015</strong>) defined each concept as follows:</p><p><strong><em>Data</em></strong> can be considered as raw material given as input to an algorithm. Since it cannot be reproduced when lost, it must be carefully preserved and harvested. It can be of different forms: a continuous signal as in the recording of an electroencephalogram (EEG), an image representing a magnetic resonance imaging (MRI), a textual data, or a sequence of numerical values representing a series of physiological measurements or decisions taken via an application. Data can be complete, partial, or noisy. For example, if only a portion of an EEG recording is available, then the data are partial. Conversely, an EEG recording that has been completed but that has some parts unusable is said to be noisy because the noise alters the completeness of the recording. Two types of data can be distinguished: <strong>unstructured data</strong>, i.e., data directly after their collection or generation, and <strong>structured data, </strong>i.e., data that have been analyzed, worked on, and put in relation to each other to put them in a format suitable for the analysis considered afterwards. In the second case, it is considered <strong><em>information</em></strong>. Importantly, data by themselves are worthless.<br><strong><em>Information</em></strong> is dependent on the original data and the context. If it is lost, it can be reproduced by analyzing the data. Depending on the data processed at time t, information must be accurate, relevant, complete, and available. Information is intelligible by a human operator and can be used in a decision-making process. It is therefore significant and valuable since it provides an answer to a question. It can take various forms such as a text message, a table of numerical values, graphs of all kinds, or even in the shape of a sound signal. When semantics are added to a set of information, it becomes <strong><em>knowledge</em></strong>. Information, depending on the context, will not have the same impact. It is the context and the semantics brought by it and the human operator involved that will determine the value of that knowledge.</p><p>To illustrate these definitions in a mental health setting, in the case of a patient undergoing a follow-up with a psychiatrist: the psychiatrist can make his patient pass numerous tests in order to collect data: MRI, EEG, and textual answers to questionnaires. These data, once processed, formatted, and analyzed together, will represent a set of information on the condition of the patient. It is the combination of the knowledge and experience of the doctor, combined with his knowledge of the patient, his family context, and the current socio-economic context, that will enable him to have a global knowledge of his patient and to provide him with the best possible support.</p><p>The passage from data to information thus requires a majority of digital processing to highlight correlations according to a given context. However, the passage to the knowledge stage requires<br>considering individuals involved (see Project design). Figure 4 illustrates data transformation into information through digital processing and into knowledge through human evaluation.<br>There is thus an increasing complexity in this process of transforming data into information and then into knowledge which makes it difficult to identify and extract.</p><h3>Knowledge Data extraction in the Litterature</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/0*qAfExVx9TmgRhlLG" /><figcaption>FIGURE 4 in (Boulos et al, 2021)| The nine Knowledge Discovery of Data stages adapted from (<strong>Fayyad, 1996</strong>): <strong>(1) </strong>understanding the scope of the application field, <strong>(2)</strong> creation of the target dataset,<strong> (3)</strong> data cleaning and preprocessing, <strong>(4)</strong> data reduction and projection: dimensionality reduction and extraction of invariant representations and relevant characteristics, <strong>(5) </strong>matching the KDD goals with the right data mining method(s), <strong>(6)</strong> exploratory analysis and selection model and hypothesis: selection of the data mining algorithm and method that will be used for the pattern search, <strong>(7)</strong> data mining: searching for interesting patterns in a particular form of representation (e.g., tree classification, regression, and clustering),<strong> (8)</strong> data postprocessing and visualization: interpretation of the patterns found with a possible return to any step from 1 to 7 for a new cycle, and <strong>(9)</strong> action on discovered knowledge.</figcaption></figure><p><strong>Knowledge Discovery of Data (KDD)</strong> is defined as the process of discovering useful knowledge from data <strong>(Fayyad, 1996).</strong> As a three-step process, the KDD includes (1) a preprocessing step which consists of data preparation and selection, (2) a data mining step involving the application of one or many algorithms in order to extract information (i.e., patterns), and (3) a postprocessing step to analyze extracted information manually by a human operator and lead to knowledge discovery. <br>As an iterative and interactive process, KDD involves many steps and decisions of the users. Iterations can continue as long as extracted information does not satisfy the decisionmaker (see <em>Identifying cognitive biases in digital health to improve health outcomes</em>).<br>Concretely, as illustrated in Figure 5, the KDD stages encompass the following: (1) understanding the scope of the application field; (2) creation of the target dataset; (3) data cleaning and preprocessing; (4) data reduction and projection: reducing the number of variables to be analyzed by reducing the dimensionality of the data, extracting invariant representations, or searching for relevant characteristics; (5) matching the goals of the KDD process with the right method(s) in data mining; (6) exploratory analysis and selection model and hypothesis: selection of the data mining algorithm and method that will be used for the pattern search; (7) data mining: searching for interesting patterns in a particular form of representation, which includes rule and tree classification, regression, and clustering; (8) data postprocessing and visualization: interpretation of the patterns found with possible return to any step from 1 to 7 for a new cycle; and (9) action on discovered knowledge. <br>Here, we mainly focus our approach on the technical aspect, i.e., data and their transformation into information. We aim to present a complete and global approach by covering the KDD stages in the life cycle of a digital health product from the definition of the scientific question to data collection and analysis (for further details, see <a href="https://hal.archives-ouvertes.fr/hal-03352727/document">Appendix 3</a> in the original article).</p><p><strong>To cite this article :</strong></p><p>LJ. Boulos, A. Mendes, A. Delmas,<strong> I.Chraibi K</strong>. An Iterative and Collaborative End-to-End Methodology Applied to Digital Mental Health. <em>Frontiers in Psychiatry</em>, Frontiers, 2021, 12, <a href="https://dx.doi.org/10.3389/fpsyt.2021.574440">⟨10.3389/fpsyt.2021.574440⟩</a>. <a href="https://hal.archives-ouvertes.fr/hal-03352727">⟨hal-03352727⟩</a></p><h3>Reference</h3><ul><li>Grazzini J, Pantisano F. <em>Guidelines for Scientific Evidence Provision for Policy Support Based on Big Data and Open Technologies</em> (2015).</li><li>Fayyad U, Piatetsky-Shapiro G, Smyth P. From data mining to knowledge discovery in databases. <em>AI Magazine. </em>(1996) 17:37.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52c42a13973c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interpretability, bias, ethics and transparency: what is the relationship? (3/3)]]></title>
            <link>https://medium.com/@ichraibik/interpretability-bias-ethics-and-transparency-what-is-the-relationship-3-3-63fa0ad17401?source=rss-c59511f426bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/63fa0ad17401</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[bias]]></category>
            <category><![CDATA[ethics]]></category>
            <category><![CDATA[transparency]]></category>
            <category><![CDATA[interpretability]]></category>
            <dc:creator><![CDATA[Ikram Chraibi Kaadoud]]></dc:creator>
            <pubDate>Fri, 29 Jul 2022 07:03:38 GMT</pubDate>
            <atom:updated>2022-07-29T07:03:38.510Z</atom:updated>
            <content:encoded><![CDATA[<p>This article was originally published in <strong>french</strong> on the <a href="https://scilogs.fr/intelligence-mecanique/interpretabilite-vs-explicabilite-comprendre-vs-expliquer-son-reseau-de-neurone-1-3/">Scilog blog related </a><a href="https://scilogs.fr/intelligence-mecanique/interpretabilite-vs-explicabilite-linterpretabilite-selon-differentes-approches-2-3/">to</a><a href="https://scilogs.fr/intelligence-mecanique/interpretabilite-vs-explicabilite-comprendre-vs-expliquer-son-reseau-de-neurone-1-3/"> the magazine “Pour la Science”</a> and on the <a href="https://www.lemonde.fr/blog/binaire/2020/10/20/linterpretabilite-selon-differentes-approches/">Blog Binaire, blog of the magazine “Pour la Science”,</a> for scientific popularization. Here is the English version, also available on my personal website: <a href="https://ikramchraibik.com/blog/">https://ikramchraibik.com/blog/</a></p><p><strong>Target audience: </strong>all audiences, high school students</p><p><strong>Keywords</strong>: <strong>Biases, ethics, Deep learning, Explainable AI, Artificial Intelligence, Interpretability, Popularization</strong></p><blockquote><strong>Short introduction</strong>: This article is the result of the collaboration of <a href="https://web.archive.org/web/20210225043640/https://www.linkedin.com/in/marine-lhuillier/"><strong>Marine LHUILLIER</strong></a><strong> </strong>and myself in 2020. Marine was my intern and we both worked on an explainable AI project. During our collaboration, Marine get specialized in research at the crossroad of AI and Cognitive Sciences. She is now a Research Engineer / Data Engineer.</blockquote><p><em>The third and last article of our series that questions the concepts of interpretability and explicability of neural networks, we end here with an opening on the particular relations between interpretability, bias, ethics and transparency of these networks.</em></p><h3>Representations and biases of a neural network</h3><p>The <strong>goal of an interpretability approach</strong> is to allow <strong>access to the implicit memory of a neural network </strong>in order to extract the rules and representations encoded (i.e. learned or extracted from the data) during its training; the extraction can take place during or after the training. In other words, one can explore and bring into the explicit domain the implicit “reasoning” that it has built for itself thanks to the numerous examples seen and repeated practice (in a way quite similar to that of a human who acquires knowledge through experience and living).</p><p>Just as a human has a representation of the world in which he lives, built according to his experience, a neural network, as it is fed with data, also builds its own representation. Although this representation is limited by the data it has learned, it still contains its knowledge and by extension the reasons on which its behavior and predictions are based.</p><p>But in this case, what happens when an individual learns only one vision of the world (a deliberately exaggerated example: the sky is green during the day and grey at night)?</p><p>Then no matter how many images of sunrises and sunsets, starry nights or rainy skies he has seen, he will always consider that all these images are wrong and only what he has learned is right. As human beings, we nevertheless have a capacity for questioning and reconsideration which can, with time, lead us, through our experience, to relativize our learning and to realize that there are other shades of color just as beautiful to observe in the sky.</p><p>Unfortunately, neural networks do not have this capacity because they often carry out a single learning phase (especially in the case of supervised learning), of several examples certainly, but in the most common cases in one go. In other words, <strong>once a rule is learned, it becomes immutable!</strong> In the case of the classification of the colors of the sky, this does not have much impact, but let’s imagine that this algorithm is used to classify the value of individuals according to their results, whatever they are. The algorithm could then consider, according to the type of data it has received and therefore the rule(s) it has implicitly encoded, that only women’s profiles correspond to a secretary’s job and, on the contrary, that only men’s resumes should be retained for a technical job in the automobile industry. These stereotypical examples, although quite simple and basic, reflect a sad reality: <strong>the biases present in AI can be dangerous and discriminatory.</strong></p><p>Indeed, AI algorithms and their decision making are influenced by the data they learn. This raises a problem around the biases present in these data but also those coming directly from the developers implementing these algorithms. Bad data sets can lead to unfortunate consequences and biased implementation. If an AI algorithm learns from the data, wrong or biased rules, then its decisions will be just as wrong and will reproduce what is called<strong> selection or judgment bias</strong>. An infamous example is the description provided by Google in 2015 on a photo of an African-American couple classified as gorillas because the corpus of data was not representative enough of the diversity of the population…</p><p>As we can see, the impact of these biases can be very serious, especially in critical areas such as health, defense or finance. Nowadays, many works exist on this subject to study these biases. Nevertheless, <strong>it is important to underline that knowing about them does not necessarily mean that we know how to eliminate them </strong>[Crawford, 2019]! Moreover, the removal of these biases in an algorithm can be very costly in terms of resources because it would be necessary to restart the learning of neural networks after correction. However, depending on their depth, complexity and the volume of data to be processed, this cost can be very, even too high. These biases are therefore sometimes retained by default because they would be too expensive and uncertain to correct. For example, <a href="https://web.archive.org/web/20210225050944/https://medium.com/fair-bytes/how-biased-is-gpt-3-5b2b91f1177">GPT-3</a>, the language model developed by the company OpenAI in 2020 [Brown, 2020], <a href="https://web.archive.org/web/20210225050944/https://venturebeat.com/2020/06/01/ai-machine-learning-openai-gpt-3-size-isnt-everything/">would require a memory of more than 350 GB, with a cost related to training the model that would exceed $12 million</a>.</p><h3>What to remember?</h3><p>In the context of the demystification of neural networks, AI algorithms also called “black boxes”, <strong>interpretability</strong> and <strong>explainability</strong> are two distinct but closely related research fields. If interpretability consists in studying and explaining the internal logic of neural networks, in other words their functioning, explicability focuses on “how” to make sense of this internal logic in the most suitable way possible. Thus, interpretability is often carried out by Machine Learning experts for other experts, whereas explicability can be aimed at neophytes and experts.</p><p>Our interest in these two domains, as well as that of many researchers, makes sense in light of the societal and ethical issues discussed above: the impact of bias, discrimination, compliance with the RGPD, etc. Indeed, succeeding in better understanding how these algorithms work allows us to help demystify AI and make these “black boxes” more transparent, intelligible and understandable. In a more general way, it also allows us <strong>to improve the confidence</strong> of each and every one of us in these tools that tend to become omnipresent in our daily life or, on the contrary, to refuse those that would not be sufficiently transparent.</p><p>If we had access to the “how” and “why” decisions of AI tools are made, we could then intervene on their functioning as well as on the data they use as a learning base, which are often our own data. We could then make sure that we are moving <strong>towards a more inclusive society </strong>where each of us is respected in all his or her diversity.</p><blockquote><strong>A word from the authors: </strong>This educational article presents one approach to interpretability. There are others and for the more curious, the bibliography of each post is there for you! The posts are not intended to give an equivalence between a human being and a neural network because a human is much more complex! The examples provided are just for illustration purposes to allow a better assimilation of the concepts. However, we would be happy to discuss them further with you! <strong>Marine LHUILLIER &amp; Ikram</strong></blockquote><p><strong>To cite this article :</strong></p><p>Lhuillier M and Chraibi Kaadoud I, Interpretability, bias, ethics and transparency: what relationships? (3/3) . Published on the blog of <a href="http://www.scilogs.fr/intelligence-mecanique,">http://www.scilogs.fr/intelligence-mecanique,</a> October 2020</p><h3>Reference</h3><ul><li>Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., … &amp; Agarwal, S. (2020). Language models are few-shot learners. a<em>rXiv, </em>arXiv:2005.14165.</li><li>Crawford Kate (2019) « Les biais sont devenus le matériel brut de l’IA ». URL : <a href="https://www.lemonde.fr/blog/internetactu/2019/10/03/kate-crawford-les-biais-sont-devenus-le-materiel-brut-de-lia/">https://www.lemonde.fr/blog/internetactu/2019/10/03/kate-crawford-les-biais-sont-devenus-le-materiel-brut-de-lia/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63fa0ad17401" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interpretability vs. explainability: Interpretability according to different approaches (2/3)]]></title>
            <link>https://medium.com/@ichraibik/interpretability-vs-explainability-interpretability-according-to-different-approaches-2-3-3b7f44bb8f3b?source=rss-c59511f426bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b7f44bb8f3b</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[interpretability]]></category>
            <category><![CDATA[scientific-popularization]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Ikram Chraibi Kaadoud]]></dc:creator>
            <pubDate>Thu, 28 Jul 2022 07:01:30 GMT</pubDate>
            <atom:updated>2022-07-28T07:01:30.223Z</atom:updated>
            <content:encoded><![CDATA[<p>This article was originally published in <strong>french</strong> on the <a href="https://scilogs.fr/intelligence-mecanique/interpretabilite-vs-explicabilite-comprendre-vs-expliquer-son-reseau-de-neurone-1-3/">Scilog blog related </a><a href="https://scilogs.fr/intelligence-mecanique/interpretabilite-vs-explicabilite-linterpretabilite-selon-differentes-approches-2-3/">to</a><a href="https://scilogs.fr/intelligence-mecanique/interpretabilite-vs-explicabilite-comprendre-vs-expliquer-son-reseau-de-neurone-1-3/"> the magazine “Pour la Science”</a> and on the <a href="https://www.lemonde.fr/blog/binaire/2020/10/20/linterpretabilite-selon-differentes-approches/">Blog Binaire, blog of the magazine “Pour la Science”,</a> for scientific popularization. Here is the English version, also available on my personal website: <a href="https://ikramchraibik.com/blog/">https://ikramchraibik.com/blog/</a></p><p><strong>Target audience: </strong>all audiences, high school students</p><p><strong>Keywords</strong>: <strong>Black box, Deep learning, Explainable AI, Artificial Intelligence, Interpretability, machine learning, neural networks, Popularization</strong></p><blockquote><strong>Short introduction</strong>: This article is the result of the collaboration of <a href="https://web.archive.org/web/20210225043640/https://www.linkedin.com/in/marine-lhuillier/"><strong>Marine LHUILLIER</strong></a><strong> </strong>and myself in 2020. Marine was my intern and we both worked on an explainable AI project. During our collaboration, Marine get specialized in research at the crossroad of AI and Cognitive Sciences. She is now a Research Engineer / Data Engineer.</blockquote><p>Although artificial neural networks are among the most accurate Machine Learning algorithms, they are currently the most obscure for humans (Figure 1).</p><p>Indeed,<a href="https://web.archive.org/web/20201201142554/http://www.scilogs.fr/intelligence-mecanique/interpretabilite-vs-explicabilite-comprendre-vs-expliquer-son-reseau-de-neurone-1-3/"> in the approach that researchers and developers have had for years, they have often neglected or forgotten the interpretability and transparency of these algorithms in favor of performance.</a></p><blockquote>This article is the <strong>second in a series of three articles </strong>questioning the concepts of interpretability and explicability of neural networks. It provides an overview of existing interpretability approaches according to the objectives sought and attempts to provide some answers to the question “how to open the “black boxes” that are neural networks?”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/0*s7G5ghibqyJMuE6N" /><figcaption>Figure 1 — Interpretability levels of Machine Learning algorithms. Image adapted from Dam et al.[2018]. Translation of terms: 1) Précision : Accuracy, 2) Interprétabilité: interpretability, 3) Faible/Moyenne/Haute: Weak/Average/High, 4) Réseaux de neurones : Neural networks, 5) Méthode d’apprentissage ensemblises: Ensemble methods, 6) Modèles graphiques: Graphical models, 7) Arbre de décision: Decision Tree, 8) Algorithmes de régression: Regression Algorithm, 8) Règles de classification: Classification rules. SVM does not need translation, it stands for Support Vector Machine. The acronym is also used in french.</figcaption></figure><h3>Local or global interpretability?</h3><p>When we talk about the interpretability of artificial neural networks, it is essential to know what we are trying to explain and how we want to use them. Indeed, the techniques that are derived from them are distinguished according to several criteria.</p><p>First, it is necessary to know <strong>what type of behavior needs to be analyzed</strong>. Are we looking for an explanation of the behavior of the whole model or of its behavior on a particular result?</p><p>The first approach, called <strong>global interpretability</strong>, tends to provide an explanation of the global behavior of the network on all the data it has learned. It makes the decision making process transparent for all the data and proves to be a valuable way to evaluate the relevance of what the model has learned [Clapaud, 2019]. On the other hand, <strong>local interpretability</strong> tends to provide an explanation for a specific outcome, i.e., for a particular decision on a very small scale. It is particularly relevant when it is necessary to analyze a particular case for the needs of a person (patient, client…) for example [Guidotti, 2018].</p><p>Let’s illustrate this with an example: Let’s imagine having access to a high-tech microwave oven of which we would have lost the instructions for use, and therefore, of which we were unaware of the operation. We can then always observe the reaction of the microwave when we press the buttons as we go along. By doing this, we associate a result (heated dish) with the arrival of an information (I pressed a particular button). This is what amounts, in fact, to interpreting locally the behavior of the machine (here the microwave oven) according to incoming information since we associate an action and a result through a function. We are then in a dimension of local interpretability:</p><blockquote>In the case of the microwave: Functions of the microwave (heating button pressed) = heated dish. <br>In a more generic case: Functions of a neural network (incoming data at time t) = result/prediction at time t</blockquote><p>The global interpretability corresponds to a simple and global description of the complete functioning of the microwave. In other words, it is a question of outlining its mode of operation: we will know which type of buttons allows defrosting, which type of buttons allows heating, which type of buttons allows adjusting the heating time, etc. But we will not have direct access to the precise behaviors, such as “to defrost a food you must first select the type of food”. But we will not have direct access to precise behaviors, such as “to defrost a food you must first select the type, then the weight, then check the time regularly to reach the desired result*.</p><p>In summary, global interpretability allows access to the chapters and main sections of the operating instructions, whereas local interpretability allows access to specific functions of the microwave.</p><h3>Exploring “a reasoning in progress” or “ a constructed reasoning”?</h3><p>The second use that is important to determine is <strong>to define when the extraction of knowledge from the network will be done.</strong></p><p>If the interpretability phase is carried out during the learning of the neural network, it is a <strong>pedagogical method</strong> because it is possible to observe the behavior of the neural network at several moments during its learning. Indeed, it is admitted that during this stage, the neural network manages to identify and extract characteristics from the data that allow it to adjust itself, i.e. to learn by modifying the weights.</p><p>On the other hand, if the extraction is done after learning, i.e. during the test phase, we speak of a <strong>decompositional method</strong> because in this case, we observe the activity of each hidden layer of the network separately before combining them.</p><p>For example, consider a supervised learning task where a network must learn to classify images of turtles and kittens. As it learns, and therefore as it receives examples, this network will identify the characteristics of the images that are specific to each class: pixels, color code, positioning of a set of pixels, etc. Thus, the more examples it processes, the more it will identify these features and the more it will be able to adjust itself to match the right image to the right class.</p><p>In this particular case, if we apply a pedagogical method to this network, it is possible to observe the evolution of this feature identification phase at different moments of its learning (for example, every 100 examples presented). In other words, we can observe <strong>what it learns and the construction of its “reasoning”.</strong></p><p>On the other hand, if we use the decomposition method, we will study the behavior of the neural network (i.e. the activities of the units of the hidden layer(s)) each time it receives a posteriori image of its learning.</p><p>At this stage, therefore during the test phase, the network has finished learning and it is possible to extract and analyze its activity at the level of its hidden layers (its behavior or internal activity) in front of each image according to the learning carried out previously. In other words, during this process, we extract individual behaviors from the hidden layers before combining them to <strong>obtain the global behavior of the network </strong>and thus <strong>understand its “reasoning”.</strong></p><p>Analyzing a neural network after learning via a decomposition method, therefore, allows us to evaluate its implicit learning by making it explicit.</p><h3>What to remember?</h3><p>If there is only one thing to remember, it is that in terms of interpretability, as in Machine Learning in general, there is not one possible approach but several. Depending on the question we wish to answer (explaining a local or global behavior of the network) and what we wish to understand (how does it learn from the data or how does it make its predictions?) the approach(es) of interpretability of neural networks adopted will be different.</p><p>Indeed, it is possible to want to explain or understand the behavior of the network in its totality, i.e. to do <strong>global interpretability</strong>: what are the set of rules that the network has learned implicitly to classify turtles and kittens? Or on the contrary, we can focus on a specific example by doing <strong>local interpretability</strong>: why was this specific image classified in this way?</p><p>On the same principle, do we seek to understand h<strong>ow the reasoning of the network is built in “real time”</strong> <strong>or once it is built</strong>?</p><p>Wishing to answer these questions requires choosing, implementing and/or using the appropriate interpretability technique among the many existing ones. To learn more about this subject, it is possible to explore research areas in Machine Learning such as Representation learning also known as feature learning [Bengio et al, 2013] and Rules Extraction [Jacobson, 2005].</p><p>Moreover, questioning the interpretability and explicability of neural networks leads us, as researchers and users of these algorithms, to consider their impact on our daily lives and by extension to questions of <strong>ethics</strong> and <strong>acceptability</strong>: are we ready to accept more AI if we have no guarantee of ethics, <strong>inclusion,</strong> and <strong>justice</strong>? Above all, do we have the means to answer our questions on this subject? These complex relationships between interpretability, bias, ethics, and transparency will be precisely presented in the third and last article of this series!</p><p>* Thanks to Frédéric Alexandre, Director of the Mnemosyne team, INRIA Bordeaux, for his help in the elaboration of this “general public” example of local and global interpretability</p><p><strong>To cite this article:</strong></p><p>Lhuillier M and Chraibi Kaadoud I, Interpretability vs explainability: understanding vs explaining one’s neural network (1/3)? Publication on the blog of <a href="http://www.scilogs.fr/intelligence-mecanique,">http://www.scilogs.fr/intelligence-mecanique,</a> August 2020</p><p>References</p><ul><li>Bengio, Y., Courville, A., &amp; Vincent, P. (2013). Representation learning: A review and new perspectives. <em>IEEE transactions on pattern analysis and machine intelligence</em>, 35(8), 1798–1828.</li><li>Clapaud, Alain (2019). Explicabilité des IA : quelles solutions mettre en oeuvre ? Publié sur LeMagIT. URL : <a href="https://www.lemagit.fr/conseil/Explicabilite-des-IA-quelles-solutions-mettre-en-uvre">https://www.lemagit.fr/conseil/Explicabilite-des-IA-quelles-solutions-mettre-en-uvre</a></li><li>Dam, H. K., Tran, T., &amp; Ghose, A. (2018). Explainable software analytics.<em> In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results</em> (pp. 53–56).</li><li>Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., &amp; Pedreschi, D. (2018). A survey of methods for explaining black box models. <em>ACM computing surveys (CSUR)</em>, 51(5), 1–42.</li><li>Jacobsson, H. (2005). Rule extraction from recurrent neural networks: A taxonomy and review. <em>Neural Computation, </em>17(6), 1223–1263</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b7f44bb8f3b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Interpretability vs. explainability: understanding vs. explaining one’s neural network (1/3)]]></title>
            <link>https://medium.com/@ichraibik/interpretability-vs-explainability-understanding-vs-explaining-ones-neural-network-1-3-3084c9bb9eda?source=rss-c59511f426bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/3084c9bb9eda</guid>
            <category><![CDATA[interpretability]]></category>
            <category><![CDATA[explainable-ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[popularization]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Ikram Chraibi Kaadoud]]></dc:creator>
            <pubDate>Wed, 27 Jul 2022 07:06:06 GMT</pubDate>
            <atom:updated>2022-07-27T07:06:06.705Z</atom:updated>
            <content:encoded><![CDATA[<p>This article was originally published in <strong>french</strong> on the <a href="https://scilogs.fr/intelligence-mecanique/interpretabilite-vs-explicabilite-comprendre-vs-expliquer-son-reseau-de-neurone-1-3/">Scilog blog related to the magazine “Pour la Science”</a> and on the <a href="https://www.lemonde.fr/blog/binaire/2020/09/04/comment-saisir-ce-que-font-les-reseaux-de-neurones/">Blog Binaire, blog of the magazine “Pour la Science”,</a> for scientific popularization. Here is the English version, also available on my personal website: <a href="https://ikramchraibik.com/blog/">https://ikramchraibik.com/blog/</a></p><p><strong>Target audience: </strong>all audiences, high school students</p><p><strong>Keywords</strong>: <strong>Black box, Deep learning, Explainable AI, Artificial Intelligence, Interpretability, machine learning, neural networks, Popularization</strong></p><p><strong><em>Short introduction</em></strong><em>:</em> <em>This article is the result of the collaboration of </em><a href="https://web.archive.org/web/20210225043640/https://www.linkedin.com/in/marine-lhuillier/"><strong><em>Marine LHUILLIER</em></strong></a><strong><em> </em></strong><em>and myself in 2020. Marine was my intern and we both worked on an explainable AI project. During our collaboration, Marine get specialized in research at the crossroad of AI and Cognitive Sciences.</em> <em>She is now a Research Engineer / Data Engineer</em>.</p><p>As human beings, how can we understand and explain an automated decision made* with an algorithm from what is called Artificial Intelligence, or AI, (for example a neural network)?</p><blockquote>*The expression “decision making” of an algorithm refers here to the decision made by a human following a proposal (i.e. prediction) from this algorithm. It is therefore a decision proposal.</blockquote><p>It is more than important to be able to explain and interpret these results, sometimes bluffing or simply counter-intuitive, which often guide our human decisions. This problem has its origins in the very concept of Machine Learning.</p><p><em>The</em><strong><em> first of a series of three articles </em></strong><em>that question the concepts of interpretability and explainability of neural networks, we begin here with an introduction to the problems related to the understanding of these Machine Learning algorithms.</em></p><p>Machine Learning (ML) is a sub-field of artificial intelligence domain that consists in giving a machine (a computer) a learning capacity without it having been explicitly programmed for this [WhatIs, 2016] and this, by adjusting its calculation according to so-called learning data. In other words, an ML algorithm consists in asking our machine to perform a task without precisely coding the different steps and actions it will have to perform to get there but by adjusting the parameters of a very general calculation to the data provided.</p><p>It is precisely these automated decisions, linked to the use of Machine Learning algorithms, especially artificial neural networks, known as deep learning when there are many layers of calculation, that are raising more and more ethical and legal issues [Hebling, 2019]. These problems are mainly due to the <strong>opacity</strong> of most of these Machine Learning algorithms. They, therefore, lead more and more researchers, developers, companies, and also users of these tools to ask themselves <strong>questions about the interpretability and explainability</strong> of these algorithms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QmV-MeBuYRH8EmLX" /><figcaption>Figure 1 — DARPA’s “Explainable AI” project. Image from Turek [2017].</figcaption></figure><h3>From the black box to explainable AI</h3><p><em>“The main difference between the AI of the 1970s and that of today is that it is no longer a purely deterministic approach” [Vérine and Mir, 2019].</em></p><p>Indeed, the implementation of AI algorithms has evolved significantly in recent years. Previously based mostly on the developers’ choices, the arrangement of formulas and mathematical equations used, these algorithms and their decision-making were strongly influenced by the developers, their culture, and their own way of thinking. Now, these rules are much more <strong>derived from the “hidden” properties in the data</strong> from which the algorithms will learn, i.e. adjust their parameters. Of course, the implementation will still be influenced by the developer, but the algorithm’s decision-making will not be as much influenced. As a result, <strong>automated decision-making by these algorithms has become opaque</strong>.</p><p>Thus, today, if a neural network is assimilated to a “black box” it is because the input and output data are known but its internal functioning, specific after learning, is not precisely known. When we study the deep hidden layers of a network (represented in figure 2 by the darkest frame) we observe that <strong>the internal representation of the data is very abstract</strong>, and therefore complex to decipher, as are the implicit rules encoded during learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/0*vT_OvkDeeaTMwdbK" /><figcaption><em>Figure 2 — Schematic representation of a neural network according to the “black box” principle</em></figcaption></figure><p>This phenomenon is notably due to the approach that many researchers and developers in AI have had for years: the performance of these algorithms has been put at the center of concern to the neglect of their understanding.</p><p>Let’s also underline another aspect involved in this “black box” phenomenon: the increasing volume of data! Indeed, the volume of data used has exploded in recent years, which was not the case at the beginning of AI. The now gigantic size of the data taken into account by algorithms hinders, if not prevents, the analysis and understanding of their behavior by a human brain. For example, if a person can analyze one or ten receipts simultaneously, it quickly becomes impossible for them to look for similarities in more than a hundred of them to understand and predict the behavior of consumers in a supermarket. This is even more true when thousands or millions of receipts are processed in parallel. This makes it difficult to understand the decisions resulting from the analysis and predictions of these algorithms.</p><p>If the demystification of the “black boxes” represented by neural networks has become a major topic in recent years, it is due, among other things, to the fact that these tools are increasingly used in critical sectors such as medicine or finance. It is therefore essential to understand the criteria taken into account behind their decision proposals in order to limit as much as possible, for example, the moral and ethical biases present in these proposals.</p><p>Precisely, the decision proposals resulting from AI algorithms raise crucial and general questions, notably in terms of the acceptability of these tools. <strong>The need for trust and transparency is thus very present and highly valued</strong>. There is therefore today a real <strong>reorientation of the problems</strong> linked to Machine Learning, particularly <strong>the need to explain them</strong> [Crawford, 2019].</p><p>Indeed, how can we trust <strong>the decision proposal of an AI algorithm</strong> if we cannot explain where it comes from? The entry into force of the <strong>GDPR</strong> (in Europe), particularly <a href="https://web.archive.org/web/20210225043640/https://www.gdpr-expert.eu/article.html?id=22#textesofficiels">Article 22–1</a> stipulating that a decision cannot be based exclusively on automated processing, added to the fact that a human needs explicit and non-opaque elements to make a decision, has greatly accelerated research in this area.</p><h3>Interpretability or explainability?</h3><p>The need for transparency and confidence in Machine Learning algorithms (e.g. neural networks or reinforcement learning mechanisms) has led to the emergence of two concepts: interpretability and explicability. Often associated, it seems important to clarify that these are two different concepts: let’s define them!</p><p><strong>Interpretability</strong> consists in providing information representing both the reasoning of the Machine Learning algorithm and the internal representation of the data in a format that can be interpreted by an ML expert or data. The result provided is strongly linked to the data used and requires knowledge of the data but also of the model [Gilpin et al., 2018].</p><p><strong>Explainability</strong>, on the other hand, consists in providing information in a complete semantic format that is self-sufficient and accessible to a user, whether he or she is a neophyte or a technophile, regardless of his or her expertise in Machine Learning [Gilpin et al., 2018]. For example, the job of a researcher will not be explained in the same way to high school students as to computer science students. The jargon used is adapted to concepts shared by those issuing the explanation and those receiving it.</p><p>In other words, interpretability answers the question “how” does an algorithm make a decision (what calculations? what internal data? …) while explicability tends to answer the question “why” (what links with the problem at hand? what relations with the application elements? …). Note that interpretability is the first step to be taken in order to achieve explainability. An explainable model is therefore interpretable, but the opposite is not automatically true!</p><p>This also explains why many people are working on <strong>the interpretability of artificial neural networks</strong>. A vast and exciting field of research, which everyone can approach through their own prism, it has seen a veritable explosion in the number of works and publications since the end of 2017 with, in particular, the <a href="https://web.archive.org/web/20210225043640/https://www.darpa.mil/program/explainable-artificial-intelligence">Explainable AI (XAI)</a> project of DARPA, the U.S. Department of Defense’s research and development agency (Figure 1).</p><p>Interpretability and, by extension, explainability, are therefore fast-growing fields in which many current questions are still far from being resolved!</p><p><strong>What to remember?</strong></p><p>Machine Learning, Deep Learning, and therefore neural networks, which are the subfields of AI research, are nowadays of great interest because their impact on our lives is so important. In parallel to improving accuracy and performance, the AI community needs to understand the internal logic and therefore the causes of the decisions made by these algorithms. But how can we be sure of the relevance of a neural network prediction if the reasons behind it are obscure?</p><p>Moreover, can we explain or interpret everything? What about decisions coming from very complex contexts requiring large volumes of data where the set of elements is far too vast to be assimilated by a human brain? These open questions are precisely the objective of the research work that tends towards a human understanding of neural networks;</p><p>It is therefore with the aim of demystifying these “black boxes” that researchers, companies, and users are increasingly interested in the field of interpretability, i.e. the understanding of the internal logic of neural networks, and that of explicability, i.e. the ability to explain the reasons behind a prediction of these same networks;</p><p>We will see in the rest of this series of three articles the questions that need to be answered in order to choose between two interpretability approaches to be used according to the use case but also how they help to make AI more transparent and understandable.</p><p><strong>To cite this article:</strong></p><p>Lhuillier M and Chraibi Kaadoud I, Interpretability vs explainability: understanding vs explaining one’s neural network (1/3)? Publication on the blog of <a href="http://www.scilogs.fr/intelligence-mecanique,">http://www.scilogs.fr/intelligence-mecanique,</a> August 2020</p><h3>References :</h3><ul><li>Crawford Kate (2019) « Les biais sont devenus le matériel brut de l’IA ». URL : <a href="https://www.lemonde.fr/blog/internetactu/2019/10/03/kate-crawford-les-biais-sont-devenus-le-materiel-brut-de-lia/">https://www.lemonde.fr/blog/internetactu/2019/10/03/kate-crawford-les-biais-sont-devenus-le-materiel-brut-de-lia/</a></li><li>Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., &amp; Kagal, L. (2018). Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA) (pp. 80–89). IEEE.</li><li>Helbing, D. (2019). Societal, economic, ethical, and legal challenges of the digital revolution: from big data to deep learning, artificial intelligence, and manipulative technologies. In Towards Digital Enlightenment (pp. 47–72). Springer, Cham.</li><li>Turek, M. (2017). DARPA — Explainable Artiﬁcial Intelligence (XAI) Program,URL: https://www.darpa.mil/program/explainable-artiﬁcial-intelligence</li><li>Vérine, Alexandre et Mir, Stéphan. (2019). L’interprétabilité du machine learning : quels défis à l’ère des processus de décision automatisés ? URL : <a href="https://www.wavestone.com/fr/insight/interpretabilite-machine-learning/">https://www.wavestone.com/fr/insight/interpretabilite-machine-learning/</a></li><li>WhatIs, 2016. Machine Learning definition. Publié sur le site WhatIs.com URL : <a href="https://whatis.techtarget.com/fr/definition/Machine-Learning">https://whatis.techtarget.com/fr/definition/Machine-Learning</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3084c9bb9eda" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cognition or what is cognitive science?]]></title>
            <link>https://medium.com/@ichraibik/cognition-or-what-is-cognitive-science-894e3e840e21?source=rss-c59511f426bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/894e3e840e21</guid>
            <category><![CDATA[cognition]]></category>
            <category><![CDATA[cognitive-computing]]></category>
            <category><![CDATA[cognitive-science]]></category>
            <category><![CDATA[human-machine-interaction]]></category>
            <dc:creator><![CDATA[Ikram Chraibi Kaadoud]]></dc:creator>
            <pubDate>Tue, 26 Jul 2022 07:06:04 GMT</pubDate>
            <atom:updated>2022-07-26T07:06:04.532Z</atom:updated>
            <content:encoded><![CDATA[<p>This article was originally published on the <a href="https://scilogs.fr/intelligence-mecanique/">Scilog blog</a> in french related to the magazine “Pour la Science”. Here is the English version, also available on my personal website: <a href="https://ikramchraibik.com/blog/">https://ikramchraibik.com/blog/</a></p><p><strong>Target audience: </strong>all audiences, high school students</p><p><strong>Keywords</strong>: <strong>Cognitive Science, Artificial Intelligence</strong></p><p><em>Short introduction:</em></p><p>This article is the result of a collaboration with <a href="https://web.archive.org/web/20210130014200/https://www.linkedin.com/in/alexandra-delmas-48200b90/"><em>Alexandra Delmas</em></a>, Ph.D. in Cognitive Sciences</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/503/0*SCrnSBG3cuLLNYCS.png" /></figure><p>How can a human memorize information? How does he manage to express himself, reason, solve a problem, and make a decision…? We do not realize it, but the human being evolves throughout his life via the intervention of mental functions which are used to process the information which surrounds us, we speak then of cognition (from the Latin cognitio, action of knowing, derived itself from cognoscere, to seek to know, to inquire). Cognition refers to the set of mental processes associated with knowledge and information processing. This current of thought, which emerged in the 1950s, replaced behaviorism, which stipulated that humans acted and functioned solely by reflex to a given <a href="https://web.archive.org/web/20210130014200/https://fr.wikipedia.org/wiki/Stimulus">stimulus</a>, or by the history of the individual’s interactions with his or her <a href="https://web.archive.org/web/20210130014200/https://fr.wikipedia.org/wiki/Environnement">environment</a>. In order to study in more detail the mechanisms of human, animal, or artificial thought, to understand its origins and its evolution in order to be able to model it at several levels, a scientific discipline was born: these are the cognitive sciences, and it involves Artificial Intelligence (AI).</p><p>Marvin Lee Minsky defined artificial intelligence as “<em>the construction of computer programs that perform tasks that are currently best done by humans because they require </em><strong><em>high-level mental processes </em></strong><em>such as perceptual learning, memory organization, and critical reasoning</em>”.</p><p>In other words, building or being interested in AI is first about <strong>understanding mental processes in humans</strong>. But what is a mental process? And how do we study humans?</p><p>Cognitive sciences are a neglected and unknown science, and they are here to try to answer these questions!</p><h3>Cognitive science: a science of sciences!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/0*VLrpiXEeTiH2PM89" /><figcaption>Figure A — Image from the french article <a href="https://web.archive.org/web/20210130014200/https://www.echosciences-grenoble.fr/communautes/le-magazine-des-sciences-de-rcf-isere/articles/quand-la-memoire-nous-fait-defaut">“When memory fails us?” by Mathilde Chasseriaud, 2017</a></figcaption></figure><p>Cognitive science is a multidisciplinary science in itself that lies at the junction of several other sciences :</p><ul><li><strong>Philosophy</strong>, allows us to address very abstract questions about human nature, the human mind, and the world in which we evolve.</li><li><strong>Linguistics</strong>, studies language, its emergence, and its impact on humans.</li><li><strong>Anthropology</strong> addresses the question of the human in all its aspects and the development of civilizations.</li><li><strong>Neuroscience</strong> deals with the question of brain development, the nervous system, and its biological functioning.</li><li><strong>Psychology</strong> is interested in mental processes, psychic facts, and psychological states.</li><li>Finally, <strong>computer science,</strong> makes it possible to artificially simulate the cognitive and/or biological functioning of the brain, which is called modeling!</li></ul><p>The goal of cognitive science is to describe, specify, explain and simulate cognitive processes, also called mental processes, which correspond to all processes of acquisition, processing, and use of information.</p><p>Behind this technical definition lies what we do every day in a spontaneous way: thinking by using our memory, concentrating on a task by focusing our attention, or simply discussing and communicating by using language!</p><p>You, dear reader, while reading these lines, are using your cognitive processes to analyze words, associate them together, extract a semantic and then analyze them! And for this, you use your word memory (semantic memory), your working memory, your reasoning, etc.</p><h3>The cognitive processes</h3><p>It seems important to remember that the brain, the seat of our cognitive processes, is a “machine” that functions permanently! During the sleep phases, it does not put itself to sleep, far from it, since it continues to analyze and process the information acquired during the day. In preparation for exams, you should not neglect your sleep, far from it!</p><p>It is by using all the cognitive processes, working in synergy (all together), that we succeed in carrying out intellectual or manual actions on a daily basis.</p><p>We will distinguish 5 cognitive processes: Memory, Attention, Language, visuospatial functions, and executive functions.</p><h3>Memory</h3><p>Great news, we do not have only one memory, but several! Behind the term “memory” lies a whole range of concepts. To mention only a few, we have for example episodic memory which allows us to record the episodes of our lives. This is what allows us to experience an event only once and to be able to remember it or to learn a list of telephone numbers. Procedural memory is the memory of reflexes and motor learning. In other words, it is what allows us to ride a bike and especially to do it automatically after several training episodes. In fact, the functioning of all these memories allows us to be what we are!</p><p>During the development of Alzheimer’s disease, a neurodegenerative disease (the neurons are destroyed), the hippocampus, an essential structure of episodic memory, is the first to be affected. This leads to an inability to acquire new memories and thus to keep in mind names, places, and events experienced after the onset of the disease.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/0*6vEP9lKZssc-5N1g" /><figcaption>Figure B — Different long-term memories. Image adapted from [1992]. Translation of terms: 1) Mémoire à long terme : Long-term memory, 2)Mémoire déclarative(explicite) : Declarative memory (explicite) 3) Mémoire non déclarative (implicite): Non-declarative memory (implicite) 4) Episodique/sémantique/Procédurale : Episodic/Semantic/Procedural</figcaption></figure><h3>Attention</h3><p>Without attention, it is difficult to focus on a goal long enough to achieve it. Attention is paramount for explicit learning (of which I am aware) of a phone number or a poem. We are not all equal when it comes to the ability to concentrate: some people can focus their attention for 30 minutes (or even hours) while others can only focus for 10 minutes. It is therefore important to know “ your cruising speed “ in order to adapt your learning methods.</p><p>Here is a well-known experiment that highlights selective attention :</p><p><a href="https://www.youtube.com/watch?v=vJG698U2Mvo&amp;ab_channel=DanielSimons">https://www.youtube.com/watch?v=vJG698U2Mvo&amp;ab_channel=DanielSimons</a></p><h3>Language</h3><p>We communicate every day, all the time with everyone. It is so obvious that it seems elementary, even simple as the rise of connected speakers (Alexa, Google home) and voice assistants (Siri, Cortana, Google now) would have us believe. However, this is not the case. Language comprehension is a complex domain in its own right. A very important cognitive process, Language is the basis of social interaction and involves two important parts: the transmission of a message in a format that the recipients can understand, and the reception and understanding of the message received.</p><p>The field of translation with Natural Language Processing (NLP) is precisely at the junction of the study of the cognitive process and its modeling in AI.</p><p>Some researchers go even further to understand language by looking at its genesis, or how children learn to speak without knowing the grammatical rules beforehand! Here is for example the interview of <a href="https://web.archive.org/web/20210130014200/https://www.inria.fr/europe-international/espace-europeen-de-la-recherche/laureats-2009-2010/pierre-yves-oudeyer">Pierre-Yves Oudeyer</a>, research director at Inria Bordeaux Sud-Ouest, who is interested in language in children and its modeling in robotics:</p><p><a href="https://www.youtube.com/embed/ZEf6oYfIR7I?feature=oembed">https://www.youtube.com/embed/ZEf6oYfIR7I?feature=oembed</a></p><h3>Visuospatial functions</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/267/0*H7eCiZONbZ8zC1h8" /><figcaption>Figure C — Spatial navigation (<a href="https://web.archive.org/web/20210130014200/https://www.flickr.com/photos/61423903@N06/7369580478">source</a>)</figcaption></figure><p>Have you ever closed your eyes to imagine a place? Mentally imagined a missing object? Memorized a route? If so, then you’ve engaged your visuospatial functions!</p><p>These functions allow you to orient yourself and move around in space while avoiding obstacles in order to reach a goal (the exit door for example). These functions not only allow navigation but also actively participate in thought processes, reasoning, object recognition, and text production.</p><h3>Executive functions</h3><p>When we are looking for a solution to a given problem, we need to analyze the problem and develop strategies and even scenarios. This is accompanied by making one or more decisions. If you agree with all this, you have used your executive functions! Located in the prefrontal cortex, these functions allow us to reason, make decisions, analyze our environment, interact with it and even inhibit our behavior when necessary. This set of cognitive processes is essential in everyday life! As our knowledge evolves over time, so does our reasoning. The experience of each person also has an impact on the rules that govern our reasoning.</p><h3>The alliance of AI and Cognitive Science: Why?</h3><p>So why are we interested in cognitive science when we are doing Artificial Intelligence? IBM asked the question in 2013.</p><p>Wanting increasingly adaptive and responsive AI that evolves as we progress as humans require an understanding of how we develop. By gathering insights into this we may be able to move toward increasingly intelligent tools that would then best assist us.</p><p>Another reason why the alliance between computer science and cognitive science is interesting is the accompaniment of a world population that is getting older and older: the life expectancy at the world level has increased, which implies a larger and larger elderly population. Unfortunately, this does not necessarily imply being in good health. In recent years, scientific advances have revealed the existence of neurodegenerative diseases such as Parkinson’s, Alzheimer’s, and senile dementia. In order to allow these people to remain as long as possible in their homes, where they have their memories and habits, it is important to study the factors that lead to the appearance of these diseases, their development, and their evolution. For this purpose, numerical simulations provide a more accessible study framework for scientists and physicians than asking patients to come several times a day to the laboratory or overloading them with physiological sensors. Children’s cognitive development is also the focus of researchers’ attention. Indeed, they are subjected to screens and new technologies at an increasingly early age, and the relationship between humans and machines is undoubtedly impacted!</p><p>The fields of AI and cognitive sciences are thus strongly linked. Studying the former often involves asking “But how do we do as humans?”. Whether it’s to develop smart tools or to help medicine, it’s important to understand what cognitive science is all about. Aside from the fact that it’s thrilling, it turns out that it also helps us<strong> better understand the world around us</strong>.</p><p><strong>To cite this article :</strong></p><p>Ikram Chraibi Kaadoud &amp; Alexandra Delmas. “Cognition or what is cognitive science?” English version of the publication “La cognition ou qu’est-ce que les sciences cognitives?” Published on the blog of <a href="http://www.scilogs.fr/intelligence-mecanique,">http://www.scilogs.fr/intelligence-mecanique,</a> December 2018</p><h3>References</h3><p>Squire, Larry R, 1992. Declarative and nondeclarative memory: Multiple brain systems supporting learning and memory. Journal of cognitive neuroscience, 4(3):232–243.</p><p>For more information on Attention:</p><ul><li><a href="https://web.archive.org/web/20210130014200/https://www.has-sante.fr/portail/jcms/c_2025618/fr/trouble-deficit-de-l-attention-avec-ou-sans-hyperactivite-tdah-reperer-la-souffrance-accompagner-l-enfant-et-la-famille-questions-/-reponses">Haute Autorité de Santé : Trouble déficit de l’attention avec ou sans hyperactivité (TDAH)</a></li><li><a href="https://web.archive.org/web/20210130014200/https://www.ameli.fr/gironde/assure/sante/themes/trouble-de-deficit-de-lattention/hyperactivite/soins-prescrits">sur amelie.fr, le site de la sécurité sociale (France)</a></li></ul><p>For more information on the visuo-spatial functions:</p><ul><li><a href="https://web.archive.org/web/20210130014200/http://www.happyneuron.fr/cerveau-et-entrainement/visuo-spatial">Happy neuron : Le traitement visuo-spatial</a></li><li><a href="https://web.archive.org/web/20210130014200/http://www.micheldenis.fr/wp-content/uploads/2013/07/1997-Langage-et-cognition-spatiale-txt.pdf">Avant-propos du livre “Langage et Cognition Spatiale” de 1997</a></li><li><a href="https://web.archive.org/web/20210130014200/https://www.dys-positif.fr/comprendre-le-trouble-visuo-spatial/">Comprendre le trouble visuo-spatial</a></li></ul><p>For more information on executive functions:</p><ul><li><a href="https://web.archive.org/web/20210130014200/http://www.cite-sciences.fr/fileadmin/fileadmin_CSI/fichiers/vous-etes/enseignant/Documents-pedagogiques/_documents/Expositions-permanentes/Cerveau/cerveau-dossierenseignant-fonctionexe.pdf">Médiation scientifique les savanturiers : les fonctions exécutiv</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=894e3e840e21" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>