<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Maixent Chenebaux on Medium]]></title>
        <description><![CDATA[Stories by Maixent Chenebaux on Medium]]></description>
        <link>https://medium.com/@mchenebaux?source=rss-9f7fba5cdbe0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*_AvudIQpDDvh1nIV01sP-Q.jpeg</url>
            <title>Stories by Maixent Chenebaux on Medium</title>
            <link>https://medium.com/@mchenebaux?source=rss-9f7fba5cdbe0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 05:57:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mchenebaux/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A novel text classification pipeline using Self-Attention and the Condenser layer]]></title>
            <link>https://medium.com/@mchenebaux/a-novel-text-classification-pipeline-using-self-attention-and-the-condenser-layer-9d1fddb0c2c4?source=rss-9f7fba5cdbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d1fddb0c2c4</guid>
            <category><![CDATA[bert]]></category>
            <category><![CDATA[probability]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[transformers]]></category>
            <dc:creator><![CDATA[Maixent Chenebaux]]></dc:creator>
            <pubDate>Wed, 15 Jun 2022 08:09:50 GMT</pubDate>
            <atom:updated>2022-09-27T09:37:10.593Z</atom:updated>
            <content:encoded><![CDATA[<p>Self-Attention is the main building block of the Transformer models that have recently taken over the field of NLP.</p><p>Intuitively, Self-Attention allows each token to have its own attention system: the embedding of a given word becomes a linear combination of the embeddings of the other words with which it “matches” in the sequence. For example, embeddings of words like <em>it</em>, <em>they</em> or <em>she</em> will become combinations of the various words they refer to in a sentence. This allows the neural network to better learn the relationships between the elements in the sequence, and is the key that make Transformers able to capture long-range dependencies.</p><p>Transformers are sequence-to-sequence models that take in <strong>n</strong> inputs and return <strong>n</strong> outputs. Using these models to classify text is often done by fine-tuning pretrained models, like BERT or GPT. A disadvantage of this network structure is that no document embedding is computed, which makes it difficult to classify texts of varying lengths. To bypass this, we usually use pooling strategies (average or max-over-time of the output vectors), or take the embedding of the special token like <strong>[CLS]</strong> as the whole document embedding. Another common approach is to compute a weighted-sum of the output vectors where weights are determined by an attention mechanism that scores each token in the sequence. The fixed-sized document vector can then be fed into a standard feed-forward neural network for text classification.</p><p>The problem with these pooling approaches is that a lot of information get lost in the aggregation, like positional information. But then, how is it possible to summarize or condense a sequence of variable length into a vector of fixed size?</p><h3>The characteristic function</h3><p>Let’s imagine for a moment that the input sequence is only composed of scalars. It is then possible to interpret the values of a sequence as realizations of a discrete random variable, where the unique elements of the sequence constitute the set of values that the random variable can take.</p><p><a href="https://en.wikipedia.org/wiki/Characteristic_function_(probability_theory)">The characteristic function</a> for a discrete random variable X is defined as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yd8L-4rlZKmBDnavTo4jdw.png" /></figure><p><strong>x</strong> being each possible value of the sequence and <strong>p_x</strong> the probability of occurrence of <strong>x</strong>.</p><p>Behind this intimidating formula lies an extremely useful function in probability theory. This complex function totally and uniquely determines the probability law it describes, hence its name. Knowing Φ is knowing everything there is to know about X. To be convinced of it, it is enough to notice that the successive derivatives of Φ applied to 0 give us a multiple of the different statistical moments of the random variable:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*F7U7pwOozWaQlU_mo2-uVA.png" /><figcaption>the characteristic function contains information about statistical moments</figcaption></figure><p>Intuitively, the characteristic function defines a curve in the complex plane which uniquely identifies a probability distribution.</p><p>Now you may ask yourselves: how do I feed the characteristic function whose domain is infinite into a neural network that accepts only finite tensors of numerical values? The answer is simple: sample the characteristic function for a fixed number of values of t, and concatenate all the real and imaginary parts.</p><h3>Putting it all together</h3><p>The Condenser layer does just that. For <strong>each dimension</strong> of its input vectors, it uses an attention mechanism that estimate the <strong>p_x</strong> of each observed value <strong>x</strong>, evaluate the characteristic function at <strong>k</strong> values, concatenate the real and the imaginary parts, then outputs a fixed sized vector that summarizes the whole sequence. Now, everything in this process is differentiable, hence the neural network learns the attention scores for each dimension independently, as well as the values of the sampling points that summarizes the input sequence.</p><p>You may notice that the order of the elements do not change the values of the characteristic function, but it is not a problem. If the Condenser layer is added after an RNN or a Self-Attention layer, it will still be able to summarize features faithfully, and hence recover positional information.</p><p>The last issue is the large size of the resulting vector : for input vectors of dimensionality <strong>d</strong> and <strong>k</strong> sampling values, the size of the embedding becomes <strong>2dk </strong>(2 because of the imaginary and the real part). Note that the size is independent of the sequence length. For an input dimensionality of 200 and 15 sampling points, the resulting vector has a size of 6000. Such a large size slows down training and can degrade the quality of convergence. The solution is to reduce the input dimensionality using the random projection technique. The core idea behind random projection is given in the <a href="https://en.wikipedia.org/wiki/Johnson-Lindenstrauss_lemma">Johnson-Lindenstrauss lemma</a> which states that if points in a vector space are of sufficiently high dimension, then they may be projected into a suitable lower-dimensional space in a way which approximately preserves the distances between the points. In the Condenser implementation, this projection is made by multiplying the resulting large vector with a <strong>non-trainable</strong> random matrix <strong>R</strong> generated using a gaussian distribution. This works very well in practice and dramatically reduces learning time.</p><h3>Hands-on example using the 20 newsgroups text dataset</h3><p>We will use the following packages: <a href="https://github.com/kerighan/convectors">convectors</a> for text preprocessing and <a href="https://github.com/CyberZHG/keras-self-attention">keras-self-attention</a> for the self-attention layer. Convectors is an easy-to-use library that allows to build NLP models by stacking layers like Lego bricks.</p><pre>pip install convectors<br>pip install keras-self-attention<br>pip install keras-condenser</pre><p>To view the example in one block, <a href="https://github.com/kerighan/condenser/blob/master/example.py">check this github page.</a></p><p>Let’s import the required objects, define global variables and fetch the training data:</p><pre>from convectors.layers import Lemmatize, Sequence, Tokenize<br>from keras_self_attention import SeqSelfAttention<br>from sklearn.datasets import fetch_20newsgroups<br>from tensorflow.keras.layers import Dense, Embedding, Input<br>from tensorflow.keras.models import Model<br>from condenser import Condenser</pre><pre>MAX_FEATURES = 100000<br>EMBEDDING_DIM = 400<br>MAXLEN = 800</pre><pre># get training data<br>newsgroups_train = fetch_20newsgroups(subset=&#39;train&#39;)<br>newsgroups_test = fetch_20newsgroups(subset=&#39;test&#39;)</pre><p>Create our text preprocessing pipeline:</p><pre>nlp = Tokenize()<br>nlp += Lemmatize(lang=&quot;en&quot;)<br>nlp += Sequence(maxlen=MAXLEN, max_features=MAX_FEATURES, pad=True)</pre><p>The Sequence layer, once fitted, will turn a list of strings into a padded sequence of integers transparently.</p><p>The first call of the <strong>nlp</strong> object will fit the Sequence layer and transform the input data. Subsequent calls will just vectorize the input texts.</p><pre># process train data<br>X_train = nlp(newsgroups_train.data)  # fits here<br>y_train = newsgroups_train.target<br># process test data<br>X_test = nlp(newsgroups_test.data)<br>y_test = newsgroups_test.target</pre><p>Now, let’s build our Keras model using the functional API and fit the model:</p><pre># get number of features<br>n_features = nlp[&quot;Sequence&quot;].n_features + 1</pre><pre># build model<br>inp = Input(shape=(MAXLEN,))<br>x = Embedding(n_features, EMBEDDING_DIM, mask_zero=True)(inp)<br>x = SeqSelfAttention(units=48, attention_width=10, kernel_regularizer=regularizers.l2(1e-6), attention_type=SeqSelfAttention.ATTENTION_TYPE_MUL)(x)<br>x = SeqSelfAttention(units=48, attention_width=10,   kernel_regularizer=regularizers.l2(1e-6), attention_type=SeqSelfAttention.ATTENTION_TYPE_MUL)(x)</pre><pre>x = Condenser(n_sample_points=10, reducer_dim=96)(x)<br>x = Dense(48, activation=&quot;tanh&quot;)(x)<br>out = Dense(20, activation=&quot;softmax&quot;)(x)</pre><pre># create and fit model<br>model = Model(inp, out)<br>model.compile(&quot;nadam&quot;, &quot;sparse_categorical_crossentropy&quot;, metrics=[&quot;accuracy&quot;])<br>model.summary()<br>model.fit(X_train, y_train, batch_size=50, epochs=5, validation_data=(X_test, y_test), shuffle=True)</pre><p>At the end of the training process, you will get a solid validation_accuracy above 87%. This model significantly outperforms any other common strategy such as average or max pooling, using a preprended <strong>[CLS]</strong> token as document embedding, or the usual weighted attention scheme.</p><p>On other datasets (R8, R52, Ohsumed), this method achieves state-of-the-art or comparable results. Benchmarks will soon be available on the git page.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d1fddb0c2c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Choosing the right number of clusters using the Gaussian Likelihood Score]]></title>
            <link>https://medium.com/@mchenebaux/choosing-the-right-number-of-clusters-using-the-gaussian-likelihood-score-30bce5ad6eac?source=rss-9f7fba5cdbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/30bce5ad6eac</guid>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[elbow-method]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Maixent Chenebaux]]></dc:creator>
            <pubDate>Mon, 20 May 2019 08:56:19 GMT</pubDate>
            <atom:updated>2019-05-20T08:56:19.408Z</atom:updated>
            <content:encoded><![CDATA[<h3>Silhouette analysis and the Davies-Bouldin Index don’t have to be the only options.</h3><p><em>You can find the package repo </em><a href="https://github.com/kerighan/gls"><em>here</em></a><em>.</em></p><p>In data science, clustering is a very common technique used to assign labels, or clusters, to a set of data points. Many algorithms exist for this purpose. K-means, Spectral Clustering, Gaussian Mixture Models or Agglomerative clustering are some that require the number of clusters as a parameter to operate. Some don’t, like DBSCAN or the more recent HDBSCAN, but the tradeoff is that you usually have to tweak some other parameters to get it just right. Sadly, there is no magic wand for such a task.</p><p>Luckily, some metrics have been proposed to infer the number of clusters in a dataset, like Silhouette Analysis, the Davies-Bouldin Index or the lesser known Calinski-Harabasz Index. Most of them follow the same principle: for a set of possible numbers of clusters, evaluate the clustering quality, then find the optimal number by choosing the highest / lowest score in the list, or by using the Elbow method. Of course, the “right” number of clusters is not something that can always be defined rigorously, and it depends a lot on the context and the dataset you’re working with. But sometimes you’re faced with a set of points in space that unambiguously scream for a certain “k”.</p><p>In such cases, one is often surprised to find out that a scoring method fail miserably at determining this number, when it looks so obvious on your scatter plot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*9OtLidIednbUiFqpl4_Vxw.png" /><figcaption>Why Silhouette, why ?</figcaption></figure><p>The reason is that these scoring methods don’t use a probabilistic generative model of your dataset, ie. they don’t know how the dataset was generated, which is not a bad thing in itself, as these scores were created to be useful in many diverse cases.</p><p>I found myself in a peculiar case when working on a speaker diarization system: the dimensionality of my voiced samples was reduced to 2 (yes, only 2) with UMAP, and the speakers could easily be inferred with the naked eye. I suspected the data to be (closely enough) normally distributed, so I decided to use this as prior knowledge, and came up with a scoring method I called the Gaussian Likelihood Score, or GLS.</p><p>The GLS can be thought out as (and actually is) the likelihood of observing the clustered points, given that they were generated by <em>k</em> normal distributions. The basic steps of my first draft were the following:</p><p>For each possible number of clusters:</p><ul><li>cluster your data points</li><li>for each cluster, compute the mean and standard deviation</li><li>for each dimension, bin the data points, and evaluate the log-probability of generating the data using the normal distribution with the computed mean and standard deviation</li><li>use the average of the log-probabilities as a score to evaluate clustering quality</li></ul><p>Now, some things should be mentioned here. First, the more clusters you use, the higher the score. In order to avoid choosing the highest value every time, we can use <strong>the elbow method </strong>to find the value of <em>k</em> after which the clustering quality improves only marginally. Second, computing the log-likelihood for each dimension independantly might not be the best idea. What if on each dimension, the data looks normally distributed ?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*uBf_WCAp8fJS627bRwnvjw.png" /><figcaption>we can clearly see two blobs on this plot, but not when we look at the distribution of the points for each dimension independantly</figcaption></figure><p>As this scatter plot clearly shows, it’s not because you can see two blobs in 2D, that they will be apparent on each axis independantly. In this case, our score would return a high value of the likelihood of a single cluster, as on each axis, a normal distribution with a high standard deviation could well explain the data. We need to find a way to make it more easy for our algorithm to distinguish clusters that only appear when looking at the big picture. In other words, we would like to find a new coordinate system on which this seperation would be sharper. Luckily, PCA can provide us with that. What PCA does is finding a new coordinate system that maximizes the variance along its axes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*_E3V-n43_N68UOBofX9Vbg.png" /><figcaption>Applying PCA make the separation more visible on the first axis</figcaption></figure><p>Using the distribution of points along the first axis, we now have a clearer picture of what’s happening: this bimodal distribution sure cannot come from a single normal distribution. Now, our processing pipe looks like this:</p><p>For each possible number of clusters:</p><ul><li>cluster your data points</li><li>for each sets of points belonging to a cluster, reduce its dimension using PCA.</li><li>for each cluster, compute the mean and standard deviation</li><li>for each dimension, bin the data points, and evaluate the log-probability of generating the data using the normal distribution with the computed mean and standard deviation</li><li>use the average log-probability as a score to evaluate clustering quality</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*79SYrIYrbeW5rI7itLIa1A.png" /><figcaption>Inferring the number of clusters using Silhouette</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*36P24yn8_jWxr3g6ugoDZQ.png" /><figcaption>Inferring the number of clusters using the GLS</figcaption></figure><h3>Now, what results ?</h3><p>Using scikit-learn <em>make_blobs</em> function, I was able to evaluate the accuracy of GLS (with PCA) compared to DBI or Silhouette. It outperformed silhouette by +31% and DBI by +48.5%, sometimes successfully guessing the right number of clusters when it was not always obvious on the scatter plot, or when the standard deviation varied greatly accross clusters.</p><p>Of course it may sound like a self-fulfilling prophecy: we used normal distributions to generate the data, so it comes as no surprise that evaluating the likelihood of observing the data using gaussian distributions would get some good results. Of course, this method won’t work on highly non-normal data, but so wouldn’t DBI or Silhouette. Every tool has its scope and domain of maximum efficiency.</p><p>Visit the <a href="https://github.com/kerighan/gls">github page</a> for more information and installation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=30bce5ad6eac" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Données personnelles : inquiets, mais peu enclins à changer nos pratiques]]></title>
            <link>https://medium.com/reputation-squad/donn%C3%A9es-personnelles-inquiets-mais-peu-enclins-%C3%A0-changer-nos-pratiques-fb726c1ace68?source=rss-9f7fba5cdbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/fb726c1ace68</guid>
            <category><![CDATA[réseaux-sociaux]]></category>
            <category><![CDATA[rgpd]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[maif]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Maixent Chenebaux]]></dc:creator>
            <pubDate>Fri, 26 Jan 2018 16:37:46 GMT</pubDate>
            <atom:updated>2018-01-26T16:37:46.621Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*y4Iz2TZUC0djCQU8Hbm9lQ.png" /></figure><p>L’apparition du <strong>World Wide Web</strong> en 1990 a révolutionné notre façon de communiquer. Peuplé de forums et de chatrooms en tout genre, le web est alors anonyme : les interactions s’y font dissimulées derrière un pseudo. Exception faite des fournisseurs d’accès à Internet, il n’y a alors aucun acteur du web capable d’identifier la navigation d’un internaute d’un site à un autre, ni d’identifier formellement la personne devant l’écran. Cette ère d’anonymat relatif s’étend des années 90 jusqu’au début du second millénaire.</p><p>Tout change en 2004, lorsque <strong>Mark Zuckerberg</strong> et quelques amis développent Facebook. Conjointement au succès du réseau social viendront les premières craintes de trafic de données intimes. Ces craintes, on les trouve notamment incarnées dans le réseau <strong>Tor</strong>, inventé en 2001, à l’origine de la popularisation du terme “darknet”. Si aujourd’hui Tor fait surtout parler de lui pour ses supermarchés illégaux, il est brandi par des militants comme un moyen de résister à la vente de ses données personnelles par les <strong>GAFA</strong> et à la surveillance généralisée, telle que dévoilée par <strong>Edward Snowden</strong>. Nous sommes à l’aube de 2018. Comment les Français réagissent à cette nouvelle ère de collecte massive d’informations ?</p><p>Place de la République, Paris. Nous interrogeons les passants et constatons immédiatement que la notion de donnée personnelle est méconnue. Une carte bancaire enregistrée sur un site e-commerce est-elle une donnée personnelle ? Et votre adresse IP ? Une photographie de vous ? Soyons fous : votre style d’écriture manuscrite ou digitale est-il une donnée personnelle ?</p><p>Cette dernière question n’est pas aussi absurde qu’on le pense : en se basant sur des algorithmes stylométriques, la NSA aurait identifié le créateur du Bitcoin. En d’autres termes, l’agence de renseignement américaine a analysé le style d’écriture de ses diverses interventions sur internet et l’a comparé avec les milliards de conversations qu’elle a collectées, afin d’établir formellement son identité. Si l’on se réfère à l’article 2 de la loi “<strong>Informatique et libertés</strong>” : Constitue une donnée à caractère personnel toute information relative à une personne physique identifiée ou qui peut être identifiée, directement ou indirectement, le style d’écriture d’un commentaire laissé anonymement sur un forum peut alors être considéré comme une donnée personnelle.</p><h3>La pilule rouge ou la pilule bleue</h3><p>Nous vouons une confiance aveugle aux grands acteurs du numérique. Nous avons pris l’habitude d’utiliser un Internet où de nombreux services sont gratuits sans contrepartie apparente. A l’instar de Neo dans <strong>Matrix</strong>, nous faisons face à deux options. La première, avaler la pilule bleue : céder nos données aux géants du web sans se soucier de leur usage ni de leur détenteur. En échange, nous bénéficions de services de qualité et en apparence gratuits. La seconde option : choisir la pilule rouge, lever le voile sur les usages et le parcours de nos données. La <strong>MAIF</strong>, dans sa volonté d’être un acteur responsable de cette ère, a commencé le déploiement de son dispositif “<strong>Mes Datas et Moi</strong>” il y a deux ans et demi, en collaboration avec <strong>Reputation Squad</strong>.</p><p>A travers cette initiative, nous avons récolté 350 000 données statistiques issues de questionnaires qui mesurent, depuis octobre 2015, les niveaux de connaissance et les pratiques des Français en matière d’e-réputation, de gestion de leurs données personnelles et de maîtrise des risques numériques. Comment les Français réagissent-ils à ces enjeux ? Nous pouvons vous l’affirmer d’emblée : ils sont pessimistes dans leurs avis, paradoxaux dans leurs usages.</p><h3>Nous ne sommes qu’en partie lucides sur nos pratiques numériques</h3><p>Si l’on croit les chiffres du Baromètre du numérique 2017 de l’ARCEP, trois Français sur quatre possèdent un smartphone (contre moins d’un tiers il y a 5 ans !). Chaque jour, nous passons 1h20 sur les réseaux sociaux et publions des statuts pour nos quelques 177 amis <strong>Facebook </strong>(<a href="https://www.mesdatasetmoi-observatoire.fr/article/donnees-personnelles-nous-sommes-inquiets-mais-peu-enclins-a-changer#anchor1">1</a>), qu’une large proportion d’entre nous (39%) admet ne connaître qu’en partie. Les frontières entre vie personnelle et numérique s’effacent. Mais à quel point ? Pour y voir plus clair, nous avons analysé les horaires de publication de 3,6 millions de commentaires Facebook (anonymisés), sur les 100 plus grandes pages Facebook de médias français.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/0*lZGQ7_tpnGk9i_rO.JPG" /><figcaption><em>Les valeurs bleues foncées correspondent à un volume de commentaires plus important.</em></figcaption></figure><p>52% des commentaires sont publiés pendant les horaires de travail usuels (9h-12h et 14h-18h), 8,9% des commentaires sont publiés la nuit (minuit à 6h) et 8,5% le sont en matinée (7h-9h). Il est clair que nos journées sont de plus en plus ponctuées de likes, de commentaires et de tweets en tout genre. Comme le déplore<strong> Tristan Harris</strong>, ancien ingénieur de <strong>Google</strong> et auteur de l’essai “Comment la technologie pirate l’esprit des gens” : “Nous sommes interrompus en moyenne toutes les 15 minutes par une notification, un mail ou un message sur Messenger.” Notre double numérique s’est immiscé progressivement dans nos jours et nos nuits.</p><h3>Nous n’avons pas conscience des données que nous produisons</h3><p>L’enquête menée par le dispositif <strong>Mes Datas et Moi </strong>a porté un intérêt tout particulier à l’usage que font les Français des réseaux sociaux. Après tout, c’est sur ces plateformes que nous produisons une majorité de nos contenus : photos et vidéos de vacances, pensées diverses, diatribes et débats enflammés. Dans le cadre de notre étude, nous avons demandé aux internautes d’estimer le nombre de statuts Facebook qu’ils avaient rédigés au cours du mois précédent. Quatre réponses leur étaient proposées : le nombre exact, 2 fois moins, 2 fois plus et 3 fois plus. Résultat : un internaute sur deux sous-estime de moitié le nombre de ses publications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CufAa3toMECtRKSK.jpg" /></figure><p>Et si nous oublions certains des statuts nous avons écrits, il en est de même pour leur contenu : notre enquête révèle qu’un tweet sur cinq n’est pas reconnu par son auteur, et qu’une fois sur quatre, nous retweetons sans avoir pris la peine de consulter le lien inclus dans le message initial. Toutefois, les réseaux sociaux ne sont pas les seuls témoins de cette anesthésie collective. Si nous sommes 58% à utiliser la géolocalisation pour nous déplacer à pied, 44% d’entre nous ignorent qu’elle reste active quand nous ne l’utilisons pas ; autrement dit : qu’une application à qui nous aurions cédé les <strong>droits GPS</strong> de notre téléphone pourrait collecter et surveiller tous nos trajets. Un constat intéressant dans la mesure où une vaste majorité d’entre nous (84%) prétend ne divulguer que très peu d’informations personnelles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8UPuYOYU9aS53sKV.jpg" /></figure><h3>L’avenir de nos données intimes nous préoccupe</h3><p>Préoccupés, les Français le sont. Les trois quarts des sondés nous ont confié être inquiets quant à la sécurité de leurs données personnelles, une inquiétude qui croît avec l’âge : si 66% des 0–18 ans sont plutôt pessimistes à cet égard, ce chiffre atteint 83% des 61–75 ans. Facebook, réputé pour regorger de données intimes, est une source d’appréhension plus forte encore : ils sont 87% à redouter que l’entreprise ait accès à leurs données personnelles. Les sociétés privées américaines ne sont pas les seuls réceptacles de cette défiance : l’Etat français l’est tout autant. Seuls 2% d’entre nous lui confieraient leurs données de santé. Sans doute ces sondés oublient-ils leur immatriculation à la Sécurité Sociale.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ur9u8jtOBQ5uR3LD.jpg" /></figure><h3>Notre prise de conscience ne se ressent pas dans nos usages</h3><p>Le pessimisme généralisé aurait pu entraîner une révolution dans les usages. La prise de conscience est toutefois plus théorique que pratique. Un internaute sur deux déclare n’avoir jamais lu les Conditions Générales d’Utilisation des services qu’il emploie. Et pour cause ! Il y a en moyenne 6508 mots dans les <strong>CGU </strong>de Facebook, Twitter, YouTube, Instagram, Pinterest et Reddit. Par ailleurs, 10% des 0–18 ans assument ne rien mettre en oeuvre pour protéger leurs données personnelles. Ces derniers sont 80% à conserver une application qui ferait une collecte abusive de leurs données. Parmi les 19–30 ans, même constat : 1 sondé sur 5 utilise déjà un tracker d’activité, et 68% accepteraient de partager leurs données de santé à leur médecin traitant ou leur entourage proche. Nous sommes inquiets, mais pas prêts à renoncer à ces services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OO6XwuTK1KkhiOJP.jpg" /></figure><p>En dépit d’une mise en vigueur imminente (mai 2018), l’initiative européenne de protection des données à caractère personnel reste peu connue du grand public. Pour preuve : nous avons échantillonné et analysé les biographies d’utilisateurs Twitter ayant fait mention du “<strong>RGPD</strong>”. Nous y trouvons juristes, avocats, experts comptables, experts du numériques, entrepreneurs, journalistes et développeurs. Mais peu, voire pas de profils non spécialistes.</p><h3>Epilogue</h3><p>Il est évident que nombre des résultats de notre enquête confirment nos présomptions sur le rapport des Français au numérique. Si l’inquiétude est partagée par une large majorité, la prise de conscience n’a pas encore fait évoluer les pratiques. Face à cet état de fait, la démocratisation d’un savoir trop souvent technique et élitiste est nécessaire. Le dispositif que nous avons déployé entend mettre en relation le grand public et les plus éminents experts du domaine afin d’accompagner les internautes vers un web plus éthique et plus respectueux des données personnelles.</p><p><strong>(Re)Découvrir Mes Datas et Moi :</strong> <a href="https://www.mesdatasetmoi.fr/">https://www.mesdatasetmoi.fr</a></p><p><strong>Explorer les données :</strong> <a href="https://www.mesdatasetmoi-observatoire.fr/explorer">https://www.mesdatasetmoi-observatoire.fr/explorer</a></p><p><em>(</em><a href="https://www.mesdatasetmoi-observatoire.fr/article/donnees-personnelles-nous-sommes-inquiets-mais-peu-enclins-a-changer#anchorlink1"><em>1</em></a><em>) Blog du modérateur, chiffres réseaux sociaux 2017 : </em><a href="https://www.blogdumoderateur.com/chiffres-reseaux-sociaux/"><em>https://www.blogdumoderateur.com/chiffres-reseaux-sociaux/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb726c1ace68" width="1" height="1" alt=""><hr><p><a href="https://medium.com/reputation-squad/donn%C3%A9es-personnelles-inquiets-mais-peu-enclins-%C3%A0-changer-nos-pratiques-fb726c1ace68">Données personnelles : inquiets, mais peu enclins à changer nos pratiques</a> was originally published in <a href="https://medium.com/reputation-squad">Reputation Squad</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Virtuelle ou augmentée : la réalité se cherche ailleurs]]></title>
            <link>https://medium.com/reputation-squad/virtuelle-ou-augment%C3%A9e-la-r%C3%A9alit%C3%A9-se-cherche-ailleurs-58dc0ae926e?source=rss-9f7fba5cdbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/58dc0ae926e</guid>
            <category><![CDATA[virtual-reality]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[google-daydream]]></category>
            <category><![CDATA[oculus-rift]]></category>
            <dc:creator><![CDATA[Maixent Chenebaux]]></dc:creator>
            <pubDate>Tue, 29 Nov 2016 13:54:27 GMT</pubDate>
            <atom:updated>2018-06-26T12:08:29.376Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BYjWBA8TCKwyV1bNGJY3eg.jpeg" /><figcaption>Illustration :<a href="https://www.behance.net/JuanCarlosOsorno"> Juan Carlos Osorno</a></figcaption></figure><h4>Mon ventre est noué, mes yeux écarquillés. Cramponné au siège de ma capsule, je crois pouvoir éviter les missiles en tourbillonnant des épaules. Cela ne change rien.</h4><p>Dans leur trajectoire chaotique, ils tamponnent mon pare-brise, pétaradent dans tous les sens, ondulent en rythme autour de mon vaisseau. Mon cœur fait du tambour, mes yeux sont à l’affût. Non sans effort, je parviens à m’extraire de ce bourbier, puis ma capsule volante poursuit sa course dans l’infini numérique. D’un seul geste, j’enlève mon casque. Je respire un grand coup. Retour au monde réel. C’était ma première expérience avec la réalité virtuelle. — Comment c’était ? demande une voix. — A couper le souffle.</p><blockquote>Jamais notre œil n’aura été aussi sollicité, happé, dopé, exalté.</blockquote><p>Fatigué ? Peut-être. La vue est sans aucun doute le sens le plus développé chez l’être humain : nous distinguons les couleurs, les formes et les reliefs, nous suivons et appréhendons les mouvements grâce à elle. Elle nous permet de communiquer avec nos semblables, détecter un mensonge dans les mouvements à peine perceptibles d’un visage, transmettre un large spectre d’émotions. Pas étonnant que la vue soit le sens le plus prometteur concernant les nouvelles technologies. Bref, voici le topo : nous sommes sur un champ de bataille. D’un côté de l’arène, la réalité augmentée. De l’autre, la réalité virtuelle.</p><p>La réalité augmentée est le mélange symbiotique du réel avec le virtuel : des images sont générées, projetées, intégrées dans notre réalité. Quant à la réalité virtuelle, c’est la pilule rouge. On descend avec le lapin blanc dans la matrice. Déconnexion complète de la réalité, immersion dans la 3D. Ces deux concepts, pourtant de nature différente, sont en guerre. Portés par différentes entreprises majeures ou renommées, les deux dispositifs sont coûteux et doivent encore faire leurs preuves sur un marché ultra compétitif. A l’heure où j’écris ces lignes, il n’est pas envisageable pour un particulier de disposer des deux technologies chez lui. L’avenir de la réalité augmentée et de la réalité virtuelle dépendra donc de l’issue de cet affrontement.</p><h3>Qui en sortira victorieux ?</h3><p>Côté smartphone, la première bataille a d’ores et déjà été remportée. Le vainqueur ? Sans contestation, la réalité augmentée. Si vous n’avez pas entendu parler de Pokémon Go, c’est que vous n’êtes pas sorti de chez vous depuis des mois. Des cohortes de dresseurs Pokémon s’agglutinent dans les parcs avec l’espoir non dissimulé d’attraper Pikachu.</p><p>Déjà téléchargé des millions de fois, Pokémon Go a fait s’envoler la capitalisation boursière de Nintendo : 14% supplémentaire à la Bourse de Tokyo, soit un total de 38 milliards d’euros (chiffres fin juillet 2016). Sony, autrefois en tête, a dû s’incliner devant la firme de Super Mario. Nintendo n’est pas seul à profiter de cet engouement, puisque McDonald’s a vu son titre s’envoler de 23% dans les premiers échanges boursiers. La raison ? Des figurines Pokémon dans les “Happy meals”.</p><h3>Il faut casquer</h3><p>Si Pokémon Go est disponible exclusivement sur les appareils mobiles, y jouer avec un casque de réalité augmentée serait une belle promesse, pas vrai ? CapitolaVR, une agence néerlandaise, a produit une petite vidéo pour présenter un prototype : Pokémon Go porté sur l’Hololens. L’Hololens ?</p><p>C’est le casque de réalité augmentée de Microsoft. Son prix (3 000 $ en pré-commande) est prohibitif et ne permettra pas une commercialisation grand public cette année, mais il s’agit du casque de réalité augmentée le plus prometteur de sa catégorie. Sa date de sortie officielle n’a toujours pas été communiquée, mais les pré-commandes sont ouvertes. Ses applications sont entre autres la navigation internet (avec un navigateur projeté sur un mur), la visioconférence (“Help me Obi Wan Kenobi, you’re my only hope”), la modélisation 3D, les jeux, le Home Cinéma, et la prise de photo et de vidéo.</p><blockquote>“Le toucher est le plus démystificateur de tous les sens, au contraire de la vue, qui est le plus magique.”<br>Mythologies, Roland Barthes</blockquote><p>Tout ça mélangé avec notre environnement. Côté casques de réalité virtuelle, Facebook et HTC se tirent la bourre avec leurs produits respectifs : l’Oculus Rift et l’HTC Vive. Malgré un lancement plutôt satisfaisant pour les deux entreprises, l’avenir du marché est difficile à prédire, notamment car ces deux casques nécessitent des machines trop coûteuses et gourmandes en ressources. Ces casques restent cependant d’excellentes attractions dans les salons et provoquent en général de vives réactions chez ceux qui l’essayent.</p><p>Comme pour clore le triptyque “thèse antithèse synthèse”, Google travaille prétendument sur un casque mêlant réalité virtuelle et réalité augmentée. Suite à l’échec de ses Google Glass, le géant californien aurait décidé de revoir sa copie, et de se positionner transversalement à ses concurrents. Il faudra attendre encore un peu avant d’en savoir plus sur ce nouvel appareil. Parallèlement, sa plateforme de réalité virtuelle pour mobile DayDream verra le jour courant automne 2016.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WkIkPy6e_DaCWOZk9vY8Sw.jpeg" /><figcaption>Illustration :<a href="https://www.behance.net/JuanCarlosOsorno"> Juan Carlos Osorno</a></figcaption></figure><h3>Pas que des jeux vidéo</h3><p>Il est légitime de se poser des questions sur l’impact qu’une telle technologie aurait sur notre position dans le réel. Pokémon Go a déjà été la source de nombreux accidents de la route, de braquages et de rackets. De plus, certaines barrières empêchent encore l’adoption de masse : le prix, d’une part, mais des effets physiologiques indésirables (nausées, désorientation) d’autre part.</p><p>Toutefois, des effets bénéfiques ont déjà été identifiés dans le domaine de l’éducation : en 2010, des chercheurs ont créé un système d’apprentissage de la chimie en réalité augmentée. Les étudiants ont obtenu des performances supérieures à l’apprentissage par ordinateur. Les perspectives technologiques, notamment avec les nouveaux casques AR/ VR, pourront améliorer ces résultats.</p><p>La réalité augmentée et virtuelle a donc de beaux jours devant elle, mais son succès ne sera pas forcément là où on l’attend : pour l’instant, les casques excellent dans des domaines tels que l’éducation et l’événementiel. Sur smartphone, la voie pour les jeux est grande ouverte depuis le succès de Pokémon Go. A quand un jeu avec Pikachu sur Oculus Rift ?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58dc0ae926e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/reputation-squad/virtuelle-ou-augment%C3%A9e-la-r%C3%A9alit%C3%A9-se-cherche-ailleurs-58dc0ae926e">Virtuelle ou augmentée : la réalité se cherche ailleurs</a> was originally published in <a href="https://medium.com/reputation-squad">Reputation Squad</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Debates — What words reveal about Trump and Clinton]]></title>
            <link>https://medium.com/reputation-squad/third-debate-what-words-reveal-about-trump-and-clinton-d28c9bddbfd5?source=rss-9f7fba5cdbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/d28c9bddbfd5</guid>
            <category><![CDATA[2016-election]]></category>
            <category><![CDATA[donald-trump]]></category>
            <category><![CDATA[hillary-clinton]]></category>
            <category><![CDATA[politics]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Maixent Chenebaux]]></dc:creator>
            <pubDate>Fri, 21 Oct 2016 07:43:59 GMT</pubDate>
            <atom:updated>2018-06-26T10:31:58.573Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yd_3kZzu6i83kDQSuqRmQA.jpeg" /></figure><h4>The third Presidential debate in Las Vegas was the last chance to frontally attack, interrupt and sometimes insult, the other candidate.</h4><p>This is what our data analysis reveals about the final match between Clinton and Trump.</p><h3><strong>Did he prepare the debate ?</strong></h3><p>Before the first debate, Trump stated he would not prepare his TV interventions. After almost all headlines claimed he had lost the first debate, Trump might have changed his mind. At least, this is what our figures suggest : at every debate, Trump consistently slowed down his tempo. We counted 700 sentences in the first debate, 624 in the second, and 590 in the third. Moreover, his speech rate dropped by 29%. Despite the claims that “<a href="http://uk.businessinsider.com/snl-second-presidential-debate-2016-10?r=US&amp;IR=T">the second debate” was “the worst</a>”, it was during this confrontation that Trump displayed the widest vocabulary (902 different stemmed words, compared to 770 in the final debate) and repeated himself the least. What now?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*UTrBptjTHnZXPlZfAWX71A.jpeg" /></figure><p>Let’s dive deeper. Parts of speech are lexical categories such as nouns, verbs, adjectives, adverbs, etc. We tagged the last debate transcript with these, allowing us to have more detailed features of the two orators’ styles. Let’s start with an example : how do Clinton and Trump refer to each other, and do they use their last names or personal pronouns (colder, in our opinion)?</p><p>During the first debate, Trump referred to Clinton as “Secretary Clinton” (28 times), but he stopped using this expression during the second confrontation, mostly using “she” (from 46 to 103 occurrences in the second debate). He then went back to “Clinton” for the final debate. Is it a sign of moderation ?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*DAcOgE40JzOfJny1oppzOw.jpeg" /><figcaption>Throughout the three debates : she, she, she, Secretary Clinton, she, Clinton…</figcaption></figure><p>Clinton usually referred to Trump with “Donald” (30 times on average for each debate), but during the final, she adopted the pronoun “he” (+80%). Something worth noting regarding personal pronouns: in the first debate, Trump used ‘I’ 293 times, down to 151 and 161 in the other debates. Symmetrically, Clinton used ‘I’ increasingly, from 160 times in the first to 194 in the last.</p><p><strong>Protocol vs TV</strong></p><p>Mark Twain once wrote: “A man’s character may be learned from the adjectives which he habitually uses in conversation”. And it seems to be true if we consider the adjectives used by the candidates. By computing the <a href="https://en.wikipedia.org/wiki/Odds_ratio">odds ratio</a> of these words, we get the “Trumpian” and the “Clintonian” adjectives. It allows us to find the words that occur the most in one candidate’s speech and the least in the opponent’s. Trumpian words (odds ratio positive) are monosyllabic for 62.5% of them, and seem to carry “emotional” polarity: bad (18 vs 0), tremendous (8 vs 0), strong (7 vs 0), great (17 vs 8), sad (4 vs 0), greatest (4 vs 0).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*OBU5NT9byvJQtH1GfjEAmw.jpeg" /></figure><p>We do not find this polarity in the Clintonian adjectives (odds ratio negative). They reveal the nature of the american public administrator she is: clear (12 vs 0), american (12 vs 3), national (6 vs 1), important (6 vs 0), wealthy (5 vs 0), federal (5 vs 0), able (5 vs 0), sure (6 vs 1).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*y8DRImSNaMG57jvw7tBAGg.jpeg" /></figure><p><strong>Las Vegas, the last chance</strong></p><p>The final debate was also the last chance to attack and interrupt. During the first one, Trump interrupted Clinton 55 times, he calmed down during the second debate (14 interruptions) but finally couldn’t resist, and interrupted Clinton 37 times in Las Vegas. Clinton remained quite constant throughout the three debates, both in terms of interruptions (9 / 4 / 8) and vocabulary (1016 different stemmed words on average).</p><h3>Trumps dreams of the red phone</h3><p>By automatically recognizing international references in the last debate, we see that Trump makes many more references to international matters than Clinton, using words like Mosul (17 vs 6) , Iran(ians) (13 vs 1), ISIS (10 vs 6), Russia(ns) (11 vs 6), Iraq (9 vs 5), Nato (3 vs 1). Whether it is a good thing or bad thing, it shows that he insisted on naming these locations, something Clinton avoided.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*LoCadGfzU3L_dSLNOEZPbg.jpeg" /></figure><h3><strong>Nasty woman, Mexican hombres</strong></h3><p>Even if pronounced only once, the word that marked the last debate before the elections is “nasty”. Trump called Clinton “a nasty woman” and considering his past declarations about women, the public responded quite negatively to this statement: on October the 19th, 237k tweets #ImwithHer were sent, a boost of +230% from the day before and #nastywoman became a trending topic. The second unexpected word used by Trump was the “hombres” (men in Spanish) talking about Mexican migrants. The trending topic #BadHombresAMovie also emerged. Women and immigration have always been two very controversial topics for Trump who decided to stress the accent on his position, once again. With “nasty” and “hombre”, Trump gets all the press. In the meantime, Clinton might get the votes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*9y6nfsrzcndCWqAxzdebww.jpeg" /></figure><p><em>Co-written with </em><a href="https://twitter.com/giuliozucchini"><em>Giulio Zucchini</em></a></p><p><em>Illustrations : </em><a href="https://twitter.com/fannette_"><em>Fanny Algeyer</em></a><em>.</em></p><p><a href="https://twitter.com/reputationsquad">@ReputationSquad</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d28c9bddbfd5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/reputation-squad/third-debate-what-words-reveal-about-trump-and-clinton-d28c9bddbfd5">Debates — What words reveal about Trump and Clinton</a> was originally published in <a href="https://medium.com/reputation-squad">Reputation Squad</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Semantics — What does data science reveal about Clinton and Trump ?]]></title>
            <link>https://medium.com/reputation-squad/semantics-what-does-data-science-reveal-about-clinton-and-trump-afdf427e833b?source=rss-9f7fba5cdbe0------2</link>
            <guid isPermaLink="false">https://medium.com/p/afdf427e833b</guid>
            <category><![CDATA[politics]]></category>
            <category><![CDATA[hillary-clinton]]></category>
            <category><![CDATA[2016-election]]></category>
            <category><![CDATA[donald-trump]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Maixent Chenebaux]]></dc:creator>
            <pubDate>Wed, 05 Oct 2016 13:05:27 GMT</pubDate>
            <atom:updated>2018-06-26T10:30:02.517Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z-RXp3t_vyuXKI91A04meg.jpeg" /></figure><h4>Data science has many fields of applications. From image processing to AI, its use is ubiquitous. One of its applications, semantic analysis, is very helpful <a href="http://veille.reputationsquad.com/">in social media monitoring</a>. Here, we did not focus on tweets or Facebook comments, but on politics.</h4><p>On the 21st of July, Donald Trump accepted the Republican Nomination for President of the US on the last day of the Republican National Convention (RNC) in Cleveland, Ohio. One week later, on the 28th, Hillary Clinton was accepting the Democratic Party’s nomination for president in Philadelphia.</p><p>Supported by their family and hundreds of thousands of fans, they wrote a new page of the history of the United States delivering their acceptance speech. We’ve analysed their words to better understand the hidden components of their political communication. This study focuses on three main features : vocabulary, style and rhythm.</p><h3>You have my word</h3><p>One way to evaluate who has <strong>the largest vocabulary</strong> is to see how many unique words the speakers use. In order to do that, we need to remove some of the most common words used in the English language (“the”, “a”, “of”, …) as they don’t bear any informative value. These words are called stopwords : a list of these words can be found <a href="http://www.ranks.nl/stopwords">here</a>. Secondly, we don’t want to count words twice : “leaders” and “leader” must be viewed as equal, like “problems” and “problem” or “am” and “are”. For this, we used the <a href="http://snowball.tartarus.org/texts/introduction.html">Snowball Stemmer algorithm</a>.</p><p>We find out that Trump’s speech is made up of roughly 13% distinct root words (965 distinct stemmed words for a 7460 word-long text). Each word is, on average, repeated 7.7 times. On the other hand, Clinton’s has 17% distinct words, and each word is on average repeated ~6 times. The difference is significant : <strong>only 480 words are needed to write 80% of Donald Trump’s speech</strong>, while <strong>Clinton gets the same result using 665 words</strong>. That’s a 38% difference. Good, it means we are starting to see some results !</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*dbPsuWbUZrzL9vZLZIfjJg.jpeg" /><figcaption>How many words are needed to write 80% of a candidate’s speech ?</figcaption></figure><p>The efficiency of a speech relies partly on the style of its orator. In our case, we would like to find which are the candidates’ favorite words. To get the “Trumpian” and the “Clintonian” vocabularies, we have to find the words that occur the most in one candidate’s talk and, at the same time, the least in the opponent’s. For example, the word “really” is found 15 times in Trump’s speech but only once in Clinton’s. One way to determine this is to calculate the<a href="https://fr.wikipedia.org/wiki/Odds_ratio"> odds ratio</a> for each word. The odds ratio (here named OR) was, for each word, computed using the following formula :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*P9Xg4RHb5IrsDEuH0F4N1w.png" /></figure><p>The first term of the ratio is the probability of a word being in Trump’s vocabulary, and the other one is the probability of the same word being in Clinton’s. The log function allows us to efficiently sort each word in one category or the other : when the probabilities are equal, the log function is null. In any other cases, it is either negative (a word is Clintonian) or positive (a word is Trumpian). Here are the results we got :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*32LqA-P-uEIC0SdviTIUkQ.jpeg" /><figcaption>Words almost exclusive to Donald Trump</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*PVTIYTdI3cAC4Li9ATG5XQ.jpeg" /><figcaption>Words almost exclusive to Hillary Clinton</figcaption></figure><p>The first thing we notice is that Trump uses short and common words and tends to repeat them a great deal : “really”, “nice”, “great”, “problems”. On a side note, we can get a feeling of some preoccupations of the Republican candidate : “Mexico”, “China” and “Iran”. Globally, Trump’s concerns seem to be more focused on international “problems” than Clinton’s. Most of his mentions of the outside world aimed at inspiring fear and scape goating.</p><p>On Hillary’s side, the word range is wider. “Clintonian” words tend to be much rarer than Trump’s. Hillary Clinton actually refers to America a whole lot more than Trump : 27 occurrences (only 5 for Trump). The Clintonian wordset suggests that Clinton’s speech is more focused on national themes. Her typical words include “together”, “campaign” and “hard[working]”. Donald Trump is also mentioned many times in her speech.</p><p>Fine observers will note that “Trump” does not appear in the Clintonian wordset above, the reason being that Trump himself made numerous mentions of his last name in his speech (10 times), bringing the ratio way down. As a point of comparison, Clinton’s name is only used twice : once in Hillary’s speech (about her husband Bill Clinton), and once in Trump’s. Moreover, the Clintonian word “Wants” that shows up in the list is mostly used to criticize her opponent (“He wants to divide us […]”, “He wants us to fear the future and fear each other.”). It clearly shows that<strong> Clinton talked about Trump, and Trump talked about… himself !</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*FLnf9vYmTHu4MAeFod0H1A.jpeg" /><figcaption>Everyone talks about Trump.</figcaption></figure><p>We can also look at the words that both candidates use equally. They represent, in a way, their common concerns. Not surprisingly, this is the case for “job(s)”, “country” and “thinking”. They both use “thank(s)” numerous times, but in a different manner : while Clinton specifically thanked a group of people or an individual, Trump’s “thanks” were mostly employed when the crowd was applauding him.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*D0Ljinuj7Cm9jLdeYxK8jg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*H8PbIbG_Tkkn-2MHFSfQog.jpeg" /></figure><h3>What’s your tempo ?</h3><p>Each candidate has his own tempo, depending on her/his background as a speaker. A good start is to evaluate the inner rhythm of their language : we can slice each talk into a pack of sentences, and sentences into a pack of words. We discover that Trump has the longest speech, with 625 sentences and 7460 words, while Clinton’s has 405 sentences and 6088 distinct words. That being said, <strong>Trump’s talk has 54% more sentences than his opponent’s while being only 23% longer.</strong></p><p>The average sentence length of Trump is around 12 words per sentence. Clinton writes slightly longer sentences with an average of 15 words per sentence. Most of Trump’s sentences are short : more than 21% of Trump’s speech is made of sentences that contain 5 or 6 words. Clinton’s sentence lengths are more evenly distributed, <strong>12-word sentences being the most frequent</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*SwZLKTIUoSEh7EbOrjbc4Q.jpeg" /><figcaption>Obama employed as many words per sentence as Trump and Clinton combined</figcaption></figure><p>We see a clear difference between the two candidates : Trump’s speech is simpler and faster, Clinton’s more diverse and chilly. But wait ! She did not nail the conversation : Obama, during his first nomination speech, employed an average of <strong>25.7 words per sentence</strong>, which is almost equal to Clinton and Trump combined. Obama also repeated himself 24% less than Clinton, and 42% less than Trump. It shows, I think, that while Clinton’s tempo is a bit slower and the sentences a bit more complex in their structure, her speech rhythm is still very close to her opponent’s.</p><h3>Last word</h3><p>Natural Language Processing is not an exact science. It only gives us clues and elements by which to understand how speeches are delivered. The dataset is short, and further analysis is required to extract more precise features. But what did we learn from this analysis?</p><p>1. Trump thinks everything is “really” “great” and “nice”, and Clinton keeps talking about how to “work” “together” for “America”.</p><p>2. Trump talks about himself, Clinton talks about Trump. While Clinton uses a larger vocabulary and more complex sentence structures, it seems that she more or less <strong>adapted herself to Trump’s way of talking</strong>.</p><p>3. Obama’s nomination speeches (both of them) enjoyed a larger vocabulary and a significantly more intricate sentence structure suggests that <strong>Trump has disruptively simplified the national discourse</strong>.</p><p><em>Original artwork by </em><a href="https://twitter.com/fannette_"><em>Fanny Algeyer</em></a><em>, graphic and web designer @</em><a href="https://twitter.com/reputationsquad"><em>ReputationSquad</em></a></p><p><em>With the contribution of </em><a href="http://twitter.com/giuliozucchini"><em>Giulio Zucchini</em></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=afdf427e833b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/reputation-squad/semantics-what-does-data-science-reveal-about-clinton-and-trump-afdf427e833b">Semantics — What does data science reveal about Clinton and Trump ?</a> was originally published in <a href="https://medium.com/reputation-squad">Reputation Squad</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>