<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Alvin Lindstam on Medium]]></title>
        <description><![CDATA[Stories by Alvin Lindstam on Medium]]></description>
        <link>https://medium.com/@alvinlindstam?source=rss-f4c1d27db844------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*xL03mO-Icp3pqX7s.jpeg</url>
            <title>Stories by Alvin Lindstam on Medium</title>
            <link>https://medium.com/@alvinlindstam?source=rss-f4c1d27db844------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 03:08:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@alvinlindstam/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Ungdomsbostäder i Sthlm — har du nån chans?]]></title>
            <link>https://medium.com/@alvinlindstam/ungdomsbost%C3%A4der-i-sthlm-har-du-n%C3%A5n-chans-eller-ej-d6f270759946?source=rss-f4c1d27db844------2</link>
            <guid isPermaLink="false">https://medium.com/p/d6f270759946</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[stockholm]]></category>
            <category><![CDATA[housing]]></category>
            <dc:creator><![CDATA[Alvin Lindstam]]></dc:creator>
            <pubDate>Sat, 21 Apr 2018 11:11:42 GMT</pubDate>
            <atom:updated>2018-04-23T18:32:23.783Z</atom:updated>
            <content:encoded><![CDATA[<p>Bostadssituationen i Stockholm är djupt dysfunktionell på grund av bostadsbristen. Som ung har du väldigt svårt att få bostad (om du inte är rik) då din kötid inte kommer räcka i vanliga bostadskön förrän du närmar dig 30.</p><p>För att motverka detta hyrs en del bostäder ut som ungdomsbostäder, där förmedlingen sker med vissa åldersgränser för att hålla oss äldre utanför. Jag har utforskat förmedlingsstatistiken för dessa lägenheter, genom verktyget jag byggt på <a href="https://alvinlindstam.github.io/bostadsdata/">https://alvinlindstam.github.io/bostadsdata/</a>.</p><h4>Sammanfattning</h4><ul><li>Ungdomsbostäder förmedlas nästan bara till de äldsta i det tillåtna åldersspannet och som registrerat sig tidigt.</li><li>De allmännyttiga bolagens system med förmedling till en specifik årskull är ett effektivt sätt att möjliggöra även för yngre att få bostad. (Men hur kommer man fram till vilka åldersgrupper som behöver/förtjänar bostad mest?)</li><li>För att ha en realistisk chans till ungdomsbostad måste du ha ställt dig i bostadskön väldigt snart efter att du fyllt 18. <strong>Säg till alla unga du känner och bryr dig om att ställa sig i kön på sin 18-årsdag</strong>, det ska göras innan första lagliga alkoholköpet!</li><li>Håll noga koll på bostadsförmedlingen månaderna innan du fyller år! Det är då du har bäst chans att få ungdomsbostad. Så fort du fyllt år konkurrerar du med personer som är nästan ett år äldre än dig, och då har längre kötid. Det här gäller framförallt månaderna innan du fyller 26 (en vanlig gräns för tillsvidarekontrakt för ungdomar).</li><li>Om du inte ställt dig i kön snart efter att du fyllde 18 är det ganska kört med ungdomsbostad. Ställ dig i kön ändå, och rikta din ilska mot politiker som inte bygger tillräckligt och anhöriga/bekanta som klagar på nybyggnationer.</li></ul><h3>Vad säger datan egentligen?</h3><p>All information här bygger på skrapad data från <a href="https://bostad.stockholm.se/">bostad.stockholm.se</a>, presenterad och filtrerad genom <a href="https://alvinlindstam.github.io/bostadsdata/">https://alvinlindstam.github.io/bostadsdata/</a>. Jag kan inte garantera att all förmedlingsdata finns där, att all hantering har blivit rätt eller att andra fel inte finns, men jag tror att resultatet återspeglar verkligheten på ett rättvist sätt. <em>Ta dock inte de faktiska siffrorna som absoluta sanningar.</em></p><p><strong>För alla siffror har jag bara inkludera bostäder i Stockholms Stad och som där sista datum för anmälan var från och med 2012 om inget annat framgår.</strong></p><h4>Allmänt</h4><p>Jag har filtrerat ut de som i statistiken markerats som <em>Ungdom</em> eller <em>Ungdom korttid</em>. Det verkar som att de under kategorin <em>Ungdom</em> förmedlas vanliga förstahandskontrakt, som inte är tidsbegränsade eller går ut vid någon viss ålder, men bara kan förmedlas till folk under en viss ålder. <em>Ungdom korttid</em> är i stället vanliga korttidskontrakt, ofta gällande i 4 år.</p><p>Sedan 2012 har 2186 bostäder förmedlats som <em>Ungdom </em>och 2389 bostäder som <em>Ungdom korttid</em>, alltså totalt 4575 bostäder. Lite översiktlig data för dessa:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dqQr5-Dj8mVLlMJfMgC7JQ.png" /><figcaption>Förmedlade ungdomsbostäder från 2012 i Stockholms Stad genom bostad.stockholm.se</figcaption></figure><p>Bilden ovan visar en översikt över de förmedlade ungdomsbostäderna. Det krävs dock ytterligare filtrering för att förstå dess mönster. Till exempel finns det en del lustiga spikar i grafen högst upp till vänster, över förmedlade bostäder baserat på ködagar.</p><p>Det olika hyresvärdarna har lite olika system för sin förmedling, och bör därför förstås var för sig.</p><h4>Stockholmshem, Familjebostäder och Svenska Bostäder</h4><p>Stockholms tre “vanliga” allmännyttiga bostadsbolag står för 2677 (nästan 60%) av de förmedlade ungdomsbostäderna. De använder sina vanliga gemensamma regler för att tilldela kontrakten, men verkar ha undantag för sina (redan låga) inkomstkrav för ungdomsbostäderna om någon annan kan gå i borgen för en. Som åldersregel väljer de ut en <strong>specifik ålder mellan 18–29 för respektive annons</strong>, så att 18-åringar bara konkurrerar med andra 18-åringar osv.</p><p>Jag antar att syftet med det är att möjliggöra även för de yngre åldersgrupperna att få någon bostad. Om man i stället hade låtit alla 18–26-åringar söka till samma bostäder skulle de ju i princip bara fördelas till 25/26-åringar som hunnit få 7–8 års kötid (se övriga hyresvärdar nedan).</p><p>I stället blir kötidsfördelningen den här:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9AX3GSnNTL3wBXb16_ZQ-w.png" /><figcaption>Förmedlade ungdomsbostäder hos Familjebostäder, Stockholmshem och Svenska Bostäder genom bostad.stockholm.se</figcaption></figure><p>Det mest intressanta här är grafen över kötid:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*RzwweDGwLo0dV97xnUhxuw.png" /><figcaption>Antal ködagar för förmedlade ungdomsbostäder hos de allmännyttiga bostadsbolagen (varje stapel motsvarar en 30-dagarsperiod).</figcaption></figure><p>Varje spik motsvarar ett år. Den första spiken slutar tvärt vid ett års kötid, vilket vi kan anta är bostäder för 18-åringar. Nästa är för 19-åringar osv.</p><p>Det är tydligt att en stor del av dessa ungdomsbostäder förmedlats till personer som ligger nära den teoretiska gränsen för hur mycket kötid de kan ha samlat på sig givet sin ålder. Det innebär att <strong>de som inte anmält sig till bostadskön strax efter att de fyllt 18 år i princip inte har någon chans på dessa bostäder</strong>. Mönstret är som mest tydligt för åldrarna 18–22, men finns kvar ända upp till 25-års-annonserna.</p><p>Om dessa bostäder främst fördelades vissa årstider skulle det också innebära en fördel för folk födda en viss tid på året. Det verkar dock inte vara fallet! Om man sorterar bort alla bostäder i Västberga (där över 300 bostäder förmedlades på en gång i början av 2016) kan man se att det är ganska jämnt utspritt både vad gäller när de förmedlades (övre grafen) och när de som fick lägenheten hade ställt sig i kö (undre grafen):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gzu4U0UMnNcQoUO8I9tcJw.png" /><figcaption>Förmedlade ungdomsbostäder (ej Västberga) från de tre stora allmännyttiga bostadsbolagen.</figcaption></figure><h4>SHIS Bostäder</h4><p><a href="https://www.shis.se/">SHIS Bostäder</a> är “Stockholm stads bostadssociala resurs”, och hyr uteslutande ut bostäder som genomgångsbostäder till unga. En del förmedlas genom genomsocialtjänsten, men under aktuell period förmedlade de också 926 bostäder genom bostadsförmedlingen (främst i stadsdelarna Liljeholmen, Stadshagen och Gamla Enskede).</p><p>En stickprovskontroll av deras bostäder verkar signalera att de fördelas till den med längst kötid i åldersspannet 18–24 (18–25 från och med september 2017). De förmedlas i upp till fem år, men hyrestiden förkortas på något vis beroende på hur lång kötid man har (oklart hur mycket). De har gränser på högsta inkomst (verkar oftast vara ca 20–25 tusen/månad), och kräver att man är folkbokförd i Stockholms Stad, och att man varit det en tid.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5YuQuFBnU86uhtlYy-pupA.png" /><figcaption>Förmedlade ungdomsbostäder (korttid) till SHIS Bostäder genom bostad.stockholm.se</figcaption></figure><p>Här finns inte samma spikar i grafen över kötid som för de allmännyttiga bostadsbolagen, eftersom alla åldrar hamnar i samma hink. Det är samtidigt tydligt att det främst är de som faller i de högre åldersspannen (med 4–6 års kötid) som får lägenheterna. Jag hade väntat mig ännu högre koncentration till höga kötider, men tror att de mer omfattande kraven på maxinkomst, folkbokföringshistorik och nuvarande bostadssituation ökar spridningen.</p><p>Ju senare i tid man filtrerar, desto högre blir kötiden. Av de 149 bostäder som förmedlats från SHIS sedan 2017 har bara 42 (28%) förmedlats till personer med mindre än 4,5 års kötid.</p><h4>Övriga hyresvärdar</h4><p>972 ungdomsbostäder har under perioden förmedlats genom övriga hyresvärdard (eller annonser som jag inte kunnat knyta till en hyresvärd). 24 lägenheter från Stadsholmen (stadens fastighetsbolag för kulturhistoriska byggnader) och övriga från privata värdar.</p><p>Jag noterade dock att annonserna innan juni 2012 hade ett annat format, där jag inte kunde hitta rätt hyresvärd i min skrapning, så <em>jag visar här i stället data från och med juni 2012</em>. Jag har också filtrerat bort 51 bostäder i Kista, som förmedlades samtidigt i 2014 och var en tydlig outlier. Det som då återstår är 710 förmedlingar. De flesta av dessa har förmedlats till personer mellan 18–25 år, och bara ca 10% är korttid:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WHGeG_tFMQtLOPBJwkXbhA.png" /><figcaption>Förmedlade ungdomsbostäder från övriga hyresvärdar från juni 2012 (ej Kista)</figcaption></figure><p>Här är det extremt tydligt att bostäderna förmedlats till personer som ligger nära det teoretiska maxgränsen (25-åringar kan ha som mest 2921 ködagar). En stickprovskontroll av de förmedlade bostäderna med kortare kötider visade att de allra flesta annonser där hade en lägre åldersgräns än 25.</p><h3>Slutord</h3><p>Testa gärna <a href="https://alvinlindstam.github.io/bostadsdata/">https://alvinlindstam.github.io/bostadsdata/</a> (på en dator, inte mobil enhet) för att undersöka vad du tycker är intressant med förmedlingsstatistiken, eller vilka möjligheter du faktiskt har med din kötid. Skriv gärna om du hittar intressanta mönster.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d6f270759946" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unicode text length - Python vs Elixir]]></title>
            <link>https://medium.com/@alvinlindstam/unicode-text-lengt-python-vs-elixir-2d528b801d53?source=rss-f4c1d27db844------2</link>
            <guid isPermaLink="false">https://medium.com/p/2d528b801d53</guid>
            <category><![CDATA[unicode]]></category>
            <category><![CDATA[elixir]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[internationalization]]></category>
            <dc:creator><![CDATA[Alvin Lindstam]]></dc:creator>
            <pubDate>Sat, 05 Aug 2017 16:55:52 GMT</pubDate>
            <atom:updated>2017-08-07T09:45:15.005Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Summary: </strong>Some characters (as perceived by users) are represented by multiple Unicode characters. Programmers should be aware of this when measuring text length or splitting text. This article describes such cases, and introduces a Python library for handling such character clusters.</p><h3>Learning Elixir, or what is a grapheme?</h3><p>I come from a Python and JS background. As most programmers, I write code that handles text every once in a while. I’m used to writing len(string) or string.length to get the number of characters in a string, and have been doing so without much thought.</p><p>When I started working with <a href="https://elixir-lang.org/">Elixir</a>, I read about the standard <a href="https://hexdocs.pm/elixir/String.html">String</a> module. It’s documentation introduced the concept of graphemes, which apparently meant that some of my understanding of Unicode text was wrong.</p><p>A <a href="http://unicode.org/glossary/#grapheme">grapheme</a> is, according to the Unicode glossary:</p><blockquote>(1) A minimally distinctive unit of writing in the context of a particular writing system. For example, ‹b› and ‹d› are distinct graphemes in English writing systems because there exist distinct words like big and dig. Conversely, a lowercase italiform letter <em>a</em> and a lowercase Roman letter a are not distinct graphemes because no word is distinguished on the basis of these two different forms. (2) What a user thinks of as a character.</blockquote><p>For traditional ASCII text <em>(without CRLF-line endings)</em>, all graphemes are represented by a single Unicode code point, but that is not always the case. Some graphemes are represented by a sequence of two or more Unicode characters (code points).</p><p>The default rules for graphemes are defined in <a href="http://unicode.org/reports/tr29/">Unicode Standard Annex #29</a>.</p><h3>Grapheme examples</h3><p>These are some of the types of multi code point graphemes that may exist. Note that all Python code examples are in Python 3, so strings are unicode strings by default (byte strings is a whole other beast).</p><h4>Diacritic marks</h4><p>Some characters have diacritics added to them, such as <em>å, ä </em>and <em>ö</em> (used in my native language, Swedish). Some common characters with diacritic marks have a single Unicode character (including å, ä and ö), but they may also be represented by the base character followed by a the Unicode character for the diacritic: `</p><pre>&gt;&gt;&gt; &quot;a\u0308&quot;</pre><pre>&#39;ä&#39;</pre><p>Even though &quot;ä&quot; and &quot;a\u0308&quot; represent the same character/grapheme, they are not equal in Python or most other languages. Nor do they have the same length. This can be handled by using <a href="https://docs.python.org/3/library/unicodedata.html#unicodedata.normalize">unicodedata.normalize</a>, but that is just usable for the few diacritic combinations that have a designated Unicode character.</p><p>Some characters with diacritic marks does not have a designated Unicode character. One example is the special characters used in <a href="https://en.wikipedia.org/wiki/Esperanto">Esperanto</a>: ĉ, ĝ, ĥ, ĵ, ŝ, and ŭ. <em>ĉ </em>is represented as U+0065 (Latin Small Letter e) + U+005E (Circumflex Accent).</p><pre>&gt;&gt;&gt; len(&quot;ĝi&quot;)<br>3</pre><h4>Korean</h4><p>The Korean alphabet, <a href="https://en.wikipedia.org/wiki/Hangul">Hangul</a>, has 40 letters. They are, however, not written in sequence as in latin script, but grouped into blocks which form a single syllable. I don’t know Korean, but will include an example from the wikipedia article:</p><blockquote>That is, although the syllable 한 <em>han</em> may look like a single character, it is actually composed of three letters: ㅎ <em>h</em>, ㅏ <em>a</em>, and ㄴ <em>n</em>.</blockquote><pre>&gt;&gt;&gt; len(&quot;한&quot;)<br>3</pre><h4>Emoji modifiers</h4><p>Some emojis support modifiers, which alter their appearance. The most common is probably the skin tone modifiers. For example <a href="http://emojipedia.org/emoji/%E2%9C%8A/">U+270A</a> (Raised Fist) is rendered as ✊ . But if the following character is 🏾 (U+1F3FE - Emoji Modifier Fitzpatrick Type-5), it’s rendered as ✊🏾.</p><p>This is probably the most common issue that modern web developers might encounter if not accounting for graphemes. How would you get the first character in the string <em>”✊🏾 some text”</em> in Python?</p><pre>&gt;&gt;&gt; &quot;✊🏾 some text&quot;[0]<br>&#39;✊&#39;<br>&gt;&gt;&gt; &quot;✊🏾 some text&quot;[1:]</pre><pre>&#39;🏾 some text&#39;</pre><h4>Emoji flags</h4><p>National emoji flags are represented as a sequence of code points matching the country’s two letter country code. So 🇸🇪 (Sweden/SE) is represented as two Unicode characters: U+1F1F8 (Regional Indicator Symbol Letter S) + U+1F1EA (Regional Indicator Symbol Letter E):</p><pre>&gt;&gt;&gt; &quot;Welcome to Sweden: 🇸🇪&quot;[-1]<br>&#39;🇪&#39;</pre><h4>Other wierd shit</h4><p>Today is pride day in Stockholm, so there is no better day to examine the text representation of 🏳️‍🌈. That simple and beautiful flag is actually represented by four Unicode characters:</p><ul><li>🏳 <a href="https://emojipedia.org/emoji/%F0%9F%8F%B3/">U</a>️<a href="https://emojipedia.org/emoji/%F0%9F%8F%B3/">+1F3F3</a> (White Flag). <em>A white flag, shown waving on a post. Traditionally used as a sign of surrender, the meaning of this flag as an emoji is less defined.</em></li><li><a href="https://emojipedia.org/emoji/%EF%B8%8F/">️ U+FE0F</a> (Variation Selector-16). <em>An invisible codepoint which specifies that the preceding character should be displayed with emoji presentation. Only required if the preceding character defaults to text presentation.</em></li><li><a href="https://emojipedia.org/emoji/%E2%80%8D/">‍ U+200D</a> (Zero Width Joiner). <em>Zero Width Joiner (ZWJ) is a Unicode character that joins two or more other characters together in sequence to create a new emoji.</em></li><li><a href="https://emojipedia.org/emoji/%F0%9F%8C%88/">🌈 U+1F308</a> (Rainbow)</li></ul><pre>&gt;&gt;&gt; len(&quot;🏳️‍🌈&quot;)<br>4</pre><h3>Python vs Elixir — what is a character?</h3><p>(Modern) Python represents strings as a sequence of Unicode code points. All string operations related to character count is based on the code points without any consideration to graphemes. So len(string) will return the number of Unicode code points in a string, and slicing and indexing will use the code point sequence and not the sequence of graphemes.</p><p>Elixir’s String module defaults to considering graphemes. String.length(string) will count the number of graphemes and String.slice(string, 0, 10) will return the first 10 graphemes (even if more than 10 Unicode code points is used to represent those graphemes).</p><p>For most use cases, the Elixir way of using graphemes for character boundaries is probably more correct. It does, however, come at a performance cost. Counting graphemes require traversing the string, checking the type of character against the previous state and the boundary rules. As such, it will require linear time in relation to the string length. Getting the number of code points in a Python string can be done in constant time.</p><h3>Introducing the Python library <em>grapheme</em></h3><p>When reading up on this, I couldn’t find a simple to use Python library for grapheme aware string operations. So I wrote <a href="https://pypi.python.org/pypi/grapheme">grapheme</a>, which contains a number of functions for counting and slicing based on grapheme character boundaries.</p><pre>&gt;&gt;&gt; import grapheme</pre><pre>&gt;&gt;&gt; grapheme.length(&quot;a\u0308&quot;)</pre><pre>1</pre><pre>&gt;&gt;&gt; grapheme.length(&quot;ĝi&quot;)</pre><pre>2</pre><pre>&gt;&gt;&gt; grapheme.length(&quot;한&quot;)</pre><pre>1</pre><pre>&gt;&gt;&gt; grapheme.slice(&quot;✊🏾 some text&quot;, 0, 1)</pre><pre>&#39;✊🏾&#39;</pre><pre>&gt;&gt;&gt; grapheme.slice(&quot;✊🏾 some text&quot;, 1)</pre><pre>&#39; some text&#39;</pre><pre>&gt;&gt;&gt; list(grapheme.graphemes(&quot;Welcome to Sweden: 🇸🇪&quot;))[-1]</pre><pre>&#39;🇸🇪&#39;<br>&gt;&gt;&gt; grapheme.length(&quot;🏳️‍🌈&quot;)</pre><pre>1</pre><p>Please check it out on <a href="https://pypi.python.org/pypi/grapheme">PyPi</a> or <a href="https://github.com/alvinlindstam/grapheme">GitHub</a>.</p><h4>Related work</h4><p>There is also <a href="https://pypi.python.org/pypi/uniseg">uniseg</a>, with a very similar purpose. It is, however, still at Unicode version 6.2.0, so it lacks many recent rule changes and newly added characters. The current Unicode version is 10.0.0.</p><p>One should also consider <a href="https://pypi.python.org/pypi/PyICU">PyICU</a>, which is a Python wrapper for the ICU C library.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d528b801d53" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Phoenix Presence for social networks]]></title>
            <link>https://medium.com/@alvinlindstam/phoenix-presence-for-social-networks-5fb67143f0ad?source=rss-f4c1d27db844------2</link>
            <guid isPermaLink="false">https://medium.com/p/5fb67143f0ad</guid>
            <category><![CDATA[elixir]]></category>
            <category><![CDATA[phoenix-framework]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[social-network]]></category>
            <category><![CDATA[websocket]]></category>
            <dc:creator><![CDATA[Alvin Lindstam]]></dc:creator>
            <pubDate>Sat, 15 Apr 2017 13:59:27 GMT</pubDate>
            <atom:updated>2017-04-15T13:59:27.068Z</atom:updated>
            <content:encoded><![CDATA[<p>Phoenix Presence is awesome! It makes tracking present users (or any other connections or services) easy and reliable. If you haven’t checked it out yet, you should:</p><p><a href="https://dockyard.com/blog/2016/03/25/what-makes-phoenix-presence-special-sneak-peek">What makes Phoenix Presence special, and a sneak peek</a></p><p>But how do you track the connection state of multiple users in a social network, where we require scalability and user privacy and where each users social network is different from the others?</p><p>I tried to find out. This article describes a simple solution, demonstrating such a solution using the standard Phoenix PubSub and Presence framework.</p><h3>Social networks are not (just) chat rooms</h3><p>The official examples, and most online tutorials, uses the concept of chat rooms when demonstrating the features of Phoenix Presence. Setting up a room based chat system that tracks connected users in each room is incredibly easy with Phoenix.</p><p><a href="https://blog.diacode.com/building-a-simple-chat-using-phoenix-framework-presence-and-react">Building a simple chat using Phoenix Framework&#39;s Presence and React</a></p><p>However, the room based approach is not a good fit for most social networks. You are not connected to a few groups/rooms where each users is visible to everyone else. Instead, you should only be notified of the connection status of your social circle which is not shared with any other user.</p><p>At first, I thought of a few naive solutions using the room based approach.</p><h4>A single presence “room”</h4><p>You could connect all users to the same room, and sync their presence state to the clients while only displaying their friends. This would however be a bad solution for a few reasons:</p><ul><li>It would not scale well. Imaging having tens of thousands of connected users. Making all those clients sync the full state would require quite a lot of network and computing resources.</li><li>It’s important that each user receives data for each of their friends, but not for any user that is not their friend. Letting all users subscribe to all other users presence state would be quite a privacy issue.</li></ul><h4>Intercept and filter</h4><p>You could create a single room that all users are connected to, while using <a href="https://hexdocs.pm/phoenix/Phoenix.Channel.html#intercept/1">intercept</a> to filter out the presence_diff events that the user should not receive. You would also have to filter the initial presence state before pushing it to the client. While this would preserve user privacy and use minimal network resources, it would not scale well server side:</p><ul><li>You have to intercept each presence diff, which is slower than using the default fastlane.</li><li>The PubSub layer would have to send a message to every connected user’s process, even though very few of them would actually use the message. This would scale quadratically in relation to the number of users.</li></ul><h4>A room for each friendship</h4><p>You could also create a separate friendship room for any pair of friends, and join each user to the N rooms required to match their social network. This would preserve the privacy and scaling issues of transmitting all presence data to all users. But it’s also a no go for a few reasons:</p><ul><li>It would require a more complicated setup on the client side, with a N Channel instances that need to be initialised and handled.</li><li>Since each phoenix channel is a process on the server side, you would en up with N processes per user.</li><li>For each present user, you would have to track their presence across N topics, which would have to be synced to and stored on all connected nodes putting N times more load on your servers than necessary.</li></ul><h3>Presence tracking without chat rooms</h3><p>We have to separate the presence state and the Phoenix Channels used to transmit it, in order to track your social circle in a scalable and privacy aware manner. To do this, we’ve got to understand the basics of how Phoenix Presence works and how it’s related to the PubSub layer.</p><h4>Understanding Phoenix.Tracker</h4><p>At the core of Phoenix Presence, there is <a href="https://hexdocs.pm/phoenix_pubsub/Phoenix.Tracker.html">Phoenix.Tracker</a>. It is what actually tracks and synchronises presence data across the cluster. Any given process may be tracked with a given topic and key (and any arbitrary metadata). The topic may be any string, and neither the topic nor the process has to have any connection to Phoenix Channels.</p><p>Phoenix.Tracker is a behaviour, and implementing modules must can use the <a href="https://hexdocs.pm/phoenix_pubsub/Phoenix.Tracker.html#c:handle_diff/2">handle_diff</a> callback to be notified of state changes.</p><h4>Phoenix.Presence behaviour</h4><p>The <a href="https://hexdocs.pm/phoenix/Phoenix.Presence.html">Phoenix.Presence</a> behaviour, which is what most Phoenix apps will use, is mostly a wrapper around Phoenix.Tracker. It provides a cleaner API for usage with Phoenix Channels, well as some process and supervisor handling for the tracker and it’s Tasks.</p><p>It also implements the Phoenix.Tracker.handle_diff callback. It’s <a href="https://github.com/phoenixframework/phoenix/blob/v1.3.0-rc.1/lib/phoenix/presence.ex#L195-L205">implementation</a> broadcasts the diff using Phoenix.PubSub to the same topic as the presence topic as the presence_diff event. This is important, since this means that we can use the presence topic to control where diff events are sent.</p><p><em>At first, I assumed that </em><a href="https://hexdocs.pm/phoenix/Phoenix.Presence.html#c:track/3"><em>Presence.track</em></a><em> was involved in adding some subscription to the presence events. But that is not the case, simply joining a Channel with a given topic will mean that you will receive </em><em>presence_diff events. The “track” in the function name does not imply that you wish to track something/someone else, but that you wish for the presence framework to track you.</em></p><p>The Phoenix.Presence track, untrack and update functions all take a Phoenix.Socket struct as their first argument. They do, however, all have an alternate form where the socket struct is replace with a pid and a topic. When using the socket as the first argument, the pid and topic is simply the same as those of the channel socket. You may call them with any pid and topic you like.</p><h3>Tying it all together: A working example</h3><p>I’ve created a <a href="https://github.com/alvinlindstam/phoenix_social_presence/">phoenix_social_presence</a> demo on GitHub. It’s a basic Phoenix project, which simply gives each user an ID and uses that to connect to a private Phoenix Channel. Think of this channel as a topic that is only to be used by the corresponding user. It would be perfect for broadcasting notifications to the user, so each user should always be connected to their user channel.</p><h4>Before social presence</h4><p>In <a href="https://github.com/alvinlindstam/phoenix_social_presence/tree/classic-presence">this revision</a>, the <a href="https://github.com/alvinlindstam/phoenix_social_presence/blob/classic-presence/lib/social_presence/web/channels/user_channel.ex">channel module</a> is set up like most other Phoenix Presence examples.</p><p>It has a join function, which verifies the user and sends an :after_join message to itself:</p><pre>  def join(&quot;user:&quot; &lt;&gt; user_id_str, payload, socket) do<br>    if to_string(socket.assigns.user_id) == user_id_str do<br>        send(self, :after_join)<br>        {:ok, socket}<br>    else<br>        {:error, %{reason: &quot;unauthorized&quot;}}<br>    end<br>  end</pre><p>The :after_join callback is just as the default generated Presence suggests. It fetches the current presence state (of the same topic as the socket) and pushes it to the client. It also tracks the current channel socket given it’s user ID.</p><pre>  def handle_info(:after_join, socket) do<br>    push socket, &quot;presence_state&quot;, MyPresence.list(socket)<br>    {:ok, _} = MyPresence.track(socket, socket.assigns.user_id, %{<br>      online_at: inspect(System.system_time(:seconds))<br>    })<br>    {:noreply, socket}<br>  end</pre><p>Using this revision works, but since only a single user ID is allowed in to the topic, it will not distribute any presence state to the users friends.</p><h4>Syncing presence state asymmetrically</h4><p>In order to sync user data for a given set of friends, I wrote <a href="https://github.com/alvinlindstam/phoenix_social_presence/commit/ce8c6732d39649bca1209247baa875e4973a6cb7">this fairly simple commit</a>.</p><p>Instead of calling MyPresence.track socket, ... we’ll track the user on a separate topic:</p><pre>  defp track_user_presence(user_id) do<br>    {:ok, _} = MyPresence.track(self(), presence_topic(user_id), user_id, %{<br>      online_at: inspect(System.system_time(:seconds))<br>    })<br>  end</pre><pre>  defp presence_topic(user_id) do<br>    &quot;user_presence:#{user_id}&quot;<br>  end</pre><p><em>Note that we’re using </em><em>track/4 instead of </em><em>track/3 since we want to control the topic. We’re using </em><em>self() for the pid (since it’s used from the channel process) and our presence_topic to reliably map a user to a presence topic. With this setup, each presence topic will only contain a single key, and each user will only be tracked on a single topic.</em></p><p>Next up, we have to retrieve the current presence state of all of the users friends. Since they are now distributed across multiple topics, we have to request the presence state of each presence topic:</p><pre>    user_ids<br>      |&gt; Enum.map(&amp;presence_topic/1)<br>      |&gt; Enum.uniq<br>      |&gt; Enum.map(&amp;MyPresence.list/1)<br>      |&gt; Enum.reduce(%{}, fn map, acc -&gt; Map.merge(acc, map) end)</pre><p><em>user_ids is a list of the user ids that the current user should get presence updates for.</em></p><p>Each user is not connected to a Phoenix Channel for each presence topic that they should get updates for, so we have to make sure updates are pushed to their socket as well. We’ll do this by subscribing the current process to each relevant presence topic on the PubSub layer. I’ve wrapped this in the same pipeline as the initial presence state retrieval above:</p><pre>  defp get_and_subscribe_presence_multi(socket, user_ids) do<br>    user_ids<br>      |&gt; Enum.map(&amp;presence_topic/1)<br>      |&gt; Enum.uniq<br>      |&gt; Enum.map(fn topic -&gt;<br>           :ok = Phoenix.PubSub.subscribe(<br>             socket.pubsub_server,<br>             topic,<br>             fastlane: {socket.transport_pid, socket.serializer, []}<br>           )<br>           MyPresence.list(topic)<br>         end)<br>      |&gt; Enum.reduce(%{}, fn map, acc -&gt; Map.merge(acc, map) end)<br>  end</pre><p><em>The subscribe call above will subscribe the current process to events broadcasted to the given topic. The fastlane option tells Phoenix to send the events directly to the transport process (and only serialize it once, even when there are multiple subscribers). The last elements are any intercepts (topics that should not be fastlaned), which I set to the empty list for simplicity.</em></p><p>With the modifications I’ve shown so far, the initial state would be displayed by the client. Any updates will be pushed to the client, but since the client doesn’t have a channel for the given topic it would simply be ignored. Let’s make sure they’re handled through this modification in socket.js :</p><pre>socket.onMessage(({topic, event, payload}) =&gt; {<br>  if (event == &quot;presence_diff&quot; &amp;&amp; /^user_presence:\d+$/.test(topic)){<br>    handlePresenceDiff(payload)<br>  }<br>})</pre><p><em>The default Phoenix JS client has the </em><em>socket.onMessage function. It will call the provided function with any message received from the server, with any topic. We’ll find the messages matching the “presence_diff” event and expected topic pattern, and handle them just as “presence_diff” events on the user’s channel itself.</em></p><h4>Performance</h4><p>While the example code would need some tuning to be production ready (<em>related to general performance and possible race conditions</em>), I think that this kind of setup would scale well.</p><ul><li>Each user’s presence is <strong>tracked and stored only once</strong>. Since the full presence state has to be synced and stored on each node any duplications would be a possible scalability issue.</li><li>It uses the default PubSub mechanisms for transmitting messages. On each actual presence change, the diff will be handled and serialised once and distributed directly to each listening user’s socket.</li></ul><h4>Summary</h4><p>Distributing presence state to a social network is both possible and surprisingly easy with Phoenix, once you understand how to wire the components together.</p><p>Check out the <a href="https://github.com/alvinlindstam/phoenix_social_presence/">demo project</a>.</p><p><em>Any questions, comments or suggestions? Please leave a response, reach out on </em><a href="https://twitter.com/alvinlindstam"><em>Twitter</em></a><em> or comment on the repo.</em></p><p><em>Looking for work? </em><a href="https://jobb.kundo.se/"><em>Come join me at Kundo</em></a><em>, a Stockholm based SaaS-company building modern tools for digital customer service. We are looking for both junior and experienced developers for Python, Elixir and JS work. Swedish language skills required.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5fb67143f0ad" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>