<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Andon Sikavica on Medium]]></title>
        <description><![CDATA[Stories by Andon Sikavica on Medium]]></description>
        <link>https://medium.com/@sikavica?source=rss-1b326d3c8148------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*jFKDzb1DnK5jS6PYE8iQKg.jpeg</url>
            <title>Stories by Andon Sikavica on Medium</title>
            <link>https://medium.com/@sikavica?source=rss-1b326d3c8148------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 14:00:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sikavica/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Applying Clojure Spec]]></title>
            <link>https://medium.com/@sikavica/applying-clojure-spec-4bf0d2fac54?source=rss-1b326d3c8148------2</link>
            <guid isPermaLink="false">https://medium.com/p/4bf0d2fac54</guid>
            <category><![CDATA[clojure]]></category>
            <dc:creator><![CDATA[Andon Sikavica]]></dc:creator>
            <pubDate>Fri, 18 Sep 2020 09:04:16 GMT</pubDate>
            <atom:updated>2020-09-18T09:04:16.641Z</atom:updated>
            <content:encoded><![CDATA[<p>I will explore and code in realistic domain. Clojure Spec will help in understand and documenting the domain while coding the “business logic”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*r6dPi_CfwzIw9jhVn251WQ.png" /></figure><p>The following dependencies are used throughout this article. It is highly recommended to experiment with the examples in a <a href="https://clojure.org/guides/repl/introduction">REPL</a> session.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/23c839b0f6971a1e7d6e08605d6c6907/href">https://medium.com/media/23c839b0f6971a1e7d6e08605d6c6907/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8cb3b1600818fe9f533614ee3d2240a5/href">https://medium.com/media/8cb3b1600818fe9f533614ee3d2240a5/href</a></iframe><h3>Clojure Spec</h3><p>Clojure has been an inspiration since I’ve become more involved in UI development (about the time <a href="https://reactjs.org/">React</a> came out).</p><p>Many of the Clojure practices are applicable in other languages.</p><ul><li>functional programming</li><li>persistent-data structures</li><li><a href="https://github.com/stuartsierra/component">life-cycle management</a></li><li><a href="https://github.com/Day8/re-frame">reactive framework</a></li></ul><p>They have found a <a href="https://github.com/netceteragroup/skele">way to become everyday use</a> for me.</p><p>One feature that I haven’t seen being used outside of the the Clojure realm is <a href="https://clojure.org/about/spec">Clojure Spec</a>. This article is an exploration of using it in an almost realistic scenario.</p><p>We won’t introduce what spec is here. The official <a href="https://clojure.org/about/spec">rationale</a> and <a href="https://clojure.org/guides/spec">guide</a> are very clear and easy to follow.</p><h3>Spec-ing a Domain</h3><p>Our goal is to explore a simple domain. One that is not hard to grasp, but has enough substance to show possible usage contexts and how they can be coded. Along the way, we will see how Clojure Spec (and also Clojure itself) will help us in writing code.</p><p>Let’s familiarize ourselves with <a href="https://pulse.eco/">Pulse Eco</a>. It’s a great platform for gathering environmental data from cities around the world. It helps in raising the awareness, and also fostering improvement and development.</p><p>We start exploring by “just talking” with the REST services provided.</p><h4>Sensor</h4><p>The first available resource from the <a href="https://pulse.eco/restapi">API documentation</a> is about Sensor data. It’s rather simple for consumption. There are two variants: “give me all the sensors” and “give me a specific one by id”. In terms of input, there is not much to discuss, so the documentation focuses on the shape of the output.</p><p>Clojure doesn’t have types. When we code our data operations, it might not be so evident what is the shape of the data. This happens mostly when we are reading Clojure code base that we haven’t written, or are coming back to it after a while. So, let’s try to specify what we know.</p><p>The documentation gives info about two specific sensor properties, type and status. They are sets of possible values, so lets spec them:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0fa070a6391a5a06ba957813a0b35346/href">https://medium.com/media/0fa070a6391a5a06ba957813a0b35346/href</a></iframe><p>All the other properties, the previous two as well, are strings. We don’t have more info. For now, we spec them as:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/88575e8c0470c3f8814482d17081a8eb/href">https://medium.com/media/88575e8c0470c3f8814482d17081a8eb/href</a></iframe><p>At a point when information is sparse, we can do some exploration to dig deeper. Clojure <a href="https://clojure.org/guides/repl/introduction">REPL</a> is perfect for this.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/97663d6d64460468dc9b3bd4e45ea629/href">https://medium.com/media/97663d6d64460468dc9b3bd4e45ea629/href</a></iframe><pre>[{:sensor-id &quot;1002&quot;,<br>  :position &quot;41.9875,21.6525&quot;,<br>  :comments &quot;MOEPP sensor at Miladinovci&quot;,<br>  :type &quot;0&quot;,<br>  :description &quot;MOEPP Miladinovci&quot;,<br>  :status &quot;ACTIVE&quot;}<br> {:sensor-id &quot;11888f3a-bc5e-4a0c-9f27-702984decedf&quot;,<br>  :position &quot;41.995828195848325,21.484215259552002&quot;,<br>  :comments &quot;V1 wifi sensor in MZT area&quot;,<br>  :type &quot;2&quot;,<br>  :description &quot;MZT&quot;,<br>  :status &quot;ACTIVE&quot;}<br>...</pre><p>In the above example we are:</p><ul><li>Using <a href="https://github.com/dakrone/clj-http">http-client</a> to get the results from the sensor end-point.</li><li>Taking the body from the response.</li><li><a href="https://clj-commons.org/camel-snake-kebab/">Transforming</a> the <em>camelCase</em> keys of the the maps in <em>kebab-case</em> clojure keywords.</li><li>Fetch sensors and explore.</li></ul><p>Some quick “data slicing” might give a clearer picture about what we are seeing.</p><p>Number of sensors in Skopje:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4c0a9a2b542043d3c5fff7cd8c638e1/href">https://medium.com/media/e4c0a9a2b542043d3c5fff7cd8c638e1/href</a></iframe><pre>40</pre><p>In what status are the sensors:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3dbea235fde88e7deb7658df714baf0d/href">https://medium.com/media/3dbea235fde88e7deb7658df714baf0d/href</a></iframe><pre>Map {&quot;ACTIVE&quot;: 35, &quot;NOT_CLAIMED&quot;: 5}</pre><p>How many are WiFi-based:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b6f966a1b207a96a22cc11871e2bf9b9/href">https://medium.com/media/b6f966a1b207a96a22cc11871e2bf9b9/href</a></iframe><pre>25</pre><p>What are the locations of the sensors in human-readable form:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8cfee311b89d688296a25634fa19bb06/href">https://medium.com/media/8cfee311b89d688296a25634fa19bb06/href</a></iframe><pre>(&quot;MOEPP Miladinovci&quot; <br> &quot;MZT&quot; <br> &quot;Bardovci&quot; <br> &quot;Biser&quot; <br> ...)</pre><p>Show me the sensor data of my sensor:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3827b4c5049a1c9ac6dc95a732fbafc9/href">https://medium.com/media/3827b4c5049a1c9ac6dc95a732fbafc9/href</a></iframe><pre>{:sensor-id &quot;c6424b37-1d84-4ccf-831e-512f1bcf19a4&quot;, <br> :position &quot;41.982045,21.477023&quot;, <br> :comments &quot;pulse.eco Wi-Fi v2. sensor at Novo Lisice&quot;, <br> :type &quot;3&quot;, <br> :description &quot;Novo Lisice&quot;, <br> :status &quot;ACTIVE&quot;}</pre><p>Here we can see a single sensor and its properties. We decide that we want to be stricter then just string?-validation for some of them. sensor-id and position are good candidates. It is clear that sensor-id is a UUID and position a latitude-longitude tuple. Let&#39;s improve ::sensor-id first.</p><p>Clojure has a <a href="https://clojuredocs.org/clojure.core/uuid_q">predicate uuid?</a> that is used for checking if a value is UUID. Unfortunately, when reading the JSON response, we didn’t transform it to a UUID, but left it as a string. We are deciding now to create a new predicate that checks for a UUID represented as a string:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9c1486c69ce5fa09dde052a9caf7469c/href">https://medium.com/media/9c1486c69ce5fa09dde052a9caf7469c/href</a></iframe><p>Now we can redefine ::sensor-id and also create a spec for sensor(s):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/04d67f746b3b0ede8747128aba10ee70/href">https://medium.com/media/04d67f746b3b0ede8747128aba10ee70/href</a></iframe><p>With these specs in place, we can try to validate the data.</p><p>First just one sensor:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cd32177c7f6fa50f8ad5030fba326cf2/href">https://medium.com/media/cd32177c7f6fa50f8ad5030fba326cf2/href</a></iframe><pre>true</pre><p>And then all of them:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a9f059633ba66f3ae3c469abcb165257/href">https://medium.com/media/a9f059633ba66f3ae3c469abcb165257/href</a></iframe><pre>false</pre><p>Hmm… The result being false is unexpected. Let&#39;s see what spec says:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c25c4a38209d27ed84ad353fc1987aa2/href">https://medium.com/media/c25c4a38209d27ed84ad353fc1987aa2/href</a></iframe><pre>&quot;1002&quot; - failed: uuid-string? in: [0 :sensor-id] at: [:sensor-id] spec: :user/sensor-id <br>&quot;1005&quot; - failed: uuid-string? in: [6 :sensor-id] at: [:sensor-id] spec: :user/sensor-id <br>&quot;1004&quot; - failed: uuid-string? in: [12 :sensor-id] at: [:sensor-id] spec: :user/sensor-id <br>&quot;1003&quot; - failed: uuid-string? in: [21 :sensor-id] at: [:sensor-id] spec: :user/sensor-id <br>&quot;1000&quot; - failed: uuid-string? in: [31 :sensor-id] at: [:sensor-id] spec: :user/sensor-id <br>&quot;1001&quot; - failed: uuid-string? in: [36 :sensor-id] at: [:sensor-id] spec: :user/sensor-id</pre><p>Spec is providing us here with nice facility that shows errors and paths where the problems were found.</p><p>In case the above official explanation from spec isn’t clear enough, we can use the <a href="https://github.com/bhb/expound">expound library</a> for a more human-readable output:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c13c9734d27bb42b816f3828abe5dd8/href">https://medium.com/media/4c13c9734d27bb42b816f3828abe5dd8/href</a></iframe><pre>-- Spec failed --------------------    </pre><pre>[{:sensor-id &quot;1002&quot;,  <br>             ^^^^^^   <br>  :position ..., <br>  :comments ...,   <br>  :type ...,   <br>  :description ...,  <br>  :status ...}  <br> ...  <br> ...  <br> ...]</pre><pre>should satisfy  <br>  uuid-string?</pre><pre>  -- Relevant specs ------- </pre><pre>:user/sensor-id: <br>  user/uuid-string? <br>:user/sensor:  <br>  (clojure.spec.alpha/keys  <br>   :req-un  <br>   [:user/sensor-id <br>    :user/position <br>    :user/comments <br>    :user/type <br>    :user/description <br>    :user/status]) <br>:user/sensors: <br>  (clojure.spec.alpha/coll-of :user/sensor)</pre><pre>... &lt;retracted&gt;</pre><p>For some reason, certain sensor-ids are not UUIDs. They are just four-digit string numbers. Lets say this is a legacy leftover. The API producers are promising that these are the only such cases. We extend the spec for it and validate again:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/830622bda5b18136c801e2102cb9793a/href">https://medium.com/media/830622bda5b18136c801e2102cb9793a/href</a></iframe><pre>Success!</pre><p>The location property we leave for later.</p><h4>Raw Data</h4><p>Off to the <a href="https://pulse.eco/restapi#rawdata">next service</a> to find a bit more complexity.</p><p>We can see that the raw-data API has more input parameters:</p><ul><li>the already introduced sensor-id</li><li>type for the different types of measurements</li><li>from/to date-time to specify the requested period of measurements</li></ul><p>Since we will use date-times, and we will need to perform validation, let’s use a <a href="https://juxt.pro/tick/docs/index.html">library</a> and some helpers:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7f7015f8ef3af6f5b61836c3c3859083/href">https://medium.com/media/7f7015f8ef3af6f5b61836c3c3859083/href</a></iframe><pre>true</pre><p>In case we are building a Clojure-client for fetching raw-data, we might create a function to do that. The API documentation has example requests. They tell us that the sensor-id and type are <strong>semi-optional inputs</strong>, i.e. at least one has to be provided. It&#39;s an interesting function, so lets give it a go:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e692bda4ac27868bfcf0cfddcb606865/href">https://medium.com/media/e692bda4ac27868bfcf0cfddcb606865/href</a></iframe><p>While this is working now, for someone that might try to use or maintain this code, it would certainly help to:</p><ul><li>know about the relation of the inputs (from before to, one of sensor-id or type mandatory)</li><li>shape of data</li></ul><p>These leads us to creation of a <strong>function spec</strong>. But before we tackle that, lets see if we are on a good track with the shape of the data:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/630b2d1c4262c5765a5ca2dc648231a5/href">https://medium.com/media/630b2d1c4262c5765a5ca2dc648231a5/href</a></iframe><pre>false</pre><p>Again, something is invalid, when we expected it to be valid. What’s the explanation?</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d549d9443b2c0d5ad7b4808627bc7186/href">https://medium.com/media/d549d9443b2c0d5ad7b4808627bc7186/href</a></iframe><pre>...<br>   {:sensor-id ...,<br>    :stamp ...,<br>    :type &quot;co&quot;,<br>          ^^^^<br>    :position ...,<br>    :value ...}<br>   ...</pre><pre>   ...<br>   ...]</pre><pre>should be one of: &quot;humidity&quot;, &quot;noise&quot;, &quot;pm10&quot;, &quot;pm25&quot;, &quot;temperature&quot;</pre><pre>-- Relevant specs -------</pre><pre>:user/type:<br>  #{&quot;pm10&quot; &quot;noise&quot; &quot;temperature&quot; &quot;humidity&quot; &quot;pm25&quot;}<br>:user/measurement:<br>  (clojure.spec.alpha/keys<br>   :req-un<br>   [:user/sensor-id :user/position :user/stamp :user/type :user/value]<br>   :opt-un<br>   [:user/year])<br>:user/measurements:<br>  (clojure.spec.alpha/coll-of :user/measurement)</pre><pre>-------------------------<br>Detected 380 errors</pre><p>We see a lot of values for ::type that are not in the expected set. To be precise:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1e1d34c3b37cad458cccebc3289684b3/href">https://medium.com/media/1e1d34c3b37cad458cccebc3289684b3/href</a></iframe><pre>Map {&quot;co&quot;: 93, &quot;no2&quot;: 96, &quot;o3&quot;: 96, &quot;pm10&quot;: 96, &quot;so2&quot;: 95}</pre><p>These are air-pollution metrics. Maybe the sensor only reported them in 2017, let’s check what the sensor reported in 2020:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d9a760040bfe1fd0946ff4af64203bf1/href">https://medium.com/media/d9a760040bfe1fd0946ff4af64203bf1/href</a></iframe><pre>Map {&quot;co&quot;: 85, &quot;o3&quot;: 94, &quot;pm10&quot;: 94, &quot;so2&quot;: 94, &quot;no2&quot;: 90}</pre><p>Still the same. OK, let’s see my sensor, since it’s probably a newer version:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a3da70193c2ab7cfb7fe7267bcf81870/href">https://medium.com/media/a3da70193c2ab7cfb7fe7267bcf81870/href</a></iframe><pre>#{&quot;gasResistance&quot; &quot;pm10&quot; &quot;noise&quot; &quot;noise_dba&quot; &quot;temperature&quot; &quot;humidity&quot;   &quot;pm25&quot; &quot;pressure&quot;}</pre><p>This confirms that the newer sensors are measuring values of the documented types. Our subjective opinion is that only the measurements documented and present in my sensor are the important ones. We filter the others and verify the output:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d9a02c55e24d829d6720eb526bab39b6/href">https://medium.com/media/d9a02c55e24d829d6720eb526bab39b6/href</a></iframe><pre>true</pre><p>Now that we know a bit more about the data, let’s get back to the main challenge, providing a function spec:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/13d6a9bcdf95606317baaf87b8a7e408/href">https://medium.com/media/13d6a9bcdf95606317baaf87b8a7e408/href</a></iframe><pre>Vector(1) [ 0: user/fetch-raw-data-filtered ]</pre><p>In the function spec definition above we can notice the following:</p><ul><li>The output is validated using ::measurements with :ret</li><li>The arguments of the function are validated using :args</li><li>The shape of the arguments is validated by using ::date-time spec for the from and to arguments. We are using using the more complex ::raw-data-opts spec for checking that <strong>at least one of</strong> (valid) sensor-id or type is present in opts</li><li>In the end, we also make sure that the to date-time <strong>&quot;is later&quot;</strong> then from</li></ul><p>Function specs can validate not only <strong>arg</strong>ument<strong>s</strong> and <strong>ret</strong>urn of function, but their relation as well. In our case, such an example might be to check that the :stamp time in each measurement is in the range [from-to].</p><p>The <strong>instrument</strong> function sets up fetch-raw-data-filtered to be validated with its own spec. This is very useful during development, where &quot;wrong usage&quot; of the function can be caught early. After the instrumentation setup, the function spec will make the code to error in case it does not conform. We can test this behavior with some of the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/92820d339aba859e146f121991ab0dcd/href">https://medium.com/media/92820d339aba859e146f121991ab0dcd/href</a></iframe><pre>List(1480) (</pre><pre>0: Map {:sensor-id: &quot;c6424b37-1d84-4ccf-831e-512f1bcf19a4&quot;, :stamp: &quot;2020-03-15T00:05:36+01:00&quot;, :type: &quot;humidity&quot;, :position: &quot;41.982045,21.477023&quot;, :value: &quot;60&quot;},</pre><pre>1: Map {:sensor-id: &quot;c6424b37-1d84-4ccf-831e-512f1bcf19a4&quot;, :stamp: &quot;2020-03-15T00:05:36+01:00&quot;, :type: &quot;noise&quot;, :position: &quot;41.982045,21.477023&quot;, :value: &quot;26&quot;},</pre><pre>2: Map {:sensor-id: &quot;c6424b37-1d84-4ccf-831e-512f1bcf19a4&quot;, :stamp: &quot;2020-03-15T00:05:36+01:00&quot;, :type: &quot;pm10&quot;, :position: &quot;41.982045,21.477023&quot;, :value: &quot;10&quot;},</pre><pre>...</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ae5ef61c8e879c32cdf67284ee6c6af/href">https://medium.com/media/7ae5ef61c8e879c32cdf67284ee6c6af/href</a></iframe><pre>-- Spec failed --------------------<br>Function arguments<br>Part of the value<br>(#time/zoned-date-time &quot;2020-03-19T00:00+01:00[Europe/Skopje]&quot;    <br> #time/zoned-date-time &quot;2020-03-15T00:00+01:00[Europe/Skopje]&quot;   \  <br> {:sensor-id &quot;c6424b37-1d84-4ccf-831e-512f1bcf19a4&quot;})</pre><pre>when conformed as</pre><pre>{:from #time/zoned-date-time &quot;2020-03-19T00:00+01:00[Europe/Skopje]&quot;, :to #time/zoned-date-time &quot;2020-03-15T00:00+01:00[Europe/Skopje]&quot;, :opts [:sensor-id-opt {:sensor-id [:uuid-string &quot;c6424b37-1d84-4ccf-831e-512f1bcf19a4&quot;]}]}</pre><pre>should satisfy</pre><pre>(fn [%] (tick.alpha.api/&lt; (:from %) (:to %)))</pre><pre>-------------------------</pre><pre>Detected 1 error</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5a9697262a46d60b5774b347df2c0080/href">https://medium.com/media/5a9697262a46d60b5774b347df2c0080/href</a></iframe><pre>-- Spec failed --------------------</pre><pre>Function arguments</pre><pre>(... ... {:wrong &quot;thing&quot;})</pre><pre>          ^^^^^^^^^^^^^^^^</pre><pre>should contain keys: :sensor-id, :type</pre><pre>| key        | spec                                              |</pre><pre>|============+===================================================|</pre><pre>| :sensor-id | &lt;can&#39;t find spec for unqualified spec identifier&gt; |</pre><pre>|------------+---------------------------------------------------|</pre><pre>| :type      | &lt;can&#39;t find spec for unqualified spec identifier&gt; |</pre><pre>-- Relevant specs -------</pre><pre>:user/raw-data-opts:</pre><pre>(clojure.spec.alpha/or</pre><pre>:sensor-id-opt</pre><pre>(clojure.spec.alpha/keys :req-un [:user/sensor-id])</pre><pre>:type-opt</pre><pre>(clojure.spec.alpha/keys :req-un [:user/type]))</pre><pre>-------------------------</pre><pre>Detected 1 error</pre><p>The function spec is part of the function documentation. A potential users might look for it, and they will get a very useful insights:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/71a70cc432bb3e9cc46cc99e1dc86b53/href">https://medium.com/media/71a70cc432bb3e9cc46cc99e1dc86b53/href</a></iframe><pre>user/fetch-raw-data-filtered<br>([from to {:keys [sensor-id type]}])<br>Spec<br>  args: (and (cat :from :user/date-time <br>                  :to :user/date-time <br>                  :opts :user/raw-data-opts)<br>        (&lt; (:from %) (:to %)))<br>  ret: (coll-of :user/measurement)</pre><h4>Average Data</h4><p>Reading the average data service seems to bring few differences from the raw-data:</p><ul><li>The sensor-id can also have additional value -1, indicating average from all city sensors</li><li>It is not explicitly written if sensor-id and type are mandatory or not</li><li>The service has 3 variants: <em>daily</em>, <em>weekly</em> or <em>monthly</em> average</li></ul><p>Quick fetch-calls to find answers. This is what we got:</p><ul><li>type is mandatory</li><li>sensor-id is optional; omitting it gives separate averages for all sensors (not like -1 that returns aggregated value as one measurement)</li></ul><p>With this in mind, here is an implementation (and specification) of the function:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c99f3cac7fbc6452c671f19296c9e88a/href">https://medium.com/media/c99f3cac7fbc6452c671f19296c9e88a/href</a></iframe><p>The function and spec are similar with the one from raw-data. The argument specification is simple, except the new specification for the sensor-id. It has few caveats:</p><ul><li>It includes &quot;-1&quot; as a valid value</li><li>It can be optional (by spec-ing it with s/?)</li><li>It can be nil-able (because of the implementation of the 4-arity function)</li></ul><p>If we want to be more clear, we should also update the return type specification. The currently defined ::measurements need to have different spec for sensor-id. The spec-for position will work, but if we want to be precise it has to be noted that when &quot;-1&quot; is used (to get data for aggregated sensors) it returns empty string. All of these can be specified! It might be even cool to specify the relation &quot;when sensor-id <strong>arg</strong> is -1 position in <strong>ret</strong>urn will be empty string&quot;.</p><p>As we don’t see the immediate need to do this, we leave it like it is.</p><p><strong>Instrument</strong>ation <a href="https://clojure.org/guides/spec#_instrumentation_and_testing">will not fail</a> when the return result does not conform with the spec. This is by design. It is expected that :ret specs are validated using tests.</p><h4>Latest Data</h4><p>This is the last service from Pulse Eco API. It has methods that are suited as a first values to show the current state in a city.</p><p>They are easy to be implemented and specified with the infrastructure we already have.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/842fd04c7d5474dbc05797f22b067aa1/href">https://medium.com/media/842fd04c7d5474dbc05797f22b067aa1/href</a></iframe><pre>Map {<br>  :city-name: &quot;skopje&quot;,<br>  :values: Map {<br>    :no-2: &quot;28&quot;,<br>    :pm-25: &quot;7&quot;,<br>    :o-3: &quot;65&quot;,<br>    :pm-10: &quot;11&quot;,<br>    :temperature: &quot;26&quot;,<br>    :humidity: &quot;51&quot;,<br>    :pressure: &quot;979&quot;,<br>    :noise-dba: &quot;51&quot;}}</pre><p>We use the opportunity to update the spec for the measurement ::type. I find that filtering was not adequate. Finding out the possible values is easy, so better to be correct.</p><p>The overall function has a (new) different response, but we will not need it, so we don’t bother with spec-ing :ret.</p><h3>Business Requirements</h3><p>Our goal is to draw charts with the data from Pulse-Eco API.</p><p>The charts that are available for <a href="https://skopje.pulse.eco/explorer">Skopje Pulse-Eco</a> site are interactive with the map being the center of attention. There are not many types of charts available. There is the basic daily view (with possibility of selecting sensors).</p><p>We want to use the <a href="https://vega.github.io/vega-lite/">Vega-Lite</a> library to create interactive charts that will showcase additional views of the data. Hopefully this will be useful for visually exploring and getting insights from the data.</p><h4>Daily Plot</h4><p>We start by the most general case, the daily line plot. We want to see the data for a specific pollutant, I mean measurement type, during the day.</p><p>Here is a very basic implementation:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb53f757c48ea11b60d8991e931c64e4/href">https://medium.com/media/fb53f757c48ea11b60d8991e931c64e4/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*Qsu3SCh6MHEZ10DYvv_Qpg.png" /><figcaption>Daily Plot</figcaption></figure><p>Short summary of the code and reasoning:</p><ul><li>The plot function is a helper from Nextjournal for drawing vega-lite charts. Here you can see the exported image. You can play with it the following <a href="https://vega.github.io/editor/#/gist/3abdd35886e277760e46a0e74ef0aaa2/apply-clj-34-first-plot-spec.json">gist in the Vega Editor</a></li><li>line-plot is our main (somewhat) generic function. It is meant to plot &quot;lines&quot; from a set of measurements.</li><li>Even though the measurements contain the sensor, we wanted to display a sensor-name in a human-readable form, so we are adding also the list of sensors here. We have a collision for the :type, so we remove it before we perform the join. This looks like a good example that we should try to prefer a usage of <a href="https://blog.jeaye.com/2017/10/31/clojure-keywords/">namespaced keywords</a>.</li><li>Expectation is that all the measurements are for a same measurement type. We might try to spec this at some point. The implementation takes this for granted and uses the type from the first measurement as a description of the y-axis.</li><li>daily-plot takes a date and type, fetches the data and sensors and uses line-plot.</li></ul><p>Up until now, we implemented logic that communicates with 3rd-party API. We leverage spec to:</p><ul><li>perform data validation</li><li>conform the usage of function</li><li>documentation</li></ul><p>With the above plot, we have now some basic logic. It would be nice to try out some <a href="https://clojure.org/guides/test_check_beginner">property-based testing</a>. We want to test the logic in the line-plot function. Its arguments are already spec-ed. The beauty is that we can relatively easy generate example data and verify correct behavior.</p><p>Let’s try to generate some data from the specs ::measurements and ::sensors.</p><p>We start off, by looking what can be generated. For example, UUIDs can be generated from the corresponding predicate:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c6409cad1f962c8828c6bd98f696ec91/href">https://medium.com/media/c6409cad1f962c8828c6bd98f696ec91/href</a></iframe><pre>(#uuid &quot;af9a17b7-d36a-4010-b662-ff8574f6218a&quot;<br> #uuid &quot;c33e2af2-068c-4949-9a29-26e24f6d7a4b&quot; <br> #uuid &quot;923fd3e2-0ee4-4e47-8996-a217fdf3f224&quot;)</pre><p>Of course, not every predicate will “magically work” and generate data that conforms to it. For details about this, check the documentation about <a href="https://clojure.org/guides/spec#_using_sand_generators">using and creating custom generators</a>.</p><p>A typical example that doesn’t work is to use the pos? predicate. It will not work:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c3ebee1aa44634b65aeef92e4058de51/href">https://medium.com/media/c3ebee1aa44634b65aeef92e4058de51/href</a></iframe><pre>&quot;Unable to construct gen at: [] for: clojure.core$pos_QMARK_@71e073b4&quot;</pre><p>On the other hand:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ad62f98fb0ec6585088f33122f6d318b/href">https://medium.com/media/ad62f98fb0ec6585088f33122f6d318b/href</a></iframe><pre>List(3) (<br>  0: 1,<br>  1: 1,<br>  2: 2<br>)</pre><p>If you start with integers, positive <strong>can</strong> be generated (in the 100 tries that are going to be attempted). Still, it’s not that simple. If you start with string, uuid-strings probably <strong>can not</strong> be generated.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/49333d1508fc043ac63f0e8875b3d9e9/href">https://medium.com/media/49333d1508fc043ac63f0e8875b3d9e9/href</a></iframe><pre>List(1) (<br>  0: Map {<br>  :via: Vector(1) [Map],<br>  :trace: Vector(74) [Vector(4), 73 more...],<br>  :cause: &quot;Couldn&#39;t satisfy such-that predicate after 100 tries.&quot;,<br>  :data: Map {:pred: Vector(4), :gen: Map, :max-tries: 100}}<br>)</pre><p>We can, however, start with generators and map them to a string. So, the following is a way of generating a sample of few uuid-strings:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/df888b37571135c56b992edbddaed276/href">https://medium.com/media/df888b37571135c56b992edbddaed276/href</a></iframe><pre>List(3) (<br>  0: &quot;3603307c-f023-4a97-aa19-b6d3f408d99a&quot;,<br>  1: &quot;42d99fa2-6a84-467d-98bf-b276d7acc6b1&quot;,<br>  2: &quot;6588a7ee-c4a1-4a76-a842-13fd03706e94&quot;<br>)</pre><p>Combined with the “legacy sensors”, we can create a fairly reliable generator for sensor-ids for Pulse Eco, that can be used for any testing needs:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d25c7018e6b59d65a06f6ca77260a336/href">https://medium.com/media/d25c7018e6b59d65a06f6ca77260a336/href</a></iframe><pre>List(10) (<br>  0: &quot;0e3269b1-d3e0-4af1-bb2b-d916b0047c3d&quot;,<br>  1: &quot;1004&quot;,<br>  2: &quot;1a88743a-625d-4731-bc10-e36f3657539a&quot;,<br>  3: &quot;60452333-a07a-48fa-a76f-bb46c567bab6&quot;,<br>  4: &quot;5d66606c-a5a3-452e-9eff-b2dec71c2ffd&quot;,<br>  5: &quot;19ba4191-32b5-4d8e-85fe-96ab0897fef3&quot;,<br>  6: &quot;fb689a8c-1420-4bc3-a786-a3f135320381&quot;,<br>  7: &quot;a06c7833-a4ef-4d2d-9b3d-c141dd89c08c&quot;,<br>  8: &quot;df341060-f8e5-4b1d-bdb7-2bcb55d234e3&quot;,<br>  9: &quot;1005&quot;<br>)</pre><h4>Position</h4><p>We want a generator for sensor (measurement) position. Let’s see how what the real data looks like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0cb8be244a78d26fb5e58a0ea6f183a7/href">https://medium.com/media/0cb8be244a78d26fb5e58a0ea6f183a7/href</a></iframe><pre>List(40) (<br>  0: &quot;41.9875,21.6525&quot;,<br>  1: &quot;41.995828195848325,21.484215259552002&quot;,<br>  2: &quot;42.027618975073544,21.38741970062256&quot;,<br>  3: &quot;41.98370716725851,21.467853784561157&quot;,<br>  4: &quot;42.00788313967865,21.39364242553711&quot;,<br>  5: &quot;42.003768,21.394638&quot;,<br>  6: &quot;41.9992,21.4408&quot;,<br>  7: &quot;42.00207919255764,21.423720717430115&quot;,<br>  8: &quot;41.997162962969476,21.385660449563975&quot;,<br>  9: &quot;41.995238146587674,21.402708292007443&quot;,<br>  30 more...<br>)</pre><p>We will improve the spec for position, and create a generator like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/52e22ab455f275d0004eb88766205965/href">https://medium.com/media/52e22ab455f275d0004eb88766205965/href</a></iframe><pre>List(10) (<br>  0: &quot;-1.0,-2.0&quot;,<br>  1: &quot;-0.75,3.0&quot;,<br>  2: &quot;2.0,1.0&quot;,<br>  3: &quot;1.625,-1.375&quot;,<br>  4: &quot;-1.0,0.5625&quot;,<br>  5: &quot;-1.75,-0.5&quot;,<br>  6: &quot;-2.0,-2.0&quot;,<br>  7: &quot;1.3125,-0.59375&quot;,<br>  8: &quot;3.125,0.7421875&quot;,<br>  9: &quot;-0.375,3.453125&quot;<br>)</pre><p>A possible (naive) generator for Skopje, might be:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/68931af3955d6145c383a8d9a021f86e/href">https://medium.com/media/68931af3955d6145c383a8d9a021f86e/href</a></iframe><h4>Interactive Plot</h4><p>The following is an example of interactive plot. It is shown with a certain sensor preselected, but you can use it to explore the others by hovering over.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/07d41428e9149fcb123c859a55300714/href">https://medium.com/media/07d41428e9149fcb123c859a55300714/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/777/1*c4pZ_gSm2vQtoYeBD6SceA.png" /><figcaption>Interactive Plot (image), check it in <a href="https://vega.github.io/editor/#/gist/1e95228dbfcf0858cce029f6716be9d8/apply-clj-45-interactive-plot-spec.json">Vega Editor for the real deal</a></figcaption></figure><h3>Final Words</h3><p>This exploration into Clojure Spec was very insightful. I find spec-ing to be very useful and I didn’t have too much hurdles in doing what I wanted. The power of what you can spec is enormous. You get to define much more than you would do with type systems, and I really like how spec is orthogonal over the main business code (which stays clean and uncluttered).</p><p>If I had more time, I would like to explore more property-based tests. Would have been nicer to compile an example of finding a minimal set of data that “breaks” the business logic based on some of the generators here.</p><p>For some near future, I hope I can create plots for use-cases that are not yet in pulse.eco.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4bf0d2fac54" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[React Native Project Setup for large projects]]></title>
            <link>https://medium.com/@sikavica/react-native-project-setup-for-large-projects-9a3acbd92903?source=rss-1b326d3c8148------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a3acbd92903</guid>
            <category><![CDATA[react-native]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Andon Sikavica]]></dc:creator>
            <pubDate>Sun, 17 Jan 2016 22:44:17 GMT</pubDate>
            <atom:updated>2016-01-17T22:44:17.449Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://facebook.github.io/react-native/">React Native</a> does a lot with its project initialisation using the following command:</p><pre>react-native init AwesomeProject</pre><p>It creates a basic setup for the <em>iOS</em> and <em>Android</em> projects, as well as giving a simple structure how the code is organised. This is very nice for smaller projects, or when you just need to get into creating apps straight-away.</p><p>As soon as you work on a <strong>larger projects</strong>, you will need to:</p><ul><li>split up the project into multiple projects</li><li>allow proper code sharing between the projects</li></ul><p>This seem like a simple task for any <em>dependency management tool</em>, and indeed using <strong>npm</strong>, you can easily do this. The thing is, you also want to <strong>develop</strong> in all the projects <strong>in parallel</strong>. For this, you usually use:</p><pre>npm link</pre><p>Unfortunately, the <a href="https://github.com/facebook/react-native/tree/master/packager">React Packager</a> at the moment does not resolve symlinks properly and having this kind of setup is not easily possible. There is a <a href="https://productpains.com/post/react-native/symlink-support-for-packager/"><strong>Product Pain</strong></a> already for this, so give it a bit of <strong>up-vote please</strong>.</p><blockquote>Developer experience in React Native is really <strong>awesome</strong>.</blockquote><p>… so it was at upmost importance to have the same in any kind of setup we choose. Weather you work in a sub-project or a target-project, it should not matter. You change something and you want the refresh to work.</p><h4>Solution</h4><p>The solution we came is based on the following:</p><ul><li><a href="https://git-scm.com/book/en/v2/Git-Tools-Submodules"><strong>Git Submodules</strong></a> for the core projects</li><li>Custom <strong>configuration</strong> for React Packager <strong>delivered via </strong><a href="https://github.com/facebook/react-native/blob/3a92f2017fc2e51c2a915f0e535397cb3937d412/local-cli/util/Config.js"><strong>rn-cli-config</strong></a></li></ul><p>Using git submodules is straightforward. You create your sub-projects as any other project. In your top-most target project, where you want to use the sub-projects, you reference them as git submodules.</p><p>Lets go with an <strong>example</strong>. We have a main project named <em>awesome-project</em>; and a sub-project named <em>awesome-components.</em> To include <em>awesome-components</em> into <em>awesome-project</em>, in <em>awesome-project</em> you do:</p><pre>git submodule add &lt;git-repo-awesome-components&gt;<br>cd awesome-components<br>git fetch<br>git pull origin master</pre><p>In case you have multiple projects, and you want to move them to one folder (example modules), than you can do this with:</p><pre>mkdir modules<br>git mv awesome-components modules/awesome-components</pre><p>Next on the list is using <em>awesome-components</em> into <em>awesome-projects</em>. Many suggestions out there are just using something like the following in the main project:</p><pre>require (../../../modules/awesome-components/play.js);</pre><p>To solve this, you can put a custom configuration in the main project, like the following:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e97996112aa79860741ef0f55bf07fd3/href">https://medium.com/media/e97996112aa79860741ef0f55bf07fd3/href</a></iframe><p>With this, you can have the previous <em>require</em> written like you usually do for any dependent library you use:</p><pre>require (awesome-components/play.js);</pre><h4>Conclusion</h4><p>This setup was made with several considerations in mind:</p><ul><li>Larger code-base split up in multiple projects that share code.</li><li>Larger development team that can work on multiple projects.</li><li>Great developer experience, same as the one you have with the vanilla React Native setup.</li></ul><p>While every non-trivial project can easily fall into this category, there are several drawbacks to this approach that are worth considering:</p><ul><li>Using <em>git submodules</em> will have an impact on your workflow. We believe that the default behaviour of <em>git</em> in this kind of setup is pretty good and did not had any hurdles. Still, you need to understand how git is tracking the commits in relation to the pointers of the sub-modules.</li><li>The relations and impact of the transitive dependences still needs to be explored and checked. In our setups we sticked to having all the dependences in the main (top-most) project.</li></ul><p>Hope this setup will help you bring better structure to your React Native projects.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a3acbd92903" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>