<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anonymous Writings on Medium]]></title>
        <description><![CDATA[Stories by Anonymous Writings on Medium]]></description>
        <link>https://medium.com/@meradroidg6?source=rss-47d216b76453------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Anonymous Writings on Medium</title>
            <link>https://medium.com/@meradroidg6?source=rss-47d216b76453------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 02:46:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@meradroidg6/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Given a start time, end time and a step, generate a time series in scala]]></title>
            <link>https://medium.com/@meradroidg6/given-a-start-time-end-time-and-a-step-generate-a-time-series-in-scala-97347c0a0b62?source=rss-47d216b76453------2</link>
            <guid isPermaLink="false">https://medium.com/p/97347c0a0b62</guid>
            <category><![CDATA[scala]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Anonymous Writings]]></dc:creator>
            <pubDate>Fri, 02 Nov 2018 15:12:16 GMT</pubDate>
            <atom:updated>2018-11-02T15:13:55.802Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/493/1*eBAXaBIw_jyxHEPsnPxe4w.png" /></figure><p>Today, while I was working on a Spark job written in Scala. I was stuck at a particular problem of generating a set of `WHERE` clauses, that would be used to query a table in a database. The condition in the where clause was simple.</p><p>SELECT * FROM TABLE WHERE LAST_MODIFIED BETWEEN TIMESTAMP(START_TIME) AND TIMESTAMP(END_TIME)</p><p>Basically, I had to get all the records from a table whose last modified time was between start_time and end_time . Now, I wanted to generate a series of queries with incremented time without losing any time in between. To make it more clear, this is what I wanted to achieve.</p><pre>Given: </pre><pre>START TIME: 2018-01-01 00:00:00<br>END TIME: 2018-02-01 00:00:00<br>INCREMENT: 1 day</pre><pre>Output:</pre><pre>SELECT * FROM TABLE WHERE LAST_MODIFIED BETWEEN TIMESTAMP(&#39;2018-01-01 00:00:00&#39;) AND TIMESTAMP(&#39;2018-01-02 00:00:00&#39;);</pre><pre>SELECT * FROM TABLE WHERE LAST_MODIFIED BETWEEN TIMESTAMP(&#39;2018-01-02 00:00:00&#39;) AND TIMESTAMP(&#39;2018-01-03 00:00:00&#39;);</pre><pre>SELECT * FROM TABLE WHERE LAST_MODIFIED BETWEEN TIMESTAMP(&#39;2018-01-03 00:00:00&#39;) AND TIMESTAMP(&#39;2018-01-04 00:00:00&#39;);</pre><pre>....<br>.....<br>......<br>.......<br>........</pre><pre>SELECT * FROM TABLE WHERE LAST_MODIFIED BETWEEN TIMESTAMP(&#39;2018-01-31 00:00:00&#39;) AND TIMESTAMP(&#39;2018-02-01 00:00:00&#39;);</pre><p>Since I am new to Scala language, I first searched in stackoverflow, I found a few Java examples which were implemented using Joda library, but they were all done in a very Java’ish way. After a few minutes of searching, I stumbled upon Scala ranges. It really blew my mind as to how elegant scala API’s are designed. I kind of got the hint as to how to go about writing the logic. I opened up scala shell and started writing it. Here’s how I did it. I used natively available java.time package classes.</p><pre>val dateTimeFormatter = DateTimeFormatter.ofPattern(&quot;yyyy-MM-dd HH:mm:ss&quot;)</pre><pre>val startTime = LocalDateTime.parse(&quot;2018-01-01 00:00:00&quot;,dateTimeFormatter);</pre><pre>val endTime = LocalDateTime.parse(&quot;2018-02-01 00:00:00&quot;,dateTimeFormatter);</pre><pre>//Get the epoch time</pre><pre>val startTimeEpoch: Long = startTime.toInstant(ZoneOffset.of(&quot;+00:00&quot;)).toEpochMilli();</pre><pre>val endTimeEpoch: Long = endTime.toInstant(ZoneOffset.of(&quot;+00:00&quot;)).toEpochMilli();</pre><pre>//Generate time ranges using some scala sauce</pre><pre>val step: Long = 86400000</pre><pre>startTimeEpoch //Start time<br>   .to(endTimeEpoch) // Till end time<br>    .by(step) // Increment by step, i.e. A day</pre><pre>     /**<br>      Create an array of 2 by moving 1 position at a time<br>      E.g val a = Array(1,2,3,4)</pre><pre>            Output: Array(Array(1,2),Array(2,3),Array(3,4))</pre><pre>       */</pre><pre>     .sliding(2,1) <br>      .map(epochRanges =&gt; {<br>        s&quot;SELECT * FROM TABLE WHERE LAST_MODIFIED BETWEEN <br>         <strong>$</strong>{<br>           epochRanges<br>             .map(epoch =&gt;<br>                          s&quot;TIMESTAMP(&#39;<strong>$</strong>{<br>                            LocalDateTime.<em>ofInstant</em>(<br>                                Instant.<em>ofEpochMilli</em>(epoch),<br>                                ZoneId.<em>of</em>(&quot;UTC&quot;)<br>                            ).format(dateTimeFormatter)}&#39;)&quot;<br>                  ).reduce(_+&quot; AND &quot;+_)</pre><pre>       }<br>&quot;})</pre><pre>// Desired Output<br>SELECT * FROM TABLE WHERE LAST_MODIFIED BETWEEN TIMESTAMP(&#39;2018-01-01 00:00:00&#39;) AND TIMESTAMP(&#39;2018-01-02 00:00:00&#39;);</pre><pre>SELECT * FROM TABLE WHERE LAST_MODIFIED BETWEEN TIMESTAMP(&#39;2018-01-02 00:00:00&#39;) AND TIMESTAMP(&#39;2018-01-03 00:00:00&#39;);</pre><pre>SELECT * FROM TABLE WHERE LAST_MODIFIED BETWEEN TIMESTAMP(&#39;2018-01-03 00:00:00&#39;) AND TIMESTAMP(&#39;2018-01-04 00:00:00&#39;);<br>....<br>...<br>..<br>..<br>..<br>..</pre><p>I know, the method chaining looks terribly formatted. I did my best 😅. Copy and paste this code snippet into IntelliJ or any other IDE that you use. Things will become clear.</p><p>So what did here is used Scala’s range functions. You can read more about it <a href="https://alvinalexander.com/scala/how-to-use-range-class-in-scala-cookbook">here</a>.</p><p>It was really fun writing this code snippet. I learnt something new today 😃.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=97347c0a0b62" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>