<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mayur Prajapati on Medium]]></title>
        <description><![CDATA[Stories by Mayur Prajapati on Medium]]></description>
        <link>https://medium.com/@mayurprajapatiin?source=rss-589f5e99fc58------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*zYShlKW76bQJdpg1FFznYQ.jpeg</url>
            <title>Stories by Mayur Prajapati on Medium</title>
            <link>https://medium.com/@mayurprajapatiin?source=rss-589f5e99fc58------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 22:30:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mayurprajapatiin/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Solving the Cost vs. Viewport Problem for Windows UI Automation on Ephemeral Cloud VMs]]></title>
            <link>https://medium.com/@mayurprajapatiin/solving-the-cost-vs-viewport-problem-for-windows-ui-automation-on-ephemeral-cloud-vms-b4d6a52b0002?source=rss-589f5e99fc58------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4d6a52b0002</guid>
            <dc:creator><![CDATA[Mayur Prajapati]]></dc:creator>
            <pubDate>Sat, 15 Aug 2026 16:51:48 GMT</pubDate>
            <atom:updated>2026-08-17T16:47:25.602Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Automating 100+ Windows desktop applications across clients means paying for idle VMs and Windows licenses around the clock — or does it? Here’s how we used Google Compute Engine’s ephemeral VMs to cut costs, and why we had to fork the Jenkins plugin to solve the missing piece: RDP viewport connectivity for UI automation.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bcpGEZUemSarqys8" /></figure><h3>⚠️Problem</h3><p>Desktop based automation is still a challenging problem due to availability of required tools, and one of the most required thing is Viewport. Unlike Web apps, <strong>Desktop apps needs a screen</strong> to render components. <strong>They can’t run on headless mode.</strong></p><ul><li>One major thing is, use of Windows VMs, be it Azure, GCP or AWS — running a VM 24/7<strong> incurs cost along with Windows license</strong>.<br>While <a href="https://docs.cloud.google.com/compute/docs/instances/windows/ms-licensing"><strong>major cloud providers</strong></a> bundle the hourly Windows Server license fee directly into the running compute cost of the VM. And, that’s a relief.</li><li>GCE Plugin manages Agent connection using SSH — which means Headless mode for UI automation. <strong>This is what we need to solve.</strong></li><li>Another problem, <a href="https://appium.io/">Appium</a>, <a href="https://github.com/microsoft/winappdriver">WinAppDriver</a>, <a href="https://launchpad.net/sikuli">Sikuli</a>, <a href="https://github.com/FlaUI/FlaUI">FlaUI </a>— whichever you choose, at the end you still need a viewport.</li><li>For connecting to a viewport we need a dedicated 24/7 VM which operates a viewport connection service requests for any short-lived Windows VM.</li></ul><h3>💡Solution</h3><ul><li>Using GCP Compute Engine’s short-lived VMs along with modified <a href="https://plugins.jenkins.io/google-compute-engine/">Jenkins GCE Plugin</a> to connect to those VMs.</li><li>By default, GCE Plugin maintains <a href="https://github.com/jenkinsci/google-compute-engine-plugin/blob/develop/src/main/java/com/google/jenkins/plugins/computeengine/ComputeEngineWindowsLauncher.java">SSH</a> to connect with Jenkins. We’ll have to convert it to JNLP connection.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q9W2j8Z2gCeoJg9jjI_8cA.png" /></figure><ul><li>For viewport connection we’ll assume that there’s a REST service which automates RDP viewport connection.</li></ul><h4>Workflow</h4><ol><li>Jenkins job runs and it’s Node manager demands a new Node from GCE plugin.</li><li>Jenkins GCE plugin creates a new VM in GCP Compute Engine</li><li>Our custom code in Plugin sends a request to our RDP viewport connection service, which invokes RDP login and a startup script we’ll setup when configuring plugin</li><li>RDP login invokes our startup script which connects to Jenkins by launching a new interactive PowerShell window (Non-headless mode)</li><li>A new windows VM node gets connected to Jenkins and ready to run automation.</li></ol><h4>Let’s get started</h4><ol><li>Clone <a href="https://github.com/jenkinsci/google-compute-engine-plugin">Google CE Plugin</a> &amp; open folder in Editor</li><li>Do below changes</li></ol><ul><li>Modify <strong><em>ComputeEngineComputerLauncher</em></strong><em> </em><strong>and</strong><em> </em>extend <strong><em>JNLPLauncher</em></strong></li><li>Change constructor of <strong><em>ComputeEngineComputerLauncher</em></strong><em> </em>and add this line</li></ul><pre>// ComputeEngineComputerLauncher.java<br>public ComputeEngineComputerLauncher(<br>            String cloudName, String insertOperationId, String zone, boolean useInternalAddress,<br>            boolean launchJnlpAgentOnBoot) {<br>        super();<br>        super.setWebSocket(false); // set this to false<br>        this.cloudName = cloudName;<br>        this.insertOperationId = insertOperationId;<br>        this.zone = zone;<br>        this.useInternalAddress = useInternalAddress;<br>    }</pre><ul><li>Add a method which will be overridden by child class</li></ul><pre>// ComputeEngineComputerLauncher.java<br>  /**<br>   * Hook that is called after the instance reaches RUNNING and before agent<br>   * launch.<br>   * Default implementation does nothing; subclass may override.<br>   */<br>  protected void onInstanceRunning(ComputeEngineComputer computer, TaskListener listener) {<br>  }</pre><ul><li>Modify <em>launch(SlaveComputer slaveComputer, TaskListener listener)</em> method</li></ul><pre>@Override<br>    public void launch(SlaveComputer slaveComputer, TaskListener listener) {<br>  // .....<br>  // .....<br>  // .....<br>  onInstanceRunning(computer, listener); // call this in launch<br>}</pre><ul><li>Modify <em>private void launch(ComputeEngineComputer computer, TaskListener listener)</em></li><li>Focus on PS command which is responsible to create an env file which will be used by Startup Script to connect to Jenkins through Agent.</li></ul><pre>private void launch(ComputeEngineComputer computer, TaskListener listener) {<br>    ComputeEngineInstance node = computer.getNode();<br>    if (node == null) {<br>        logWarning(computer, listener, &quot;Could not get node from computer&quot;);<br>        return;<br>    }<br><br>    Connection conn = null;<br>    Optional&lt;Connection&gt; cleanupConn;<br>    PrintStream logger = listener.getLogger();<br>    logInfo(computer, listener, &quot;Launching instance: &quot; + node.getNodeName());<br>    Session sess = null;<br>    try {<br>        cleanupConn = setupConnection(node, computer, listener);<br>        if (!cleanupConn.isPresent()) {<br>            return;<br>        }<br>        conn = cleanupConn.get();<br>        String javaExecPath = node.getJavaExecPathOrDefault();<br>        if (!checkJavaInstalled(computer, conn, logger, listener, javaExecPath)) {<br>            return;<br>        }<br>        String jenkinsDir = node.getRemoteFS();<br>        copyAgentJar(computer, conn, listener, jenkinsDir);<br>        sess = conn.openSession();<br><br>        // Prepare Windows Startup script to launch JNLP agent on login/boot<br>        try {<br>            String nodeName = node.getNodeName();<br>            String rootUrl = Jenkins.get().getRootUrl();<br>            if (rootUrl == null || rootUrl.isEmpty()) {<br>                logInfo(computer, listener,<br>                        &quot;Jenkins Root URL is not configured; cannot create startup script.&quot;);<br>            } else {<br>                // Construct URLs and secret for this agent<br>                String normalizedRoot = rootUrl.endsWith(&quot;/&quot;) ? rootUrl.substring(0, rootUrl.length() - 1) : rootUrl;<br>                String secret = computer.getJnlpMac();<br><br>                // Execute PS script remotely to create env file, which will be used by startup script lateron<br>                String remoteAgentDir = &quot;C:\\path\\to\\remoting\\dir&quot;;<br>                String filePath = remoteAgentDir + &quot;\\env.env&quot;;<br>                String agentKey = secret;<br>                String agentName = nodeName;<br>                String serverAddress = normalizedRoot;<br><br>                String psCommand =<br>                        &quot;cmd.exe /C powershell.exe -NoProfile -NonInteractive -ExecutionPolicy Bypass -Command &quot;<br>                                + &quot;\&quot;$content = \\\&quot;AGENT_KEY=&quot; + agentKey<br>                                + &quot;`nAGENT_NAME=&quot; + agentName<br>                                + &quot;`nSERVERADDRESS=&quot; + serverAddress<br>                                + &quot;`n\\\&quot;; &quot;<br>                                + &quot;New-Item -Path &#39;&quot; + remoteAgentDir + &quot;&#39; -ItemType Directory -Force | Out-Null; &quot;<br>                                + &quot;Set-Content -Path &#39;&quot; + filePath + &quot;&#39; -Value $content -Encoding UTF8\&quot;&quot;;<br><br>                sess.execCommand(psCommand);<br>                try (InputStream psOut = sess.getStdout();<br>                        Scanner psScan = new Scanner(psOut, StandardCharsets.UTF_8.name())) {<br>                    if (psScan.useDelimiter(&quot;\\A&quot;).hasNext()) {<br>                        psScan.next();<br>                    }<br>                }<br>                logInfo(computer, listener, &quot;Configuration file created at: &quot; + filePath);<br>            }<br>        } catch (IOException ioe) {<br>            logException(computer, listener, &quot;Failed to create Windows startup script for Jenkins agent&quot;, ioe);<br>        }<br><br>        if (sess != null) {<br>            Session finalSess = sess;<br>            Connection finalConn = conn;<br>            computer.setChannel(sess.getStdout(), sess.getStdin(), logger, new Channel.Listener() {<br>                @Override<br>                public void onClosed(Channel channel, IOException cause) {<br>                    try {finalSess.close();} catch (Exception ex) {}<br>                    try {finalConn.close();} catch (Exception ex) {}<br>                }<br>            });<br>        } else {<br>            if (sess != null) {<br>                try {sess.close();} catch (Exception ex) {}<br>            }<br>            if (conn != null) {<br>                try {conn.close();} catch (Exception ex) {}<br>            }<br>            return;<br>        }<br><br>        return;<br>    } catch (Exception e) {<br>        if (sess != null) {<br>            try {sess.close();} catch (Exception ex) {}<br>        }<br>        if (conn != null) {<br>            try {conn.close();} catch (Exception ex) {}<br>        }<br>        logException(computer, listener, &quot;Error: &quot;, e);<br>    }<br>}</pre><ul><li>Now, modify another file: <em>ComputeEngineWindowsLauncher.</em></li><li>This is where main RDP Viewport connection service gets invoked</li></ul><pre>// ComputeEngineWindowsLauncher.java<br>@Override<br>protected void onInstanceRunning(ComputeEngineComputer computer, TaskListener listener) {<br>    ComputeEngineInstance node = computer.getNode();<br>    if (node == null) {<br>        logWarning(computer, listener, &quot;onInstanceRunning: node was null&quot;);<br>        return;<br>    }<br>    WindowsConfiguration windowsConfig = node.getWindowsConfig();<br>    if (windowsConfig == null) {<br>        return;<br>    }<br>    try {<br>        Instance instance = computer.refreshInstance();<br>        if (instance == null) {<br>            logWarning(computer, listener, &quot;onInstanceRunning: instance not found (terminated?)&quot;);<br>            return;<br>        }<br><br>        // Retrieve hostname and IPs<br>        String privateIp = &quot;&quot;;<br>        String publicIp = &quot;&quot;;<br>        if (instance.getNetworkInterfaces() != null &amp;&amp; !instance.getNetworkInterfaces().isEmpty()) {<br>            NetworkInterface nic = instance.getNetworkInterfaces().get(0);<br>            privateIp = nic.getNetworkIP();<br>            if (nic.getAccessConfigs() != null) {<br>                for (AccessConfig ac : nic.getAccessConfigs()) {<br>                    if (NetworkInterfaceIpStackMode.NAT_TYPE.equals(ac.getType())) {<br>                        publicIp = ac.getNatIP();<br>                    }<br>                }<br>            }<br>            if (nic.getIpv6AccessConfigs() != null &amp;&amp; publicIp.isEmpty()) {<br>                for (AccessConfig ac : nic.getIpv6AccessConfigs()) {<br>                    if (&quot;DIRECT_IPV6&quot;.equals(ac.getType())) {<br>                        // Prefer IPv4 if available, otherwise use IPv6 external<br>                        publicIp = ac.getExternalIpv6();<br>                    }<br>                }<br>            }<br>        }<br><br>        // Build payload: ipAddress expects public IP; fallback to private if no public<br>        String ipForPayload = publicIp != null &amp;&amp; !publicIp.isEmpty() ? publicIp : privateIp;<br>        String nodeName = node.getNodeName();<br>        String payload = &quot;{&lt;rdp connector payload&gt;}&quot;;<br>        sendJsonPost(<br>                &quot;https://&lt;endpoint-to-call-rdp-connector-service&gt;&quot;, payload, computer, listener, &quot;RDP Connector&quot;);<br>    } catch (Exception e) {<br>        logException(computer, listener, &quot;exception while calling microservice&quot;, e);<br>    }<br>}<br><br>private int sendJsonPost(<br>        String urlString,<br>        String payload,<br>        ComputeEngineComputer computer,<br>        TaskListener listener,<br>        String logContext)<br>        throws IOException {<br>    URL url = new URL(urlString);<br>    HttpURLConnection conn = (HttpURLConnection) url.openConnection();<br>    conn.setRequestProperty(&quot;Authorization&quot;, &quot;Bearer &lt;token here&gt;&quot;);<br>    conn.setRequestMethod(&quot;POST&quot;);<br>    conn.setConnectTimeout(15000);<br>    conn.setReadTimeout(30000);<br>    conn.setDoOutput(true);<br>    conn.setRequestProperty(&quot;Content-Type&quot;, &quot;application/json&quot;);<br>    try (OutputStream os = conn.getOutputStream()) {<br>        os.write(payload.getBytes(&quot;UTF-8&quot;));<br>    }<br>    int status = conn.getResponseCode();<br>    if (status &gt;= 200 &amp;&amp; status &lt; 300) {<br>        logInfo(<br>                computer,<br>                listener,<br>                String.format(&quot;%s: call succeeded for &#39;%s&#39; with HTTP %d. Response: %s&quot;, logContext, computer.getName(), status, conn.getResponseMessage()));<br>    } else {<br>        logWarning(<br>                computer,<br>                listener,<br>                String.format(&quot;%s: call failed for &#39;%s&#39; with HTTP %d. Response: %s&quot;, logContext, computer.getName(), status, conn.getResponseMessage()));<br>    }<br>    conn.disconnect();<br>    return status;<br>}<br><br>// ...<br>// ...<br>// ...<br><br>// Add below extension, or else there&#39;ll be an error<br>@Extension<br>public static final class DescriptorImpl extends JNLPLauncher.DescriptorImpl {<br>    @Override<br>    public String getDisplayName() {<br>        return &quot;Compute Engine Windows Launcher&quot;;<br>    }<br>}</pre><h4>All changes are done, Pack plugin as HPI file</h4><pre>mvn clean compile -Dspotless.check.skip=true<br>mvn hpi:hpi -Dspotless.check.skip=true</pre><p>Install plugin on Jenkins, Go to <em>Manage Jenkins &gt; Plugins &gt; Advanced Settings</em></p><p>Do not forget to set startup script in Cloud configuration. Set below startup script content</p><p>Below script will wait for <strong><em>.env</em> </strong>file to be created by plugin and when created it’ll use that file to connect current Computer to Jenkins. This script will be invoked when RDP viewport is connected to it.</p><pre># 1. Define paths<br>$StartupFolder = &quot;C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp&quot;<br>$RemotingDir = &quot;C:\path\to\remoting\dir&quot;<br>$BatFilePath = Join-Path -Path $RemotingDir -ChildPath &quot;agent.bat&quot;<br>$PSScriptPath = Join-Path -Path $RemotingDir -ChildPath &quot;agent.ps1&quot;<br>$ShortcutPath = Join-Path -Path $StartupFolder -ChildPath &quot;AgentStartup.lnk&quot;<br>$EnvFile = Join-Path -Path $RemotingDir -ChildPath &quot;env.env&quot;<br><br># Ensure folder exists<br>if (-not (Test-Path $RemotingDir)) {<br>    New-Item -ItemType Directory -Path $RemotingDir -Force | Out-Null<br>}<br><br># -----------------------------------------------------------<br># 2. Create the PowerShell Runner Script (agent.ps1)<br># -----------------------------------------------------------<br><br>$Ps1Content = @&#39;<br>Function Agent-Runner {<br> $AgentProps = convertfrom-stringdata (get-content &quot;C:\path\to\remoting\dir\env.env&quot; -raw)<br> echo &quot;INFO - Running agent&#39;&quot;<br>    curl.exe -sO &quot;$($AgentProps.SERVERADDRESS)/jnlpJars/agent.jar&quot;<br>    java -jar agent.jar -url $AgentProps.SERVERADDRESS -secret $AgentProps.AGENT_KEY -name $AgentProps.AGENT_NAME -webSocket -workDir &quot;c:\jenkins&quot;<br>}<br><br>Function Run {<br>    while($true) {<br>        try {<br>            Agent-Runner<br>            Timeout /T 60<br>        }<br>        catch {<br>            $string_err = $_ | Out-String<br>            write-host $string_err<br>        }<br>    }<br>}<br><br>Run<br>&#39;@<br><br>$Ps1Content | Out-File -FilePath $PSScriptPath -Encoding UTF8 -Force<br><br><br># -----------------------------------------------------------<br># 3. Create agent.bat — infinite loop waiting for env.env<br># -----------------------------------------------------------<br><br>$BatContent = @&quot;<br>@echo off<br>cd /d C:\path\to\remoting\dir<br><br>set MAX_WAIT_SECONDS=600<br>set INTERVAL_SECONDS=2<br>set /a ELAPSED=0<br><br>echo ==========================================<br>echo   Agent Auto Launcher<br>echo   VM Name: %COMPUTERNAME%<br>echo   Time: %DATE% %TIME%<br>echo ==========================================<br><br>:WAIT_LOOP<br>if exist env.env (<br>    echo env.env found. Launching PowerShell agent.ps1...<br>    powershell -WindowStyle Minimized -ExecutionPolicy Bypass -File &quot;C:\path\to\remoting\dir\agent.ps1&quot;<br>    goto END<br>)<br><br>echo Waiting for env file to be created...<br>timeout /t %INTERVAL_SECONDS% &gt; nul<br>set /a ELAPSED+=%INTERVAL_SECONDS%<br><br>if %ELAPSED% GEQ %MAX_WAIT_SECONDS% (<br>    echo Waited 10 minutes. env.env not found. Exiting.<br>    goto END<br>)<br><br>goto WAIT_LOOP<br><br>:END<br>pause<br>&quot;@<br><br>$BatContent | Out-File -FilePath $BatFilePath -Encoding ASCII -Force<br><br><br># -----------------------------------------------------------<br># 4. Create shortcut in Startup<br># -----------------------------------------------------------<br><br>$WshShell = New-Object -ComObject WScript.Shell<br>$Shortcut = $WshShell.CreateShortcut($ShortcutPath)<br>$Shortcut.TargetPath = $BatFilePath<br>$Shortcut.WorkingDirectory = $RemotingDir<br>$Shortcut.IconLocation = &quot;C:\Windows\System32\cmd.exe&quot;<br>$Shortcut.Save()<br><br>Write-Host &quot;BAT file created at: $BatFilePath&quot;<br>Write-Host &quot;PS1 file created at: $PSScriptPath&quot;<br>Write-Host &quot;Shortcut created at: $ShortcutPath&quot;</pre><h3>Conclusion</h3><p>By forking the GCE plugin’s launcher to <strong>swap SSH/headless JNLP</strong> for a triggered RDP session, we turned a 24/7 Windows VM cost into an on-demand one — <strong>the viewport only exists for the lifetime of the automation run</strong> itself. It’s a fairly small, targeted change (a new hook, a modified launcher, and a startup script that waits on an env file), but it removes the single biggest blocker to running desktop UI automation on ephemeral cloud infrastructure: the assumption that a GUI needs a machine that’s always on. The same pattern should generalize to any CI system where GCE, Azure, or AWS ephemeral VMs are in play — the specifics of the Jenkins plugin will change, but the core idea (trigger a real login session on boot instead of connecting headlessly) carries over.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4d6a52b0002" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How TimescaleDB Compression Cut a 152M-Row Query from 27s to 0.8s]]></title>
            <link>https://medium.com/@mayurprajapatiin/how-timescaledb-compression-cut-a-152m-row-query-from-27s-to-0-8s-e1b25242f0da?source=rss-589f5e99fc58------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1b25242f0da</guid>
            <category><![CDATA[time-series-data]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[timescaledb]]></category>
            <category><![CDATA[sql]]></category>
            <dc:creator><![CDATA[Mayur Prajapati]]></dc:creator>
            <pubDate>Sat, 08 Aug 2026 18:07:07 GMT</pubDate>
            <atom:updated>2026-08-14T07:51:29.624Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GN9xYP72kWh_m9_VyfzTag.jpeg" /></figure><p>A practical guide to optimizing a 152M-row timeseries Postgres table storing 1-minute OHLC candle data using TimescaleDB compression — cutting a 27-second query down to 0.8 seconds.</p><h3>⚠️Problem &amp; Diagnosis 🔍</h3><p>Here we go,</p><p>groww_option_candles_1_min table contains multiple NSE Stock Option and Index Option contracts for year 2023–2026.</p><blockquote>Below is a query to fetch OHLC data from 1 min candles table. It’s having total 152,938,137 rows.</blockquote><pre>SELECT <br>    date_trunc(&#39;week&#39;, openingtime) AS week,<br>    contractid,<br>    MAX(high) AS high,<br>    MIN(low) AS low,<br>    SUM(volume) AS volume<br>FROM groww_option_candles_1_min<br>WHERE contractid LIKE &#39;BANKNIFTY%&#39; <br>  AND openingtime &gt;= &#39;2023-01-01&#39; AND openingtime &lt; &#39;2024-01-01&#39;<br>GROUP BY 1, 2<br>ORDER BY 1, 2;<br><br>-- This query takes 27 seconds, which is not better for performance</pre><blockquote>Let’s get under the hood, debug and understand what’s happening</blockquote><pre>GroupAggregate  (cost=2661367.98..2891402.05 rows=1500670 width=51) (actual time=19043.270..20803.583 rows=5214.00 loops=1)<br>  Output: (date_trunc(&#39;week&#39;::text, openingtime)), contractid, max(high), min(low), sum(volume)<br>  Group Key: (date_trunc(&#39;week&#39;::text, groww_option_candles_1_min.openingtime)), groww_option_candles_1_min.contractid<br>  Buffers: shared hit=6638 read=2178463, temp read=14900 written=14907<br>  -&gt;  Gather Merge  (cost=2661367.98..2853001.59 rows=1571367 width=51) (actual time=19043.250..20244.575 rows=1852949.00 loops=1)<br>        Output: (date_trunc(&#39;week&#39;::text, openingtime)), contractid, high, low, volume<br>        Workers Planned: 6<br>        Workers Launched: 6<br>        Buffers: shared hit=6638 read=2178463, temp read=14900 written=14907<br>        -&gt;  Sort  (cost=2660367.88..2661022.61 rows=261894 width=51) (actual time=18887.414..18943.876 rows=264707.00 loops=7)<br>              Output: (date_trunc(&#39;week&#39;::text, openingtime)), contractid, high, low, volume<br>              Sort Key: (date_trunc(&#39;week&#39;::text, groww_option_candles_1_min.openingtime)), groww_option_candles_1_min.contractid<br>              Sort Method: external merge  Disk: 21624kB<br>              Buffers: shared hit=6638 read=2178463, temp read=14900 written=14907<br>              Worker 0:  actual time=18927.702..18993.014 rows=282876.00 loops=1<br>                Sort Method: external merge  Disk: 18208kB<br>                Buffers: shared hit=685 read=309900, temp read=2276 written=2277<br>              Worker 1:  actual time=18938.664..19004.025 rows=287508.00 loops=1<br>                Sort Method: external merge  Disk: 18512kB<br>                Buffers: shared hit=453 read=312929, temp read=2314 written=2315<br>              Worker 2:  actual time=18813.002..18869.134 rows=241580.00 loops=1<br>                Sort Method: external merge  Disk: 15512kB<br>                Buffers: shared hit=1152 read=310104, temp read=1939 written=1940<br>              Worker 3:  actual time=18771.284..18809.565 rows=200560.00 loops=1<br>                Sort Method: external merge  Disk: 12896kB<br>                Buffers: shared hit=2507 read=307376, temp read=1612 written=1613<br>              Worker 4:  actual time=18807.480..18861.749 rows=226151.00 loops=1<br>                Sort Method: external merge  Disk: 14552kB<br>                Buffers: shared hit=1010 read=308399, temp read=1819 written=1820<br>              Worker 5:  actual time=18911.198..18973.052 rows=278305.00 loops=1<br>                Sort Method: external merge  Disk: 17896kB<br>                Buffers: shared hit=421 read=311024, temp read=2237 written=2238<br>              -&gt;  Parallel Seq Scan on public.groww_option_candles_1_min  (cost=0.00..2631555.32 rows=261894 width=51) (actual time=11582.572..18495.395 rows=264707.00 loops=7)<br>                    Output: date_trunc(&#39;week&#39;::text, openingtime), contractid, high, low, volume<br>                    Filter: (((groww_option_candles_1_min.contractid)::text ~~ &#39;BANKNIFTY%&#39;::text) AND (groww_option_candles_1_min.openingtime &gt;= &#39;2023-01-01 00:00:00&#39;::timestamp without time zone) AND (groww_option_candles_1_min.openingtime &lt; &#39;2024-01-01 00:00:00&#39;::timestamp without time zone))<br>                    Rows Removed by Filter: 21583598<br>                    Buffers: shared hit=6368 read=2178463<br>                    Worker 0:  actual time=11528.648..18483.558 rows=282876.00 loops=1<br>                      Buffers: shared hit=640 read=309900<br>                    Worker 1:  actual time=11533.114..18501.248 rows=287508.00 loops=1<br>                      Buffers: shared hit=408 read=312929<br>                    Worker 2:  actual time=11598.585..18447.714 rows=241580.00 loops=1<br>                      Buffers: shared hit=1107 read=310104<br>                    Worker 3:  actual time=11528.296..18496.137 rows=200560.00 loops=1<br>                      Buffers: shared hit=2462 read=307376<br>                    Worker 4:  actual time=11625.690..18474.302 rows=226151.00 loops=1<br>                      Buffers: shared hit=965 read=308399<br>                    Worker 5:  actual time=11602.499..18507.620 rows=278305.00 loops=1<br>                      Buffers: shared hit=376 read=311024<br>Planning Time: 0.172 ms<br>Execution Time: 20814.720 ms</pre><ul><li>Postgres used <strong>6 parallel workers</strong> to sequentially scan the <em>entire table</em>.</li><li>Total rows scanned ≈ 21,583,598 (removed) + 1,852,949 (kept) ≈ <strong>23.4 million rows</strong>, and only ~1.85M matched in filter (~8% selectivity).</li><li>Buffers: shared hit=6368 read=2178463 — only 6,368 pages (≈50MB) came from cache; <strong>2,178,463 pages (≈17GB) were read from disk</strong>. That&#39;s the dominant cost. Notice that the scan&#39;s <em>actual time</em> starts at 11582 — over 11 seconds pass before the first row even comes out, which is classic &quot;waiting on disk I/O&quot; behavior, not CPU-bound filtering.</li></ul><h3>🧩Root Cause</h3><blockquote>Understand why this breaks PostgreSQL and how <a href="https://github.com/timescale/timescaledb">TimescaleDB PostgreSQL Extension</a> helps to solve this.</blockquote><p><strong>Why this breaks PostgreSQL:</strong></p><p>Even with indexes, PostgreSQL now has to fetch millions of rows from disk into memory because LIKE &#39;BANKNIFTY%&#39; matches hundreds of contracts spread across the whole table. A prefix match like this <em>can</em> use a btree index in theory, but at ~8% selectivity (1.85M of 23.4M scanned rows match), the planner reasonably decides a sequential scan is cheaper than random-accessing an index over that many matching rows. The real problem isn&#39;t the LIKE itself — it&#39;s that a 152M-row, row-oriented table forces Postgres to read every column of every candidate row from disk just to check three conditions and aggregate five columns. That&#39;s a massive I/O bottleneck no index rewrite fixes on its own.</p><h3>🛠️Solution</h3><p><strong>How TimescaleDB can improve this:</strong></p><p>A standard TimescaleDB would improve performance due to its <strong>columnar compression</strong>.<br><strong>Standard Postgres stores data row-by-row</strong>. <strong>Compressed TimescaleDB stores data column-by-column.</strong><br>When you run this query on a compressed hypertable, TimescaleDB completely ignores the unused columns (like open, close, etc.) and reads highly compressed arrays of high, low, and volume directly from disk.</p><h4>Installation of TimescaleDB</h4><p><a href="https://towardsdev.com/timescaledb-for-postgresql-a-dbas-guide-to-time-series-data-performance-82f105421887">TimescaleDB for PostgreSQL: A DBA&#39;s Guide to Time-Series Data Performance</a></p><p>After installation, enable TDB extension.</p><pre>-- Enable extension<br>CREATE EXTENSION IF NOT EXISTS timescaledb;<br><br>-- Convert table to TDB hypertable (This process can take 3+ hours)<br>SELECT create_hypertable(<br>    &#39;groww_option_candles_1_min&#39;, <br>    &#39;openingtime&#39;, <br>    migrate_data =&gt; true<br>);</pre><p><strong>To see the massive performance difference, need to turn on compression.</strong></p><p>Tell TimescaleDB to group all the minute-candles for a specific options contract together before compressing them</p><pre>ALTER TABLE groww_option_candles_1_min SET (<br>    timescaledb.compress,<br>    timescaledb.compress_segmentby = &#39;contractid&#39;,<br>    timescaledb.compress_orderby = &#39;openingtime DESC&#39;<br>);<br>-- This can improve searching for contractid column, such as &quot;like &#39;BANKNIFTY%&#39;&quot; clauses</pre><p><strong>Manually Compress Existing Data</strong></p><blockquote>Enabling the compression settings in the previous step does not automatically compress old data. To see the performance difference in A/B test immediately, you must manually compress the existing chunks.</blockquote><pre>SELECT compress_chunk(i.chunk_schema || &#39;.&#39; || i.chunk_name)<br>FROM timescaledb_information.chunks i<br>WHERE hypertable_name = &#39;groww_option_candles_1_min&#39;<br>  AND is_compressed = false;<br>-- took 8 mins</pre><h3>📊Results</h3><h4>Try running the same query again</h4><pre>SELECT <br>    date_trunc(&#39;week&#39;, openingtime) AS week,<br>    contractid,<br>    MAX(high) AS high,<br>    MIN(low) AS low,<br>    SUM(volume) AS volume<br>FROM groww_option_candles_1_min<br>WHERE contractid LIKE &#39;BANKNIFTY%&#39; <br>  AND openingtime &gt;= &#39;2023-01-01&#39; AND openingtime &lt; &#39;2024-01-01&#39;<br>GROUP BY 1, 2<br>ORDER BY 1, 2;<br><br>| Metric               | Before (uncompressed)  | After (compressed)         |<br>|--------------------- |------------------------|----------------------------|<br>| Execution time       | 20,814.72 ms           | 811.88 ms                  |<br>| Buffers from cache   | 6,368 pages (~50MB)    | 52,823 pages (all cache)   |<br>| Buffers from disk    | 2,178,463 pages (~17GB)| 0                          |<br>| Rows scanned         | ~23.4M                 | ~1.85M (via chunk pruning) |<br><br>-- Now it takes 0.8 seconds — a ~27x improvement</pre><blockquote>This explains how it has done optimization under the hood</blockquote><pre>Sort  (cost=144889.22..145352.40 rows=185271 width=52) (actual time=793.580..793.773 rows=5214.00 loops=1)<br>  Output: (date_trunc(&#39;week&#39;::text, groww_option_candles_1_min.openingtime)), groww_option_candles_1_min.contractid, (max(groww_option_candles_1_min.high)), (min(groww_option_candles_1_min.low)), (sum(groww_option_candles_1_min.volume))<br>  Sort Key: (date_trunc(&#39;week&#39;::text, groww_option_candles_1_min.openingtime)), groww_option_candles_1_min.contractid<br>  Sort Method: quicksort  Memory: 559kB<br>  Buffers: shared hit=52823<br>  -&gt;  Finalize HashAggregate  (cost=126362.79..128678.68 rows=185271 width=52) (actual time=778.917..780.146 rows=5214.00 loops=1)<br>        Output: (date_trunc(&#39;week&#39;::text, groww_option_candles_1_min.openingtime)), groww_option_candles_1_min.contractid, max(groww_option_candles_1_min.high), min(groww_option_candles_1_min.low), sum(groww_option_candles_1_min.volume)<br>        Group Key: (date_trunc(&#39;week&#39;::text, groww_option_candles_1_min.openingtime)), groww_option_candles_1_min.contractid<br>        Batches: 1  Memory Usage: 4633kB<br>        Buffers: shared hit=52823<br>        -&gt;  Append  (cost=8736.04..91624.48 rows=2779065 width=52) (actual time=145.015..776.098 rows=8859.00 loops=1)<br>              Buffers: shared hit=52823<br>              -&gt;  Partial HashAggregate  (cost=8736.04..11051.92 rows=185271 width=52) (actual time=145.014..145.702 rows=1457.00 loops=1)<br>                    Output: (date_trunc(&#39;week&#39;::text, _hyper_1_77_chunk.openingtime)), _hyper_1_77_chunk.contractid, PARTIAL max(_hyper_1_77_chunk.high), PARTIAL min(_hyper_1_77_chunk.low), PARTIAL sum(_hyper_1_77_chunk.volume)<br>                    Group Key: date_trunc(&#39;week&#39;::text, _hyper_1_77_chunk.openingtime), _hyper_1_77_chunk.contractid<br>                    Batches: 1  Memory Usage: 4377kB<br>                    Buffers: shared hit=8086<br>                    -&gt;  Custom Scan (ColumnarScan) on _timescaledb_internal._hyper_1_77_chunk  (cost=4.01..3983.73 rows=380184 width=52) (actual time=0.048..77.690 rows=355632.00 loops=1)<br>                          Output: date_trunc(&#39;week&#39;::text, _hyper_1_77_chunk.openingtime), _hyper_1_77_chunk.contractid, _hyper_1_77_chunk.high, _hyper_1_77_chunk.low, _hyper_1_77_chunk.volume<br>                          Bulk Decompression: true<br>                          Buffers: shared hit=8086<br>                          -&gt;  Seq Scan on _timescaledb_internal.compress_hyper_2_196_chunk  (cost=0.00..181.89 rows=993 width=168) (actual time=0.030..0.519 rows=978.00 loops=1)<br>                                Output: compress_hyper_2_196_chunk._ts_meta_count, compress_hyper_2_196_chunk.contractid, compress_hyper_2_196_chunk._ts_meta_min_1, compress_hyper_2_196_chunk._ts_meta_max_1, compress_hyper_2_196_chunk.openingtime, compress_hyper_2_196_chunk.close, compress_hyper_2_196_chunk.createdat, compress_hyper_2_196_chunk.high, compress_hyper_2_196_chunk.low, compress_hyper_2_196_chunk.open, compress_hyper_2_196_chunk.updatedat, compress_hyper_2_196_chunk.volume<br>                                Filter: ((compress_hyper_2_196_chunk.contractid)::text ~~ &#39;BANKNIFTY%&#39;::text)<br>                                Rows Removed by Filter: 1413<br>                                Buffers: shared hit=152<br>              -&gt;  Partial HashAggregate  (cost=11558.88..13874.77 rows=185271 width=52) (actual time=221.671..222.417 rows=1874.00 loops=1)<br>                    Output: (date_trunc(&#39;week&#39;::text, _hyper_1_78_chunk.openingtime)), _hyper_1_78_chunk.contractid, PARTIAL max(_hyper_1_78_chunk.high), PARTIAL min(_hyper_1_78_chunk.low), PARTIAL sum(_hyper_1_78_chunk.volume)<br>                    Group Key: date_trunc(&#39;week&#39;::text, _hyper_1_78_chunk.openingtime), _hyper_1_78_chunk.contractid<br>                    Batches: 1  Memory Usage: 4377kB<br>                    Buffers: shared hit=11631<br>                    -&gt;  Custom Scan (ColumnarScan) on _timescaledb_internal._hyper_1_78_chunk  (cost=4.30..5266.59 rows=503384 width=52) (actual time=0.108..119.439 rows=532313.00 loops=1)<br>                          Output: date_trunc(&#39;week&#39;::text, _hyper_1_78_chunk.openingtime), _hyper_1_78_chunk.contractid, _hyper_1_78_chunk.high, _hyper_1_78_chunk.low, _hyper_1_78_chunk.volume<br>                          Bulk Decompression: true<br>                          Buffers: shared hit=11631<br>                          -&gt;  Seq Scan on _timescaledb_internal.compress_hyper_2_197_chunk  (cost=0.00..232.75 rows=1224 width=168) (actual time=0.048..0.718 rows=1308.00 loops=1)<br>                                Output: compress_hyper_2_197_chunk._ts_meta_count, compress_hyper_2_197_chunk.contractid, compress_hyper_2_197_chunk._ts_meta_min_1, compress_hyper_2_197_chunk._ts_meta_max_1, compress_hyper_2_197_chunk.openingtime, compress_hyper_2_197_chunk.close, compress_hyper_2_197_chunk.createdat, compress_hyper_2_197_chunk.high, compress_hyper_2_197_chunk.low, compress_hyper_2_197_chunk.open, compress_hyper_2_197_chunk.updatedat, compress_hyper_2_197_chunk.volume<br>                                Filter: ((compress_hyper_2_197_chunk.contractid)::text ~~ &#39;BANKNIFTY%&#39;::text)<br>                                Rows Removed by Filter: 1952<br>                                Buffers: shared hit=192<br>              -&gt;  Partial HashAggregate  (cost=1349.93..3665.82 rows=185271 width=52) (actual time=22.830..23.368 rows=456.00 loops=1)<br>                    Output: (date_trunc(&#39;week&#39;::text, _hyper_1_79_chunk.openingtime)), _hyper_1_79_chunk.contractid, PARTIAL max(_hyper_1_79_chunk.high), PARTIAL min(_hyper_1_79_chunk.low), PARTIAL sum(_hyper_1_79_chunk.volume)<br>                    Group Key: date_trunc(&#39;week&#39;::text, _hyper_1_79_chunk.openingtime), _hyper_1_79_chunk.contractid<br>                    Batches: 1  Memory Usage: 4185kB<br>                    Buffers: shared hit=1695<br>                    -&gt;  Custom Scan (ColumnarScan) on _timescaledb_internal._hyper_1_79_chunk  (cost=2.49..676.95 rows=53838 width=52) (actual time=0.050..12.526 rows=54573.00 loops=1)<br>                          Output: date_trunc(&#39;week&#39;::text, _hyper_1_79_chunk.openingtime), _hyper_1_79_chunk.contractid, _hyper_1_79_chunk.high, _hyper_1_79_chunk.low, _hyper_1_79_chunk.volume<br>                          Bulk Decompression: true<br>                          Buffers: shared hit=1695<br>                          -&gt;  Seq Scan on _timescaledb_internal.compress_hyper_2_198_chunk  (cost=0.00..138.56 rows=272 width=168) (actual time=0.032..0.229 rows=274.00 loops=1)<br>                                Output: compress_hyper_2_198_chunk._ts_meta_count, compress_hyper_2_198_chunk.contractid, compress_hyper_2_198_chunk._ts_meta_min_1, compress_hyper_2_198_chunk._ts_meta_max_1, compress_hyper_2_198_chunk.openingtime, compress_hyper_2_198_chunk.close, compress_hyper_2_198_chunk.createdat, compress_hyper_2_198_chunk.high, compress_hyper_2_198_chunk.low, compress_hyper_2_198_chunk.open, compress_hyper_2_198_chunk.updatedat, compress_hyper_2_198_chunk.volume<br>                                Filter: ((compress_hyper_2_198_chunk.contractid)::text ~~ &#39;BANKNIFTY%&#39;::text)<br>                                Rows Removed by Filter: 571<br>                                Buffers: shared hit=128<br>              -&gt;  13 more as above...<br>Planning:<br>  Buffers: shared hit=379<br>Planning Time: 3.421 ms<br>Execution Time: 811.881 ms</pre><h3>⚖️Trade-offs</h3><blockquote>Compression isn’t free, and it’s worth knowing the costs before reaching for it:</blockquote><ul><li><strong>Updates/deletes get expensive. — </strong>Compressed chunks are effectively read-only — updating or deleting rows in a compressed chunk requires decompressing it first.</li><li><strong>New data needs a policy.</strong> Manually running compress_chunks()only fixes existing data. Without a add_compression_policy()job, every new chunk stays uncompressed and you’re back to slow scans on recent data.</li><li><strong>Point lookups can get slower. — </strong>Compression is optimized for scanning many rows across few columns. Single-row lookups by primary key can actually regress compared to an uncompressed, well-indexed table.</li></ul><h3>📌Takeaways</h3><p>— The table is converted to TimescaleDB’s hypertable physically split into time-based <strong>chunks</strong> (_hyper_1_77_chunk … _hyper_1_95_chunk, likely monthly)</p><p>— TimescaleDB compression is a good fit when you have append-mostly time-series data, wide tables where analytical queries touch few columns, and can tolerate periodic compression jobs; less ideal for tables with frequent updates to old rows.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1b25242f0da" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automating Java Code Generation, Git, and GitLab Merge Requests using Java]]></title>
            <link>https://medium.com/@mayurprajapatiin/automating-java-code-generation-git-and-gitlab-merge-requests-using-java-1e11f1a4bf45?source=rss-589f5e99fc58------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e11f1a4bf45</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[java-poet]]></category>
            <category><![CDATA[gitlab4j]]></category>
            <category><![CDATA[jgit]]></category>
            <category><![CDATA[automation]]></category>
            <dc:creator><![CDATA[Mayur Prajapati]]></dc:creator>
            <pubDate>Sat, 30 Dec 2023 17:59:11 GMT</pubDate>
            <atom:updated>2023-12-30T17:59:11.987Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Glimpse!</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*nyA734yhmBEVQlBgPctz1g.gif" /><figcaption>Final output!</figcaption></figure><p>This procedure becomes essential at the CI/CD level, particularly in environments where numerous projects are concurrently underway within a company. Establishing interconnections between projects becomes imperative to ensure seamless coordination and integration throughout the development lifecycle.</p><blockquote>Consider the scenario where a Python microservice project continually updates a database with information.<br>In response to these updates, another team is tasked with making regular code modifications to a separate project to ensure alignment with the evolving data landscape. This necessitates a robust process for maintaining synchronization and coherence between the two projects.</blockquote><h4>Certainly! Let’s start by creating a simple Java project that performs the specified actions.</h4><h4>Pre-requisites</h4><ul><li>You’ll need to update props.properties file before running this project</li><li>Java ≥ 11 &amp; Maven is required</li></ul><blockquote>props.properties</blockquote><pre>gitlabUsername=&lt;Your gitlab username&gt;<br>gitlabPersonalAccessToken=&lt;Gitlab PAT&gt;<br>gitlabProjectId=53433329<br>gitlabMrAssigneeId=8650933<br>mainBranchName=main<br>gitlabRemoteRepoUrl=https://gitlab.com/mayurengineer/example-java-project.git</pre><ol><li>Generate Java Code</li><li>Clone <a href="https://gitlab.com/mayurengineer/example-java-project">this</a> Repository and Checkout a New Branch</li><li>Commit using JGit</li><li>Create a Merge Request on GitLab using GitLab4j</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*-8wFCZuub0fOX8VzE6-5yw.gif" /></figure><p>We’ll use <a href="https://github.com/square/javapoet">JavaPoet</a>, <a href="https://github.com/gitlab4j/gitlab4j-api">Gitlab Java Client (GitLab4j)</a> &amp; <a href="https://www.eclipse.org/jgit/">Git Java Client (JGit)</a> as Maven dependencies</p><blockquote>pom.xml</blockquote><pre>  &lt;!-- GitLab Java Client --&gt;<br>  &lt;dependency&gt;<br>   &lt;groupId&gt;com.github.gitlab4j&lt;/groupId&gt;<br>   &lt;artifactId&gt;gitlab4j-api&lt;/artifactId&gt;<br>   &lt;version&gt;main-SNAPSHOT&lt;/version&gt;<br>  &lt;/dependency&gt;<br>  &lt;!-- Java Code Generation --&gt;<br>  &lt;dependency&gt;<br>   &lt;groupId&gt;com.squareup&lt;/groupId&gt;<br>   &lt;artifactId&gt;javapoet&lt;/artifactId&gt;<br>   &lt;version&gt;1.13.0&lt;/version&gt;<br>  &lt;/dependency&gt;<br>  &lt;!-- Git Java Client --&gt;<br>  &lt;dependency&gt;<br>   &lt;groupId&gt;org.eclipse.jgit&lt;/groupId&gt;<br>   &lt;artifactId&gt;org.eclipse.jgit&lt;/artifactId&gt;<br>   &lt;version&gt;3.5.0.201409260305-r&lt;/version&gt;<br>  &lt;/dependency&gt;</pre><h3>Steps Expanded!</h3><blockquote><strong>1. Generate Java Code</strong></blockquote><pre>private static String generateJava() {<br>  String className = &quot;Main&quot;;<br>  Builder builderClass = TypeSpec.classBuilder(className);<br><br>  int a = random(0, Integer.MAX_VALUE);<br>  int b = random(0, Integer.MAX_VALUE);<br><br>  MethodSpec methodSpec = buildSumMethod(a, b);<br>  builderClass.addMethod(methodSpec);<br><br>  JavaFile.Builder javaFile = JavaFile.builder(&quot;com.example&quot;, builderClass.build());<br><br>  // convert our work into java source<br>  return javaFile.build().toString();<br> }</pre><blockquote>2. Clone Repository and Checkout a New Branch</blockquote><pre>private static Git gitCloneRepositoryAndCheckoutBranch()<br>   throws IOException, GitAPIException, InvalidRemoteException, TransportException, RefAlreadyExistsException,<br>   RefNotFoundException, InvalidRefNameException, CheckoutConflictException {<br>  // delete existing cloned repository<br>  File dir = new File(projectDir);<br>  if (dir.exists()) {<br>   FileUtils.delete(dir, FileUtils.RECURSIVE);<br>  }<br><br>  Git git = Git.cloneRepository().setCredentialsProvider(gitCreds).setURI(gitlabRemoteRepoUrl).setDirectory(dir)<br>    .call();<br><br>  git.branchCreate().setName(branch).call();<br>  git.checkout().setName(branch).call();<br><br>  return git;<br> }</pre><blockquote>3. Commit using JGit</blockquote><pre>private static void gitCommit(Git git)<br>   throws GitAPIException, NoFilepatternException, NoHeadException, NoMessageException, UnmergedPathsException,<br>   ConcurrentRefUpdateException, WrongRepositoryStateException, InvalidRemoteException, TransportException {<br>  git.add().addFilepattern(&quot;.&quot;).call();<br>  git.commit().setMessage(commitMsg).call();<br>  git.push().setCredentialsProvider(gitCreds).call();<br>  git.getRepository().close();<br><br>  System.out.println(&quot;Code commit was successful to git&quot;);<br> }</pre><blockquote>4. Create a GitLab MR</blockquote><pre>GitlabHelper gitlab = new GitlabHelper(gitlabPersonalAccessToken, gitlabProjectId);<br>gitlab.createMergeRequest(branch, mainBranchName, &quot;Adding sum method&quot;, commitMsg, gitlabMrAssigneeId);</pre><blockquote>This also uses helper methods and can be accessed from below GitHub repository.</blockquote><p><a href="https://github.com/mayurprajapati/java-code-generation">GitHub - mayurprajapati/java-code-generation: Automating Java Code Generation, Git, and GitLab Merge Requests using Java</a></p><p>Thank You!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e11f1a4bf45" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Efficient TCP Tunneling using WebSockets: A Step-by-Step Guide]]></title>
            <link>https://medium.com/@mayurprajapatiin/efficiently-transfer-http-traffic-using-websockets-a-step-by-step-guide-3f2991b591f7?source=rss-589f5e99fc58------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f2991b591f7</guid>
            <category><![CDATA[proxy]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[https]]></category>
            <category><![CDATA[selenium]]></category>
            <category><![CDATA[websocket]]></category>
            <dc:creator><![CDATA[Mayur Prajapati]]></dc:creator>
            <pubDate>Fri, 21 Apr 2023 13:09:46 GMT</pubDate>
            <atom:updated>2023-11-21T16:43:49.639Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Explanatory steps to create a Web Socket service, Proxy Client &amp; Proxy Server using which one will be able to transfer HTTP traffic back and forth.</blockquote><h4>Use Cases</h4><p>This can be thoroughly used in following scenarios …<br>1. A VPN connection is required to access a particular service<br>2. In Selenium, a website/desktop app/mobile app can only be accessed from client system<br>3. Can be used as a proxy service which hides your public IP<br>4. And, Many more, just can’t think of 😉 …</p><h4>Visuals</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*UYXKm7HqFkRBn5yhrkWIAQ.png" /></figure><ul><li><strong>Websocket Server</strong>: A public Websocket server which acts as a bridge between Proxy Server and Proxy Client and can be accessed easily.</li><li><strong>Proxy HTTP Server: </strong>A local HTTP server and Websocket Client running on User’s machine, which receives HTTP requests, sends back to Websocket Server and waits for the response.</li><li><strong>Proxy HTTP Client: </strong>An HTTP Client and Websocket Client running on a machine, from where the Actual HTTP Server can be accessed.<br>Receives HTTP requests wrapped in Websocket Message and sends back to Actual HTTP Server and then sends back the response to Websocket Server.</li></ul><h3>Let’s dig deeper 🌊</h3><h4>A working Proxy Connection &amp; Bridge …</h4><ul><li>Connection from both Proxies to Websocket Server has to be established. Which creates a working bridge between both Proxies.</li><li>To make a connection to the Websocket Server<br> — The Proxy HTTP Client has to connect first with an HTTP header containing an UUID.<br> — Then, the Proxy HTTP Server has to connect to the Websocket Server with two HTTP headers, 1 for it’s own UUID and other one with <strong>P</strong>roxy HTTP Client’s UUID.<br> — Thenafter, the Websocket Server creates a mapping (Bridge) between both Proxies using UUID and redirects each other’s messages.</li></ul><blockquote>Websocket communication is always async so one needs to use synchronization to make it sequential similar to HTTP protocol. Because Websockets doesn’t support Request &amp; Response, only Messages are supported.</blockquote><blockquote>We need to do a configuration changes in Actual HTTP Client to use local IP of Proxy HTTP Server so that the communication can happen.</blockquote><blockquote>We can use any HTTP Server, HTTP Client &amp; Websocket library to make this work.</blockquote><blockquote>Also, it’s good practice to use messages of smaller size. Because it is convenient and doesn’t add load to the server.</blockquote><h4>Communication</h4><ol><li>A normal HTTP Request contains below things<br> — Method<br> — URI<br> — Cookies<br> — Headers<br> — Body</li><li>A normal HTTP Response contains below things<br> — Status Code<br> — Cookies<br> — Headers<br> — Body</li><li>We also need to send some metadata about the request payload so that the Proxy HTTP Client can wait for whole payload to download and then process.<br> — totalPayloadBytes — Total size of HTTP Request<br> — we can add more here …</li><li>A metadata message must be sent by both Proxies before each HTTP Request/Response.</li><li>Websocket Server receives the metadata message and send to Proxy HTTP Client.</li><li>Proxy HTTP Client receives the metadata message and waits for actual message.</li><li>Proxy HTTP Server sends actual payload wrapped in Websocket Message and waits for the response.</li><li>Websocket Server receives the message and send to Proxy HTTP Client.</li><li>Proxy HTTP Client receives the message, unwraps and creates an actual HTTP Request and fires a request to the given URL &amp; gets the response.</li><li>Proxy HTTP Client sends metadata about response.</li><li>Websocket Server receives the metadata message and send to Proxy HTTP Server.</li><li>Proxy HTTP Server receives the metadata message and waits for actual response message.</li><li>Proxy HTTP Client sends actual response wrapped in message.</li><li>Websocket Server receives the message and send to Proxy HTTP Server.</li><li>And now Proxy HTTP Server gets the response and gives back to Actual HTTP Client.</li></ol><h4>Conclusion</h4><ul><li>In conclusion, using WebSockets to transfer HTTP traffic can be a useful solution for a variety of scenarios.</li><li>Whether you need to access a service that requires a VPN connection, access a website or app from a client system, or hide your public IP through a proxy service, WebSockets can offer a reliable and efficient solution.</li><li>By establishing a bridge between a WebSocket server, a local HTTP server and WebSocket client, and an HTTP client and WebSocket client, HTTP traffic can be easily transferred back and forth.</li><li>However, it’s important to keep in mind that WebSocket communication is asynchronous and requires synchronization to make it sequential.</li><li>It’s also important to use messages of smaller size to avoid adding unnecessary load to the server.</li><li>With proper configuration and attention to detail, WebSockets can offer a powerful solution for efficient HTTP traffic transfer.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f2991b591f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Animate Dialogs In Flutter? Here Is Answer]]></title>
            <link>https://medium.com/flutter-community/how-to-animate-dialogs-in-flutter-here-is-answer-492ea3a7262f?source=rss-589f5e99fc58------2</link>
            <guid isPermaLink="false">https://medium.com/p/492ea3a7262f</guid>
            <category><![CDATA[flutter-widget]]></category>
            <category><![CDATA[dialog]]></category>
            <category><![CDATA[flutter-app-development]]></category>
            <category><![CDATA[flutter]]></category>
            <category><![CDATA[animation]]></category>
            <dc:creator><![CDATA[Mayur Prajapati]]></dc:creator>
            <pubDate>Sun, 12 May 2019 04:45:19 GMT</pubDate>
            <atom:updated>2019-05-14T14:43:14.447Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X8cqtVRD2FQviFOkAILsuw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/photos/4W5WWKaxsKs?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Markus Spiske</a> on <a href="https://unsplash.com/search/photos/animation?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Hello, I’m <strong>Mayur. </strong>This is the first article of my life and on <em>Medium</em> also.</p><p>Animations give a better experience to the user. I know and you might be knowing that Flutter has great support for Animations with ease of use. But wait, not in the case of Dialogs. Flutter’s Dialogs have some simple and predefined Animations.</p><p>There are tons of great articles for getting started with Animations in Flutter. But unfortunately, it doesn’t cover Animations for Dialogs.</p><p>Hey, don’t worry. I’m writing this article to solve this problem. In this article, I will show you how to animate dialogs in <strong>Flutter. </strong>Not only some simple animations but you can use any or many curves from <a href="https://docs.flutter.io/flutter/animation/Curves-class.html"><strong><em>Curves</em></strong></a>.</p><h3>Simple Dialog</h3><p>Flutter has <strong><em>showDialog()</em></strong> to show basic dialogs with basic animation. Following GIF shows a basic dialog in Flutter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GaS2qhF2WL2-3TxU5GlIEg.gif" /><figcaption><a href="https://docs.flutter.io/flutter/material/showDialog.html">showDialog()</a></figcaption></figure><h3>So, let’s animate it.</h3><p>Internally Flutter calls <a href="https://docs.flutter.io/flutter/widgets/showGeneralDialog.html"><strong><em>showGeneralDialog()</em></strong></a> via <a href="https://docs.flutter.io/flutter/material/showDialog.html"><strong><em>showDialog()</em></strong></a>.</p><p>Let’s dive into <strong><em>showGeneralDialog()</em></strong>. Here are some parameters.</p><blockquote>Future&lt;T&gt; showGeneralDialog&lt;T&gt;({<br> @required BuildContext context,<br> @required RoutePageBuilder pageBuilder,<br> bool barrierDismissible,<br> String barrierLabel,<br> Color barrierColor,<br> Duration transitionDuration,<br> RouteTransitionsBuilder transitionBuilder,<br>})</blockquote><p><strong>Note:</strong> <em>All arguments are required otherwise some error will occur.</em></p><p><strong>Some main arguments in showGeneralDialog&lt;T&gt; are</strong></p><ol><li><strong><em>barrierDismissible -&gt; </em></strong>Defines whether dialog may be dismissible or not<em>.</em></li><li><strong><em>barrierColor -&gt; </em></strong>Background color of dialog.</li><li><strong><em>transitionDuration -&gt; </em></strong>Duration of animation when the dialog appears and disappears.</li><li><strong><em>transitionBuilder -&gt; </em></strong>Takes <a href="https://docs.flutter.io/flutter/widgets/RouteTransitionsBuilder.html">RouteTransitionsBuilder</a> A builder function, called with animation values whenever dialog needs a rebuild. The implementation is shown below.</li></ol><blockquote>typedef RouteTransitionsBuilder = Widget Function(BuildContext context, Animation&lt;double&gt; animation, Animation&lt;double&gt; secondaryAnimation, Widget child);</blockquote><blockquote>The main parameter of <em>RouteTransitionsBuilder</em> is <strong><em>animation</em></strong>. You can use it to animate things in a dialog.</blockquote><h3>So, let’s do some practicals</h3><ol><li><strong>Rotate</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4u7QtC3_QYc5OY6UlJWGXg.gif" /><figcaption><strong>Rotate</strong></figcaption></figure><pre><strong><em>import </em>&#39;package:vector_math/vector_math.dart&#39; as math;</strong></pre><pre>showGeneralDialog(<br>    context: context,<br>    pageBuilder: (context, anim1, anim2) {},<br>    barrierDismissible: <em>true</em>,<br>    barrierColor: Colors.<em>black</em>.withOpacity(0.4),<br>    barrierLabel: &#39;&#39;,<br>    transitionBuilder: (context, anim1, anim2, child) {</pre><pre>      <em>return </em><strong>Transform.rotate(</strong><br>        <strong>angle: math.radians(anim1.value * 360),</strong><br>        child: AlertDialog(<br>          shape:<br>              OutlineInputBorder(borderRadius: BorderRadius.circular(16.0)),<br>          title: Text(&#39;Hello!!&#39;),<br>          content: Text(&#39;How are you?&#39;),<br>        ),<br>      );<br>    },<br>    transitionDuration: Duration(milliseconds: 300));</pre><p>2. <strong>Rotate with Opacity</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5T7PQKUa5FCz_09Ne5_4pg.gif" /><figcaption><strong>Rotate with Opacity</strong></figcaption></figure><pre><em>import </em>&#39;package:vector_math/vector_math.dart&#39; as math;</pre><pre>showGeneralDialog(<br>    context: context,<br>    pageBuilder: (context, anim1, anim2) {},<br>    barrierDismissible: <em>true</em>,<br>    barrierColor: Colors.<em>black</em>.withOpacity(0.4),<br>    barrierLabel: &#39;&#39;,<br>    transitionBuilder: (context, anim1, anim2, child) {</pre><pre>      <em>return </em>Transform.rotate(<br>        angle: math.radians(anim1.value * 360),<br>        child: <strong>Opacity(</strong><br>          <strong>opacity: anim1.value,</strong><br>          child: AlertDialog(<br>            shape:<br>                OutlineInputBorder(borderRadius: BorderRadius.circular(16.0)),<br>            title: Text(&#39;Hello!!&#39;),<br>            content: Text(&#39;How are you?&#39;),<br>          ),<br>        ),<br>      );<br>    },<br>    transitionDuration: Duration(milliseconds: 300));</pre><blockquote>Ah, these are some simple animations let’s do some complex one. Let’s use <a href="https://docs.flutter.io/flutter/animation/Curves-class.html"><strong>Curves</strong></a>.</blockquote><p>3. <strong>Curves.<em>easeInOutBack</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mW3X1ityawnEcu1rVPheRw.gif" /><figcaption><strong>Using curve <em>easeInOutBack</em></strong></figcaption></figure><pre>showGeneralDialog(<br>    barrierColor: Colors.<em>black</em>.withOpacity(0.5),<br>    transitionBuilder: (context, a1, a2, widget) {<br>      <strong><em>final </em>curvedValue = Curves.<em>easeInOutBack</em>.transform(a1.value) -   1.0;</strong></pre><pre>      <em>return </em>Transform(<br>        <strong>transform: Matrix4.translationValues(0.0, curvedValue * 200, 0.0)</strong>,<br>        child: Opacity(<br>          opacity: a1.value,<br>          child: AlertDialog(<br>            shape: OutlineInputBorder(<br>                borderRadius: BorderRadius.circular(16.0)),<br>            title: Text(&#39;Hello!!&#39;),<br>            content: Text(&#39;How are you?&#39;),<br>          ),<br>        ),<br>      );<br>    },<br>    transitionDuration: Duration(milliseconds: 200),<br>    barrierDismissible: <em>true</em>,<br>    barrierLabel: &#39;&#39;,<br>    context: context,<br>    pageBuilder: (context, animation1, animation2) {});</pre><p>4. <strong>Scale</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XAqKL0Rq92rhj7UJDTLUfQ.gif" /><figcaption><strong>Scale</strong></figcaption></figure><pre>showGeneralDialog(<br>    barrierColor: Colors.<em>black</em>.withOpacity(0.5),<br>    transitionBuilder: (context, a1, a2, widget) {<br>      <em>return </em><strong>Transform.scale(</strong><br>        <strong>scale: a1.value,</strong><br>        child: Opacity(<br>          opacity: a1.value,<br>          child: AlertDialog(<br>            shape: OutlineInputBorder(<br>                borderRadius: BorderRadius.circular(16.0)),<br>            title: Text(&#39;Hello!!&#39;),<br>            content: Text(&#39;How are you?&#39;),<br>          ),<br>        ),<br>      );<br>    },<br>    transitionDuration: Duration(milliseconds: 200),<br>    barrierDismissible: <em>true</em>,<br>    barrierLabel: &#39;&#39;,<br>    context: context,<br>    pageBuilder: (context, animation1, animation2) {});</pre><blockquote>If you need more customization you can visit <a href="https://flutter.dev/docs/development/ui/animations"><strong>Flutter Animations</strong></a> guide and can apply your software specific logic for Animations. From this method you can use any curve, any type of animation.</blockquote><blockquote>You just need to play with the values 😀.</blockquote><p>If you like the article just click on 👏. Rate it out of 50. Goodbye.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=492ea3a7262f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/flutter-community/how-to-animate-dialogs-in-flutter-here-is-answer-492ea3a7262f">How To Animate Dialogs In Flutter? Here Is Answer</a> was originally published in <a href="https://medium.com/flutter-community">Flutter Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>