<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Abhyuday Bharat on Medium]]></title>
        <description><![CDATA[Stories by Abhyuday Bharat on Medium]]></description>
        <link>https://medium.com/@abhyudaybharat?source=rss-2f6b427d4dfc------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*TObIsyIp--l7hoQFZ000ig.jpeg</url>
            <title>Stories by Abhyuday Bharat on Medium</title>
            <link>https://medium.com/@abhyudaybharat?source=rss-2f6b427d4dfc------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 08:51:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@abhyudaybharat/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Website screenshot overlay tool: A tool based on HTML Canvas, Selenium and Java libraries]]></title>
            <link>https://medium.com/miq-tech-and-analytics/website-screenshot-overlay-tool-a-tool-based-on-html-canvas-selenium-and-java-libraries-14f945056b34?source=rss-2f6b427d4dfc------2</link>
            <guid isPermaLink="false">https://medium.com/p/14f945056b34</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[selenium]]></category>
            <category><![CDATA[chromedriver]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[html-canvas]]></category>
            <dc:creator><![CDATA[Abhyuday Bharat]]></dc:creator>
            <pubDate>Fri, 03 Apr 2020 07:34:44 GMT</pubDate>
            <atom:updated>2020-04-03T07:34:44.370Z</atom:updated>
            <content:encoded><![CDATA[<h3>Screenshot overlay tool: A tool based on <em>HTML Canvas, Selenium </em>and<em> Java libraries (java.awt, javax.imageio)</em></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cZJH15m3CnobKSrQ" /><figcaption><strong>Fig-1</strong></figcaption></figure><p>In this write-up, we’ll look into a basic screenshot overlay tool developed using basic capabilities provided by HTML Canvas and Selenium. The tool captures a screenshot of the requested website, allows the user to draw a rectangle over the screenshot, stores the coordinates of the rectangle and then overlays another image on top of that screenshot at the coordinates selected at a later time. At <a href="http://www.wearemiq.com/"><em>MiQ</em></a>, we get to work with different ad-creatives. This tool is part of a <em>creative-overlap workflow</em> that is used for quality testing purposes.</p><p>As you can see in <em>Fig -1</em>, there are 3 key things happening in the tool:</p><ul><li><em>Fetch screenshot</em></li><li><em>Draw rectangle</em></li><li><em>Overlay an image over the screenshot</em></li></ul><p>Let’s see how these functionalities are achieved in the tool.</p><h3>1 — Fetch screenshot</h3><p>There are many online services that provide APIs to download screenshots of websites like <a href="https://urlbox.io/">URL box</a>, <a href="https://apiflash.com/">API flash</a>, etc. They are not completely free though. We developed our own service to download screenshots and it’s quite easy to develop this using <em>chromedriver</em> and <em>selenium</em>.</p><ul><li>Once you have set up chromedriver and a basic spring-boot rest app according to your needs, you can use the snippet shown in Snippet-1 to get a screenshot.</li></ul><pre><strong><em>&#39;Snippet — 1&#39;</em></strong></pre><pre>import org.apache.commons.io.FileUtils;<br>import org.openqa.selenium.OutputType;<br>import org.openqa.selenium.chrome.ChromeDriver;<br>import org.openqa.selenium.chrome.ChromeOptions;</pre><pre>...</pre><pre> System.setProperty(“webdriver.chrome.driver”, “/usr/bin/chromedriver”);<br> ChromeOptions options = new ChromeOptions();<br> String chromeWindowSize = “ — window-size=” + width + “,” + height;<br> options.addArguments(“ — headless”, “ — disable-gpu”, “ — no-sandbox”, “ — disable-extensions”,<br> “ — disable-geolocation”, chromeWindowSize, “ — ignore-certificate-errors”);</pre><pre> ChromeDriver driver = new ChromeDriver(options);<br> driver.get(url);<br> Thread.sleep(sleep);<br> File source = driver.getScreenshotAs(OutputType.FILE);</pre><pre>...</pre><ul><li>Basic knowledge of Java 8 is a prerequisite to understanding what’s happening in the snippet in <em>Snippet-1</em>.</li><li>We used this snippet to create a generic screenshot API which is now used across our organisation for various screenshot needs.</li></ul><p>The next step is to fetch the screenshot of a website and show it on our webpage. This is where <strong>HTML Canvas</strong> comes to the rescue.</p><ul><li>HTML Canvas or <strong><em>&lt;canvas&gt;</em></strong> tag in html allows you to create graphics and play with pixels on a webpage.</li><li>With proper use of different mouse events like <strong><em>onmousemove</em></strong>, <strong><em>onclick</em></strong> etc. and appropriate JS functions, we can even create a paint tool in our web browser.</li><li>If you are using AngularJS, the events on the canvas would look like the code snippet shown in <em>Snippet-2</em>.</li></ul><pre><strong><em>&#39;Snippet — 2&#39;</em></strong></pre><pre>&lt;canvas id=”smpSgDomainCanvas” ng-mouseleave=”removeUndrawnSlots()” <br> ng-mousemove=”drawSlot($event)” ng-click=”storeSlot($event)”&gt;<br>&lt;/canvas&gt;</pre><ul><li>Canvas allows you to load an image tag over it and then manipulate pixels. We used JS <strong><em>Image()</em></strong> to define an image tag and dynamically load the downloaded image using our <em>Screenshot API.</em></li><li>Another code snippet from our implementation (AngularJS) is shown in <em>Snippet-3. </em>The code snippet shown in <em>Snippet-3</em> is self-explanatory with a bit of experience in JS. Here, <strong><em>response.data.responseString</em></strong> is the PNG image which is set as the image tag’s src.</li></ul><pre><strong><em>&#39;Snippet — 3&#39;</em></strong></pre><pre>var imgTag = new Image();<br>imgTag.src = &#39;data:image/png;base64,&#39; + response.data.responseString;<br>document.getElementById(&#39;smpSgDomainScreenshot&#39;).appendChild(imgTag);<br>canvas = document.getElementById(&#39;smpSgDomainCanvas&#39;);<br>canvas.height = $scope.domainModel.screenHeight;<br>canvas.width = $scope.domainModel.screenWidth;<br>imgTag.onload = function () {<br> canvas = document.getElementById(&#39;smpSgDomainCanvas&#39;);<br> var context = canvas.getContext(&#39;2d&#39;);<br> canvas.height = $scope.domainModel.screenHeight;<br> canvas.width = $scope.domainModel.screenWidth;<br> context.clearRect(0, 0, canvas.width, canvas.height);<br> context.drawImage(imgTag, 0, 0, $scope.domainModel.screenWidth,<br>   $scope.domainModel.screenHeight);<br>};</pre><h3>2 — Draw rectangle</h3><p>The next step is enabling the user to draw a rectangle. There are many ways to do this which would depend on how much flexibility you want to provide to the user. In our use case, we had pre-decided sizes of rectangles where the user selects the size of the rectangle, moves the rectangle over the screenshot by dragging her mouse pointer and clicks wherever she wants to place the rectangle.</p><ul><li>This functionality is achieved by using AngularJS versions of HTML/JS events like <strong><em>onmousemove</em></strong>, <strong><em>onmouseleave</em></strong> and <strong><em>onmouseclick</em></strong>.</li><li>Please refer to <em>Snippet-2</em> to see how these events are used on a canvas element. The functions used on these events (<strong><em>drawSlot</em></strong>, <strong><em>storeSlot</em></strong> and <strong><em>removeUndrawnSlot</em></strong>) do the real work of drawing rectangles, moving rectangles over the canvas and storing the coordinates on a click.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RElVDUJcIXYk-fL4" /><figcaption><strong>Fig-2</strong></figcaption></figure><ul><li><em>Fig-2</em> shows the user flow. This flow is achieved using <strong><em>onmousemove</em></strong> event handler in canvas. The <strong><em>mousemove</em></strong> event is passed to the function which has properties clientX and clientY.</li><li>The 2 properties which represent the current mouse position combined with the <strong><em>getBoundingRect()</em></strong> function of JS’s canvas returns the relative position of the mouse pointer on the canvas.</li><li>Using this info, the canvas is cleared and then the rectangle is drawn again.</li><li>All this happens very fast making it seem like the rectangle is moving. Refer to the code snippet shown in <em>Snippet-4</em> for an AngularJS implementation of this flow.</li></ul><pre><strong><em>&#39;Snippet — 4&#39;</em></strong></pre><pre>$scope.drawSlot = function (e) {<br> canvas = document.getElementById(&#39;smpSgDomainCanvas&#39;);<br> var pos = getCursorPosition(canvas, e);<br> pos.x = pos.x - 20 &gt;= 0 ? pos.x - 20 : 0;<br> pos.y = pos.y - 20 &gt;= 0 ? pos.y - 20 : 0;<br> var context = canvas.getContext(&#39;2d&#39;);<br> context.clearRect(0, 0, canvas.width, canvas.height);<br> context.drawImage(imgTag, 0, 0, $scope.domainModel.screenWidth, $scope.domainModel.screenHeight);<br> drawRectangleFilled(context, pos, $scope.slotSelected.selected.width, $scope.slotSelected.selected.height);<br>};<br><br>function getCursorPosition(canvas, event) {<br> var rect = canvas.getBoundingClientRect();<br> var x = 0;<br> var y = 0;<br> x = event.clientX - rect.left;<br> y = event.clientY - rect.top;<br> return {x: x, y: y};<br>}</pre><ul><li>These functionalities are supported by most of the latest browsers. This is one way of implementing this functionality. You can find out many other ways to implement the same thing using canvas object and the <em>mousemove</em> event.</li><li>Please ignore the <em>pos.x -20</em> calculation. That is just a part of our implementation where we draw the rectangle with 20px padding on top and left of the cursor.</li><li>On user click, relative coordinates of the rectangle are stored and saved for future use.</li></ul><h3>3 — Overlay an image over the screenshot</h3><p>A little background here, <a href="http://www.wearemiq.com/"><em>MiQ</em></a> works with different sizes of ad-creatives which need to be quality-tested by placing them over webpage screenshots. In the previous step, the user has already selected the coordinates where she wants to see the creatives placed. In this step, the ad-creative is placed on the webpage screenshot at the appropriate coordinates selected by the user. We will focus on the image processing aspect of the flow which is quite easy to do.</p><ul><li>Since the rest of our workflow uses Java, we went ahead with a Java implementation.</li><li>Java has some straightforward implementations for doing basic image processing. We have used 2 such libraries to achieve this —<strong><em> java.awt, javax.imageio</em></strong></li></ul><pre><strong><em>&#39;Snippet - 5&#39;</em></strong></pre><pre>public BufferedImage overlapImages(InputStream baseStream, InputStream creativeStream,<br>   CreativeCoordinates coordinates) {<br> try {<br>   BufferedImage baseImage = ImageIO.read(baseStream);<br>   BufferedImage creativeImage = ImageIO.read(creativeStream);<br>   Graphics2D baseImageGraphics = baseImage.createGraphics();<br>   baseImageGraphics.drawImage(creativeImage, coordinates.getXStart(),</pre><pre>coordinates.getYStart(), null);<br>   return baseImage;<br> } catch (IOException e) {<br>   log.error(&quot;Error reading stream in Overlap image call: {}. Error: &quot;, coordinates, e);<br> }<br> return null;<br>}</pre><ul><li>Refer to the code snippet in <em>Snippet-5</em>. As you can see it’s a simple, straightforward code.</li><li>The function accepts 3 parameters — the website screenshot as InputStream, the ad-creative image also as an InputStream object and the starting coordinates of the selected rectangle in the previous step.</li><li>The code is quite self-explanatory. <strong><em>Graphics2D.drawImage</em></strong> modifies the website screenshot’s BufferedImage object and returns the final image.</li></ul><p>Thanks for reaching this far. As you can see, Selenium (known for browser-app testing) can be used for non-testing purposes as well. Though this was a very specific use-case, you can see how basic HTML canvas and JS events can be combined and used to do lots of graphics manipulation in websites. Please write a response if you have any queries, feedback or suggestions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14f945056b34" width="1" height="1" alt=""><hr><p><a href="https://medium.com/miq-tech-and-analytics/website-screenshot-overlay-tool-a-tool-based-on-html-canvas-selenium-and-java-libraries-14f945056b34">Website screenshot overlay tool: A tool based on HTML Canvas, Selenium and Java libraries</a> was originally published in <a href="https://medium.com/miq-tech-and-analytics">MiQ Tech and Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>