<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Gene McKenna on Medium]]></title>
        <description><![CDATA[Stories by Gene McKenna on Medium]]></description>
        <link>https://medium.com/@mckennagene?source=rss-77bca457938b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*IwNK7DRmEJZkKaQs.</url>
            <title>Stories by Gene McKenna on Medium</title>
            <link>https://medium.com/@mckennagene?source=rss-77bca457938b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:06:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mckennagene/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Making Art from the 30k Photos Clogging Your Computer]]></title>
            <link>https://medium.com/@mckennagene/https-medium-com-mckennagene-make-art-from-the-30k-photos-clogging-your-computer-50ac7753db24?source=rss-77bca457938b------2</link>
            <guid isPermaLink="false">https://medium.com/p/50ac7753db24</guid>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[visual-art]]></category>
            <category><![CDATA[mosaic]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[photography]]></category>
            <dc:creator><![CDATA[Gene McKenna]]></dc:creator>
            <pubDate>Tue, 25 Oct 2016 04:34:53 GMT</pubDate>
            <atom:updated>2016-10-25T04:34:53.168Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’re a typical, digital-age family, you have two kids and 10–15k photos of each of them, more of you and your spouse, extended family, friends, … But honestly, who is ever going to look at all those photos? And what are you going to do with them besides worry about losing them?</p><p>Freshly returned from a big, 120+ person family reunion and armed with hundreds of photos from dozens of relatives, it felt like too much. What are we going to do with 700 photos?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7ut5g9jnZzYjQFQSfICVQg.jpeg" /><figcaption>some of the 1300 photos of a family reunion</figcaption></figure><p>I thought it might be time to try a photo mosaic. Everyone is familiar with these. There are some very famous works of art like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/402/0*llM4heqE81juw1zB.jpg" /><figcaption>Lincoln in Dalivision by Salvadore Dali</figcaption></figure><p>That’s not exactly a photo mosaic. Lincoln in Dalivision is hand painted by a genius. Not only does Dali work a nude of his wife into Lincoln’s face, he even uses a small Lincoln within the larger Lincoln mosaic. It’s like a fractal that goes down the rabbit hole forever. It was good inspiration but I wanted something more achievable and done with photos.</p><p>There are <a href="http://www.artensoft.com/artensoftphotocollagemaker/">commercial software packages</a> to create mosaics. Some come with many stock images of different colors to give you a useful library. And as I learned, the library of photos you have to work with is super important as is the choice of the target image you are trying to produce.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/0*_Ixmu6NK13F46kpV.jpg" /><figcaption>Artensoft Photo Collage Maker</figcaption></figure><p>I wasn’t ready to buy software to do this. I wanted to use only my own library of photos so it had more meaning to me. I also wanted to scratch a coding itch. In short I convinced myself to build my own mosaic maker.</p><p>Here is my process, what I learned and samples of the “art” I am now foisting on my family whether they want it or not.</p><p>First I gathered all my photos. I had about 700 from the family reunion I just attended. I thought that would be a lot. However quick math told me it was small potatoes. If a single photo in the mosaic is one pixel, you need roughly 40 to 100 pixels to make a reasonably detailed image. The <a href="http://obama2008.s3.amazonaws.com/mosaic.html">Obama Headlines Mosaic</a> is about 46x46 pixels, or over 2300 pixels total.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0a4HuN_Pu7uPCOQS.jpg" /></figure><p>In a 46x46 mosaic, a library of 700 pictures would require every picture to be used 5 or 6 times on average, some many more times. I had grand visions of not even repeating an image once. I quickly realized however they would have to repeat, as they do in the Obama mosaic. Even so, I would need a lot more images. I eventually added another 600 photos from the family reunion three years prior. It was a good start and kept my family theme.</p><h3><strong>Getting Organized</strong></h3><p>I needed all my photos all the exact same size. Some photos were portrait (tall) and some were landscape (wide). I wanted to just throw out the portraits but given my need for as many photos as possible, I thought of two ways they could be used. 1) I could crop them (take out some of the height) or 2) I could write smarter code that would use a portrait photo in the same width and twice the height as a landscape photo.</p><p>I opted for the “crop it out” choice. I first tried to do it automatically while the system also scaled photos to a uniform size. I thought most of the portrait mode pictures would have faces at the top and feet at the bottom, so chopping off the bottom would work. As the below shows, it often failed right in the middle of someone’s face.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j-YKmWr9VQ3CF0Z8S9VzwA.jpeg" /><figcaption>from left: original portrait photo, automatic cropping out of the bottom and manual crop to get all faces</figcaption></figure><p>I tried variations on automated cropping, but there was no good rule of thumb that was effective. But neither could I afford to reject a couple hundred photos just because they were portrait. So I ended up manually cropping well over 100 of them. That was time consuming.</p><p>If I have time for another version, I will try option 2), writing the mosaic system to allow portraits in more or less “as-is” but using up two pixels in the mosaic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j-Z0Nw0AgI7SfJ55uI4bkQ.jpeg" /><figcaption>this portrait image would need only minimal cropping to fit in the mosaic as a “double height” photo.</figcaption></figure><p>Armed with an initial library now of 1300 photos, it was time to do some color matching and see if I could construct an image.</p><h3>Color Matching</h3><p>Color Matching means finding the right photo from your photo library to fill in each spot in your target image. If you want a mosaic of Ziggy Stardust, you get a picture of Ziggy Stardust and break it up into little pixels and then find the right photo to represent each pixel.</p><p>To do this my software profiled all my photos and found out what color each photo was. Obviously every photo has many colors in it, but when a photo is viewed from a distance it looks tiny and comes through as more or less a single, average, color. To build a mosaic you have to know the color of each pixel in your target image, and then find a source photo from your library whose average color best matches the target pixel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4NL5fMBUMP4ATcaow7aYig.jpeg" /><figcaption>Finding the best photo in my library to match a blue pixel in Ziggy Stardust’s stripe.</figcaption></figure><p>In the image above, I give numeric color values with the label Lab. There are more details in the technical notes at the bottom of this article, but in short, matching colors using the standard Red Green Blue (RGB) that everyone is familiar with isn’t very good. Lab is another way of representing colors that better matches human perception of the difference between colors with a mathematical measurement of that difference.</p><p>So after scaling and cropping all the images to the same size, then profiling each image and recording its average Lab color, it was time to pick a target image and see if I could construct a mosaic.</p><h3><strong>Make a Mosaic Already</strong></h3><p>I quickly learned that a target image that is mostly one or two faces would be difficult. Apparently not many photos in my library have an average color similar to the faces of the people in my family. The system would find one or two reasonable color matches and then repeat those photos over and over and over again giving the skin a pretty monotone look.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7yFsgrXUnhgU_33Pj1mvJw.jpeg" /></figure><p>My son came out looking like his head had some splotchy disease. Yech.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uUjwQ8HlOy0ESWEYIR7J6Q.jpeg" /></figure><p>At this point the project became a lot more about art than coding. And I’m not much of an artist. I did make a decision that I didn’t want to artificially alter the color of my source photos. I have seen mosaics where people’s faces are green or blue because that is what was needed. We have one of those on the wall where I work. Meh.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/864/1*3z4KGVGUFUZt33isCmFmpg.jpeg" /><figcaption>A photo mosaic where photos are artificially colored (we really don’t have green people here)</figcaption></figure><p>I realized the choice of target image is really important. To represent something well at a size of about 50x50 pixels, the target image has to be very recognizable. It needs to be iconic in both senses of the word. This process is like constructing an icon to be used on a old computer desktop with bad resolution.</p><p>Abe Lincoln is such a good choice because his look is iconic. You can recognize Honest Abe in about 13x18 pixels in the Dali painting. Obama’s mosaic works in about 46x46 pixels because we’ve all seen the <a href="http://www.openculture.com/2009/02/the_obama_hope_poster_the_new_copyright_controversy.html">Obama Hope photo</a> a million times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bEZe07qd0bYE6dPdeQQUYw.jpeg" /></figure><p>I wanted a target photo that would have meaning to my family, and be recognizable in about 50x50 to 100x100 pixels, and of course had colors found in my photos. I thought maybe a picture of our grandmother would work, but given the major skin tone problems in my early testing, that seemed a stretch. Since our family is Irish, a shamrock was an easy choice.</p><p>This first attempt at a shamrock seemed decent. From the target at left I created the mosaic at right (you can see it zoomed in further below). I cheated a bit and allowed the all white areas around the shamrock to be filled with just plain white instead of a photo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zybti5rmPL1rAKXIqUEVjw.jpeg" /><figcaption>Shamrock</figcaption></figure><p>Fortunately a lot of the pictures from our reunion were taken outside with lots of grass around, so my library had a lot of green. But I was allowing unlimited repeating of images and when you looked closely, it was obvious.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U8Hu7EPyT2B5_iefOKJ7ow.jpeg" /></figure><p>The shamrock target image was made at 128x128 pixels. Shamrocks are easily recognizable at that size. Each photo was scaled down to 128x84 pixels, so the total image size was 16k pixels across or about 54 inches wide if printed at 300 dpi. There were a few problems with this setup. A 54&quot; inch print is really big. The repeating was bad. And with each photo at only 128x84 pixels it was hard to see who was actually in the photo.</p><p>In the version below, the target shamrock was reduced to 48x48 enabling the photos comprising it to be at a larger 256x158. This would print at about 41 inches wide. In addition a repeat penalty was added in the photo selection process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4ENQtUs4FSBZa9pO5_As1g.jpeg" /></figure><p>Here the repeat penalty helped reduce the number of times any photo was used, but it was applied in such a way that the color matching gets progressively worse as we go down in the image. The system filled in the best matching photos (that haven’t been used too much already) starting at the top left and moving across in rows, then down. By the time it got to the bottom rows, all the photos with lots of green have been used many times already and couldn’t be used anymore.</p><p>The larger size of the photos here made it much easier to see who was in them. This made it clear that many relatives were not in the mosaic at all. They weren’t in any photos whose color matched the shamrock well.</p><p>Here a political decision outweighed an artistic one — I wanted to ensure as many relatives as possible were included.</p><h3>Iteration</h3><p>In this next version I increased shamrock size from 48x48 to 56x56 pixels. Instead of filling the picture from top left to bottom right, the fill was randomized and the repeat penalty was made to be more severe if the photo repeated nearby and less so if it repeated far away. Instead of filling in the area around the shamrock with pure white, I let the border fill in with whatever photos were the best color match to white. That made the edges smoother. I also made a report to show me which source photos were used and how many times. That made counting cousins easier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4iWYxByazDcTVeCjKU6OTA.jpeg" /></figure><p>It was looking better all around but I was still only using about half of the source photos. The repeat rate was still pretty high, especially in the white areas, and still a lot of relatives weren’t getting in the picture. I didn’t really have that many pictures that were white on average, yet about 30% of the target image was white. I had a lot of blue and gray pictures going unused.</p><p>I also had to guarantee that some VIPs (my parents, all my aunts and uncles) were included no matter what. I couldn’t guarantee that for all 30 odd cousins and 40 odd children of cousins or I’d go crazy, but I had to do better.</p><p>So I introduced a color mask using a really bright purple and changed the matching rules so anything at all could match bright purple, but VIPs got selected preferentially to guarantee they’d get in. Finally I put a blue border all the way around to help frame it. This is the point where you can say, yes, clearly you are not an artist. But I was able to use a lot more of the photos.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T6EPshOMlJYAl8Bxm5ucSg.jpeg" /></figure><p>At this point I felt the system worked pretty well so I kicked out a few alternative shamrocks and Irish icons as options. My wife assures me the leprechaun will never be hung on any wall in our home. But with the help of <a href="https://www.vivyxprinting.com/">Vivyx Printing</a>, the only online site that allows uploading 100MB image files, printing is not a problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TdFzE6CUACF2O3L0YA0jkg.jpeg" /></figure><h3>The Whole Enchilada</h3><p>Finally, I wanted to explore what other images I might make. In all the above samples I was using a library of about 1300 images from two family reunions (same family, two different years), but it was very limiting to have so few photos and so many missing colors. How much can you do with green and grey?</p><p>In my whole photo library I have about 40k images from the last 6 years. I’m still working on what to do with them besides worry about backing them up all the time. I have two thoughts left to share.</p><ol><li>when you use your ENTIRE photo library the whole question of “repeating images” becomes more nuanced and problematic. The digital camera age means every life moment, every dinner party, every grandma-holding-the-baby, results in 3 to 33 pictures of pretty much the same thing. The repeat penalty has to apply to a whole set of similar pictures, or the pictures need to be de-duped ahead-of-time. No one in my house likes to sit down at the photo library and choose one picture of each set to keep. So I was exploring algorithms to detect dupes automatically and think this should be quite possible to work in on a future version. Also because they will be reduced in size to about 256 pixels, the question of which one of a set is the mythical “best one” is less relevant.</li><li>To choose a target image it is helpful to know what colors you have in your photo library. I wanted a way to visualize this. In the image below, you can see a cone of colored chips. The cones go from fairly bright at left to dull &amp; grey at right. In the 2nd row, the squares are filled in according to how many of my photos match each color chip. As you can see, reducing a photo to its average color gives you lots of grey.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*FH5w1fZqyc9hso3H7xlafA.jpeg" /><figcaption>inventory of available colors</figcaption></figure><p>But since I do appear to have some blues, greens and oranges, I found a little abstract art that uses those colors. Now if I just could find a wall that is in need of a one hundred inch wide print, I’m all set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O-K8DhyJyY5NzBPVAbbubA.jpeg" /></figure><h3><strong>For the technically curious</strong></h3><p>There were four basic steps to my process: Scaling, Profiling, Selection and Composition. All the code I wrote was done in Java but I also made use of ImageMagick, a pre-made software library for image manipulation and batch processing.</p><p><strong>Scaling: </strong>The first software I wrote was to scale the images. All source images were scaled to a size specified in a config file. If it was a portrait photo (height &gt; width) it was rejected (and many of these I manually cropped so they’d become landscape.</p><p>For landscape photos the fact that I was getting pictures from many different relatives taken on many different cameras meant I had a lot of variation in sizes and aspect ratios. But I needed to get everything to one size. I used a couple simple rules.</p><p>If an image’s aspect ratio was just a little off (+/- 5%), i would squish it (allow different scaling in x and y dimensions) to get to the desired aspect ratio. If the aspect ratio was 5–20% off, I would crop the photo equally from top and bottom. There may have been a few of these where I also had to manually crop the source and send it through again but this small amount of cropping was usually ok. If the aspect ratio was more than 20% off target, the image was rejected (or again, I might manually fix it).</p><p>To scale the images I just opened them as a BufferedImage, created a Graphics2D with rendering hints KEY_INTERPOLATION, and VALUE_INTERPOLATION_BILINEAR. The Graphics2D would then let me draw the image at a new size and ImageIO would write it out. A key issue was calling dispose() on the Graphics2D so the resources would properly get recycled. When processing thousands of images, resource management in Java can be an issue if not done right.</p><p>Given that these pictures are usually scaled down quite small (about 256 pixels in my case) I had a thought to try to exclude pictures where the people in the picture are very tiny and wouldn’t be recognizable at such small scale. Typically this would be because they are far away. Many cameras record the focal length of the lens when the photo is taken and this <a href="http://regex.info/exif.cgi?dummy=on&amp;imgurl=http%3A%2F%2F1hr-tempcache%2F92726050806102-231-84-542-371-DSC_0966.JPG">data can be extracted from the “xif” data of an image easily</a>. I never got around to trying this or even determining what % of my source photos have this data. Maybe in the next version.</p><p><strong>Profiling: </strong>Initially I tried identifying the top few color clusters in every source photo. Photos may have several distinct color groups like green grass and blue sky in one picture. The thought was that if the target called for green, a photo with a green cluster comprising at least 40% or more of total pixels in the photo could be used. And by contrast the fear of simply averaging all pixels into one color was that they’d all come out close to grey as my color inventory above shows. When a picture with a lot of green has an average color of almost grey, it might not get chosen when green is needed.</p><p>For clustering I used <a href="https://commons.apache.org/proper/commons-math/javadocs/api-3.0/org/apache/commons/math3/stat/clustering/KMeansPlusPlusClusterer.html">KMeansPlusPlusClusterer</a> which was good for generating the clusters. I tried a few iterations: 3 clusters, 2 clusters or a hybrid approach starting with 3 clusters and reducing the cluster count until one cluster had at least 60% of the pixels. But the quality of mosaics of any clustering method was not as good as using a single cluster, aka, avg color.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W_uqHe5MXLxEompFRQc2Jw.jpeg" /><figcaption>from left: 1 cluster (avg color) — top 3 color clusters — top 2 color clusters — hybrid clustering</figcaption></figure><p>I used Lab color space instead of RGB for the profiling (selecting the average color of a photo) and selection (matching a photo to a pixel in the target). In RGB color space it is not necessarily the case that one color a certain distance from your target will be perceived by your eyes to be closer than another color that is a greater distance away. Distance in RGB space is not perceptually linear. In Lab space (officially <a href="https://en.wikipedia.org/wiki/List_of_color_spaces_and_their_uses#CIELAB">CIE Lab</a>), the distance is more uniform with what your brain thinks about as the distance or difference between colors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*v3bRj07MAd-h26OjFPiQ6Q.jpeg" /><figcaption>CIE Lab Colorspace — ideal for measuring perceptual color difference</figcaption></figure><p>Below you can see a comparison between using standard RGB color vs Lab. The RGB mosaic loses detail in the lower left leaf and is much darker overall. The blue border is very subtle, as intended, in the Lab mosaic. In the RGB mosaic its a cartoony blue/green. The same repeat penalties were applied in both cases, but the RGB has visibly more repeats.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QvxQbHszETMN255U-cNJRA.jpeg" /><figcaption>Colorspace comparison: Lab color space at left and RGB color space at right</figcaption></figure><p>I also experimented with HSV (or HSB) color space but similarly found Lab to be superior.</p><p>Since photos are stored typically in RGB by the time they land on your computer in a JPG file, I had to convert from RGB to Lab. I used code from <a href="http://stackoverflow.com/users/790979/charleslparker">charleslparker</a> posted <a href="http://stackoverflow.com/questions/4593469/java-how-to-convert-rgb-color-to-cie-lab">here</a> for the conversion.</p><p><strong>Selection (Color Matching): </strong>Here I loaded the target image and I stretched it vertically a bit because using landscape photos means they aren’t as tall as they are wide. So to avoid the resulting image being vertically compressed, I stretch the target vertically by a corresponding amount.</p><p>Then I make a List of all the x,y coordinates in the target image and shuffle the List so the fill order is random. This presents all the best color matches from being “used up” in the top rows of the image.</p><p>For each point, if the target color is my special purple mask color, I skip it, otherwise I find the photo whose average color is the best match, subject to the repeat penalty. The repeat penalty has both a global aspect (how many times has this photo been used before anywhere) and a localized repeat penalty (how many times does this photo appear within a 5x5 area around this point).</p><p>After all the points that weren’t the special purple mask color were filled in, I cycle through all the points (again in random order) that are the mask color and I fill them with photos that have not yet been used anywhere, starting with the VIP photos (my parents, aunts and uncles) marked in a configuration file. The repeat penalty is again applied so it cycles pretty quickly through all the photos that have not yet been used.</p><p>The output of the Selection step is not the final image. When I tried to assemble the final image in Java, it would continually run out of memory. The easiest way to make an image in Java and save it as a JPG involves holding the entire image in memory and these are 200 million pixel images. Instead the Selection step outputs ImageMagick <a href="http://www.imagemagick.org/script/montage.php">montage</a> commands.</p><p><strong>Composition </strong>The final step is actually constructing the mosaic image file. ImageMagick is very good at this with the montage command. The Selection step above essentially writes out a text file of montage commands. The commands specify which photos are laid down in what order for every row of the target image. The montage command is used to creates a series of images, one per row. And then a final montage command instructs it to combine all the row-images into a final mosaic.</p><p>ImageMagick doesn’t run out of memory. I suspect it isn’t holding all the data in memory but rather just concatenating the image data directly on disk, in whatever way the jpg format accommodates that.</p><p><strong>Useful Reads:</strong></p><p><a href="http://pages.cs.wisc.edu/~dyer/cs534/hw/hw5/sample-project-reports/epps2009.pdf">The World in Dalivision: Crowdsourced Photographic Mosaics, Brett Epps</a></p><p><a href="http://filipjaskolski.pl/doc/fjaskolski.pm.pdf">Photomosaic Maps, Filip Jaskolski</a></p><p><a href="https://www.google.com/search?q=chuck+close+art&amp;source=lnms&amp;tbm=isch&amp;sa=X&amp;ved=0ahUKEwiKl63ziO_PAhUF4SYKHUmcBPMQ_AUICCgB&amp;biw=1440&amp;bih=711">Artwork of Chuck Close</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50ac7753db24" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>