<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by D41 on Medium]]></title>
        <description><![CDATA[Stories by D41 on Medium]]></description>
        <link>https://medium.com/@d41dev?source=rss-9d15bed2c1fd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Z38qoaXAJbA9uOJ-6zlGuA.png</url>
            <title>Stories by D41 on Medium</title>
            <link>https://medium.com/@d41dev?source=rss-9d15bed2c1fd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:08:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@d41dev/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How Google’s AI-Created Apps Could Disrupt the Android Ecosystem]]></title>
            <link>https://medium.com/@d41dev/how-googles-ai-created-apps-could-disrupt-the-android-ecosystem-7e18e1b1727c?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e18e1b1727c</guid>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mobile-apps]]></category>
            <category><![CDATA[google-play-store]]></category>
            <category><![CDATA[android]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 15:01:04 GMT</pubDate>
            <atom:updated>2026-10-14T15:01:04.441Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yP4P-5mIdf_jtKibXqYqwg.png" /><figcaption>Illustration of Google’s personal app library feature. The phone screen is taken from an actual screenshot in Google’s blog post; the surrounding photograph is AI-generated.</figcaption></figure><p>Google has recently announced that users will be able to <a href="https://blog.google/products-and-platforms/platforms/google-play/google-play-updates-october-2026/">build vibe-coded apps via Google AI Studio and run them on Google Play</a>. This felt inevitable after LLMs burst onto the scene, but I hadn’t considered how it could disrupt the Android ecosystem until now.</p><h3>How does this impact users?</h3><p>If you go on the Subreddits <a href="https://www.reddit.com/r/androidapps/">r/androidapps </a>or <a href="https://www.reddit.com/r/fossdroid/">r/fossdroid</a>, you’ll frequently see app requests like “<em>Is there an app that…?</em>”, “<em>Is there a FOSS alternative to…?</em>” or “<em>Is there an app like X that also does Y?</em>”. These questions highlight two problems Android users face: app discovery and a lack of personalisation.</p><p>The ability to create personal apps addresses both problems, which is great for users.</p><p>At the core of both is the desire for 100% personalisation. I’ve always recognised its importance because I’ve experienced the need for it myself. But this became even more apparent when I became a developer and started receiving niche feature requests from users who were using my app differently from how I’d intended.</p><p>With this new approach to app creation, users no longer need to wait for developers to implement a feature they want. They can simply create an app that does what they need.</p><p>Of course, this assumes that creating these apps is free or affordable. Knowing Google, a subscription or freemium model is likely. If users have to pay to generate and run these apps, the benefits may be limited for those who only need a simple utility.</p><p>So, if users can now create their own apps, where does this leave developers?</p><h3>How does this impact developers?</h3><p>Android is a multisided network consisting of developers, who create third-party apps, and consumers, who use them. The ability for users to create their own personalised apps reduces their reliance on third-party developers. This represents a fundamental change to how mobile operating systems and their app ecosystems have traditionally worked.</p><p>However, not all developers will be affected equally.</p><p>While Google hasn’t provided full details on how this will work, it’s reasonable to expect that this approach will initially be most practical for simple, client-side apps. Apps that require backend infrastructure or complex client-side functionality, such as on-device ML models, will be harder to build and maintain this way. Apps that rely on third-party APIs may also be impractical to vibe-code, depending on their integration requirements.</p><p>I don’t think this will result in developers being completely replaced, but it’s entirely possible that it could replace a whole class of apps: simple, client-side utilities.</p><p>And to be honest, that’s not necessarily a bad thing. We don’t need 1,000 different TODO apps or 1,000 different note-taking apps if users can create something that meets their exact needs.</p><p>But if developers lose out, how does Google benefit from this change?</p><h3>How does this impact Google?</h3><p>If this approach gains widespread adoption, it could further consolidate Google’s power over the Android ecosystem. User-created apps distributed through Google Play would bring more activity into Google’s platform, potentially increasing users’ reliance on Google Play and Google Play services while reducing the visibility and relevance of independently developed apps.</p><p>There is also a potential financial incentive. If users have to pay to create these apps, Google could establish a new revenue stream. Rather than relying solely on its share of in-app purchases, it could monetise the app creation process itself. Depending on the pricing model, Google could capture revenue directly from users who would previously have downloaded free utilities from third-party developers.</p><p>Another potential benefit for Google is improved app quality on the Play Store. If users can easily create their own simple, client-side apps, developers may face less incentive to build and maintain generic utilities. This could shift the focus of professional app development towards apps that offer more complex functionality, such as backend infrastructure or advanced on-device processing, which are harder to replicate through vibe coding. In theory, this could raise the average quality of apps available on Google Play, although that would depend on how the ecosystem evolves.</p><h3>Conclusion</h3><p>Ultimately, users and Google stand to benefit the most from this change, while developers of simple, niche apps stand to lose the most. Users gain the ability to create software tailored to their needs without waiting for developers to implement specific features, while Google stands to benefit from potential subscription revenue, greater control over the Android ecosystem, with improved app quality on the Play Store as a possible indirect consequence.</p><p>Developers, meanwhile, could lose users to personalised alternatives that are easier to create than ever before. Although apps requiring complex functionality or backend infrastructure are less likely to be affected, developers of simple utilities may find it increasingly difficult to justify building and maintaining apps that users can now create themselves.</p><p>The biggest question is whether this will simply change how people discover and use apps or fundamentally change the role third-party developers play in the Android ecosystem.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e18e1b1727c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SmartScan Update: SmartScan Is Not a Gallery App, It’s a Personal Knowledge Management App]]></title>
            <link>https://medium.com/@d41dev/smartscan-update-smartscan-is-not-a-gallery-app-its-a-personal-knowledge-management-app-d783ff85759d?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/d783ff85759d</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[personal-knowledge-system]]></category>
            <category><![CDATA[mobile-app-development]]></category>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[android]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Fri, 21 Aug 2026 14:02:37 GMT</pubDate>
            <atom:updated>2026-08-21T14:02:37.209Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wi4-Vqv9XLPLbGUqZBeJlQ.png" /></figure><p>When I initially made SmartScan I referred to it as a “smart gallery” app, for lack of a better description. But I always knew I didn’t want it to be a typical gallery app like Google Photos or Samsung Gallery app.</p><p>The idea for it came about because I’m someone who frequently takes screenshots of reddit posts, articles, pictures of book pages and screen recordings of podcast clips when they trigger interesting thoughts. At some point they end up getting lost in my camera roll and I say “I’ll clean this shit up at some point”, but never do.</p><p>This is the real problem I want to solve and it has made me realise that SmartScan was always supposed to be a personal knowledge management (PKM) app.</p><h4>What our camera rolls says about us</h4><p>There’s 2 sides of the spectrum. On one side you have Type 1, knowledge-capture focused camera rolls. This includes things like screenshots of articles and social media threads, pictures of book pages, screen recordings of podcast clips. The phone functions as an external brain, collecting ideas and information for future use.</p><p>On the other side of the spectrum, you have Type 2, memory and experience focused camera rolls. These include things like photos of friends, family, travel, hobbies, nature, and everyday life. The phone functions as a memory archive, preserving relationships, experiences, and personal moments.</p><p>We are all somewhere along this spectrum and it may fluctuate during different parts of our life. Personally my camera roll has always been more Type 1 than Type 2, but even more so since I became a dev. Your typical gallery app is mostly catered to Type 2. I wanted SmartScan to be balanced within this spectrum but with slightly more focused on Type 1 camera rolls.</p><p>The existing Auto Collections feature aligns with Type 2 as it’s able to automatically group visually similar media, which can capture groups of memories or experiences. For example, party photos will likely be grouped together. It aligns less with Type 1 because Type 1 requires grouping by meaning rather than visual similarity.</p><p>The new Concepts feature aims to address this and is the start of SmartScan’s transformation into a PKM app.</p><h4>Capturing Concepts</h4><p>The Concepts feature lets you create a “Concept” by describing it, and all matching media is automatically added to it, making it easy to track the development of ideas or rediscover them later.</p><p>To use it, you’ll either need to add descriptions to media manually or use the “generate image descriptions” feature if you have an OpenAI API key which you can add to the app. If there is enough user demand I plan on providing this directly via SmartScan cloud so users don’t need an OpenAI API key. This of course would be a paid feature to cover server cost. Once descriptions are generated, they’re stored and embedded locally on-device, so everything continues to work offline from that point onward.</p><p>Grouping by concepts is just the first stage. To truly extract value from the created concepts there are 2 supporting features I plan to add in the near future, Concept Reminders and Chat with Concepts.</p><h4><strong>Concept Reminders</strong></h4><p>Concepts become more useful when the information within them is resurfaced at the right time.</p><p>Concept Reminders will periodically bring relevant ideas back based on things like recent searches, newly added media, or concepts that have become more active. This would make it easier to rediscover useful information without having to remember that it exists or actively search for it.</p><h4>Chat with Concepts</h4><p>This feature would allow users to interact with the knowledge collected within a concept using AI.</p><p>For example: “How does Google’s developer verification effect the FOSS Android ecosystem?”</p><p>The AI would answer using the media stored within that concept.</p><p>This would make Concepts more than just a way to organise media. They would become focused knowledge spaces where you can explore, connect, and extract insights from the information you’ve collected over time.</p><h4>The Future of SmartScan</h4><p>I felt the need to make the direction of SmartScan explicitly clear because I’ve had numerous users mistake it for a typical gallery app, which is understandable because there is heavy overlap. But to be frank, for users looking for a traditional gallery experience, there are many better alternatives. Going forward, knowledge management will be the focus.</p><p>The <a href="https://github.com/smartscanapp/smartscan-android/releases/tag/v2.0.0"><strong>latest version of SmartScan</strong></a>, has been released with the Concepts feature, as well as several other improvements to the app. One of the other notable changes is improved search through the use of media descriptions. My next post will provide technical insight into the changes made to search, as well as the challenges involved in implementing it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d783ff85759d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cosine similarity isn’t enough for search relevance — why you need re-ranking]]></title>
            <link>https://medium.com/@d41dev/cosine-similarity-isnt-enough-for-search-relevance-why-you-need-re-ranking-c9d416972e61?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9d416972e61</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[vector-search]]></category>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[semantic-search]]></category>
            <category><![CDATA[information-retrieval]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Wed, 17 Jun 2026 17:34:24 GMT</pubDate>
            <atom:updated>2026-06-17T17:39:46.754Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9HdK0nCiT64a1_iTMnpAUA.png" /></figure><p>In a <a href="https://medium.com/@d41dev/smartscan-update-cluster-based-search-and-new-collections-feature-064f81b3eee5">previous article</a>, I wrote about a solution I used to try and improve the relevance of search results in my android app <a href="https://github.com/smartscanapp/smartscan-android">SmartScan</a>. As a quick recap, I used a cluster-based retrieval approach using the top 3 cluster matches and only items within those clusters are searched. While it did improve relevance in some cases, it ultimately fell short because expected images and videos were often excluded from the results due to the top 3 clusters limit, and I ended up scrapping it.</p><p>In hindsight, that approach was quite naive and failed to address the real issue. Instead of solely relying on cosine similarity, I needed a re-ranker, which seems quite obvious now.</p><h3>Re-ranking search results</h3><p>My previous cluster-based retrieval approach was along the right lines (using clusters matches as a strong signal), but it missed the key re-ranking step.</p><p>The approach shown below re-ranks results by combining each item’s similarity to the query with the similarity of the item’s cluster to the query.</p><pre><br>val itemToCluster = buildMap {<br>    clusterToMediaIdsMap.forEach { (clusterId, items) -&gt;<br>        items.forEach { itemId -&gt; put(itemId, clusterId) }<br>    }<br>}<br><br>val clusterSims = itemToSimMap.keys.mapNotNull { itemId -&gt;<br>        itemToCluster[itemId]?.let(clusterToSimMap::get)?.toDouble()<br>    }.sorted()<br><br>val itemSims = itemToSimMap.values.map { it.toDouble() }.sorted()<br>val iP10 = itemSims.getOrNull((itemSims.size * 0.1).toInt()) ?: 0.0<br>val iP90 = itemSims.getOrNull((itemSims.size * 0.9).toInt()) ?: 0.0<br>val itemSpread = iP90 - iP10<br><br>val p10 = clusterSims.getOrNull((clusterSims.size * 0.1).toInt()) ?: 0.0<br>val p90 = clusterSims.getOrNull((clusterSims.size * 0.9).toInt()) ?: 0.0<br>val clusterSpread = p90 - p10<br><br>val eps = 1e-3<br>val ratio = clusterSpread / itemSpread.coerceAtLeast(eps)<br>val k = ratio.pow(3).coerceAtLeast(1.0)<br><br>val scoredItems = itemToSimMap.keys.map { itemId -&gt;<br>    val itemScore = itemToSimMap[itemId]?.toDouble() ?: 0.0<br>    val clusterScore = itemToCluster[itemId]?.let(clusterToSimMap::get)?.toDouble() ?: 0.0<br>    itemId to (itemScore * (1 + k * clusterScore))<br>}.sortedByDescending { it.second }<br><br></pre><p>It estimates how much influence cluster relevance should have by comparing the similarity spread of cluster-query similarities against item-query similarities, increasing cluster influence when cluster similarities vary more strongly.</p><p>The `<strong><em>ratio`</em></strong> is a proxy for comparing the signal strength of item-query similarity and cluster-query similarity. If the ratio is above 1, the cluster signal is stronger; if it is below 1, the item signal is stronger. Typically, the cluster signal is often stronger because cluster embeddings represent the aggregate semantics of many related items, which emphasise the dominant semantic features across related items.</p><p>The variable `<strong><em>k</em></strong>` represents the boost factor derived from the ratio. Because cosine similarity scores between text and images usually fall within a tight range, I use the cube of the ratio to amplify the boost:</p><pre>val k = ratio.pow(3).coerceAtLeast(1.0)</pre><p>The results are then sorted by:</p><pre>val score = itemSimilarity * (1 + k * clusterSimilarity)</pre><p>This boosts items that belong to highly relevant clusters while still preserving the influence of the item’s own similarity score, ensuring that the most relevant results are pushed to the top.</p><h3>Filtering irrelevant results</h3><p>Re-ranking improves ordering, but it does not remove weak matches, this requires a filtering step.</p><p>The challenge is determining where to draw the cutoff. A fixed threshold is simple but tends to perform poorly across different queries because score distributions vary significantly. Some queries produce a few very strong matches, while others generate a much flatter distribution.</p><p>To handle this, I use two separate cutoff mechanisms and take the larger of the two as can be seen in the code snippet below.</p><pre><br>val scores = scoredItems.map { it.second }.sorted()<br>val maxScore = scores.max()<br>val minScore = scores.min()<br>val medianScore = scores[scores.size / 2]<br>val meanScore = scores.average()<br>val stdScore = sqrt(scores.map{(it - meanScore).pow(2)}.average())<br>val centrality = medianScore - minScore<br>val safeStrictness = strictness.coerceIn(0f, 1f)<br>val baseCutOff = baseCutOffPercent * maxScore<br>val dynamicCutOff = medianScore - stdScore - (1 - safeStrictness) * centrality.pow(1.0 + safeStrictness)<br>val minAllowedScore = max(baseCutOff, dynamicCutOff)<br><br>return scoredItems<br>    .filter { it.second &gt;= minAllowedScore }<br>    .map { it.first }<br></pre><h4>Base cutoff</h4><p>The first threshold is a simple percentage of the highest-ranked score:</p><pre>val baseCutOff = baseCutOffPercent * maxScore</pre><p>With the default value of 0.6, any result scoring less than 60% of the top result is rejected.</p><p>This acts as a safety net when there is a clear standout result. The main advantage of this approach is its simplicity and predictability since results are judged relative to the best match for the query.</p><h4>Dynamic cutoff</h4><p>The second threshold adapts to the shape of the score distribution:</p><pre>val dynamicCutOff = medianScore - stdScore -<br> (1 - safeStrictness) * centrality.pow(1.0 + safeStrictness)</pre><p>This threshold uses several statistics from the re-ranked scores:</p><ul><li><strong>Median score </strong>represents the centre of the distribution.</li><li><strong>Standard deviation</strong> measures how spread out the scores are.</li><li><strong>Centrality</strong> measures how far the middle of the distribution is from the weakest result.</li></ul><p>Unlike the base cutoff, this threshold is not tied to the highest-scoring item. Instead, it attempts to identify where the relevant portion of the distribution ends and the irrelevant tail begins.</p><p>The `<strong><em>strictness</em></strong>` parameter is user-controlled and determines how aggressively low-scoring results are filtered. Higher values return fewer, more relevant results, while lower values return a broader set of matches.</p><h4>Why use both?</h4><p>The two thresholds address different characteristics of the score distribution.</p><p>The base cutoff works well when there is a clear best result, while the dynamic cutoff adapts to queries where scores are more evenly distributed. Using the larger of the two thresholds provides a consistent minimum quality level while still adapting to the shape of individual result sets:</p><pre>val minAllowedScore = max(baseCutOff, dynamicCutOff)</pre><p>Only items whose re-ranked score exceeds this value are returned. This re-ranking approach can significantly reduce irrelevant results without discarding relevant results from broader or more ambiguous queries, making it vastly superior to my previous naive attempt at improving the relevance of search results.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9d416972e61" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SmartScan Update: Cluster-Based Search and New Collections Feature]]></title>
            <link>https://medium.com/@d41dev/smartscan-update-cluster-based-search-and-new-collections-feature-064f81b3eee5?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/064f81b3eee5</guid>
            <category><![CDATA[android]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[android-apps]]></category>
            <category><![CDATA[semantic-search]]></category>
            <category><![CDATA[android-app-development]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Tue, 21 Apr 2026 17:54:32 GMT</pubDate>
            <atom:updated>2026-04-21T17:54:32.068Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*toGTjnzJIECa05nMrA6EGQ.png" /></figure><p>The <a href="https://github.com/smartscanapp/smartscan-android/releases/tag/v1.3.0"><strong>latest version of SmartScan</strong></a><strong> </strong>improves search with faster, more relevant results and implements a new Collections feature, both enabled by incremental clustering of media.</p><p>Below I will outline the challenges of on-device semantic search, the limitations of the previous search implementation, and how I addressed them, followed by an introduction to the new Collections feature.</p><h3>Challenges and issues with search</h3><p>SmartScan’s core functionality is enabled by CLIP models. While the <a href="https://github.com/smartscanapp/smartscan-android-lib"><strong>SmartScan SDK</strong></a> supports other models, only CLIP is multi-modal. CLIP has lower accuracy compared to modern multi-modal models. This limitation primarily affects text queries, while image queries perform better. Replacing it with larger models is impractical due to mobile constraints.</p><p>This issue manifest as many irrelevant images or videos being included in results. For example, a search with the query “man” ends up including over 200 irrelevant images as shown below (left), whereas the new search approach (right), which i will go into detail on later, instead shows 38 relevant images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sXPx9tOjqjZu_FOzykF4JA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vzIYsZxMJmB8beLXd5fK9w.png" /><figcaption>Test example showing search results using old approach (left) vs new cluster-based retrieval approach (right)</figcaption></figure><p>Mitigating this by tuning the required similarity threshold is a brittle solution because with text queries, the difference between the cosine similarity of relevant results and irrelevant results is often extremely small (0.005–0.1), where even the most similar items would probably have a max sim of roughly 0.3.</p><p>Another separate issue with the previous search implementation is that it relies on a brute-force similarity search, which makes searches slower as the number of items grows.</p><h3>The solution to the search problem</h3><p>To mitigate the described problems, I initially considered using Approximate-Nearest-Neighbour (ANN) search, which queries a <a href="https://www.pinecone.io/learn/series/faiss/hnsw/"><strong>HNSW</strong></a> index to retrieve the top-k closest embeddings in sublinear time, reducing the number of comparisons and returning only the most similar results, improving both speed and relevance.</p><p>The major drawback with this approach is that it’s incompatible with exhaustive search, which is a key aspect of the app. For example, say a user has 1000 images of cats, the ANN approach would only show the top-k results e.g 30 rather than all cats. This made it a no go.</p><p>Instead, I decided that the optimal approach is cluster-based retrieval. During search, the query is first compared to cluster centroids, the top 3 cluster matches are selected and only items within those clusters are searched. This restricts brute-force comparison to a smaller, semantically related subset of the data, reducing the number of comparisons and excluding unrelated items, which improves both speed and relevance.</p><p>The main benefit of this over the ANN approach is that it still allows for an exhaustive search of items within those clusters. Using the previous cat example, the 1000 cat images would all be in the same cluster (or at least in the top 3), and therefore all will be included in the results. That being said, this solution is not perfect. Similar to the ANN approach, though to a much lesser extent, it can still exclude results a user might expect. Therefore, I made it a toggle-enabled setting users can control.</p><p>The clusters are created using an incremental clustering approach after indexing, allowing clusters to adapt as new media is indexed. This same technique is core to the new Collections feature.</p><h3>Collections: The evolution of auto-organisation</h3><p>For those of you who have followed the journey of SmartScan, you may remember the “auto-organisation” feature, which was removed.</p><p>It was removed because it didn’t provide the best UX, as it required users to essentially manually create and seed clusters. The incremental clustering approach avoids this and instead allows for dynamic unsupervised creation of clusters. The second problem with the old auto-organisation feature is that it moved user media on the device file-system, changing the actual file locations. This became problematic, especially due to the underlying model accuracy issues already described.</p><p>In contrast, the new collections feature leaves all media files untouched on the device and only manages an internal, in-app layer of organisation. Collections define a private in-app grouping of media without affecting how or where files are stored on the file-system. There are 2 types of in-app collections. User created collections aka “Tag Collections” and cluster created collections aka “Auto Collections”. Users can choose whether to manually build their collections with tags, use only auto collections or some hybrid approach.</p><p>Because Auto Collections are derived from clusters created during indexing, which are used for search, they’re treated as “read-only”. Users cannot modify them directly, except for renaming. This avoids conflicts between user-defined groupings and the clustering algorithm, which could affect search retrieval. If changes are needed, users can bulk-copy items into Tag Collections, where full modification control is available.</p><h3>Reflection on the changes</h3><p>The cluster-based retrieval approach to search is definitely a significant improvement over the previous implementation. That being said, it’s still subject to the same underlying model accuracy issues, since cluster formation is directly tied to the quality of embeddings produced by the model. This also affects Auto Collections in the same way, meaning in some cases they can fall below user expectations and contain incorrect media.</p><p>Furthermore, CLIP is optimised for image–text alignment rather than high-quality visual representation learning. This means that pure vision models supported by the SmartScan SDK, such as DINOv2, would likely produce more accurate clusters but these clusters wouldn’t be compatible with text queries.</p><p>In theory, I could use separate models for Auto Collections and cluster-based retrieval, but then this would add complexity that may not yet be justified. For now I’ll wait to see user feedback on the Collections feature.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=064f81b3eee5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sereleum: Building a prompts analytics platform]]></title>
            <link>https://medium.com/@d41dev/sereleum-building-a-prompts-analytics-platform-b174468cb021?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/b174468cb021</guid>
            <category><![CDATA[writing-prompts]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Sat, 31 Jan 2026 12:30:54 GMT</pubDate>
            <atom:updated>2026-03-12T17:53:46.471Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DwrQ9sHvXA2bNOP-rNsm2w.png" /><figcaption>Sereleum dashboard page</figcaption></figure><p><a href="https://sereleum.vercel.app/">Sereleum</a> is a prompts analytics platform that helps businesses turn user prompts into actionable insights. It uncovers semantic patterns, tracks LLM usage, and informs product optimisation.</p><p>After building, <em>SmartScan</em>, an android app for searching images and videos offline using text or by reverse image search with on-device AI, I became very interested with vector embeddings and how they can be universally applied to a wide range of use cases that require search and or classification. It also led to me making the <em>SmartScan</em> python library which Sereleum is built on top off.</p><p>To truly understand the value of Sereleum its important to understand the relationship between system prompts and user prompts and their roles.</p><h3>System Prompts vs User Prompts</h3><p>System prompts provide LLMs with foundational instructions that define behaviour, constraints, and objectives.</p><p>User prompts are dynamic inputs from end users, usually questions or commands, reflecting real-user intent and needs.</p><p>Analysing user prompts reveals actionable insights to enhance UX, optimise system prompts to better align with the most significant use cases and manage prompt-related costs.</p><p>Sereleum provides businesses with a platform to do this effectively.</p><h3>What Makes Sereleum Fundamentally Different</h3><p>A lot of the existing prompt analytics platforms focus on system prompts rather than user prompts. Sereleum does the opposite, using the approach outline below.</p><ul><li><strong>Incremental clustering:</strong> Continuously updates clusters as new prompts arrive, avoiding costly full re-clustering. Producing durable clusters that evolve over time.</li><li><strong>Semantic labelling: </strong>LLM auto-labelling (with manual fallback) converts clusters into business-readable concepts.</li><li><strong>Cost-linked insights:</strong> Connects intent clusters to token usage, translating semantic patterns into actionable financial signals.</li><li><strong>Direct product feedback:</strong> Insights feed into system prompts, prompt templates, UX, and cost controls.</li></ul><p>In short, Sereleum is designed to answer the following questions:</p><ul><li><strong>What are users trying to do?</strong></li><li><strong>How often does each intent occur?</strong></li><li><strong>How much does each intent cost?</strong></li><li><strong>And how should the product change as a result?</strong></li></ul><h3><strong>How Sereleum Works</strong></h3><p>At Sereleum’s core is a 3 step process of indexing, clustering and labelling.</p><h4>Step 1: Indexing</h4><p>For now users can only add prompts via JSON file upload (other methods will be supported). Vector embeddings are then generated for each prompt, and the prompt, vector embedding and associated metadata are stored.</p><h4>Step 2: Clustering</h4><p>Prompt embeddings are incrementally clustered, allowing clusters to adapt as new prompts arrive.</p><h4>Step 3: Labelling</h4><p>Sample prompts from each cluster are selected and passed to an LLM to label the cluster. If confidence for the label is below the threshold, then the cluster is left unlabelled and the system falls-back to manual labelling. Labels can be edited manually at any time in the app.</p><h3>How Sereleum Helps Businesses</h3><ul><li><strong>User intent visibility:</strong> Reveals what users are trying to achieve and how frequently each intent occurs.</li><li><strong>Cost attribution:</strong> Connects intents to token usage, identifying high-cost behaviours.</li><li><strong>Actionable insights:</strong> Directly informs product decisions, system prompt adjustments, and prompt template optimisations.</li></ul><h4>Example Outcomes</h4><p><strong>Use case discovery and prioritisation<br></strong> <em>Example: Frequent prompts around “automating reports” surface reporting automation as a core workflow worth investing in.</em></p><p><strong>System prompt alignment<br></strong> <em>Example: User intent clusters show a preference for casual writing, prompting an adjustment to the system prompt’s tone constraints.</em></p><p><strong>Cost reduction<br></strong> <em>Example: High-cost use cases identified and optimised prompt templates are implemented to lower token usage.</em></p><p>The result is faster product iteration, better alignment with real user needs, and controlled LLM costs.</p><h3>Want to try Sereleum?</h3><p>Try the early access version of <a href="https://sereleum.vercel.app/">Sereleum</a>, where you can upload sample prompts and get a free report.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b174468cb021" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Turning SmartScan into a System-Wide Embedding Platform via AIDL Services]]></title>
            <link>https://medium.com/@d41dev/turning-smartscan-into-a-system-wide-embedding-platform-via-aidl-services-d170c14acefa?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/d170c14acefa</guid>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Wed, 05 Nov 2025 20:16:52 GMT</pubDate>
            <atom:updated>2026-01-04T11:13:08.275Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pOM3XrbIewoGdynNvIqk6Q.png" /></figure><p>Up until recently, I had no clue what <a href="https://developer.android.com/develop/background-work/services/aidl"><strong>AIDL services</strong></a><strong> </strong>were. But I had an idea which led to me stumbling on it. I was curious to see if there was a way I could leverage <a href="https://github.com/smartscanapp/smartscan-android"><strong>SmartScan’s</strong></a> existing ML infrastructure to power similarity search for one of my old notes app.</p><p>This seemed more logical than re-implementing all the same code again and allows keeping the notes app lean. While searching for a solution I came across AIDL Services. And then I had what felt like a light bulb moment — turn SmartScan into a System-Wide Embedding Platform, to enable similarity search and classification features for any app I make on my device.</p><p>If you are wondering about what AIDL Services are I will be explaining it shortly, as well as how i integrated it into SmartScan and how client apps can use the services.</p><h3>What are AIDL Services</h3><p>AIDL defines an interface for inter-process communication (IPC) between Android components in different processes. It automates the marshalling and unmarshalling of data so services and clients can communicate using defined methods and data types.</p><h3>Building the AIDL Services</h3><p>SmartScan, as mentioned previously, already had the core ML infrastructure for generating embeddings, powered by my <a href="https://github.com/dev-diaries41/smartscan-sdk"><strong>SmartScan SDK</strong></a>, including the management of models (importing and exporting) — though this is not currently available to users in the latest public releases on F-droid and GitHub. This made creating the AIDL service quite simple.</p><p>The steps can be outline as follows.</p><h4><strong>Step 1: Enable AIDL in build features in the build.gradle file</strong></h4><pre>  buildFeatures {<br>        // other build features...<br>        aidl = true<br>    }</pre><h4>Step 2: Create the interfaces</h4><p>AIDL services have limited data type support. As a result, in the IImageEmbedderService interface, a byte array is used to represent the input image(s) instead of Bitmap. In the case of the embedBatch method, the input data is a concatenated byte array of byte arrays (more on this on Step 3). Similarly, the returned float array is also concatenated (also for the ITextEmbedderService).</p><pre>package com.fpf.smartscan;<br><br>import java.util.List;<br><br>interface IImageEmbedderService {<br>    int getEmbeddingDim();<br>    byte[] getDelimiter();<br>    void closeSession();<br>    float[] embed(in byte[] data);<br>    float[] embedBatch(in byte[] data);<br>    List&lt;String&gt; listModels();<br>    boolean selectModel(in String name);<br>}</pre><pre>package com.fpf.smartscan;<br><br>import java.util.List;<br><br>interface ITextEmbedderService {<br>    int getEmbeddingDim();<br>    void closeSession();<br>    float[] embed(in String data);<br>    float[] embedBatch(in List&lt;String&gt; data);<br>    List&lt;String&gt; listModels();<br>    boolean selectModel(in String name);<br>}</pre><h4>Step 3: Create a service which implements the interfaces</h4><p>For brevity I will only show the implementation of the IImageEmbedderService as they are similar. As you can see the embed method requires converting the byte array to a bitmap before calling the embed method on the imageEmbedder, which expects a bitmap. For the embedBatch method, you can see the byte array is first decoded using the decodeByteArrayPayload, this is required to extract each individual image correctly. The client side uses the same delimiter for encoding.</p><pre>class ImageEmbedderAidlService: Service() {<br><br>    companion object {<br>        const val TAG = &quot;ImageEmbedderAidlService&quot;<br>        private const val DEFAULT_MODEL = &quot;Default&quot;<br>    }<br>    private lateinit var imageEmbedder: ImageEmbeddingProvider<br><br>    override fun onCreate() {<br>        super.onCreate()<br>        imageEmbedder = ClipImageEmbedder(application, ResourceId(R.raw.image_encoder_quant_int8))<br>    }<br><br>    override fun onBind(intent: Intent): IBinder {<br>        return binder<br>    }<br><br>    override fun onDestroy() {<br>        super.onDestroy()<br>        imageEmbedder.closeSession()<br>    }<br><br>    private val binder = object : IImageEmbedderService.Stub() {<br><br>        override fun getEmbeddingDim(): Int {<br>            return imageEmbedder.embeddingDim<br>        }<br><br>        override fun closeSession() {<br>            imageEmbedder.closeSession()<br>        }<br><br>        override fun embed(data: ByteArray): FloatArray? {<br>            return runBlocking {<br>                try {<br>                    if (!imageEmbedder.isInitialized()) imageEmbedder.initialize()<br>                    val embedding = imageEmbedder.embed(byteArrayToBitmap(data))<br>                    embedding<br>                } catch (e: Exception) {<br>                    Log.d(TAG, &quot;EMBEDDING_ERROR: ${e.message}&quot;)<br>                    null<br>                }<br>            }<br>        }<br><br>        override fun embedBatch(data: ByteArray): FloatArray? {<br>            return runBlocking {<br>                try {<br>                    if(!imageEmbedder.isInitialized()) imageEmbedder.initialize()<br>                    val decoded = decodeByteArrayPayload(data, delimiter)<br>                    val bitmaps = decoded.map { byteArrayToBitmap(it) }<br>                    val embeddings = imageEmbedder.embedBatch(bitmaps)<br>                    flattenEmbeddings(embeddings, imageEmbedder.embeddingDim)<br>                }catch(e: Exception){<br>                    Log.d(TAG, &quot;EMBEDDING_ERROR: ${e.message}&quot;)<br>                    null<br>                }<br>            }<br>        }<br><br>        override fun listModels(): List&lt;String&gt; {<br>            return getImportedModels(application).filter { it.type == SmartScanModelType.IMAGE_ENCODER }.map { it.name }<br>        }<br><br>        override fun selectModel(model: String): Boolean {<br>           // Currently no other image models<br>            return true<br>        }<br><br>        override fun getDelimiter(): ByteArray {<br>            return byteArrayOf(0xFF.toByte(), 0x00, 0xFF.toByte(), 0x00)<br>        }<br>    }<br><br>    private fun byteArrayToBitmap(bytes: ByteArray): Bitmap{<br>        return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)<br>    }<br><br>    private fun decodeByteArrayPayload(bytes: ByteArray, delimiter: ByteArray): List&lt;ByteArray&gt; {<br>        val recoveredPayload = mutableListOf&lt;ByteArray&gt;()<br>        var pos = 0<br>        var i = 0<br><br>        while (i &lt;= bytes.size - delimiter.size) {<br>            var match = true<br>            for (j in delimiter.indices) {<br>                if (bytes[i + j] != delimiter[j]) {<br>                    match = false<br>                    break<br>                }<br>            }<br><br>            if (match) {<br>                recoveredPayload.add(bytes.sliceArray(pos until i))<br>                i +=  delimiter.size<br>                pos = i<br>            } else {<br>                i++<br>            }<br>        }<br><br>        if (pos &lt; bytes.size) {<br>            recoveredPayload.add(bytes.sliceArray(pos until bytes.size))<br>        }<br><br>        return recoveredPayload<br>    }<br>}</pre><h4>Step 4: Add the service to AndroidManifest.xml</h4><p>Define the permissions.</p><pre>&lt;permission android:name=&quot;com.fpf.smartscan.permission.TEXT_EMBED&quot; android:protectionLevel=&quot;normal&quot; /&gt;<br>&lt;permission android:name=&quot;com.fpf.smartscan.permission.IMAGE_EMBED&quot; android:protectionLevel=&quot;normal&quot; /&gt;</pre><p>Define the services (with intent filters)</p><pre>&lt;service<br>    android:name=&quot;com.fpf.smartscan.services.TextEmbedderAidlService&quot;<br>    android:exported=&quot;true&quot;<br>    android:permission=&quot;com.fpf.smartscan.permission.TEXT_EMBED&quot;<br>    &gt;<br>    &lt;intent-filter&gt;<br>        &lt;action android:name=&quot;com.fpf.smartscan.intent.TEXT_EMBED&quot;/&gt;<br>    &lt;/intent-filter&gt;<br>&lt;/service&gt;<br>&lt;service<br>    android:name=&quot;com.fpf.smartscan.services.ImageEmbedderAidlService&quot;<br>    android:exported=&quot;true&quot;<br>    android:permission=&quot;com.fpf.smartscan.permission.IMAGE_EMBED&quot;<br>    &gt;<br>    &lt;intent-filter&gt;<br>        &lt;action android:name=&quot;com.fpf.smartscan.intent.IMAGE_EMBED&quot;/&gt;<br>    &lt;/intent-filter&gt;<br>&lt;/service&gt;</pre><p>The permissions part of this is frustrating for me because its caused me to have to think about some annoying trade-offs. Using the permission level as `normal` works so long as the SmartScan app is installed on the device first, if the client app is installed before SmartScan, then a SecurityException is thrown stating the the client can’t bind to the service.</p><p>This is because normal permissions are granted upon installation. If the server app (SmartScan) is not available at the time of installation ( or deleted after), then the system is unaware of the required permissions, and therefore cannot grant them to the client app.</p><p>This problem is avoided when the permission is set to signature because the OS doesn’t check for permissions at runtime, instead it just checks both apps signature match when a bind is attempted. However, this is restrictive because other devs won’t able to use SmartScan’s AIDL Services, since their apps won’t be signed with my key. The other alternative is to maybe use no permissions at all, but I’m unsure about doing this.</p><h3>How Client Apps can use the Services</h3><p>I created a lightweight <a href="https://github.com/dev-diaries41/smartscan-client"><strong>library</strong></a><strong> (</strong>not officially released yet<strong>)</strong>, which enables Android apps to connect to the SmartScan app’s AIDL services for generating embeddings from text or images. It provides simple client interfaces for the Text Embedder and Image Embedder services and handles all the setup. The README at the provided repo link also covers how to set it up yourself if you decide not to use the library.</p><h3>What this means for SmartScan going forward</h3><p>SmartScan originally started of as a consumer type app, but since finding out about AIDL Services, it can now become a &quot;system-level&quot; type app. As of now it just provides Embedding services, but as the on-device AI ecosystem matures I plan on adding additional AIDL related features, making it a central hub providing on-device AI features.</p><p>I could maybe add the use of LLMs now, but TBH I don’t see the point. IMO the LLM models suitable for use on Android right now are pretty useless / will sacrifice performance.</p><p>In an ideal world I would love if I could actually make it an official OS System App, but that seems unlikely unless I make my own or partner with an existing Custom ROM, who would need to see the value in this service.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d170c14acefa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SmartScan Update: Performance Lessons, SmartScan SDK and The Road Ahead]]></title>
            <link>https://medium.com/@d41dev/smartscan-update-performance-lessons-smartscan-sdk-and-the-road-ahead-2ac589f381e2?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ac589f381e2</guid>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[devlog]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[androiddev]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Sat, 04 Oct 2025 06:48:05 GMT</pubDate>
            <atom:updated>2026-01-04T11:14:11.456Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yCfzjxP3hDQX5I25fXvhBw.png" /></figure><p>This week I released <a href="https://github.com/smartscanapp/smartscan-android"><strong>SmartScan v1.1.5</strong></a>. I haven’t made a dev log about SmartScan since the release of v1.1.0, where the last major change was the addition of video search — though somewhat still in its infancy due to some current codec issues. From the users perspective, the changes since then have mainly been incremental improvements of UX, rather than any major changes to key functionality, with one of these incremental UX improvements being driven by a fundamental change in the way embeddings are handled on device — more on this later.</p><p>However, behind the scenes there have been big changes to the codebase, driven by my personal growth and goals as a developer, and my long-term vision for the app, which I will run through in this catch-up.</p><h3>How I made Index loading x100 faster</h3><p>One of the challenges with building an app with on-device AI powered search capability is that the ability to search requires the entire index to be loaded in memory, since there’s no native VectorDB for Android. Originally, the app naively used RoomDB to store embeddings. With this approach, loading the index for a few hundred to a couple thousand images only took a few seconds. However, as the number of images grows large e.g 50k, the loading time, can take minutes, which essentially nullifies the search functionality and kills UX.</p><p>The solution? Storing the embeddings as a binary file and loading the embedding using a memory mapped approach.</p><pre>    override suspend fun getAll(): List&lt;Embedding&gt; = withContext(Dispatchers.IO){<br>        cache?.let { return@withContext it.values.toList() };<br><br>        FileInputStream(file).channel.use { ch -&gt;<br>            val fileSize = ch.size()<br>            val buffer = ch.map(FileChannel.MapMode.READ_ONLY, 0, fileSize).order(ByteOrder.LITTLE_ENDIAN)<br><br>            val count = buffer.int<br>            val map = LinkedHashMap&lt;Long, Embedding&gt;(count)<br><br>            repeat(count) {<br>                val id = buffer.long<br>                val date = buffer.long<br>                val floats = FloatArray(embeddingLength)<br>                val fb = buffer.asFloatBuffer()<br>                fb.get(floats)<br>                buffer.position(buffer.position() + embeddingLength * 4)<br>                map[id] = Embedding(id, date, floats)<br>            }<br>            if (useCache) cache = map<br>            map.values.toList()<br>        }<br>    }</pre><p>This made index loading nearly up to x100 faster in some cases, as shown in the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zYUxkl-AFi1ujHCSaNp1_g.png" /><figcaption>Instrumented test benchmarks of Index loading</figcaption></figure><p>Its important to note the image shows results for instrument tests, not real life tests. I did run the same test for 2.5k images on my Android device and the results were somewhat similar.</p><p>Solving challenges related to AI on edge devices, like this and several others I’ve discussed in previous posts, has helped me decide what I want to specialise in and directly influenced the next major change behind the scenes — the creation of the SmartScan SDK.</p><h3>The birth of the SmartScan SDK</h3><p>Over the years, I’ve jumped between software domains and projects like most devs. Only now have I realized it is time to decide what I want to specialize in, something I’ve personally been wrestling with for a while. I plan to go into more detail in a separate post about the journey from generalist to specialist and why I think it’s necessary in today’s tech landscape.</p><p>My experience with SmartScan led me to focus on on-device AI for edge devices, as well as mechanistic interpretability, which I aim to use to improve AI integration on edge devices.</p><p>This focus led to the creation of <a href="https://github.com/dev-diaries41/smartscan-sdk">SmartScan SDK</a>, a modular Android SDK that powers the SmartScan app. It provides tools for:</p><ul><li>Image &amp; video processing</li><li>On-device ML inference</li><li>Semantic media indexing and search</li><li>Few shot classification</li></ul><p>The long-term vision for the SDK is to become a C++ cross-platform SDK for search and classification, capable of running both offline on different edge devices and in cloud environments for bulk processing.</p><p>The latest version of SmartScan now uses this SDK for all core ML logic, letting the app focus mainly on UI. Developers can use the SDK similarly to add search and classification features to their own apps. Separating the ML logic from the app also makes it easier for me to implement new search types, which I’ll cover in the SmartScan roadmap.</p><h3>SmartScan Roadmap: What to expect in the future</h3><h4>New search types</h4><p>Currently, SmartScan’s search and classification functionality is powered by a CLIP model, which is best suited for general-purpose image processing. I’m considering adding facial recognition and object detection-based search and classification. These would work similarly to the current system, except the new types would use image-to-image search rather than text-to-image search.</p><p>This will be enabled by providing downloadable models. The current SmartScan SDK design makes it easy to swap models while maintaining the same search functionality.</p><h4>Multi-language support</h4><p>I’m aware that many users of the app are from different countries so multi language support for both UI and search queries is important. I’ve been slow to add it but definitely will at some point.</p><h4>Potential removal or deprecation of the auto-organisation feature</h4><p>As mentioned earlier, CLIP’s accuracy is mediocre, which limits the effectiveness of the auto-organisation feature. While I’ve implemented some mitigating solutions, ultimately it needs to be replaced for auto-organisation to work reliably. This is something i plan to do if the app grows enough to support the model training required.</p><p>From user interactions, search is the feature people seem to care about most. Given this, I’m considering removing or deprecating the auto-organisation feature, as maintaining it may not be the best use of time. I haven’t made a final decision yet and will wait to hear user feedback before proceeding.</p><p>If you want to help shape the roadmap join the <a href="https://www.reddit.com/r/SmartScan/"><strong>r/SmartScan</strong></a> subreddit to share your feedback and ideas.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ac589f381e2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Google’s War on the FOSS Ecosystem]]></title>
            <link>https://medium.com/@d41dev/googles-war-on-the-foss-ecosystem-733cc79a8d23?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/733cc79a8d23</guid>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[privacy]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Wed, 27 Aug 2025 07:22:20 GMT</pubDate>
            <atom:updated>2025-08-27T07:24:04.420Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DXsfiQAH-t2nxKBE0QUHzg.png" /></figure><p><em>Do you really own your device if you can’t control what software can run on it?</em></p><p>Android was originated by a group of companies called the Open Handset Alliance (OHA), led by Google, with the goal of creating an open platform for mobile innovation. But that no longer seems to be the case.</p><p>Google’s new D<a href="https://android-developers.googleblog.com/2025/08/elevating-android-security.html"><strong>eveloper Verification</strong></a><strong>,</strong> branded as “a new layer of security,” might be the final nail in the coffin for an open Android ecosystem. In this post, I’ll break down what developer verification is and what it means for sideloading and the FOSS ecosystm.</p><h3><strong>Sideloading: The Gateway to the FOSS Ecosystem</strong></h3><p>One of the key features that Android users like myself, love over iOS, is the ability to sideload apps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*Gguqtd7CPJDVBP-KgBBAjg.png" /><figcaption>Redditor highlights sideloading freedom as a major advantage over iOS</figcaption></figure><p>Sideloading is the process of installing an app from a source other than the Google Play Store (the Gatekeepers), typically by downloading an APK file and manually installing it via the Package Manager, providing users with a greater level of freedom and control.</p><p>Sideloading is the fundamental feature that has enabled the growth of the FOSS Android ecosystem, the resistance against the Gatekeepers. This ecosystem includes apps that avoid proprietary Google services (e.g., Play Services, Play Store), prioritising privacy, control, and customizability, as well as custom ROMs like <strong>GrapheneOS</strong> or <strong>LineageOS </strong>that rely on sideloading to remain viable.</p><p>This capability has become increasingly crucial for developers since Google has made it harder, especially for privacy-conscious developers, to publish apps on the Play Store due to restrictive requirements. FOSS alternatives such as <strong>F-Droid</strong>, made possible by sideloading, provide developers with a way to distribute their apps simply and in a privacy-respecting manner — the way it should be.</p><p>But it seems like Google is covertly trying to kill this, with their new weapon, Developer Verification.</p><h3>Developer Verification: What It Means for FOSS</h3><p>Google recently introduced Developer Verification, a new requirement for apps to be installable on certified Android devices, where developers, who distribute apps outside of Play Store, must verify their identity by providing their name, address, email, and phone number via the Android Developer Console, as well as the apps package name. This will begin rolling out in September 2026.</p><p>For developers who only distribute on FOSS platforms like F-Droid, Obtainium, or direct sideloading, this could be a make-or-break shift.</p><h4>Impact on Developers</h4><p>FOSS developers typically build for three groups of Android users:</p><ol><li>Custom ROM users on compatible devices.</li><li>Stock OS users who “degoogle” their devices.</li><li>Rooted device users.</li></ol><p>Most FOSS developers are unlikely to participate in verification because of privacy concerns. That means their apps would only run on custom ROMs or rooted devices, cutting off users on certified stock devices. Since custom ROM support and the number of rooted users is already limited, this could significantly shrink the potential audience, enough to push some developer away from Android app development altogether. Something I’m personally considering.</p><h4>Impact on Users</h4><p>Users on custom ROMs or rooted devices may still access unverified apps, but developer interest in maintaining them could fade as the audience shrinks. Potentially impacting future usability of popular FOSS apps.</p><p>Degooglers who stick with stock OS because their devices don’t support alternatives will now be locked out of apps that skip verification, leaving them effectively trapped in Google’s ecosystem.</p><h3>Additional Barriers for FOSS on Android</h3><p>Previously released Google services such as <a href="https://developer.android.com/google/play/integrity/overview"><strong>Play Integrity API</strong></a> and <a href="https://developers.google.com/android/play-protect/client-protections"><strong>Google Play Protect</strong></a> create additional hurdles for the FOSS ecosystem.</p><p>Play Integrity can block apps from running on devices that do not meet Google’s certification standards, which includes most custom ROMs and many rooted devices. This effectively locks out users who rely on alternative operating systems or modify their devices for privacy and control.</p><p>Similarly, Play Protect can flag sideloaded or FOSS apps as potentially harmful, even when they are safe, creating a false perception that all apps outside the Play Store are unsafe and discouraging users from exploring FOSS alternatives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/431/1*Sjv7ooXoDuZv1OeWDzgSiQ.png" /><figcaption>Popular FOSS app Revanced blocked by Google Play Protect</figcaption></figure><p>Both tools do have legitimate security benefits. Play Integrity protects apps from fraud or tampering, and Play Protect scans for malware to alert users to genuine threats. However, these benefits do not outweigh the restrictions they impose on user freedom and the growth of independent developers.</p><p>Combined with Developer Verification, these measures make it increasingly difficult for FOSS apps to reach users on stock Android devices without compromising openness or privacy.</p><h3>Final Thoughts</h3><p>Framing Developer Verification as a “layer of security to protect against malware” is misleading, after all, isn’t that already the role of Google Play Protect? Developer verification masks the real issue, which is that some Android users simply lack sufficient security knowledge. A more effective approach would focus on user education and or providing meaningful security alerts for sideloaded apps.</p><p>If security were genuinely the motivation, there were far less restrictive ways to implement it. For example, a <strong>developer option </strong>in device settings that allows toggling the sideloading of apps. This would preserve freedom while still offering protection, and though it would be somewhat inconvenient, it would be a reasonable compromise.</p><p>What I love about Linux is its emphasis on user responsibility for device security. Android used to strike a similar balance, but recent changes, including developer verification, increasingly abstract that responsibility away. We don’t need this hand holding.</p><p>For me personally, these changes have made me seriously reconsider continuing Android development. I was already thinking of stepping away due to other personal goals. If I do continue, it will most likely be at the OS level, which seems prudent given Google’s monopolistic control over the Android ecosystem.</p><p>I think it’s time for a third major mobile OS.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=733cc79a8d23" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SmartScan v1.1.0: Video Search, Undo, and Killing Over-engineering]]></title>
            <link>https://medium.com/@d41dev/smartscan-v1-1-0-video-search-undo-and-killing-over-engineering-334199eb7900?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/334199eb7900</guid>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[androiddev]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Sat, 31 May 2025 07:46:37 GMT</pubDate>
            <atom:updated>2026-01-04T11:14:47.206Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ECTijHzl4j8ERic7PiKIQ.png" /></figure><p><a href="https://github.com/smartscanapp/smartscan-android"><strong>SmartScan</strong></a> started off as a simple tool to organise my images. Then I added text-based search for images. And now, with the latest release v1.1.0, users can also search videos. I could be wrong, but I think this might be the first FOSS Android app that supports on-device video search based on content similarity. Either way, I’m proud of how far it has come.</p><p>Another major addition in this release is the ability to “undo last scan”, which reverts all file changes made by the image auto-organisation feature. So let’s dive into how and why they came about. Let’s take a closer look at how and why these features came to be.</p><h3>Video search</h3><p>The integration of the video search feature is nearly identical to the way in which the image search feature was integrated, which you can read more about <a href="https://medium.com/@d41dev/design-decoded-the-architecture-design-choices-behind-smartscan-fea2940fb2c9"><strong>here</strong></a> with the main difference being in how the embedding is generated during the indexing process. Generating an embedding for an image is straightforward, the bitmap for the image is passed to the CLIP model, which outputs the embedding.</p><p>Videos, on the other hand, are trickier. CLIP was designed for text and images, not video. To make it work, I extract 10 evenly distributed frames from each video, generate embeddings for them, and then average those into a single prototype embedding. This is the same technique I use in the auto-organisation feature.</p><p>This approach seems to be working well so far, but it does come with a cost. Because I need to compute 10 embeddings per video, the indexing time is significantly higher, anywhere from 5 to 10 times longer than images. I plan to run tests to benchmark how search accuracy changes with fewer frames, likely in the 5 to 10 range, since reducing the number of frames is the only way to speed things up.</p><h3>Smarter Auto-Organisation: Undo and Sanity Checks</h3><p>The auto-organisation feature moves images from target folders to destination folders based on content similarity. If the similarity passes a certain threshold, the image is moved. The issue is that the accuracy of CLIP isn’t great, and false positives happen and as a result some users had images moved into unrelated folders.</p><p>One way to mitigate this, is to create a fallback folder like “Others” for low-confidence matches but that doesn&#39;t provide the best UX and seems more like a hacky work around rather than a real solution. One user who had a bunch of images moved incorrectly requested a way to undo image moves. That made total sense to me. So I added an “undo last scan” feature that gives users full control to revert the last auto-organisation operation.</p><p>To further reduce false positives, I introduced an additional check. Instead of relying solely on the similarity threshold, I now also compare the top two folder matches. If the difference in similarity between them is less than 5 percent, the image is not moved. This acts as a sanity check to ensure there is a clear best match.</p><h3>Realising when you are over-engineering</h3><p>Before the latest SmartScan release, image indexing used a dual WorkManager setup: a periodic worker, <em>ImageIndexWorker,</em> to query all images on the device and chain enqueue one-time worker’s, <em>ImageBatchWorker, </em>to handle image batches of 500. I went into detail on this in the blog post linked earlier under the video search section. Honestly, even when I wrote that post, I was already realising it was over-engineered. Integrating the video search feature just confirmed it.</p><p>If I’d tried to use the same setup for videos, I would’ve had to reduce the batch size from 500 to maybe 50, just to stay within the 10-minute execution limit of Android Workers, due to the increased processing time of videos. That would’ve meant a much larger number of enqueued workers, possibly hitting Android’s worker limit depending on the number of videos the user has, and some videos might not even get indexed as a result.</p><p>While this issue also existed with images, it becomes a real problem with video because the batch size has to be much smaller. Furthermore, doze mode which triggers for workers when the app is in the background, causes the indexing processes to be much slower, though this was not something users complained about it still seemed sub-optimal.</p><p>Because of all this, I ended up simplifying the whole thing by moving the indexing logic for both images and videos into a single Service. A flag determines whether to index images, videos or both. This change not only cut out the over-engineered dual worker setup but also cleaned up other parts of the app that had become too tightly coupled to it.</p><p>One of the main reasons I originally chose the WorkManager approach was to easily schedule periodic indexing of new images. But now, after each index run, I just store the <em>lastIndexTime</em> in Shared Preferences. Then, in the <em>onResume</em> of MainActivity, I check that value. If it’s past the user-defined indexing interval, I trigger the Service. Simple and effective.</p><h3>Wrapping Up</h3><p>SmartScan is still evolving. There’s plenty I want to explore such as, improving the accuracy of both search results and auto-organisation (which may mean training a new CLIP-style model), and eventually supporting multiple languages search queries, which is something I’m keen on doing since many of my users seem to be from countries where English is not the first language.</p><p>If you try v1.1.0, let me know what works, what breaks, and what’s missing. Bug reports, feature requests, or quick thoughts. All of it helps improve the app.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=334199eb7900" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Design decoded: The Architecture Behind Unmute]]></title>
            <link>https://medium.com/@d41dev/design-decoded-the-architecture-behind-unmute-11cd77ef9621?source=rss-9d15bed2c1fd------2</link>
            <guid isPermaLink="false">https://medium.com/p/11cd77ef9621</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[conversational-ai]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[systems-thinking]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[D41]]></dc:creator>
            <pubDate>Wed, 28 May 2025 17:32:27 GMT</pubDate>
            <atom:updated>2025-05-28T17:32:27.443Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8NK3xLn7MGmfBbHx0yGLAw.png" /><figcaption>Screenshot of Unmute platform</figcaption></figure><p>In this second installment of my <em>Design Decoded</em> series, where I break down the architectural and implementation choices of software systems and analyze their trade-offs, I’m looking at <a href="https://unmute.sh/">Unmute</a>, a new conversational AI platform released by Kyutai, which strongly embodies what this series is about. The team’s implementation choices come with clear trade-offs that reveal what they’re prioritizing. It’s not open source yet, so I can’t get into specific code-level decisions, only the general design, but they’ve stated they plan to open-source it soon. You can check out more by watching the video below.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F7_WbY4qtE-w%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D7_WbY4qtE-w&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F7_WbY4qtE-w%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/44b176c76af4fc54345b2cf77c194493/href">https://medium.com/media/44b176c76af4fc54345b2cf77c194493/href</a></iframe><p>Let’s start decoding the design choices.</p><h3>Design overview</h3><p>Unmute is designed following a cascading system which is composed of a speech-to-text model (STT), a text-to-speech (TTS) model, and a swappable LLM layer (they use Gemma 3 12B). The speech-to-text model transcribes what the user says, this is then fed into the LLM which generates the response text, and finally the text-to-speech model converts that response into audio.</p><p>The STT model is streaming-based and integrates semantic voice activity detection instead of relying on an external model, which helps reduce latency in the first leg of the pipeline. On the other end, the TTS model is also streaming, both in terms of audio output and text input, meaning it can begin speaking before the LLM has finished generating the entire response. This parallelization is a key part of how they’ve kept end-to-end latency low.</p><p>Another key feature of the design is its modularity: each component (STT, LLM, TTS) is decoupled, which allows you to swap in a different LLM or voice model without requiring fine-tuning or adaptation. In the current demo on their website, you can adjust the system prompt of Gemma 3 to experiment with different personality behaviours, and independently change the voice of the TTS via a short 10-second sample.</p><h3>Trade-offs analysis</h3><p>The design is modular, flexible, and well-suited to rapid iteration. The use of a decoupled cascading architecture (STT → LLM → TTS) improves testability, facilitates debugging, and allows those managing the system to identify and address performance bottlenecks more easily. Critically, it enables rapid experimentation since the LLM component can be swapped out without retraining the rest of the pipeline, which is ideal given the fast-evolving nature of foundation models (o3, Sonnet 4, Gemini 2.5 Pro), where benchmark leaders seems shift frequently. Assuming they open-source it as planned, this will give developers, as well as the team, the flexibility to select the model most appropriate for their use case, and control over inference costs by choosing models based on cost-efficiency.</p><p>However, this approach isn’t without drawbacks. While streaming architectures minimize latency, a major win for responsiveness, they limit the system’s ability to process global context and capture emotional nuance or non-literal cues such as sarcasm. These elements are particularly important in use cases like mental health support or emotionally aware conversational agents. Overall, the trade-offs are rational and aligned with their primary goal of low latency, which is the most critical factor in real-time conversational AI. In this context, the advantages, especially modularity, substantially outweigh the limitations.</p><p>For developers, Unmute highlights the importance of defining your core goals early, since those choices affect every layer of implementation and ultimately impact user experience. Clear priorities guide architectural decisions, trade-offs, and even the flexibility to adapt as technologies evolve. Without this clarity, it’s easy to lose focus or make costly compromises that undermine the product’s purpose.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11cd77ef9621" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>