<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Akshay Gawade on Medium]]></title>
        <description><![CDATA[Stories by Akshay Gawade on Medium]]></description>
        <link>https://medium.com/@akshaygawade91?source=rss-292e25c387ee------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*cENy5srWkYYnigqh.</url>
            <title>Stories by Akshay Gawade on Medium</title>
            <link>https://medium.com/@akshaygawade91?source=rss-292e25c387ee------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 17:26:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@akshaygawade91/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building an On-Device Automotive Damage Detection App Using Multimodal Prompts in Apple’s…]]></title>
            <link>https://akshaygawade91.medium.com/building-an-on-device-automotive-damage-detection-app-using-multimodal-prompts-in-apples-6a32f183aaf9?source=rss-292e25c387ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a32f183aaf9</guid>
            <category><![CDATA[apple]]></category>
            <category><![CDATA[apple-foundation-models]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[swift-programming]]></category>
            <category><![CDATA[generative-ai]]></category>
            <dc:creator><![CDATA[Akshay Gawade]]></dc:creator>
            <pubDate>Sat, 11 Jul 2026 15:08:10 GMT</pubDate>
            <atom:updated>2026-07-11T15:14:22.640Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building an On-Device Automotive Damage Detection App Using Multimodal Prompts in Apple’s Foundation Models (iOS 27)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kIfJab8Fw7LreOJ2kGCH2g.jpeg" /></figure><blockquote>Learn how iOS 27’s new multimodal prompting capability lets you combine images and natural language to build an AI-powered vehicle inspection app that runs entirely on-device.</blockquote><p>Apple continues to expand the capabilities of Foundation Models, and one of the most exciting additions in <strong>iOS 27</strong> is <strong>Multimodal Prompting</strong>.</p><p>Until now, interacting with the on-device language model primarily involved text prompts. With iOS 27, developers can now attach images directly to a prompt, allowing the model to reason about both <strong>visual content and textual instructions</strong>together.</p><p>This opens the door to many practical applications such as:</p><ul><li>Manufacturing quality inspection</li><li>Medical image assistance</li><li>Retail product verification</li><li>Document understanding</li><li>Industrial automation</li></ul><p>In this article, I’ll demonstrate how I built an <strong>on-device automotive damage detection prototype</strong> using Apple’s Foundation Models without relying on any cloud AI service.</p><h3>Why Multimodal Prompting?</h3><p>Imagine asking the model:</p><blockquote><em>“Inspect this hood and identify dents, scratches, paint bubbles, missing labels, uneven panel gaps, and alignment issues.”</em></blockquote><p>Along with that prompt, you attach an image of the vehicle hood.</p><p>Instead of describing the image yourself, the Foundation Model interprets the visual information and combines it with your prompt to generate a structured inspection report.</p><blockquote>Everything happens locally on the device.</blockquote><blockquote>No image upload.</blockquote><blockquote>No external APIs.</blockquote><blockquote>No privacy concerns.</blockquote><h3>Prompt Engineering Matters</h3><p>During development I noticed that prompt quality significantly affects inspection accuracy.</p><p>For example:</p><ul><li>Act as an automotive quality inspector.</li><li>Inspect only visible defects.</li><li>Look specifically for dents, scratches, paint bubbles, panel gaps, missing labels, and alignment issues.</li><li>Return a professional inspection report.</li></ul><p>Being explicit dramatically reduced ambiguous responses.</p><h3>Post Processing the Model Output</h3><p>While Foundation Models produce impressive results, production applications still benefit from lightweight business rules.</p><p>For example, after receiving the inspection result I normalize:</p><ul><li>defect severity</li><li>defect locations</li><li>inspection status</li><li>recommendations</li><li>quality score</li></ul><p>This allows the application to produce consistent reports while still leveraging AI for visual reasoning.</p><p>Think of the Foundation Model as an intelligent assistant — not the entire business logic.</p><h3>Why On-Device AI?</h3><p>Running the inspection entirely on-device provides several advantages.</p><h3>Low Latency</h3><p>No waiting for cloud inference.</p><h3>Offline Support</h3><p>The application continues working inside manufacturing plants with limited connectivity.</p><h3>Lower Infrastructure Cost</h3><p>No GPUs.</p><p>No inference servers.</p><p>No API billing.</p><p>For enterprise applications, these benefits are significant.</p><h3>Potential Use Cases Beyond Automotive</h3><p>Multimodal prompting isn’t limited to vehicle inspection.</p><p>The same approach can be applied to:</p><ul><li>Manufacturing quality control</li><li>PCB defect detection</li><li>Warehouse inventory verification</li><li>Medical image assistance</li><li>Construction site inspection</li><li>Safety compliance checks</li><li>Product packaging validation</li><li>Equipment maintenance</li></ul><p>Any workflow that combines <strong>an image with domain-specific instructions</strong> can benefit from this capability.</p><h3>Code Snippet 1 (Building the Prompt)</h3><p>After this paragraph:</p><blockquote><em>The biggest difference in iOS 27 is the ability to attach images directly to a Prompt.</em></blockquote><p>Insert:</p><pre>let prompt = Prompt {<br> promptText<br>Attachment(image)<br> .label(&quot;inspection-image&quot;)<br>}</pre><h3>Code Snippet 2 (Calling Foundation Models)</h3><p>Immediately after explaining the prompt, add:</p><pre>let response = try await session.respond(<br>    to: prompt,<br>    generating: InspectionResult.self,<br>    includeSchemaInPrompt: true<br>)</pre><h3>Input Screenshot</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/316/1*Oq0eljM5CU1oDT3ZW9EFuQ.jpeg" /></figure><h3>Output Screenshot</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*Bbdg0N-y7maDpYUQbD5ytA.png" /></figure><h3>What I Learned</h3><p>Developing this prototype highlighted several important lessons:</p><ul><li>Prompt engineering is just as important as model selection.</li><li>Strongly typed responses simplify SwiftUI integration.</li><li>Keeping business rules outside the AI model improves maintainability.</li><li>Multimodal prompting significantly reduces the need for traditional computer vision pipelines for many inspection tasks.</li></ul><h3>Conclusion</h3><p>Multimodal Prompting in <strong>iOS 27 Foundation Models</strong> is one of the most exciting additions for Apple developers.</p><p>By allowing images and natural language to be combined into a single prompt, Apple enables entirely new classes of intelligent, privacy-first applications that run directly on-device.</p><p>In this project, a simple iPhone camera becomes an automotive inspection assistant capable of identifying dents, scratches, paint defects, missing labels, and alignment issues — all without sending a single image to the cloud.</p><p>I’m excited to see how developers use this capability in manufacturing, healthcare, retail, and beyond.</p><h3>References</h3><ul><li><a href="https://developer.apple.com/documentation/FoundationModels/analyzing-images-with-multimodal-prompting">Apple Developer Documentation: <em>Analyzing Images with Multimodal Prompting</em></a></li><li><a href="https://www.google.com/url?sa=t&amp;source=web&amp;rct=j&amp;opi=89978449&amp;url=https://developer.apple.com/videos/play/wwdc2025/286/&amp;ved=2ahUKEwiBzrnq7sqVAxWbSWwGHYABLuUQtwJ6BAgREAI&amp;usg=AOvVaw0o0hKZQcaf8RVhIcD5dSCJ">WWDC25 — Foundation Models Framework Sessions</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a32f183aaf9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to compare image on pixel data?]]></title>
            <link>https://akshaygawade91.medium.com/how-to-compare-image-on-pixel-data-6ecc8c9c6289?source=rss-292e25c387ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ecc8c9c6289</guid>
            <category><![CDATA[cifilter]]></category>
            <category><![CDATA[swift]]></category>
            <category><![CDATA[colorabsolutedifference]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[image-comparison]]></category>
            <dc:creator><![CDATA[Akshay Gawade]]></dc:creator>
            <pubDate>Sat, 24 Apr 2021 07:52:07 GMT</pubDate>
            <atom:updated>2021-04-24T07:52:07.499Z</atom:updated>
            <content:encoded><![CDATA[<p>How to compare image on pixel data? And How to compare color of two images using colorAbsoluteDifference CIFilter?</p><p>To compare image on pixel data you need CGImage object of UIImage.</p><p>Sometime when you create computed image through CIFilter and directly create UIImage from CIImage. It does not have CGImage data, so we need to covert CIImage to CGImage first and then get pixel data to compare</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*pA40ktKsGBjyouIk2BjtWA.png" /></figure><p>Get pixel data of images and compare through == operator.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*IPW1jQ7FIC8S_M6WANIq3w.png" /></figure><p>There is other way to do this through CIFilter.colorAbsoluteDifference() which get available from iOS 14 <a href="https://developer.apple.com/videos/play/wwdc2020/10673/">Explore Computer Vision APIs — WWDC 2020 — Videos — Apple Developer</a></p><p>Following code shows how to compare two images using colorAbsoluteDifference which return the image with difference of two images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*G2Ssmgjhmy3bOCd3baJPTA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ecc8c9c6289" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Xcode 9.0 Beta (Wireless debugging on iOS 11)]]></title>
            <link>https://akshaygawade91.medium.com/xcode-9-0-beta-wireless-debugging-on-ios-11-4892039f52b9?source=rss-292e25c387ee------2</link>
            <guid isPermaLink="false">https://medium.com/p/4892039f52b9</guid>
            <category><![CDATA[xcode]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[debugging]]></category>
            <category><![CDATA[ios-11]]></category>
            <category><![CDATA[xcode-9]]></category>
            <dc:creator><![CDATA[Akshay Gawade]]></dc:creator>
            <pubDate>Thu, 08 Jun 2017 11:45:59 GMT</pubDate>
            <atom:updated>2017-06-08T12:51:49.073Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X3Ow_qJnHZUqSu3MrWjvqA.jpeg" /></figure><p>I hope you enjoyed the WWDC keynote with awesome hardware and software upgrades. And for developers there is new Xcode 9.0.</p><p>Xcode 9.0 is full of new features</p><ul><li><strong>All new editor.</strong> Fast, structure-based editor that lets you intelligently highlight and navigate your code. Includes great Markdown support.</li><li><strong>Refactoring.</strong> Refactoring built right into the editing experience and works across Swift, Objective-C, Interface Builder, and many other file types. Select variable by <strong>holding cmd ⌘ </strong>on popup menu you will see rename option.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/501/1*k6tzt7Q4UMoCE-yia4Xxbg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*efvLC_AngMhYlPunhBQ7wg.png" /><figcaption>Refactor Shortcuts</figcaption></figure><ul><li><strong>Debugging.</strong> Wirelessly debug iOS and tvOS devices over the network, new debuggers for Metal, and more features throughout Xcode.</li><li><strong>Source Control.</strong> All new source control navigator and integrated support for GitHub accounts for quickly browsing repositories and pushing your repositories to the cloud.</li><li><strong>Xcode Server built-in.</strong> Continuous integration bots can be run on any Mac with Xcode 9, no need to install macOS Server.</li><li><strong>New Playground templates.</strong> Includes iOS templates designed to run well in both Xcode and Swift Playgrounds in iPad.</li><li><strong>New Build System.</strong> An opt-in preview of Xcode’s new build system provides improved reliability and performance.</li></ul><h3>Wireless Debugging</h3><blockquote>Network debugging requires Xcode 9.0 or later running on macOS 10.12.4 or later, and on the device, requires iOS 11.0 or later, or tvOS 11.0 or later.</blockquote><h3>Set up an iPhone, iPad, or iPod touch</h3><ol><li>Choose Window &gt; Devices and Simulators, then in the window that appears, click Devices.</li><li>Connect your device to your Mac with a lightning cable.</li><li>In the left column, select the device, and in the detail area, select Connect via network.</li><li>Xcode pairs with your device. If Xcode can connect with the device using a network, a network icon appears next to the device in the left column.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GkHcDhP25AiLvAdZ0a4tNg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/0*sJP4YZwELwKdXt5q.png" /></figure><p>5. Disconnect your device.</p><p>6. It is now ready for debugging over the network.</p><h3>Launch your app on a networked device</h3><p>Most of the time the Mac and device will be on the same WiFi network and Xcode will connect using Bonjour. Sometimes you need to connect to a remote device or can not use Bonjour. When that happens, you can connect using the IP address of the device.</p><h3>Connect using bonjour</h3><ol><li>Connect your Mac and the device to the same network.</li><li>In the <a href="https://help.apple.com/xcode/mac/9.0/index.html?localePath=en.lproj#/devf0d1df47a">toolbar</a>, choose the device from the scheme menu.</li><li>In the toolbar, click the Run button.</li></ol><h3>Connect using an IP address</h3><ol><li>Make sure your Mac and device are connected to a network.</li><li>Choose Window &gt; Devices and Simulators, then in the window that appears, click Devices.</li><li>Control-click on the desired device in the list of devices.</li><li>In the pop-up menu that appears, choose Connect via IP Address.</li><li>Enter the the IP address of the device in the dialog that appears and click Connect.</li></ol><blockquote><strong><em>iOS: </em></strong><em>Open Settings and choose Wi-Fi. Click the Info button next the active network to see the IP address</em></blockquote><ol><li>Once the device is connected, run your app.</li></ol><h3>Unpair a network debugging device</h3><ol><li>Choose Window &gt; Devices and Simulators, then in the window that appears, click Devices.</li><li>In the list of devices on the left, Control-click the desired device.</li><li>In the pop-up menu that appears, choose Unpair Device.</li></ol><h3>Troubleshoot network debugging</h3><p>Before you begin, ensure that:</p><ul><li>The Mac and the device are on the same network.</li></ul><h3>Verify that Xcode can see the device</h3><ul><li>Choose Window &gt; Devices and Simulators, then in the window that appears, click Devices.</li><li>The device should appear under Connected in the left column.</li></ul><h3>Ping the device</h3><ol><li>Find the IP address of a device of the device.</li><li>On your Mac, open Terminal.</li><li>Type ping followed by a space and the IP number you found above.</li><li>If the Mac can see the device, the output should look like:</li><li>&gt; ping 10.0.1.2PING 10.0.1.2 (10.0.1.2): 56 data bytes64 bytes from 10.0.1.2: icmp_seq=0 ttl=64 time=97.413 ms64 bytes from 10.0.1.2: icmp_seq=1 ttl=64 time=141.892 ms</li><li>If there is an error, you will see lines that look like:</li><li>Request timeout for icmp_seq 0</li></ol><p>If you can ping your device that means it is visible on the network. If you have not already done so, <a href="https://help.apple.com/xcode/mac/9.0/index.html?localePath=en.lproj#/devac3261a70?sub=devae20a6022">check that port 62078 is open</a>.</p><p>If you can not ping your device it can mean that ICMP is blocked or that there is some other connectivity issue. <a href="https://help.apple.com/xcode/mac/9.0/index.html?localePath=en.lproj#/devac3261a70?sub=devdf32a9e5d">Use an Ethernet cable to connect the device</a>.</p><h3>Check that port 62078 is open</h3><ul><li>Network debugging uses port 62078 for communication. Some networks block certain ports. Depending on your situation, you will need to check your own network settings, or check with your network administrators.</li></ul><h3>Use an Ethernet cable to connect the device</h3><ul><li>Connect the device to the same network using an Ethernet cable.</li><li>For an Apple TV, connect using the Ethernet port in the back of the device.</li><li>For an iPhone, iPad or iPod Touch, use the Lightning to USB Camera adapter and a USB to Ethernet adapter.</li></ul><h3>Re-set up the network device</h3><ul><li>Unpair the device and go through the set up process again.</li></ul><p>Overall wireless debugging is not robust yet &amp; has it’s loose connection but give it a try.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4892039f52b9" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>