<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Ollie - Medium]]></title>
        <description><![CDATA[Ollie the smartest, fastest and most fun photo curation experience on the planet. We use powerful AI and delightful UI to take you from a cluttered camera roll to photo-zen, so that you can enjoy your memories again. - Medium]]></description>
        <link>https://medium.com/goodones?source=rss----66cd6b56a989---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Ollie - Medium</title>
            <link>https://medium.com/goodones?source=rss----66cd6b56a989---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 11:05:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/goodones" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Marketing tech is huge timesuck on iOS, this is how we would do it again]]></title>
            <link>https://medium.com/goodones/marketing-tech-is-huge-timesuck-on-ios-this-is-how-we-would-do-it-again-59e26e7193f3?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/59e26e7193f3</guid>
            <category><![CDATA[in-app-purchase]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[email-marketing]]></category>
            <category><![CDATA[ios]]></category>
            <dc:creator><![CDATA[Mahyar McDonald]]></dc:creator>
            <pubDate>Mon, 29 Apr 2024 22:10:06 GMT</pubDate>
            <atom:updated>2024-04-29T22:10:22.295Z</atom:updated>
            <content:encoded><![CDATA[<p>Marketing tech is some of the most fragile stuff we have ever dealt with at <a href="http://getollie.ai/">Ollie</a>. Little bug after little bug just pops up and makes the entire house of cards fall down. I’ve found engineering focused vendors like Datadog and Sentry to be far more reliable, but anything in the marketing tech side just breaks in all sorts of ways or don’t have features that are braindead basic because nobody likes working on analytics in the engineering world.</p><p>If we were starting from scratch this is how we would set it up. If we did it this way we probably would’ve saved 100s of hours accumulated dealing with all sorts of bugs and would have a data set that I didn’t have to clean up a bunch of times to understand the long term trends in my app.</p><h4>Context</h4><p><strong>This advice is for small startups.</strong> Big companies can and should do it differently! The author worked on mobile logging systems at Uber 6 or 7 years ago, and there are a bunch of things he wouldn’t do here if he was in a large iOS app company.</p><p>We are a small startup ourselves working on a native B2C iOS only app with a subscription. iOS apps often rely on paid advertising for their go-to-market, and in order to be successful in that channel, proper attribution and therefore instrumentation is a core competency. Nobody on the team had prior expertise with this, and we worked with an in-house marketing person as well as a couple of agencies to guide us in the process. Despite that, the process has still been frustatingly slow and janky.</p><h4>Core Concepts</h4><p>There are a few core concepts in marketing tech:</p><ol><li>Most marketing tech, deep down inside, are just fancy logging systems that make special logs for you and others and then make queries on those logs and show it to you in visualizations. Having a good logging architecture from the start will save you so much pain and is not complicated once you understand what is needed.</li><li>The key to not go crazy in marketing tech is good ID management.</li><li>The key to messing with ad tech is messing with anything that can be an ID and making the data incredibly dirty.</li><li>Keep things simple. You will also be working with non-engineering marketing people, so try to also keep the data you report to marketing data platforms structured very simply. This also reflects in the lack of sophistication in marketing tech vendors. Appsflyer for example cannot extract properties from events beyond currency amounts &amp; SKUs to use it to set SKAN attribution, you need to separate them into separate events.</li></ol><p>This is also how we currently do it or are planning to do it soon.</p><h3>You need your own user microbackend</h3><p>Set up some sort of backend that tracks user logins, emails, etc. It should handle OAuth, it should be able to handle login with apple and google and so on. It should generate a single user UUID that is always consistent. The user should be independent of login with google or apple, and if they decide to connect google also, bind it all into one user. Use Django + their built in user management backend or use some firebase / supabase service, whatever. It will be incredibly cheap to run because it’s one table holding barely any data if you make your own. Even if your making just a mobile app and really don’t need or want a backend, make this little micro backend service.</p><p>You want this because the key to the house of cards is <strong>CONSISTENT USER IDs USING ONE CONSISTENT UUID SCHEMA!</strong> Emails don’t work well, phone numbers don’t work well, login with apple is not consistent because they designed it assuming you will have this backend to store some special tokens that they only give you once and so on. All you get in an app install is IDFV which is a device ID and a UUID you can just generate on first launch as a install ID. Even for these anonymous users, make a user UUID.</p><p>Also by keeping all user ids a consistent User UUID just in a table, you make GDPR compliance much easier since deletion requests for a lot of PII is just one row in one table on a thing you control %100. All the other vendors store just the UUID, or hinge on the user’s UUID for deletion requests.</p><p>Even if your app doesn’t need a login to work, you should still try to get a login or some sort of concept of user that you can keep consistent. If you don’t, users who use multiple devices and start getting confusing issues such as expecting their purchase works on multiple devices or so on will be far harder to debug and fix than when they are 3 anonymous device login users than they would be if they were 1 user with 5 device / app install ids.</p><p>With anonymous user IDs, you will probably want to have a cleanup mechanism for them if they are idle after a year or similar, since a lot of services charge by ‘user’ and one user can become 25 anonymous device users fairly quickly. If they didn’t log in and never did after a year of inactivity, they are not high intent users and not worth really thinking about much.</p><h4>Think about a single source of truth for analytics data</h4><p>For your analytics data, having one source of truth that aggregates all the data such as purchases from store fronts, email campaign openings and so on will keep your sanity intact. We were pretty good on this ourselves, but this an common pitfall. We use Mixpanel for that purpose currently.</p><p>Also think how this data will be exported to a ‘data warehouse’, essentially something that exposes the ability to do SQL queries on the analytic logs you upload to your analytics provider. The trend is to make analytics providers and data warehouse providers the same thing with services such as PostHog.</p><h3>Use one interface for logging, user properties and analytics</h3><p>Inside your mobile app, make a central logger that takes in logs of various types and then distributes them to various services as you like. This way if you want to integrate another service that needs your analytics yet again, to work, you just make a 50 to 100 line LogOutputManager instance for the service and you saved yourself a lot of time.</p><h4>Structure your logs</h4><p>Structure your log API to work similar to this way:</p><pre>import Foundation<br><br>enum LoggingLevel: Int {<br>    case trace<br>    case debug<br>    case info<br>    case analytic<br>    case warn<br>    case error<br>    case critical<br>    //etc<br>}<br>enum AnalyticSubtype: Int {<br>    case screen<br>    //etc<br>}<br>typealias LoggableProperties = [String: Encodable &amp; Sendable]<br><br>final class Logger {<br>    func log(_ level: LoggingLevel, <br>             _ message: String,<br>             error: Error? = nil,<br>             file: StaticString = #fileID,<br>             line: UInt = #line,<br>             function: StaticString = #function,<br>             info:LoggableProperties? = nil) { /* ... */ }<br>    func track(_ subType: AnalyticSubtype? = nil, name: String, file: StaticString = #fileID, line: UInt = #line, function: StaticString = #function, info: LoggableProperties? = nil) { /* ... */ }<br>}<br><br><br>//typical usage<br>let logger = Logger() // dependency injected<br>logger.log(.trace, &quot;file loaded&quot;, info:[&quot;path&quot;: &quot;/usr/log&quot;])<br><br>//errors<br>let err = NSError()<br>logger.log(.warn, &quot;image load problem&quot;, error: err, info:[&quot;id&quot;: 393])<br><br>//custom analytic events<br>logger.log(.analytic, &quot;triage-photo&quot;)<br>logger.track(.screen, name: &quot;home&quot;)<br>logger.track(.purchase, sku: &quot;aaa-333&quot;, amount: 3833)</pre><h4>Do not string interpolate!</h4><p>Do not string interpolate your base log message, this makes it an exercise in regexing to figure out what is going on. Keep it one static string that is very easy to query, and then put any variables that you would be tempted to interpolate in a structured dictionary that only accepts types that are serializable to JSON.</p><p>This will x10 the power of your logs and is called structured logging. It also speeds things up by avoiding interpolation. Feeling like you need to interpolate a string for logging is a huge code smell. Always try to put it in a structured dictionary if you can. If whatever service can’t handle your structured info blob, you can always cast it into a single string later, and that string would at least be parsable back into json again very easily.</p><p>By keeping the base string static, this will let you merge the same log into one item and then see trends in it’s cause based on the values of the properties. It makes errors show up in Sentry as one error vs 20 different errors because you interpolated some value into the title. It helps keep your data clean and sanity intact.</p><p>Also handling error objects as a separate type enables you to integrate them with all sorts of special things in other services.</p><p>Also don’t put the file or class name where your log is coming from, since you get that for free with #file, #line, #function variables you can add to your ‘extra info’ dictionary by default.</p><h4>Follow a two stage init &amp; identify pattern</h4><p>Most 3rd party SDKs need to be initialized early in the lifecycle when you have minimal info available about your user. They all recognize this, so they also have an identify call of some sort. Follow this structure inside your own logger so you can distribute this properly. You will also call identify again once your user logs in or updates their profile, so make sure it works properly when you call it multiple times.</p><h4>Have a base UserProperty &amp; Identity manager to sync to various services</h4><p>Most analytics &amp; marketing tech systems work on an events &amp; user properties data architecture. Having a UserPropertyManager that would centralize updates to things like totalPhotosTriaged to 7 services and so on will also save you a bunch of pain.</p><p>Since you will need to use the info that you create for identify calls in many places, making a central IdentitySet service or object will help you keeping things consistent. You would also put things like your sessionID in there and track special properties like email which many services need to be handled in a special way. In Mixpanel for example, you set the user email with a specially named $email user property, while in Qonversion, it’s a special enum typed .email call.</p><p>The identity manager can also help you deal with vendor quirks such as only being able to take an identify call once, which would make you wait until your ‘final’ login state is finished, or separate the identify call into a ‘create’ vs. ‘update’ or make it just call a single identify function multiple times for both creation and update actions.</p><p>This is also where you want to manage “super properties” which are properties you attach to every log event, such as sessionID. This is different than a userProperty which is changing one user’s data for the provider and is detached from any specific event.</p><p><strong>Make sure to put in this info for each log as a super property if the vendor doesn’t include it:</strong></p><p>Each item will help you solve a class of bugs if you include it.</p><ul><li>Session ID: you generate this on every new app launch as a UUID</li><li>User ID: The user UUID from your microservice</li><li>Device: The device UUID from the IDFV API</li><li>App Install ID: you generate this on first launch and persist it, don’t change it otherwise</li><li>Thread ID: 0 is main, other numbers are not. Create it with: `(Thread.isMainThread ? 0 : pthread_mach_thread_np(pthread_self())) % 1000`. This can make some bugs very, very clear and is a cheap sanity check to add.</li><li>The log context of #file, #line, #function</li><li>Raw Device Model Name: Which iPad is this?</li><li>OS Version</li><li>Any other extras you think are important</li></ul><h4>Keep one consistent variable naming scheme</h4><p>Choose camelCase or snake_case and stick with it.</p><p><strong>DO NOT USE “SPECIAL”</strong> characters like $ in your variable names, this is a big pain to migrate away from and can cause bugs in other services!! If the service you use needs some special names like that for various reasons, do it in the integration layer for that service. Don’t pollute your general interface with these special characters that you might use for one service. Mixpanel for example uses $email to set the user’s email. Don’t be lazy and use $email as the name of the user property for everything.</p><h4>Use the Segment Spec for how you structure your analytics</h4><p>Segment is in the the business of being glue for this marketing data mess, so they have thought of the base things that almost all services can use in some form or another. We suggest you follow it and set up your app to track things in a similar way: <a href="https://segment.com/docs/connections/spec/">https://segment.com/docs/connections/spec/</a></p><p>This means making special type screen or tap analytic events, using the sub specs for things like e-commerce if necessary, putting in special revenue properties and so on. Then set up your app to always emit them. This will make so many other services work very well when you integrate with them.</p><h4>Think about SKAN when you create your analytics</h4><p>SKAN 4.0 uses revenue, events and funnels on appsflyer. Make sure that revenue is being tracked properly in your analytics using the segment schema and the events you want go to appsflyer with proper unique user counts.</p><p><strong>Do all the above and vendors will say things like:</strong></p><p>“ah, so this is why you were able to add ____ integration so quick”</p><p>— real quote from one vendor</p><h3>Don’t use data proxy services like Segment to integrate your vendor if possible, just do it in the client directly</h3><p><strong>Most vendors only test their happy path with their own SDK properly.</strong> They don’t test things like Segment sending the same data itself into their service that well, and their own integrations with other services can work better than the equivalent one with Segment (or not). Often vendors also only properly integrate with each other when you have their first party SDKs connecting with each other via special IDs they generate inside your mobile app, which is something we experienced with Qonversion, Customer.io and Appsflyer.</p><p>Segment might make some things faster in the end and help you avoid mobile integration bugs and some data cleanness bugs, but you trade one set of potential bugs for another. Whatever Segment receives data wise with their generic spec events often doesn’t not translate %100, and sometimes their integration creates their own data cleanness bugs too.</p><p>As a startup, you can afford to just directly integrate their SDKs into their app. Yes it eats more network bandwidth because you send the same event 8 times to 8 different vendors, but it <strong>saves you so much time</strong> because you avoid bugs. There have been so so many bugs because we tried to be clever with Segment and the vendor is really just expecting you to use their SDK and thats it. You don’t want to waste your time with these bugs. Once you have product market fit and network usage &amp; binary size is becoming an issue then you can worry about such things and consolidate them to a backend like Segment.</p><p>This might be a controversial choice for some, but I do think there is some value in keeping it simple like this at the start for a small startup and adopting a CDP like Segment when the value and need become very obvious, and going slow at it if possible.</p><h3>Small pieces of advice</h3><h4><strong>Just use a subscription / IAP service wrapper from the start</strong></h4><p>If you start charging a subscription, just use a subscription service wrapper like RevenueCat or Qonversion from the start. The main reason why is app store connect &amp; segment doesn’t integrate with any other service, so if you want to see your subscription or revenue events in things like appsflyer or mixpanel or whatever else, then you need one of these anyway or waste your time making it yourself. <a href="https://medium.com/goodones/our-ios-subscription-management-shopping-trip-fbe22de72717">You can see the results of our research here</a>. Also what apple shows for revenue analytics isn’t as good as these services.</p><h4>Make sure you actually need a vendor</h4><p>Every vendor has an integration time cost that can often take more time than you think it would. Each one will come with their own bugs. No vendor is that easy to integrate, but still less time than making it yourself in most cases. Do you really need that shake to report bug vendor, or can you just use your log service and have a shake to report bug system that just sends an email?</p><h4>If you use appsflyer, install the FB SDK anyway if your going to run ads on FB</h4><p>Facebook is a special flower in this way unfortunately.</p><h4>Things like appsflyer is a bigger pain to integrate than you think</h4><p>It will take much longer than you think it will to integrate, and loves to get into big chicken / egg cycles where it doesn’t work well if you haven’t published your app on the app store yet.</p><h4>Just use google analytics &amp; firebase to start</h4><p>Google analytics and firebase give you a whole bunch of services out of the box that are not half bad. The analytics they surface by default are usually the ones you care about the most anyway. We use sentry &amp; mixpanel in one app and and crashlytics &amp; google analytics in another and found that crashlytics can surface more details in some crash cases and the default analytics that google creates are actually fairly useful.</p><p>If we were to start again we would’ve just used google analytics and firebase to start out and then transition out if I needed something different.</p><h4>Look at Posthog</h4><p>You’ll notice a bunch of the tools listed below are all in posthog and it all comes preintegrated, I’d consider researching it if we were starting over.</p><h4>Don’t bother with apple offer codes for partner attribution for now</h4><p>Apple doesn’t tell you which offer code was used for a purchase, just the generic offer identifier, and you can only make 10 offers per purchase, so you can work around it that way. So you can’t use offer codes to connect an organic purchase with a discount code they saw from an influencer or similar. We filed FB13568481 to fix it, please dupe the radar in feedback assistant too!</p><p>And also don’t bother to make your own discount code system for IAPs so you can record the code used, because that is also supposedly against the rules, even though it’s not written anywhere despite 1 hour of searching. You might be able to convince the reviewer to let it through, but there is a chance they might reject also.</p><p>This and their newness is why probably nobody uses them in apps from what I’ve seen, but we don’t have the data Apple has to know what actually is happening.</p><h3>Current experiences with vendors</h3><h4>Third Party Marketing &amp; Engineering SDKs in our App</h4><p>Notice how almost every vendor can be boiled down to either fancy logs, log glue or log analysis? Data is pain! Posthog is probably on to something here.</p><ul><li><strong>Qonversion:</strong> Works well enough, responsive support team. Haven’t used it that long</li><li><strong>Mixpanel:</strong> Our central source of analytics truth. Big shrug, works, but not really satisfied with it, but no reason to migrate.</li><li><strong>LaunchDarkly: </strong>Too small to be worth it, will migrate away probably once it makes sense</li><li><strong>Google Analytics &amp; Firebase:</strong> We have another app that uses it, it’s much better than what we were expecting compared to our other services, but we don’t use it extensively either to start getting annoyed at it.</li><li><strong>Sentry:</strong> Decent, but a bit annoyed that Crashlytics catches watchdog “crashes” with all the stacks while Sentry only catches one thread’s stack. Would make me want to migrate to Crashlytics if we started having an issue there again.</li><li><strong>Datadog:</strong> We just use this as Kibana in the end. Works well as a Kibana. Tried things like RUM, found it lacking. Overpriced, but not worth the effort yet to migrate away.</li><li><strong>Customer.io :</strong> Holy crap, it’s been a slog to get this to integrate and work properly. So. Many. Bugs. They only really test their happy path with their mobile SDK we have found.</li><li><strong>Segment:</strong> Works in the segment -&gt; mixpanel integration, customer.io does not. We did it because customer.io doesn’t have a good mixpanel integration but at this point we don’t know if we will still need it. Might only limit it to customer.io -&gt; mixpanel to cover for that missing integration and reduce it’s complexity cost</li><li><strong>Appsflyer:</strong> It’s bit of a janky confusing UX. It was a lot of work to integrate. But it works well enough for now. Appsflyer doesn’t support event properties beyond revenue in their config dashboards despite being able to ingest them in their SDK, which is confusing because why have support for it then!? Qonversion reports sale events for subscriptions as generic trial_converted style events, and then specifies the product that was sold as an event property. You can’t configure ad conversion targets in SKAN with these properties in appsflyer for example.</li><li><strong>FB:</strong> I’m going to look into using it more and we will see! Currently debugging why data dropped a lot despite us not touch it’s code at all other than a version bump.</li><li><strong>Session Replay:</strong> We tried, none of it works well enough to use yet. <a href="https://medium.com/goodones/15-years-later-there-is-still-no-good-session-replay-for-ios-f8d335999737">See more here.</a></li><li><strong>Judo:</strong> Really cool, great team, but we need to hire a growth PM to start using it properly as a rapid iteration SDUI system for our onboarding.</li></ul><h4>Internal Tools</h4><ul><li><strong>Bitrise:</strong> Works well enough</li><li><strong>Github:</strong> Works well</li><li><strong>Linear:</strong> This is my favorite tool. You might think “linear for X” is a startup cliche to the point where it’s a bad idea, but look at how much meh I’m showing for everything else here. My biggest pain was no email -&gt; ticket support, which they fixed recently. JIRA is a really easy target although. It’s still a good startup formula to this day, find a big business that everyone hates using and make the version everyone loves using.</li><li><strong>Pagerduty:</strong> Works</li><li><strong>Slack:</strong> The author likes Slack, it’s probably one of the first “well made &amp; fun” B2B apps out there.</li><li><strong>Notion:</strong> Fun, but also kind of slow &amp; clunky to use at times.</li><li><strong>Multi:</strong> Actually pretty cool, but they need a mobile app &amp; web call experience! A bit confusing for non-engineers IMO. We still need to use google meet as a result. People don’t like the ‘yet another app’ aspect of it, Slack Huddles are their biggest competitor.</li><li><strong>Teamplify:</strong> Small tool that actually solved some things, responsive support team.</li><li><strong>Google Workspace:</strong> It works</li><li><strong>TestLodge:</strong> Better than a spreadsheet</li><li><strong>Runway:</strong> It’s really cool, but too overcomplicated for our use case to spend more time integrating</li><li><strong>Zapier:</strong> It works, but I’d rather not use it if we could</li></ul><h4>Vendors we dropped</h4><ul><li><strong>Superwall: </strong>Cool tool, too many bugs with storekit to the point just making it ourselves natively. Once we actually start using it I’ll write a blog article about it!</li><li><strong>Instabug:</strong> This just wasn’t that good of a tool.</li><li><strong>Logrocket:</strong> Buggy 1 year ago, still buggy the exact same way 1 year later.</li></ul><h3>Feedback Wanted!</h3><p>What was your experience like? Would you do it differently? Any gotchas we should be aware of? We are at a small iOS startup, so we are interested in the wisdom of people further along this journey. Please tell us via comment or elsewise!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59e26e7193f3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/marketing-tech-is-huge-timesuck-on-ios-this-is-how-we-would-do-it-again-59e26e7193f3">Marketing tech is huge timesuck on iOS, this is how we would do it again</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How SwiftUI & Concurrency could improve]]></title>
            <link>https://medium.com/goodones/how-swiftui-concurrency-could-improve-16c13664e697?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/16c13664e697</guid>
            <dc:creator><![CDATA[Mahyar McDonald]]></dc:creator>
            <pubDate>Thu, 15 Feb 2024 01:05:53 GMT</pubDate>
            <atom:updated>2024-02-22T06:21:59.932Z</atom:updated>
            <content:encoded><![CDATA[<h3>How SwiftUI &amp; Concurrency Could Forward Fix Their Issues</h3><p>After our series of blog articles, what could apple do to forward fix the issues we had?</p><p><a href="https://twitter.com/mahyarm8/status/1709305138605953118">I also wrote a tweet thread about this partly</a>.</p><h3>Swift Concurrency Deadlocks</h3><ul><li>This is a great presentation about Swift Concurrency issues:<a href="https://www.youtube.com/watch?v=HqjqwW12wpw"> <strong>Matthew Massicotte — The Bleeding Edge of Swift Concurrency</strong></a><strong>. I would strongly suggest addressing all the scary aspects he raises.</strong></li><li><strong>A ‘watchdog’ extension</strong> that lets you pause and inspect all call stacks of the app, and upload it to an endpoint. The extension would live in a different process, and also be called when the app crashes or gets killed by the system watchdog. MetricKit has this partly, but being able to put in our own logic (such as a heartbeat checker to check if specific subsystems are deadlocked) would help us catch the cases that MetricKit does not catch.</li><li>A watchdog extension would’ve helped us with our production threading heisenbug issues, because we would be able to guarantee that we are observing all deadlock states and know the actual rates for them. Also there are some watchdog terminations that only have full call stacks on the phone, while sentry only has one thread’s call stack. This watchdog extension should also be able to recognize if Swift Concurrency is specifically deadlocked while the main thread is still working.</li><li><strong>An OS logging system</strong> that uploads to a URL endpoint reliably, that doesn’t rely on the app to implement it’s own logger properly or worry about deadlocks.</li><li><strong>Make in order execution of Tasks the default</strong>, out of order execution be a special mode that you have to wrap in a scope identifier. Sometimes out of order execution is a huge surprise and most do not need it’s benefits.</li></ul><pre>Async.outOfOrder {<br>  await Task { /* ... */ }<br>  await Task { /* ... */ }<br>}</pre><ul><li><strong>Debugging / pausing: </strong>You can only see the thread call stack, you can’t see the task count or the task tree that a debugger breakpoint is currently in. You have threads &amp; queues on the debugger pane, add a third ‘tasks’ versions that lets you see this dependency tree so we can understand what is going on.</li></ul><h3>SwiftUI</h3><h4><strong>State Lifecycle Control and Inspectability</strong></h4><ul><li>We should be able to answer questions like: Why did this object get created 5 times or 1 time? How do I control the state lifecycle of this object and understand how it is working?</li><li><strong>Linting:</strong> Warnings for not making a @State or @StateObject private, or setting it directly can help people in using it properly. This should be a hard warning from the compiler but this very bad behavior has no warnings whatsoever.</li><li>SwiftUI presents itself as a mutable system of sorts with @State or @StateObject, when it actually is not and is acting like a pure stateless function under the hood. Exposing the AttributeGraph underneath and how it works can help reduce the magic.</li><li>Consider exposing the source of AttributeGraph and SwiftUI so we can dive into it with breakpoints. It doesn’t have to be actual OSS.</li></ul><h4><strong>Debuggability</strong></h4><p>Make it easy to answer: “Why did this change happen in this order?”</p><ul><li>With UIKit, you usually could determine with a breakpoint why this view got updated with a new value by putting a breakpoint on the property set call of the view, and that set call would only hit once in normal circumstances and go to the root modifier up the call stack. If a dispatch call on another thread caused it on your thread, you would see that call stack in the debugger too with all of it’s jumps.</li><li>In SwiftUI, there is no real reliable way to do this, and you often do not see the piece of code that edited a binding value get modified in the breakpoint for the creation of your state view, and often that gets called 5 times for no discernible reason, breaking the chain of causality in the debugger.</li></ul><p>What I estimate what is causing the disjoint behavior:</p><ul><li>Main Thread Event Loop Cycle 1: Modify variable in ObservedObject</li><li>Cycle 2: AttributeGraph checks it’s dependency graph for any ‘dirty’ nodes, marks all downstream nodes as dirty</li><li>Cycle 3: AttributeGraph remakes View structs down a tree with new values (your breakpoint hits here, with no connection to what happened in cycle 1, you don’t know what caused your view to change anymore, and which upstream dependency it came from)</li><li>SwiftUI encourages something I call the ‘side effect rube goldberg machine’. In UIKit, you directly modify your thing you want to change. In SwiftUI, you modify a state variable, and then the black box SwiftUI rube goldberg machine finds your View you want to change via AttributeGraph and then modifies it on your behalf as a side effect. I estimate this creates a lot issues in debugging because of it’s lack of inspectability and how it disjoints a lot of actions.</li></ul><h4>(Non)<strong>Determinism</strong></h4><ul><li>How can I reliably control when certain objects get created, this is especially a problem with things like NavigationLinks.</li></ul><h4><strong>Navigation View Mutability / Feature Backporting</strong></h4><ul><li>You can’t edit the navigation view, so if you want to remove previous items in a navigation stack, ex: “login → push onboarding (and remove login so you cant go back)” you have to do a subview swap instead and then animate a side transition to make it look like a navigation push currently.</li><li>This is fixed in iOS 16 with NavigationStack, but we can’t use it due to being at minimum version of iOS 15, and ~%5 of users being on iOS 15 consistently. Backporting would solve this for us.</li></ul><h4><strong>Collection View Performance</strong></h4><ul><li>It’s much easier to make a performant collection view in UIKit vs. SwiftUI due to easily inducing non-lazy behavior by mistake in SwiftUI compared to UIKit.</li></ul><h4><strong>View Debugger Inspectability</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3qO8bRo1R7Peahl8zZqXiA.png" /><figcaption>Why does SwiftUI create a huge deep view tree for 3 labels in a list with 3 toggles?!</figcaption></figure><ul><li>This is a lot better in Xcode 15, now your SwiftUI view names show up in the graph, but SwiftUI still creates significantly more complicated view debugger states than UIKit does. There is a lot of empty container states that create a lot of noise, a way to compress and hide them would help in understanding, kind of like the debugger hiding system call stack frames as a button option.</li><li>A lot of sentry and InstaBug view profiling is messed up because of this, the names are not meaningful or helpful</li></ul><h4><strong>Hidden Magic / Unlabeled Functionality</strong></h4><ul><li>Fundamentals of SwiftUI such as identity are hidden by design. StateObject &amp; State honestly should be separated systems idependent of SwiftUI vs. things associated with a view via AttributeGraph.</li><li>What objects are created lazy or not is not obvious. @StateObject should be force tagged as lazy so people know it’s not eager initialized for example.</li></ul><h4><strong>Defaults &amp; Examples encourage bad practices</strong></h4><ul><li>SwiftUI via @StateObject encourages embedding business logic heavy objects into a View hierarchy. Examples that encouraged fully separating business state &amp; logic in a separate class based state hierarchy and then injecting it from the top into a SwiftUI View tree as the default example would significantly reduce people doing it the wrong way by default and using a lot of SwiftUI foot guns.</li></ul><h3>Last Article</h3><p>This is the last article of a series of blog posts where we go into:</p><ul><li><a href="https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d">Why we decided to make a native iOS app vs React Native and an a outline of our stack.</a></li><li><a href="https://medium.com/goodones/ollies-app-architecture-continued-2b572e23c204">Baseline architecture — including our data model which is most of the app, and then the views.</a></li><li><a href="https://medium.com/goodones/how-ollies-ai-system-works-ac7d8838df11">AI model — this is a bit of the secret sauce so we’ll keep it closer to the vest, but we’ll share our basic insights. It turns out it’s quite difficult to figure out which photos people will like!</a></li><li><a href="https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c">The async/await quagmire — what issues we ran into with SwiftUI &amp; Concurrency, and why we decided to move on to UIKit &amp; Dispatch.</a></li><li><a href="https://medium.com/goodones/pareto-optimal-apple-devtools-b47e79b6676f">Imagining a world where Apple made some more pragmatic decisions for their new dev platforms vs. the current ones they made today.</a></li><li><a href="https://medium.com/goodones/how-swiftui-concurrency-could-improve-16c13664e697">How Apple can forward fix their issues with SwiftUI and Swift Concurrency.</a> (This Article)</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=16c13664e697" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/how-swiftui-concurrency-could-improve-16c13664e697">How SwiftUI &amp; Concurrency could improve</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pareto Optimal Apple Devtools]]></title>
            <link>https://medium.com/goodones/pareto-optimal-apple-devtools-b47e79b6676f?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/b47e79b6676f</guid>
            <dc:creator><![CDATA[Mahyar McDonald]]></dc:creator>
            <pubDate>Thu, 15 Feb 2024 00:53:22 GMT</pubDate>
            <atom:updated>2024-02-21T21:52:16.093Z</atom:updated>
            <content:encoded><![CDATA[<p>In <a href="https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c">our previous article</a>, we explained how a combination of new Swift features and how we were using them led to a difficult to diagnose stability crisis at our company, <a href="https://getollie.ai/">Ollie</a>. It led to us deciding to move away from these new systems and to adopt UIKit. This article goes into how Apple Developer Tools could’ve made different decisions in the past that would’ve led to better outcomes.</p><p><strong>Warning:</strong> It’s a rant.</p><h3>You held SwiftUI wrong</h3><figure><img alt="Steve Jobs holding an iPhone 4" src="https://cdn-images-1.medium.com/max/1024/1*30dprT63LgGB2YSzy1hRww.png" /></figure><p>Yes we definitely held it wrong. Our <a href="https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c">last article</a> definitely has a bunch of details about Swift UI and Swift Concurrency being held completely wrong. But why is it so easy to hold it wrong, even after spending literally over 100 hours trying to understand these problems?</p><h4><strong>Why make hard mode concurrency the default?</strong></h4><p>Why make a concurrency system for fairly standard, low complexity actions have something equivalent to <a href="https://www.techtarget.com/whatis/definition/superposition#:~:text=Superposition%20is%20the%20ability%20of,the%20English%20physicist%2C%20Thomas%20Young.">quantum superpositioning</a> <strong>as the default</strong>, where %99 of await calls and Tasks execute in the order you write in the code, except when they don’t, and will execute in a different order when you apply breakpoints vs. when you don’t and about a million different special cases?</p><p>The vast majority of mobile apps don’t need this level of power to implement their insurance apps, or even their photo processing apps because most of their heavyweight operations are in easy to reason about small sections such as a DB read, network call, logging call, GPU processing call, etc that a simple set of deterministic preemptive Dispatch queues works great for. Hard mode should not be default mode, and Tasks should’ve executed in order. Have out of order execution for the pros who can declare it in a scope operator for example.</p><h4><strong>SwiftUI is an exercise of conflicting abstractions</strong></h4><p>SwiftUI I feel is an exercise of having your cake and eating it too. They want to be a declarative pure function that you could potentially reason about, but probably quickly found out that most engineers are not that smart, or find a stateless binding only view systems create a lot more boilerplate than an equivalent imperative one, so they needed to graft in a state management system on top of a system that isn’t really supposed to have state ‘live’ inside it’s stateless function in its design.</p><p>Because they wanted it to make it ‘easy’ to use and for it to ‘just work’, they hid a ton of the complexity and implicit assumptions in front of some StateObject property wrappers and opaque some View types and try really hard to hide all the identity dependencies that SwiftUI needs to work performantly. Thus all the footguns of SwiftUI easily shoot you in the foot, because the developer UX design implies things work how your used to, but actually work really badly when you try to use it how you were expecting it to.</p><p>The solution to these problems ends up being opting out of SwiftUI state management beyond some very simple cases, or learning how to hold it very, very carefully. Worse of all, it’s a magic system where you can’t inspect the library code to understand what is really happening, unlike most of Android.</p><h3>This is the Era of Second System Apple</h3><p>This is a pattern I’m seeing with Apple foundational technology is a repeat of an overall trend within Apple starting around Swift, which I call ‘<a href="https://en.wikipedia.org/wiki/Second-system_effect">second system</a> Apple’. It started around iOS 7 with the completely new UX revamp &amp; the Swift language.</p><h4>First iOS System Apple</h4><p>I have about 15 years of experience with apple platform tech, starting with iOS 2 when the first public iPhone SDK came out. Some things were 10 minute blog article unintuitive, like interface builder links and table view cell recycling, but after you got past the surface ugliness of Objective-C, it was a pretty well thought out compete native UI toolkit built on top of an elegant compiled reimplementation of Smalltalk that I quite admired.</p><p>UIKit &amp; Objective-C had some warts, but everything worked predictably well, you were only 2x slower than raw C, didn’t have to contend with a garbage collector giving you random frame drops. No Samsung reimplementing the world under your feet and no web world that went for the javascript trend of the week, had multiple browser targets and bad performance.</p><p>New ‘change the world’ transitions like Grand Central Dispatch, GCC → LLVM and ARC were actually pretty smooth transitions, with simple rules on how to use them and deterministic, predictable execution with small unintuitive corners. If you fucked up in ‘normal’ ways, it led to memory leaks and slowdowns, which were recoverable and detectable and didn’t deadlock your app in innovative, fragile new ways. Autolayout was probably the least smooth transition that first system apple had.</p><h4>Second System Apple: Swift &amp; iOS 7</h4><p>Then Swift &amp; iOS 7 came out, and the reliability and complexity of what apple delivered in the iOS space <strong>changed.</strong> I was at Uber in 2016 when we did the Swift rewrite. Before the rewrite Uber’s Rider app compile time for 1 million lines of Objective-C code was 5 minutes, had a fast startup time, a small binary size and a working debugger.</p><p>After the rewrite to 1 million lines of Swift code we got 50 minutes build times, a broken debugger (which has never gotten to the same performance level and reliability that it did back then) and a dynamic linker induced startup time issue where we had to hack in static linking into swift years before it was available officially in order to not have a 1 minute pre-main startup time.</p><p>It took about 7 or 8 years for Swift to start getting to an almost equivalent developer experience to Objective-C, and what mostly got it over the hump was the apple silicon chips in the end, not a great revolution in compiler performance.</p><p>I was the manager of the iOS developer experience team at Uber when the M1 came out and I pushed hard to get iOS engineers early upgraded because it made that much of a difference. If Uber still had an Objective-C codebase then, the M1 mac probably would’ve made build times go down to almost nothing.</p><p>Swift is still missing things like OCMock to this day. Your swift previews would probably be instantaneous at this point with Objective-C, since a lot of the slowness comes from Swift’s build time issues. You can actually hack in UIKit views into Swift UI previews, and you could probably do the same with Objective-C and get a faster experience with Swift UI Previews than you do with Swift UI itself.</p><p>Swift UI and Swift Concurrency feel like repeats of the same pattern with the Swift rollout. For these to become complete systems, you will probably have to wait another 7 years until they are really ready to be %90 as good as the original systems they are replacing at this rate, while with things like ARC, Autolayout and Dispatch, maybe avoiding v1 for a year or two is the only thing you had to worry about back then.</p><p>I admit that “second system apple” might’ve been happening for a long time, as I wasn’t there for the macOS &amp; AppKit → iOS transition, or the Carbon transitions and so on. I am curious how it was like for developers with more experience for me, and what we missed (or not) from back then. I’ve heard that UI development today in many ways still hasn’t beat the productivity of Visual Basic 6 and Pascal, even though I was a child when that era of software was around, and there are many platforms I haven’t developed with, like Visual Studio and Windows in general. You often hear from devs that the original Visual Studio is really good.</p><p>Jonathan Blow is probably on to something about how unproductive we are today in software compared to 20 years ago.</p><h3>A Thought Exercise: Could these new tools compete with a pareto optimal version of themselves that had the benefits of the original systems?</h3><p>There are a lot of nice features in languages like Swift, but <a href="https://github.com/apple/swift/blob/main/docs/CompilerPerformance.md">what are we missing</a> from our tools as a result of that complexity?</p><h4>PurelyObjective</h4><p>Imagine a version of Objective-C that just implemented the “easy &amp; fast” stuff from swift and called itself PurelyObjective:</p><ul><li>Had Swift’s syntax and just implemented the square bracket part of Objective-C, no C ugliness would be available.</li><li>Autogenerated header files from source files implicitly, no header files needed</li><li>Had private/public/protected</li><li>Had easy namespacing</li><li>Did API renames to get rid of all the NSWhatevers</li><li>Implemented swifts ADT enums (which are relatively simple in compiler land)</li><li>Auto created an implicit enum type for all selector calls called ‘selector enums’ for each class that you could record and feed into various class types, basically giving you a key part of the elm / TCA architecture for free as a first class citizen.</li><li>Implemented strong nullability (Objective-C’s version was not strongly enforced by the compiler)</li><li>Implemented a java-like generics system that enforced types at compile time properly</li><li>A container data typeset that could handle nils</li></ul><h4><strong>Missing Features Compared to Swift</strong></h4><ul><li>Needed to keep per file imports vs per module imports</li><li>Might still need to type out more types instead of having type inference almost everywhere significantly slowing down performance and hiding complexity that can bite you in the future.</li><li>You couldn’t do systems programming with it</li><li>You wont get operator overloading</li><li>Whatever else was a minor feature that significantly increased compile &amp; debugger times</li></ul><h4>And in exchange what you got in comparison to Swift:</h4><ul><li>Wicked fast compiler speed</li><li>Rock solid IDE debugging</li><li>Extremely inspectable memory runtime during debugging</li><li>All sorts of tools like dynamic Mock Generation</li><li>Near instant UIKit Previews, enabled by the dynamic dispatch nature of Objective-C, delivered 5 years earlier, finally catching up with<a href="https://youtu.be/EPoI2UfIW44?feature=shared&amp;t=115"> web development 10+ years later</a>.</li><li>Tiny binaries that would be smaller than equivalent javascript programs and could execute on the browser and run at 60fps</li><li>A build system that could literally scale to 10 million lines of code without much of a sweat, and scaled linearly with CPU core count if you needed to build faster</li><li>Make beautiful fluid 60fps UIs on processors weaker than a raspberry pi 2 today.</li><li>First class interop with C &amp; C++ via the old Objective-C(++) code base</li><li>1:1 equivalence and interop with Objective-C code.</li></ul><p>And it would’ve taken Apple a lot less manpower and a few years to implement.</p><p>Would you take that deal?</p><p>I would!</p><h3>Imagine DataUIKit</h3><p>Now imagine <a href="https://componentkit.org/">ComponentKit</a> ^H^H^H^H DataUIKit:</p><ul><li>Same SwiftUI Syntax</li><li>Makes UIKit equivalent View trees of what you declare</li><li>Is literally just inert data</li><li>You take the root DataUIView tree object and then call a .generate()</li><li>It generates a UIView and a (BindingDelegatePacket)</li><li>You hook into the Binding delegate</li><li>You take that root view and put it wherever you want.</li><li>ActionGraph is a public library that you can use to make equivalent stuff with SwiftUI. We basically did the same thing with RxSwift at Uber before SwiftUI was ever public. RxSwift was pretty rock solid despite it’s conceptual complexity.</li><li>Think SnapKit, but for all of UIKit, or code based XIB files</li><li>You’re done!</li></ul><h4><strong>In exchange you get:</strong></h4><ul><li>Instant DataUIKit previews</li><li>Serializable view trees you could unpack over the wire, creating a SDUI system for free.</li><li>Yes this is XIB files, but it’s code, not XML files that are not designed to be human editable!</li><li>All the features of UIKit, nothing is missing</li><li>All the performance of UIKit</li><li>All the debugability and legibility of UIKit</li><li>A view debugger that works already</li><li>Few to no <a href="https://en.wikipedia.org/wiki/Leaky_abstraction">abstraction leaks</a>, because it’s basically equivalent to SnapKit for UIKit + a few nicer to use data binding libraries</li><li>Data update flows you can put legible breakpoints on</li><li>Way less work to implement for apple</li><li>You get that DOM-esque &amp; React-esque niceness, but with way less bullshit!</li></ul><h3>SwiftRust</h3><p>Instead of trying to be an everything language with Swift and end up with “New Apple Flavored C++”, Apple makes PurelyObjective, have %90-%99 of development for apps and more be in PurelyObjective, and put all the slow compiling, fast executing, hard generic binary size exploding, high control, high safety system language stuff in SwiftRust, putting innovative concurrency things that the Rust language is also trying to solve inside it.</p><p>Since it’s made by Apple, they can share a lot of compiler infrastructure, use a very similar syntax, probably reuse a lot of the same macro infrastructure, syntax traversal and linting, have first class interop and so on and avoid a shit ton of pain from shoehorning a systems language into an app development use case and make some very safe high performance system, data and engine libraries. Have 5 more years of ABI instability because no root app of 2 million lines of code is depending on it, and your targeted, high performance library can afford a longer ABI instability time. Bake the language properly.</p><h3>DispatchAwait with New Actor Feature in PurelyObjective</h3><p>Get compiler enforced thread execution for single threaded libraries like UIKit without quantum superposition behavior! AsyncAwait uses Dispatch and is not an easy to deadlock “<a href="https://en.wikipedia.org/wiki/Cooperative_multitasking">cooperative</a>” system that the computing industry moved away from in the 90s, before multithreaded processors were ever widespread! Things execute in order, and they use a lot more locking in exchange for simplicity and predictability. Leave the quantum superpositioning fancy behavior to SwiftRust, when you actually need it for that extra performance.</p><h3>Next Article</h3><p>All of these things won’t happen anyway, but they are the alternatives not publicly considered. Hopefully in 5 to 10 years this will all resolve itself via Apple forward fixing it, or we will all be using AI to make software in some human language → assembly way and none of this will matter anymore. To the future!</p><p>The <a href="https://medium.com/goodones/how-swiftui-concurrency-could-improve-16c13664e697">next article</a> has some suggestions on how Apple could forward fix SwiftUI and Swift Concurrency.</p><p>This is part of a series of blog posts where we go into:</p><ul><li><a href="https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d">Why we decided to make a native iOS app vs React Native and an a outline of our stack.</a></li><li><a href="https://medium.com/goodones/ollies-app-architecture-continued-2b572e23c204">Baseline architecture — including our data model which is most of the app, and then the views.</a></li><li><a href="https://medium.com/goodones/how-ollies-ai-system-works-ac7d8838df11">AI model — this is a bit of the secret sauce so we’ll keep it closer to the vest, but we’ll share our basic insights. It turns out it’s quite difficult to figure out which photos people will like!</a></li><li><a href="https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c">The async/await quagmire — what issues we ran into with SwiftUI &amp; Concurrency, and why we decided to move on to UIKit &amp; Dispatch.</a></li><li><a href="https://medium.com/goodones/pareto-optimal-apple-devtools-b47e79b6676f">Imagining a world where Apple made some more pragmatic decisions for their new dev platforms vs. the current ones they made today.</a> (This article)</li><li><a href="https://medium.com/goodones/how-swiftui-concurrency-could-improve-16c13664e697">How Apple can forward fix their issues with SwiftUI and Swift Concurrency.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b47e79b6676f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/pareto-optimal-apple-devtools-b47e79b6676f">Pareto Optimal Apple Devtools</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Ollie is moving away from SwiftUI to UIKit]]></title>
            <link>https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/cfdefe918d1c</guid>
            <dc:creator><![CDATA[Mahyar McDonald]]></dc:creator>
            <pubDate>Thu, 15 Feb 2024 00:17:42 GMT</pubDate>
            <atom:updated>2024-06-06T18:09:59.905Z</atom:updated>
            <content:encoded><![CDATA[<p>A few months ago at <a href="https://www.getollie.ai/">Ollie</a>, we made the decision to move away from SwiftUI &amp; Swift Concurrency for our application and to move the core of our app to UIKit and Dispatch. We have migrated all screen and navigation management to UIKit recently in our app and it allowed us to remove several categories of hacks and solved a bunch of performance issues &amp; bugs we had.</p><p>We will still use SwiftUI for simple screens like onboarding or settings screens, but complicated things like collection views or anything else we need to not be bug prone, we will use UIKit.</p><p>People who are making more typical server client apps with SwiftUI and Swift Concurrency have a good chance of not running into our issues due to the lack of threading &amp; resource stress. They do not need to maintain local state consistently on their device, since they can rely on a backend to properly maintain state for themselves. This kind of app will probably have a much nicer time with SwiftUI.</p><p>Multiple people have been interested in the details of why we made this decision, so this article is for you.</p><h3>Our Adventure</h3><p>At the beginning we were pretty excited to work on a new app with all the latest Swift libraries, but we have found it a painful, incomplete system to work with once you start making something substantial with it.</p><p>When the author joined the company, we were under a stability crisis whose cause was mysterious and we were not in a state to release it to the app store. It took us 2 to 3 months to partly fix the stability issues to be in a releasable state.</p><p>Overall we had to spend over a working year of cumulative engineering hours to fix our stability issues, figure out why they were happening and fix them for most of our cases. Since the cause of our stability issues were hard to reproduce deadlocks and infinite spinners, which also blocked reporting and observability of those issues, it was much harder to figure out why things were happening and to get call stacks of what induced these states. Our app also didn’t not rely on a backend at all to function, so we couldn’t inspect server logs to see what network calls were the ‘bad behaving’ ones. All we had is remote logging with datadog and sentry.</p><p>After releasing to the app store, several months of production usage and several deep dives of books, docs &amp; WWDC video binges, we finally figured out what combination of bad things were causing our issues. To understand what we did wrong although, you have to understand what our discovered issues were with SwiftUI &amp; Swift Concurrency.</p><h3>Our Issues</h3><h4>SwiftUI has too much magic &amp; abstraction leaks. AttributeGraph is non-deterministic from a outsider perspective</h4><p>SwiftUI:</p><ul><li>Is filled with missing stair steps features that are easy to trip over.</li><li>Lacks features that have existed since iOS 2.0 in UIKit.</li><li>Has issues with dealing with debugging, observability &amp; the ability to reason about what is happening in our app.</li><li>Is hard to easily control the lifecycle of state objects, callbacks and so on.</li></ul><p>We have found SwiftUI to have a lot of “magic” within it that makes it hard to reason about. SwiftUI has a lot of implicit hidden assumptions in the system that, if you don’t follow, can cause a lot of surprising painful results. SwiftUI also calls its view body function many times, at a frequency that is difficult to reason about or predict. Performance of large image collection views are also still an issue today, which are essentially the core of our app.</p><p>It can be near impossible to break point how a specific binding got updated that way this time, but only occurs %5 of the time for example. To fix such issues, you basically have to deduce what is happening like Sherlock by testing hypothetical inputs and seeing the outputs. SwiftUI itself is a black box wrt to it’s behavior.</p><p>SwiftUI is a deceptive system which is easy to get started in, but hard to make it work right without knowing the implicit assumptions unless you’re making something fairly simple. This especially expands once your entire app view tree and navigation is managed by SwiftUI.</p><p>In some ways, it reminds me of the trade off you get with React Native due to its abstraction leaks because of the need to seamlessly integrate with UIKit. The need to become an expert in multiple platforms simultaneously to properly debug all issues down the stack vs. working with one native one, and being a declarative vs. imperative view system that uses a diffing algorithm to create view changes.</p><p>I fear that SwiftUI is suffering from a <a href="https://en.wikipedia.org/wiki/Second-system_effect">second system effect</a>, much like its parent Swift itself did.</p><blockquote>The <strong>second-system effect</strong> or <strong>second-system syndrome</strong> is the tendency of small, <a href="https://en.wikipedia.org/wiki/Elegant">elegant</a>, and successful systems to be succeeded by <a href="https://en.wikipedia.org/wiki/Overengineering">over-engineered</a>, <a href="https://en.wikipedia.org/wiki/Software_bloat">bloated</a> systems, due to inflated expectations and overconfidence.<a href="https://en.wikipedia.org/wiki/Second-system_effect#cite_note-1">[1]</a></blockquote><blockquote>The phrase was first used by <a href="https://en.wikipedia.org/wiki/Fred_Brooks">Fred Brooks</a> in his book <a href="https://en.wikipedia.org/wiki/The_Mythical_Man-Month"><em>The Mythical Man-Month</em></a>, first published in 1975. <a href="https://en.wikipedia.org/wiki/Second-system_effect">Wikipedia</a></blockquote><h4>Swift Concurrency deadlocks too easily &amp; it‘s very hard to observe that it deadlocks in iOS apps</h4><p>Swift Concurrency we have found to be fairly fragile. It’s too easy to deadlock it by surprise due to internal apple libraries such as PhotoKit using standard dispatch_sync calls on unpredictable schedules.</p><p>Due to everything being on one concurrent Dispatch thread pool with Swift Concurrency and any downstream call of that thread pool inheriting the peculiar properties of the Swift Concurrency thread pool, it also makes these deadlock states unobservable.</p><p>So if you have an async function calling into another module that uses an independent dispatch queue to manage it’s multithreading, Swift Concurrency will ‘infect’ that dispatch queue with the special properties of the main Swift Concurrency dispatch queue, and that queue will start acting like a Swift Concurrency dispatch queue.</p><p>So if you call a logging function in your async function, that logging system will also start to deadlock because it happens to be within a Swift Concurrency calling context, even though that logging system does not use any Swift Concurrency primitives, complies with the forward progress contract and so on. If behavior elsewhere deadlocks your swift concurrency queue, it will end up deadlocking your independent logging queue!</p><p>If you rely on a logging system to help debug or detect these deadlocks, you have to be extremely careful to not have swift concurrency ever touch those queues. Which can be really difficult to pull off and still be useful, since you often need the data collected by the logging system to diagnose WTF is going on in the first place!</p><p>On top of this, the MainActor / main thread of the app gets special privileges from Swift Concurrency’s standpoint. Swift concurrency interactions running on the main thread does not execute in the shared dispatch queue that the rest of Swift Concurrency executes on, and the ‘swift concurrency queue behavior virus’ does not infect the main thread!</p><p>So as a result, when swift concurrency deadlocks, it deadlocks everything it touches, except the main thread. This means your UI will still change in response to taps, it’s just anything that calls into an async await call will never return once you’re in this state.</p><p>This is actually worse, because your plain old independent logging systems such as Datadog and Sentry who are using plain Dispatch cannot upload the fact that you are in a deadlock from another independent thread pool. Your app will also never crash under a watchdog crash by the system because the UX / main thread stays responsive, making it completely undetectable beyond user reports and internal testing or some really heavy duty and fragile engineering hacks.</p><p>On top of this, you can never figure out the prevalence of what is happening with the way iOS apps are set up currently. I would solve this with creating a watcher process that would spawn the actual app process as a child for example, but you are restricted from doing so in iOS unlike macOS.</p><h4>Tasks, await calls, actors, etc in swift concurrency execute in order most of the time, but sometimes do not, and it is non-obvious on how to force it to execute it in a specific order.</h4><p><strong>Note / Update:</strong> This is the hardest to believe part of our article, but after spending 6 hours with about a dozen iOS engineers on a private slack with 100s of slack messages going back and forth about this, pasting a lot of code and checking a lot of edge cases that didn’t apply in the end, we couldn’t figure out why it happened, just that it did.</p><p>It could very well be a bug in Swift Concurrency, or we did use Swift Concurrency wrong in some really edge case way, but in the end it doesn’t matter. The fact that this is possible to do and really hard for a lot of people figure out why it’s happening is the main problem in the first place. It’s too easy to screw it up. Just know the code below is simplification to illustrate our point, the actual code has a lot more checks to prevent double execution and such.</p><pre>class MyViewModel: ObservedObject {<br>  @Published var showLoadingOverlay = false<br>  <br>  /// Significantly simplified from the original code<br>  func refreshPhotos() async {<br>    await photoManager.refresh() // 1 (long running, has await calls &amp; Tasks to other systems inside of it)<br>    await MainActor.run {<br>       showLoadingOverlay = false // 2<br>    }<br>  }<br>}</pre><p>%99 of the time when you run code like this, it will execute in order. 1 will finish executing and then 2 will execute. When you write this code, this is what you would assume from all your years of programming. But sometimes, especially when you stress the threading system with a lot of work happening elsewhere, 1 will start executing, and before 1 completes, 2 will execute, causing all sorts of race condition bugs, SOMETIMES!</p><p>Actors similarly, also do not guarantee serial execution, but act serially, most of the time!</p><p>Once you run into a bug like this and try to fix it by reasonably forcing serial execution, it’s not obvious how to do so with Swift Concurrency. If you try to force these two await calls into their own dispatch queues instead, you start getting a whole bunch of warnings and errors as swift-concurrency complains endlessly about it.</p><p>Do searches about it and you see obscure things in swift forum &amp; reddit threads about AsyncStream and SerialExecutors and it’s also not obvious on how to force two tasks to execute serially from each other reading these forum threads. The dispatch API on the other hand made it obvious directly from its documentation, while to do something similar from async await, you need to make a <a href="https://github.com/dfed/swift-async-queue/blob/main/Sources/AsyncQueue/ActorQueue.swift">50 line wrapper</a> to make it as simple as Dispatch did. Most engineers are not going to figure this out easily, or even expect any of this to happen.</p><p><strong>The interaction of all 3 of the above issues together led to our stability crisis.</strong></p><h3>How we held SwiftUI and Swift Concurrency Wrong</h3><ol><li>We used MVVM which we translated into @StateObject business logic heavy ViewModels that was ok to do in UIKit, but fundamentally a bad idea with SwiftUI due to it’s unpredictable initialization behavior.</li><li>Most SwiftUI tutorials and documentation do not make this obvious unless you read a few key paragraphs in documentation or deep dive into WWDC videos such as ‘<a href="https://developer.apple.com/videos/play/wwdc2021/10022/">demystifying swift ui</a>’ &amp; ‘<a href="https://developer.apple.com/videos/play/wwdc2023/10160/">swift ui performance</a>’ that partly explain the internals of how SwiftUI work and implicit assumptions. Most will not learn about these fragile gotchas when using SwiftUI.</li><li>If I was to do the data model again, I would avoid usage of State management within SwiftUI and inject everything in as an observable from the top. This lets you manage the lifecycle of all data, business logic and multithreading code completely independent of SwiftUI.</li><li>SwiftUI’s internal C++ AttributeGraph library that determines the state lifecycle of state objects attached to views is hard to reason about, especially in the way we used it. Multiple instances occurring of our StateObject ViewModels when we were only expecting one induced a lot of further issues when it started interacting with Swift Concurrency and combine, because it caused us to stress the threading system more than it needed to with repeated actions.</li><li>We call a lot of heavyweight apple libraries, like Vision, CoreML, PhotoKit, etc. We also use a complicated Google Photos library as part of an optional feature. These codebases internally do not conform %100 to the Swift Concurrency contract, sometimes leading to our deadlocking issues under times of threading stress, which was made even more intermittent by SwiftUI’s AttributeGraph non-deterministic object lifecycle behavior.</li><li>Combine can spawn 100 threads unexpectedly when you use it wrong. Dispatch stops itself after a certain point.</li></ol><h3>How we solved some of the above issues</h3><ol><li>We migrated away from Combine. This started before I joined, but some vestiges remained causing bugs.</li><li>We moved heavyweight operations into explicit dispatch queues so we don’t stress the shared swift concurrency ‘cooperative’ thread pools.</li><li>We made some ViewModels ObservedObjects that parent StateObject vended into views, but this wasn’t easy in some spots due to our reliance on StateObject. We are still using StateObject improperly in a bunch of spots, but have improved our usage of it as we started to understand it’s implicit requirements better.</li></ol><h3>How we are going to solve the rest of the issues: UIKit &amp; Dispatch</h3><p>To solve the rest of our issues, we were facing down needing to significantly refactor our view models and business logic to be more SwiftUI friendly, so we could reason about the lifecycle of our data models and move away from StateObject usage. We are also facing down moving a lot of async await code to dispatch so we avoid unobservable deadlocks and predictably allocate threading resources for our various subsystems. Overall it is a huge refactor of our code base with an already tricky multithreading system.</p><p>After talking a while about it, we realized that moving the views from SwiftUI to UIKit would actually be the smaller lift. We would free ourselves of SwiftUI’s indeterministic state management, our ViewModel lifecycles will work predictably and well with UIKit and overcomplicated things like SwiftUI navigation would be made easy. On top of that, we wouldn’t have to spend weeks in the future to implement things that are 5 liners in UIKit that are multi-day projects to implement equivalents in SwiftUI.</p><p>After we made that decision, one of our engineers, made a UIKitCollectionView prototype in a few hours of a core screen that performed literally 10x faster than our SwiftUI version with way less headache, and he did it out of out excitement with nobody asking him.</p><p>Several months after we made this decision, we finished the migration of all navigation management, app management, etc into UIKit. We don’t have a top level App struct anymore, use Application &amp; Scene delegate instead. Each SwiftUI screen is wrapped with a hosting UIViewController and we use UIKit’s UINavigation to manage screen management, modal displays, etc. Everything works much better now. We partly estimate that this works better because each SwiftUI attribute graph tree is not part of one App mega tree, but instead many separated smaller trees that AttributeGraph is probably having an easier time dealing with performantly.</p><h3>Next Article</h3><p><a href="https://medium.com/goodones/pareto-optimal-apple-devtools-b47e79b6676f">The next article</a> will be a bit of a rant about how we held these tools wrong, but asking why are they so hard to hold in the first place, and us imagining a world where apple made some more pareto optimal choices in their devtools, saving literal billions of dollars of human time &amp; life.</p><p>This is part of a series of blog posts where we go into:</p><ul><li><a href="https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d">Why we decided to make a native iOS app vs React Native and an a outline of our stack.</a></li><li><a href="https://medium.com/goodones/ollies-app-architecture-continued-2b572e23c204">Baseline architecture — including our data model which is most of the app, and then the views.</a></li><li><a href="https://medium.com/goodones/how-ollies-ai-system-works-ac7d8838df11">AI model — this is a bit of the secret sauce so we’ll keep it closer to the vest, but we’ll share our basic insights. It turns out it’s quite difficult to figure out which photos people will like!</a></li><li><a href="https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c">The async/await quagmire — what issues we ran into with SwiftUI &amp; Concurrency, and why we decided to move on to UIKit &amp; Dispatch.</a> (This Article)</li><li><a href="https://medium.com/goodones/pareto-optimal-apple-devtools-b47e79b6676f">Imagining a world where Apple made some more pragmatic decisions for their new dev platforms vs. the current ones they made today.</a></li><li><a href="https://medium.com/goodones/how-swiftui-concurrency-could-improve-16c13664e697">How Apple can forward fix their issues with SwiftUI and Swift Concurrency.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cfdefe918d1c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c">Why Ollie is moving away from SwiftUI to UIKit</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ollie’s App Architecture Continued]]></title>
            <link>https://medium.com/goodones/ollies-app-architecture-continued-2b572e23c204?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/2b572e23c204</guid>
            <category><![CDATA[ios]]></category>
            <dc:creator><![CDATA[Mahyar McDonald]]></dc:creator>
            <pubDate>Wed, 14 Feb 2024 23:22:39 GMT</pubDate>
            <atom:updated>2024-02-22T17:42:32.431Z</atom:updated>
            <content:encoded><![CDATA[<p>This is a continuation of our <a href="https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d">last article</a> about our app architecture and our reasoning behind being a native app vs. a cross platform one.</p><h3>Local Data State Model</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XcjSBBGGQEVy6Bq6o2kGdQ.png" /></figure><p>Our app is more data model &amp; business logic than view logic. Due to the reactive nature of SwiftUI, we made it a bit too reactive, which we plan to move away from. But the basic data model types are:</p><ul><li><strong>Photo:</strong> A singular photo, from multiple providers or ‘destinations’, be it Google Photos or PhotoKit. Has many attributes such as location, AI image tags, source, time, album membership, etc.</li><li><strong>Album:</strong> This represents “interesting” sets of photos. It could be all the photos in a month, a photos of certain category (e.g. pets), or a certain trip. This is somewhat replicating functionality from Google Photos and Apple Photos but we needed to do that in order to help users sort photos with a certain theme.</li><li><strong>PhotoStack:</strong> A small subset of photos from an Album that is not too overwhelming to sort or train an AI model with. Usually 15 to 30 photos, which by most people, can be sorted through in less than a minute.</li></ul><p>All of these basic models and their attributes are persisted in our sqlite3 DB, wrapped by GRDB, which is then wrapped by our own Repository &amp; Query pattern that generates and persists these model objects.</p><p>We also generate some Apple albums as a backup source of truth &amp; form of export. All photos you mark as favorite, trash or ‘keep’ in our app is also put into an equivalent Apple album inside the Apple Photos app with PhotoKit. So even if you delete our app and its database, we can restore most of your app state via these Apple albums.</p><h3>Event Generators &amp; On Demand Actions</h3><p>We then generate model objects to populate the database and albums via various event generators and on demand actions:</p><p><strong>Event Generator: PhotoKit Import Watcher</strong></p><ul><li>Most photos come in via an import action every time you launch the app, which queries photokit for the metadata of your photos to kick start various other services and put them in our DB. It also detects when photos are deleted to mark in our DB as ‘deleted’.</li><li><strong>Album Calculator:</strong> Since this is a pretty quick operation, this doesn’t need to be done as a background service action and is just done on demand for new photo imports</li><li><strong>Stack Generator:</strong> From the raw stream of albums, we then generate organization ‘stacks’ for the PhotoStackLoader to use within those albums.</li></ul><p><strong>Photo Processor Service:</strong></p><ul><li>This is a ‘background’ service that does the heavyweight raw image → vision tag scanning and trip album calculations to put in the DB to make other actions act quickly. It acts continuously in the background until it is done scanning your entire photo library, watching the DB photo stream.</li><li><strong>Tag Extractor:</strong> Associates tags for each photo, such as ‘beach’ or ‘flower’.</li><li><strong>Trip Calculator:</strong> Takes photos and puts them into various trip albums or generates new ones for them.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8joW9DQL8xjiBwQKvLqu7Q.png" /><figcaption>We even have some fun progress views in our developer panel.</figcaption></figure><p><strong>On Demand: PhotoStackLoader</strong></p><ul><li>This is used whenever a view needs to load a stack of photos for the user to organize. It’s fed a PhotoStack object to load.</li><li>Loads the full image thumbnail + photo from PhotoKit for all photos in the stack.</li><li>Generates the AI model rating for each photo via on device inference. We do this on demand since the model changes every time a PhotoStack is committed, so we can’t do it in the background photo service.</li><li>Calculates related / similar photos, called a ‘bracket’, via a 2 minute time window, detects if they are similar or not and chooses the best photo out of them. A stack can have multiple brackets within it.</li></ul><p><strong>On Demand: Post Stack Committer</strong></p><ul><li>After a user finishes a batch, we need to save and process the results.</li><li>Commits user’s decisions to the DB</li><li>Trains AI model with new photo data and decisions</li><li>Modifies Apple albums with new state (new favorites, etc)</li><li>Deletes photos marked as trash by user with a PhotoKit prompt</li></ul><p><strong>Subscribing Event Watcher: Gphotos Uploader</strong></p><ul><li>Any new ‘favorite’ images, if the user has enabled it, gets uploaded to google photos also via the GPhotos library.</li></ul><h3>Major Dispatch Queues</h3><p>We found that with Swift Concurrency’s single shared concurrent queue, putting all network, PhotoKit, CoreML, vision calls as basic async/await functions to be an exercise in creating heisenbugs and unobservable deadlocking. To mitigate this we explicitly created a set of dispatch queues for each major subsystem that has a heavy workload:</p><ul><li>Photo model object update queue</li><li>GPhotos Upload &amp; Queries</li><li>Apple Vision Calls</li><li>PhotoKit Queries and Photo Loads (This includes adding / removing things from Apple albums)</li><li>Database Actions</li><li>CoreML model inference &amp; updates (this is what our custom model runs on)</li><li>Logging systems that already came with their own queues</li></ul><p>Each queue was an exercise in hoping we didn’t need to do it with Swift Concurrency / Combine, but ending up needing to in the end to avoid deadlock and heisenbug issues. More details about this in the next blog post!</p><h3>View Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JpRjjNyFOr0J99SJuE3Kqg.png" /></figure><p>Like most SwiftUI apps, we are side effect state driven. Flip a state variable somewhere, and your view changes in reaction to that state change as a side effect with observer bindings! We recently changed screen management to be purely UIKit driven, but in the past when we were SwfitUI driven we had top level screens being controlled by several state variables within the top level App &amp; AppState objects.</p><p>Top level views were swapped in to a root view:</p><ul><li>Login</li><li>Onboarding</li><li>Main</li></ul><p>We had an overlayView as a special global view that any view model can set over the rootView, as an alternative form of .fullScreenCover() that gets around some bugs and limitations in that API.</p><h3>View Models</h3><p>Many full screens have their own associated view model which executes a lot of business logic on their own. A lot of PhotoStackLoader actually occurs in the StackLoadViewModel, Temporal stack generation happens within the TemporalStackViewModel, interacting with various managers and so on. A lot of services get generated as part of DI initialization and get started by app launched or app foregrounded notifications from the app to manage the lifecycle properly.</p><p>This heavy ViewModel architecture is something that doesn’t jive well with SwiftUI in some fairly subtle ways and is something we eventually plan to move away from. Also SwiftUI’s performance for large image collection views is hard to make performant compared to the equivalent UICollectionView we have found, since it is really easy to trigger SwiftUI to act in a non lazy loading way compared to UIKit</p><h3>Navigation Stack Manipulation Alternatives</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*kjm11M3t5QiZrspe68I9Mw.png" /></figure><p>The StackLoadView is kind of a goofy hack, with the loading view being the parent view of the grid view / carousel detail view navigation stack. This was mostly put in because you don’t have the fine grained control that you have in UIKit that allows you to manipulate the navigation stack while also being compatible with iOS 15. We wanted to be able to load a new stack once an old stack was done without going back to the home tab and without being able to ‘go back’ to the old stack essentially. In UIKit this isn’t a hard thing to do.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B4zPY1S8U-S367hLnqmnuQ.png" /></figure><p>Similarly with our login → onboarding → grid → main tab navigation, we had to implement it as a root view swap + overlay because you similarly cannot manipulate the navigation stack in iOS 15 that well to prevent going back to previous views or other complicated transitions. This has created a lot of hacky complexity in our transitions that has been the source of several major bugs that we got rid of with transitioning our screen management to UIKit.</p><h3>Next Articles</h3><p>In the next articles, we will go into the lessons learned from using Swift Concurrency and SwiftUI in a photo heavy, multithreading heavy application, the surprising footguns you can run into with using them. In another article we will go into why we decided to migrate to UIKit &amp; Dispatch to solve them quickly compared to radically changing our data model architecture.</p><p>This is part of a series of blog posts where we go into:</p><ul><li><a href="https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d">Why we decided to make a native iOS app vs React Native and an a outline of our stack.</a></li><li><a href="https://medium.com/goodones/ollies-app-architecture-continued-2b572e23c204">Baseline architecture — including our data model which is most of the app, and then the views.</a> (This Article)</li><li><a href="https://medium.com/goodones/how-ollies-ai-system-works-ac7d8838df11">AI model — this is a bit of the secret sauce so we’ll keep it closer to the vest, but we’ll share our basic insights. It turns out it’s quite difficult to figure out which photos people will like!</a></li><li><a href="https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c">The async/await quagmire — what issues we ran into with SwiftUI &amp; Concurrency, and why we decided to move on to UIKit &amp; Dispatch.</a></li><li><a href="https://medium.com/goodones/pareto-optimal-apple-devtools-b47e79b6676f">Imagining a world where Apple made some more pragmatic decisions for their new dev platforms vs. the current ones they made today.</a></li><li><a href="https://medium.com/goodones/how-swiftui-concurrency-could-improve-16c13664e697">How Apple can forward fix their issues with SwiftUI and Swift Concurrency.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b572e23c204" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/ollies-app-architecture-continued-2b572e23c204">Ollie’s App Architecture Continued</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[15 Years Later, There Is Still No Good Session Replay For iOS]]></title>
            <link>https://medium.com/goodones/15-years-later-there-is-still-no-good-session-replay-for-ios-f8d335999737?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/f8d335999737</guid>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[session-replay]]></category>
            <dc:creator><![CDATA[Mahyar McDonald]]></dc:creator>
            <pubDate>Wed, 14 Feb 2024 22:59:52 GMT</pubDate>
            <atom:updated>2024-03-21T00:27:51.006Z</atom:updated>
            <content:encoded><![CDATA[<p>At <a href="https://getollie.ai/">Ollie</a>, I went on a shopping trip for a session replay solution and found there still isn’t a good one 15 years after the release of iOS. We wanted one so:</p><ol><li>Our designer could do their user studies &amp; interviews without an awkward video recording flow and to not see the contents of people’s photos, and to let him see how users interact with new features to catch design problems sooner.</li><li>To help users with tricky support issues</li><li>To debug some bugs that you just need to see what is actually happening in the UX to truly fix, internally and externally.</li></ol><p>iOS development, 15 years after the introduction of the iPhone, still does not have a good public session replay system to this day. When you compare what is available for the web, and the fact that the main computing device of most people now is their smartphone, this is a sad state of affairs. There is one hopeful out there although that I think could make it soon although.</p><p>We wrote this article to save other people the many hours of searching to find out, no, this stuff still doesn’t properly exist. And for PMs, Sales &amp; Founders at these companies to have an very public example they can point to show that the bad stuff they are doing really does hurt them.</p><p><a href="https://medium.com/goodones/our-ios-subscription-management-shopping-trip-fbe22de72717">You can also see our recent subscription management shopping trip here too.</a></p><h3>Our Requirements</h3><ol><li>Works with a mainly SwiftUI app, but also works in a mixed UIKit + SwiftUI context.</li><li>Can hide views for privacy and bandwidth reasons, and do it well within subviews instead of just censoring regions of the overall screen. We are a personal photo organization app, and we really want to avoid looking at people’s photos. This is not possible when people do simple screen-shares, screenshots or video recordings.</li><li>Does not degrade app performance</li><li>Is not just a series of screenshots</li><li>No manual instrumentation beyond which views you want to hide, what you want to label screens and piping logs.</li><li>Transparent and predictable pricing, does not waste your time with “contact us” &amp; “the price is what we think you can pay”. Easy self service trial system to save us time.</li></ol><h3>Top Contenders</h3><h4><strong>Bitdrift</strong></h4><p>Out of all contenders, this is probably the most exciting one even though it’s very raw. They have no performance penalties, they were able to add SwiftUI support very quickly and record the view tree as an actual view tree vs. a series of screenshots. They also have a workflow triggering system that is very useful to use them as an efficient debugging tool after the fact.</p><p>The only missing main feature is the fact they don’t record the contents of views. I have talked to them, and they are open to implementing this as a form of a fidelity parameter. By them recording the view tree as a tree, this allows for advanced view censorship that doesn’t censor subviews unnecessarily like UXCam. They also don’t require any manual instrumentation beyond log piping.</p><h4><strong>UXCam</strong></h4><p>This was the second one that actually works, has SwiftUI support and doesn’t require massive amounts of manual annotation. They record sessions like a low framerate video essentially, and from my testing do view censorship as a ‘censorship zone’ on the video, which means views that happen to be in the same space are also censored. This leads to a lot of issues, where things we want to censor still ‘leak through’ sometimes through animations, and sometimes areas that we don’t want censored get censored (like table cells under a button overlay censoring the button overlay too) get censored also.</p><p>It’s the only complete system we’ve found so far, so this might be the one we use for now. I hope that bitdrift can deliver although so we get view censorship with proper layer support. Also their managment dashboard feels a bit sloppy quality wise.</p><h4>LogRocket: Close but no Cigar</h4><p>We tested logrocket in Feburary 2023, but found it had horrible performance issues with our app and only recorded half of the screen contents. Reaching out to their support led nowhere, so we abandoned adopting it.</p><p>Trying again in January 2024, almost a year later, and we see the exact same issues as last time. This reduces our confidence they will figure it out. They were the least effort to integrate although, and automatically recorded network interactions and logs without any work on our end, which makes this extra disappointing, since it does seem like a nice product on the surface.</p><h3>One Extra Bad One: FullStory</h3><p>FullStory has a SwiftUI library according to their docs, but signing up for their self service trial seems to pretend that mobile apps don’t exist in this age of literal billions of smartphones. Reaching out to get mobile access to their trial, I was told I was enabled, but their dashboard didn’t have any set up instructions to make it obvious, which is the <strong>only</strong> vendor that had issues like this. Every other vendor we tried that had iOS support did not put up multiple barriers to try their product.</p><p>This overall mobile onboarding neglect and attitude does not bode well so we decided to skip over them. They also have a reputation of being a pain in the ass to work with and overpriced, which seems to correlate with our own experience and the price quotes we got.</p><p>I think FullStory is missing the full story on their own product.</p><h3>Everyone Else</h3><p>It’s pretty crazy how many others just don’t support Swift UI, 3 years after release, or have some other fatal flaw like LogRocket.</p><ul><li><strong>Posthog:</strong> no iOS support for now, although you can see activity on github. I look forward to what they create in a few years.</li><li><strong>Datadog:</strong> no SwiftUI support, which isn’t documented in an obvious way. You find out by looking at the content of a recording which just shows a big “SwiftUI” error box. Not very confidence inspiring.</li><li><strong>Instabug: </strong>Instabug’s “session replay” is just a series of screenshots. We abandoned Instabug due to a general bad experience with them and replaced them with an email bug reporter.</li><li><strong>DynaTrace:</strong> SwiftUI with manual instrumentation. Doesn’t seem like it’s their main focus, very ‘business sales process’ focused . A lot of web / backend focus, the service overall reminds me of datadog. We decided to skip trying due it looking like a neglected product and a lack of an obvious self service trial mechanism.</li></ul><h4><strong>Quick Skim Vendors</strong></h4><p>These are vendors we skimmed over quickly and decided to skip and save time or their docs were clear about their lack of iOS support, which we appreciate.</p><ul><li><strong>Plerdy:</strong> seems web only?</li><li><strong>UserX.pro:</strong> seems kind of janky looking at their website, but they have something for ios.</li><li><strong>Embrace:</strong> nothing obvious</li><li><strong>Acoustic Tealeaf:</strong> nothing obvious</li><li><strong>Quantum Metric:</strong> they have a swiftui logo, but no public docs and pricing</li><li><strong>Glassbox:</strong> nothing obvious, no public docs or pricing</li><li><strong>ContentSquare: </strong>They have SwiftUI docs, but no self-service free trial or public pricing. They want to force the demo and waste your time. I asked them for a 2m demo video to see if I can save time with the process and didn’t get a reply back.</li><li><strong>SessionFox: </strong>Doesn’t even have their own domain, signing up for a free trial leads to DNS errors. Nope.</li><li><strong>HotJar:</strong> No iOS support</li><li><strong>Heap:</strong> no iOS support</li><li><strong>Sprig:</strong> no iOS support</li><li><strong>Sentry: </strong>no iOS support</li><li>I’m probably missing 20 other providers who don’t have iOS support, but have a web product!</li></ul><h3>Our Suggestions To Vendors</h3><ol><li>Finally prioritize mobile, it’s the actual platform the world is using. Web session replay is a commodity, mobile session replay is not.</li><li>Implement subview friendly view censorship. Apps layer views and have scroll views.</li><li>Stop doing “contact us pricing, high touch, high time amounts consumed, demo only trials”, be transparent in very obvious ways about everything, otherwise you reduce our trust in you. If you have a reputation for being a pain, fix it! You don’t know what your missing by forcing this.</li><li>Make self service trials as easy as possible. You don’t know who you are missing with people who don’t want to waste their time with pushy sales people. Most people are not going to be as through as we are, and as you see, we evalutated literally dozens of vendors.</li><li>Engineering opinion matters, because if your solution breaks the customers product, they’re not going to adopt you and nobody is doing it that well yet! Who is responsible to make sure it doesn’t break things? Engineering.</li><li><strong>Performance of your SDK &amp; dashboard matters, if it makes everything freeze and stutter, we won’t use you!</strong></li><li>Answer your support / sales emails within a day or two. If the prospective customer doesn’t book a time for a time wasting sales meeting and instead replies to the email to ask questions, please reply to the email instead. It’s the business equivalent of being an old person who insists on phone calls vs. the faster mode of texting. “This meeting could’ve been an email” applies outside of companies too!</li><li>Make demo videos of what you can provide, on youtube, of your actual software for each major feature / platform, SEO keyword optimized. It’s not hard to do! “How Session replay for iOS in UXCam works” should exist, it does not. Feel free to disable comments and direct people to support in your video description for questions to reduce support watch space.</li><li>A janky low quality marketing website or dashboard does not inspire confidence and can cause people to skip you. If you can’t bother to get your fonts and margins right in your UX focused product, what else are you sloppy with deeper inside?</li><li>It is possible to make mobile session replay in a good way, there are companies that have all the technical pieces to do it well out there. Even if it only exists within large tech companies internally such as Facebook and Lyft. Bitdrift might eat everyone’s lunch in a couple of years.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f8d335999737" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/15-years-later-there-is-still-no-good-session-replay-for-ios-f8d335999737">15 Years Later, There Is Still No Good Session Replay For iOS</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ollie’s iOS Subscription Management Shopping Trip]]></title>
            <link>https://medium.com/goodones/our-ios-subscription-management-shopping-trip-fbe22de72717?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/fbe22de72717</guid>
            <category><![CDATA[subscription]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[ios-app-development]]></category>
            <category><![CDATA[in-app-purchase]]></category>
            <dc:creator><![CDATA[Mahyar McDonald]]></dc:creator>
            <pubDate>Mon, 08 Jan 2024 21:52:58 GMT</pubDate>
            <atom:updated>2024-01-11T01:16:18.181Z</atom:updated>
            <content:encoded><![CDATA[<p>Our app <a href="https://www.getollie.ai/">Ollie</a> is an iOS only app. We implemented subscription support directly via StoreKit 2, and didn’t think we needed to use subscription management services due to being a single platform app, and a lack of a desire to pay a %1 tax on our revenue.</p><h3>Our Requirements</h3><p>After a while we realized we still needed them, mostly because:</p><ol><li>Apple’s app store connect dashboards are not that good compared to 3rd party services.</li><li>We were not tracking subscription events server side, so users that pay but haven’t launched the app were not being tracked properly in Mixpanel and other services to optimize our ads and internal data.</li><li>Traditional data glue providers like Segment do not pipe data from app store connect into analytics providers like Mixpanel. It’s a strangely missing gap. You have to implement it manually yourself or use a subscription management provider directly to do so.</li></ol><p>This need led us on a shopping trip to choose a subscription management platform. During our research, we found there isn’t much out there in articles reviewing which ones are better, or articles like this one outlining company’s experiences in using them, thus us writing our own.</p><p>One thing we were trying to avoid is the need to install their SDKs in our app to get good server side subscription tracking support via an app store connect configuration and to ship faster. We found out that every provider requires an SDK to track subscriptions, so save yourself some doing integration testing and realize no matter what you choose, you will need to integrate an SDK.</p><h3><strong>What We Tried</strong></h3><ul><li><a href="http://appflow.ai/">Appflow</a> : %0.38-%0.68 of MTR (monthly tracked revenue ). Seems like a worse RevenueCat IMO, website seemed a bit sloppy which made us worry about quality. Cheapest option.</li><li><a href="https://www.qonversion.io/">Qonversion</a>: %0.6-%0.8 of MTR, free basic ‘subscription analytics only’ option. <strong>This is what we ended up using.</strong> We upgraded from free because we wanted the mixpanel glue. Does not have a merged all app revenue view yet, but it’s on their roadmap.</li><li><a href="https://www.revenuecat.com/">RevenueCat</a>: %1 of MTR, the OG &amp; most popular service, we didn’t end up using them.</li><li><a href="https://adapty.io/">Adapty</a>: %1-%1.2 of MTR. Most expensive of them all, the web GUI looked the nicest out of all of them. Getting the MTR rates was a bit hidden in their pricing pages, but they are there when you look at the table headers after you scroll down when we looked.</li></ul><h4><strong>Paywall Focused Providers That We Skipped</strong></h4><ul><li><a href="https://www.namiml.com/">NamiML</a> : Seems to be more paywall orientated?? $50 / 1000 impressions, which is incredibly expensive compared to competitors. Had to dig for pricing. Quickly dropped investigating further</li><li><a href="https://superwall.com/">Superwall</a>: We tried it to speed up paywall iteration several months ago, it had problems actually purchasing with storekit when we were testing in testflight. It ended up being faster to make our paywall ourselves, I don’t think they do the subscription tracking &amp; management glue stuff like RevenueCat. The founder has a really good twitter thread on iOS app store sales although: <a href="https://twitter.com/jakemor/status/1549281861687451648">https://twitter.com/jakemor/status/1549281861687451648</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fbe22de72717" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/our-ios-subscription-management-shopping-trip-fbe22de72717">Ollie’s iOS Subscription Management Shopping Trip</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Ollie’s AI System Works]]></title>
            <link>https://medium.com/goodones/how-ollies-ai-system-works-ac7d8838df11?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/ac7d8838df11</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[photography]]></category>
            <dc:creator><![CDATA[Mahyar McDonald]]></dc:creator>
            <pubDate>Tue, 05 Dec 2023 19:15:16 GMT</pubDate>
            <atom:updated>2024-02-15T01:17:08.125Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://www.getollie.ai/">Ollie</a> helps people organize their photos to make them useful. People take a lot of photos, but often get overwhelmed going through their personal photo collection, having to sort through pictures of screenshots, a blurry photo of a funny car and a beautiful sunset photo. It’s easier to take a photo than to browse through or organize your photos.</p><p>To make this stream of photos useful for people, we help them organize them into three categories:</p><ul><li><strong>Favorite (Top %20):</strong> The beautiful sunset photo, or the best photo of ten in a burst shot</li><li><strong>Keep: </strong>The receipt you need to keep for work</li><li><strong>Trash:</strong> That blurry photo or screenshot you don’t care about anymore.</li></ul><p>It gets tedious fast to manually sort your photos one by one into categories to make them useful, so our app suggests which category a photo should be put in via a set of AI models we have created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*GRW1y8zoFaW1DHm0o9C78Q.gif" /></figure><p>This presents another problem though. We have found what people like in photos is unique to themselves. Some may like food photos, others might like artistic blurry photos, others hate blurry photos and food photos. As a result you usually cannot make a generic model that will determine if a photo is ‘good’ or not. If so, we would be satisfied with the “zero explicit input” models that Apple and Google have to create highlight reel albums for their users today.</p><p>This led us to the current strategy of bundling a set of seed models with our app and training the model on device as the user expresses what their preferences are.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jgKwUxrku6Wbr3Be0Mb-3w.png" /></figure><p>To solve this issue, we fine tune part of our initial generic AI model based on the user’s explicit input. As people organize their photos with our app, we train our model with new photos from the users camera roll and their organization decisions. As the user corrects our model with what their real preferences are, we create a model that becomes more accurate in guessing their personalized preferences.</p><p>Since people’s photos are private and it creates a lot of slowness to upload people’s photos to a centralized server to train a personalized model, we do this model training on the users device in a rate / update cycle, <a href="https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d">which you can see the architecture of here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CTbYoiQn5FeOJQsYL49RGA.png" /></figure><h3>Current Model Architecture</h3><p>We use multiple independent models in Ollie to implement its functionality today. We have various models for:</p><p><strong>Photo Set Classifiers:</strong> Evaluates photos as part of a set</p><ul><li>Determining if a photo is a duplicate or related in a cluster</li><li>Choosing the best photo in a cluster</li></ul><p><strong>Single Photo Classifiers:</strong> Evaluates each photo independently of the other</p><ul><li><strong>Trash vs. Non-Trash Photo:</strong> “Should we delete this photo?” classifier</li><li><strong>Favorite vs. Non-Favorite Photo:</strong> “Is this a good photo?” classifier</li><li>Detecting screenshots</li><li>Initially targets about %20 of photos to be marked as ‘favorite’ to follow the <a href="https://en.wikipedia.org/wiki/Pareto_principle">pareto distribution</a><strong>.</strong></li></ul><h3>Basic Favorite Model Example</h3><p>To make our current favorite rating model, we use <a href="https://en.wikipedia.org/wiki/Transfer_learning">transfer learning</a>. Transfer learning is a machine learning technique where a model developed for a specific task is repurposed as the starting point for a model on a second task. It leverages knowledge (features, weights, etc.) from the initial task to improve learning efficiency and performance on the new task, especially useful when the new task has limited data. This approach is akin to applying insights gained from solving one problem to a related but different problem, capitalizing on prior learning to accelerate subsequent learning processes.</p><p>In our case, we use a few layers on top of <a href="https://medium.com/@godeep48/an-overview-on-mobilenet-an-efficient-mobile-vision-cnn-f301141db94d">MobileNet</a>. MobileNet is a small image classifier model that takes a photo for input, and returns classification tags such as “beach” or “car” about the contents of the photo. It’s a fairly small and lightweight model that executes quickly with good performance for its size. We then transfer the learning from that model, into our custom layers on top of it, and then train the new composite with our internal employee dataset of curated photos to create our base initial model.</p><p>We are currently developing a far more complicated multi-modal model set that makes our model determine if it is favorite, keep or trash in one shot, while still being fast to execute and updatable on the user’s device. We are trying many different experiments in this and it is an active work in progress, but our current first attempt has been a surprisingly effective one.</p><h3>AI on the Edge</h3><p>Currently we train our initial seed model on employee photos with our own desktop &amp; cloud computers. Once we create a topology &amp; initial data set that works well with employee data, we add it to the model experiment list on our CDN.</p><p>Once it’s added to the list, user’s apps download the new models dynamically and our app starts using it alongside the current model the user is using. We then watch the accuracy, recall &amp; precision results it generates on users phones after multiple training runs with the user’s own photo data, and compare it against the current top model.</p><p>If the new model is a winner, it will replace the previous model that the user is using, if it isn’t, then we don’t fully deploy it. We don’t know the contents of the user’s photos when we are tracking this, the app just reports what our AI recommends a photo be rated as, and if the user readjusted that rating to something else or not along with which model they used. Basically a (random_photo_uuid, ai_rating, user_rating, model_id) log. Essentially we are doing model A/B testing on our user’s phones.</p><p>We also plan to create a ‘retraining’ system that retains an embedding cache (25kb per photo) from our base MobileNet model of deleted photos on the user phone. This allows us to get new models back up to speed faster, but we haven’t implemented it yet. The embedding cache will remain on the user’s phone to retain their privacy.</p><h3>Preparing the Data</h3><p>A significant part of building a great model starts from data engineering — collecting the raw information, understanding the “shape” of it, and preparing it for the model.</p><p>This is a big part of what we do also in training our initial generic model¹ for first installs, and how we feed data into the fine tune process to update the user’s personalized model.</p><p>[1] Our generic initial model is only trained with employee data or otherwise donated photo curation sets. We don’t use any photos or data from our actual users to train our initial generic model.</p><h3>Example AI Photo Organization Data Issues</h3><p>With photos, you still need to apply some heuristics in how you initially feed data into the model. A few classes of ‘bad’ data we discovered caused problems are:</p><h4>Not Handling Similar or Duplicate Photos Properly</h4><p>Often when people take photos, they often take 5+ photos in short succession of the same scene, and then choose only one or two to keep and delete the rest of the photos. Another similar situation is keeping one identical photo and deleting the rest of them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oV8lh9iMBr269N1HZ-05KQ.png" /><figcaption><em>In this sunset photo cluster, they’re all good photos on their own, but because you have many similar ones, you would probably only pick one of them as a favorite, and delete or keep the rest.</em></figcaption></figure><p>Naively just feeding these results into any model will confuse it greatly, because the base model you use to transfer learn with will probably output identical results, but have different training labels for the same thing. This introduces noise into the dataset, and certain user behavior patterns can cause it to get extra confused if you naively train with photo clusters. Have the right “bad” combo of user preferences and clusters and you create a useless photo organization model.</p><p>To deal with this, we filter out clusters from the training set. Currently we only take the single “top” photo according to a simple decision tree and feed that into the model. So if you have 1 favorite and 4 “keeps”, we only train the model with the one favorite and ignore training with the “keeps” to avoid this issue.</p><p>This also means you need a completely different model for evaluating ‘what’s the best photo in a cluster vs. ‘what rating should this individual photo get’.</p><h4>Data Imbalance Issues</h4><p>People tend to only like the top %20 of their photos, and often do not have a set of ‘deleted’ photos lingering on their phone like they tend to do with their current favorites. This makes bootstrap model personalization more difficult, because you don’t want to make a model that just coin flips with the distribution of best, keep or trash photos that the user tends to choose. This means you have to use various techniques, such as a sampling to keep the training data set even, so the model actually learns the content of photos, along with other techniques we won’t go into here.</p><h4>Data Drift (The “Ex-Partner” Issue)</h4><p>Another issue is something we call the ‘ex-partner’ problem. You favorite all the photos of your partner, then you break up, and now all the photos of your partner are put into the trash, but you always want to favorite the photos of your kids, even though your partner is in the photo. Or you are really into motorcycling, have a motorcycle accident and never want to see motorcycle photos again as a result. Like the duplicate photo issue above, this creates chaotic noise. The user liked this a lot, then they started not liking it on a dime.</p><p>This is a pretty complicated issue that will depend on the nature of the data drift and the user’s preferences, and is not something we have gotten to fixing yet since our app hasn’t been around long enough for this to develop into too much of an issue yet.</p><p>We have several ideas, with things like ‘period models’ where we create multiple models that govern specific time regions, or even making a feature that removes certain things like ‘motorcycles’ or ‘this face’ from the users training set and retrains their personal models with the new data set. It will be a pretty fun issue to solve.</p><p>We even have the funny idea for a mini app that will edit your ex out of all of your photos via cropping or AI subject erasing, but that is for the future 🙂.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pFSRS9ChIQiUnh8ygHeBGQ.png" /><figcaption>Stalin would’ve loved an “erase your ex” app!</figcaption></figure><h3>The Issue with Apple &amp; Google Photo Data</h3><p>As you see, there are many, many interesting data curation issues with personalized photo curation. If you observe the types of AI albums that apple and google photos create, they are very thematic, based on a sophisticated combo of heuristics &amp; AI models to find various themes such as “beach time”, “vacation”, “cluster of photos with a face you take a lot photos of”, etc. They then observe user engagement with these generated album types and make more of them if they increase engagement metrics.</p><p>The problem with these albums is what I call the “line up at the stadium” issue. They can add photos that tend to be not that great on their own, such as a photo of the line up at the stadium, attached to an event you really care about and engage with, which is going to the stadium and seeing the ball game. They tend to pick photos that are a mix of meh and good. If they add a filter for lineups, they might exclude good photos with your buddies in the line up. It’s a hard thing to balance.</p><h4>The Luxury of the Explicit Signal</h4><p>Apple and Google work with the assumption that the user gives no explicit signal (up-or-down for every photo), and few implicit signals (favoriting, sharing, engagement). Because of that, it is incredibly difficult for them to gather the data set about what a user actually cares about within their personal photos.</p><p>To be clear, it’s a completely reasonable product decision to build the “hands off” model.</p><ul><li><strong>Human nature: </strong>It’s <em>really</em> difficult to get people to engage with their photos habitually. It’s just a lot of work. We’re dedicated to making that a reality, but right now still a small percentage of user, who care a lot about their photos and are drawn to it. For us, we can build a business out of it. For Google and Apple, who have a 1B+ users it doesn’t move the needle at all. This is the classic “<a href="https://en.wikipedia.org/wiki/The_Innovator%27s_Dilemma">innovator’s dilemma</a>”.</li><li><strong>Business incentives: </strong>Once you really help users organize their photos, since users tend to delete about %50 of them when they really get going, that would reduce cloud storage revenue, so working against their own business interests then.</li></ul><p>With our app although, our incentives are aligned. We give the users the final choice on every photo. This allows us to quickly learn through their explicit preferences and create better results than what Apple Photos and Google Photos do. Our aim is not to replace these products, which are frankly excellent in their own right. Our aim is to work together and make the experience of using them even better. For example, say you spent a weekend in Tahoe, and Apple Photos creates a “memory” for it. If you didn’t give any explicit signals, the output would be a rather indiscriminate set from Tahoe. With Ollie though, because you’re favoriting ~20% of the photos from that trip, the “memory” that is created will be reflective of your own preferences. Eventually our goal is to auto-organize a user’s photo stream once accuracy hits a certain point and only show the good photos to them.</p><h3>Summary</h3><ul><li>We have made a local, private AI app that learns what you like in organizing photos.</li><li>We use many small models at Ollie to implement our features currently, with one example being a MobileNet based model specialized for our rating case.</li><li>We are actively developing a multimodal model for all rating types that should be much better.</li><li>A significant chunk of our work is data management, like most AI projects. Learning to deal with these issues and learning about the problem space is the interesting part for the most part. There are still many things to learn!</li><li>Apple &amp; Google are stuck in organizing photos well due to various internal structural counter incentives that prevent them from collecting it from their users, which makes it prime for a small AI startup to solve.</li></ul><h3>Next Articles</h3><p>This is part of a series of blog posts where we go into:</p><ul><li><a href="https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d">Why we decided to make a native iOS app vs React Native and an a outline of our stack.</a></li><li><a href="https://medium.com/goodones/ollies-app-architecture-continued-2b572e23c204">Baseline architecture — including our data model which is most of the app, and then the views.</a></li><li><a href="https://medium.com/goodones/how-ollies-ai-system-works-ac7d8838df11">AI model — this is a bit of the secret sauce so we’ll keep it closer to the vest, but we’ll share our basic insights. It turns out it’s quite difficult to figure out which photos people will like!</a> (This Article)</li><li><a href="https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c">The async/await quagmire — what issues we ran into with SwiftUI &amp; Concurrency, and why we decided to move on to UIKit &amp; Dispatch.</a></li><li><a href="https://medium.com/goodones/pareto-optimal-apple-devtools-b47e79b6676f">Imagining a world where Apple made some more pragmatic decisions for their new dev platforms vs. the current ones they made today.</a></li><li><a href="https://medium.com/goodones/how-swiftui-concurrency-could-improve-16c13664e697">How Apple can forward fix their issues with SwiftUI and Swift Concurrency.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac7d8838df11" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/how-ollies-ai-system-works-ac7d8838df11">How Ollie’s AI System Works</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Making Ollie]]></title>
            <link>https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/88f533e0b21d</guid>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[photography]]></category>
            <category><![CDATA[swift-concurrency]]></category>
            <category><![CDATA[swiftui]]></category>
            <category><![CDATA[ios]]></category>
            <dc:creator><![CDATA[Mahyar McDonald]]></dc:creator>
            <pubDate>Sat, 28 Oct 2023 00:18:00 GMT</pubDate>
            <atom:updated>2024-02-15T01:18:34.239Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Making Ollie: </strong>Creating a local iOS app in a time of React Native</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*anRsUJz0QeKpyH6WogwupA.png" /><figcaption>But what’s the 4th hidden face we are missing in this trilemma pyramid?</figcaption></figure><p>This is the first in a series of blog posts about how the Ollie iOS app is made, architected and what decisions we made along the way to create it. This post will start with the context of our app and why we decided to make a native iOS app first. Later posts will go into lessons learned and decisions we have made, including spicy ones about why we are deciding to move away from SwiftUI going forward.</p><p><a href="http://getollie.ai/">Ollie</a> (formerly GoodOnes), if you don’t know, is an app that helps you organize your photos with a self-updating local AI model, privately, without uploading your photos to our backend.</p><p>As you sort your photos into 3 piles of favorites, keep and trash, the AI learns what you like and updates itself on the device. Many of us joined because we all have a lot of photos to sort, not enough time to sort them and want to get the best photos from our phone. We believe “<a href="https://www.theverge.com/2013/11/5/5039216/everpix-life-and-death-inside-the-worlds-best-photo-startup">the photo mess</a>” is still fundamentally unsolved.</p><h3>Why We Chose To Go Native</h3><p>When we started about a year ago, going native was a controversial choice. Most people expected us to go in the React Native or Flutter direction, for speed of development, ease of hiring and cross-platform app delivery.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pINEamDprenkIzdtgecF3Q.png" /><figcaption>The typical startup react native ‘shopping’ app.</figcaption></figure><p>However, from a product perspective, we knew we had to solve for these trio of constraints:</p><ul><li><strong>Speed:</strong> We wanted an app that ran locally on the users phone, integrated very well with their local native photo library and performed quickly, because people have zero patience for waiting or inconvenience. Needing to upload large photo files on a slow mobile connection kills this, not to mention issues that come from using a cross platform library. Deep PhotoKit, CoreML &amp; Vision integration is not a strong suit of cross platform toolkits without diving into native code anyway.</li><li><strong>Privacy:</strong> We didn’t want to rely on a backend because photos are very personal and private. Most people are not comfortable uploading their photos to an unknown service, especially an AI one. On top of this, this is something we personally believe in as a company.</li><li><strong>Cost:</strong> Photo storage and running AI training &amp; inference is a very expensive business to be in, and if we relied on uploading photos our costs as a small startup would balloon considerably.</li></ul><p>At Ollie, we always try to apply the combination of first-principles thinking with <a href="https://fs.blog/chestertons-fence/">Chesterson’s fence</a>. We evaluate every decision by its merits, but if we’re going against the standard practice, we want to understand why others went with that option and what makes us different.</p><p>Where we landed is that most non-game apps for people’s mobile devices are what we call ‘shopping apps’. They are basic client-server applications that don’t maintain much state on the client and rely on backends as the ultimate source of truth.</p><p>As a result, their client apps are not data model heavy and could be replaced with a well made web app for the most part. Even Amazon’s mobile app is mostly webviews under the hood. These apps tend to not stress the phone resources and it is fairly well understood how to make them. If you are a startup making a shopping or “<a href="https://www.coinbase.com/blog/announcing-coinbases-successful-transition-to-react-native">banking</a>” app using a cross platform toolkit like Flutter, React Native or being web only is an understandable choice.</p><p>For us though, because of the constraint trio above, <strong>we stress all subsystems of the phone</strong>, with many threads leveraging the GPU, Neural Engine, CPU, network, database and storage systems simultaneously, since there is no server to offload this to. Because we make a high load app, it ends up being a stress test on new multithreading &amp; UI libraries as a result, which means we need to be more conscious on how we manage threads than the typical client-server mobile app. We also need to do it quickly to not eat the user’s battery life.</p><p><em>TL;DR: Ollie more resembles a local desktop app than a server client app as a result, which makes data state screw ups and phone stress far more significant. Other startup founders are often surprised we didn’t use something like react native to make our app, but now it should be obvious why we chose this route.</em></p><h3>App Stack: Standard choices</h3><p>Let’s start with the basics — those are pretty standard choices for a native app these days:</p><ul><li>Pure SwiftUI app with a few exceptions around playing transparent videos.</li><li>Swift Concurrency as the default multithreading model, though this will get a big caveat in a second.</li><li>MVVM: Model, View, ViewModel</li><li>GRDB for database ops and state with a repository pattern</li><li>The standard SaaS backend support: Datadog for debug logs, Mixpanel for analytics, Sentry for error handling, PagerDuty for alerting, Linear for bug tracking, LaunchDarkly for feature flags. We’re thankful we live in the age these all come out-of-box 🙂</li></ul><p>Here’s a couple of places where we somewhat deviated from the standard:</p><ul><li><strong>Dependency Injection:</strong> We created our own basic dependency injection system that is like @Environment but is checked at compile time vs. run time with our own @Inject property wrapper. In exchange for compile time checking and simplicity of implementation, we gave up view tree scoping which we feel was a good trade off at our current app size. In practice, most environment objects live at the top of the app view tree anyway.</li><li><strong>Performance Infra:</strong> such as a leak detector, performance span monitoring, AI performance monitoring and so on. Sentry provides some of this, but we wanted something simple to start.</li><li><strong>Bug Reporting:</strong> we used Instabug initially, but it had a lot of weird limits and a surprising cost, to the point where we replaced it with our own code in the end and get bug reports via email + attachments now. This also allowed us to copy thread state for user reports.</li></ul><h3>App Stack: Where it got gnarly</h3><p>We wanted folks to be able to submit their best photos to Google Photos, so we added Google Photos support with the the <a href="https://github.com/deivitaka/GPhotos">GPhotos</a> library. We even hired the library maintainer for a while for some improvements in the library and are planning to submit the improvements back to open source. Gphotos support although proved to be a big source of complexity in our app that consumed a lot of engineering time in the end.</p><p>We use pretty much all the bells and whistles of Apple’s advanced libraries</p><ul><li>CoreML for running inference and model updates (more on this in a future spot)</li><li>Apple’s <a href="https://developer.apple.com/documentation/vision/">Vision</a> framework for feature extraction</li><li>PhotoKit to access the user’s photo library and modifying local albums.</li></ul><p>All of the above combined with Swift Concurrency really tested the limits of what is possible to keep stable within a SwiftUI app.</p><p>Like many, we tried Combine due to it being the major interface for state with SwiftUI and after using it a lot, we realized it wasn’t a good idea with it’s many footguns and thus moved away from using it.</p><h3>Looking Back: Are we happy that we went native?</h3><p>When we were just getting started, we got quite a few incredulous looks when we decided to go native. When we were hiring, it was 10x harder to find great iOS developers, and so it was tempting to go with a cross-platform framework. When people ask us “when will you be on Android”, our answer is 🤷.</p><p>And yet, every single roadblock we hit would have been worse if we stayed on a platform like react native or flutter, not to mention that we would not be able to achieve our “holy trio” without going native. At the end of the day, we would have had to do a full-rewrite pretty quickly, and either re-skill or rehire, neither of which is a great prospect for a pre product / market-fit company.</p><p>So the purpose of this first article was to essentially advocate for native development, for all the woes that come with it. If one developer or would be startup reads it and it saves them the agony we went through, it will have been worth it 🙂</p><p>This is part of a series of blog posts where we go into:</p><ul><li><a href="https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d">Why we decided to make a native iOS app vs React Native and an a outline of our stack.</a> (This Article)</li><li><a href="https://medium.com/goodones/ollies-app-architecture-continued-2b572e23c204">Baseline architecture — including our data model which is most of the app, and then the views.</a></li><li><a href="https://medium.com/goodones/how-ollies-ai-system-works-ac7d8838df11">AI model — this is a bit of the secret sauce so we’ll keep it closer to the vest, but we’ll share our basic insights. It turns out it’s quite difficult to figure out which photos people will like!</a></li><li><a href="https://medium.com/goodones/why-ollie-is-moving-away-from-swiftui-to-uikit-cfdefe918d1c">The async/await quagmire — what issues we ran into with SwiftUI &amp; Concurrency, and why we decided to move on to UIKit &amp; Dispatch.</a></li><li><a href="https://medium.com/goodones/pareto-optimal-apple-devtools-b47e79b6676f">Imagining a world where Apple made some more pragmatic decisions for their new dev platforms vs. the current ones they made today.</a></li><li><a href="https://medium.com/goodones/how-swiftui-concurrency-could-improve-16c13664e697">How Apple can forward fix their issues with SwiftUI and Swift Concurrency.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=88f533e0b21d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/raising-ollie-how-we-built-a-fully-local-multithreaded-ai-app-to-solve-your-photo-mess-88f533e0b21d">Making Ollie</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting Organized with Ollie]]></title>
            <link>https://medium.com/goodones/getting-organized-with-ollie-reimagining-a-clean-camera-roll-eca3521d374e?source=rss----66cd6b56a989---4</link>
            <guid isPermaLink="false">https://medium.com/p/eca3521d374e</guid>
            <dc:creator><![CDATA[Israel (Iz) Shalom]]></dc:creator>
            <pubDate>Thu, 26 Oct 2023 16:29:48 GMT</pubDate>
            <atom:updated>2023-10-30T22:15:05.045Z</atom:updated>
            <content:encoded><![CDATA[<h3>Getting Organized with Ollie: Reimagining a Clean Camera Roll</h3><p>When we started six months ago, our goal was to help parents, avid travelers, pet owners and everyday phone users organize their images and make sorting through their camera roll a more enjoyable experience.</p><p>Our users have collectively sorted through hundreds of millions of photos, and we’ve learned some interesting insights along the way. The average user has more than 20,000 photos in their phone’s camera roll (equivalent to 100 GB). About half of it is trash. And only a small fraction of people meaningfully use Apple’s “Favorites” function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/0*9rL7NavlMFXeLEyZ.png" /><figcaption>Say hello to Ollie!</figcaption></figure><p>Photo clutter can be so paralyzing for some that instead of cleaning out their unnecessary photos and videos, they just buy more storage. This can include either a $100-$300 memory upgrade for a new phone, or monthly payments for added iCloud storage to lug around unwanted images, or both.</p><p>However, adding more storage isn’t getting to the root of the problem. In fact, this only allows more room for clutter. If we’re constantly accumulating clutter in our camera rolls, we’re inevitably going to bury our favorite memories further and lose the photos that <em>actually</em> matter to us. It’s no wonder that less than 1% of mobile photos are shared with others.</p><p>While our users were uncovering memories and building manageable camera rolls, your feedback painted a clear picture: you wanted to spend less time organizing photos and more time enjoying memories.</p><p>In the last year, with the recent advances in AI, we also saw a big shift in user sentiment — and an increasing amount of people leverage AI to help get work done for them. Luckily, we were in a great position to build exactly this, so we used our learnings to implement the world’s first AI photo assistant.</p><p>Enter our new AI-centric app: Ollie, which is available for free download in the Apple App Store today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*_173nkInSHouDo88.gif" /></figure><p>We’re thrilled to continue our journey while building on the stepping stones the GoodOnes app provided. Before, you had to manually swipe to choose your best and worst photos. Now, Ollie uses AI to automatically find your best photos for you, and all you have to do (if you like) is adjust these. And every time you Ollie’s choices — Ollie learns your preferences, taking you one step further to a perfect camera roll!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/0*09tswJWCUb7UIbUj.gif" /></figure><p>We’ve been hard at work doing this, while keeping the same serverless and private infrastructure our users love. The entire app runs on-device (which means you can declutter on the way back from your vacation), and we never see your photos.</p><p>Ollie’s new learning process uses transfer learning, CoreML and the Apple Neural Engine Accelerator to predict which photos users will want to favorite, keep and discard (separate engineering blog post to come). In the last few weeks we’ve been experimenting with this, we find that our system can predict the user’s choices with 90% accuracy, and make sorting your photos 10x faster.</p><p>Ollie makes it faster and easier than ever to share photos with friends and family, document your favorite life events, share memories via social media or even scroll through old photos for a trip down memory lane.</p><p><a href="https://apps.apple.com/us/app/goodones-photo-organizer-app/id1551399205">Download</a> the new Ollie app now, and experience the reimagined way to organize your camera roll. We can’t wait to hear your feedback!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eca3521d374e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/goodones/getting-organized-with-ollie-reimagining-a-clean-camera-roll-eca3521d374e">Getting Organized with Ollie</a> was originally published in <a href="https://medium.com/goodones">Ollie</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>