<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Danning Ge on Medium]]></title>
        <description><![CDATA[Stories by Danning Ge on Medium]]></description>
        <link>https://medium.com/@danning.ge?source=rss-d6d650ea37b1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*c2a5-gbZJkC0a3zs.</url>
            <title>Stories by Danning Ge on Medium</title>
            <link>https://medium.com/@danning.ge?source=rss-d6d650ea37b1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 18:42:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@danning.ge/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building an At-Mentions Framework]]></title>
            <link>https://medium.com/frame-io-engineering/building-an-at-mentions-framework-33ca79bc4740?source=rss-d6d650ea37b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/33ca79bc4740</guid>
            <category><![CDATA[swift-programming]]></category>
            <category><![CDATA[interceptors]]></category>
            <category><![CDATA[swift]]></category>
            <category><![CDATA[swift-framework]]></category>
            <dc:creator><![CDATA[Danning Ge]]></dc:creator>
            <pubDate>Tue, 07 Jul 2020 21:25:44 GMT</pubDate>
            <atom:updated>2020-07-07T21:36:42.644Z</atom:updated>
            <content:encoded><![CDATA[<p>At-mentions are an easy way to draw someone’s attention to a high-priority event like a comment or message. The standard way to mention someone is to type “@” followed by the recipient’s name in the message and then select the recipient from a list of results. At-mentions are ubiquitous in apps that have a social element because they facilitate communication and increase user engagement. Last year we released at-mentions in both our iOS and macOS apps.</p><p>While there are many third-party libraries that enable the feature, we decided to implement our own at-mentions framework. We needed our framework to be compatible with both UIKit and AppKit so we could have either created two custom text fields or extended the existing functionality of the UIText fieldDelegate and NSTextFieldDelegate. We chose the second strategy because it centralized the at-mentions logic for both platforms while maintaining the existing text field delegation flow. This article highlights the key components and lessons learned.</p><p>This article will cover the main components of the framework:</p><ol><li>A trigger reporter that processes the current string in the text view</li><li>Intercepting UITextViewDelegate and NSTextViewDelegate</li><li>A mentions manager that processes tokenized mentions</li><li>A coordinator that ties together the components and reports updates to the client</li></ol><p><strong>Creating a trigger reporter</strong></p><p>We called the object used for processing the current string a trigger reporter. A trigger can be any string symbol — in our case, it is the “@” symbol. When a user types “@” in the app the trigger reporter will notify its delegate. The trigger reporter checks for trigger matches in the text input and reports the result to its delegate.</p><p>The interface for the trigger reporter is shown below. An in-depth discussion on the regex pattern matching used by the reporter will be covered in a separate article — stay tuned.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/01dd2435b780783123be0b02a0651ec5/href">https://medium.com/media/01dd2435b780783123be0b02a0651ec5/href</a></iframe><p><strong>Intercepting the delegate of the text view</strong></p><p>We know how to check for the at-mention trigger but also need to figure out <em>when</em> to check for the trigger in real-time. Leveraging the system text view delegates seems fitting since they are designed to report text view events; however, we cannot just conform to the text view delegate because the clients are already using it to handle other comment functionalities. What we need then is a way to keep the existing delegate but also respond to text field events for the new at-mentions feature.</p><p>There is an established way of solving this type of problem: the interceptor pattern. The pattern is used to augment a part of a framework without altering current functionality. In our case, we want to expand the functionality of the text view delegate callbacks while keeping the client side’s use of the text view delegate unchanged. The diagram below illustrates the concept.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/1*F05TlXmpUZJlVuj_R4jcxg.png" /></figure><p>Figure 1.0 <em>The Interceptor Pattern</em></p><p>To use the interceptor pattern, we first define what the interceptable interface is.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c11b53034ba160a789bf4653489915ea/href">https://medium.com/media/c11b53034ba160a789bf4653489915ea/href</a></iframe><ol><li>We define an Interceptable protocol that exposes an interceptableDelegate.</li><li>Next, we define a MentionTextInput protocol that provides two callbacks to update the display text with attributed text after a mention is selected (see the video example below).</li><li>Because we have both iOS and macOS clients, NSTextField and UITextField conform to MentionTextInput. For brevity purposes, we only show the iOS extensions.</li></ol><p>Notice that the interceptableDelegate still conforms to UITextViewDelegate. The idea is that when the client creates a text view and sets the delegate of the text view (the delegate is called passThroughDelegate and the text view is called input), it only needs to conform to the UITextViewDelegate while allowing our framework to also respond to the delegate methods.</p><p>Our interceptor class is called MentionInterceptor and is responsible for forwarding the text view’s events to the existing text view delegate (set in the client and called passThroughDelegate here) and intercepting those same events.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0addefb5e5eeb10dd5c6d77c812c644/href">https://medium.com/media/d0addefb5e5eeb10dd5c6d77c812c644/href</a></iframe><ol><li>The mention interceptor is instantiated with an interceptable input, which is either an NSTextView or UITextView object. It then holds on to the original delegate or passThroughDelegate and text view or currentInterceptedInput.</li><li>We only care about delegate callbacks that either the passthrough delegate or mention interceptor responds to so we override responds(to:)</li><li>Any events not handled by the mention interceptor are forwarded to the text view delegate.</li></ol><p>Finally, the mention interceptor implements text view callbacks that it cares about and conforms to the text view delegate. As mentioned earlier, we need our framework to work with both macOS and iOS targets so the MentionInterceptor needs to conform to both NSTextViewDelegate and UITextViewDelegate.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/03cac00628d14fd42d6266ac05e9817e/href">https://medium.com/media/03cac00628d14fd42d6266ac05e9817e/href</a></iframe><ol><li>The interceptor defers to the passThroughDelegate to see if the text should be replaced.</li><li>The text view did change event is first passed to the passThroughDelegate to be handled before the interceptor passes it to its own delegate.’</li></ol><p>It is interesting to note that in both of the callbacks that the mention interceptor intercepts, it also chooses to forward the events to the passThroughDelegate. We could just have easily blocked the event from reaching the passThroughDelegate. The interceptor pattern is a powerful tool that allows us to augment functionality, in this case, the text view delegation, in a transparent manner without requiring actual changes to the design or existing implementation.</p><p><strong>Implementing storage for mentions</strong></p><p>The video below illustrates the user at-mentions flow in our iOS app. When the user types in “@”, the interceptor in the mentions framework receives the trigger, processes the text, and delegates back to the app with the search text. The app then uses the search text and presents a table of mentionable results for the user to choose from.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*ASFF46W__9dSDljt.gif" /></figure><p>We need a way to temporarily store the mentions selected by the user so we create a mentions maintainer that is responsible for keeping track of added and removed mentions. It is also in charge of updating the mentions’ indices as the user edits the comment: if a user updates the text upstream of the existing mentions, the indices of the mentions are shifted accordingly while downstream changes are ignored. Below we define the interface for the mentions maintainer.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0addefb5e5eeb10dd5c6d77c812c644/href">https://medium.com/media/d0addefb5e5eeb10dd5c6d77c812c644/href</a></iframe><p>A mention is required to have an itemID, start and end indices, and the corresponding text. The itemID is specific to our needs but there needs to be something that indicates where the mention is located in the text — we use start and end indices.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0addefb5e5eeb10dd5c6d77c812c644/href">https://medium.com/media/d0addefb5e5eeb10dd5c6d77c812c644/href</a></iframe><ol><li>In the mention maintainer, we keep a private reference to a mentions map that has the start index of each mention as the key. This makes it easy for us to find the mentions that need to be shifted when the text is updated.</li><li>We create an extension on Mention to return an updated mention shifted by the offset. We use this new start index and update the mentions, which are structs, map accordingly.</li></ol><p><strong>Putting it together</strong></p><p>The final component is the mention coordinator which facilitates communication among the mention text input, the trigger reporter, and the mention maintainer. The mention coordinator is injected with the three components and sets itself as their delegates. It reports events to its own delegate, which is the Swift client.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0addefb5e5eeb10dd5c6d77c812c644/href">https://medium.com/media/d0addefb5e5eeb10dd5c6d77c812c644/href</a></iframe><ol><li>We define the mention coordinator delegate that exposes two methods: both the iOS and macOS clients need to know when to make API requests using the parsed search string. The second method triggers keystroke events for macOS.</li><li>When the trigger reporter detects an @-mention search, the mention coordinator is notified and it, in turn, notifies its delegate, our Swift client, of the search string. Our client then makes a backend request for users matching the search string and displays the results. When the user ends a mention search, the mention coordinator similarly notifies the client.</li></ol><p>The diagram below illustrates the flow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/0*VivAS3vHmuGVThxH" /></figure><p>Implementing at-mentions required multiple independent components each with its own responsibility. Investing in planning and designing the components paid off for us and enabled us to release the feature with minimal maintenance and stability. Because the at-mentions business logic is contained in a separate framework, it was easy to write unit tests — we are very proud to have <em>100 percent code coverage</em> in the framework!</p><p>Our standalone at-mentions framework is also flexible enough to accommodate future iterations that require additional trigger events like hashtags. Finally, it was a great learning experience using the interceptor pattern in our framework.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33ca79bc4740" width="1" height="1" alt=""><hr><p><a href="https://medium.com/frame-io-engineering/building-an-at-mentions-framework-33ca79bc4740">Building an At-Mentions Framework</a> was originally published in <a href="https://medium.com/frame-io-engineering">Frame.io Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Huge Images, Small Phone]]></title>
            <link>https://medium.com/frame-io-engineering/huge-images-small-phone-15414b30d39e?source=rss-d6d650ea37b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/15414b30d39e</guid>
            <category><![CDATA[swift-4]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[swift-programming]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[swift]]></category>
            <dc:creator><![CDATA[Danning Ge]]></dc:creator>
            <pubDate>Tue, 27 Nov 2018 17:21:14 GMT</pubDate>
            <atom:updated>2018-11-27T17:21:14.505Z</atom:updated>
            <content:encoded><![CDATA[<h4>Efficient Large Image Loading on iOS</h4><p>Scroll views with images are featured ubiquitously in iOS apps. Images help engage users but showcasing them on a small device such as the iPhone comes with performance challenges. Directly loading large images incurs a high memory cost and limits performance. It would be great if we could conserve memory by showing only areas of the image the user has zoomed into; and if zoomed out, show a lower resolution version of the image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/230/0*9tRP9loJmhW1RVLB" /><figcaption>Image credit: NASA/JPL-Caltech</figcaption></figure><p>As it turns out, the Core Animation framework offers a built-in solution called <a href="https://developer.apple.com/documentation/quartzcore/catiledlayer">CATiledLayer</a>. Showing tiles of an image in higher resolution rather than the entire image means that only parts of it are loaded into memory at once, resulting in faster load time and lower memory usage. To see tiles in action, zoom in on an area in the Maps app and see the zoomed-in area sharpen after a short wait. This post will show how to load large images efficiently using tiles.</p><p>This article will cover:</p><p>1) Implementing a scroll view backed by a CATiledLayer.</p><p>2) Cutting and caching image tiles using CATiledLayer, introduced In Apple’s <a href="https://developer.apple.com/videos/archive/">WWDC 2010 video</a> (<em>Designing Apps with Scroll Views</em>).</p><p><strong><em>Implementing a scroll view backed by a CATiledLayer</em></strong></p><p>To implement a scroll view that uses CATiledLayer, we must first create a custom tile view. This view will be responsible for requesting the parts of the image that we need to show and drawing them. You can read more about the properties of CATiledLayer <a href="https://developer.apple.com/documentation/quartzcore/catiledlayer">here</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/84aa25a56d332598cccd8a9db627c822/href">https://medium.com/media/84aa25a56d332598cccd8a9db627c822/href</a></iframe><ol><li>First, we implement a custom view and override its layer class to use CATiledLayer.</li><li>Content scale factor defines the relationship between points and pixels (1:1 for non-retina displays, 1:2 for retina). We always return a content scale factor of 1 since we will be drawing images directly.</li><li>We inject a tileManager into the tiling view. This tileManager will be responsible for providing the image sections that the tiling view requests. We will cover its core functionality in the next section.</li><li>We override drawRect(_rect: CGRect) which will be called by the scroll view when the user zooms in on an image. When drawRect is called, we can access its rect property to determine the tiles we need to supply.</li></ol><p>Next, we implement drawRect(_rect: CGRect):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9e60888b17e75919d5e630c75df21dcd/href">https://medium.com/media/9e60888b17e75919d5e630c75df21dcd/href</a></iframe><p>There’s a lot of math here but essentially we calculate the frames of the tiles we need and ask the tile manager for the part of the image that maps to the tile. We then draw each tile using its respective image. This code is adapted from the WWDC tutorial on CATiledLayer (<a href="https://developer.apple.com/videos/archive/">WWDC 2010 video</a> <em>Designing Apps with Scroll Views</em>).</p><ol><li>We get the current scale of the view and apply that same scale to the tile size width and height.</li><li>Calculate the number of columns and rows we will need to draw in the view. We ask the tile manager for the image tile that corresponds to the tile scale and dimensions.</li><li>We draw the image tile in the rect.</li></ol><p>The final step is to add an instance of TilingView to a custom scroll view. We’ll also need to set the maximum and minimum zoom scales of the scroll view depending on the image’s aspect ratio.</p><p><strong><em>Cutting and Caching Image Tiles</em></strong></p><p>At Frame.io, our API returns both high and low-resolution images. We use a download task to download the resource from the high-resolution URL. If successful, we move the downloaded resource to a path (called highResolutionImagePathURL below) in the local cache directory using the file manager. RayWenderlich.com has a <a href="https://www.raywenderlich.com/567-urlsession-tutorial-getting-started">great tutorial</a> on download tasks and local caching.</p><p>In order to cut the image tiles we need, we first need to efficiently access the high-resolution image (remember calling UIImage(named:) will load it directly into memory).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/53b1b23b05e65cef409d7143ba93f20f/href">https://medium.com/media/53b1b23b05e65cef409d7143ba93f20f/href</a></iframe><p>Now that we have a file location path reference, we can access it using CGDataProvider and CGImage (using jpegDataProviderSource). This will return a CGImage.</p><p>Next, we use the tile manager to cut and cache the image tiles:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b041939c7de3bf58714e34b1bbc58ce6/href">https://medium.com/media/b041939c7de3bf58714e34b1bbc58ce6/href</a></iframe><ol><li>First, we need to figure out the corresponding rect on the image. The mappedImage represents the image we want to display depending on the zoom scale. For example, we may choose to only show the high-resolution image if the resolution is above 4000. Otherwise, we show a smaller image.</li><li>We crop the image for the tile rect and use FileManager to save the tile. Remember to clear the cache when dismissing the scroll view.</li></ol><p>We recently launched our Image Review feature at Frame.io that utilizes CATiledLayer to enable zooming, panning, and annotating on high-resolution images on the iPhone. We found that CATiledLayer solves the performance problems that result from loading high-resolution images by conserving memory usage without limiting user interaction. Below is a side-by-side memory usage comparison of loading images with and without CATiledLayer (“Memory Report” is accessible from the debug navigator):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FDuYoV7HayGbly67" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*87VbFx582oNEQAyQ" /></figure><p>The non-tile image loader used almost 8x as much memory as the tile image loader! The difference may not be noticeable on an iPhone simulator on the laptop, but it is significant on a small device like the iPhone and a high memory pressure will likely crash the app or hinder performance. So the next time you need to load large images, consider using CATiledLayer.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15414b30d39e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/frame-io-engineering/huge-images-small-phone-15414b30d39e">Huge Images, Small Phone</a> was originally published in <a href="https://medium.com/frame-io-engineering">Frame.io Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>