<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ertem Biyik on Medium]]></title>
        <description><![CDATA[Stories by Ertem Biyik on Medium]]></description>
        <link>https://medium.com/@ertembiyik?source=rss-d1b6967a471a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*CYh3KlZuzjVP3fSZIfn3JQ.png</url>
            <title>Stories by Ertem Biyik on Medium</title>
            <link>https://medium.com/@ertembiyik?source=rss-d1b6967a471a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 18:02:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ertembiyik/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Built a Tool That Lets You Point at iOS Views and Tell AI What to Change]]></title>
            <link>https://medium.com/@ertembiyik/i-built-a-tool-that-lets-you-point-at-ios-views-and-tell-ai-what-to-change-ebfea007fa37?source=rss-d1b6967a471a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ebfea007fa37</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[swift]]></category>
            <dc:creator><![CDATA[Ertem Biyik]]></dc:creator>
            <pubDate>Fri, 06 Feb 2026 06:35:25 GMT</pubDate>
            <atom:updated>2026-02-06T06:35:25.639Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Abv2uTaQmS1LdsjOosiwUg.gif" /><figcaption>swift-agentation demo</figcaption></figure><blockquote>We’ve all been there. You’re looking at a button in your app that needs to be bigger, or a label that’s misaligned, or a screen that’s missing a loading state. You open Cursor or Claude Code, and now you have to somehow *describe* which button you mean. You screenshot it, paste it in, say “the button in the middle of the screen, the blue one, no not that one” and pray the agent figures it out.</blockquote><p>I built <a href="https://github.com/ertembiyik/swift-agentation">swift-agentation</a> to fix this. You tap on elements in your running iOS app, leave notes like “make this larger” or “add a loading state here,” and copy structured output that AI coding agents can actually use. The output includes view types, accessibility labels, frame coordinates, and the full hierarchy path — everything an agent needs to locate and modify the exact right view.</p><p>But this post isn’t about <strong>what</strong> it does. It’s about <strong>how</strong> it works under the hood, and the problems I ran into making it work.</p><h3>The Two Ways to See Your App</h3><p>The first design decision was: how do you actually inspect what’s on screen? iOS gives you two fundamentally different lenses into the view hierarchy, and they reveal surprisingly different things.</p><p><strong><em>The accessibility</em></strong><em> tree</em> is what VoiceOver sees. It’s semantic — elements are classified as buttons, links, headers, images. It respects the accessibility contract the developer set up, which means it often gives you the *meaningful* elements rather than every implementation detail. When you tap a “Submit” button, the accessibility tree knows it’s a button labeled “Submit.”</p><p><strong><em>The view hierarchy</em></strong> is the raw UIView tree. It’s structural — you see every container, every layout helper, every invisible spacer. It gives you the actual type names (`UIButton`, `UILabel`, `_UIHostingView`) and the exact nesting.</p><p>Both are useful for different reasons. The accessibility tree is great out of the box — most apps have reasonable accessibility setups, so you immediately get semantic meaning. The view hierarchy gives you full coverage, including elements that might not be exposed to VoiceOver. I made it so you can switch between them mid-session.</p><p>The interesting part is how each one is traversed.</p><h3>Walking the Accessibility Tree</h3><p>Apple’s accessibility API is designed for assistive technologies, not inspection tools. There’s no “give me all accessible elements” method. You have to recurse through the tree yourself.</p><p>The logic starts from every window in the app and recursively collects elements. For each element, we check three different sources of children, in order:</p><p>1. Custom `accessibilityElements` array (if the developer set one up)<br>2. `accessibilityElementCount()` with indexed access (for custom containers)<br>3. UIView subviews as fallback</p><pre>private func collectAccessibleElements(<br>    from element: NSObject,<br>    path: [String],<br>    depth: Int<br>) {<br>    guard depth &lt; 50 else { return }<br>    <br>    if element.isAccessibilityElement {<br>        let frame = element.accessibilityFrame<br>        let label = element.accessibilityLabel<br>        let traits = element.accessibilityTraits<br>        // … store element<br>    }<br>    if let customElements = element.accessibilityElements as? [NSObject] {<br>        for child in customElements {<br>            collectAccessibleElements(from: child, path: updatedPath, depth: depth + 1)<br>        }<br>    } else if element.accessibilityElementCount() != NSNotFound<br>                &amp;&amp; element.accessibilityElementCount() &gt; 0 {<br>        for i in 0..&lt;element.accessibilityElementCount() {<br>            if let child = element.accessibilityElement(at: i) as? NSObject {<br>                collectAccessibleElements(from: child, path: updatedPath, depth: depth + 1)<br>            }<br>        }<br>    } else if let view = element as? UIView {<br>        for subview in view.subviews {<br>            collectAccessibleElements(from: subview, path: updatedPath, depth: depth + 1)<br>        }<br>    }<br>}</pre><p>The 50-level depth limit is a safety net. I’ve seen accessibility trees where custom containers reference each other in unexpected ways. Without the limit, you’d spin forever.</p><p>Traits get mapped to roles — `UIAccessibilityTraits.button` becomes “Button”, `.link` becomes “Link”, `.header` becomes “Header”. This semantic information is exactly what makes the accessibility data source so useful for AI agents: they don’t just get “a view at coordinates (100, 200),” they get “a Button labeled Submit.”</p><h3>Walking the View Hierarchy</h3><p>The view hierarchy traversal is more straightforward in concept — you just recurse through subviews — but it surfaces a problem that doesn’t exist with the accessibility tree.</p><p>The traversal starts from the app’s windows, filtering out our own overlay window and system windows. For each view, we record the class name, accessibility identifier, accessibility label, and frame.</p><p>This works great for UIKit. But then you hit SwiftUI.</p><h3>The SwiftUI Problem</h3><p>This is where things get non-obvious. When you write a SwiftUI view like:</p><pre>Button(&quot;Confirm&quot;) {<br>    viewModel.confirm()<br>}</pre><p>…what actually ends up in the UIView tree? Not a `UIButton`. SwiftUI doesn’t create a UIButton for this. Depending on the context, it might render through CoreAnimation layers, or wrap it in a generic hosting container, or — in some cases — not create a dedicated UIView at all.</p><p>When our view hierarchy traversal encounters this, it sees something like a `_UIHostingView` containing a `SwiftUI.AccessibilityNode`, but there’s no `UIButton` and no view labeled “Confirm” in the UIView tree. The button just doesn’t exist as a view.</p><p>The accessibility tree handles this fine because SwiftUI registers accessibility metadata regardless of how it renders. But the view hierarchy data source? It’s blind.</p><h4>Fixing It With `agentationTag()`</h4><p>The solution was to introduce a ViewModifier that bridges SwiftUI’s world into the UIView world through geometry:</p><pre>public struct AgentationTagModifier: ViewModifier {<br>    <br>    let tag: String<br>    <br>    public func body(content: Content) -&gt; some View {<br>        content<br>            .onGeometryChange(for: CGRect.self) { proxy in<br>                proxy.frame(in: .global)<br>            } action: { frame in<br>                AgentationTagRegistry.shared.set(tag: tag, frame: frame)<br>            }<br>            .onDisappear {<br>                AgentationTagRegistry.shared.remove(tag: tag)<br>            }<br>    }<br>    <br>}</pre><p>The idea: since we can’t find the SwiftUI view in the UIView tree by type, we match it by *position*. The modifier tracks the view’s frame in global coordinates and registers it in a singleton. When the view hierarchy data source encounters a UIView, it checks the registry: “is there a tag registered at approximately this position?”</p><pre>func tag(for frame: CGRect, tolerance: CGFloat = 2) -&gt; String? {<br>    for (tag, tagFrame) in entries {<br>        guard abs(tagFrame.origin.x - frame.origin.x) &lt; tolerance,<br>              abs(tagFrame.origin.y - frame.origin.y) &lt; tolerance,<br>              abs(tagFrame.width - frame.width) &lt; tolerance,<br>              abs(tagFrame.height - frame.height) &lt; tolerance<br>        else {<br>            continue<br>        }<br>        return tag<br>    }<br>    return nil<br>}</pre><p>Usage is dead simple:</p><pre>Button(&quot;Confirm&quot;) {<br>     viewModel.confirm()<br>}<br>.agentationTag(&quot;confirmButton&quot;)</pre><p>Now when the hierarchy data source encounters whatever UIView backs this button, it finds the “confirmButton” tag and uses it in the output path. Instead of `ViewController &gt; UIView &gt; _UIHostingView`, you get `ViewController &gt; [confirmButton]`. An AI agent can actually search for `agentationTag(“confirmButton”)` in the codebase and find the exact view.</p><h3>The Overlay: Invisible Until It Isn’t</h3><p>The capture interface lives in its own `UIWindow` that sits above everything else. The tricky part: this window needs to be completely invisible to touch when you’re not using it, but capture all touches when you are.</p><p>The solution is a small pass-through system. There’s a `HitTestable` protocol:</p><pre>protocol HitTestable {<br>     func contains(_ point: CGPoint) -&gt; Bool<br>}</pre><p>The root SwiftUI view conforms to this and knows which regions are interactive (the toolbar, the element highlights, the feedback sheet). Then `PassThroughHostingView` — a subclass of `_UIHostingView` — overrides `point(inside:with:)` and delegates to the SwiftUI view:</p><pre>final class PassThroughHostingView&lt;Content: View &amp; HitTestable&gt;: _UIHostingView&lt;Content&gt; {<br>    <br>    override func point(inside point: CGPoint, with event: UIEvent?) -&gt; Bool {<br>         let globalPoint = convert(point, to: nil)<br>        <br>        guard rootView.contains(globalPoint) else {<br>             return false<br>        }<br>        <br>        return super.point(inside: point, with: event)<br>    }<br>    <br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*27RoMcmPumXws_AwNBViOw@2x.png" /></figure><p>When the SwiftUI view says “I don’t contain this point,” the hosting view reports it’s not inside — and the touch falls through to the app underneath. `PassThroughHostingViewController` ties it together by using this view as its `loadView`.</p><p>The overlay window’s own `hitTest` also returns nil for the window itself, so even the window background never intercepts touches. This means you can interact with your app normally while the collapsed toolbar floats in the corner.</p><h3>Hit Testing From Above</h3><p>When the user taps during a capture session, we need to figure out which element they meant. The tap hits our overlay, and we do a reverse lookup: find the smallest element whose frame contains the tap point.</p><p>“Smallest” is key. Multiple elements overlap — a button is inside a stack which is inside a view controller’s view. The user almost always means the leaf element. So we sort candidates by area and pick the smallest one containing the touch point.</p><p>The element frames were captured at snapshot time. We grab the hierarchy once when you start (or resume) a capture session, not continuously. This was a deliberate choice — continuous tracking would mean fighting against animations, scroll offsets, and layout changes. A snapshot gives you a stable set of elements to tap on. If the layout changes, you pause and resume to re-snapshot.</p><p>There is an experimental frame tracking mode that uses a `CADisplayLink` to update element positions every frame. It’s useful for elements that move (like items in a scroll view), but it’s off by default because the performance cost isn’t worth it for most use cases.</p><h3>What the Output Looks Like</h3><p>After tapping on elements and leaving feedback, you hit copy. The output is structured for AI agents to parse:</p><pre>**Viewport:** 390×844<br><br>## MainScreen<br><br>### 1. Button “Submit”<br>**Location:** MainViewController &gt; FormView &gt; [submitButton]<br>**Frame:** x:155 y:680 w:80 h:44<br>**Feedback:** Make this full-width and add a loading spinner state<br><br>### 2. Text “Welcome back”<br>**Location:** MainViewController &gt; HeaderView &gt; “Welcome back”<br>**Frame:** x:16 y:100 w:358 h:32<br>**Feedback:** Use the user’s first name here</pre><p>Or as JSON if you prefer:</p><pre>{<br> &quot;viewport&quot;: {<br>  &quot;width&quot;: 390,<br>  &quot;height&quot;: 844<br> },<br> &quot;screens&quot;: [{<br>  &quot;screen&quot;: &quot;MainScreen&quot;,<br>  &quot;items&quot;: [{<br>   &quot;type&quot;: &quot;button&quot;,<br>   &quot;displayName&quot;: &quot;Submit&quot;,<br>   &quot;path&quot;: &quot;MainViewController &gt; FormView &gt; [submitButton]&quot;,<br>   &quot;frame&quot;: {<br>    &quot;x&quot;: 155,<br>    &quot;y&quot;: 680,<br>    &quot;w&quot;: 80,<br>    &quot;h&quot;: 44<br>   },<br>   &quot;feedback&quot;: &quot;Make this full-width and add a loading spinner state&quot;<br>  }]<br> }]<br>}</pre><p>The path is the most important field. It gives the agent a breadcrumb trail to find the exact view in code — by searching for the accessibility identifier, the agentation tag, or the view class name.</p><h3>The Toolbar Itself</h3><p>The toolbar morphs between a collapsed circle and an expanded capsule. Both states use the same glass material (via <a href="https://github.com/nicklama/universal-glass">UniversalGlass</a>) and a matched geometry transition, so the animation feels like the circle is stretching open rather than being replaced.</p><p>When expanded, the toolbar is draggable. It snaps to the horizontal center and clamps to the safe area so it never floats off-screen. The drag has a 3-point minimum distance threshold to distinguish taps from drags.</p><p>One detail I like: when you’re not in a capture session but have feedback items from a previous session, the collapsed toolbar shows a badge with the count. Your annotations survive across sessions — you can stop, go do something else, come back, and your feedback is still there.</p><h3>Try It</h3><pre>// Package.swift<br>.package(url: “https://github.com/ertembiyik/swift-agentation&quot;, from: “0.1.0”)</pre><pre>// AppDelegate or App init<br>Agentation.shared.install()</pre><p>That’s it. A toolbar appears. Tap it, tap on UI elements, leave notes, copy. Paste the output into your AI coding agent. It knows exactly what you’re talking about.</p><p>iOS 17.0+, Swift 5.9+. Both SwiftUI and UIKit. The <a href="https://github.com/ertembiyik/swift-agentation">source is on GitHub</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ebfea007fa37" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create a beautiful Network Layer With Swift]]></title>
            <link>https://medium.com/@ertembiyik/create-a-beautiful-network-layer-with-swift-f10a06e2c4b5?source=rss-d1b6967a471a------2</link>
            <guid isPermaLink="false">https://medium.com/p/f10a06e2c4b5</guid>
            <category><![CDATA[network-layer]]></category>
            <category><![CDATA[swift]]></category>
            <category><![CDATA[networking-tips]]></category>
            <category><![CDATA[urlsessions]]></category>
            <category><![CDATA[networking]]></category>
            <dc:creator><![CDATA[Ertem Biyik]]></dc:creator>
            <pubDate>Tue, 31 May 2022 19:16:54 GMT</pubDate>
            <atom:updated>2022-05-31T19:16:54.468Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Barely all swift developers once upon a time face with a task of interacting with API’s. Have you ever wondered how to make it look and feel seamless?</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f9EaqC1CQHDt3H2qzCxW4A.png" /></figure><p>Recently I’ve been working on swift-ethereum project: <a href="https://github.com/sparrowcode/swift-ethereum.git"><em>https://github.com/sparrowcode/swift-ethereum</em></a> and wrote some cool stuff that I want to share with you guys. This article shows how to make a scalable, pretty Network Layer for your iOS app or swift package.</p><p>For this tutorial, I have chosen the Rick and Morty API (<a href="https://rickandmortyapi.com/">https://rickandmortyapi.com</a>). It has 3 main routes:</p><ul><li>Character — <a href="https://rickandmortyapi.com/documentation/character">https://rickandmortyapi.com/api/character</a></li><li>Location —<a href="https://rickandmortyapi.com/documentation/location"> https://rickandmortyapi.com/api/location</a></li><li>Episode — <a href="https://rickandmortyapi.com/documentation/episode">https://rickandmortyapi.com/api/episode</a></li></ul><p>So for tutorial purposes we will take the character’s route and play with it.</p><h3>Service</h3><p>Let’s create an enum with the baseURL inside it</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6b9e5d8e75538f46a4d3d5a435ce4045/href">https://medium.com/media/6b9e5d8e75538f46a4d3d5a435ce4045/href</a></iframe><h3>RoutePrototcol</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ccb8f7aadbf6759cf122e1bc461e99da/href">https://medium.com/media/ccb8f7aadbf6759cf122e1bc461e99da/href</a></iframe><p>A type that confirms to this protocol will give us the stringValue which we add to baseURL and get the API endpoint. The method is the httpMethod being used for this request. You can also add httpHeaders property</p><h3><strong>CharacterRoute</strong></h3><p>Let’s create an enum that confirms to the protocol above:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d6d7d7217e5343dc313583cac75deed/href">https://medium.com/media/4d6d7d7217e5343dc313583cac75deed/href</a></iframe><p>From each of the cases we can get the stringValue which should be added to baseURL and httpMethod needed for request.</p><p>The character struct that I need looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b6abee20a1b6ac6d7fc29f0bd06715a1/href">https://medium.com/media/b6abee20a1b6ac6d7fc29f0bd06715a1/href</a></iframe><p>The info struct:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f7b1103758e259bd526fa309ca60f79f/href">https://medium.com/media/f7b1103758e259bd526fa309ca60f79f/href</a></iframe><h3>NetworkManager</h3><p>This is where the magic happens. This class will manage encoding, sending, receiving and decoding data from/to server.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/128a1ed495af28b52d6a6d735d59a338/href">https://medium.com/media/128a1ed495af28b52d6a6d735d59a338/href</a></iframe><p>We provide an initialization for class with baseURL, a session for dataTask calls and a concurrent queue to make our manager thread safe. We also call finishTasksAndInvalidate() inside a deinit block to finish and stop all outgoing requests after an instance of our manager is being deinitialized.</p><p>Next let’s take a look at sendRequest method: it is a generic function, which generates a request based on given inputs, sends it and decodes a response. We have two of them, because the params field is generic and for some requests we don’t need any httpBody.</p><p>Now we can add an instance of NetworkManager to the Service.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/548ebce1018e5a9d8d7eb592ea6eb6b4/href">https://medium.com/media/548ebce1018e5a9d8d7eb592ea6eb6b4/href</a></iframe><p>We also need to implement methods inside CharacterRoute using manager:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7f11b68f7871ef68bfc8a6dee4469227/href">https://medium.com/media/7f11b68f7871ef68bfc8a6dee4469227/href</a></iframe><p>That’s it! Take a look, how beautiful and logical it looks when you use it:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7463f02107550204cd1d7d8020690729/href">https://medium.com/media/7463f02107550204cd1d7d8020690729/href</a></iframe><h3>So what are the beauties of this network layer?</h3><ul><li>Quick fixes if the API changed: if the URL of some endpoint or the params needed for request changes, you always know where and what to fix</li><li>Scalability: if a new endpoint is added just write new case to enum, the compiler will force you to write stringValue and method</li><li>Independence of NetworkManager: any params can be sanded/received due to its generic nature</li></ul><p>That’s it! Thank you for reading, have a nice one)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f10a06e2c4b5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>