Introduction

Recently, I came across Medidate – Mindfulness app. The intricate mandalas on the app looked like a work of art. This motivated me to dive into understanding how such complex structures can be created.

In this article, we’ll explore how to draw, customize, and animate shapes using SwiftUI and CoreAnimation. We’ll start by exploring the coordinate system, then create a flower petal, adjust its shape, assemble it into a flower, and finally animate it.

Understanding the Coordinate System for Paths

When drawing paths, understanding the coordinate system is critical. The origin (0,0) is at the top-left corner of the view, with the x-axis increasing to the right and the y-axis increasing downward. This system allows precise control over the placement of points and the creation of paths.

Creating a Custom Shape

To create custom shapes in SwiftUI, you can conform to the Shape protocol. This involves defining the path that outlines your shape. Here, we’ll draw a flower petal using the Shape protocol.

struct FlowerPetalView: Shape {
    func path(in rect: CGRect) -> Path {
        var path = Path()
        
        let width = rect.width
        let height = rect.height
        let controlHeight = height
        // 1. Start at the center of the shape
        path.move(to: CGPoint(x: width / 2, y: height / 2))
        
        // 2. Draw the top half of the petal using a quadratic curve
        path.addQuadCurve(
            to: CGPoint(x: 0, y: height / 2),
            control: CGPoint(x: width / 4, y: height / 2 - controlHeight)
        )
        
        // 3. Draw the bottom half of the petal using another quadratic curve
        path.addQuadCurve(
            to: CGPoint(x: width / 2, y: height / 2),
            control: CGPoint(x: width / 4, y: height / 2 + controlHeight)
        )

        return path
    }
}

Note - In this example, we use a rectangular frame for the petal.

In this code snippet:

1. We start by moving to the center of the rectangle.
2. We draw the first half of the petal with a quadratic curve, using a control point that extends upward.
3. We draw the second half of the petal with another quadratic curve, using a control point that extends downward.

Adjusting the Shape

Now that we’ve created a basic flower petal shape, we can further modify the petal by adjusting the control points. By changing the the control points, we can create variations in the petal’s appearance. For example, if the y-coordinate of the control point is closer to the baseline, the petal will appear skinnier. Some examples:

Now, we try to draw two petals in this rectangle. Another petal on the right side matching the one on the left:

struct FlowerPetalView: Shape {
    func path(in rect: CGRect) -> Path {
        var path = Path()
        
        let width = rect.width
        let height = rect.height
        let controlWidth = width / 4
        let controlHeight = height / 2.5
        
        // Move to the center of the rectangle
        path.move(to: CGPoint(x: width / 2, y: height / 2))
        
        // 1. Draw the left petal
        path.addQuadCurve(
            to: CGPoint(x: 0, y: height / 2),
            control: CGPoint(x: width / 2 - controlWidth, y: height / 2 - controlHeight)
        )
        path.addQuadCurve(
            to: CGPoint(x: width / 2, y: height / 2),
            control: CGPoint(x: width / 2 - controlWidth, y: height / 2 + controlHeight)
        )
        
        // 2. Draw the right petal
        path.addQuadCurve(
            to: CGPoint(x: width, y: height / 2),
            control: CGPoint(x: width / 2 + controlWidth, y: height / 2 - controlHeight)
        )
        path.addQuadCurve(
            to: CGPoint(x: width / 2, y: height / 2),
            control: CGPoint(x: width / 2 + controlWidth, y: height / 2 + controlHeight)
        )
        
        return path
    }
}

controlWidth and controlHeight are the distances used to define the control points for the quadratic curves.

Notice, for right petal the x postion is width/2 + controlWidth contrary to width/2 — controlWidth .

Assembling the Super Shape

To create a complete flower shape in SwiftUI by assembling multiple petals around a central point, we can utilize the Flower shape. This shape combines multiple instances of a petalPath rotated around the center of a rectangle. Here’s how it works:

struct Flower: Shape {
    
    func path(in rect: CGRect) -> Path {
        var path = Path()
        let iterationAngle = Double.pi / 3
        
        // Iterate through each petal angle
        for angle in stride(from: 0, to: Double.pi, by: iterationAngle) {
            // Create a rotation transformation for the petal
            let rotation = CGAffineTransform(rotationAngle: angle)
            
            // Apply translation to center the petal in the rect
            let translation = CGAffineTransform(translationX: rect.width / 2, y: rect.height / 2)
            
            // Combine rotation and translation
            let transform = rotation.concatenating(translation)
            
            // Apply the transformation to the petal path
            let petal = petalPath(in: rect).applying(transform)
            
            // Add the rotated petal path to the main path
            path.addPath(petal)
        }
        
        return path
    }
    
    func petalPath(in rect: CGRect) -> Path {
        var path = Path()
        
        let width = rect.width
        let height = rect.height
        
        path.move(to: CGPoint(x: width / 2, y: height / 2))
        
        path.addQuadCurve(to: CGPoint(x: 0, y: height / 2),
                          control: CGPoint(x: width / 4, y: height / 2 - height / 6))
        
        path.addQuadCurve(to: CGPoint(x: width / 2, y: height / 2),
                          control: CGPoint(x: width / 4, y: height / 2 + height / 6))
        
        path.addQuadCurve(to: CGPoint(x: width, y: height / 2),
                          control: CGPoint(x: width * 3 / 4, y: height / 2 - height / 6))
        
        path.addQuadCurve(to: CGPoint(x: width / 2, y: height / 2),
                          control: CGPoint(x: width * 3 / 4, y: height / 2 + height / 6))
        
        return path
    }
}

Flower Shape (path(in rect: CGRect) -> Path):

• Iteration: The for loop iterates through angles (angle) from 0 to 2 * π (360 degrees) in steps of π / 3 (60 degrees), which positions each petal around the center of the rectangle.
• Rotation Transformation: For each angle, a rotation transformation (rotation) is created using CGAffineTransform(rotationAngle: angle).
• Translation: A translation transformation (translation) centers the petal in the rect using CGAffineTransform(translationX: rect.width / 2, y: rect.height / 2).
• Combining Transformations: The rotation and translation transformations are combined (concatenating) into a single transformation (transform).
• Applying Transformations: The combined transformation (transform) is applied (applying) to the petalPath to rotate and position each petal around the center.
• Adding to Main Path: The rotated and positioned petalPath is added (path.addPath(petal)) to the main paththat represents the complete flower shape.

The number of petals in this pattern is 2*(number of times loop is run) . Thus, if iterationAngle is updated to pi/2 , the number of petals would be 2*2 = 4 . After a cetain number of petals, the rotations would lead to overlapping petals.

Animating the Shape

Animating shapes in SwiftUI involves leveraging SwiftUI’s declarative syntax and animation modifiers to create dynamic and engaging visual effects.

struct FlowerView: View {
    //1
    @State private var visiblePetalTrim = 0.0
    
    var body: some View {
        ZStack {
            RandomlyMovingShapesView()
                .frame(width: 400, height: 400)
            //2
            Flower()
                .trim(from: 0.0, to: visiblePetalTrim)
                .stroke(.cyan, lineWidth: 2)
                .frame(width: 300, height: 300)
                .onAppear() {
                    //3
                    withAnimation(.easeInOut(duration: 9)) {
                        visiblePetalTrim = 1.0
                    }
                }
        }
    }
}

1. State Declaration: This state variable controls how much of the flower’s petals are visible. It starts at 0.0 (fully hidden) and will animate to 1.0 (fully visible).
2. Flower Shape: We use 12 petals in Flower for this example. iterationAngle is set as pi/6 for the same.
   trim(from: 0.0, to: visiblePetalTrim): This modifier animates the drawing of the flower shape from 0.0(start) to visiblePetalTrim (end), controlled by the state variable.
3. Animation Block: Initiates an animation block with a specified animation type (here, easeInOut) and duration (9 seconds).

Inside the animation block, visiblePetalTrim = 1.0 gradually changes visiblePetalTrim from 0.0 to 1.0 over the duration of the animation.

Conclusion

In this article, we explored the fascinating world of creating custom shapes and animations in SwiftUI, highlighting its powerful capabilities and ease of use. Here are the key takeaways:

1. Creating Custom Shapes: We learned how to define intricate shapes using SwiftUI’s Shape protocol and its various path-drawing methods. From basic polygons to complex floral patterns, SwiftUI empowers developers to craft visually compelling UI elements effortlessly.
2. Flexibility and Creativity: SwiftUI’s approach to creating and animating shapes is remarkably flexible, allowing for seamless integration with other SwiftUI views and gestures. This flexibility encourages creativity in UI design, enabling developers to innovate and deliver unique user experiences.

Have you used custom shapes in your app? I would love to hear about your experiences and any creative ways you have found to implement it! Reach out to me on X (formerly Twitter) on @garimabothraa!

In the vibrant world of modern dining, platforms like Swiggy’s Dineout offer endless culinary adventures. However, the excitement often wanes when navigating extensive restaurant menus becomes cumbersome. Searching for a favorite dish, like pizza, amidst a lengthy menu can be time-consuming and frustrating, leading to a sub-par user experience.

A solution is required to swiftly and precisely extract and search items from these menus, facilitating customers in effortlessly locating specific dishes. With Optical Character Recognition (OCR) technology, users can now effortlessly type in their desired dish and be swiftly directed to relevant menu sections with their preferences highlighted. This article delves into the iOS implementation of OCR, showcasing its potential to transform menu navigation and enhance the user experience.

What’s the Problem?

In our quest to reimagine menu item navigation, we tried to target one major problem with the existing menus:

• Tedious Search: Searching for specific dishes like pizza can be time-consuming, and important items might be overlooked.

The Vision Behind the Magic:

To simplify menu navigation, we utilized Apple’s VisionKit. This powerful framework provides a range of solutions for image analysis, including text recognition, which we have leveraged for our OCR process.

Unlocking the Power of on-device OCR:

We utilise VisionKit’s OCR capabilities to extract text from images. On-device OCR processes images directly on the user’s device. VisionKit analyzes pixel information and isolates important data and extract text without the need for internet connectivity, ensuring privacy and real-time performance.

This is a crucial step as it transforms the visual data from the images into a format that can be processed and analysed in real-time. The extracted text is then indexed and made searchable, thus, enabling users to easily look for specific items.

Seamless Search, Sensational Results:

Following text extraction, a search functionality allows users to input their desired item, highlighting these items within the menu. This simplifies the process of finding specific dishes or beverages, enhancing the overall user experience by making navigation more intuitive and less time-consuming.

The end result of this implementation is a more user-friendly solution for browsing restaurant menus. This is particularly beneficial for menus that span multiple pages, as it highlights the search results on all pages.

Now when you crave the same authentic Italian Margherita pizza, simply input “Margherita”, and like magic, the app dynamically highlights appropriate pizza options within the menu and takes you to the response.

In conclusion, our solution significantly improves the ease and efficiency of menu navigation, providing a more enjoyable and convenient pre-dining experience for all users.

Implementation:

Ready to peek under the hood? In this section, we’ll dive into the technical magic behind our OCR integration. Discover how we transformed searching menus from a chore into a delightful experience.

We have created a function — fetchTexts to fetch the text from the given image and draw highlights around the text blocks identified when isRender is true else we call the function didFinishDetection with the string results. This function is explained in detail below:

/// Fetches the text from the given image and calls the appropriate method based on the `isRender` flag.
///
/// - Parameters:
///   - cgImage: The CGImage representation of the image.
///   - isRender: A Boolean value that determines whether to render the image or not.
///   - image: The UIImage representation of the image.
///   - searchText: The text to search for in the image.
    
    fileprivate func fetchTexts(_ cgImage: CGImage, _ isRender: Bool, _ image: UIImage, _ searchText: String?) {
        // 1
        let requestHandler = VNImageRequestHandler(cgImage: cgImage)
        // 2
        let request = VNRecognizeTextRequest { (request, _) in
            // 3
            guard let observations = request.results as? [VNRecognizedTextObservation] else {
                return
            }
            if let searchText, isRender {
                let boundingRects: [(frame: CGRect, string: String)] = observations.compactMap { observation in
                    
                    // 4
                    guard let candidate = observation.topCandidates(1).first else { return (.zero, "") }
                    
                    // 5
                    let stringRange = candidate.string.startIndex..<candidate.string.endIndex
                    let boxObservation = try? candidate.boundingBox(for: stringRange)
                    
                    // 6
                    let boundingBox = boxObservation?.boundingBox ?? .zero
                    let frame = self.getConvertedRect(boundingBox: boundingBox, inImage: image.size, containedIn: self.scaledSize)
                    return (frame, candidate.string)
                }
                for item in boundingRects {
                    // 7
                    self.drawBounds(item.frame, text: item.string, searchText: searchText)
                }
            } else {
                let mappedStrings: [String] = observations.compactMap { observation in
                    guard let candidate = observation.topCandidates(1).first else { return nil }
                    return candidate.string
                }
                // 8
                didFinishDetection(mappedStrings)
            }
        }
        // 9
        request.recognitionLevel = .accurate
        
        do {
            // 10
            try requestHandler.perform([request])
        } catch {
            print("Unable to perform the requests: \(error).")
        }
    }
}

// Steps 1 - 10 are explained below

This method utilizes VisionKit’s text recognition capabilities to extract text from a given image represented as a CGImage. It accepts parameters such as isRender to determine whether to render the image, image representing the UIImage representation of the image, and searchText specifying the text to search for in the image.

Text Recognition Process: Within the fetchTexts method, a VNRecognizeTextRequest is created to perform text recognition on the provided image. Detected text observations are obtained from the request results, and based on the isRender flag, either bounding rectangles and strings are extracted for rendering highlights on the image, or the recognized strings are mapped and passed to the didFinishDetection method. Step-by-step explanation of how the code works:

1. Create image request handler for searching in the image
2. Create a VisionKit request to convert the image to smaller blocks with text
3. Extract the result of OCR as array of smaller text blocks
4. Find the top observation — i.e. most probable result
5. Find the bounding-box observation for the string range.
6. Get the normalized CGRect value for the frame of observation.
7. Draw the highlights for the searched text around the frames from last step. (Implement a drawBounds function for this)
8. Call completion handler for detected texts with the array of strings identified
9. The recognition level for image request handler is set as accurate. There are two recognition levels provided for the same: `accurate` and fast
10. Perform the text-recognition request in try catch block

Menu Mastery: Impact and Beyond

The adoption of OCR technology heralds a new era of dining discovery on Swiggy Dineout, with far-reaching implications for users and restaurateurs alike. The impact extends beyond mere convenience, encompassing:

• Informed Decision Making: Users can effortlessly search for their favorite items amidst extensive menu listings.
• Intuitive Interface: Our solution offers a visually engaging and user-friendly experience.

Conclusion

As the culinary landscape continues to evolve, our commitment to innovation remains unwavering. With OCR technology as our guiding light, we’ve embarked on a journey to redefine the way users discover and savor the flavors of the world. Together, let us unlock the menu magic and embark on a culinary adventure like never before. Welcome to a world where every dish tells a story, and every bite is a revelation — welcome to the future of dining discovery on Swiggy Dineout.

The potential of VisionKit extends beyond Swiggy Dineout. Other apps can also leverage its powerful text recognition capabilities to enhance user experiences across various domains, from document scanning to real-time translation. As for Swiggy, the possibilities with VisionKit are vast. Imagine using it to detect allergens, suggest popular dishes, or even provide nutritional information at a glance. By continuing to explore and innovate with VisionKit, Swiggy can further revolutionize the way we interact with food, making dining not only more convenient but also more informed and enjoyable.

Special thanks to Darshita Golchha, Priyam Dutta, Agam Mahajan, Raj Gohil, Tushar Tayal, Preetichandak and Saptarshi Prakash for the execution of this project.

Streamlining Menu Browsing with OCR Text Recognition was originally published in Swiggy Bytes — Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.