Financial services and healthcare companies have a new innovation roadmap, and it starts with on-device intelligence.

A few months ago, Apple quietly flipped a switch at WWDC: developers can now tap its on-device foundation model directly from apps. This week, OpenAI dropped open-weight models you can run on your own hardware — even on those with more modest specs. For leaders in banking and healthcare, these announcements represent more than “just another AI feature” to think about. They open a whole new lane: ship AI features that keep sensitive data on the device, cut latency, and dodge a ton of compliance drag.

So what?

Local AI shrinks the riskiest surface area — data in transit and at rest — so developers can move faster with fewer lawyers in your standups.

Local models are the cheat code: they turn security and privacy from constraints into accelerants.

Apple and OpenAI Are Making “Local First” A Viable Strategy

Apple’s Foundation Models framework exposes the on-device model that powers Apple Intelligence — available with just a few lines of Swift. It’s fast, private, works offline, and Apple explicitly calls out no per-token inference cost for on-device use. For heavier lifts, Private Cloud Compute uses Apple-controlled servers running on Apple silicon with a verifiable privacy posture. Translation: you can build real AI features without transmitting customers’ sensitive data to a generic cloud.

OpenAI’s new gpt-oss-120b and gpt-oss-20b are open-weight models under Apache 2.0. The 20B variant is optimized to run with as little as 16 GB of memory — a sweet spot for laptops, kiosks, or clinical workstations. And you can host either model yourself or via AWS (Bedrock/SageMaker) if you want local control with managed infrastructure.

Privacy, Latency, and Cost Turn from Risks to Opportunities

Keeping inference on the device minimizes the data you transmit and store — the exact stuff that inflates breach blast radius and legal exposure. Apple’s docs spell it out: for requests that go to larger models, Private Cloud Compute doesn’t store or share data with Apple, using it only to fulfill the request. Meanwhile, on-device inference cuts round-trips and delivers snappier UX for tasks like categorization, autofill, or triage. And done right, local models eliminate per-call API fees for high-volume micro-tasks.

In regulated markets, reducing that exposure isn’t a nice-to-have — it’s existential. IBM’s 2025 Cost of a Data Breach pegs the U.S. average at a record $10.22M, with healthcare breaches averaging $7.42M. The same report found 13% of organizations already experienced AI-related breaches, and 97% of those lacked proper AI access controls. Local AI trims the target.

13% of organizations have experienced AI-related breaches, and 97% of those lacked property AI access controls.

Payments teams face PCI DSS 4.0/4.0.1 requirements hitting hard after March 31, 2025. Less data egress means fewer systems in scope. Health teams now operate under an expanded FTC Health Breach Notification Rule, explicitly covering health apps and tightening breach notice timing. And HHS OCR is prioritizing enforcement around tracking tech in covered entities’ sites and apps. All three trends reward data minimization, which local inference naturally supports.

Five Ideas to Juice Your Customer-Facing AI Roadmap

• Card-present smarts for mobile wallets. Use on-device models to detect risky payees or devices before a P2P push — no server round-trip. Demand is there: Zelle just crossed $1T in annual volume and 151M enrolled users.
• Transaction insights that never leave the phone. Categorize, summarize, and explain recent activity locally — then ask permission before anything syncs. Consumers are already behaving mobile-first: 70% of U.S. consumers made a mobile payment in the 12 months ending in October 2024, with mobile hitting 32% of all payment counts.
• Ambient clinical documentation with a smaller footprint. Run speech-to-summary locally during visits, and only export structured fields to the EHR. Early evidence shows reduced note time and better clinician experience; Kaiser reports 15,000 hours saved across deployments. Start small with specialties that favor templated notes.
• Member/patient inbox triage. Classify sensitive messages on-device, draft replies, and escalate only the minimal context. Faster response, less PHI moving around. (Apple’s guided generation + tool-calling in the Foundation Models framework makes formatting structured outputs straightforward.)
• Assistive workflows at the edge. From branch kiosks to bedside tablets, gpt-oss-20b can run on 16 GB machines, enabling offline ID verification prompts, form prefill, and risk checklists. With leading banks like Chase, Bank of America, Fifth Third, PNC, and others doubling down on branch expansion, it might be time to rethink the “phygital” in-branch experience.

How to Choose Your Local AI Stack for Speed and Scale

1. Pick the right jobs. Target high-volume, low-risk text tasks first: summarization, entity extraction, classification, next-best-action. (Apple’s on-device ~3B model is built for exactly these.)
2. Stand up the model. On iOS/macOS, wire in Apple’s Foundation Models framework; for cross-platform or workstation needs, deploy gpt-oss locally or in your VPC. AWS has managed paths if you want faster infrastructure boot-up with your own controls.
3. Constrain outputs. Use guided generation / structured outputs so the model returns typed data you can validate. This is how you keep downstream systems clean.
4. Keep data local by default. Do redaction and entity extraction before any sync; prompt users to opt-in for cloud features. Apple’s Private Cloud Compute covers heavier inference when needed — under strict privacy guarantees.
5. Instrument safety. Add prompt filters, rate limits, per-feature kill-switches, and human-in-the-loop for escalations. Open-weight models make internal red teaming and fine-tuning easier — and safer — inside your perimeter.
6. Measure what matters. Track deflection, handle time, accuracy, and user trust (did they accept the suggestion?). Tie wins to tangible costs: API calls avoided, seconds shaved, escalations reduced.

Bonus: Bake compliance into the product pipeline, not after it. Bake compliance into the product pipeline, not after it:

• Data maps for every prompt path (what’s on-device vs. whatever leaves).
• Model cards and eval suites for bias, safety, and performance — checked into source.
• PIAs that reference your local-first design choices and applicable standards (PCI, HIPAA, HBNR).
• Human-centered UX for consent and transparency; Apple’s Generative AI HIG has pragmatic patterns to borrow.

Where Every Bank and Healthcare Executive Should Go From Here

Local AI is the fastest way to get real, intelligent value into customers’ hands without creating new data liabilities for your company. It’s how you pilot quickly, learn safely, and scale what works, especially when your security team (rightly) runs a tight ship. If you’re exploring Apple’s on-device models, OpenAI’s gpt-oss, or any of the other device-optimized models from Google, Samsung, Meta, or Microsoft (to name a few), let’s co-design a roadmap and ship something useful together — in 90 days or less.

Curious where local AI could move the needle in your app? Reach out — we’ll whiteboard ideas specific to your workflows, data, and risk posture.

What’s the smallest risky workflow you could move on-device this quarter — and what’s stopping you?

Photo by Roberto Hund: https://www.pexels.com/photo/close-up-photo-of-person-using-an-atm-5357171/

Build Intelligent Features Safer and Faster with Local AI was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.

When mobile app developers talk about engagement analytics, we’re usually talking about what kinds of things users have been doing inside the context of our apps. This conversation about engagement started to change when Apple launched app extensions with iOS 8. Rather than asking, “What are users doing in our app,” brand leaders started asking, “How are our audiences experiencing our brand on mobile?” But why the change?

iOS app extensions enable brands to provide content and functionality to users outside their core application.

Back in 2014 when extensions were launched, app developers flocked to create Today view widgets (those are the card-looking content blocks that appear when you swipe your iPhone home screen all the way to the left). Since checking balances is one of the primary use cases for banking apps, financial services brands used Today widgets to show users a quick view of their account status without requiring those users to go through the hassle of logging in.

iOS app extensions have come a long way since 2014.

While biometric authentication has made complex login a thing of the past, Apple continues to offer app developers entirely new and advanced extensions that make it possible to expose their content across the operating system, creating opportunities to enhance app discoverability, engagement, and ease-of-use. Not every available extension is relevant for financial services brands, however, if you’re looking for inspiration on how you can extend the features of your existing iOS app, consider these three unique possibilities.

Be Part of the Conversation with Messages

Messages apps became an almost overnight hit because of how easy Apple made it for non-developers to create custom branded “sticker packs” and release a Messages app to the store, no coding experience necessary. But, the Messages extension truly shines when brands develop a custom app for Messages that’s linked to the features, functionality, and content of their application.

Some of the most recognizable names in mobile payments have built Messages extensions that let users complete or request peer-to-peer payments, for example, without ever having to leave a text message thread with their friends. Apple Cash provides one of the most recent examples for how brands can create a unique peer-to-peer payment experience.

Messages extensions work best when their functionality feels like it should be shared with others, which is why peer-to-peer payments have stood out from the crowd. However, financial services brands can do much more with the robust scope made possible through Messages. Shared account holders could send their account balances or relevant account alerts with one another without needing to login to their app. A primary cardholder could use a Messages conversation to add an authorized credit card user. There are even opportunities where members of a shared account can converse about and split up monthly payments for an account to ensure everyone pays their fair share. Some financial services brands have even adopted the sticker approach in hopes that Messages will become a new source of customer referrals.

Make Payments and Pay Bills with SiriKit

SiriKit is one of the most recent pieces of Apple technology made available to third-party developers. While the number of intents that SiriKit is currently configured to manage is still somewhat limited, Apple does make it possible for developers to create app extensions that allow users to pay their friends or their bills just by invoking Siri.

While it may not sound earth-shattering, it’s worth remembering that most banking apps that support bill payments require their users to complete lengthy and sometimes complex forms. These forms can introduce a variety of potential headaches for your users and even disincentivize them from using certain features in your app all together. This is where making bill payments using Siri, for example, could significantly improve your brand’s overall user experience while you overhaul a legacy process reliant on old technology that may be doing nothing more than frustrating your users.

Create Rich Context with Interactive Notifications

At this point, most financial services brands have been using push notifications to keep customers updated on the status of their accounts and to encourage app engagement. However, the simple act of sending a notification barely scratches the surface of what’s possible with iOS rich notifications.

Rich Notifications offer two unique opportunities for brands who are looking to extend content and functionality outside their core apps. First, Rich Notifications on iOS can include Quick Actions — CTA buttons that your users can tap in response to the content of the notification. Let’s suppose that, to combat fraudulent charges, you send your app users a push notification when you detect fraud on their account. When the customer users 3D Touch to expand the notification, he or she could see two buttons, one that opens the app to start a fraud report and one that freezes their credit card.

Second, Long Looks allow iOS users to view and interact with even more rich content without leaving the context of the notification. Just like Quick Actions, a notification’s Long Look is exposed using 3D Touch, and Long Looks can be developed to show the user just about anything, including images, video, audio, and even a completely custom user interface. Your users could configure and submit a payment, transfer funds, pay bills, and much more right from a notification.

These enhancements may sounds like window dressing. However, if you’re already sending push notifications for scenarios when a bill payment is approaching or a transaction was made, for example, rich notifications can significantly shorten the time between a user seeing a notification and ultimately doing something because of it.

The Extensions Universe is Growing, and So Should You

Notifications, SiriKit, and Messages aren’t the only extensions relevant for financial services brands. In fact, Today view widgets are still hugely popular among apps of all flavors, and extensions for WatchKit and Spotlight content indexing continue to help developers extend the possibilities of their apps outside the core user interface.

And, Apple will continue to invest in building extensions that provide incremental value for developers and users. Apple’s newest hardware HomePod, for example, supports just a few of the available intents users can invoke through Siri.

Consider adopting one or many of the available app extensions for you iOS app. You’ll give users more opportunities to discover and engage with your brand on their smartphones and tablets, and you’ll have a better chance of catching Apple’s eye when their teams are making decisions about which apps they want to feature in the App Store.

3 Unique Ways Financial Services Brands Can Boost App Engagement with iOS Extensions was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.

When drafting an RFP for a new digital product, there are a number of questions that you’ll typically want to include. What kind of development process does the vendor employ? What is the experience level of the consultants on the project? What will the price and timeline be for the expected feature set, and when can we expect the product to launch?

But beyond the necessary details are the qualitative questions that separate a great product development partner from a lousy one — the questions that will identify whether a vendor relationship will be enjoyable or frustrating. These questions aren’t always easy to quantify, and admittedly won’t make sense within the text of an RFP, but are essential components of the agency selection process that clients typically neglect — and regret it.

Most products are focused on the needs of the end users — but the best agencies consider their clients as users in their own right, users of the agency’s services that are just as important as the end users who will use the product. In essence, consider yourself the user of the agency. How well will your agency address your real needs? Beyond how well they’ll execute the project, also be critical of what these agencies will do for you. To assess that, you can begin by asking yourself these questions during your agency search.

When evaluating agency partners, remember to not only evaluate how well the vendor will execute the project, but also be critical of what these agencies will do for you as their client.

Does the agency practice what it preaches?

If you’re currently in the process of researching software design and development agencies, you’ve probably noticed how common it is for those agencies (Punchkick Interactive included) to market themselves as “user-first” or “user-centric.” For a long time, being focused on the user as a principle was a competitive differentiator for software agencies like ours, but today, agencies tend to use the concept of user-first product development as little more than a base requirement for website SEO, or as a way to get their foot in the door for early-stage sales meetings. Simply put, the marketing of “user-first” has become hackneyed. User-first is a buzzword.

And that’s a good thing. The more that agencies like Punchkick and our client partners focus on building products that deliver actual value for users — as opposed to simply checking boxes from lists of features, which were developed by several business units in multiple solution design meetings — the better we all start to look in the eyes of our diverse audiences. As you conduct your next agency search, be sure to include companies that really market and defend the principles of user-first design and development in practice.

The more that agencies focus on building products that deliver actual value for users, the better we all get at addressing the needs of our diverse audiences.

How well do you feel listened to?

A catalyst for a healthy, growing relationship is good listening. Good listening is something we’ve all been trained on from the time we were kids, and bad listening is easy to spot early on. Great listening — the kind you should be looking for — requires the listener be capable of picking up on other signals, things that aren’t said in conversation, but are just as critical to understanding who you are and what you need.

At Punchkick, our product owners are intentional about “eliciting” requirements for new software products or features. Eliciting requirements is much different than gathering or listing them because it places a proactive focus on seeking to understand the user and business needs behind a particular product or feature. Some of the best features Punchkick has helped build over the last decade were ideated after initial contracts were signed, during early solution design periods, where empathetic listening allowed us to help our clients invent something even better than they had originally imagined.

Will your support team make your life easier?

One of the biggest horror stories that companies can tell is about the agency that made their job unbearable. When you choose to work with a third party, you put your trust in their team to design, build, and launch something amazing. As a client, a very realistic assumption you should have of your new full-service design and development agency is that they’re going to make your life both easier and more enjoyable.

Building custom software should be fun. And your agency partner should ensure that every stage of the product development process makes clients’ jobs easier and more effective.

One question you should ask every agency is about who your daily contacts will be, and what specifically they will provide you with. At Punchkick, for example, every employee — from account managers to quality assurance testing engineers — can pick up the phone and ensure your needs are met. For daily operations, however, our support teams cover three bases. Account managers ensure you’re happy with the experience you’re having each week. If anything feels wrong, they fix it. Project managers keep development teams running efficiently so your budget requirements are met, your products are delivered on time, and communication lines are open and transparent. Finally, strategists ensure that your requirements are not only accurately communicated to designers and developers, but are also continually ideated upon as things change to ensure the product will be industry-leading when it launches.

Will you learn something new?

The coolest thing about working with a software agency is that you get to play a leading role in designing and building something that never existed before. That generally means you’re going to be exposed to concepts and ways of working that are unfamiliar to you, at least in the beginning. Any great custom software agency will treat you like a partner in the process — rather than just a paycheck and a rubber stamp — and will go out of its way to ensure that the process is one that supports an objective of continuous education.

At Punchkick, we practice this idea of continuous education starting early, with design. Rather than spending months drafting the most “pretty” artwork we can, Punchkick designers use the process to teach clients about best practices in user experience design, color theory, accessibility, and the effectiveness of specific interactions, animations, transitions, and other UX considerations.

Does the agency make recommendations that help your brand long-term?

Software is never a one-and-done activity because technology changes faster than almost anything else you’ll deal with in your day-to-day job. The best software development agencies approach projects with this in mind. But, more important than scaleable technology, your agency should be relentlessly focused on your long-term brand vision. Believe it or not, not every development agency has the marketing chops to bring a vision to life. Corey Wainwright from HubSpot says, “[T]he easiest way to tell if I could trust an agency (and the best way I tried to build a sincere and effective relationship with my clients) is to see whether they were willing to be honest with me. Even when the truth kind of hurt.”

Because the best software development agencies approach your business as if it were their own, they’re unafraid to offer advice about better ways to use technology to create growth for you. If the advice sounds counter-intuitive to your method of thinking, it could be that they’ve uncovered a critical insight that could only be found by someone outside your company.

The best software development agencies think of your business as if it were their own, and ground every decision around how well it’ll help your team meet its goals.

Questions like these most likely won’t make it into your RFP, and they rarely get answered in traditional agency review cycles. The only way learn whether a new agency will be a true partner, making your life easier and ensuring long-term success for your mobile marketing efforts, is by picking up the phone and asking these questions directly. You’ll feel even better if you ask to speak to a fully functioning product development team that’s responsible for creating successful products.

5 questions most agencies never answer during the RFP process was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Accessibility is a topic that in many ways is as old as the web. Because designers and developers have been focused on making digital products accessible for decades, there are many guidelines, standards, and even automated assessment tools that already exist to ensure websites are accessible. There are also regulations that many companies need to comply with, including the Americans with Disabilities Act, which has been updated to include digital product guidelines.

In the technology landscape of 2017, guidelines designed for the web as it was in 1997 are beginning to look dated.

But in the technology environment of 2017, guidelines based around the needs of disabled web users in 1997 are looking increasingly dated. These accessibility guidelines and frameworks were developed in the early days of the web, and in many ways the evolution of consumer technology has outpaced them. Many accessibility guidelines and best practices are focused on the web as it was in 1997 — static pages of content.

Designers should endeavor to define accessibility guidelines for the web and mobile as they are in 2017 — populated with advanced web and native applications that have their own accessibility best practices. This is based around a commitment to user-centered design, with an understanding that every user is valuable — regardless of their age, familiarity with technology, or disability.

Alienating users — any users — is bad design. Everyone should be able to use any digital product, regardless of disability.

Product designers can’t claim to fully embrace user-centric design unless they design for the needs of every user. Below are a few target areas that expand upon the common guidelines for fonts, colors, and web markup to better address the technologies and use cases of modern users.

Applying accessible design to native app navigation

There are many paradigms and best practices for how mobile and web experiences to offer fluid into navigation. On mobile devices, standards put forth by Apple and Google help ensure that navigation is consistent between apps and predictable for new experiences. In many cases, simply complying with Apple’s Human Interface Guidelines or Google’s Material Design best practices can ensure an accessible experience of the box. However, there are some common themes that can help any kind of application or user experience prevent alienating the users who depend upon them.

For navigation, it’s essential that the app’s many feature areas are as accessible as possible. In most cases, this needs making them visible the moment the user opens the app, potentially in a tab bar at the bottom of the screen. Google’s Material Design best practices even recently adopted the bottom tab bar as a recommended navigation option, shifting away from its previous stance of hamburger menus.

Hamburger menus obscure navigation options behind an invisible drawer — an issue that’s compounded when disabled users are navigating the app through a means other than their fingertip.

About hamburger menus. The three horizontal lines at the top left corner of many interfaces — while prevalent on both web and mobile — are doubly inaccessible in many cases. They not only obscure the navigation items available to the user behind a drawer, but they also place an essential piece of the app’s navigability at the furthest corner of the screen from users’ thumbs. This issue is compounded when users might be interacting with the screen with something other than their fingertip, like switch control or screen reading technologies, as this approach requires two distinct interactions to reach the most essential parts of the app.

Considering the unique needs of mobile users with disabilities can have a halo effect that improves the user experience for all users, even those without such disabilities. Here it is the underlying lesson of accessible design: good UX design, especially in the context of navigation, offers benefits for all users by making the app easier to understand and simpler to use.

Accessible design delivers benefits for all users — disabled or not — by making the app easier to understand and simpler to use.

There are many areas of accessible design that need to be reevaluated for the unique needs and considerations of modern mobile users. However, whether for compliance with federal regulations or for the betterment of all audience members, employing accessible design in mobile app design and development is a surefire way to make your app more usable and more fundamental to your users’ lives.

Read more about accessible mobile design here:

Apple’s new design style in iOS 10 is a statement on accessibility

Check out Punchkick’s webinar on Designing for Accessibility here:

Designing for Accessibility Webinar

Accessible design in modern app navigation — beyond the static web was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.

During the holiday season, as millions of people are setting up their brand-new iPhones and iPads, there is perhaps no better time to have your iOS app featured on the front page of the App Store. Apple has many avenues to feature apps — from promotional banners on the App Store home page, to more carefully curated Editors’ Choice and Design Awards lists. But there’s one thing Apple loves featuring more than anything else — apps that make use of its latest technologies.

There’s one thing Apple loves to feature above all else — apps that make use of its latest technologies.

Below is a list of the low-hanging fruit that the best iOS developers can explore to not only get Apple’s attention, but also to substantively improve the user experience of their iOS apps. And — who knows — investing in these new channels could open revenue opportunities and feature ideas that attract a whole new audience to your app.

1. Design your app to be universal on iPhone and iPad.

This one might seem like a lot of work, but building iOS apps the right way from the beginning can make iPad-specific views a nominal task down the road. One iOS app can run device-specific views on iPhone and iPad while sharing a universal code base for app logic and communicating with an API. Smart iOS app designers can even find ways to be efficient with in-app views, using Auto Layout and size classes to make the app look great on larger iPhones and iPad. With a little extra design and development effort, apps can target iPad relatively easily — and for certain apps, being universal is the price of entry to be considered for an App Store feature.

2. Find creative ways to implement 3D Touch.

3D Touch is the new pressure-sensitive technology that Apple introduced with iPhone 6s, and many top-tier app developers were quick to adopt it for previewing in-app content or getting quick access to features from the home screen icon. But a year after its debut, the 3D Touch use cases that impress App Store editors and users alike are those that use the technology more inventively, using the various amounts of on-screen pressure to perform different actions within the app. Every iOS app could use 3D Touch differently, but developers would be best served to think of ways to use the feature that will surprise and delight their users.

3. Use HealthKit to share data between apps.

HealthKit is an iOS SDK that allows health app developers to share their app’s health datawith one another. For example, a dietary app can compare notes with a workout app, and each can write their data to a common dashboard in the iPhone Health app. For any health- or fitness-related iOS app in 2017, HealthKit compatibility is an absolute must, and can augment the user experience for an audience that increasingly uses a number of apps to track their wellness. What’s more, your app doesn’t have to only write its own data to HealthKit, but can also read data provided by other apps, allowing developers to build smarter experiences with more awareness of users’ overall health.

4. Build a version for Apple TV and tvOS.

Apple’s fourth-generation Apple TV set-top box opened the door to third-party developers with an all-new App Store in 2015, and Apple has been aggressive about promoting apps designed for its tvOS operating system ever since. Because tvOS apps can share code with their iOS app counterparts, many video streaming and game developers have introduced successful tvOS apps. But there are many kinds of apps that could make sense on Apple TV, and it’s worthwhile for iOS developers to consider how their apps could make the jump to the silver screen.

5. Ensure your app is built with native code.

There are many ways to build apps for iPhone, including so-called “write-once” app platformsthat aim to make development across iOS and Android more efficient. However, sharing code between platforms comes at great expense to the end user experience, and can introduce major complications with updating the app for new versions of iOS over time. Apple favors apps written in native code — Objective-C and its new Swift programming language — that take full advantage of the APIs and frameworks built into iOS. Not only do these apps offer a best-in-class user experience on the platform they’re designed for, but they also are better equipped to jump into the new technologies Apple has in store for the next version of iOS.

6. Use Apple Pay for payment within native apps.

Apple Pay has reinvented how millions of people pay in-person at retail stores, but its in-app payment component is perhaps more impactful to deliver a great iOS user experience. With Apple Pay, iOS app developers can expedite the payment process, not requiring users to enter their payment details or billing address, and instead offering a one-click secure payment option in their apps. For any app that includes transactions and in-app payment, implementing in-app Apple Paycan make for a more frictionless in-app purchasing experience.

7. Build an iMessage SDK extension — or sticker pack.

With iOS 10, Apple introduced a new iMessage App Store that allows users to bring apps into their iMessage conversations. There are many ways for iOS app developers to extend their apps into iMessage threads, including interactive multiplayer games or collaborative shopping carts. Having an app for iMessage as part of your larger iOS app is a great way to add value for users, who won’t have to open your app to use certain features, but also is a great new avenue to have your app discovered in the iMessage App Store. What’s more, an incredibly basic iMessage app extension can simply add a sticker pack to the conversation, and Apple has built basic templates to speed up development.

8. Design an Apple Watch version for watchOS.

Apple makes it simple for iOS app developers to bundle Apple Watch apps with their iOS apps, and native apps on Apple Watch in watchOS 3 make watch apps more powerful and useful than ever. Apple Watch apps are a great way to extend simple pieces of your app’s functionality and deliver up-to-the-moment content to users’ wrists. There are many best practices that can help a watchOS appseparate itself from the pack, but Apple is keen to help developers who invest in watchOS get their apps noticed on the App Store, even including a dedicated Apple Watch App Store within the Watch app for iPhone. In the end, exploring watchOS and Apple Watch is a great way to extend the iOS app experience to an entirely new — albeit smaller — context, and for your app to begin populating the micro-moments in users’ lives.

9. Integrate with connected home hardware and HomeKit.

HomeKit is a connected home platform that is only just beginning to find its legs, and there is an ever-expanding portfolio of hardware products that connect to Apple’s HomeKit APIs. Not every iOS app that uses HomeKit needs to have a hardware component, but it certainly makes sense for companies who make hardware for the home to find ways to connect their products to HomeKit. Since Apple introduced the dedicated Home app in iOS 10 for HomeKit features, it’s prioritizing HomeKit-specific apps across the App Store to showcase its latest features. If you make lightbulbs or thermostats, your app simply needs to be there.

10. Design an experience people will love to use.

In the end, the apps Apple chooses for its Editors’ Choice or Essentials lists come down to a user experience that iOS users will love. Having a household brand name simply isn’t enough to get noticed on the App Store these days, especially since new Silicon Valley startups are finding ways to improve UX to disrupt established industries. With customer expectations for mobile apps in 2017 being higher than ever, user experience matters most, and most of these technologies are really in service of removing friction or context switching from the user flow. All that matters is how well the iOS app user experience addresses end users’ real-world needs — and if the app fails to do that with a visual and interactive design users enjoy, it’ll fail to meaningfully connect with those audiences.

10 Ways to Get Your App Featured in the iOS App Store

This story was originally published by Punchkick Interactive. Check out punchkick.com for more about iOS app development, mobile UX design, and more.

10 Ways to Get Your App Featured on the iOS App Store was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Over the last several years, our expectations for what computers should do for us have transformed dramatically. What were functional tools that sat on our work desks only ten years ago have evolved to fit in our pockets, capture our memories, and reshape how we communicate with family and friends. With each passing year and each new leap in consumer electronics, the devices we love to use have continually become more accessible, more integrated, and more adaptable to our everyday lives.

These trends are a continuation of what computers have been doing since the first PC. The first 50 years of computer science were a story of humans adapting to computers, rethinking their thoughts around word processors and discovering new business models that fit into the web. The next 50 years will be a story of computers becoming more human — changing how we interface with them beyond mice and keyboards and instead incorporating elements of how we interface with other humans.

The first 50 years of computer science were a story of humans adapting to computers. The next 50 years will be a story of computers becoming more human.

One of the first trends in this new trajectory is conversational UI, a new user interface technique that’s becoming increasingly popular on mobile. Better known as chat bots, these programs are an evolution of user interface design that allow users to communicate with a digital experience using their own words and on their own terms. We’ve examined how brand apps in particular can leverage conversational UI best practices to enhance their own apps, and what considerations should go into developers’ experiments with their first chat bot.

Chat bots are a quintessential mobile trend — something flashy and tech-forward that some brands are sloth to adopt. But what their conversational UI model represents is something greater: a revolution in how humans interact with computers, the descendants of which will define our mobile experiences for decades to come.

Understanding Bots

The idea of a chat bot is nothing new, but it’s growing in popularity as a user interface technique for mobile experiences. This is mostly due to the massive adoption and ubiquity of messaging apps, many of which have hundreds of millions of users and are poised to become development platforms unto themselves. While chatting with a computer is a concept that dates back well before the age of AOL Instant Messenger, the idea of marrying this interaction paradigm with new advancements in natural language processing is actually the latest in a series of transformations in how we interact with digital products.

Marrying a chat-like interaction paradigm with new advancements in natural language processing represents a transformation in how we interact with computers.

To understand the significance of this shift, it’s important to recognize its context in the evolution of user interface models. One of the biggest transformations in how people interact with computers was the graphical user interface (GUI), which replaced the command-line interfaces of the 1980s with dynamic and visual user interfaces populated by buttons and windows. The change was intended to bring digital experiences closer to our experience in the real world, which is filled with physical objects that represent our work and play spaces.

In 2007, the capacitative touchscreen on the first iPhone allowed for a new input method, where users could simply tap on-screen items to perform actions. This was bringing the interface language of computers further into human context — there’s no abstraction of a mouse-and-cursor when people interact with objects in the real world, so why should there be such barriers when they interact with information in the digital world?

The trend is obvious in hindsight: each of these changes brings the digital world a step closer to our physical human experience. We understand spacial relationships better than a lexicon of computer commands. We understand direct manipulation better than indirect mouses and cursors. As computers become more powerful, and the services that run on them become more sophisticated, new interface paradigms are necessary for humans to more closely relate to the digital experiences in our lives.

And this is where bots come in — a new way to interface with computers that more nearly approximates the experience of interacting with a friend.

What is a chat bot, and why should brand marketers care?

What are bots?

Chat bots are web-assisted programs that allow users to perform actions or get information that would have traditionally required opening an app or navigating to a website. Need the weather? Shoot off a chat message to a Slack bot and get an immediate response. Want to shop for new clothes? Ping a Messenger bot for images and prices from your favorite store. Put simply, bots represent a way to perform the same actions we’re accustomed to on mobile devices in a new way, a way that’s more intuitive and expressive than ever before.

Bots perform the same actions we’re used to on mobile devices in a new way — a way that’s more intuitive and expressive than ever before.

This approach works because technologies exist to understand users in natural language, allowing anyone to compose requests and have the computer understand them. Whereas app user experiences before mandated that users engage with them on the apps’ terms — having static buttons and a limited set of input options — chat interfaces and natural language processing allow users to engage with apps on their own terms, choosing how they express their requests from within their natural speaking style and using their own language.

Just as the GUI’s “desktop metaphor” helped bring an increased level of familiarity to digital experiences in the 1980s, so too do bots bring an increased level of familiarity and comfort to engaging with digital systems in 2016. Billions of people chat with friends and loved ones on their mobile devices, sending trillions of texts and iMessages and WhatsApp messages every day. It’s a natural extension that they would begin chatting with their mobile apps, too.

Going forward, this new experience will evolve to support voice input, and products like Amazon Echo and Google Home are the first players in this new category. The closer to ordinary life that a digital experience can feel, the better — asking Siri about the weather will be the modern equivalent of having “folders” on your “desktop.”

Why Bots Work

The most compelling thing about chat bots is that they don’t need a manual. They allow users to engage with digital products using their own words and on their own terms. Natural language processing has reached a degree of maturity that bots won’t need to know the exact phrasing of every possible request in order to act upon every single request — and that intelligence is only accelerating as bots proliferate.

But using a digital product on the user’s own terms extends beyond literal “terms.” Consider the mobile best practices for organizing an application’s content and navigation. Perhaps there’s a tab bar at the bottom of the screen, or a hamburger menu at the top left corner. Maybe options are laid out in a hierarchical table, and certain content or actions are obscured from view until the user drills down through enough menus. There’s a certain language to it, which can be optimized for as many users as possible, but it’s a language that requires a degree of UX fluency all the same.

With messaging bots, users don’t have to learn anything. If they assume the bot can perform nearly any action they’d expect of the brand — and if there’s graceful error messaging when it can’t — the interaction they have with a bot requires no thought or prior learning. This is enormously powerful as mobile reaches new audiences of varying ages and technical literacy. Imagine someone using an ecommerce app who’s never used a smartphone before. Now imagine that same person dictating requests to a chat bot that can peruse the entire store without the need to drill through various menus.

Example in Practice

Consider these two incarnations of the Spring app, a popular fashion-focused ecommerce product. On one hand, you have the traditional mobile app user experience, complete with menus and navigation paradigms that would be familiar to most smartphone users. For all its gloss and flat UI design, the app still requires users to be familiar with the trends of how ecommerce sites and apps have always been organized, meaning users would have to navigate through “Women,” “Tops,” “Blouses,” and so on to find the item they want.

Alternatively, consider the Spring bot for Facebook Messenger. Here, the user can immediately request the kind of item they’re shopping for, and the bot provides them with actionable results. No searching, browsing, or poking through menus organized by even the most competent UX designer. Just a request and a response.

Messaging services are the most heavily used apps across mobile platforms, with more than 2.5 billion active users sending trillions of iMessages, WhatsApp messages, SMS messages, and Messenger stickers every day. These services account for a significant portion of the reason that mobile users pull their phone out of their pocket every few minutes, and these communication methods form the backbone of billions of people’s mobile experiences.

These figures indicate that the user experience of composing a message and sending it to a friend is now universally understood among nearly half the planet — and Activate projects messaging app usage increase to 3.6 billion in the next few years. This is an interaction method that doesn’t need to be taught, because it’s most of what billions of people are using their devices for anyway.

For brands and third-party app developers, this universality of experience should indicate that your apps have a thing or two to learn. Third-party apps should reflect the interaction paradigms users are most familiar with. And in the age of WhatsApp and Messenger, that interaction is more likely to look like a chat conversation than anything from UIKit or Material Design.

Third-party apps should reflect the interaction paradigms users are most familiar with — and in the age of WhatsApp and Messenger, that’s more likely to look like a chat conversation than anything from UIKit or Material Design.

Further, because of the explosive growth of these messaging services, they’ve become hotbeds for innovation among developers and evolved into platforms unto themselves. Today, services like Facebook Messenger, Kik, Skype, Slack, and even iMessage have introduced features that allow third-party developers to build apps and services within users’ messaging conversations. And considering that the top four chat apps have a larger combined user base than the top four social networks, there’s already an enormous user base waiting to discover your app.

Building apps for Messenger or iMessage puts your brand squarely where your audience already lives, and within the services they already love. This not only aids discoverability of your service, but also accessibility for users who might not be as familiar with mobile UX patterns that others might consider obvious. Bots are the way that brands can take advantage of the messaging app phenomenon — but that isn’t to say they’re easy to get right.

How Bots Fit into Your App’s UX

When considering how bots can integrate into an existing app or service’s user experience, it’s important to note that the changeover to embrace the bots-centric conversational UI paradigm doesn’t have to be total. The chat bot–type interface is flexible, and can extend to include as much or as little of a digital product’s experience as makes sense for the user. There are three primary ways that bots can be integrated into a mobile user experience.

Bots can become the core user experience for certain types of apps. This doesn’t work for every app’s user experience, but some apps can afford to reinvent themselves to be fully conversational UI–focused. This works best for apps and products that already feel conversational, rather than transactional, where the primary methods of user feedback and interaction are a give-and-take of information between the human and the system. If this experience can be made to feel more personal through a chat bot–like interface, then rethinking the primary UX around a text conversation could make a lot of sense.

Bots can be integrated into an app to support one feature or functionality. While not every app can support a conversational UI approach as the primary interaction model, some specific features of those apps translate well to a text-based chat bot UI. For example, if the primary function of the app’s user experience is best represented as a traditional button- or table-view–centric interface, but there’s a customer support portal that requires two-way communication between the user and the app, it could lend itself well to a chat-bot-style UX makeover. Further, if the on-boarding flow for a particular app is long and arduous, requiring the user to complete several forms and volunteer information about themselves in a specific order, a chat bot can make that process feel lightweight and friendly rather than boring or intimidating. Again, this doesn’t work for every app, but many larger mobile products have areas or features that could be better represented as a conversation.

According to data from the Evercore Group in the Wall Street Journal, capturing half of the predicted business for customer support via a messaging interface would represent a brand-new revenue opportunity of $4 billion for brands.

Bots can be an extension of your app or service within third-party platforms. This approach is perhaps the lowest barrier to entry for third-party developers, but one of the toughest justifications for inserting a chat bot interface for a mobile product into users’ lives. Bots can extend the product’s services or data into third-party platforms like Facebook Messenger, Slack, or even Siri with SiriKit in iOS 10. Each of these platforms comes with its own strengths and limitations, but integrating digital services into new contexts like messaging platforms presents an opportunity to interact with customers in the apps they already use. For users, it’s much easier to strike up a conversation with your brand in the Messenger app they already have installed rather than seeking out your app on the App Store and waiting for it to download.

So how should brands approach this problem, and determine how bots would best intersect with their mobile products? For starters, it’s important to consider how and where users already expect to interact with your brand, your services, or your data. Considering users’ contexts when they seek out the services your digital product provides can help qualify or eliminate any of the places bots could be incorporated into your experience. If users need quick and easy access to your app’s data, perhaps adding a Messenger bot and other bot environments to your brand’s digital portfolio makes sense. However, if users only have a conversational relationship with your brand when they’re seeking out one-on-one customer support, perhaps adding a chat bot help line as one feature of a larger mobile experience is a better option.

Gartner claims that 85% of all customer interactions won’t require human customer service representatives by 2020, and instead will lean upon automated analysis of social media channels, CRM software, and personalized chat bots to greatly reduce the need for traditional call centers.

Secondly, it’s critically important to be mindful of providing bot-like assistance only when users actually need or request it, and avoid nagging users with unhelpful or irrelevant messages. The infamous Clippy character from Microsoft Word is a perfect example of what is ostensibly a “chat bot” verging on the annoying. The reason users hated Clippy wasn’t just that he was so often wrong — offering help with composing a letter when users had no intention of writing one — but it was because if a bot isn’t providing any real value, and instead is just a gimmick, it’s a nuisance that can sour users’ perceptions of your app. Before anything else, brands need to identify a place where a conversational UI can provide real value to users, and that may be within the confines of an app or website, and it may be outside of them.

Building Your Bot

Because many of the largest players in the messaging space also happen to be the largest players in the conversational UI space, there are many technologies that already exist to facilitate building chat bots that integrate with these services. With open-source platforms offering APIs that scale the bot’s conversational logic across multiple messaging services — and even into proprietary brand apps and websites — third-party developers have ample opportunity to experiment with the conversational UI model without having to develop the learning and conversation modeling algorithms for themselves.

While there is a diverse toolset of technologies at developers’ disposal, they share many of the same fundamental components and functionality. Central among these is teaching bots how to learn — helping them parse the unique vocabulary and request types for each distinct application and service. Thankfully, this is particularly what the open-source artificial intelligence platforms are focused on solving for, removing a massive obstacle for developers. All brands need to bring to the table is a handful of key reference data, and in most cases the bots can handle the rest.

Teaching Bots to Learn

Many of the open-source AI platforms rely on developers to submit reference data or requests to the bot that will help it extrapolate meaning from users’ messages. Here’s an example of how to teach a bot to adapt to varying user requests, in this case for a bot that gives weather forecasts.

1. Reference query. First, developers need to provide a sample query that a bot should field. This could be something like, “How will the weather be in Cleveland this weekend?”
2. Identifying intent. Next, developers should help the bot understand its purpose in this instance, which would be to retrieve a weather forecast. Bots can have several purposes, called users’ “intents” when they submit a message, and these can be identified by a number of different keywords or variables. Identifying the intent for this message helps the bot to understand what kinds of supplementary information it should look for — or ask for — and what kind of data it should ultimately respond with.
3. Elaborating with entities. Entities are the variables of the user’s request, which in this instance could include things like cities (“Cleveland” might be substituted for “New York” or “Chicago”), times (“this weekend” could just as likely be “this evening” or “next Wednesday”), or even forecast types (a general request like “weather” could also be more specific values like “precipitation,” “temperature,” or “humidity”). By providing reference entities and some examples of alternatives, bots can begin to parse users’ sentences and better narrow their response options. In particular, platforms like wit.ai offer ready-made entities for common requests like locations or times of day, eliminating the need for third-party developers to volunteer that data themselves and accelerating development.
4. Correcting for errors. In instances when the user doesn’t submit all of the entities necessary for the bot to appropriately field their question and address their intent, the bot needs to be able to gracefully handle those errors and ask for more information. In the example given above, if the user doesn’t mention Cleveland or any particular location, the bot might respond with “Where?” or give results for the user’s current location. If the user gives more detail, the bot can then append the request with that new entity value and provide a more accurate response.
5. Testing and enhancing. Like any software, the best way to test for bugs is to start using it. Many of the bot AI platforms offer a console-like interface for chatting with the bot and seeing examples of where it responds with appropriate data or where it falls down. Over time, developers can plan for common problems and ensure that conversations progress in an organic, logical way.

Teaching users to interact with your bot during on-boarding is a popular best practice in preventing embarrassing errors. For the most part, the leading conversational UI programs offer “what I can answer” sections that help users understand what they should expect of the bot and its capabilities. Giving examples of the requests your bot is equipped to field helps prevent frustrating bugs and gaps in conversation logic from derailing the user experience, but it’s important to remember that no software is perfect — especially bots. For this reason, graceful error handling is imperative for users to trust the technology. Users are more forgiving of a system that’s honest about its faults (“I’m sorry User, but I’m not sure I understand. Could you try asking a different way?”) versus offering flippant and cheeky responses (Apple’s Siri often says “That’s an interesting question” in response to queries it misunderstands).

The leading bot intelligence platforms have APIs that allow them to interface with many different technologies, including apps on both iOS and Android native platforms, within SMS conversations, on a myriad of chat platforms like Slack, Messenger, or Skype, and within websites and web applications. It bears repeating that many of these bot platforms are designed and open-sourced by the stewards of the messaging platforms themselves — Facebook developed and open-sourced wit.ai, which interfaces nicely with Messenger, while Microsoft’s Bot Framework is designed to fit well into Skype. Most of these platforms are free to use and easy for developers to program, meaning that technology limitations are no longer inhibitive of adding a great conversational UI experience to a third-party app.

Technology is no longer an obstacle to developing a great conversational UI experience for your app — what matters most is finding ways that bots can offer value to users.

In fact, while these technologies have equalized the playing field for developers, the challenge remains to find a worthwhile user experience that bots can facilitate within your apps. Brands need to determine the user experience they want the conversation to deliver before deciding on any one platform or technology, because — like with native development — there are many technology solutions that are better suited for one UX goal over another. However, the right marriage of a conversational UI experience coupled with a compelling value proposition for the brand’s audience can yield a more engaging and interesting mobile product than anything possible with traditional digital tools — which is reason enough to begin exploring.

There are already many compelling examples of conversational UI experiences in the market, some more experimental than others. But already, users can order an Uber from a chat conversation within Facebook Messenger, catch up on the news from CNN, shop for clothes with Spring, or discover their company’s marketing statistics using Statbot in Slack. Further, there are solid examples of conversational UI–centric experiences within native apps, like Lark for health and wellness or Quartz for a choose-your-own-adventure style messaging thread for the news.

The largest technology players are already active in this space, offering frameworks like SiriKit to extend iOS apps’ functionality into Apple’s tentpole virtual assistant, or enhancing Google Assistant and messaging app Allo to allow for third-party input. This is a trend that only promises to accelerate, and the apps that begin collecting insights and analytics about users’ interactions early will be those who can begin streamlining and enhancing their conversational experiences first.

Bots are certainly in vogue as a mobile UX strategy, but trends and usage statistics suggest that they’re anything but a fad. In the future, certain types of experiences will be delivered exclusively as chat bots, and will deliver the same functionality we’re accustomed to in a more intuitive and expressive way than ever before. Users’ expectations for mobile experiences have begun to shift dramatically, such that they expect dialogues with bots more frequently than apps’ dialogue windows. Just as brands have had to adapt to an evolving mobile landscape with smart UX design and digital services, so too will they need to adapt to an emergent conversational trend — tomorrow, the brands we rely on for everything in our digital lives will need to be conversant in “LOL.”

How chat bots fit into your app's UX

This guide was originally published by Punchkick Interactive. For more information about user experience, mobile development, and more, check out punchkick.com.

Guide to how conversational UI and chat bots can help reinvent any app’s UX was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Even with the Cubs winning the World Series and national politics inspiring breathless headlines, perhaps the strangest thing about 2016 has been the transformation of some of the industry’s biggest hardware companies. Between the big three operating system makers — Google with Android, Apple with iOS, and Microsoft with Windows — a dramatic shift in strategic thinking for each company has become clear in the last few months.

Punctuating this strategic change are some not insignificant technology releases over the past year. iOS 10 was a significant evolution of Apple’s mobile aesthetic, Google released its first-ever smartphone, and Samsung has struggled with less-than-optimal temperature control issues. But looking back on 2016, and especially looking forward into 2017, a few trends have emerged that will redefine the mobile landscape in coming years.

Looking back on 2016, a few trends have emerged that will redefine the mobile landscape in 2017.

Google is bridging its platform into all-new hardware categories

Based on advertising revenues, Google is a search provider. Full stop. Its myriad array of products and online services, including everything from the ubiquitous Gmail through its ubiquitous Android operating system, exist solely in service of making its digital ads ubiquitous. But in the past few months, Google has diversified its product offering to include a new family of hardware products, branded by Google and designed to more deeply integrate the Google brand into users’ lives.

Google was successful in establishing a deep and wide platform with Android. But upon that platform, Google is now positioning itself to build a hardware empire.

The company’s announcement of its Pixel and Pixel XL handsets underscore its prioritization of first-class smartphone user experience, but also represent its tacit admission that closely integrated hardware and software products add up to a user experience greater than the sum of the two. And beyond smartphones, Google is smartly readying its product line to dominate the platforms of the future.

First is Google Home, the company’s best impression of Amazon Echo writ large with machine learning and its decades of search intelligence. The connected home voice-assistant product category is just getting started, and is rumored to see a new entrant from the likes of Apple, but Google is making sure to establish a foothold within the conversations and aberrant questions that populate users’ lives.

But more interesting than Home is Daydream, and its dedicated View headset for virtual reality within Android. Pure virtual reality is an arena that many have ventured into — Samsung’s Gear VR, Sony’s PlayStation VR, and Facebook’s Oculus to name a few — but one that Google seems confident it can dominate by building on the ubiquity and ecosystem of Android. The VR–ready Pixel and Daydream mode of Android are only the beginning, but represent perhaps the best example of Google building on its strengths to segue into a new product category.

Google is confident it can use Android’s ubiquity and strong ecosystem to underpin its virtual reality ambitions.

Apple is balancing a superabundance of disparate platforms

The entirety of Apple’s product lineup seems mired in half-step limitations and holding-pattern compromises. The new iPhone doesn’t have a headphone jack, but next year’s promises to be a revolution. The new MacBook Pro is limited to 16GB of RAM by its dated Intel chipset, but Apple seems increasingly on the precipice of ARM processors in the Mac. And Apple TV’s fabled over-the-top streaming service is bogged down in negotiations with the cable industry, but Netflix is due to jump in at any moment.

This is a vision of an Apple trying to balance wide and disparate platforms — and failing to achieve that balance more frequently than they have in decades.

Of course, Apple’s revenues have been no slouch — the iPhone is still generating more profit for the company than any other smartphone maker, Apple Watch has become the top-selling product in its category, and iOS remains the mobile user experience that defines the digital lives of billions. But with its product iterations this year, more and more pundits are feeling as though Apple is losing the thread of the mobile revolution it started.

Whereas Apple was once the quintessential widget company, masterfully integrating hardware and software together to differentiate its products, its competition has begun adopting its integrated strategy to widespread acclaim. Meanwhile, the consequence of Apple’s runaway success and expanding product lineup is that it now maintains more platforms and services than ever before, with four full operating systems and a myriad of APIs, platforms, and even a full programming language it actively maintains.

With the A10 through the W1 chips, Apple is a company that now has an alphabet soup of processor families that nearly outnumber its actual product line.

But again, Apple’s perceived challenges might only be perception, the result of the company’s famous penchant for secrecy. Its rumored self-driving car project and numerous patents for innovative new device form factors might yet come to bear fruit in the next few product generations. But as Apple gradually evolves its product lineup with more subtractions than substance, many pundits are beginning to assume stagnancy rather than secrecy.

Microsoft is the new industrial design powerhouse

Microsoft reached unimaginable heights of dominance in the 1990s, building on its OEM partnerships and operating system lead to become ubiquitous within business and personal computing. But as mobile began to revolutionize the consumer electronics industry in the 2000s, Microsoft was slower to adapt, ceding ground to Apple and Google as traditional PCs lost relevance.

Part of Microsoft’s strategy in the late 2000s was to prioritize its popular Office and cloud service offerings, doubling down on the enterprise and building tight integrations into competitive operating systems like iOS and Android. But significant acquisitions of Nokia’s mobile hardware division and aggressive hiring of some of the industry’s best industrial designers have helped position Microsoft as perhaps the leading hardware design powerhouse for the next decade.

Strategic acquisitions and aggressive hiring have positioned Microsoft to be perhaps the leading industrial design powerhouse of the next decade.

This strategy has spawned a series of new hardware products from Microsoft in the past few years, formally starting with the Surface tablet in 2012. The company built upon propriety multitouch technologies from its experimental PixelSense tabletop computers, and pioneered the convertible form factor and detachable keyboard that many competitors—Apple’s iPad Pro included—have come to imitate.

With last month’s introduction of the revolutionary Surface Studio, a move clearly aimed at creatives and designers, Microsoft’s gray and magnesium-centric design aesthetic is carving out a niche for itself among industrial design aficionados. The import of this transition is that Microsoft is positioned to define the hardware form factors that Apple and Google will follow for their products — Microsoft debuts the Surface in 2012, Google and Apple introduce the similar Pixel C and iPad Pro in 2015.

While Google is keen to shoehorn virtual reality into its Android platform, Microsoft is hedging its bets between full-fledged virtual reality support in Windows 10 and more transitional — and potentially more useful — augmented reality with its HoloLens product. HoloLens blends the digital and physical worlds together with pioneering head-tracking and camera systems, allowing Windows to dynamically outline objects within physical reality on a transparent screen in front of users’ eyes. It’s a middle ground between a smartphone and full virtual reality, but could prove to be more practical than either.

HoloLens and augmented reality are a middle ground between smartphones and full virtual reality — but it could prove to be more practical and useful than either.

Just as the explosion of mobile upended the industry in the 2010s, trends are coming into focus that will define these giants’ product strategies in the next decade.

• Google is betting big on its platforms and cloud services, and beginning to build new hardware products on top of that foundation.
• Apple is committed to maintaining its massive library of products and platforms, but the opportunities for convergence or differentiation remain unclear.
• And Microsoft has taken the baton of hardware design innovation, while positioning itself not to miss the boat on the next big technology revolution like it did with mobile.

As these trends continue to accelerate, and as new players like Facebook and Uber begin to make enormous strategic moves, the 2020s promise to be a more volatile and interesting time for technology than even the 2010s were.

Pixel and Surface turn the electronics industry on its head

Originally published by Punchkick Interactive.

Pixel and Surface are turning the electronics industry on its head was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Until the introduction of Notification Center in iOS 5, receiving and managing notifications on iOS was something of a nightmare. For all its user experience differences, Android long held an advantage over iOS in terms of how easily users could interact with, organize, and dismiss the myriad messages and alerts we’ve come to receive throughout our days. In the ensuing years, Apple has evolved how its premiere operating system presents notifications from third-party apps, but has limited developers to titles and strings of text, reserving certain functionality — like the inclusion of image previews or high degrees of interactivity — to first-party apps like Apple Messages or Photos.

Interactive notifications in iOS 8 represented the first true enhancement to the functionality of notifications on the operating system, allowing third-party developers to surface a handful of options for user action following a swipe on a new alert. Receive a new chat message in Slack? Swipe to initiate a reply. Get alerted about a new comment on Facebook? Tap the “Like” button without leaving the lock screen. This simple functionality, coupled with a smarter Notification Center shade that helped users react to multiple missives over time, helped iOS stave off criticisms of its notifications system while Apple focused on other features.

But in a release that broadly opens components of the operating system to third-party developers — from the Messages and Maps apps to interactive widgets and access to Siri — Apple saw fit to reinvent what notifications themselves can be. From the types of content they present to how users can interact with them, notifications in iOS 10 represent an enormous opportunity for third-party developers to extend their apps’ functionality to the lock screen, meaning users can perform sophisticated actions or receive new types of information without ever having to open the app. Apple’s changes to notifications in iOS 10 aren’t to play catch-up with Android’s notifications system — they’re leap-frogging the competition in a substantive way that will change how we interact with apps for years to come.

What’s new in Notifications in iOS 10

In conjunction with Apple’s changes to the lock screen, making it more of a staging area for users to perform simple actions before they even unlock their devices, Apple has given developers new powers to make their notifications richer and more powerful than ever before. Notifications can now include complete views, just as an iOS app would, presenting users with discrete windows into the application from the convenience of the lock screen. Examples from Apple’s first-party apps include conversation threads from iMessage, previews of schedule availability from Calendar, and message content from an email in Mail. All users need to do is 3D Touch a notification (or use a new gesture, on older devices) to pop directly into a compact view of the app.

Notifications can now include complete views, just as an iOS app would, presenting users with discrete windows into the application from the convenience of the lock screen.

The value to users is obvious — no longer will they need to navigate their home screens, open folders, and launch apps just to take action on a simple notification. But the value this functionality presents to third-party iOS app developers has only begun to be realized, and could redefine the possibilities on the iOS platform as a whole. These notification views can include animations, images, and full-featured interactivity, blurring the lines between messages that the app sends you and the user experience within the app itself. In the past, brands and third parties could simply send a string of text to users, linking them to one piece of content within the app. Now, with rich notifications in iOS 10, developers can build apps that bring their functionality to you.

Now that users can perform sophisticated actions from within notifications on their lock screen, designers need to deliver a great mobile app UX without the app.

The intention is to accelerate users’ interactions with the operating system, making those two-second operations — which once required finding and opening an app — only a tap away on the lock screen. User expectations for mobile session lengths is shrinking as devices become more capable and as apps become more streamlined around common red routes. Moving key interactions out of the app itself and exporting them to a notification on the lock screen helps to ensure that users can more quickly and efficiently perform common actions with their apps, effectively trimming a multi-step process to only one.

Notifications, brought to you by the future of iOS

As apps on iOS transform themselves into new and different forms, the trend that began with iOS 8’s Extensibility features has reached a fever pitch. Apps don’t just exist within their traditional containers anymore. They’re fluid and extensible entities that use the connective tissues of iOS to permeate the entire user experience across the system. Using richer and more interactive notifications, just like the enhancements to notifications that came in iOS 8, eliminate the user experience friction of locating and opening apps, instead surfacing functionality and content immediately, wherever the user might be throughout the operating system.

Apps don’t just exist within their traditional containers anymore. They’re fluid and extensible entities that use the connective tissues built into iOS to permeate the entire user experience across the system.

The iOS Notification Center and lock screen are increasingly growing in prominence as part of the iOS user experience, demonstrating Apple’s insistence on Touch ID and 3D Touch as essential components of the overall iOS ecosystem. Notifications are perhaps the best case yet for iPhone 6s and 7 customers to begin taking advantage of 3D Touch, which is no coincidence — the more valuable and functional these hardware-dependent features become, the more likely Apple is to outperform iPhone sales estimates year over year.

With rich notifications in iOS 10, users don’t need to figure out for themselves what apps can do. Developers can build apps that bring their functionality to you.

But above all else, notifications in iOS 10 are exciting for developers for the same reason the first-generation implementation of push notifications were in 2009 with iPhone OS 3: unfettered access to customers’ attention. Notifications are an excellent mechanism to alert users to time-sensitive information, and the enhancements in iOS 10 only serve to enrich and enliven the type of information they can convey. As apps evolve to become less static, more impermanent, and more adaptive to our contexts, notifications are one avenue for brands to make their apps more immediate and useful. This also means that, as long as users engage with and act on notifications from their lock screen, the days of trying to eliminate those pesky red dots are over.

This piece was originally published by Punchkick Interactive. Visit punchkick.com for more about iOS, user experience design, and mobile app development.

Thinking post-app interactivity with iOS 10 Notifications

Thinking post-app interactivity with iOS 10 Notifications was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.

In a year when the new iPhone hardware design is a subtle refinement, holding out for the dramatic reinvention in 2017, the new version of iOS is the headline. This year, iOS 10 makes the platform competitive in new spaces, showcases the latest hardware improvements like Touch ID and 3D Touch, and pushes Apple’s machine learning agenda forward. And, most importantly, iOS 10 opens the platform to developers in all-new ways, giving access to new corners of the operating system that most closely impact the user experience.

We examined how developers can begin to take advantage of these new features, and how new frameworks like SiriKit, richer notifications, and the Messages SDK can help brand marketers build better experiences for their customers. Apple has become the kind of company that only makes big moves — and iOS 10 is one of their biggest yet.

Messages is the most heavily used app of any kind on iOS devices, growing in jumps and spurts over the years to incorporate a wide range of diverse features. What was simply called “SMS” in iPhone OS 1, handling nothing beyond green-bubble text messages, has grown to absorb nearly every conceivable means of communication from images and video to voice recordings, contact cards, GPS coordinates, documents, and more.

But Apple’s iMessage platform, while ubiquitous across iOS, macOS, and watchOS devices, exists in an era of accelerating competition. While Apple has sold a staggering 1 billion iPhones to date, Facebook Messenger has more than 1.1 billion users across iOS, Android, and web, and other messaging apps like WhatsApp, WeChat, and LINE report similar global usage statistics. The messaging ground has shifted under Apple’s feet since Messages saw its last major update in iOS 8, and the lock-in benefits of iMessage usage are as critical to Apple’s long-term strategy as ever. With iOS 10, Messages is ripe for something fresh.

So with its tenth release, Apple is opening iMessage conversations to apps, introducing a specialized App Store and a range of new app categories for experiences that can coexist with iOS users’ conversations. The Messages SDK represents the opening of one of iOS’s most prominent and popular arenas, and a strategic shift in terms of how developers can interact with the iOS platform. iOS 10 fundamentally changes what “apps” can be — and in so doing, effectively obviates a pocket industry of sticker- and GIF-focused third-party keyboards from the App Store.

Introducing the Messages SDK

The Messages SDK in iOS 10 is a new framework that allows third-party developers to create new kinds of apps that exist within iMessage conversation threads. These apps typically replace the on-screen keyboard with a custom interface, allowing users to select objects or content to insert into their conversations in place of ordinary text messages.

But while iMessage until this point has seemed best-suited for emoji sticker packs or copy-and-paste GIF generators, typically delivered in the form of a third-party custom keyboard, the Messages SDK opens up the possibility for entirely new types of apps to populate users’ conversations. Sticker packs are still there — and easier than ever for artists and designers to create — but there are new collaborative types of apps that allow users on both sides of the conversation to interact with content within the message thread. It’s a dramatic reinvention of how apps can deliver multi-user experiences, and introduces a new real-time collaborative mode that many types of third-party apps are perfectly suited for.

One example of this functionality is a restaurant menu, where multiple members of a group iMessage thread can add their individual dishes to the order in real time. Another is purchasing movie tickets, where several members of the conversation can select showtimes and seats simultaneously, ensuring they can all sit together. The Messages SDK gives developers access to the message thread in real time, and logs each users’ contributions to these shared sessions as events within the iMessage timeline.

It’s not entirely a novel idea to have users interacting with the same content at the same time, but it’s a feature that has never been so elegantly implemented.

This type of shared-session real-time collaboration is certainly possible within the contexts of native apps, but required a lot of the users — they each needed to download and use the same app, somehow coordinate and schedule a time to work together, and likely enter a cumbersome code or password to ensure they were in the same session. Now, one user simply initiates the app and everyone in the iMessage conversation can jump in within seconds.

Important to note: only one user needs to initiate the iMessage app. The others don’t need to have downloaded the app from the Messages App Store themselves, or even necessarily heard of it. Messages helpfully places a Download call-to-action beneath any messages contributed by third-party apps, allowing users to jump straight into an experience their friends have implicitly shared with them.

While the most common and likely most widely used iMessage apps will be these reimagined sticker packs — which Apple has helpfully included as an out-of-the-box template in the new version of Xcode — the nature of conversation threads introduces an all-new opportunity for different kinds of mobile experiences. Brands whose customer experiences are intended to be enjoyed by groups — think splitting the bar tab, agreeing on a restaurant reservation, picking a movie showtime, holding a fantasy football draft, making a shopping list, and uncountable other examples we can’t even imagine yet — there’s a new outlet to deliver them to more users than ever before.

Customer experiences that are designed for groups — splitting the bar tab, agreeing on a restaurant reservation, ordering group takeout, picking a movie showtime, scheduling a meeting, holding a fantasy football draft, making a shopping list, and more — are perfect for the Messages SDK.

You have a new message from a brand — and you’re weirdly cool with it

iMessage apps represent an opportunity for brands to offer real value to customers in an arena that simply didn’t exist before. Designing custom sticker packs to effect brand affinity is one thing — and a silly trend that far too many brands will probably fall victim to. But reinventing a pain point of consumer software that’s as old as the web — how to get all your friends doing the same thing at the same time — will genuinely help users in their everyday lives, and become important differentiators in an increasingly competitive mobile landscape.

Solving a consumer pain point as old as the web — getting all your friends doing the same thing at the same time — will genuinely improve everyday users’ lives.

Further, the nature of iMessage apps’ growth strategy is significant. While iMessage apps can exist solely on the Messages-specific App Store, or be bundled with a larger iOS app distributed through the primary store, Messages is a new organic avenue for new customers to discover amazing iMessage apps. With the call to action that helpfully prompts users to download the apps their friends use, the best iMessage apps can spread virally, offering developers with the most useful ideas an unprecedented level of discoverability.

The mobile experiences that matter are those that legitimately impact their users’ lives — and the Messages SDK offers a toolset for developers to do exactly that. And no, that doesn’t mean sticker packs. Users will come to expect this level of real-time, social, shared-session collaboration before long, and the apps that adopt it first and best will reap the viral rewards. In many of these cases, until brands rethink their mobile apps take advantage of this new type of experience, their connection to users will feel like a holding pattern, an ellipsis within a blinking speech bubble that feels like the beginning of something amazing. And that real-time collaborative future is the long-anticipated message users don’t even know they’re waiting for.

This piece was originally published by Punchkick Interactive. Visit punchkick.com for more about iOS, user experience design, and mobile app development.

What are iMessage apps, and how can brands join the conversation?

How apps can use iOS 10’s Messages SDK to join the conversation was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.

In a year when the new iPhone hardware design is a subtle refinement, holding out for the dramatic reinvention in 2017, the new version of iOS is the headline. This year, iOS 10 makes the platform competitive in new spaces, showcases the latest hardware improvements like Touch ID and 3D Touch, and pushes Apple’s machine learning agenda forward. And, most importantly, iOS 10 opens the platform to developers in all-new ways, giving access to new corners of the operating system that most closely impact the user experience.

We examined how developers can begin to take advantage of these new features, and how new frameworks like SiriKit, richer notifications, and the Messages SDK can help brand marketers build better experiences for their customers. Apple has become the kind of company that only makes big moves — and iOS 10 is one of their biggest yet.

While virtual assistants and chat bots are common today, it’s surprising to remember that Siri was one of the first of its kind when the iPhone 4s launched in 2011. When Siri was first demoed, it was a novelty, fielding questions and requests that controlled many components of iOS 5 and integrating data from sources like WolframAlpha and Major League Baseball. Today, as virtual assistants have come to populate the other operating systems and services billions of people use every day, Siri has aged gracelessly, earning a somewhat sour reputation for unreliability and limited utility.

Since Siri’s introduction, the chat bot–centric model called conversational UI has exploded in popularity, with new conversational interfaces sprouting up in Google Assistant on Android, Cortana on Windows 10, and within dozens of apps and services integrated with Facebook Messenger, Skype, or Slack. While Siri computes its responses server-side, Apple’s commitment to users’ privacy has prohibited the company from collecting user data at any meaningful scale for many years, making Siri slower to evolve than some of its competition.

Now, with iOS 10, Apple has decided to act. Responding to increased competition from the likes of Amazon’s popular Alexa, which inhabits its Echo connected home hardware and allows for powerful third-party integrations from a variety of apps and services, Apple has turned to its expansive and active developer community to give Siri the legs it so desperately needs. Enter SiriKit, a new developer framework that will allow users to interact with certain kinds of apps through Siri — and which fundamentally transforms how Siri fits into Apple’s ecosystem strategy forever.

What is SiriKit?

SiriKit is a new iOS developer framework that allows third-party apps from the App Store to integrate their features with Siri. Using SiriKit, certain types of apps can offer an extension that teaches Siri how to pass along information from users’ voices, extending the app’s functionality into new corners of the operating system. Since iOS 8, users have been able to access app functionality from other parts of iOS using extensions in Notification Center or the system share sheet. With iOS 10, SiriKit is another type of app extension that lets any app integrate a conversational UI.

SiriKit is an app extension that allows any app to add a conversational UI.

At launch, SiriKit will be compatible with a limited range of app types — this is because Siri will need to gradually learn the unique syntax and sentence structures associated with new kinds of apps over time. Apple has said it’s committed to adding support for new types of apps to SiriKit over time, but on day one iOS 10 will support the following kinds of apps.

• Messaging apps. From Slack to WhatsApp, users can compose and respond to messages in apps that support SiriKit.
• Ride sharing apps. Apps like Uber and Lyft can be summoned to request a ride from anywhere in iOS using Siri.
• Photo library apps. Apps that store and organize users’ photo collections can be searched using Siri.
• Peer-to-peer payment apps. Use Venmo or Square Cash to pay friends back for dinner? Now Siri can handle those requests without opening the app.
• Workout apps. Apps that report workouts to HealthKit can now start, pause, or stop workouts using Siri.
• Climate control and radio apps. In CarPlay-compatible vehicles, drivers can now adjust the air conditioning or the radio station with their voice.

The reason for Apple’s limited range of Siri interaction options is simply a problem of syntax. Siri can’t possibly learn how to field every type of app-facing user request overnight — especially considering the user privacy limitations Apple altruistically places upon itself. For brands whose apps fall into any of these categories, it’s open season for adding SiriKit functionality in iOS 10. (For those whose apps don’t quite fit into these categories, sit tight — Apple is almost certain to broaden compatibility in iOS 10.1, 10.2, and so on.)

Developers need to offer the unique vocabularies of their apps to Siri, and the intelligence of SiriKit can handle the rest. First, they specify the types of “intents” that their apps can field — like sending a message. Then, they identify the different entities that will vary from user to user, including public and private data types.

For example, a messaging app might use a string that includes an intent and various entities, like “Hey Siri, send a [message type] to [contact name, private user data] using [app name].” Once SiriKit has an understanding of how each app’s requests are structured, it can appropriately route the request.

Hey Siri, send a [message type] to [contact name] using [app name].

Hey Siri, send a text to James using WhatsApp.

Giving every app a voice.

For third-party developers, every new extension point in iOS represents an opportunity to extend their apps into other parts of the operating system, reducing the distance between users and the apps and services they love. But unlike extensions in years past, which relied on users manually enabling them or adding them to a queue of similar widgets, SiriKit requires zero setup on the user’s part to become useful. They just install the app, summon Siri as they always have, and go.

SiriKit, like many of the new frameworks Apple has introduced in the past few years, is another argument in favor of building iOS apps with native code. Android apps can’t take advantage of SiriKit — and unless your app is developed independently for each platform, it likely can’t either. By developing apps for iPhone and iPad with Objective-C or Swift, rather than a lowest-common-denominator development platform that targets multiple operating systems, developers can ensure their apps are ready to take advantage of SiriKit on day one.

Android apps can’t use SiriKit. And if your iOS app is built for the lowest common denominator across both platforms, it probably can’t either.

Even if developers’ apps aren’t supported by SiriKit’s six app types on day one, brands should begin thinking about how Siri could interface with their apps now. Voice-based interfaces are increasingly looking like the future of mobile user experience, and with new products like Amazon Echo and Google Assistant launching every month, that future is fast approaching. SiriKit is the first note of what’s sure to be an opera of Apple’s endeavors with voice-based interfaces. By taking advantage of the new framework today, developers can ensure their apps are part of the ensemble.

This piece was originally published by Punchkick Interactive. Visit punchkick.com for more about iOS, user experience design, and mobile app development.

SiriKit in iOS 10 gives third-party app developers a voice

SiriKit in iOS 10 gives third-party app developers a voice was originally published in Punchkick Interactive on Medium, where people are continuing the conversation by highlighting and responding to this story.