This article is the last chapter of a series about the design of Facebook Spaces.

A new type of interface

One of the biggest challenges for our design team was to design the user interface of Spaces. Unlike with traditional web, desktop or mobile design where we can rely on existing UI elements and interaction patterns that people have learned over the years, most of those patterns have yet to be invented for VR.

An important goal of our interface design was to make sure the UI was here to serve the conversation, not get in its way. While we could have had interfaces that take full advantage of the space and isolate people while they focus on performing an action, we instead looked for interfaces and interactions that empower people to reach their goals while still being in the space with their friends and continuing their conversation. We wanted the UI to feel transient and useful, not interruptive.

To help people distinguish “real” objects and people from the user interface, we adopted early on a holographic visual language for our UI. This not only allowed us to visually separate interactive elements from the rest of the environment, it also allowed us to be more flexible with physics and temporality for our UIs, as people already have a preconception of holograms being lightweight and ephemeral.

Throughout our progress building Facebook Spaces, we came up with a few interfaces and interactions that are worth mentioning. Some of them didn’t turn out successful and had to be thrown away, while some worked out better. In the next few paragraphs, I’ll share some of the concepts we explored to solve some of our problems and the learnings we made.

Virtual Table

During our early days, we built prototypes in which people could freely roam around a space. We tried different solutions to the locomotion problem and tried different types of space. We quickly realized one of the biggest problem with that approach: it was hard to actually have a conversation with someone. When able to freely move around, people tended to get lost and weren’t really interacting with each other.

In another experiment, we built a tool in which people could create music together. In its first iteration, people would be side by side, looking at a giant keyboard-style instrument. While really fun and playful, people were never really looking at each other and talking.

A quick observation of how we tend to operate in the real world when spending time with people having conversations pointed us at the essential element we were missing: a virtual table, to anchor the conversation and create a social construct.

At F8 2016, our first pass at a table was a rectangular tray people would stand around and on which they could pick up/drop objects. We also played with different sizes and configurations (square, rectangle, large, small, solid, transparent, etc). The solution that felt best for us was a circular holographic table that can sit 4 people comfortably.

Main Navigation

VR Watch

One of the first interfaces we worked on was our main navigation UI. Our first prototype was a virtual watch that would be attached to your virtual hand and contain the top level menus of our app. While very discoverable and cool, we learned how hard it was to aim at buttons attached to a moving hand. It was also quite tiring for the arm, as it required people to raise their elbow quite significantly to make the interfaces readable and reachable. This indicated to us that the watch could be a great place to surface short-length temporal information with limited interaction (for example, notifications) but wouldn’t feel as good for complex interfaces like a main navigation system.

VR Tool-belt

Attached to your body, the tool-belt felt very personal. Appearing the moment you’d look down at it, it would allow you to make selections or grab objects with one hand, which felt pretty nice. But after a few days of using it, we realized how uncomfortable it was on the neck to repeatedly have to look down at the tool-belt. It was also challenging for people using the app seated who would keep hitting their legs when using it.

VR Buddy

The idea behind VR Buddy was to provide a single object you could move around and place wherever you wanted it in the space. Hitting that object would open a radial menu around it, giving you access to its submenus.

The benefits of this approach were multiple:

• You could manipulate the UI with one hand and it was anchored to the world, allowing you to be precise and take your time.
• You were also able to choose its position in space, allowing you to bring it with you when you were drawing on the side, etc.
• Moving it was quite fun and delightful, especially when the submenus were open as they were placed on a spring and were lagging behind the core in a fun and bouncy way.

But the drawbacks were also multiple:

• The radial menu approach meant having to wave your hand in the air multiple times to hit the different submenus and get to what you wanted. This created fatigue in your arm and was quite inefficient.
• Opening up a 2D interface meant either having you turn around to face it, or opening it in a weird position in which we couldn’t ensure your comfort.
• The fact that you could move it around also meant you could easily misplace it and have to look for it.
• It was hard to build a spatial model/muscle memory of how to use the UI as you were regularly changing its position and orientation.

VR Dock

With the learnings from those past explorations and a few others, we designed the first version of what we called the VR Dock: a 3-dimensional interface anchored to the table in front of you that lets you access your content easily and quickly without having to move around too much.

This solution had the same benefits as the tool-belt without the main inconvenient of forcing you to look down too much. The main concerns with this were scalability and the amount of space this was occupying on the table. The floating orbs were also getting in the way of you interacting with the objects on the table.

VR Plate

The idea with VR Plate was to incorporate an interactive surface on the table with an outer ring and an inner ring as a way to structure the information. The inner ring could also serve as a region for delivering 3-dimensional objects like a selected tool or media.

Similarly to the first VR Dock, the main drawback of this exploration was the amount of space used on the table, which we considered solving by adding a open/close button near the edge of the table. Another problem was to communicate the origin of any UI that would come out of a selection you’d make in that interface, which wasn’t easy with this concept.

VR Dock 2.0

The solution we shipped with was very much a marriage of the first Dock and the Plate: a 3-dimensional Dock with a 2D interface on it, that can project a UI or objects in front of you, and can be closed down into the table when you don’t need it.

This solution worked out for a few reasons:

• People seemed to immediately know how to interact with it. The resemblance with a tablet was a signal for people that they could touch it with their fingers to make a selection.
• The projector metaphor worked really well to communicate the origin of the UIs, how to turn them on/off or how to switch between them.
• The minimal amount of real estate occupied in front of you gave you a feeling of control and it didn’t feel in your way.

Display Center

Another problem that we had to figure out was how to allow people to show media content to the other people in the room. From our first prototypes, we knew that turning a selected media into a 3-dimensional object felt right. Holding a photo or a 360 orb in your hand feels real, and passing it to a friend feels natural and delightful.

But we also needed a way for people to display media big, where everyone in the room can comfortably see it. Our early interaction was to let you stick a 360 orb to your face to activate. While pretty delightful, it wasn’t very discoverable and we saw some people hitting their headset with their controller the first few times they’d try. We also felt the need to be able to display a 2D photo or video big where everyone can comfortably watch it.

We called the solution we came up with the Display Center: a circular zone at the center of the table, which activates when you drop a piece of media inside of it and automatically starts displaying that media. A 360 photo or video suddenly becomes the environment, while a 2D media starts projecting on a large screen.

Once projecting, people can control their media from a distance using their dock. They can also end the projection, either from the dock or by taking the media out of the Display Center.

Wrist UI

Another challenge was to figure out a way to interact with a particular object in the space. After a few concepts, we created the Wrist UI, a contextual interface that appears on the inside of your wrist when holding an object and displays the action buttons you can use for that particular object. It’s the equivalent of a right-click on a desktop computer.

This interface worked out pretty well, as it was very discoverable and easy to understand and use. The biggest challenge with it was to make it feel visually okay to have a UI floating off your arm. The hologram metaphor turned out very useful in this case, as seeing a hologram projected on your forearm feels much better than a solid object attached to your invisible forearm.

VR Watch 2.0

With our first prototype of the VR Watch as a main navigation system, we learned that the concept of a virtual watch should probably be kept for fast, easy interactions, to prevent you from keeping your arm in mid-air for a long period of time. We also learned that flipping your wrist to access information felt fast and magical.

With those learnings in mind, we rethought our watch interface to serve as a notification center. When a friend sends the person a message on Messenger, tries to call them or sends them an invite to join their space, their controller vibrates and a sound comes out of their watch, catching their attention. By a flip of the wrist, the person can read the notification and take a quick action.

An evolving system

Together, those different interfaces allow people to find their friends, access their content and tools, display their media to the people in the space, and stay connected with the outside world. They’re just our first attempt at solving some of those problems and it’s likely that most, if not all of them will eventually be replaced by more robust solutions in the future.

The journey ahead

Building this first version of Facebook Spaces has been the most challenging, fun and rewarding time of my career. I’ve gotten a chance to tackle something both crazy ambitious and very fun with a team full of talented and passionate people. Designing for VR is both frightening and exciting, as you can’t rely on the skills you’ve been perfecting over the years and have to get out of your comfort zone to go acquire new ones. And designing social VR experiences turned out to be extra challenging, as the feeling of presence and immersion is very strong and the opportunities and challenges are numerous.

But as with everything, both for Facebook Spaces and for VR as a whole, this is just the first step of a long journey ahead and I’m grateful to be a part of it.

I’d like to thank everyone on the Spaces team for their incredible contribution to making this a reality. In particular, I’d like to thank Mike Booth for allowing me to embark on this crazy adventure, Charlie Sutton for his wisdom, talent and support, Rachel Rubin Franklin for her devotion and leadership, Mike Schroepfer and Mark Zuckerberg for their trust and support, Brad Dickason and Alicia Berry for helping us cross the finish line, and the entire immersive design team that helped forge this experience: Chris No, Katya Kostyukova, Gabriel Valdivia, Cliff Warren, Stephanie Engle, Richard Emms and Brian Perone.

If you’re interested in VR design, you can find more resources at facebook.design/vr.

You should also check out “The Making of Facebook Spaces”, by Mike Booth.

This article is the third chapter of a series about the design of Facebook Spaces.

Social presence

One of the new cool buzzwords you can hear in every talk about VR is “Presence”. The reason is quite simple: the goal of any virtual reality experience is to immerse you in a world that is different from the physical world around you and make you believe you’re there, present in that reality. A true feeling of presence is the ultimate goal of VR.

Presence is a phenomenon enabling people to interact with and feel connected to the world outside their physical bodies via technology.

Anything that reminds you that what’s around you isn’t real breaks that feeling of presence. That’s why it’s important for designers of immersive experiences to be on the constant lookout for anything that can break the magic.

With social VR, the challenge becomes even more interesting: not only does your experience need to make people feel present in a space, but it needs to make them believe their friends are also physically present with them, no matter where they are in the World.

This becomes particularly challenging when people start having real conversations with their friends in VR. Quickly, they start laughing, are surprised, or amazed by something they see or say. Today’s VR headsets aren’t able to detect facial expressions and the avatar face stays neutral, creating a distance between the avatar and the person animating it.

To help overcome this problem, we created a series of avatar emotes that people can use to express their emotions through their avatar. By performing certain hand gestures (e.g. hands in the air for yay!), or by using the joysticks on their controllers, people can make their avatar smile, act surprised, scared, etc.

A lot of our work over the last year has been focused on finding the key ingredients to enable that feeling of social presence. Here are some of the things we’ve found to be contributing in a meaningful way:

• Hand gestures — Seeing your friend’s hands moving and matching their real hand gestures plays a crucial role in making you believe they’re with you.
• Eye gaze — Even if the current VR headsets don’t support eye tracking, animating the eyes of the avatars of your friends to indicate where they’re looking is crucial. It also allows you to make eye contact with a friend, which is truly visceral and special.
• Lipsync — When a friend is speaking, animating the mouth of their avatars in real time, combined with spatialized audio helps you believe you’re looking at your friend and not an abstract representation of them.
• Facial expressions — Adding facial expressions like smiling, laughing, being confused… to the avatar faces makes them feel alive and believable. It also makes for great selfies!
• Distance — Interpersonal distance plays a crucial role in building social presence. If your friends are too far or too close from you to chat, you stop believing you’re having a normal and real conversation with them.
• Self reference — Being aware of what you look like in VR plays a really important role too. Having a mirror or selfie stick in the space makes people feel even more present with their friends.
• Hand contact — Being able to grab a friend’s hand or high-fiving them is incredibly powerful and makes you really forget they’re not physically close to you.

If you’re interested in the particular aspect of designing for hands in VR, I wrote a note a few months ago that shares some of the learnings we made.

A safe space

With Facebook Spaces, we focused a lot of our attention towards creating a space in which people would feel safe and included. We chose to limit the experience to small groups of 4 people and to only let your friends join your space.

Because VR is so new and different, people want to understand what they can and can’t do. When seeing a friend’s avatar next to them, people want to try to touch it, to see how it feels. Or they want to test what happens if their avatar collide with their friend’s. While this may be done with a harmless intention, we still wanted to make sure everyone always feels comfortable in their space.

For those reasons, we created a personal safety bubble that acts as a shield and protects you from anything or anyone entering it. For example, if a friend tries putting their hand in your face, their hand will start disappearing, and they’ll see the edges of your shield. If they get too close from your body, your entire body will disappear to them and theirs to you. This pattern has already been tested in a few other VR experiences as a way of creating some safe boundaries between people.

Another important element of creating a safe space is giving people the tools to control their experience.

Whenever people want to take a break from their experience, whether that’s because the pizza delivery guy is knocking at the door, there’s a destabilizing shaky 360 video around them, or they’re feeling uncomfortable, they can pause their experience by pressing the pause button located on the inside of their wrist, or by taking their headset off. This teleports them out of the space momentarily and into a “paused space”, giving them a chance to catch their breath and take action if needed (resetting the space, muting/kicking people out, reporting content…).

Building a safe experience also means protecting people from some of their own fears: to prevent people afraid of heights from experiencing vertigo in certain 360 photos or videos, we designed a platform that appears whenever you look down.

Doing anything, anywhere

Once we’ve successfully managed to connect people with their friends in a safe virtual space, it’s important for us to provide them the tools to enrich their conversation and ensure they’re having a great time together.

One of the core aspects of the Facebook Spaces experience is the ability to access the same photos and videos you’re used to seeing on Facebook and show them to your friends in the space. This means you can watch the latest Star Wars trailer together or show your friends the photos from your recent vacation trip.

It also means you can explore the countless amazing 360 photos and videos available on Facebook. When you display one of them, it feels as if you had been teleporting where that photo or video was taken. You can now be together on Mars, front row at the stadium, or swimming underwater with sharks. The feeling of presence is real.

The challenge was to design a user interface that would allow people to access and navigate all of that great content while continuing their conversation. While people are used to certain paradigms and interactions on 2D screens (grids, scrolling, selecting, etc), finding ways to create a similarly intuitive browsing experience in VR took quite a few iterations. The result is an intuitive interface that immediately feels familiar. 360 content appears as mini-windows into a new world, leveraging your head motion to create a parallax effect that helps you preview the content.

Alongside exploring media content together, Facebook Spaces allows people to express their creativity and have fun. Using our 3D markers, people can draw in 3D and create objects and decorations.

Those drawings can be used for anything and can also be attached to an avatar, allowing people to decorate themselves. While we could have added countless advanced features and tools to it, we decided to keep our marker as simple and approachable as we could, leaving it to people to be creative within its constraints.

Bringing everyone in

While being present in a VR space with a friend is where the real magic is, we were aware that some people don’t own a VR headset yet. As VR continues to grow, we — as an industry — need to find ways to make VR approachable and attractive. One good way is to try to break the wall between VR and the real world and allow people to share their VR moments with the rest of their friends.

The first feature we built is a selfie stick, which allows people to capture their best moments in VR. This feature quickly turned out to be one of people’s favorites. When holding the selfie stick in VR, people often end up smiling in real life, demonstrating the strong attachment to their avatar. Once captured, people can immediately see the selfie they took and share it with their friends on Facebook via a simple tap of a button.

The second feature we built is a way for people to video call their friends on Messenger. This makes it possible for them to share a VR moment with a friend who doesn’t own a headset or is on the move. A simple tap on a button initiates the call, and the friend on Messenger will see the caller’s avatar and the space around them. This makes it possible for the person in VR to show their friend what they’re seeing and doing, giving them presence in the space.

This is the second chapter of a series about the design of Facebook Spaces.

Being yourself in VR

If we were to build a people-first platform, we were going to focus our efforts on understanding what makes you feel truly present with someone else in VR. Very early on, we decided to build an experience for the Oculus Touch controllers, which were providing the highest fidelity VR experience. We were particularly interested in exploring what hands and hand gestures could provide in a social context. Our early research demonstrated that people were able to recognize their significant other from other people in VR just by recognizing their hand motion.

The first challenge was to figure out how to represent people in VR. While the first instinct is to try to represent people exactly like they are in real life, in very high fidelity, we had to find a way to avoid falling into the uncanny valley.

We played around with different types of avatar systems and different visual treatments. A lot of explorations that seemed interesting in theory turned out pretty disturbing or disappointing in VR.

One of our first explorations that ended up being pretty successful and delightful was what we called the Rabbitars, some very cute and expressive rabbits with flopping ears. With the Rabbitars, we learned the importance of a torso and arms in making you feel like the avatar next to you was present in the space. Since only the head and hands were tracked by the hardware, we used a technique called inverse kinematics to predict where the arms and torsos would be. While this technique doesn’t work really well when applied to your own body, we found that it works surprisingly well when looking at other people in a social context. Another really positive aspect of those avatars was how expressive their flat textured eyes were.

On the other hand, while very charming, those rabbits were far from ideal when having a conversation with friends. The moment you’d be with multiple people in a space, you were constantly looking around to figure out who was talking and it was hard to remember who was who. You were constantly reminded that you were not really looking at your friends but just at an abstract representation of them.

With every iteration, we learned new things that informed the next one. Here are some of the things we learned along the way:

• To achieve the feeling of being present with your friends in VR, you need to be able to bring some elements of your real-world identity.
• Attaching body and arms to head and hands makes your avatar feel more realistic and give it presence and volume in the space. Even if the current generation of VR headsets don’t track the body and arms, generating them based off the hand and head data goes a long way.
• Adding motion to certain elements (hair, for example) when a person moves gives the avatar more life. For example, your glasses can move up and down on your nose when you shake your head.
• For eyes, 3D eyeballs are a lot of work, and they are not as charming as we thought they were. We found that 2D eyeballs were easier to design for emotes (e.g. laugh) and more inviting/less intimidating.
• When starring at an object or person, our eyes rarely stay still and wide open for more than a few seconds. Adding a fake “look away” intermission and some fake blinking helps make eyes feel alive.

In the end, we landed on very stylized avatars that act as a cartoon representation of yourself. Charming and inviting, those work really well within today’s constraints of VR systems and allow expressiveness and self representation without falling into the uncanny valley. This style also means the avatars are quite flattering and tend to be more of an aspirational version of who you are.

Creating your avatar

Alongside figuring out what our avatars would look like, we had to design how people would create their own. Immediately, our instinct was to look for ideas that would prevent people from having to do a step by step, pick a facial feature after the other, type of interaction. Too many games and experiences force you to spend 20 minutes painfully creating an avatar you’re not sure yet how you’re going to be using. And with VR, there’s the extra difficulty that you need to remember what you look like in real life, as you have a headset on your face.

Thankfully, the advancements in AI over the last few years made it possible for us to create an experience in which we can automatically create an avatar for you out of one of your Facebook photos. This makes the process of creating your avatar blazing fast and fun.

The moment we started testing our algorithm, we realized it was going to play a big role in the success of our product. The experience of creating your virtual self suddenly felt simple, fun and magic instead of clunky and time consuming.

Because so much of our understanding of what we look like and how others perceive us comes from us looking at ourselves in the mirror, we designed this experience around a virtual mirror in which people can discover their new virtual face and customize it. The floating photos around the mirror serve as a reference point as people tweak their avatar, but can also be picked up to generate an avatar automatically out of the selected photo.

To edit, people can select a facial feature they’d like to change by simply tapping on it on their reflection, and a set of alternative options appear on the mirror for them to choose from.

For choosing special elements like clothing and glasses, we came up with the concept of a virtual magazine in which people can pick something they’d like to wear and it automatically applies to their avatar.

The simplicity and fun of this experience combined with the expressiveness and friendly look of the avatars made our avatar creation experience something that feels fun and magical.

This article is the first chapter of a series about the design of Facebook Spaces.

On April 18th, we introduced an early beta of Facebook Spaces, Facebook’s first social VR experience, on Oculus Rift. With Facebook Spaces, you can spend time with your friends in VR, no matter where they are in the world and feel like they’re here with you.

A people-first computing platform

In early 2016, I joined the newly formed Social VR team as its first Product Designer. The goal for that team led by Mike Booth was both ambitious and exciting: explore the future of social interactions in Virtual Reality. As I stepped into this new medium, I quickly realized how incredible its potential was: it could be a huge step in our journey towards making people feel more connected and closer to each other and to the world around them.

It also became clear to us very quickly that the Facebook experiences would have to be reimagined to double down on what this new platform could provide.

As Mark Zuckerberg described in his keynote at Oculus Connect 3, we’ve spent decades building software platforms that are organized around apps. When we pull up our phones, we see a grid of apps. We have apps for making phone calls, apps for sending messages, apps to browse the web, apps to order food… But unsurprisingly, the ones we spend the most time with are the ones that let us connect with our friends. This way of organizing software around apps doesn’t align really well with our human nature and the way we process the world around us. This doesn’t align with how, as humans, we approach the world. But there is a unique opportunity for VR to be a people-first computing platform. A platform in which people are at the core of the experience, with a true feeling of presence, and where they can choose to do the things they want, together.

Designing for VR

Designing a virtual reality experience is both intriguing and daunting: few of the interaction and visual design patterns we’ve been crafting and polishing over the last decades are immediately applicable to VR. The industry is in its early days, it is a time of exploration and experimentation. Some of the interactions and design patterns we are inventing now may become standards in just a few years, or become completely obsolete as we make more progress and learn.

Facebook Spaces is the result of months of those experimentations. A lot of trial and error combined with a constantly refined design process. But even if VR is still uncharted territory and we don’t yet have all the processes and tools to help us explore it, some of the tools we have crafted over the years for software design are still useful. One of the most significant one is to define your design principles early on in the project, as they will guide a lot of your day-to-day decisions. We early on defined our design principles as follows:

• Promote Agency — People should always be able interact with the content and environment around them and feel like they’re in control.
• Embrace Asymmetry — VR interactions should work asymmetrically when possible (i.e. have a counterpart in the 2D world / with different input systems). Connecting with people both inside and outside of VR should be celebrated.
• A Safe Space — Give people the tools to feel safe.
• Feels like Facebook — The experience should feel new and exciting, yet familiar and have Facebook character.

Another aspect of our traditional design process for building new products or feature at Facebook that still applies to building VR experiences is to answer those three questions:

• Who are we designing for?
• What is the people problem we’re trying to solve?
• What does success look like?

Doing this exercise helped the team understand what we were trying to achieve and what our high level goals should be, which allowed us to stay focused on the right things along the way. We decided to focus on delivering an experience that would allow you to:

• Be “in person” with your friends regardless of where they are
• Experience meaningful content together
• Spend quality time together doing things you enjoy

Process & Tools

Once we have a better understanding of what we’re trying to accomplish, we move on to exploring how we can solve the problem. With VR, exploring that spectrum of solutions is often done with traditional design tools. Whether it’s by drawing on paper, creating a mockup in Sketch or Photoshop, or making a prototype in Origami or After Effects, we try to visualize some of the different ideas we have for interactions or interfaces. This allows us to move quickly and explore a large breadth of concepts and solutions.

But unlike with traditional software design, a lot of the ideas and concepts that seem very powerful and effective in mockups or videos can end up very disappointing or ineffective once in the headset.

• 2D mockups don’t always give a good perspective on the spatial context and ergonomics of the interface or interaction you’re designing. Is this interface 2m large and 1m away, or 1m large and 50cm away? How far do I need to reach out to grab this object or touch this button?
• It’s often hard to predict how an interaction will feel when using virtual hands, in a virtual world. Swiping windows in the air ala Minority Report looks cool in a movie, but when you can’t actually feel the windows at the tip of your finger and your hand goes through them, how effective is that solution?

One of the important lessons when designing for VR is that you should move your idea or concept as fast as possible to prototyping. Trying the interaction or interface in headset gives you a far better understanding of whether or not the solution will solve the problem.

Our team developed Facebook Spaces using Unity, a robust game engine used by many to develop 3D applications and games. Unity is also fairly approachable to designers as it offers a powerful editor that allows them to visualize the environment and make changes to every object without having to jump into the code. You can try your scene, make changes and see them reflected in the headset in real time, which is very useful when trying to fine tune the position, size, physics of an object or interface, for example.

We early on decided to do our prototyping in Unity as well. This means we can not only test our ideas rapidly, but can add them to our app and test them in a social context, with other people in a space with us. To move fast on complex prototypes, designers often pair up with engineers with the goal of standing up the core of the idea quickly to test it.

Another important aspect of making a concept come to life is creating the 3D assets and effects that will support the feature or interaction. Designers often pair up with 3D artists who can model, texture, animate or create the effects that will help craft the solution.

The last part of this process is to test those prototypes with as many people as possible to validate that the solution works with everyone. For us, this means having as many internal employees as possible trying it out and giving us feedback, as well as bringing external people in our research labs to understand how they interact with that feature. It also means looking at how people use the feature once launched to understand what works and what doesn’t.

Building a VR team

One of the biggest challenges of building Spaces was to assemble the team that would solve those new types of problems. Traditional software companies like Facebook have been crafting their team structures and hiring process over the years to facilitate agility and empower people to express their talents. But VR requires new types of skills that are hard to find.

A lot of our time while building Spaces was spent identifying those necessary skills and finding that talent: 3D art generalists, character artists, prototypers, real-time game engine experts… Different industries have developed and leveraged those skills in the past (movie and video game industries, for example). We had to search for those talents, get them excited about our mission, mix them with more traditional software designers and engineers and create the right process for them to collaborate and learn from each other. In the end, we built a team of incredibly talented people with diverse backgrounds and perspectives which allowed us to tackle those new types of problems.

The next step in building natural human-machine interactions

Earlier this year, I joined the Social VR team at Facebook. The goal of our team is to explore how virtual reality can help create new rich and meaningful ways of connecting with the people you care about.

In October, Mark Zuckerberg gave a live demo of some of our recent explorations during the Oculus Connect keynote.

VR is still very much a fragmented and diverse medium with dozens of different headsets and input controllers. Our team decided to focus our work on the recently released Oculus Touch controllers. We recognized early on that bringing full-hand capabilities into VR through these controllers creates unique opportunities for meaningful social interactions.

In the next few years, VR is likely to grow beyond what we know today, but there are still valuable lessons for designing for what we already know. Following are a few lessons learned as we created natural and comfortable interactions for hands in VR. At the rate of the discoveries we’re making each week, some of this might be irrelevant or silly in just a few years—but we hope it can help everyone continue to make progress.

Creating natural interactions

I sometimes describe designing for VR as being dropped into the ocean and not knowing how to swim. You don’t really know where to go and don’t always know the way to move forward. The only things that keep you from drowning are re-using some of the problem solving tricks we use in product design to solve complex problems, looking for any opportunity to grow every new skill useful for this medium and helping each other out. Coming from a traditional product design background — where most interactions have many established and proven patterns — the most effective way I’ve found so far to make progress and learn in VR is through trial and error.

When it comes to designing for Oculus Touch, we realized early on that anyone new to VR had to make two radical shifts in their thinking. The first was the immersive aspect: looking around, having a feeling of presence and distinguishing the virtual space from the physical one. The other was how this new type of input controller suddenly gives you virtual hands and lets you interact with your surroundings.

When designing for control, it’s important to look at the most natural and intuitive ways of doing a specific action. Doing so allows you to create experiences that are approachable and delightful: whether that’s reaching out, picking up objects, interacting with a 2D panel, something in the distance, and so on.

Through countless demos and usability studies, we discovered that relying on controller buttons had a few drawbacks. People who weren’t used to video games had trouble finding the buttons without being able to see the controller. And using invisible buttons could potentially break the experience by reminding you your virtual hands weren’t real. This problem was even more obvious in a social context where people were trying to perform actions while having a conversation with someone else in VR.

We decided to always make the default interaction something that can be done with the virtual hand itself, and doesn’t require the use of any of the front-facing buttons of the controllers. Picking up an object and holding it is done by pressing and holding the grip. Interacting with a 2D user interface is done by simply touching it. Selecting something in the distance is done through pointing at it with your index finger. Scrolling through a list is done by either grabbing the list with your hand and moving it up and down or by grabbing and moving its scrollbar.

We’ve also learned how crucial visual, audio and haptic feedback can be in creating natural interactions. When approaching an object with your hand, giving feedback such as the object enlarging or brightening, or seeing your hand adapt by opening or otherwise adjusting its pose helps understanding that object can be interacted with. When touching the object, getting haptic and audio feedback adds to the immersion and creates a connection between the virtual and physical hands. Think of the sound your hand makes when you clap, or the subtle sensation of picking up and setting down your phone.

Designing comfortable experiences

While using your hands to interact with a virtual world is incredibly exciting and immersive, it presents some physical challenges for designers to consider. Here are some of the things I’ve learned:

• The higher the person has to raise their arm to perform an interaction, the faster that interaction should be in order to avoid fatigue.
• Raising your arm high enough that your elbow is no longer in contact with your body can be tiring.
• Interacting with a moving interface (for example, a UI attached to the secondary hand, like a watch) can be challenging and should be primarily used for quick actions.
• Some people have trouble visually reading depth, and may need some time and practice in order to learn how far they have to reach out to manipulate objects in VR.
• When sitting, reaching out towards objects that are too close to your virtual body can be challenging.
• Hands that visually look realistic can be creepy. They give people a feeling of being inside someone else’s body. They can also break the immersion when those same hands go through what appears to be a physical object, like a table.
• A familiar object often communicates how it should be picked up, held and used. For example, people expect to use a gun-shaped prop to aim at things.
• A physical object may need to have different physics when being held. If you’re holding a pen and go through the table, you probably don’t want the pen to hit the table and bounce, but rather go through the table with your hand.
• People lose track of their real world physical surroundings when in VR. Hand gestures can result in them hitting a wall or throwing the controller away.
• Hand gestures are inherently social. When your friend is interacting with virtual objects in VR, having a visual indication of what they’re interacting with removes the potential awkwardness of seeing their hands move in mid-air with no understanding of what they’re performing.

More Resources

Hopefully these insights help you when considering designing for hands in VR. We’re still in the early days, and as we make progress in this journey our design process also evolves and improves. Today, we’re excited to release a set of Virtual Hands we’ve created that can help designers mock up hand interactions for VR in traditional design tools. And if you’re curious about some other aspects of building a Social VR experience, checkout Mike Booth’s talk at Oculus Connect.

Check out more stories and resources from the Facebook VR design team at facebook.design/vr.

Designing for Hands in VR was originally published in Design at Meta on Medium, where people are continuing the conversation by highlighting and responding to this story.

A journey in design & storytelling

A little more than a year ago I joined the Facebook Photos team with the goal of exploring what could be the future of storytelling on Facebook. We felt like it was time for us to modernize our platform and better help people share their stories. We gathered a small team and started doing some research and prototyping.

The Problem

Telling a good story is hard. Photos and videos go a long way (even though we didn’t support letting you post both at the same time), but it’s only when they’re ordered and presented in the right way with the right context that suddenly a story becomes interesting for the viewer. People usually end up sharing all of their photos from a moment into an album, without curating or captioning, which is often quite hard to consume/relate to for the viewer.

We uncovered the problem by working with our Research team and Data Analysis team on identifying what were the main issues today that were preventing people from telling their stories. Our findings indicated that a lot of photos in people camera rolls were photos they took with the intention to share them, but never did. People often pointed out how difficult it is to pick the best photos out of a moment: once you’ve taken tons of selfies, five or six photos of that same waterfall… picking the few photos that will really show your friends how great your hiking trip or your dinner party was feels like a lot of work — and people very often end up forgetting to share the story. This seemed like a problem we could try to tackle and help people share the photos that mattered to them the most.

Our process

Based on our research, our team worked on figuring out some principles that would guide us through our design journey. When building a product, a lot of decisions have to be made on a daily basis that will influence a lot the final outcome. A good set of design principles can add a point of view and influence every decision, to make sure they’re aligned with the original problem you’re trying to solve.

We agreed that we wanted to build a product that would:

• Help people remember they have stories to share
• Make it fast and easy to share a rich & visual story
• Design a format that makes people proud of the story they share, and makes the consumption experience engaging & fast.

We then started exploring what the ideal consumption experience could be. After a few white-boarding sessions, we came up with a few ideas that we started designing and prototyping: a flip-book, a long vertical story with blocks of texts, maps and photos, a book-like interface, a collage, a video…

Design Prototyping

It’s no secret that many of us on the Facebook Design team use prototyping tools to create high-fidelity demos that show how a feature would look and feel when actually using it. To design collage, I mainly used Framer.js and Origami, and prototyped end-to-end demos of some of these concepts and interactions. Building quick prototypes allowed us to get a better feeling of what would be the experience for someone consuming one of these stories on their phone. While some interactions felt very compelling — for example, swiping down to reveal an interactive timeline of the story — we knew we needed to build a flexible system that would require as little input as possible from the user. We also worked on stripping away the distractions and focus on what really mattered.

Prototyping allowed us to quickly eliminate the concepts that didn’t feel good enough, and to validate/test the ones that had more potential.

A dozen of prototypes later, we settled on the collage grid as being the most flexible foundation for our product. We realized this format had the benefits of supporting all kinds of media types (photos, videos, text, maps…), and was very dense, which meant stories would be fast to scroll through and consume, and the individual photos and/or videos wouldn’t have to stand on their own and be high quality.

As we were making progress on the consumption experience, we started thinking about what would be an easy way for people to create these stories. We also decided to only focus on photos and videos at first, and started exploring how to automatically generate drafts of the stories people would then be able to curate and share.

Designing our North Star

As our Director of Design Julie Zhuo suggests in her article “Design’s North Star”, it’s very important in the design process of a product to communicate what the high-level narrative of your vision is. What do you want your product to look like 2 years from now? Why is this product so exciting and how will it improve people’s lives?

Combining all of the learnings we had made during our explorations, we started designing what we imagined the final state of our product would be, putting aside the details or technical constraints, and shared it internally with the other teams at Facebook to kick off some discussions.

Research & Internal Testing

Once we had defined our North Star, we started working on simplifying our product to its core minimum.

• What were the fundamental things that this product was doing that Facebook wasn’t offering already?
• Out of all these things, which ones were most likely to immediately provide value to people?

We designed our minimum viable product and started working with our Research team on how we could validate some of our assumptions by putting our designs into real people’s hands. To make sure people would interact with a prototype that would use their actual camera roll, our engineers built a standalone iOS app that would group photos and videos in the camera roll by time and location, and would display each moment as a collage.

In our usability testing lab, our Research team helped us identify a number of valuable criteria about how people perceived their photos and what they consider interesting to share. We also had a chance to answer some of our questions: What do people consider a good story? What curation tools do people need? What do people want to see in their News Feed?

We also made the feature available to all Facebook employees and started gathering feedback. Employees were able to look at their camera roll organized through stories, and give us the feedback wether a story was shareable or not, and why. Using this data and what we had learned in the Research lab we started to tweak our algorithm and make some improvements to the stories themselves. We quickly identified that people didn’t want to share collages that had a bunch of the same photo repeated again and again, so we worked to combine similar photos into animated slideshows — people loved this.

Building the real thing

Now that we had something we felt pretty good about, we started moving some of our code into the main Facebook app. It was also time for me, as a designer, to start polishing some of the elements of the UI and craft the interactions and layouts.

We started refining the details of our MVP, stripping out the elements that weren’t needed, and making sure we had the best solution we could come up with for every step of the flow. As part of this refinement, I designed dozens of entry points for the suggestions, dozens of headers for the stories, or attachments for the News Feed post that would go with a story. Making all of these iterations and explorations is long and hard process that forces the designer to never fall in love with any of any single design, and to continuously look for a better solution.

Public test

When building products at scale, you make tons of design decisions on a daily basis that are mostly based on hypothesis and your understanding of the problem. There is nothing more effective than testing these hypothesis by launching some public experiments and getting data as early as possible. Usability testing can go a long way and help unblock some difficult design decisions. But a public test will give you a sense of what impact your product or feature has on the overall system. It will also give you a better understanding of how people interact with the product:

• When and where are people dropping out of the flow?
• Do people easily discover X or Y, etc. How do they use Z?

It’s also usually a good way to test the performance and stability of your product without breaking the experience for all of your users.

As we often do at Facebook, we ran a public test for our product. Our main goal was to confirm that our product was actually helping people by solving the problems we had identified, as well as testing some of our hypothesis and better understand how people would use our features. We launched the product in Sweden, Norway and Denmark so we could see the social effects of launching this brand new format.

Shipping

Alongside running the test in Scandinavia, we continued polishing our experience, mainly by focusing on our set of curation tools, and on the performance and stability of the product. We also started ramping up our marketing efforts, working with our research, marketing and content strategy teams on finding a name (Collage!), preparing screenshots, shooting a video... and other efforts to get ready for launch.

Lessons learned

When designing products, it’s always good to look back at the journey and try to identify the things you could have done better, so you can apply your learning to your future projects. It’s also important to reflect on the learnings you’ve made about people’s behaviors, since that’s often the key to identifying the next set of problems to solve.

We gathered as a team, and looked back at our original goals. We ran a survey with the people who had used the product, which helped us better understand how close we got to achieving our goals:

• Support people, and give them control — we were able to create a product that suggests stories to people by organizing their photos and videos into Collages, reduces the number of duplicates, and offers them the tools to be creative and give their voice to their stories.
• Create a way to share pride — Photos are pride. We found time and time again in testing this product that people loved that we made it fast and easy to share a rich & visual story that makes them proud of their photos.

We live in an age of information overload. Everyone takes a bazillion photos. Over time, it has become a real need to be reminded of what you’ve taken. Memories are meant to be shared, and it’s important for technology to help us figure out what and how we can share our daily moments.

While designing Collage, we learned there’s no perfect way to generate stories that will satisfy every user. A subset of people will be perfectly fine with a few blurry selfies they took at the mall and are okay with sharing them, while other users will only care about sharing the photos of their travels. We also learned that people are generally happy with auto-generated stories that aren’t perfect as long as you provide straightforward tools to curate/craft them. Finally, we learned that building a product that impacts the rest of the system is hard to get right, and will often expose the product to a lot of internal feedback.

Designing Collage has been an incredible adventure so far, and I’m looking forward to improving the product over the next few months, and to seeing the great stories people will share on Facebook.

I’d like to thanks the entire team, for their hard work in building this new experience, another big step into making the World more open and connected.

Special thanks to Priyanka Kodikal, Jonathan Colman , Laura Javier, Mills Baker, Jasmine Friedl and Hannah Pileggi for their help in proof-reading this.

Designing Facebook Collage was originally published in Design at Meta on Medium, where people are continuing the conversation by highlighting and responding to this story.

Designers are fascinating. Each and every one of them has a unique skill-set, area of expertise, experience, and yet most of them want to work on the same kind of projects. What Jeff Domke calls “shiny products” in his great piece “I only work on shiny products”. Glamorous products that will look great on their portfolio.

Almost two years ago, I went on a quest to find a second designer for the Parse team. Finding good designers who aren’t already busy building things isn’t easy, but as it turns out, finding one who’s excited about designing developer tools is even harder. I’ve talked to a lot of talented designers, and most of them had the same reaction when I explained what Parse was doing. “Hum, yeah, that’s cool. By the way, have you heard of that new coffee place in the Mission?”. They all ended up either joining another team at Facebook or another company, to work on a “shinier” product. Developer tools were just not appealing enough.

Anything can be shiny

Now here’s the thing: developer tools can be “shiny”. Like any piece of software, they offer room for beautifully crafted interactions, gorgeous illustrations and iconography, and minimalistic interfaces. The problem you’re trying to solve is different, the ways of solving it are similar.

Developers are people. And just like anyone else, they appreciate well-crafted, easy to use software that provides an amazing user-experience. One thing we often hear from Parse users (who are mostly developers) is how easy and nice it is to get started with Parse. They often mention that our quickstart guide gets them started in minutes, and our documentation is some of the best out there. Or they tweet about some little details we add to the experience. Since the very beginning, the Parse team has been focusing on simplifying mobile development, and offering beautiful and easy to use developer tools. The fact that we got popular in the developer community, and that our users keep pointing out the simplicity of our tools and the attention to details, demonstrates that developers care about great tools, and appreciate us going the extra mile when crafting the experience.

As designers, our job is to take a hard problem, and find its best solution. The problem can be “How can we make sharing photos on mobile simpler?”,
but it also can be “How can we help developers build apps faster?” or “How do we help developers understand what’s right or wrong in their apps?”. All of these problems have millions of possible outcomes, and the process of finding the best solution remains the same.

Identifying the problem. Identifying the constraints. Finding ideas. Saying no. Designing wireframes. Designing Mockups. Experimenting. Prototyping. Implementing. Shipping. Communicating. Iterating. This is what you do when you design the next Snapchat or Square. It’s also what you do when you design the next AWS or the next Google Analytics. Different problem, same process.

Overlooked problems

This goes beyond developer tools. My girlfriend Claire is busy designing business interfaces at Fliptop. My friend Maykel is busy designing Business Tools for Instagram. Margaret leads a great design team that builds amazing advertising interfaces for Facebook. After working on apartment search at Lovely, Kerem is now rethinking healthcare delivery at Omada Health. Those probably don’t precisely correspond to the definition of “shiny products”. Yet they’re fascinating, challenging, rewarding products to work on. They belong to undervalued industries, and actually offer designers a chance to have a much bigger impact.

These are great examples of designers who decided to dedicate their precious time rethinking these complicated, often ugly and hard to use user interfaces, to make them functional and beautiful.

Don’t get me wrong though, I love shiny products too. I spend way too much time on Tweetbot, Messenger, or Rdio. These products are amazing and inspire us all every day. My point is that given the state of the product design world in 2014, (most) designers are very fortunate to be in a position where they can decide what problem they want to solve next. And as a community, we should encourage some of them to also try to tackle these complicated, often-overlooked problems.

Designing developer tools is fun too. You should give it a try.
Start designing for the unshiny.

Design for the unshiny. was originally published in Design at Meta on Medium, where people are continuing the conversation by highlighting and responding to this story.