Problem

Lawmakers are not known to be familiar with technologies. Since the beginning of Donald Trump’s presidential run, Trump and conservative senators have been claiming that search engines are biased against them.

At a recent hearing with Google’s CEO Sundar Pichai, some US senators were not content with how they were portrayed in search engines. This prompted me to investigate how exactly they are portrayed.

Approach

Instead of conducting natural language analysis of web search results, I decided to target something more intuitive: image search results.

My goal is to gather photos from the top search results (from the whole web, not from specific websites) of all 100 senators and Donald Trump, and examined the emotions shown in those photos.

After getting the search results, I then visualized the data in a graph, with the x-axis ranking all senators in the order of “the most conservative” to “the most liberal”, and the y-axis showing their emotion scores. With a bit of research, I decided to use the “Ideology Score” (https://www.govtrack.us/congress/members/report-cards/2017/senate/ideology) from govtrack.us for the ranking, and some APIs provided by Microsoft to get and analyze the photos.

Working with APIs

After setting the goal, as a designer who codes, I realized that I need at least two APIs and some JavaScript libraries: one API for obtaining the image search result from a search engine, another for analyzing the emotion in the photos, and some libraries like d3.js to help me draw the graph on the webpage.

I looked into the developer documentation of a few of the most popular search engines in the US: Google, Bing, Yahoo and DuckDuckGo.

However, Yahoo deprecated their search API in 2016 and DuckDuckGo never offered an image search API at all. Google replaced its old search API which was able to search the whole web with the new custom search API which only allows users to search in specific websites.

Fortunately, Microsoft’s Bing Image Search API (https://azure.microsoft.com/en-us/services/cognitive-services/bing-image-search-api/) meets my criteria perfectly. The API provides an experience similar to Bing’s native image search function by returning images that their algorithm determines are relevant to a user’s query.

As for the emotion analysis API, I had to choose between Microsoft’s Face API (https://azure.microsoft.com/en-us/services/cognitive-services/face/) and Parallel Dot’s Emotion Detection API(https://www.paralleldots.com/facial-emotion). Although Parallel Dot’s API seems to generate better results, the rate limit was so high that it will take the browser 16 hours to finish sending the HTML requests. Also, since I am using the Bing Image Search API already, it is easier for me to manage the Ajax calls, which share similar endpoints and structures in my code.

Challenges

Rate limit
Originally, I hoped that users could get fresh image search results when they visit the website, but due to the Face API’s rate limit of 10 calls per second, I had to manually slow down the processing and would not be able to process images in real-time, at least not without sacrificing the user experience.

SVG drawing
Although I have the 300,000 pieces of data at my disposal, how can I draw them on a graph efficiently? Even with my experience using the d3.js charting library, I decided I shouldn’t reinvent the wheel. In the end, I chose to use plotly.js (https://plot.ly/).

Solution

See the website live

Take a look at the screencast of the website or visit it at https://senators-debunk.glitch.me!

Current state & next steps

The site is still under development, and there will be more functions implemented soon. Some of the functions I plan to add include:

1. Use regression lines to summarize the trend in the graph.

2. Use a 3D Scatterplot to present two kinds of emotions at once.

3. Allow users to change the range of search results they want to see. In my own testing, we can see different trends of emotions from different ranges (for example, between the first to the 30th and between the first and the 100th)of search results.

In addition, since I’m not a mathematician or data scientist, please let me know if you have suggestions about where I can take this project next at elvin_hu(at)outlook(dot)com!

Thank you for reading! For more projects like this, stay tuned for future articles or visit my portfolio site at https://www.elvin-hu.com 🎉

It tells the story of the Phantom and Raoul falling in love with Christine Daeé because of her voice, and it is the musical that inspired me to practice musical theatre, and I always wondered, why don’t things like that happen in real life?

Problems

As a professional single person (and third wheel), I use dating apps quite often, and I’ve been on both good and bad dates thanks to those apps.

Popular dating apps consistently build around one certain narrative: they resemble real-life human experiences. You meet someone, see their face, start conversations, then either develop a kind of relationship, or move on. However, in those apps, one key element of real-life interaction is missing: you can’t hear the other person, and that brought forward a few problems in dating apps.

After conducting polls and interviews with more than 50 participants, I condensed the results into a few main points:

“I didn’t expect your voice to sound like this…”

70% of people I’ve talked to said that they have gotten surprised by their dates’ voice before, and half of those surprises were negative ones. Currently, the only dating app with the voice messaging feature is Grindr, but I am not quite sure why someone on Grindr might want to send a voice message instead of pictures of their 🍆 or 🍑.

“It’s all about looks.”

Most dating apps make users’ appearances at priority, and make it easy for the users to swipe. 36% of the interviewees said that they swipe on people without reading profiles, but the thing is, people are more than a mere picture.

“We don’t talk anymore.”

Not the aftermath of bad dates, but literally, people don’t know how to converse anymore. 65% of people responded that they felt nervous when they talked in person with their dates instead of when they texted.

Prototype Version 1.0

After conducting interviews and designing a low fidelity prototype, I made this animated interactive prototype in Sketch and Flinto, and here are the solutions I came up with in response to the problems.

Problem: “I didn’t expect your voice to sound like this…”

Solution: The only way of chatting in this app is by talking (02:53)
In Matchat, people can only send voice messages. No text, no photos (except for profile pictures).

Problem: “It’s all about looks.”

Solution 1: Profiles will only show blurry pictures when people first swipe (01:53)
After two people match, the more they talk to each other, the less blurred the pictures become.
Solution 2: Page based profile (01:53)
Each profile is formatted to look like a miniature blog, and people can only pass on or like a profile with the buttons on the top of the page which is harder to reach than buttons on the bottom.

Problem: “We don’t talk anymore.”

Solution: Audio self-introduction and answers to 3 questions in profiles (01:30)
People introduce themselves in their profile with their own voice, and answer questions that function as conversation starters.

User Testing and Feedback

After presenting the prototype to my class, guest critics from the Wall Street Journal, New York Times, and my friends, I received mostly positive feedback. People especially appreciated my considerations of the role of ethics in design, but we noticed some issues that I overlooked:

“I don’t feel safe or comfortable “liking” someone who I can’t see.”

As I mentioned above, existing dating apps are successful because they resemble real-life experience when people meet each other. Despite the fact that Matchat offers a voice feature, users still miss out on the imagery offered in other dating apps. Solution to follow ;)

“I don’t know how to introduce myself.”

People spend hours crafting their written dating profile. Although there is no written dating profiles in Matchat, people are required to record a “profile message”. Some people feel more comfortable with speaking whereas others don’t. To counter this friction in the UX, I added sample profiles for people to listen to, in order to help them record the message.

“Does this app auto-play profile message?”

Nobody likes auto-play. I repeat. Nobody. My first iteration did not take this into account, but I have solved this issue in the new version.

“I really don’t care about voice/the set up process is too complicated”

Get Tinder here ➡️ www.tinder.com
Jokes aside, on one hand, this app does face a more limited target audience than generic dating apps. On the other hand, the somewhat complicated onboarding process filters out people who are not serious enough about finding a relationship. However, the recording screen did feel unnecessary after I tested the prototype with people, so I simplified the recording process.

“The blog-looking layout is nice but…”

People agreed that the page based layout made it harder for people to swipe through the profiles, but they can push the buttons quickly to achieve the same thing. Also, people thought that the photos were related to the audio clips which they were not.

Prototype Version 2.0

Sign in

People can register with their phone number, and there is no requirement to connect to a social network account. Gender options include male, female and customized input.

Profile message

Instead of writing a profile, people introduce themselves with their voices. Examples are provided in the app to inspire people just in case.The color of the progress bar in the ‘hold to record’ feature changes from red to yellow to green to encourage people to talk more. Users then have the chance to record again if they are not satisfied with their initial recording.

Questions as conversation starters

Now people don’t have to start every conversation with “Hey how’s it going!” or “How was your weekend?”

Countdown before auto-playing profiles

The app automatically plays profile messages 3 seconds after it is launched.

Blurred profile picture becomes clearer as the profile message plays

Only the first photo is gradually un-blurred and shown to people before they match, which builds up trust, and encourages positive engagement when they decide to match (or not).

Like/dislike buttons become activated after 5 seconds

In order to prevent users from swiping aimlessly, the like/dislike buttons become activated after the profile message plays for 5 seconds.

Sending voice messages before matching

Users can choose to respond to messages they receive even before they match.

Voice messages only

People can only chat by sending voice messages.

Non-featured profile photos are revealed gradually

The longer people talk, the clearer the rest of the photos become. The indicator next to the conversation suggests how far away it is to unblur the rest of the photos.

Challenges & Next Steps

There are a few key questions that I need to answer about the functionality of the app. For example, how can I decide the threshold of un-blurring a profile picture after people start talking?

Currently, I am building the minimal viable product as a one-person team. If you are interested in hearing more about, it or you are an iOS developer and want to work on it together, feel free to shoot me an email at elvin_hu (at) outlook (dot) com!

Thank you for reading! For more projects like this, stay tuned for future articles or visit my portfolio site at https://www.elvin-hu.com 🎉 Currently I am looking for an internship opportunity for Summer 2019, if you are either hiring for your team or just wanna chat about design, don’t hesitate to reach out 😊