In 2012, Dr. Amy Cuddy made a splash through her TED talk, aptly named Fake it Till You Make it, arguing that simply acting powerful will make you more powerful/confident.

The phrase resonated with many people. At the end of the day, “faking” is an economical act — low cost, no actual work needed. “Make it” also implies path-independence. Whichever way you travel, supposedly you get the same reward in the end, so why not picking the easiest one?

But what about authenticity?

Aren’t we suppose to be true with ourselves and others? How can you be an authentic leader when you fake.

Thesis and antithesis

To answer this question, Dr. Herminia Ibarra wrote an article on Harvard Business Review in early 2015, pointing out that authenticity cannot be defined too rigidly, such that there is no space to reinvent yourself. For example, since leadership style is not a personality trait but a tool, a leader should adopt a playful frame of mind to experiment with different leadership styles. Getting fixated upon your true style (if there ever was one) would not be effective.

After fall of Theranos and Elizabeth Holmes, Professor Bill George from Harvard Business School offers a rebuttal to Ibarra, with Holmes as an example. The thesis can be summarized as you cannot fake leadership. And if you fake it, you won’t make it. A leader has to be “honest and open” to the world.

The Synthesis

To me, the valuable question is not about which scholar is right. In fact they are both right. The question is, what is your it.

Holmes did not just fake leadership. She faked reality. In fact, she faked key data points, laboratory reports and test results. When the it is reality, faking it would never pave the way for making it. One will have to break some fundamental law of physics if faking reality can lead to a different reality. So far in my life, I have not seen anyone who has the superpower to bend reality.

Contrarily, when the it is a skill one can acquire through practice, faking it puts you on a fast track to making it.

This summer, I found myself in the middle of a conflict between two team members. My “authentic self” avoids conflicts at all cost. I would not have stepped out of the comfortable zone to resolve the conflict. My coach gave me a conflict resolution framework and asked me to wear the conflict resolver hat.

Wearing the “conflict resolver” hat for an hour, I went through the steps with these two members and got surprisingly good outcome. I did feel “unauthentic” during the conflict resolving session — I would not sit in any conflict in the first place. However, the responsibilities I have as a manager triumphed over my narrowly defined “self”. Today I no longer fear conflicts.

To me, the wisdom of this aphorism is not in fake or make, it is in it. I will quote the Serenity Prayer to conclude this article, with my annotation.

God, grant me the serenity to accept the things I cannot change, [don’t fake it]
Courage to change the things I can, [fake it till you make it]
And wisdom to know the difference. [know your “it”]

Large-scale natural language processing (NLP) problems almost always involve a large corpus, which almost always contains “unbounded” number of unique words. Here, words are defined as a string of characters, separated by space, punctuation or other meaningful boundaries.

This problem was evident when I did my startup, AI.codes. Programmers use unique names for their variables, methods or classes (AbstractSingletonProxyFactoryBean is a classic example). For machine learning models to understand code, it has to understand these words.

This problem is also ubiquitous at reddit, my current employer, where we deal with Internet-scale, user-generated text. In fact, we assume that every NLP model at reddit would encounter rare words at some point, no matter how big the vocabulary is.

Strategies

1. Use a large vocabulary and discard everything else.

The idea is rather simple. We build a reasonably large vocabulary (say, up to 10 million words) based on usage frequency of words, and discard words outside the vocabulary in machine learning. For instance, if `covfefe` is not a word in vocabulary, we simply ignore it. The vocabulary can be built periodically to capture language evolution.

This strategy is sufficient for applications where useful signals are not in rare words. From the representation learning perspective, we assume rare words do not affect the representations. Sentimental analysis is a good example, where people tend to express sentiments using a known set of words and structures, and rare words rarely affect the sentiments of the whole sentence or paragraph.

There is a variant of this method, where a special token called UNK is used to denote OOV (out-of-vocabulary) words. This is usually only useful when the distribution of rare words play a role in the model. For instance in Bayesian model, it might be good to model rare worlds outside the vocabulary.

2. Hashing words to a finite range of integers

The idea is to use a non-cryptographic hash function (such as murmur3, FNV) to map an arbitrary word to a finite integer (usually 32 or 64 bit int, modulo a prime number). This is called the hashing trick, and is widely used in text classification tasks.

The key insight is that natural language words follow a Zipfian distribution, such that hash collisions are (if any) between infrequent words, or more rarely, between a frequent word and an infrequent word. Thus, the collided feature used for classification is either unlikely to be selected as a feature (since both words are infrequent) or will almost always represent the word that led the classifier to select it (if it is a collision between a frequent one and an infrequent one).

The above argument, of course, is a probabilistic one. No machine learning model is perfect in capturing all information from features, so is the hashing schema. Admittedly, the “how I learned to stop worrying and love the collision” mindset is not easy to grok. The following two papers provide detailed analyses on the hashing trick

• Extremely Fast Text Feature Extraction for Classification and Indexing by George Forman and Evan Kirshenbaum
• Feature Hashing for Large Scale Multitask Learning by Kilian Weinberger et al.

3. Character and subwords model

The idea is to split word characters or subword units. The cardinality of characters or subword units are low (~100 printable characters in English and ~200 for latin languages). A natural extension here is to use Unicode bytes, with a cardinality of 256 (2⁸ per byte). Subword combines some frequently co-appearing character sequences into a subword. For English, sequences like “er”, “co” may be useful to reduce the model complexity while not blowing up the vocabulary space.

This approach may be surprising for traditional ML practitioners. How would character and subword features even work for tasks like text classification? At the end of the day, a word is not a simple combination of its character or subword features. Indeed, models with more complexity, such as deep neural networks, are required to work with these features. The idea is to use a deep neural network (with sizable layers and parameters) to learn the correct representation of words from its spelling. The idea is successfully applied in machine translation applications. The following two papers illustrate the idea.

• Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation by Y Wu et al (use character and subwords for machine translation)
• Neural Machine Translation of Rare Words with Subword Units by Rico Sennrich et al (detailed explanation on how to construct subwords from data)

Another closely related technique in the context of deep learning is to use a Convolutional Neural Network (as oppose to use LSTM) to effectively read words as character n-grams (with padding). A few notable papers:

• Exploring the Limits of Language Modeling (use CNN to read words for language modeling/word embedding)
• Convolutional Neural Networks for Sentence Classification (again use CNN to read words for classification)

4. Char n-gram

The idea of using CNN to detect n-gram patterns in words is in fact not new. Classic NLP techniques such as stemming and lemmatization are all about using shorter char sequences to represent a class of words. In the information retrieval world, it is also well-known to use character N-grams to improve the recall rate while not sacrificing much on precision. IR systems such as Elasticsearch has native support for character n-gram.

The assumption here is that we can represent (rare) words by its substring, if the substring is reasonably long. While the character- or subword-level models uses a deep neural network to map building blocks to word representation, a longer sequence of characters may capture enough information of a word. For instance, the char sequence frequ is unique enough to capture the meaning of words derived from frequent or frequency, so taking it as a feature would give us a way to represent other derived, rare words, such as frequenter.

In practice, we can, for instance, represent a word by all of its character 5-grams. This trick maps open vocabulary to a finite character 5-grams feature space. Hashing trick also comes in handy if the char 5-gram space is too large. As an example, Facebook’s FastText uses both char-ngram and FNV hashing to handle rare words.

A caveat here is that the trick works for languages with natural “stems”. The technical may not work for languages such as Japanese and Chinese, where there is no clear definition of n-gram character in a word, or that a word cannot be approximated by some of its substrings.

An Efficient Way to Understand Code

Tying everything discussed together, here I briefly describe how AI.codes’ system understand the meaning of classes and methods while doing code auto-completion.

For the first version, I picked Strategy 1, where I constructed a huge vocabulary of class and method names, covering top 1000 popular Java frameworks on Github. It has about 10M unique words. Language model on this is reasonable, but the model has no idea about unknown words.

I did not use the hashing trick, because I do care about the meaning of rare words when doing code completion. Instead, I picked strategy 3, where I tried to train a Siamese network to map words to their embedding. The beautiful thing about this approach is that the network picked a bunch of useful n-grams that are so indicative in predicting code. For instance, the CNN picked up Builder$ and build( , and mapped them very close, indicating that when you see a Builder class in Java, you’d likely to autocomplete it with a build() invocation.

While you are here, let’s talk about fake news. I will ask three questions and try to answer them.

• Does Facebook have the responsibility to filter fake news for its users?
• Do we have the right technology to filter fake news?
• Does filtering fake news promote high quality journalism?

On Facebook’s responsibility

When you advocate for Facebook dealing with the fake news problem, whichever way it may be, the hidden assumption is that Facebook is responsible for feeding its users fake news.

Let’s dig deeper into this assumption, with something tasty.

Blame the pizza delivery person

Say your friend ordered a big pizza to cheer you up after the bitter election. She cares about you, except the fact that she ordered the wrong topping — you hate olives on pizza. The delivery person came to your door, showed you the pizza. You realized that it is delicious expect the olives. Would you ask them to remove the olives for you? Yes, that’d be crazy.

We don’t let pizza delivery person adulterate our pizza. We do not blame UPS for delivering low-quality or counterfeit goods we ordered from some sketchy eStore. The logic is obvious and intuitive in the physical world: we don’t blame the carrier for the quality of our atoms.

Atoms are hard to copy, modifier, filter and alter. So, we don’t hold messenger accountable for what’s in the package. Bytes, on the other hand, are quite easy to copy, modifier, filter and alter. The feasibility to filter bytes that enables us to hold Facebook accountable, but it should not be the reason to ask.

Algorithmic censorship

To hold Facebook accountable, we ask Facebook to move away from a neutral content distributor, and become a content filter. As glory as it is, we are essentially trying to grant grant Facebook the right to censor what people perceive in the Facebook world.

In Mark Zuckerberg’s post, he outlined a plan to combat fake news, and the first one would be to use algorithm to decide what’s fake and what’s not before user reports it themselves. In other words, we are letting an algorithm decide what is fake and what is not.

In good old days Richard Nixon had to call New York Times, a publisher, to try to censor some information he didn’t want the public to know. And of course, when he made that request, NYT held the executive branch accountable through the landmark supreme court case. It’s so much easier now: the algorithm will do it at the pipe. No matter how hard publishers or subscribers try, and there will be no evidence for them to sue when the algorithm is biased. Facebook will have ultimately control what people see and hear, because it controls all the goods in the marketplace of ideas. If that is not scary, probably you have not heard of North Korea.

Algorithmic neutrality is a myth

You may rightfully argue that the intention of the algorithm is to filter out only fake news, not good journalism. Facebook will not control what we see, just, you know, remove weeds from a walled garden.

Algorithm is fundamentally biased

Forget about unbiased algorithms, it doesn’t exist. No matter how well intended the algorithm or its designer is, it is going to be biased. We the people work in the AI field know it all too well.

You don’t have to know computer science to understand why, because it is really common sense. Algorithms are rules, extracted from experience. The data we use to train the algorithm are inherently biased. Just like people with different upbringing are entitled to have different opinions, algorithms trained on a set of data will inherit the bias in that data. The subtlety in data/experience is way beyond anybody’s ability to understand. As a result, the algorithm is born with bias. Bias that implicitly censor one view over another.

Algorithm is fundamentally superficial

The algorithm would not know the tension in the Arabic world in Tunisia in 2011, and it would categorize a fruit seller being slapped by a policewoman as fake news. It would not know the racial tension in Ferguson in 2014, and would fact check “Hands up, don’t shoot” and deemed it factually inaccurate (some people would report that too, and thus bury the news). Arab Spring or BLM movements would never happen if we just focus on reported facts in news, not what’s driving the news. Human world is complicated, while algorithms are shallow by design. The algorithm that tells fake news from genuine ones, does exactly that. True force is dampened, and news become collection of trivia.

Algorithm can be exploited

Deciding what’s fake is fundamentally a binary classification problem. If you don’t trust an algorithm to pick the right president for you (mostly because the algorithm cannot factor in every concerns and considerations you have), you shouldn’t trust an algorithm to pick what is fake and what is genuine news for you. It is your judgement, not the algorithm’s.

By training an algorithm to filter contents for us, we are creating the ultimate echo chamber. It is safe to predict that with the implementation of fake news, users will flag information they do not like as misinformation, and thus teach the algorithm to filter unwanted content, not fake content.

Once Facebook has a way to filter fake news, it will become the best friend of totalitarian government. In China where I grew up, government is actively censoring the Internet. Now it becomes easier: just call Mark and tell him that this and that is fake. Facebook will become a mediator. I can guarantee you that the party who has more resources will get to tell what is fake and what is not. Facebook becomes a propaganda leverage for government, for special interest groups, for people with more resources. It is not the intention, but the destiny of such an algorithm.

On Journalism and the marketplace of ideas

One amazing thing I learnt from economics is that the invisible hand acts in unexpected way. I am predicting here that Facebook’s fake news policy will kill good journalism, instead of saving it.

It is well-known that high quality journalism is a good but rare thing. We as a society should support high quality journalism. Now the question is: is fake news hurting high quality journalism? If so, should we eliminate fake news?

Fake news is definitely attracting traffic from sites like New York Times and Washington Post — you have a constant pool of reader attention. However, the very existence of fake news is to reflect the value of good journalism. Eliminating it has unintended consequences.

Average content drives out good, paid content

Eliminating fake news, if that is all possible, would allow two goods: average-journalism and good-journalism, to float the marketplace of ideas. Now if I am a consumer, after Facebook’s filtering, I lose the signal of the differences of what’s good and what’s bad, and I lose the signal of what is average and what is good because I have already relied on Facebook to make the judgement. We hope to distinguish good and bad, and as a result we now cannot distinguish average and good. So how do I pick: I will go with what’s cheap.

This is a commonly observed phenomenon: majority of the people buy store brand when choosing between a store brand and a premium brand (supposedly better quality but consumers cannot tell). Average-quality journalism (your friends’ posts, blog articles, tweets) is the store brand of Facebook. With fake news floating around, you know the value of good journalism and you don’t even necessarily trust something old John posts.

Now with the filtering, the average quality seems better, and everything seems to be “true”. In this world, I certainly would not pay $9.99 per month to get New York Times if I can get a somewhat true, filtered news source through Facebook. The value proposition of NYT is diminished when fake/bad quality news is out of the marketplace.

In addition, it reduces readership to high-quality journalism. New York Times relies on a healthy funnel from Facebook/Google to drive traffic to their site, so they can get subscribers. In the short-term after the fake news policy is implemented, they may get a few percentage more traffic because fake news is out of the marketplace. In the long term, however, the vacuum will be filled by some average, barely passing Facebook standard, news sources. They are cheaper to produce, cheaper to maintain, and of course, cheaper for the customers. No doubt customers will pick them instead.

Facebook should focus on exposing the signals that reflect the quality of the news, not silently filter fake news and hide the quality signal. This way, it eliminates the asymmetry of information and lets users know what is good and what is not.

Finally words

The only newspaper ever known to the human race that never said anything fake is of course, Pravda, or “Truth”, from USSR. We can blame Facebook for fake news, but I don’t think we really want a world where Facebook would use its algorithm to turn NTY, Washington Post and every news agency there all to one version, the Pravda.

There is an easy and human approach to this: just unfriend that sucker who spreads fake news. Problem solved. Don’t trust the machine, especially the machine that feed you information. Fast food fabricated by machine gives you empty calories, fast food style news feed fabricated by machine gives you empty and biased mind.

I sincerely feel that forcing Facebook to deal with fake news is dangerous. The consequence is way beyond “I’d never need to see the crazy propaganda your high school classmate posts”. Any intellectual with integrity, liberal or conservative, should oppose Facebook’s fake news policy.

If you are reading this, it is likely that you are a knowledge worker (KW). By knowledge worker I mean an engineer, scientist, doctor, writer, manager, lawyer et cetera, who tends to work in an office, and uses their brain instead of their body to earn a living.

There are many fitness goals out there, it is important to find the one that suits you. Why? Because fitness goals like 20 push-ups or 10 pull-ups are meaningless to your need, unless you know what works for you.

For instance, the default goal of many fitness trackers is 10,000 steps per day, or about five miles per day if you are neither a giant nor a dwarf. This is a simple goal to have, but it does not fit everyone. Depending on your gender, age, body weight and the terrain, it may be too little or too much. It is not tailored to an individual’s needs.

As another example , the Physical Activity Guidelines for Americans, published by the federal government in 2008, recommended that American adults should do at least 150 minutes a week of moderate-intensity, or 75 minutes of vigorous-intensity aerobic physical activity. These fitness goals, while covering the cardiovascular element of fitness, leave out other areas of fitness such as balance, coordination and strength.

Over the years, I worked out with many knowledge workers in boot camps, martial arts schools and gyms. As I observe myself and others, I developed the following fitness goals for myself. In these goals, I factored in occupation, life style, posture and movement patterns, and work-related challenges. These goals are useful to me. I hope they are useful to you as well.

1. Attaining General Wellness

Wellness is a state of physical, mental, and social well-being. For KWs, it is particularly important to focus on nutrition and sleep.

• Nutrition

Knowledge of basic nutrition concepts is vital to physical fitness. When I worked at Google, oftentimes I would see people making all kinds of crazy diet choices at lunch. I saw Googlers skipping breakfast everyday, just to come in to eat a gargantuan meal at 11:30; I saw a young lady making a plate of pasta and three chocolate mousse cakes, and nothing else; I saw a skinny guy making a mountainous plate of extra-lean beef and egg whites, and adding a ton of salt on top. Free food provided by companies is better than nothing, but if you’ve got the wrong notion about nutrition, there is little benefit in these gourmet foods.

There is a lot of hype in the nutrition and diet arena. Diet itself is a multi-billion dollar business, so naturally there are all kinds of companies trying to sell you magical food. It is good to stick to the basics. A good starting point is the Smart Nutrition section of nutrition.gov. There, you will find Dietary Guidelines for Americans published by USDA.

I also try to cook often, with ingredients picked to make a balanced meal. Free food offered by companies can be attractive, and Soylent can be handy, but the truth is, nutrition is too important to put on auto-pilot. Being mindful of what fuels my body encourages me to think about my overall physical health.

• Sleep

A wonderful night of sleep can leave us feeling refreshed. Research shows that about 70 million Americans suffer from chronic sleep problems. For KWs, it is extremely important to have a good night of sleep, as sleep restores memory, relieves stress and spurs creativity. We all have personal experiences that sleep deprivation squarely leads to low productivity and poor quality of life.

Getting a good night of sleep means properly-timed exercise during the day. Activities such as yoga and meditation can also improve sleep. Caffeine and excess use of social media negatively affects circadian rhythm. Once the circadian rhythm is broken, a rough night is destined to come. If you want to further your research on sleep, I recommend “Improving Sleep: A Guide to a Good Night’s Rest” published by Harvard Medical School. This book is insofar the best resource on sleep. You can also find practical tips for sounder sleep in this book, such as how to moderate the temperature and light in your bedroom.

2. Enhancing Posture and Stability

KWs often sit in their offices all day, with attention focused on computer screens. Ergonomic chairs and stand up desks are getting popular. The truth is, no matter how ergonomic your chair or your desk is, if you are not mindful of your posture at work, you are likely to put unnecessary tension on certain muscle groups. Our office environments put a lot of strain on the body, especially shoulders, neck, and lower back. Our ancestors never got a chance to work in office, and we are not evolutionarily adapted to the office environment.

Shoulder, neck and lower back pains are all posture issues that can be addressed by improving muscle balance. That said, just pumping iron in the gym will not necessarily fix those imbalances. For example, say you’ve got a lower back pain after sitting in front of computer all day. You may have heard that “core” or abdominal support for the spine is the key to prevent lower back pain, so you hit the gym and work on your six pack. Unfortunate, a stronger six pack may even worsen lower back pain. I will explain this in later posts when discussing posture.

Strength training is great, but it should not be the fitness goal for most KWs. General muscle stability takes priority. As KWs, we need to focus on muscle stabilization exercises first. Calisthenic (body weight) exercises are great for muscle stability and endurance.

3. Battling Sedentary Life Style

There is a Chinese saying: when you sail against the current, you either move forward or get swept downstream (逆水行舟，不进则退). Our physical body is no exception — it is very easy to get de-conditioned. De-conditioning means beneficial effects that came with exercise are gradually lost or reversed after four weeks without training. If you live a sedentary life style (active for less than 30 minutes a day), assume your body is de-conditioned; it is performing sub-optimally.

As I mentioned before, it is suggested in “Physical Activity Guidelines for Americans” that adult Americans should exercise at least 150 minutes per week to maintain cardiovascular fitness. In fact, agility, balance, coordination, speed, power and reaction time all degrade when you live a sedentary life.

You may do running, biking or hiking once in a while, but if these activities are not regular, they are unlikely to help maintaining your physical health. A sedentary life style is the current that KWs are constantly sailing against.

4. Striving for Elite Fitness

When I am at work, I think of myself as a Spartan warrior or a Navy SEAL. This warrior mindset helps me to get things done. Warrior mindset means that I plan ahead, utilize all possible resources, take calculated risk, focus on the goal, and strive for excellence. Years of Zen meditation and martial arts training taught me one thing: mind and body are really one thing. When my body is at its peak performance, so is my mind. When my body is tired, I tend to have a weak mind. One of my martial arts training buddies once said this to me, “Eric, if you can kick that punching bag hard, I am sure you will kick the project’s ass hard.” And I did.

It is my belief and training philosophy that through elite fitness training, one can obtain a warrior mind. In Silicon Valley, people tend to put “warrior mind” into the “entrepreneurship” bucket. You don’t have to run a startup, own a business or have a vision to change the world to be a warrior. Once you have the mindset, you can be a warrior everywhere: in your life, family and relationships. The foundation stone of warrior mind, I believe, is a warrior body. I myself use the Navy SEAL training program to train, and I cannot recommend it more to people who want to obtain the warrior mentality.

Take Away Message

We just turned the page to 2016. As you are making New Year’s resolutions, think about these four fitness goals, and see how they relate to your profession and life style. From my experience, these fitness goals are not hard to attain if you have a fitness routine and stick to it.

It does not take much to start physical training. I started training by doing P90X at home. Although I still go to martial arts classes to polish my BJJ skills, I do most of my training in my living room and backyard. Pick a program, any program, let it be P90X, Yoga, Pilates or Navy SEAL Fitness, and stick to it. Start now.

If you need some help to kick start this journey, I can help. I host fitness boot camps and 1:1 private fitness classes near the SF Ferry Building some weekdays. If you think these four goals make sense, and would like to have someone who can understand your concerns and constraints in fitness training, try me. We can work together to achieve your fitness goals.

This is the first article of my “Wellness & Leadership” series. In this series, I want to share my thoughts on how to become a lean, mean tech lead.

What I believe

We all want better self everyday. We want to be a better parent, a better son/daughter and a better sibling. In relationships we strive to be a better partner and friend. At work, we want to be a better employee, better manager or a better CEO. These life goals, and the actions we take to achieve them, define who we are.

On Wellness

I firmly believe that a fit body and mind is necessary condition to achieve all of these goals in life. I call the health state of body and mind “wellness”. Too often, we see ambitions hampered by energy level, dream frustrated by poor physical health, and vision curbed by unmatched emotional stability. Whatever walk you take in your life, wellness is the cornerstone to any achievements in life.

On Leadership

We face challenges every day. They are like waves, pushing and pulling our vessel, as we propel ourselves through the life ocean. It takes motivation, planning, commitment and hard-work to tackle these challenges and achieve our goals. Specifically, we should take the lead in our lives, not getting pushed around by the waves and flow of challenges. Leadership is an active force that we use to shape our lives.

Two Pillars

I sincerely believe that general wellness and leadership are two foundational pillars that support all these goals and actions in our life. That said, both wellness and leadership are means to the end. Though achieving great physical fitness and becoming a great leader in life have their own virtue, the purpose of life is usually above these two. You, the reader and the doer, will be the one to figure out your own purpose. It is my hope and my personal experience that, by nurturing wellness and leadership, the purpose of life is more attainable.

About me: I lead an engineering team at Fitbit, focusing on Personal Wellness. I am also a certified personal trainer, taking Navy SEALS training programs to knowledge workers in SF bay area.