Finding bugs is a technical problem, fixing them is a human problem.

Hacking: Exciting.
Finding bugs: Exciting.
Fixing those bugs: Not exciting.

The thing is, the finish line for our job in security is getting bugs fixed¹, not just found and filed. Doing this effectively is not a technology problem. It is a communications, organizational² and psychology problem.

A decade ago on the Microsoft vista pentest we³ found some bugs. Then as we worked to get those bugs fixed we got a lot of excuses back: “but that would be illegal”, “just a denial of service”, “ but its a perf hit”, “but the victim would have to click on it”. This happened enough times that a bingo board of the excuses took shape:

The point is it is sometimes a hassle to get security bugs fixed and this isn’t a new problem. It is common for critical security flaws are fixed quickly but the long tail of lower-priority issues (that should still be fixed!) to drag on.

Over the years I’ve gotten a lot better at getting these type of issues fixed.

Things that worked for me:

1. Is prioritization correct? Bad security teams think everything is critical, in which case nothing is. Everything in security, engineering, and life is tradeoffs. Don’t push for something that doesn’t matter. Preserve your credibility by not causing security fatigue for engineering teams.
2. Is the bug clear? All security bugs should cover:
   * Impact.
   * How the issue works.
   * Clear next steps. Either a fix or what to investigate next.
   * Why you assigned it to that person.
   * What your expectation is for the assignee.
   Jack, Philippe and Stephen are all examples of clearly communicating complicated security issues.
3. Explain why security matters. I believe all security tasks should link to something explaining why security matters, how fixing discrete bugs is part of that and a clear-eyed description of what might happen if it doesn’t get fixed. Doing this well you draw the reader into the world and mission of the security team.
4. Empathize. Can you imagine the stress of getting a critical security bug in your code? The stress is multiplied through a murky and accusatory bug report demanding swift action. That is again multiplied when English is not your first language. Show empathy through clear, concise, non-emotional bug reports free of FUD. Empathy also means using the right medium for the person: email, slack, IRC, note on a diff or just showing up at their desk. Respecting different people’s workflows matters.
5. Nagging or guilting engineers into fixing stuff is miserable for both sides. If you go down this path consider carefully the “obvious” solution of automating it via a bot that pings tasks. I’ve had good luck with it but only when done carefully. Sometimes it really is as simple as reminding someone a few times and they get to work on it.
6. Escalating to a manager. Not ideal but not as “bad” as it feels to most people. Often the manager has the most context on priorities and who has free time on their team. Good to ensure email has more the tone of “I want to make sure you are aware of this and I would like it to be fixed” not “engineer $foo has dropped this task n times”.
7. Fixing them yourself is great sometimes, non-optimal most times. Any security team should be capable of this but it but exercised sparingly. Doing the work feeds the lesson back into the people its most relevant to, and needed for.
8. A/B test wording. “you are in the small minority of people who have not fixed your open security bug” is a line that works wonders. I stole this wording from an excellent planet money episode about a/b testing the best words to get people to pay their taxes in Italy. It works for our field too.
9. Visualization, leaderboards, gamification. I’ve seen security debt graphs to nudge senior leaders to invest in security work, leaderboards to praise the teams fixing their bugs the fastest and many types of gamification “fix 10 bugs, get a t-shirt!”. All can work or fail depending on the composition of your engineering org and values of your company. Kelly pointed me to this whole area of social proof and it sounds promising but I’ve personally only tried the above.
10. Not a tactic but an observation: Sometimes you can actually see the 5 stages of grief happen with a security bug: Anger → Denial → Bargaining → Depression → Acceptance. It can be an emotional thing to be told your code put the company and its users at risk and I’ve seen this happen many times: first teams are mad, then try to argue down the severity, then slow to fix then finally thank us for improving their code. Not much to do here except to recognize there is some emotion with this and try to not take it personally.
11. Cupcakes. All else fails try cupcakes. Upon the 1 year birthday of bugs we delivered cupcakes to the desks of 30 or so engineers who had not fixed longstanding bugs assigned to them. About a third of them had their issues fixed within the next two weeks. SUCCESS.

Conclusion

I’ve seen many exasperated security people lean too heavily on the moral component of fixing security bugs (its the right thing to do, how could you not!) and be surprised when that plea doesn’t resonate.

Security is an inherently cross-functional discipline. We have to work across many teams of varying qualities and enthusiasm for security. The social engineering to getting bugs fixed can be complex so hopefully these notes help.

Footnotes

1. Not just found and fixed but prevented for next time, eradicated across the codebase, well-understood by the company. I am using “fixed” as shorthand.

2. This all assumes you work at a company that cares about security and has the appetite to fix things but just isn’t getting it done. I’ve written about this with more color around how to start from zero here.

3. I played a small part here finding a few bugs and mostly heard this story from others after walking past this board.

Background

Reasonable engineers rely on the work of others. This work comes from other parts of the company, the open source community or a commercial product. Once used this code is part of the surface area of your codebase and from a security perspective must be treated just like homegrown code.

The problem is when security vulnerabilities come to light in this code, they don’t get updated. It is not top of mind that this stuff requires ownership, updating and care.

This can be especially important when, by lines of code, 30, 50, or 80% of your codebase is actually 3rd party code.

This is basic patching. OWASP calls it insecure components but the generalized form of it goes by the name “vulnerability management”. Vulnerability management exists across many areas: servers, routers, laptops, ipads on the conference rooms, etc.

How to fix

I’ve found this breaks down into 3 problems:

1. Inventory — knowing all your inventory of code, systems, etc for which security vulns might be found and need updates.
2. Vulnfeed — A feed of security updates against your inventory.
3. Remediation — A process to actually patch those things. Usually requires human intervention but may be as easy as apt-get update.

My first cut of solving this problem was writing a script to guesstimate if something was a 3rd party package. If it was, get its name and version and stuff it in a csv (inventory). Then download the cvedb, compare (vulnfeed) and if there was a hit send an email to whomever checked the package and ask them to update (remediate). Then auto-nag them if they didn’t within a few days.

The biggest variable here is how difficult it is to inventory your code: sometimes 3rd party code is packaged and nicely versioned, other times its a tarball of code checked into source control. The trickiest case is when someone downloads the source code to openssl, pulls out range of lines from a file and copy/paste it into your codebase. There are commercial solutions that can help if the code is nicely packaged, I’ve had luck with src:clr.

To operationalize this system involves stuffing it in a script, automatically rebuilding the inventory index and most importantly figuring out how to effectively motivate others to fix the findings in a way that scales.

On motivation one trick that helped was careful wording. Our first few waves of emails about updating were ignored. We eventually found success with the wording: “you are currently in the very small minority of people who have not updated yet”. I stole this idea from Italian tax collectors who after lots of a/b testing found it to be the most successful.

Prioritization

Starting from scratch on this problem can be intimidating. There are so many things to patch but what to go for first? Some ideas here: CVE score, the closeness of the system/code to the edge/internet, the areas that have owners, the areas that touch sensitive data.

Conclusion

Like with all security work the biggest obstacle is apathy. Pointing to this Equifax breach should help motivate other to help fix this grungy but important problem.

Much ink has been spilled in defining, or refuting, the concept of the 10x engineer.

A concept sometimes used in Silicon Valley to describe an engineer that is 10x more productive than an average engineer although the 10x metric is figurative.

10x or not there are definitely patterns in how strong engineers approach their work and career. Generalizing these patterns into lessons illuminates how to be a better engineer, and employee.

This draws from my experience as an programmer, team lead, manager and manager of managers.

Be reliable

This is the bedrock of you as an employee. Be careful when you say yes to something because you must now see it through to completion. You must be trustable.

Results > effort

Broadly companies care about problems solved well, not how. If you built it yourself, used a library or purchased something doesn’t matter. The company also doesn’t care how hard you worked, only what got done. The job doesn’t end at code, you must own the outcome even if means putting on a project manager hat.

Solve problems; look for more.

Your job is not only to solve X, its to be closer to the problem of X than anyone else in existence and surface further work. Over time this leads to a stack of better solutions. Problems are not strictly technical, maybe the new engineer training sucks and you could improve it.

Embrace ambiguity

Taking on increasingly ill-defined problems is something generally entrusted only to the most senior engineers. When your manager says “go figure out X” understand the problem, articulate a solution and get to work on it.

Make others better.

Peter Seibel on Twitter

How to be a 10x engineer: help ten other engineers be twice as good.

Lead the meeting when no one steps up to. Be the person to collect all opinions on something to inform a group decision. Email some persons manager when they did awesome work. Question why we are doing X if it feels suboptimal. Take on interns. Make your weaker teammates better. This effort betters you, them, and the company.

Do scalable things

Mentoring, documentation, training, libraries, crowdsourcing translation instead of hiring hundreds of translators. All these things make your entire team, company and sometimes world better in an efficient way.

Don’t be silent about your work

I did a lot of cool work for years that I made no effort to tell anyone about. I expected people to organically find and be impressed by it but that rarely happened. Don’t be an obnoxious self-promoter but tell people about your work so they know and may use it. Its unlikely they will seek it out themselves

Impact drives promotions

I define impact as the mark you leave on a problem, area, field, company, world, etc. Work that solves big problems is what you want to optimize for.

Don’t avoid discomfort

Human beings naturally hate conflict. If a timeline will slip or an idea didn’t work the worst thing you can do is hide it from your manager. This applies in many areas: don’t like a teammate? Feel underpaid? Other team blocking you on something? Got the flu? All of these are tractable problems once your manager knows.

Communicate well

Huge topic. Many of us love details which can make succinct writing hard. Know your audience and only include the essentials. The worlds best email gives context and a decision along with a suggestion for that decision. I view communication in 3 vectors/audiences — up (to my boss), across (to my peers), and down (to anyone who reports to me).

Clear and precise language is a critical tool for us. My favorite illustration of this is when a mathematician spends a week defining the word “context” for a room of people who all held different definitions

In the event, I never got beyond the first content slide. But not because I was thrown out. Rather, the rest of the session was spent discussing what appeared on that one slide…[I was told afterwards,] “That one slide justified having you on the project.”

So what had I done? Nothing really — from my perspective. My task was to find a way of analyzing how context influences data analysis and reasoning in highly complex domains involving military, political, and social contexts. I took the oh-so-obvious (to me) first step. I need to write down as precise a mathematical definition as possible of what a context is. It took me a couple of days…I can’t say I was totally satisfied with it…but it was the best I could do, and it did at least give me a firm base on which to start to develop some rudimentary mathematical ideas.

The fairly large group of really smart academics, defense contractors, and senior DoD personnel spent the entire hour of my allotted time discussing that one definition. The discussion brought out that all the different experts had a different conception of what a context is — a recipe for disaster.

Conclusion

These are far from comprehensive but they have helped me in coaching others and becoming a better employee myself.

There is lots of good stuff in this arena, some of my favorites are: The Effective Engineer, The Managers Path especially the part about “how to be managed”. Jackson Gabbard’s interview videos. Anything by rands.

Each area has a different sized feedback loop.

Building software has a very tight and immediate feedback loop. Get an idea, try to build the idea, fix/tweak/improve then finally ship. At the micro level its write feature, fix bug, get it worked, commit the diff and go home for the day. This all feels fantastic!

Security is essentially about understanding how software and systems work together better than the creators so you can find, and fix, security weaknesses in the aggregate system. The work here is simple enough: understand everything. And then be paranoid. The feedback loop for here takes longer, months of groundwork to understand all the moving parts¹, a deep code audit of a given subset from which you draw both bugs and lessons to apply to the rest of the codebase.

Leadership/management has the longest feedback cycle of all. This is because we as human beings are way more complex than computers. Turning an employee from a low performer into an exemplary one takes time. Months and years. It takes trust and the understanding of motivations, goals, circumstances, skills, etc, etc, etc. But when it happens I’ve found it the most rewarding of all 3.

Footnotes

1. It takes a lot of groundwork to understand how all the technical parts of a company fit together. I found I didn’t feel 100% until 6 or 9 months in to any security job.

A bug is a discrete flaw in software.

An outcome is a bug + how it was used. How did the story of the bug end? Did it lead to a breach or was it snuffed out early in development. Similar to medicine doctors ultimately care about the outcome of a patient, the type of cancer and how early it was found are footnotes to that story.

For example: an RCE bug that no one has found is less important than an XSS that is being actively exploited to harm users.

Imagine we have perfect knowledge of all the bugs in existence in our codebase¹, the breakdown might look like:

From this our goal is clear: Shift left².

The more bug outcomes that land in leftmost buckets the better for security.

We can view all security efforts through this lens.

• Static analysis run at checkin time moves a bug that would be found externally to found internally.
• Design reviews catch bugs before they become code.
• A bug bounty program provides a pressure release for bugs that might otherwise be exploited.
• Good security training moves bugs into prevented

None of these ideas are new, the SDL and manufacturing are all about surfacing flaws earlier in the process. Its worth explicitly calling out — most bugs will be found in these latter stages once software is shipped because that is the stage most software spends most of its life in: Shipped and running. This does not mean you suck at security.

Our goal isn’t the impossible one of building flawless software, it is to strike the right balance of risk vs effort. Outcomes are the best proxy I’ve found to model this for my teams.

Next article — lessons learned from using this model to drive product security work.

Footnotes

1. This is of course sadly impossible.
2. This is not novel, sdlc, manufacturing, etc all talk about this

With that came a whole new set of manager-y words that I had previously nodded along with but not deeply understood. This article is my definition of these concepts in a way engineer-Collin would understand.

Preamble

Engineers build¹ a $thing. Managers build the team that builds a $thing.

Everything revolves around each groups respective focus, including the vocabulary.

Conversation around software engineering is rooted in detail: scale, design, trade-offs. The manager-side equivalent are more abstract but revolve around: team direction, health, output, etc.

As an engineer I remember hearing these terms primarily during planning or review season.

Vision

Vision is the version of the future you want.

Roadmap

The near-term manifestation of the vision, made concrete. Comprised of projects or tasks.

Mission

The underlying purpose of a team, project or company. Doubles as a heuristic to determine if a given project/goal/task should fall to a given team. It defines the edges of your team or problemspace.

Impact

Impact is the effect you and your team have on the company/world. Revenue, cpu cycles, engineer time saved, monthly active users, data breaches avoided, etc. The magic of software is that high impact projects are often scalable.

Influence

The ability to excite, convince or motivate. Often used in service of selling a vision because big ideas are rarely achievable solo. An engineer convincing other engineers to join an effort is utilizing influence.

Vision++

Vision is one of the most interesting concepts, and one I struggled with, so it gets extra attention.

Again, vision is articulating the version of the future you believe should exist. It is often a big bold sort of pronouncement.

Vision is inherently anti-detail. This is because it defines the ends, not the means. This can be uncomfortable for someone who likes to wallow in the details, or who immediately starts thinking of ways something could go wrong.

Vision must be succinct. Like a joke if you have to explain it it has failed.

By the end of the decade, we will put a man on the moon.
– John F. Kennedy, 1962

A computer on every desk and in every home.
– Bill Gates, 1980

We will sell our excess compute capacity
 — Jeff Bezos

These are audacious ideas that are easy to get excited about, possible to define success for and succinct.

Vision is useful even if its unachievable, ex: We ship zero security bugs, ever.

Vision by no means has to come from the manager, it often bubbles up especially with some encouragement.

A clear vision has strong downstream benefits:

Seeing a SpaceX employee assembling a large part, he stopped to ask him, “What is your job at SpaceX?” He answered, “The mission of SpaceX is to colonize Mars. In order to colonize Mars, we need to build reusable rockets because it will otherwise be unaffordable for humans to travel to Mars and back. My job is to help design the steering system that enables our rockets to land back on earth. You’ll know if I’ve succeeded if our rockets land on our platform in the Atlantic after launch.” The employee could have simply said he was building a steering system for landing rockets. Instead, he recited the company’s entire “Mission-to-Metrics” framework.

— Ali Rowghani

Vision is a top level concept, from which roadmap, projects, tasks all derive.

The most interesting instances of vision, and the most difficult, is recognizing a future that has never existed. Its much easier to bring an idea you have seen work to your current company. E.x: Facebook had a great built system, lets get something similar here.

Conclusion

As an engineer your manager owes you their own definitions of the above concepts with plenty of examples to illustrate. To further understand your manager check out some good writing on the topic.

These are just my definitions for these terms right now, I am sure they will drift and change over team. I would be interested in hearing how others think about these.

Footnotes

1. Build is a stand in verb for conceive, design, build, ship, maintain and are generally responsible for

Hacker Noon is how hackers start their afternoons. We’re a part of the @AMI family. We are now accepting submissions and happy to discuss advertising & sponsorship opportunities.

If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!

Manager lingo for engineers was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

And the two young fish swim on for a bit and eventually one of them looks over at the other and goes, “what the hell is water?”

David Foster Wallace, 2005

In security, risk is that stuff all around us whether we realize it or not. This article is my take on how businesses think about risk and how that affects our domain of information security.

On day 1 hour 1 of my first security job I mainly cared how gnarly a given security bug I found was. The more complex and dangerous the better. On a good day I might have considered how the bug would affect the larger application I was auditing but not much beyond that.

In time my perspective broadened and I recognized that the root cause of insecurity is both technical and organizational.

We must ask what the engineering organization that built the software is optimizing for? Features, performance, cheap engineering talent, comprehensibility, security, are all competing concerns. Which prevails? The company that houses the engineering organization decides that. And the company makes makes those decisions based off what it sees as risks and opportunities.

So lets explore how and why companies make these business risk decisions.

Business risk

CEOs think of our security work as risk management.

From the perspective of a CEO your company faces many simultaneous risks — A fire in a manufacturing plant, a supplier going out of business, a boycott of your product or your mail spool being put on the internet must all be considered. Every risk carries a probability of happening and impact to the company.

CEOs have the difficult job of judging and prioritizing these risks, often with imperfect information. High-impact risks are called “risk factors” and are disclosed by public companies in their 10-K reports. Reading them as a security engineer illuminates what the company is most worried about, it is essentially a map.

Here is what Target reported after their breach in 2013

The data breach we experienced in 2013 has resulted in government inquiries and private litigation, and if our efforts to protect the security of information about our guests and team members are unsuccessful, future issues may result in additional costly government enforcement actions and private litigation and our sales and reputation could suffer.

Here are some of Facebooks:

Security breaches and improper access to or disclosure of our data or user data, or other hacking and phishing attacks on our systems, could harm our reputation and adversely affect our business.

Pretty straightforward. Most 10-ks you have to read between the lines. What else is in here…

If we are not able to maintain and enhance our brand…our business and financial results may be harmed.

Our business is subject to complex and evolving U.S. and foreign laws and regulations regarding privacy, data protection…otherwise harm our business.

Our products and internal systems rely on software that is highly technical, and if it contains undetected errors or vulnerabilities, our business could be adversely affected.

We can take Facebooks risks point by point and see why they need top notch security, comms and legal teams working on these issues.

Or look at this one:

The size of our user base and our users’ level of engagement are critical to our success.

This realization about the business shows why Facebook has the worlds best team at limiting spam and keeping users from being mean to one another. Users wont stick around when they have a miserable time.

Looking at Snaps S-1 filing

Our products are highly technical and…may contain undetected software bugs, security vulnerabilities…Spectacles, as an eyewear product, is regulated by the U.S. Food and Drug Administration, or the FDA, and may malfunction in a way that physically harms a user.

A security flaw that blinds someone is a bad thing.

If our security is compromised … our users, advertisers, and partners may cut back on or stop using our products and services altogether, which could seriously harm our business.

Getting hacked: also bad.

From these 10-Ks we get a clear sense of the high-level business risks. Such risks can define both how we prioritize and tell the story of our work on the security team.

Example risk profiles

Companies are all different and thus their individual risks are all different as well. Lets think through the infosec risks of three theoretical businesses

SockCo. They sell socks online. Pretty standard company. Makes money from the 10% margin of buying socks wholesale and selling them online.

SockBook. A social network for sock lovers. Makes money from advertisers paying to show users ads as they talk about socks.

MtSox. An online cryptocurrency exchange. Makes a profit whenever trades happen and takes a cut. Holds the sockCoins on behalf of others like a bank.

The risk profiles of these companies are quite different.

SockCo risk profile — SockCo worries about consumer sock sentiment, fewer people buying socks, keeping good connections with sock factories, differentiating themselves from other sources of socks via unique design, cost, bad press harming sales, etc.

The infosec-related risks here are: That someone DoSses their website. That a bug in code harms the ability to order. That someone is phished and their corporate network is taken over for ransom. That credit card fraud levels rise to harmful levels.

SockCo getting hacked would be a bad day but it would probably not kill the company. Their 10-Ks risk factors would reflect this.

Sockbook risk profile — Sockbooks revenue is directly tied to the engagement of its users. So SB must not only protect itself as a business from infosec risk but it must engender a place users want to spend time, which means it protects user experience, no spam, no graphic images and nothing that would drive people away.

Sockbook getting hacked would also probably not end the company as it enjoys the network effect. This effect is an attribute of the business but shapes how to evaluate the risk of technical security issues. Spam and abuse become first-class problems in ways they wouldn’t be at another company.

MtSox risk profile — mtSox must care a lot about information security as it holds the precious sockCoins on behalf of its users. One security flaw allows sockCoin theft and the entire company sinks.

This is the very rare case of the venn diagram of “infosec risk” and “business risk” overlapping 100%. Credit to Ryan for observation. See his Blockchain graveyard for examples of bitcoin companies blowing up as the result of a security issue.

So every company in the world has its own unique risk profile. Even within companies different sections are more or less risky, its bad if someone hacks my gmail account but much worse if they control the autonomous car that is driving me.

Conclusion

Understanding business risk lets you make better local security decisions and tie your work to the larger risks of the business.

Different companies have different risks and tolerances for those risks. Infosec risk is one among many.

Risk can be ignored, transferred, reduced (mitigated) or eliminated (remediate). The correct answer to risk is not always to eliminate it and it is our job to know the appropriate action given the many variables.

The advice to the me of a decade ago is to set down vi, nmap and IDA for a moment to read a 10-k because to make the best tradeoffs when evaluating security we must understand vastly more than just security.

References

1. Dan Geer recognized all this and explained it. In 2002. Of course.

We see security as a subset of quality. Alongside performance, usability, maintainability, correctness and cleanliness, security should be a quantitative consideration when writing software. At Uber, we have found success by following three key principles.

Solve for Yes

Culture is an important tool for improving security. It’s easy for security teams to simply say “no.” It’s much harder to figure out how to safely get to yes, but this is the job of a good product security team. At Uber, we challenge historical assumptions about what can be secured. Establishing a reputation of saying “no” with your product and engineering colleagues kills a lot of potentially work and weakens your influence with these teams.

We committed to a culture where other teams aren’t afraid of security. We don’t allow shaming. Instead, we work hard to make something both possible and safe. This attracts a lot of inbound work for our product security team, builds strong relationships with product engineers, and forces us to think critically about short and long-term challenges — — and new ways to solve them.

As a result of these efforts, the product security team Uber is a welcomed advocate throughout the development process. Before a single line of code is written we investigate the impact new builds will have on current architecture and ensure it’s not inherently impossible to secure.

During this conceptual phase, we ask questions such as:

• What data will be stored and/or used by a new service, feature, or product?
• Where is input validation done? How and where is authorization enforced?
• Is the software directly accessible to external parties, e.g. vendors, partners, public, etc.? If so, how will they access it and what controls are in place?

We work to identify hard tradeoffs that might weaken our security posture and help product teams make safer decisions. Catching issues at this stage means potential flaws are never written to begin with, and solidifies the partnership between security and engineering.

Follow the Bugs

Each bug is a lesson, a story of mistakes and a valuable data point in finding potentially larger problems. It is our job as the product security team to follow bugs back to their root and understand the deeper cause.

Software is written in stages and our security team gets involved in each one because different types of vulnerabilities can be introduced during different stages. As a result, different security strategies and processes are used during each stage to identify as many bugs as we can before code ships into production. The early we find a bug, the easier it is to fix it with minimal impact on subsequent stages. This is good for both security, and cost.

Phases of software development at Uber and some of the corresponding security tools for detecting bugs. Our goal is to identify potential security issues in the earliest stage possible.

Here’s a brief outline of what our security efforts typically look like during each stage:

• Design — We guide development away from insecure decisions through reviewing the design. This is the best stage to snuff out a bug before it ever becomes code.
• Build — Code exists but it is in flux. Finding & fixing individual mistakes in the code through advice and awareness. The product security team serves as consultants to product teams, answering questions like “is this secure?” or “how should I write this?” and proactive outreach.
• Ship — A thorough review by the product security team, if appropriate before launch. Our product security team holds itself to an SLA and we work hard to keep launches on track.
• Live — Software spends most of its lifetime in this stage, so our security efforts don’t stop at launch. In addition to ongoing internal tests, we also invite external researchers to audit our code in production through the Uber bug bounty program.

Every bug is a lesson — an indicator of inexperience or misunderstanding of how something works. This is true of the engineers writing code and us on the security team looking for bugs. Over time, these bugs provide a long tail of tactics to help improve our code, which we can enforce on new projects.

At Uber, we shamelessly follow leads from sources that routinely produce bugs — a specific part of the codebase, abstraction, or team. This helps us formalize bug patterns to catch them before they go live. Sometimes it’s a grep-based checkin rule, static analysis rule, or a specific individual needing more training. Either way, by working back from each bug, we accumulate customized tools and processes tailored specifically to the problematic characteristics of our code.

Measure & Prove Your Worth

Security can be difficult to quantify because you cannot measure an unknown. The most common perception, even within our own industry, is that security is expensive insurance and a cost center. In terms of measurement, this approach leads security professionals to ask abstract questions like, is one critical bug worth 20 low priority bugs, or if we look harder for bugs and find more is that a good thing?

At Uber, we decided to look at measurement differently and track the effort required to prevent bugs from being created in the first place. We believe security is more than stopping bad things from happening — it should be valued for its ability to unlock new types of business by providing a trusted foundation. The ultimate goal is to contribute to both the technical and financial health of the company.

We know bugs are more expensive to fix the longer they go through the development process before they’re found, and so, identifying security issues in earlier stages of the process benefits both cost and safety. This is one way we prove our worth to the business — demonstrating how our security team moves these fixes “up the chain” to earlier stages of development, ultimately preventing many bugs from ever being created.

Even bugs found in live code lead to improvements earlier in the process, maybe a static analysis rule, better education, or a library. Additionally, our bug bounty program compliments internal discovery of bugs in the wild. All of these signals can be used in reporting our value to the business by tying directly to the impact on development.

How it Works in Practice

Here’s an example of how the approach above plays out. Jinja2 RCE was reported by a security researcher through our bug bounty program in April 2016. After fixing the bug, we built a code commit rule that looks for any patterns like this in our codebase and alerts the product security team for manual review. This effectively moved detection of this bug earlier in the development process. We also modified the jinja2 engine to take out the ability to run python code inside an expression. Now, even if a developer mistakenly writes this bug again, it is defanged.

Security by Default

As more and more bugs move toward the best end of the spectrum, we prove our value to the business in a way they understand — — development time, risk, and cost. It also puts us in a position of strength for conversations about resources, tooling, and product strategy.

We settled on these guiding principles to help keep Uber’s code clean and secure. They also let the builders at our company build new and better experiences for users while achieving increasingly more security by default.

From the Ground Up: Building Product Security at Uber was originally published in Uber Privacy & Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

All of these ideas are seen through the lens of product/application security but may apply to other aspects of information security as well.

Purpose

Taking a simplified view companies hire a security team to prevent them from being hacked. Product security is responsible for the portion of that goal surrounding the security of the codebase. This is largely done through finding, fixing and preventing security vulnerabilities (“bugs”).

All security efforts are inherently asymmetric. The defenders have to win everywhere, every day but an attacker only has to win once.

Put together these two truths suggests that the ultimate goal is to eliminate, and forever prevent, all security flaws in the codebase. Lets call this goal “perfect security”. It is unquestionably an aspirational goal but for definitions of purpose it feels like a good target to aim for.

What is bug (baby dont hurt me)

Everything in product security revolves around bugs. Bugs are really a proxy for “insecurity” and perhaps more accurately a “observed example of insecurity”.

Every bug is a story and a lesson. The story describes a failure of some kind that can teach us something. Taken in aggregate security bugs localized to a feature, a framework, a pattern, a team or a codebase are big blinking arrows saying “look here”.

Bug fates

I believe the fate of a security bug is what we really care about. This is the core idea that guides lots of work.

Myspace had many XSS bugs before (and after) the samy worm but the difference was how the bug affected the company and its users. Simply tallying the raw number of XSS bugs wouldn’t reflected the impact of Samy’s finding.

Since quantity and severity don’t fully capture the impact we need to describe the outcomes.

These outcomes can be broadly categorized as:

• Exploited — This bug was used for harm
• Unfound — The bug still lurks, unfound by us or others
• Found — We become aware of this issue and fixed it
  - Found outside
  - Found by security team in code
  - Found by security team pre-code
• Prevented — A bug that is unlikely to happen again
  - Mitigated
  - Prevented

Exploited is the worst outcome for a bug and prevented is the best. Because it needs a name we can call this idea “outcome buckets”. I feel like many of us security practitioners think of things this way. Lets look at each outcome in detail

Exploited — This is bad. This is Gmail being hacked by China, this is the Tunisian government finding dissidents via lack of https. For more examples OWASP has a corpus of a decade of security flaws some of which were exploited.

Unfound — Unfound by anyone is not a good state as it is essentially a race between your internal security team and the entire world. Unfortunately if you are unaware of a lurking security flaw there is nothing you can do about it, a core part of why security is hard. Thus a primary goal for a product security team is to excavate security bugs turning them from unfound to found. Basically for bugs in this bucket there is only one action you can ever take, find the bug.

Found — There is a lot of range in this bucket and I believe sub-buckets inside it.

• Pre-code — before the insecure idea or design becomes code
• Code — Found as the software is being written or before its shipped outside the company to the greater world.
• Live/Externally — Found once the software is live often by someone outside the company like when through a bug bounty program

While finding is always good, finding it in earlier stages of the software development lifecycle is better because as the bug is cheaper to fix, safer for users and easier for an engineering team. This is the essence of the SDLC.

So our goal when finding a bug is considering how we could have found it earlier in the development process while accepting the reality that software is built in stages and as such security involvement will differ by stage. Design reviews, more or better security audits, external audits, static analysis and other tooling all apply to this bucket.

Prevented — Preventing a bug is the best possible outcome. The canonical example is sql injection being prevented by parameterized queries but I like to take it further and believe truly preventing a class of bugs includes:

• Library/framework support, like an ORM or parameterized queries
• Enforcement/safeguards in the form of commit hooks or tooling to enforce the secure patterns and surface non-blessed ways of doing things to the security team.
• Training & Culture to explain and emphasize what the secure path is and why taking it is important.

Prevention encompasses two separate ideas: mitigation and prevention. Mitigation I see as reducing the damage if an issues does occur while prevention would be reducing the likelihood of an issue ever happening in the first place.

Bug success

This is pretty abstract stuff, lets look at an example.

Imagine we have perfect knowledge of all the bugs in existence for a month of security work. The breakdown might look like this:

From this our goal is clear — move the maximum number of bug outcomes in the direction of “prevented”. Anytime we can find a bug before someone outside the company its a win, anytime we can find a security bug at all before its exploited for real harm is a win, etc.

The primary goal of a security team is to move every bug outcome in to prevented. This is our “perfect security” defined earlier.

We can view all security efforts through this lens. Static analysis run at checkin time moves a bug that would be found externally to found internally. Design reviews catch bugs before they become code. A bug bounty program reduces the likelihood of bugs in Exploited by providing a pressure release and moves Unfound into the Found category, its a two for one.

Its worth explicitly calling out — most bugs will be found in these latter stages once software is shipped because that is the stage most software spends most of its life in: Shipping and running. This does not mean you are bad at product security.

Bug reality

Like all models this one is imperfect. Key weaknesses:

• Its impossible to measure unfound bugs.
• Its difficult to honestly measure prevented bugs.
• You don’t always know when a bug is exploited.

As practitioners the exact outcome we are striving for, preventing a bug from being written in the first place, is really hard to measure. That sucks. The best I’ve found is comparing the before/after to approximate the impact of fixing a class of bugs. The best example of this I have seen is this work from the Microsoft security team where you can see the zero day flaws decrease over time due to investment.

My categorization ignores plenty of subtlety, what if a bug is found but unfixed. Prevented but regressed. Unfound but unexplainable, etc. In general though I’ve found it a useful way to view things.

Another big weaknesses is we have been treating all security bugs as equal to one another, which is obviously wrong. The most accurate answer is a blog post long but for now know that some bugs are more meaningful than others and CVSS with an emphasis on the environmental factors is a decent approximation of importance.

Opinions

If we accept all these primitives then I will share a few opinions that have guided how I’ve invested effort among the security teams I’ve lead in the past. These is all about optimizing the effort expended for maximum “security” in return.

Opinion — Shifting bugs from a less to more favorable outcome bucket is always good, but some jumps are more impactful than others. Known -> prevented is the biggest win possible. Wiping out XSS with CSP in block mode is the rare tangible, easily explainable, clear-cut security work

Opinion — Unknown -> Known is huge. All security work relies on a base layer of awareness and visibility into what is going on at the company. Awareness project X is shipping, awareness a known shaky team is actively working on a sensitive codebase, awareness or what is going is a prerequisite to prioritizing what to look at and diving deeper into concerning areas. It might not feel like “real work” to meet with PMs and managers to ensure you are aware of their roadmap but it is.

Opinion — Keep a balance of finding and fixing. Product security is unique in that we essentially make our own work. If we look harder for flaws it generates projects we could complete to then prevent those flaws. Keep in mind a balance between finding more work and completing what is already on your plate

Opinion — Measurement. Meaningful metrics are very hard. The two most useful over time to me have been:

• % of “really bad” bugs vs total security bugs
• Classes of bugs going down with investment to prevent them

There are many ways to twist metrics and in security its especially hard, Are more bugs a good thing or a bad thing? If its turning unfound into found then its good. If its simply writing more bugs overall its bad. Explaining a security team’s work to higherups can be hard, accentuating the positive, or the security as enablement, has worked for me at times.

Some good metrics for a product security team itself:

• Inbound vs outbound-initiated security work
• Engineer-hours saved with a new library or framework
• How long critical bugs were live before fix

Opinion —As a group of primarily engineers we discount the usefulness of nontechnical solutions. Socializing the impact of a bug across the company is a powerful and underused tool to improve security. Its like a seasoning, don’t overdo it but I believe most orgs are not doing it enough. I was surprised that one of the most powerful things I’ve done for security is compiling and sharing a pseudo-monthly newsletter of interesting, exciting or representative security bugs we have found in our travels through the codebase. It gets the company talking, and thinking, about security.

Conclusion

There is a lot of Fear, Uncertainty and Doubt when it comes to security. This FUD is unfortunate because it obscures, enhances, trivializes or otherwise misrepresents the true risk of a security issues and thus how best to battle and remedy them.

This is a way of thinking about security that has been useful for me but its one of many ways, I wrote it primarily to hear how other people think about this topic. Gathering this understanding of how to approach security optimally means we can work towards higher quality software which will lead to a safer world.

Thanks

Much thanks to Magoo, Jim O’Leary, Hudson, Robert Fly and Dev for talking over these ideas. Be sure to see Jim’s talk about ideas like this at Appsec California 2017.

All our work in security rests on these difficulties and this article aims to collect the specifics inherent in application security so followup articles can offer solutions. It is not meant to be defeatist.

Writing software is hard

Securing software is hard but even writing it somewhat correctly is difficult. Finding security bugs in a subcategory of finding bugs in software.

Software in a pacemaker, a satellite or a battleship should probably be perfect but for the vast majority of software time to market, more features and other goals understandably come first.

We know it is possible for us as human beings to write flawless software, it is just really hard and often not prioritized. An example optimized towards quality/zero bugs is NASA and the curiosity rover which is appropriate because the cost of a bug in that software is very high. Djbdns is another.

Unknown unknowns

Testing shows the presence, not the absence of bugs

— Dijkstra in 1969 report

If we knew all the vulns we were searching for it would obviate the need to look in the first place.

The number of vulns in the 100,000 lines of code comprising service $foo is unknown ahead of time. The universe doesn’t give us a yardstick to measure against, there is no “you have found 16/27 vulns, keep looking!” progress bar.

While writing bug free code is difficult enough if we managed to accomplish that it its even harder yet to prove that it is bug-free. Proving a negative is hard. NIST has an awesome paper about security + formal verification which is the state of the art in this area.

The absence of evidence is not necessarily evidence of absence.

Inconsistent

Even the process of measuring the full set of human-findable bugs in a codebase differs between who is looking, how, and what they had for lunch.

I ran a trial at Facebook where 10 security consulting companies audited the same code. Code my team had already carefully audited. All 10 found the same pool of shallow bugs (about half) but the remaining issues were all over the map, including one we ourselves had missed. Each person brings their own long tail of security knowledge to bear. Contrast this with something like performance (another attribute of “quality” in software) where it is trivial to measure progress.

Inconsistent is another way to say probabilistic. Like mining for gold finding security bugs involves expending effort for a possibility, not a promise, of discovery.

Opposition

We don’t always get to know who our competition is.

It is not FUD to say nations may actively be trying to do this.

I sometimes think of product security as a race. A race for us to find the lurking security flaws before someone else does who would use them for harm.

Weighting

Consistently judging the severity of a vuln is hard. We have this discussion every week at work when rewarding bug bounty issues. Is one really good account takeover bug worth 10 less critical bugs? 100?Its hard to agree on this among any two security engineers but is a prerequisite to optimizing time investment in the areas that will benefit holistic “security” the most. CVSS is currently the best answer we have to grade the criticality of bugs.

Constraints

Despite best efforts and big budgets we will never be able to deeply review the whole codebase for security flaws. If we somehow did by the time we completed it will have changed significantly. This isn’t fatalism but fact.

Timeline

Security is often on a longer timeline than you expect. Its possible to harm security with a decision whose result wont arrive for years.

Hacker Noon is how hackers start their afternoons. We’re a part of the @AMIfamily. We are now accepting submissions and happy to discuss advertising &sponsorship opportunities.

To learn more, read our about page, like/message us on Facebook, or simply, tweet/DM @HackerNoon.

If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!

Why product security is hard was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.