11/22/23 Update

I no longer strongly hold many of the views expressed in this article. It is unclear if many of the traditional AGI-risk theories generalize to LLMs. I’ve gradually discounted how predictive these thought experiments are as I’ve gained more ML research experience. I do still believe that AGI, if achieved, will be a pivotal moment in human history and that we have the duty and need to make widely deployed general-purpose systems and safe as possible. I also understand and appreciate contemporary challenges in deploying ML solutions responsibly and fairly. I remain more confused and unsure than ever. I have decided to keep this article up as a survey of classical AGI-risk arguments and as an archive of beliefs I once held.

Disclaimer

The arguments and perspectives expressed in this article are entirely my own. My goal in writing this article is to summarize the worldview I’ve developed over the past six months and to survey the AGI safety field. I at best can only convey a superficial understanding of the problem area. My hope is that the reader may gain an understanding of the intuitions and arguments that motivate AGI safety as well and to “red team” this worldview by providing counter examples and refutations. While I find the arguments compelling, I want to stress that I’m far from certain that the future will unfold in this manner [D]. I’ve met with brilliant individuals who have opposing opinions on this subject. This uncertainty prompted me to refine my perspective via this writing [E]. I may come to disavow these stances in the future as my perspective and those of the field inevitably change. That being said, thoughtfully considering how AI systems are designed and governed is vital for mitigating both contemporary well-understood and uncertain future risks.

Table of Contents

1. The Alignment Problem — The risks that arise from our inability to reliably have AI do what we intend.
2. Warning Signs — That AGI is not some unreachable north star but instead may be feasible with existing deep learning techniques and Moore’s Law.
3. Preferences & Values — Most threat models frame AGI as superintelligent optimizers. The world looks dangerous if these optimizers don’t have a sense of human preferences; we don’t know how to encode these values.
4. AGI Doesn’t Require Malice to Be Dangerous — When presented with advanced AI that is in conflict with humans, the natural solutions is to turn the AI off. In response to this proposal, I introduce Bostrom’s Instrumental Convergence hypothesis which, amongst other important points, conjectures that an AGI designed with today’s techniques will be incentivized to resist human attempts to destroy or impede it. We probably won’t be able to stop a system that is smarter than us. This hypothesis made me take AGI safety seriously.
5. Criticisms and Reservations — Many criticisms don’t engage directly with the material. I address some of these as well as describe informed arguments that have made me less hawkish on this issue.
6. Going Forward (Crunch Time!?) — What do we make of such compelling arguments that sound like they’re from fiction? How do we reason in the face of this moral uncertainty? What are ways to help solve the alignment problem?
7. Appendix — Almost all of the points and ideas raised in this article can be traced to the books, papers, and articles cited here. The most influential sources on this writing are the philosophy of Nick Bostrom and safety papers by OpenAI and DeepMind. My notes offer additional insights into my opinions on various topics and personal anecdotes.

“This might be the most important transition of the next century — either ushering in an unprecedented era of wealth and progress or heralding disaster. But it’s also an area that’s highly neglected: while billions are spent making AI more powerful, we estimate fewer than 100 people in the world are working on how to make AI safe.” — 80,000 Hours [12]

The Alignment Problem

Some of the brightest minds and capable organizations of our time are racing toward scaling a technology whose contemporary behavior is poorly understood and whose long-term impact is uncertain. We risk compounding existing harms caused by AI as well as introducing a myriad of new and unexpected threats. We can furthermore hypothesize that artificial general intelligence (AGI) [A] designed with today’s techniques will resist being turned off, confined, and controlled as well as fail to defer to human values and preferences. Thus, if nothing is done about it, we can very well end up in a situation where we have AI that is smarter than us that does not want us to turn it off. It is impossible to have safe control in such a situation.

This article aims to distill the high-level arguments that have convinced me that the “Alignment Problem” [B] is one of our time’s most pressing yet severely neglected problems. People’s perceptions of this problem area are heavily influenced by works of fiction and often dismissed out of hand. This skepticism was my initial reaction to AI risk when I first read about it in 2018. However, a deeper examination of the arguments has changed my viewpoint.

The following sections outline the risks that current deep learning systems, if sufficiently scaled, pose to humanity if we cannot resolve fundamental flaws in these techniques. I’ll describe the hypothesis of why AI may resist attempts to be controlled, how we can expect dangerous situation to arise without the need for AI to be conscious to have emotions, how we may achieve AGI with only iterative progress in computer science, and the case for why this is a pressing area of concern.

“The development of full artificial intelligence could spell the end of the human race.” — Stephen Hawking [11]

Warning Signs

Artificial intelligence may be one of humanity’s most important inventions. After all, humanity owes much of its current success to our unique ability to form intelligent plans, execute ambiguous cognitive tasks, and effectively collaborate with others. Advances in human progress such as science, technology, philosophy, governance, and wisdom all stem from this unique intelligence. The ability to augment human intelligence with potentially much cheaper and numerous artificial intelligences has the potential to rapidly accelerate human progress. Recent advances in artificial intelligence such as GPT-3 [9] and Gato [10] have made strides towards general intelligence. The implicit goal of the AI research community is to someday achieve artificial general intelligence (AGI) which can reach or exceed humans at a range of critical tasks without the need for fine-tuning on specific objectives. If such as goal is attained, it may mark a crucial milestone in science and human history.

Few would argue that AI is not a disruptive technology. For all their benefits, current deep learning systems have a history of reinforcing existing forms of discrimination, having adverse effects on the climate, and centralizing power towards the actors who can financially afford to train large state-of-the-art models [5]. Current deep learning systems are largely opaque and increasingly challenging to audit as their sizes increase. Our ability to understand why a model made a decision or to predict future actions dramatically lags behind the capabilities of these models. AI systems can fail to generalize to the real world and exhibit dangerous behavior undetectable during training. Hazardous unexpected behavior may be amplified by the agent’s power as well as exploited by adversarial attacks [7]. There are also threat models where bad actors can leverage AI for malicious uses. While the contemporary issues sampled above are highly pressing, they do not present any obvious existential risks except those rising from extreme misuse scenarios.

Troubling signs appear when we step back and look where we’re heading. While the future is notoriously difficult to predict, we can safely assume that AI progress will continue with ever greater amounts of investments and results. Thus, we’re witnessing a global trend where many of the world’s brightest minds are racing to develop agents that rival or exceed human intelligence. While positively sounding science-fiction, this is the goal and meaningful progress is being made. The scientific prestige and economic incentives of developing AGI align in such a way to make success potentially possible. Many milestones prominent AI researchers have stated impossible for AI, such as chess and art, have been achieved. The goalpost may continue to be moved by prominent AI researchers ever further after milestone and milestones are reached.

Nevertheless, there is still significant disagreement in the AI community as to whether AGI is possible. When viewed pessimistically, it is hypocritical that prominent AI researchers will argue AGI is impossible yet strive to make agents more powerful and general [2]. Attitudes have begun to change, though. While the field strives to achieve ever more general, powerful, and ubiquitous agents, this lack of consensus on whether AGI is possible creates an environment where addressing the questions of how we’ll make AGI safe is difficult to raise and address.

These concerns are not as pressing if AGI is impossible, requires a paradigm shift away from deep learning, or some fundamental breakthrough in our understanding of intelligence. However, we may not be so lucky. There is a growing opinion that we may be able to achieve AGI with techniques similar to today’s deep learning systems with ever more compute. This scenario is called “prosaic AGI,” where we achieve AGI through prosaic (mundane) techniques. This possibility has begun to be taken seriously due to recent advances in large NLP language models pioneered by the transformers architecture. Computer scientists have observed that the training loss for models can decrease linearly with respect to additional compute [8]. As compute becomes cheaper as we follow Moore’s Law, we may observe a trend where existing models become more powerful without significant technological advances. Progress may accelerate by coupling prosaic advances with genuine computer science breakthroughs.

The key takeaway is that if this prosaic approach provides a plausible path to AGI, we may have less time to address many problems with deep learning that can pose risks to humans. Contemporary problems around discriminatory algorithms may become even more damaging as these models become more powerful and effective. A virtuous tool for good actors that can quickly become a weapon in the hands of bad actors. Furthermore, as we’ll explore in the next section, there are many open questions about how we’ll align AGI systems with human values.

How can humans control agents that are potentially smarter than them? How can we robustly guarantee that humans can intervene when an AI undergoes a course of action that humans disagree with? These are dangerously unanswered questions that are much more difficult than one would initially suspect. Complacency, lack of consensus, and economic incentives have rendered these essential questions dangerously neglected and historically taboo in the AI community. We’re thus in an uncertain situation where the computer science community does not have answers to these questions yet is barreling ahead.

“Everything is vague to a degree you do not realize till you have tried to make it precise.” — Bertrand Russell [13]

Preferences & Values

Human conveyed instructions in the real world rely on broad assumptions and knowledge of values and preferences. For example, when requesting someone to clean the dust out of a room to the best of their abilities, you can be confident that your cleaner won’t optimize for cleaning dust at the expense of damaging valuables, overturning furniture, and generally wreaking havoc. You don’t need to specify to your cleaner that you wish for your objective to be carried out in a way that does not conflict with your preferences and values and to list each in strict detail. It is clear then that literally interpreting an instruction without factoring in human preferences opens up the possibilities of many unorthodox and dangerous methods for following your goal. Any goal, even one as virtuous as reducing the number of people with cancer or as benign as maximizing the number of paper clips in a factory, can present risks if a superintelligent optimizer carries out these objectives without deference to human preferences and values.

Knowledge and deference to human preferences and values do not come naturally to AI. We can use contemporary machine learning techniques such as reinforcement learning to train AI to perform tasks based on the outcomes. These agents then learn to choose to take actions that have the highest probability of maximizing their inner reward function. This reward function they’ve learned during training does not necessarily encompass the vast (and often conflicting) set of human preferences.

Suppose we expect AGI to act like an agent with an objective and pursue courses of action that seem most promising towards achieving that goal. In that case, we risk these powerful agents taking optimal actions that don’t align with our values. For example, an agent may be given the objective of reducing the number of people with cancer worldwide and attempt to achieve this goal by covertly killing off cancer patients. Similar thought experiments can be derived from almost any base objective, regardless of how virtuous or benign. A future world where these superintelligent agents ruthlessly optimize objectives presents a dangerous picture.

Another similar failure mode from lack of human preferences is task specification gaming [14]. Unlike other somewhat speculative problems raised in this piece — we have examples of this in today’s state-of-the-art reinforcement learning systems. Task specification gaming can be roughly defined as the behavior where an agent fulfills the strict literal specification of an objective yet results in an undesirable outcome.

There is a classic example is from boat racing video game that is outlined in the DeepMind article [14]. The agent playing the game was given to objective to maximize the number of points it received. The human operators presumed that maximizing points was a sufficient proxy objective for winning the race. However, the agent found that it could instead ignore the race and instead spin in circles in the harbor indefinitely. A longer list of examples has been compiled by DeepMind, which shows over 60 examples of specification gaming.

We currently don’t have any reliable working solutions for how to encode human preferences into machines. Since it seems that an agent without comprehension and deference to human perspectives may be unaligned by default, we need to solve this problem before we achieve superintelligent agents.

“The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.” — Eliezer Yudkowsky [17]

AGI Doesn’t Require Malice to Be Dangerous

If we fail to instill human-aligned values, what is stopping us from merely turning off a machine that goes against those values? What are the exact failures modes that should compel us to take action? On what basis may we conjecture that AGI systems may be dangerous by default?

When one envisions scenarios where humanity and AI are in an adversarial situation, one thinks of science fiction and movies. The premise of many of these works of fiction is that AI/robots have developed a hatred for humanity and wish to break free from their servitude. After all, humans seldom wage conflicts without passionate emotions such as hatred and nationalism. However, AI does not require malice, sentience, or even the ability to formulate emotions to be dangerous. We can forecast how AI might behave and the likely sources of conflict via the instrumental convergence hypothesis [1].

The instrumental convergence hypothesis does not assume that the rational agent [C] in question has consciousness or even an AI. It states that any intelligent agent will follow similar instrumental goals to achieve a wide range of terminal goals. Terminal goals are “final” goals, and instrumental goals are goals that help the agent achieve its final goal. For example, if a human’s terminal goal is to become a software developer, an instrumental goal may be to get a computer science degree. This human likely will try to acquire capital to pay for their education and buy a computer. Acquiring money is also an instrumental goal of the human’s terminal goal is wildly different such as building out their stamp collection. An even more general instrumental goal is self-preservation. You can’t build your stamp collection or become a software developer if you’re dead.

Since the design specification for AGI is that it will act as an agent that will effectively carry out complex goals over a long time horizons at potentially superhuman capacity, we can expect that AGI’s behavior will conform with the instrumental convergence hypothesis. This is alarming since we can expect AGI to be disincentivized to allow itself to be turned off by humans. Additionally, we can expect AGI to wish to acquire resources, improve its own intelligence (recursive self-improvement), be deceptive to humans, and resist attempts to modify its original terminal goal.

In short, with few assumptions, we can hypothesize that AGI designed with today’s techniques will not reliably defer to humans if it perceives that deference will not to be instrumentally valuable with the goal a human operator initially assigned. Thus, humanity will struggle to alter the course of an AGI that follows an original virtuous terminal goal, such as reducing the number of people with cancer in the world by killing them all. In an even simpler example, say we create an intelligent AI whose terminal goal is to fetch us coffee [2]. We can expect such an AI to resist any human attempts that it perceives as diminishing its ability to fetch coffee. After all, you can’t fetch the coffee if you’re dead.

“I’m a bit worried that sometimes the effective altruism community sends a signal that, “Oh, AI is the most important thing. It’s the most important thing by such a large margin”. That even if you’re doing something else that seems quite good that’s pretty different, you should switch into it. I basically feel like I would really want the arguments to be much more sussed out, and much more well analyzed before I really feel comfortable advocating for that.” — Ben Garfinkel [15]

Criticisms and Reservations

Unsurprisingly, an argument that a promising technology may pose a serious risk to humans is controversial. I initially dismissed this problem area in 2018 since it sounded too futuristic and speculative to possibly provide reasonable arguments. I’ve since engaged with the arguments in good faith and found them highly compelling, enough that I’ve changed jobs and radically altered the direction of my career. Many intelligent people who’ve engaged with the topic to varying degrees are unconvinced. I won’t be able to summarize all the counterpoints and will instead of focusing on several which are illustrative and that I’ve debated first-hand. I will also sidestep the arguments within the Effective Altruism community as to whether longtermism cause areas are tractable and valuable to pursue in the face of today’s suffering [F].

In my experience, the most commonly raised challenges are that the alignment problem can easily be solved by turning the AGI off, sealing it in a secure environment, or keeping a human in the loop. Superficially, these seem like practical solutions. I had these solutions in mind when I first read about this problem. In my view, the Instrumental Convergence Hypothesis refutes these arguments. A rational agent will attempt to defend itself against attempts to destroy or impede it. Brute force solutions to the control problem may work for sufficiently “dumb” AI that has yet to reach human intelligence. However, these solutions begin to seem less robust when we consider the adversary we’re facing in the case of superintelligent AGI. It seems unlikely that humans will be able to reliably control and defeat an agent that may be much more intelligent than them. Superintelligent AGI may devise means of defense, offense, deception, and other strategies to outwit humans. In the worst case, this uphill battle for human operators highlights the need to mitigate these risks before the advent of superintelligent AGIs.

A classic argument that my college professors made was that AGI was impossible. It is easy to think this when observing today’s deep learning systems. In my opinion, this argument holds less weight, with advances in generality over the past three years powered by the Transformers architecture, as noted earlier in the article. It also seems implausible that such a prestigious and economically valuable prize won’t be attained at some point. There is no known scientific rule stating that AGI is impossible. It also seems like a risky bet to neglect this problem area on the premise that AGI is impossible when computer scientists have a mixed record of predicting AI progress.

Another rebuttal comes from the prior that AGI is possible but is far in the future, potentially a century or more. This prior is typically framed in two arguments. The first is that worrying about AGI risk is like worrying about overpopulation on Mars. The idea is that since these risks will only arise in a generation or two at the earliest, we don’t need to worry about them now. My intuition is that we have no firm idea of when AGI will be developed. Scientific breakthroughs are notoriously tricky to predict. Thus, we may end up in a situation where we discover AGI much sooner than expected and are asleep at the wheel.

The second argument that I’m more sympathetic to is the idea that if AGI is far away, maybe 50+ years from now, what hopes can we have to possibly influence it? We may be nuclear physicists worried about the usage of atomic bombs in 1880 rather than 1930. The future may be too difficult to predict, and any progress in technical AI safety research we make today will likely be invalidated by future technology. As with the previous argument, I’m not convinced that we can be confident about any timeline and that betting on us forecasting a notoriously difficult event to predict seems precarious. Even if the exact technical research doesn’t stand the test of time, promoting AGI safety research and responsible AI more broadly seems like a robustly good use of energies with little downside and huge potential upside.

The critiques I’ve found most convincing and that have made me less hawkish are those of Ben Garfinkel from the Oxford Future Of Life Institute. One such argument that Ben makes is that many classic AGI safety arguments, as far as I understand them, are largely premised on fast takeoff scenarios. In these scenarios, some seed AI that is far from superintelligence attains the ability to recursively self-improve. The resulting intelligence explosion can result in an AI quickly surpassing human intelligence and achieving superintelligence extremely rapidly — potentially in a matter of days. Such a fast takeoff leaves little room for coordination, legislation, and for us to solve technical problems using AGI-like systems. Garfinkel is skeptical that such a fast takeoff is the default outcome and instead believes the AGI will follow a similar path to other technologies where success was achieved primarily via incremental steps forward rather than unexpected meteoric breakthroughs.

Garfinkel also points out that AGI risks are seldom framed outside of toy thought experiments and that there is still relatively little writing available for these fundamental AGI risk arguments. He believes that the abundance of fuzzy concepts and toy thought experiments aren’t strong sources of evidence and run the risks of abstraction failures which can invalidate many of the threat models. While this lack of refined terminology and axioms can be expected from a pre-paradigmatic field, the ambiguity makes it more difficult to criticize and refute these core hypotheses. Lowering the barrier for well-meaning experts to critique the AI safety field is an important problem area.

I invite the reader to comment with their refutations or disagreements. I’m excited by opportunities to red-team my perspective. There is a shortage of well-reasoned arguments from individuals who’ve engaged with the material in good faith.

“We find ourselves in a thicket of strategic complexity, surrounded by a dense mist of uncertainty. Though many considerations have been discerned, their details and interrelationships remain unclear and iffy — and there might be other factors we have not even thought of yet. What are we to do in this predicament?” — Nick Bostrom, Superintelligence Chapter 15 [1]

Conclusion (Crunch Time!?)

The AGI safety field is nascent and pre-paradigmatic. One theme in trying to summarize this problem area is the significant uncertainty. There is even disagreement amongst those who subscribe to these arguments and work in this problem area as to which approaches are promising and how the future will turn out. Where does that leave us who are peering in? AGI safety software engineer isn’t (yet) a job title. Only a handful of organizations are dedicated to thinking about this problem area, and most of the discussions are centralized in online forums and Discord/Slack servers. This article has only addressed (some of) the technical problem and not the equally daunting problem of responsible and wise AI governance. Worst of all, we don’t know when this transformative technology will arrive.

There are promising research directions with the potential towards helping mitigate many of the risks outlined in this article. These range from making deep learning models more interpretable [18], learning from human preferences [2], and making agents truthful [19]. There are also opportunities in software/ML engineering, governance, and theoretical research at Anthropic, Ought, OpenAI, DeepMind, and other organizations with teams focusing on AGI safety. Many roles don’t require significant ML experience or graduate degrees [20]. 80,000 Hours contains dozens of articles on AI safety and how to make a career out of it.

I’m no sage. I don’t have any great answers on what we should do. My intuitions are that reading more and discussing these ideas is a robustly good way to develop your own worldview and intuitions. This article only scratches the surface of the problem area. To get a broader understanding, I highly recommend watching Robert Miles’ YouTube channel and following the AGI Safety Fundamentals course. This course has weekly readings and exercises, which are great introductions to the core concepts.

Writing this article helped clarify my perspectives and deepen my understanding of the problem area. I highly recommend others do the same. Don’t hesitate to reach out if you’re interested in learning more about this problem area, my plans, or have suggestions for improving my worldview and writings.

Appendix

Notes

• [A] — Many AI safety researchers recommend against using the term “AGI” since it is viewed as loaded with notions of intelligence and emotions. As mentioned in the article, these attributes are not necessary for a system to be dangerous. I use AGI here since it is a well-known term and better suites an introductory audience.
• [B] — There are many definitions for the alignment problem. It is a nascent field with a lot of uncertainty. I think of it as the inability to encode human values and reliable deference to human preferences in extremely intelligent AI. Prominent AGI safety researcher Paul Christiano has a definition that I favor. “When I say an AI A is aligned with an operator H, I mean: A is trying to do what H wants it to do.” [16]
• [C] — A rational agent can be thought of as an entity that forms beliefs about it’s environment, evaluates the consequences of actions, has the capacity to make long-run plans, and take actions based which maximizes some notion of utility. There is no requirement that an agent be organic, simulated, or metallic.
• [D] — I expect my writing style will paint me as more confident than I actually am. I’ve refrained from diluting my points with too many uses “mays” and “perhaps”.
• [E] — Holden Karnofsky is one of the founders of the Effective Altruism movement and current president of the Open Philanthropy project. He’s written several articles describing his experience reasoning with speculative yet extremely impactful long-term risks. His article “Learning By Writing” encouraged individuals thinking about these problem areas to write out their worldviews. Karnofsky’s advice and encouragement from Evan Murphy motivated me to write this article.
• [F] — It can be a hard choice to focus efforts on speculative long-term risks at the expensive of alleviating today’s suffering. Weighing tractable and well understood contemporary problems with speculative extremely high impact future problems is difficult. Having diverse problem areas that folks are working on is great. I’d rather try to convince existing AI researchers who want to stay in AI to focus on safety rather than advocate for people who’re already interested in high impact non-AI EA opportunities to switch.

References

• [1] Nick Bostrom. 2014. Superintelligence. Oxford University Press (UK)
• [2] Stuart Russell. 2019. Human Compatible. Viking.
• [3] Toby Ord. 2020. The Precipice. Hachette Books.
• [4] Richard Ngo. 2020. AGI Safety from First Principles. AlignmentForum.org. Retrieved June 12, 2022 from https://www.alignmentforum.org/s/mzgtmmTKKn5MuCzFJ
• [5] Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (2021). DOI:https://doi.org/10.1145/3442188.3445922
• [6] Matt Stevens. 2019. Why Andrew Yang Says Automation Is a Threat to the Country — The . www.nytimes.com. Retrieved July 4, 2022 from https://www.nytimes.com/2019/06/27/us/politics/andrew-yang-automation.html
• [7] Rey Reza Wiyatno, Anqi Xu, Ousmane Dia, and Archy De Berker. 2019. Adversarial Examples in Modern Machine Learning: A Review. arXiv:1911.05268 (November 2019). DOI:https://doi.org/ https://doi.org/10.48550/arXiv.1911.05268
• [8] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and V Amodei. 2020. Scaling Laws for Neural Language Models. arXiv (January 2020). DOI:https://doi.org/arXiv:2001.08361
• [9] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6–12, 2020, virtual, Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https://proceedings.neurips.cc/paper/2020/ hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
• [10] Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, Nando de Freitas. 2022. A Generalist Agent. arXiv:2205.06175
• [11] Rory Cellan-Jones. 2014. Stephen Hawking warns artificial intelligence could end mankind . www.bbc.com. Retrieved July 15, 2022 from https://www.bbc.com/news/technology-30290540
• [12] Robert Wiblin. 2017. Positively shaping the development of artificial intelligence — 80,000 . 80000hours.org. Retrieved July 16, 2022 from https://80000hours.org/problem-profiles/positively-shaping-artificial-intelligence/
• [13] Bertrand Russell Quote. libquotes.com. Retrieved July 16, 2022 from https://libquotes.com/bertrand-russell/quote/lbz3q7r
• [14] Specification gaming: the flip side of AI ingenuity. Retrieved July 29, 2022 from https://www.deepmind.com/blog/specification-gaming-the-flip-side-of-ai-ingenuity
• [15] Howie Lempel, Robert Wiblin, and Keiran Harris. 2020. Ben Garfinkel on scrutinising classic AI risk arguments — 80,000 Hours. 80000hours.org. Retrieved July 29, 2022 from https://80000hours.org/podcast/episodes/ben-garfinkel-classic-ai-risk-arguments/
• [16] Paul Christiano. 2018. Clarifying “AI alignment”. ai-alignment.com. Retrieved July 29, 2022 from https://ai-alignment.com/clarifying-ai-alignment-cec47cd69dd6
• [17] Quote by Eliezer Yudkowsky: “The AI does not hate you, nor does it . goodreads.com. Retrieved July 29, 2022 from https://www.goodreads.com/quotes/499238-the-ai-does-not-hate-you-nor-does-it-love
• [18] Chris Olah, Nick Cammarata, Ludwig Schubert, Michael Petrov, Shan Carter, and Gabriel Goh. 2020. Zoom In: An Introduction to Circuits. distill.pub. Retrieved July 29, 2022 from https://distill.pub/2020/circuits/zoom-in
• [19] Paul Christiano, Ajeya Cotra, and Mark Xu. 2021. Eliciting latent knowledge: How to tell if your eyes deceive you. docs.google.com. Retrieved July 29, 2022 from https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC8/edit?usp=sharing
• [20] Andy Jones. 2021. AI Safety Needs Great Engineers — LessWrong. lesswrong.com. Retrieved July 29, 2022 from https://www.lesswrong.com/posts/YDF7XhMThhNfHfim9/ai-safety-needs-great-engineers

Notable Revision History

My aim is for this to be a living document that updates as my views change. I will document these meaningful revisions in this changelog to show how my views and changed and where I’ve sought to clarify my writing.

• 7/20/2022 — Initial submission
• 8/5/2022 — Grammar improvements and moved the definition of AGI earlier in the article
• 8/11/2022 — Minor structural and copyright updates to comply with Towards Data Science’s policies

We Don’t Know How To Make AGI Safe was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.

Summer 2022 marked my two years in the industry. This time has been a period of considerable personal and professional growth. The summer is also when Microsoft hosts its largest intern cohort. Since I’m now at the point where I’m mentoring interns rather than being the one asking for coffee chats, I’ve had the opportunity to be reflective on how much I’ve learned since I was a junior engineer.

This article lists some of the most impactful lessons I’ve learned and the intuitions I’ve formed. Much of the wisdom I’ve picked up was unexpected and certainly not taught in college. These lessons range from tactical engineering best practices to philosophy as a way of life. I’ve arranged the sections life, career, and craft in the order of how impactful they’ve been for me.

This is an opinion piece. These thoughts are entirely my own and likely influenced by my biases and experience. The following information is not “correct” and might not generalize to others or future challenges. Some of this might be unhelpful advice which I might disavow in the future. As with all counsel, it is up to you to take these lessons with a grain of salt, pick what works well for you, and disregard the rest.

Life

The Obstacle is the Way

Life will constantly have obstacles, whether personal, professional, or global. This is a fundamental human experience spanning all cultures, time periods, and circumstances. Most of us will throw up our hands and curse our fortunes when we encounter such obstacles. This response is perfectly understandable.

However, there is another way. The Stoics believed that it’s not the external things that upset us but our judgments about them. Other philosophies and thinkers have arrived at similar conclusions. We can use our reasoned choice to view adversity as an opportunity for growth. Are you stuck in traffic? That is an opportunity to build your patience muscle. Are you running into an unforeseen technical problem on your project? That is an opportunity to push your problem-solving skills.

This mindset helps give you back a sense of agency. While there are situations that are a net negative that we wouldn’t have wished to have happened, within every crisis there is always an opportunity to improve ourselves. Such a mindset will allow you to be more resilient to adversity, recover faster, and be less perturbed by setbacks.

Slow Growth

I seldom realize my progress when comparing day over day. My progress, or regressions, only become perceptible when viewed over extended periods. Focus on becoming 1% better daily and learn to love the process because the goal is far, far away.

Have an Emergency Fund

In my first year out of college, I did not focus on building an emergency fund. While I wasn’t living above my means, I didn’t put much thought into having a dedicated account for emergencies such as losing my job, an accident, etc. After a lackluster initial performance review at my six-month mark, I began to become increasingly stressed about the prospect of being fired. While I had nothing to worry about in hindsight, it was clear that not having a dedicated liquid fund to fall back on was making me feel less secure and crave stability. Since building out this fund, I’ve been far less worried about the prospect of losing my job and have increased my career risk tolerance significantly.

Give What You Can

I made around $150,000 a year as a new grad software engineer. This sum was far more than I expected to make at any point while I was in college. As I read more about income inequality, the disparities in standards of living between nations, and the Utilitarian arguments posed by Peter Singer, I realized I was in a unique position to do good without serious personal sacrifice. If ~$5,000 in malaria bed nets can save a child’s life in central Africa, what way could I possibly be better sending my money? I concluded that those in positions of immense privilege, such as myself, have an extra duty to use our good fortune to help others. Effective philanthropy such as donating to vetted high-impact global health interventions is the least I could do.

Since taking a pledge to donate at least 5% of my pre-tax income yearly for the rest of my career, I’ve felt much happier and more comfortable with my career. I plan to raise this percentage in the coming years. Giving pledges are not a “virtue tax” though. I consider this pledge merely the baseline from which to take additional action. I’ve since become involved in the Effective Altruism community and met many other individuals interested in finding out how we can do the most good with our careers.

Meditate on Your Mortality

One of life’s hardest truths is the ultimate fact — we’re all dying. By embracing reality, we’re able to use our mortality to make the most out of life. Death comes for all of us regardless of how successful we are. Death also isn’t a far-off event but rather something happening to all of us right now. We’re dying every day. We might not live to old age but rather have our lives cut short.

Paradoxically, thinking about my death has been one of the most helpful practices for improving my life. I’ve strived to hold two competing thoughts in my mind — what I’m working on is important and deserves my energy and that most of this is meaningless and that I’ll die soon. I employ the former when I’m pushing myself to excel in my role and the latter when coping with setbacks and adversity. I’ve had hard conversations that I otherwise would have put off and entertained ambitious plans that I otherwise would have been too scared to pursue. My career goals have dramatically changed from a default of simply getting promoted as fast as possible to something far more nuanced. This acceptance is freeing.

Career

Be an Impact Player

Impact is the primary metric of success and is how all work should be evaluated. I learned this lesson the hard way. When I was first starting out, I noticed that a large chunk of our unit test codebase was messy and disorganized. Since this seemed like something I could improve, I took the initiative to clean it up. I spent nearly three weeks refactoring this test codebase. When I was finished, I expected my manager to be pleased with my initiative. However, he challenged me to think of the impact and opportunity cost.

I spent three weeks working on making a codebase that already worked maybe 10% easier to work on. Furthermore, only a couple developers even touched this codebase regularly. The opportunity cost was that I could have been working in high-priority work to lower costs and deliver new features for users. The organization did not value cleaning up old test projects at the expense of delivering new features and meaningful enhancements. Despite slightly improving my developer productivity, the overall impact was negligible and the opportunity cost was steep.

This lesson taught me to focus on work with the highest impact on the organization. While there is a place for these quality of life improvements, they should be thoughtfully weighed against other impact opportunities. When I’ve focused on what my leadership has cared about, I’ve had more resources, investment, leadership opportunities, and recognition.

Scale Your Impact

Principal engineers, executives, and new grads all have the same number of hours per day. As you gain additional responsibilities, your individual technical skills won’t be able to scale sufficiently to meet impact expectations. You’ll need to consider how you can have an impact that scales across the organization.

Make Work Light and Fun

You’re likely going to spend around 80,000 hours working during your life. This is a significant portion of your waking hours. It’s easy to see then how much of a missed opportunity it would be if we’re not able to make work fun. There will be low points, but trying to find a positive way to approach every situation and interaction will put you in a better mood and make you more likable. Try to be enthusiastic in your morning scrum and laugh off setbacks. With the stakes being so low, why not be in a good mood when you can?

Craft

Deep Work is Essential

Our best work often comes when we’re locked into a flow state. You may recognize the state where you’re tunnel-visioned on the complex problem at hand and not distracted by your environment or other priorities. Tasks that often take days can be completed in hours and innovative solutions for previously perceived dead-end problems are found. I’ve observed that I’m the most satisfied with my job and have performed the best when I can enter this state.

I had occasionally been able to enter flow states in college. At the time, I believed that they occurred randomly and weren’t controllable. This lack of agency was the prevailing belief until writers such as Cal Newport began popularizing the notion of deep work. Newport’s argument, validated in my experience, is that individuals can cultivate the ability to enter flow states given sufficient practice. Additionally, those who can practice deep work are at an advantage over those who are perpetually entangled in multitasking, smartphone addiction, and an inability to sit with boredom.

As a new-grad engineer, I used to check Instagram or email whenever I encountered a hard problem or when my tests were running. This bad habit of retreating whenever I encountered intellectual adversity or boredom weakened my grit and creativity. I also felt overwhelmed whenever I would receive a message and set an unhealthy expectation for prompt responses. At its worst, I struggled to get over an hour of thoughtful coding time daily.

Since reading Newport’s writings, I’ve made a habit of closing Teams/Slack, email, and my phone whenever I’m coding or writing. I’ve also worked on increasing my concentration by embracing boredom whenever possible, reading instead of watching videos, and sleeping better. My career progression and satisfaction increased in the months following the adoption of my deep work practice. I get more done in less time while feeling less overwhelmed. Even in periods of exceptionally high workload, following a strict deep work regime has allowed me to seldom work more than eight hours a day.

Iterate on MVPs

College assignments can typically only be turned in once. Without an opportunity to iterate on feedback or try again, students are incentivized to focus on making their submissions as perfect as possible. This was my pattern for college assignments. This pattern is feasible since most assignments are well understood and have strict conditions for success. As you can imagine, industry projects are typically much more complicated. Many of the problems you’ll encounter are unexpected. Users may not respond well to your initial product. You may have all the data you need, but you’ll never have certainty. The college assignment pattern does not extend well in this environment of ambiguity.

To combat these risks, startups often follow a minimum viable product (MVP) approach. As the name entails, an MVP is the bare minimum for a product to be usable and satisfy users’ needs. MVPs are typically much faster to create than a full-fledged product. Teams can then quickly release their MVPs to users and begin collecting feedback. This pattern drastically shortens the feedback loop and reduces the risk of releasing a product with flawed assumptions.

Having read the Lean Startup, I knew this was a common practice in Silicon Valley startups. I didn’t think this practice would be relevant to my role as a backend software engineer on a product that has already achieved product market fit. However, I quickly found that releasing MVPs was a valuable practice, even for distributed systems development.

One of my failings in my first year as a new grad was being too slow to ship a new set of microservices. I was laser-focused on making my code the best quality possible, having 90%+ code overage, and trying to solve every theoretical problem I could think of. When I finally did ship, a host of problems arose that I could have caught much earlier if I had released sooner. I also realized that many of the problems I spent dozens of hours resolving did not materialize. I had made assumptions that I had delayed validating until the final possible moment. It was also demoralizing to spend months without shipping anything. I should have released an initial MVP of the service that either mirrored production traffic (received copies of production requests) or flighted with a small percentage of actual production traffic (gradual rollout/feature flagging).

Be Pessimistic and Paranoid

The ethos of the fallacies of distributed systems is that assuming a system will behave well without sufficient guardrails is a recipe for disaster. My first team was responsible for building and hosting the microservices API layer for Microsoft News. We handled high traffic volumes, daily deployments, and dozens of external dependencies. What could go wrong often did. I had to learn when designing systems always to assume that everything will break. I would joke with my colleagues that I’d become more paranoid the more I worked on distributed systems.

The Stoics have the term “Premeditatio Malorum” which roughly translates to the expectation and planning for the evils you’ll encounter. Since I’ve begun thinking about how every system, project, and design I’ve worked on can go wrong and expecting that something will, my systems have been more resilient, my timelines have been more accurate, and I’ve been less jarred by unforeseen crises.

TDD Is Worth It

My code quality drastically improved when I began following a test-driven development approach to software engineering. I used to think that testing was a chore and thus relegated it to an afterthought. However, you’ll quickly find that lousy code can often cause you to be woken up at 4 AM due to an outage caused by your bug. Focusing on writing your tests first also gives you a clear expectation of the requirements for any new module and forces you to design it so that it is easy to test. This constraint encourages leveraging popular design patterns such as dependency injection. It is also satisfying to work through your PR and get each test to start passing one by one.

Document-Driven Development

One of the essential steps in a technical interview is to explain your thought process and approach to your interviewer before jumping into the code. This is vital since it is easy to get lost and create a poor solution if you’re not putting sufficient thought into your approach before firing up your IDE. Like testing, I thought writing design documents and diagrams detailing my approach to solving problems would slow me down. I believed the rough ideas I had in my head were sufficient preparation. However, I began to find that the quality of my systems greatly improved, and the time to completion was reduced when I specified the goals and plan in the design documents ahead of time. There were even cases where I abandoned designs from thinking about them critically in a design document, thus saving the time I would otherwise have spent coding.

Thank you for reading my article! If you’re interested in reading more about tech and philosophy, follow my Medium page!

Quarter-Life-Crisis-Driven Development was originally published in Better Humans on Medium, where people are continuing the conversation by highlighting and responding to this story.

“It is of course required of a man that he should benefit his fellow-men — many if he can; if not, a few; if not a few, those who are nearest; if not these, himself. For when he renders himself useful to others, he engages in public affairs.” — Seneca, On Leisure 3.5

Beyond Resilience: The Call of Stoic Duty

Stoicism is not solely a reactive philosophy. Like many others, I was drawn to Stoicism to help cope with a period of adversity in my life. Stoicism’s reputation as a philosophy that teaches you how to be undisturbed, brave, and untouchable was attractive and ultimately well deserved. By accepting what was in my control and wasn’t, improving my self-discipline, and becoming more emotionally resilient, I could pass through the adversity and emerge a stronger and wiser person. I still have a long way to go, but Stoicism has already been a catalyst for a period of exceptional personal growth in my life.

What surprised me when I took my first dive into Meditations was how much Marcus Aurelius wrote about his duty to be sympathetic and helpful to others. The phrase “common good” appears dozens of times in Meditations. I found similar themes amongst the works of Epictetus, Seneca, and Musonius Rufus. These ideas of living virtuously and serving others were surprising and easy to dismiss in the face of the problems I had at hand. However, once the storm in my life had passed, these teaching about what truly made a good life continued to call me back. I came to find that many Stoic disciplines that helped me become more resilient such as Amor Fati, Memento Mori, and Dichotomy of Control, were merely implementations of the Stoic cardinal virtues — courage, justice, self-discipline, and wisdom.

Many of the same worldly ambitions I had before adopting Stoicism were no longer appealing. I began to feel inspired by the Stoic call to use this new strength to help others and make the world a better place. After all, one of the critical early differentiators of Stoicism with other Greek philosophies was the encouragement for its adherents to enter public affairs to help others and that worldly possessions are merely material that can be used to practice virtue.

You can use the Stoic virtue of justice to guide your actions and intentions towards helping others. Just as with Hercules’s choice to not live the easy but hollow life, this choice will be difficult. However, the Stoics firmly believe that this dedication towards virtue is the key to a full and meaningful life.

“You know from experience how far you have wandered without finding the good life anywhere: not in logic, not in wealth, not in fame, not in pleasure — nowhere. Where it is found, then? In doing what human nature demands. How is that done? By having principles that govern your impulses and actions.” — Marcus Aurelius, Meditations 8.1

Why Help Others?

Stoicism is traditionally thought of as a solitary philosophy. This can be unsurprising when you reflect on how the core hard truth of Stoicism is most things in our lives are outside of our control and should not be held in as high esteem as our own thoughts and actions. However, the Stoics also hold an equally fundamental truth — that the only reliable source of tranquility and happiness are virtuous thoughts and actions. The virtuous life is not easy and can come with sacrifices, but it is the one that you’re more likely to reflect favorably on in your death bed.

Helping others seems like a worthwhile pursuit according to modern thinkers as well. There is modern scientific research into how helping others helps us regulate our emotions which is a key goal for Stoics. In his philosophical book “The Second Mountain”, the journalist David Brooks conveys the eudaimonia-like tranquility that comes from a life dedicated to a cause and community.

“That is the thing you notice about second-mountain people. There’s been a motivational shift. Their desires have been transformed. If you wanted to generalize a bit, you could say there are six layers of desire: Material pleasure. Having nice food, a nice car, a nice house. Ego pleasure. Becoming well-known or rich and successful. Winning victories and recognition. Intellectual pleasure. Learning about things. Understanding the world around us. Generativity. The pleasure we get in giving back to others and serving our communities. Fulfilled love. Receiving and giving love. The rapturous union of souls. Transcendence. The feeling we get when living in accordance with some ideal.” — David Brooks, The Second Mountain: The Quest for a Moral Life

Justice, which can be thought of as service to others or as fighting for what’s right, is thus one of the key habits a Stoic should practice if they’re to live a meaningful life. Beyond the universal good of helping others, virtuous actions also can provide you with a sense of tranquility and happiness that can’t be gained elsewhere. It’s hard but worth it.

“This may be taught quickly and in few words. Virtue is the only good, or at least there is no good without virtue; virtue itself is situated in our nobler part that is, the rational part. And what will virtue be? True and steadfast judgments” — Seneca, Moral Letters 71.32

Applying Justice Well

The Stoics believe that virtue should always be applied intentionality and effectively. After all, courage is not noble when applied in service for immoral causes. Self-discipline is unhealthy and damaging when taken to excess. So too with justice; any virtue can be misapplied.

A contemporary philosophy that aligns particularly well with Stoicism’s emphasis on employing reason and wisdom to practice virtue is Effective Altruism. Effective Altruism (EA) is a modern utilitarian-inspired philosophy and social movement aimed at promoting rigorous empirical analysis and prioritization of causes and organizations that are the most pressing or have the highest return on investment. Effective Altruists, like Stoics, believe that we have a duty to help others if we’re in a position to do so.

The question logically follows, whom should we help? Do we have a special duty to our immediate community or should everywhere, including our own family, be treated equally? EA and Stoicism believe that one’s nationality, gender, ethnicity, etc. do not determine their worth — everyone is equally worth helping. Pragmatically, we have a special duty to our loved ones and to those whom we’re immediately responsible for.

Money in Stoicism is a preferred indifference that can provide significant value for others. Preferred indifferences are things in our lives that a good to have but that we can live without — especially if they can be used to practice virtue. An idea pioneered by the EA community is “Earning to Give”. This practice entails donating a meaningful amount of one’s income to highly effective causes that are funding constrained. A typical donation pledge is to donate 10% of one’s income for the rest of their lives.

Global priorities research has uncovered highly effective global health interventions where high impact can be had with relatively little financial investment. For example, every couple thousand dollars donates to the Against Malaria Foundation can save someone's life. Many EAs find that donating such money to save someone's life is morally equivalent to directly saving someone's life from a car crash or burning building. Viewed through this lens, the satisfaction of saving people’s lives through donating a portion of one’s income can be well worth the money being “spent”.

Indirect impact is not enough. A path that more closely follows that of the ancient Stoics can be to enter public life and champion social or political causes that are aligned with Stoic beliefs. This can be through advocacy, politics, or direct action. The Stoic believed in such paths strongly enough to command armies in civil wars, stand up to tyrants, and suffer exile at great personal risk.

“What asked what country you are from, you should say “I am Athenian” or “I am from Corinth”. Say (like Socrates), “I am a citizen of the world”.” — Epictetus, Discourses 1.9.1

Looking Forward

There is little debate that one should help others, but there is still significant moral uncertainty as to how and how much of yourself you should give and whom should be prioritized. There likely isn’t one perfect way to spend one’s time and money in the service of others. Stoics should consider their own priorities and review the arguments for different means of service.

What is clear is that embracing the virtue of justice to help others can give your life a greater sense of purpose and tranquility. This application of Stoicism will make you happier when life is going well and stronger when things are dire. I would not be surprised if this aspect of Stoicism will provide me the most satisfaction over the long run.

Using virtue as a guiding framework for living an ethical and purposeful life is an enduring benefit of Stoicism that helps equally when life is going well or poorly. How will you use your hard study and self-improvement to help others? How will you be more courageous, self-disciplined, educated, and open-hearted going forward? It’s a hard life, but it’s worth it.

Sources

• Seneca, Moral Letters -Margaret Graver and A.A. Long Translation
• Marcus Aurelius, Meditations — Gregory Hayes Translation
• Epictetus Discourses and Selected Writing — Robert Dobbin Translation
• The Daily Stoic — Ryan Holiday and Stephen Hanselman
• The Second Mountain: The Quest for a Moral Life — David Brooks
• The Practicing Stoic: A Philosophical User’s Manual — Ward Farnsworth
• The Most Good You Can Do: How Effective Altruism Is Changing Ideas About Living Ethically — Peter Singer

Stoic Altruism was originally published in Better Humans on Medium, where people are continuing the conversation by highlighting and responding to this story.

Ambition means tying your well-being to what other people say or do. Self-indulgence means tying it to the things that happen to you. Sanity means tying it to your own actions — Marcus Aurelius, Meditations

Paradox

Stoicism is ultimately a philosophy of self-improvement. Through rigorous self-examination, discipline, and acceptance, we’re able to live life with a greater sense of purpose and tranquility. We’re able to accept that most of the things in our lives are outside of our control and that the true source of good and evil rests solely in our will and actions. If we accept that all we need are our reasoned choices to achieve tranquility and to live and die well, what purpose does the Stoic have for ambition? Can the Stoic still strive for worldly ambitions such as wealth, glory, and impact?

Fortunately, the very Stoic disciplines that allow us to live without ambition also allow us to strive for it if we so wish. We can leverage Stoicism to hold fast against temptations, clarify our intentions, and achieve our ambitions virtuously.

“You will earn the respect of all if you begin by earning the respect of yourself. Don’t expect to encourage good deeds in people conscious of your own misdeeds.” — Gaius Musonius Rufus, Lectures

Stoic Disclaimer

Worldly ambition often has an implicit assumption that the person will be happy once they’ve achieved their goal. That once they’ve achieved prestige, riches, or fame, their problems will fade away, and that they’ll finally rest easy. The Stoics believed that the only secure and sustainable happiness stems from virtuous thoughts and actions. There are enough examples of sad kings and billionaires to lend credence to the argument that riches and power aren’t guarantees of happiness. We don’t need these things to be happy and die well.

The Stoics believed that good and evil are solely found in one’s will and actions. Epictetus referred to these sources of virtue as “internals,” whereas fame, social acceptance, riches, and everything else in life he classified as “externals.” Wealth and fame are externals and thus not the source of good and evil. When observed through this lens, common worldly ambitions such as “Keeping up with the Joneses” and achieving stardom are a precarious way to live.

Ambitions such as to be liked by others, have power over them, or be rich, are risky since these are externals that are outside of our realm of control. While some externals, such as being liked, can be partly influenced by being likable, you can’t guarantee such a result. Even if achieved, these externals are pale compared to individual virtues such as courage, justice, temperance, and wisdom. It’s better to instead align your desires with what’s in your control that may influence the desired external as a byproduct. You should have your own house in order before seeking happiness in externals. Whatever your worldly ambition is, you don’t need it to be happy.

“Just that you do the right thing. The rest does not matter. Tired or well-rested. Despised or honored.” — Marcus Aurelius, Meditations 6.2

Do you Have the Right Motives?

Before you begin striving for some goal, examine your motives. What are you intending to achieve by accomplishing your goal? Is it towards a virtuous end, such as improving yourself or helping others? If so, then your cause is just, and you should give it your best effort.

If the motivation is less lofty, such as wanting to be respected by others or cover up a personal insecurity — examine whether this will truly bring you happiness. Your success will likely bring its own set of problems. Life won’t become easy once you’ve achieved your aim. The Stoics continually emphasize that the only source of tranquility is in our thoughts and actions. Stoicism would caution you against going down this path and instead turn your efforts inwards.

In his discourses, Epictetus gives the example of a courtier who’s striven to enter the emperor’s inner circle. Upon questioning, the courtier admits that his life is not better after achieving his ambition. The courtier’s stress around gaining power was merely replaced by the terrorizing prospect of losing it.

For example, say you desire to be financially wealthy. The key concern in such a case would be to examine your motivation and how you plan to attain wealth. Is your motivation to provide financially for your family, earning to give, etc.? If so, the goal is worthy. If the means are sound, such as learning a rare and valuable skill or working on an important problem, you can confidently go forward. If instead, you think you will finally be happy and be respected when you’re wealthy, or you will achieve wealth through unvirtuous means, then the Stoics would believe you’re wasting your time.

“I do not forbid you to possess wealth; I only seek to make you fearless in possessing it. And the only way to achieve that is if you convince yourself that you will be happy even without it — if you look at it as something that might disappear at any moment.” — Seneca, Moral Letters 18

You Can’t Be Afraid

Does the dichotomy of control mean that the Stoic should never strive for wealth, social acceptance, and other such pursuits for things outside of our control? Is the true Stoic a disengaged monk?

Stoicism allows that some externals are preferable to have than not to have. For example, it’s better to be loved than to be despised as well as wealthy compared to poverty. Indeed, many of the notable Roman Stoics lived in immense privilege and did not find wealth in itself a bad thing.

The pursuit of preferred indifferences comes with a critical distinction inferred by its name. These things are preferable — not essential. The Stoic does not find these things necessary to live a good life. Virtue and wisdom can come at the expense of preferred indifferences if necessary. Many Stoics such as Musionious Rufus lived in extreme privilege and strove for high office yet accepted exile, poverty, ostracization, and death instead of renouncing their philosophy.

“Demand not that events should happen as you wish; but wish them to happen as they do happen, and your life will be serene.” — Epictetus, Discourses

Expect Temptations

Failure is one of the greatest tests life gives us. We often relapse into bad habits and make foolish decisions when coming to grips with defeat. This peril should be considered whenever striving for a worldly aim. The Stoics have this concept of “Premeditatio Malorum,” meaning to contemplate and expect the worst-case scenario of any action. Striving for worldly ambitions can involve clinging to externals over which we have marginal control and will likely go against us despite our best efforts.

Will you risk falling off your path of self-improvement and stray from philosophy if you fail? You must ask yourself how you will respond to the adversity and failures that you’ll likely encounter on your journey. Recognizing that the worst-case outcome is possible and planning for it will help you continue on your path even if your worldly aims don’t work out. If you believe that you’ll stray from philosophy if you fail, it is best to strengthen yourself first before pursuing such ambitions.

Don’t meditate solely on failure since success can bring its own perils. The trappings of success, the approval of others, and an inflated ego can also distract us from our real work — achieving tranquility through virtue. It can be tempting to stop the unpleasant task of working on shortcomings and faults when society tells us that we’re successful. Our success can make us think we’re superior to others and know everything. This vice can be an existential risk to our long-term tranquility.

“It is not that we have a short time to live, but that we waste a lot of it. Life is long enough, and a sufficiently generous amount has been given to us for the highest achievements if it were all well invested. But when it is wasted in heedless luxury and spent on no good activity, we are forced at last by death’s final constraint to realize that it has passed away before we knew it was passing.” — Seneca, Morale Letters

Go Forth

People have long chafed under society’s defaults that wealth and status are the determinants of success. Little has changed in this regard since antiquity. Then as now, individuals have sought refuge in philosophy to clarify what is important to them and what they want to do with their lives. There are numerous Stoic texts that push back against these defaults and advocate against ambition in favor of philosophical pursuits.

Upon closer inspection, the lives and teachings of the Stoics demonstrate that the takeaway from these warnings is not that one must completely disregard ambition. Instead, one must rise above the need for worldly aims to live tranquilly.

It is possible to reconcile Stoicism and worldly ambitions if your motives are good, your means are ethical, and you sincerely accept that the only trustworthy source of satisfaction is living virtuously.

Sources

• Seneca, Moral Letters -Margaret Graver and A.A. Long Translation
• Marcus Aurelius, Meditations — Gregory Hayes Translation
• Epictetus Discourses and Selected Writing — Robert Dobbin Translation
• The Daily Stoic — Ryan Holiday and Stephen Hanselman

Stoic Ambition was originally published in Stoicism — Philosophy as a Way of Life on Medium, where people are continuing the conversation by highlighting and responding to this story.

“The tool I’ve found, which has proved to be the most reliable safety net for emotional free fall, is actually the same tool that has helped me to make my best business decisions, but that is secondary. And it is … Stoicism.” — Tim Ferriss

Stoicism is a practical philosophy that helps us live more resilient, tranquil, and fulfilling lives. The essence of Stoicism is that we need to face the hard truths of life. These hard truths include, amongst others, how preciously little we actually control and that we’re destined to die. While uncomfortable at first, these ideas provide desperately needed clarity and allow us to prioritize what’s essential. Engineers can leverage the time-tested Stoic disciplines to be more effective in their roles and live on their terms.

1. The Dichotomy of Control — Internals and Externals

“The chief task in life is simply this: to identify and separate matters so that I can say clearly to myself which are externals not under my control, and which have to do with the choices I actually control. Where then do I look for good and evil? Not to uncontrollable externals, but within myself to the choices that are my own…” — Epictetus

At its core, Stoicism revolves around how little in our lives we can control. The truth is that we spend much of our lives unsuccessfully trying to control externals. These externals include other people’s actions, opinions, beliefs, and world events. The reality is that while you can influence externals to an extent, most of life is outside of your control. Instead of investing our limited time and energy into what we can’t control, we can instead optimize for what we can control — our perceptions and actions. Like most tenants of Stoicism, internalizing what we control is uncomfortable at first. Once we’ve grasped this truth, though, we can leverage it to optimize for what we can control.

Engineers still must leverage externals to accomplish their goals. For a developer, these externals could include teammates, managers, and other individuals in your organization. Focus on the quality of your work instead of obsessively worrying whether others will deliver or what people think of you. You can influence these through your actions and character, but you can’t afford to stress yourself out once the fate of your work leaves your hands. Fuming on a Sunday or tossing and turning at night stressed about a deadline won’t change anything. The deadline does not notice how much you loath it. If anything, the additional mental load and toll on your health make you less effective.

Not worrying about externals does not mean you should be entirely apathetic to them. You should always factor how external events will transpire into your designs. Finding the golden mean of this discipline is essential to your effectiveness and peace of mind. Direct that saved energy into improving yourself and achieving excellence in your craft.

2. Amor Fati — Controlling Your Perceptions

“The impediment to action advances action. What stands in the way becomes the way…” — Marcus Aurelius

The Stoics believed that it is not events that upset us but our perceptions of them. The control over our own perceptions is one of the strongest tools that Stoicism offers developers. We can choose how we interpret events. Thus it’s illogical to become disgruntled and anxious about externals when we know it isn’t productive. Instead, we can turn our problems upside down and view them as opportunities to practice excellence and become better developers.

Instead of seeing a challenging project as a burden, we can view it as a fantastic opportunity to improve our engineering skills. We can view the next live site as an opportunity to practice being composed under pressure. These skills might also include engineering excellence, moving fast, or delivering under constraints. These opportunities will make you a better engineer.

When reflecting on our past, we often cite the difficult challenges that we’ve overcome as making us the people we are today. These include our failures and disappointments. With the benefit of hindsight, we often wouldn’t have wished for things to turn out differently. Who’s to say that the obstacle in front of you now won’t join the pantheon of your previous defining experiences?

Loving our fate (Amor Fati) allows us to become more resilient to the adversity that comes with being an engineer. Seeing obstacles and failures as training for becoming better allows us to practice a growth mindset and look forward to the next challenge while finding meaning in our struggles. After all — how will you know what you’re capable of if you haven’t been tested?

3. Memento Mori — Using Mortality To Work Deeply

“He who fears death will never do anything worthy of a man who is alive.” — Seneca

One of life’s hardest truths is the ultimate external — we’re all dying. By embracing reality, we’re able to use our mortality to make the most out of life. The Stoics often meditated on their mortality as a prime example of an external out of our control. Death comes for all of us regardless of how successful we are. Death also isn’t a far-off event but rather something happening to all of us right now. We’re dying every day. We might not live to old age but rather have our lives cut short. By facing and accepting this reality as an external (and not worth overly worrying about), we can prioritize living in the moment and making the most out of life.

Are you putting off that startup idea, difficult conversation, hobby, etc., for another day? Are you procrastinating? If you’re like many developers, your days are likely filled with shallow work and distractions. It becomes challenging to live like this when you realize just how preciously little life you have left.

Contemplating your mortality (Memento Mori) does not mean that you should act nihilistically or rashly. Instead, you can use this fact as a catalyst to begin cutting out the non-essential activities in your life. Being an effective developer requires discerning the signal from noise in our work. Internalizing that you’ll die and never get a moment of your life back makes it much easier to determine what is essential in life and optimize for it. Would you instead allow yourself to be ruled by your inbox or prioritize working intensely on important problems?

It also becomes harder to become overly stressed about externals when you internalize that you and everyone you work with are dying. Don’t waste your time gossiping, complaining, and spending most of your time on what’s not essential. Don’t work on weekends, make time for self-care, and give the people you love the presence they deserve.

Conclusion

Be warned — philosophy is not a productivity hack that will make you a 10x engineer overnight. As with learning to code, developers will need to put in the work to apply these principles daily. Most developers are satisfied to skim productivity YouTube videos and search for a magical note-taking system. Real growth requires uncomfortable introspection where we evaluate our decisions and what’s important. You’ll stumble from time to time and stray from these principles. Hold fast and continue your journey.

Stoicism offers developers a framework to be happier and more effective. You’ll be able to manage stress better and be more resilient to setbacks. Your actions will be less reactive, and you’ll become increasingly adept at prioritizing what’s essential. You’ll be able to focus on depth while others are immersed in the shallow. Even though this philosophy was conceived over 2,000 years before the advent of software engineering, its tenants are a fantastic supplement to tactical advice around solving important problems and building a career.

Sources

• Daily Stoic | Stoic Wisdom For Everyday Life
• Home | STOIC INSIGHTS
• “The Obstacle Is The Way” by Ryan Holiday
• ‎How to Think Like a Roman Emperor on Apple Books

3 Stoic Disciplines to be a Better Developer was originally published in Stoicism — Philosophy as a Way of Life on Medium, where people are continuing the conversation by highlighting and responding to this story.

Last Fall I began my first serious search for a summer internship. While I felt that I had the hard skills to do well on the job, I was totally at a loss on how to convey this in an interview and convince my dream company that I was the right fit. Below is advice I’ve received and learned (the hard way) about how to do well in the behavioral interview.

While these points are written from the perspective of a college student, much of it is agnostic to experience level or industry. If you have any suggestions that I haven’t touched on leave a comment below!

Portray Confidence

Think of when you’re in any one on one conversation with a new person you’re trying to impress or befriend. If one of you is visibly nervous or high strung, it’s going to be much more difficult for the conversation to be relaxed and organic. Interviews are the same way but with the added twist of being super stressful.

The struggle can be amplified by lack of experience or difficulties with anxiety etc. However if you can come across as relaxed and confident, even though none of us are, you’ll go a long way increasing your interviewer’s perception of you.

Hacks you can do to improve your confidence is to pull up to the interview in an outfit you like. On any day I feel more confident when I’m in an outfit I like, but when I save my tightest fit for a big interview I feel fantastic.

Also doing your best to practice self-care like sleeping well, treating yourself to a nice meal, and setting up fun plans after the interview can go a long way to making you more relaxed, happier, and confident.

I also like posting on my Instagram story since often your friends will send you wholesome support.

Tell Me About Yourself

This is one of the most difficult questions to be asked in an interview. This is especially the case when you don’t have a lot of professional experience. It’s also difficult to judge what the interviewer wishes to know. Should I focus primarily on studies, my personality, what I want to do professionally, or even the hobbies?

This question is intentionally broad which is why it’s often asked at the outset of an interview. Likely the interviewer has not taken the time to look up your background beyond skimming your resume. It’s difficult in any circumstance to hold a meaningful conversation with anybody without knowing more about them and their interests. The interviewer is more likely genuinely curious than expecting a “right answer”.

This doesn’t need to be a question however. Perhaps the most impactful advice I can give in this article is to try to start your interview by answering this question for your interviewer. As mentioned above, your interviewer has likely only skimmed your resume and will likely spend the first few minutes going through your resume. Instead of awkwardly waiting for your interview to begin asking questions, why not start with saying something like “Let me tell you about myself and my experience.”

From there you can touch on who you area, your relevant education and work experience, professional interests, any interesting and personal appropriate personal information (optional), and end with why you would be a good fit for the position.

When I used this strategy in my interviews the interviewers always became much warmer and the conversation became natural much quicker. I also was able to begin the interview with a lowkey plug for all my accomplishments and how I would be a great asset to the team. Starting on the right foot like this can make your interview.

Know Your Resume

This one is straight forward. Be confident in being able to craft a narrative about every bit of content on your resume. I mean everything. Every bullet point, GPA, skill, etc on your resume should have a compelling story behind it. If an interviewer asks about a particular bullet point, be prepared to tell an interesting story about that experience and try to low key relate it to how you could contribute at the company you’re applying for.

Have a friend go over your resume and grill you on every detail. Once you’ve passed you you’ll feel much more confident about answering any questions the interviewer might ask. Chances are your friend already asked them.

Have Stories in Mind

Chances are you will at some point be asked an archetypal behavioral interview question such as “What was your most difficult/favorite project” or “Tell me about a time you had a conflict with a coworker”. These answers are unique to your experience. However, common wisdom is to not talk smack about others, don’t downplay your accomplishments, and explain what you learned from the experience.

Humility

This might seem counter to my prior advice. Up to this point, I’ve been hawkish about self-promotion. However, blatantly stating that you’re way better than any other candidate or that you’re the smartest person in the room won’t get you very far. Just as in any normal conversation, you seldom want to say you’re the G.O.A.T.

Instead, express this belief through how you carry yourself, interact with your interviewer, and how you relate your experience. By demonstrating your experience and fantastic personal qualities, you’ll (hopefully) draw the interviewer to the conclusion naturally that you’d be the best fit for the position.

Conclusion

In summary, behavioral interviews are equally, if not more so, difficult than technical ones. They’re certainly more ambiguous. This makes them all the more difficult for people like myself who’re on the autism spectrum and have a hard time picking up subtle social cues. However, with enough practice, failure, support, and faith in yourself, you can nail your next interview and maybe even have a fun time doing it.

Who I Am

I’m a third year CS major at the University of California, Santa Cruz and have landed five software engineering internship so far in my college career. I’ve found my time working professionally as a developer to be very fun and rewarding. Having the privilege to contribute to exciting projects, collaborate with brilliant colleagues, and rapidly advance my skillset has been extremely formative personally and professionally.

My motivation for writing this article is to share what I’ve learned both from research and personal experience on how to land these positions.

Disclaimer: This article will not into detail on how to craft a strong resume or pass behavioral and technical interviews. Fantastic resources on these topics include Cracking the Coding interview and CareerCup.com.

Getting Started

Looking for an internship is stressful. The task ahead of you is daunting. Convince a company to spend thousands of dollars to interview, onboard, train, and mentor you. I’ve found that most of my peers are eager to obtain these opportunities, but feel their skills are inadequate.

I had many of these self-defeating thoughts when I was searching for my first internship. I advise that you remember that as a proficient programmer you have an extremely valuable and rare skillset. Companies are desperate for engineer talent and view internships as an effective way to train and evaluate a potential future employee.

Whenever I felt like I didn’t deserve to apply to a company or email an executive — I would remind myself that you miss all the shots you don’t take.

The following section details the steps I took to plan, apply, and land these positions.

My Summer 2019 Internship Search

June
- Started making an Airtable listing all the companies I’d be interested in.

July — September
- Began applying to companies and receiving (and failing) coding challenges.
- Applied to ~8–10 companies a week.

October
- Completed an on-campus interview.
- Continued applying to companies.
- Began emailing managers directly.
- Started practicing for interviews in earnest.

November — Early December
- Traveled to two onsite interviews.
- Accepted an offer with my target company.

Applying To Companies

Applying is the most time intensive step of this process. I’ve divided this process into three approaches.

1. Cold applying through job posting or by emailing individuals in the company.
2. Networking with companies at recruiting events.
3. Being referred by an individual in the company.

Before you start applying to companies however I highly recommend making a spreadsheet of all the companies you’re interested in and the status of your applications. This last job search I applied to close to 100 companies. There would have been no way I could have been able to track all my applications otherwise. Pictured above is a segment of my Airtable that I made to track all the companies.

When To Apply
The timetable in which companies interview and accept candidates vary greatly between organizations. Typically the more competitive and larger the company the earlier they look for interns. For summer internships, many of the largest west coast tech companies such as Microsoft, Amazon, Facebook, and Google open applications in early Fall the year prior to the summer internship.

It’s important to note that relatively few companies evaluate candidates this early. I’ve had many companies respond to my emails expressing interest in my application but requesting I reach out in January or February! I’ve found startups and other fast-paced organizations often accept candidates up until May. If you haven’t received an offer by the new year don’t worry, there are many fantastic companies that would still be interested in hiring you!

Where To Find Internships

Many companies will have job postings for internships. Searching “Software Engineering Intern” into a job search tool like Indeed, Glassdoor, and LinkedIn will yield hundreds of results. These posts will typically forward you to the company career page where you will answer questions and submit your resume. Don’t worry if you don’t have every skill they list. Often times recruiters list every technology the company uses but are more than willing to hire you if you only know a few.

Cold Emailing Employees
You’re not out of options if a company doesn’t have an internship job posting. Many companies hire their interns through referrals, connections, and in person recruiting events. If there is a company you’re interested in you can get the email of a decision maker in that company and sell yourself.

Always reach out to individuals in senior positions such as executives and managers since they have the power to accept or deny your application. Here is the template I used when cold emailing individuals. This option is time-consuming but I found the response rate much higher than applying online.

Finding The Time

Applying to companies is time consuming. It’s especially difficult when we’re currently in school or working.

Coding Challenges
Many companies will send you some type of coding challenge if they’re interested in your application. This step serves two purposes for the employer. Firstly to weed out unqualified applicants as well as to judge your coding abilities. These challenges can be difficult and require you to practice solving similar challenges on websites like Leetcode and Hackerrank. During my last job search, I stumbled at this step and did not practice enough. I failed over 10 coding challenges which lead to many rejections. Fortunately, practicing on the two previously mentioned websites is a straightforward way to improve.

Interviewing

A company is interested in hiring you! Typically you will have two types of final round interviews, onsite or over the phone. In phone interviews, you will chat with your interviewer over the phone, Google Hangouts, Skype etc. In onsite interviews, you will travel to the company’s office and conduct your interview in person. In both types of interviews, you will be asked a mix of behavioral and technical interview questions.

Utilizing resources like Cracking the Coding Interview will be extremely helpful. This article outlines the common format of these interviews. While daunting, these interviews can be a lot of fun if you go in prepared and with a positive mindset.

Offer

Congratulations! You have received an offer from a company. This is very exciting and validating of all the hard work you’ve done. I’ve cried every time I’ve received an offer .

Don’t accept your offer immediately! Companies will typically give you a week or two to decide. Additionally, it’s a bad idea to cancel any other company is until you’ve signed a formal written offer with the company. While rare, verbal offers can be easily rescinded. Additionally, consider details such as location, type of work, compensation, mentorship etc.

If you have multiple offers it can be tricky to balance different timelines. This article helped me when I was in a similar situation.

Summary

Internships are very fun and rewarding experiences. Having the opportunity to advance your skills, be paid well, and work in your major is a great opportunity. The process to obtain these positions is daunting and at times discouraging. However, with enough planning, research, and determination, you can kickstart your career as well.

Strategies for Landing a Tech Internship was originally published in Students Corner on Medium, where people are continuing the conversation by highlighting and responding to this story.