<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ben Kozary, Ph.D. on Medium]]></title>
        <description><![CDATA[Stories by Ben Kozary, Ph.D. on Medium]]></description>
        <link>https://medium.com/@benkozary?source=rss-6fa08adf767f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*4TbyU1Qt9i6gcDycEUfLqA@2x.jpeg</url>
            <title>Stories by Ben Kozary, Ph.D. on Medium</title>
            <link>https://medium.com/@benkozary?source=rss-6fa08adf767f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 16:05:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@benkozary/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[MONEY FOR NOTHING]]></title>
            <link>https://medium.com/@benkozary/money-for-nothing-754aec2e12b0?source=rss-6fa08adf767f------2</link>
            <guid isPermaLink="false">https://medium.com/p/754aec2e12b0</guid>
            <category><![CDATA[strategy]]></category>
            <category><![CDATA[psychology]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[academia]]></category>
            <category><![CDATA[behavioral-science]]></category>
            <dc:creator><![CDATA[Ben Kozary, Ph.D.]]></dc:creator>
            <pubDate>Wed, 05 Jul 2017 00:00:00 GMT</pubDate>
            <atom:updated>2018-01-20T06:43:08.028Z</atom:updated>
            <content:encoded><![CDATA[<h3>How collaboration between academia and industry can bolster the rigor, relevance, and value of research in both sectors</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/1*eFm5jri48u6uOJQdrAmHAA.png" /></figure><p><strong><em>Only have two minutes? Here are the key takeaways.</em></strong></p><p><strong><em>Want the full story? Read on below.</em></strong></p><ul><li>Initiatives introduced into the broader social psychology field in the last 5 years, in the wake of several scandals, have been beneficial. The field is now heading in the right direction, with rigor as a focal point for academic research. The next priority should be relevance — both to other academics, and for industry practitioners.</li><li>Academia is not currently asking the right questions to effectively collaborate with industry practitioners. The broader social psychology field is too focused on the <em>existence</em> of effects, whereas organizations are more interested in the <em>magnitude</em> of effects, and how these affect their business.</li><li>From the industry side, despite substantial (and growing) worldwide spend on market research, much of the output goes unused. There are typically three reasons for this outcome: 1) a lack of stakeholder buy-in, client-side; 2) the inability for clients to implement initiatives based upon the insights provided; and 3) the research does not solve the client’s business challenge. Put simply, industry also suffers from issues of relevance and rigor.</li><li>The first step in collaboration is building rapport and cultivating a relationship — and as with any successful relationship, it is necessary to find common ground. Mutual benefit comes from understanding and empathy which, coincidentally, are the foundations of design-driven innovation.</li><li>Academics can bring vast knowledge, new perspectives, and scientific rigor to industry practitioners. Industry practitioners can provide a practical lens to the heavily theoretical academic sector, and have far greater access to data than academics — both of which can help academics with securing funding and publications. Collaboration between the two sectors will ultimately enhance the rigor and relevance of both academia and industry.</li></ul><p>— — —</p><p>Operating within paradigmatic boundaries is not something I am particularly good at. My Ph.D. was completed in a business school, but my dissertation was grounded more in social psychology than it was in marketing, and borrowed quite heavily from several other fields. This breadth of scope can be attributed to my belief that innovation and progress stem from a cross-pollination of ideas and perspectives — and that one cannot “think outside the box” if constrained by artefactual disciplinary or sector boundaries.</p><p><strong>Stepping back in time…</strong></p><p>To that end, I feel that academics and industry practitioners have a great deal to offer one another, and that by understanding each other’s respective goals and challenges, it is possible to bridge the gap between the two sectors. The aim of this post is to facilitate some of this understanding for mutual benefit; specifically, to emphasize the importance of rigor and relevance of research conducted in both sectors, thereby maximizing its value.</p><p>In early 2014, <a href="https://indecisionblog.com/2014/04/07/viewpoint-why-im-leaving-academia/">I wrote a post about my decision to leave academia to pursue a career in industry</a>. I had become disillusioned with the state of the broader social psychology field (including the consumer behavior discipline from which I hail), owing to questionable, and sometimes downright fraudulent, research practices, which led to numerous retractions of published articles. Additionally, the relevance of much of the published literature was poor — <a href="http://www.sciencedirect.com/science/article/pii/S1057740813000624">both substantively, and even within academia itself</a>. Together, a lack of rigor and relevance translated into money for nothing: a proportion of the funding going towards academic research was, essentially, being wasted.</p><p>So, where are we today? Whilst <a href="https://www.theguardian.com/science/head-quarters/2017/mar/02/fresh-concerns-raised-over-academic-conduct-of-major-us-nutrition-and-behaviour-lab">questionable research practices continue to be reported upon in the media</a>, the difference now is that these instances appear to have been brought to light <em>as a result</em> of the changes instituted 3–5 years ago (e.g., <a href="http://fortune.com/2016/10/02/power-poses-research-false/">replication projects</a>). I am heartened by some of the initiatives I have seen in recent times (having attempted to keep a toe in the academic sector; a foot just isn’t practical when working a demanding industry role) — such as the <a href="https://academic.oup.com/jcr/pages/tutorials">Tutorials in Consumer Research</a> series, released as part of the Journal of Consumer Research; as well as the <a href="https://cos.io/prereg/">Preregistration Challenge</a>, which is conducted by the Center for Open Science.</p><p><strong>Relevance: The “so what?” factor in academia</strong></p><p>I believe the broader social psychology field is heading in the right direction, with a strong emphasis now on the rigor of the research that is being produced, and I am excited at its prospects for the future. However, the positive effects of implemented change will take time to fully emerge. For example, in 2012, the Association for Consumer Research (ACR) board approved the establishment of an ACR journal that would focus more on substantively relevant issues — but it took until <a href="http://www.journals.uchicago.edu/toc/jacr/2016/1/1">January 2016</a> for the first issue to be published.</p><p>Despite this positive progress with respect to rigor, there remains a question of relevance — and whether initiatives such as the ACR journal will adequately address this. In his <a href="http://www.acrwebsite.org/volumes/1900001/volumes/v40/NA-40">2012 ACR Presidential Address</a>, Professor Jeffrey Inman spoke about “the increasing need for useful insights”, and noted that a “quick scan of the implications section of a lot of consumer behavior articles reveals that the implications offered there are often rather far removed from the findings.”</p><p>Professor Inman went on to describe how the relative attendance of academics, government officials, and industry practitioners at ACR conferences had “grown quite lopsided”, with academics comprising over 99% of those in attendance at the 2012 conference in Vancouver. I doubt this attendance has balanced itself out since then. Case in point: I had the pleasure of attending a boutique Society for Consumer Psychology (SCP) conference at Columbia University last month, as the sole attendee not currently employed in academia — and I suspect it will be a similar story again in October, when I travel to San Diego to attend my first ACR conference since 2014.</p><p>Yet, attending the SCP Boutique Conference was tremendously insightful — and not just in terms of the high quality and incredibly engaging research that was presented on the day, regarding emotions and motivations, and the roles they play in consumer behavior. For me, it was fascinating to step back into the academic world with a completely new perspective, which has been shaped by solving client challenges for the past few years. What struck me, specifically, was how much my approach to addressing research questions has evolved, and how differently I interpreted the findings presented, relative to others in attendance (i.e., putting aside my own intellectual curiosity, would any of my clients want to know about these effects?).</p><p>I’ve heard it said that social psychologists care mostly about the existence of an effect, whereas economists are concerned more with the magnitude of an effect. I can understand this distinction; the rationale being akin to: to build knowledge, it is necessary to lay a foundation of understanding what effects exist, and identify those factors that drive or mediate the effects, those which moderate them, and those which attenuate and/or reverse them.</p><p>The economics field is older than the broader social psychology field, and already has a strong foundation from which to further develop; hence, by this rationale, it is free to address the question of effect size magnitudes. This misguided approach to building a discipline, however, prevents the progression of academia, causing the enduring lack of relevance that we see in the broader social psychology field now, <a href="http://www.sciencedirect.com/science/article/pii/S1057740812000952">both within academia, as well as from a practitioner perspective</a>.</p><p><strong>The other side of the equation: Rigor and relevance in industry</strong></p><p>And therein lies the problem: out of the multitude of projects I have worked on since moving to industry (and bearing in mind that I, personally, am not involved in qualitative research), I cannot recall a single one in which a client did not ask for an effect or a relationship to be quantified. Indeed, understanding the magnitude of effects is an essential step in providing clients with a hierarchy of strategic priorities, and narrowing their focus to proactively solve the unique business challenge for which they have requested assistance. Put simply, academia is not currently addressing the right questions to effectively collaborate with industry practitioners.</p><p>If you’re an industry practitioner reading this (particularly in market research), you might be thinking that the grass isn’t any greener on this side of the fence. Based upon <a href="https://www.esomar.org/knowledge-center/reports-publications">ESOMAR data</a>, global spend on market research grew almost 7% (accounting for inflation) between 2009 and 2015, with the spend for 2015 totaling $44.35 million USD — but those in the industry know all too well that a sizable proportion of that spend goes towards research that is uninformative, or even unused. Typically, there are three reasons for this outcome.</p><p>First, there is no buy-in and, as such, research is seen as merely a box to be ticked. Whether it’s managers relying upon their own experience and intuition, and who trust their gut above all else; or a case of poor stakeholder management within a client’s business, such that no-one with decision-making responsibilities is an advocate for what the research is addressing — the result is the same: the research may get commissioned and completed, but the insights and recommendations go unused. Unfortunately, this is a client-side challenge; however, research firms can help to mitigate the issue, by clearly communicating both the objectives and benefits of the research (i.e., its relevance), to rally stakeholders within the client’s organization to become advocates for change.</p><p>Second, there is an issue in applying the findings from the research. When research is sold as a product, rather than a means to address a business challenge; or the research firm lacks strong advisory capabilities, the recommendations will be poorly applied, or will go unheeded altogether. Currently, the industry appears to be self-correcting here, <a href="http://fortune.com/2016/12/21/tech-consulting-digital/">as evidenced by larger, strategic advisory firms acquiring smaller research agencies en masse</a> — both <a href="https://newsroom.accenture.com/news/accenture-to-acquire-labanswer-expanding-its-life-sciences-and-cross-industry-research-and-laboratory-informatics-capabilities.htm">locally in the US</a>, as well as <a href="https://www.cmo.com.au/article/620563/kpmg-joins-consulting-jostle-own-marketing-services-new-acquisition/">around the world</a> — including the knowledge and talent that goes into the bespoke solutions that boutique suppliers can design and offer.</p><p><strong>What can industry offer academia?</strong></p><p>Finally, the research simply does not solve the business challenge. There may be numerous antecedents to this outcome, but the most pertinent is that of poor design (i.e., a lack of rigor) — and this is the area where academic input is most applicable and valuable, because basic principles dictate that before we can accurately measure something, we must first understand it. Coincidentally, this understanding also helps to solve for the other two reasons, above, by facilitating clearer communication of the relevance of the research, including its aims, objectives, benefits, insights, and recommendations.</p><p>The volume of data that organizations generate and have access to is astounding, and has created an obsession with “big data” in industry. But data without design is worthless; you can have the most sophisticated analytics capabilities in the world, but these will ultimately be useless if you cannot translate the outputs into an outcome that 1) stakeholders care about and will buy into; 2) can be implemented to drive change; and 3) will solve a specific and important business challenge.</p><p>The most effective initiatives often come from the marriage of behavioral science and data science; those firms that can operate at the nexus of these two disciplines will, in my view, be the most successful. We have seen this already, particularly with the giants of the tech industry, such as Facebook and Google — as well as with the acquisition of research agencies by management consulting firms seeking to bolster these capabilities, as above.</p><p><strong>What can academia offer industry?</strong></p><p>And with the increasing ease of access to data, as well as the greater amounts of data being generated, comes an opportunity for academic expertise to be applied. For example, industry practitioners can collaborate with academics on individual projects, to enhance rigor (e.g., with input from academics regarding the research design and/or analytics). Conversely, these academics could use some of that industry data to generate and/or explore hypotheses as part of their research endeavors (i.e., enhancing the relevance of their own work).</p><p>The unfortunate reality of the consulting industry is that the popularization of fields like behavioral economics (including the plethora of books published on the subject) has resulted in simplified principles, derived from published “effects papers”, that laypeople mistakenly believe apply universally. In turn, it can be difficult for organizations in the market for research solutions to discern genuine expertise from a good sales pitch.</p><p>To communicate effects such as “choice overload” in a popular-science setting, it is necessary to apply relatable anecdotes (e.g., “how difficult is it to choose what to eat at a restaurant when you are faced with ten pages of options?”) — <a href="https://www.psychologytoday.com/blog/pop-psych/201602/is-choice-overload-real-thing">but these generalized examples neglect the nuances of these effects</a>. In fact, these nuances are just as important as the core effect; for instance, if a business owner concluded that choice overload meant they should reduce the number of options they make available to consumers, they may inadvertently make their offerings <em>less</em> attractive, not more.</p><p>How can this be the case? Contrary to what a “generalized principle” of choice overload suggests, <a href="http://www.sciencedirect.com/science/article/pii/S0022435913000419">consumers prefer larger assortments (e.g., because of greater freedom of choice), as these generate positive affect; however, when it comes to actually making a choice, consumers’ rational cognitions take over, and choosing from a large assortment set <em>can</em> become more difficult (e.g., depending upon whether the category is hedonic or utilitarian)</a>.</p><p><strong>Understanding rigor: The important role of context</strong></p><p>Put simply, it is an understanding of these nuances, <em>and their importance</em>, that can separate a rigorous research firm from one that is not — and this understanding can come from collaboration with those whose work is the generation of knowledge: academics. In other words, academia can assist industry by providing consulting services to practitioners, and training the talent that goes to work in the sector (i.e., educating practitioners about these nuances), hence bolstering the rigor of industry-oriented research.</p><p>Academics know all too well that the best research is conducted neither quickly nor cheaply. But in industry, time and money are not luxuries that most organizations have; in today’s dynamic business environment, organizations must be agile, and research budgets are often slim. These circumstances have paved the way for some firms to produce poorly designed and executed research, which can be turned around fast and for low cost, but which is lacking in analytical rigor, thereby leading to the problems outlined above.</p><p>Many of the scandals seen in academia have emerged from the misuse of statistics; however, as with behavioral effects, understanding the nuances of analytical techniques and statistics can be very powerful — if used correctly. For instance, whilst the conventional alpha level for statistical significance (i.e., <em>p</em> &lt; .05) is typically not relaxed in academia, it sometimes makes sense to relax this threshold in industry.</p><p>Take the case of a brand tracker: Is it riskier for an organization a) to institute an operational change based upon a statistically significant movement in tracked metrics that has a higher probability of being a false-positive (i.e., by relaxing the alpha level to define statistical significance as <em>p </em>&lt; .10)? Or b) to keep the conventional alpha level and infer no meaningful movement in metrics (e.g., a statistically non-significant dip), and hence make no operational changes?</p><p><strong>Identifying a middle ground: Collaboration between academia and industry to bolster rigor and relevance for research in both sectors</strong></p><p>Obviously, the answer is dependent upon the cost to the organization; specifically, that of making a <em>potentially unnecessary</em> change, versus that of <em>not</em> making an <em>early</em> change. In other words, the context is crucial. The key point, however, is that understanding which rules can be bent and, more importantly, <em>when</em> and <em>how</em> to bend them, can save clients a lot of time and money, without compromising the quality of the research. Achieving this understanding requires insights from academia, to determine the implications of, and considerations for, relaxing statistical constraints.</p><p>In both academia and industry, research that lacks rigor and/or relevance equates to a waste of both time and money — the extent of which is dependent upon how invested stakeholders are in its outcomes. The purpose of research in academia is to contribute to knowledge, answering questions <em>that matter</em> to the discipline; in industry, it is to solve a business challenge. At the end of the day, though, the research process is the same in both sectors: objectives are identified, studies are designed and executed, and insights are derived. Put simply, research is an ongoing process of problem identification and resolution.</p><p>The best ideas and outcomes when problem solving will often be generated as a result of collaboration — <a href="http://www.newyorker.com/books/page-turner/the-two-friends-who-changed-how-we-think-about-how-we-think">and you need look no further than everyone’s favorite behavioral science duo for a case in point here</a>. Consequently, it is collaboration which, in my view, is the optimal way forward — because it will yield positive outcomes for both industry and academia, including helping to ensure that neither sector continues to fund research that is not useful. And, at the end of the day, no-one wants to spend money for nothing, particularly when it comes to research.</p><p>— — —</p><p><em>Dr Ben Kozary is a behavioral scientist and research design specialist living and working in New York. He holds a Ph.D. in Consumer Behavior from the University of Newcastle, Australia.</em></p><p><em>Enjoy what you read? Share, like, and comment, to continue the conversation — and feel free to reach out: b.kozary@gmail.com</em></p><p>— — —</p><p><em>Originally published at </em><a href="https://www.linkedin.com/pulse/money-nothing-how-collaboration-between-academia-industry-ben-kozary/"><em>https://www.linkedin.com</em></a><em> on July 5, 2017.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=754aec2e12b0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[:InDecision: Viewpoint — Nudging Nudgers’ Nudges]]></title>
            <link>https://medium.com/@benkozary/indecision-viewpoint-nudging-nudgers-nudges-222a0aba443f?source=rss-6fa08adf767f------2</link>
            <guid isPermaLink="false">https://medium.com/p/222a0aba443f</guid>
            <category><![CDATA[behavioral-economics]]></category>
            <category><![CDATA[science]]></category>
            <category><![CDATA[throwback]]></category>
            <category><![CDATA[conference]]></category>
            <category><![CDATA[psychology]]></category>
            <dc:creator><![CDATA[Ben Kozary, Ph.D.]]></dc:creator>
            <pubDate>Fri, 27 Jun 2014 08:39:05 GMT</pubDate>
            <atom:updated>2018-02-19T00:40:05.334Z</atom:updated>
            <content:encoded><![CDATA[<p><strong><em>In June 2014, the </em></strong><a href="https://indecisionblog.com/2014/06/27/viewpoint-nudging-nudgers-nudges-2/"><strong><em>InDecision</em></strong></a><strong><em> team asked Ben to report on the first International Behavioral Insights Conference in Sydney, from the perspective of someone moving from academia into industry. Here, he reflects on his experiences at the conference, and some insights he obtained.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*rGGddEAvzwnsaduF." /></figure><p>A few days ago, I read <a href="http://www.huffingtonpost.com/adam-grant/why-behavioral-economics_b_5491960.html">an interesting piece on the differences between behavioral economics and psychology</a>. Truth be told, even after attending the inaugural Behavioural Exchange conference in Sydney on June 2 and 3, 2014, I’m still not sure what exactly the difference is. But does the distinction even matter? The academic in me screams, “YES!” but, outside of the ivory tower, it seems that most people aren’t concerned. Behavioural Exchange (hereafter referred to as <a href="http://bx2014.org/"><strong>bx2014</strong></a>) was, after all, a public policy conference — and as such, the emphasis of the conference was on actionable and applicable insights from the behavioral sciences.</p><p>It was the promise of these insights that drew me to bx2014. As a final year Ph.D. candidate researching Consumer Psychology and Behavior, I was looking to ground the heavily theoretical work I do in some concrete practical applications. My hope was that, in doing so, my work would take on greater meaning, even if only to me — not to mention that I’d be able to pad out the practical implications component of my dissertation. Additionally, <a href="https://indecisionblog.com/2014/04/07/viewpoint-why-im-leaving-academia/">as someone transitioning from academia into industry</a>, I was curious as to what the transition was going to entail. Who are the sorts of people I’m going to be dealing with in industry, and how are they similar/different to academics? What value is placed on the skills and knowledge I have, and how might I be best able to apply myself? And how well received are the ideas that I’ve been exposed to throughout my time in academia? In this post, I hint at the answers to these questions but, as you may have guessed, they’re by no means definitive.</p><p><strong>bx2014: an overview</strong></p><p>Speakers at the event included academics, the majority of whom were from Harvard University; as well as members of government, primarily from Australia, but also from the US, UK, and Singapore; and business-people from around the world, including CEOs, consultants, and designers. For me, the first day seemed to focus more on government, whilst the second day was primarily about business — but regardless of the relevance (or not) to me, I found all of the sessions fascinating. Individual presenters and panels alike dealt with such issues as:</p><ul><li>The importance and benefits of nudging</li><li>How governments can embrace, and have already undertaken, nudging</li><li>Opportunities, risks, and common challenges of nudging</li><li>The fundamentals of nudging, including data, design, and delivery</li><li>How the integration of findings from academia and experiences in business and government can make nudges more effective</li><li>Reflections and insights from business and academia on the application of nudges in the corporate world</li><li>The future of nudging and behavioral science</li></ul><p><strong>Nudging: is it just a fad?</strong></p><p>At this point, you may have noticed that the term “nudging” seems to have been applied as a catchcry for any behavioral intervention. <a href="https://indecisionblog.com/2013/04/13/7sins-6-overgeneralisation/">Overgeneralization may be a common sin of consumer psychology researchers</a>, but our thinking appears more nuanced than that of our industry counterparts. For instance, many of the initiatives suggested or discussed at bx2014 were based upon research describing numerous cognitive biases, including the sunk-cost fallacy, present bias, hindsight bias, confirmation bias, anchoring, and framing effects — but there was almost no mention of the intricacies of these biases, nor any real emphasis placed on the conditions specific to particular case studies upon which they had been used as part of a successful behavioral intervention. Given that I was asked on more than ten separate occasions during the two days whether I’d read Kahneman’s <em>Thinking, Fast and Slow</em> (I haven’t, which apparently put me firmly in the minority of conference attendees), my concern here is that many of the attendees were searching for quick, easily applied solutions to issues affecting their stakeholders.</p><p>This overgeneralization is dangerous ground to walk, because it flirts with the prospect of nudging becoming yet another apparent management or political panacea, when it’s anything but. Instead, we need to heed what Professor Cass Sunstein said in the first presentation of the conference; that effective nudging is about recognizing individual differences. “Nudges are like GPS units: they tell you the most efficient, or ‘best’, route, but you don’t have to take it; you can go your own way and choose the scenic route, if you like.” In other words, nudges should preserve individual choice by not being overly paternalistic; this is what separates them from mandates. In that sense, I feel that Professor Sunstein was nudging us (if you will) to not overgeneralize.</p><p><strong>Experimentation: “test, learn, adapt”</strong></p><p>If you’re thinking executing Professor Sunstein’s advice is easier said than done, you’re right — but that was also addressed at bx2014. One of the recurring points of the conference was the need for experimentation, despite how challenging it may be. Dr David Halpern, Chief Executive of the UK Behavioural Insights Team, told us to live by one simple principle: “Test, learn, adapt.” Another presenter suggested, “It’s better to say, ‘I don’t know,’ and then test something, than to skip trials and push ahead to a full roll-out on a hunch.”</p><p>We were also reminded that the most successful companies, especially in the technology sphere — Google, Facebook, Amazon, etc. — perpetually experiment. Randomiszd controlled trials (RCTs) are the gold standard of experimentation, but they’re not always viable. When that’s the case, we were advised to “do whatever experiments or tests you can, provided the costs don’t outweigh the potential benefits — but always strive to get the best data available.” And therein lay two of the foremost challenges of behavioral interventions in industry: funding, and time. Fortunately for me, my time in academia has me well versed in both of these issues…</p><p><strong>Replication: it’s essential in industry, too</strong></p><p>The issue of replication is one that we should all be familiar with by now — and it didn’t go unmentioned at bx2014, either. For instance, Professor Richard Thaler, from the University of Chicago, told us that managers and policy makers generally think they’re right, and they don’t like taking risks; however, they are often too impatient to run experiments, and don’t see the point of replication, with their philosophy being, “It worked already, so why do we need to spend more time and money to test it again?” This problem is an obvious one, but it can be overcome with education and training.</p><p>A more serious problem with replication was highlighted during the Design breakout session I attended on the afternoon of the first day, where one of the presenters said, “Relative to hard sciences, social science is difficult, because the results will not always replicate. You can implement a nudge or a system of some sort as an effective intervention, but in 6–12 months’ time (or maybe more), people might have adapted and changed their behaviors such that it no longer works — and therefore won’t replicate in any RCTs or experiments you run.” From an academic standpoint, I find this idea intriguing, because it’s something that we rarely consider; we tend to take the more general view that, if an effect is real, it will replicate. I’m yet to hear people’s adaptability offered as a reason for some of the recent failures for studies to replicate — and I’m not saying that it’s necessarily a legitimate reason, but it does highlight something that we as researchers risk forgetting: that there are real people behind our statistics, and they can be unpredictable and subject to change.</p><p><strong>Big data: how do we use it?</strong></p><p>On the second day, I attended the Data breakout session, during which several interesting points were made. The focus of the session was on big data, which Dr James Guszcza, of the Deloitte Analytics Institute in Singapore, told us referred to predictive analytics and modeling. These models, he said, can point us in the right direction, and tell us who to target our interventions to, but they don’t tell us how to prompt the desired behavior change. For that, behavioral insights are required; thus, he recommended that behavioral insights and predictive modeling be infused, because — to echo Professor Sunstein — solutions will often need to be nuanced and individually focused. Dr Guszcza also advised that we be flexible with our data, and be open to the possibility that it can be useful in ways that you wouldn’t previously have imagined. “Old” datasets are particularly useful in this regard, he said, so you should also be mindful of “digital exhaustion” (that is, the deletion of older data). “With today’s storage capabilities, you shouldn’t need to delete anything simply because it’s old.”</p><p><strong>Collaboration in nudging: how academia and industry can work together</strong></p><p>Strangely, the importance of collaboration between academia and industry wasn’t strongly highlighted at the conference; however, one presenter did note its significance. For collaboration to be effective, he said, it’s a matter of recognizing each other’s needs. That means that academics should look at questions important to organizations, and organizations should allow academics to publish — especially given the type of rich data they have access to. Furthermore, collaboration could help solve what was highlighted as a critical issue affecting research into behavioral insights. That is, Professor Max Bazerman, of Harvard University, described how the judgement and decision-making field originated decades ago under the notion that, “If we understand what’s wrong with the human mind, we can fix it — but this approach is flawed. Instead, we should focus on understanding and accepting that this is the way the world works, and therefore we can learn to adapt and be effective.”</p><p><strong>Final thoughts</strong></p><p>At the conclusion of the conference, we were asked to fill out a short questionnaire. One of the questions asked us to describe bx2014 in one sentence; I wrote: “Nudging nudgers’ nudges”, because I felt that neatly summed up the notion that most people were there to learn how to implement more effective behavioral interventions (plus, my brain was fried after an intense two days, as well as having being punished by my downing more than a few drinks at the reception the night before…). But, the truth is, this conference can’t be compressed into a sentence; the ideas are just too big. So, with that in mind, I’d like now to share with you a few thought provoking ideas that I jotted down over the two days:</p><blockquote>If I were young and wanted to start a business, I would start a choice engine, because they will do for other industries what travel websites did for that industry. The amount of data emerging and becoming available is monumental.<em> -Professor Richard Thaler, University of Chicago</em></blockquote><blockquote>Nudges lead people to engage in behaviour — but, as a psychologist, I’m interested in the outcome beyond that behaviour; for example, are people happy or unhappy? <em>-Professor Mike Norton, Harvard University</em></blockquote><blockquote>When thinking about nudges, consider this piece from the late author, David Foster Wallace:<em> There are these two young fish swimming along, and they happen to meet an older fish swimming the other way, who nods at them and says, “Morning, boys, how’s the water?” And the two young fish swim on for a bit, and then eventually one of them looks over at the other and goes, “What the hell is water?”</em> Remember: there will always be water — and there will always be nudges, even if we don’t realise they’re there. We must open our eyes to the extensive possibilities of nudging.<em> -Professor Cass Sunstein, Harvard University</em></blockquote><p>And, finally:</p><blockquote>Done is better than perfect.<em> -Mia Garlick, Head of Policy Australia and New Zealand, Facebook</em></blockquote><p>So, in the interests of applying something I learned at bx2014, I’m calling this post done. It’s not perfect — but as several speakers remarked at the conference, satisficing is better than optimizing. And, at the end of the day, I think that’s a pretty good example of behavioral economics in action.</p><p><em>Originally published at </em><a href="https://indecisionblog.com/2014/06/27/viewpoint-nudging-nudgers-nudges-2/"><em>indecisionblog.com</em></a><em> on June 27, 2014.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=222a0aba443f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[:InDecision: Viewpoint — Why I’m Leaving Academia]]></title>
            <link>https://medium.com/@benkozary/indecision-viewpoint-why-im-leaving-academia-223768114ba2?source=rss-6fa08adf767f------2</link>
            <guid isPermaLink="false">https://medium.com/p/223768114ba2</guid>
            <category><![CDATA[academia]]></category>
            <category><![CDATA[psychology]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[throwback]]></category>
            <category><![CDATA[science]]></category>
            <dc:creator><![CDATA[Ben Kozary, Ph.D.]]></dc:creator>
            <pubDate>Mon, 07 Apr 2014 09:00:31 GMT</pubDate>
            <atom:updated>2018-02-19T00:27:31.028Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/300/0*tQP9fono9XxXwj3f." /></figure><p><strong><em>In April 2014, </em></strong><a href="https://indecisionblog.com/2014/04/07/viewpoint-why-im-leaving-academia/"><strong><em>InDecision blog</em></strong></a><strong><em> featured a guest post from Ben Kozary, a Ph.D. candidate at the University of Newcastle in Australia. After getting to know Ben at various conferences over the past year, the InDecision team was disappointed to hear about his decision to leave academia — partly because he’s an excellent and passionate researcher, partly because we wouldn’t benefit from his jovial company at future conferences! However, his reasons for leaving echoed many dinner conversations we’ve had with fellow Ph.D. students so we asked him to write about his experience and his decision to move to industry. Over to Ben…</em></strong></p><p>To say I’ve learnt a lot during my Ph.D. candidature would be an understatement. From a single blank page, I now know more than most people in the world about my particular topic area. I understand the research process: from planning and designing a study; to conducting it; and then writing it up clearly — so that readers may be certain about what I did, how I did it, what I found, and why it’s important. I’ve met a variety of people from around the world, with similar interests and passions to me, and forged close friendships with many of them. And I’ve learnt that academia might well be the best career path in the world. After all, you get to choose your own research area; you have flexible working hours; you get to play around with ideas, concepts and data, and make new and often exciting discoveries; and you get to attend conferences (meaning you get to travel extensively, and usually at your employer’s expense), where you can socialize (often at open bars) under the guise of “networking”. Why, then, you might be wondering, would I want to leave all of that behind?</p><p>My journey through the Ph.D. program has been fairly typical; I’ve gone through all of the usual stages. I’ve been stressed in the lead-up to (and during) my proposal defense. I’ve had imposter syndrome. And I’ve been worried about being scooped, and/or finding “that paper”, which presents the exact research I’m doing, but does it better than me. But now, as I begin my final year of the four year Australian program, I’m feeling comfortable with, and confident in, the work I’ve produced so far in my dissertation. And yet, I’m also disillusioned — because, for all of its positives, I’ve come to see academia as a broken institution.</p><p><a href="http://www.nature.com/news/scientific-method-statistical-errors-1.14700">That there are problems facing academic research is not news</a>, especially in psychology. <a href="http://www.timeshighereducation.co.uk/features/social-psychology-is-primed-but-not-suspect/2002055.fullarticle">Stapel and Smeesters</a>, <a href="http://pss.sagepub.com/content/22/11/1359.abstract">researcher degrees of freedom</a> and <a href="http://link.springer.com/article/10.1007%2Fs11192-011-0494-7">bias</a>, <a href="http://www.ncbi.nlm.nih.gov/pubmed/15137886">(the lack of) statistical power</a> and <a href="http://psycnet.apa.org/journals/amp/49/12/997/">precision</a>, the “<a href="http://psychsciencenotes.blogspot.com.au/2013/11/replication-will-not-save-psychology.html">replication crisis</a>” and “<a href="http://psychsciencenotes.blogspot.com.au/2011/11/theory-and-why-its-time-psychology-got.html">theoretical</a> <a href="http://osc.centerforopenscience.org/2013/11/20/theoretical-amnesia/">amnesia</a>”, <a href="http://psychsciencenotes.blogspot.com.au/2013/09/social-priming-of-course-it-only-kind.html">social</a> and <a href="http://pps.sagepub.com/content/9/1.toc">behavioral priming</a>: the list goes on. However, these problems are not altogether removed from one another; in fact, they highlight what I believe is a larger, underlying issue.</p><p><strong>Academic research is no longer about a search for the truth</strong></p><p>Stapel and Smeesters are two high profile examples of fraud, which represents an extreme exploitation of researcher degrees of freedom. But what makes any researcher “massage” their data? <a href="http://www.newscientist.com/article/dn20447-journal-rejects-studies-contradicting-precognition.html#.Uu23t7TRxNE">The bias towards publishing only positive results</a> is no doubt a driving force. Does that excuse cases of fraud? Absolutely not. My point, however, is that there are clear pressures on the academic community to “<a href="http://www.theguardian.com/science/2011/sep/05/publish-perish-peer-review-science">publish or perish</a>”. Consequently, academic research is largely an exercise in <a href="http://www.economist.com/news/business/21595929-business-schools-are-better-analysing-disruptive-innovation-dealing-it-those-who">career development and promotion</a>, and no longer (<a href="http://www.nytimes.com/2014/02/02/opinion/sunday/scientific-pride-and-prejudice.html?_r=2">if, indeed, it ever was</a>) an objective search for the truth.</p><p>For instance, the lack of statistical power evident in our field has been known for more than fifty years, with <a href="http://www.ncbi.nlm.nih.gov/pubmed/13880271">Cohen (1962)</a> first highlighting the problem, and <a href="http://psycnet.apa.org/journals/ccp/58/5/646/">Rossi (1990)</a> and <a href="http://www.ncbi.nlm.nih.gov/pubmed/15137886">Maxwell (2004)</a> providing further prompts. Additionally, Cohen (<a href="http://psycnet.apa.org/index.cfm?fa=search.displayRecord&amp;uid=1991-11596-001">1990</a>; <a href="http://psycnet.apa.org/journals/amp/49/12/997/">1994</a>) reminded us of the many issues associated with null-hypothesis significance testing — issues that were raised as far back as 1938 — and yet, it still remains the predominant form of data analysis for experimental researchers in the psychology field. To address these issues, Cohen (1994: 1002) suggested a move to estimation:</p><blockquote>“Everyone knows” that confidence intervals contain all the information to be found in significance tests and much more. […] Yet they are rarely to be found in the literature. I suspect that the main reason they are not reported is that they are so embarrassingly large! But their sheer size should move us toward improving our measurement by seeking to reduce the unreliable and invalid part of the variance in our measures (as Student himself recommended almost a century ago). Also, their width provides us with the analogue of power analysis in significance testing — larger sample sizes reduce the size of confidence intervals as they increase the statistical power of NHST.</blockquote><p>Twenty years later, and we’re finally starting to see some <a href="http://pss.sagepub.com/content/25/1/3.full.pdf+html">changes</a>. Unfortunately, the field now has to suffer the consequences of being slow to change. Even if all our studies were powered at the conventional level of 80% (Cohen, 1988; <a href="http://www.ncbi.nlm.nih.gov/pubmed/19565683">1992</a>), they would still be imprecise; that is, the width of their 95% confidence intervals would be approximately ±70% of the point estimate or effect size (<a href="http://www.ncbi.nlm.nih.gov/pubmed/8017747">Goodman and Berlin, 1994</a>). In practical terms, that means that if we used Cohen’s d as an effect size metric (for the standardized difference between two means), and we found that it was “medium” (that is, d = 0.50), the 95% confidence interval would range from 0.15 to 0.85. This is exactly what Cohen (1994) was talking about when he said the confidence intervals in our field are “so embarrassingly large”: in this case, the interval tells us that we can be 95% confident the true effect size is potentially smaller than “small” (0.20), larger than “large” (0.80), or somewhere in between. Remember, however, that many of the studies in our field are underpowered, which makes the findings even more imprecise than what is illustrated here; that is, the 95% confidence intervals are even wider. And so, I wonder: How many papers have been published in our field in the last twenty years, while we’ve been slow to change? And how many of these papers have reported results at least as meaningless as this example?</p><p>I suspect that part of the reason for the slow adoption of estimation techniques is due to the <a href="https://indecisionblog.com/2013/10/28/guest-post-michael-blastland-on-uncertainty/">uncertainty</a> they bring to the data. Significance testing is characterized by dichotomous thinking: an effect is either statistically significant or it is not. In other words, significance testing is seen as easier to conduct and analyze, relative to estimation; however, <a href="http://www.psychologicalscience.org/index.php/publications/observer/2014/march-2014/theres-life-beyond-05.html">it does not allow for the same degree of clarity in our findings</a>. By reporting confidence intervals (and highlighting uncertainty), we reduce the risk of committing one of the cardinal <a href="http://www.sciencedirect.com/science/article/pii/S1057740813000624">sins of consumer psychology</a>: <a href="https://indecisionblog.com/2013/04/13/7sins-6-overgeneralisation/">overgeneralization</a>. Furthermore, you may be surprised to learn that <a href="http://pss.sagepub.com/content/25/1/7.full.pdf+html">estimation is just as easy to conduct as significance testing, and even easier to report</a> (because you can extrapolate greater meaning from your results).</p><p><strong>Replication versus theoretical development</strong></p><p>When you consider the lack of precision in our field, in conjunction with <a href="http://link.springer.com/article/10.3758/s13423-014-0601-x">the magnitude of the problems of researcher degrees of freedom and publication bias</a>, is it any wonder that so many replication attempts are unsuccessful? The issue of failed replications is then compounded further by <a href="http://psychsciencenotes.blogspot.com.au/2013/11/replication-will-not-save-psychology.html">the lack of theoretical development that takes place in our discipline</a>, which creates <a href="http://osc.centerforopenscience.org/2013/11/20/theoretical-amnesia/">additional problems</a>. The incentive structure upon which the academic institution is situated implies that success (in the form of promotion and grants) comes to those who publish a high number of high quality papers (as determined by the journal in which they are published). As a result, we have a discipline that <a href="http://www.sciencedirect.com/science/article/pii/S1057740813000624">lacks both internal and external relevance, due to the multitude of standalone empirical findings that fail to address the full scope of consumer behavior</a> (Pham, 2013). In that sense, it seems to me that replication is at odds with theoretical development, when, in fact, the two should be working in tandem; that is, replication should guide theoretical development.</p><p>Over time, some of you may have observed (as I have) that single papers are now expected to “do more”. Papers will regularly report four or more experiments, in which they will identify an effect; perform a direct and/or conceptual replication; identify moderators and/or mediators and/or boundary conditions; and rule out alternative process accounts. I have heard criticism directed at this approach, usually from fellow Ph.D. candidates, that there is an unfair expectation on the new generation of researchers to do more work to achieve what the previous generation did. In other words, that the seminal/classic papers in the field, upon which now-senior academics were awarded tenure, do less than what emerging and early career researchers are currently expected to do in their papers. I do not share this view that there is an issue of hypocrisy; rather, my criticism is that as the expectation that papers “do more” has grown, there is now less incentive for academics to engage in theoretical development. The “flashy” research is what gets noticed and, in turn, what gets its author(s) promoted and wins them grants. Why, then, would anyone waste their time trying to further develop an area of work that someone else has already covered so thoroughly — especially when, if you fail to replicate their basic effect, you will find it extremely difficult to publish in a flagship journal (where the “flashiest” research appears)?</p><p>This observation also begs the question: where has this expectation that papers “do more” come from? As other scientific fields (particularly the hard sciences) have reported more breakthroughs over time, I suspect that psychology has desired to keep up. The mind, however, in its intangibility, is too complex to allow for regular breakthroughs; there are simply too many variables that can come into effect, especially when behavior is also brought into the equation. Such an issue is highlighted no more clearly than in the case of behavioral priming. Yet, with the development of <a href="http://pps.sagepub.com/content/9/1/49.full.pdf+html">a general theory of priming</a>, researchers can target their efforts at identifying the varied and complex “unknown moderators” of the phenomenon and, in turn, design experiments that are more likely to replicate (<a href="http://pps.sagepub.com/content/9/1/40.full.pdf+html">Cesario, 2014</a>). Consequently, the expectation for single papers to thoroughly explain an entire process is removed — and our replications can then do what they’re supposed to: <a href="https://indecisionblog.com/2014/03/31/viewpoint-a-great-research-paper-explains-a-lot-not-everything/">enhance precision and uncover truth</a>.</p><p><strong>The system is broken</strong></p><p>The psychology field seems resistant to regressing to simpler papers that take the time to develop theory, and contribute to knowledge in <a href="http://pss.sagepub.com/content/25/1/7.full.pdf+html">a cumulative fashion</a>. Reviewers continue to request additional experiments, rather than to demand greater clarity from reported studies (for example, in the form of effect sizes and confidence intervals), and/or to encourage further theoretical development. Put simply, there is an implicit assumption that papers need to be “determining” when, in fact, they should be “contributing”. As <a href="http://pss.sagepub.com/content/25/1/7.full.pdf+html">Cumming (2014</a>: 23) argues, it is important that a study “be considered alongside any comparable past studies and with the assumption that future studies will build on its contribution.”</p><p>In that regard, it would seem that the editorial/publication process is arguably the larger, underlying issue contributing (predominantly, though not necessarily solely) to the many problems afflicting academic research in psychology. But what is driving this issue? Could it be that the peer review process, which seems fantastic in theory, doesn’t work in practice? I believe that is certainly a possibility.</p><p>Something else I’ve come to learn throughout my Ph.D. journey is that successful academic research requires mastery of several skills: you need to be able to plan your time; communicate your ideas clearly; think critically; explore issues from a “big picture” or macro perspective, as well as at the micro level; undertake conceptual development; design and execute studies; and be proficient at statistical analysis (assuming, of course, that you’re not an interpretive researcher). <a href="http://dynamicecology.wordpress.com/2014/01/23/william-shockley-on-what-makes-a-person-write-a-lot-of-papers-and-the-superstar-researcher-system/">Interestingly, William Shockley, way back in 1957, posited that producing a piece of research involves clearing eight specific hurdles — and that these hurdles are essentially all equal</a>. In other words, successful research calls for a researcher to be adept at each stage of the research process. However, in reality, it is often that the case that we are very adept (sometimes exceptional) at a few aspects, and merely satisfactory at others. The aim of the peer review process is to correct or otherwise improve the areas we are less adept at, which should — theoretically — result in a strong (sometimes exceptional) piece of research. Multiple reviewers evaluate a manuscript in an attempt to overcome these individual shortfalls; yet, look at the state of the discipline! The peer review process is clearly not working.</p><p>I’m not advocating abandoning the peer review process; I believe it is one of the cornerstones of scientific progress. What I am proposing, however, is for an adjustment to the system — <a href="http://news.sciencemag.org/2012/01/online-social-network-seeks-overhaul-peer-review-scientific-publishing">and I’m not the first to do so</a>. What if we, <a href="http://www.theguardian.com/science/blog/2013/jun/05/trust-in-science-study-pre-registration">as has been suggested</a>, move to a system of pre-registration? What if credit for publications in such a system were two-fold, with some going towards the conceptual development (resulting in the registered study), and some going towards the analysis and write-up? Such a system naturally lends itself to specialization, so, what if we expected less of our researchers? That is, what if we were free to focus on those aspects of research that we’re good at (whether that’s, for example, conceptual development or data analysis), leaving our shortfalls to other researchers? What if the peer review process became specialized, with experts in the literature reviewing the proposed studies, and experts in data analysis reviewing the completed studies? This system also lends itself to collaboration and, therefore, to further skill development, because the experts in a particular aspect of research are well-recognized. The Ph.D. process would remain more or less the same under this system, as it would allow emerging researchers to identify — honestly — their research strengths and weaknesses, before specializing after they complete grad school. There are, no doubt, issues with this proposal that I have not thought of, but to me, it suggests a stronger and more effective peer review process than the current one.</p><p><a href="http://www.newyorker.com/online/blogs/newsdesk/2012/12/cleaning-up-science.html"><strong>A recipe for change</strong></a></p><p>Unfortunately, I don’t believe these issues that I’ve outlined are going to change — at least not in a hurry, if the slow adoption of estimation techniques is anything to go by. For that reason, when I finish my Ph.D. later this year, I will be leaving academia to pursue a career in market research, where obtaining truth from the data to deliver actionable insights to clients is of the utmost importance. Some may view this decision as synonymous with giving up, but it’s not a choice I’ve made lightly; I simply feel as though I have the opportunity to pursue a more meaningful career in research outside of academia — and I’m very much looking forward to the opportunities and challenges that lay ahead for me in industry.</p><p>For those who choose to remain in academia, it is your responsibility to promote positive change; that responsibility does not rest solely on the journals. <a href="http://www.theguardian.com/higher-education-network/blog/2014/jan/29/boycott-academic-journals-early-career-researchers">It has been suggested that researchers boycott the flagship journals if they don’t agree with their policies</a> — but that is really only an option for tenured professors, unless you’re willing to risk career self-sabotage (which, I’m betting, most emerging and early career researchers are not). The push for change, therefore, needs to come predominantly (though not solely) from senior academics, in two ways: 1) in research training, as advisors and supervisors of Ph.D. students and post-docs; and 2) as reviewers for journals, and members of editorial boards. Furthermore, universities should offer greater support to their academics, to enable them to <a href="http://www.tandfonline.com/doi/abs/10.1080/1047840X.2012.705245?journalCode=hpli20#.UvlzoYVqO20">take the time to produce higher quality research that strives to discover the truth</a>. Grant committees, also, may need to re-evaluate their criteria for awarding research grants, and focus more on quality and meaningful research, as opposed to research that is “flashy” and/or “more newsworthy”. And the next generation of academics (that is, the emerging and early career researchers) should familiarize themselves with these issues, so that they may make up their own minds about where they stand, how they feel, and how best to move forward; the future of the academic institution is, after all, in their hands.</p><p><em>Originally published at </em><a href="https://indecisionblog.com/2014/04/07/viewpoint-why-im-leaving-academia/"><em>indecisionblog.com</em></a><em> on April 7, 2014.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=223768114ba2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>