<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Peter Campbell on Medium]]></title>
        <description><![CDATA[Stories by Peter Campbell on Medium]]></description>
        <link>https://medium.com/@petecam?source=rss-a2a774fe7d7d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*pMVAVKsTXp50t-f0mtbmPw.jpeg</url>
            <title>Stories by Peter Campbell on Medium</title>
            <link>https://medium.com/@petecam?source=rss-a2a774fe7d7d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 23:00:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@petecam/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[My monitor uses more watts than my computers!]]></title>
            <link>https://medium.com/@petecam/my-monitor-uses-more-watts-than-my-computers-11166ef87c4f?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/11166ef87c4f</guid>
            <category><![CDATA[energy]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[carbon-emissions]]></category>
            <category><![CDATA[mac]]></category>
            <category><![CDATA[sustainability]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Tue, 19 Sep 2023 10:23:21 GMT</pubDate>
            <atom:updated>2023-09-19T10:31:48.482Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Today, the world is in the midst of the first truly global energy crisis, with impacts that will be felt for years to come. Russia’s unprovoked invasion of Ukraine in February has had far-reaching impacts on the global energy system, disrupting supply and demand patterns and fracturing long‐standing trading relationships. The crisis is affecting all countries, but at the International Energy Agency (IEA), we are particularly concerned about the effect it is having on the people who can least afford it</blockquote><blockquote>World Energy Outlook 2022, International Energy Agency</blockquote><p>In 2023 we no longer view electricity like we used to. Many are noticing its cost for the first time. Others are switching off to reduce bills. But how are we meant to understand which appliances use the most electricity?</p><p>Electricity generation is also the largest contributor to carbon emissions globally given most countries burn fossil fuels to generate supply. This is why it is a priority for many countries to transition to renewable energy sources. But how are we meant to understand which appliances cause the most carbon emissions?</p><h4>Doing some unscientific tests</h4><p>Every electrical device has a power rating which describes how many watts it draws. If you multiply this usage in watts by time used you get a kWh value. And given we pay for electricity per kWh we can easily calculate the cost per device.</p><p>But it’s tricky with electronic devices because they can run much lower than their stated power rating when different features are enabled. For example, your computer monitor draws less power when you turn down the brightness.</p><p>We need to do some real-world tests to measure this. I haven’t done these tests in a lab nor are they following the scientific method. They are individual tests I did using two tools:</p><ol><li>Software method using <a href="https://bjango.com/mac/istatmenus/">iStat Menus app</a> on Mac.</li><li>Hardware method using this <a href="https://www.amazon.co.uk/dp/B085S7Q1T4?ref=ppx_pop_mob_ap_share">power meter plug</a>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zy-wsoM9pGgZ-SdRYS8exQ.jpeg" /><figcaption>Mac mini power usage over 9 days</figcaption></figure><h4>Insights from my wattage tests</h4><p>My individual tests running power meter tests over a few weeks give a number of insights:</p><ol><li><strong>The iPad as expected is very efficient</strong>. The iPad Air (5th generation) typically draws 3–4W on 50% brightness. This spiked up to 13W on heavier use like playing a video.</li><li><strong>The Mac mini is just as efficient as the iPad</strong> <strong>(excluding the screen)</strong>. The M2 Mac mini (2023) typically draws 5–7W for light use with Microsoft Teams video calls spiking it to c.11W. Over 9 days of use (6 working days, 3 non-working days) this amounted to 0.6KWh which costs about £0.30. The maximum watts recorded was 28W on startup.</li><li><strong>If you are using an external monitor the Mac mini is more efficient than the MacBook Pro</strong>. However, if you aren’t using an external monitor and just using the MacBook as a laptop, it is more efficient. The M1 MacBook Pro 14 inch (2021) uses 15–20W with 50% brightness for normal use.</li><li><strong>The M1 MacBook Pro uses significantly more than the M2 Mac mini when stressed</strong>. Running Cinebench R23, the Mac mini ran at 26W and 100% CPU for the multi-core test. The MacBook Pro strangely ran at 48W for the same test (and even more weirdly returned a higher score than the M2).</li><li><strong>Reducing the MacBook screen brightness reduces the energy needed significantly</strong>. At idle 1 bar brightness uses about 15W while full brightness uses 26W. So if you want a simple way to reduce your electricity use, reduce your brightness.</li><li><strong>Sleep mode still uses energy</strong>. On the Mac mini, it’s about 1W when in sleep mode. When switched off its uses zero (obviously). But <a href="https://support.apple.com/en-gb/guide/mac-help/mh35848/mac">starting up from cold uses more energy</a> than from sleep. I found the MacBook Pro oddly used about 10W in sleep mode, presumably to keep the battery topped up?</li><li><strong>Macs have become much more efficient in the Apple Silicon era</strong>. The lifetime carbon emissions for M2 Mac minis have been reduced by 55% from the 2018 Intel i7 Mac mini down to 112kg CO2e (256Gb). This is a combination of reduced carbon emissions from production and more efficient use. But worth noting that most carbon emissions are emitted during the production of new computers. Lifecycle carbon emissions analysis from Apple for the M2 Mac mini shows <a href="https://www.apple.com/environment/pdf/products/desktops/Mac_mini_PER_Jan2023.pdf">63% from production</a> while 36% from lifetime use over 4 years.</li><li><strong>My monitor uses more electricity than the Mac mini, MacBook and iPad combined</strong>. I have the LG 34 inch ultra-wide screen (LG34WL85C) which draws 42W with 50% brightness. This was quite a shock but it shouldn’t have been given it has a G energy rating. The eco settings seem to make no difference to the power use. Brightness changes do though. 0% brightness is still visible and draws 26W with 55W at 90% and 58W at 100% brightness.</li><li><strong>Dark mode helps make a small reduction in energy use</strong>. Switching on dark mode in macOS does not change the energy draw of the Mac mini, but it does reduce the monitor energy draw by 1–2W on 50% brightness.</li><li><strong>Bigger screens mean bigger electricity bills</strong>. I tested a 55&quot; LG LCD TV and it uses 85–95W when in use. It uses the higher level when there are more white pixels and the lower range when more black colours. So it seems that the larger the panel, the more electricity is consumed.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*esoPlKNBMq5eWnzx8n2zoA.png" /><figcaption>Mac mini power use over 7 days (iStat Menus)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*V5XPHNhYTfKPrS26kmcwYQ.png" /><figcaption>Mac mini power use over 24 hours (iStat Menus)</figcaption></figure><p>Other appliances were also an eye-opener. I tested an electric clothes dryer and was shocked to see it uses 325W continually. So not one to run 24x7 then.</p><h4>Conclusion</h4><p>The conclusion of these tests is not to rush out to buy the latest energy-efficient laptop or monitor. There are significant carbon emissions from the manufacturing of new appliances (circa 60–70%) and increasing e-waste when we dump unused electronics.</p><p>Instead, increased awareness of energy efficiency when buying new appliances will help us reduce our electricity bills in future. Understanding the performance-per-watt ratio will also allow us to get more from our new computer purchases. Two 4K second screens may look cool but it will burn through more electricity. And if major buyers such as organisations implement this policy, manufacturers will prioritise energy efficiency more as a feature.</p><p>For now, changing how we use the devices we have, e.g. reducing brightness is the simplest way to reduce our bill.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11166ef87c4f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ESG: Why Businesses Must Care]]></title>
            <link>https://medium.com/@petecam/esg-why-businesses-must-care-80f4338c66cb?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/80f4338c66cb</guid>
            <category><![CDATA[sustainability]]></category>
            <category><![CDATA[green]]></category>
            <category><![CDATA[esg]]></category>
            <category><![CDATA[esg-investing]]></category>
            <category><![CDATA[regulation]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Thu, 02 Mar 2023 10:24:36 GMT</pubDate>
            <atom:updated>2023-03-02T10:24:36.184Z</atom:updated>
            <content:encoded><![CDATA[<p><em>ESG is an emerging priority for many businesses. Given the new ESG reporting regulation coming in the next two years, businesses need to act now to ready themselves for a substantial increase in detailed disclosure, expected by investors, employees and regulators.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*m2fSNyJbYDpY7EBH" /><figcaption>Photo by <a href="https://unsplash.com/@kellysikkema?utm_source=medium&amp;utm_medium=referral">Kelly Sikkema</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>For almost 20 years ESG has been expanding its reach for businesses. “Green” funds and investment opportunities are booming with a predicted 84% increase to <a href="https://www.pwc.com/gx/en/news-room/press-releases/2022/awm-revolution-2022-report.html">21% of all assets under management</a> by 2026. It is reported that <a href="https://www.greenbiz.com/report/state-profession-2022-report">60% of CEOs</a> are very interested in ESG issues. Businesses are seeing the opportunity to incentivise more sustainable customer behaviours, such as Green Mortgages that offer lower interest rates for homeowners who invest in energy-efficient homes. And in the last two years, a majority of public companies now voluntarily publish sustainability reports describing their ESG goals and performance.</p><h4>Greenwashing and Green-hushing</h4><p>So much so good. But there are a number of issues to recognise early.</p><blockquote><a href="https://dictionary.cambridge.org/dictionary/english/greenwashing"><em>greenwashing</em></a><em>, noun</em></blockquote><blockquote><em>behaviour or activities that make people believe that a company is doing more to protect the environment than it really is</em></blockquote><p>The UK’s Competition and Markets Authority (CMA) found that up to <a href="https://www.gov.uk/government/news/global-sweep-finds-40-of-firms-green-claims-could-be-misleading">40% of company websites</a> appear to be breaking consumer law with vague sustainability claims that are lacking in evidence or omit negative information. This is not a surprise given that even after 20 years, ESG is still an emerging market where standards are still forming and any comparison between businesses remains subjective.</p><p><a href="https://www.penningtonslaw.com/news-publications/latest-news/2023/greenwashing-and-the-rise-of-green-hushing#_edn1">Green-hushing</a> is the opposite of greenwashing, where sustainability goals are understated or not published. This doesn’t seem as serious as greenwashing but it means investors and customers cannot make informed decisions to invest or buy products. Whether washing or hushing, businesses need clear, standardised guidance to ensure they have credible goals that will make a positive impact on the world around us that is published regularly. Without this, we will not make significant progress towards ESG goals such as reducing the impact of climate change with reduced carbon emissions.</p><h4>What is ESG and where did it come from?</h4><p>You may have read this far and still be wondering, what is “ESG”? It’s a broad term that can be hard to pin down.</p><blockquote>It stands for Environmental, Social and Governance.</blockquote><blockquote>It translates to Planet, People and Processes.</blockquote><blockquote>It is non-financial reporting.</blockquote><blockquote>It is Green.</blockquote><blockquote>It used to be called Corporate Social Responsibility.</blockquote><blockquote>It includes climate change (reducing carbon emissions) and nature stewardship (protecting the environment, circular economy)</blockquote><blockquote>It includes societal benefits like protection of human rights, community impact and Diversity &amp; Inclusion</blockquote><blockquote>It includes regular Corporate Governance.</blockquote><p>So where did ESG come from? ESG had its beginnings within finance during the 2000s. A group of banks and insurers worked with the UN to publish recommendations that advocated for more ESG measures in company decision-making because of their belief that doing so would improve shareholder value. They indicated a role in this virtuous circle from analysts (data), investors (reward), governments (regulate), companies (action) and stock markets (list).</p><blockquote>Companies that perform better with regard to [environmental, social and corporate governance] issues can increase shareholder value by, for example, properly managing risks, anticipating regulatory action or accessing new markets, while at the same time contributing to the sustainable development of the societies in which they operate, <a href="https://www.ifc.org/wps/wcm/connect/de954acc-504f-4140-91dc-d46cf063b1ec/WhoCaresWins_2004.pdf?MOD=AJPERES&amp;CVID=jqeE.mD">Who Cares Wins, 2004</a></blockquote><p>Climate change continues to get the most attention of all the ESG concerns given the national commitments made by governments at successive UN COP conferences. Last year was COP27 while the Paris Agreement on climate change was agreed at COP21 in 2015. Given the projected risk without a substantial reduction in carbon emissions to the atmosphere, the climate will continue to be the top priority for many organisations. However, national ESG regulation will require businesses to describe a wider, more complete description of their risks and impact than just climate.</p><p>But the environment does not just include climate. There are other parts that are collectively grouped together in ESG frameworks as “nature”. Nature is arguably just as critical as carbon emissions given it contributes to a balanced environmental ecosystem. This includes a positive and negative impact on water usage, biodiversity, energy efficiency, deforestation, and pollution.</p><h4>Standardisation Soup</h4><p>SDG, GRI, PRI, SBTi, CDP, TCFD, SASB, ISSB, TNFD. These are just some of the ESG standards that have been launched in the past few years. All of these are voluntary standards from regional or international bodies that aim to define reporting measures for ESG. Most of them are complex, while all of them overlap and also yet differ in places, e.g.</p><ul><li>Some are aimed at investors as primary stakeholders (e.g. SASB) while others are aimed at consumers (e.g. GRI).</li><li>Some are interested in reporting on the impact of the world on companies (e.g. ISSB) while others are also interested in the impact of companies on the world around us (e.g. GRI). This is called “single materiality” compared with “double materiality” and has a substantial increase in expectations.</li><li>Some are aimed at one topic (e.g. SBTi climate, TNFD nature) while others cover all of ESG (e.g. GRI).</li><li>Some are high-level principles (e.g. UN SDG) while others are detailed specifications of reporting statements and metrics (e.g. GRI).</li><li>Some are industry agnostic (e.g. TNFD) while others are industry-specific (e.g. GRI has 5 sector-specific standards so far).</li></ul><p>The good news is that standardisation is happening, and it’s happening quickly. In 2021 SASB merged with IIRC while the SASB and CDSB merged into the ISSB in 2022 following an announcement at COP26. The GRI and ISSB have helpfully announced a collaboration to harmonise standards. This starts to reduce some of the competition and allows global comparison though it does not deal with the real issue of complexity within the resulting standards winners (e.g. GRI, ISSB).</p><h4>National Regulation is Coming</h4><p>While emerging standards are essential to allow companies to talk a common language and report in a consistent way to pre-defined measures, they remain voluntary disclosures. Companies will self-disclose, publish and there it ends. There is no external audit, and no check back to confirm the impact of the changes.</p><p>This is all about to change. Countries are now rushing to introduce ESG reporting regulations to supplement existing laws. This will by definition enforce a standard similar to financial reporting (at least in each country), set penalties for non-compliance and introduce external auditing to maintain compliance. This should as a result enable meaningful comparison between different companies (at least within the same country).</p><p>The EU is trying to lead the world on ESG regulation with its CSRD bill. This is due to be finalised in 2023 and requires reporting from the first batch of companies in early 2025 for the fiscal year 2024. It will be huge, affecting <a href="https://finance.ec.europa.eu/capital-markets-union-and-financial-markets/company-reporting-and-auditing/company-reporting/corporate-sustainability-reporting_en">50,000 companies</a> across Europe including SMEs. The EU has defined their own standard (ESRS) that will be published in 2023 and is closely related to the GRI standard. It will be the Rolls-Royce of ESG regulation and covers all of ESG in detail including double-materiality and sector-specific additions.</p><p>The UK has already introduced mandatory TCFD reporting for premium-listed companies in 2022 which you can read in each company’s annual report. Kainos, as a publicly listed FTSE 250 company <a href="https://go.kainos.com/rs/272-PGO-379/images/270727_KAI_6647_AR22_FINAL.pdf">published its TCFD disclosure last year</a> (along with voluntary CDP submission). This disclosure is only for climate risks and you can read the <a href="https://www.fca.org.uk/publications/multi-firm-reviews/tcfd-aligned-disclosures-premium-listed-commercial-companies">UK FCA quantitative analysis</a> of this year’s submissions to get a sense of the increasing compliance. The UK is consulting on further reporting but has indicated it will add to the climate disclosure with SDR legislation that will use the ISSB international standard.</p><p>The SEC in the US is also conducting an ongoing consultation that has received 4000 responses about its legislation that is mooted to arrive for publicly listed companies in 2024. The scope of the SEC legislation is limited to climate risks and in draft form includes scope 1, 2 and 3 emissions reporting (though scope 3 reporting is one of the contentious items from the consultation).</p><p>What this means is companies need be putting in place governance for ESG in 2023. Governance will include educating executives and staff, setting credible goals (at the very least for climate), mapping out a plan and measuring a baseline. Two years will allow for a baseline year and subsequent year to compare performance against this. From this operational teams can make informed decisions about how to best meet goals, make a positive impact on the world and report this ready for regulators. And because of the complexity, specialist help may be needed so start early.</p><h4>The rise of the Chief Sustainability Officer</h4><blockquote><em>Only 6 percent of large organizations lack a full-time sustainability leader</em></blockquote><blockquote><em>GreenBiz, </em><a href="https://www.greenbiz.com/report/state-profession-2022-report"><em>State of the Profession Report 2022</em></a></blockquote><p>Large organisations are already hiring full-time sustainability roles. These roles are not yet consistent or well-formed so there is a lot of variety in titles and responsibilities. From LinkedIn I can quickly see 10+ closely related job titles:</p><ol><li>Chief Sustainability Officer</li><li>Head of Sustainability</li><li>Head of ESG</li><li>ESG Director</li><li>Director Sustainability and Social Impact</li><li>Director Corporate Social Responsibility</li><li>Climate and Sustainability Officer</li><li>ESG Data Analyst</li><li>Environmental Sustainability Officer</li><li>ESG, Sustainability and Governance Manager</li></ol><p>ESG is an enterprise-wide concern. It brings together issues across people (HR), risk (Finance), suppliers (Procurement), governance (Compliance) and offices (Property). And then it adds on brand new responsibilities like climate, nature and local community impact. So which department are these roles reporting to? It varies widely. This can be the CEO, Chief Risk Officer, Chief Marketing Officer, Chief Information Officer or Chief HR Officer. Some predict it could end up as a Chief Finance Officer&#39;s responsibility given the importance of non-financial disclosures to the market. Time will tell.</p><p>What is clear is the trend will be for organisations who recognise climate risk (and ESG risks more widely) as primary risks to their businesses to create a C-level executive role with responsibility for ESG. Deloitte assessed the <a href="https://www.deloitte.com/global/en/Industries/financial-services/perspectives/the-future-of-the-chief-sustainability-officer.html">CSO role to be just 15%</a> of its survey respondents in 2021 but predicted it would grow rapidly. Given ESG needs group-wide goals, group-wide data collection and group-wide disclosures it necessitates a multi-departmental, multi-disciplinary approach, similar to the journey organisations took with Cyber Security or Data Privacy. Operationalising ESG performance will be a responsibility for all employees, not just the Chief Sustainability Officer.</p><h4>Why Businesses Must Care</h4><p>ESG performance is now vital to understand the value of an organisation to investors. It is also vital to make a positive impact on the world around us in the future. And it will become a mainstream business process within the next couple of years given regulation. Some have mistakenly aligned ESG with a woke agenda. Some have highlighted the lack of impact from nearly 20 years of ESG. Neither of these concerns takes away from the need for companies to do more to improve their positive impact on the world around us and ready themselves in 2023 for the significantly heightened mandatory ESG reporting demands that are coming within the next two years.</p><p><em>If your business has hired a Chief Sustainability Officer recently or you’re already planning for the coming ESG regulation please reach out to me on </em><a href="https://www.linkedin.com/in/petercampell/"><em>LinkedIn</em></a><em>. I’d love to hear how your company is approaching it. And if you are right at the beginning of this journey and want to discuss, reach out anyway.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80f4338c66cb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Kainos is hiring a Data Ethicist]]></title>
            <link>https://medium.com/@petecam/why-kainos-is-hiring-a-data-ethicist-e2fb3ff90b0d?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2fb3ff90b0d</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[governance]]></category>
            <category><![CDATA[trust]]></category>
            <category><![CDATA[ethics]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Fri, 11 Mar 2022 10:33:00 GMT</pubDate>
            <atom:updated>2022-03-15T12:41:09.881Z</atom:updated>
            <content:encoded><![CDATA[<p>Kainos is hiring its first Data Ethicist to improve trust in AI/ML systems. But why? To do what? And why now?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iV2Zmr_bdBeuPrzO" /><figcaption>Photo by <a href="https://unsplash.com/@faithgiant?utm_source=medium&amp;utm_medium=referral">Alex Shute</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>What is Data Ethics anyway?</h4><p>Data Ethicist. AI Ethicist. Responsible AI Lead. Digital Ethicist. Machine Learning Trust Officer. Machine Learning Trust Programme Manager. Trust and Safety Lead.</p><p>On first glance this is an odd collection of job titles. They appear to be philosophical policy-setting roles. But each of these is a real job, with well-known internet-era IT companies Workday, Accenture, Google and TikTok. They are all closely related in their purpose, to ensure the responsible use of AI/ML.</p><p>There isn’t yet a “standard” job title across the industry for this responsible adoption of AI, which is evident by the variety of the job titles. Some prefer to talk of “Ethical AI”, others “Responsible AI” and “Trustworthy AI”. In a sense the label doesn’t matter so much because the underlying concern is to ensure that AI systems are fair (not biased), understandable (not opaque), safe (not harmful), secure (not leaky) and have clear accountability (with a person).</p><p>At Kainos we have adopted “Data Ethicist” because it best encapsulates the breadth of the role across data and AI systems. Data Ethicists will collaborate closely with Data Scientists, Data Engineers and other data job families.</p><p>There has been a rise in the past year or two for dedicated roles to address these unique non-functional concerns because of the increasing reliance on assistance from AI systems and the corresponding rise of high-profile issues. For example,</p><ul><li>Racial bias is accused of facial recognition AI models that haven’t been trained with the right balance of skin colours and racial diversity. <a href="https://www.wired.com/story/the-apple-card-didnt-see-genderand-thats-the-problem/">Privacy concerns</a> and <a href="https://www.reuters.com/technology/exclusive-amazon-extends-moratorium-police-use-facial-recognition-software-2021-05-18/">use for law enforcement</a> are also a challenge for facial recognition systems. If you have time, read the the <a href="https://www.turing.ac.uk/sites/default/files/2020-10/understanding_bias_in_facial_recognition_technology.pdf">Alan Turing Institute’s detailed examination</a> of the systemic challenges with facial recognition.</li><li>Regulated industries such as Financial Services have been slower to adopt some AI systems, e.g. deep learning models because of a difficulty to explain AI recommendations. Public issues like the gender bias reported by AI systems for the “<a href="https://www.wired.com/story/the-apple-card-didnt-see-genderand-thats-the-problem/">Apple Card</a>” demonstrate this challenge.</li><li>Physical safety risks are a priority with Tesla’s Autopilot mode as it develops into Full Self-Driving feature. Media coverage has <a href="https://www.theguardian.com/technology/2021/aug/16/teslas-autopilot-us-investigation-crashes-emergency-vehicles">focused on crashes involving Teslas</a> but it’s clear that one of the most complex computer vision systems will require an equally considered approach to trusted and safe AI.</li></ul><h4>What skills does a Data Ethicist need?</h4><p>Kainos is hiring its first Data Ethicist because it wants to ensure its technology is trusted by users. And as we deliver more AI systems it is time to hire the right person to improve our processes, help our customers to succeed and use their strong voice to educate others.</p><p>The Data Ethicist will have an interest in following and applying the latest data ethics standards, regional legislation, and practices. But more than this, the candidate will be passionate about going beyond ethical AI research to work within multi-disciplinary software delivery teams to ensure customer solutions are trusted by users. This trust will result from user engagement, improved software development processes, better tooling and tighter QA controls. It will allow even our most ambitious customers to build trust with their users.</p><p>Linkedin tells us there are not many data ethicists (in 2022). But this is difficult to quantify given the variety of job titles and lack of standardisation. It is a young discipline that I expect to grow significantly, merging with other forms of data governance, such as data privacy.</p><p>Given this is a young discipline we open to hiring someone already in similar role or someone pivoting from a different role. A technology background is an advantage but this can be learned by someone who is curious about technology. We’re interested in someone who can apply their experience to a new discipline, paving the way for others to follow. We are looking for someone experienced in,</p><ol><li>Developing new processes and practices in an emerging area.</li><li>Developing risk-based governance frameworks or policy, e.g. data privacy.</li><li>Simplifying and clearly communicating risks in conversation, documentation and presentation to C-level stakeholders.</li><li>Critical-thinking and confident engagement with technologists.</li></ol><p>This is important now in 2022 for Kainos because we are building more AI/ML with our customers than ever before and we want to ensure our tech benefits peoples lives. We are working in new industries like Defence that has huge ambitions for Data &amp; AI while operating with higher-risk scenarios. We are pioneering new research on Trusted AI with our academic and media partners. And we expect regulation for AI to significantly increase in the next 12 months with the EU AI Act and its conformity assessment based on risk. This is why Kainos is investing now in Data Ethics.</p><p><em>If you are interested in joining the Kainos team as a Data Ethicist </em><a href="https://www.linkedin.com/jobs/search/?currentJobId=2945506925&amp;keywords=data%20ethicist"><em>you can apply here</em></a><em>. Or if you know someone who is a good fit for the role please refer us.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2fb3ff90b0d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[FOSDEM: More Popular Than Ever]]></title>
            <link>https://medium.com/hackernoon/fosdem-more-popular-than-ever-87e886c2e4d2?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/87e886c2e4d2</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[bitcoin]]></category>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[software]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Thu, 15 Feb 2018 23:15:43 GMT</pubDate>
            <atom:updated>2018-02-15T23:15:43.108Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>8000 geeks.</blockquote><blockquote>42 devrooms.</blockquote><blockquote>2 days in Brussels.</blockquote><blockquote>0 celebrity motivational speakers.</blockquote><blockquote>0 cost to get in.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EY3SpeDznYxWlj4I9-UrMg.jpeg" /><figcaption>A popular talk in the main FOSDEM theatre.</figcaption></figure><p>It’s the most relaxed tech conference I’ve ever been to. Part of this is because it happens on a weekend, in your own time. But another part is because it’s not really a conference at all, but a giant open source meet-up. Dig a little deeper and it’s actually the location for about 40 global open source communities to get together. These are the devrooms where tracks are organised by their own open source communities. A meet-up of meet-ups.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OCLmhKHRqp4IriMR6qQPzw.png" /><figcaption>42 devrooms</figcaption></figure><p>There is a rigid and relentless speaking schedule like other conferences but there is no registration and no entry fee. Entry is on a first come, first served basis. And there are plenty of full rooms. This year especially I got turned away from about 5 rooms. But there are also lots of people milling about, talking, coding, drinking beer. All of this helps reenforce the relaxed feel. And you get to meet with friends you haven’t seen in a while. I had some great chats (and steak) with <a href="https://medium.com/u/fdc2919b4f7b">Rob Lazzurs</a> and friends.</p><p>Free and open source software to give it its full title, or <a href="https://www.gnu.org/philosophy/floss-and-foss.en.html">FLOSS</a> for those in know, is what the meet-up is about. It’s because of this that it attracts lots of geeks. And not so much of the geeks in suits (or maybe geeks in chinos if you work in USA). Even the enterprise geeks are in a minority unlike most tech conferences. Instead the focus of the conference attracts the geeks who contribute to open source projects. This brings with it a different vibe, kind of like an alt-techconference vibe. I love it.</p><p>So FOSDEM is a mirror of open source itself. It’s free. It’s all about the grassroots. It doesn’t have any marketing but has lots of value. It has diverse content. It is a network of communities. And it is put together by community volunteers.</p><p>Some of the largest open source projects have commercial organisations that sell services that surround the code. These organisations help drive adoption of open source to enterprises by selling help and support and sometimes exclusive “enterprise” features. You can see the same big names that sponsor other conferences — Google, RedHat and AWS for example. Beyond this their influence is subtle. Thankfully there are no “sponsored talks” like other conferences that act like product placement advertising for companies. They are a more natural part of the community at FOSDEM. For example one Terraform talk was given by a community guy who was comparing the maturity of Terraform to the maturity of Puppet — but during the talk he called out the Hashicorp CTO who was in the audience.</p><h3>Trending Open Source</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QKsLrG5rRIyxPtPcd0cymw.jpeg" /><figcaption>Everyone in the doorways were turned away from this one.</figcaption></figure><p>The popularity of open source products can be correlated to the popularity of FOSDEM devrooms. From what I seen this year that means Cryptocurrency, Go, Postgres, Containers and Configuration Management are hot. This is no real surprise,</p><ul><li>Containers remain one of the most hyped and popular technologies.</li><li>Some think that Configuration Management has peaked given the rise of container orchestration tooling such as Kubernetes but the Container Management devroom was still filled out.</li><li>Postgres remains the <a href="https://db-engines.com/en/ranking_trend">most popular open source database</a> having strong appeal for relational-store, document-store and enterprise needs.</li><li>Go is an interesting one. I haven’t seen much adoption of Go with businesses yet. But it has <a href="https://opensource.com/article/17/11/why-go-grows">jumped in popularity</a>, seemingly <a href="https://www.infoworld.com/article/3177310/application-development/googles-go-popularity-doesnt-translate-to-the-enterprise.html">driven by developers</a>.</li><li>And cryptocurrency is almost certainly <strong>the hyped</strong> technology of 2017, breaking outside of tech circles to appeal to the mainstream.</li></ul><h3>Lessons from the History of Unix</h3><p>I’ve been thinking recently about the immaturity of the software profession. It’s a very young profession, just an infant compared to ancient professions such as medicine but also to industrial revolution industries such as the automotive and utility sectors. We have an extremely high degree of change in software technology. And so engineers are always looking forward to the next new thing. Because of this, engineer’s don’t spend much time thinking about the history of software. But FOSDEM had a whole track devoted to history which I think is a brilliant antidote to the faddism of tech.</p><p>The History of Unix talk is a great case study. We think of software having a short half-life. 5 years maybe. 10 years at most. However, the first Unix code was created in the early 1970s. That’s 45 year old code. <a href="https://fosdem.org/2018/schedule/speaker/diomidis_spinellis/">Diomidis Spinellis</a> who gave the talk has put together a github repo that has all extant PDP, Unix Research Edition and BSD unix source code history from the 70s to the latest FreeBSD. This <a href="https://github.com/dspinellis/unix-history-repo">repo</a> is a fascinating source for the evolution of Unix.</p><p>Much of the thinking from the early 70s lives on in today’s Linux distributions — kernel, filesystem, virtual devices, pipes, filters, shell, C, dynamic memory, virtual machines, regular expressions. Apparently some code even still lives. You can see the overlap by comparing the <a href="https://dspinellis.github.io/unix-architecture/arch.pdf">architecture diagram from today</a> with the one from the <a href="https://dspinellis.github.io/unix-architecture/arch-V1.pdf">Research Edition of 1971</a>. What can we learn from this? For me, it’s the success of joining many smaller simpler things and the effective use of abstraction and modularity.</p><p>And it isn’t just good designs or code that was decided early. Documentation in Unix distributions remains such high quality because of early decisions about manual pages. Early on distributions mandated this documentation and enforced deletion of any user commands that didn’t have man pages. Clearly draconian but effective.</p><h3>Monero, BitCoin 2.0?</h3><p>Cryptocurrency has broken into the mainstream in 2017. We now see over <a href="https://coinmarketcap.com/all/views/all/">100 cryptocurrencies</a> that can be mined, traded, stolen and (sometimes) used for purchases. Investors are clamouring to invest in the next Bitcoin. VC firms are backing new cryptocurrencies and Independent Coin Offerings (ICO) have entered investor conversation. And of course <a href="https://coincenter.org/entry/what-is-open-source-and-why-is-it-important-for-cryptocurrency-and-open-blockchain-projects">open source is vital to cryptocurrencies</a>.</p><p>Many of the big cryptocurrencies are open source software, led by Bitcoin. Take a look at the <a href="https://github.com/bitcoin/bitcoin">Bitcoin code in GitHub</a>. This makes sense given the aim of cryptocurrencies led by Bitcoin is to offer a de-centralised system of currency using mathematics and independent distribution as trust. Openness increases trust. That’s not to say of course there is no need to trust with open source or cryptocurrencies. Instead trust is placed with the developers of the blockchain, cryptocurrency client or smart contracts. This is how an <a href="https://medium.freecodecamp.org/a-hacker-stole-31m-of-ether-how-it-happened-and-what-it-means-for-ethereum-9e5dc29e33ce">Ethereum wallet bug allowed Ether to be “stolen”</a> from their blockchain.</p><p>Anyhow back to FOSDEM (it’s so easy to get distracted by cryptocurrency). I listened (along with hundreds of the other conference attendees) to a <a href="https://www.youtube.com/watch?v=DI4S6ek7fUI">great talk by one of the Monero</a> contributors. <a href="https://getmonero.org/">Monero</a> is a new open source private cryptocurrency. XMR currently has a price of $272 and is 13th largest cryptocoin by market cap. But we weren’t at FOSDEM to hear about money.</p><p>I hadn’t heard of Monero before this. It was described as “Bitcoin 2.0”. Isn’t that what all cryptocurrencies are attempting to be? Hmmm. What is clear is that Bitcoin has significant challenges as a currency.</p><p>Bitcoin has a finite supply (by design). Hence the long and vicious block-size debate inside the community with the inevitable fork of new cryptocurrencies by those who don’t win the debate. It uses an increasingly large amount of effort to mine (by design). The downside is this means we are using an increasing amount of electricity to create new coins that only have value because of trust (not by how difficult they are to mine). <a href="http://fortune.com/2018/02/13/iceland-bitcoin-mining-electricity/">Iceland is apparently using more for mining that domestic use</a>. It’s mining has become geographically centralised: the majority of Bitcoin mining is said to be in China using dedicated hardware and cheap electricity. <a href="https://www.ft.com/content/adfe7858-f4f9-11e7-88f7-5465a6ce1a00">China has responded by recently clamping down</a>. It also is not anonymous. Bitcoin transactions are stored publicly on the blockchain and when you transact with someone you see their identity and all their previous transactions with this identity.</p><p>Monero looks to solve these by being an anonymous, private digital currency that closely mirrors cash (the original anonymous, rate-limited paper currency favoured by privacy advocates). It uses ring-keys instead of single keys to reduce the likelihood of tracing transactions to one private key. It will in future use i2p multi layer-routing to reduce the likelihood of sender IP tracing. Their proof of work also is designed to use a lot of RAM (2Mb) to try and make it prohibitive for ASIC miners to work at scale.</p><p>So will Monero succeed? No idea. We are undoubtedly in the wild-west of cryptocurrency right now. A few people are Bitcoin millionaires. Many more want to be cryptocurrency millionaires. Eventually things will normalise into a smaller number of truly valuable cryptocurrencies. I suspect that the ones to succeed will have value outside of their backers. If Monero really did become a digital equivalent to cash and it remained anonymous, private and de-centralised this would be valuable to many.*</p><h3>Succession Planning</h3><p>Succession planning is a mandatory topic for senior leaders in organisations who care about the future. So I was surprised to hear a talk about <a href="https://fosdem.org/2018/schedule/event/community_passing_the_batton_foss_leadership/">succession planning for FOSS</a>. Of course many open source communities care about the future of their projects so they are interested in succession. But for open source this is not just leadership succession, it’s succession for anyone in a key contributor role.</p><p>Meritocracy is something often referenced within the open source community. <a href="https://fosdem.org/2018/schedule/speaker/vm_vicky_brasseur/">Vicky</a> pointed out that good as this is, meritocracy is implicitly discriminatory if merit is not defined and when there is no mentoring programme to allow others to succeed. A good point I think — meritocracy needs to be explicit and clearly described to be effective. <a href="https://opensource.com/open-organization/16/6/presenting-framework-meritocracy">Like Redhat did</a>.</p><h3>Funding Open Source</h3><p>The XWiki founder Ludovic Dubost told us about his <a href="https://fosdem.org/2018/schedule/event/community_oss_in_small_company/">funding model and business model for open source</a>. He described how the XWiki code is open source, licensed permissively as LGPL. But he has created a <a href="http://www.xwiki.com/en/">for-profit organisation</a> to provide a commercial SAAS XWiki. So if you want to roll-your-own, use the open source code but if you want a cloud service operated and managed by someone else you can pay the subscription. They also provide some pay-for modules (which is a common thing with other for-profits). This is an interesting model and as more software moves to become a cloud service it’ll be interesting to see if this becomes pervasive. Ludovic also described how his company was non-VC funded because this is incompatible with open source. He told us that VC funding leads to primacy of making money for investors which is not what open source is about.</p><h3>Security Theatre</h3><p>Is Open Source <strong>mandatory</strong> for information security? One speaker about <a href="https://en.wikipedia.org/wiki/Security_theater">Security Theatre</a> believes that it is. There are few today who believe open source is less secure — that view has been <a href="http://www.zdnet.com/article/six-open-source-security-myths-debunked-and-eight-real-challenges-to-consider/">discredited for many years</a>. But is it <em>more</em> secure? I haven’t seen any evidence to support this but I do agree there is a psychological benefit to <a href="https://gds.blog.gov.uk/2017/09/04/the-benefits-of-coding-in-the-open/">coding in the open</a> that encourages better security practice (as well as better practices generally).</p><p>Being open does make things better. Teams who code in the open know they cannot hide behind obscurity and so it changes their approach. Developers know others will look at their code and their commit comments and so they approach it differently. This is big advantage for open source / coding in the open.</p><p>There were of course some less interesting talks. But I’m not going to focus on these because this is inevitable in every tech conference. Instead I’ll finish with one question:</p><blockquote>Why do Belgian frites taste so much better than chips from other countries?</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G_vvm-p0S9INFacy5Hf0vQ.jpeg" /><figcaption>mmm belgian frites double-cooked in horse/cow fat</figcaption></figure><p>You can check out all the <a href="https://fosdem.org/2018/schedule/">FOSDEM talks from this year</a> or watch most of them on the <a href="https://www.youtube.com/channel/UC9NuJImUbaSNKiwF2bdSfAw">YouTube channel</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87e886c2e4d2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/fosdem-more-popular-than-ever-87e886c2e4d2">FOSDEM: More Popular Than Ever</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Strata London 2017]]></title>
            <link>https://medium.com/data-science/strata-london-2017-a2bd12ffa4c?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/a2bd12ffa4c</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[big-data]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Mon, 29 May 2017 11:36:29 GMT</pubDate>
            <atom:updated>2017-05-29T11:36:29.129Z</atom:updated>
            <content:encoded><![CDATA[<p>I went to Strata London last week to see the state of data engineering and analytics in the UK. Of course it is just a tech conference and so while you’re there it’s is a kind of artificial reality. But I was ready for this, I knew that <a href="https://medium.com/daphni-chronicles/why-are-tech-conferences-disappointing-6837d6dd7860">tech conferences can be disappointing</a>. Here’s what I learned.</p><h3>Deep Learning is the latest marketing</h3><p>Some older folks still call it “Artificial Intelligence”. For others it is “Machine Learning” or “ML”. Now it seems like the marketing machines have turned it into “Deep Learning”.</p><p>There was a significant focus in the keynotes on ML and data science. Much of the application of ML today is still focused on recognition: image recognition, speech recognition and driving recognition. This is why the global internet companies dominate this space: huge amounts of data are needed to make ML effective.</p><p>One keynote by an academic showed that when data science was combined with ML we could go beyond image recognition to infer beauty and sentiment. Interesting but still just image recognition.</p><p>I was keen to see where ML was being applied beyond recognition. McKinsey reported that they are meeting an increasing amount of tech-savvy CEOs who have a deep understanding and are trying to work out how to apply it. Beyond this I couldn’t see many speakers shouting about ML integrated to their user services.</p><p>AWS had a low-key presence at the conference but was there to promote Apache MXNet as their framework of choice for AI on AWS. Google have TensorFlow while AWS are backing MXNet. Both TensorFlow and MXNet seems focused on non-functional improvement such as performance and scale.</p><h3>Big Data is just starting</h3><blockquote>“Big data has just started”</blockquote><blockquote>Uber</blockquote><p>Uber collect sensor data from their self-driving car programme to test their cars are making the right decisions when faced with real data. Today each car collects 250Gb per hour. So for 1000 uber cars this is up to 6,000 Tb a day.</p><p>I suspect those organisations with self-driving car programmes will become leaders in hadoop-ecosystem technology. Everyone else will be followers.</p><p>Given all this it’s time to stop using the phrase “Big Data”, it’s become meaningless.</p><h3>Tim O’Reilly does the best keynotes</h3><p>Tim’s talk was the highlight of the conference. He is a technology futurologist, thinking things that are years ahead of everyone else. And even though this was Strata, he spoke more about the socio-economic impact and responsibilities of technologists not so much about data or technology.</p><p>Tim insisted that automation including ML should be used to <em>augment people</em> not to replace people. Automation should be used to allow us to do more by helping humans not produce the same with less people. This is the lesson from the industrial revolution where we are richer than ever before with better life expectancy.</p><p>He also challenged us. Technology is a powerful enabler for good and should be used to improve the life experiences for the many not just the few. It made me wonder whether the social side of our industry should be stronger?</p><h3>Data Engineering and Data Science are different tribes</h3><p>Strata combines three different data-related topics:</p><ol><li>Data engineering. This is hadoop and all the technology to support large-scale data processing.</li><li>Data science. This is more focused on the data not the technology and producing analysis. Many tools vendors were at the expo to support analytics such as Arcadia Data and Trifacta.</li><li>ML. This is about learning algorithms with popular frameworks to make it easier such as TensorFlow and MXNet. All of these need substantial datasets to be able to learn.</li></ol><p>I was trying to resist the urge to spend time learning about the detail of how to scale Kafka/Spark/Cloudera to cope with <em>n</em> million transactions. Instead I wanted to understand more about advanced analytics. What struck me was the oh-so-different audience for the science and ML talks compared to the deep engineering talks.</p><h3>Data engineering is still the Wild West</h3><p>Data engineering is all about processing large amounts of data and storing it somewhere. This storage can be relational storage, NoSQL data store, Hadoop or the filesystem.</p><p>Processing can be done real-time, near real-time or in batches. In the early days of Hadoop it was all about batch processing. However data streaming (near real-time) is now increasingly popular to support a perceived need to process more quickly.</p><p>Trying to choose the technologies for a data pipeline is like a child in a sweet shop. There is too much choice and too much obsessing over minutiae. And because the market is immature, technologies are changing so quickly. Even <a href="https://www.slideshare.net/mobile/hadooparchbook/architecting-a-%20nextgeneration-data-platform">among co-presenters there was differing opinions</a>. What I took away was,</p><ol><li>Kafka is winning for producing and buffering.</li><li>Spark is winning for streaming and querying.</li></ol><h3>Data engineering is complex and often fails</h3><p><a href="https://www.oreilly.com/learning/on-complexity-in-big-data?platform=hootsuite">Jesse Anderson</a> gave one of the only data engineering culture talks. He told us that 85% of big data projects fail. And in his experience this was because of a skills and experience gap in most data engineering teams. He argues strongly for <a href="http://www.jesse-anderson.com/2016/07/ability-gap-why-we-need-data-engineers/">specialist data engineers</a> to counter this.</p><p>I was also interested to hear Jesse talk about the exponential complexity of large-scale data processing compared with SQL or data warehousing. This resonated with me having seen the <a href="http://mattturck.com/wp-content/uploads/2017/05/Matt-Turck-FirstMark-2017-Big-Data-Landscape.png">technology landscape</a>. And so technical architects must resist the urge to deploy large-scale data processing technologies when the volume of data and processing throughput does not demand it.</p><h3>2017 is Year of the Cloud for Data Processing</h3><p>Hadoop solutions have lagged other solutions in their move to the cloud. This has often been for security or performance reasons. But now Cloudera has launched <a href="http://www.zdnet.com/article/cloudera-introduces-altus-offering-hadoop-jobs-as-a-service/">Altus</a> I expect the cloud will become the default for hadoop.</p><p>With Cloudera Altus you persist your data in AWS, e.g. in S3 while your cluster is auto-provisioned by Altus based on defined data pipeline jobs. Afterwards it is de-provisioned. Today it is just focused on data engineering but it is expected to follow this with analytics.</p><p>So who are Cloudera’s competitors? Well Google had a big presence at Strata pushing their cloud, tools and deep learning. Microsoft had a booth at the expo but were not prominent. And AWS were largely absent. I would expect Microsoft Azure and AWS to dramatically ramp-up their big data PAAS services this year at their own conferences.</p><p>It is apparent that Hadoop is just infrastructure for big data. If organisations can forget about managing servers, installing hadoop and managing clusters by using a PAAS data solutions have a better chance of succeeding. This move to PAAS for data services will both move data projects up the stack towards higher-value and allow the market to mature.</p><p>Want to work at Kainos as a data engineer or data scientist? <a href="https://www.kainos.com/careers/find-a-role">Find out more here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2bd12ffa4c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/strata-london-2017-a2bd12ffa4c">Strata London 2017</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Technology Governance]]></title>
            <link>https://medium.com/kainos/technology-governance-c1e308733050?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/c1e308733050</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[agile]]></category>
            <category><![CDATA[community]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Mon, 20 Mar 2017 19:11:26 GMT</pubDate>
            <atom:updated>2017-03-21T08:32:46.694Z</atom:updated>
            <content:encoded><![CDATA[<p><em>by </em><a href="https://medium.com/u/a2a774fe7d7d"><em>Peter Campbell</em></a><em> and </em><a href="https://medium.com/u/8354b7809951"><em>Rory Hanratty</em></a></p><p>Does your organisation have technology governance? Does it allow you to be a growing, dynamic organisation? Or is it focused on compliance not delivery? We want to share how we think about technology governance within Digital Services at Kainos.</p><p>Most of us will be familiar with “governance”. Organisations have <em>corporate</em> governance because they needs rules and processes to function, particularly when company directors have legal responsibilities and liability. In the UK this results in an elected Board of Directors who regulate its behaviour.</p><p>Many organisations also have formalised <em>project</em> governance. This is an attempt to govern the actions and decisions of major projects. The shape of this project governance is typically a series of boards when decisions are made (or pushed up). This shape of governance is of course suited to waterfall projects — it is completely unsuitable for agile software delivery.*</p><p>The reason for this is that agile delivery is centred on delivering working software early to users and iterating on this. Teams need to be responsible for delivering software to users. If it doesn’t get to the users or isn’t working this is an issue for the teams. And so teams must be empowered to make decisions to deliver. If not then they cannot be agile.</p><p>This topic is worthy of its own post in the future to explore how project governance needs to be adapted from PRINCE2 thinking to work for agile delivery. But for now I want to talk about technology governance.</p><p><em>Technology</em> governance is often adopted by organisations in a similar way to project governance. Multiple boards are setup that decide on a recommended technology or design. Architects will present to these boards to get advice or authority to proceed. In larger organisations these can grow to ridiculously large sizes: boards with 25–50 members. And many descend into technology talking shops because that is what hordes of architects like to do.</p><p>But again this way of working is not aligned to an organisation that delivers early. And this is not how we choose to govern technology decisions.</p><p>Within Digital Services we work with our customers to build software that delivers value and meet user needs. Our multi-disciplinary teams work with many different customers with diverse technology contexts. Much of our digital work is ground-breaking for customers and uses latest thinking to continuously deliver software. We work in an environment with high change.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ukUPkiY70uCgTOIkkMfENw.png" /><figcaption>The Governance Pyramid</figcaption></figure><p>The Governance Pyramid summarises our approach to technology governance. On the ground we rely on empowered teams to deliver. And our teams share openly allowing peer-review. Meanwhile we are developing Capabilities and Communities of Practice to collaborate and develop best practice across teams. On top we have accountable persons who define our shared principles and senior architects who provide external assurance.</p><h3>Trusting People over Process</h3><p>This is a core principle from the Agile Manifesto. And it applies directly to our governance. Every year we spend time searching for the best engineers and designers. We take on large numbers of graduates and apprentices that grow up through the organisation. Our staff are supported to learn and grow with both internal content and external training.</p><p>Given all this we want our people to think and innovate. It will be their creativity that finds new ways of working, develops new patterns and explores new technologies. With good execution this creativity will result in success for our customers and for Kainos.</p><p>So we expect to trust our staff to make the right decisions. We haven’t created an all-knowing technology playbook that makes decisions for them. Instead we’re spending time iterating a <a href="https://k-team-roles.herokuapp.com/">pre-flight team checklist</a> for each role. This is why we invest so much to find and retain the best engineers, designers and managers.</p><p>This isn’t to say we don’t have processes. We do.</p><h3>Principles over Homogeneity</h3><p>Our teams share similar principles. These principles mean that we view good software in a similar way. Our intent is to deliver value to users by designing software that is as simple as possible that meets their needs.</p><p>When we come to recruit architects and engineers <a href="https://hackernoon.com/how-to-get-an-architect-job-with-kainos-63d0ddc0e530#.5a623whxv">we are open about the type of people we’re looking for</a>. These are people that think about users and software the way we do.</p><p>But what we don’t try and do is homogenise technologies, design and products. We don’t have some of the things that other organisations do:</p><ul><li>A pre-approved list of technologies that teams must use. Experienced engineers and architects are free to choose the best technologies for the context they are working within — based on customer policy.</li><li>Exclusive relationships with vendors so that must recommend a single product. Teams are free to recommend the right product — commercial or open source — for the context they are working within.</li><li>A board to approve designs. Experienced team members are trusted to decide on how the software they are building should be constructed.</li></ul><h3>Technology Assurance</h3><p>So how do we know that teams are making the right technology choices? And how can we see if they are applying the guidelines well? Well as we said this is primarily by trusting people not by compliance processes.</p><p>But we manage this risk by assuring what teams are doing. This takes a number of forms:</p><ol><li>We have an <a href="https://medium.com/@petecam/open-by-default-communication-a640a68a32d5#.ixjg7qjr5">open-by-default culture</a> and encourage teams to publish their designs as <a href="https://medium.com/kainos/you-must-think-first-before-you-move-7ac4af0346a9#.h0mxpduti">Open Design Proposals</a>. This allows peer-governance and collaboration.</li><li>Assurance by a solution architect attached to the project who works with the teams.</li><li>Assurance by visits from one of our senior architects.</li></ol><h4>Visits</h4><p>During these visits our most senior architects will check-in with teams to understand their decisions and design choices. This cannot be a check of every decision. Instead we will select based on risk-factors such as projects moving from alpha to beta or for major technical decisions such as hosting.</p><p>We are still developing the way we do this external assurance — today this can range from:</p><ul><li>a conversation between architects.</li><li>a day visit to speak with the team. We have started to translate this into risks that can be given to the team to action. It can be shared with customers too.</li><li>a multi-day visit that might report back on engineering and ways of working across teams.</li><li>a code-level review that looks at the state of the code produced by the teams.</li></ul><p>It’s important to note that our assurance is not a big scary thing. It is designed to <em>help</em> teams by celebrating good choices and highlighting change where there are risks. It shouldn’t be seen as a technology performance metric but as important advice from experienced eyes.</p><h3>Peer Governance</h3><p>We also improve governance by encouraging peer-review.</p><p>This happens naturally when people mix from different teams and disciplines. This can happen over Slack or over lunch back in the office on a Friday.</p><p>We’re trying this year to put more effort into tending this natural peer-governance process. A little structure and ownership makes a big difference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FbHCrv8QhwWOaZSfmIIKdw.png" /><figcaption>Communities of Practice, Capabilities and Teams</figcaption></figure><h4>Capabilities</h4><p>We have identified capabilities we believe are important to delivering working software to production and given a single person ownership of that capability.</p><p>These capabilities are grouped around:</p><ol><li><em>Engineering</em> for development. Given the size of engineering, it also includes sub-capabilities such as QA.</li><li><em>WebOps</em> for deployment, infrastructure, monitoring and operations.</li><li><em>Performance</em> for system performance.</li><li><em>Security</em> including application, infrastructure and operational security.</li><li><em>Data</em> for data science and analytics.</li></ol><h4>Communities of Practice</h4><p>We encourage people to create communities of practice to share learning around areas that matter to them, for example development approaches, tooling, practices, machine learning and so on.</p><p>One example of a mature community of practice is our Technology Leaders group. You can read <a href="https://medium.com/@petecam/7-rules-for-building-a-technology-leaders-community-e0205aebce37#.26rm5mgxg">more about how we run our technology leader community</a>.</p><h4>Teams</h4><p>Capabilities and communities will both inform and take direction from what is learned by our delivery teams who implement the architecture, use the tooling and write the code.</p><p>Our intention is that the technology communities and capabilities will drive up innovation and increase peer-governance for each discipline.</p><p>Effective feedback loops between teams, capabilities and communities help to make sure that learning that informs peer-governance is by design, rather than by evolution.</p><p>Without these distinct voices and effective feedback loops you run the risk of reverting to an (ivory tower) review board approach that has lost touch with reality.</p><p>This is how we approach technology governance. We recommend you re-think your technology governance if you’re an agile organisation with waterfall governance.</p><p><em>If you’d like to build great software with </em><a href="https://kainos.com/"><em>Kainos</em></a><em>, please </em><a href="https://www.kainos.com/careers/architects/"><em>find out more here</em></a><em>.</em></p><p>* this is why <a href="https://speakerdeck.com/tastapod/agile-revisited">Dan North calls many projects water-scrum-fall</a> because the agile bit is sandwiched between waterfall governance and processes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c1e308733050" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kainos/technology-governance-c1e308733050">Technology Governance</a> was originally published in <a href="https://medium.com/kainos">Kainos</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to get an architect job (with Kainos)]]></title>
            <link>https://medium.com/hackernoon/how-to-get-an-architect-job-with-kainos-63d0ddc0e530?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/63d0ddc0e530</guid>
            <category><![CDATA[recruiting]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Thu, 01 Dec 2016 14:08:41 GMT</pubDate>
            <atom:updated>2017-07-14T13:42:29.901Z</atom:updated>
            <content:encoded><![CDATA[<p>Job application and interview processes can be long and difficult. It’s really tricky to know what an employer is looking for or whether they meet your expectations. Job descriptions try to explain a role but often don’t reveal much about the selection criteria for candidates. How brilliant would it be if employers told you what <em>their</em> <em>criteria</em> was from the beginning?</p><p>Here at Kainos our multi-disciplinary teams develop software supporting digital transformation. Teams get to work on creating and building large-scale digital services and products that are transforming citizen and user experiences.</p><p>We like (software) architects to work <em>within</em> our teams, guiding them to make the right design and technology choices as they build a software product. We look for our architects to inspire and influence those around them not just in how they work, but how they grow and learn. We trust our architects to make important decisions so it’s important we hire those who will make good ones.</p><p>But we have been finding it really difficult to find the right type of architect to match our growth. We think openness and transparency about what we are looking for could help with this.</p><h3>What do we mean by “Architect”?</h3><p>An architect (in the software sense) is a dedicated role that requires breadth to understand how different elements work best together. It also needs a depth of understanding of code, components and products. This is why our architects are engineer-architects not process-architects or ivory-tower-architects.</p><p>Architect and architecture have no consistent industry definition. When we say “architect” we mean someone who is responsible for making decisions about software design. <a href="https://github.com/Rory80Hz/blog/blob/master/general/architect.md">Rory said it well</a>,</p><blockquote>An Architect is someone with sufficient experience to make decisions and guide others in the right direction, balancing dogmatism and pragmatism, know that they don’t know everything, understand that they have more they can learn, and above all else understand how they can use experience and knowledge to guide their decision making.</blockquote><p>We scope the decision-making of architects at three different levels,</p><ul><li><em>Technical</em> <em>Architects</em> make design decisions about the application or infrastructure for a single team. Technical Architects will spend most time with code compared with other architects.</li><li><em>Solution</em> <em>Architects</em> make design decisions for a whole solution which may stretch across many teams. They may specialise in infrastructure or application or data but have an understanding of the others.</li><li><em>Principal</em> <em>Architects</em> make design decisions for larger solutions or across many software products. They do this with responsibility for application, infrastructure and data design.</li></ul><p>This forms the core of our architect roles. There are of course additional responsibilities that Technical Architects, Solution Architects and Principal Architects will have — these are described in the job descriptions. The point of this article isn’t to describe these roles but instead to focus on the characteristics of the architects we want to hire.</p><h3>Who are we searching for?</h3><p>Our search is for individuals who have the right experience and a similar mindset to us when building software. However, someone with all the right technical experience but who can’t work within our teams is a bad fit. It is important that we find those who care about building software that is well-considered, tested and meets user needs.</p><p>Specifically we are searching for those who are,</p><ol><li><strong>Switched on to what architecture is</strong>. Architecture for us is about making good design decisions with the right technologies. It is more about design than selecting technology products (though this is part of it). It is more about building software products well than writing options-papers (this is also necessary sometimes).</li><li><strong>Excellent decision-makers</strong>. We want to see evidence of thoughtful, well-considered decision-making in the recent past. This is critical because this is the primary role of an architect.</li><li><strong>Engineers who understand code</strong>. Architects need to have a current understanding of code to be able to make good design decisions. Most of our architects don’t have time to write code every day. This is understandable given their primary responsibility is not to write code but many still make the time because they enjoy it and it keeps them current.</li><li><strong>Used to delivering working production software to real users</strong>. Architects will have experience of delivering through to production. This will ensure a better understanding of non-functional demands, operational needs and user feedback.</li><li><strong>Switched on to avoid non-functional failure</strong>. Architects need to advise on the priority of non-functional features (often over functional features) to product owners. This is necessary to avoid <a href="https://hackernoon.com/non-functional-failure-489013986d50#.5fn8v0syy">technical failures</a>. To do this they need to understand operational issues and how this will affect their design decisions.</li><li><strong>Excellent communicators</strong>. Clear thinkers also need to be able to communicate their thoughts to others verbally, in prose and in drawings. Architects should be able to communicate easily with both developers and those less-technical. Waffle in CVs or interviews or inability to draw designs is a sign of unclear thinking and/or communication.</li><li><strong>Able to influence customer thinking</strong>. Architects will be advising customers regularly. They will need to be able to talk with and influence the decision-making of both technical and non-technical customers.</li><li><strong>Prefer Open to Closed</strong>. Open standards and open source over the last 20 years have helped make it easier to build better software. We are looking for those who recognise this and who prefer open to closed.</li><li><strong>Collaborative Leaders</strong>. Architects are experienced but they don’t know everything. They have to rely upon their teams to make the best decisions. We want architects who learn from the development team instead of dictating designs. We look for those who lead collaboratively, encourage and rely upon specialist knowledge and opinions from within their team.</li><li><strong>Simplifiers not Complicators</strong>. Building software is difficult. Those who make it more complex than it needs to be create unnecessary problems for teams, maintenance and operations. So we look for those who are striving to simplify.</li><li><strong>Always Learning</strong>. Like many of the attributes described here learning does not just apply to architects. Active, thoughtful-learners are typically focused on improving how things are done and are not content with the status quo. If architects are always learning this leads to better decisions.</li><li><strong>Helping and Growing other People</strong>. Kainos is a growing business that needs our more experienced people to grow others. Creating opportunities and growing others should be a natural thing for our architects.</li><li><strong>Pathfinders.</strong> We love to work with people who challenge the way things are done. We actively encourage people to spend time finding new ways to design and build software.</li></ol><h3>Things we’re flexible about</h3><p>Many of the characteristics we’re looking for are more about an architect’s mindset that are evidenced through recent experience. This might seem like a high bar: it isn’t as simple as just finding people with experience of <em>x</em> and <em>y</em> technologies for <em>z</em> years.</p><p>However, there are some things that we are flexible about.</p><ul><li><strong>Java</strong>. We don’t have a technology policy that mandates which tech stacks we will adopt. However, we prefer open source and most of our teams over the last few years have chosen to use Java/JVM (or .NET) and Javascript. We are prepared to help upskill hires who don’t have deep Java experience.</li><li><strong>Agile</strong>. We prefer to work with an agile delivery model. However, we do not require agile experience as a prerequisite for hiring. What we are looking for is a mindset that is suited to work in a way that doesn’t plan everything up-front, accepts change, works collaboratively and is focused on improvement and value. We are prepared to help upskill hires who don’t have deep agile experience if they have the right mindset.</li><li><strong>Commercials</strong>. We are a commercial organisation and all staff especially more senior people need to have a realisation of the commercial impact of their decisions. Architects will also support the sales process with their technical expertise to imagine, illustrate and size solutions. We are prepared to help upskill hires who haven’t worked in commercial organisations before.</li><li><strong>Location</strong>. We hire architects from all over the UK, Ireland and Poland. Some live close to our major offices in Belfast, London and Gdansk. Others are home-based. Depending on customer needs we will often co-locate with our customers. We don’t ask people to travel 5 days a week but co-located projects need its architects to travel to be with the customer and teams.</li><li><strong>Certifications</strong>. We don’t look for architecture professional certifications such as TOGAF because we don’t find that these distinguish good software architects. So we prefer to use our own criteria.</li></ul><p>If you recognise yourself in the description above and you’d like to build great software with <a href="https://kainos.com/">Kainos</a>, please <a href="https://www.kainos.com/careers/architects/">find out more here</a>.</p><p>Thanks for the contributions and edits from Kainos architects: <a href="https://medium.com/u/8354b7809951">Rory Hanratty</a>, <a href="https://medium.com/u/c556fc2be07f">Will Hamill</a>, <a href="http://kylethompson.co.uk/">Kyle Thompson</a>, <a href="https://medium.com/u/33dcab2bd0c1">alastair allen</a>, <a href="https://medium.com/u/5d5105b1a3e7">Joseph McKavanagh</a> and Damien Taylor.</p><blockquote><a href="http://bit.ly/Hackernoon">Hacker Noon</a> is how hackers start their afternoons. We’re a part of the <a href="http://bit.ly/atAMIatAMI">@AMI</a>family. We are now <a href="http://bit.ly/hackernoonsubmission">accepting submissions</a> and happy to <a href="/service/mailto:partners@amipublications.com">discuss advertising &amp;sponsorship</a> opportunities.</blockquote><blockquote>To learn more, <a href="https://goo.gl/4ofytp">read our about page</a>, <a href="http://bit.ly/HackernoonFB">like/message us on Facebook</a>, or simply, <a href="https://goo.gl/k7XYbx">tweet/DM @HackerNoon.</a></blockquote><blockquote>If you enjoyed this story, we recommend reading our <a href="http://bit.ly/hackernoonlatestt">latest tech stories</a> and <a href="https://hackernoon.com/trending">trending tech stories</a>. Until next time, don’t take the realities of the world for granted!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63d0ddc0e530" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/how-to-get-an-architect-job-with-kainos-63d0ddc0e530">How to get an architect job (with Kainos)</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Slave to Software Architecture Trends?]]></title>
            <link>https://medium.com/hackernoon/slave-to-software-architecture-trends-5cd8899af381?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/5cd8899af381</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[trends]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Mon, 17 Oct 2016 18:45:01 GMT</pubDate>
            <atom:updated>2017-07-14T11:54:52.950Z</atom:updated>
            <content:encoded><![CDATA[<p>Have you noticed the latest trends in software architecture? These trends are pervasive practices that <em>everyone</em> seems to be talking about and doing. Or at least claim they’re doing.</p><p>Now we’re used to trends in fashion. Some of us are slavish followers of fashion. Most of us aren’t — we don’t care so much about following the latest fashion. But it is difficult to avoid the impact of trends even if you’re a late-follower or one of the late-majority. Take a look back at your family photos from 20 years ago, you will be dressed in the fashion of the day. The trend wave carries us all along.</p><p>This trend wave appears to be similar for software. For example in the 2000s we had the mass adoption of the internet and the resulting SOA wave. This Service Oriented Architecture trend was very appealing to organisations and architects. Everyone was talking about it like it was a must-have approach for new software. And yet no-one is talking about SOA today. Instead there is much talk about Microservices, Reactive Programming and Big Data.</p><h3>A Brief History of Software Architecture Trends</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wtc4eedL7ZNCGd2RKKWEWw.png" /><figcaption>Peter Campbell, 2016</figcaption></figure><p>I couldn’t find much on the history of software architecture trends on the interwebs. The best I could find was <a href="http://arch.simplicable.com/arch/new/a-brief-history-of-solution-architecture">A Brief History of Software Architecture</a> by simplicable. So I attempted to draw one taking inspiration from this and my experience. You can see the major trend types from Machine Code to DevOps (above).</p><p>I then added some of the practices and technologies that arose from these trend types (below). I haven’t tried to make it comprehensive or align it with a timeframe because I wanted to avoid tumbling down a rabbit hole. I’ve grouped all the practices by trend type (even if they arose later than other trends). There are also missing links, such as virtualisation to cloud computing: I avoided drawing cross-trend type relationships to simplify the picture. This is my brief history of software architecture trends.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jooAZNM7Lh8qVtFvJ7dyig.png" /><figcaption>Peter Campbell, 2016</figcaption></figure><p>I started programming in the midst of the Distributed era when high-level languages were standard (for business applications at least). My experience of distributed computing has seen it move from binary and proprietary to simpler open standards through RPC, CORBA, RMI, EJB, SOAP to REST. The internet is perhaps the single biggest impacting trend in software architecture and has given birth to major trends such as SOA and Microservices.</p><p>Today the effects of previous trends remains strong. The internet remains a major facilitator of advances and trends. “Big Data” is spoken about less than it used to be (perhaps because it’s a poor name) though Hadoop and its ecosystem is still an underexploited technology. The cloud vendors continue to expand their platforms from the basic virtualisation (infrastructure-as-a-service) to anything-as-a-service. The ubiquity of smartphones has plenty of room for more advanced apps, particularly for business apps. And DevOps toolchains have become mainstream and yet Continuous Delivery remains elusive for many.</p><h3>Don’t Be a Slave</h3><p>Slavery to fashion trends is dangerous for a software architect. Drinking too much of the kool-aid can easily distort your vision of the context and value of technologies.</p><p>This is important because microservices are not <em>always</em> the right solution. A relational database might be <em>just</em> what your software product needs. A <a href="https://m.signalvnoise.com/the-majestic-monolith-29166d022228#.14izxw2jk">Majestic Monolithic</a> could meet the needs of your organisation. Using shell scripts for deployment to production could lower the bar to allow <em>anyone</em> to deploy code.</p><p>This reminds me of being caught up in the Enterprise Java Beans wave in the early 2000s — when I was a lot younger. Back then it offered standardisation across the Java vendors (for the code though not so much for the configuration), easier Java for distributed computing, open source implementations and a new advanced application server runtime. I happily dived in with stateless session beans, entity beans, application containers and message beans. And yet a few years later there was a (necessary) backlash to the heavyweight model, awkward persistence model and monolithic runtime from Spring and others. The trend weakened as a result. I learned the lesson back then that software architecture trends are fickle.</p><p>So lets learn from history. Lets all have a clearer self-awareness of software architecture trends and evaluate them in the context of the needs we see in front of us.</p><p>I’m sure there’s a least one historical inaccuracy hiding in here. If you spot it let me know.</p><figure><a href="http://bit.ly/HackernoonFB"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0hqOaABQ7XGPT-OYNgiUBg.png" /></a></figure><figure><a href="https://goo.gl/k7XYbx"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vgw1jkA6hgnvwzTsfMlnpg.png" /></a></figure><figure><a href="https://goo.gl/4ofytp"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gKBpq1ruUi0FVK2UM_I4tQ.png" /></a></figure><blockquote><a href="http://bit.ly/Hackernoon">Hacker Noon</a> is how hackers start their afternoons. We’re a part of the <a href="http://bit.ly/atAMIatAMI">@AMI</a> family. We are now <a href="http://bit.ly/hackernoonsubmission">accepting submissions</a> and happy to <a href="/service/mailto:partners@amipublications.com">discuss advertising &amp; sponsorship</a> opportunities.</blockquote><blockquote>If you enjoyed this story, we recommend reading our <a href="http://bit.ly/hackernoonlatestt">latest tech stories</a> and <a href="https://hackernoon.com/trending">trending tech stories</a>. Until next time, don’t take the realities of the world for granted!</blockquote><figure><a href="https://goo.gl/Ahtev1"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*35tCjoPcvq6LbB3I6Wegqw.jpeg" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5cd8899af381" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/slave-to-software-architecture-trends-5cd8899af381">Slave to Software Architecture Trends?</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Always Learning]]></title>
            <link>https://medium.com/kainos/always-learning-708a1d8b8954?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/708a1d8b8954</guid>
            <category><![CDATA[education]]></category>
            <category><![CDATA[self-improvement]]></category>
            <category><![CDATA[learning]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Mon, 10 Oct 2016 08:26:07 GMT</pubDate>
            <atom:updated>2016-10-10T08:26:07.261Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I66Yzj-Ne2mWjCurOIzZ0g.jpeg" /><figcaption>Learning about the Gdansk shipyard in the <a href="http://www.ecs.gda.pl/title,Jezyk,pid,2,lang,2.html">Solidarity Museum </a>with <a href="https://twitter.com/nlynas">@Nigel Lynas</a></figcaption></figure><blockquote>What have you learned in the last six months?</blockquote><p>This is a question I sometimes ask when interviewing. It helps to tease out if someone has stopped learning — or worse, stopping thinking <em>they</em> need to learn.</p><p>When you are younger it is easy to think you have lots to learn. Then when you’ve worked for a while and learned a bit it can become natural to think you’ve learned lots. And that can lead some to consider they have less to learn. But after a few more years you will likely come to realise there is <strong>so much</strong> left to learn. So much that it doesn’t feel like less but <strong>more</strong> to learn, because you can see how much more there is.</p><p>Continually learning is important because it shows those who have a mindset to continually improve. So, what have you learned in the last six months?</p><h3>Body and Mind</h3><p>Learning can be for the body (skills) or for the mind (ideas). Some like to extend their abilities by learning new skills:</p><ul><li>Learning a new <em>language</em>.</li><li>Learning a new musical <em>instrument</em>.</li><li>Learning a new <em>sport</em> or exercise.</li><li>Learning a new <em>technique</em>.</li></ul><p>Others want to expand their mind by learning new ideas:</p><ul><li>Learning a new <em>concept</em>.</li><li>Learning some new <em>facts</em>.</li><li>Learning <em>how</em> something works.</li><li>Learning a new <em>technology</em>.</li></ul><h3>Experience</h3><p>Very often when hiring we distinguish between “experienced” people and recent graduates or undergraduates. Lets deconstruct what we mean by this.</p><p>By “experience” we mean those who have spent time doing the job as described. Those who have learned the skills and technical information needed to succeed at their job.</p><p>During interviews we attempt to assess whether candidates have met their goals or not by asking probing questions. We break down knowledge and skills needed for the role and ask questions to allow people to demonstrate how they have met these. And yet given this we often fallback on duration to help us. Does this really help?</p><blockquote>“Have you got at least five years of experience in this role?”</blockquote><p>This is not helpful. A person who learns more will have better experience that someone who learns less in the same time period. We should be on the lookout for those who are faster and more voracious learners. Faster learners are the people to hire — not just because of five years duration in a role.</p><p>But how do we recognise these people?</p><h3>Self-Aware Learners</h3><p>It’s difficult to quantify someone’s learning rate. What we can do though is understand how self-aware someone is about the ways in which they learn. Then we can find out the size and rate of things they have learned recently. This is why the interview question is valuable.</p><p>For example, if someone learns by “googling things” or “from the internet” it’s a sign they haven’t really thought too much about how they learn best.</p><p>I find I learn best by,</p><ol><li><strong>Learning from doing</strong>. It is well known that you can learn more from your failures than your successes. Of course it is easy to <em>not</em> learn from your mistakes and that’s Not A Good Thing. This is why agile practices encourage experimentation with a short feedback cycle: to allow iteration to improve. I like to learn by doing and I find this is a good model for learning — start small, test it and iterate it.</li><li><strong>Learning from people around me.</strong> We can also learn from others if they are open enough. This is especially true when learning from others failures. It’s not easy to share your failures but learning from a trusted team is <em>so</em> much more accelerated than doing it all yourself. This is mostly unplanned learning. I find that this is the primary way I learn nowadays. There is only so much I can read. There is only so much I can do. But a team can do and read many multiples of what I can.</li><li><strong>Learning from reading</strong>. Reading broadens our minds even when we don’t know the writer. It doesn’t have to a tome either, browsing my twitter feed has introduced me to new and interesting thoughts. I have also found <a href="https://medium.com/">Medium</a> to be a rich source of relevant ideas to me over the past year (especially when I tuned my recommendations by blocking the writers that I’m not interested in — there’s only so much self-help articles I can take).</li><li><strong>Learning from writing</strong>. I have been trying to write more this year. This has been both educational and cathartic. I find I enjoy writing (even if few are listening). It’s been educational because I now appreciate more than ever the importance of self-editing, third-party editing and empathetic reviewing. But more than this, it helps me refine my thinking. It pays to be more careful if your words are public.</li><li><strong>Learning from listening to a course</strong>. This is academic learning where you choose to take time out to learn something specific. Of course nowadays e-learning and MOOC have made this more convenient than ever. But this is always planned learning. I find multi-topic conferences work better for me rather than specific courses or post-graduate courses.</li></ol><h3>More New Skills Please</h3><p>While writing this (more learning) I’ve realised I don’t try to learn new skills as much as I could. I spend more time learning new ideas. So during the summer I did a sprint duathlon for first time (and second time too). This was tough but reminded me why I prefer cycling to running. Next I’m going to have a go at <a href="http://www.ulstercyclocross.com/">cyclocross</a> racing.</p><p>Peer group pressure can be an encouraging way to learn new skills. The <a href="https://www.justgiving.com/teams/scrumteambadass">Kainos Tough Mudder</a> team have dramatically increased their exercise (skills?) throughout 2016. They also ignored their collective injuries and made it round the course. <a href="https://medium.com/u/619220208bda">Kieran Barber</a> even <a href="https://medium.com/@kieran.barber/i-survived-tough-mudder-65182c4a6463#.vau3vye1y">wrote about it</a>.</p><p>It’s easy to be inspired by other’s learning. Many were amazed that <a href="https://www.theguardian.com/technology/video/2014/oct/23/mark-zuckerberg-beijing-speaks-mandarin-video">Mark Zuckerberg spent time learning Mandarin</a> to allow him to address a 30 minute Q&amp;A session in China. <a href="https://medium.com/u/d37ebc7e9ff2">davey.mcglade</a> did this too with his dedication to <a href="https://medium.com/@daveym/couch-to-half-marathon-6546e835d659#.lzerhgjea">become a half-marathon runner</a> in such a short time.</p><p>Employers can help too. <a href="https://twitter.com/KainosSoftware">@KainosSoftware</a> introduced a <a href="https://www.kainos.com/training-in-kainos/">Skills for Me</a> programme last year that encourages all staff to learn a new skill for free once a year. I spent a day learning to ride a trail bike in a forest (great fun). This has been a great initiative and has encouraged many to try new skills.</p><p>What new skills, ideas and lessons have you learned in the past six months?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=708a1d8b8954" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kainos/always-learning-708a1d8b8954">Always Learning</a> was originally published in <a href="https://medium.com/kainos">Kainos</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Non-Functional Failure]]></title>
            <link>https://medium.com/hackernoon/non-functional-failure-489013986d50?source=rss-a2a774fe7d7d------2</link>
            <guid isPermaLink="false">https://medium.com/p/489013986d50</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[failure]]></category>
            <category><![CDATA[digital]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Peter Campbell]]></dc:creator>
            <pubDate>Tue, 26 Jul 2016 15:01:02 GMT</pubDate>
            <atom:updated>2017-07-13T21:58:35.685Z</atom:updated>
            <content:encoded><![CDATA[<p>One thing stands out most from what I have learned in the past 10 years.</p><blockquote>Non-functional failure is the most dangerous technical risk in software.</blockquote><p>Agile is designed to allow for change, encouraging experimentation. And if you are experimenting — with your design, user experience or technology — you should expect to fail. By failing you learn more, allow innovation and will have a better product afterwards. We expect this from controlled experimentation, but are caught out when a whole service fails. It is this macro-failure that we must beware of.</p><h4>Macro Failure</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YqF23v0AaE8E5TodBIn6MA.jpeg" /><figcaption>credit: <a href="https://medium.com/u/8354b7809951">Rory</a> Hanratty</figcaption></figure><p>There are five macro reasons I can see why digital services fail.</p><ol><li><em>Building a product that doesn’t work</em>. This is most obvious type of failure when software is not functional. It is the right product but it’s defective.</li><li><em>Starting but not finishing the product</em>. Fundamentally the software was not delivered to its users.</li><li><em>Spending more money than expected</em>. Running over budget is not automatically failure but <em>major</em> budget overrun added to one of the others is.</li><li><em>Building the wrong product</em>. You might have the best developers with the best technologies but if the product you give to your users is the wrong one then you’ve failed.</li><li><em>Building a product that may work but can’t be used</em>. This is the silent but deadly reason for failure. It is often badly understood by product owners. This is the failure I want to explore.</li></ol><h4>What is Non-Functional?</h4><p>My definition is,</p><blockquote>non-functionals are concerned with <em>how</em> your software product works not <em>what</em> it does (functional).</blockquote><p>So for example, if your service allows users to obtain a fishing license but in doing this, your data is exposed, then it is insecure. This is a non-functional issue.</p><p>Some have abandoned the word “non-functional” altogether and have adopted words like “<a href="https://www.mountaingoatsoftware.com/blog/non-functional-requirements-as-user-stories">constraint</a>” but I’m not sure this adequately covers what is needed.</p><p>It is common to see software products that have non-functional failures because these concerns are often badly understood for software. If you think about a new car, its non-functional concerns are well understood: it should be drivable by one adult (accessibility), do 100 mph without falling apart (performance), prevent others from stealing it (security) and be able to do 20,000 miles until its first service (reliability).</p><p>It is dangerous to think non-functional issues are more relevant to software architects than users. There is a close relationship between <em>what</em> your service does and <em>how</em> it does it. Issues with <em>how</em> your service works are often barriers to <strong>any</strong> use of your service. If your service is not accessible on mobiles or tablets users will avoid using your service. If your service has performance issues users will not be able to use your service. Look what happened to <a href="https://www.engadget.com/2016/07/08/pokemon-go-global-rollout-pauses-to-fix-server-problems/">Pokémon Go</a> in its launch week.</p><p>So we all need to take more care to ensure non-functionals are taken seriously.</p><h4>Silent But Deadly: The NFR Trap</h4><p>I have written before about the <a href="https://hackernoon.com/top-10-performance-optimisations-18cdad54155c#.qsjb8tcuq">NFR Trap</a> in relation to system performance. The trap is to believe your team won’t have non-functional issues because you’ve got “The NFRs”.</p><blockquote>There are some who get performance optimisation. They test, they analyse and they fix. But when real users start to use their service they have performance problems. Unfortunately these folks often get snared by the NFR Trap.</blockquote><blockquote>The NFR — Non-Functional Requirements — did not describe the level of real-world usage that real users impose on the service. Instead very complicated, hard to understand and detailed requirements were constructed by a smart software architect that didn’t get the user need or user context.</blockquote><p>With service design and agile development we now have a focus on users and their needs represented as user stories instead of business requirements. Yet non-functional requirements are often represented as a list of abstract statements about things like system performance, security, usability, accessibility, availability, maintainability and business continuity. They are abstract because they doesn’t relate to users (bad), don’t mean much to most people (bad) and are difficult to test (very bad).</p><p>Often The NFR are kept separate and referenced, but are very difficult to corroborate or approve. The NFR are often derived from templates that carry an undue reverence. Take <a href="https://www.google.co.uk/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=3&amp;cad=rja&amp;uact=8&amp;ved=0ahUKEwi_wqGk1IHOAhUGG5oKHYrlCtgQFggoMAI&amp;url=http%3A%2F%2Fwww.cragsystems.co.uk%2Fdevelopment_process%2Fdocs%2FNon-FunctionalRequirementsDocumentTemplate.doc&amp;usg=AFQjCNHTdByX_IWNXT121NpqBRFLAp_WQQ&amp;sig2=WVQ2v3KsinMwnomd0mM7sA">this NFR template for example</a>. It is typical of what is perpetuated by many teams but unfortunately its inadequate.</p><h4>Two Examples</h4><p>Lets look at two common examples to understand why traditional NFRs are inadequate. Ask yourself for each one what it means for the users.</p><blockquote>Availability must be no less than 99.9%</blockquote><p>What this means is that the system should be available (working) 99.9% of the time — except when it is down for scheduled maintenance.</p><p>There are a few problems with this. It doesn’t relate to availability when <em>users</em> need it. If peak usage is in the morning time but the service is used in the daytime only then availability is business critical in the morning, important in the afternoon and not required in the evening.</p><p>How will scheduled downtime affect users? It is rare to see scheduled downtime targeted in NFRs but users don’t care about this — downtime of any kind means they can’t use the service. So for a 24x7 service understanding what users can tolerate and the resulting design for minimising or zero downtime will be important.</p><p>How can you be sure 99.9% is even necessary? It isn’t untypical for these numbers to be guessed, written in The NFR by The Architect and never questioned ever again. A better question to ask is, what is the impact to users when the service is not available and what alternatives will they have? Designing a contingency or having high impact areas of your service less complex (to allow easier redundancy) may be time better spent for your users instead of a focus on an uptime threshold.</p><blockquote>90% of all page requests must be completed within 1.5 secs</blockquote><p>This is an attempt to describe how responsive your service should be based on experiences with popular websites. It is written to provide confidence that your service will be “fast” for its users. Lots of time may have been spent in meetings discussing whether the target should be 1.5 seconds, 2 seconds, 3 seconds or something. But isn’t this missing the point?</p><p>Surely the point is to understand what performance is expected by your users? What performance level will allow them to use your service without frustration? To understand this, it’s necessary to speak to users and do effective research with them and analyse performance data. Some parts of your service might be time-critical, others less so. Use this to prioritise critical features within your service avoiding generalised page response time targets like the one above.</p><p>While you’re doing this make sure you validate actual performance by testing it.</p><h4>Test It, Don’t Just Require It</h4><p>It’s vital to ensure features are <em>measured, </em>especially for performance. Performance testing is more important than performance requirements. This is because you can easily iterate and optimise performance based on test results and user research <em>if you are doing it</em>. Active performance testing of features by teams should become the new normal. And the results can even be used to guide acceptable performance for users.</p><p>There is a risk though that your testing could be providing false confidence. To help avoid bogus performance testing, you need to ensure a number of realism-factors are present.</p><ul><li><em>Test on production-sized infrastructure</em>. Even better test on Production if it’s a new service.</li><li><em>Test with production-sized datasets</em>, ideally production data or anonymised production data.</li><li><em>Test response time at peak load</em>. You want your service to be as responsive on its worst day as its best day. This means testing at peak load.</li><li><em>Ensure the testing is end-to-end</em>. Often response time testing is done from edge of the data-centre only (largely because it’s easier to measure). However this doesn’t help your users.</li><li><em>Ensure the testing throttles connectivity and client performance</em> based on user devices. If you have a significant group of users with older devices on sub-broadband speeds their performance will be substantially slower.</li></ul><p>So we’ve seen that The NFR are often too abstract and are rarely considered in context of the users. Let’s rip up The NFR and start again with non-functional needs that are user-focused, testable and a regular aspect of team development.</p><h4>Non-Functional Features</h4><p>Non-functionals can be normalised within agile development by considering them as features. Many non-functionals as we’ve already seen heavily impact on user experience and so can be <a href="https://www.mountaingoatsoftware.com/blog/non-functional-requirements-as-user-stories">written as user stories</a>. Alternatively orthogonal features such as performance expectations can be integrated into your stories as acceptance criteria.</p><p>When is the right time to do this? Beta. The Beta phase is where you build out an end-to-end service and starting using it with production data. Just make sure your non-functional features are developed in your backlog at the <strong>beginning</strong> of Beta. Waiting until near the end of the Beta phase is an invitation to fail.</p><p>At Kainos we have written some guidance for teams moving into Beta. These were written by a bunch of Kainos technical architects who have seen the lows of non-functional failure. These will help guide some of the more important non-functional features you should be thinking about.</p><ol><li>You have a Backlog that contains non-functional features.</li><li>You have a Backlog that ensures operational (live running) aspects of functional stories are accounted for.</li><li>You have a production environment for the start of private Beta.</li><li>You have a deployment pipeline to build, package and release features into production rapidly.</li><li>You have a deployment pipeline to build, package and release patches into production rapidly.</li><li>You have invested in automation for builds, tests and deployment (application and infrastructure).</li><li>You have instrumentation to understand what your users are doing with the service.</li><li>You have got aggressive scale and performance targets. Don’t be satisfied with historic peaks. You have load, stress, bandwidth and soak tested your service beyond these performance targets.</li><li>You have tested all integration points (internal and external to your service) for performance, scale and stability.</li><li>You have performance, application and infrastructure monitoring to understand what your service is doing.</li><li>You have alerting to proactively identify performance and stability issues.</li><li>You are aggregating log information to a central point and making it available for all developers.</li><li>You have accessibility testing planned with real users.</li><li>You understand the security classification of the service and its data — and what this means for development, testing and production.</li><li>You have in-sprint security testing or regular checkpoints with the security specialists.</li><li>Security controls for both application and infrastructure required for live operations are present across all environments in the pipeline.</li><li>You are able to deliver the service to a wide range of browsers and devices.</li><li>You know how you will open source your code repositories without including sensitive configuration.</li><li>You are able to test failure of your service and know how it will respond if parts aren’t available.</li><li>You have discussed and agreed the contingency options for the digital service with the customer and where appropriate prioritised work on building contingency up front (particularly important when working with hard deadlines).</li></ol><h4>The Finish</h4><p>Let’s not be complacent when building digital services for citizens and customers. Macro-failure is bad for everyone, let’s work hard to avoid it.</p><p>Thanks to <a href="https://twitter.com/JohnStrudwick">@johnstrudwick</a> who has very kindly edited this post into something much more readable.</p><p>Credit also to the bunch of Kainos architects who are co-authors of the 20 points: <a href="https://medium.com/u/8354b7809951">Rory</a>, <a href="https://medium.com/u/d37ebc7e9ff2">davey.mcglade</a>, <a href="https://medium.com/u/44457f66b58d">Gareth Workman</a> and <a href="https://medium.com/u/77b156708fb2">Caoimhin Graham</a>.</p><p><em>And if you’re interested in working with us at </em><a href="https://twitter.com/KainosSoftware">@KainosSoftware</a> <em>to build great software, we are </em><a href="https://www.kainos.com/careers/professionals/">hiring</a>.</p><blockquote><a href="http://bit.ly/Hackernoon">Hacker Noon</a> is how hackers start their afternoons. We’re a part of the <a href="http://bit.ly/atAMIatAMI">@AMI</a>family. We are now <a href="http://bit.ly/hackernoonsubmission">accepting submissions</a> and happy to <a href="/service/mailto:partners@amipublications.com">discuss advertising &amp;sponsorship</a> opportunities.</blockquote><blockquote>To learn more, <a href="https://goo.gl/4ofytp">read our about page</a>, <a href="http://bit.ly/HackernoonFB">like/message us on Facebook</a>, or simply, <a href="https://goo.gl/k7XYbx">tweet/DM @HackerNoon.</a></blockquote><blockquote>If you enjoyed this story, we recommend reading our <a href="http://bit.ly/hackernoonlatestt">latest tech stories</a> and <a href="https://hackernoon.com/trending">trending tech stories</a>. Until next time, don’t take the realities of the world for granted!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=489013986d50" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hackernoon/non-functional-failure-489013986d50">Non-Functional Failure</a> was originally published in <a href="https://medium.com/hackernoon">HackerNoon.com</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>