BotNets, You Hate’em Right?:
IoT is a broad term that covers everything from baby monitors and security cameras to industrial control systems and your “smart” locks/fridge/toothbrush. These devices are equipped with computer chips and basic networking software that often runs stripped-down (and insecure) forms of the Linux operating system. A “botnet” is a collection of these IoT devices (the “bots”) that have been taken over by an outside party.

On October 12, 2016, a college student from Rutgers initiated a massive DDoS (Distributed Denial of Service) attack that crippled the East Coast. The initial intent was to disrupt some competitors on the online game Minecraft but at its peak over 500,000 internet devices had been compromised and taken over to form a digital army that could shut down or compromise even the largest of computer services, financial institutions or even small countries. This was the Mirai botnet and it was all made possible because IoT (the Internet of Things) is a security disaster.

As naive users of technology we understand that there is such a thing as computer security and we are often aware of major attacks or data-breaches from major companies. Our mistake is that we think that these cyber attacks are the work of elite teams of foreign operatives or genius hacker teenagers.

Sadly, instead of this:

Iot security is SO bad that it is closer to this:

Seriously. The Mirai botnet was so effective because all that it had to do was log into IoT devices using passwords like “123456”, “password” and “default” (here’s a more complete list if you’re curious: https://www.grahamcluley.com/mirai-botnet-password/) and convert them into a bunch of silicon zombies.

OWASP (the Open Web Application Security Project) is the go-to organization for internet security. OWASP has a special ‘Internet of Things (IoT) Project’ which lists these as the ‘Top 10 IoT Issues’:
1: Insecure Web Interface
2: Insufficient Authentication/Authorization
3: Insecure Network Services
4: Lack of Transport Encryption
5: Privacy Concerns
6: Insecure Cloud Interface
7: Insecure Mobile Interface
8: Insufficient Security Configurability
9: Insecure Software/Firmware
10: Poor Physical Security

This is, for the record, is basically EVERY possible security issue that you could ever theoretically have.

WAVE TO THE BABY:

Now you’re thinking, “That’s a pretty serious list of issues, is that the worst of it?”. To answer that let me introduce you to Shodan. Shodan is basically Google, but for IoT devices. What could be worse than millions of small computers spread around the world that are all either using “admin” as a password or no password at all? How about the fact that you can go to a single website and search for them all individually with a nice user interface.

That’s bad, right?

“… Even the devices that do require authentication mostly use default credentials, so you just go on Shodan and you can search for the default password and access them as easily as that.”

-Adrian Hayter (Founder, Shodan)

Now anyone in the world can basically Google the baby-cam in you child’s bedroom and, if it is by default insecure, access it.

'I'm in your baby's room': A hacker took over a baby monitor and broadcast threats, parents say

IT CAN’T GET WORSE, CAN IT?:

At least the problem is known and people are working on it right?

“Attacks from and on IoT will be more severe over the coming year.”

-Kunal Agarwal (Symantec’s general manager for IoT)

More specifically, while DDoS attacks are financially costly and highly inconvenient they are at least rarely life threatening. Future attacks against IoT devices however, will undoubtedly evolve to targeting critical infrastructure. This critical infrastructure, such as Industrial Control Systems, can shut down vital public services and, in extreme cases, possibly trigger explosions or serious equipment damage. The code that runs Industrial Control Systems is notoriously outdated and ill-equipped to handle these attacks and that leaves them highly-susceptible to ransomware, destructive malware and targeted attacks. 40% of industrial sites are now also connected to the internet making these systems defacto IoT devices.

CAN YOU HEAR US NOW?
One final indirect way that things are about to get a whole lot worse is the widespread adoption of 5G which will greatly increase data speeds. Increased data speed means increase attack speed and that means that security defenders will often have mere seconds to identify and mitigate a new DDoS attack. This will be a serious threat to organizations and countries around the world and could lead to attacks that will dwarf the scale of anything that we have seen to date.

SOLUTIONS- Manufacturer Compliance:
New IoT devices are released every day and often contain both known and as-of-yet undiscovered vulnerabilities. Manufacturers must adopt universal IoT security standards or they will continue creating poorly secured devices. Techniques such as Security-by-Design (SbD) could be used to solve most of these security short-comings. If vendors fail to self-regulate then governments may have to step in.

The Top 5 Areas For Manufacturer Improvements:
1. Complex, non-Hard-Coded Passwords
2. Secure Hardware
3. Secure Update Mechanisms
4. Methods For Patching Older Systems and Software
5. Secure Data Storage & Transfer

“ALTERNATIVE” SOLUTIONS :
One of the more fascinating “alternative” solutions is in some ways the most logical. If the root of the problem is that devices are insecure and easily taken over, then we should just take them over. In 2017 a hacker named Janit0r did just that with the creation of BrickerBot. Much like a farmer might need to quarantine and kill livestock that were infected with a deadly disease in order to save the rest of the herd, BrickerBot takes over the insecure device and promptly destroys them (i.e. “bricks” them) by executing a chain of commands that causes them to be permanently disabled. These commands include corrupting the device’s storage, blocking the internet connection, tampering with device performance, and erasing all of the files.
Recently another piece of software called Silex has been discovered doing similar things to IoT devices but the motivation of the hacker that released it is lesser known.

CONCLUSION:
The Mirai attack was the first major example of IoT devices being weaponized as an army of bots but it is only the beginning. Internet connectivity is being added daily to things that we would never have previously guessed needed their own computer chips. For the moment the size, speed and variety of these insecure IoT devices is dwarfing the effort to securely contain them. This will have to change in the coming years but the path forward is not clear. Eventually Universal IoT standards will need to be enacted by governments as well as trade groups and International organizations. We will see if this is able to happen proactively or if it will be after we fall victims to a truly disastrous cyber attack.

EVEN MORE AMAZING REAL-LIFE IoT FAILURES!:

A Real “Oceans 11” Heist

CyberSecurity firm Darktrace announced that one of its’ casino clients had 10 GB of data that were hacked and sent to a device in Finland. The attack vector, the internet-connected aquarium thermometer in the lobby.

Unsafe Sex

Tech firm SEC Consult announced that an unsecured sex toy released a database of over 50,000 customers. ‘The database containing all the customer data (explicit images, chat logs, sexual orientation, email addresses, passwords in clear text, etc.) was basically readable for everyone on the internet.’ This highlights both the data protection issues with IoT as well as the potential privacy issues.

Read This Hilarious, and frightening, take on the hack: ‘Panty Buster’ sex toys can be hacked to ‘remotely pleasure people without their consent’, researchers claim:

https://metro.co.uk/2018/02/01/panty-buster-sex-toys-can-hacked-remotely-pleasure-people-without-consent-researchers-claim-7279177

Open Sesame!

When Lesley Carhart (Full Spectrum Cyber-Warrior Princess) vented on Twitter about her apartment building forcing a switch to “key-less” electronic locks there were many people that thought she was over-reacting. Soon after a Dallas hackers group demonstrated how easy it was to gain remote control of the locks in question and could basically open & close the locks in Lesley’s building at will (this hacker group pro-actively shared all of this data with the lock company and they were able to fix that specific vulnerability).

Security Things to Consider When Your Apartment Goes ‘Smart’

https://tisiphone.net/2019/01/28/security-things-to-consider-when-your-apartment-goes-smart/

Baby It’s Cold Inside

In 2016, the Valtia Group (apartment managers in Lappeenranta, Finland) disclosed that two of their apartments had their “smart” thermostats hacked which left residents in the freezing cold for almost a week.

Hackers leave Finnish residents cold after DDoS attack knocks out heating systems (https://www.ibtimes.co.uk/hackers-leave-finnish-residents-cold-after-ddos-attack-knocks-out-heating-systems-1590639)

Hack My Heart

In 2017, the US Food and Drug Administration (FDA) confirmed that St. Jude Medical’s implantable heart monitors were vulnerable to hacking. If someone was able to gain control of a device they could possibly deplete the battery or administer incorrect pacing or shocks.

FDA confirms that St. Jude’s cardiac devices can be hacked:

https://money.cnn.com/2017/01/09/technology/fda-st-jude-cardiac-hack/

Worse Than ‘Elf-On-The-Shelf’

“Concealed surveillance device” Is not something that should ever be associated with a children’s toy but in 2017 Germany was forced to ban the ‘My Friend Cayla’ doll because an insecure Bluetooth device installed in the toy was an opening for hackers to use its cameras and microphones to see and hear whatever Cayla saw and heard including the childs name, parents’ names, what schools they go to, and where they live.

This pretty blond doll could be spying on your family:

https://www.washingtonpost.com/news/worldviews/wp/2017/02/23/this-pretty-blond-doll-could-be-spying-on-your-family/

Backseat Driver

In a now infamous conference presentation at the 2015 Blackhat Security Conference held in Las Vegas, a team of researchers were able to take total control of a Jeep SUV and make it speed up, slow down, and veer off the road.

Black Hat USA 2015: The full story of how that Jeep was hacked

FURTHER READING:

Fun with Shodan and IOT
https://www.kiledjian.com/main/2018/7/23/fun-with-shodan-with-iot

OWASP Internet of Things Project
https://www.owasp.org/index.php/OWASP_Internet_of_Things_Project

What is a botnet? When armies of infected IoT devices attack
https://www.csoonline.com/article/3240364/what-is-a-botnet.html

In his 2018 keynote address Dr. James Mickens (Professor of Computer Science at Harvard) warned that, “You should never let things that you do not understand do important things”. Specifically he was talking about Artificial Intelligence (AI) and Machine Learning (ML). Despite the fascinating academic prospects for the often interchangeable AI & ML, the jump to apply them to real world problems becomes in itself a huge problem when the scientists that are using these technologies don’t actually, really know how they work. It becomes an even more tangible problem when they fail (and they fail A LOT) but scientists can’t always figure out why or how to prevent failures in the future.

This would be, and should be, primarily an academic problem. Something that frazzled grad students wrestle with until late into the night. Instead scientist and tech companies have often jumped right past the “Does this actually work”-phase and have moved directly to the “Can we make money on this”-phase.

Dr. Mickens reduces this problem to 3 basic steps:

1. Technology is not value neutral

2. We often move faster than our ability to understand technology

3. We often do not learn basic lessons for when these conditions have failed catastrophically in the past.

But first, let’s take a quick step back and get an overview of what we are taking about.

What is a Square?

I know what a square is. You know what a square is. Does a computer know what a square is? [Spoiler Altert: It Does Not]

But what if I wanted to teach the computer what a “Square” is? I might start with the image below.

I can start by providing the computer with some basic formulas for a square (4 equal sides, 90-degree corners, etc.) but that’s just math. If I really want the computer to be able to look at a picture and “understand” that the object is a Square then I’m going to have to do some more work.

Step 1- Since figure A provides the clearest idea of what a square is we can start by translating the black lines of the square into numbers that can be understood by the computer (on a scale where 0 represents pure white/no color and 255 represents pure black/maximum color).

Next, to make it even more understandable by the computer we would break these numbers down line-by-line and translate this to an array:

[[255, 255, 255, 255, 255, 255, 255, 255, 255, 255], [255, 0, 0, 0, 0, 0, 0, 0, 0, 255], [255, 0, 0, 0, 0, 0, 0, 0, 0, 255], [255, 0, 0, 0, 0, 0, 0, 0, 0, 255], [255, 0, 0, 0, 0, 0, 0, 0, 0, 255], [255, 0, 0, 0, 0, 0, 0, 0, 0, 255], [255, 0, 0, 0, 0, 0, 0, 0, 0, 255], [255, 0, 0, 0, 0, 0, 0, 0, 0, 255], [255, 0, 0, 0, 0, 0, 0, 0, 0, 255], [255, 0, 0, 0, 0, 0, 0, 0, 0, 255], [255, 255, 255, 255, 255, 255, 255, 255, 255, 255]]

Now the computer knows that if it sees an array with numbers like these then it is looking at a square. In real life however, there are an almost infinite examples of squares and very few of them are as clear to recognize as this simple black box.

Step 2- We can use this logic on Figure B but the results are much less clear.

The resulting array:

[[100, 190, 190, 190, 190, 190, 190, 190, 190, 100], [190, 210, 190, 190, 190, 190, 190, 190, 210, 190], [190, 190, 0, 0, 0, 0, 0, 0, 190, 190], [190, 190, 0, 0, 0, 0, 0, 0, 190, 190], [190, 190, 0, 0, 0, 0, 0, 0, 190, 190], [190, 190, 0, 0, 0, 0, 0, 0, 190, 190], [190, 190, 0, 0, 0, 0, 0, 0, 190, 190], [190, 190, 0, 0, 0, 0, 0, 0, 190, 190], [190, 190, 0, 0, 0, 0, 0, 0, 190, 190], [190, 210, 190, 190, 190, 190, 190, 190, 210, 190], [100, 190, 190, 190, 190, 190, 190, 190, 190, 100]]

Step 3+: Now what happens if we show the computer this picture?

The short answer is, with AI we are able to show the computer so many pictures of squares (literally thousands, and possibly millions, of pictures) that it will be able to start recognizing the minute similarities in all of the examples that you have shown it and be able to do things like pick squares out of a picture of the Roman Pantheon. Show it even more pictures of the Roman Pantheon and it will reliably start being able to recognize and label pictures of that landmark with no other reference data available. This is called “Training The Model” and, just like training in real life, if you train incorrectly you will not always get the results that you are looking for since the farther that we move away from a direct representational model of the square (our Figure A above), the more interpretation is required by the computer and, if not done correctly, the more potential for error that is exposed.

Even worse, if researchers use “Black Box” algorithms, i.e. algorithms that lack clear, causal connections between what goes in and what comes out, then it is often difficult to detect and fix these models when they go wrong.

“Blackboxes” & “Adversarial Examples”:

“Blackboxes” refers to algorithms that are highly complicated and non-linear so that it is impossible, sometimes deliberately so, to understand how the algorithms came to the conclusions that they did. When the algorithm gives you incorrect results, it is therefore often impossible to tell exactly why or how to fix it so that you get better results in the future. Despite this, Blackboxes are often preferred by companies with AI-based products to sell because it makes their tools more unique and proprietary. This issue is expounded upon by Cynthia Rudin (Director, Prediction Analysis Lab, Duke University) and Joanna Radin (Associate Professor, History of Medicine, Yale Medical School), “Over the last few years, the advances in deep learning for computer vision have led to a widespread belief that the most accurate models for any given data science problem must be inherently uninterpretable and complicated. This belief stems from the historical use of machine learning in society: its modern techniques were born and bred for low-stakes decisions such as online advertising and web search where individual decisions do not deeply affect human lives. In machine learning, these black box models are created directly from data by an algorithm, meaning that humans, even those who design them, cannot understand how variables are being combined to make predictions. Even if one has a list of the input variables, black box predictive models can be such complicated functions of the variables that no human can understand how the variables are jointly related to each other to reach a final prediction.”(1)

Adversarial Examples are pictures and data that provide tangible examples of why Black Box models are so problematic. They are concrete demonstrations of where and how AI/ML fails in real world applications but unfortunately, because the lack of transparency provided by Black Boxes, do not clearly show what is necessary to definitively fix them.

In 2019 an MIT research team published a fascinating paper that showed how easy it was to fool AI/ML into mistaking a cat for a bowl of guacamole. (2) Anish Athalye, PhD student at the Massachusetts Institute of Technology(MIT) and lead author of the MIT report has stated,“The Machine Learning community doesn’t fully understand what’s going on with adversarial examples or why they exist. This is an open problem. There have been many proposed techniques, and almost all of them are broken”. He goes on to warn, “More and more real-world systems are starting to incorporate neural networks, and it’s a big concern that it would be possible to subvert these systems or attack using adversarial examples” (3)

When this system works it can be astounding. When it doesn’t work in an academic setting, it is often actually quite humorous. When it doesn’t work in real world setting peoples lives and well being can be put at risk.

Implications of Bias:

All of this would be interesting academically if the implications of these issues were not already being felt in the real world. Many people’s fears of AI/ML are often along the lines future Terminator/Matrix dystopias. While we wait for the arrival of our future robot overlords however, there are unfortunately a wide range of ways in which AI/ML are currently negatively impacting our daily lives.

Two of the most troubling areas of bias are criminal and financial because they have the possibility to touch and affect individuals lives in such an intimate and powerful way.

“You should never let things that you do not understand do important things” -Dr. James Mickens

Criminal Bias:

The use of AI/ML for “Risk Assessments” in the criminal justice system is uniquely troubling since it is often using technology to codify existing social bias and overt racism. This is highlighted by the MIT Tech Review comment that, “The technology is a formidable way to invade people’s privacy, and biases in training data make it likely to automate discrimination.”(7) . In 2016 ProPublica ran an expose highlighting racial disparities of computer generated risk assessments by software (most often a program called COMPAS by the Northpointe company) that is used in a growing number of states including Arizona, Colorado, Delaware, Florida, Kentucky, Louisiana, New York, Oklahoma, Virginia, Washington and Wisconsin. Specifically ProPublica reported that, “… many jurisdictions have adopted Northpointe’s software before rigorously testing whether it works” and “In forecasting who would re-offend, the algorithm made mistakes with black and white defendants at roughly the same rate but in very different ways.

• The formula was particularly likely to falsely flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants.
• White defendants were mislabeled as low risk more often than black defendants.

Could this disparity be explained by defendants’ prior crimes or the type of crimes they were arrested for? No. We ran a statistical test that isolated the effect of race from criminal history and recidivism, as well as from defendants’ age and gender. Black defendants were still 77 percent more likely to be pegged as at higher risk of committing a future violent crime and 45 percent more likely to be predicted to commit a future crime of any kind. (Read our analysis: https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm)”(4)

Financial Bias:

Opaque and proprietary algorithms are also being used more and more in the financial industry to regulate such things as loan approvals and credit ratings. The impact of these biases were recently highlighted when a Tech Industry leader (David Heinemeier Hansson, creator of Ruby on Rails) called out Apple/Goldman Sachs for the sexist approval practices on its credit card. As outlined by Gabriela Barkh of The Observer, “Since news of the Apple Card’s gender discrimination broke last week, Goldman Sachs has been in damage control mode. The financial institute, which partnered with Apple to issue the newly-launched credit card, insists its approval process doesn’t favor male applicants, as users have reported. It’s safe to say the public isn’t buying Goldman’s claim of innocence; even Apple co-founder Steve Wozniak noted his wife’s considerably lower Apple Card spending limit. “What Goldman Sachs failed to take into account is that machine learning algorithms excel at finding latent features in data,” explained Lux Research analyst Cole McCollum. “These are the features that aren’t directly used in training a machine learning model, but are inferred from other features that are.” McCollum told Observer that this type of bias that’s found in historical datasets is one of the biggest challenges in implementing machine learning. It’s also why companies need to explore bias detection and artificial intelligence (AI) explainability tools that can help alleviate potential discrimination. “In the case of the Apple Card’s spending limits, even if gender was not specifically considered (as Goldman Sachs claims), other related features in the dataset can still embed those biases and lead to unfair decisions,” he said.”(5)

Additional Problematic Examples:

- “Adding stickers to a stop sign can make a self-driving car believe the sign is posting a 45-mile-per-hour speed limit, for example, while adding them to a road can make a Tesla swerve into the oncoming traffic lane” (6)

- “Self-crashing Cars: After a fatal accident involving one of Uber’s self-driving cars in March, investigators found that the company’s technology had failed catastrophically, in a way that could easily have been prevented”(7)

- That’s Funny, You Don’t Look Gay: In 2017 two graduate students from Stanford put together a research project to have AI face recognition distinguish between gay and heterosexual individuals using online dating profiles and pictures. The experiment was limited but claimed up to a 91% success rate with men and up to a 71% success rate with women. This seems exceptionally dangerous given that in many countries non-traditional orientation is illegal and is punishable by death in eight countries including Iran, Mauritania, Saudi Arabia, and Yemen (UN Human Rights Council, 2015).(8)

What Should We Do Now?

Artificial Intelligence and Machine Learning have enormous potential for changing the world but, as a wise man once said, “With great power comes great responsibility” (9).

Based on Dr. Mikens original 3 basic issues about AI/ML, here are some general suggestions for how to most responsibly use AI/ML:

1. Technology is not value neutral

Therefore, we need a better understanding of implicit bias. Technology is not neutral because the people who build it are not neutral and often the datasets used to train the algorithm are not neutral. We must understand and test for all of the ways that the algorithm is getting it wrong.

2. We often move faster than our ability to understand technology

Therefore, we need MORE TESTING!!! AI/ML is a fascinating, powerful technology that does some things very well. Let’s figure out what those things are first, before we put them in charge of peoples lives.

3. We often do not learn basic lessons for when these conditions have failed catastrophically in the past.

Therefore, we need to have less reliance on “Black Box” vs interpretable algorithms. If you don’t know, for sure, why you are getting the answers that you are getting then it is far harder to understand and correct for incorrect results. Let’s situate ourselves to make the most informed and educated decisions as is possible.

REFERENCES:

(1) Cynthia Rudin, Director, Prediction Analysis Lab, Duke University, and Joanna Radin, Associate Professor, History of Medicine, Yale Medical School, “Why Are We Using Black Box Models in AI When We Don’t Need To? A Lesson From An Explainable AI Competition”, Harvard Data Science Review, November 1, 2019

(2) Anish Athalye, Logan Engstrom, Andrew Ilyas, Kevin Kwok, “Synthesizing Robust Adversarial Examples”, Cornell University, Computer Vision and Pattern Recognition (cs.CV), Submitted on 24 Jul 2017 (v1), last revised 7 Jun 2018 (this version, v3))

(3) “Computers can be fooled into thinking a picture of a taxi is a dog just by changing one pixel, suggests research”, BBC, November 3, 2017

(4) Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner, “Machine Bias: There’s software used across the country to predict future criminals and it’s biased against blacks.”, ProPublica, May 23, 2016

(5) Gabriela Barkh, “How Goldman Sachs Can Regain User Trust After Apple Card Discrimination”, The Observer, November 16, 2019

(6) Karen Hao, “A New Set of Images that fool AI could help make it more hacker-proof”, TechnologyReview.com, June 21, 2019

(7) Will Knight and Karen Hao, “Never mind killer robots- here are six real AI dangers to watch out for in 2019”, MIT Technology Review, January 7, 2019

(8) Yilun Wang, Michal Kosinski, “Deep Neural Networks can detect sexual orientation from faces”, (PREPRINT) Journal Of Personality and Social Psychology, July 2, 2018

(9) Spiderman