We are on the precipice of another government shutdown if the United States Congress fails to act by September 30. This would be the 22nd U.S. government shutdown since 1976, with the most recent shutdown lasting 35 days from December 22, 2018, to January 25, 2019 — by far the longest government shutdown in U.S. history. Nothing is preventing another event of this magnitude — or worse — but with the growing turmoil between and within the Republican and Democratic parties, the likelihood increases by the day.

This internal and cross-party squabbling is distracting our Members of Congress, preventing them from effectively governing and creating significant security risks by opening the door for insider threats.

Financial (In)security of Federal Workers

The salaries of approximately 4.3 million federal workers, including 2.2 million civilians, 2.1 million military and reservists, and a substantial number of government contractors, are directly affected by a government shutdown.

While for many people, headlines shouting about an impending government shutdown results in nothing more than a roll of the eyes and an off-color remark, it’s essential to recognize that millions of federal civilian, military, and contractor personnel are genuinely concerned about how this shutdown will affect their finances and their families’ well-being.

A recent survey reveals that more than 61% of U.S. adults face financial stress as they live paycheck to paycheck. Poverty is also rising in the U.S., with the largest one-year increase on record happening from 2021 to 2022. Federal workers are not immune from these realities.

During the 2018–19 shutdown, 800,000 federal employees and 4 million federal contractors were furloughed or compelled to work without pay. Although federal employees eventually received back pay, the Brookings Institute stated that “more than 60% of affected federal workers reported exhausting their savings.”

Some may remember the story of Lila Johnson (pictured above), a federal contractor employed as a custodial worker for the Department of Agriculture who came forward to tell her story during the 2019 shutdown. Federal contractors aren’t paid for time they aren’t working, and Lila, like many others, was already working a second job to help pay the bills. The loss of her income during the shutdown made it impossible to meet ends.

Cleared and Present Danger

Government shutdowns not only cause stress for federal workers as they grapple with how they will meet their families’ needs (like paying rent, buying food, paying medical bills) but also fuel feelings of anger, resentment, and frustration. Both military and government civilians are actively deployed when the shutdowns happen. It’s hard for them to concentrate on their duties when they get letters from home about not being able to feed kids or make ends meet. This places a target on them.

Here’s how.

At last check, at least 4.7 million people with security clearances work in civilian, military, and contractor roles supporting the federal government. Of these:

• 1.25 million individuals hold a Top Secret clearance.
• 2.2 million individuals have an active security clearance AND currently using it to access classified information for their job.
• 2.5 million individuals have a security clearance but are not currently using it to access classified information, due to retirement, unemployment, or employed in a non-cleared role.

To obtain a security clearance, it is necessary to undergo a background check that includes an assessment of the candidates’ financial situation. Why?

“Failure to live within one’s means, satisfy debts, and meet financial obligations may indicate poor self-control, lack of judgment, or unwillingness to abide by rules and regulations, all of which can raise questions about an individual’s reliability, trustworthiness, and ability to protect classified or sensitive information. . . . An individual who is financially overextended is at greater risk of having to engage in illegal or otherwise questionable acts to generate funds. Affluence that cannot be explained by known sources of income is also a security concern insofar as it may result from criminal activity, including espionage.” — National Security Adjudicative Guidelines

When the federal government shuts down, federal workers with security clearances are forced into economic insecurity — by the government they support.

This, as clearly articulated by NSA guidelines, makes these workers quickly become targets for those that wish to obtain protected information.

To be or Not to Be…an Insider Threat

“Not all spies seek to be spies. Some are recruited; others divulge information unknowingly…[some are] vulnerable to blackmail, extortion, or being coerced into providing confidential proprietary information.” — Verizon Insider Threat Report 2019

What is an insider threat? Insider threat is the threat that an insider will use their knowledge or authorized access, wittingly or unwittingly, to do harm to an organization or agency’s mission, resources, personnel, facilities, information, equipment, networks, or systems.

There are four common motivators that nations, businesses, or organizations leverage to recruit and develop an insider threat.

1. Money: Whether financial gain intended to enable someone to live a lavish lifestyle or financial hardship and the money is meant to fill a critical gap, money is the primary motivator for why people become insider threats.
2. Ideology: People whose beliefs (political, economic, religious, or otherwise) align with that of the recruiting entity are often the most concerning to intelligence agencies.
3. Coercion or Compromise: This includes an “act that influences a person to take an action that may or may not be in their best interest,” such as blackmail or threats of injury or death.
4. Ego or Excitement: People with high self-importance or who feel they are not fully appreciated (like Edward Snowden) may be targets as agents attempt to stoke their egos and make them feel important, while others may be chasing excitement and are motivated by the adrenaline rush of illicit activity.

Money is a powerful motivator. Some of the most well-known cases of insiders turned spies resulted from the incentive of financial gain:

• Robert Hanssen was motivated by money. His personal finances were in the worst shape of his professional life.
• Ron Rockwell Hansen worked for the DIA and received over $800k to be a spy.
• Aldrich Ames was motivated by money for luxury items.
• John Anthony Walker had debts which compelled him to spy for the Soviet Union for anywhere between $350k-$1 million.
• Most recently, two US Navy servicemen were paid to pass information to China. Both had security clearances and access to sensitive defense information.

These are the names and events that were allowed to be declassified and made public. There are undoubtedly cases that haven’t.

Anyone can be a target, but especially when they encounter financial hardship.

Imagine you are a federal worker with a security clearance. You have access to sensitive information (nothing that seems really important and Top Secret) and you are furloughed. You have to stop working and will not receive pay during the time you are furloughed. Your health insurance benefits will remain active while you are furloughed and the employer portion of the premium is paid. You have the option of paying your portion of the premium or delaying payment (which accumulates) until your paid employment resumes. You have 1 month of savings that will cover your usual costs — but not anything additional. Your son is diagnosed with a serious illness. His medical bills will be extensive. You have not been paid in 3 weeks and do not know when your next paycheck will occur.

Someone contacts you with a solution to this problem. All you have to do is answer a question for them. No big deal.

What do you do?

Congressional Inaction Has Consequences

It’s time for the Members of Congress who are playing petty personal and party politics to stop. Their actions are harmful in so many ways, but are without question putting targets on millions of people and jeopardizing our national security.

November is a moment of reflection. For me, it’s reflecting on the question of how can we understand the barriers between awareness and implementation?

To answer this, it’s imperative to understand the organization, how it functions, the people inside of it, and the tech that runs it.

This week, I decided to share my thoughts elsewhere about my take on integrating security practices and controls — and with it a big question: is awareness the problem, or is it accessibility?

I also pose this question: What happens if organizations can’t afford cyber insurance because the minimum requirements cost a premium? We may make the market thrive, but are we meeting the objective?

A few days ago, Senator Mark Warner spoke on Policy Report that was released, stating,

“Industry experts have shared that cyber insurance is increasingly expensive and that the application process is extensive and sometimes burdensome. A thriving market for cyber insurance could lower overall risks by introducing minimum cyber hygiene requirements.”

Can we afford “accessible” security? Tell me what you think: https://buff.ly/3UQENbS

Shadow-IT — Deployed information technology systems or services by employees or non-IT departments, to work around the shortcomings of the central information systems; actual or perceived.

Example: The marketing team decides to use Third-party Software-as-a-Service (SaaS) applications outside the control of their IT department

Scamicry — A “legitimate” interface or experience that confuses people, breaks mental models or imitates scams (scam mimicry)

Example: A bank contacting a customer requesting they share their password. Common consumer anti-virus pop ups invoke fear unless you purchase them

Insanity — Has been described as the act of performing the same thing over and over again and expecting different results

Example: A user hitting CTL + ALT + DEL repeatedly on a frozen computer

Insan-IT — A faulty IT process defined by statements or policy put in place to achieve a business outcome for the user, but which requires the user to circumvent the process to achieve their goal. An inefficient system that is never fixed, but still recognized as the policy, practice, or process even though the system owner does not support it.

Example: To comply with policy, a user must submit a request through the IT ticketing system. After several weeks without a response, someone in IT directs the user to email an administrator directly as a non-documented standard practice. Without fixing the workflow employees will be left to perform the same function over and over again expecting the results to be different.

Example 2: IT implements a password policy expecting users to generate a different, complex password on a regular short term basis without offering single sign-on. IT then wonders why so many credential instances of <company><year> appear to be used by staff, but refuses to acknowledge their role in generating a solution.

I have long been watching discussions around IT and security Policy, Process, and Procedure and seeing the violent melting pot of opinions on root cause and fixes.

All too often the user is defined as the sole source of problems and at the same time the solution:

We the BUSINESS write the policies that the USERS are accountable for following.

We the BUSINESS create agility and decrease staff by ‘shifting left’ the processes for the USERS to be more self-serving.

We the BUSINESS provide security awareness training once a year so the USERS can fight “social engineering” attacks as the first last and only line of defense for themselves and the organization.

We the BUSINESS define password complexity requirements so USERS can create lots of unique passwords for all the services they interact with.

We the BUSINESS provide tools that we think the USERS need, without offering a feedback loop to understand their individual or departmental needs and requirements (collaboration, integration, automation, etc.)

The list stretches on for as long as there are days.

We the BUSINESS are the security and IT practitioners that are supposed to be translating best practices into meaningful stories for USERS to successfully use. And what does failure look like?

• Shadow-IT
• “P@ssword2022!!”
• Frustration
• Tech debt
• Confusion
• Outdated documentation
• Slow, grinding wheels of the BUSINESS
• Attrition
• Budget spent without ROI
• Increased Risk to compromise

Do these sound familiar? How do we fix it?

Unfortunately, “Zero-Trust” is the Zero answer. IT and Security need to step back and work together to “Stop the Insan-IT”

That means, step back and think smarter, not harder:

• Are your business Policies, Processes, and Procedures actually doing what you intended?
• What are the supporting metrics for your PPP?
• Where is single sign-on in the plan?
• Where is password-less authentication and hardware tokens?
• Where is the support for the Helpdesk and SOC to build, fix and manage the workflows for your business process management?
• Where is the button to report a suspect email that has gotten through the perimeter defenses? (also, where are the perimeter defenses?)

Understand the impacts your business processes have in the speed of your business, the barriers to entry, and the desire for your users to circumvent and figure out their own solutions. If you can’t understand why USERS aren’t doing the needful, you may be contributing to the Insan-IT.

Looking back 5 years, the FDA marked National Cyber Security Awareness Month with a statement [1] (available on third party site) I feel is important to reflect on:

“At FDA, we strongly believe that medical device cyber safety is a large and shared responsibility that requires diligence from all stakeholders, including: medical device manufacturers, government agencies, health care organizations, health care professionals, cybersecurity researchers, and medical device users. To that effect, the FDA has partnered with several public and private sector organizations including the Department of Health and Human Services, the National Health Information Sharing and Analysis Center (NH-ISAC), and the Medical Device Innovation, Safety, and Security Consortium (MDISS), and the Department of Homeland Security’s Industrial Control Systems Cyber Emergency Response Team (ICS-CERT). These partnerships serve to raise awareness of medical device safety and cybersecurity in healthcare and public health, and also to foster collaboration, information sharing, and coordinated vulnerability disclosure policies and practices.”

The FDA’s statement doesn’t preclude legacy devices. It doesn’t focus on future (pre-market) devices. It also doesn’t say “shift the liability,” or “find someone else to be accountable.” The statements of shared responsibility from US regulatory bodies and mirrored by international regulatory forums [2] are non-discriminatory and show that each organization and entity has a role to play in the safety and security of the eco-system.

Based on historical data, its easy to see that many healthcare organizations presently struggle to hire, retain, and support IT security professionals, let alone afford to implement the necessary tools and leadership practices to protect their organizations from current and future threats such as ransomware.

The increase of ransomware attacks on hospitals, the impacts that this has had on the National Critical Function to “Provide Medical Care” [3], and now the first patient death and lawsuit [4] as a result, it’s important we understand the risk and responsibility involved.

Do not wait to incorporate lasting security into the medical devices you manufacture and sell.

Do not wait for international regulatory bodies to tell you what to do.

Do not wait until your Medicare funding is on the line to understand the security of your medical devices. [5]

Do not wait until after the ransom or the death to improve your security practices

Do not wait until next October to think about why digital security of your organization or your medical devices is critical.

Your adversary will not wait for you.

[1] https://www.medwrench.com/article/47300/october-is-national-cybersecurity-awareness-month-the-importance-of-cybersecurity-and-medical-devices

[2] http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-200318-pp-mdc-n60.pdf

[3] https://twitter.com/cisagov/status/1444023843400691712

[4] https://www.wsj.com/articles/ransomware-hackers-hospital-first-alleged-death-11633008116

[5] https://oig.hhs.gov/oei/reports/OEI-01-20-00220.pdf

There was much fanfare that Moody’s put another “triple-A stamp of approval” on Security Ratings firm BitSight to the tune of $250 million. While this is a substantial, newsworthy investment, it isn’t “new” though. The direct correlation between security ratings organizations and Moody’s is evident, and, a point of concern for many, including this author two years ago.

That said, could we see another Crisis Inquiry Report but this time for Security Ratings with statements like this from 2011:

“You will also read about the forces at work behind the breakdowns at Moody’s, including the flawed computer models, the pressure from financial firms that paid for the ratings, the relentless drive for market share, the lack of resources to do the job despite record profits, and the absence of meaningful public oversight.”

“The captains of finance and the public stewards of our financial system ignored warnings and failed to question, understand, and manage evolving risks within a system essential to the well-being of the American public. Theirs was a big miss, not a stumble. While the business cycle cannot be repealed, a crisis of this magnitude need not have occurred. To paraphrase Shakespeare, the fault lies not in the stars, but in us.”

“Panic fanned by a lack of transparency of the balance sheets of major financial institutions, coupled with a tangle of interconnections among institutions perceived to be “too big to fail,” caused the credit markets to seize up. Trading ground to a halt. The stock market plummeted. The economy plunged into a deep recession.”

Inertia and comfort with the known (even when known to be wrong) are powerful drivers. But the definition of insanity (and certainty for disappointment) is to continue to repeat the same behaviors while expecting a different result.

On top of being funded by the numerous customers they publicly champion, security ratings organizations are being bought or infused with tens if not hundreds of millions of dollars, and yet what they offer still is considered laughable by most industry leaders, even though they still buy their services.

GET YOUR STUFF OFF SEARCH

Across Industry, we all have the power to control the narrative and what attackers including the security ratings vendors, see:

Get your Stuff Off Search and prevent organizations like these from selling a more than likely inaccurate perception of your security posture, and with it the negative statements that can hurt you and your customers.

Security Ratings Vendors: ‘If my statements upset you it’s because I’m doing my job. You have yet to do yours.’ For years I’ve asked you to join me and my industry colleagues in making this industry better. My offer to you, and folks like Mr. Venables, the co-creator and champion of security ratings, still stands.

Let’s talk.

Maybe this doesn’t feel new. The stories have been playing on the news more and more and the impacts to hospitals, especially during a pandemic, continue to impact community hospitals.

“So it used to be that if you had a ransom or an outage or some power problem, you would merely divert ambulances to the next nearby facility and that’s kind of predicated on the next nearby facility being able to receive anybody. So when everyone’s at a saturated level…a failure in any single hospital tends to have cascading stressors or failovers in nearby facilities.”[2]

While ransomware specifically was only starting to impact hospitals in 2017, the medical community was already studying and publishing how disruptions and stressors impacted mortality:

“Large marathons frequently involve widespread road closures and infrastructure disruptions, which may create delays in emergency care for nonparticipants with acute medical conditions who live in proximity to marathon routes. Medicare beneficiaries who were admitted to marathon-affected hospitals with acute myocardial infarction or cardiac arrest on marathon dates had longer ambulance transport times before noon (4.4 minutes longer) and higher 30-day mortality than beneficiaries who were hospitalized on non marathon dates.” [3]

“The cdc tracks something really important called “excess deaths”, and this is the difference between expected deaths and actual deaths. When the US hit that February milestone of 500,000 lost Americans to covid, we also hit a different milestone of 150,000 lost Americans to non-covid conditions that are otherwise treatable — very treatable. The number one age demographic of that was 25 to 44 year olds so young folks that could have been saved but for excessive loads on our healthcare delivery across the country.” [2]

Stop for a moment and let that number sink in. 150,000

But that was just 2020… right? 2021 is totally different… RIGHT?

“Memorial Health System is a nonprofit organization, which makes it an even more attractive target for cybercriminals because nonprofits are often viewed as having lower defensive maturity and limited cybersecurity expertise,” — @stephanchenette

So while this is a reasonable statement, we can’t look at a non-profit organization’s status as the reason why they may lack security. Hospitals choose what resources they will invest in based on needs and requirements.

That said, In 2017 a federal task force estimated that 85% of hospitals did not have a single IT security person on staff.[5] Starting in 2020, with hospitals having to turn away their revenue generating elective procedures because of the pandemic, many were forced to lay off and furlow IT and IT Security staff, so it’s almost certain that the percentage of hospitals without IT security staff is now higher.

Hospitals can’t afford the staff with experience they need, and can’t seem to keep pace with the speed of cyber, regulatory, or the attackers, and yet they also can’t afford not to. Meanwhile our attackers’ cup (and popcorn bowl) runneth over.

The number of stories showing the impact of ransomware on communities and health providers is only growing.

Where do we go from here?

Hope — that this will all fix itself, that our community is not like those others, that we have skirted adversaries impacting our medical devices, hospitals, and our networks for years so why would they start now, or that 4.4 minutes is perfectly adequate time to wait, is not a good plan.

It takes a community of security folks to work the problem and get involved. Check out and get involved with Biohacking Village[5], and I Am The Cavalry [6] if you haven’t already. Please take time to understand the crossroads of biosecurity and the tangential impacts to hospitals and communities and be a part of the solution.

[1] https://arstechnica.com/gadgets/2021/08/hospitals-hamstrung-by-ransomware-are-turning-away-patients

[2] https://www.youtube.com/watch?v=phtBJciRz54

[3] https://www.nejm.org/doi/full/10.1056/nejmsa1614073

[4] https://www.youtube.com/watch?v=WqD-ATqw3js

[5] https://www.villageb.io/

[6] https://iamthecavalry.org/

#biohacking #biohackingvillage #iamthecavalry #security101 #defcon #hospitalsecurity #healthcareCybersecurity #Security101 #CyberSecurity #securinghealthcare #informationsecurity #privacy #ransomware #covid19

Prologue

Shortly after seeing this story on Security Ratings published, attempts to reach Mr. Phil Venables were made to comment and offer assistance. Unfortunately, after a reasonable amount of time passed, it was determined that Mr. Venables did not wish to be reached for comment. I’m now releasing a response to the statements made by Mr. Venables with reasonable public disclosure.

Mr. Venables’s B.L.U.F.

In his post, Mr Venables asserts that Security Ratings made in the way that industry vendors deliver today is something we need, but not what we need them to be, and we should join forces together to improve them.

My view and that of others who understand third party risk, however, is quite the opposite.

Nowhere in Maslow’s hierarchy of needs (or even the Hierarchy of Compliance/Ethics Program Needs) can we find Security/Risk Ratings alluded to, let alone required. When you analyze and attempt to understand the reports from the rating companies you find that the results are biased, misrepresentative, untruthful, or simply provide bad advice to those that choose to buy their data. This recent report provides a perfect case in point. True security measurement can not be found using the methods the industry employs today, but requires a completely different approach.

Since 2017 when these Principles for Fair and Accurate Security Ratings were published, has anyone seen them improved, amended or modified? In that time, ratings vendors have repeatedly tried to soothe the public with a vision of the future as a magical place where all things are possible, when in actuality they have proven themselves as proclaimers of “Fair and Accurate Organizational Security Ratings Here (available tomorrow).” While this industry may not want “perfection to get in the way of good,” how can it expect to be taken seriously if there is no actual effort shown on their part towards being better?

The Missing How

As is noted in his bottom line, Mr. Venables asks if we want to be part of the process for improving “it.” It’s quite concerning that the industry Mr. Venables helped create and continues to champion is not one of openness and transparency — neither open to feedback, nor open to suggestions for how to improve, nor open to help in getting it done.

Mr. Venables: please show me where in either your post or your Principles you provide your audience ‘the how’ for contributing to the process for improving ‘it.’ This seems to be missing from my copy.

For the last three years I and others have sat on the business end of Security Ratings vendors, their customers, and their data. As a result, some of us have independently and collectively researched ratings vendors and the industry to better understand the direction, changes and impact being made, and have published some of the findings publicly. Many efforts have been made to work with those that claim to be ‘industry leaders’ in the security and risk ratings field to try and effect positive change, but it has fallen on deaf and unaccountable ears.

Instead, the Security Ratings industry has come at a cost of countless millions of dollars, unnecessary pain and suffering, and steps backward instead of forward. But sadly the sentiment remains that there’s nothing to be concerned about as long as we carry on and focus on the future.

The Missing [Credit Scoring] Link

Every industry vendor I have spoken to who is associated to the Principles (and a few who aren’t) have cloaked themselves behind a curtain of intellectual property and trade secrecy by refusing to openly and publicly show a list of:

• what they measure
• how they measure via their methodology (do they even have a methodology?)
• which standardized metric are they citing and then measuring against
• the basic remediation steps required to address the issue as defined by the aforementioned bodies

These Principles themselves lack transparency and a means for seeing what iterative improvement and change have occurred, when, and how. Any interpretation has been left largely to the ratings vendors themselves to define in their own marketing words, leaving the practice of those involved in the Principles to be fractured.

To that point, in his post Mr Venables tries to simplify and align this industry to financial credit scores as being “useful enough within a margin of error to permit a gigantic market to mostly function reasonably well.”

Wow.

Frankly, this statement breaks down under inspection. Is this the level of confidence you want in statements about an industry claiming to identify and analyze the risk your organization presents to your customers?

While Security Ratings providers often associate their industry to that of the credit rating industry, they neglect to include that the credit ratings industry (in both the US and abroad) is a highly regulated industry with limits and scope of data use, consumer protections, and accountability for the organizations providing the scoring.

I encourage Mr. Venables to please provide reference to whatever he’s using to support this association. In the meantime, I encourage everyone, especially Mr. Venables to think about how the 110 page Fair Credit Reporting Act, or even the Summary of Your Rights Under the Fair Credit Reporting Act would compare and contrast to that compact and airy single page Principles document defining their industry success as “useful enough”.

My Bottom Line

What’s supposedly guiding the Security and Risk Ratings industry is sad and broken. Sure the product output may have a shiny and colourful veneer, but below the surface lays something poorly built and fit to be condemned.

These Principles need to be addressed by more than a back room of 45 organizations. If Mr. Venables motherly statements of inclusion are truly sincere, then I as well as others will welcome the opportunity to roll up our sleeves and provide assistance though direction and action for positive and lasting change.

Otherwise, we are left to assume his silence is acceptance.

Until then my advice to fellow security, risk, and compliance colleagues is not to validate these services by buying from them or fixing their flawed data, but rather to ignore their advice or sue them for malpractice or defamation. Their “advice” is not funded in reality unless heavily curated by the ones being rated at their own cost and at their own risk. Lets collectively move the risk to the rating companies and stop accepting this abuse from these security charlatans.

Footnote: I’d like to thank the numerous executives, compliance, risk, and security practitioners who helped contribute to this response. Your input is invaluable.

Let’s talk about contextual evidence, or rather the lack thereof, in the Security Ratings industry:

For more than a year, I’ve witnessed the various Security Ratings vendors tell their customers and assessed organizations what they deem as “severe” findings in their reports, without any contextual evidence to support either investigation or remediation efforts. These include hundreds of domains, hosts, IPs, and other related findings, each added to their reports with the claim that they are associated to the assessed organization, but without proof or supporting evidence to back up these statements with any accuracy.

The only way to verify these claims and to mitigate the findings requires the report reader to have both knowledge and access to the contextual reporting on, and intimate knowledge of, the organization being rated, and to further dedicate time and resources to investigate each of the reported claims.

For example, this week I witnessed new ‘malware event’ identifications added into a report from Security Scorecard. Their report provided an “Identified Date” of several days prior, an associated IP address, a detection method of “Email,” and a statement of “14 observations,” purportedly occurring over ports that are “not available.” That’s all.

Security Scorecard provided no actionable, or even verifiable, data to support these “findings.” The result: the organization, eager to understand these “findings,” spun-up multiple resources to investigate, and in the end, nothing was actually found. Further, external threat intelligence that should have corroborated the finding showed no results.

I’ve repeated this same exercise with Security Scorecard and similar vendors countlessly — each time requesting the ratings vendor to supply contextual supporting evidence for their “findings.”

This week was no different. I just received SecurityScorecard’s responses to the tickets submitted against the malware events and almost 20 other findings — all were set for removal from their reporting without any supporting data, reasoning, or explanation and simply “Marked for removal.”

So, it would appear that we have a vendor making possibly false, and definitely unverifiable, negative statements about an organization, in a semi-public manner, and refusing to provide evidence to back up their claims. Does anyone have thoughts or insight on what the legal parlance for this sort of behavior might be called? I’m fairly certain the term isn’t “false positives.”

Dear RiskRecon (and other Security Ratings vendors passively assessing organization’s patching posture),

Failing to recognize, investigate, and implement the many well-documented ways that have been shared with you for how you identify backported security fixes in applications does not equate to a “false-positive” like you claim. It’s just plain lazy, and inaccurate.

But instead of belaboring this point, I think we should talk about something more basic and “fundamental”:

Please explain how it is that you can passively collect header information that tells you the OS like this: Apache/2.2.15 (CentOS), and then tell your customers that it’s vulnerable to CVE-2010–0425, CVE-2007–6423?

Some important questions for the U.S. Chamber of Commerce:

• Is this the level of acceptable accuracy and empirical evidence you define in your principles?
• Where is the accountability for Security Ratings vendors?
• Who investigates the investigators?

Maybe you could start a new initiative like: The US Chamber of Commerce Center for Security Ratings Vendors Who Can’t Read Good and Want To Do Other Stuff Good Too

“High Severity Vulnerabilities found”

Imagine viewing a report that contains the data types above, associated with pages upon pages of findings of software patching vulnerabilities. Imagine that a company you are evaluating is the subject of this report and ask yourself the following questions:

• What applications are impacted here?
• Is my company’s use of those impacted applications critical?
• Is my company’s data adversely impacted by these findings?
• Any idea what OS platform is being used, and does it matter?
• Is the IP address owned by the organization or is it a dynamic IP from a third party provider?
• What level of confidence do you have in this data? What level do you think the Security Ratings vendor has?
• What methodology was used for collecting this data, and where’s the contextual evidence to support it?

If you’re a non-technical decision-maker, your eyes probably glaze over and you just move on. These questions may not even occur to you to ask. You may simply view the size of the report as evidence that, “there’s just too much evidence that the assessed organization isn’t patching; therefore, their patching practices are bad, and that must mean they have bad processes everywhere.” Time to re-think the relationship.

If you’re an Information Security professional evaluating another company, you might spend a few extra minutes to see if there’s something that stands out in the data.

However, if you’re an Information Security professional evaluating your own company that has implemented open source operating systems like Fedora, Centos, RHEL, or Ubuntu, you might look at this a bit differently.

Below is an excerpt from the FAQ section on “A PCI audit says I am running a version which has CVE exploits in it” by CentOs.

Security patches and bug fixes are backported into the shipped version. See here for details: https://access.redhat.com/security/updates/backporting. Simply reading a version number on a package or a banner from network scanning is not sufficient to indicate a vulnerability, in light of this approach. Most reputable vendors understand this, but some seem to not account for the upstream approach in their product’s reporting interface.

…

If a scan report is complaining about package versions, the person providing it is probably not doing it right, as the popular meme goes. CentOS and its upstream are continuously updated, and the CVE’s addressed are reflected in the aforementioned changelog, so running a protective backup, updating, and rebooting or restarting the affected daemon service should address the matter. Other approaches, such as using one keyed to package version numbers, are simply wrong.

While Centos and Red Hat have had these articles in place for some time, Red Hat recently felt the need to add greater specificity in addressing backporting and the Security Ratings industry.

Security Ratings organizations often talk about the “volume of data,” such as the millions of critical data points they collect and analyze. They also often claim to have the greatest breadth and quality of intelligence available — all with less than 1% of false positives. However, without a published methodology or any contextual evidence to support the findings, there really is no assurance of their statements as fact, other than glossy marketing statements regurgitated by sales.

Don’t take my word for it

It would appear that security rating vendors think that all networks run on Windows, and fit neatly into uniform boxes. However, even with published evidence (such as the articles referenced above) that there are servers with backported security fixes connected to the internet, and that their analysis and ratings processes are inaccurate, the Security Ratings industry is still doing it wrong.

Security metrics shouldn’t be reduced to the unreliable banner scraping of software versions. But, for Security Ratings vendors, it seems that is just what they have done.

Here’s another example:

If your eyes were immediately drawn to that number on the right, I wouldn’t blame you. It’s easy to create a bias based on an artificial statistic that has a large number. In this case, the author wishes you to know that this host hasn’t been patched in over 7 years.

Or has it?

Some industry colleagues and I wanted to see how bad these Security Rating findings really are. Are there truly millions of data points being used and do they really reflect the truth? Or is this something more…. Simple.

We secretly replaced banner information on Linux machines running applications such as Apache, with an alternative string of text such as IIS/4.0 and watched to see what would happen.

Within days of this simple text modification (that anyone can perform without impacting the operation of the application or system), the hosts started populating in the reporting of Security Ratings vendors being rated against the modified header information.

An executive for a Security Ratings vendor recently said:

“We want to understand the state of every device that’s connected to the internet”

“We want to understand what is the level of security hygiene of this asset and what is the level of compromise of this asset.”

It may be a surprise to the person behind these statements, but I don’t believe they’re going to find this in the banner (even if it’s not modified). The good news is that what’s being described IS possible, but not without consent. It requires working WITH assessed organizations.

This Security Ratings executive went on to say:

“Security can’t be reduced to just: ‘I see that you have this unpatched vulnerability’. It’s more about: who have you hired to run your security, is the team competent, do they have processes and procedures in place?”

However, none of these factors are captured or evaluated by Security Ratings vendors — who instead capture information passively and make determinations based solely on that passive information.

The Ratings Vendor Response

When bringing this information regarding backporting to one Security Ratings vendor, their initial response was that “it is hard to discover these patches and specifically the CVE’s they patch.”

This vendor found this particular problem so difficult in fact, that instead of stopping, investigating, implementing and testing a proper fix, they simply suppressed the alerts. This means that while data was still visibly showing up in their platform as identifying vulnerabilities, the findings were not being listed or rated against the assessed organization — and only that organization. If it’s out of sight, it should be out of mind, right? Perhaps for just one organization being assessed, but definitely not for everyone being rated. That to me would appear to be a pretty clear sign that they don’t care about accuracy.

Another Ratings organization stated that they primarily determine the version information based on what is visible in the data they collect, and don’t perform vulnerability scans/tests, even though their reporting states that they have used ‘passive analysis to identify systems running end-of-support software that have security vulnerabilities’. The question remains as to how they can determine that vulnerabilities exist without running vulnerability scans or tests. Instead of correctly addressing the backporting issue, they instead deemed their findings to be ‘false positives.’

How can Security Ratings organizations who put so much weight on patching also claim their results, which include these inaccuracies, as empirical and accurate? This approach is similar to taking a single IP address coming from one country, and making definitive statements about nation state attribution without using any other data to confirm.

Security Ratings organizations need to stop referring to their poor processes and lack of testing as ‘false positives.’ Further, Security Ratings organizations shouldn’t wait until an assessed organization objects to these false positives, and start being accountable for their accuracy.

The process of assessing the security posture of an organization’s infrastructure should be largely defined publicly by the Security Ratings organization’s methodology. Without a published methodology, the data and ratings cannot be considered transparent nor accurate and verifiable.

Verify until you can Trust but Verify

Many Security Ratings vendors have publicly committed to honoring the Principles for Fair and Accurate Security Ratings (and might have even put their name next to and then blogged about.) Organizations being assessed by, as well as those relying on, Security Ratings vendor reports are strongly encouraged to understand and apply the Principles and to include the following below listed points in their discussions and analysis of the report:

Accuracy and Validation:

Ratings should be empirical, data-driven, or notated as expert opinion. Rating companies should provide validation of their rating methodologies and historical performance of their models. Ratings shall promptly reflect the inclusion of corrected information upon validation.

— Principles for Fair and Accurate Security Ratings

When provided with access to third party Security Ratings reporting and data:

• Ensure the report includes contextual, time bound evidence to support the findings within. If the report does not include this, ask the Ratings vendor to provide these details and/or explain why they are not provided.
• With regards to patching, ask how they take security backporting into account and where it is stated in their methodology. If they do not include backporting, ask why that is not clearly articulated in the report, along with an explanation of what the backporting does in terms of changing the findings.
• Ask where their confidence rating is in their reported findings.
• Does the report provide a means to verify that the findings are accurate, including that the information from the banner (which is modifiable) is what it says it is on face value? How do they know it is accurate?

Transparency:

Rating companies shall provide sufficient transparency into the methodologies and types of data used to determine their ratings, including information on data origination as requested and when feasible, for customers and rated organizations to understand how ratings are derived. Any rated organization shall be allowed access to their individual rating and the data that impacts a change in their rating.

— Principles for Fair and Accurate Security Ratings

Anyone provided with access to third-party Security Ratings reporting and data should:

• Ensure understanding of how the Ratings company arrived at their conclusions.
• Review their methodologies and see how comprehensive they are.
• If they do not provide their methodology either as part of the report or via a publicly-accessible link, question why they do not.
• Ensure that you and your organization are allowed access to full and complete information related to the score, and not just a summary report that includes only a subset of measured data .

Confidentiality:

Information disclosed by a rated organization during the course of a challenged rating or dispute shall be appropriately protected. Rating companies should not publicize an individual organization’s rating. Rating companies shall not provide third parties with sensitive or confidential information on rated organizations that could lead directly to system compromise.

— Principles for Fair and Accurate Security Ratings

Organizations should think about the type of information they are viewing in these reports — even (and especially) if the findings don’t belong to them. Ratings vendors provide their customers with reports that they describe as an unprecedented, accurate analysis from millions of data points of passively captured data. This does not mean that it’s ethical to share all the collected data with anyone that pays, while restricting the same information from those being assessed. This could lead to a pay-for-exploitation business model.

Anyone provided with access to third-party Security Ratings reporting and data should:

• Ask their Security Ratings company who they share reports and data with, and if different levels of contextual evidence are provided to assessed organizations than to customers. If not, why not?
• Review the reports provided and determine if the information shares IP addresses and hosts in conjunction with vulnerabilities. Ask if there is a reason they are providing the IP address and hostname next to the finding they claim makes it vulnerable.
• Ask if this information has already been disclosed to the assessed organization. If so, when, and how. If not, why not?

Transparency Through Social Engineering

For many organizations being assessed, the results and findings come as a shock, and with it, a sense of urgency to set the record straight. Without context and confidence, these bold statements that portray of a lack of operational security, can lead organizations to believe that the statements and findings are ‘ground truth’ when there is so much instability its quicksand.

It is not your responsibility to confirm or clarify the operational security posture to your attackers. By doing so you leave yourselves open to increased risk of attack.

Would your legal team honor a request by any organization to provide comprehensive data about your organization’s infrastructure and security controls that are not already published publicly? Is there a difference between an attacker attempting to socially engineer data from your organization and staff, and a Security Ratings vendor that may have a financial relationship with an organization you know?

Social engineering, in all its many forms continues to be an effective means for attackers to achieve their goals. One security vendor recently reported that they saw a 500% increase in social engineering attacks during the first half of 2018 stating, “Social engineering is increasingly the most popular way to launch email attacks. Criminals continue to find new ways to exploit the human factor.”

Through personal observation as well as confidential conversations with risk assessed companies it’s been clear that the types of data that have been found to be inaccurately represented or misleading has been great and far reaching. What has come out of the probing into Security Ratings vendors’ methodologies and practices is a question that should rattle organizations to the core:

Why don’t you just tell us?

When presented with evidence that their methodology is flawed, Security Ratings vendors offers up platitudes and apologies for misrepresenting the assessed organization. They also asserted that these can only be corrected by disclosing all of the assessed organization’s IP blocks and domains, as well as insight into other areas such as compensating controls. This was done under the guise to ensure that their customers viewed the organization correctly.
Even if all of this confidential data had been provided, the Security Ratings vendor could only assure that the finding would be marked as ‘false positives,’ and that no changes to their process or methodology would result.

As a reminder, Security Ratings organizations are using passive, non-invasive means to attain an attacker’s eye view of organizations. They tout accuracy in their data, some even saying they have a false positive rating of less than 1%, though they offer no data to support that metric. Yet multiple inaccuracies in their reporting have been presented, and their processes have not improved. If their data truly were highly accurate, why would Security Ratings vendors need your organization to provide confirmation and additional context about your infrastructure?

…Something is rotten in the state of Denmark.