TL;DR — Congressional law surrounding the federal regulation of Laboratory-Developed Tests (LDTs) must be augmented in response to advances in genetic testing technology. The Food and Drug Administration (FDA) software testing procedures must also be augmented to ensure software that affects the health of millions of people is correctly implemented and leads to accurate results.

Today, for around $99 to $199, novel technology has enabled anyone to use a saliva sample to understand not just their ancestry but also crucial disease risk information based on their personal genome [1]. However, because of the significantly influential nature of the health data being received, it is crucial that the information provided to customers are accurate and have gone through the necessary validation procedures. Thus, for the past month, we have been in contact with the Food and Drug Administration (FDA), genetic services companies and experts in the field to get a better idea of how these processes work and whether consumers should trust these companies. Interestingly, we found an important differentiating factor between companies that required FDA approval versus those that do not — being direct-to-consumer or physician-initiated.

23andMe — A Direct-to-Consumer Genetic Services Company

Founded in 2006, the genetic services company 23andMe was one of the first companies to offer a direct-to-consumer genetic testing service. This service launched in 2007 and won Time Magazine’s Invention of the Year for 2008 by delivering genetic information related to a person’s ancestry and their risk levels for a provided list of diseases [2]. By October of 2010, 23andMe had increased the number of diseases tested to over 200 in their health reports (see Figure 1)[3] which included diabetes, arthritis, lactose tolerance as well as testing for BRCA-related genes for cancer risk. During this time, 23andMe contracted out its services through laboratories had also been accredited by the College of American Pathologists (CAP), which regulates laboratory operation standards, and had Clinical Laboratory Improvement Amendments (CLIA) certification, which facilitates sites that test human specimens. Dr. Alberto Gutierrez, the former director of the FDA’s Office of In Vitro Diagnostics and Radiological Health mentions, “This is important, particularly in the early days since 23andMe does not portray itself as a laboratory and did not want California to regulate them as a laboratory.”

However, in November of 2013, the FDA sent a warning letter to 23andMe that ordered the company to stop marketing and offering its health services. In the warning letter, the FDA expressed particular concern regarding the accuracy of information provided to consumers and the potential health consequences that could result from false positive and false negative assessments. In particular, the FDA suggested, “the false genotype result for warfarin drug response test [one of the BRCA-related tests] could have significant unreasonable risk of illness, injury or death”.[5]

When asked about why it took the FDA so long to respond, Dr. Gutierrez, who also signed the 2013 warning letter to 23andMe, explained to us, “it was their inability to put a submission together, as stated in so many words in the warning letter.”. In the letter, Dr. Gutierrez stated that, “months after you (23andMe) submitted your 510(k)s and more than 5 years after you began marketing, you still had not completed some of the studies and had not even started other studies necessary to support a marketing submission for the PGS (Personal Genome Service). It is now eleven months later, and you have yet to provide FDA with any new information about these tests.”[5]

23andMe decided to comply with the FDA so after this incident, 23andMe stopped offering its health risk testing service and began focusing primarily on its ancestry service. When asked about services offered by the company before 2013, Annie from 23andMe’s Customer Support responded that she could not comment on those tests because of “regulatory reasons” and that they were no longer offering those products. When asked about how 23andMe views the tests offered before 2013, Annie responded, “Our mission has always been to help people access, understand and benefit from the human genome and that has not changed. However I can’t comment on the stance of tests before.”

After a two year hiatus, 23andMe brought back its health risk tests in October of 2015 [6], giving customers carrier-status information for 36 diseases, including cystic fibrosis and sickle cell anemia as reported by the New York Times. The FDA approved 10 more diseases in April of 2017 [7] and three specific BRCA1/BRCA2 breast cancer gene mutations (out of more than 1000 known BRCA mutations) in March 2018 [8].

The Growth of Physician-Initiated Genetic Services Companies

Since the emergence of 23andMe, there has been a growing number of companies in the genetic testing space, specifically physician-initiated genetic services companies (see Figure 3). As an example, we have chosen to highlight Color Genomics, which offers physician-initiated genetic testing for BRCA1, BRCA2, and 28 other genes so that customers can understand their risk for genetic diseases such as hereditary breast and ovarian cancer. Due to unresolved congressional law and the FDA’s interpretation of said law, Color’s processes do not require FDA oversight and approval but only require CLIA certification and CAP accreditation (regulations explained in the section above). Physician-initiated genetic testing companies can avoid requiring FDA approval on the validity of these tests due to being classified as Laboratory-Developed Tests (LDT). However, Michelle Wu, the CEO of Nova Approval, a company that works with the FDA approval process for medical devices, mentions, “If these (physician-initiated) companies grow big, they will be impacting more customers. When that time comes, the FDA will need to be involved. The challenge for the FDA is that it does not have the capability or manpower to do those approvals as innovation is always ahead of the regulation.”

When asked about how Color Genomics validates its tests, Cynthia, one of their genetic specialists, cites several peer-reviewed publications on their website from blind studies conducted with 99% success at UCSF and University of Washington, which they follow for all their tests and continue to improve upon with ongoing research. She also states that they have certified professionals review every result and that they allow for genetic counseling for all of their customers as a part of their service.

It appears that being a physician-initiated genetic services company does not affect Color Genomics’ ability to provide validated tests. This leads us to ask — why is it that the FDA is even involved in the space and why do only direct-to-consumer companies require this type of oversight?

The “Physician-Initiated” Loophole

Based on current laws surrounding Laboratory-Developed Tests (LDTs), Physician-Initiated services from Laboratory-Developed Tests do not require FDA oversight whereas Direct-To-Consumer Companies do require FDA oversight.

The FDA views companies that sell physician-initiated tests as providing a service so that they will only need to follow the requirements for LDTs which require only CAP accreditation and CLIA certification. For “direct-to-consumer” companies that do not have physician-initiated tests, the FDA classifies this company as selling a medical device. In the example of 23andMe, the FDA viewed their “Personal Genome Service (PGS)” (see Figure 4) as a medical device that needed regulation.

When we were talking about this medium post with a friend who was working on a medical device startup, he immediately noted, “Hey, we know about this loophole. Don’t ruin it for us!”. For most startups, having to deal with FDA regulation on top of creating a successful business turns a difficult task into one that is seemingly impossible. That’s why so many companies that are using Artificial Intelligence, novel algorithms and other medical hardware/software systems to assist in the healthcare sector are bringing physicians in the loop to avoid the scrutiny of the FDA.

While the ability to avoid the FDA medical device approval processes with physician-initiated services has certainly led to more companies in the medical device industry, there are serious consequences if and when companies abuse this process and inadvertently provide inaccurate information to consumers, especially for the genetic testing space we are focusing upon. Although, we have mentioned the rigorous processes Color Genomics utilizes without FDA oversight, there will eventually be companies that cut corners and inadvertently provide inaccurate information with harmful effects.

First off, when health reports and information are provided to physicians for the sake of their patients, Dr. Gutierrez warned, “it is not possible for a physician to be able to interpret all the test results” due to lack of expertise in every area of healthcare. At some point, they might care for only one result (e.g. BRCA) and discard the rest. The genetic test result itself might also contain the Variant of Uncertain Significance (VUS) which makes interpretation difficult for even geneticists.

Furthermore, Dr. Gutierrez even admits himself that, “there is a loophole in the current regulation” of genetic testing companies and that the FDA has been aware that the regulation must be changed for years. In 2010, the FDA announced that it would hold a public meeting to discuss “how the agency will oversee” LDTs due to many misleading, deceptive studies that had yield inaccurate results. In 2014, the FDA released two draft guideline documents regarding to LDT regulation but after two years of debate, the clinical laboratory and healthcare communities could not settle on an agreed proposal so this plan was put on hold. In 2017, after collecting feedback from the public, the FDA released yet another document discussing LDTs and proposed updating the CLIA regulations overseeing laboratory operations.[9]

Unfortunately, this paper has yet to be escalated, implemented or resolved. This unending debate regarding LDT regulation continually enables this unresolved loophole that may affect thousands or even millions of consumers whom receive inadvertently inaccurate data from physician-initiated genetic services companies in the future.

How does and would the FDA regulate these medical devices?

Let’s pretend for a moment that laws are changed and the FDA now has oversight into validating the tests any genetic services company provides. How do they make sure these devices are working correctly and accurately?

In the past, medical devices had mainly been involved with hardware testing where the FDA thoroughly inspected the entire process of how the device works. Nevertheless, as technology has progressed, medical devices has included much more software which has been more difficult to test. Today, “FDA only looks at the output data and compares it with the study to make sure that the data is appropriate and that we can rely on it”, Dr. Gutierrez informed. “FDA does not look at the codebase and test the program themselves”.

From a deep dive into the FDA’s Software Testing Regulations, it appears there is an engineer that does look at the submission, which includes Level of Concern, Software Description, Device Hazard Analysis, Software Requirement Specification, Architecture Design Chart, Software Design Specification, Traceability Analysis, Software Development Environment Description, Verification and Validation Documents, Revision Level History, and Unresolved Anomalies. Still, this does not imply that the engineer looks at the source code and that they actually test the program themselves since all provided pieces of information are text descriptions (design and algorithm).

This raises a concerns that the program might be programmed with a partial hardcode fashion to yield high performance during the test similar to that of the Volkswagen “Diesel Dupe” Scandal. Altered data is difficult to catch and is a valid point of concern for consumers that depend upon FDA’s approval processes.

What can be done?

There are two significant changes that must occur to ensure the safety of genetic testing service consumers.

1. Congress needs to pay more attention to the issues surrounding LDTs so that laws can be updated to reflect modern day technologies.
2. FDA needs to improve upon its software testing procedures for medical devices and become more involved during the software development process as well as the data processing pipeline.

First, the issue around having more regulation surrounding LDTs must become a higher priority for congress. Only by setting a deadline for a compromise can this sector of the medical device industry become better regulated to ensure more accurate tests and safer products. After a quarter of a century, laws surrounding LDTs and CLIA certification have still not been changed. (CLIA was last finalized in 1992!) Specifically, LDTs must have its tests be put under higher levels of regulation for their accuracy and must go through the same validation processes the FDA has for other medical devices.

Second, the FDA must become more involved during the software development process so that it can ensure that the company is following standard development processes throughout, rather than just checking at the end of a product’s development lifecycle. Furthermore, for genetic tests, the regulation should cover the whole test pipeline, from obtaining the data to interpreting the concluding result regardless of external physician involvement; today CLIA and CAP only cover the laboratory operation standards.

Parting Thoughts

The FDA approval process for genetic services companies is an ethical issue that requires further examination. The advent of new technology in the field of genetics is supposed to increase our access to a better and healthier lifestyle by decreasing cost of disease risk analysis and by providing more valuable information. However, we must balance the ability to immediately help consumers who need access to this type of information with regulations that may hinder the adoption of the new technologies but can ensure more accurate information presented.

For the case of 23andMe, even after 4 years of complying with the FDA, they are still nowhere close to providing the same number of tests they had previously allowed that could have benefited many people. On the other hand, as the FDA pointed out for BRCA risk assessments, with a “false positive, it could lead a patient to undergo prophylactic surgery, chemoprevention, intensive screening, or other morbidity-inducing actions, while a false negative could result in a failure to recognize an actual risk that may exist.”

Nonetheless, there are a few “common sense” steps we can take to ensure the safety of consumers starting with updating our antiquated laws to keep up with the present day technological advances and implementing more transparent software testing processes between companies and the FDA.

Special thanks to Dr. Alberto Gutierrez for taking the time to speak with us over the phone and answering our questions. We also truly appreciate the help of Michelle Wu for providing invaluable insight into this matter. Finally we are grateful for the gracious customer support and genetic specialists at 23andMe and Color Genomics for lending their time to talk to us.

We’d love to hear your thoughts on this issue, especially if you’ve ordered a genetic testing kit yourself. Drop us a line at alfeng6[at]stanford[dot]edu!

[1]: https://www.23andme.com/

[2]: http://content.time.com/time/specials/packages/article/0,28804,1852747_1854493,00.html

[3]: https://web.archive.org/web/20131107064641/https://www.23andme.com/health/all/

[4]: https://web.archive.org/web/20140114140414/23andme.com/health

[5]: https://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2013/ucm376296.htm

[6]: https://www.accessdata.fda.gov/cdrh_docs/reviews/DEN140044.pdf

[7]: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm551185.htm

[8]: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm599560.htm

[9]: http://clinchem.aaccjnls.org/content/63/10/1575.long