<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Health Data Science - Medium]]></title>
        <description><![CDATA[Precision Medicine through Data Science and Machine Learning - Medium]]></description>
        <link>https://medium.com/health-data-science?source=rss----e07efed498a6---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Health Data Science - Medium</title>
            <link>https://medium.com/health-data-science?source=rss----e07efed498a6---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 10:01:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/health-data-science" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Artificial Intelligence Can Now Use Instagram to Predict Substance Abuse Risk]]></title>
            <link>https://medium.com/health-data-science/artificial-intelligence-can-now-use-instagram-to-predict-substance-abuse-risk-b805e78354af?source=rss----e07efed498a6---4</link>
            <guid isPermaLink="false">https://medium.com/p/b805e78354af</guid>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[mental-health]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Jerry Wei]]></dc:creator>
            <pubDate>Fri, 03 Jan 2020 22:45:22 GMT</pubDate>
            <atom:updated>2020-01-03T22:45:22.597Z</atom:updated>
            <content:encoded><![CDATA[<h4>A paper published recently developed a method of predicting someone’s risk of substance abuse based on their Instagram accounts (<a href="https://www.nature.com/articles/s41386-018-0247-x">link</a>). Let’s take a look at how they did it.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PWFvYeHpbzJaTSSD" /><figcaption>Photo by <a href="https://unsplash.com/@zambon?utm_source=medium&amp;utm_medium=referral">Alessandro Zambon</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Substance Abuse. </strong>The authors of the paper recognized the pressing issue of substance abuse — it is an issue around the world that affects people regardless of socioeconomic status. Here’s a quote from the paper:</p><blockquote>“Altogether with lifestyle choices and metabolic risk factors, the use of alcohol, tobacco, and drugs are among the top ten causes of preventable deaths in the United States. The misuse of prescription and illicit drugs causes over 100 deaths from overdose alone each day, dethroning motor vehicle accidents as the leading cause of injury deaths.”</blockquote><p>The current opioid epidemic in the United States has only made the problem worse, and the authors hoped to employ modern technological innovations to address the problem.</p><p><a href="https://www.datadriveninvestor.com/2019/03/01/the-myth-of-ais-predictive-power/">The Myth of AIs&#39; Predictive Power | Data Driven Investor</a></p><p><strong>Instagram Data.</strong> The paper chose to use Instagram because of its popularity among young users and its rapid growth. The authors note that the images and text that users post can be used to get an idea of each user’s personality. To capitalize on this feature of social media, the authors crowdsourced thousands of participants (with their consent, of course) to share their Instagram information and participate in the study. Specifically, they looked at each user’s posted photos, comments, and captions (interestingly, the average number of posts per user was 183.5 posts). The authors used NIDA Modified ASSIST to assign “high-risk” and “low-risk” labels to participants. Let’s look at some extreme examples so we can get a better understanding of what kinds of things artificial intelligence would be looking for.</p><ul><li>If a user posts a picture of themselves drinking alcohol, the machine learning model would probably mark that as high-risk for substance abuse.</li><li>If a user comments and says “I hate drugs”, the machine learning model would probably mark that as low-risk for substance abuse.</li><li>If a user posts an image with the caption “Currently abusing substances”, the machine learning model would probably mark that as high-risk for substance abuse. (OK unrealistic but you get the point)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/978/1*EvsWRikQmhxmF_BTn-A-HQ.png" /><figcaption>Substance use risk distribution among Instagram users in the dataset. Image credits to the authors of the original paper.</figcaption></figure><p><strong>Machine Learning Model.</strong> Since image and text have different data structures, the authors used multiple different machine learning architectures. For analyzing the images that users posted, they used a ResNet-18 model that was pre-trained on the ImageNet dataset. They fine-tuned the model on their own Instagram dataset to maximize accuracy. For analyzing the text (captions and comments), the authors first used Word2Vec on their data, a method of changing words into vector representations, and then fed the resulting vectors into a Long Short-Term Memory (LSTM) network. They then used a final layer to combine the results from the image and text networks to get a final prediction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*exuCNIln5Rlo3Gmjt5gNbg.png" /><figcaption>The architecture that was used — CNNs extract features from images and LSTMs extract features from text. A final layer uses those features to get a final prediction. Image credits to the authors of the original paper.</figcaption></figure><p><strong>Results.</strong> The authors found that their AI could identify alcohol risk based on Instagram content significantly better than what is expected due to random chance. They got an F1 score (a standard measurement of accuracy for this type of problem) of 72.4%, compared to a guessing F1 score of 50%. However, the results were not as promising when the AI was trying to identify risk for tobacco, prescription drugs, and illegal drugs; the AI was not able to achieve F1 scores above 40% for those classes.</p><p><strong>Conclusion. </strong>The paper was the first to show that machine learning could be used to identify potential substance abuse behavior using social media. The authors note that their method is completely automatic, an improvement on previous research that required manual analysis to predict substance abuse risk. They attribute the model’s inaccuracies on classes outside of alcohol to an imbalanced dataset (there were relatively few people that were high-risk for those classes).</p><p>I’ve listed some resources below that may be of interest.</p><ul><li><a href="https://www.nature.com/articles/s41386-018-0247-x#Sec3">Original Paper.</a></li><li><a href="https://www.drugabuse.gov/related-topics/trends-statistics">Statistics on drug abuse in the US.</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b805e78354af" width="1" height="1" alt=""><hr><p><a href="https://medium.com/health-data-science/artificial-intelligence-can-now-use-instagram-to-predict-substance-abuse-risk-b805e78354af">Artificial Intelligence Can Now Use Instagram to Predict Substance Abuse Risk</a> was originally published in <a href="https://medium.com/health-data-science">Health Data Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Artificial Intelligence Can Now Detect Bone Fractures From Osteoporosis]]></title>
            <link>https://medium.com/health-data-science/how-artificial-intelligence-can-now-detect-bone-fractures-from-osteoporosis-fb9deddcefe8?source=rss----e07efed498a6---4</link>
            <guid isPermaLink="false">https://medium.com/p/fb9deddcefe8</guid>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[health]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Jerry Wei]]></dc:creator>
            <pubDate>Fri, 03 Jan 2020 22:42:44 GMT</pubDate>
            <atom:updated>2020-01-03T22:42:44.545Z</atom:updated>
            <content:encoded><![CDATA[<p>I read a research paper in which the authors used deep neural networks to automatically detect osteoporotic vertebral fractures on CT scans (<a href="https://www.sciencedirect.com/science/article/pii/S0010482518301185?via%3Dihub">link</a>). Let’s examine what they did.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8QlBZI8T-B0il2w3" /><figcaption>Photo by <a href="https://unsplash.com/@kentreloar?utm_source=medium&amp;utm_medium=referral">Ken Treloar</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Osteoporotic Vertebral Fractures.</strong> Abbreviated as OVFs, Osteoporotic Vertebral Fractures are fractures that occur in the spine as a result of osteoporosis. <em>You keep saying “osteoporosis”, but what is that? </em>Osteoporosis is a disease where bones become fragile and filled with holes. This means that the bones are losing their density and quality — the main issue is that osteoporosis happens over a long period of time; there are often no symptoms shown until the first fracture happens. Not only is osteoporosis very difficult to detect, but it is also extremely prevalent in the general population. This disease sees more than 3 million cases in the United States each year, and women older than 50 are most likely to get the consequent spine fractures. In fact, spine fractures that result from osteoporosis will happen to about 40% of women by age 80. <em>So how do we currently tell if someone has OVFs? </em>The current standard to identify spine fractures is through CT scans and X-rays, which will be manually looked at by medical professionals. Knowing that, let’s talk about the innovation…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vOU5kI24uQj0EsM5cKXPVg.png" /><figcaption>CT scans identified as positive for OVF by the deep learning model. The heatmaps show regions that were strongly associated with the model’s prediction. Image credits to Tomita et al., the original authors of the paper.</figcaption></figure><p><strong>Data Collection.</strong> The authors used over one-thousand CT scans from the Dartmouth-Hitchcock Medical Center, applying a filter so that the CT exam had to be of the chest, abdomen, and pelvis. If the CT scan showed “compression deformity or compression fracture”, then that scan was marked as positive for Osteoporotic Vertebrae Fracture. The other scans were marked as negative for OVF. The exams were then randomly split such that 80% was used as a training set, 10% were used as a validation set, and 10% were used for testing.</p><p><a href="https://www.datadriveninvestor.com/2019/01/07/the-ultimate-learning-path-for-deep-learning-in-2019-more/">The Ultimate Learning Path for Deep Learning in 2019 &amp; more... | Data Driven Investor</a></p><p><strong>Deep Learning Model.</strong> The authors created a system with two major components: a Convolutional Neural Network (CNN) for feature extraction and a Recurrent Neural Network (RNN) to use the extracted features to make a final prediction. The CNN was a 34-layer Residual Network (ResNet), which is a state-of-the-art machine learning model for image processing. The RNN was made of Long Short-Term Memory (LSTM) networks, which have the capability to identify and ‘remember’ important information through special gates that control information weights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*PuXMyJSWVBIOSTFbLDCYGg.png" /><figcaption>A representation of the authors’ deep learning model, where the CNNs are feature extractors and the LSTMs are feature aggregators. Image credits to Tomita et al., the original authors of the paper.</figcaption></figure><p><strong>Results. </strong>The authors tested their model on a test set comprised of images the model has never seen before with the following results. The deep learning model in this paper achieved the best overall accuracy (89.2% accuracy) in comparison to other baseline models (highest accuracy of 88.4%) and even outperformed professional radiologists (88.4% accuracy). The authors also plotted Receiver Operating Characteristic (ROC) curves for the models, and their neural network’s curve was similar to that of radiologists, which is further evidence that the model’s performance was on par with practicing radiologists.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o3tjaDNI1_7mzcpcZK6wxg.png" /><figcaption>Performance of the paper’s model in comparison to that of radiologists — the model proposed by the paper achieved the highest overall accuracy. Credits to Tomita et al., the original authors of the paper.</figcaption></figure><p><strong>Conclusion. </strong>The authors showed that their CNN→RNN model was capable of doing Osteoporotic Vertebrae Fracture diagnosis with similar performance to that of practicing radiologists. They note that most commonly, OVFs are missed by radiologists (false negative) because of “staff shortages and/or excessive workload”. Their machine learning model would be able to reduce time and manual burdens that are placed on radiologists and could also improve overall diagnostic accuracy.</p><p>While the world may not be ready for artificial intelligence to completely conduct medical diagnoses, perhaps sometime in the future we will be examined by machines with the same (if not better) capabilities as humans.</p><p>I’ve put some additional resources below that may be interesting to look at.</p><ul><li><a href="https://www.sciencedirect.com/science/article/pii/S0010482518301185?via%3Dihub">Original paper</a></li><li><a href="https://www.webmd.com/osteoporosis/guide/osteoporosis-and-spine-fracture#1">More information on osteoporosis</a></li><li><a href="https://arxiv.org/pdf/1512.03385.pdf">ResNet paper</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb9deddcefe8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/health-data-science/how-artificial-intelligence-can-now-detect-bone-fractures-from-osteoporosis-fb9deddcefe8">How Artificial Intelligence Can Now Detect Bone Fractures From Osteoporosis</a> was originally published in <a href="https://medium.com/health-data-science">Health Data Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Generative Adversarial Networks (GANs) for Data Augmentation in Colorectal Images]]></title>
            <link>https://medium.com/health-data-science/using-generative-adversarial-networks-gans-for-data-augmentation-in-colorectal-images-565deda07a22?source=rss----e07efed498a6---4</link>
            <guid isPermaLink="false">https://medium.com/p/565deda07a22</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[health]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[computer-science]]></category>
            <dc:creator><![CDATA[Jerry Wei]]></dc:creator>
            <pubDate>Fri, 03 Jan 2020 22:41:04 GMT</pubDate>
            <atom:updated>2020-01-24T05:40:29.948Z</atom:updated>
            <content:encoded><![CDATA[<h4>A new way to generate augmented data for medical datasets could provide new insights for improving deep learning classifiers.</h4><p>So a pretty recent development in machine learning is the Generative Adversarial Network (GAN), which can generate realistic images (shoutout to my boi Ian Goodfellow for having the big brains to do this). An even more recent development is the use of GANs to generate images that can be used to augment datasets. Over the past few months, I’ve been working on a project that uses CycleGANs (more specific info about CycleGAN in <a href="https://towardsdatascience.com/cyclegan-how-machine-learning-learns-unpaired-image-to-image-translation-3fa8d9a6aa1d">my previous blogpost</a>) to generate synthetic data for a colorectal polyps dataset. I had the privilege of presenting my work at NeurIPS 2019 (I’ll write a blog post on my NeurIPS experience in the near future).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*B3O-7sBu_HOUzd_0" /><figcaption>Photo by <a href="https://unsplash.com/@nkachanovskyyy?utm_source=medium&amp;utm_medium=referral">Nikita Kachanovsky</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>The Problem. </strong>Using deep learning models for image classification tasks in a medical context is very popular, but accurately analysis medical images with deep learning classifiers requires large and balanced datasets. This isn’t always the case, though. For many diseases, the distribution of disease sub-classes in collected datasets is heavily skewed by each class’s prevalence among patients, and so detecting rare diseases in medical images with deep learning can be challenging. In other words, it’s hard to get enough training data if the disease is super uncommon. In these situations, a reliable method of data augmentation can mitigate the effects of data imbalance by preventing overfitting and thus improving overall performance.</p><p><strong>The (General) Solution.</strong> GANs!!! Use GANs to generate images of the rare diseases so that you don’t have that problem anymore. More specifically, CycleGAN might be better for medical images because information from one class might be useful to create synthetic data in another class. Think about it this way: it’s probably easier to generate an image of lung cancer if you’re already given the general structure of a lung than if you have to make it from random noise. For this reason, an image-to-image translation model would be suited for the task.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RjRuEd2PBvNAamCSG-qGbw.png" /><figcaption>General overview for how to obtain useful images for data augmentation using GANs. For more details about this, go to “The Team of Networks” section.</figcaption></figure><p><strong>The Dataset.</strong> I used a dataset of colorectal polyp images collected from the Dartmouth-Hitchcock Medical Center in New Hampshire, USA. I had 427 high-resolution whole-slide images, which I split into a training set of 326 whole-slide images and a testing set of 101 whole-slide images. For the training set, I had pathologists annotate all whole-slide images with bounding boxes representing regions of interest, for a total of 3517 variable-size image crops. Each image crop was labeled with a single class for the polyp type, which was either benign (normal or hyperplastic), or precancerous (tubular adenoma, tubulovillous/villous adenoma, or sessile-serrated adenoma). A lot of medical jargon — for the important aspect, look at the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MuDh1qbMi3saKXDTttJZuQ.png" /><figcaption>Visual representation of the dataset that I used. Don’t get boggled down by the medical wording — just keep in mind that the rare class (SSA) only makes up 3% of the dataset by size. Or just remember that the dataset is imbalanced.</figcaption></figure><p><strong>The Team of Networks.</strong> For my project, I used CycleGAN to convert images of normal colon tissue to images of precancerous colon tissue (the real medical lingo is kinda complex so we’ll leave it in that simplified form). But when you try to train CycleGAN with all of your precancerous tissue images, the generated images might look pretty normal. That’s because not all images that are labeled by pathologists as precancerous have strong precancerous features. In other words, some images might just be a little bit precancerous, but they still have to be labeled as precancerous; the images range from slightly precancerous to very precancerous. This is different from your common GAN to generate pictures of dogs: you don’t have images that are slightly dog vs very dog. So what’s the fix? Pretrain a classifier on your precancerous images and only use the images that are most precancerous to train your CycleGAN. How do you do that? Run the classifier on all of your training images and take the images with the highest output probabilities for each class based on what your classifier says. Don’t believe this works? Check this out:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3KVg-tSC25aPsD87zvGHJA.png" /><figcaption>What percent of CycleGAN’s generated images were correctly recognized as the intended class when we train CycleGAN using only the most precancerous images? As you filter out and select only the images with the most precancerous features to train CycleGAN, more of the generated images are recognized as tubular.</figcaption></figure><p><strong>Proof that this filtering method works!</strong> So you’ll see in the table above that I got numbers to back me up. Let me first define the alpha for you: I only use the top alpha images for training CycleGAN. So for alpha = 1/4, I only choose the top 25% most adenomatous images to train CycleGAN. As alpha gets smaller, I’m choosing fewer and fewer images and restricting my training set to only the most precancerous images. So the table above is representing the percent of CycleGAN’s generated images that are recognized by a classifier as the class they’re supposed to be. Let’s take the TA class as an example. When I train CycleGAN with all TA images, only 35.4% of the generated images are classified as tubular. But when I train CycleGAN with only the top 1/32 most precancerous TA images, over 90% of the generated images are classified as tubular. That’s an A! So yeah, use this filtering method to help your pal CycleGAN out. The picture below this is further proof for the h8rs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RXJbcI_zIZFKsczTRSYlew.png" /><figcaption>Visual representation of the filtration method: when only using the most precancerous images to train CycleGAN, the resulting images have more precancerous features.</figcaption></figure><p><strong>Quality of Resulting Synthetic Images.</strong> Here’s the heavy hitter — I showed my CycleGAN-generated images to some expert pathologists and they had a hard time telling the difference between the real images and the fake images. I gave 4 pathologists 100 real images and 100 fake images and asked them to classify each image as real or fake. No more than half of the pathologists were able to do so at a statistically significant level. What does this mean? It means that the fake images are similar to real images to the point that the differences are almost indistinguishable. So that’s some cool proof that CycleGAN learned something. Check out the picture below for some more details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HLzouPsh73Vn_XW6g2FIGg.png" /><figcaption>Turing tests with pathologists for tubular adenoma and sessile serrated adenoma images, the two most underrepresented classes in the dataset. No more than half of the pathologists were able to distinguish between real and fake images for both classes.</figcaption></figure><p><strong>Improving Classifier Performance.</strong> Now that you have realistic images of your rare classes, you can augment your dataset! I tossed some generated images into my dataset and trained a ResNet classification model to identify images of sessile serrated adenomas, the smallest class in the original dataset. When I compared the performance of this ResNet to the performance of the ResNet trained using the original dataset, the differences were clear.</p><ul><li>ResNet trained using real data only achieved an AUC of <strong>.78</strong>, similar to ResNets trained using real data and augmented data from DCGAN and DiscoGAN baselines.</li><li>ResNet trained using real data and augmented data from CycleGAN achieved an AUC of <strong>.89</strong>.</li><li>ResNet trained on only synthetic data from CycleGAN outperformed ResNets trained on only synthetic data from DCGAN and only synthetic data from DiscoGAN</li><li>ResNets trained using only synthetic data did not achieve performances comparable to ResNets trained using real data.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8tq2JvmoOIs_dxCSkkMhsA.png" /><figcaption>Performance of ResNets trained using real data and synthetic data (A) and ResNets trained using synthetic data only (B). In both experiments, ResNets trained using CycleGAN’s synthetic images performed the best.</figcaption></figure><p><strong>Closing thoughts.</strong> I’d say this was a pretty cool project (although I am biased). The implications are definitely huge, since class imbalances are a huge problem in machine learning everywhere — being able to use GANs to address this issue would be a huge development. If you have an unbalanced medical dataset and want to improve the performance of your classifiers, I’d definitely suggest that you try this method out. Your results might get much better!</p><p>The full paper for the work can be found <a href="https://ml4health.github.io/2019/pdf/42_ml4h_preprint.pdf">here</a>.</p><p>Source code is available on GitHub at <a href="https://github.com/BMIRDS/HistoGAN">this link</a>.</p><p>If you happen to use these methods, feel free to cite this paper as</p><p>Jerry Wei, Arief Suriawinata, Louis Vaickus, Bing Ren, Xiaoying Liu, Jason Wei, Saeed Hassanpour, “Generative Image Translation for Data Augmentation in Colorectal Histopathology Images”, Full Paper in Machine Learning for Health Workshop at NeurIPS, In Press, 2019.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=565deda07a22" width="1" height="1" alt=""><hr><p><a href="https://medium.com/health-data-science/using-generative-adversarial-networks-gans-for-data-augmentation-in-colorectal-images-565deda07a22">Using Generative Adversarial Networks (GANs) for Data Augmentation in Colorectal Images</a> was originally published in <a href="https://medium.com/health-data-science">Health Data Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Artificial Intelligence Now Diagnoses Lung Cancer as well as Pathologists]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/health-data-science/artificial-intelligence-now-diagnoses-lung-cancer-as-well-as-pathologists-6e8b4a473df?source=rss----e07efed498a6---4"><img src="https://cdn-images-1.medium.com/max/2546/1*UZAH2-y4BfpxhTZbe7mR6A.png" width="2546"></a></p><p class="medium-feed-snippet">Towards AI-assisted healthcare</p><p class="medium-feed-link"><a href="https://medium.com/health-data-science/artificial-intelligence-now-diagnoses-lung-cancer-as-well-as-pathologists-6e8b4a473df?source=rss----e07efed498a6---4">Continue reading on Health Data Science »</a></p></div>]]></description>
            <link>https://medium.com/health-data-science/artificial-intelligence-now-diagnoses-lung-cancer-as-well-as-pathologists-6e8b4a473df?source=rss----e07efed498a6---4</link>
            <guid isPermaLink="false">https://medium.com/p/6e8b4a473df</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[pathology]]></category>
            <category><![CDATA[lung-cancer]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Jason Wei]]></dc:creator>
            <pubDate>Tue, 05 Mar 2019 17:59:05 GMT</pubDate>
            <atom:updated>2019-03-07T01:23:13.539Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Classification of Histopathology Images with Deep Learning: A Practical Guide]]></title>
            <link>https://medium.com/health-data-science/classification-of-histopathology-images-with-deep-learning-a-practical-guide-2e3ffd6d59c5?source=rss----e07efed498a6---4</link>
            <guid isPermaLink="false">https://medium.com/p/2e3ffd6d59c5</guid>
            <category><![CDATA[pathology]]></category>
            <category><![CDATA[healthcare-technology]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[lung-cancer]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Jason Wei]]></dc:creator>
            <pubDate>Sat, 02 Feb 2019 22:37:38 GMT</pubDate>
            <atom:updated>2019-03-04T18:15:07.104Z</atom:updated>
            <content:encoded><![CDATA[<h4>Everything you need to know to train your own classifier on histopathology images.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xuL7hmvpWgqWiOM5a4JDYA.jpeg" /><figcaption>Histopathology images of the lung. Ask me about them. <a href="https://github.com/BMIRDS/deepslide"><strong>[Github code]</strong></a> <strong>[</strong><a href="https://www.nature.com/articles/s41598-019-40041-7"><strong>paper</strong></a><strong>]</strong>.</figcaption></figure><p>Automated classification of high-resolution histopathology slides is one of the most popular yet challenging problems in medical image analysis. The development of deep learning has allowed for accurate classification of <a href="https://arxiv.org/abs/1901.11489">lung cancer</a>, <a href="https://medium.com/@Petuum/deep-learning-for-breast-cancer-identification-from-histopathological-images-f38de0a658a5">breast cancer</a>, <a href="https://sites.google.com/site/miccaibrats2014/">brain tumors</a>, <a href="http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2017;volume=8;issue=1;spage=30;epage=30;aulast=Korbar">colorectal polyps</a>, <a href="https://arxiv.org/abs/1811.08513">esophageal cancer</a>, <a href="https://arxiv.org/abs/1901.11447">celiac disease</a>, and <a href="http://www.cancerimagingarchive.net/">so much more</a>. How exactly do these algorithms work, and how do you build one?</p><p>For a quick implementation, see the <a href="https://github.com/BMIRDS/deepslide">DeepSlide Github repository</a>, which has been optimized for histopathology image analysis:</p><p><a href="https://github.com/BMIRDS/deepslide">BMIRDS/deepslide</a></p><h3>Deep Learning &amp; Convolutional Neural Nets</h3><p>Recently, deep learning has produced a set of image analysis techniques that automatically extract relevant features, transforming the field of computer vision. <a href="https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53">Convolutional neural networks</a> (CNNs) use a data-driven approach to automatically learn feature representations for images, achieving super-human performance on benchmark image classification datasets such as ImageNet. CNNs are the current state-of-the-art architecture for medical image analysis. More info on deep learning and CNNs: <a href="http://neuralnetworksanddeeplearning.com/">[deep learning book]</a> [<a href="https://www.coursera.org/specializations/deep-learning">coursera</a>].</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vkQ0hXDaQv57sALXAJquxA.jpeg" /><figcaption>Example of a convolutional neural net. <a href="https://www.mathworks.com/solutions/deep-learning/convolutional-neural-network.html">[source]</a></figcaption></figure><h3>Challenges</h3><p>Because of the nature of histopathology images, there are two main challenges:</p><ol><li>The images in benchmark datasets that CNNs do well on are low resolution (something like 224x224 pixels). Histopathology images, on the other hand, are for pathologists to examine under the microscope, so they tend to be <em>extremely</em> high resolution (sometimes 100,000x100,000 pixels and 10GB+). We can’t train directly on whole-slide images since their resolution is too high.</li><li>Benchmark datasets have hundreds of thousands of images for training. In histopathology slide analysis we don’t have that many images, and we can’t just collect more data by downloading images from the internet. So we have to figure out a way to train a generalizable network without overfitting on the training set.</li></ol><h3>Sliding Window Approach</h3><p>The sliding window approach is the simplest (and currently standard) way of dealing with both of these issues. The idea is the following:</p><blockquote>At training time, we slide a 224x224 window across the entire whole-slide image to generate small patches. We label these patches with the original label of the whole-slide image (or bounding box labels if you have them), and train a CNN to classify patches.</blockquote><p>By generating patches, you now have manageable-sized inputs for training a CNN. At the same time, this helps prevent overfitting because you’ve generated many unique patches from each whole-slide image.</p><p>You’ve now solved both main challenges!</p><h3>Whole-Slide Inference</h3><p>You can predict the labels of small patches, but ultimately you will need a way of aggregating these patch predictions to predict a label at the whole-slide level.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ynb2sFK10FgYvBdjwKCxEg.jpeg" /><figcaption>Sliding window framework for histopathology image classification.</figcaption></figure><p>There are multiple ways of aggregating patch predictions. Here they are, in order from complexity:</p><ol><li><strong>Majority voting.</strong> Take the most frequently predicted class.</li><li><strong>Patch averages.</strong> Average the probabilities of all patch predictions, and take the class with the highest probability.</li><li><strong>Thresholding.</strong> Throw away predictions of low confidence. Then, only predict certain whole-slide labels if the number of patch predictions hit some other threshold. Otherwise, predict normal. This is ideal for when you have normal slides mixed in, and if you have multi-label slides.</li><li><strong>Machine learning. </strong>Feed your patch predictions into some machine learning model like support vector machines, random forest, or logistic regression. Only try this approach if you have &gt;1,000 whole-slide images for training, otherwise you may have problems with overfitting.</li><li><strong>Attention-Based Classification. </strong>Train an end-to-end model to take in a whole-slide image and slide some feature extractor with an attention module to identify the most important patches. This approach is the most complex but also the most scalable, plus it works without bounding box annotations in some cases where you would have otherwise need them. Read more in <a href="https://arxiv.org/abs/1811.08513">[this paper]</a>.</li></ol><h3>Evaluation</h3><p>Congratulations! You’ve now trained a model for classification of whole-slide histopathology images. Time for evaluation on the testing set. Calculate metrics such as accuracy, precision, recall, and <a href="https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html">F1 score</a> per class. You can also plot <a href="https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html">receiver operating characteristic</a> (ROC) curves and calculate area under the curve (AUC). Do you have pathologist’s annotations in addition to ground truth labels? Compare your model’s performance to that of pathologists.</p><p>Visualization of your model’s predictions can be a good way to confirm your model’s predictions and also help debug your model to improve performance. You can try both a whole-slide level visualization of patches as well as <a href="https://jacobgil.github.io/deeplearning/class-activation-maps">CAM</a> or <a href="https://github.com/marcotcr/lime">LIME</a> for patch-level visualizations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ReDYR2wG8Y9KQPZCmSTeRg.jpeg" /><figcaption>Visualization of my deep learning model’s predictions of histologic subtypes in lung cancer.</figcaption></figure><p>My code, which has been optimized for histopathology image classification, is easy to use and can be found here:</p><p><a href="https://github.com/BMIRDS/deepslide">BMIRDS/deepslide</a></p><h3><strong>Frequently Asked Questions</strong></h3><ul><li><strong><em>I have data but I’ve never coded a neural network from scratch. How do I start? </em></strong>Great question! You can pull my PyTorch <a href="https://github.com/BMIRDS/deepslide"><strong>code</strong></a> from Github, which is optimized for histopathology image classification, and use it as a baseline. Then, work on patch aggregation methods or your own custom implementation.</li><li><strong><em>How deep should my neural network be?</em></strong> I encourage you to try something deeper than AlexNet and VGG like <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py">ResNet</a> or <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/inception.py">InceptionNet</a>. Deeper networks have achieved higher accuracies on the 1000 classes for ImageNet, but since histopathology images are mostly purple and look pretty similar, you probably don’t need that many learnable parameters. ResNet-18 has worked pretty well for most of the tasks that I’ve worked on.</li><li><strong><em>How many whole-slide images do I need?</em></strong> Obviously more is better, but I think if you have something like 10–20 whole-slide images per class with a decent area of classifiable tissue, you can get decent accuracy after extracting patches and doing data augmentation.</li><li><strong><em>What resolution should my images be?</em></strong> I would recommend something in the range of (the lowest resolution that is still readable) to (twice that resolution). ~5MB per whole-slide image has worked pretty well for me. If your resolution is too high, you’ll be dealing with hefty computation times and memory issues, and patches may not be classifiable if they are too zoomed in. On the converse, if your resolution is too low, then the images aren’t readable.</li><li><strong><em>How do I deal with class imbalances?</em></strong> If you’re using my <a href="https://github.com/BMIRDS/deepslide">code</a>, patches from less frequent classes are extracted with a lower stride and then automatically duplicated to create perfectly balanced classes. Another solution would be to write a custom loss function that has a higher penalty for less frequent classes.</li><li><strong><em>Do I need bounding box annotations from pathologists?</em></strong> It depends on the homogeneity of your classification problem. For example, in celiac disease detection, any given patch of a whole-slide image can be classified as having celiac disease, so you would not need bounding annotations. In this case you would not need bounding box annotations. On the other hand, in lung cancer classification, each image has multiple classes as well as normal tissue, so any given patch is not that likely to have the same label as its whole-slide image. You would need bounding boxes in this case. I would suggest trying it without bounding boxes first, and if the results are poor then go back and find a way to get bounding boxes.</li><li><strong><em>What should I do with all the white space on the whole-slide image? </em></strong>Please do not train on white space. You can write a simple image processing function to filter out these images, or just take the one I have written <a href="https://github.com/BMIRDS/deepslide/blob/master/code/utils_processing.py">[here]</a> for H&amp;E stained slides (purple images).</li></ul><h3>Want more?</h3><ul><li>If you haven’t yet, try out the <a href="https://github.com/BMIRDS/deepslide">DeepSlide Github repository</a>. Feel free to post in the “issues” section and someone will get back to you.</li><li>Read <a href="https://www.nature.com/articles/s41598-019-40041-7">my paper</a> on pathologist-level lung cancer classification.</li><li><a href="https://www.hassanpourlab.com/publications/">Dr. Saeed Hassanpour</a> from Dartmouth College does incredible work in image classification for both histopathology and radiology images.</li><li>My personal website is below. Feel free to reach out by email :)</li></ul><p><a href="https://jasonwei20.github.io/">About Me</a></p><blockquote>Thank you to my supervisor, Saeed Hassanpour, for advising me on the content and methodology in this guide.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e3ffd6d59c5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/health-data-science/classification-of-histopathology-images-with-deep-learning-a-practical-guide-2e3ffd6d59c5">Classification of Histopathology Images with Deep Learning: A Practical Guide</a> was originally published in <a href="https://medium.com/health-data-science">Health Data Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>