To anyone who is reading… or to myself when I read this in the future, Hello! I am penning down my personal experience and research during my ACL reconstruction.

Knocking out my ACL!

My trip to my alma mater for an alumni basketball match had both bitter and sweet outcomes. Though we won the game, towards the weekend, while playing with friends and college juniors on 9th October, I twisted my right knee two times. My right shoulder also started hurting during the same weekend. Initially, there wasn’t pain but I took precautions in not moving the knee too much.

After a few days, I visited Dr Navin Kumar in Bangalore at the Sports Orthopedic Clinic in HSR Layout. He recommended physical therapy and some shoulder exercises for my supraspinatus tendonitis i.e. my impinged shoulder. For the knee, I was recommended to do an MRI and then re-consult again with the diagnostic report.

My MRI was done on the night of 15th October at Ebisu Diagnostic Centre in HSR Layout which confirmed the tear in my Anterior Cruciate Ligament (ACL). I had almost a complete tear with a few fibres intact, however, I did not have any clinical abnormalities or pain.

Hello doctors, what’s next?

Over the next few weeks, I met with a couple of doctors in Bangalore to evaluate my options further.

First, I went back to Dr Navin Kumar who looked at the MRI image film, and the diagnostic report and recommended ACL reconstruction surgery since the tear was more than 50%. He informed me about the physiology of the knee joint and mentioned that the ACL does not grow back itself. Being young, he suggested that ACL reconstruction has a good chance that I could go back to playing in 9–12 months' time.

On my aunt’s recommendation, I went to another orthopaedic hospital near Mattikere. There the doctor examined the injured knee before looking at the MRI film after which their prognosis about the injury was similar to my last consultation. This was a rather useful visit since the doctor told me about the process during the surgery, where they place the second hamstring tendon in the knee cavity in place of the actual ACL. The cost that the doctor mentioned was ₹1,80,000/- for the entire 2-day operation.

The Surgery Decision: Who and Where?

After this consult, I had to look carefully at my options, and my health insurance and plan who and where the surgery might happen.

On 16th November, while I was visiting my folks in Greater Noida, we got an appointment with Dr Prateek Gupta at A+ Sports Orthopaedic Centre in Safdarjung Enclave. Although I was clear that going under the arthroscope for reconstructing my torn ligament was my only option, the doctor made it quite clear that getting operated sooner than later was better for my knee recovery. Waiting and repeated use of my injured knee was likely to injure my meniscus even more leading to a longer post-surgery recovery period.

Dr Prateek and his team would perform the surgery at Sir Ganga Ram Hospital and explained all the commercials as well as provided his availability for us to decide. The possible dates were much sooner than what I was prepared for — 28th November. At that moment, I was not sure whether the cost of the procedure and the hospital would be covered by my insurance. However, getting operated by a doctor whom my parents trusted was a moot discussion.

Now, I needed to get my insurance in order. At Wise, we have employee insurance from Plum. A quick lookup on their website for Network hospitals as well as a response from their customer service about my cashless hospitalisation query confirmed that Sir Ganga Ram was a network hospital. Also, the insurance agent at A+ Sports Orthopaedic Center asked me to visit on the 24th of November with my documents to start the process for cashless insurance.

What now?

Today, there are 7 days to go! I am figuring out how to perform some physical therapy for my shoulder and knee before undergoing the operation.

Henceforth, we began with a vision to personalise the products for each user shown in a list on the 1mg mobile app homepage. We wanted to build a recommendation system that can leverage all users’ short and long term views and purchase histories and predict what they would like in future.

The Idea was ‘Collaboration’

Collaborative filtering models are fundamental to recommendation systems which are based on assumption that people like things similar to other things they like, and things that are liked by other people with similar tastes.

The underlying assumption of the collaborative filtering approach is that if a person A has the same opinion as a person B on an issue, A is more likely to have B’s opinion on a different issue than that of a randomly chosen person

- Wikipedia

CF is a general concept and there are several algorithms to implement it. One of the popular approaches is based on low-dimensional factor models (model-based matrix factorisation).

The idea behind matrix factorisation models is that the preferences of a user can be determined by a set of hidden factors. We can call these factors Embeddings. Intuitively, we can understand embeddings as low dimensional hidden factors for products and users. Each element in the product embedding vector could represent different characteristics about the product and likewise, the elements in the user embedding matrix might represent different characteristics about the user. A higher number from the dot product of user-X and product-A matrix means that product-A is a good recommendation for user-X.

Collecting Clickstream Data

Our internal data pipeline collects and stores request-response logs for all API calls in compressed hourly files in our data warehouse in Amazon S3. This way we have been able to gather users’ clickstream activity on the Android and Web. These files are read and processed using Spark deployed on small clusters and then stored back in a columnar data format which serves as our Tracking Data Store.

Matrix Factorisation — Implicit Preferences and ALS

A typical user might specify their preference for a product by giving it a high rating if they like it or a low rating if they dislike some aspect of the product. Collaborative filtering models are typically built using explicit preferences, where available. We had no explicit ratings for products and hence, we had to look for alternative methods to assess a user’s preference towards a product. In some ways, a user’s purchase of a product can also be seen as an implicit indicator of their preference. We started with a fairly simple implicit preference function which has become more nuanced over time (takes weighted sums of a user’s views/add to carts/purchases of products). A discerning reader can tell that preference functions can be constantly tweaked and improved.

Matrix Factorisation works on a dataset with users, products and users’ preferences for the product. We used our data pipeline as described above, to find which user interacted with which product and extracted aggregated interactions for each user-product combination for a period of nine months.

An out-of-the-box implementation of the Alternating Least Squares (ALS) technique in the MLlib library in Apache Spark has been used widely for solving CF problems. Using a three instance cluster with Spark and Spark’s high-level API in Python, we trained an ALS model with implicit ratings. We evaluated the model by measuring the root-mean-square error of rating prediction.

Ensemble Models

ALS has a number of hyperparameters like dimensions of latent factors, iterations, regularisation etc. which can be tuned to improve the evaluation metric. Through our experiments, we found that an ensemble of matrix factorisation models (labelled as MF — Bagging) was better than individual models.

Deep Learning

The exploration of deep neural networks on recommender systems has received relatively less scrutiny. We came across Neural Collaborative Filtering (NCF) where we trained a Generalised MF and Multi-Layer Perceptron to learn separate embeddings, and combined the two models by concatenating their last hidden layer. This model combines the linearity of MF and the non-linearity of DNNs for modelling user-item latent structures. We evaluated the NCF model using Hit Ratio (No. of correct predictions / No. of total predictions) and Normalised Discounted Cumulative Gain (nDCG).

Personalisation in Production

The final output of the modelling exercise was 20 product recommendations for all the users in the dataset. We pushed the new list into production using an AB Framework and appropriate fallbacks to the static list. Both the metrics that we used to track — clicks through rate (CTR) and conversion in same session, jumped up. These initial results were very promising and have pushed us further into improving the personalisation that we have been able to achieve.

Growth of Orders through the List from Oct ’17 to Feb ‘18

As we moved ahead, some new questions came up

• Were users satisfied with the list, or in other words, were users clicking the recommendations that appear first in the list more often now?
• Were users able to discover more items using this list?
• Was the list decaying with time?
• Were the recommendations changing when we periodically recomputed the recommendations?

Deep Diving into Evaluation

These questions forced us to identify better and more appropriate metrics that would answer these questions and more. Below, we describe some metrics we were able to calculate.

Positions of Products in the List

We took a course on metrics for recommender systems and came up with two metrics that could tell us more about the positions where the clicks were happening -

• Mean Reciprocal Rank (MRR) — Reciprocal rank is defined as the inverse of the rank of the first item in the list with which the user interacts (clicks/purchase).
• Discounted Cumulative Gain (DCG) — looks at the utility of the item at each position in the list and discounts this utility or gain by position in the list. So we place more emphasis on the items at the top of the list and then we normalise by comparing the discounted cumulative gain of the list the recommender produced versus the discounted cumulative gain of a perfect recommendation list.

Do the Recommendations Change with Time?

Lathia’s Diversity talks about How temporal diversity is an important factor in recommender systems. How do the recommendations change when new data is introduced with respect to time.

Lathia’s Diversity Histogram for ~500K users
Over two recommendation lists computed at the difference of one month
(0 means no change in list and 1 means a completely new list)

From a Business Standpoint

• Revenue Per Mille (RPM) could tell us how the price value of converted items from the list is varying
• Conversions Per Mille (CPM) could tell us how the number of converted items from the list is varying

Do the Clicks on the List Decay with Time?

We analysed the New Users vs Old Users who click on the list on daily basis. We observed that the list was not decaying as there was a set of new users who had never interacted with the list clicking on the products in the list.

What about Cold Start?

Collaborative filtering does not provide any recommendations for new users with less or no tracking history and also does not include new products in the recommendations for users. In our case, we show a static list of products where we do not have any recommendations from the CF model. This problem is popularly known as the Cold Start Problem. One of the ways it can be addressed is by relying on content-based vectors for products and profile-based vectors for users and using them alongside the embedding vectors in the model to learn the preferences of all users for all products.

The Future still holds promise

As we start to dwell further on personalising product recommendations for users, we have many exciting paths to tread. Working out better solutions for tackling implicit ratings for products, addressing cold start for products and users and updating recommendations in real time could also lead to better click-through and conversion and thereby better personalisation.

One of the biggest problems of healthcare in India is the lack of digital prescriptions. Remember your last doctor’s visit? Most likely you walked out with a paper prescription. Even though digital health records enable critical healthcare delivery functions such as decision support, improved care management, adherence, etc. their prevalence in India continues to be dismal.

At 1mg, we process a large number of prescription orders every day. These typically come attached with a picture of the prescription. A central feature of 1mg’s order processing system is our digitization system wherein a trained pharmacist codifies the attached prescription image using a smart software system. Once digitized, the system decides eligibility and duration for dispensation of medicines and also quantities that can be dispensed. The system is built to maximize patient safety and eliminate dispensation errors.

Given that 1mg handles tens of thousands of orders each day, and the core nature of the digitization system, we have always been keen to improve automation in the digitization system without compromising safety OR accuracy. This article looks at the challenges in creating ‘a digital prescription ecosystem’, and our approach which does not require any change in a doctor’s behavior.

Challenges in creating a Digital Prescription Ecosystem

Urban India generates an estimated 3 billion prescriptions each year. And almost all of them are on paper! A typical medical prescription contains doctor and patient identifiers, diagnosis, tests required, medication, dosage, and other details. It is in an unstructured format and is mostly written in individual handwriting styles on letterheads. Challenges in digitizing the large swath of prescriptions include:

Firstly, doctors in India deal with high patient loads and as a result low consultation time per patient in most hospitals and clinics. This is largely due to a low number of doctors present per 1000 patients in India. This means that doctors spend a very small amount of time with each patient and cannot afford to type prescriptions.

Secondly, the process of manually digitizing the prescription image is time-consuming. A typical prescription has 4–6 medicines and takes a pharmacist some time to digitize as it involves extracting the doctor and patient identifiers, appointment or visit dates, and medicines along with their strengths and dosages.

Thirdly, the hard-to-understand handwriting of doctors adds to the complexity of the process by making it time-consuming and error-prone.

Early Approaches for Medicine Suggestions

As we understand from the above discussion, the bottleneck in speeding up the digitization process is finding, typing, and searching for medicines from a prescription image. Accurate medicine suggestions to the pharmacist digitizing the prescription can circumvent this activity and make the process of adding medicines faster. To achieve this, we initially resorted to the following techniques.

Optical Character Recognition (OCR)

We thought that generating medicine suggestions was possible simply by detecting the text in a prescription. A widely used algorithm to detect text in documents that we used was Optical Character Recognition (OCR), in which we solved a pattern recognition problem to classify individual characters or symbols. To perform OCR on a document, we cropped the image into multiple word images. Each of these images was then split horizontally into symbols and each symbol was classified to a letter in the English alphabet. OCR provided by Google Vision APIs gave reliable results for computerized text and legible handwriting. Once the text was extracted, we matched it with medicine brands from our product catalog using various fuzzy matching techniques to get relevant suggestions. This worked well for some computer-generated prescriptions with printed medicines but failed miserably for a majority of prescriptions where the medicine names were handwritten. The extracted text after performing OCR on prescription was extremely inaccurate which either could not be matched to any medicine or matched with wrong brands and gave false positives.

Multi-Instance Learning (MIL)

Our manual digitization process generates weakly annotated data, where the region corresponding to each medicine is not explicitly annotated and only the entire image is associated with all medicines found in the image. Even though we could extract word images, matching them to a medicine brand was challenging. To overcome this issue, we employed a Multi-Instance Learning (MIL) approach.

Attention-Based Multi-Instance Learning Model
used for predicting medicine brand labels on a bag of word images

Attention-based Multi-Instance Learning is a variation of supervised learning where a single class label is assigned to a bag of instances. In other words, our hypothesis was that given enough prescriptions containing a particular medicine brand, a MIL model would be able to identify which words have a similar pattern and thus, correspond to the particular medicine brand. Even though these set of experiments showed slightly better results, the accuracy of medicine suggestions was not good enough for deployment in production forcing us to go a step further.

Present Approaches using Labeled Prescriptions

As we moved further into our investigation to build better and more accurate models for suggesting medicine brands and doing more, we recognized that multiple bounding boxes and labels on prescription images like those for medicine brands, doctor credentials, appointment dates were necessary. There are a few tools that are used to annotate images, however, none of them provided essential features like the ability to rotate images. Moreover, they could only be used on local machines rather than be deployed on cloud instances. These constraints forced us to build our own image annotation tool to mark regions in a prescription and label them appropriately.

Image Annotation Tool to label prescriptions built in-house

Once we started generating annotations on prescription images, it opened up a plethora of avenues:

• Object Detection using convolutional neural networks (CNN) was one of the approaches we used. It helped detect instances of semantic objects of a certain class in digital images without specifically defining features. You only look once (YOLO) is a state-of-the-art, real-time object detection system which applies a single neural network informed by the global context in the image, making it extremely fast and accurate. We trained multiple YOLO models using labeled annotations on prescriptions to segment various regions in the image like bounding boxes for doctor details, patient name, medicines and medicine brands.

YOLO Model for Detecting Regions in a Prescription

• Word level handwriting classification model was also one of the techniques which trained on labeled word images where labels corresponded to medicine brands. It helped us extract medication information from prescription images and infer medicine brand labels among all the word images found in a prescription.

Word Level Classification Model to detect Medicine Brands

Taking Help of Intelligent Heuristics

In India, it is a common practice among doctors to prescribe branded generics since they have come to trust these companies and their brands for manufacturing quality drugs over time. Also, Indian branded generic companies have been innovative in terms of drug delivery systems to improve absorption, reduce side effects, thereby increasing the efficacy of the drug. This has led to an extremely large catalog of medicines that are found in medical prescriptions causing problems for any algorithmic approach focused on finding the medicines in the prescription. This is because classifying bounding boxes on text found in images or matching the extracted text to such a huge number of unique medicines is not feasible.

Simple heuristics and rule-based machine learning help in reducing the search universe for each prescription image.

• Finding the Doctor or their Specialisation — Even though there are a large number of medicines that can be prescribed, a single doctor typically specializes in one of 18–20 domains of medicine like cardiology, gynecology, etc., and usually prescribe medicines in only a few therapeutic and pharmacological classes. A valid prescription image always contains the doctor’s credentials along with their specialization which can be used to reduce the candidate set for searching for medicines.
• Association rule mining is a data mining process of finding the rules that may govern associations and causal objects between sets of items. We leverage a large number of digitized prescriptions by 1mg to observe frequent patterns of co-prescribed medicines to generate accurate suggestions as soon as a single medicine is added.

It is 3.7 times more likely that ‘Duphaston’ occurs in a prescription if ‘Ecosprin’ is present, than otherwise

Towards a Faster and Scalable Rx Digitization

At 1mg, we have developed models to classify valid prescription images, segment regions in the image where medicines are written, extract bounding boxes for brand and drug keywords and build hierarchical classifiers to identify medicines. This has enabled us to set up a process to obtain relevant medicine suggestions that go beyond what can be provided by using the best OCR technologies leading to speeding up the prescription digitization process with much higher levels of automation.

References

https://economictimes.indiatimes.com/industry/healthcare/biotech/healthcare/india-facing-shortage-of-600000-doctors-2-million-nurses-study/articleshow/68875822.cms

UPSC Live is an open platform where educators start building attractive and engaging online classrooms for any subject they want to teach. The UPSC Live platform is open to teachers teaching in any language, currently, we have Hindi and English educators taking more than 100 live sessions each week.

One-Stop Open Classrooms

On UPSC Live, these online classrooms are openly available to students to subscribe to. The content in these classrooms is completely owned and run by each educator with help from the UPSC Live team. Each educator has complete autonomy on how they run these online classrooms. They are free to choose what topics they teach, how they teach them, and when they conduct their classrooms. Within their online classrooms,

• Educators can take unlimited Live Sessions with as many participants as they would like. Students can openly subscribe and join any number of Live Sessions with any teacher of their liking
• These sessions are easily recorded and made available for their students as Class Recordings for those who weren’t able to attend the Live Session
• Educators also take MCQ tests which are best for preparation of Prelims examination
• Study Materials including Web Links, YouTube Videos, PDFs for live sessions are shared in the classrooms in the Resources section
• Many educators teaching for Civil Services Mains examination give Answer Writing assessments which they evaluate and share detailed feedback
• Last but not the least, UPSC Live classrooms are a place to have lively Discussions where all students can share their ideas and thoughts through comments

These features have made quite a few educators shift their student audience from other social media platforms like Telegram, Youtube to their UPSC Live open classrooms.

Success means Marketing-Managing-Monetizing your Classrooms!

A successful educator is one who is able to Market their classrooms, Manage the engagement of students and finally Monetise different aspects of their teaching like app subscriptions, partial or full courses, regular complete or summarised study material, conduct test series, live interaction with educators through their scheduled sessions or through helping students in answer writing evaluation.

Many online educational institutes spend a lot of money to run ads for their websites, apps, and other channels. Many also follow more coercive tactics as soon as they identify a potential student like relentlessly calling students and their guardians pursuing and soliciting them for paid offerings. For a lot of smaller institutes and individual educators, it becomes a time-consuming and costly activity to market, manage and then monetize their classrooms.

Students want to talk to their teacher!

The team at UPSC Live is focused on building an engaging platform for students and teachers to interact and engage with each other.

Student-Teacher interaction on YouTube is one-way, students can only comment. On UPSC Live, Students directly talk to teachers during their Live Sessions

On YouTube, students can see teachers teaching. But on UPSC Live, students directly talk to teachers during their Live Sessions. Nothing can convince a student more than talking to an actual teacher asking them to solve their doubts. One conversation with a teacher can have a tremendous effect on a student to improve their knowledge, skills, and aptitude. Students want to give a test from teachers rather than choosing from 1000s of tests on an app! This is what UPSC Live makes possible, direct Student-Teacher interaction.

Serious Students can be taught privately!

All educators know, free classes attract students! That’s why YouTube has so many teachers taking classes for almost everything. But only serious students would end up studying diligently. Educators pull serious students from their free open classrooms into their paid private classrooms where they run the entire curriculum for making sure that the student’s preparation is top-notch.

On UPSC Live, Educators have complete autonomy and freedom to run private classrooms and manage fees

The educator is free to set fees for their private classrooms and gets no deadlines or interference for running their classrooms in any particular way. Many teachers just give tests or share study material regularly in their open classrooms. Educators have complete autonomy and freedom in their online classrooms on UPSC Live.

Teachers get New Students on UPSC Live!

On the platform, UPSC Live, educators are not teaching alone. They are teaching along with a group of educators specific to competitive examinations conducted by Union Public Services Commission like Civil Services Examination, State Public Service Commission exams, National Defence Academy exams, State Staff Commission exams, and more.

Educators on UPSC Live are not alone, they teach along with other educators running their open classrooms.

Now, when the UPSC Live app is marketed to students looking to pursue a career in civil services, they are able to find good educators and mentors to help them through their preparation. Students subscribe and engage with classrooms maintained and managed by multiple educators for different subjects and topics. Once they subscribe to their classrooms, students are notified about happenings inside the classroom like live sessions, MCQ tests, answer writing assessments, and more from the educators. They can ask doubts through discussions and in live sessions. Basically, these students are able to find educators that they can stay in touch with through their online UPSC Live classrooms.

Our progress, so far …

We started UPSC Live a week after the 2021 Prelims examination around mid-October and it has been almost two months since we have been working on architecting this platform to suit both teachers and students. In the past few weeks, we have been able to help educators, who are teaching different subjects and taking regular live sessions, get more than 500 students in each of their classrooms. Educators are extremely pleased with the way the platform helps them interact with the students and actually indulge in teaching their subject properly, and primarily as they wish and as their students want them to teach.

If you are a teacher and want to start your classes on UPSC Live today, please fill the form using this link and we will get back to you as quickly as we can.

If you are a student preparing for Civil Services, try out the UPSC Live app 📱 by downloading the app from the Google Play store by clicking this link and subscribing to the classrooms that you are looking for.

Share your thoughts in comments 💬 or clap for the article 👏