My master’s research project on SVM
My master’s research project focused on leveraging hypergraphs in machine learning (ML) to develop novel frameworks for classification and multi-label learning in semi-supervised settings. The project addressed significant applications such as text categorization, recommendation systems, and bioinformatics. Under the guidance of Dr. Reshma Rastogi at South Asian University, we initiated this project in August 2022. We successfully concluded it in December 2024, resulting in two research publications at the International Conference on Pattern Recognition (ICPR) 2024, Kolkata.
The project began with an extensive literature review to identify gaps in existing methods for multi-label learning. While standard graph-based methods captured binary relationships, they often overlooked higher-order associations inherent in real-world data. Hypergraphs offered a promising avenue to address this limitation. However, hypergraph methods also posed challenges and difficulty in balancing binary and higher-order relationships. These gaps shaped our research direction.
For the first phase, we aimed to simplify the multi-label learning problem by converting it into independent binary classification problems. TSVM (Twin Support Vector Machine) was chosen for its computational efficiency, operating three times faster than the traditional SVM proposed by Vapnik. We introduced additional terms into the optimization function to enhance robustness: one capturing label correlations and another leveraging hypergraph-based locality information. We used the least square loss function for its simplicity and easeness. My process involved iteratively experimenting with benchmark datasets. I optimized the model using accelerated proximal gradient descent and analyzed metrics like Hamming loss, F1 score, AUC, Average Precision and ranking loss. These iterative experiments were instrumental in fine-tuning the framework for better accuracy using grid search.
In the second phase, we identified a conceptual gap: while graph Laplacians captured binary relationships, they were inadequate for higher-order associations, which hypergraph Laplacians could address. After brainstorming and theoretical modeling, we proposed a linear combination of the two to balance these properties. This led to the development of IHLSVM, which employed manifold regularization and a square hinge-loss function. Using Newton’s method, the decision to solve the quadratic programming problem in primal space stemmed from a need for computational efficiency without sacrificing accuracy. We validated the approach through rigorous experimentation on datasets from the UCI ML repository, MNIST Fashion, and multi-label benchmarks, achieving improved performance.
Get Dev Nirwal’s stories in your inbox
Join Medium for free to get updates from this writer.
Throughout the project, challenges such as tuning hyperparameters, handling imbalanced datasets, and ensuring computational efficiency required iterative problem-solving. I often brainstormed with my supervisor, tested hypotheses, and refined the algorithms based on experimental outcomes. Additionally, conducting the Nemenyi test for statistical significance was critical to establishing the superiority of our methods.
My contributions spanned formulating optimization problems, identifying research gaps, coding algorithms, conducting experiments, and analyzing results in MATLAB and Python. The experience honed my ability to think critically, iterate on solutions, and translate theoretical insights into practical frameworks. Furthermore, an additional paper on imbalanced multi-label learning, completed under the same project, was accepted at CODS-COMAD 2024, showcasing the project’s broader impact.
Research Output:
DOI: https://doi.org/10.1007/978-3-031-78383-8_15
DOI: https://doi.org/10.1007/978-3-031-78192-6_16
