Supercharge Your Recruitment Process:

A Deep Dive into Automated Resume Screening with RAG Fusion and LlamaParse

Introduction

Manually screening resumes is a common yet inefficient practice in recruitment. HR professionals invest countless hours poring over resumes, a process that is often inconsistent, resource-heavy, and unscalable. Add the challenge of diverse resume formats, and the task becomes even more daunting.

The Solution? Automating resume screening with advanced tools like LlamaParse, RAG Fusion, and an intelligent Chatbot. Together, these technologies transform how resumes are matched to job descriptions, ensuring speed, accuracy, and scalability in recruitment.

Defining the Tools

LlamaParse

LlamaParse is an AI-powered document parsing technology built for handling complex documents such as resumes, which often include tables, graphs, and different layouts. It supports a wide range of file types like PDFs, DOCX, and EPUB, and it converts them into easy-to-read formats such as markdown or text. This parsed data can then be fed into a retrieval system, such as a RAG (Retrieval-Augmented Generation) pipeline, allowing for efficient information extraction and further processing. LlamaParse also supports custom parsing instructions, enabling pre-processing tasks like sentiment analysis or summarization, and outputs data in formats like JSON for easy integration with other systems.

For a full rundown on LlamaParse check out this blog from LlamaIndex.

To get started, visit this page and click on login. Then go to the API key section:

This service is available in a public preview mode: available to everyone, but with a usage limit (1k pages per day) with 7,000 free pages per week. Then $0.003 per page ($3 per 1,000 pages).It operates as a standalone service that can also be plugged into the managed ingestion and retrieval API

Groq

Groq, an AI technology startup, brings high-performance language processing capabilities to the table. With its unique Language Processing Unit (LPU), Groq accelerates the processing of Large Language Models (LLMs) by overcoming bottlenecks associated with traditional GPUs and CPUs. The LPU’s architecture allows for faster generation of text, enabling near real-time responses in AI-driven applications like resume parsing and chatbot systems. Groq’s performance has been proven, with the system handling over 300 tokens per second using Meta AI’s Llama-2 70B model.

Getting Started with Groq

Right now, Groq is providing free-to-use API endpoints to the Large Language Models running on the Groq LPU — Language Processing Unit.

Groq guarantees to beat any published price per million tokens by published providers of the equivalent listed models. Other models, such as Mistral and CodeLlama, are available for specific customer requests.

To get started, visit this page and click on login. The page looks like the one below:

AI Agent

LLM-powered agents, while not universally defined, can generally be described as systems that leverage large language models (LLMs) to reason through problems, develop plans for solving them, and execute those plans using a set of integrated tools. In essence, these agents combine advanced reasoning abilities, memory functions, and task execution capabilities.

The concept of such agents first emerged in projects like AutoGPT and BabyAGI, where they demonstrated the ability to solve complex problems with minimal human intervention. To provide a clearer understanding, here’s an overview of the typical architecture of an LLM-powered agent system.

An LLM-powered agent is built around several key components:

1. Agent Core: This central module manages decision-making and coordinates the agent’s behavior. It includes:
   General Goals: The agent’s objectives.
   Tools for Execution: A set of tools the agent can use.
   Planning Modules: Guidance for different situations.
   Relevant Memory: Retrieves past interactions relevant to current queries.
   Persona (optional): Customizes the agent’s responses and preferences.
2. Memory Module: The agent uses two types of memory:
   Short-term Memory: Logs interactions related to the current task.
   Long-term Memory: Stores past interactions for future reference using semantic similarity.
3. Tools: Predefined workflows that allow the agent to execute specific tasks.
4. Planning Module: For complex tasks, the agent employs:
   Decomposition: Breaks down complex questions.
   Critique Methods: Improves reasoning and accuracy using techniques like ReAct and Chain of Thought.

RAG Fusion

In many cases, humans are not efficient and accurate at writing prompts to LLMs. This may result in multifaceted and ambiguous queries to the retriever and the LLM generator. This issue is especially common in the recruitment context where job requirements are often composed of many different aspects. Simple RAG systems such as Naive RAG may further amplify this problem if the retriever searches for documents similar to the poor query to feed the LLM, leading to a less accurate and relevant response.

Get Wissem Hammoudi’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

RAG Fusion is a novel technique designed to address this limitation. The key idea is to first deconstruct the original input queries into smaller but more focused sub-queries. For each of these queries (including the original one), the retriever then performs similarity-based retrieval to obtain a relevant document list. A specialized algorithm is then applied to combine and re-rank these lists based on their ranking across multiple lists. The final result is a new ranked list representing the most relevant documents to the original complex query. This figure illustrates an overview of this process.

The central process of the RAG Fusion framework is its re-ranking mechanism. This re-ranker is responsible for fusing multiple lists of documents into one ranked list so that every new rank reflects the original rankings of documents. To achieve this, the re-ranker utilizes an algorithm called Reciprocal Rank Fusion (RRF), an unsupervised way to effectively re-rank the results of multiple ranked lists with only a few simple initial conditions. Given a set D of documents to be ranked, a set of rankings R, and a constant k to indicate the influence of the current rank on the final score, RRF sorts each document d according to the scoring formula:

The main objective of RRF is to give a higher final ranking to those near the top of more lists. It is clear to observe that the scoring formula closely reflects this goal. For each item, the algorithm computes the inverse rank scores obtained from each list, so that a lower r(d) value, which represents a higher rank, results in a higher individual score. By summing all of the scores from each list, it is possible to produce a combined score for the final ranking that reflects every individual ranking score.

Réalisation

Resume parser

Initially, I used the LlamaParse API to extract text from resumes.

This llm agent focuses on extracting essential details such as the candidate’s name, contact information, skills, education, work experience, and more.

To achieve accurate extraction results, we crafted a detailed message that guides the LLM in identifying and structuring the required information. The messages variable consists of a list of dictionaries representing the conversation between the system and the user. The system message outlines the extraction task, specifying the types of information to be retrieved from the resume text, such as the candidate’s name, contact details, skills, and education. The user message contains the actual resume text for analysis. This approach ensures that the LLM receives clear instructions and relevant data for effective processing.

We utilized the Groq API, which is optimized for handling large-scale language models efficiently. The chosen model for this task is LLaMA 3–8B-8192, a lightweight model recognized for its capability to process extensive contexts. The LLaMA 3–8B-8192 model analyzes the text and identifies structured information such as personal details, professional experience, education history, skills, and relevant projects.

Chatbot:

The process begins by converting resumes into vector storage. When a job description query is received, the LLM generates sub-queries. The vector storage retrieves the top-K most similar documents for each sub-query. These documents are then combined, re-ranked, and used as external context for the LLM to generate accurate and relevant responses, helping match resumes with complex job descriptions.

Preprocessing:

The sub-queries generation process involves loading the user’s job description into an LLM agent, which analyzes and breaks it down into multiple sub-queries, each addressing different job requirements. For example, a job requirement like “Data Scientist with R/Python skills, teamwork ability, and 3+ years of experience” is split into sub-queries such as “Data Scientist with R/Python skills” and “Data Scientist with teamwork ability.”

The system generates 3 to 4 sub-queries by crafting a prompt with two components:

1. System Prompt: Provides consistent instructions and context to guide the LLM.
2. User Prompt: A dynamic query based on the specific job description.

The prompt directs the LLM to act as an expert recruiter, focusing on the job query to minimize inaccuracies. It can be customized to adjust the number of sub-queries generated, making the process adaptable to specific needs.

The system uses the **all-MiniLM-L6-v2** model, a lightweight sentence transformer, for text representation in semantic search tasks. This model provides a balance between speed and performance, offering competitive sentence embeddings and semantic search accuracy while being smaller and faster than other models. It has a token limit of 256 words, and the output is a 384-dimensional vector representation of the text.

For handling large documents like resumes, the system employs **document chunking**, which breaks long documents into manageable segments to improve retrieval efficiency. Recursive chunking, a method that hierarchically splits text using separators like periods, is used for its simplicity and ability to maintain coherence. The system sets chunk sizes to approximately 1024 characters with a 50% overlap (around 500 characters) between chunks, ensuring that context is preserved across sections for better information retrieval.

Retrieval

We use a vector store in this case, which is a lightweight storage system designed for efficient management and retrieval of dense vectors. Facebook AI Similarity Search (FAISS), a library developed by Meta, is employed for this purpose. FAISS provides fundamental vector store functionalities, including local vector storage and highly efficient in-memory similarity searching. It achieves this through various specialized index structures that enable rapid retrieval of similar vectors.

Data ingestion involves indexing resume embeddings and text chunks into a vector space using a flat index, which prioritizes search accuracy over speed. The indexed data is stored locally and serves as the knowledge base for the LLM agent.

Similarity-based retrieval uses cosine distance to compare query and document embeddings, focusing on semantic similarity. FAISS retrieves the most similar resume chunks for each sub-query, ranking them based on their cosine distance.

Finally, Reciprocal Rank Fusion (RRF) is employed to combine and re-rank results from multiple sub-queries. It aggregates inverse ranks from different ranked lists into a final score, balancing their influence with a coefficient (set to 60), and produces a re-ranked list of documents based on reciprocal scores.

Algorithm of Reciprocal Rank Fusion implementation.

Generation

The system employs the LLaMA 3–8B-8192 model for generating responses and creating sub-queries. This high-performance model processes up to 30,000 tokens per minute and uses a low temperature setting (0.1) to ensure consistent, predictable output, minimizing randomness and hallucinations.

The Small-to-Big Retrieval technique first retrieves smaller chunks of resumes and uses metadata to fetch complete documents. This approach enhances retrieval efficiency and provides full context for more accurate assessments.

In prompt design, the system uses a System Prompt to instruct the model to act as an “expert in talent acquisition” and focus only on the provided context, helping to prevent hallucinations. The User Prompt combines re-ranked resumes with job descriptions, ensuring the model has the necessary context to generate accurate responses.