Reference Verification Techniques

ICLR'26 has decided to 𝗱𝗲𝘀𝗸-𝗿𝗲𝗷𝗲𝗰𝘁 papers with 𝗵𝗮𝗹𝗹𝘂𝗰𝗶𝗻𝗮𝘁𝗲𝗱 𝗿𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲𝘀 generated by LLMs. That raises a practical question for every author: How do we verify citations reliably? We’re excited to share our new paper, 𝗕𝗶𝗯𝗔𝗴𝗲𝗻𝘁, an agentic citation verification framework designed to make reference checking auditable BibAgent traces where a claim is supported, surfaces evidence spans, and reports confidence rather than guessing. When a cited paper is behind a paywall, it can switch to a community-based “evidence committee” approach: collect downstream open-access citers, distill what they attribute to the paywalled work, and decide with consensus—or abstain if evidence is insufficient. We also propose a unified miscitation error-code taxonomy and release 𝗠𝗜𝗦𝗖𝗜𝗧𝗘𝗕𝗘𝗡𝗖𝗛, a large cross-disciplinary benchmark of miscitation cases. If you’re building LLM writing assistants, submission pipelines, or research integrity tooling, this is meant to be a step toward: draft fast → verify rigorously → publish faithfully. Ultimately, we hope this research helps 𝗳𝗮𝗰𝗶𝗹𝗶𝘁𝗮𝘁𝗲 𝗳𝗮𝗶𝘁𝗵𝗳𝘂𝗹 𝗮𝗻𝗱 𝘁𝗿𝘂𝘀𝘁𝘄𝗼𝗿𝘁𝗵𝘆 𝘀𝗰𝗶𝗲𝗻𝘁𝗶𝗳𝗶𝗰 𝗽𝘂𝗯𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀—so that emerging agents for scientific discovery can build on literature that’s genuinely grounded, not citation-shaped. Paper link: arxiv.org/abs/2601.16993 #AI #LLMs #ResearchIntegrity #OpenScience #NLP #ScientificDiscovery #TrustworthyAI

As AI tools become deeply embedded in academic workflows, one challenge keeps resurfacing: fabricated references. Many students, and even experienced researchers, unknowingly rely on citations produced by AI, only to discover later that some of those articles never existed. This not only weakens the credibility of their work but can also lead to academic and ethical consequences. The issue is not with AI being “wrong”; it is with how we use it without verification. Below is a clear, structured guide to help researchers understand why this happens, how to detect fake citations, and how to avoid them altogether. 𝗪𝗵𝘆 𝗔𝗜 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝗙𝗮𝗸𝗲 𝗥𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲𝘀 AI models predict text based on patterns. When you ask for references, the model often creates: plausible-sounding article titles real author names paired with non-existent works journals that look authentic publication years within a reasonable range This is called hallucination, and it can easily mislead an unsuspecting researcher. 𝗛𝗼𝘄 𝘁𝗼 𝗩𝗲𝗿𝗶𝗳𝘆 𝗥𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲𝘀 𝗣𝗿𝗼𝗽𝗲𝗿𝗹𝘆 Never add any AI-generated reference directly into your manuscript. Always follow a proper verification cycle: 1. Cross-check in credible databases Use platforms such as: → Google Scholar → IEEE Xplore → PubMed → Scopus → Web of Science → ScienceDirect If the article does not appear in at least one major library, treat it as suspicious. 2. Search by title, not just authors Sometimes AI mixes authors and titles. Always verify the exact title in quotation marks. 3. Confirm DOI numbers Real academic articles always have a valid DOI. If the DOI cannot be resolved on doi.org, it is fake. 4. Inspect journal websites Visit the actual journal’s archive and check if the publication exists in the stated volume and issue. 5. Be cautious with recent years Most fake references appear in the last 2–4 years. Cross-verify these with extra care. 𝗛𝗼𝘄 𝘁𝗼 𝗔𝘃𝗼𝗶𝗱 𝗙𝗮𝗸𝗲 𝗖𝗶𝘁𝗮𝘁𝗶𝗼𝗻𝘀 𝗶𝗻 𝘁𝗵𝗲 𝗙𝘂𝘁𝘂𝗿𝗲 1. Use AI only for direction, not citation lists Ask AI to suggest keywords, themes, or authors, not final references. 2. Generate references after doing your literature search Your citation list should come from databases, not AI-generated output. 3. Train students and research assistants on verification tools A quick 5-minute check can prevent a major academic embarrassment. 4. Use research managers wisely Tools like Mendeley, Zotero, and EndNote allow you to import references directly from credible databases. 5. Ask AI to format, not fabricate Instead of asking: “Give me references on machine learning in healthcare.” Try: “Format the following five verified references in APA 7th style.” This keeps you in control. As researchers, mentors, and educators, we must guide students to use AI wisely, not blindly. 📌 This is Dr. Laraib Abbas, and I help researchers accelerate their thesis journey with clarity, confidence, and credibility. 

𝐇𝐨𝐰 𝐰𝐞 𝐛𝐮𝐢𝐥𝐭 𝐜𝐢𝐭𝐚𝐭𝐢𝐨𝐧-𝐠𝐫𝐨𝐮𝐧𝐝𝐞𝐝 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐢𝐧 𝐅𝐃𝐑𝐘𝐙𝐄® We spent the last several months working on citation accuracy. Here's the process we landed on. 𝐋𝐚𝐲𝐞𝐫 1 - 𝘎𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘰𝘯 𝘸𝘪𝘵𝘩 𝘤𝘪𝘵𝘢𝘵𝘪𝘰𝘯 𝘪𝘯𝘴𝘵𝘳𝘶𝘤𝘵𝘪𝘰𝘯𝘴 The LLM generates responses with inline citations pointing to retrieved chunks. But we found this alone gives roughly 78% citation precision. The model confidently cites chunks that don't actually support the generated claim. 𝐋𝐚𝐲𝐞𝐫 2 - 𝘗𝘰𝘴𝘵-𝘩𝘰𝘤 𝘷𝘦𝘳𝘪𝘧𝘪𝘤𝘢𝘵𝘪𝘰𝘯 Every generated sentence goes through a verification pass. We check if the cited chunk actually entails the sentence using an NLI model fine-tuned on MNLI. Embedding similarity alone wasn't enough because semantically similar doesn't mean factually supported. For borderline cases we run an LLM-as-judge step. Expensive but necessary. 𝐋𝐚𝐲𝐞𝐫 3 - 𝘋𝘦𝘵𝘦𝘳𝘮𝘪𝘯𝘪𝘴𝘵𝘪𝘤 𝘷𝘢𝘭𝘪𝘥𝘢𝘵𝘪𝘰𝘯 This is where we catch what neural methods miss. We extract all named entities from the generated output and verify each one exists in the source documents. We extract all numbers with their surrounding context and require exact matches. Dates, percentages, and monetary values go through the same check. If the generated output contains any entity or number not present in sources, it gets flagged for regeneration. 𝐓𝐡𝐞 𝐬𝐲𝐧𝐭𝐡𝐞𝐬𝐢𝐬 𝐩𝐫𝐨𝐛𝐥𝐞𝐦 The hardest case is when the LLM combines information from multiple chunks into a single sentence. For this we decompose the generated sentence into atomic claims and attribute each claim independently. If a claim requires more than one logical hop from the source, it gets routed to human review. 𝐖𝐡𝐞𝐫𝐞 𝐰𝐞 𝐥𝐚𝐧𝐝𝐞𝐝 99%+ citation precision on our production benchmark. Zero hallucinated entities or numbers in our last 1000 generated documents. 100% of numerical claims verified against source documents. Full audit trail for every generated sentence back to source chunks. #EnterpriseAI #RAG #GenerativeAI #LLM #AIEngineering