<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Semantic Scholar - Medium]]></title>
        <description><![CDATA[Semantic Scholar is a tool that empowers scientific exploration and discovery. A free product of Ai2, we help scientists achieve impact. - Medium]]></description>
        <link>https://medium.com/semantic-scholar?source=rss----7e2b8c39c505---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Semantic Scholar - Medium</title>
            <link>https://medium.com/semantic-scholar?source=rss----7e2b8c39c505---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 12:51:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/semantic-scholar" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Announcing a new way to read papers.]]></title>
            <link>https://medium.com/semantic-scholar/announcing-a-new-way-to-read-papers-df0dec59d53b?source=rss----7e2b8c39c505---4</link>
            <guid isPermaLink="false">https://medium.com/p/df0dec59d53b</guid>
            <category><![CDATA[researchers]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[academia]]></category>
            <category><![CDATA[arxiv]]></category>
            <category><![CDATA[science]]></category>
            <dc:creator><![CDATA[Sam Skjonsberg]]></dc:creator>
            <pubDate>Wed, 21 Feb 2018 19:59:36 GMT</pubDate>
            <atom:updated>2018-02-21T22:35:38.701Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>TLDR; Read ArXiv papers on Semantic Scholar as </strong><a href="https://www.semanticscholar.org/paper/Self-Paced-Deep-Learning-for-Weakly-Supervised-Obj-Sangineto-Nabi/1a5151b4205ab27b1c76f98964debbfc11b124d5/read"><strong>responsive HTML documents</strong></a><strong>, rather than static, hard to read PDFs.</strong></p><p>Researchers at <a href="http://allenai.org/">AI2</a> both read and write a lot of papers, so it’s of no surprise that when walking through our office you’ll regularly see a bunch of screens broadcasting academic literature. What is surprising is that researchers are still consuming these documents as PDFs, static documents that don’t reflow to improve content legibility for the particular device you’re using. This is particularly surprising given the proliferation of devices we all have at our fingertips at any given moment.</p><p>Today we’re excited to announce a feature that changes that. Over the last month we partnered with the team behind <a href="https://www.arxiv-vanity.com/">ArXiv Vanity</a> to bring their technology to <a href="https://www.semanticscholar.org/">Semantic Scholar</a>. Starting today you can read papers published via <a href="https://arxiv.org/">ArXiv</a> on <a href="https://www.semanticscholar.org/">Semantic Scholar</a> as responsive HTML documents. This means they’ll render correctly whether you’re on a phone, tablet, laptop, or desktop computer. The reading experience also includes improvements like consistent formatting from one paper to another, optimal typography for legibility, and the ability to see the full title and author pertaining to a cited paper inline within the document.</p><p>Behind the scene’s <a href="https://www.arxiv-vanity.com/">ArXiv-Vanity</a> uses <a href="https://dlmf.nist.gov/LaTeXML/">LatexML</a> to convert the raw LaTeX to it’s HTML representation, after which some post-processing further enhances the output and styles it using the <a href="https://github.com/distillpub/template">Distill</a> template. We execute this process offline en-masse as to ensure as fast of an experience as possible.</p><p>In order to access the new reading experience, look for papers <a href="https://www.semanticscholar.org/search?venue%5B0%5D=ArXiv&amp;q=deep%20learning&amp;sort=relevance">published in ArXiv</a> on Semantic Scholar with the “Read Paper” button. Some ArXiv papers won’t have this button due to to formatting incompatibilities, but the vast majority should be available. If you just want to preview the new experience without searching for a paper, check out <a href="https://www.semanticscholar.org/paper/Self-Paced-Deep-Learning-for-Weakly-Supervised-Obj-Sangineto-Nabi/1a5151b4205ab27b1c76f98964debbfc11b124d5/read">this paper</a>.</p><p>Keep in mind that this is a first draft, so there’s bound to be a few rendering bugs and other issues. We’re curious to see what people think — and like any prototype if the idea resonates we’re planning on further investing in it. Here’s a few things we’re thinking about adding in the future:</p><ul><li>The ability to highlight passages (and see what others are highlighting).</li><li>The ability to comment inline within a document, and collaborate with a small group in doing so.</li><li>The ability to share a link that takes a user to the exact line / figure / citation in a document that you’d like them to see.</li></ul><p>That’s all for now. Happy reading, and if you’re excited about this feature or have ideas about what we can add to it, don’t be afraid to drop me a line at <a href="/service/mailto:sams@allenai.org">sams@allenai.org</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df0dec59d53b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/semantic-scholar/announcing-a-new-way-to-read-papers-df0dec59d53b">Announcing a new way to read papers.</a> was originally published in <a href="https://medium.com/semantic-scholar">Semantic Scholar</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Measuring research impact using AI: opportunities and challenges]]></title>
            <link>https://medium.com/semantic-scholar/measuring-research-impact-using-ai-opportunities-and-challenges-48f012c10868?source=rss----7e2b8c39c505---4</link>
            <guid isPermaLink="false">https://medium.com/p/48f012c10868</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[science]]></category>
            <category><![CDATA[research]]></category>
            <dc:creator><![CDATA[Erika Deal]]></dc:creator>
            <pubDate>Wed, 25 Oct 2017 19:06:24 GMT</pubDate>
            <atom:updated>2017-10-25T21:40:06.241Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EJDzHlRqcRFr_8zskIrmiw.jpeg" /></figure><p>Recently, Science magazine <a href="http://www.sciencemag.org/news/2017/10/who-s-most-influential-biomedical-scientist-computer-program-guided-artificial">published an article</a> with a ranking of top biomedical researchers based on citation data provided by Semantic Scholar. The list rightly sparked a discussion on social media about citation data and bias in algorithms and academia more generally.</p><p>After reflection we realize that publishing this data was a mistake both due to the quality of the data and because it does not benefit the scientific community. It is also a good opportunity to talk about the challenges and limitations of analyzing researcher impact using algorithms.</p><h3>How can bias show up in research impact analysis?</h3><p>Bias can come into play in two major ways when analyzing citation data:</p><ol><li>The data that we select for analysis</li><li>The assumptions built into the model that performs the analysis</li></ol><p>In producing the list of top scientists, we realize that we had flaws in both steps. Here’s what happened and why this kind of analysis is hard to do right.</p><h3>Bias in the data: garbage in, garbage out</h3><p>Researchers like to keep tabs: who’s famous, who’s on the rise, whose lab is ahead. Scientific research is highly competitive and it stands to reason that researchers might want to see the citation record sliced and ranked in new ways to show who’s on top.</p><p>But many recent studies have exposed problems with citations as a measure of scientific impact. Large-scale analyses of available citation data have taught us that:</p><ul><li><a href="https://www.semanticscholar.org/paper/Quantitative-Evaluation-of-Gender-Bias-in-Astronom-Caplar-Tacchella/aab323d6801972b3aa10045a1785f1e2eeb46ffe?tab=abstract">Men tend to be cited at a higher rate than women</a>, at least in some STEM fields.</li><li><a href="https://arxiv.org/pdf/1607.00376.pdf">Men tend to cite themselves more often than women</a>.</li><li>Demographic differences in publication output have a cumulative effect on citation numbers (<a href="https://www.semanticscholar.org/paper/Gender-Differences-in-Publication-Output-Towards-a-Symonds-Gemmell/e971ce3c87735e1ac553ba6709df8f457bddd6c2">1</a>, <a href="https://www.semanticscholar.org/paper/Are-female-researchers-less-cited-A-large-scale-st-Aksnes-R%C3%B8rstad/24dd9294fffae5de28f4e304d5b16ceb70a950d5">2</a>, <a href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0183301">3</a>)</li><li><a href="https://www.nature.com/nature/journal/v396/n6708/full/396210b0.html">Research tends to be regionally isolated.</a></li><li><a href="http://science.sciencemag.org/content/333/6045/1015">Minorities are granted funding at a lower rate</a>, which impacts output.</li></ul><p>All of this adds up to increasing awareness in the academic community that citations, while critical for tracing and measuring scientific work, are a flawed or at least insufficient way to measure impact.</p><p>Multiple studies have found evidence that citation metrics favor some groups over others. Some possible causes of unequal distribution and benefit of citations include:</p><ul><li><strong>Self-citations.</strong> As citation counts and h-index increasingly become part of tenure considerations, researchers are incentivized to pad and otherwise game the system. As <a href="https://arxiv.org/pdf/1607.00376.pdf">noted above</a>, this happens at different rates among different groups. We filter out self-citations in our metrics for this reason.</li><li><strong>Institutionalized bias in fields of study.</strong> Particularly in the sciences, women have long struggled with lower visibility and the <a href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0183301">vicious cycle</a> of less prestigious positions, lower output, and lower citation rates. <a href="https://elifesciences.org/articles/21718">Women are also underrepresented</a> as part of the peer review process, even correcting for numerical differences in representation.</li><li><strong>Researcher bias.</strong> This may take the form of gender and racial bias, but it may also include hard-to-quantify preferences for scholars who work in the same country or with whom the researcher has a professional relationship.</li></ul><p>Despite the fact that the deck is stacked against underrepresented groups, citation-based measurements still remain a primary measure of achievement for researchers. As more citation data becomes publicly available, however, Semantic Scholar and others are starting to experiment with alternative ways to analyze and measure researcher impact.</p><h3>Bias in AI models</h3><p>When we take that biased data and feed it into an algorithm, we’re subjecting it to an additional layer of assumptions as we try to produce an analysis. These assumptions may not account for complications like:</p><ul><li>A “good” citation count may look different from one field of study to the next.</li><li>Self-citations can greatly inflate citation counts, even indirectly.</li><li>Researchers may change their publishing name or have names that don’t conform to the “firstname lastname” convention.</li><li>Innovative research is not always published in top journals.</li></ul><p>Relying on community consensus is an easy starting point for the issues above, but it runs the risk of amplifying and perpetuating community biases under the banner of objectivity.</p><p>On Semantic Scholar team, we have attempted to address these issues in several ways, most important of which is our measure of “highly influential citations.” This metric is an attempt to identify citations that indicate one article was strongly impacted by another. It also excludes self-citations — in other words, we try to capture citations that more closely represent “scientific impact” and weren’t just added for the sake of the review process.</p><p>However, we realize that this metric remains imperfect. It is still fundamentally based on cumulative citation counts, vulnerable to coverage gaps, and makes assumptions about <em>how </em>scientists discuss influential papers that likely need closer examination. Even as we try to correct for known issues with the data, there are challenges that we have yet to account for.</p><p>Since we know there will be issues with the data and our ability to analyze it, any analysis of research impact we make will either amplify or expose bias in the scientific community — it’s up to us to decide which route to take.</p><h3>What can we do better?</h3><p>The difference between amplifying and exposing bias starts with contextualizing the data. This was a major learning for us on Semantic Scholar: when you know you have flawed data, it becomes all the more important to clarify what conclusions can and cannot be reasonably drawn from it. And when you see flaws that come from bias, you have a responsibility to try to address them.</p><p>As of today, here is how we’re thinking about appropriate use and communication of citation data:</p><p><strong>Be careful about using AI to rank individuals and groups of people.</strong> AI models are only as good as the data that goes into them, and they are a lot less accurate and deterministic than many people assume.</p><p><strong>Consider the public good when releasing citation data.</strong> What are we adding to the research community when we talk about citations as an impact indicator for researchers, papers, and journals? If a ranking or score does not assist researchers in their work, consider not publishing it.</p><p><strong>Be transparent about training data and assumptions built into the models.</strong> And if we can’t explain those assumptions and how we chose the data very clearly, we have some work to do before we can release our findings. This is the standard we hold peer-reviewed research to, and we should do the same for automated analyses of that research.</p><p><strong>Make sure flaws in the data can be corrected.</strong> This is another big learning for the Semantic Scholar team: our output will never be as good as we want it to be if we can’t improve the underlying data. This is also an important principle of user-centered design and something we hope to address in 2018.</p><h3>Open and ongoing research</h3><p>Finding more accurate ways to measure impact is an open area of research. Several researchers are now focusing specifically on finding ways to combat bias when measuring scholarly impact.</p><p>At the University of Washington, Jevin West and Carl Bergstrom are digging into citation data to uncover new ways to measure impact. The <a href="http://www.eigenfactor.org/">Eigenfactor</a> project explores problems of gender bias, journal ranking, and structural issues in science while providing alternate metrics for measuring the impact of scientific research.</p><p><a href="https://www.altmetric.com/">Altmetric</a> and <a href="https://profiles.impactstory.org/">Impactstory</a> attempt to analyze additional sources of information that capture scientific impact: discussions, open reviews, and more. As more and more scholarly output moves online, altmetrics provide an interesting avenue for new measurements of research impact.</p><p>In biomedicine, researchers are <a href="https://www.semanticscholar.org/paper/A-Scientific-Diversity-Intervention-to-Reduce-Gend-Moss-Racusin-Toorn/85f0ebf8ada3191cd8d614be1980e841396864c0">investigating interventions</a> and supporting <a href="https://www.nigms.nih.gov/training/dpc/Pages/build.aspx">large-scale initiatives</a> to improve scientists’ willingness to participate in bias training, ensure more equal access to funding and other professional opportunities, and fight institutional barriers to ensuring that everyone can benefit from medical advances.</p><p>Of course, this is just scraping the surface. Normalizing citation metrics is an open research problem, and breaking down the issues that can lead to unequal research impact analysis will take time and experimentation. We hope that the spirit of scientific inquiry helps us have more open, more self-critical conversations and find more effective ways to equalize the access, reporting and impact of science over time.</p><p>Did we miss relevant research we should share here? Please comment below!</p><p><em>Thanks to Miles Crawford for contributions to this post and several members of the Semantic Scholar team for reading and editing.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=48f012c10868" width="1" height="1" alt=""><hr><p><a href="https://medium.com/semantic-scholar/measuring-research-impact-using-ai-opportunities-and-challenges-48f012c10868">Measuring research impact using AI: opportunities and challenges</a> was originally published in <a href="https://medium.com/semantic-scholar">Semantic Scholar</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Semantic Scholar Cuts Through the Clutter of Biomedical Research]]></title>
            <link>https://medium.com/semantic-scholar/semantic-scholar-cuts-through-the-clutter-of-biomedical-research-c933935f063c?source=rss----7e2b8c39c505---4</link>
            <guid isPermaLink="false">https://medium.com/p/c933935f063c</guid>
            <category><![CDATA[research]]></category>
            <category><![CDATA[ai2]]></category>
            <category><![CDATA[biomedical]]></category>
            <category><![CDATA[semantic-scholar]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Marie Hagman]]></dc:creator>
            <pubDate>Tue, 17 Oct 2017 15:06:02 GMT</pubDate>
            <atom:updated>2017-10-17T22:33:24.393Z</atom:updated>
            <content:encoded><![CDATA[<p>Estimates indicate that global scientific output <a href="http://blogs.nature.com/news/2014/05/global-scientific-output-doubles-every-nine-years.html">doubles every nine years</a>. For many researchers, this translates to hours spent every week just trying to keep up with the latest discoveries. As a result, scientists are overwhelmed with information and a sense that they may be missing something — and in some cases, they may be right.</p><p>At the Allen Institute for Artificial Intelligence, our mission is to pursue AI for the common good. We started <a href="https://semanticscholar.org/">Semantic Scholar</a>, a free academic search engine, to help scientists find research that accelerates their work and to help the public get informed on advanced research topics. Using AI, we are creating a scientific knowledge graph out of the insights trapped in over 40 million PDFs to understand the research on important topics and how they interrelate.</p><h4>Delayed Discoveries: The Cost of Information Overload</h4><p>To understand the impact of missed connections on medical research, look no further than the development of trastuzumab, a lifesaving drug for treating HER2-positive breast cancer. The most important advance in breast cancer therapy in the last thirty years could have been discovered seven years earlier had technology been able to extract and identify the relevant information.</p><p>Trastuzumab, known under the brand name Herceptin, was approved by the FDA in 1988 for the treatment of HER2-positive breast cancer. 25% of all breast cancer cases are “HER2-positive,” meaning that the HER2 gene is overexpressed; it’s one of the most aggressive types of cancer in humans and for a long time, the prognosis was grim. Trastuzumab reduces the risk of death by 33%, an astonishing breakthrough that rested on the discovery of a single gene, a protein and its antibody.</p><p>Trastuzumab was developed at Genentech based on research by Axel Ullrich and his colleague from UCLA, Dennis Slamon. Ullrich and Slamon established that a specific gene, which they called HER2, overexpressed in a certain type of breast cancer and that the antibody to the HER2 protein had potential as a treatment. But they weren’t the first to make this discovery — the same gene had been discovered years earlier at a different lab.</p><p>In 1982, Lakshmi Padhy was a postdoc in Robert Weinberg’s lab at MIT. Weinberg had discovered that oncogenes cause cancer and they were working on identifying specific genes that might be responsible. Padhy extracted DNA from neurological tumors in rats and injected it into normal mouse cells, which then turned cancerous. Padhy then discovered that these cancerous mouse cells triggered an immune response in the mice, creating specific antibodies to the proteins created by these rat-tumor genes. He called this gene and antibody <em>neu</em> — but never followed up on it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*d7DNpWmoQ82dUj9I." /></figure><p>Although Padhy’s discovery was <a href="https://www.semanticscholar.org/paper/Identification-of-a-phosphoprotein-specifically-in-Padhy-Shih/149575661605e11743a92d6025e9c7dae6b5605c">published in Cell</a>, a high-profile scientific journal, nobody noticed that they might have stumbled on a potential anticancer drug because the neu-binding antibody was buried in an obscure figure in the article. This is the kind of buried information that AI has the potential to bring to the surface for researchers, and that Semantic Scholar tries to foreground: discoveries that may not be recognized as important at the time, but are essential to the advancement of science overall.</p><h4>Important Discoveries in Obscure Research</h4><p>Some breakthroughs turn an entire field on its head. One of these actually inspired me to join the Semantic Scholar team. My personal experience made very real for me the cost of delay in spreading research findings to the broader medical community.</p><p>In 2002, I was working as a software developer for a small startup. Things were going well until my health took a turn for the worse. Everything I ate or drank caused intense stomach pain and heartburn — even a glass of water made me double over in pain. I saw a gastroenterologist who did an endoscopy and the results weren’t good — I had an inflamed stomach lining (gastritis) and two duodenal ulcers.</p><p>I was shocked! I was young, healthy, and none of the usual ulcer causes explained my condition. The doctor handed me a six month prescription for Prevacid, a drug that reduces stomach acid, and said I would need to be on it for the rest of my life. The medication made my symptoms go away but I was worried about the idea of taking a pill for the rest of my life, especially when the cause of my condition was unknown.</p><p>I saw a second gastroenterologist and heard the same story. At my wits’ end, I searched online for medical research papers and learned that Helicobacter Pylori (H. Pylori) causes ulcers and gastritis. I saw a third doctor, armed with research about H. Pylori. The treatment was a 2-week antibiotic cocktail, and she agreed to let me try it.</p><p>Since that course of treatment I have been completely cured. Just a few years later, I learned the <a href="http://discovermagazine.com/2010/mar/07-dr-drank-broth-gave-ulcer-solved-medical-mystery">amazing story</a> of two Australian doctors, Marshall and Warren, who were awarded a Nobel Prize in 2005 for their work on H. Pylori. Marshall had to prove that H. Pylori causes ulcers through drastic measures — by experimenting on himself. Their work was originally published in the Australian Journal of Medicine in 1985 and languished for another 10 years before it started gaining acceptance in the US. Today antibiotics are the standard of care for ulcer treatment as well as stomach cancer, which is now almost gone from the Western world as a result of their work.</p><h4>Helping Researchers and Patients Connect the Dots</h4><p>With these examples in mind, we set out to turn Semantic Scholar into a tool that could surface interesting connections in the literature and help researchers and patients alike discover important new insights. In particular, we focused on a problem that kept popping up as we talked to biomedical researchers: finding information about an unfamiliar topic without getting overwhelmed.</p><h4>What we did</h4><p>To help people get up to speed on new research topics, we first needed to identify which topics are important in research papers and how the topics relate to each other. Most existing work like this has identified people, organizations, and locations. However, identifying scientific topics is harder than identifying proper nouns because there are fewer lexical cues like capitalization. For example, in the paper “Acute Lymphoblastic Leukemia in Children,” we need to identify “acute lymphoblastic leukemia” as the important topic — not just “leukemia” — and we also need to know that “acute <em>lymphocytic</em> leukemia” is the same disease.</p><p>We started by enlisting medical experts to spend hundreds of hours labeling topics like diseases, genes, and research techniques in papers. These examples were used to teach our algorithm to identify medical topics in <em>any</em> paper. This was an expensive and time consuming endeavor so the challenge was to build an algorithm that could understand the meaning of many varied medical topics without human annotated examples for each subject area.</p><p>PubMed includes research from 80+ medical fields, it would be prohibitive to collect labeled data for each field. To get around this restriction, we used two neural networks: one to capture the grammatical context of information in papers, and a second to use that language understanding to make predictions based on the examples annotated by our medical experts. The result was highly accurate identification of salient topics in papers with less training data than might normally be needed for a project of this scope. Our algorithm also relies on several knowledge bases, in addition to the grammatical context and labeled data, to help it understand the meanings of words. For example, in medical literature the word “hedgehog” can refer to the signaling protein or the animal. If you want to know more, don’t worry — the Semantic Scholar Research team is working on a paper for publication that will describe this system in more depth.</p><h4>How it works</h4><p>Once we’ve identified topics and scored how important they are in a particular research paper, we use that information to decide what to show on the site when someone does a search. Topics show up in two ways:</p><ul><li>Suggested terms to augment or refine your search</li><li>Topic pages that define the topic and offer a summary of recent and review literature</li></ul><p>To take an earlier example, here’s what we show on a search for “<a href="https://www.semanticscholar.org/search?q=stomach%20ulcer&amp;sort=relevance">stomach ulcer</a>”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oEdv39jz62XfzCYd." /></figure><p>Already the link to H. Pylori is apparent. Further down the page we link to a topic page for “<a href="https://www.semanticscholar.org/topic/Gastric-ulcer/16989">gastric ulcer</a>” (the synonymous medical term):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*enuFYi1pwiMYmVLJj9hQ3A.png" /></figure><p>Following that link, the topic page shows a list of reviews, some of which discuss the link to H. Pylori. I wish I had this years ago when I was exploring this topic!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vcdDCdKqbjq5S118." /></figure><p>These topic pages are for medical experts that are stepping outside their comfort zone or the general public who want to learn about their own conditions: a short summary of the topic and recent reviews, intended to give people a running start with the literature. For researchers who are more familiar with a topic, search term suggestions can be a great way to call to mind topics or tangents that might take the search in an interesting direction. And for those who know what they’re looking for, we’ve kept many of the filters that help “cut through the clutter” and zero in on papers with a PDF, papers from a particular year or author, and more.</p><h4>What’s next</h4><p>Semantic Scholar started in 2015 by covering computer science literature and expanded to include neuroscience a year later in partnership with the <a href="https://www.alleninstitute.org/">Allen Institute for Brain Science</a>. We are now expanding to include literature for all biomedical sciences because we believe that accelerating scientific discoveries is one of the highest impact applications for artificial intelligence today.</p><p>PDFs are the standard electronic format for scientific articles today. That means a scientist often has to read the whole paper — as much as 40 pages of it — to find the information that helps them move their research forward. The next lifesaving breakthrough may be buried in the caption of a figure, or published in an obscure journal that the researcher hasn’t subscribed to. This is where artificial intelligence can shine: helping researchers separate the wheat from the chaff so that they can spend more of their time on high-impact work. We hope this release is just the first step toward freeing scientists to spend more time on research and less time worrying about missed connections.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c933935f063c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/semantic-scholar/semantic-scholar-cuts-through-the-clutter-of-biomedical-research-c933935f063c">Semantic Scholar Cuts Through the Clutter of Biomedical Research</a> was originally published in <a href="https://medium.com/semantic-scholar">Semantic Scholar</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Don’t write an SPA*]]></title>
            <link>https://medium.com/semantic-scholar/dont-write-an-spa-57aaa40e0314?source=rss----7e2b8c39c505---4</link>
            <guid isPermaLink="false">https://medium.com/p/57aaa40e0314</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[front-end-development]]></category>
            <dc:creator><![CDATA[Sam Skjonsberg]]></dc:creator>
            <pubDate>Wed, 12 Jul 2017 16:41:20 GMT</pubDate>
            <atom:updated>2017-07-12T16:41:20.346Z</atom:updated>
            <content:encoded><![CDATA[<p><em>*Unless it makes sense to.</em></p><p>Ask almost any frontend developer out there about the state of JS tooling and frameworks, and they’ll probably tell you that it’s impossible to keep up with what’s new. Angular? React? Vue? <a href="http://stateofjs.com/2016/frontend/">Aurelia</a>? Selecting the right one for a new project is a daunting task, making it easy to choose one without thinking critically about the long term implications.</p><p>The vast majority of projects choose React or Angular, and accordingly adopt a single page application (SPA) architecture. This approach has its benefits, but they don’t come without a cost — SPAs introduce several layers of complexity that aren’t always easy to recognize up front. An SPA might be the right choice for your project, but sometimes boring old technologies are the right choice too — it just depends on the goals of your project.</p><p>When we created <a href="https://semanticscholar.org/">Semantic Scholar</a> in 2015, we built a client rendered SPA using React that we later made isomorphic. While this approach definitely paid off in some ways, it also came with a few challenges that we didn’t anticipate. Below are some of the benefits and drawbacks we’ve experienced over the past two years.</p><h3>What went well.</h3><h4>Our site feels fast.</h4><p>The decision to build an SPA really boiled down to one key requirement — speed. Performance is an extremely important part of a good user-experience, as <a href="https://blog.codinghorror.com/speed-still-matters/">demonstrated by experiments conducted by both Google and Amazon</a>:</p><blockquote><em>Half a second delay caused a 20% drop in traffic. Half a second delay killed user satisfaction.</em></blockquote><p>The SPA architecture enables our application to render additional pages / content without downloading additional Javascript and/or CSS — the only bytes we have to send down are those which make up the data we’re rendering. While the net benefit of this from a raw bandwidth perspective is small (as the initial payload users download is larger), the benefit lies in a reduction of “perceived latency.” Client-side rendering enables the display to be immediately updated with every interaction, making the application feel snappier, even if the API is chugging hard to get you the results you’re looking for.</p><p>The architecture seems to have paid dividends in this regard. We’ve received a lot of feedback from end users that the site is incredibly responsive and fast — and our metrics back this up. The <a href="https://www.semanticscholar.org/paper/Latent-Dirichlet-Allocation-Blei-Ng/1e56ed3d2c855f848ffd91baa90f661772a279e1">paper detail page</a> renders within 1.16 seconds for 90% of our users, including those on mobile devices / slow networks. What’s really interesting about this is that our initial load time isn’t so great — for 90% of users it takes around 5 seconds to load and render the page! Yikes! This supports our intuition that end users care more about “perceived” latency than actual latency. If the application feels responsive via immediate visual cues, the actual time spent retrieving search results becomes less noticeable. The SPA architecture made building a responsive, snappy interface more straightforward than it otherwise might have been (that said, you can definitely accomplish the same thing with other techniques).</p><h4>It’s easy to write tests.</h4><p>The popularity of the JavaScript ecosystem and SPA architecture has a great side-effect : there’s a ton of great libraries for testing UI components (we use <a href="https://github.com/airbnb/enzyme">enzyme</a>, <a href="https://github.com/tmpvar/jsdom">jsdom</a> and <a href="https://mochajs.org/">mocha</a> specifically). Granted, these toolsets aren’t exclusive to SPAs — but React was developed with testing in mind, which in turn makes the code you write with it easier to verify. Riding the wave of tools and libraries driven by the popularity of an approach has very real advantages — granted, the wave can (and in all likelihood will) break at some point. Surf’s up.</p><h4>It’s easy to build interactive features.</h4><p>As the web and the applications we build with it continue to advance, so do the expectations of our users. Tools like React and angular simplify the code behind developing highly interactive experiences because the specifics of interacting with the DOM and its differences across browsers are handled for you. Plus, the resulting abstraction caters to well encapsulated, testable components (it’s so important it’s worth repeating).</p><h4>It’s fun to work with.</h4><p>The SPA is a double-edge sword — it’s the shiny, unwieldy thing we all want to pick up and use, even though we might hurt ourselves in the process. We find working with React and SPA-based architectures really interesting, more so than tried and true approaches like server-rendered templates with a dash of jQuery on top. We’re engineers after all — we like solving tough problems! That said, we’re also our own worst enemy in this regard, as keeping thing simple lends itself towards stable software that’s easy to change. This in turn empowers more iteration (and a better end product). The key is finding the right balance by applying technologies we’re excited about in a fashion that produces robust systems that can be changed rapidly.</p><h3>Lessons learned.</h3><h4>Analytics aren’t free.</h4><p>With a traditional, server rendered application the analytics story is really simple:</p><ol><li>Login to Google Analytics.</li><li>Copy and paste the snippet into your base template.</li><li>Deploy your website.</li></ol><p>Bam! A few days later your Google Analytics dashboard is full of useful information about the most popular pages on your site and how long it takes people to load the page in some remote location on a satellite internet connection.</p><p>With an SPA the notion of a traditional “pageview” goes out the window — and without custom instrumentation the data provided by Google Analytics becomes pretty useless. Several companies like New Relic and MixPanel are trying to fix this — but we still found the metrics provided by their SPA offerings pretty limited and/or inaccurate.</p><p>If you’re going to write an SPA, be prepared to do more work instrumenting your analytics platform. You’ll have to add hooks to your codebase for tracking pageviews (given the transitions occur on the client) and instrumentation for tracking performance related metrics (as GA nor any of the solutions mentioned above effectively track the time it takes to transition). This might sound simple, but it’s important to recognize that it’s not free. You’ll spend time writing and testing these hooks, and you’ll definitely run into unanticipated bugs / issues (no software is perfect, after all). We made a big mistake in that we didn’t test any of this initial instrumentation — and suffered the consequences when our metrics stopped working due to unrelated changes / regressions. That said, we’re happy to report that we’ve now got nearly completely coverage the analytics we track — losing sight of what our end users are experiencing is pretty drastic.</p><h4>Stateful applications are complex.</h4><p>Stateful systems are inevitably more complex — the developer has to reason about concurrency, mutability, locking, etc. — and with these concerns come hard to debug bugs. Embracing the SPA architecture requires diving head-first into this world, as the JavaScript runtime is one where mutable, global application state reigns supreme. Sure, things like <a href="https://facebook.github.io/immutable-js/">Immutable</a> and <a href="http://redux.js.org/">redux</a> help make this more manageable — but under the hood you’re really still just modifying global state. Server side rendered web applications lack this complexity — each request is processed as a single transaction with clear I/O (a request in, a big ole’ blob of HTML out). There’s a lot less failure scenarios, and the runtime is entirely in our control.</p><h4>Client-side rendered applications don’t index well.</h4><p>Search engines like Google and Bing claim to index JavaScript rendered content without issue, but the specifics aren’t well documented and a test we executed demonstrated that the capabilities of these crawlers are really limited. We experimented by launching our site first as a client-rendered SPA only — requesting any page on the site simply resulted in a bare-bones HTML file that bootstrapped the experience:</p><pre>&lt;!DOCTYPE html&gt;&lt;html&gt;<br>  &lt;head&gt;<br>    &lt;meta charset=&quot;utf-8&quot;&gt;<br>    &lt;title&gt;Semantic Scholar&lt;/title&gt;<br>  &lt;/head&gt;<br>  &lt;body&gt;<br>    &lt;script src=&quot;main.js&quot;&gt;&lt;/script&gt;<br>  &lt;/body&gt;<br>&lt;/html&gt;</pre><p>We then implemented a sitemap with links to each paper in our corpus, making it easy for Google and other engines to crawl our site. As it turns out, Google indexed each and every URL, we ended up with results that looked like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Am7jle3y5Wg-bWVOo8X6XQ.png" /></figure><p>Clearly Google was indexing the URLs, but none of the content was being ingested. Not only does this look bad, but it causes each page to be indexed as an empty document — not very relevant to most queries.</p><p>The experiment revealed that client-side rendering wasn’t going to cut it , so we moved to an “isomorphic” application that rendered on the client and server. Luckily React made this possible — but not without jumping through a few hoops (which were on fire, and moving).</p><h4>“Isomorphism” is hard.</h4><p>While React allowed us to forget about the complexity of interacting with Internet Explorer’s DOM abstractions, SEO required that we implement an application that could render on the server or client. For us, this was uncharted territory. There weren’t a lot of supporting libraries or tools, and the code samples we found were relatively incomplete. Something as simple as reading a cookie now forced us to think about the abstraction in more detail:</p><pre>function getCookieValue(cookieName) {<br>  if (typeof document !== &#39;undefined&#39;) {<br>    const cookieStore = document.cookie;<br>    ...<br>  } else { <br>    if (!requestContext) throw &#39;No cookies for you.&#39;<br>    const cookieStore = requestContext.getHeader(&#39;Cookie&#39;)<br>    ...<br>  }<br>}</pre><p>The complexity is ultimately manageable — it just requires more effort during the software design phase to make sure the abstractions are well thought out.</p><h3>Where we’ve landed, and where we’re headed.</h3><p>So you’re probably thinking at this point that we’d like to nuke n’ pave — but in actuality we’re quite happy with where we are today:</p><ol><li>Our site is fast — users rave about its responsiveness.</li><li>The content indexes remarkably well. We continue to connect with a broader audience via search engines.</li><li>We have a robust suite of tests that help us change our system quickly and with confidence.</li><li>Our team enjoys working with the codebase and continues to look for ways to simplify the architecture.</li></ol><p>That said, it didn’t come without a cost — and the codebase is far from perfect. Things we’re thinking about:</p><ul><li>Carving out clear boundaries that represent the core “application.” For instance, one might argue that the search experience merits a SPA, while the content pages could be server-rendered.</li><li>How to manage the continued growth of our JavaScript bundle. It’s only 312 KB (compressed) currently, but as we build new features it’ll grow and at some point significantly impact the load time for new visitors. This requires navigating another perilous passage — we either “hot-load” dependencies or split our application into several, separate applications with individual JavaScript bundles.</li></ul><p>At the end of the day, we just want to make sure the architecture we choose doesn’t get in the way of our mission: to reduce information overload for researchers by providing them with innovative, comprehensive, and fast ways to keep up with the latest advances in their fields.</p><h3>So are you telling me I shouldn’t write an SPA?</h3><p>Not necessarily. SPAs are a good choice if:</p><ul><li>The interface you’re building is highly interactive and needs to be fast / responsive — and you want an architecture and abstractions that supports this out the gate.</li><li>You’re not worried about each and every bit of content being indexed by search engines correctly.</li></ul><p>On the other hand, an SPA might not be worth the complexity if:</p><ul><li>Your application is simple (there’s not a lot of interactive content)</li><li>SEO is of pivotal importance.</li></ul><p>But at the very least, take a step back and consider what you need before selecting an SPA — you’ll thank yourself down the road.</p><p>~ @codeviking</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=57aaa40e0314" width="1" height="1" alt=""><hr><p><a href="https://medium.com/semantic-scholar/dont-write-an-spa-57aaa40e0314">Don’t write an SPA*</a> was originally published in <a href="https://medium.com/semantic-scholar">Semantic Scholar</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an Elasticsearch Learning to Rank Plugin for Semantic Scholar]]></title>
            <link>https://medium.com/semantic-scholar/building-an-elasticsearch-learning-to-rank-plugin-for-semantic-scholar-40731196823a?source=rss----7e2b8c39c505---4</link>
            <guid isPermaLink="false">https://medium.com/p/40731196823a</guid>
            <category><![CDATA[search-engines]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Derick Anderson]]></dc:creator>
            <pubDate>Fri, 30 Jun 2017 17:39:42 GMT</pubDate>
            <atom:updated>2017-06-30T17:39:41.750Z</atom:updated>
            <content:encoded><![CDATA[<p>Semantic Scholar is an academic search engine for scientists, which means that relevance and ranking are core components of the site. We use Elasticsearch to power our search experience, and for a long time we relied on a handcrafted query that did a reasonable job with our initial corpus of computer science papers.</p><p>As we scaled up, though, we realized we needed more than what Elasticsearch can provide out of the box — it doesn’t provide support for expressing complicated models, or for an easy model training workflow. We therefore built a plugin to extend that default functionality to include a more normalized machine learning environment, i.e. evaluating arbitrary models on features from feature templates and featurizing training datasets for training without a cluster.</p><h3>The Problem</h3><p>If we look at an official ES <a href="https://www.elastic.co/guide/en/elasticsearch/guide/current/controlling-relevance.html">guide</a> to improving search relevance we can see what is available out of the box. Most of what is available centers around two things: matching multiple fields (title, abstract), and matching in multiple ways (phrase, bigram). These are ways to get different numerical features describing the document and its match with the query. That’s great — we want all the information we can get.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/550704d44c2097f56df657dbbd3dac1b/href">https://medium.com/media/550704d44c2097f56df657dbbd3dac1b/href</a></iframe><p>A problem arises when we have all this information though. How do we take in different features and combine them into a final score? The essential Elasticsearch technique is to write your query as a tree with similar features grouped together. This allows you to attempt to tune your query by coming up with better combinations of subtrees or transformations on subtree scores (e.x. exponentiation).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d69bc2ad22eebf1bb37f7a35d6fa0b99/href">https://medium.com/media/d69bc2ad22eebf1bb37f7a35d6fa0b99/href</a></iframe><p>That is fine as far as it goes — but it does not go as far as we would like. That is, we cannot write the arbitrary models we want. We can write custom scoring scripts, but they get only one _score value - they cannot see how much of the score came from title and how much came from abstract. Without visibility into individual feature values it is not clear how to express something like a decision tree. Furthermore, even if a script could see multiple values it is not clear how to train a model with this setup.</p><h3>The Solution — a plugin!</h3><p>The high level goal of our plugin is to enable a normalized learning to rank workflow with Elasticsearch at the core. We implement a Lucene Query that iterates over documents that match a query, and captures the values of low-level features from matching the query to the document. The features are collected and can be sent to an arbitrary scoring function to compute the final document score, or sent over the wire for use by an offline system.</p><p>Instead of writing a plugin to provide a modified Query, we could have written a service to take a large sample (say 1000 documents) from a basic Elasticsearch query and rank the documents again. This would have been nice because it would not have complicated our Elasticsearch deployment and we could calculate even the most expensive of features. With the plugin we have to redeploy whenever we make changes, and features that are evaluated on many documents need to be cheap. On the other hand, we believed that the plugin would be the more performant final solution given cheap features, and its integration into Elasticsearch meant the overall system architecture would be simpler. Ultimately we decided in favor of the plugin model because of the conceptual simplicity and its composability with reranking in the worst case.</p><h3>Feature templates</h3><p>Features templates are specifications for the features to calculate and pass to a model. Features are implemented by wrapping and capturing the results of normal Lucene queries and scripts. This made sense for us because it allows the use of the wealth of preexisting functionality without a lot of implementation cost.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/27e86c8099cc39252786e7d44ec8a8f9/href">https://medium.com/media/27e86c8099cc39252786e7d44ec8a8f9/href</a></iframe><p>An example of a feature:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/630fbf248690b8e2f2b51f7cacc4a6d6/href">https://medium.com/media/630fbf248690b8e2f2b51f7cacc4a6d6/href</a></iframe><p>The above will return one value: the score on the match between the query and the keyPhrases field in a document.</p><p>Normal queries make use of something called the <a href="https://www.elastic.co/guide/en/elasticsearch/guide/current/practical-scoring-function.html">Lucene Practical Scoring Function</a> which incorporates several lower-level features (such as TF and DF). We also implement a featurization wrapper that extracts some of those lower-level inputs and then reports them as features that the model can learn its own scoring function on.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a9165d36e10f314120fe020414376f6b/href">https://medium.com/media/a9165d36e10f314120fe020414376f6b/href</a></iframe><p>The above will return 4 values: the score, the number of matching terms, and the sums of those terms’ TF and DF.</p><p>We access numeric fields with simple call outs to scripts:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0cd122e987a3d4522ccb7c4a59df75b/href">https://medium.com/media/e0cd122e987a3d4522ccb7c4a59df75b/href</a></iframe><h3>Arbitrary scoring functions</h3><p>With features collected into vectors we can write our ranker as a function that takes an array of floats and returns a single float (the score for the document). To decide on actual implementation we considered a few factors. Firstly, we were using <a href="https://sourceforge.net/p/lemur/wiki/RankLib/">RankLib</a>’s implementation of several models. Secondly, we needed as much runtime performance as possible for production. Lastly, we also wanted to be able to reload models for rapid experimentation without restarting the cluster.</p><p>To meet the competing demands of production and rapid experimentation we implemented two ways of providing models. The first is just calling out serialized RankLib models by filename — the models are loaded and fed data in the RankLib format. Calling out filenames met the rapid experimentation use case because we can scp models onto the cluster without restarting or recompiling anything.</p><p>RankLib normally evaluates models sort of as an interpreter — it has the model parameters in a data structure and traverses the data structure. In order to improve the performance of evaluating models in production we compiled RankLib regression trees to Java code. We distribute these in the plugin binary and then call them out by classname.</p><p>Calling to a compiled model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eb47897c8c92490379ee5a53e25de5c3/href">https://medium.com/media/eb47897c8c92490379ee5a53e25de5c3/href</a></iframe><p>A snippet from the included compiled model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2eddfa58c20dc51222d4cc27cff8b7fb/href">https://medium.com/media/2eddfa58c20dc51222d4cc27cff8b7fb/href</a></iframe><h3>Scraping and storing offline</h3><p>The point of the plugin is to enable a normal machine learning workflow so it needs to support training (without a cluster running) on the same data as available at runtime. To enable this we add a REST endpoint to the Elasticsearch cluster that takes a query with document IDs specified and returns a map from the document ID to the document’s feature vector. Thus the cluster is needed to turn (document ID, query, label) triples into (feature vector, query, label) triples but not to learn the (feature vector) -&gt; score model.</p><h3>Performance</h3><p>One potential issue is that by default the feature and model based query is purely disjunctive — if any feature fires for a document it would run the whole model on it. The first step was to do something we already did in our baseline — to filter out documents that didn’t contain some match for all mandatory terms in the query. The second was to tightly tune the implementation of featurization and model evaluation to be as performant as possible.</p><p>Tuning the performance of featurization was actually quite difficult. This difficulty was connected with difficulty experienced in writing the featurization code in the first place — the web of Lucene contracts and possible Elasticsearch calling patterns meant it was easy to experience hard to debug problems. The solution was to faithfully adhere to the contracts of the Lucene components touched on and to implement all of the optional functionality available.</p><p>After the performance work was finished the new ranker performed with limited degradation against the baseline (load tests against the Semantic Scholar search endpoint measured 10% increase in latency at p99). We were happy to avoid making any difficult decisions about changing our retrieval model to included reranking or tightening our filter.</p><h3>Conclusion</h3><p>The development of the plugin and a set of offline training tools creates a powerful learning to rank environment based on Elasticsearch. The primary benefit of the plugin, and other relevance work that will be discussed in future posts, is the ability to train rankers easily. This includes ranker optimized for different query or corpus distributions. We retrained recently with the introduction of many new medical papers to Semantic Scholar, and will continue to adapt our ranking as we expand Semantic Scholar.</p><p>Thanks for reading, and hope you found this helpful.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=40731196823a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/semantic-scholar/building-an-elasticsearch-learning-to-rank-plugin-for-semantic-scholar-40731196823a">Building an Elasticsearch Learning to Rank Plugin for Semantic Scholar</a> was originally published in <a href="https://medium.com/semantic-scholar">Semantic Scholar</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Open Academic Search]]></title>
            <link>https://medium.com/semantic-scholar/open-academic-search-792593e3b3d?source=rss----7e2b8c39c505---4</link>
            <guid isPermaLink="false">https://medium.com/p/792593e3b3d</guid>
            <category><![CDATA[open-access]]></category>
            <category><![CDATA[semantic-scholar]]></category>
            <category><![CDATA[open-academic-search]]></category>
            <category><![CDATA[ai2]]></category>
            <category><![CDATA[semantic-search]]></category>
            <dc:creator><![CDATA[Marie Hagman]]></dc:creator>
            <pubDate>Tue, 20 Jun 2017 14:01:01 GMT</pubDate>
            <atom:updated>2017-06-20T14:01:01.473Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l2UC8Rh2oHIWBJPAju-hyQ.png" /></figure><p>The internet was originally created for the distribution of academic information, but decades later much of the scholarly work produced is not easily accessible. Fortunately, many disparate efforts in the open access movement are working to change that. <a href="http://allenai.org/">AI2</a> formed the <a href="http://www.openacademicsearch.org/">Open Academic Search (OAS)</a> working group to accelerate the pace of progress for groups engaged in these efforts.</p><p>Working in silos means duplication of foundational pieces and time and resources wasted. For example, anyone working on an academic search solution needs to parse and normalize metadata from PDFs, which means everyone has developed a different solution to this problem. What’s the best way and how much better could it be if we collaborated on a single solution?</p><p>Researchers rely on a comprehensive knowledge graph of authors and papers created from citations in order to assess impact and attribute scientific work; this is a foundational component of any academic search system today. But as machine reading improves and academic search grows more sophisticated, there are endless possibilities for building on and adding value to this foundation. Through collaboration we can openly share author-paper data and build our unique solutions on top of it to accelerate research.</p><p><strong>OAS Core Principles</strong></p><ol><li>Collaboration drives academic search innovation.</li><li>AI can play a unique role in surfacing and analyzing information contained in millions of online research papers and academic journals.</li><li>Our core mission is advancing the pace of research and aiding breakthroughs in critical research areas.</li></ol><p>We’ve assembled an advisory board of like-minded influencers in the academic search space for the benefit of their expertise and collective mindshare. Members come from both nonprofit and for-profit organizations, many of whom already participate in other open access initiatives. The key focus of this particular effort is on <strong>technology and data to support academic search and discovery</strong>. Our long-term goal is to ensure the longevity and long-term viability of academic search systems.</p><p>One of the biggest challenges for research scientists is staying informed of the latest developments with the ever increasing pace of <a href="http://blogs.nature.com/news/2014/05/global-scientific-output-doubles-every-nine-years.html">global scientific output</a>. Members of the OAS working group have already put useful tools into the public domain to help with this challenge. The group will also identify new opportunities for collaboration that build on openly available technologies and data sets to speed the pace of innovation in academic search.</p><p>To show our commitment to this cause, the <a href="https://www.semanticscholar.org/">Semantic Scholar</a> team has recently released several tools to accelerate innovation in academic search:</p><ul><li><a href="http://labs.semanticscholar.org/corpus/">Open Corpus</a></li><li><a href="https://github.com/allenai/science-parse">ScienceParse</a></li><li><a href="http://pdffigures2.allenai.org/">PDFfigures</a></li><li><a href="http://api.semanticscholar.org/">Semantic Scholar API</a></li></ul><p>For more information and a comprehensive list of resources see the <a href="http://openacademicsearch.org/">OAS webpage</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=792593e3b3d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/semantic-scholar/open-academic-search-792593e3b3d">Open Academic Search</a> was originally published in <a href="https://medium.com/semantic-scholar">Semantic Scholar</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>