<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Libraries.io - Medium]]></title>
        <description><![CDATA[Stories on the development, design, and data behind Libraries.io - Medium]]></description>
        <link>https://medium.com/libraries-io?source=rss----42fe172790d9---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Libraries.io - Medium</title>
            <link>https://medium.com/libraries-io?source=rss----42fe172790d9---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 01:26:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/libraries-io" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Our second Libraries.io open data release has arrived]]></title>
            <link>https://medium.com/libraries-io/our-second-libraries-io-open-data-release-has-arrived-703422b1ad88?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/703422b1ad88</guid>
            <category><![CDATA[open-data]]></category>
            <category><![CDATA[librariesio]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Andrew Nesbitt]]></dc:creator>
            <pubDate>Thu, 30 Nov 2017 14:32:46 GMT</pubDate>
            <atom:updated>2017-11-30T14:31:31.611Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<p>Today we’re publishing another Libraries.io open data release with over 311 million rows of metadata about open source projects and the network of dependency data that connects them all.</p><p>Six months ago we <a href="https://medium.com/@BenJam/libraries-io-releases-data-on-over-25m-software-repositories-ab1db665826e">published our first open data</a> as part of our commitment to the<a href="https://medium.com/libraries-io/libraries-io-joins-brave-new-software-c8cb7bba93bf"> Alfred P. Sloan and Ford Foundations</a>. The data supports <a href="https://arxiv.org/abs/1710.04936">academics looking into trends in software development</a>, investors to understand the success of projects they support, and <a href="https://libraries.io/rubygems/split/usage">developers to understand how their software is used</a> more effectively than ever before.</p><p>Last week <a href="https://medium.com/libraries-io/why-libraries-io-and-dependency-ci-are-joining-tidelift-1caf72f4d1b0">we announced</a> that Libraries.io has joined forces with <a href="https://tidelift.com/">Tidelift</a> to make open source software work better for developers and users. Libraries.io’s mission hasn’t changed and we’re going to continue publishing open data releases every quarter to build a stronger, more informed open source ecosystem.</p><p>Since our last release the Libraries.io dataset has grown significantly, today we’re releasing data on:</p><ul><li>34 package managers</li><li>2.7 million projects</li><li>11 million versions</li><li>66 million project dependencies</li><li>31 million repositories</li><li>161 million repository dependencies</li><li>10 million manifest files</li><li>46 million git tags</li></ul><p>The data is available in its raw format on <a href="https://zenodo.org/record/1068916">Zenodo</a> and we’re working on getting it published as a structured, queryable dataset on Google’s BigQuery. If you’d like to build tools on top of the most recent data, or top up your dataset to keep it current, check out the <a href="https://libraries.io/api">Libraries.io REST API</a>.</p><p>For further documentation, check out our dedicated <a href="https://libraries.io/data">open data page</a>. Also check out<a href="https://opensource.com/article/17/7/librariesio-data"> the article Ben wrote for opensource.com</a> to get more ideas of things you can do with the data.</p><p>This data is published under a <a href="https://creativecommons.org/licenses/by-sa/4.0/">Creative Commons BY-SA-4.0 licence</a>. It’s an open and free licence that commits the user to redistributing their work, and their understanding. And don’t forget, Libraries.io is <a href="https://github.com/librariesio/libraries.io">open source</a>, so if you’d like to get involved we can help you get started — check out the <a href="https://docs.libraries.io/contributorshandbook">Contributors Handbook</a>:</p><p>Finally, if you’d like regular updates from Tidelift on news like this, <a href="https://tidelift.com/#mailing">sign up here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=703422b1ad88" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/our-second-libraries-io-open-data-release-has-arrived-703422b1ad88">Our second Libraries.io open data release has arrived</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Libraries.io Releases Data on Over 25m Open Source Software Repositories]]></title>
            <link>https://medium.com/libraries-io/libraries-io-releases-data-on-over-25m-software-repositories-ab1db665826e?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/ab1db665826e</guid>
            <category><![CDATA[librariesio]]></category>
            <category><![CDATA[open-data]]></category>
            <category><![CDATA[sustainoss]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[digital-infrastructure]]></category>
            <dc:creator><![CDATA[Benjamin Nickolls]]></dc:creator>
            <pubDate>Wed, 29 Nov 2017 12:07:56 GMT</pubDate>
            <atom:updated>2017-06-26T13:30:30.229Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<p>Today’s software relies on a core set of of free, openly licensed components, frameworks and systems. But our shared, digital infrastructure is under threat. It’s overburdened and under-supported.</p><p><a href="https://www.fordfoundation.org/library/reports-and-studies/roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure/">Nadia Eghbal’s <em>Roads and Bridges</em> study for the Ford Foundation</a> gave us a series of personal vignettes on the state of open source — stressed maintainers, fractured communities and financial trouble. The stories we read resonated with our own experiences, our concerns legitimised and amplified.</p><p>Something has to change.</p><p>For nearly three years, <a href="https://libraries.io/">Libraries.io</a> has been gathering data on the complex web of interdependency that exists in open source software. We’ve published a series of experiments using harvested metadata to highlight projects <a href="https://libraries.io/help-wanted">in need of assistance</a>, <a href="https://libraries.io/bus-factor">projects with too few contributors</a> and <a href="https://libraries.io/digital-infrastructure">too little attention.</a></p><p>Many of these are projects that are essential to making much of today’s software work. And these are projects that need our help. This problem is already attracting the attention of our community, as shown by <a href="https://sustainoss.org/">this month’s Sustain conference</a> in San Francisco.</p><p>Today Libraries.io is releasing data on over two million unique infrastructure-type projects under a permissive licence. We believe that this information will empower and accelerate the work of those seeking solutions, and kickstart the conversation as to what will lead a sustainable economy for key open source projects.</p><p>The data is available in its raw format at on <a href="https://zenodo.org/record/808273">Zenodo</a> and soon we’ll be publishing a structured, queryable dataset on Google’s BigQuery. This data is published under a <a href="https://creativecommons.org/licenses/by-sa/4.0/">Creative Commons BY-SA-4.0 licence</a>. It’s an open and permissive licence that commits the user to re-distributing their work, and their understanding.</p><p>We’ll be updating this data on a regular basis. Find out more at <a href="http://libraries.io/open-data">https://libraries.io/data</a>.</p><p>As a community, we’re yet to agree a way to ensure the continuing health of individual open source projects. But we hope this will be another step in the right direction.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab1db665826e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/libraries-io-releases-data-on-over-25m-software-repositories-ab1db665826e">Libraries.io Releases Data on Over 25m Open Source Software Repositories</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Libraries.io is joining Tidelift]]></title>
            <link>https://medium.com/libraries-io/libraries-io-is-joining-tidelift-8da700862cde?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/8da700862cde</guid>
            <category><![CDATA[high-fidelity]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[sustainability]]></category>
            <category><![CDATA[librariesio]]></category>
            <dc:creator><![CDATA[Benjamin Nickolls]]></dc:creator>
            <pubDate>Wed, 15 Nov 2017 11:08:57 GMT</pubDate>
            <atom:updated>2017-11-14T14:41:09.601Z</atom:updated>
            <content:encoded><![CDATA[<p>In my last two posts I rejected the notion that <a href="https://medium.com/@BenJam/is-open-source-a-public-good-210594a69fd7">open source projects are a public good</a>. I then identified some of <a href="https://medium.com/@BenJam/sustaining-open-source-software-ac4a46a7c01a">the challenges that open source projects have to navigate</a> and charted some of the courses projects have taken to become sustainable. I signed off with the simple question:</p><p>Can we do better?</p><p>Can we build a future where open source maintainers can choose to leave the work they do for the work they love without having to become a sales team, a SaaS provider, and a support desk all at once? Can we build a network of open source software that users know will continue to provide for them in a year, maybe even a decade? Where projects are able to commit to big goals, make brave decisions, and own their project roadmap?</p><p>Maybe. Yes. <em>Maybe</em>.</p><p>I’m not about to tell you I have the solution.</p><p>These are difficult problems to solve. Problems that we have known about for decades and have fought to <a href="https://www.fordfoundation.org/library/reports-and-studies/roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure/">bring to the attention of the world</a> for years.</p><p>At <a href="https://libraries.io/">Libraries.io</a> we have been building software that creates a stronger, more informed open source ecosystem. At <a href="https://dependencyci.com/">Dependency CI</a> we have been building tools that lower the barriers to using and maintaining open source software. But <a href="https://docs.libraries.io/strategy">our role in solving the bigger problem</a> has been only as an enabler, helping others prioritise their work and understand their impact. This is about to change.</p><p>Today <a href="https://medium.com/@teabass">Andrew</a> and I are joining <a href="https://tidelift.com/">Tidelift</a> to tackle this problem head-on, and we’re bringing Libraries.io and Dependency CI with us. Tidelift is a new company, with a mission of making open source software work better for developers and users. You can read a little more over on the <a href="https://www.tidelift.com/blog/2017/11/14/hello-tidelift">Tidelift blog</a>.</p><p>Now as part of Tidelift, we will continue our work with our new colleagues and supporters. Libraries.io will remain an open source project, we’ll continue our open data program, and we’ll continue to collaborate with partners in the open source and research communities. And we’re excited to take Dependency CI to the next level, with an eye to supporting both creators and users of open source.</p><p>We hope you’ll continue to support us, our new colleagues and each other as we move toward a more sustainable future for open source software.</p><p>Because difficult problems are easier to solve together.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8da700862cde" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/libraries-io-is-joining-tidelift-8da700862cde">Libraries.io is joining Tidelift</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Libraries.io and Dependency CI are joining Tidelift]]></title>
            <link>https://medium.com/libraries-io/why-libraries-io-and-dependency-ci-are-joining-tidelift-1caf72f4d1b0?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/1caf72f4d1b0</guid>
            <category><![CDATA[librariesio]]></category>
            <category><![CDATA[announcements]]></category>
            <category><![CDATA[dependencyci]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Andrew Nesbitt]]></dc:creator>
            <pubDate>Tue, 14 Nov 2017 14:06:27 GMT</pubDate>
            <atom:updated>2017-11-14T14:06:27.497Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F7Nkg9PI_k_jBDohHO4SIQ.jpeg" /></figure><p>Today we have some pretty big news. <a href="https://twitter.com/benjam">Ben</a> and I are joining <a href="https://tidelift.com/">Tidelift</a>, a new company focused on making open source software work better for developers and users, and we’re bringing <a href="https://libraries.io/">Libraries.io</a> and <a href="https://dependencyci.com/">Dependency CI</a> into the fold as well. For more details check out the <a href="https://tidelift.com/blog/2017/11/14/hello-tidelift">Tidelift blog post</a> and <a href="https://medium.com/@BenJam/libraries-io-joins-tidelift-8da700862cde">Ben’s blog post</a>.</p><p>This post is a little more about how we got here and why I think this is the right next step.</p><p>I started working on Libraries.io in my spare time back in 2014, a couple of months after I left GitHub. I knew that open source discovery was an important problem to solve, and I wanted to concentrate my efforts on it.</p><p>Based on what I’d learnt building <a href="https://24pullrequests.com/">24 Pull Requests</a> and my analysis of the open source landscape while at GitHub, I knew that I’d need better usage data than just download and star counts, so Libraries.io started at the package manager level and started to index the network of interconnected dependencies across all of open source, in the same way that Google does for the web with PageRank.</p><p>The site was a hit, and quickly grew in size. Ever-increasing hosting costs and time requirements started to take their toll. I needed to find a way to make Libraries.io sustainable, whilst keeping the principles of the free and open communities that it supported.</p><p><a href="https://dependencyci.com/">Dependency CI</a> was my first real attempt at building a business on top of the Libraries.io data set, allowing projects and companies to codify policies on their open source usage, making sure they didn’t add any bad dependencies to their apps, while integrating directly into their workflow rather than as an afterthought when it was time to do some due diligence.</p><p>The problem I quickly ran into there was that I didn’t have anywhere near the resources to provide support for the big, paying customers who need on-premise enterprise deployments and round-the-clock support.</p><p>It was around that time that I met Ben, who’d been investigating ways of finding potentially vulnerable, high-profile projects in the wake of the <a href="https://en.wikipedia.org/wiki/Heartbleed">OpenSSL Heartbleed</a> fallout. Libraries.io was the perfect tool for highlighting high-risk projects because of the connected network of usage data. Together we started planning how we could use that data to help with the wider issue of sustainability in open source.</p><p>Thanks in part to an introduction from <a href="https://medium.com/@nayafia">Nadia Eghbal</a>, we managed to secure a joint grant between the <a href="https://www.fordfoundation.org/work/challenging-inequality/internet-freedom/">Ford Foundation</a> and <a href="https://sloan.org/programs/digital-technology">The Alfred P. Sloan Foundation</a>, the first of its kind. This gave us a one-year runway.</p><p>The high-level plan we put together was broken out into three sections:</p><ul><li><strong>Discovery</strong>: Helping developers make faster, more informed decisions about the software that they use.</li><li><strong>Maintainability</strong>: Helping maintainers understand more about the software they depend upon and the consumers of their software.</li><li><strong>Sustainability</strong>: Supporting undervalued software by highlighting shortfalls in contribution and funneling support to them.</li></ul><p>Libraries.io already had <strong>Discovery</strong> covered and Dependency CI focused on <strong>Maintainability</strong>. As we reached the end of the grant, after completing our funding commitments, we started thinking about the third goal, <strong>Sustainability,</strong> and how to ensure that Libraries.io could continue to operate a free and open service that the wider open source community has come to depend upon.</p><p>Once again, an introduction from Nadia, this time to <a href="https://tidelift.com/">Tidelift</a>, helped us take the next step towards supporting sustainability in open source.</p><p>Over the past few years we’d spoken to a number of companies who were interested in employing us based on the work we’d done on Libraries.io, but none of them felt quite right. There are quite a few companies building tools to help companies manage their open source usage but they almost always sell a band-aid over the underlying problem.</p><p>Problems like poor security, conflicting licensing, lack of backwards compatibility and support often stem from the fact that the majority of open source maintainers are working as unpaid volunteers.</p><p>Rather than just selling products that try to shield companies from open source projects, Tidelift is aiming to solve the root causes and give maintainers the resources they need to strengthen the foundations of our digital infrastructure and enable all kinds of new innovation at the same time.</p><p>So what’s going to happen to Libraries.io and Dependency CI now?</p><p>We’re going to continue working on Libraries.io in the open, under an open source license and releasing open data as before, now with more resources to make it even faster and more stable.</p><p>Dependency CI may take on a different form in the future as part of Tidelift, though we intend to continue providing tools to help companies and open source projects with their usage of open source.</p><p>I’m looking forward to sharing more of what we’re working on at <a href="https://tidelift.com/">Tidelift</a> over the next few months and working with a great team to make all of open source even better and stronger.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1caf72f4d1b0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/why-libraries-io-and-dependency-ci-are-joining-tidelift-1caf72f4d1b0">Why Libraries.io and Dependency CI are joining Tidelift</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What does a sustainable open source project look like?]]></title>
            <link>https://medium.com/libraries-io/what-does-a-sustainable-open-source-project-look-like-bf9b8cf824f8?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/bf9b8cf824f8</guid>
            <category><![CDATA[governance]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[community]]></category>
            <category><![CDATA[sustainability]]></category>
            <dc:creator><![CDATA[Andrew Nesbitt]]></dc:creator>
            <pubDate>Fri, 10 Nov 2017 17:19:44 GMT</pubDate>
            <atom:updated>2017-11-10T17:19:44.276Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sAuEg3TGbR6VaZg7UXE4eA.jpeg" /></figure><p>Over the past couple years, thanks in part to Nadia Eghbal’s <a href="https://www.fordfoundation.org/library/reports-and-studies/roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure/">Roads and Bridges</a> report, the world is starting to see the need for long-term sustainability in key open source projects that are the foundation of the ecosystems that our society relies on.</p><p>Getting financial support to work on an open source project can be difficult, <a href="https://github.com/nayafia/lemonade-stand">but it’s not impossible</a>. Thankfully a growing number of people are coming up with ways to make working on open source projects a financially secure prospect. But many of these models involve offering something in return for money.</p><p>Advertising offers marketing exposure to sponsors. Consulting, books, and training offer insights and knowledge not easily available from the project itself. SaaS services offer a hassle-free hosting option and <a href="https://en.wikipedia.org/wiki/Open_core">‘open core’</a> models offer advanced add-ons under proprietary licenses.</p><p>The key thing for me is that many of these models don’t actually support the project directly. Often the time invested in open source project development is paid for with the money left over after delivering something else you <em>sold</em>.</p><p>Why is this the case? By being freely available for everyone, open source makes it a challenge to sell direct improvements, as only the first customer needs to fund the improvements and everyone else gets them for free.</p><p>So we end up in a sort of standoff, with everyone waiting for someone else to support the project unless there is a time sensitive issue that’s cheap to fix. This is one of the biggest challenges in funding open source work and a particularly tricky problem to work around in a capitalist economy.</p><p>So I decided to articulate what a successful, sustainable open source project looks like, including what kinds of work do people do on it, and what kind of community of users and contributors it has. Then with that as an end point, we can work backwards to figure out how to bring that world to life.</p><p>I’m going to focus on high level things that apply to as many projects as possible rather than practices that may be specific to a programming language or ecosystem.</p><p>So here’s my list of things that make up the ideal sustainable open source project:</p><h3>Governance</h3><ul><li>The project has structures in place for making high-level decisions and enforcing communication standards, codes of conduct etc.</li><li>Decisions are publicly documented and communicated to all interested parties.</li><li>The maintainers of the project have taken steps to ensure it does not rely on any single person to be able to get work done.</li><li>Plans for future development direction, ideas, and goals are kept up to date in a roadmap document.</li><li>Taking steps to ensure that the project fosters a <a href="http://sarah.thesharps.us/2015/10/06/what-makes-a-good-community/">good, diverse community</a> and is welcoming and friendly to users and contributors alike.</li></ul><h3>Documentation</h3><ul><li>The project has good-quality documentation, covering all the public APIs and interfaces. They are updated with each release.</li><li>Commit messages should describe what and why the change was made as per <a href="https://chris.beams.io/posts/git-commit/">Chris’ great guidance</a>.</li><li>Human-focused <a href="http://keepachangelog.com/en/1.0.0/">release notes</a> should be published with every release, listing notable changes and deprecations.</li><li>If possible, documentation should be available in multiple languages or at least open to contributions from translators.</li></ul><h3>Code Quality</h3><ul><li>Code should have a consistent style throughout the project, ideally programmatically enforced with linters and documented with style guides where necessary.</li><li>The project should have good test coverage, with tests being run automatically on a CI environment after every commit.</li><li>There should be a documented code review process for all contributions involving both automated checks and human approval process to keep code quality levels high.</li></ul><h3>Support</h3><ul><li>Contributions and support requests and should be responded to in a timely manner even if a fix isn’t possible straight away.</li><li>Outstanding tickets should be triaged on a regular basis to ensure stale issues don’t fall through the cracks.</li><li>If there is a long-term support release available, the policies around it should be documented and future release dates included in roadmaps.</li><li>The project should document all supported runtime/language versions and major external dependency version compatibility, which should also have automated testing setup.</li><li>New release candidates should be tested against as many upstream dependency versions as realistically possible to ensure backwards compatibility or enable communication of breaking compatibility changes.</li><li>It’s also a good practise to keep an eye out for posts on Q&amp;A sites like Stack Overflow where users often go to get support with open source projects.</li></ul><h3>Ecosystem Collaboration</h3><ul><li>Maintainers should identify and coordinate with related projects to reduce potential for conflicts on new releases and breaking changes</li><li>For projects that are heavily depended upon, automated integration testing against key downstream dependencies should be set up for early warning detection of unseen breakages and conflicts.</li><li>Projects should have a clear process for proposing and discussing large changes such as an RFC.</li></ul><h3>Security</h3><ul><li>There should be a documented process for privately reporting security issues to the project’s maintainers as well as clear guidelines for maintainers on how to handle reported security issues.</li><li>Projects should acquire a <a href="https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures">CVE</a> for all known security vulnerabilities and document which released versions the CVEs apply to.</li><li>Commits and releases should be signed by the authors so that users can verify whether what they downloaded matches the same contents you released.</li><li>Maintainers should have 2FA and strong passwords on all related accounts (GitHub, package manager registries, email etc)</li><li>If releases include publishing compiled binaries, there should be a provenance chain for those binaries, ideally compatible with the <a href="https://reproducible-builds.org/">Reproducible Builds</a> program.</li><li>If necessary a <a href="https://en.wikipedia.org/wiki/Threat_model">threat model</a> should be documented to highlight where the software is most vulnerable to attack and how to mitigate those threats.</li><li>Any reports produced whilst researching the security aspects of the project should be published within a reasonable timeframe.</li></ul><h3>Legal</h3><ul><li>The project should be made available under one of the <a href="https://opensource.org/licenses/alphabetical">OSI-Approved Licenses</a>, that’s any license that fits with the <a href="https://opensource.org/osd">Open Source Definition</a>.</li><li>All licenses and trademarks for the project should be properly documented and ideally available in machine readable format like <a href="https://spdx.org/">SPDX</a> as well.</li><li>There should be a succession plan in place in case of the death of maintainer to allow other maintainers to legally take control of the assets of the project <a href="https://www.wired.com/story/giving-open-source-projects-life-after-a-developers-death/">incase the worst happens</a>.</li></ul><h3>Finance</h3><ul><li>As with any entity that is dealing with money and people, correct accounting and tax reporting should be done based on the laws of the countries that the maintainers reside in.</li><li>In some cases a legal organisation should be set up to to protect the liabilities of individuals involved, either as a regular business or a not-for-profit organisation.</li><li>With people being paid to work on the project, potentially from a variety of countries, policies around pay rates and expenses should be set up.</li><li>For ultimate transparency, an open ledger of all project income and outgoings could be used to show exactly how funds are being spent on the project.</li></ul><h3>Marketing</h3><ul><li>Having a recognizable brand can help the project build a strong audience of users and contributors, that includes having a logo and website of it’s own to help users understand what the project is about and also control the brand of the project outside of the GitHub repository page.</li><li>Projects should aim to keep their users and contributors up to date with what’s going on with the project, including larger announcements and highlighting interesting goings on in the development process as well as sharing useful related content via an email newsletter, blogging and twitter.</li><li>Surveys can also be a useful tool to collect quantitative and qualitative information about how and why the use the project and what else they would find useful to help inform future roadmapping decisions</li></ul><h3>Dependency Hygiene</h3><ul><li>If the project has dependencies it should ensure that each dependency is properly licensed and that license is compatible with the project.</li><li>Dependencies should also be checked for any potential security or compatibilities on a regular basis, including transitive dependencies.</li></ul><p>One thing you might notice here is that many of the tasks involved in running a sustainable project don’t involve writing any code at all, behind every successful, sustainable open source project are many people making non-code contributions that are necessary to keep everything working.</p><p>We can also start to see which attributes different stakeholders in the project are going to care about most, from there we can connect the dots and start to get a clearer picture of which funding models fit with a project whose stakeholders care about those attributes being present in the project.</p><p>It shouldn’t be too surprising that almost all of those attributes are also attributes of a successful software business. Every sustainable open source project ends up being run like a business in some ways, even if there aren’t any stakeholders that want to profit from it directly. Infact every successful open source project <em>is</em> a business. It’s just there are no paying customers.</p><p>Let’s change that!</p><p>If you’d like to add something to the list, I’ve put it up on GitHub, pull requests are very welcome: <a href="https://github.com/librariesio/sustainable-oss-attributes">https://github.com/librariesio/sustainable-oss-attributes</a></p><p>As always, follow me on Twitter at <a href="https://twitter.com/teabass">@teabass</a> and <a href="https://twitter.com/librariesio">@librariesio</a> for more updates.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bf9b8cf824f8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/what-does-a-sustainable-open-source-project-look-like-bf9b8cf824f8">What does a sustainable open source project look like?</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Benefit Of Doubt]]></title>
            <link>https://medium.com/libraries-io/word-counter-version-0-1-e386cbc5c1c0?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/e386cbc5c1c0</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Thomas Buckley-Houston]]></dc:creator>
            <pubDate>Mon, 17 Jul 2017 10:58:24 GMT</pubDate>
            <atom:updated>2017-07-17T10:24:33.504Z</atom:updated>
            <content:encoded><![CDATA[<h3>Word Counter Version 0.1</h3><p>Consider the following code to serve an API that can count words;</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b7ee6a21dd951de671751fa969cd9331/href">https://medium.com/media/b7ee6a21dd951de671751fa969cd9331/href</a></iframe><p>The problem of counting words is a surprisingly difficult one. The ‘algorithm’ used here, namely, text.split(&#39; &#39;).length, is an understandable first attempt, but it clearly has a multitude of problems. The most immediate being its inability to deal with the occurrence of double spaces, or indeed any kind of white space that isn&#39;t a single space character. For example Accidental&amp;nbsp;&amp;nbsp;&amp;nbsp;spaces counts as 4 words and Some\nnew\nlines counts as 0. So an improvement might be to use text.match(/\S+/g).length, where \S has the special regex meaning of &#39;non-whitespace&#39;. Now we&#39;re counting sequences of characters that don&#39;t come under the definition of whitespace, hopefully that means <em>words</em>. But this still isn&#39;t the &#39;right&#39; answer! What about accidental missed spaces, like,this? Ultimately, the point being, this seemingly innocent problem leads to cases such as schEissE!myKEybrdis破碎, whereupon ideas such as dictionary lookups and machine learning nervously stir, hoping not to anger the gods of nondeterministic polynomial time.</p><h3>Negative Capability</h3><p>I’m writing this post to explore the idea of doubt, or uncertainty, in the world of software. Perhaps somewhat obviously I am interested in this theme from a human perspective, rather than whether computer science formally encodes such concepts. (Although I would love to know more about such doubt-teasing ideas as the Bounded-error Probabilistic Polynomial time complexity class or how Bell’s Inequality experiments suggest even Quantum uncertainty doesn’t have so-called Hidden Variables.) Doubting is one of those signature traits of human behaviour that largely seems to get a bad press. However, in contrast, I believe doubt isn’t just manageable but, in certain circumstances, beneficial.</p><p>The first doubt one might have in such a doubt-positive philosophy is its potential to justify laziness and inexperience. So often good and even right answers come from, what appears to be, impossible or impractical situations. It is only after the sustained effort of sincere research and imagination that the fogs of uncertainty part to reveal an unmistakably certain solution. If we indulge so easily in justifying our doubts how can such breakthroughs ever occur? I want to call this <em>shallow doubt</em> because it generally arises from a lack of excavation into the problem.</p><p>However, I’m concerned here with a deeper category of doubt, perhaps even in the essential sense, a doubt that can neither be proven to be soluble <em>nor insoluble</em>. This may remind some of you of the famous <a href="https://en.wikipedia.org/wiki/Millennium_Prize_Problems">Millennium Prize Problems</a> of Mathematics — to solve any of the problems and claim a prize of one million US dollars, then proving the problems to be either false conjectures or impossible to solve is as equally valid as discovering solutions. These prized problems are indeed more reflective of the <em>deep doubt</em> I’m attempting to point to. But there is just one more human element I want to bring to bear, which I find best described by John Keats, the 19th Century Romantic Poet, in his idea of Negative Capability;</p><blockquote><em>that is, when a person is capable of being in uncertainties, mysteries, doubts, without any irritable reaching after fact and reason.</em></blockquote><p>Irritable? This is the key to the distinction for me. If I’m simply having a bad day, my irritation is inevitably going to lobby my discernment faculties to buy into the propaganda of an imminent solution. But what happens if there isn’t an imminent solution? What if the problem is yet to reveal its nuanced subtleties, each of which requires careful and patient attention? Well then clearly my knee-jerk reaction commits me to investing a lot of time and energy into backing a losing horse. In such circumstances I would have been better to swallow my pride and impatience in order to let the problem be just as it is — a currently insoluble problem.</p><p>“But I can’t just sit here doing nothing!”, I hear you cry. Yet this is precisely the coal-face of deep doubt, that wafer thin seam between acceptance and action. It’s not easy.</p><h3>Word Counter Version 0.2</h3><p>Consider this updated version of the previous word counter, it uses exactly the same algorithm;</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/af77666e328f9708295f7f49636ca8bb/href">https://medium.com/media/af77666e328f9708295f7f49636ca8bb/href</a></iframe><p>I’ve added 5 things.</p><ol><li>New Relic (it magically hooks into a lot of low level Node JS internals so it can track metrics like; HTTP response times, database queries, uncaught exceptions, etc).</li><li>The algorithm refactored into its own function.</li><li>Tests.</li><li>Console logs.</li><li>Comments.</li></ol><p>This isn’t one of those design pattern lessons. In fact I’d hope there’s nothing much new here for most programmers. These are just examples of general code improvements that can be made without working on the core algorithm. I would also add things to this list like; getting a good night’s sleep, eating properly and seeing friends. Deep doubt need not short circuit us into unproductive inaction like a rabbit caught in headlights. There’s no doubt about the <em>conditions</em> that at the very least <em>contribute</em> to good code.</p><h3>Word Counter Version 1.0</h3><p>A major version bump represents a significant landmark for software, either breaking changes or, in the case of version 1.0, a declaration to the world that it’s ready. Whole number change communicates something about the process of feedback between the author’s best guesses and how those implemented guesses interact with the world. The initial lack of conclusive certainty has coagulated into a confidence that can now be shipped.</p><p>Our Word Counter isn’t at that point yet. Which, for the purposes of this reflection, is actually where I want to dwell. I have a lot of ideas for what to try next, but ultimately I just don’t know what the ‘right’ way is. In the back of mind I can see archetypal bosses irritably reaching for me to get on with it. I feel the eyes of my peers pondering my hesitation as a possible symptom of imposteritis. Even <em>I</em> feel the itch to have something to show — so I can have that comforting feeling of productivity at the end of the day.</p><p>But. And this is the main point. None of these concerns … Serve. The. Software.</p><p>I think a good dose of courageous honesty is needed. Are we writing software, or trying to please our boss? Are we tinkering on a hobby project, or avoiding the real world of human relationships? Are we bootstrapping a disruptive startup, or seeking self-gratifying fame and fortune?</p><p>I certainly see it in myself, that ‘irritable reaching after fact and reason’, as Keats puts it, is often, if not always, a coping mechanism against the inextricable problems of simply being a human. I can’t control climate change or my love life, but I can control this computer. When a software problem is too hard, when I’ve exhausted all avenues, it’s hard to raise my head above the parapet and away from the screen, to simply let the problem be a problem. I want there to be a solution, no I <em>need </em>there to be a solution.</p><h3>GNU’s Not Unix</h3><p>The ‘G’ in GNU stands for GNU’s (as in GNU’s Not Unix) and the ‘G’ in GNU’s stands for GNU’s and the G in that GNU’s stands for … Anyway, point taken. I don’t believe it’s an accident that there’s a connection between GNU — one of open source’s (<em>ahem</em> Free Software’s) primary totems — and the kind of doubt I’m exploring here. First of all of course we have the infamous concept of recursion in computer science (the ability for a function to call itself), which can be quite difficult to get your head around. But we also have the GNU General Public License (GPL), which, if you look at closely, is actually defined by a form of recursion itself. Paraphrasing; you can do whatever you want with this software as long as you continue to provide that same freedom to others. So the license just preserves the license. But what is the license? It is a license to preserve the license. But what is … etc and so on.</p><p>I think a common perception of the impetus behind Richard Stallman’s creation of the GPL is that it was some kind of anti-capitalist statement against the financial greed of traditional Copyright licenses. That may well be true, I don’t actually know enough about the history, but as I’ve come to understand my own software development habits, I see the GPL as applying more broadly to all human self-interest, full stop. To continue the recursive theme, what if even Stallman himself, in his impatient irritability assumed the GPL was a solution to the ‘software licensing problem’? Some argue that spiritual successors such as the Apache and MIT licenses are even more free than the GPL because they give the freedom for derivative works to restrict freedoms. Anyway, the point being that human self-interest is everywhere and it’s subtle, it’ll sabotage even the most noble of intentions.</p><p>To give a more concrete example of how I see the GPL and other such licenses protecting against <em>human</em> problems rather than just financial ones, consider the ability to resuscitate dormant projects on Github. Open source licenses mean that we aren’t at the mercy of the author, whether that’s a profit-focussed corporation slashing its developer budget, or a parenting-focussed mother choosing to spend more time with her children.</p><p>What the GPL and its ilk do is … Put. The. Software. First.</p><p>They have nothing to say about <em>what</em> the right solution to a problem is, they just create the <em>conditions</em> in which the software has the best opportunity to flourish — <em>whatever</em> flourishing might look like.</p><h3>Doing Something Good</h3><p>I certainly don’t believe that all software has to be permissively licensed. Money is a reality to be constructively engaged with, not something to be idealistically destroyed. However I do believe that the open source paradigm — and by extension the ability to doubt — is a good solution, in both the moral and technical sense. By contributing to open source we are giving to the greater good, a greater good which we in turn benefit from. This transaction is neither direct nor guaranteed, one could say there is some doubt in it. Yet on balance it offers a clear efficiency improvement. I mean, if every single one of us had to write our own OpenSSL-esque encryption library, what kind of a world would we be in now? To misuse computer science’s concept of efficiency, open source improves the total cumulative financial expenditure (and in turn human work) for certain common problems from O(N) to O(1), where N is the number of users.</p><p>The price, for want of a better word, that we pay for open source is uncertainty. It is an untameable beast. Our fickle and fallible human agendas role off like water on a duck’s back. This, for me, is something approaching the deepest sense of goodness. In as much as only humans can be evil, then anything that’s ultimately inert to human self-interest, must at least not be evil. For myself and I know for many others too, this is a significant motivation in being part of open source. It’s the opportunity to contribute to something of genuine and lasting value that transcends direct, contractual return on investment.</p><h3>A Garden Engineer</h3><p>To come back to our second version of the Word Counter. We still don’t know anything more about the problem’s domain. All we have is a bunch of scaffolding and software paraphernalia, like seismic sensors listening in for noteworthy tremors. For all software engineering’s association with binaries like off/on, passing/failing, compatible/incompatible, and so on, it’s easy to forget that so much of the work we do, or more to the point, most of the work we <em>can</em> do, is prosaic and objectively indeterminate in its contribution to the ultimate solution. It has no guarantee that it will honour our investment of time.</p><p>By design, deep doubt, like a recursive function, defies our ability to comfortably behold. It keeps the solution within our view, but reminds us that solving the solution at the expense of everything else isn’t always the best way to, in fact, solve the solution. Perhaps there is something of the gardener to software engineering. If a ripe harvest is our ‘solution’, then it would be arrogant of us to claim that we literally constructed potatoes and blossoms with our bare hands. Indeed, we are mostly ‘just’ facilitators of working software, we sow the seeds and cultivate the optimal conditions that, should circumstances be favourable, reap the very 2,581 words of which you now read.</p><h3>Appendix</h3><p>I chose a word counter as the example in this post because it was something my late father described to me when I was perhaps about 10 years old. He was passionate about programming and I was passionate that he was passionate. He walked me through the various stages of thought that one might go through in writing a simple word counter, as we have done here. He was setting the stage to reveal the surprising winner of a competition in the then popular Computer Weekly magazine.</p><p>The competition was to write a word counter where the winner would be the highest scoring code based on the metrics of; total lines of code, speed and accuracy. Translating into modern Nodejs, this was the winning entry;</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9a60bbf046e93d54731dc8c7fd4a0543/href">https://medium.com/media/9a60bbf046e93d54731dc8c7fd4a0543/href</a></iframe><p>It blew my little mind. It was an almost Neo-seeing-the-matrix moment, of feeling like I understood something deep about the secret inner workings of computers. Of course this solution, like the others we’ve discussed, is not perfect. Yet for all its brevity, it is as notably not-perfect as most other complex solutions. For instance, Vim counts 2,595 words in this post, only 16 words different from the competition winning method’s answer of 2,581.</p><p>Nowadays, as someone who writes software for a living, I can almost hear the cynical pragmatism in the winner’s mind. To me the simple solution betrays at least some, if not years, of hard fought problem solving. I like to think of it as someone who has grown to nurture a healthy scepticism of their mind’s irritable, knee-jerk reaching for fact and reason. Someone who doesn’t reach for the nearest design pattern, framework or algorithm as soon as they can. Someone who can bear doubt, at least for a moment, at least for long enough to let the initial wave of ‘obvious’ solutions pass.</p><p>So there you are ladies and gentleman — doubt. Another little tool for the toolbox?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e386cbc5c1c0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/word-counter-version-0-1-e386cbc5c1c0">The Benefit Of Doubt</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Libraries.io June Progress Update]]></title>
            <link>https://medium.com/libraries-io/libraries-io-june-progress-update-f4df0420ab4e?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/f4df0420ab4e</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[librariesio]]></category>
            <category><![CDATA[progress]]></category>
            <category><![CDATA[software]]></category>
            <dc:creator><![CDATA[Andrew Nesbitt]]></dc:creator>
            <pubDate>Mon, 10 Jul 2017 20:13:35 GMT</pubDate>
            <atom:updated>2017-07-11T08:23:15.914Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<p>Continuing from <a href="https://medium.com/libraries-io/libraries-io-april-may-progress-update-c501fc7a6926">last month</a> with the regular progress updates on Libraries.io since <a href="https://hackernoon.com/libraries-io-joins-brave-new-software-c8cb7bba93bf#.30dtnjsmm">securing a grant from both the Sloan Foundation</a> and <a href="https://medium.com/@BenJam/brave-sloan-ford-and-you-8a0eb266d73f#.nidpm2ozp">the Ford Foundation</a>, here’s what we got up to in June 2017.</p><h3>Notable Features</h3><p><strong>Open Data Release</strong></p><p>The big feature that we worked on in June was our first ever open data release.</p><p>25GB of metadata on all of the packages, versions and dependencies from the 33 package managers we support plus repositories, tags and dependency information from over 25 million open source repositories found across GitHub, GitLab and Bitbucket.</p><p>All this data is available in CSV format, split into a number of different files, with the total number of rows reaching 200 million!</p><p>You can download and reference the data from <a href="https://zenodo.org/record/808273">Zenodo</a> under a <a href="https://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International Licence</a>, more information and updates available on <a href="https://libraries.io/data">https://libraries.io/data</a>.</p><p>Ben also wrote up a great blogpost about the release: <a href="https://medium.com/@BenJam/libraries-io-releases-data-on-over-25m-software-repositories-ab1db665826e">https://medium.com/@BenJam/libraries-io-releases-data-on-over-25m-software-repositories-ab1db665826e</a></p><p><strong>Updated homepage</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LgdRznmTXyD4t41TH3V3ZA.png" /><figcaption>New, simplified homepage for the site</figcaption></figure><p>Along with the new data page, Ben updated the homepage of the site to better communicate the wide range of things you can do with <a href="https://libraries.io/">Libraries.io</a>, which aligns nicely with our documented set of <a href="https://docs.libraries.io/personas.html">user personas</a>.</p><p><strong>Sustain OSS</strong></p><p>Towards the end of the month both Ben and I travelled over to the US for the <a href="https://sustainoss.org/">Sustain OSS</a> conference in San Francisco, as described on the website:</p><blockquote>“Sustain” will be a one day conversation for open source software sustainers. There will be no keynotes, expo halls or talks. Only a guided discussion and concrete ideas about getting and distributing money or in-kind services to the Open Source community.</blockquote><p>We got to meet a lot of great people who are also focused on helping solve long term sustainability for important Digital Infrastructure projects.</p><p>One of the key things I took away from the discussions we had there is the need for more data of changes over time, allowing for analysis of trends and generation of feedback based on real data.</p><p>We then flew over to New York City and took part in a joint Ford/Sloan meeting to help them craft their strategy on supporting digital infrastructure.</p><p>Both of these meetings were really positive for us both in terms of what we’ve already achieved and informing our work for next year.</p><p>Ben has started a small, working group focussed on <a href="https://github.com/librariesio/metrics">collaboratively developing measures for classifying and measuring aspects of projects</a> which should fill in a few holes in our own knowledge and our data.</p><p><strong>Brighton Ruby Talk</strong></p><p>At the end of the month I put together a data driven talk for the <a href="https://brightonruby.com/">Brighton Ruby Conference</a> about some of the risks involved in depending on open source software</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fspeakerdeck.com%2Fplayer%2F754f03779e5542a89db24b2e9fa23d0d&amp;url=https%3A%2F%2Fspeakerdeck.com%2Fandrew%2Fcan-my-friends-come-too&amp;image=https%3A%2F%2Fspeakerd.s3.amazonaws.com%2Fpresentations%2F754f03779e5542a89db24b2e9fa23d0d%2Fslide_0.jpg&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=speakerdeck" width="710" height="596" frameborder="0" scrolling="no"><a href="https://medium.com/media/587bb13cc5ef68f7fc51349ae6664d40/href">https://medium.com/media/587bb13cc5ef68f7fc51349ae6664d40/href</a></iframe><h3>Other Notable Changes</h3><ul><li><a href="https://github.com/librariesio/libraries.io/pull/1519">Fixup Docker configuration for local dev</a></li><li><a href="https://github.com/librariesio/libraries.io/commit/c39e09eaf627cf5fc8e5821eb9d05bd6c7fe3cbf">Added paginated list of most used dependencies for a user</a></li></ul><h3>Statistics</h3><p>41,155 new libraries and 298,570 new versions found, bringing the total to 2,339,193 libraries indexed.</p><p>134 commits, 13 pull requests and 323 issues opened across all Libraries.io repositories on GitHub: <a href="https://github.com/librariesio">https://github.com/librariesio</a></p><h3>Contributors</h3><p>This month we had a quite a few patches from outside contributors whom we would like to thank:</p><ul><li><a href="https://github.com/splashinn">Kyle Maune</a></li><li><a href="https://github.com/brynary">Bryan Helmkamp</a></li><li><a href="https://github.com/abrophy">Arthur Brophy</a></li></ul><h3>Plans for July</h3><p>After full time working for 6 months on Libraries.io we’re halfway through our grants from Sloan and Ford, for the next 6 months we’ll be focusing on the commitments we’ve made to the Ford Foundation, as outlined in our <a href="https://github.com/librariesio/supporters/blob/master/Ford/Ford%20Foundation%20Project%20Grant%20Application.pdf">proposal</a>, which includes the following:</p><ul><li><strong>Establishing a baseline for open source metrics</strong></li><li><strong>Collecting and recording time series data around those metrics</strong></li><li><strong>Exploring routes to long term sustainability for the project</strong></li></ul><p>As always, follow us on Twitter at <a href="https://twitter.com/teabass">@teabass</a>, <a href="https://twitter.com/benjam">@benjam</a> and <a href="https://twitter.com/librariesio">@librariesio</a> for more updates.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4df0420ab4e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/libraries-io-june-progress-update-f4df0420ab4e">Libraries.io June Progress Update</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Libraries.io April/May Progress Update]]></title>
            <link>https://medium.com/libraries-io/libraries-io-april-may-progress-update-c501fc7a6926?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/c501fc7a6926</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[bitbucket]]></category>
            <category><![CDATA[librariesio]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[Andrew Nesbitt]]></dc:creator>
            <pubDate>Fri, 02 Jun 2017 15:49:41 GMT</pubDate>
            <atom:updated>2017-06-02T16:19:08.710Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<p>We missed our April update last month due to holidays, so we’ve got a combined April/Month progress report on Libraries.io since <a href="https://hackernoon.com/libraries-io-joins-brave-new-software-c8cb7bba93bf#.30dtnjsmm">securing a grant from both the Sloan Foundation</a> and <a href="https://medium.com/@BenJam/brave-sloan-ford-and-you-8a0eb266d73f#.nidpm2ozp">the Ford Foundation</a>, here’s what we got up to.</p><h3>Notable New Features</h3><p><strong>Importing GitLab and Bitbucket users and organisations</strong></p><p>After much refactoring and some painful migration of large sections of the database, our concept of repository owners is now longer tied to the specifics of the GitHub API, we now have support for Bitbucket Users and Teams and GitLab Users and Groups.</p><p>It’s also going to be much easier for us to add support for more kinds of repository owners in the future.</p><p><strong>Importing GitLab Bitbucket Issues and Pull Requests</strong></p><p>Along similar lines, the support for storing Issue and Pull Request metadata for repositories is no longer tied to the GitHub API, and we have support for indexing both GitLab and Bitbucket Issues and Pull Requests/Merge Requests, which will be useful in the future for calculating community engagement, responsive and activity metrics.</p><p><strong>Redesigned Project pages</strong></p><p>After Ben’s comprehensive <a href="https://github.com/librariesio/libraries.io/issues/1366">UX review</a> of the search experience, we’ve made a number of simplifications and moved some elements around to make searching and browsing the site much more enjoyable.</p><p>We’re also loading some of the intensive elements on the page as you scroll down the page which has reduce the load time by approximately 50%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T3zkwLjZLVsoJgVPZUW3Mw.png" /><figcaption><a href="https://libraries.io/npm/base62">https://libraries.io/npm/base62</a></figcaption></figure><p><strong>Search page refinements</strong></p><p>You can also now search both libraries and repositories directly from the navigation header from any page of the site.</p><p>We also took the time to enable selecting multiple filters from the interface, this allows you to do more comprehensive searches, for example, searching for <a href="https://libraries.io/search?languages=CoffeeScript%2CTypeScript&amp;q=testing">Testing Libraries written in CoffeeScript or Typescript</a>, or any <a href="https://libraries.io/repos/search?license=GPL-3.0%2CGPL-2.0%2CLGPL-3.0%2CAGPL-3.0%2CLGPL-2.1%2CGPL-3.0%2B%2CGPL-2.0%2B&amp;q=">repository available under some kind of GPL license</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*04G0eI6WLFm3fLqi6C3w4w.png" /><figcaption>Yes, that’s 18 libraries called “covfefe”!</figcaption></figure><p>The search pages are also completely driven by elasticsearch now, skipping the need to load records from postgres after querying elasticsearch as we did before, making the search pages considerably quicker to load.</p><h3>Civic Tech Research</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yu7CJT2iSzFz4a_0hb2NCA.png" /><figcaption>Contribution to Digital Infrastructure by the Civic Tech Community</figcaption></figure><p>One other large piece of work we did in April was researching <a href="https://medium.com/@BenJam/the-impact-of-civic-tech-on-open-source-81816ca2aed3">The Impact of Civic Tech on Open Source</a> which was presented by <a href="https://medium.com/@BenJam">Ben</a> at <a href="http://tictec.mysociety.org/">TICTec</a>.</p><p>We compiled <a href="https://libraries.io/digital-infrastructure">a list of the top one thousand most depended upon projects</a> amongst the 93m declared dependencies tracked by Libraries.io which we’re calling “Digital Infrastructure”, then looking into how the Civic Tech community uses and contributes to it.</p><p>Some highlights include:</p><blockquote>We found that amongst the 5,034 projects we call Civic Tech, 6,815 users have contributed 1,135,846 commits. This compares with 82,514 people who have contributed 2,731,564 commits to any open source dependency of a Civic Tech project.</blockquote><blockquote>The Civic Tech Community contributes around ~15% of the work needed to support its own foundations.</blockquote><blockquote>The Civic Tech Community contributes ~6% of work needed to support our shared, digital infrastructure.</blockquote><h3>Other Notable Changes</h3><ul><li><a href="https://github.com/librariesio/libraries.io/commit/a7c9f9930e049035d0a1410c4caf0cdd5a6eb8a2">Allow users to resync out of date repositories</a></li><li><a href="https://github.com/librariesio/libraries.io/issues/1434">Add support for GitHub and GitLab Topics</a></li><li><a href="https://github.com/librariesio/libraries.io/pull/1322">Extract JSON generation for API into serializer</a></li><li><a href="https://github.com/librariesio/bibliothecary/commit/fb9f13efef5b88109353009991d3eb04bbbdc09e">Added support for passing dependencies from package-lock.json</a></li><li><a href="https://github.com/librariesio/bibliothecary/pull/229">Added support for passing dependencies from *.csproj files</a></li><li><a href="https://github.com/librariesio/libraries.io/commit/3354c0a91dfad208803f14706e74c3812e58c575">Allow sorting of repository projects via the API</a></li><li><a href="https://github.com/librariesio/libraries.io/commit/4a99e5ebfb6155fdf9428e19d6271f6b5b7a34e5">Add schema_plus for better index configuration</a></li><li><a href="https://github.com/librariesio/libraries.io/commit/36e6c9360773fa8e0e47d2c0aa45a075761a3bc6">Add pghero dashboard for visualizing db stats</a></li><li><a href="https://github.com/librariesio/libraries.io/commit/20ae4887b78a0bd239e1282c1b3ac4a96313ac2a">Match HISTORY and COPYING when checking for repo meta data files</a></li><li><a href="https://github.com/librariesio/libraries.io/commit/06439a1b4efacfa5588ddcc4d84c5ca390bca94a">Add support for Maven dependencies</a></li><li><a href="http://Add #potentially_outdated? method to projects">Add #potentially_outdated? method to projects</a></li></ul><p>A full list of all changes right across the Libraries.io org on GitHub is available in two gists, <a href="https://gist.github.com/andrew/2d842f9aee411e7abc764dd2177ef04a">April</a> and <a href="https://gist.github.com/andrew/80a8adc97525999496adc860811d3683">May</a>, generated by: <a href="https://github.com/librariesio/org-pulse">https://github.com/librariesio/org-pulse</a></p><h3>Statistics</h3><p><strong>April: </strong>230 commits, 11 pull requests and 129 issues opened across all Libraries.io repositories on GitHub: <a href="https://github.com/librariesio">https://github.com/librariesio</a></p><p><strong>May: 279</strong> commits, 15 pull requests and 228 issues opened across all Libraries.io repositories on GitHub: <a href="https://github.com/librariesio">https://github.com/librariesio</a></p><p>We’ve now indexed <strong>9,375,765</strong> published versions of <strong>2,291,604</strong> libraries, <strong>23,470,130</strong> open source repositories, <strong>23,644,149</strong> issues/pull requests and <strong>93,965,312 </strong>dependencies from GitHub, GitLab and Bitbucket.</p><h3>Contributors</h3><p>We had a few patches from outside contributors in April and May whom we would like to thank:</p><ul><li><a href="https://github.com/tabrath">Trond Arne Bråthen</a></li><li><a href="https://github.com/benmann">Ben Mann</a></li><li><a href="https://github.com/rtfpessoa">Rodrigo Fernandes</a></li></ul><h3>Plans for June</h3><p>The main aim for June is to release our first public data dump of dependency graph and metric data. This will including information about the 2.3 million libraries we have indexed, their versions and dependencies and the 25 million open source repositories and all of their dependencies.</p><p>It will be available under a Creative Commons Attribution-ShareAlike license, we’ve already had some amazing researchers start to play around with the data and find some interesting facts around dependency complexity and expect to see many more fascinating insights once people start to drill down into it.</p><p>Keep track of our progress via this GitHub issue: <a href="https://github.com/librariesio/supporters/issues/9">https://github.com/librariesio/supporters/issues/9</a></p><p>We’ll also be heading over to America for the <a href="https://sustainoss.org/">Sustain OSS</a> conference and a Digital Infrastructure workshop with Ford and Sloan Foundations to share what we’ve been working on.</p><p>As always, follow us on Twitter at <a href="https://twitter.com/teabass">@teabass</a>, <a href="https://twitter.com/benjam">@benjam</a> and <a href="https://twitter.com/librariesio">@librariesio</a> for more updates.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c501fc7a6926" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/libraries-io-april-may-progress-update-c501fc7a6926">Libraries.io April/May Progress Update</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Libraries.io March Progress Update]]></title>
            <link>https://medium.com/libraries-io/libraries-io-march-progress-update-bb4869f03fc7?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/bb4869f03fc7</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[librariesio]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[Andrew Nesbitt]]></dc:creator>
            <pubDate>Mon, 03 Apr 2017 19:22:29 GMT</pubDate>
            <atom:updated>2017-04-03T19:23:08.942Z</atom:updated>
            <cc:license>https://creativecommons.org/licenses/by-sa/4.0/</cc:license>
            <content:encoded><![CDATA[<p>Continuing from <a href="https://medium.com/libraries-io/clibraries-io-february-progress-update-e51b458676c8">last month</a> with the regular progress updates on Libraries.io since <a href="https://hackernoon.com/libraries-io-joins-brave-new-software-c8cb7bba93bf#.30dtnjsmm">securing a grant from both the Sloan Foundation</a> and <a href="https://medium.com/@BenJam/brave-sloan-ford-and-you-8a0eb266d73f#.nidpm2ozp">the Ford Foundation</a>, here’s what we got up to in February 2017.</p><h3>Notable Features</h3><p><strong>Documentation</strong> - The first week of March we be continued our documentation push, getting involved in <a href="https://bids.github.io/docathon/">Docathon</a>.</p><p>Many improvements were made to our <a href="https://github.com/librariesio/documentation">documentation repo</a>, bringing the all the old wiki content over as well as adding new docs on <a href="https://docs.libraries.io/personas">Personas</a> and documenting <a href="https://github.com/librariesio/libraries.io/blob/master/docs/add-a-package-manager.md">how to add support for a package manager</a> to <a href="https://libraries.io/">Libraries.io</a>.</p><p><strong>Upgraded infrastructure</strong> - We had two big projects start using the <a href="https://libraries.io/api">Libraries.io API</a> this month, <a href="https://www.visualstudio.com/">Visual Studio</a> and <a href="https://glitch.com/">Glitch</a>, both of which utilize the unified package manager search functionality.</p><p>This started to put a strain on our already overloaded Elasticsearch cluster so this month we spent some time upgrading key pieces of infrastructure to handle the higher loads without slowing the site down.</p><p>This included upgrading ruby to 2.4.0 on all machines and elasticsearch from 1.7 to 2.4 on our cluster, all done automatically via the growing collection of ansible playbooks in our <a href="https://github.com/librariesio/infrastructure">infrastructure repo.</a></p><p><strong>Release and diff urls for versions and tags</strong> - For projects that have their source hosted on GitHub, GitLab or Bitbucket and push tags their when they release new versions we can do some cool things:</p><ul><li>Link to view the changelog or browse source for the tagged release: <a href="https://github.com/splitrb/split/releases/tag/v2.2.0">https://github.com/splitrb/split/releases/tag/v2.2.0</a></li><li>Link to a diff between the current and previous version: <a href="https://github.com/splitrb/split/compare/v2.1.0...v2.2.0">splitrb/split@v2.1.0…v2.2.0</a></li></ul><p>This month we added those links for versions and tags on projects and tags on repositories with a slightly redesigned list view:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/1*wu9K59ypeQmzUmbOmUFPiw.png" /></figure><h3>Other Notable Changes</h3><p>We started using <a href="https://appsignal.com/">Appsignal</a> for performance monitoring which has already helped us improve the performance of many of the background jobs we run:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GyYDo5flLGgDjWgq_Ov25Q.jpeg" /><figcaption>90th percentile response time of background jobs</figcaption></figure><p><a href="https://github.com/benjam">Ben</a> added contribution notices around the site to encourage people to add support for other package managers if they can’t find what they are looking for.</p><p><a href="https://github.com/maclover7">Jon</a> has been refactoring the code we use for monitoring the GitHub firehose and the many RSS/JSON feeds of new/updated packages to make it more efficient and less coupled to the sidekiq work names that we use.</p><p>Improvements to testing setup, removes a lot of boilerplate and recorded webmock requests, making it easier to add and change tests for everyone.</p><p>Switched from redcarpet to commonmarker for markdown rendering in readmes to support the official spec for <a href="https://github.com/blog/2333-a-formal-spec-for-github-flavored-markdown">GitHub Flavored Markdown</a>.</p><p><strong>Team page</strong></p><p>A full list of all changes right across the Libraries.io org on GitHub is available in <a href="https://gist.github.com/andrew/648d8bc111ae5504dc0b5c674d8a2063">this gist</a>, generated by: <a href="https://github.com/librariesio/org-pulse">https://github.com/librariesio/org-pulse</a></p><h3>Statistics</h3><p>37,187 new libraries and 401,341 new versions found, bringing the total to 2,194,374 libraries indexed.</p><p>446 commits, 30 pull requests and 182 issues opened across all Libraries.io repositories on GitHub: <a href="https://github.com/librariesio">https://github.com/librariesio</a></p><h3>Contributors</h3><p>This month we had a quite a few patches from outside contributors whom we would like to thank:</p><ul><li><a href="https://github.com/maclover7">Jon Moss</a></li><li><a href="https://github.com/chasenlehara">Chasen Le Hara</a></li><li><a href="https://github.com/samjacobclift">Sam Clift</a></li><li><a href="https://github.com/karanjthakkar">Karan Thakkar</a></li><li><a href="https://github.com/abrophy">Arthur Brophy</a></li></ul><h3>Plans for April</h3><p>In April I’m planning on finishing up some last bits of GitLab and Bitbucket support, then revisiting the support we have for each package manage, documenting it and filling in the gaps where we’re missing bits.</p><p>As always, follow us on Twitter at <a href="https://twitter.com/teabass">@teabass</a>, <a href="https://twitter.com/benjam">@benjam</a> and <a href="https://twitter.com/librariesio">@librariesio</a> for more updates.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb4869f03fc7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/libraries-io-march-progress-update-bb4869f03fc7">Libraries.io March Progress Update</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving Open Source Discovery]]></title>
            <link>https://medium.com/libraries-io/solving-open-source-discovery-db43a04cd9e7?source=rss----42fe172790d9---4</link>
            <guid isPermaLink="false">https://medium.com/p/db43a04cd9e7</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[development]]></category>
            <dc:creator><![CDATA[Andrew Nesbitt]]></dc:creator>
            <pubDate>Fri, 24 Mar 2017 12:44:59 GMT</pubDate>
            <atom:updated>2015-03-16T12:50:39.812Z</atom:updated>
            <content:encoded><![CDATA[<p>Today I’m launching <a href="http://libraries.io/"><strong>Libraries.io</strong></a>, a project that I’ve been working on for the past couple of months.</p><p>The intention is to help developers find new open source libraries, modules and frameworks and keep track of ones they depend upon.</p><p>The world of open source software depends on a lot of open source libraries. We are standing on the shoulders of giants, which helps us to reach further than we could otherwise.</p><p>The problem with platforms like Rubygems and NPM is there are so many libraries, with hundreds of new ones added every day. Trying to find the right library can be overwhelming.</p><p>How do you find libraries that help you solve problems? How do you then know which of those libraries are worth using?</p><p><a href="https://github.com/explore">GitHub Explore</a> and <a href="https://github.com/search">Search</a> are the best tools we have at the moment, but stars often turn searches into a popularity contest, telling you which projects people currently admire, rather than the ones they are actually using and depending on.</p><p>The search approach I’ve been working on within <a href="http://libraries.io/"><strong>Libraries.io</strong></a><strong> </strong>attempts to solve this, taking a leaf out of Google’s book.</p><p>The dependency graph of all of the available libraries in a package manager fits quite nicely with the PageRank algorithm. According to <a href="http://en.wikipedia.org/wiki/PageRank">Wikipedia</a>:</p><blockquote>PageRank works by counting the number and quality of links to a page to determine a rough estimate of how important the website is. The underlying assumption is that more important websites are likely to receive more links from other websites.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*f5P5MbtzLlVc27a2kbzwQg.png" /></figure><p>If we swap ‘linking to’ for ‘depending on’ you have a fairly good model for discovering which libraries are best to use in your apps.</p><p>Within <a href="http://libraries.io/"><strong>Libraries.io</strong></a><strong> </strong>I’ve aggregated over 700,000 projects, written in <a href="http://libraries.io/languages">130 languages </a>from across <a href="http://libraries.io/platforms">22 package managers</a>, including dependencies, releases, license information and source code repository infomation. This results in a rich index of almost every open source library available for use today.</p><p>The index is kept up to date with information about new libraries and new releases of existing libraries every day. This ensures upcoming projects get exposure, as well as existing well known libraries.</p><p>This also means you can subscribe to a project to receive notifications when new releases are published, including new tags on GitHub repos for platforms like <a href="http://libraries.io/bower">Bower</a> and <a href="http://libraries.io/go">Go</a> that don’t store release information in a central repository.</p><p>I welcome you to try it and let me know how we can improve it to make the world of open source an easier place to navigate.</p><p>Follow me on Twitter at <a href="https://twitter.com/teabass">@teabass</a> and <a href="https://twitter.com/librariesio">@librariesio</a> for updates. Discussion on Hacker News: <a href="https://news.ycombinator.com/item?id=9211084">https://news.ycombinator.com/item?id=9211084</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db43a04cd9e7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/libraries-io/solving-open-source-discovery-db43a04cd9e7">Solving Open Source Discovery</a> was originally published in <a href="https://medium.com/libraries-io">Libraries.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>