DOC on issue triaging process (scikit-learn#17907)

Co-authored-by: Nicolas Hug <[email protected]>

Author: GaelVaroquaux

.github/ISSUE_TEMPLATE/config.yml

@@ -1 +1,11 @@
 blank_issues_enabled: false
+contact_links:
+  - name: Stack overflow
+    url: https://stackoverflow.com/questions/tagged/scikit-learn
+    about: Please ask and answer usage questions on stackoverflow
+  - name: Mailing list 
+    url: https://mail.python.org/mailman/listinfo/scikit-learn
+    about: General discussions and announcements on the mailing list
+  - name: Gitter
+    url: https://gitter.im/scikit-learn/scikit-learn
+    about: Users and developers can sometimes be found on the gitter channel

.github/ISSUE_TEMPLATE/usage_question.md

@@ -1,6 +1,6 @@
 ---
 name: Usage question
-about: If you have a usage question
+about: If you have a usage question please use another channel
 title: ''
 labels: Question
 assignees: ''
@@ -15,6 +15,12 @@ assignees: ''
 - **For more information, see User Questions: http://scikit-learn.org/stable/support.html#user-question**
 
 The issue tracker is used only to report issues and feature requests. For
-questions, please use either of the above platforms. Most question issues are
-closed without an answer on this issue tracker. Thanks for your understanding.
+questions, please use either of the above platforms. **Issues that are
+usage questions are closed without an answer.**
+Indeed, this issue tracker is a tool used for the development of
+scikit-learn. The additional activity created by usage questions impedes
+this development.
+
+Thank you for your understanding.
+
 -->

doc/about.rst

@@ -40,13 +40,16 @@ in the FAQ.
 
 Triage Team
 -----------
-The following people are active contributors who also help with triaging
-issues, PRs, and general maintenance:
+
+The following people are active contributors who also help with
+:ref:`triaging issues <bug_triaging>`, PRs, and general
+maintenance:
 
 .. include:: triage_team.rst
 
 Emeritus Core Developers
 ------------------------
+
 The following people have been active contributors in the past, but are no
 longer active in the project:
 

doc/developers/bug_triaging.rst

@@ -0,0 +1,146 @@
+.. _bug_triaging:
+
+Bug triaging and issue curation
+================================
+
+The `issue tracker <https://github.com/scikit-learn/scikit-learn/issues>`_
+is important to the communication in the project: it helps
+developers identify major projects to work on, as well as to discuss
+priorities. For this reason, it is important to curate it, adding labels
+to issues and closing issues that are not necessary.
+
+Working on issues to improve them
+--------------------------------------
+
+Improving issues increases their chances of being successfully resolved.
+Guidelines on submitting good issues can be found :ref:`here
+<filing_bugs>`. A third party can give useful feedback or even add
+comments on the issue, while core-developpers or members of the triage
+team can edit the issue description and title.
+
+The following actions are typically useful:
+
+  - documenting issues that are missing elements to reproduce the problem
+    such as code samples
+
+  - correcting incorrect use of code formatting
+
+  - making sure that the title and description are explicit about the
+    problem to be solved
+
+  - linking to related issues or discussions while briefly describing how
+    they are related, for instance "See also #xyz for a similar attempt
+    at this" or "See also #xyz where the same thing happened in
+    SomeEstimator" provides context and helps the discussion.
+
+.. topic:: Fruitful discussions
+
+   Online discussions may be harder than it seems at first glance, in
+   particular given that a person new to open-source may have a very
+   different understanding of the process than a seasonned maintainer.
+
+   Overall, it is useful to stay positive and assume good will. `The
+   following article
+   <http://gael-varoquaux.info/programming/technical-discussions-are-hard-a-few-tips.html>`_
+   explores how to lead online discussions in the context of open source.
+
+Triaging operations for members of the core and triage teams
+-------------------------------------------------------------
+
+In addition to the above, members of the core team and the triage team
+can do the following important tasks:
+
+- Update labels for issues and PRs
+
+- Follow up on stalled PRs, to see if they must be relabeled as
+  stalled and needing help (this is typically very important in the context
+  of sprints, where the risk is to create many unfinished PRs)
+
+- Triage issues:
+
+  - **close usage questions** and politely point the reporter to use
+    Stack Overflow instead.
+
+  - **close duplicate issues**, but only after checking that they are
+    indeed duplicate. Ideally, the original submitter moves the
+    discussion to the older, duplicate issue
+
+  - **close issues that cannot be replicated**, after leaving time (at
+    least a week) to add extra information
+
+:ref:`Saved replies <saved_replies>` are useful to gain time and yet be
+welcoming and polite when triaging.
+
+
+.. topic:: Closing issues: a tough call
+
+    When uncertain on whether an issue should be closed or not, it is
+    best to strive for consensus with the original poster, and possibly
+    to seek relevant expertise. However, when the issue is a usage
+    question, or when it has been considered as unclear for many years it
+    should be closed.
+
+A typical workflow for triaging issues
+----------------------------------------
+
+The following workflow [*]_ is a good way to approach issue triaging:
+
+1. Thank the reporter for opening an issue
+
+   The issue tracker is many people’s first interaction with the
+   scikit-learn project itself, beyond just using the library. As such,
+   we want it to be a welcoming, pleasant experience.
+
+2. Is this a usage question? If so close it with a polite message
+   (:ref:`here is an example <saved_replies>`).
+
+3. Is the necessary information provided?
+
+   If crucial information (like the version of scikit-learn used), is
+   missing feel free to ask for that and label the issue with "Needs
+   info".
+
+4. Is this a duplicate issue?
+
+   We have many open issues. If a new issue seems to be a duplicate,
+   point to the original issue. If it is a clear duplicate, or consensus
+   is that it is redundant, close it. Make sure to still thank the
+   reporter, and encourage them to chime in on the original issue, and
+   perhaps try to fix it.
+
+   If the new issue provides relevant information, such as a better or
+   slightly different example, add it to the original issue as a comment
+   or an edit to the original post.
+
+
+5. Make sure that the title accurately reflects the issue. Edit it
+   yourself if it's not clear.
+
+6. Is the issue minimal and reproducible?
+
+   For bug reports, we ask that the reporter provide a minimal
+   reproducible example. See
+   https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports
+   for a good explanation. If the example is not reproducible, or if
+   it's clearly not minimal, feel free to ask the reporter if they can
+   provide and example or simplify the provided one. Do acknowledge that
+   writing minimal reproducible examples is hard work. If the reporter
+   is struggling, you can try to write one yourself.
+
+   If a reproducible example is provided, but you see a simplification,
+   edit the original post with your simpler reproducible example.
+
+4. If a reproducible example can't be provided, add the “Bug: triage”
+   label.
+
+5. Add the relevant labels, such as "Documentation" when the issue is
+   about documentation, "Bug" if it is clearly a bug, "Enhancement" if it
+   is an enhancement request, ...
+
+   If the issue is clearly defined and the fix seems relatively
+   straightforward, label the issue as “Good first issue”.
+
+   An additional useful step can be to tag the corresponding module e.g.
+   `sklearn.linear_models` when relevant.
+
+.. [*] Adapted from the pandas project https://dev.pandas.io/docs/development/maintaining.html

doc/developers/contributing.rst

@@ -55,11 +55,11 @@ find a typo in the documentation, or have made improvements, do not hesitate to
 send an email to the mailing list or preferably submit a GitHub pull request.
 Full documentation can be found under the doc/ directory.
 
-But there are many other ways to help. In particular answering queries on the
-`issue tracker <https://github.com/scikit-learn/scikit-learn/issues>`_,
-investigating bugs, and :ref:`reviewing other developers' pull requests
-<code_review>` are very valuable contributions that decrease the burden on the
-project maintainers.
+But there are many other ways to help. In particular helping to
+:ref:`improve, triage, and investigate issues <bug_triaging>` and
+:ref:`reviewing other developers' pull requests <code_review>` are very
+valuable contributions that decrease the burden on the project
+maintainers.
 
 Another way to contribute is to report issues you're facing, and give a "thumbs
 up" on issues that others reported and that are relevant to you.  It also helps
@@ -173,6 +173,9 @@ feedback:
   <https://help.github.com/articles/creating-and-highlighting-code-blocks>`_
   for more details.
 
+If you want to help curate issues, read :ref:`the following
+<bug_triaging>`.
+
 Contributing code
 =================
 

doc/developers/index.rst

@@ -19,5 +19,6 @@ Developer's Guide
    utilities
    performance
    advanced_installation
+   bug_triaging
    maintainer
    plotting

doc/developers/tips.rst

@@ -110,7 +110,10 @@ replies <https://github.com/settings/replies/>`_ for reviewing:
 Issue: Usage questions
     ::
 
-        You're asking a usage question. The issue tracker is mainly for bugs and new features. For usage questions, it is recommended to try [Stack Overflow](https://stackoverflow.com/questions/tagged/scikit-learn) or [the Mailing List](https://mail.python.org/mailman/listinfo/scikit-learn).
+        You are asking a usage question. The issue tracker is for bugs and new features. For usage questions, it is recommended to try [Stack Overflow](https://stackoverflow.com/questions/tagged/scikit-learn) or [the Mailing List](https://mail.python.org/mailman/listinfo/scikit-learn).
+
+        Unfortunately, we need to close this issue as this issue tracker is a communication tool used for the development of scikit-learn. The additional activity created by usage questions crowds it too much and impedes this development. The conversation can continue here, however there is no guarantee that is will receive attention from core developers.
+
 
 Issue: You're welcome to update the docs
     ::

doc/governance.rst

@@ -22,10 +22,19 @@ Roles And Responsibilities
 
 Contributors
 ------------
+
 Contributors are community members who contribute in concrete ways to the
 project. Anyone can become a contributor, and contributions can take many forms
 – not only code – as detailed in the :ref:`contributors guide <contributing>`.
 
+Triage team
+------------
+
+The triage team is composed of community members who have permission on
+github to edit, label and close issues. :ref:`Their work <bug_triaging>` is
+crucial to improve the communication in the project and limit the crowding
+of the issue tracker.
+
 Core developers
 ---------------
 Core developers are community members who have shown that they are dedicated to

doc/support.rst

@@ -46,6 +46,8 @@ what target are your trying to predict: binary, multiclass (1 out of
 ``n_classes``) or multilabel (``k`` out of ``n_classes``) classification
 or continuous variable regression.
 
+User questions should **not be asked on the bug tracker**, as it crowds
+the list of issues and makes the development of the project harder.
 
 .. _bug_tracker: