diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..dd84ea7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,38 @@ +--- +name: Bug report +about: Create a report to help us improve +title: '' +labels: '' +assignees: '' + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +Steps to reproduce the behavior: +1. Go to '...' +2. Click on '....' +3. Scroll down to '....' +4. See error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Desktop (please complete the following information):** + - OS: [e.g. iOS] + - Browser [e.g. chrome, safari] + - Version [e.g. 22] + +**Smartphone (please complete the following information):** + - Device: [e.g. iPhone6] + - OS: [e.g. iOS8.1] + - Browser [e.g. stock browser, safari] + - Version [e.g. 22] + +**Additional context** +Add any other context about the problem here. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index b93799b..5ddef0c 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -55,7 +55,7 @@ further defined and clarified by project maintainers. ## Enforcement Instances of abusive, harassing, or otherwise unacceptable behavior may be -reported by contacting the project team at [jon.tennant.2@gmail.com](mailto:jon.tennant.2@gmail.com) or [bianca.kramer@gmail.com](mailto:bianca.kramer@gmail.com). All +reported by contacting the project team at [chris@libscie.org](mailto:chris@libscie.org) or [bianca.kramer@gmail.com](mailto:bianca.kramer@gmail.com). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index dff4e27..d0025ae 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,4 +1,26 @@ # Contribution Guidelines +These are the main contributing guidelines for the development of this MOOC, and apply to each module within. The development structure for this is based on a combination of two things: +1. Invited experts as part of a core development team, led by one or two managers for each module. +1. Open participation, where anyone can contribute using the standard processes on GitHub. + +Feedback and contributions of any form are welcomed. Feel free also to [contact us](https://opensciencemooc.eu/contact-us/) to discuss anything further. + +At the present, development is in very early stages, as this is an entirely crowd-sourced and volunteer-led project. We are focusing inititally on [Module 5](https://github.com/OpenScienceMOOC/Module-5-Open-Research-Software-and-Open-Source) to run as a pilot for testing and receiving feedback. After this, the protocol and content will be revised, and then applied accordingly to the development of the remaining modules. + +## Contact us + +If you want to contribute, add yourself to the [contributors list](https://docs.google.com/spreadsheets/d/1FEbnxBkmIQFDEViTgZDn0AJ0MSK6eDSBEgZuCkW_B5s/edit?usp=drive_web) and join our open [Slack channel](https://openmooc-ers-slackin.herokuapp.com/). + +If you have questions about the project, please [email us](info@opensciencemooc.eu) directly. + +Stay tuned on what's happening on Twitter with [@OpenScienceMOOC](https://twitter.com/OpenScienceMOOC/). + +## Getting started +1. [Forming a team for collaborative design](https://github.com/OpenScienceMOOC/Main/blob/master/Production_Files/MODULE_DESIGN_PROTOCOL.md#forming-a-team-for-collaborative-design). +1. [The development process](https://github.com/OpenScienceMOOC/Main/blob/master/Production_Files/MODULE_DESIGN_PROTOCOL.md#the-development-process). +1. Familiarise yourself with the [script writing guide](https://github.com/OpenScienceMOOC/Main/blob/master/Production_Files/Writing_a_script.md), the [script template](https://github.com/OpenScienceMOOC/Main/blob/master/Production_Files/Script_template.md) and the [video management protocol](https://github.com/OpenScienceMOOC/Main/blob/master/Production_Files/Video_management_protocol.md). + +Each team will adhere to the [MOOC planning template](https://github.com/OpenScienceMOOC/Main/blob/master/Production_Files/MOOC%20planning%20template.docx) to structure development in a systematic way. ## Reporting issues @@ -10,11 +32,26 @@ Refer to the [README](README.md). ## Content style +This is flexible to each module as required, and defined by each development team in advance as part of the protocol. ## Code style +Flexible, as long as it is consistent. Ideally, all content would be drafted in markdown, for increasing re-use. This can be easily performed in R Studio, for example, which also has a GitHub interface to make collaborating on this project even simpler. +Please read [this guide](https://support.rstudio.com/hc/en-us/articles/200532077-Version-Control-with-Git-and-SVN) to familiarise yourself with this process. Which in itself, is actually fairly powerful for Open Science! ## Pull requests + +Please refer to each project's style guidelines and guidelines for submitting patches and additions. In general, we follow the "fork-and-pull" Git workflow. + +- Fork the repo on GitHub +- Clone the project to your own machine +- Commit changes to your own branch +- Push your work back up to your fork +- Submit a Pull request so that we can review your changes + +NOTE: Be sure to merge the latest from "upstream" before making a pull request! + + - Try not to pollute your pull request with unintended changes – keep them simple and small. If possible, squash your commits. - Try to share how your code has been tested before submitting a pull request. - If your PR resolves an issue, include **closes #ISSUE_NUMBER** in your commit message (or a [synonym](https://help.github.com/articles/closing-issues-via-commit-messages)). diff --git a/Glossary.md b/Glossary.md new file mode 100644 index 0000000..8321939 --- /dev/null +++ b/Glossary.md @@ -0,0 +1,41 @@ +# Terminology used in the Open Science MOOC explained +OS-MOOC specific terminology being clarified according to "our" (OS-MOOC community) view therefore streamlining Open Science, Open Research etc into the same concept. +And of course also linking to the research coalition and other OS glossaries for further info. + +Obvious terms like Open Access and such mentioned broefly referring to a wikipedia article and recent research literature for further info. + + +## Open Science + +## Open Research + +## Open Scholarship + +## Open Access + +## Open Source +(Software and Hardware used in Science) + +## Open Peer Review + +## Open Methodology + +## Open Data + + + +# Other Glossaries on Open Science topics + +Open Research Glossary by SPARCOpen - https://openresearchglossary.herokuapp.com/ + +Open Science Dictionary by the Center for Open Science - https://osf.io/t84kh/ + +Glossary of Open Access terms by the Imperial College London - https://www.imperial.ac.uk/research-and-innovation/support-for-staff/scholarly-communication/open-access/glossary/ + +SHERPA Glossary of Open Access Abbreviations, Acronyms and Terms - http://www.sherpa.ac.uk/glossary.html + +Open Glossary - https://blogs.egu.eu/network/palaeoblog/files/2015/02/OpenGlossary1.pdf + +Open Science Definition by FOSTER - https://www.fosteropenscience.eu/foster-taxonomy/open-science-definition + +Open Data Handbook - http://opendatahandbook.org/ diff --git a/Key_Reports/Altemerics - rewards and incentives for open science .pdf b/Key_Reports/Altemerics - rewards and incentives for open science .pdf new file mode 100644 index 0000000..882803e Binary files /dev/null and b/Key_Reports/Altemerics - rewards and incentives for open science .pdf differ diff --git a/Key_Reports/Amsterdam call for action on open science.pdf b/Key_Reports/Amsterdam call for action on open science.pdf new file mode 100644 index 0000000..800c88c Binary files /dev/null and b/Key_Reports/Amsterdam call for action on open science.pdf differ diff --git a/Key_Reports/Digital Education Action Plan.pdf b/Key_Reports/Digital Education Action Plan.pdf new file mode 100644 index 0000000..fc00e2b Binary files /dev/null and b/Key_Reports/Digital Education Action Plan.pdf differ diff --git a/Key_Reports/Eurodoc Statement on Framework Programme 9.pdf b/Key_Reports/Eurodoc Statement on Framework Programme 9.pdf new file mode 100644 index 0000000..9ec2466 Binary files /dev/null and b/Key_Reports/Eurodoc Statement on Framework Programme 9.pdf differ diff --git a/Key_Reports/Evaluation of Research Careers fully acknowledging Open Science Practices.pdf b/Key_Reports/Evaluation of Research Careers fully acknowledging Open Science Practices.pdf new file mode 100644 index 0000000..d1c4fd8 Binary files /dev/null and b/Key_Reports/Evaluation of Research Careers fully acknowledging Open Science Practices.pdf differ diff --git a/Key_Reports/FOSTER Roadmap for Open Science.pdf b/Key_Reports/FOSTER Roadmap for Open Science.pdf new file mode 100644 index 0000000..eb583cd Binary files /dev/null and b/Key_Reports/FOSTER Roadmap for Open Science.pdf differ diff --git a/Key_Reports/Horizon 2020 Work Program.pdf b/Key_Reports/Horizon 2020 Work Program.pdf new file mode 100644 index 0000000..10165ab Binary files /dev/null and b/Key_Reports/Horizon 2020 Work Program.pdf differ diff --git a/Key_Reports/LERU - Open Science and its role in universities.pdf b/Key_Reports/LERU - Open Science and its role in universities.pdf new file mode 100644 index 0000000..0406b00 Binary files /dev/null and b/Key_Reports/LERU - Open Science and its role in universities.pdf differ diff --git a/Key_Reports/MLE on Open Science - Altmetrics and Rewards - Final Report.pdf b/Key_Reports/MLE on Open Science - Altmetrics and Rewards - Final Report.pdf new file mode 100644 index 0000000..23247af Binary files /dev/null and b/Key_Reports/MLE on Open Science - Altmetrics and Rewards - Final Report.pdf differ diff --git a/Key_Reports/Providing researchers with the skills and competencies they need to practise Open Science.pdf b/Key_Reports/Providing researchers with the skills and competencies they need to practise Open Science.pdf new file mode 100644 index 0000000..93b9e32 Binary files /dev/null and b/Key_Reports/Providing researchers with the skills and competencies they need to practise Open Science.pdf differ diff --git a/Key_Reports/Recommendation on access to and preservation of Scientific Information.pdf b/Key_Reports/Recommendation on access to and preservation of Scientific Information.pdf new file mode 100644 index 0000000..f40fd8f Binary files /dev/null and b/Key_Reports/Recommendation on access to and preservation of Scientific Information.pdf differ diff --git a/LICENSE b/LICENSE.md similarity index 100% rename from LICENSE rename to LICENSE.md diff --git a/LICENSE_CONTENT.md b/LICENSE_CONTENT.md new file mode 100644 index 0000000..670154e --- /dev/null +++ b/LICENSE_CONTENT.md @@ -0,0 +1,116 @@ +CC0 1.0 Universal + +Statement of Purpose + +The laws of most jurisdictions throughout the world automatically confer +exclusive Copyright and Related Rights (defined below) upon the creator and +subsequent owner(s) (each and all, an "owner") of an original work of +authorship and/or a database (each, a "Work"). + +Certain owners wish to permanently relinquish those rights to a Work for the +purpose of contributing to a commons of creative, cultural and scientific +works ("Commons") that the public can reliably and without fear of later +claims of infringement build upon, modify, incorporate in other works, reuse +and redistribute as freely as possible in any form whatsoever and for any +purposes, including without limitation commercial purposes. These owners may +contribute to the Commons to promote the ideal of a free culture and the +further production of creative, cultural and scientific works, or to gain +reputation or greater distribution for their Work in part through the use and +efforts of others. + +For these and/or other purposes and motivations, and without any expectation +of additional consideration or compensation, the person associating CC0 with a +Work (the "Affirmer"), to the extent that he or she is an owner of Copyright +and Related Rights in the Work, voluntarily elects to apply CC0 to the Work +and publicly distribute the Work under its terms, with knowledge of his or her +Copyright and Related Rights in the Work and the meaning and intended legal +effect of CC0 on those rights. + +1. Copyright and Related Rights. A Work made available under CC0 may be +protected by copyright and related or neighboring rights ("Copyright and +Related Rights"). Copyright and Related Rights include, but are not limited +to, the following: + + i. the right to reproduce, adapt, distribute, perform, display, communicate, + and translate a Work; + + ii. moral rights retained by the original author(s) and/or performer(s); + + iii. publicity and privacy rights pertaining to a person's image or likeness + depicted in a Work; + + iv. rights protecting against unfair competition in regards to a Work, + subject to the limitations in paragraph 4(a), below; + + v. rights protecting the extraction, dissemination, use and reuse of data in + a Work; + + vi. database rights (such as those arising under Directive 96/9/EC of the + European Parliament and of the Council of 11 March 1996 on the legal + protection of databases, and under any national implementation thereof, + including any amended or successor version of such directive); and + + vii. other similar, equivalent or corresponding rights throughout the world + based on applicable law or treaty, and any national implementations thereof. + +2. Waiver. To the greatest extent permitted by, but not in contravention of, +applicable law, Affirmer hereby overtly, fully, permanently, irrevocably and +unconditionally waives, abandons, and surrenders all of Affirmer's Copyright +and Related Rights and associated claims and causes of action, whether now +known or unknown (including existing as well as future claims and causes of +action), in the Work (i) in all territories worldwide, (ii) for the maximum +duration provided by applicable law or treaty (including future time +extensions), (iii) in any current or future medium and for any number of +copies, and (iv) for any purpose whatsoever, including without limitation +commercial, advertising or promotional purposes (the "Waiver"). Affirmer makes +the Waiver for the benefit of each member of the public at large and to the +detriment of Affirmer's heirs and successors, fully intending that such Waiver +shall not be subject to revocation, rescission, cancellation, termination, or +any other legal or equitable action to disrupt the quiet enjoyment of the Work +by the public as contemplated by Affirmer's express Statement of Purpose. + +3. Public License Fallback. Should any part of the Waiver for any reason be +judged legally invalid or ineffective under applicable law, then the Waiver +shall be preserved to the maximum extent permitted taking into account +Affirmer's express Statement of Purpose. In addition, to the extent the Waiver +is so judged Affirmer hereby grants to each affected person a royalty-free, +non transferable, non sublicensable, non exclusive, irrevocable and +unconditional license to exercise Affirmer's Copyright and Related Rights in +the Work (i) in all territories worldwide, (ii) for the maximum duration +provided by applicable law or treaty (including future time extensions), (iii) +in any current or future medium and for any number of copies, and (iv) for any +purpose whatsoever, including without limitation commercial, advertising or +promotional purposes (the "License"). The License shall be deemed effective as +of the date CC0 was applied by Affirmer to the Work. Should any part of the +License for any reason be judged legally invalid or ineffective under +applicable law, such partial invalidity or ineffectiveness shall not +invalidate the remainder of the License, and in such case Affirmer hereby +affirms that he or she will not (i) exercise any of his or her remaining +Copyright and Related Rights in the Work or (ii) assert any associated claims +and causes of action with respect to the Work, in either case contrary to +Affirmer's express Statement of Purpose. + +4. Limitations and Disclaimers. + + a. No trademark or patent rights held by Affirmer are waived, abandoned, + surrendered, licensed or otherwise affected by this document. + + b. Affirmer offers the Work as-is and makes no representations or warranties + of any kind concerning the Work, express, implied, statutory or otherwise, + including without limitation warranties of title, merchantability, fitness + for a particular purpose, non infringement, or the absence of latent or + other defects, accuracy, or the present or absence of errors, whether or not + discoverable, all to the greatest extent permissible under applicable law. + + c. Affirmer disclaims responsibility for clearing rights of other persons + that may apply to the Work or any use thereof, including without limitation + any person's Copyright and Related Rights in the Work. Further, Affirmer + disclaims responsibility for obtaining any necessary consents, permissions + or other rights required for any use of the Work. + + d. Affirmer understands and acknowledges that Creative Commons is not a + party to this document and has no duty or obligation with respect to this + CC0 or use of the Work. + +For more information, please see + diff --git a/MOOC_presentation.odp b/MOOC_presentation.odp new file mode 100644 index 0000000..ad4d27c Binary files /dev/null and b/MOOC_presentation.odp differ diff --git a/MOOC_presentation.pdf b/MOOC_presentation.pdf new file mode 100644 index 0000000..8fee8b0 Binary files /dev/null and b/MOOC_presentation.pdf differ diff --git a/MOOC_presentation.pptx b/MOOC_presentation.pptx new file mode 100644 index 0000000..8263ff4 Binary files /dev/null and b/MOOC_presentation.pptx differ diff --git a/Main.Rproj b/Main.Rproj new file mode 100644 index 0000000..8e3c2eb --- /dev/null +++ b/Main.Rproj @@ -0,0 +1,13 @@ +Version: 1.0 + +RestoreWorkspace: Default +SaveWorkspace: Default +AlwaysSaveHistory: Default + +EnableCodeIndexing: Yes +UseSpacesForTab: Yes +NumSpacesForTab: 2 +Encoding: UTF-8 + +RnwWeave: Sweave +LaTeX: pdfLaTeX diff --git a/Open Letter to EU Commission - Open Science MOOC.pdf b/Open Letter to EU Commission - Open Science MOOC.pdf new file mode 100644 index 0000000..52aa68e Binary files /dev/null and b/Open Letter to EU Commission - Open Science MOOC.pdf differ diff --git a/OpenScienceMOOC-flyer2.pdf b/OpenScienceMOOC-flyer2.pdf new file mode 100644 index 0000000..e5c5cb5 Binary files /dev/null and b/OpenScienceMOOC-flyer2.pdf differ diff --git a/Production_Files/MODULE_DESIGN_PROTOCOL.md b/Production_Files/MODULE_DESIGN_PROTOCOL.md index daef104..c55df42 100644 --- a/Production_Files/MODULE_DESIGN_PROTOCOL.md +++ b/Production_Files/MODULE_DESIGN_PROTOCOL.md @@ -15,26 +15,14 @@ This is a collaborative, team-based effort. Each module has its own repo on GitH 9. [Open Educational Resources](https://github.com/OpenScienceMOOC/Module-9-Open-Educational-Resources) 10. [Open Advocacy](https://github.com/OpenScienceMOOC/Module-10-Open-Advocacy) -Each module also has its own specific set of key resources to aid with content development: - -1. [Open Principles](https://opensciencemooc.eu/open-science-resources/#one) -2. [Open Collaboration](https://opensciencemooc.eu/open-science-resources/#two) -3. [Reproducible Research and Data Analysis](https://opensciencemooc.eu/open-science-resources/#three) -4. [Open Research Data](https://opensciencemooc.eu/open-science-resources/#four) -5. [Open Research Software and Open Source](https://opensciencemooc.eu/open-science-resources/#five) -6. [Open Access to Research Papers](https://opensciencemooc.eu/open-science-resources/#six) -7. [Open Evaluation](https://opensciencemooc.eu/open-science-resources/#seven) -8. [Public Engagement with Science](https://opensciencemooc.eu/open-science-resources/#eight) -9. [Open Educational Resources](https://opensciencemooc.eu/open-science-resources/#nine) -10. [Open Advocacy](https://opensciencemooc.eu/open-science-resources/#ten) ## Forming a team for collaborative design -1. Hopefully, you've identified which module program to work on. It is probably best that you are already familiar with this topic. Building your core team is the next step. You will need a combination of: +1. Hopefully, you've identified which module program to work on. It is probably best that you are already familiar with this topic. Building your core team is the next step. You will need a combination of: 1. Experts. These will be your bread and butter. This can comprise people from any background, including academics, librarians, journalists, policymakers, librarians and information specialists, advocates, and students. 2. It might also be useful to supplement this with learning designers, including those familiar with learning technologies. 3. Being a trainer, educator, or educational expert is not a prerequisite. However, it would not hurt to have someone familiar with this on your team. -2. Having an initial briefing meeting will be the second step. This will be in conjunction with either a member of the MOOC Steering Committee, or if you're really unlucky, Jon. In this meeting, the following will be covered: +2. Having an initial briefing meeting will be the second step. This will be in conjunction with either a member of the MOOC Steering Committee, or if you're really unlucky, Jon. In this meeting, the following will be covered: 1. An introduction to the MOOC, the managerial team, the core team, and an understanding of the requirements of each. 2. Expectations of what each will contribute and receive from taking part. 3. An explanation of time scales and commitments. @@ -50,74 +38,74 @@ Each module also has its own specific set of key resources to aid with content d ## The development process +For a checkbox version of this, see the [MOOC planning template](MOOC_planning_template.md). This is design to keep track of the development in a structured manner. + 1. Initiate and plan - 1. Define the target group - 2. Refine the learning objectives and outcomes - 3. Design the course outline - 1. Combination of lectures, reading, and activities - 4. Design the project plan and timeline - 5. Identify promotion channels + 1. Define the target group + 2. Refine the learning objectives and outcomes + 3. Design the course outline + 1. Combination of lectures, reading, and activities + 4. Design the project plan and timeline + 5. Identify promotion channels 2. Designing and scripting - 1. Identify key resources (including those already gathered) - 2. Design learner activities with clear instructions - 3. Find resources (video, illustrations, screencasts, podcasts, assignments, quizzes, presentations) - 4. Write concise, engaging video (and audio) scripts - 5. Review all learning resources, adjusting as needed - 6. Finalise all scripts - 7. Copyright strategy + 1. Identify key resources (including those already gathered) + 2. Design learner activities with clear instructions + 3. Find resources (video, illustrations, screencasts, podcasts, assignments, quizzes, presentations) + 4. Write concise, engaging video (and audio) scripts + 5. Review all learning resources, adjusting as needed + 6. Finalise all scripts + 7. Copyright strategy 3. Recording and editing - 1. Record either on location, online, or within studio - 2. Preference to non-'floating head' styles - 3. Edit all audio/visual material + 1. Record either on location, online, or within studio + 2. Preference to non-'floating head' styles + 3. Edit all audio/visual material 4. Quizzes and assignments - 1. First draft for feedback to instructional design - 2. Follow up feedback - 3. Define regularity of quizzes - 4. Final examination - 5. Review assignments - 6. Other assignments (e.g., discussion prompts) + 1. First draft for feedback to instructional design + 2. Follow up feedback + 3. Define regularity of quizzes + 4. Final examination + 5. Review assignments + 6. Other assignments (e.g., discussion prompts) 5. Internal reviewing - 1. All members of team cross-review content - 2. Checks from Steering Committee -6. External testing and review - 1. All content via GitHub - 2. Use existing channels from communication strategy -7. Internal reviewing and finalisation - 1. All members of team cross-review content - 2. Checks from Steering Committee -8. Implementation - 1. Agreement on platform - 2. Module logo (either designed or copyright free) - 3. Module description and introduction - 4. Instructor and guest lecturer agreement - 5. Instructor and team member profiles - 6. Course readings all acquired (copyright free) - 7. Port all content to selected platform - 8. Make sure all content is also deposited in the Open Science Framework - 1. For future indexing via Zenodo + 1. All members of team cross-review content + 2. Checks from Steering Committee +6. External testing and review + 1. All content via GitHub + 2. Use existing channels from communication strategy +7. Internal reviewing and finalisation + 1. All members of team cross-review content + 2. Checks from Steering Committee +8. Implementation + 1. Agreement on platform + 2. Module logo (either designed or copyright free) + 3. Module description and introduction + 4. Instructor and guest lecturer agreement + 5. Instructor and team member profiles + 6. Course readings all acquired (copyright free) + 7. Port all content to selected platform + 8. Make sure all content is also deposited in the Open Science Framework + 1. For future indexing via Zenodo 9. Promotion - 1. Content and communication calendar/strategy/timeline + 1. Content and communication calendar/strategy/timeline 2. Identify relevant channels, including: - 1. Mailing lists - 2. Social media (including relevant hashtags) - 3. Organisations - 4. Individuals - 5. Websites - 6. Conferences - 3. Images for use in social media - 4. Course title marketing check + 1. Mailing lists + 2. Social media (including relevant hashtags) + 3. Organisations + 4. Individuals + 5. Websites + 6. Conferences + 3. Images for use in social media + 4. Course title marketing check 10. Launch - 1. Publicity start - 2. Open and free for all; continuous, self-paced learning; 100% online - 3. Soft launch date - 4. Course launch date - 5. Monitor learner experiences and reactions - 6. Prepare to provide additional information if required -11. Reviewing and optimisation - 1. Collate and review learner feedback at regular intervals - 2. Track any potential new information during course duration - 3. Prepare evaluation report - 4. Evaluation meeting - 5. Optimise content where relevant - -Note. This planning template was adapted based on material provided by the University of Copenhagen and the Centre for Innovation, Leiden University. + 1. Publicity start + 2. Open and free for all; continuous, self-paced learning; 100% online + 3. Soft launch date + 4. Course launch date + 5. Monitor learner experiences and reactions + 6. Prepare to provide additional information if required +11. Reviewing and optimisation + 1. Collate and review learner feedback at regular intervals + 2. Track any potential new information during course duration + 3. Prepare evaluation report + 4. Evaluation meeting + 5. Optimise content where relevant \ No newline at end of file diff --git a/Production_Files/MOOC_planning_template.md b/Production_Files/MOOC_planning_template.md new file mode 100644 index 0000000..17ad449 --- /dev/null +++ b/Production_Files/MOOC_planning_template.md @@ -0,0 +1,93 @@ +--- +title: "Mooc planning template" +author: "Jon Tennant" +date: "11 May 2018" +output: html_document +--- + +# MOOC planning template + +## How to use this template + +This is to provide a structured check list to track content development. + +- For the 'Delivered' column, a simple Yes/No Scheme should be used. +- For the 'Status' column, please use one of the three symbols below. +- For the 'Deadline' column, please use a traditional dating scheme: 2018/05/10. +- For the 'Comments' column, insert any text as neccessary. + + +Status traffic light scheme: + +Green: All looks good + +![Green](https://img.shields.io/badge/Status-text-green.svg) + + +Orange: Issues that can impact launch date + +![Orange](https://img.shields.io/badge/Status-text-orange.svg) + + +Red: Launch date in danger + +![Red](https://img.shields.io/badge/Status-text-red.svg) + + + +| Design Phase | Delivered | Status badge | Deadline | Comments | +| ------------------ | ------------ | ----------------- | ------------ | ------------- | +| **Initiate and plan** | | | | | +| Kick off | Yep | ![Green](https://img.shields.io/badge/Status-text-green.svg) | 2018/05/10 | Sprint success! | +| Define target group | Nope | ![Green](https://img.shields.io/badge/Status-text-green.svg) | 2018/05/31 | | +| Refine learning objectives/outcomes | Nope | ![Green](https://img.shields.io/badge/Status-text-green.svg) | 2018/05/31 | | +| Design course outline | Nope | ![Green](https://img.shields.io/badge/Status-text-green.svg) | 2018/05/31 | | +| Design project plan and timeline | Nope | ![Green](https://img.shields.io/badge/Status-text-green.svg) | 2018/05/31 | | +| Identify promotion channels | Nope | ![Green](https://img.shields.io/badge/Status-text-green.svg) | 2018/05/31 | | +| **Design and scripting** | | | | | +| Identify key resources | | | | | +| Design learner activities | | | | | +| Find existing key resources | | | | | +| Write audio/video scripts | | | | | +| Review all learning resources | | | | | +| Finalise all scripts | | | | | +| Copyright strategy | | | | | +| **Recording and editing** | | | | | +| Record on location/in studio | | | | | +| Edit all audio/visual material | | | | | +| **Internal reviewing** | | | | | +| Cross-check and review content | | | | | +| Checks from Steering Committee | | | | | +| **External testing and review** | | | | | +| All reviewing conducted via GitHub | | | | | +| Existing channels from communications strategy | | | | | +| **Internal reviewing and finalisation** | | | | | +| Cross-review and check content | | | | | +| Final checks from Steering Committee | | | | | +| **Implementation** | | | | | +| Agreement on platform | | | | | +| Module logo designed | | | | | +| Module description and introduction | | | | | +| Team member and guest lecturer agreements | | | | | +| Team member and guest lecturer profiles | | | | | +| Course readings acquired | | | | | +| Port content to selected platform | | | | | +| All content deposited in the Open Science Framework | | | | | +| **Promotion** | | | | | +| Content and communication calendar/strategy/timeline | | | | | +| Identify relevant channels (mailing lists, social media and hashtags, organisations, individuals, websites, conferences) | | | | | +| Images for use in social media | | | | | +| Course title marketing check | | | | | +| **Launch** | | | | | +| Publicity start | | | | | +| Open and free for all, continuous, self-paced learning, 100% online | | | | | +| Soft launch | | | | | +| Course launch | | | | | +| Monitoring of learner experiences and reactions | | | | | +| Prepare to provide additional information if required | | | | | +| **Reviewing and optimisation** | | | | | +| Collate and review learner feedback at regular intervals | | | | | +| Track any new information during course duration | | | | | +| Prepare evaluation report | | | | | +| Evaluation meeting | | | | | +| Optimise content where relevant | | | | | diff --git a/Production_Files/MOOC planning template.docx b/Production_Files/OLD/MOOC planning template.docx similarity index 100% rename from Production_Files/MOOC planning template.docx rename to Production_Files/OLD/MOOC planning template.docx diff --git a/Production_Files/README.md b/Production_Files/README.md new file mode 100644 index 0000000..f30b1df --- /dev/null +++ b/Production_Files/README.md @@ -0,0 +1,18 @@ +# Production files + +This repo is aimed to provide the production files for content development and management for each MOOC module. Each MOOC repo has its own copy of these files, which can be customised for each module as needed. + +## Table of contents +- [MOOC planning template](MOOC_planning_template.md) +- [Module design protocol](MODULE_DESIGN_PROTOCOL.md) +- [Writing a script](Writing_a_script.md) +- [Script template](Script_template.md) +- [Video management protocol](Video_management_protocol.md) + +## Licenses + +### Content +MOOC content license: [![CC0 Public Domain Dedication](https://img.shields.io/badge/License-CC0%201.0-lightgrey.svg)](https://creativecommons.org/publicdomain/zero/1.0/) + +### Code and software +Software associated with the MOOCs, or any code snippets contained in the MOOCs, carry the following default license: [![License:MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT). If needed a different [OSI approved software license](https://opensource.org/licenses) may be chosen. \ No newline at end of file diff --git a/Production_Files/WritingStyleGuide.md b/Production_Files/WritingStyleGuide.md new file mode 100644 index 0000000..97c8bd8 --- /dev/null +++ b/Production_Files/WritingStyleGuide.md @@ -0,0 +1,14 @@ +In construction. + +# tips + +## checklists, +let's use a special character and a bullet? +- ❏ + +## Basic data management + +- **never use spaces** in folder or filenames (it breakes the continous integration checks) +- Same for othe special characters and language specific characters +- filenames can go up to 32 characters: make it informative +- filenames should be unique in one repo, if possible in all our repos. diff --git a/README.md b/README.md index 6d1d5a7..7edcbd7 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,42 @@ -# Main -Management materials and content +# Main management materials and content -![Open Science MOOC modules](moocpng.png) +Welcome to the GitHub home of the Open Science MOOC! -### Table of contents +This repo is aimed to: +1. Provide general information about the MOOC, which can also be found on our main [website](https://opensciencemooc.github.io/site/). +1. Provide the core tools for managing the content development process for the MOOC. + +For content development for individual modules, please see the relevant repositories contained within the project. + +## Table of contents +- [Rationale](#Rationale) - [Contributing](#Contributing) -- [Module design protocol](#Module design protocol) +- [Module design protocol](#Module_design_protocol) - [Roadmap](#Roadmap) - [Code of conduct](#COC) -- [Licenses](#Licenses) +- [Licenses](#Licenses) +- [Disclaimer](#Disclaimer) +- [Glossary](https://github.com/OpenScienceMOOC/Main/blob/master/Glossary.md) + +## Rationale + +Research is getting a global makeover, in part thanks to the power of the internet and the tools it provides for us, and in part due to a growing call for accountability (e.g., reproducibility and data provenance) in research. Global policies are emerging at different levels that include some aspect of 'Open Research', 'Open Scholarship', or 'Open Science', and inclusive of all research disciplines. But our universities are often letting us down, and they are not teaching us the knowledge, tools and skills we need to do research effectively in the 21st century. + +Open Science is about increased rigour, accountability, reproducibility for research. It is based on the principles of inclusion, fairness, equity, and sharing. Open Science can be viewed as research simply done properly, and it extends across the Life and Physical Sciences, Engineering, and Mathematics, to Social Science and Humanities. + +This MOOC is designed to help equip students and researchers with the skills they need to excel in a modern research environment. It brings together the efforts and resources of hundreds of researchers and practitioners who have all dedicated their time and experience to create a community platform to help propel research forward. + +The content of this MOOC is distilled into 10 core modules. Each module will comprise a complete range of resources including videos, research articles, dummy datasets and code, as well as 'homework' tasks to complete as individuals. Because you don't learn how to do Open Science by reading; you learn by doing it. + +The MOOC will be hosted through an open source provider. We expect that in the future different systems of certification will be developed, including completion badges. We also intend to build a forum for the open discussion of the MOOC and any relevant topics. + +![Open Science MOOC modules](https://github.com/OpenScienceMOOC/Main/blob/master/logos/moocpng.png) + ## Contributing See [CONTRIBUTING](CONTRIBUTING.md). -## Module design protocol +## Module design protocol See [Module design protocol](MODULE_DESIGN_PROTOCOL.md). ## Roadmap @@ -25,7 +48,13 @@ See [the code of conduct document](CODE_OF_CONDUCT.md). ## Licenses ### Content -MOOC content license: [![Content license](https://img.shields.io/badge/License-CC0%201.0-lightgrey.svg)](https://creativecommons.org/publicdomain/zero/1.0/) +MOOC content license: [![CC0 Public Domain Dedication](https://img.shields.io/badge/License-CC0%201.0-lightgrey.svg)](https://creativecommons.org/publicdomain/zero/1.0/) ### Code and software -Software associated with the MOOCs, or any code snippets contained in the MOOCs, carry the following default license: [![License:MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT). If needed a different [OSI approved software license](https://opensource.org/licenses) may be chosen. +Software associated with the MOOCs, or any code snippets contained in the MOOCs, carry the following default license: [![License:MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT). If needed a different [OSI approved software license](https://opensource.org/licenses) may be chosen. + +## Disclaimer + +We, the contributors, are fully aware that there is no magic "one size fits all solution" when it comes to implementing Open Science philosophies and best practices, especially when covering all research disciplines. However, we also believe that we should not limit ourselves from the onset, just because pragmatic solutions may not exist today for certain disciplines. We aim to set a highly inclusive standard, fully accepting the risk that for some disciplines this strategy may not be fully appropriate. Through that failure, we hope that you, the course users, will join us as course contributors and help us co-create bespoke solutions to your discipline based on principles of transparency, provenance, reproducibility and reuse of knowledge. + + diff --git a/communication_strategy.md b/communication_strategy.md new file mode 100644 index 0000000..c8433d5 --- /dev/null +++ b/communication_strategy.md @@ -0,0 +1,132 @@ +# [Open Science MOOC platform](https://opensciencemooc.eu/) Communications Strategy + +Two-phase release strategy: + +(1) Open review of final draft by Graduate Schools and professional societies aiming for a good discipline coverage, and use the review process as part of the promotion; + +(2) Full public release and promotion as below. + +## Primary mailing lists + +* [Schol Comm](mailto:scholcomm@lists.ala.org) +* [Open Science](mailto:open-science@lists.okfn.org) +* [GOAL](mailto:goal@eprints.org) +* [FORCE11](mailto:f11discussion@force11.org) +* [RadicalOA](mailto:RADICALOPENACCESS@JISCMAIL.AC.UK) +* [LIS bibliometrics](mailto:LIS-BIBLIOMETRICS@JISCMAIL.AC.UK) +* [Accès ouvert](mailto:accesouvert@groupes.renater.fr) (note: from Serge Bauin) +* [Italian OA list](mailto:oa-italia@openarchives.it) (note: from Elena Giglia) +* [EU Open Data](mailto:euopendata@lists.okfn.org) +* [Fossil Bank](mailto:fossil-bank@lists.okfn.org) +* [Open Development](mailto:open-development@lists.okfn.org) +* [Open Education](mailto:open-education@lists.okfn.org) +* [Open Humanities](mailto:open-humanities@lists.okfn.org) +* [EURODOC open science](mailto:open-science@eurodoc.net) + +(More lists available here: https://lists.okfn.org/mailman/listinfo/open-science-de) + +## Social Media +- Dedicated [Twitter handle](https://twitter.com/OpenScienceMOOC). +- Thunderclap. +- Dedicated [Facebook page](https://www.facebook.com/OpenScienceMOOC/). +- Dedicated Slack/Gitter channel for all participants [join here](https://osmooc.herokuapp.com/). + +### Twitter organisations + +* [LIBER](https://twitter.com/LIBEReurope) +* [EIFL](https://twitter.com/EIFLnet) +* [SPARC NA](https://twitter.com/sparc_na) +* [SPARC EU](https://twitter.com/sparc_eu) +* [R2RC](https://twitter.com/R2RC) +* [FORCE11](https://twitter.com/force11rescomm) +* [OpenAire](https://twitter.com/OpenAIRE_eu) +* [OpenUP](https://twitter.com/projectopenup) +* [The Carpentries](https://twitter.com/thecarpentries) +* [Creative Commons](https://twitter.com/creativecommons) +* [FOSTER](https://twitter.com/fosterscience) +* [Mozilla Open Leaders](https://twitter.com/mozopenleaders) +* [OpenCon](https://twitter.com/open_con) +* [OKFN](https://twitter.com/OKFN) + +### Facebook groups and pages +* https://www.facebook.com/www.mariecuriealumniassociation/ +* https://www.facebook.com/panddk/ +* https://www.facebook.com/carpentries/ +* https://www.facebook.com/FutureScienceOA/ +* https://www.facebook.com/CenterForOpenScience/ +* https://www.facebook.com/GrazOpenScienceInitiative/ +* https://www.facebook.com/eff/?rf=104017266300739 or https://www.facebook.com/eff/ +* https://www.facebook.com/openaccessglobal/ +* https://www.facebook.com/eurodoc/ +* https://www.facebook.com/oasouthasia/ +* https://www.facebook.com/IFLA.org/ +* https://www.facebook.com/InternationalScience/ +* https://www.facebook.com/OANepal/ +* https://www.facebook.com/openscholarshipinitiative/ +* https://www.facebook.com/africarxiv/ +* https://www.facebook.com/OKMaps/ +* https://www.facebook.com/pubmet/ +* https://www.facebook.com/IGDORE/ +* https://www.facebook.com/open.osodos/ +* https://www.facebook.com/EurSciEd/ +* https://www.facebook.com/socarxiv/ +* https://www.facebook.com/heterodoxacademy/ +* https://www.facebook.com/OpenAccessBd/ +* https://www.facebook.com/OOOcanada/ +* https://www.facebook.com/DirectoryofOpenAccessJournals/ +* https://www.facebook.com/Altmetric/ +* https://www.facebook.com/OpenKnowledgeEngland/ +* https://www.facebook.com/access2perspectives/ +* https://www.facebook.com/sci.hub.org/ +* https://www.facebook.com/fosteropenscience/ +* https://www.facebook.com/AcademiaObscura/ +* https://www.facebook.com/FORCE11ResearchCommunication/ +* https://www.facebook.com/OpenAccessNG/ +* https://www.facebook.com/OpenLibHums/ +* https://www.facebook.com/groups/370560229815368/?ref=group_browse_new +* https://www.facebook.com/groups/398759490316633/?ref=group_browse_new +* https://www.facebook.com/groups/1541992832751466/?ref=group_browse_new +* https://www.facebook.com/groups/1650961761843383/?ref=group_browse_new +* https://www.facebook.com/groups/ossdeveloper/?ref=group_browse_new +* https://www.facebook.com/groups/1457457734499379/?ref=group_browse_new +* https://www.facebook.com/groups/AllThingsOpen/?ref=group_browse_new +* https://www.facebook.com/groups/open.science.fi/?ref=group_browse_new +* https://www.facebook.com/groups/reviewer2/?ref=group_browse_new +* https://www.facebook.com/groups/oaindia/?ref=group_browse_new + +## Other non-social media + +- University mailing lists. +- Assets: EURAXESS, EURODOC. +- National Open Science contact points (OpenAIRE NOADs = National Open Access Desks ). +- National science communication organizations. +- Etherpad for collaborative note taking. + +# Messaging + +## For students: + +- These skills will get you a job both inside and outside of academia. +- These skills will save you time during your research. +- These skills will make your output of better quality. +- These skills make peer review of your papers more efficient. + +## For lab-heads and institutions: + +- Make collaboration in the workplace better and faster. +- Continuing the work of alumni will be easier (information transfer, patenting). +- Openness often implies more citations and attention for research. +- Encourages integration of science within society, increases societal impact of research. + +## For policymakers: + +- Good for researchers as it teaches them core competencies and transferable skills to be used outside of academia. +- Contributes to innovation and economic growth. +- Contributes to a healthier society. +- Helps them meet policy objectives. + +## For librarians: + +- Can be integrated with graduate school training programs at zero cost to them. +- Helps train researchers/students in tasks that will ultimately make their jobs easier. +- Will create a new knowledge pool that they can draw upon if needed. diff --git a/Diversity statement.md b/diversity_statement.md similarity index 100% rename from Diversity statement.md rename to diversity_statement.md diff --git a/moocgif.gif b/logos/moocgif.gif similarity index 100% rename from moocgif.gif rename to logos/moocgif.gif diff --git a/Logo.png b/logos/mooclogo.png similarity index 100% rename from Logo.png rename to logos/mooclogo.png diff --git a/moocpng.png b/logos/moocpng.png similarity index 100% rename from moocpng.png rename to logos/moocpng.png diff --git a/logos/outline_ver.2.mp4 b/logos/outline_ver.2.mp4 new file mode 100644 index 0000000..3801f60 Binary files /dev/null and b/logos/outline_ver.2.mp4 differ diff --git a/module-logos/_os-mooc-module-logos.ai b/module-logos/_os-mooc-module-logos.ai new file mode 100644 index 0000000..407cbd8 --- /dev/null +++ b/module-logos/_os-mooc-module-logos.ai @@ -0,0 +1,1685 @@ +%PDF-1.5 %���� +1 0 obj <>/OCGs[17 0 R 14 0 R 18 0 R 19 0 R 20 0 R 21 0 R 22 0 R 23 0 R 24 0 R 25 0 R 26 0 R 27 0 R 28 0 R 29 0 R 30 0 R 31 0 R 32 0 R 33 0 R 34 0 R 35 0 R 36 0 R 37 0 R 38 0 R 39 0 R 15 0 R 40 0 R 16 0 R 41 0 R 42 0 R]>>/Pages 3 0 R/Type/Catalog>> endobj 2 0 obj <>stream + + + + + Adobe Illustrator CC 22.1 (Windows) + 2018-08-15T13:00:44-04:00 + 2018-08-15T13:00:44-04:00 + 2018-08-15T13:00:44-04:00 + + + + 180 + 256 + JPEG + /9j/4AAQSkZJRgABAgEASABIAAD/7QAsUGhvdG9zaG9wIDMuMAA4QklNA+0AAAAAABAASAAAAAEA AQBIAAAAAQAB/+4ADkFkb2JlAGTAAAAAAf/bAIQABgQEBAUEBgUFBgkGBQYJCwgGBggLDAoKCwoK DBAMDAwMDAwQDA4PEA8ODBMTFBQTExwbGxscHx8fHx8fHx8fHwEHBwcNDA0YEBAYGhURFRofHx8f Hx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8f/8AAEQgBAAC0AwER AAIRAQMRAf/EAaIAAAAHAQEBAQEAAAAAAAAAAAQFAwIGAQAHCAkKCwEAAgIDAQEBAQEAAAAAAAAA AQACAwQFBgcICQoLEAACAQMDAgQCBgcDBAIGAnMBAgMRBAAFIRIxQVEGE2EicYEUMpGhBxWxQiPB UtHhMxZi8CRygvElQzRTkqKyY3PCNUQnk6OzNhdUZHTD0uIIJoMJChgZhJRFRqS0VtNVKBry4/PE 1OT0ZXWFlaW1xdXl9WZ2hpamtsbW5vY3R1dnd4eXp7fH1+f3OEhYaHiImKi4yNjo+Ck5SVlpeYmZ qbnJ2en5KjpKWmp6ipqqusra6voRAAICAQIDBQUEBQYECAMDbQEAAhEDBCESMUEFURNhIgZxgZEy obHwFMHR4SNCFVJicvEzJDRDghaSUyWiY7LCB3PSNeJEgxdUkwgJChgZJjZFGidkdFU38qOzwygp 0+PzhJSktMTU5PRldYWVpbXF1eX1RlZmdoaWprbG1ub2R1dnd4eXp7fH1+f3OEhYaHiImKi4yNjo +DlJWWl5iZmpucnZ6fkqOkpaanqKmqq6ytrq+v/aAAwDAQACEQMRAD8AU8oeULTzClwG1SOyngI/ cyCOrI2wYGSWKvxbGlabeOVAW97rNYcNeniB9/6imWj6Fa6L+ZGhWUF8l+RdQNLJGE4qxf7NUeVT t741u0Zs5y6WcjHh2P45B7RqH5g2dpdXtqllPcT2ZVRFEUeRmNxDb7xIXlSrXCslU+JQSNqVteJS 6f8ANzSre1urmfTrxFtBIZ4eKmaIxxyORMlfgq0fDYtQnegqQqzzFXYq7FXYq7FXYq7FXYq7FXYq 7FUHqurWmm23rXEioWqsIc8VaQKWVC9CF5UoK99uuJBokC6ZQAMgO9IrHzcTdSJMPVgLnjQBZY1L nqNldVDDpvt+0cox5xJpzDJhl6hce/8AH9vvZJa3dtdwia3kEkZ7jqD4MDupHcHcZfbKMxIWFXFk knnO61i18u3E2kCT62slusskEYnnitXnjW7mghIf1JobYySRpwfk6gcH+wVXnj3v5r6jcxWeh3+p rZ+lqb6Zq1/Y2to1xJEmn/VDqkc9mjRRC4nu04wwRSvGgZQ1ObKoaDzL+bv6fuorVLy6eLX7tEsL uzeO0k0qBNReKNbhtOso4fW9G2SKb65OOTKzUBKMqz79Ma9/yrL9Mehc/p39E/Wvq31f/S/rP1fn w+rcf73nt6fHrtTFXiX5Ua5pWjeYbi51O4FtA9uqK7BiCwuoJKfCD+yjHKol7jtXBPJjAgLN/wC9 kqWF9a335uWV3aSCW3l1CAxyAEAj4R3oe2PVjOBhoyDzES90v9K8zSyXD2mqLCJHJgiKVCAIwX4t /wBum1KeNcseUx5cQq426fT/ADUbh3iv4xbtK7CE7ERV+FQ/BqEgkHY02IxWOTDW8d6/HVubTfMr sWi1BYQqHhGf3nJvTUUZuCD7YJ5cT8sVGTEOcbdc2XmiS5mlhuIYo2ZDFF6jkKAoDKf3YqC1TtT7 tsVjPEAAQT+PenuFxHYq7FXYq7FXYq7FXYqhr/UbeyhMkpqQA3prTmV5KhYAkVClxXG2M5cMeKtg 8q13zmmpWZi1ACPkxpNFyEW60AZGLMjUYVJqNiTx6Zj5MGSGSU8UuY5fDu/VUujn4RCcQY7j7fx5 Matb6+04KEIuLBacADQxrU7ow3Gx2HToBTKiceaX8zL95/HulzJDcQY7c4/j8dzPfJ+qvd3VtcRs y+qyhj9hiORUpIB8LcSzU7dx45LEZCZjLmHVanBCEoyhtxdE784+UptZ1zy5qcFtbSy6Rc83nnkC SRQvLDJIYUkt7yJ2P1cfsxuNikse9ctk848x/l3cW2g6T5TEGkafqVyt3Y2+rQiYPqDfoPUbFbi+ dbc+k73GoK/F5JN2ekjuwDKsd4x+YvLaanp9/DpgstL1ofX7VJpLVU1C4gs0mvvq+maVF9Xf9GXU UvKFmiXjN+zyRVPPMVrb39nqNjY2tne3mv3yajoHmCaGdbzUki1e2vHsw7WN5G8McUSpDKxdDCiP 6ZgRnCr0amof8qy9H9Fx/pD6t9W/RvFuHq8/RrT9H8evx1+o+nXfj6fxYqgv+VH+Q/8AfVx/yOP9 Mjwh2/8ALmo7x8kVpf5QeTNM1G21C1jnFzayLLCWlJHJTUVFMeEMMva+ecTE1R8ma5J1bsVdirsV dirsVdirsVdirsVdiqHvb61s4lkuJFjDsI4+Z4hpGB4pU92pQe+QyS4YmXcLZ48ZmaDzLz1ql7Nc hpavBCzGJBRWj5cVJSRQprSvXf3GUY9TjzwA+jJ+OXfy6bgbmLPFkyYTOwJ4zX3/AI2OxPV5zc33 qIQ/x06sBRxt3UdR16fjlhyShL1/NshpYT9WA8Mh/D8+/wCy7HM27SLieLUbeBXrBMzVQ7j7Jao8 DVfxyjXwjPBOVbgOVpsxnYkKlH8fj73ov5edLL/X/wCZpyz/AC8vh9zr9Z9MPe9WzJa0h83eU9M8 yWJtNQtIr6FkaJ7a4Z1jZJGRyax/EHVolZGG4I2p1wiurOHDfquvJKJfy3sDHPGkZ9HUhP8Apq0+ tSrDdC4up714ZAFP7r17yX7PEsh4PyT4chLivZkBj3sn8fFyfl7GotSqmM6av+4VYrydE08uVZxa rw2Wq8VWTmojrCAImZCd3HJl5Mj/AEZc/oT6l68v1qnL6x68nqepz58vVpX7Xbjxp8PHh8OKd682 PxeYNdlrwkUgdSVRQPmTQZ5nh9oO0Ml8Mht5RA+Z2d7LSYRzH3q9lrmsNqUFtcOAHdVdeK9D7gZl aLt3XHVQxZDsZAHYci15NLi4DKPcm915p8uWs00NxqMEc1vQTRlxyUkgdB4chXwqK9c9DdQsPm7y wI2lbU7dYl9T960gCN6KNJJwY7PwSNmPGuwOKpvirsVdirsVdirsVdirsVdirHPPYB0iIHp64/4g +Hofc7Ds3+8Pu/U8sv7m4gIjRucA2EL9FA7Ieq9tum1BTNdm00ZjzdpPBGZJ5H8c2M6slq9q1zCp jkU0delGK8u23fsaZZo55OPw5+qNdf19fjvfV0ut0/heobSHKvx9n2LNNB/S1j/rH/k2+DU/4rP+ r+pyMH95P3vSfy86WX+v/wAzTln+Xl8PucHWfTD3vVsyWt2KuxV2KuxVhdtF9csbSwjjozyO/wBY rVQRWoIC+AHfPMNNi/M6fFp4x3lKR47sA9QRXdXXud5OXBOUyenJWtpVMmmQmIxvbzcAxP2/i+M0 4g05Zk6bIDLTQMeGWPJw8/q39Rqhtf6ujCcdpm7sIHV38ofpC5N5pguXSaRTFcXI9HntNIxt5ZPT jEjRg/Y+Nt++drl7RjEkVyNcwP7PLvdbHASFIaT+Xt5azxXGlottfC+nubczSBg8UvAhYww4PKsh YBacegyUNfE7+Uvf6TXLvKDhP3famjfmHaRyxQz2U0c0kRkdSePB15go3qCNgB6R+IgbbkZR/K0Q QDE3X6++u5n+WPeqL59tmkaNLOVnCOY+BDq8iMUCKYw/LmwopWp3G2S/lON1wnl9vw7/AMBH5c96 9/OnpwrO9n+5do0BEtGVpIy9HDogTgw4vU/Due2E9o0LMdtuveOu21de5Rg6Wrad5uS+1CC0S0dF nqFnLBkqiyF6FQQd4xxNfiBqOmTw68TmI8PPr8/1bd43Yyw0LtkGbBpdirsVdirsVY756/45MX/G cf8AEHw9D7nYdm/3h936nk+rf3n0n+GYx5O6HVILqMmwugR/u3/mUuRwf34936XU9sH0j+r+ko3S tMkk1WxKgt8TdPtD9zIen0YNRIflZ3t6WrTyrJP+t+l6j5D8uTR2dtcueCISyMKENxlaopWo6ZkT xEZZH3OLqJRyRjXMFP8AWPNEWn3CwrEZAjFblmS5HEhFkAjMVvMjngxanIdPY0m1If8AxkrKGhhj k4ANcrzn5qBXmERbdmdl4saELtxOwYYqrJ5us4LKS51RfqvotGsghS6nAMpIWhNvExFVpXjsdjQ4 qrjzboRnhgE0vqzyLCi/V7jaRy3FXPp0Q/AT8VNqHoRVVN8VYnF5V1iJuUVxHG3iruD+C553i9ld bjNwyQifKUh/vXcS1+I8wfs/WrWXlrUob6G4lljcRuGb4mLGhqeq5k6L2Z1WPUQyzlAiMgTvInnf WLDLrYGBiAdwyQxxmtVB5U5VA3p0+7O6oOqtdhVoxxluRUFqceVN6eGCgtuCIKUUCmwoOnbGgrZA IIIqDsQcKuAAAAFANgBirsVdirsVdirsVY756/45MX/Gdf8AiD4QLB9zsOzf7w+79TzK9sjIS3Xu CO30Zj+RdzuDaVtYE6fekiqiY79v7lD9GQwxPjj3fpdT2vUoj+r+mTO/Kvkyd7y0vj8FrHVnV6hi skMiD0zQht3HXLZ445cBxz6inAhOWPLksb8X6WW6t5itbP1rW24vdCNpF4SWvwsGPMtHLPbseFCz 9Ngd65a1sbtbXT4kieERoJJo40thKnxwcSTxA1Axkjm/FuoBpxNMVQ8GlaRLeJBqRSf0ZpBHE84H FyT6saqdSmIkZmBPw/MdKKpwR5baxjjbSZFhgZ4RD9ZtQVdQjHkRdULhRVSTyUKendVq00rydcO0 ctq9oCpaNpb4UljkmUB19K5kJV5VWlR1267Yqyf6zp31H1PXi+pU9P1uY9OleFOdadfh69cVee/8 r38u/wDVvvP+SX/NeZ/8ny7w6r+VodxRmj/nLoWqapa6dDZXSS3cqwo7+nxBc0BNGJyM9FKIJsbM 8facJyEQDuzR9W01Jmha4QTK3Ex1q1aV6DfMTgLnnJG6tb+m9I58DdxBuXChYD4qhaGviSAPfD4c u5Hix7211jS29Ol1GfVbhH8Q3aqin/DrT5jxwcB7k+JHvXNqumqQGuYxUch8Q6cQ1f8AgWGPAe5f Ej3tx6lYSSSRJOhkiJEiV3HEkHb5qfux4SonHvVbe4huIllhbnG32WFd/vwEUyBBFhUwJdirsVdi rsVUL2xtb6AwXMYkjJrToQR3BG4OEGmePJKBuJYVrHky8tqy2VbqD+QD96v0D7X0fdkzwz5u5wdo RltLY/YhvK3l2PVLPUQ59MfWSnqUDD+4i2ZCRXrmNCM8ee+ceFxO2cYnwi6NfpLL9au3srNLSygb nMGRfSWcCKML8TK8EFyFda/ArKK9stdeSSbLG7uwJtUeVLmOR2M/qwJM8plfkjK3HTyypKoV5DxH xbUp1UIaLyxchYiiyKVK24QURVReAVxTS1+Iu/xE9AteWzcVWQaPoN5ZafeQz2VrcS3kjyzrNMJF kLcSA7raQ8gSXJ5Iae9fhVVYtLuYlV4tC0uOeJg0XGYgA/ECwYWtQaBeg7nw3VRn6GsJLiJ59Msy IYgscvFXdCpqqIDGKIORI+L6MVRf1Gx+rfVfq8X1atfQ4L6deXL7NKfa3+eKvFPyti8oW1nPq2uQ LJLFMIo53DTRwqVqGkiVSI6nZXfqelKHNvqzMnhi89oBiAMpj9Kpf2Og2/5k+XptFt2tba7mhmaF iUIJkoCIHAeIMNxXYjpTBGUjilxb0mUYDPAwFAkfiuj0HULry8upypPYP6/rKDccmRWejUbkD8xm FESrYuznKHFuN7Wvf+VWfhLp6r6ZmZyQC3KLjI32a82ai9/bHhn3rx4+7vbOp6DyhraXDEgS8nlf iJC0asszO/EhGjQNyJAp4Y8Eu9fEh3H8d6NsV0SW8/R8UEyO8DM7erKY2jWOEcS4crJRZ1A3NN8j LiAv8dWyHATwgHl+r9aYJ5e0hJ5J0hZJZnaSVlkkHJmLM1aN0JdtumV+JJsGGN3SLs7K2srZLa2T 04I/sJUtSpr1Yk5GUiTZZxiIigrYGTsVdirsVdirsVdiq1IYo2do0VGlbnIVABZqBeTU6migYpMi eaXTeWfL88tzLNp8Ekl4hjumZAfUUtzPKvX4gDX2HgMUKE/kvytOYzLp0RMVPTI5ArSnShH8or8s VTlVCqFXZVFAPYYq3irsVdirsVfOGit5u0yIW50q+e0DmVVjjmgljkZeJeOVFqpK7EEFSO2b2fBL exby2PxY7cMq+I+1MNBi8z3vnLQ57zTbiC1tLiNYl9GUJFH6nM1dwWO5JLMxOV5DAY5Uebbhjkll iSDQPc9b1CLz273wspbSOM3EZ08k/EIPTIkElY2H95xPc0qPDNM9IpRW/wCYgnSaW4sivqqZLYE+ n6PIsyqfSD86EICWptWmKrLi2/Mp7qd4buwjty7/AFeEhiQjh1Qs3p/ajPFv8r27qr5LX8wyYJEu rRDzX6zBXkvAzczwYwggiI8O/SvXFUy8tQ+ZYrWZNfmhuLn1T6EkGw9LitKjim/KuKpvirsVdirs VdirsVQ0+p2EF1HaSzBbiUckjoSeNaVNB8Ir3OC2YgSL6JHrutaTfaYRb6hEER1mkV2kjSaGL43U yIrMEZf2lBB9xXIykCG/FjlGW4SOzsI/0pHc3GrRW0MMSXCWoM7TRRrdueETSekfTNRGzFDUfDRR TIAb82+U/TQjfy7vv6oOysJrKG2jbWYoLuxuS0k8cjziSQRyRKnBYITIvqMA6lmPX4g1MAFdWcpi RPp2I/HUs88t289t5d0u2uJBNPBZwRzTAsQ7pEqs1XCtuRX4gDlseQcDMQZyI7ymOSanYq7FXYq7 FUm/xp5O/wCr7p3/AElwf815DxY94cr8jn/1Of8ApSqW/mzytczpBb6zYzTysFjijuYWdmPQKoYk nEZInqES0eaIswkB/VKa5NxnYq7FXYq7FXYq7FXYq7FXYq7FUr1Xy7Z6ndwXE8ki+iOJSP015rWp VpOBlCt+0quAe+RMbbceYxFD8foY35ks9EtGstI1e8vZLa4tLtYJv3FIY41iiKHjEKl/VVU5Vq1B vXISA5FysMpm5RAux3+fmlgg8t3VjFez290lpCllDJcIbeF7eS4K3UaJFBHGGB+uBZCBQ16GhOR2 bLmDQIvfv3rbr7tkROLK6htdJlju1W9kiutJsGazkipJ6s45O8UlKLG9Vk5dgtcPkxFgmW22xO/k O/7qZ5YIY7G3jKlSkaKVbhUcVA39MKn/AAIA8MtDgTO5V8LF2KuxV2KuxV846B+V82raRZ6j+kfQ +u8hFF9VuZd1Lbc0UqdkJ2zWQwWLt7vU9rjFMw4b4f6UQmOm+Q5PL3mfy7efXfrUc2pRwEfV54KM jGu8qqDupGSji4ZA31aMvaIz4ckeGqgTzB+56lqHmfzNFdXtvZaG9w0JVbbaUBq3EMVS7pHCwaOV pBwlPEIeVD02LxaXT+efOcFrdTHypcST24k5WsfqMeSRyFfTk4UnBkVV+EDY1HLYFVnmKuxV2Kux V2KuxV2KuxV2KuxVTe2tpJkmeJHmjBWORlBZVZlZgCdwC0ak+4HhjSRI1SgukaSs8NwtlAJ7dRHB KIkDxoooqo1KqANgBgoMvElVWVg0LQ1hlgXTrYQTENNEIY+DspqCy0oSPfHhCfFnd2UZDDFDEkMK LHFGAscaAKqqBQAAbADCwJvcrsUOxV2KuxV2Kvnny9+ZVhpmjabYS2s7vYM7MyG24tyEg25ws3+7 O5P9NbDOAAHuNT2VLJklIEer+t5efkjrHzva675i8u2UEEsRi1Vbhml9Gn7yRiAPSjQ/t+NPbJDL xSA82nJ2fLDiySJH93XXu8y9auvLV1NeyXKalIiySCQwFeSAAEcach4jMww35vNQ1URGuHot/wAM 6gJGdNVdf77hWMsV9VQAQS5PJDuCfpx4D3p/NRr6O77Pg0fK95VGOqSn00WIKQwVkRgVD8XVm+Ec W+Lfrj4Z71/Nx/mj8fBE/oK79WV/0hJxktpLdYeJ4I0kcKCRatz+Ewk7sftdfE8B72v8xGh6eoP2 nb7fsQs3la/lSRP0q6I8c0aRorhE9YNutZC3wtwK/Ftx2oDTI+Ge9sjq4j+DqPs+Hv8An5JnY6bc Wt7cTm6aSCYKqWxHwxiMUTgan9n7Xid9smI0WjJlEogVuOvej8k0OxV2KuxV2KuxV2KuxV2KuxV2 KuxV2KuxV2KsU/5VV+X/AP1Z4/8Ag5f+a8p/Lw7nZfyxqv55+z9SvY/lx5JsbyG8tNLSK5t3EkMg eUlWU1B3YjCMEAbAYZO1NROJjKVg+5kmWuA7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7 FXYq7FXYq8Nt9c87XMXqwXt5JHyK8xI1OQAJFa+4yiy8rHPnIsGSYaDrXmxfMWmQX15delPOitHI 7FWXkAR13wgm27BnzeJESJ3LN9Q8z+Zorq9t7LQ3uGhKrbbSgNW4hiqXdI4WDRytIOEp4hDyoelz 0iXT+efOcFrdTHypcST24k5WsfqMeSRyFfTk4UnBkVV+EDY1HLYFVnmKuxV2KuxV2KuxV2KuxV2K uxV2KuxV2KuxV2KuxV2KuxV2KvE9JsPLV1o0MepastrOs0soiVXJAkCLRvgIr+6rse+UgCnl8UMU oASlRs/o8vJHWNlodrruhppepLegXoMiFXVxzKb7qop8GGhYbccMcckOCXF6maajLpkd3eTya16P 1edFuYCrOqO0ZdUKg7/AeVe1D9E6d5LCSbtKx5m8seq7R+Zl2WVxSKV+CyfCG5VO4bdSetR9LSPB PejJrzR4p3ik11vVteVvLHwlI/cUcq/Egt8Gx+L4vnjS+Ce9edf8via6La4aR27xy24jlAi5xRn1 R/uyiiEtUk/a616mmXhHv6fqR9lp36Qs7e7g1JpbWZUeOVVdC8dB8IDNxCnrunX7sFI8E96Il8vO VkEF7LAZJFbmpNQixKnDZgN3XnUDvTDSnD3FY/l+9dZFbU5P3qoD8J2ZAKFasaVbkW8a+wwUg4T/ ADkbpenS2QmElwbj1G5KWFGAqTQkluXXCA2Y4GPW0dhbHYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXm 3/KnP+1v/wBO/wD19yvw3SfyP/S+z9qM0j8q/wBHapa336U9X6tKsvp+hx5cTWlfUNPuxEG3D2Xw TEuLl5ftZbL5e0KVrhpdPt3a7ZZLotEhMjJspeo3pU5Y7ZaPLPl0S+r+jLX1PTMNfRT+7Nar06bn FW38uaA8zzyadbPNIzvI7RIxZpFCuTUb8lFDiqyTyt5akDB9LtSGAVh6KUoAVA6dKMRiqPtbW2tL eO2to1ht4VCRRIKKqjoABiqrirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVeOR/n1qMpIi0J ZCo5MFmY0A7mkebD8kO9z/yY70doP51Xmp67ZaXJpCQfW50hZ/WYleZpXiUGRnoxGJNsZ6QCJNvU JLq1jDmSZEEVDJyYDiG2XlU7V7ZguE4XdqWkUTRlov71eQqu1fiFdumKrpJ4YwxkkVAgDOWIAAOw Jr0G2KrXvLRH4PPGr8gnFmUHkwqFoT1IxVuO5t5GCxyo7EEgKwJIU8SdvA7YqqYq7FXYq7FXYq7F XYq7FXYq7FXYq7FXYq7FXYq7FXh3krfytBx5p8b+uyO0ShUclGdlJJ5tJQFFL/u6D22eb6nY5vqb mljPnfy2JFZ72W9SR7iRaSGJXEKAsxaU/HG4/eszAAdOmIHol3Uo+iXdTONR8u+VLi/1J7xrs1kP 1qJFChaSQXkhEsUYnZawxklnNFPFabAa8QLg8BSzU/I/kGJbvSJrm9hQBxFDG6gW6zR+kwhYp8I9 O64/ETUGhrviMZItRjJFple6F5Xutdu7yS+uo7+9lAnVvRMQa04ohMU8bx8YhyC8gftM3WjY+GV4 Ch4tF8mrdWM8V/dK31g3tokUUKcpDJMWjYRwLIyl+Y4N06ClaE+FJPhlF6B5a8saLqsE1teXf1uC JoVtZig/dqhkAZFjRuMayHh28MHAWPAU+TzdoTxhxOd0d+PE1onX2+W+9D4HD4UmXhSd/i3RfReY SsYo+PNuNPt0psd++PhSXwijtP1Ozv1ka2YsI24tyBU/Ohoe2RlEjmxlEjmisixdirsVdirsVdir sVdirsVdirsVdirsVfPmmaF+ZWn2i2qaG00Ubc4WljHqx1NWEUqMksYY7niwzbSnjJu3aSnjJu0f pHl7z5d+dNG1HUtKmgtrOWBF2qkUMb8vtMzOd2LEsSSTkJ5MYgQDzYynAQIBe2y2FhNX1raKTk/q NzRWq4UJyNR14gCvhmtEi64SLprCxnJM1vFKSakuitU7eI/yF+4YiRCiRWPpOlOxZ7OBmapYmJCT y612798PEe9PEe9cdN04hgbWEhvtD0132I328GI+nBxFHEVyWNkihUt4lVa8QEUAVXiabd12+WPE VstPYWMjK8lvE7qAFZkUkBa8QCR2qaY8RXiLnsLF5Hke2iaSTaR2RSzbU3JG+wx4ivEV8NrbQFzD EkRkNZCiheR8TTriSSpKpgQ7FXYq7FXYq7FXYq7FXYq7FXYq7FXYql/+INH/AOWlfub+mab/AEQ6 L/VB8j+pyfymXuXRa3pcsixR3AZ3PFVo25P0ZZi7c0mSQhGYMpbDY/qYy0uQCyEdm1aHYq7FXYq7 FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FWI6Jp9pcWbSTWjTtzIDh+IoAD05LnnHYfZ2 HNgMsmIzPEd+Kun9aLudVmlGVCVfD9ipPZQ2us6eIrY24ZwTV+XIhh/lNTLtRoseDW4BDGcdy/nX e4/pSYwymeKdni2Waje+fGvL+Cwit1VSv1WSUxqir9YhHLkJJJD/AKO0rPyhFGAC1/a9A4g6mkvn v/zbjtbsxWNlNeRCT015osMhEbhDDWQOn7zg1JTuNvhrUNoZ20sairOqjxJA9sTIBad6ke3xDcEj cdB1xsLS4EHoa9vuwq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYqwXT9Ss7WApLaCeQtXk XK7U+Rzybs7tPBgxmM8XiSvnxV+gu/zYZTNiVK8V/Bd6vYmK3+rhHVacy9fir3ApmXi7QhqdbgMM fh8MgPq4uvuDXLEYY5Wb2ZFJoVo80srSS1mkErLyFNgVZRt9llJDDPRTpYkk2dzf497qPEKyLy/b xCQJcT0lWQSglDyMwUO1Sh3PAHbvgjpALone+7rz6Kchak8uWcru8ksztIrI5JShVmLkU48ftnl0 wHRxO5J/G/d3qMpXT+XbCdnMhcrIHV0+ChEnqV3K8gP3zbA0/GplpIS59f03+tRlIRdhYxWVsLeJ maMPI4Lnk37x2kIr3oW2y7FiEI8I8/tNsZSs2iMsYuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2K uxVDfovTP+WSH/kWn9MwP5L0v+pY/wDSR/U2+Pk/nH5tpp2no4dLaJXU1Vgigg+xAycOztNEiUcc AR1ER+pBzTOxJ+aIzMa3Yq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXzP5T8t eefM8c09heyQ2dueMt5c3EiRBgK8QV5sTQ12Gb/Nlx49iN/c48QSjPL9r5l0rzzoMF/dTSQXN1C0 UnqSmOROYB+GTiw91ZQehpQjI5DCWORA6KLBew3v5hWNjeXsN1Y3MVtZvJH9fcBbd3jheUosjFVL tw4qBXfwzROQpxfmbobalf2MqNA9jcG1aSRkVHkAuSApYjc/UmoP8pfeiqsPzA086fqF+IGEVgYw 6GSIyEvL6VWjRnkQctwWX4l3GKoEfmrpJuo7T6nOk0sbSJLKY4rcqvoDn67sqhC1zx3oarSnxLyV Xyfmt5eAuGjjlKQtGFkkMcUbq0ckjsrOwr6ZhaNh158R+0DirpvzT0mF7xWs7iT6nLewH0QHJksZ BGVYfDw9Rj8LH4f5mGKrZvzU0mG4miazuHWG8nsXkiCsEkguUtayVK8VeSUcTuOxIOKsl8vawdY0 tL827Wod5EETsrH905jJqu3VTiqY4q7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq8F8l+ZbOTyLLocWt HQdYspZbq1kDrElz8JKRtI/FRVzQjkNqdc3OfEfE4uHiifsaInakwv8AzTYat5q8l2MUsV3qFrcQ yarewLSN7l1jVwrDZ/sbsNumQjhMYTPIHl7kmVkPULmbzUL6aOCCFrMBvRmNORPDmtVLj9r4PxzX AQrfm27uEnmlJSvpRSxGdRzJVWEHrqCQoJ5H0eR3p9PTGoLu2k/mT6/wkgAsxM9JVEbEx8gEqDIp A4Vqd2r27Y1CvNd0JJc+dxYzPFaRG8UkxxOycWX0pNlIb4T6nD7Rp8uokBjvnsjdFJN5lM8AeAek ZSLgj0lCpUfZPqOWWlewb9WRqFJ3TrKkuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV2KuxV5d/wBC /eWv+rje/wDJL/mjNj/KM+4NfhBG6L+SWgaTq1pqcN9dyS2cqzIj+nxJQ1ANEBpkMmvlKJFDdRjA eiZgtjsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirzj/lZGuf74tf+Ak/6qZq fz8+4fj4vRfyRi75fZ+pF6R591i81O1tZYbcRzyKjlVcMAxptVzk8WtnKQBA3ac/ZmOEDIGWw8v1 MpuPM3l+3kuIpr+FZbXj9Yj5AsvJlQfCKn7Uig+BYV6jNm6NS/xh5W9My/pW29GjMsxkURsscbSu Uf7L8Y42Y8SdgTiqcYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq8+8hQ6dc+ukt nby3MPEo8pJZg1a0Vua/DTsM1eiETdgW7/tOU40RIgHuRWp2Vjb+YtKe1tYoSboJM8JPHkODBabL +3X7OTyQiMkaHVqw5JSwzEiT6dr+KJ1Gy8qtc3jXdh9YpLxmhlmYxFuS3bFIZH9EcpLZWIUfEw+L qczTkAdZHATXmh7ryv5BnjureTSleK6dkuIjK0aBiZFIFZFWI0U8QtOopTs+IEDCWQJ5it5Gb04n ZFEhLEgfDGobkAeoYHbHxAk4COa2XzNaxwtK0MhVVLnhxagClhyoTxPw0I7VFcByhI05Jq0Ta63Z XE5hUlG5Mql6Ly4lQCteobl8PyyQyAlhLCQLQy+ZbczCIwurFmUFqAHjTpXx5fL3yPihn+XNXa6P zHauoYI1ODsd1r8Csx2JDU+AjlSldsIyhB05DpPMdskpjETuu/GUFeBo/CpYkAL3ricoUac05vMV ssrRlD8DRqxDA/3sYkB22IoabY+KFGA0m2WNDsVdirsVdirsVdirsVdirsVdirsVeW/4E8y/8s6/ 8jE/rmm/J5O56f8AlPD3/YUx0byt5ntr6y9dALO3nWYr6iEA7cmoD4DLcWnyCQvkC42o1uCUZV9R FcnoObR0LsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirEv+Vo+VP9+Tf8ij/X IcYdd/KmHz+SvY/mL5avryGzgeUzTuI4wYyByY0FThEwzh2jilIRF2WTZJznYq7FXYq7FXYq7FXY q7FXYq7FXYq7FXYq7FXYq7FXYq7FXYq7FXg/lrypqWu3KLCpitOXGW7YfApA5EDpyanbKBG3k9Np ZZTty72Qt5Zi0TzlpkEIkMa3UJSVgW5q5G5YBVWhHEDeu/tkqouYdMMWaIHeGSata+anu9QSLXIb OF2QRty5mMGeKRR6KxxshFuky/3x5k1PH9m23fGYHVLLm08/LZ3KweabP6w4lFrdSdJPgkiVpEEZ SLi5D1j2BXfkKgi0eJHvZ8NRsS3ATpy5+lTkPtmvw/P4T92G08Y72otSsJmCxTo7MaKFINT128dh XBaBOJ5FE4WbsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdirwDTvMOp2Fjc2UEpEFwAQOTAx uCD6kZUrxagofEZQC8jj1EoRMRyKJ8tXl3deatKe5nkndbiMK0jM5A5dAWJxHNnppmWWNm9w9pbR tOad5/TZZ5DyeVJJEYmhFaqw6A0Hh2y6np/CjdrRoOkgMFg4BlCNwd1qirwCfCw+Gn7PTv1xpHgx 7lz6LprsrPGzFGLxgySURiweqDlRfiHbGknFFfDpllC0TIh5QBhCWd34BwAQvImgoo2xpIxgfBFY WbsVdirsVdirsVdirsVdirsVdirsVdirsVdirsVdir//2Q== + + + + 3 + + + + open advocacy + open advocacy + + + open educational resources + open educational resources + + + public engagement with science + public engagement with science + + + open evaluation + open evaluation + + + open access to research papers + open access to research papers + + + open research software & open source + open research software & open source + + + open research data + open research data + + + reproducible research & data analysis + reproducible research & data analysis + + + Open Collaboration + Open Collaboration + + + Open Principles + Open Principles + + + OPEN + OPEN + + + + + + xmp.did:3381cc39-e5c5-4647-aa5d-986966253dc2 + + + application/pdf + + + os-mooc-module-logos + + + uuid:6a28da28-2754-43fe-8104-c27d3ded64ad + xmp.did:5898fcc8-c100-7744-93ad-7b8ecc658ce7 + xmp.did:a97db6c3-688d-7045-a98e-fc0efec1be67 + proof:pdf + + + + created + xmp.iid:a97db6c3-688d-7045-a98e-fc0efec1be67 + 2018-08-08T09:02:19-04:00 + Adobe Photoshop CC 2017 (Windows) + + + saved + xmp.iid:2b2c7454-aa01-8e41-8ff4-46113cf52fcc + 2018-08-08T09:10:17-04:00 + Adobe Photoshop CC 2017 (Windows) + / + + + saved + xmp.iid:5898fcc8-c100-7744-93ad-7b8ecc658ce7 + 2018-08-15T13:00:42-04:00 + Adobe Illustrator CC 22.1 (Windows) + / + + + + + xmp.iid:980fd760-1509-ab48-8681-642420ba12f1 + xmp.did:980fd760-1509-ab48-8681-642420ba12f1 + xmp.did:a97db6c3-688d-7045-a98e-fc0efec1be67 + proof:pdf + + 1 + 720000/10000 + 720000/10000 + 2 + 65535 + 1280 + 1474 + 1 + True + False + + 789.820992 + 151.000000 + Pixels + + + + + Montserrat-SemiBold + Montserrat + SemiBold + Open Type + Version 7.200 + False + Montserrat-SemiBold.ttf + + + + + + Cyan + Magenta + Yellow + Black + + + + + + Default Swatch Group + 0 + + + + OS MOOC Yellow + PROCESS + 100.000000 + RGB + 230 + 184 + 43 + + + OA MOOC + PROCESS + 100.000000 + RGB + 24 + 137 + 171 + + + white + PROCESS + 100.000000 + RGB + 255 + 255 + 255 + + + OS MOOC RD + PROCESS + 100.000000 + RGB + 217 + 64 + 67 + + + OSM Software + PROCESS + 100.000000 + RGB + 58 + 181 + 133 + + + OSM 2nd TExt + PROCESS + 100.000000 + RGB + 112 + 112 + 112 + + + OSM OA + PROCESS + 100.000000 + RGB + 151 + 83 + 158 + + + OSM OE + PROCESS + 100.000000 + RGB + 253 + 209 + 51 + + + OSM Grayblue + PROCESS + 100.000000 + RGB + 89 + 98 + 166 + + + OSM Edu + PROCESS + 100.000000 + RGB + 242 + 147 + 117 + + + OSM Accessible Green + PROCESS + 100.000000 + RGB + 55 + 165 + 118 + + + OSM Yellow Accessible + PROCESS + 100.000000 + RGB + 181 + 143 + 35 + + + OSM Peach Accessible + PROCESS + 100.000000 + RGB + 209 + 125 + 102 + + + + + + + Document + Adobe PDF library 15.00 + + + + + + + + + + + + + + + + + + + + + + + + + +endstream endobj 3 0 obj <> endobj 44 0 obj <> endobj 45 0 obj <> endobj 10 0 obj <>/Resources<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 11 0 obj <>/Resources<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 12 0 obj <>/Resources<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 13 0 obj <>/Resources<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 46 0 obj <>/Resources<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 47 0 obj <>/Resources<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 48 0 obj <>/Resources<>/ExtGState<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 64 0 obj <>stream +H�lTMo1 ��W�{o���8�m��TJ�����������B�L��l�?��NvW{���v������b�I֦�6��Iצ�6����Mm��(�A�<�<�� <�<��2�!�>u��n����ɳ�O��{ x��v�`�o\l>��?NLR�hL)���{���ο\� �?W�>P� +�������E�7��������T�M z�5��)l�l+㘘�����vB��OW�b���7+/����|�a.f�$�0> + ���c �Z�?������~�����e�YB��R�)��}��w��h�Rn��9�ĩy��/ ���$T���D���*� 2ဃ���B@K� ��+��J!g@!�xC�o�Ԫ�/���-�r�.F#M�C����b�ŏ�� +W6�ڨb�&(�g�p1%(N���p��P���U+ʔ9/�"v˜LQ�:˾oV��B�N�ͼq�j�)_�B�!F� +5���rM�E�j�X4[!�<�T�P�{B �tp6��!���F�s񪘯�']�h4f�kONB�r�Nï�S��h8fÈ�e�q+��c�͌:Tj��*pDJ� S�fH���B��,���*[\���ig)��\�, �6�m�F2RH�"T1������!D�p����#�R�$�ү�8f2[�,K��MolH�QC�� v_n�ͥ��v�7����K� 9�ú�’�r�6'D5./̝�$�5 � � ��/u}�\ �z��2x�e����[��0djvX +endstream endobj 65 0 obj <> endobj 17 0 obj <> endobj 14 0 obj <> endobj 26 0 obj <> endobj 27 0 obj <> endobj 28 0 obj <> endobj 29 0 obj <> endobj 30 0 obj <> endobj 31 0 obj <> endobj 32 0 obj <> endobj 33 0 obj <> endobj 34 0 obj <> endobj 35 0 obj <> endobj 18 0 obj <> endobj 36 0 obj <> endobj 37 0 obj <> endobj 38 0 obj <> endobj 39 0 obj <> endobj 15 0 obj <> endobj 40 0 obj <> endobj 16 0 obj <> endobj 41 0 obj <> endobj 42 0 obj <> endobj 19 0 obj <> endobj 20 0 obj <> endobj 21 0 obj <> endobj 22 0 obj <> endobj 23 0 obj <> endobj 24 0 obj <> endobj 25 0 obj <> endobj 123 0 obj [/View/Design] endobj 124 0 obj <>>> endobj 121 0 obj [/View/Design] endobj 122 0 obj <>>> endobj 119 0 obj [/View/Design] endobj 120 0 obj <>>> endobj 117 0 obj [/View/Design] endobj 118 0 obj <>>> endobj 115 0 obj [/View/Design] endobj 116 0 obj <>>> endobj 113 0 obj [/View/Design] endobj 114 0 obj <>>> endobj 111 0 obj [/View/Design] endobj 112 0 obj <>>> endobj 109 0 obj [/View/Design] endobj 110 0 obj <>>> endobj 107 0 obj [/View/Design] endobj 108 0 obj <>>> endobj 105 0 obj [/View/Design] endobj 106 0 obj <>>> endobj 103 0 obj [/View/Design] endobj 104 0 obj <>>> endobj 101 0 obj [/View/Design] endobj 102 0 obj <>>> endobj 99 0 obj [/View/Design] endobj 100 0 obj <>>> endobj 97 0 obj [/View/Design] endobj 98 0 obj <>>> endobj 95 0 obj [/View/Design] endobj 96 0 obj <>>> endobj 93 0 obj [/View/Design] endobj 94 0 obj <>>> endobj 91 0 obj [/View/Design] endobj 92 0 obj <>>> endobj 89 0 obj [/View/Design] endobj 90 0 obj <>>> endobj 87 0 obj [/View/Design] endobj 88 0 obj <>>> endobj 85 0 obj [/View/Design] endobj 86 0 obj <>>> endobj 83 0 obj [/View/Design] endobj 84 0 obj <>>> endobj 81 0 obj [/View/Design] endobj 82 0 obj <>>> endobj 79 0 obj [/View/Design] endobj 80 0 obj <>>> endobj 77 0 obj [/View/Design] endobj 78 0 obj <>>> endobj 75 0 obj [/View/Design] endobj 76 0 obj <>>> endobj 73 0 obj [/View/Design] endobj 74 0 obj <>>> endobj 71 0 obj [/View/Design] endobj 72 0 obj <>>> endobj 69 0 obj [/View/Design] endobj 70 0 obj <>>> endobj 67 0 obj [/View/Design] endobj 68 0 obj <>>> endobj 53 0 obj <> endobj 125 0 obj <> endobj 126 0 obj <>stream +H��VgtU���3gN�z؞{͍%t��(��C�"$� !M@�� �>���R�Io�)�PM���P�����ͽ ��z��;g�3{f���o��� %`"XС}��:]S?���S����S������NrF��u*�?�K���S$���: ���' 2������"W��L���c�d���:��@ H^�W�_�~�����f7g6U�(��5dhB|�c&��]�VR|zJ��a�:���nr|R��M3�ݝ'К���ӊ6���V;0����r����wT�80M���ֹ�h��bo����<�\�p5�C��is����4��`� 8+0F�(;��M�����G�A���p��j�V���%m����yrH�Hg�+ �l�֒,��}�>d��n������� �yI��v�*ۤ�|��䁪P Z�K /�@ ɐc` +,��`��]�5��|(� pC��E��{��I�i� ��Z܂��8���x��/d�C��>=M�P3jG])��R�� �[nK�2�@e����L����Y8��\��p+����x/�%*Eeȥ�I�����K��-�#�䚴���Fn�u�� vR$ dI�쒝�C��9(�e��_��0N���R�)��NvK�̗/d���i#��T�����`$�e���q�E�).Kcɒ�2���ܕ�2^����Z3��)�^��›���L� �p��Pg��� q�9#�dg�3�I.����� + �x '(�3��q#T\��kT������ٝfP6��\:m�U� +�J[��(��UӪm=c}`-�����e��p}n�}8����Ev��]͎1�&ܔ7U���L c�6�L+���4���f��b���f��k~r��%ݲn%׸^7ڭ��u�Ϲ�n�;�}��r��ݥ�Jw�����n���x=ў���Ξޞ���o��������xkx[z㼉�,�Z�F�6�.�߾'.-~�+���e-� "��հw�Q��U�� U�x��f�N9t�ҭ�H�R$���H��pI.Ǖ�7�Fܗq*��9vI��e�4!���0��5Q���i����icz��f�I7�T3#�D��(�� +n�����>��4w�"��]�H,sW���Mn�#$�+�<==���H�R$*>B��/ӷƷ������Q$..~�*X�Ã�^T�.��3�����5�c�ø��}�W���u��� �b�����|�9O�*�&{х��� +��-L(ؤ��_4ʅ��g�/�.�������Tr�mՖ�m���_9:��o��˟Y����=гҭ1�cM�2��V�u��[�dY����M�>�p�����'������xo�v={���^cg���v����yX���I;Ͼh�hߕ0��J%с�ԞЧꞜ��z� >��IɓSr�Q<� ���F�W<'��b�>����V� ��\��䚲�u���[R䆓�\�K�K g���Ui�$K5NUNn&M���*/�� U��aT7��r�W����s���� ���&�! �� \���F5C�e�*�"\���W5&�0WTcTap�j��*SV�%R�PMU�n����@��".T����S�� +�������q%����$n����lh�{�y<uq�œ�O@{̃v�-t�|�g`ނD�=���M�C?t��0�@��� �T,�ax��(bG!�����' +�I�Pux�"�c�P-XN��Jj+�!��氅^���l��`7u�/�|A�� �<��|s��I��h��ap�F�e�I�6��h��� G(�A n��0����z +*�����:�)h���� 2�)h�G!�c�̧(�s�.�F�4��� +^�~x �Re�C�`5���g8@=�[[�%|E���BU�ET���k4�It�:�N�3��ښ� r��N?�Շ;B�v��8�N��TpJ9��N�S�)��X� y��y.��L����Y�O��!�4�ˑ<���x~�'�D��S�M~���wT?����/<�3xO������W����;��W{xT�?3�sy���f�l�ٻ�CB�{����]B��b)w��V�V�ZP V�b��c �֯�ڹ4�j�mUT�KK[Z +h}}��~�_�}��?�5�}�̜sf���9s��GS�1qND�!�5 8�R�m���N�x Τ�3�p��t� �S�N�\�͸[�R�� 0�a���p�؍��^���e�~\,nEK���3�=�NJp��%&ů�r� ~F<����į1%.v�^\&��r�?/�+ģ�>��c�B����/��q�x�(��UT#��)�O�g�G<��q�x�D�����J��k1#~���Kx�x׋W�+�~U��׈��Z�:n�:��^¯��x�8�_Gq�8���x�xoě�q���n�-�/x�8���[��p�8���o�S������o�6�6f!�D٣c}��^��Y�9R�8\�NB;��;p=gp/�M܀-���W�V^ ����τ�q~�#��B���q���K`�CO��< ��.x�/�|=�W�i~����)����_|+Ա�a4�9T��P��.�a?�*� J�m0���l; g;���Z؋��^�v�:�a���= ��30�= s�S�a@����p;�6^��c�|,����, ;�?�a��`�w���_ �*8��p�g�(�c|5���>� ��7��f�'�ll7�0v;�]�չ7���+x#���M/�_�w�+�^#04�1vw�xظ�x�x�+�1�q�8b�I]� �~����s��~��3��{[��Oo��g�w���w���7��>@���L���V��E���|��||�������uT]�'�?<;��5k.��q��.%��OA���U��x����e`��1��}�:�~L���� +�� O�dT.M*c� +W�2�T�_#��]�����蟂L��6C!��j�א�ǣ��J�k��۲G��)%V�NeÝD&���R%a��J7d�̬+U*E�Vϔ�I��]k�O���ɦ�*O�>q�����=�㛾�y&Y�ʝ��NWAR+��c&�=��L�UCFk�!t{^O�S,�y������Ėt�� :)W�W�'��G��-B"{�K-�ͼ���J�DF�4�H�Ȭ���t �[��~�Lwz��<�Z��$3�3 +�Gڪԉ���֠��(FV<�x��2d��ƨ*��6ua�-������ߖ3���-N"���0����wa2�!ܾLd���d��`�((i��E+)�V�-��!��ɴ +̳�/a��//��0����� +;�<�z�mQUi�����Y�����{�R�:iVI����m�s.�� ���_f}���iQUm'��@��y�U�jkcT���K��<� �6ǯ��v��Au��X:��":�)��A��T�G�z�DI8��y�6���ұ��!���f^�������t���=7TC0���[���^�X.V�6T6�\Um�eB���nQ�ťO�﫩aP�x�j���1'��F��HT��Ӵ����h;(�t�M��j:� MM;(�t��i�);��m��we��aK�[�/HT5��� #�� � y�TE��7��M �$��4D�4�D�4���� ��a§i��t +��t*��Զes.M�6[�K�b�;�P�ճu��l��(��it:�Q��M����05����#*:�Դ�Y]¥��Q�䞡tf�rv��Kh��N�g�e��-��{s-���j +f�:�u��\�~�$馨�m�F7G՜�S��ΐ�\ +ԇeLv�B@�]��vXT9\�1Th�:�a�ny��*V�Ij��h8��C\ s"��1K��,�9�\5�龍/jK��ZҺ���e�4����q^\��2*�Vn��N7�9������w�K�8�s'm������5i2����N1��vݜʜ�)����TP�` %~dW�Q� +猠o*_AϞE�0�� I\l(��j&7-������Ї�(6�P��{ZA���Ի����vB��0�~&�x�l/D��)�p�%N1\�~K����1��v5�qS&uR��łE�v�9�N3u����k/��R[5E.v`�V�"Y�M��R�S1Z�� ��l�{>�ʭx�NP��O�n^~�6*L�c�K>aJw���Xc�u�٢R5(_B^���H�+�4� Y�� ���\P�����^S��_61m�Fժ�4^b��D�ڋ r�l��[���Nh������ئ�3~ι'6$�Nl'���8H�$@�&7��8B�����$$PVF��� tt�M��/��A�u��LZ�j�*JZ��$Ƥ A�C+�z�w�ܩl�g�B�{��;������wF�NA��2�O��'+�� @r�D�^� ��]�Y�h ��j�j<�d<[�<��Y�q�y�CG<� ���`�D��D�9-6"�&� ،QnW#�!n�-�.D[�]����B��ۅ(��B��ۅhb\3����Xp���b��J���k���D�.ΡI�s��vF�n��;�m@�����$ �1 � /78�o_q�~"����"�L� ��� ��8p���w��8�e"�"}v& � ���D«�+�"���m�^L�^ +��L���G} �9��u1�R̓� �EE*P*��M��QM�ɦX��No�[q)n��>�%zN?�&��[X��u؟x�/@����W+I�"�m!�""�5� �I�iii�4��(Vٜ�-r�T����B�Բ Z;N���^>x��iԯ_K@�UПA��E��� ~x#JL��I�ic��J��SM$�������ɟ;'ә�(v���^�g�*x�G`*���U��VU���;��ڄ~���M���?|�������7�5�k7�� }�ݎOޭXT^�[����K�_�o�m�k�Gj��� ցy�����v��0��m�� ���z$�F�x�*��~���Q dٞ�p������%�6l<�m��+z�;B;5mghG����*s�̖-g"��]=��˗�֯�^��� +zg��+�W>�Ֆ(��P��2Qh�B�BE����Ƞ(�Ap���!u�U�i@��<%�,�B��p��ƍon�?V��=ֶ��aWkO��{I�Y��]7j� +��^�,��OD�%�~2�M1��%H��j\�q�ea:�N{� �V�R`�����G119?�,+9�nC� +u��8�V��@�rS�vν��~����� +=�9�J�"�2���fPWBޱ�/��R&���-(¼;<ŋ��D+^Ȟ�(���t1���} ���;;��K�7-�q��oA��͋:��I������� =����U����XV3�d�+�7����82l���5K�y�b���0O���MB�]��7��J�rC���{q�922u���~��g����<שNB�� +� '���~~�կN�/_���'�}�ktB�}�4����۠WF�|��6�n�t��DVJ�Rx-ge����������[��l.�1�1���%M�.�P�_�B���D���� J�W������;t�=��p���Wp��eO�gNn�C��$��F�92d-f��6����ȮV���Z\����?|���F?G��8Hi���ރY_J��N��yչsr�s���a: �����Ž���J�ee.W����U�*���xz�/�׀�ᬆ7��I��������b>,ֿ����ë� +w��j +�͈f{���s��������N���[�Wݰ�����m���ս�c�ܑ�^VT�Ə���y�S���~���<b +����� > &���M����.�����P�.� zۨ0"�6�,O<>����s=)?�����m�<��!b�!��d�����&B �Γ��H�g������/��$��~8O[�&� e����"��� �pI|r�,�� �9��܃��@��2_��-�5-�ɀ8���SE8����w`��a���붥��"e������ ~�4lb��d7Q���I[�����_����i"؁�ȼ�ȿ��nA�!y�,�-��~�����Uq೻�7 ��P I �!� �@��2� ��&`�-��R�-��j�qD��-��괶@uju�V[e�P������͟��3�����{Ϟݳw�h�X+�5`�Y? �� +�g�b)�X�mVš���3�ՌY�_fI�:�"*C���Wѫ��t��sx��WBL +sLemxc"i��sd�^���`�� �ƚ�riC�$�W�;g�n�~Z�鱒V'��}NF�~�\t�I�?�����N�.v�S��)�������4�o#��a����֒n��7��a���l[Gm<N����WDѷ1�m�o�A߮�=����A�'^�L�8=��n�/�b^���a^�����KG�}��c^!Y������F:�Q��ψ�k�I��K'�k�U��$^<�7��QY�j7��}-���0���O�y�*��2���l`��_��*���,�*֭(`���ԪP��쥊�C�����d�zDZ� +�S����k2ᔤJ�D8�<���!؏.���p/��L�[���fy�^)3��S'�)?(�W�ཅ�(�U=c��H�t}T� �{_�2�ʬb럇�*�� ~L�=�K䔰��9������ +ƐB���u��;� �d�����#�W]�D�U����?��9r)�,u:ݎ���j�I����t�����K��z|�!l��S�[&�����R�'� �]��η�0�B&Z�#��\t�X/t2c���KU)���Gy'����2���_�,�o֏<�|�Ļ;��{�-���h��_��S��TM�F�� �P뤇z��ي�;R�?� �IRj�K�~T���u���~I��)�����)�{��%�/�,�-������}햶���+%I ��()�ϛG���9�~��SI�֑ww��p��#]�8)��u��3S)�F�ےc&�Ε�H;vK��K��~�~�C��qt9�fcLeNC��1�`� ��u �����d^��f�}�A)��R��j��˛���b���?#�]-^�4���^�,q;�oFliuv�y�m���X�0�.�9��E9�{n6Ob�,��>?��3���-f�˖v���6��3L����L*o�B����Q���M���-VK��7ݓ��^�n��&�e���$y��7o\֭CP�vA�����Z���!����5�~��G��J���qH��״�?��,&�� ~EY��p��inҐ�� +�lBn����BO�?����K�.��� )6� ѫ��/-��*�o7�� ľ;�4��z��F� s3� ��O���M��/�a�������\�Y���:ނ�.ྴ}�Pv'y�iބ��!��T1艽Zr��}�Ÿ��v]�\ �v�}?&��b���;WO3im �}���m��PeK�����l�����`�ì���:�� Blj�t��*�����y�"97�k �� +��y��>��";�:�_U��*Pw +��k�ܼ��E���d,�q^���<*��JKs eVﱡ>�Lb�-K�=�Ƕ � +�W�>�� Ɗ߳�3��%f�k�p��E��Qk���m�o�?�N|���Ƀ��挘�7H������oO�U��L}%���.gn�J���뫧�(t������nz'k�SVB.�A\ #`�+��m�a��K�A:����}�W��6��N +]���!���(��P��5E.�#�2�_��t��|���g�z0�7���6�G�������2N��er�͗H�*��e�8[f�9�r|qz���Ѝ���M-��������u ��i�52RwF��ܱ�K����/�2>�`}�}ɿ\r�jN�a������$U��o�3�v������[�H +\��"s��tr��~�=o�D+w���ʅ�i��S��U�S�ƓL�5)�,O�Ÿ�{Hf�[%�� +�L��~�������D|�Bl�y6��YO�!��x]�9l ����n���p�d��2k WV����`��8!l�'�N4.x=��HX��BSh +M�)4����BS�� 4��D��X+���\*�DK��wD�[I׋gK���H��@7r>�0�/s����Ty�Hm��]\I����Z��J7�sq���\���j�����\3Il@���X�����ŋ���-+�jI��l�sK��F.]2qYy�?�S=K�P=�G�� +"��h��C]�1�T�Z+������������"t��� �:9���iD����{��s�y$��8A��4��NX����V =��3��~*W+f�Z^�A6O햫�^�/ ࣍[~M8��6��bdQ!bs?��\�Y�aźO>����|!w�X��,V�~�wQ��h�e�QC�U>ZD��=269 g<[DGy��F&}�QE�#5���}]�d=4����};��_5ɯQ<��ZG/�(��$=V�ҫ +�vE\_�aY�Z�O��jIꎼ��W����M�P.���t O���91+�Ŕ�cBy�ʐ�!�+b" #Z�z'f_��(�0��a���q�E�^1��*�Я&�e]�M�ٴ2��" ����(�)ݐa� r0� +endstream endobj 52 0 obj <> endobj 66 0 obj [/Separation/All/DeviceRGB<>] endobj 51 0 obj <> endobj 127 0 obj <> endobj 128 0 obj <>stream +%!PS-Adobe-3.0 +%%Creator: Adobe Illustrator(R) 17.0 +%%AI8_CreatorVersion: 22.1.0 +%%For: (Mike) () +%%Title: (_os-mooc-module-logos.ai) +%%CreationDate: 8/15/2018 1:00 PM +%%Canvassize: 16383 +%%BoundingBox: -135 -1164 1336 932 +%%HiResBoundingBox: -134.729196378521 -1163.30325902354 1336 931.12314384281 +%%DocumentProcessColors: Cyan Magenta Yellow Black +%AI5_FileFormat 13.0 +%AI12_BuildNumber: 312 +%AI3_ColorUsage: Color +%AI7_ImageSettings: 0 +%%RGBProcessColor: 0.094117648899555 0.537254929542542 0.670588254928589 (OA MOOC) +%%+ 0.850980401039124 0.250980406999588 0.26274511218071 (OS MOOC RD) +%%+ 0.901960790157318 0.721568644046784 0.168627455830574 (OS MOOC Yellow) +%%+ 0.439215689897537 0.439215689897537 0.439215689897537 (OSM 2nd TExt) +%%+ 0.215686276555061 0.647058844566345 0.462745100259781 (OSM Accessible Green) +%%+ 0.949019610881805 0.576470613479614 0.458823531866074 (OSM Edu) +%%+ 0.34901961684227 0.384313732385635 0.650980412960053 (OSM Grayblue) +%%+ 0.592156887054443 0.325490206480026 0.619607865810394 (OSM OA) +%%+ 0.992156863212586 0.819607853889465 0.200000002980232 (OSM OE) +%%+ 0.819607853889465 0.490196079015732 0.400000005960465 (OSM Peach Accessible) +%%+ 0.227450981736183 0.709803938865662 0.521568655967712 (OSM Software) +%%+ 0.709803938865662 0.560784339904785 0.137254908680916 (OSM Yellow Accessible) +%%+ 0 0 0 ([Registration]) +%%+ 1 1 1 (white) +%AI3_Cropmarks: -134.729196378521 780.12314384281 655.091795410228 931.12314384281 +%AI3_TemplateBox: 682.5 384.5 682.5 384.5 +%AI3_TileBox: -126.458700827469 558.983143499489 646.821253129135 1152.26315849125 +%AI3_DocumentPreview: None +%AI5_ArtSize: 14400 14400 +%AI5_RulerUnits: 6 +%AI9_ColorModel: 1 +%AI5_ArtFlags: 0 0 0 1 0 0 1 0 0 +%AI5_TargetResolution: 800 +%AI5_NumLayers: 29 +%AI17_Begin_Content_if_version_gt:17 1 +%AI9_OpenToView: -309 1006 2 1465 1040 18 1 0 60 101 0 0 0 1 0 0 1 1 0 0 +%AI17_Alternate_Content +%AI9_OpenToView: -309 1006 2 1465 1040 18 1 0 60 101 0 0 0 1 0 0 1 1 0 0 +%AI17_End_Versioned_Content +%AI5_OpenViewLayers: 77676777777777777777777777767 +%%PageOrigin:0 0 +%AI7_GridSettings: 72 8 72 8 1 0 0.800000011920929 0.800000011920929 0.800000011920929 0.899999976158142 0.899999976158142 0.899999976158142 +%AI9_Flatten: 1 +%AI12_CMSettings: 00.MS +%%EndComments + +endstream endobj 129 0 obj <>stream +%%BoundingBox: -135 -1164 1336 932 +%%HiResBoundingBox: -134.729196378521 -1163.30325902354 1336 931.12314384281 +%AI7_Thumbnail: 92 128 8 +%%BeginData: 7650 Hex Bytes +%0000330000660000990000CC0033000033330033660033990033CC0033FF +%0066000066330066660066990066CC0066FF009900009933009966009999 +%0099CC0099FF00CC0000CC3300CC6600CC9900CCCC00CCFF00FF3300FF66 +%00FF9900FFCC3300003300333300663300993300CC3300FF333300333333 +%3333663333993333CC3333FF3366003366333366663366993366CC3366FF +%3399003399333399663399993399CC3399FF33CC0033CC3333CC6633CC99 +%33CCCC33CCFF33FF0033FF3333FF6633FF9933FFCC33FFFF660000660033 +%6600666600996600CC6600FF6633006633336633666633996633CC6633FF +%6666006666336666666666996666CC6666FF669900669933669966669999 +%6699CC6699FF66CC0066CC3366CC6666CC9966CCCC66CCFF66FF0066FF33 +%66FF6666FF9966FFCC66FFFF9900009900339900669900999900CC9900FF +%9933009933339933669933999933CC9933FF996600996633996666996699 +%9966CC9966FF9999009999339999669999999999CC9999FF99CC0099CC33 +%99CC6699CC9999CCCC99CCFF99FF0099FF3399FF6699FF9999FFCC99FFFF +%CC0000CC0033CC0066CC0099CC00CCCC00FFCC3300CC3333CC3366CC3399 +%CC33CCCC33FFCC6600CC6633CC6666CC6699CC66CCCC66FFCC9900CC9933 +%CC9966CC9999CC99CCCC99FFCCCC00CCCC33CCCC66CCCC99CCCCCCCCCCFF +%CCFF00CCFF33CCFF66CCFF99CCFFCCCCFFFFFF0033FF0066FF0099FF00CC +%FF3300FF3333FF3366FF3399FF33CCFF33FFFF6600FF6633FF6666FF6699 +%FF66CCFF66FFFF9900FF9933FF9966FF9999FF99CCFF99FFFFCC00FFCC33 +%FFCC66FFCC99FFCCCCFFCCFFFFFF33FFFF66FFFF99FFFFCC110000001100 +%000011111111220000002200000022222222440000004400000044444444 +%550000005500000055555555770000007700000077777777880000008800 +%000088888888AA000000AA000000AAAAAAAABB000000BB000000BBBBBBBB +%DD000000DD000000DDDDDDDDEE000000EE000000EEEEEEEE0000000000FF +%00FF0000FFFFFF0000FF00FFFFFF00FFFFFF +%524C459A9494949A9494A1FD49FFA8FD05FFA8FD04FF6F949494709A6FA1 +%FD3AFF84AFFD0CFFA87D7D7D527D5252527DA8FF949ACBCACBA194A1FF7D +%A87D7D7DA87DFD31FFC9AE84AEAFFD0AFFA87DA8527DA8527D7D277DFF6F +%94A1CAA29A69C4A87D7D84FD057DFD2FFF7EA1A0A7838484FD05FFA8FFA8 +%FFFF7D52A8A8FF7D7D7DA87DFF9493C3FFCA9494A1FFA8FFFFFFCAFD30FF +%CFC9A277A1A0C9A7AE84FFFFFF7D527D7DA87D52A87D7D7DA8A8A852FFFF +%94949BFFA89469C4FFC4A1C3A2C4A1C4A8CAA1C4A8CAA1C4A8FD24FFCBA7 +%CEA0A1537DA0C9A0A884FFFFFF7D7D7DFF52527DA8525252FFA827A8FF9A +%93C3FFCA9494A1FFCBFFCAFFCAFFA8FFCAFFCBFFCACBA8FD25FFA2A2A1A7 +%A0A1777E7DC9A1AEFFFFA87DFFA87DA87D7D7DA8A8FF52A8527DFF6F9494 +%9A709469A1FD33FFCB84847DA177A1A0A0767776A784FD07FF52A8277DA8 +%527DA852A8A8FF94949493949494A1FD33FFC3C37D83597D779B7CA67CA1 +%A7AEFD06FFA87D527D7DFF527DA852A8FFFFA8CAA8CAA8CAA8FD33FF847E +%A0A0767D587D767776A07DA784FD07FFA8CBA8CA7DFFA883A8A884FD3AFF +%C8A87E7E7CA09A7D587D77A07CA1A7AEFD08FFCAFFFFCFCAFFAEFFA8FD3A +%FFA1C9A6A77C59527C7576527776A076A784FD4AFFCBC4A1C9A6A67C7D53 +%7D767D77A17CA1A7AEFD13FFC7C7A5C7C7C7A4CEFD30FFA1C39BA17CA07B +%7C5376527776A077A784FD13FFC7C6C6C6C7C6C6C8FD2FFFCAC4C3A19BA1 +%9AA07C7D767D52A17CA1A7AEFD13FFA4C7C8C8A6C7A4CEA9A87D7D7DA8A8 +%7DA8FD26FFCBA1C3A1A19A9B767C5276527776A07DA784FD13FFC7CECEC6 +%CCCFC7C8FF7D7D7DA87DA852A8FD26FFCBC4C3C49BC39B9B7C7D767D77A1 +%A0A1A7AEFD13FFA4FFC9C6A5FFC7CDFD08FFCAFD09FFCFFD1CFFCBA1C3A1 +%A19AA19AA0527D587776A1A1C984FFFFFFA8FFFFFFA8FFA8FD09FFC7C8CC +%C9CDC7CDC8FFA7CFA7A7CACFFFA7A7A6A7C9A7C9A7A7A7C9A7C9A7A6A7FD +%17FFCBC4A1C4A1A19A9B7C7D768377A17DA8CAFFFFFFA8A8A8FFFD05A8FF +%A8FD06FFC6C8A5FFC8C8A4CEA8A6A0A6A0A7A0CAA0A67AA6A0A67BA7A0A7 +%A7A6A0A6A0A7FD18FFA1C39BA19AA176A0527D587DA0A784FFFFFFA8A87D +%A87DFD05A87D7D7DA87DA8FFFFC7C6CDC8CDC7C6C8FFCFFFCFFD2BFFCBC4 +%A1C4A1A19BA1A07D77847DC9CEFFFFFFA8A8A8FFA8A8A87D7DA8A8A87DA8 +%7DA87DFFFFC6C6A4C6C6C6A4CEFD30FFA1C3A1A19BA19AA7597D7D7EA2CF +%FFFFFFFD05A87DA8A8FD0BFFCEC9CEC8CEC9CECFFD2FFFCAC4C3C3A1C4A1 +%A1A07E7DA8A8FFFFFFA8FFA8A8A8FFA8A8A8FFA8A8A8FFA8A8A8FD0CFFCF +%FD2EFFCB9BC4A1C3A1A19AA77DA1A8FFFFFFFD06A87DFFA8A87DA87DA87D +%A8A8A87DA8A8FD39FFCBC4C4C3CAC3C3A785A8FFFFFFFD04A87DA87DA8A8 +%7DA8FFA8A8A8FFA8FFA8A8A8FD3BFFCBA2C4A1C49AC984AFFFFFA8A8A8FF +%7DA87DA87DA87DFD0DFF845A845A845A84A8FD33FFCAC4A2C4CEFD04FFA8 +%A8FFA8FFFD07A8FFFD05A8FFA8FD04FF0D350D2F0D350D84FFA884AFA8A8 +%84A9A8AFA8A9A8AF84AFA8A984FFA8A9FD1EFFCAC49BFFFFFFFD05A8FF7D +%A8A8FFA8FD0FFF362F5AA8842F3684FF84A984AFA8AF84AFA8AF84AF84AF +%A9AF84A9A9A984FD24FFA8A8FF7DFFA8A8A8FD12FF2F5AFFFFFF840D84FD +%54FF365AFFFFFF843684FFA8AFA8AFA9AFA9AFAFA9A8FFAFAFFD45FF0D60 +%845A5A840D84FFFD0684A87EA98484A8AF7EFD45FF365AAF5A85843684FD +%34FFCAA8CAA1CAA8A8A8CAA8FD16FF2F365A845A360D84FFA1A8FD047DA8 +%A8FD047DA8FD047DA8FD21FFA1CA9AFD04A1A876A1A8FD15FF360D363535 +%0D3684FF7DFFA8A8A8FFA8FF7DFD04A8FFA8A87DFD23FFA1FFCAFFA8CACA +%FFA8FD16FF5A845A845A8459A9FD2EFFA8A1A8A8A1CAA1A1A8CAA8CAA1CA +%A1A8FD4DFFA1A1A1FFA1A1A1CAA1A1A1CAFFA1A1FD4CFFCACAA8CAA1FFA8 +%CAA8FFA8FFA1CAA8CAFD51FFA1FFA8FFA1FFA8CAA8CAA8FD16FFA1C3A1C3 +%A1C3A1CAFD33FF9A76A1A1CA76A1A8A1A1FD17FFFD0793A1FD33FFA8FFCA +%FFA1FFCACACAFFA8FD16FFFD0793C3FFFFA8FFA8FFA8FFA8FD4BFF9AA1CA +%CACAA8BDC3FF7DA8FD047D52A8FD4BFF93CAA1CAA1CA9AC3FFA8A8FFFD05 +%A8FD4BFFBCFD05CAC3A1FFCACACAFFCAFFFFFD04CAFFCACAFFCACAFFCACA +%CACBFD3DFF93CAA1CAA1CA9AC3FFC3A1CAA8C3A1CAA1CAA1CACACAA1FFA1 +%CAA1CAA8CAA1FD3DFF9A9ABC9ABD9A9AA1FD54FFFD04938C938CC3FD54FF +%BC9ABC9ABC9ABCA1FDFCFFFD6CFFAE84AFA8AE84AEAEFD54FF343B345F34 +%3A3484FFA8A8FFA8FFA8FD4DFF5F345F345F345F84FF7DA8FD047D52A8FD +%4BFF348383835F833484CBFD08A8FD0BFFAFFD3FFF5F835F345F5F5F83FF +%AEAFA8FFAFAFAFFFAEAEA8FFAFAFAFFFAEFFA8AFA8AFA8AFAEAFAEAEA8FD +%36FF3484835F59843484FFA883A8A8A884A883AE84A8A8AF83FF84AEA884 +%A8AEA8A883AE83A88384A8FD35FF5FA8FFFFFFAF8483FD54FF345F595F59 +%5F3484FF7DA87D7D7D527DA87D7DA87D7D7DA852A8FD42FF5F345F345F34 +%5F84FF7DFFFD05A8FFFD09A8FD42FF345F345F345F3484FDFCFFFD6CFFA8 +%FFA8FFA8FFA8FD55FF77717771777177A2FD54FF4D77777877714CA2FFA8 +%A8FFA8FFA8FFFFFFA8FFA8FFA8FFA8FFA8FD41FF78A2CBA2CBA278A2FF7D +%A87D7D7DA852A87D7D7DA87DA87DA87D7DA8FD40FF71A8A2A87EA877A2FF +%A8A8FFFD06A883FD07A87DFD41FF77A2A8A2A8A277A2FFFFFFCBFD17FFCB +%FD38FF4DA2A2A8A2A24DA2FFA2A8A8A8A97EA2A2A8A2A8A2A8A2A2A8CBA2 +%A9A2A2A2A8A2A2A2A87EA2A8FD35FF78A2CBA2CBA271A2FFCBFFA8FFFFFF +%A8FFA9CBA8FFCBFFA9FFCBFFCBFFFFFFA8FFFFCBCBFFCBFD36FF4D77777E +%77774DA2FD54FF77717771777177A2FD54FFA8A9A8CBA8A9A8FDFCFFFD6D +%FFA5C7C7C7A5C7C6CEFD54FFCCC6C7C6CCC6C6C8FD54FFA4CEFFFFCFCFC6 +%CEA9A8A8A87DA87D7DA8FD4BFFC6C8CFCECFC8C6CEFF7DA87D7D7DA852A8 +%FD4BFFC6CECACFCFCFA4CEFFCAA8FFFFFFA8FFCAFFA8FFA8FFA8FFCFFD43 +%FFC7C7CEC9CEC9C7CEFFA7CFA7C9A7CFCFCFA7C9A7CFA7CFA7A7A7FD42FF +%C6CEFFFFFFCFA4CEFFCAA8CFA8CFA7CFA8CFA8CFCACFA8FFA8CFFD42FFC7 +%C7CCC6CCC6C6C8FD54FF9EC6A4C69EC6A4CDFD54FFCECECEC9CECECECFFD +%FCFFFD6CFFFD077EA8FD54FF2953292F2953297EFD54FF54537E7E7E5353 +%7EFF7DA8A8FF7DA8A8FFA8FFFD04A87DFFA8A8A8FFFD06A87DA87DFD36FF +%297EFFFF84A8297EA8FD047D52FD047DA8A8527D52FD0D7D52A8FD36FF54 +%A9FF7EFFFF7E7EFD54FF29FFA8537EFF537EFFA9A8AFA8A9A8A9A8A9A8FF +%A8A9A8A9A8FFA8A9FD40FF537EFFA8A9A8547EFF7EA9A8A9A8A9A8FF7EA9 +%A9A97EFF7E7EA8A97EFD40FF2F5353847E532984FFFFA9FD07FFA8FFFFFF +%A8FFFFFFA8FD41FF532F54295329537EFD54FFFD067E7DA9FDFCFFFD6CFF +%A7CAC9CAA7CAC9CFFD54FFC29AC2A0C29AC2C9FD54FFA0C29AC29AC29ACA +%AFA8A8A87DA87DA8A8FD4BFFC3A0CACACAC3C3CAFF7DA87D7D7DA852A8FD +%4BFFA0CAFD04FFA0CAFFFFA8FD51FFC3C2FFCFFFCAC3C9FFCAFFA8FFFFFF +%CAFFCACFCAFFCAFFA8FFCAFFFFFFCACACAFFCAFFCAFFCFCAA8FFCACFA8FD +%30FF9AC2CAFFA8CAA1C9A8CAA8CAA7CAA8CAA7CAA8CFA1CAA8CAA1CAA7CF +%A8CAA1CAA8CAA8CAA8CAA7CAA7CFA1CAA7FD2FFFC3C2C3A0C3C9C9C9FD54 +%FF9AC29AC29AC29ACAFD54FFC3A0C9C3C3A0C3CAFD56FFCFFFFFFFCAFDFC +%FFFD11FFAFAEAFA8AFAEAFAEFD54FF345F345F345F3484FD54FF5F355F34 +%5F345F84FFA8FFA8FFA8FD4EFF345F345F5F843484A87D7D7D527D7D527D +%FD4BFF5F345FAEFF835F84FFA8FFA8FFA1FFA8FD4CFF3483A8FFFFAE3484 +%FFAFA8AFA8FFA8AFA8AEA8AEA8AEA8AFFD44FF5F83FF84845F5F84FF84AE +%A8AE84AEA8AEA8AE84AEA8AFAEFD44FF34345F5F343B3484FD0AFFAEFD49 +%FF5F355F345F355F83FD54FF345F345F345F3484FD53FFFF +%%EndData + +endstream endobj 130 0 obj <>stream +%AI12_CompressedDatax��}gw�H��<��?�`�"��dL���s 6 L�3w?����[ +(fv����VUwu���>rכ�tWh�A:D���8:�Nyn.L.��U��=�6|.2"P�t)�.��㧳�0N�(*D�o �{�������>��5�yx�.̂#A����b��§0 q�5��qsh ��0E�1� W� +7�ƿ��l�h@�t�F3�b��?3�_ W��#��dIӬ+NS������ �P���q���"���C4AS�8Aё%2DR4��1���ZN�,F�x^� +~6� +Ca:K��������7���?]�!��VS�sh{���@�`� 2°�nu��]�k�6]��C�=���WL�t���3���{��y����s���v�������{i���|�txŌzt�4D����L,�G"x��T��S��R����X ?�Ebq���v]�jY<�~h���$A�I��'���X =a�(!I��Q`41 W#�'`ވ(���$�(J�6�2 ���LŸP$F�(���f �����x,�9z��\Ը�j���/!�]��,��`ID�a",K3�f�88�.��HR��(=hyWq��ce� )I�b@ L�(���F�9'�(NG� ,��!滋%Z�Sh4��4IGi��EXAe�I )�*�E�)��=\�K@�1�04������ +� ቉���b�7��2"KS$��/b��aQo(B�}�hJ��W�h� F�� F�6F��:}���C�$ �4K��Ȣ� tP�0wbD�x F��ԫ�Л��MHߡ�24� tq�(7D��q�!I +c�s�?�s��`U *�UzG����s�K�T������H�Ec�Z��`4 �d$�$(*f���?� A-c��ƨP����w�)�$I�R,��(AĀ�l����l&g@;� �!>G��$#�?MFb ���o�_�τ�Z� +LO�M����O�UL��v<@:��������P@�V}�?R�)�hq�O~&D.���ŖX@�V��<2T\���� L�������A���h&�?� 2*a���&��%��1i"����(p, Jх�t���R�?���1=���1̑�u�(���d���M�AHdRD�,�g�?6���6�6�2��{q:�*�(J�b�܍PL�h��S�O��Q���F���D:`�9 Qf0��+��$BWM4 KV!��awM=�N�m�����XL��x�����œ�DP�q6�9�~�.����F�P�hs3>���ŧݶ� 4�`2�p�����?�>�LAxzC��������bƍ��f�� +��'x +����õ<�ED��<�;LS�D������ x:��.7���gN���>`3R�c�w�A��H<���S���M���e�tʏ;��6ܞ +���́F-,�w��o ��Wi<��Q* �t��X����'}~�bu +v�#���d�I +O��/���Oa�� �ix������,�|x�١��Y��u��`8���� �Ј�uCܣX ��YpS����=��p��1?���1��ʹH��r�i4�p:�J]����<���U��ߕ�V%EIզ�l�����!�����j�jb�j��F��|0��fC���nŏoU�.�z_������v �6 s"bN���QsK &LJ;29x�s^��+(~��@l5[ T(�6<�c,"���'A�� 5Y~���"1� �Bӟ�����9&�o������z#��� ̣�*Ң�ɏ;�w�w��8�x� }/Z����zN��jM|��D6Ѣ1���aJ�Xʮ'�]�>\���TXLJ㞰��W <���`ػ�Z� ��P�)�}� /�@9��e>�]�[��)����߃@K >͏�Ca�,�s�>���ԅ_,V����I���G0AsL�, +n��o�@�� �*�d:��?��7�0%{���� 6��9��:��Ӎ�ֹy�p�M�%�? +#��Ϝ�<ˁ�ٝ�Wf���]-A.��_bR�ïЗN�6���p0-Ѩ�ԁ<�ΐo�����o3��"}VG!5��e�z��b[~e� ^�/aG Q�������_�E�d�=@xM�e��(��m#��̽�� �n.Pp��Ma4Ai@W��Mx,9r˦2�c�� +�2���}m�r�"��@߃7�M����Rل��������:�bMw����)Ch`�C�BK +����R������w(U0� ��W������J���`��57�]x��&�M����?B��!@�c�PQ�ɉ�$!(�`(�����r���7��(E�TʿE�bJH������j���'����I��Cݐ����2������\ϯ���/b���¢/q���+ � ��h� F� F�G(2�Q$.M�1���H�b��Si4 �cX*NŢh�( G�A��0�f�q^����cO�e��4a堙M��1���(��D� 5cb�DOH� CP�'��Բ�v�����i��NEi& +|���1U����@�8 �V&V���<�f}�2x�w0�: �I���P��HbA�#h ��4�(�Ǣ, ���\<��gY:�ŷ_��'��� ��&I�} �xmbv�|��! Ų�$Έ�b{怢��#�c`���#�D�$��`$X���f�=s���12BF�� ��²�h��9�ı8�h��8 &�d��o���P�� +�A2 q&�ס@��q`�^�����G�>�` +R�~�G���Q�4q0J�q����5I"� f��T�Q���(Y}���g0q��1pz`������@�Q. �~t��Yqݑ����q:��64N��F���GI���MW �b�,�c�t�Z��g:GN|wt-hi���b-��&��/[4�6Q�R`�i���� �|�hQ4�ӵ翙6��[ ��M��|5v���k��4'�� TEs�aa��s��G���Ly�.��;�!����ˈ�n��� ��E�"�"P1~@�k�]�{Y����s��(O����R�,wB�����ڝLb� >2 ƕ���8�3Z*g�b#'�Ӣ���u�4YĎ���}��+���U'\��$q�t�7���X���E�C�;C���fE��6��t��+���v����dE@���4�#q�,�8-��1�F � �f�"b���f/��O��8�J�^��lo�����i��M2w�^��l_ �x=��Kb�\s���[u`�&��IB|)."P�G~��O�0H�����9x�G�a��џ�rc��o���U�Z��&�O]޿F�14r��t�^������8=��O��U�N0�N���V�+\R-_���h� ���g￸�����NѶ��a�)6F/f& ���K���̴�ϓ�=��I'��6��5��r�����֬뚁��7�u4H��.�b{B�քp6���#�p2����GJ���ͅ�ߨ��\O�%~9%j����,��q(ǂl((��O� t��֟��Dy����o�� b2:���q\�� +7xnh1�>���;�u��B}�Ǫˈl��;�;���_1��j)�q��L��-㋢��nL�����dL0GA�O[��sU�{s�����V?���Xq6�Ŵ��z���v�_я?��y��3�;sЕ�'�xN�?�e�O��.��N�A5�?/�����`���Gm<��5j��_��cn0�M�K�6�e�^��M��9r�� \� +�[�ƣ�)@�}� �m\�p7� ڃ�`��9����`� D%U���� ?���� L��`������\=�����:�������� Հ K� +mC����fʡB��/\n�js�����CS!)�Hј+F�Q��/ɐ���_s޶�)?㧿x����NP�N�w��S�h�!C�E�2�(k���pZ�DM�h�$Cq�2�D\�h��Y���zg�W��W�եq����w�qW݇�8F��@C�xdI�X���M���<;�B��^9�Ga0�-1��%�Rd�� �^N�Y�pZʺ�wK��}�⻃�ȥ�c�-��At��gAzH�2}�Kn]��0Ց0�]�, !d���+~�w5������U�dMh>�-����#��Y�_�.P�[]��j�����S�KcB}zm��� ���̭�;t�� f�!���C�V"��v0ʪ�)�^�=Ё�o� �� Rw��'/��9P�W�T=ݐ�!\�B[�h����IL(�hL�5)�B��0�����P�f��-p�n��-s�-����՘Ԑ�1����4Z���b����'S��@�"�ё0�P�!b!��G�(Ð1�\���8�aPd�qQ&�qt~*�`�u���L��h�E21$ L�O,’���[kf� ?";�LĄ�Zs�l�N�X�V����������X?���Ī����rrmA�~��E�~���q�X��M')�2��o�Q����](���M�T�pE�Y榝��3������8��`�h&�!���LQq*D�"�H���w N���{�I�bt�%�!X��z�DB8����$E�(X�L(kvZo9M��^�m������g�W��w0U��w$���(ńh��l�;�+o:R�l�M �~‚�_UW� ���@�n�6c`Swaq&��6a�Xl��u ��{Ɗ ��i�9Z9���+Ќ���lԊ��8Li�{S�p�������3��M�,k��̯�a�',����9�ג�`ъ:��&��ɍ�7p�r�W��8��r]~6�+�1�f� A���'\W�U=]r�sa��f ?q�]��h1�T�k��p X�t�.\�c�� l�����dl>�Ƴ '�@� ��K� +�`&�#�aJx\��tZTSh;���6�ԙd�>��bY�€��=��D]ޱ����q�j��Z�4)�B�����M@ ֹ)(/t:n~��ѹʽ��Wă4jz�t�FMŮjڢ�ᦨL�R��j +����"�i�\�A��m įJ7NQ��0��]Qt������f���\ .�):��͍��µ�R��>��R���3����כ�#?s�M%FM�%���.�, �T�|S�J����ەr�%�v�n E���昛�r*��9r�{�0��_����C��Y�鳄t%�@�3��&n���?sH=}�o��Ơ��d!Ƙ+H{%B_B[����Ra�. �Uڮ��J����+���T�M��F+ۧ�@�t3[*�"9M+~�\Ԏ��g��S�!P��k�LjZ��cw��x=�����y� ��-&���D��s��.:�B���� s@��W�3@�N�B'���d��U�0�I�V�S�Uy����������';�P�󹂫�}��\�(}*�s_��)H��� �i��'كE!��g���{@���!��h���y~I���;s��v����w�5���F�\�hQ��=@��U�x�-r���hf�>$z�>8�'RC���\�����.<�" ��,���&�o�?�R���c_��f�`,v��.M�����;�o�ϼ�����c�:��Oe=��,�8Iޕ�����~��j_DMa.:~$o�nfxt� N�/�t�y���K���� + ���5�w<#���i8:z9D��Y�NO;%��� � ;���(���{��&�0�L^�{�>3��G��G�l���>�G�)#��3{��f�7��]�9A�^1�����dOд<�����*����A�A�U�|=�ʅ�����4EhX�� C�M ��/1��p&�vv���������z�J���?�����������u��gs�c�W�Dd4�*S^v�U��=o�V<��Eܙ�;&w��]0���I�r_�\/\��s��0öoo�����t-���z��O��5��<�0��w��Ol�)�N��2�C�V�}ݿg_l��7:�,d{'$�0��6���0��Uҵ�I���u+"udZ�"�X`> ��/�qHI6ƥN[��t�<_�NG\%��x��ePM��Z������'�qv�# +/)����Y!�@T�3���=�\��-n�t<���ı8ejA ?�]Բ�h����s��xGy�����+?��᷇$<�R�g�g�Ш�rҗ�gV�� ~����G��2�mR�v ��+'6�O$4��Vj@�(�������w7'uG�����hđb��[�LS|&A���h��z*-;��Z=�����<_�>/�kg���~<������o)��g7*"�$}�\�P�� +q�l�ﺓ��  �.��Jퟓ2�4�)�L���^��h�L ��� Rz��������'wU=.�_V�_`4F����!SD5j�ؐ��AT��h�M?p�HB����JZ�?'5X<�CR�eʴS+���1�DF�L��5�b\_�ܠ�ۖi0����8m9��G�<~f7bF<-ҀE�a�F�}lM�  D��E���[w�q!��[�(�����`=?�����D�}Bf��ۉ�����}��n?���wJd�q<*����F側� �Y�[pG#m��EFj���)W��|�"#4$?� P��`��Lc���ԝǛ�u��W@���M���ZHGR��ܥw|��<6ǚ�� ;��l�΂��\��ă{��U1�؇֙�L�3ǟ�x��X�»���Pzn���%ߋ�mϴ��؟<8���NlEf�FF�!�����ť4:�N�%�˗7.�C�Q�.�I��l�"Hᘈ�x ˞,C�d�Eq>�T +(��֪l�p� -c��u��4�B����W��]��Y3<�?������ij��:�tK�ъ�v{�-���5�r�A߅�%7􏐮�?���i�2Wh +bYiw ����ģ���\!��rސf���<�ȁH �\�9T]-f���F(��D� +zup��z��l��z��:�;(]�&aY�Ŀ��s�뼁��.��ӓ:j@$��!A��d+1+�jES�����d��^�@�5�i�b� 8�|J�n�qQP���s��3����b��ahzI�ٸ6�,����-�����PN��� �h��ų%)��x��d.3�"C����2$~ �}~K�O+����� �u����S x��N�� +j��3��k�(m��L}.�z�>��_� ����5�C������z����[�:w+/�!��B*ǻo�3C>Cf�g(��M�8���c<#�i�Px��=��M �}��+�;Ŧ�%]�P���:|�xI#�4� +ʜ��7iRn*I��C1� b���f/��aA�9cuO@���G<���2k�Ԑ&���Pe�� ���,H<{�� +W� x|=��$Fb��ӝ�����\z�X{�)��Z ���^�Iz^�Έ�s�l��'�\;$g9� ���_��0�ϲL?P�/�t+[k�B�n0���(&L�>1�z|�xHG_����p�H���ų�2�8���[8��X��oxG�kIM3����JE8�~|g�1�Z������)��@��_B��ݏ5O'O�v���O@���<�ۭ��{� �s�u|�,�7 +��������<��9����%��G2�����!��Uoa�n��[�'C��f��Aw��N��^�R�lUb§�J�a��� BY.��;Ǒ\��n��G_�Ԭ\���e��K���!+�c"��Lz�V7���]�6�fz�v%P��kԇ�Z:r�ZҺ����.�V�n�7��G*Y�ȸ7W��� �l�xT|yO����6{%�¾�P7¦� ι�t���Q��R����VO�ї�l*q�3���4�n��ƣ"�?���}J�/O~�/pV��T,��B�%zG�[�w!B|���I.�_������!h�x�5�-�M�s��配��p2�σ��`� W�h����G���<-j�0��L>��U �J1� +�,DV���ܘ �ܛ"������"}����]A���� ��`�TB^Q83��go�����p�2Wj3O��c�?�" i=�)�lOz�ܐ��9�Q�U����?��D-���՜�2�<�n���U3������ E�' P�������AOq��nk� F����_N��2+H�� �bA �|ބ�x�L�{lr�}�q�P�i�h4/�8��c�v-$�$J,��G9>�C�C\{�vO? �.,t�gSIJ�Df� (�ҵƬ�l|���8�!��BH T�t� �a(?�|}�S�R�f�+{��J��V�+��朓����Q�ؽĮ��׷VZ l㬴J����A7uv�`n�;�lgD�qy1�^��r�SM�����o�E����A;`c�c)qD|?�� Z{r��+����0֢y47;�&��M�*Ꙍ��i�$ �f<!N�W��Xq6� 4Z���8տ�]�ϰg�(p��E�1�>4��:��߀y���+֊JNBh7 ���;`�X�Jɔs��!�@���ɟ�r���s'�p'q姟A<������jj��^4� �O�����q Po��C̐���J�T*������t�˂�K��>��o`�yJ����<�� ;J����y��,�;� ]��/�-yP����� ��|�:�'��h� �Tږ�� �]�U8��إW�ʜv�'�OU�k?� ���e��~RY�91���O���PHDo�i�D*��=��V�)ֺM7����/N�v��Q9A3i�7�y�[����z7 RxЦ^Y�� +(�{��v��c�+�`=Y.5��\�^�B]�X���^*��\!�|2�V����!�afc6mM�R��Kc��g�|g�>Ff��&AA�7/`G3�t�rr�6� ��"�@�_Wt���'RQ�Z�q��hZ�t��'z��/�ޤ�d���5.���Y`�k�-�7z?h=b�k|T:f�d����� 2MO��h��� +�Fy��H�ɰ�NH=�(�FGo�#�m��g3��h�=�}D�=x�z�� ����ᡐ}= @wn�0����������#��t���;��9Y�~.|w�7�'o��m����}G;N���a���0��Y��[�����|�m��'�ng�2�� y��&�ۊ'c9�E��Tw��h*oNOW��L��P�x[}~ƻ�)��yWW� s��9�ya���Ϗ]3m�tqc��Z{ګ��f豲S�Yr��0H���[X�&C�H������Ep��`�h\�l���� D�w �1�r���xn2��L� �� �+��9V����iQ ��S�f���]QZݗ��'�� �;�����,#�����o�B.S��:;h[���sЋ,�?��6�@��$]�e6��)!tp�s!ږo ��9��]n*��>�YPu{��d��ux�>T;iŠFW�X���ˆ�3����n=�R$R]���y�ٳ�D,1,ˌ�cHN�w���x�*�ϥD����T.��ˆd0�ce�<�P� _��ӓ�p�;�T�7 NU�Z�X�Z�n�Z�S�~k47a���B;�W�C,�נj�dl�UkV� �@i��ĵ�*>�:Sz&,%��2�f#f*���������E��Nd1ݙ +mn.�/� +Ƭ�W�F*�)b?��Zu ҎXWc�י߬0��j�R�y��S�iu�����RmE�m%�x���ih��!��Dq���i�z���dN[0Ԙ�%��Z��ҊydO��ߚi�"��J��k!��Iq��t��o՗뉕C�E���/�Z3N�8�r�M����� �Jm+��YH�Y�U��S���$��䨥�Qu�~B �)�T)^���hL:�KZ�zJ�#.{t�X�36����^x%*���I])��7�"bUا�!�Z�OUЊ +k�[�ԻJq�XP��F!���t�ea�Rz+�Y�Nt@3�0�BK�g�� �jīiī��r hY��dIKb��5u��De{�p�`hDžZ� LKϚ�����O�&��/�j�#��JE�ڛ|��Cm� �^���~�C��j9�Y�V���9V���3 +��� �[9#����o��y6����DT����N�`���L��Y���G(���L�nM��*�p3Y�Ϭt��r1�l� �5 �5 o����6TgQ�4@��| �:{f�X�̬����\�+�-��j �U�X��������5P�������5v��駑���ȴ���*i��v3tP藕�@̈́�g�o-�#�kE` -�$e�9���R&C�m$ɖMZ���&�x��� ���Ѩ�nj> �:4TA��B,C$�X��c�N7F9_JM4�x�S%�N>���CKc��M@�4�α]����۠�B�z�7��n=W_e�����pms�n9��vվ�Z���6���x�m|��d���(���`(��� �٢=���i��e��†���L��ONuG�)80pc�FC��sj�` +f�Y���Su>°�il"������C�]+3�����PNY>�>o���5�W)�e�8����O+Ѫڍ8]�M��3i��Ku��j��"Ob{\��32r������8SeCyԁ�MS|3�2'e.c̠�����C�ԟ�uc�=��'�\��NPYh�-#.�S&5�U:L�a�m�l���J���`^QiY�Ϗ�˃�9�[����Q��{�l����iV�d�#꿸k����*���i�Ŝ�$�勛�o��)}њgr�x���q��z�cr���7ؾ�G_���{�и��`� {����w�}ēU��*�Ta��5����C"|��Ý����fFͮ���o����31:�>�����GA��𕷀��|��i��w_����M: +��}v�!|�~��/ًl�u�<���|^���kM0�;��Z��^�ŝ���P-a�,�|��ΦSjƹ�O޲�_�I��]"�6/�Ш 4��;�yd�ༀ�.$�ŷ[���)�O?D^�N�(�JN|g�@�3=�#�^!��ܾ��w�%�B�h"�ܾ�G�8 +�ո�@�}�����X�" �*q�52�o� L�Px��ߚ��&@�Y�0nQ���!�=&�x�F����3�c �<�7V��Oz�gq�?�ԃ;�=M_O�5"�4���j�E��y�W +b���̥)V�� +�Ͱ��oa���u��D|���N�U#���76c��2y�ce|hnN;�+����S�D!SI�`e�S߹��Q#~|# +��1֢��,�� �/��)� ��l}`��I����~��HVV���z��!����2�UB�p݃6Z�G"������k�2����h�4� +h�o �nji��1�3����7���d}ɟ�Qc�j��:L$M��?���1V��(T��FX 0�(~�\�n��K�A���U�/�a����Ɗ9M7��{��;|��щ^x<�wZ����F~�� k�廠`Es3e��D���6����2� �+Q�xS����[c4:��Q֚�X+��J($L �6��V@�W�L�1.`}f�V��3���7���!�;b>�X�!Xn���Ͱ���7O�k5��/tX��>xwlJ������ �x +��c�.�_�Ή�F�}m�_�b�N =e�/��[��a�]��~Ϳ?9�4��A?xM��?���������i��X!��O�J��cg������9���~�OtXޣ�)�Jy��K����Oݟ�y�5�*���Qmp����~��� !a�_�uJ�8|o�^' ~�'�Z]qC���5��:]|�=�7�X �R<�E�d%���h$���rK���m����c7��wVbe�w���C�����z +O�yt�l��#]7/���ҝR���?y. ��� Y?��~N�Ϻ�ms�RWS����;�@<��ֻQ�m��c���=��x��TD[mP[|��mޡ;�L�����*шVC��~>�(��Wu�m�G~��c@����¨����o�����~NE��O�g����oo��훗yP����7�L~2i��� rVD#�C�[��3q�1{��󷵟ĩ�������؟��oߧ���W�o�m{�J47E�FJ��Q՜���FA�VB\����X��ǚ��jl�]��r�d6.���r3w�o�bA��~+�S� 8�d /�.؇��`x R��y�|���p��(��>�9��`�e������%�Ϗ��_����8��Ό��qH��X�Y+'W*"��B<m�^M�>��b�yB�}5b�x�tk��Sj�>e�š�ۧ��4��U��#�j�����=�c�[i �H��xce|� +V��8� � �bš� V��FXEW���t�q��XQha�� pR8�/��c�k�rg�9)�y=DoC*M����Qӕvt�pЎ�8I���Ƭ)v�Ձ�O�*���`�N2~y��E />����\%DZ�L�Տl�$�L ܈�$x�P� +���3��Z���Ci��<7:5]��ß�H�'2ѹ�0��S?��M�)%'� Q�!�F��ɉ�ے�q9若���~�&��'�t&��sG����:'J�R�~�1<�H��s������d�H��l�T�)}]u}���-@?��I%m^�›��Q~`��=䔜�.-�GH�V�v#D?��P �FϾ�4�HP��!���O���e�~u#$�/���0�$�im�}�5�c�� d�uV����H/������%ArjȐ^�l�ySzi����I�hQ+"�% ��B��)n"�0�e�54$�z�XI�kɗG{ɢ;� ����ZC{M����^���7O1�g(��A�xn4���͕���F�|�X��(=Yj&�q+� �<�$���_r�퇄�E5�C����1��0�*�G�?��0�ެ���pg��yЏ����Q�I�7&ݲ��j���:�C@c��YF�9��Ρ�;�F�^w��9BQ�+� =���k *�b���;%�S�Q�&�%�$K-�[`5���$�.�6k�z�T�k�AP�����0�p;|����r%`�\<�����N�aC͏�JwT��oM ���o�5���h���=�~O�y��Q�iO��qn�E�u��D��/u�^/t���E�u�sޘ_�,`?�Ճc ��)on +���Uu�u�9�K��0��?�JNp�X ������;�F}�v� +�{0Tūy'��u�=^k��y��}q׺r��ͩ޷7�z��z��1���%C�������a�̇�� �53��b�Y�m�՞�a=W�v�<� ;}�,�ø��4�ݍ�Zaw�t�XO�0�=��Ә��w�M�)����QY��;ܪ!��1^{r[&BFe���p��$0�Bv� �Ĺ��ڶ �De�Ƭ3k�Ѧ��>9HV�t�Iz�w��(MJ�Um�ë����=m�L�l$7���-YkR�H�AO�;��4&nI��[�[k��X ��m�ݴ,��P ��Qv��]��b@Z��Y��Q�����I�TM���+�ƾ�D-2>����pg��kӥ����:��hl�Y4�6j�Z�j�@�6�� :��*WWqk��ˋ��<��'O�gj�p'����]Ʃ!{@/�� 1������T��^v��,���Ϫ�� ���:ױ����; +��S�-���Wo�p�iN�#@k���:sk�n�N�����AY5��5d��;6k��޿�l ��}���C��Q�5��> �ッ��b���>����}Y���/H����K�����H�Yn# X�n�Hڱ�����S�����o�컹�5 �aw��Oqw� ���IxC9o�v=�1�� �)]��٪�j϶� �v��/�l��vL�6ԥTm�L�/���� �̖���� �FV������i�daA���P��N_y���KF�� [�� q�EB��XɋC�Q���ҽh��=��Z>@��r>�Z>1��r>�Z>4����`mX�.nV�g]˷�*�۪�ϺݞT'�u9�u-�����r>�t���YD��|bg��=m%�:��!4E�n9�SV���}o^�8[ks���]�nUmv�:t��s���Ƥj��cS��8"��5��خ������^�؆�4���m�-��Gh��P�l��ݽX���II?@�l7~�O��.�n���:�{�(�|�)���� �A�[�n��.�Ǡ�a��>� �{tP�t,��y��ε_ ����7\nXvg���I��ݭ��2��} L�n1���y��G�F���9 �ՠ4�ഥ'��W� ݝj����62� +Jѫ�l6�{]��8(������ba�.�&M�kdVs�߱�64�E/׃ynԨ�5r_lo˚B� ����m�o{��$ڛnQ��Ҽ�ѾOV;�tVU��z��9ZlK(곞[�!ޜ�Z��!4ӃԾ�ЬKtVA�/�4���kͦhg�a�m3،h6�5�M��9O���w2o�5����j�<�p*��2�Z�g����͡'�A���?� 13?�r�4������4��]�K{Eqb���4:��l�mJ��*Vlg)9��a�8:�Z��i;��lB�j=�{��G��d'�'�Ϊ�^J�����WI�aڅ��F�|8e�� +�$w0l�\^x�6~3?�u~��ց�t� c?���~�S]-��)�r��rAf��iez�&ĢO��e�S�\/{c]�^�ƾ<�q�KC*m��>���ٛ��)#�a��� }��w���7*�({t��,�a�������9ޜ4*o���T�h�]=�q�m�z���7�5i\+�m�&�l�}�Py�� ##�+�M����؋�#���zGzoy�J}���QQ�?�`�!���cϖ�F�'n�� Nb��6�26����qT�iE�:y_q�G�Q������("�8*`1�uj ݢ�O��+{��5٦�v@g�O��SY�Mv�;��C�$������Ss��8o�=冕yl�ݤ2�p3,���<뽃;��۳-]�Ie�誯���2'!W��� +���hR��ܢ�V��ڕy�0ݝ��ʼ�xj��W�9Z@���R[pH%PV�KҶk�����h��@�9�zi[x�8�֋^��R����6\ڔ�c@k�a +���箈ݱ����Etm��S����'�m���~�`aԣ��;�G�.�=�z��w���+�1�ǻ��B�V<����qbl��#�p*�:({������h��tw�=�u"nWZ��7�2�� ����Qvb�w�*ͪ?{6uk�v�_~vS$K����V�,��(�tR$K�3;)���yU?�E��;)���U�����d�wU${���k� XVw�����P:y48�U���{����u��l��vU�'e�Tuy���Ey+D�{��쉶��<1�T���C۠(O��Ρ��Yo�ذ(o�cT��V_uM���;�9˔����%~�ރj ��-m��6̲긒�ə�����V��e�}� +���aT i����V�l��2?�ñ�P�ݣ����WKi�;�?���ͻ;p��w���;X���u�Y�ۣx<�H�k���@��E?Е��C��c]���4�h�y���P[%��S��q���> +ح��BA5�h��Ϩ������ټ���ɪ��˪دw�4� +h�e��b���o�� +�� +=t;�+�ot��?���p�ew��X����~D!N\+���6FԛY�e�ߌѮ����w�Xѥ�}��Ɛu�_��1�tv�t�bu�^�-ͼ�q?�M� @��>uG�����m;���-�Pӵ�M�H_�>ɘ�§��_��G��xWOg,,�:m�un���j�[���s�h�Ԟ����w�֎.g�������zF���� m�\qF*�O�`G��Mq'���n���n�3%ƞ�7�9��}2��mA��m7j��d��Bݲ�y�y�����ͩ�-2{o�ݢ�Ϩ�x4�-�3��S��wT�g�k|��� N�����>����9+�ܺ�Ϩ�o�t�ua�Ql��f��}ۮwYا�M��O�lvR�g���Ya�QU��*�v�}����I�첰O�3{Y�g��w��>���=}����}FzV'��(�3��ӡ�Ea��׿\#�]a��J �]�9�lk�},oR�g�u��Y܂h�o@�� ��w ���Ϩ�O���Aa��=�63���8���>���380c��>��л,�3���Cﰰ�\ 촰ψ{�cs�/�3ZDS)�� +��}����Z� F�^g�6F,Y�9�'��ꨦo�^C���J])�7o+��)0��M���(���?�{������x�ƛ��T�g�4>��ދ��'È��?�}��(`�!��W�?�͢[z�^�O*e��Ue^�֟ڐ�|���P]���������ζZm}��z����k�zL\��Q�c��� �f���MJ�4��m�z����̏1(o��*����:���^���T������?�g�U� ������V8˞��QÂ�=}������v�=���čo5��|���0���5�m����m����EvPW���hr���&5T��eI�&ݍ�g�1�݆#X>i_gskĬ�˭Vm�[���������;*��9)9Fhl�l���b�����u�0��Y�������f�Սc�M��:վ>���C[�����j���Y�{:p�P�l���f��'����Tk��[!o�WQ��6���KX/�' u�m�N�I}�~�FY�%��oGw�:��q����2;-�r�󆅘V%{�oZ��p�x���]c���x��[����3f�-͍i�I��0 �7Y,���@u�Yk:,�x�������޿�?������:M m� ����mꪯy���>��m��Nf�]�����^ �ݽjecx��shV��)�:M��۠ڗ�o���D�ݯ��Xp����^�a��W���X�r�D�� �}�[��T�"(�X���!����2R�h��{ݤ��\���}�ZHϐ0��t$�0�g�cp��:��Jx7�+� �F���(�[��V/�f���:�Dcw{„�9�dM�؃�����ݢ>9a �2�e�m��1����o�0�U�7�Nc{��e�w���2���������V���T��?��f�-g����.�l�t @�uO�u��>��a:99 ���e�8]��l�^������R)�I����İ��w[�V�E�ե:��{�?|G%Ct.|�&碴\^�ʼX=L�I����s(Lt;�4W�yۇ�o��<��5|o�U�{�EhD!���&W[#7�O����,��YC �(��K����~�ŋY��ae��LAD�I�oX����O��f����I��dbV(V!*�.}I�'a��(�^6L��G�Ỷ+�ib΢�H$L�Ng��S���V��'r����t�c($�A��&��.��,`ܴv�;�_;w��� ʂRi���:����� ��:cj_��7���4ܠ)IU�x�[NZI�ns����n$]��jD�r8����6��fm��"�9�÷�_LS?����A������q3%��p���F�q����Q�ߐ� @��f�b;�=`F[-�ڰO{+e���9��3���V�5�`C��c���,7m�>�E�kgn�:]m��:^ |�;]W1�Y����zp�>���Few�i0g�4T(�$��;�;��"T�h>(�T���aݚ>f���m`N/2_����$�)�Ia7]��Ů"�����idT�g~ޱ���:�^�<���yI��߿ܚ� +ػE�{u����7�y��jW�>�iU�&gn-��y��*�ֺTŢO�Q����ںD��wr۠������jK�ʶ�i���U��x���hCPj�C ��"�5:���4�Dk@�Y]�h6+��t��[m����m��G��6-I\���J��#����J��#:�C��$:-�3�:7)���G�S]�AI��zD�ܼ$�i=��~�QI��z�5#�M/4���(IT�+�;���tR��/+�ϳ�>Sa�� + �v���p5m��\V�x��e����N.+�¨�c�� +u{9��� +Mk^��3�nx֜HM���tzNe�����b�}4�2nO�(�O�捛J4����DƟ<��������_Oh�ϧ�B��'��E��F3�D���C\��BD��!������%�<�>qW%��I����E;G�־+��<�@���c��qD|������g:K��әp\�.����$� ���B*5��I�㾺�QQ/���wO���x.S��5��|4��V��� +uO��g�˺D�%��F��_0-�*� �����Qmp�����P׻�Ct��?dn�;2 �M���?; & �)>&>��7�c��v$7D��H��}�(��,v����rՖ�<ķu����Wx:{Nѳ�7/J�"F��ɑ&�&�R��W���䵰@��>������� ���~���)�s�����рxIt� �l4�:����������H��ҕ��7.p�(x/K%�QG/�w�sD�1�ܐC�l (�8.�i��� = K*9r�Y��E�Q��+ty�ˑ� �?ҟ�P�Z~���.Qw^苻 �uf\�$�Iƻ��+��z�ĉ�̿%������]5)�(�T/^o)�*���]�^>4V�u1�WȧF],��׆g��r�C�X#Љv~�0>�ß�~*f�`Գ~�$?�c����B,��g�n�/�?��$��_g]��A�K_��!q}�!��c O(�ɒ���Z��31:vʿ�z�>[�e�����-�����M�ʆ@6�NA����yK( ���D6}{|���G+�t+=��REe�O�D6�u�Ӫ{ +L֢�L���06k���2�hh�!D�*$�q�wA��y"��y�/��?C���o��?#Xn� �h�oC(IB���<�O{�߸��4���N0�^�/>�B��B�Y��M_�P�����>sa��8�9����̗N |#%��ǚ�2�z����su�\�:���F.��Ǫ��?i���<��GJwا��0�;��RgL��@����F@��x1��ǃ��|��'n�uj,��5�NA(�f)��]����r��;r�s?��{s�ɉ�۞��"�`�s��o�}���-��zQv~&5��)�.h�K��������0���WV�WcIQ��Qn�������#@_��9��(x�:��k�A�:8L��~"9�ltĎ<ˆ��� &���O��#��Q^�&R"Vx �oAk�s_B:��� + +���;M�~����]$���\�L ] hD�|�f���<,�e���� =�bu?$�44��+��Q`Ք��3R�����2iW�?wM`��.\�U�8Kd��(h���d�� +�/*�����?a��ϳr*<�\��xU�-���z.]E� �==x��J�'3C]�����/��7������`�~,����#CO��N�����M���p��6'v�M�f(�e���<�ҷ1�<�AV1(���N�F.g鉐C��&j�b6 � +����Y@�(d5��W�pp��+-z�!���$�T����������yj�)_�0MTr'%��S:�qQ�>ήk� _8By?Y���e�;�ٗ��H���j~��Y�$!��� +@K����/#�n�0� ��3�w2@>q'�v��HT��4�䬊2����)�/�NQy:SA�W�N�T�/��B� �l��%��1 I�w��Nh�� hح�.��ftʠS'�J)�0:Sw����l�A�X4�\�- �a���[|�R�����f�}H�x�ȠN0�r��4��܈H�h�70s����˹� �X�Y/�O��lw!*�mr�����6��z��uV�g?8�N�i�����l8s��Y0�# + ��umz,誮���5T��F�+ɓL7���џ��@�"��I}�1 ��s~�,�L'I��Ù�ʼ�Cv�a�A� �&���d����ر��� ��Z��F��4=��Kd����i~ �����nU��D�H�4�)��d��:c\F�tMk���?<~]�g��!�������k����� +@SG�f$%a`$�9�)��a58;�ȎœIS�χ (wn��v6���W�~W�MW���4с�JV]��O�Zl)�T�����h��� +Y��$pc�7���S^b�M��!�>Jy�����(m{D�c1��Sw��R�}�r�?�Z���Ϫ�V�9��nf �`��_=9�{4�M��O�;�^��i��ƙZ=�Z�E"���7Wzf���»�:�(�� |?�^�����d�����,�����I����� 5>������ %��nj@R�N�B���������Бx�%Z]fq��� +�( ���<�����cr#Ԡ��9��չN��&�;�$�l���>:����K܈�3�]$n��TU�-����I�a����p���$�sr��`���{�E���rޝ^lg��3 !�&���}�{%�wһEEł��ԣ +�[i*(]�w U ���7egfg7�-�9?烛-�ͼ������闞��W��T �j�߫1*%.���I�]j��H욤��KEs_s�[����fp�� +\���6��G�n�zzΣ ]n�+B�� |t�*$KM;u��ȡ���G� +�� x�k��b��vN i��8�b�Ȟ��� ׁ��Ij'�+tBO�n���y�n{s8ZE����u��B���,#�}��df���f�9B�降y`Xn�(�=�y��d�Q!��(K�4�S#Ɔj�����9��l./�ţ�1ٍ�u�cyQ�TX3�s}�+FKQ���+���L����a�g�ij�]i9�X +W-��E7V��^ ����K���Dyz�?�!!� }�>�ޭ�H�VejxFHjvCj@oo +͋ K WE���HÛ�}�����#k�3s�+9��\���W���ㄴW���# +\��K"��o\J�oxYeH��srxTR�2�Q#ׄE�V6G��(C��/��Jj���� �@G΃ �!44�ک.7L��Ԍ b �(c��� 5�W�X��\��"#�a�Ս�%�+G�0,TQ�|sDJܝ +�0�q�*��ލs�nyX3F���sf��i* wm� j����� +{T�oh�ߐ�[@�ٳ�w��� �F��� +��e�/�$Ǝkt��c�2t�7j8�*��"S�G�4�E��$9!",��#F#�OS�����)L.���ޔW���L��S������e-�� +(M������rx�xcN�DP��mc.ڊ� C�s���.Ԣ�Qzo B��y��y``6nun�5Z� �ܜ�F +�KE��a���g��d2� ���~1�(�؈�6Y�Bxc4���mpz���fz���ЂޔM��[#���l�����t��J��g&>5�ӄϴ��������L�d�f�����!&��y�I�}�%0���~ ğ���[nx*�"�!�`,�@�2@Rџ8��?���(�����BBڃn,m��y����E� c��:=�|�1�u�4M� �G�n �-�����L0_տ ˭�.Z�6˞&|�S +�>I�B^I�*n��U-�:��=���t{��Z�& ͋�N�?˓��2|� >*l����j��U� �*��%%��-C�3��k" +�k\"45EU�,�ZV�_i�v��t��$���9��.Gb����tSe�a�(�^蚳�2��W:������+��qC�Um�]�����%�T��=�)����>i��>�pʘb���S�Xy��8��>�zbn�9�x� ׷5gO=��7�x걳ت�O�%g���U� ��a0 �4�:�3��ɒ?se�m��Ϭ ˉH��J%�Wi�[Qf�!�������L���\9ųp�A��6�s�2v�I͌T��X�.c��Cvs�VŰ��I����6�u�he���- +����#s� *X��(�1��" ]��r�m�J�����D�f)�eٜ�E���߮�<��0@E� |�u.L�{$|��q���k��,AC��K��զл;��*�{45�xnB$��T�t�M�̾* oO�X��͒��I��rj +Ӆ4�Wb5Y�@ժ��/W���x�gO�9�6D�p�8�4���q�Uj�A�]�O���u�w�{#=��APU$�A �I6s����Ok�� �'hA�S���.uB�4�c��C��q@� �n�‰Ah +���,�E��8�R��N;v��9��b�4�P�`j s q�wAl#�Nۆ�Р鴁K� +oC�\Ó�-�s#tgD�6o�4eރ7��o�4����ě�� 0���K��No� h#)���>7�� WDV�9m@� f3��= K�H*&�{j#�Pe�=N����6�B�Ym�7uiQ T�j�ݦ�Qo��H��:m4�Smd�j�PD�g��D�\z�d1nV��9.$0������se����A3�FY�y�=��v���������J�i����i�c8VL|�V{�7�]i����ӧ��T5�jo��)�*���bƺ‴��(.k4�&��@�>�������9˂� �Hq]�&��`�.nl5��!� �g@�c���#n�9��ʣX�L��<)���妭��Ψ��JF +W�"�҄�V�9��هj��^P��>m�Rq�bgF��V��V� � .�A*o�"&2:��r�l�"��)��]z��zwZ5�pi+�l�9��(�Wf�G+��RTJ�Мl��5����� �������3��;�@*p�8�6g�����8����p� N�N�1na1mU1����0�x�2|Z��Z��"q�%�n��� +,�r� ��Jv70[��d�`*1�4�~2�e�&�o��G��Hw`��K����8�kmj+�C�r�4I���N�)�&4i�lx�bU���&NA�L)hMQ�� +��` ���y���djS�t�a��A\�~� �]�YD�meA�ɧ����fe�U�3�F�h'?��8>�E7Ȭ����IUa}$.���Z +II�'�`����OTV+*���*��J�Ԏ~9J��nq\����NM2b2��K�r���a) s�?�(��w5��=��{b��Y�x�1ju� ?�p��k��"�(b�Sd�re�/ڞ���H�����(wN��ʒ�4w��� +���g\�%) +T��! xT�J%I� C%bQ���Z����D{"O!�����=���ǪC+S�Y��꾭́�p��f��r��T;�ҼU�`�EYq��,T롤�=��� +tA���ʕR��̊��.`� ��@O���f��֢ȸ� ��l�xgv�ge�g�o��0ߣ��+)���6ǕQ J#&쪴�����Z:{PQB�I�} �c�J}��� n-�*̭!ZY\Q��l���tq8t��+�o�Y��ՌJW=š�0<9��9����ZJ#/�y��w�#�)K����Y�;u��xȜ������������B���\�#�tR;���%����L��D&�[%S@�L��)�j^�(� +��e!O��x'�xGFh�Csp�'�r@TSS`@qX�kj?�3���>��p��L>p"}k�i.>�S҃Q��H Hb���e�@э�&�C-1�̒F:�@� ��� �ʨ`' �&�va|=�Z)F��So)2T�U�fO�t� +�+T�V�;�!!��K+$3� ���Y"�2�Z�F�~��hh`��j�uj+A��1���e`�����*4��Jm�T�eDF�жp!Kjf%�X�I�����l���).u ˎ�����lQ���{��}��U+����A�ݕ%�.Q�� 0����i��� Z%�������7��'*�f���]�� _��z$�zR�4�C���Gkf5.����#X+�2����Q��g��ε�T��|�,�@0Q� �c�� kg��G1O4ޅh�-*|�g�˅�e �~�Jvi�-�#�g��G�rUF*CԚ�<_bG���(#��$��R�S囜_%�]�_%�M�]���hQ�$8�bID�~F �W�ST }��22?�_Y���-�W&�!QJ +7!���@�I��/�!���U�s�����'_�2 �������m�t�Hc9�3#СO�JԌ���8/Q%qk��r*�������8��|Q��0/2C��jUATC(|�J�*����/��}0:��P�\�eR=� ���H�*�Ol��"���ͨ _i�7� eJY<��ډ9�v#lm��"�bP�Q�!S�c�C�X Z�R�Ȫ��<�\� Rrm,> F.� �� R��ݼX�T�U�R[ِ%����d(CQ:6��#{dѓKU$Taj �E� - 9�Y���n���� y�'� 90~�qD�)�#47RW��/G���fx)ǧ��ʝڈ'�cS}��bdC��H"QG%��/i�S^s6��Ts>:A��)�� V�D%’��U�$E���p�BJ���p�_���啁��\�����Y� +�q�B*��~Tp� +��{����DU��"��U;��k6:�q�� ���\���ӂ��E��re��򖐸���D���:� y�Qj��:�7Ԉj�^��:�R#��D�mLAEo�~�(V�g*䝻�:k'C +x ��" + +(>��)�¾i��u� Y/��T(Z�HaU,@˪�|���"絸�]A�T��� f�[R]��eS�WK�'OmRdG +hҍ��1�ڐ������F��$* +S:�8Zά��o5��<*���2"�"��4s�C�f]3'ݍ3�Rާ��5ͯ��W_O�S�Q�C?;�0�;����c`k����e�-&�c��%���Q�u_��L��>�� Z.����8-�q"b��G�i��TM]�&= �:PT��l�sqQ�=���\�H.w�p�P����B�w�<���w�+��/�����nCV"�H�=��Rn�a���/�;��'A �J#�����ń���:*�*��/�$�%�WXbT�tI�cK��9��Q1D��Ȇ�,,����2�^�xBA��k%�^��H���L���\g��:�����2�rC\0�$dq�,=Ҙ����px�ǒ8J���^�4b�17^Z���/�Q֏->svDek� q��3���=Cip��3 + ��O�h � `�}Yu��㐪� +(�S���F�&��-T5�z�,U���8<�%ޒ�Qn ������G_Oړ�X�bA�v��0�uz2u�T�6)3FUV��� x֒�DH�ܵ�����4 [�<�vM���Cv������N��X�|��Y%�#Y���'p �!9S���_d��߂��L +����+��0���hY���t=4>У%%�@9��VR�L�oL��e"�:����e"��?1�Jj��� i% �T5'��k%U�x&��i�"V� c^/ &NPe�jG�Iq�qMtB\�;-C#ϱ�%�� ���0U\�HI����ٕЩ�����{B�0�iRcm���p��8�ޅ�?�����ˉgО��#� i'��x2�I�)c���H�æ|�eL6e��Ȇ��ɱ9iІٓ�U�%���@IKHP�A�Ld���$,%+#Ӝ��p�Dָ$:�0�'�9�O�k�$��%C�s�J��t��rRf����X��H�)��R�E�j��ӝ� W6:�1���䄒���h�3��D����F�TLYA=p�/�g��vG����a~��ܬ�2���:�.v.N��vΥ� �7.�<�l%�ߪ�.-�_���[a�� 0 !��H�����:m>���[\Ԏ!� �ЙHn,�^��M�0�}.��E���u�G&�23V�;�)��=��†"운EX��1 Y����b��&;G�Pw�!By�:��T����A�RD��r��.q&:��P+= pV'71�\�����V��hNJ��3:hW59hL�"wS�r� �(�>KkWe��JJ(�pOΜ��C�T��Z]�t���w�� LE�r����k��f,g�{e�=�A\�8�ԕJq3ٝ`���W'�ﳌ���� ���sS�I*�X,.nPJ�C%�=@e�G_���ı@xt�Ӆ YƠg}��8���^ �)t�L}�q\��-�,��R�c�s� ;�:�w+2���)�<���$5����y�`͂�G>�#�G`!����$a(C&� 'xir�� ���M"��L���*�{؞��k4K�DZ�X�)C���}(*�O1�0�'�MDB��#��(њ�ya��>�8�ԡ�Pk\�cQ� +]SG���R��2�%,�� ��� ��)������RY�ge^\�_k��T�\7_����poU�_v���_v�W��1�h X���'�v�V�l�֓�Ԥ��[�x��J`���7H���i�����<� a��h�GI�+=����I�%}ݢ ���U���tĠVC&U�Bg� s q�AX���C��r�;��>���>idž�2�# �8qx�;�@�����a'`QFUm��U�n� r�WF&FѶ ):)Ұ`DKE2�R�����`�q,�E!��p� t|P�J�NHaZ�#�=�����& ��-79>�ġ�Bd�,�&m�.�|� +�V��ԱU��rض +j�\‘��<8�f����:^�\H�P�x��Q��a[ +$O��$E�!@m�k0:�[)���G�� ��5�<Θ lW�dB؋6�E72����%��:V8�)�%�2�5E�� ! 6���)�Ϫ"����P�cCV��J؋y��Z؋6�4+��hc^t|Җ {�ƼৱV؋6�ņ<��*a/��V؋6�9���t��i����� ��B؋V�!��Ja/ZAwc������׸�+�` ��A<��cQ$hA�a�SԱt�Ahm��m`m�35Q+Mkm���T������.��?B}?6� I��?�TwW��:�(EZ���䤸��1��"��4D�Xm��E&��U(pΪ�0����>�{��I$Upd~��$ҋ˘���I> �G*b�R"���{�:� ZMf�����C�=�xc�c�=�cL +ˎ(H�Dy���P��«Tn�-�a���A o�w�Ҕ$�'Gz��M��R�"uڵ��i��iQAZ�y�{,�>��9�/lO�j(4eL�r(4� �+�Bc��t��_H"��!�hk?D��SLW�tIP�-�x�$9�E�eW��x�� ������3����l;Lj4��cQ��k�� +�%�?XF�R␥H�*�qs�Wz�R��/�_!���e��+>��=lތ$Հ�� W+IUwi���lף?�؞L�2r�ɘ�R�\��F�/�Zs +�FS�S���)�+(e{��m��E=!����j��+��/CqT�X� ��g�.�p*�?��Gݷ�!W*�'�Nb��O���b^����Z�O�� �~b�F���"�Y�I��}�=/2��'���W�1�{��@����Q %)j���Ɵ��cJ��x�W/򽬲&T3��RE&{�$��)� `���+�"W�F����8�:�)����~I(^2����Y����� +-�`��M����W��j�dV��-�N�&����?G[����"��Ӝ��J�6�Z�^k�PS9�[c߹D��\�ȥ T6�Z.T��P�Z˅�\T�QR���(G���\68#���$��8P+Uk��i�r���w�"ڞ zw5�N�(2�;�p)ɡ� �C���d/�=E*�Z���� %�&F!�b�P�H�TD�%`or,�YA��*�cxF?W}N]���^gQK6Df�)�K���H���`���;r �SIV(��۞�I��;�����Qohg�+��Ӹ��ļQ� �Nqf�Zӿ!@���(ʪ#BP8{��ĭ ���D�*LH�E5.� +�j�K <6��#q,T���F���J V��u����gM� kJ�J6f�p�gBHt������/I�Z�s�]}<|����R�IK� R8yK���}�J�c�Ko4� E9�L�+�XE���oT�>�f|̾ʨ�mG>�b�mX�en�J�6�4��mV�چV�6�n�6��c���r=�1R+�*1vZ#)@Y'�Nkc$}�։��#��܌�#w;��X1v���hL��ȴ +�~�KnQZ��b촋�����3�����D4�wx{ +q=�\uA�i�B ��t�#����dLl1�t���ӰR�-P��^@R�D�U +/h�~�$TDB"s�l�"�V�0lX��!�O7��!C�c�$X�]x�l uy���V;�]��%ъ�6��,��F\3��2�i=&ԛ]���S��J�P7\�)>,ż� �2b������Yn �U � dI�&:�Y ��C�@^l���Q +�����X�l�SX��h�xXR��������z��2��������Y��gΘ�8 ��ݠ3�c��5 �vn�A 贀Lt:s:�9�"�$ـ&�� +)'��SK;��.'A���ӯ�K���%���-'�WK†U��"�$�jI0��SN�.D�u��������;�efan�N2�0��wcfan��[��R�;�G�Re'e����i?)B����Pe�(�S�Α +�8X��4��W^�3�9W��Q�(���9���m6�z��Y:�i�(�P���<)����`�…e^�����8���$Y+eGk�QY!��Ύ�q���<�L���|T��y����Dt7�^k�n[[-��2�l�=A�M�Pk�}"yD{��U�� +�]�U�|��m�,<}%�&�[n�<�b����Oy&���e=[�l�yes�jT��>���������Y����i-D�6��_d_ʫ�!����N�3TT���O�a��B��@�kB#�Q@�uR�W����m��Z��@��:����_X��Q䙠�{)� +�ʳ*Pd +��9ӧ0�_��c�GH�Q +�(��J�-�ˎ�ß�l_�ڌ�C�THm͊�� eV�7G���7g���}��D��i1��!iـ +W�h +l��l@_4�#�d@_4�S*Tv���cX��<#Щ� +�J�P%�WF��#�"9.@�� +Mj�Cexv�} ->��Br\���eMڐ��8��&eM��'��!��)u���j(�}���xb^�̩���݅��k��W��ZW�%b^��ܢs�͵܆��ġRQ�-�HAHʐ�# �H�D�����P��+@#�TM�m��˒���66��@A��xT)�=�a�ؘ"� �*�i+0� ^� Bs�A�P��P:��Eh�,�G�_��PYz�苖�H�,�6�X\����.Y��\t�2s�I�ӟ*�)��B�\=qM �� >��46,6�O�4���$� ゑ�`|X� ������0��i��X��� \�)P#m(L +.�����`��� ������x���q��]������Iz�۽�����Q1=�5=N��;1=�L�S��U 7y��A�,v����9Zڞ˫��%xH�YlE�H-!��ٰ�`Ckt��[]�1HhO��A����l�$��� tc�$�H6T⿹�7��> �hؗ`������F��V�G���@C� ��ԇ�I��]U����\\�UI�����j�=Y�*���l�NA�L� F����%0�V��2T�n0��PYu8*�Ɍ��sBi恸���$:ӆؙ��8�4,Ȫ��*"�3�؊�v�X!�K�xa��g�F)mt��t�<��0�� +�J���US����~H�"VZ=(����<��JOs� +X�6T�5�zK��Wk�+,PgO��4�G�R�Zՙ7�����2��9�Ն<�����F���M��Gݘ~b;��@��6�Ϫ���dcBv�Q�!6��V�����r�?=h�����}`8:�F{$�)\B�3q��\B�3m�S�M��z0e ��@�lgC*c?UX������{��(�Q}� 0���c��V�'Ƙ�j����E(Y�V��>y�u�l��١�5Dů�f����ۍ����v�� S2w� +�c��Ff���c�K虒�cT��0%s�(W� u$��W����6*�Ò���P�ƈ̝r܄o�Z��X!`�'BT/���f�Id�P��}�;��ԩ(\���ݬ� �i��jMFv�i�z�'�U�G�j�o8���J��w:Ȃ�4�_��%�� Z�dE^��0���a(�p+rl�0J{��n����o a vwf�=~;;4���`�ؒ��:2B+��S�dܸ��D�"WR�LSa�`�Ňl46�)Y�%)ڍ��[]���'�_!�H�R� �'��n�2�vaO�y�*��LB +�Y�R–ΐ�(��0���K�C�{F$��݂G�C�ְLB��]}�d�ܨ&��c-hh2M�m��� +C�{�6h�i�2�Җ�$��R2���EVȜ��8)ӷVLOb^q��P�s�$�2���)��I1Ĝ�4~EeUxp�;B���ܘQ��E*�v���ou�;%kg{�OS �{D%d#�@���ߑ.hU��V+�8R� ����ƺ�?sE���p=��4.ī��W� Șw�R�&�U�9�w�o�3�6�&��os�`��LwJ��Բ��H����=T0��1�8�V*��>̵؞�\�ӫ 뭴~�V]�9)�d���s#� ,(FQ�� �����$/B**O��!o���� ?8W�£��'W����n�)��źjqpO� X7†���Cj��"�a� +܍WzTha�����X�o�社Ls��C���{��i� D�*_*��W`6F�C30-;+t�#�R��6xy(|l�=�����_��-a�%��eu-���rMjht��GXYI}i�-��(��׳�hmCOy�%V�Y�'%�(�\�ȭi#QGعɝ�$�����I�}��+��r��k�o**qf禈W�9S�g*�������w+����P+J]�A�ذP��6$���4zG�g�"���Ûr�8�="@PB�Ab��Yy*t��� +����Ig� ��̖Չ~'"���|�U��!��d@�{�<$\ ������K�Dy&��xE�8������ +lt���Ja$;pB&Tg�L�]�,cS��@�"�|�ݰ���6��i� �)i��u�8A�e�lO�-GS�>p��) S�a�T�|���Ȣ�HY�S�i��� �8���N�IV�� ��L%i-��=��^TaQH�͐�>pP >���W5i*�UkN>��K� +�W�52�Èc�RG��d�wT�WZE��,�OzW�VO ��U�h�J��p�� ���X�� +'�O_���P���FӐ�''v<:u��r<�WaE����M�^���Qbh�����MJ�\��b�`,�s�g옉d�����r�� ��0гoh+0�̰>���y�)��y��Θ��Hj����zF�Ӽ����#k�3s���S�p&��� ;D �J�$GB�dxGP���,7bn��uS�}]ݫ ����G� +BO�� F�� ��ej�����0�5V�,�R�m l�ƭav,���K �`��r�Gql�����/�<)�p-�L� ���X��enO�� ة�d���T�-^��Qd +�q�Ȓ�`��4���0:�����[��fS���`�J�(�� ~�𹹧��/�×��� H� ��QZ�i�5�!(�^�*g�,�V�����E*J�օ55Ի0����>"�����{�9mdf&i�6*�Y���ҵ��Ax��bJ�wäh���/��e)��sS���=��g��@���_U� [fkS��$���J�Tb�>�b ���t��]�3\�`��ÛZ T��?�8� ��=>P�X�Q�u�ܴڡ^�Iso�qDڋ�5}���j��1)Y�:r�B$�X����'��J��r��^�2N��o�F;�؈HBXs*6����֜Dz2���JOm��&u15�J� �f��S� s����8 ]��qpq��@�NPh#�26lOb�^�A@ ����ǁșǽ����AP�*,2�X�]�v��pb�B�r���\쐒�g1H���߮�����f� ��6zn����sE��Ѝ��!��{n��iR2���0��!7�A,�i��R�6��ӆTb�>g7��i���׆��T�� �3"u�k�4��`�� ��Q';;� %RR'݆�KK�������0�zw:!�HUT�}����>26�՗�E�=�:�_���Z�{�U�g�z.� ���� ň�.�]%�4$x��M]%��&W�� ����&W�s�J��{�vQ��DM�j�M�z`@� ��ܛpc��)3�� XЍ�Q y��݀�t�&ca����D”ʈ~A���/_R-���J|�[��r�����/6�5�D�`ߪ�Z?�����Ή[1<` C� ��w���]wD�H�3l]�zuŊ�ߙ��mllZ[[����iT��`��M�v��e�(|/`� ��m +��O$��}�5u���Į�k]xt��_L�D鷥;w�|��weeef"�< �H�z�� �:���G),,ܽ{���G����nՄ)�O$���\�Ł5c~\\����F-ֺ�������[ߵH"A7nܘ>}���0O�������;v�?�bП�������Ǐ�������z(--M���I"��dU�`��}A[�+tِ�?��z��o}�"�$�@�kkk{��'- @ +��ȑ#B����GP||�>oڽ�} 4��M�D�*�>������O�:����pm�ꓻ�(P��q�y���ɓ ��c�=&�H@�P����;::&��������+���J��g�y��t{,;v,/��b��?���w�ۣ�1cƹs�x�G�����d��T�>��J����\۶sq�����3�/����~�bӹ�b�H�&(L�8�駟��Ə>�htt��I�~��_~�EW�#bc����w�yT<^� + ݶm�a!P�s�zM�‰',XPRR?�����pG����h��o��;�B�G������U}$b�HV���Pg�[=d��x� ��+?���+��s�$��@��۷//+~�'F�y����ڹq��cQQ�C=��h����.]2Ј��G����G��84~���a���Y\\,�D�ɪ$���k���oD�'`���'v^�uS4��t�Ѕ ���u�[��T +ڜ�kp +��g����?���n�A3�� PB�lฬ�������{���5j��{�O$��Q�G��ٹ�|�Đ�pb`^�<���o����������g``��͛Mk�����{N�����������ܹ��Ņ�5<�n��{����� t���'�U�X�M��o�������j��W��ft<�H"Y��\���xmmm;;;u-� ����˯�������� ���n�p���?�$��L555U�4�ŋ-��w�}����s�p�:u*/�Y��M���7��^����\�1ݹ�LJ11��'b�HV%3���h�W����xpʪ�Q?�<|�#R�ў={���q���������׋��8 �YUU�v�E����h����M���˃��={���>� نK}��'� Xs�����������E��Pg�����gr�@����՘ه��!�/R��_|��?���i���M(����c�������� ����������!C8���� ۶m�|��>��Ǒ#G򚠽��AP�'��'�Q ��/7ٶ�k�+]���X����w�o1Ka��w-��zl;��g�:�˝ۢ(�����������lii�챧[�lQ(f.�Hx�',뒛���T&��./�5779r��t��Y÷j��S����,��H���&}��"��du_�zcݢ�q�F�וi�onY� �:��>⚽�{[�d}5:p�{L�^>|�U�.�K��d%�v�Zmm-����O�8��(&�B�t�����/c�@A,Z�1��7�X��"�E���X�� X��p�����RNG.S�L���������r̹�,hs��-[�����… �G��g�� ��ٮ_�^wvD��(�s���/g_�?�k�k]#[X�����i��X�Ǭ�}�5m��5�4@�*����I��)�e XtFF�.w]�r�e�MCC���9y�d+a��O<��ў{�9ޒ2������ �m_�Tꞈ!b�HF`ߍU�o��q}�ԫӆw��Ƃ�Q�W�|���ﺯu������׼���/Gr2�V|�ɾuǯ^3E��b�����U�,�}�����>��ŋ7m��ݣ�>��aaa�I�� �������f���TUU���9�O$���>˧][�q�C�F���Ǿzm��[��ݽ�K/`m}u��8�j��k;f�t��UQ�|�t�Rnn��͓� guuu����l����\SS�|)-Z�����vP��������"�0s�L�b2�&�mݺ���K��g�y[f�3�D2��؇�o��N@��]��(�]�§7.���#�y�Mڵ�r�$��Y�AՆ�v\<.ŸHfҵk�Stձ���ƺ�:u*11��������:yϮ���~����/8h>jjj�y���{���̙�� ��?���M�D2�8ا�eS�N��:�ϓ�޸~��=�\�k��y�+�,a!`�����������Q�"�J�l�y�N��C=���/[0��m���j��ý���6l� ��!h-33�@�⽓ߧ�e�OyT“'�0�D2��a�^[8��gt�@c���gF��9x݆��}�;�ה=_��qZ܊�&й�7_�)�/�)�f��ٺ�큁����"P!�O��[�D��8p@���žG}��������Ƚ�}@ǎ����}:x4B��O$���� +�9�]�h�;�q�鏫��k�1w؊�{G�p�֌�qi��IA�4��ΡM�M����=�I$]�駟tMjO<�ĪU�,��͛7����I��\^^~��e��[ +�x��\���q�����=�}��c��O>���O$��G�u���ںƼB`ߙ�������Vu,��{ + \#w,J\�' B��?�����b�"�ġ7n����2ը�(�<8�h������?9�?��#����>GG�q��]�zU���ا�& rrrڲe��}"E��2�N����3m��)�����=3���������t�i{W���Ը#8a0Y_�/<�����ݢ T$��[���=z�����ŋ�i@}���5x�ʇ}����~����_�����VTT���mܸѨ;�/�t䖖�3��"##'O�,b�H�I8���e�=6s�� �y4‹c������3��%U�'qN�[��������~1 P$�t�ҥ��`]����v�Zs>A�2d���x`���X�.�=�����#G���Ǐ�;w��͛�9�� �Gۻw/�a� ������e�O$}d4�-�zd�G�pd��'�j�0��#���oٴr���kz�@���� �W~�9)�s�[枾��#R��d֬Y666� +�3$$�СC�5{����S�>��s������ĉ�~h�3��}�}��ڶm����D����}�?:�h(��w����:���o~�������s`�K�Ft~�)����ؕ'~�r놈�"��g�&%%�GDD8p�7%�j,Y��^�m�l�ĉ~+b�a�2eʓO>)b�H�Y�ׂ!�g :>����6n[�2�X�;�7����;�_��>>dِ���pfߵۿ�(�>XY�b���=/�J��~��s�,!hj���o��&o)f���l�Z-��g��\���K/�:�D��=l��n�L�\�kᇠ�:�䘺�l�I/oZ1~ɮ������C����j4'&��Q�vt�zѪ#/�}M7n�9r��o��g�}6--m���ᠫ�k����F_;����~��a��0�۷o_hh(��.b�����^���W�\�k�!h죯����"���'�~�eA��?��~e�4N�bW�Xz|��(�>p2d�������s��͛77l�P[[k�2|��7��x"� ��k���o�O$�s�N����bfV�Қ���x���۷ �zKb��tb|� x��vۂ��0�ɻ�W��[6���-g~{����⁀"���;��k���_��W;;;ggg`�)z衇 +Ś5k�XM������V���hL�ǟ�}��}�N��=��m=aCU��M���w��t�z���U��q!h{#� x��n���vv�Z( ��� �nZ�>[6��w���;��(P�8t����Ç���� \1L�?�xHHȺu��W�>�c�p�B��� b���=\Sgmyoت��9L��.F&�%�w]��=���؇�y���k'�6p� w��q�����Z)�/��un��1�:հG��P,#���8p��?NJJ����>�� +����l��ٗ.]2A���(�.`�tSD������9g���6�>�3�|Ff�T�*?{��M���r ylu�����!h�v ��io��ff��(���Y9`����}������3� ��pO�~��s�ԩ 6L�4���F�Ѽ�� �q�}��8\]]_}��9s�l۶M`i^��te���nj3��}���r���Ҕ��‹} N�����ɜ3/_���� �J� ����Y�Ƽ��> ��׸��V7�e*�p ������F/a����99���G��<�ݳ�[�kE��q���U���~��y�k;�_8&���$�p.m�h-I�ߐ�a\�fmyw�� s�K�1u���y�f�?6o���{��R0( +�m����7������^��X3tۼ�գ������o�i徟O���"�$�H�8�>�f��>���,�?�Z��M�}vu� ���� + 6���a�'�j�L{��i����M���g$�d��`�� 3~w����DI$�~s2�}�͜�m��oA���_ާ������C�;Ϛz}y�`�`ȡ���ĉ�91�����w�\�k��y�}�{y��I�R0�K?���ٖ�E�I$�D�7�G�c��{þ*��E`_^Z{>�� �-���s �]�o��1L�G��ꩶZ@=�xl|�7_�xϪ^<p����W~�� \�~�O+O^�$F��$�H"�k$���|.2���vgI�4��*�h�4q]�gM��-�}� tλ�&�t# S�;������ �i{W �4=a�G�4��5f�t��߭x�Ν���>|x���;w���Ŷqc�z����p��7��}3���Q +��Yo��{Rî��[�ua�1��ڎ]���,x�����eC��O���_�Ma��)�͚5���!44���]&�I$'''�T�T*}}}SSS�z��˗_�|�R��ڳgϸq���ʂ���j������3t 7����4gΜ�g� +gk�}�]HH������ b �~���111%�E��v���o�� F`���&�ߕ+WJJJ8mVWW߸qôł��͛���� c�R�`=��pqq��垞����\�bŵkׄ��p� � ˎvBBL��}��d��kچ��N�8�(�pU�/�i;Ɗ�G��D�Zn)�1 �VLXԋ���5z��$�H8��Xv�����[��W;v����������OTmݺ՜~�.Y����hz��'KKKA+�xgg�n�銊 +�|��7O N�h�|� ow��dee��$00�sr�p���Ax�4 ��H:q�DE>��>�ꫯ�8q�Ǜ�����w�������m��<^z�g���֭[������}"�Dfb\�ߞ���'/�L��@k��M8�)S�uN� �M�3���9 ���aނ��T t��u�?.��09d)�D�֍3מ���,b��퉉�=�c�}�����)S�)or�֭E���ř���@�ܱc�>̽��X�?~��� �d&����|�rGG���6�} p��QގD��42���;xʬ���n�\�3� �����f��ɹ�Sz ��xp�G��q��� [<6�����E�{/ ��kF�\���(v���Ρ���;~�����G������L�y������={ w +|u���O?��Ez|��mmmgϞ��»��Fi���Ʈ43��ʕ+555�M߆ X��[��>��}"���P�ݛG����}r��[g�޽���Y�h���g��O(��A��jǯ����yC,��#�y��hE��l'�����z��E�z� 8�����Ev��oU۲��/ߺ~_o1����� t��H���Ȅ_|�E�0�>�[z�Ǟ�/���F���??a�]������� �G���x�3�}@������7j���}�ˉ'233 �B�)�`1A0�=�� -^���6�a�3挶��L�}�1?���kS����m�����{��ͺ{ː#�W�bq}�Δ��>}�P y��楏6��:��[Ћ�G�@�Q�c����y�v ༃k����j��p��������^�u�>�h ��+��zFF���� ��� +���'���������{9�|]]��*�A___�>��bѢE�0�� �{{{�qtt4�|���I��7�|s��4t��;v�v�#����\�tI����}��w��o�f`�������6l���K�.]�p�I��z���l�L�OJp,))�</�=��C!!!���ɣ���~����tK���n�����#�#���+�/��{W�mq�#���;�eru�8@=�2RGg�k�_�6v�G;f�"��㐎�xU�f#���[���jؽ3����;���Ϯ��bĠ-�v���c���.,,ԕ��OO�3ft�0���d]���ё7)�Rؗ��j8��d����z꩎��m��}'O�,((�g�1�����_����ٳgڴi~~~�?���͛7s&��`����&�� ��Ew���>:�{g���k�w��!wo��������f�=ePeGa:�}��UY0�s����i�$L��_:�1�0��� X���^�.�����@�� ��_tt�-��~C:s挭��.����۴iS�b3|aÆ ��.�} D����3���UxΜ9��֬Y��M���㏭���m���ӧ9�i�ʩ�{��l�"P����y8��� ��B"���w�T{���H���R����� ��%5Y���3xʸ�>i)��6:-�����̶a>�z��{U<��['�7�Ŋ=5�f���6�[�M��耓q-���9հ������k�����;^"���9�n�������"S}_����[�/`�111=�������333 W`mm������E[���X�#Ę���ﺝl����8q¨�P���/���z衿��o�Ǐ�m�DB�wz\ݧ�v}�{ON�v?���}�Ż�����. kc��goշ��hU?�.�1������#Sbh%C�>4��[�{e��߬�_�67���l0�ˑ�X�����w��<� ��4�9�ئ.]�t��}����o��k��1--m׮]��?0d������g�����p����ܹs=�i,�A�����=>��� W;v���D���ܖ�}�J�D���D��m�Gv��ުb)�;4�G߾{�\���}��7i��Q����Lf?�e��5��Z�S 4��Q��_��V�͛��ch��������v�T*�#���z������/��%�CM�0�nj?����O>፼%���� sdTn��}�+�`�Tu��������>?�֭۽�}�ߚ0�q�}ql�yL�H������wM��������M�Q"<�@��s��ܶp��3�~�gEź�!�>��@������G�͐��+W�6���9r�R�|�����&�^��E�Y{d�S�{����M��ӧ��͛ ��Q�J_bb�.Ԃdr��!k<2M"��ċ}l�/K���}��օ��+�������_�S�;� m�ţ&���3z���@�Ny��D���D�M+�/߹�7O��un��c���!E-6bg�K'��R0���{��gu�+�/���S�,� �h߾}9]<��������6� ݧ���c�͝;���\7`(,,L_���c`�������9_<=z�igg'�D҇}�:��}rT��7�J`���o�ߵ�% V̞o ��}p������d��1c`�φ�Q�?�{n/*�������*�t��uSz�(����zk���|�C�_���2V���gM�-ӫ�[��_���Z � \ф�ZL�:g�݈����?�����ݿطaÆ�{�* +��!$±�w̘1�Oooo���Z��}"�>� tU�6/��lU_Z�srU��şfc} ylE>7&+����[?�U�'B�/�lo�ԁA�|g�O�{������+�M��Ƀ��z����R�mmm*r�������oMv�{��יi�edd�s��p���>���'t��駟����'��>���*�y�������$�D�}�xlp��������������W�]Q������ԛ�G�@֎)����)�k����6M��wK��2��X.�кi��{�:�������=��9tď�O]����СC����h��d���s��ݾ}��˗������\�/ ��#F��Q +$Ka_XXغu붛D��{�>�aÆ�ҁIٸq#o�±�-$$���?��OӦM�'5/���,++ۼy� C�o�>ˆi�dm�}�8��Ћݻi���j���v��d��=��>"f�+c_�?&7���mjF�9����~��'��jc�=�V�w���,��U��@�����g$��ȏ3��C�,�����۳g��ӧ�����F�T���9r۶mB��ԩS�v��,X��w�,�}�|���)�'ww�-[��C���+.�RTT����Ē��ě� �Μ9������SO=�c�`M������?/��M휜�ޱ'�d)2 +�p L{����h��6 P����O���fI�������7���a/��,��@��2�^��´�P�D���w}���Kz��9E����09`�������������E��U\�%� �{���h���x�+}}}9?wtt\�~���}&`(��G상ڹs'�ѱ��WVV�r{���{h� /�`8��Rd�s@��S�H$c���n����GV�[J# 6��[Q�|�9ut@+`Տ)���v��u �h���M�_0%�;�����c7����� ���+VP@�Q;Gw����]�v%''�+R�������.^�X�����������J&������\���+������<��/����<�p�۷o�n9�Db�D�I�>�Lƾ�Vul]:n��׎�y~u0S<�*v˲AK.�=[�a#^�R1&�m���6�v�Q#v~ы�!'���ѵp��A��M�5(\y_��g��� �%&&�ֶ2@O=�Tcc��ݻu]-��G�5������_q�А!C�}�.�J�ƍ��������j�T�I�>����-K'mYڱmi���/��*����x���o���3s�aq�{������攂)�j{�i��O������ó�>>Q�8�� �g�r��މ����� �s�����w��� ��0 8|DD�Ν;9�ߡC�|||8_m�L�}�bD�u������ +� �f�<�;f�`M"��d6�� �/��OB<�e�2�n�5�#3�l�~LV7"7��c��{��"�a�_ˉ=1������{�~"���}Ll���7���L�x ==��]�?��h8�Ec���"�$�`Ķl٢ j�q/���:�w Ǿ�'O:88p���O���.搈}�9�J�篞?|��mA�ܖ�>B���Ɖ/3����k���~6������g7ٔ�d��.q��W�m n���E�#������c����w ��H��J��MB���'+�2���O�33��lM��N�^�`�b�>$�T� �S2hj+�D9;TG`�?V +�[�'�r/�$�VPb]��y�?�8z��iYY������4p�w�G������w�'t�б��q�����8�SRR�箂��+� +Ͼ)v]nhh�{����_{�5��|.��700���|�������3g���y<���J|��o����)�i!W" �J����pK��c�"ɹ>E�5m���ƛ' +"-�m=��0c՘j��z]#/G����v���d��m/WF٦�F�.�j�P�!��УB�dff��O?͖����eб���p (���������~�]�uŊ����� �>��ٳgQ?N�˽r� +>��\�V�t)�u�'y��X�{_0�D�>$<+���n��L�����k;��h�%�� Q�M^=�� ���S��~�+ꪆZ�U�4�5���S�h�q��ݻw�;�Ћp��`FxwD��?�Y[[�Ak����f���ü�koo�Ӛf��h�����f����� �� ���`;;�����sZ�M�N�is�NhWں��Um �*|/$ۑc,�{@Q��C@����"z"<�UE�D7�j,�@!b�X)P�A�Fo��\F��4N�X�C��ֶe������ƽ2t�|70RUU�ײ���@eee�'�����¯e����[� �P(�Zָ��`�`^@Q�i���@��7�C��n�����j1ه��PR���I��At&�j��]஝`��850�Zn��ٙe0rG�6לR=1O����S�3�@)�+��ҋ#E���}��j��m�����o�B�����}���ȩS�x{���cbb{tf�������wx�N�/_�\�M���g /F�����q�-%� +�K��3����qn�\���v�+�jT͍:0��g-6���Q�l�xC�#U a�?0� a��\ ��~�� ��p��J�;L�p�L6v*�p;�b�̳�#wͲxg�j� 6��7�577�>}���`~|����}��ݻ8�!�������x��޵��bQkK�}S�̧��6�-n?��c��gc�;{{{ ���_~y�޽�S'D.�}KI� +%HdPs|�2��+��3��foK�,L�7���O��8��H\]�o�%�<���h*��B(�qI��c�6D�%+ʳD��nFQ��+�שּׂ���~����������{ގwϞ=��C����A�*##3��猌����p�fl���$����z�����_�g]�tI�Z��j���g���ٷ��b_~`q�wA�i�ש�>On�#�(��2'�X��$�r�W�^�\K�Tgү��aԎ<�n� �{p������Z�ME�5��1V +4��ә�q6���F�ˌհ[�U�"m�J���3���N ���`hhXUU%|�W���������y���<����o��[�i��L����G}='�B��(qi�o���O?�4;�K��y�g-QUUmjj��=�ov�ʕW_}��������[�ٷ��b���4V +�]�e#tb�����ǒ}%>�%��� a���7p{���5�BgSL�50�ϓ]C1��RO��A(�R��خʮ}B����C�v�KrM�<<��}@ +��˿���˗/'�HB.�����ݻw��c]\\x�I�����[o��{�_��_~��N�����x6��$ؐ����۷s���{ﹻ��]�ɳ�� +~��.�%�}S�aً/�:�N�l��\�ڤ��>OOϵk�r�����7o��g�R_�!��_���w>���7��,�1[�d� D��aU�ړ�-��O@ak���3��4���!��U�"�����#d5�X�zl7��m���Y����:�9s������QSS�b� +�^0���khh��18���v���|7�}��w+++��N�����̀�m۶��� +�-�&eee�۷�wUg0������gR����"`�D��7Ŷ��eP�� x|���R��p���P�� 5���߃�G���ٷ�$�}� (J �x��G&"�˕vع���[E�� g߳0� 9�'���h��ȧ��Y�{��c�>��PU��I�Ι�ئ@�mÉO�5P၀�V7o���g �'�~��]�#��p�����斕�ݾ}~�����-�|�=~��x566�p��o��F]]�P�f+ق?"##�1|�9Z�n���bp�����ݻ|}�هܺ��8ߺ�^�>�ZFF����?�<�W�666������TTT��TTT��/fk|����Ͷ����RUU�B��Ç�^0� ه�@�)9aZ1��>���rr�v�T'�Ƹ 7�E�>�J�+���s�=aݝ���N��D2���d�o|�r�z"��:ɱ��톀і �h��2w +��{Bm��}�'���HHH��) �� �f]�~� ��໗��oڴ�on %���l �N�uy�"*��E>���,@�~�m]]��D�>�S �� ���}S�����~���%6�[����0_\3((�Ͷ�'���'�,�i���� {b,a؇�/Ϗ� T w� �!�#�'94f3��c��Ǡ*�ң�]3k`V>�]W�'bc�iF��Tg}7}�Ꮠ/�s �s������4FY<��o!�rW�>p׺Y�b��ρ~XYYY )�p���_X��tuu�����D�P����D��y� �9z������N$�b������;��0�b����y�fQ}Ap�]�v�I�Mc�|-k��� �aq�PB��{0���vН�>K|h㏱d� g�n������'��0f}2Yc�!�름���! Aݹ4���F��P�O|�����>DHu�H��e˖�i藴�����I�8<(`ڟ�}S�"�+W���T�d��Ɠ'O�=�p��+�&����ξ������I!���dCT�u�ԡ{����~����7r�і�KU��X�� J5f�@ϝ@�<+G�2w, ��t&`�5�Vv ��}Sl�P[[��������2B���RRR������NKK;p���~8�����;[�la0�g���؇l;Ż/"�هhxx800PRR�\?�_��x������D���nBu�eM�G���8F�v�%����ž)6������w��uww?|��l���e�6n�H$+**����.J�L�kjj���<{��7�|�[;=* N���q`�Ϣ�|�gc�(��������c���@1x쾾�G����[� �����UTTLHHhnn�;�ʷ�8�~�g2��0����B�ljLGF����������#Ϫ@1a�I��q��N��j]؞���3���:xO��1;��4�BQ�s=63*/G�"9��-�L�%�>^!�����@����������/���p_�i999iii���І������ϣ��W{��)ܮU�jkk�; +-�m�B�PZ�� ���0��<���������UYY�Z�NH!���ȟ6���`�ݥ��̉�ֻ� �ʺF_��\J@�sT��37��n�SgQ�>Ĕ}��9��O7DnmIX�M�G�koy� �!���0Ɵ�.U[ ��9 �9�ɡ2lѶC��.\Z�3'����lH mN��R�(l�?1)hʤ��7�u �7���R�w�0�>�\��(u���Z��W��Ů�1� ���J��k`�M��:n��50�����8���q��… ���G�Iݘ`��WHH%Y�֤�� 1'�|=��ljlrp�%Wi'T��1�.��zi��0�MG.�J]��r{����QFt���Du70���@e���C�c�@E�q��…k�jd�iHC��-�)v��q2�=�QE���b��׉�� +TWig睎��������i���8a�'�!��O�!b ��>�]��}*Ԇ�q +�Ր�œ�и���P��.\�^ `_�R�� �����t��p�u[����q挡�Eb��,�xC�d��Z��Z��y�]��KN��0Pt�CR����ƈ��@�#6e�_�����L���nW�SP)���ɦW��������Å �Y���~7˳.�Z�N,����ȍ?0��S�+ۆ�8���c������+t�>g� l6�P��'ڹ�&bɾi�y�� ه2��?kCwD9c��|XFFFYY���:!!���C$�Bp���RSS�C]�|YVV�p��QCCC�Q[[��JT!544���OU]]�ĉ��҇���766 + +�/t�����Ⰿ�5�o�oM�Y?3*�IK}R�;�*!^l�qx�2���n�-¾鐲�����^;� �"� �b_��(�֙U���+ }傭,1��i]��<�N�^>�Gv��X� ξ߫��޷o�9_����7����o/�5�kGZZZ���544�n�˭�_~y�ʕ&&&EEE�AQmX���驨�ptt��2�RII)>>H-�-�����޽��� ���o�-�9|��7�����٭�� �j� /� <�2e��2ݥ����'�deXY����ރ�����^�8����?I��v��� �p��Ǯ��Zt�A�87� �oI���[�c�Y;�k������C:~�l���9�(ξ߫��>���qn���� �{�y������S���۷ �X�W���\ؼ�?r400��-��'�̩ �H [[ۚ��:A��@1--���N�MF8mؽ{���ǃp�-\|ه�۝4Òȃ�dT +t�Mg�[Q7��a_�[H��g��㔋�+�����lC9jkN)�A��Eeb�Tj�p�@�����w��6���b��^a�vH��O� �n��}5l�}�W `H���>�kMJJ��Ksg�% ��ի�J=n���;������������ii���_����+����������`m=~��'`���7����_���k.D؇�6�rA�T� �:��:��&(:�Q��(^l�E��� < �4��6�lP)P�Y�,?��:�.>����G�~Ԧ�����%aE��Q�� ��A�L��.R䎲��!x��K��Ý���@q��^%�}��7n�������gΜ���}�����'Oί������733����?9:: +vI��Gs��/���~�\�SYL����?���H������p�z!����H<ɕ�}b�KQ>��/(%�Q� ���=B��{�#;��q{@��v�F�9�@@,ٗ����y�*��Ø��հ"�n�]�ĺ +�ݔj���*��^�(�R�9�n_+�j�8�~��>�ϟ?���š;�~��W_}5���d\������wq�ֲe���555 �YZ[[MLL�_�\H�Y�&==]�@d)W___!�eK�����[ZZ�o�>? �>ϺLʝ4���#� ��� �V ��`d��.�}���㔳�t Mց���Đ}�,�V� 9��vŠ��=IdM/�+���0�R��$u��)�D��'�q�­����*����_�3###��}p뒒��n��������\]]�t:�L666>~����~+`Ë����x��� �� ��͋Ih---h�J577WRRZ�v������KNN~!z�y�5�����O�K���S(???x۷oP 'B���(.!��! ����|�D��}?���E ���}��s���r�Uc�Ն.���R�%�X��Q��%ō?��I��R�#��i���?��d�5e���3���8���z!��8/^���UTTv(°�޽{����lY�W_}�tnnn�����&&&�ox�ɓ'qqqRRR�!>�Ν;σ������P���?xxx���<<<��PBK�́Skkk��χ��Z��ç�+���~�����lm����]\\zzz���s�bPB�����}��G���������>-J��9-!q��3�7F�ͬYE�>�����ˎ3p����B�9�5���)��d�<�����h�٘JU��p�~3]��g�����F���l�W\��� ��`�A� D;q�_��e���LLL&[, [�|9_�m���D��h|M���������[�!�����H$ +��8ZUU�y�o�����=�����>Jy�It�b�ߖP����1��r�DP/��}�)P �;+j�a�����F�1Z�,��b��}H +�❐�O��̬�YY�7���Cr�������'q�- �>Й3g�;Y�cSSSO7{�E����֖�6�|�X �*U�B^�r�/z+�^�gt��� �S�:t���tNm����O?��@�J��x�z���������6%%eN��`BB������JJJ ���gּ�gk�.N_b����Bۂ̂�2��P]V +�r����Ym0���$яo�L��X����O�"ņ�m�3k`ǭ-�?fK�d�S�}KC|���+���X���4�/���/##��,���� +f_CC�w�}��ۿ���vvv������@CCC�)�0� < �����{��� YY����y� ���]�?x122u���Q�%��m۶���ͣ ��<|d@'| +| �B�g=���;C]~!�R���]��[�"�.*�8��>ʮ�;l�$�ه��)�"!ύe�z�E`��A�����֠6�x�6���M@ �Cg�R_�mܸ��/�D�x��A!'�AG +N��?D��ƻ�+$���\�z����B 2% �6��rZ����M`���� ���k�椰�0޹p�cǎ� +k���7��(V�Xq��݅�����ߖ�D�>��(�_�P[0}�\b���b�>$��h��v�̹��*M��7b�>��v���xf(�]�7�Iׇ肳���˾'N������>��c �0�{�g�}faa������*ު @���w׵��+W���/������9 n��~��Ɵ����z�m���v�/X?�䓬�,�#===yM|���� ��irll�q���E�>Ө�k��纐U����A��Iذ��զ���"�5��8vR$/9B +Ѵ��%�� `��B}��֙�{���O&|�S�8������… 7n�@%�e/^������F]SAA��͛B�L��� /���꫊���'������Ñ�CBB���333?��s�1p�h�Ё�߹s���/������7����Ν;y�ٳg�=�� 592:��!������yA�ه�IT�Fd��0g�����{zߢ�`�>J�%�`+�3���Q�d���D�%?�b�>��z^-�?�8vj;��1�2ܕ|H΋D@�}KI|٧�����s��1^�����tԔ���z+??�֭[B�̗��8����_���DU�L�����瑪௣�#� ���F� ��-���y���?r�`Q��}�˖-�����C� 2�=}�md�~=ض�P�I������������J�b3S�?����q�������f_�%k$ʖ�C6��(y/)��,z�MG]�6T�9~5��wBwF;yE_��o)i6������F9����/�������|j���� ����}Hi�g�}�j�_|q��m~v�x��o͚5 S� +�S�N��cH D� 0���u.�m��(h4o�� ��"�9�t��&!�߳h���ճ�)?�{�"���Nd�l��Pk�V1tW���7��"'� +��>�U+_yK�}6�3�g#N�#^73-���}��,�;{/b{K�*n>�[S�+eo,�հq�-% `t�/^Du������ٮ644t��i�վ��KdtLH��� + +������;;;E��kjjV�Z�j���������.���L.x�rrr/������=q��-�/�NUp����ѳk�}�{�}��NY<�!a�"F�9���������̽<=_hE�>$�.Zy��t�jn�~F +�q�q���| [�i#;—�x��{<��Q��ޢ���ٷ�$�}�nYY�+�ޘ��@��{�=TG}��%�������E%��������촷���ʢ���޿�w�7�|���,�6������;w����޵kע������.!� +It�w��:�׶�(��{/ ��Dr~-6�� ƨD��~}�>$��i�ا����y@�o��*V���3 ��z��1BW?����1Yֆ�oa�����c��k�.4��o)I0�C`�P�wo��fFF�������Q��W�����B��#`U& +��B��v����Ammmԍ�x� '''�QII oFt�֭"O6�����i׮]�?~̻3�w�}���*�6���>�Zp���ȥ��23��X�����ɧ���`þ�,h��Y��&�!��{L�ܒ��a��.{XD< ��b=�J��!gO��p��K�"�roȚ�:��T�-�n憳"��C6b@���)++�f���YYY����6� �B�o``����(�����������7J"�P� �-[faa������l��d���D>� �;�CBBLxO^߽n�:xK�m��7���;����t�Q���Շ:�)��{�7=Ì}��� �>�Q�AZ�����Fv +ԃ�M Ɔ}!j�� K�������V���Q��5� �t +�A���i��"%�<�Z�����[J�>Pgg���<�D���n޼�M"p������߹{�w�[�>����Q �lee5'���������+~ `߆ �\>�<�b�4�@@���h��H��K= i1 E��Ϟ�fǠX3\u�����[�+h��c�����\$�=#�Cيv �}��@t�e���}��=^�&�@K� ~�.l[�������o)��C6^G-�׿��Ľ�d}}=�>}���]x�]�r5��{������;###���^{ ����Ɂ�Q�HFFF�E�� + Qk��^�����[�r�J޵�p�J|��<J�t�ض�7[�.�� M�N��eb�>NX0� + K ����)��&d:�Ƴ*�Ee��)X:� YnD�z�$�������o��B� X�'bm�Ū�q�-%��}S�q�����������ܹ�8��[4RPP�}/��Q.�/���� Q;���"�k������[o����:11v ��K���D��]BB/d��[[[�����>y`�hۀ�#�& 9�G˪c�R�����[��i��Z�C�&!Y�X�I��2Hb }n�� �;�gA���ӂ0`bTH49�����[O��J�=o�l�} +T;�S��(<<٬�wuh?2�O������]�/�>��͛Qo�OIImpq�B�M�һUL���a��z���O&�;� V�CŠ�ɰ��0A�@7^�M��cþ@5��*�G�D9B�'��{�?ˍD�ݦ!��ا@5�Tj�5'�jO���Zs��ʠ}�/��o)IH����)((�[��v��y�[�~�in.�[�}����hjA�6mjii�g/))���oQw������خPBB��e�bbbD8�?�nN���3�^���z ~uhkk���.��a'��;�^j]w�w�vNܾ���u'3�!a�pUbXm�i�e0z����F�0��f_���@r�E��@D�@)�L�,�e�~�y�"��&��@�G����p�� %$����ֽ��LJw�Op1J�z !���������X���tQ}p�����R�RRR� >����**� �ҥKs��Pp����'���oWVV"�������W_��g.���>�\�5���4���ax�;�GnR-�r���� ��QaW^[�{ĕ`Cr +2�a�>��$� �v ����v3!@�� ���⏵h%��Ø �p��i%<�������Pٹ?�p��ͨr�>���1�}�������.\�0�MZn�k�@����-��t:�L˗/�z�S�b��4mܸ���}���z��(�T*���bh���s�K&��*�;�����5��Q�t溏�_�u,�gǠX2�� �} �������|����v&� �P! �����@/)G�6R`�= �E�4��o��*�?� �����Jx�MLLDFF�̻�$`���c³�����; +�U��t�����oٲe`-9�8@����{�1p���2й�g�����'�W^����|Fx����h:���qqk>��`���U��*]��Op'?Y��{vݾt>$'3�qh�pUaX�3������������/ğH��!��"�q����z��F �Y��.���?�}r �>�f����O�[�������³����[��{<���~!ږ���;��}�v�|�������y��簾�.� ���VYYYދ��7UUU�G����~"UTTD^t�k���7g�/��x��졆=;�=`���LGSF^ '����Є�t �l��.P����T{�3�p�A@��C@�I�rC�Z��d�b�](�>\³o�M���P^��-�8����>���*��{����ӏ?��偳���6o��kW�y睤�$��iii�V������t�k�a�D"�� ����h�a�n����o�h�y;P���ܹ�Q�+��ѵP����jt�څ���fx��;˕�}�o'��2�a�t L��y��t㟸��{m-]���� ��"g_��*P��D)�4@iӐ��N:g�9�o�]��;׏#�Dff�l��+�:::�������������֔nhh8t�j��K�9����s(��:::��&����߼ys�n�������#��~�!77�u�EXX_������Tw�oHH������g�;�?�D¾ +?R�7��I�N�tÞ��)л��5������ob�>�@�!�I�bw�f��[�@CIw�+v��FT �7M@���i/�b�+q"��I��^�"�����5W�A� ]%0��H?~\������d����.l�{��;v��� ����5111^���������Wkii��@TA ��n��������x� � ��utt���zzzx7Dv�/��������������'|epS �����:t8�{S�{�� �� ?�h�� _R�����vm�����M�0� 3��a��ψ�d@a��~���/>��.���%�����}�q�d��4��d%F�=o��̚�Hg���o�m�:�vŸ|� �L@�2W�!�222�K����Н���#@kg�/�ﶶ�ZZZ��ˀhٲennn>rjj�_|��\ ���\}}��ؘ�6������~��'���/�D~�X������&�9_��5k����_������O؂?JKK������~��W�m@v�ռ�߉zG�:�j���!��~�b|eJ}�ӉY������V9�Ԫ=O��o�Z'��jێ�s�42!23�A�S5)��� ^�(���誝���!)P�<�Y�h�aF���&cc��I�هKĚ����8�� �u + �*[�{�p�Q+��z���j6��@Q��C@ ���g��'6m��C0a��d�,{"��yVbD�i����هk�b[���@�Od����7����!`����U�󎍍 �N����~C ԡC��,[�c��ENN� |���`Q-����Q�\�^o�O�����«����P)�Ed'�w4��=����> ����� @@Ѳ�C� T+ �� +/� �FT-��KV��>N�*�\�%�t�i����"�*P�}��Ǿ)�B�w�ޭ}�����k!�b϶T��_��[u9'���+��\7$觧��_�^��R/��2xU}}�y������{� +�r���ƍo޼�$�<�ъ�5�g�;U&�5��k=rU����>$ł}�i���Z�r25[�s `�yEOGgRX_.�8�8+j�I��製�[s��5M����Eȋ(�!�.Or�&Z�� �����I����w��هk�웇Ⱦ)6z����y�;� +����?�����y��������?�<��� ؒ��MLL��켎���nݺ��1 ��:88���.�D�!�(�vP�*Z67�����5tM�'c�>�OB��v�e�}�P�����W�Y)P�؇����>��>����h���\��y�Y�ܐ ���}�P��~�h������Y㛎 ���ه�ž)6zzzz233y���O?�TKK���pppp�>Ɗ� +++����zNm����$%%�S^�^ ���d2yժU�{a`%P��ť��n��9��b���W��7�o���K����}iD= `������j���@�l����5�.*�8P�j��K�go���,�!񫗦�����%��-������R]j%A �f�:�(�>\��Ç�:S��������2�\�7��Obbb�}A�>o���k��;^ @����x+--�+��n޼y�ĉ����������qڀ4�����eˠ�������MSS��} ,==�ܹs����9�+�:��g����,� '��� ��ӑ�%��VP�{���9�+��<� �ǒ}� �f�\.9~�fێ:.��lښv^�������С8�шR�:+������Ed�@�a^ +�g)P"w +�z�U�A�هk΂n3???j���-��p�������7������,��tvv���ݜ��-,,H$������kXX�8�����\K�:�G4"0.[a�>U��2���~;z+�`�d5�հq��…k�J�8t-l�I���~^��-�� +b�)�4 ��!��N��|���(�J�"��}1c�����������#q�ǵ�3@���e�J�{�"���!�s��/��;GgP��h!����վTv�P͖m��#�%S�$ݑ���bH��}���u �p5�詽r&7�t ML��-����>�X;�[��WB�'���Å ן\B� Ja�~j�����������!N��K �?ѳ �[j�e�ٴ�� ��D��-Ij�=]̅������ማ���\�y�Y������%�&� +�]����vH�8�p�…k��C ����+����$q��o�8 �{J����6�����k�n��؄��V\�FJ�h��H�+N�Θ� }{�Φ���)P1my[��3�=��ه .\Ssg�4 �!��n1�u���>��۝bR�a�<)�Ed߳H7Һ}�0�Zq�:q1Nܒ=��Hpp�E�>$ ����F�� �˽5~���23�_�!��"�Jg�?��L�������6::�%��0м����eJ��I��f&�~t��c�@ú�e'Z�o��8�}�+vlM�r�ә�" V�CBݑU��Y +�����J��~'cc�e"g�?��߿��� `���P���***��ߺ!�棅�/8 *�?��7J�=z�s�\��79�8�Fܧa�>v +4�J��#U;6�r��Z ��Cґz�]��d�����=I��t7Mw��4�Xᥱ�]󄍞����"@�}�(f��xc7*&������z�����9�|���q�ē>>'�N2'ῼo��&ǞO�G{ �۪I��D�x�#�@�$����'���[rb��krb���#C폹�E�940 W�j��l��r��������̡~�ggg����_�#{F��[��Θ+72>�p����ݣ��3����Z� +�/�M<�>�Ak��Z�� ��a�w�#��<�&������߽�3vf}K�p�q��'C��u�>\|�l�@�!��F�t�������>n�SL�U��X��j��R����c{g�O\쎤x���to�s���E@s�ut�_8�{F���E��>\ �pdM�FOT�(Ǎ��u���~�Sʻ�2k����N����D�(F�'e2��M�x� Wo���w6�����:it���9Эw���;�P��������C�p��������ƪ�MO�x�}�H^���h��^���Ѫ�����-��6�����^�ML2��R��j��0������6F�q�x�%��D��t�'��bx�X�?:�|1����CI��O�R���|%�\�=��z��昫���;��ߕN�����x{��_�u��ϝ��1ꗳ�����ۇ{ҬeoZ���|�P�MiЎ8m�(5�}�z��@���Z~+MN�O>�d +��H؇�?��+j�K��3����ZP5��3Vf��[e+wo�qC�FJ"�����=W +tQ��!��a7��jOM�7w�u��L�)P�Cg�j8��c5�[&��RB���p�5W ��:���s1��P �W= ,�DK��M޽j��꽜8P��O�F�z��S\o%�����^�����i]�{� ez���h�=���Ǻt��3G��},,j��uR��0�s7굻ԭ�K��ؽ���\`e��\��!�;�&c��p��F�՝ݦ�ǟ�G��b_I�;���L��p��i��>�vǽ]�:[b5.e9w�~%=(�`�U�Ž�G@�5��;��rν�� �B�e;�Y�Y�2)f�M@�ݞGp"��2LqG�VUW#gÝ��Z@�{u�/�AO����_��.Bŭ*�!�ۆ��n&6u��U]��^7��/e��d:�["U/e;w��`#��P>fW,�A���89��L/B�!A������)�_��I9F�y�x[��a�>vd�(�F�@�H���п�J�G< �C�kK8�d��������5O���ꘊ8�p-P��Nq�H̝I���+ +endstream endobj 131 0 obj <>stream +-�|�7D/�2�9�O�������G)�%�k��v�a�i|�76:�ѥs�֏9:��$�*�����D����C<;�X3�,i9�ץ��?�v�(��`3�S�2�>��]Ww ':�'� ���L0��M�;���t�ܷĨ[�08~a�i���d�0��0JF_ �K� �V�]X�x6ͪ�wz�?��[��3^��b&��7�Y���> l�0�tpL�`g轌� +�2E���H491�l&3��0�W2+�0�3;�&'U��"�?��;J��?���4�>����:\�ޟ��\�g�%�Ҍ��f��)>������-R9���l�\1c��-A��L��䔳��h��M����(�A��õ@ ɾ������3Q��o x�6=5������$sb����ځ>��xK7�Mt<�6��s�6�� �}=��z�:�"�&GG0`����#�MR��s�O����{�� �M��ۼ7A��v�ط3^G1�<#����>�\ +��� RS�����+���v�C���p/�O�gi�o�I��مY�����+�A�9�����l1������r���e�}�[E�k�U�q'� ��~�/�asUOT��Z+!���7I&�*8y9�9�:b�>Nhښ�订k�R�;�:V����D��هk�b�<7x ��f6�e5�"�i�k��&�C!��^�7��u� p�5蜇 �O��<��M�񴢕�����+�6<>����ɧc����Xu!����6�~��MG +R�}=Vc�Cq^H��'O��d�����x��+�.�#c��"a��D}��p�$B���݀)�wS�"�b��b��$�ܪb��>Ki�]��*v���'�wԟ�A��ِ/ +f_h}:8G�C)þl�uh�}5�MjYNk#.Vt6��-|48��Y��b_w"��f/�|� VI1��'��M�8��c�~Q�i�bα_>c;��k�C�&y۔ ğ����_�3�*wnr{�b�]�j�s����q��윓�v���k<5�X��\3��y8�p-P,���vl�������c�O�����>��j +��:�-:��7=޷ޣk�k�t�����b��c5�s��-���:���a_U>�k(ޛ��`����&'G�s�}��r��c5E��>7�.�}�9q�e�p��k{Hg91d��㰏9<8ZU�C����f�$}�����/Q�th�rk�w�;ղ�eS]#���}-p�ɛ��>V�a�J�C擲ܖ�����٧n�Ë�Z����>`X6�ߑdc����5��w���6ԭ}��:Lɱ<n���w!�N,J��� ��^XWk��� �7ٓ2�Be>�a�+1+%��+��b ��ifG����Y��>�:yG�v���5�}���*l���3͹dv���}��}�t�q�)����2)ɔcG\���0c�մ3=�/2�[(ڪ$���}8�~C�ط�c�:g4�q4�YM̮a�}���m����ݱ���~ ¾��@ɧ�[����9Ȫcd����C��Oo7sޝ�1F�˾�����N�� ������T�6V_ΪuQ�ݭ���8��.��݆2@LX9V](U�i}���UHL�4MN�s��9:<|3�K�@���Ѝ����l�UI{��D]��Q��K}T�9Fp����IF_���B���8���lG��]����I.�=�ӷ)�\ +!�>C0��Ī��p"�W{�X�)��n�#�<�Λ��QϢ$�|�����-����Jc�ō�ĎY{�' +<59X�ɓc�>�~T�vA`����U�.�/����b߳A��+�*vo�}ƾ���7n��O:q���\(( +�q�agz���|&�Vxilv�"��{�>\ ���^���*�~R&X���kF�b��1����uiI��_�1�71>t�������nc�>��Xϑ����p���,�MN"�0އ��U!3���e֥���H�Ks�b���)Z��+�W�]X��� �\�v�@�'�j|/ �F�������a��l��}�?���� W����>��+n�����%F],R ��d���6B�b�C�pw;��{�f +�Q�Y��5�����0ေ��� �X0~>$*�����6���$���A�n�L+T +t�� }��Ӈ��o��f_�F��F `��^�W��_@@ѱ�C@EG��4�t����@W�S�NZf� �>\ ���.́�^Մ.�@p������F�?�h�Fp�,߷�o$�.���ց�As�Y3�^����N oiҍ77]k�?�{,o� G���"Lt�¹��q�9x��Q�Յ��;!�f�j\㭖�.$چ���j�_ʙ��:8�Cڪ:�o������\Q�v�Z�����֢��F'�,KbQj�u)c����!m�[�N�tBf� z�$l�T���9�$`4�s�Z�qh�x�����L���$�ڞ?��= +z�Y�]s�$�#���K�Y��-it-7�]�_Y�r{�Y)P[�K��%�b�>�4@�쫧��䀹��f� 0�)2G���M�lg%����1�r.���)������U�lN508�p-P��y>�i����U������de>e�G?!��3��[�޵���n��ȱ��M|�O�?������d��豹�m,ץê��)�dN �����F���y ��e�'�+�!��}k•��do���ؔ�O1�as;X/�����9s>Z֓���ZfE��z�j]F�)���M*:�s��Ww5J6�}S���y��f�}S�b��m�Հ��\7>�l4���T�5��{��C�ه�7�# f� f�����[�_�D+jŒ}HX�����ޟa#>�����e�)��)�T}�� Njvo�~��X�yS�)Y s���!�U{�#�z���+�;��FF�����>\ �X�ۜ�e-��IV�dUۀ}�h�s;09��~o�&g����;2�Y<@�D�p@8���޷����:8W�� ��s��h89`0�e(�1V[���>�t8-|(�w������?n�v uF�hI:r�pj(s��� ���ك���ʹ���>MnK�eI �ğ�e�e�\*"�_D³:ޡ<�^��9҇J�v���ץ�+������M�8S �H0��ʹǥLf��TZU,7�����Z��J���˙�q�F����͕��� +�R���K��}�\A����^�e��HMW\٪��X������ �{��=�n�&f�&�A���+����X�UЊ1c��'\��e�N�>��V���-_�b��g��j���u>������+$ū���@�t�&o��&�<p���!�E#I�f&Չ�8�p�-m��������m����ه�7ыهx�'�zEN +�כ��;]C�\J��wQ�0PT�CB��k�c�W6����f?�S�K��k���8w��ɼ�{�7I�lx�+��R�����c�>$�mMe]�%ܵ��`��q���[�jnn���)++���GMMMNNN����uC���}��fW�C3f��|����������k��7W�_�)�q/�ǒ}V4:�J����>胙50_Z�%R]��X�����+gS�+f�@7r�@��o��?ikn:m1a2 P����+{"<�>�}�p�bKX�q�Ba>4~Z�����>$��TX`�SX��84��ϓ}�c����׉���N�D}���!�ճ�'�KI� �m��#g;�؇�?ulM��tp���Å �����8>.Vn��ܝ��'o ���%�zU�M�.�8�s�t`|jd�/������D�T�ا�<Z(��RB���Ubo;�h�gfM0�3ǀ}���A���g.\�͇}������Ϫ|�yg:�$9�#_�<�jLa�;O �ⱏSs��w�}�ǖa��NJ���n.D� �'Z�!���v�Xў�Ubb5�50���Nw0�&$�p���Å ����K � �����|�n.�u�k��v;_7���V܄�hJ��u�������ߛ��O�QDCL��6����3N.�z^S�~}�u[7�(�&Z����ه .,� �zMG�g}ޕ�<���q��Ă Ϣ:$��� ����Y���1�uB(����kK��N����}�"�|��]U�b�y��W�_�Dƾ].�&�$g�>\�p�P�aU��Nw�Ku�L�V\�.���a�>+��%Yނ����.��~@ >Lt#Iɨ2�Ef W�)���+�6r�W�~m��x�9GCmq���Å �"Kd�c+z'��Iޡ���U�O����d��%Y��������o� `�7DE�s�K�!)�$�3�rK7�W=�B��]�JgI&8�p��…Kx��Z�Yc����}����W���������On�̌"��c�>nn%y�1#��&1�[����&��cG��B����P��ذ.{�g-T�g�>\�p����pwX������c�:&Dž�)^��{�� X)P�A��5��?�%L � ��}*YP&��G �N���C�R���&�X�/���K�n�.ھ�b�znVnXsJ��H���ه�.\��l�&��K��Эۤ�@�ڛT�A��ȫEc+n�j����aM|ƾ�ԟ���\��D`�>$�-]�{~M 䮁����hj�0���%�ؑ�,�ylo�fq�rvm TV�$��ه�.\(1�F��b����,��2y��9T�(8 ���CR�e�u�:�y[�u��ў�˃뒷"u���d��%YÂ��D��e����I������I�� +��c�ک� dz�X��+*�>{�wM5["�>�}�p���d(���$�~/�NR�V��s�6� HW8�.�ξg��wxxK�#m9�>�?�M<���Ć}\)P���oC���W ���M�Ì/��a�>$�)��-Ul�q<`������M� �l�X��g.\���o��V�ˍ'����nX'���[����]��Ĉ}H���$+=L�Ҟ:��-7W��(���@�}H�Y���[�P�}H���W����X�&��cG�E���{K7����n�Q>K4ӝ�p���Åk� aߐ_ĠWN�E�ó�w7�H��1}b�J�����+��(�'��j�M�m���3�G��SiHޙ�63���C ����V��{� � �2 �Ռjlb�l|K�!�����vT��������{(*�֦8�p����'7��|�}n �s:L#�n7���I�۩�Üs�g_I�QI�aY�vc�t{��\�O�'�L +q�@ ؇��Ȓ|���3��3�&`�f�u�oc����>6���N]W8V�M��� *ׯ-۸>��>s�k�@�}8�p�ZzB�o:|��iY�)PI]n�(��w� �-�g�, k�]�ܞ������7֔�*�y��cþ�0�7 �"�~e�l� ������E��U��b��Yī�̔�.�$^9���[��UX��8�p������l�����v�bɻ�m6�Q�Ǭ�2����F`�jF501�1VKŐ}�h΁�e� X.��G��>�}�p�9%�}��y��N�����g��H��4o���۱o�����������?���+��� �� C���q���23� �f�d��&��|:��XU�d�C��6U�_��_���8�p�����`� y�b�+��-뉲��M������p��@���C7f �7b�4�㕛S6��x���[n��~������3�M�hs�@������oM,��Đ}1�NBĪ�e���X����c�Ƙb��h�-��X��n,�T�XP:G�����]P�" ��������;@,ܞ0�3�ϱ7;�������m��g�c�cb�� �� �q�l�م��F�U�O�X^�x��=��;��}’�y�r�m�%��c#t��G����g��c��|u��>�� N�B�P�>J� �=�l����,5�Q�Y�g���a �61 k�0W�,Rm%cc�����{�{���ޙ7-�,2;:V����>:F鼶��}��ϺU�k�}�h��v6����.W���<�00&l���C�S���>J��F6��<�5 j&���a���a� +B���R� +QS�c�c�cbb�}�3��������˺v'�j�P*���0�W �:�(��Rc_�ݝj�7//�����S��29,H�J���c)z�VK�zy�4 )g����Ҷ5W��F�1�1�111��ޓ}��c����r�fY2~ ��R:�H�F�������oCzĺ���L����N��M�3��ךkk�}Bhd�ad=�ȱ�a eJA�7z�S���7�����}�}LLL��g�=�P��~�7|N��UR�x_ұ;9[S �g���b�8��E�;��o_�����b,M����y�J�}#KM��B�mW�������)�1�6�6K��������=�a�w�?�o�M'�ҍ��&��=�D�����b7��',;"T3��&Ix ̫)�3I}�F���6�&���Z���Y�_�M�P����t- � J�}�}LLLP�}B��� + ��sz�^�p%��@��h��;��>bwF��n�����˩]N%��R4�p��H� +,� \�迆�&���>��[�����>�j�}������xb�z�(e�����'��33�Mp����}�ew�bq����ΥI]s�����[�I�}��+�]�S�A�� �ױ�ܨ������������U��� ��v��ű)��MB��X�7ze�� ���{�xd�͊�wM����j���� �{ L3A��:^���T3�1�111����>Qل^1t-���?�[A�'"��\ ��t��U���WN�=Q��ƍ����P�97j������:I�}t +TA`�Q��T8�YN�FB2T��o͊-����������$M� ��~�ޯX���U�>a�2o��\��t�J (� ���5�B�;�e��\�K�8�ٱ�)��a)�SN? �`�@�F"��|��@��m�b,cc��H����W� '�b#炥z�cWs �7^1OE#7�VRc_j��D?�ԀU�g���G�?Q.$�]�e�V��f�S���>b�V��_ )�P� +���qs\��������Ts�¾�.%N�%~E:v��_��0W ����#��TPRV�<���^L��H�POg�� `-���OS`�Z`�U������k���~ccS����9��;�#d +t���<�k�S�Rg-*{B扦@��S��pf[�Ԡ��“��a-�f�t�~���DD@�>�>&&������ѯX߱`��p +��'��]��j��ߛ?� ��2+b�ɸb��n���Yiec!M�i�j�/ֱ�\���������� ���X��p�_����T�\=AN� �+'����yQ�S��k`z�F��vR�7����t�Ut�Z +�������A2¾W��>�l�?^I�>����c���I'g�O�+'���EǶ�N��.��3x������cc��K��G<`��s�b�r��X�+�8w��G���5�_��OX���"����9��bb�h�(�Ŗ5�}�}LLL�H�G��)Ҳ=�>&&&̲ٔ���v��k��Ǯ&�#%w�� KsLժ7���>r���E�b_L���̶�wO677d�c�cbb�A�2�^M��,�����OH�a�s�*�i�D�V�k�}t +tu֦�'��\H�|R�����Bߠ�)P�>�>&&&^$��#娭w�j�CS��.��!��e9j*9>�L�J�}( ��w.ȏu&�5�(��f�� pV���>�>&&&^������﨑}��őeZ�P~q�T���9�z<�/�wU�ߊ]A������Sq=���������������}D�+1�/1�E)�0;:]!GgmV�6��$����F��ЕkO���<��V��������}��,�0;4s�������y+vy��}�&�*d�N?�Ѝ{�d\�͞���M���������}�Է.T�˟��'���/��h��U;ccSm�.��[���OP��8{P9Q_�Oe��5����WN�T����F�����b�9�{&ccSm����\�7_-{�e�3���_��oM��J(���rO���Ag�Q&�Nc�c�cbb�m�U�����ϙ�&{ؒ�8lQ���,�%�'E�)&��J�^�/xꩭ��������:�>}�|�Y��i��n������,��~���kxd_��x��G��0�1�111IMu�}�*J��I�� Br ʁ�K���ῖx@��oe��BA�P�>�>&&&����O�?=��u��)P��(c���fJ�!��$���,`�c�cbb��W�;D�@uL�Tu�'�������W�{��1��P����}�}LLL��}� �#��T�>�<0v�vݕI�ʉ�oA�g��Ǝ`�c�cb�窷�#P8:K����ZxpЂ��-M3RL��)�>��0�1�111�g�Q����=I��/Kn�A����^�,�vuM ��žB�"�/�f�c�cb��b�+7�:&��7�������+�h?:��D*,Գ1ձ6f�c�cb�ob�{����s�Uɒ_L�W�� ��]�j���R�����P`k&�5ճ1��2�1�11�=1����>&��'ƾJاiLJ�:A���YC�F@���-N5{�*�:�>ZDИ���������� ��0k�Ƭ +�����Q w�Z�b���R��'6������T���W=����0�EjY#��f-�?aю�+�\���(ul�5���������Ⱦצ@g(s `����)����ʧ@mM�zh2�1�11�%1�՜}B��e/����� +�����V.I�Y����X�7�SȾ�}�}LLuE�}ožr���Z�>k�r���@n��I w�Y�亲��z����J&�������1����'� B�0[U/�_��g��'��ܡ�����*+A��E���������$�b�{�qJ�Jͬ� +��e��:�>E+A���&e�0e�rX����dY�}�ɾr��5�-�'�#e5��Ᏹ���I������#�O� c�r)�O� �S�1�}���N�2�111ɲ�> �4 2�K�}������}��+��(�r �����$�b��x��J?��O�?�ʧ@��B*�@����dY�}5�>��Ll�aCܚ���OX�-D'������I�������� �P]=n����Kxf�$`��)c�̊��n�/^m9���o0���>RBWXX a�cbb�Q1��%���o�Ҹ5�xg���&�}LLL�'ƾ:ƾr��)��(E��Z 4�l�b"�������$;b�{�{A@�)P���4��0�Wp�[�����IF��W��W�(/����G�����]�d�cbb����6�8���O���>��>�UM�2�111IG�}��}���8>�G�'����������ċ�� �h98a4��%�G| +�����I:b�w�8�p�����V,��}��@+�����$1��C��� ��L��Z�ּz�*��50���{�������#ƾz�>9����u\t��t�,����G=�����z2�111ն��5�\���^��F� +J�}(�!JWz��7����������/��d�}0�V�S�Rfߵ�>�{��b������TKb��}�>+�l�}B���d�ko��'���H�}aˈXs�P#_�>&&�Zc��CI�]Y-��z@'];���}�#�-�cL�#�� �����>��d�}�"�%�������}[LCP�M�Í�� �����>��d�}I�Q�f�Z���;W�V��b��C��2M�z2�111���d�})3�F����OT�\�G���W���h�2 +��e�cbbz[1�� �H+ ����#�����*+}~�G<`���g +������m��'��#��j +T�� ��VؙD���P��}�M�B�|}ES��}LLLo+�>Yf�! �� �X�h�����ƆG��: hh��������b�q��"4���?�b_��J��G��c�j�P +� ��Z������Ts1�},���}B�i{Fj�U5*5�����]�9g�+cS �����P��BU�g���P�%b����ٷn���n�Mw �r:o�����T��>:������k�`~�� +� �'��>��tG������w&�M���~����e_k���a����>�#4\#�ڑ��|�o�]�����)!�ܫ# cS�c�G;&��_���h$��̾W����6t%��Cy�z;9�R��J�@�����>j�55 +oab!�ڮ�2YK�w�ѓ��k�����>���xn�#cW�}u�}�U��V��x��m�����z�������~r���-cc_�a)0�)�+6��>���C,��-Ҭ�s� xd񀷓J\O2�11�o1�} �f�P�����L�Tض�g�Y��n���5'�!'ИG������I��92�11�s1�}�eke�X!#�%MwE���I@�/��3/�77�3'Ԓ_��� %�Z���#�>&�z+ƾȾ,��} �� 2�>���^��}B�%���8焘��W>*��)л1��\�2�11�O1�}X���^�<�W9�wu �W��#����L���}����̉��t +T���q�U�Ú ������j�}*��TX���L�P��}��9 `���50�d߃����w�n��&��#ƾ�`_��*)5!�t�G=`��qErQxd�+F;���Ⱦ�N���<�t������l;�����T��W���_�h��&�P�;:H�XN.J�g��0�+'Ҏ�G������/ܻ��RA����}LLu[�}��>!��5�Rg���������Vy;��;�[�s­ C�yd����{O��u!�\cSc��'9�;�P�mXe�j���3�h)N4��܏G�Q=|�8�j��d�cb�3b�&�*<�kW���>�P7�`W'A�*�����t#� [G�[��Ⱦ��)�k{Jӓ��}��w�I��*�c���W���\- �C�� t�t +T���9��΁R&n��������DS��>:~����9`߂]�v�m;����s)�?}�^�xq�ĉ������Ç?y�~[RR��QPP*<��ٳg���N:p����)OO�]�v�3i + + + +���/^�XU$������;v��g�������Ѵ}WW�-[�\�pu��#G<<<�2�)t!==����ҥKdɣG������燅[�nE 40���۷�������^�ʭ��G�E��76���� 4�����\���(//���B����aaqq��֙>"1�I�}bL����{8�˾���.�OݲeThf���W&Jt/^����A����y[�{�B�����mm���۷hѢY�f������jժ�w� +aaaM9"���?�dٲeM_�g�}fbb��o۶��������lO�2����G0��`�ܹ��7o޷o_��.�F�I7�8���ˮ]�nڴ m���� �b� +�;z��^�zݸq����������h���y�f +�3g�L�<�u�֤�W_}գG��Vx���������tC��Ã1cƐu���-[����?~�S�N+W�D�He3t��~���׌}���b%`����hy��G�u��������͌�-��9�mX��U�Q!�� ���۷���>���v ����2��'M�t��5� ����@$�O�>@ طp�B|F�B��c��iժU� ��Kx$��l�7�|�gϞJC%�~�[D$�^a�!C��;w�D���ٖ�����_ZZ:mڴ�m�Ѯ��Ġsss|F} p_�xqrr2����†��� �\@�رc�;wѝ�۷�{������f͂$m�'v�֭]�v.d1�Ç� �2,!ǁ�_�c]� : Ϟ����fff\L3}tb��}(I~�E1�K������t1�ryf)�7�%�Xl�}�)�|x:$�)S��>}���{���իW�^0;������D���|�����X���_�C������T�>| �Ž�0Ο?�%_����� � ��G�lh�I�&��Ƅ)O��[�6m�-m}�x�?���������7o��� f�^0}�b��u�R����50���O�ٷ�pTt^T�h����>�y~͚5�� �؉�C��n�z�ĉ�������mmm�>P.lk��������q|������V�>��KAA�ʕ+��E�RYe�+��STTl޼ybbbIII����H��{��K�.�3�z��9a��5"` ]1������sgYń� Ξ=����~[ +�<(�q�F�+"kkk�������G���l��cc����Rr7���9s'���,����!�� ����>ZbE�튶g�#�u���ٳ�e�A�+��.]����?�}H�ׯ_�#���O�92���ua-�-�>�8��cǎ0Y HU�C}33�V�Z�sp999m۶����ʪ`���xyyy!6xUln۶md�������駟�R%G`˖-�|� ��b(�B����7�����;4�l�2�úK�,��OR�e�=��-ZL�:U �L��d�}��sG����غ(=���N 5���=P}%��#pSV�kb��$`=g�|s��a�����Pƌ �4l���/�@.|�~�z�p�>�>�� +��� +\�=y� oڴi��������?@7 ߢ�޽{���Ν+��}�",(�?sss�ƍ/_�<|��#F�d�N3Ю*�I�}�C5{�h����3��}��vN��g�={����xRPP�y�&]&�P�S����3`1r�H�nÆ ���111�*P¾j�����k׮�Z� +�8kٲ�� ���������/�&bpqqAS��#w�;?��3H�%u֬Y ����&�����`T����x:m�Ջ����o��655u���_|�b�7w�7�w� +V�u��4�H�2�$�X�f �BCC�����,�!�5�y�ԩ� �xc�l��C@�J�@��>u۽��i6A�o��G ����BM��-��e-3f�q����C�!��g����jժO�>���y�����>�)��k�>�y���^�z5h�>Q�}�jҤI?��SxxxB����{���v��U1�!Nlqذa��ՕB'::, �]�2'*�n?��#�����Q�M^^��������ķ0�����ϟ򢢢������/[� ��n4���H| ��x222�w����Ĉy���p�***�C7n����W�}b%_`��k����ϾWN�Z d�}��7�v�7�۬�+�*JP��5�C��t�_��G�L�2嫯��ɂՂ���PΞ-��.T�|*|�,����:1���֭[۷o���͝����D<~~~@ �װa�-Z�} Ѣ)�PnTgΜ��C<ӦM��)�^� +�����.1���O�H�F��! �"����N�:�^���rrr�UL�9%%寿���}l #Fod`�cb쫊��:& �,����f�[��5��#L� �7�����Re_x��Z����������+! � X�v-�Oll,w +4;;{Æ {��-���޸q����_����6++�����3�^L��p���V�����bgAXl 1�����G�E�Q K`��bC����:�ĺAAA:::���...EEE�;���������۷߸;�d_�}�@=�\m�l�7{@i��| +�}{]��l�b�ŧ@�; [�,��(x[X������}LLLo%ƾM���^? h/ �e�kx��?��[���r���O +0�111U#ƾ0G�Dĸ�=�4�G��̥�Q����| +�>8�ȗ��}e����J���z������$)ƾ����6������ܵ�q�gF��4 ���O����Z���˾2� �G��;�8��)P�>&�z.ƾ��� +TF���(�+�p+�t�����g�g3�O� �����c�b���������] �'� +4[���qd+�p+c�tC�t�5�E��G�N�0�IV���#�d��Z��)��GTξ���_����G1���hn�(��3�J7�L7ؐ��R=��>�iNc���� ��}D/�>yz�DH��M����0)�����ţ�_�|�ޫ~�ԩ ��nܸ���M�I*���ʢo[��u�d;����G�� ��ӧ��ܿ?//����6�����A^���233O�8�f��}��uĆ��rl�JFu���s��Y�ýkݼy�СC��^�=�����h\�9f��jb�Ђ�m}�AG�Ee�[&��w�č�����}���Љ�u�M��ßd�qou$�cpHװs0m�`7ro��h�رc� ������iBF��!Bk��e�����JKK�DˆU58� ƾ�%�h +4K�G�'$�Z5S��d��8��Si^s����˾2�|q����n���K���D,��g�#�._��{��?���o����?����ӷ ����u�*�����ɉ��N[[����_%''�����w�a<"##�!�8M�v��)���ѣ�.��_�u릤����H"�6l��\��At�DR&Jћ6m1b��������K�.s��E���pgg�N�:M�0� #0d�ɨ���}�����w�ؑ�q�M{zz" ��a�ܹ�3�-ű������jbb���o�رb�355EW�\ɽ�@�>7n\UϵMV$A���/)))$���X_�L�r��%THOOG$���bLL ;���vРA�/��ڵ�… ��ݻwoԨQL��&�ܹs���G���5t�dpfϞ]RRB���=�l�2�Vl[8p �Y��6mmmݫW��D���W�Z%�o�g�� ��3Μ)��Hc����2�>��v{���q���ѝ�������۠ޓ�엲w\��t�駟"����!�4l���ʕ+e"Â���gii��v�ڟ~��iӦ����Z�6����+�9ھ};VD����o5j��G��������!!!ba$&&�&�accC� + +BJ� Al�x _Dؿ!����J�[A�?��#�+� ���Ƕm�.]� 9��M�CD8x�`���'���ŕ�r������b�Z�n����?������z�&MRSS�D�eÆ X�+**^�Ǐ�}��`=A O��h�xD��S&.��00J���P�'OFzG¯t?b��� H #z��_��矄���Z���{*<< †��F̚5KUU�_�~hg����4��B��۷WQQ^�:޲eK]�2��3Ç'o'$��`ף5r�]�݇*;;; �2:�!"��0,�o���p����V�f͌��0�؃ؖ� ����ڵSPPX�x�?���III�}u�}B����J��'�� 7n�W���(/�{y���E�+������O��J�}/+T����V�WHDHS1 ��=����`2���2>� +999mڴ��AfCj��o�� ��*���@ճ�A��R&zC�G-L���b��gy:Y�-���AB"�cǎ�[��w�Ê"?��d侠�Epm�'d  a����H�������Ì��`���ٸqc� ��lW�}X��������;�dk�>�$��|}}�0���}���D�,YB�geea�c�8B�`�諊�088���0`� 8ք}�}�������PGY�2~t�_�~�} +Ԕ���׸q�F )~�40t�<2]����ا�5h��}�d�}B�~���v`��R��<�g{N�Rc�l +�䫯�B�˨0HV8?q����7o�\�z�����B�._�L���fΜ ?չs����7���9s���\J`'�>D����,���ه� +��}XR���� ��3l:** +?E<�䊱��/{�"�o�ʊ�����2�>Q�.4��)PE�ه�⒈��=N������Ὁ�I�f�������D"�233%}_ZZ�բE����L�}_!`��e��@�ˑ��a�D�#���ӂ��+�kii�?!!�z��>�&�HU50 ͛7722BV߳g"��Ԥ�U�>p��ӧ�d�Ƒb��M�> �SMM &�����eX��A^3��IHe��>������e�Î�{ +cK�m!<8Dl㏑LII������ر#\��5$pX8jq����U�4i�/������%������?�!C��F�fϞ�C&��'�l�2zQ���aه�_u��M����&�&��^����>Z�hQ: �c_���kkk9{yy�e $�իW#�#���}�o88WPP@�B:E�INN�ZHMH���� +I$��ihh<|�P__i�#�Už� v��u��ŤEEE$ز*�0mذm�`���(��߄F��Ǐ���ə+��  �# $��ٳ'�E*T�>T�믿�u�&y��#G�Ԥ샏�q�X���+V���C�#�~VV�(((w`�����>� 4 �;bĈ�-[N�8�\6I؇8鞂D t������;#G��^kڴ)\UNNΟ�����8J�:�# +�aÆqm�ɓ'a��ßp�l�…�;w�׬Ys��]Xc����.�� p���\z����#ٸqc�^�p��_����e�U��}�����( `j�ڤ�`_�̛>����Ge�}�����:���؇ ����{r M�7Hz�ƍ�i"��񶜜����W���E�o<�GN0!1�5P y�=�����;t而��}��G��>�~�Mmڴ�o�9]�n���ͽrɖ�X��S^^~�H�$ȷ����f��C�:u*��7��@����P��30��{��]����� �B g�F a`��c��� �. <G/d�Vs�C�^�j�lpp0�F�d������ +�`�^xx8��P��΋fdd���wB� +���=�x_�E�]�z_��#~�82���g@�X��> 5�N~-��0Fa��H-#b�=�����K Y�U���>��ӏ�?:x�,�����=O����+%�%K���p0�L�R���x�'�K�ɜ'�رc��I���@TC����#�5n���=�ƭ\�����>��ѷ喉�M)�T333�i��v�ځh"��`kk 7z��-MMM8Ydc� +!��tH��‰�ه�RRR0\��qqq@r����)��0A�Z¾2�iE�@�aTVVF��4 .�}���R�9���Ht�@ izzz��b����̙�QB������f#��{��*�Yh,�>}:���6Lcccr�$���ϟ�H�����؇#r� |Y�~��f�[&:B�7'/p$�[ٳg~o��L,T�>T�W�������ǀ!e��}��V�|�� �� �`���%C��l�Ұ����Qx�H��n��̾2���4h��S����?�!2�(y� W�?�r����‘��r�}D۶mC�E�U�y�M�4��C�aH7a�,D�H��… ) > �駟v��V�TUU��q�l�����_X�k �<� 9SV=��*�BT�5�ҥ ��D����>8.�H�ӦMC�>�k�������� �̂�}����ZY�|��6g̘AF��������C52Y����^;|�0z����=��A ����8���u�e��2q��U�;�Qx� +��+�|�tqqA�ZZZ�T&,�3� �,~�w�5���`��Iu�����Q�Jn����]q$;���#���戾%��f +�/��jk�l1� +� ����Rc�gu�����ѯtǔz˾2�#Pp���xd0yyy8A@����3gδ��{�����@ �E"o�ڵk�ĉh�V�g111�V`����A�d;� #��HbuP# as����� +�8h� ����ɽO*((X�v-�3*�� ��޽{������ܚh����� ���8l��� L� +蝫��СCA%�)m���>:naaA� ���/���� a���# #�HHH�N-" ��a[0�8���viHyyyd � �Z��;Ǝ+�¬Y��)�LNN�OD�`p��j�x�]@���'r��8O�8���{�ϝ��Q�~��L����KKK�O�U�LAb�r��J/!���"�?|���ŋbO�np�/�pl]���L$�.P���B�C^�Z���J+����H��w��OI�c�kl�L"�Iq@��`$��UU��hA�Y^��:�aķb�Y+�9Ā�D�Ol|�[�!����3F�`]�g���"��x��p�-��ұE���bH�* ~ �y�����[�Ag��_�����q�&��#���#`m=ݺf��09h�L��!7[l�d +T��{q����I�\;{�XV?����T��'�,6�.з��ߊ}bS�I�2�� +�DEX�yۄ���ʾœ^>?x�4�Ѕ��b�>&&��z�>J�"}����#05D�8���?�DE4b�i#��D_�,�|w_晪�@�����V��}Pt(���MkK���Η����ʧ@7G�o +��}e�'����p+%�c���~�� �#����|/�E�=�m���ΗR:�ϖ �U�X8��;��?~�_p1&�c������b +Tt(�컒�eG�\cGw+�`��Dnj ����K�W&��t��ތӱ� +�������OH���(ƾ���@ ����50�}Q�Yxx��zٻ�[���cbGE8t t���o�;��?yv�����)����t�������|����^ľc�OOOz��W5}||��-[����������� r�'uW�:�dl���.D�����b��u떗�W`` yd�+WвX#d�ЗP�0J4������������;U&�R�ѣGX1sc�����p��|��5� __���XD�%[ƞBe���M����B�砒�� |��~�M+� �%777�Inn.�W4!!!88��&���0�b� ·bG#x��}�Mii)B�^��2�Y4�NaC�6m:~�8� ���H�Dz{&/b�; �n7�@�Yz����6�2��H���(�B}2�be�}e"x�����K�yZ�i;��2� $��={6��g�}��W_ 4��M$�֭[�����H���-[�$�jMNNn�VoժբE��Md����N"//��_�7�#)m۶�G�_~�%Vo޼9V7n\aa!w���$Be��|�����C��� A}%%�o����,A�;v,G�f͚as:t033�f�cǎ 8TTT���V��G��!��w�������:t�����2c��݋%������\�QB$�;��D@V�ر�y�pҿ������h�={v�n����iӦa?Z[[S.l޼�E�� +�(~�嗉'�A�z��l�p��a��]�v�&���w�}G:��/E�t�B����y����lc�o���%K��G�o���]PP�v�E�, �$ nӦ �(�w��8 + �� ��񆡡!�|����E����8~� w��]�eD�����ժ�*!���ar#|�=��e-��^&UL�J�}� Q�n +� >�ᗖ��;���߸����e�Y�`2��ï=zX]]yi�ԩH�0&��oiiI�#����#-�'w�'j.]�F ��޽�@rKKK÷������/�(� ���?�tpp@�ݵk�����?���}b�X�THYHk��Q�F� +�}�O��I�� �`���=������d;lذ3f��`s��ۿ��;0{K����I��Qqnl�,���À4n�XCC������3�%OP���"�o�=~����A�@9B;�X9k֬�W�����F��|JV��#a`�"B��o۶-F���2��QPP�� `�~����Oh +�1d����'V��MXX� sJw��NQ!H�`455�a��̙�|�r,'�+�Y�6�z�� �,ƷӧO�1y��������������(�jf��0 �������Q�F�`b��}D5`����ܹ�!v +~��ƫz�t��WES�5= XK�#��)P��w 4'3$;3po�Ob��fY`�̊0�����'L�����[C�ѧI�_ '>>���CjE���?��:6��~:qZU�T�}�~��o����CCC��$e�p�B�@�V�Hm�e��:�<�k�:B����%�IEX�|�S$+�Y�ntdi1���I�#�>�|�w���S�ѩc����3Ө�'����s��;�s��dH�c4ާO�e}��2~�x����������ׯ���dGc�&>�8ؐ����iNN����_2� &�h���ў9s&y�}���[�w�cl��*>�}b��]�}��šM���(M]< m]e�}�d�de����c�J�Lt`g���?^�����y{{#/�=z��8q"I�od�֭[�4i3�}�C5qv���,�B�є�}h���4I����샙EZ�Ğ�)&��:8;;c�#�ҋ7�Nz. ���`�����D���������������)�}� ,��L��a�f�f���+W����Ds��'Oe�.F{�ܹ����XB^C����.����So5dF���0�B+555,c�t���8L������:ъ�A}���I^"1�mp�t���氣�Δ)S0�%lݤ���Z� +�4[�l��9{�}g���ɻ yc� X$��}��&��2¾W�۞���')1�!#j�۷Gf�W5a_�ׅ���A����4n�������@I�b���8�KJJ��������;e�����Sl+� +�̑�+�>�#G���Vh��񍫓��6j�(�^�>�hѢE0Y�.]�)��.�@�F����S`:�ܼys��V��(a��N�B$�����#G���������'AL�ه�5�"c_��V+**b7�:���ш��&������7 �>t��-𔼄���u�%&&f�޽�}��1czl�ҭ��u#S��QJb���e5���O�yٹ��p���0p��(c_YS>��3X*��_~��ս{�&�����^��%�PW�YM����SSS�TqP=��V�Z!�od���u�ֽ��i���Ja" ��1 š5�Ǎ7"q�effv�� �\�܍���f����-'M��������8l�0�퍫S�|��v�؁��>�� 맭� _L�,�>q޼ya�ˁ(��*�5x�`�p���W&z)�k����.��1t� +~mΜ9&L ;�|�̙3M+�� +�a+��#:�|``����:}���=<<Ј����.W��oÆ t�0p�e�a����K���#?`�=]�t):�ÕJ!K؇���������W����W�"zhUW�J�}«@ݽ5���5�$+� �^#��g�"L��2 �σ����H~HNN��AV�i��͛` +��sA�|�:p�H7�l�F��;w��~�Q:y�(��H�b4|��>�^<�I�m۶H�)����ȟH�����K���ɓ+V4l�y+"÷i�P� ���W��>�ծ^�����@p�-ƾ�������a �z+��&�*((� 8N�P�ۢ�8��ٳ' ;�����au�`�&���ٳ����;�Ì#F��'�;�+y�gM؇��'q�ߠA�)��K  IG�����= �\PP�<���C*,��] ,�#9@�z��;�pЅ��gq��o18�O�>`��u�5a XQQ0�a�a|p؆�Ǡ����� ���!C� �y2��;�L�e��S������h��|�f��;̴yg�+e�r�s����WS�������Kn�jڴ)R�СCI"*�8߇ �B$r�r{vvvY��>�E�ź3f�ݸ�@Rrss��ARjԨ�Br�&�N�Ɖ +h�G���b�+e`www�C�W&J����{�Fz�&�-���|����R�C^�n�����'�T�:L%aH$�@ ಏ� l�K�+=$��ѱ"�)޽{���D� ����� �9�����a��F<(O����C͢�"�z$/[o׮���6��͛71Vp�� | ����+Z%���w�MxF1�a���b�t����Dײb�a��6���n���`<��Á�ĸ�cU5�R��¾cN�%�^%fN����;�������"���>��p��n� ��#e��~��N�\';�[��D���#_Ѽ'�3g�xxx �lܸ����^)���������ț������� +�v͚5h�֭[�cddĽ�G��#r`DD���$NU ���!��Oa��6v�'''�LN���N�B����PSSspp�� �������К�>��'WlV�z���8�H26b[��|�n��8��>01����� ���H��'�`��ez�A�%�W�F l����t�]�֭[K�$#===�йnl�r������� G�"��B�&�URR����F�C#'�a�q`������@�����b�� Fc�aׄ��З�" ]Ck�`1#�m۶!l�C�dvk���J�Dr! _�h�Gݟ��P�,P~���e(�$������.�7@)��;�}����)����O�111ɬ>.��"�� ������2�W�2%�� ��9O�r��{����[�//���1�111IG#�����)P#���*92�>�?����ʦ@y`_ᥗ/��|t�卸��}����x�Gʾr��)P���/w�"�/Cu��#���7>�f��RG�'���|������ _^�b�cbb�W/�^�����Uؗ7d�kU2��vF���>J�<_��z2�>�^�|���헷w1�111񨏝}�M��:��>!�֩fo�;��'J}GDMX[���6�c�C�@�e_���\|y#Vl +�����I:��{u ��;/S�b�ˉ�����ڻ#7��>R�������ɟ9���������r_^���}%s^>�S�ZLLLLﯺľrh�Sb�Vbd�;���bt�S��e1���FE f��ʄ��9��˻/���b_��w��+i�G���kзm��}�v�r/^�(** +�)00099�<̙(--�OB䵡����?t���eee����R�v��)����srrBBB�{�=z��nڴ���u��Tۿ?}�������͛7s_�@ޅz��q����B�sn�ё�� z��7�d��_� aaa�7�"���|4� +h_��Btt4w0�ky� ���R&�JU��� +� ��@�b�{�4�e)%��5�D^&�G�R8z����g�_Wx��L�~�!!i���~���M�4i֬Y�ƍ[�j���@ވl����D��>��Th۶mDD�M{�ĉ 6$�Π"/^733kР����ɓ�}�*jjj���.ɜ9s$����&Oquu�>�H�ڵ�)S���N����d|e�?�Z� +p5j���cdd�������ɣ�a ��'��u�V��}� 0��Q���ʨ@��={!�;�[��â�|O�eggw����C�w��E��7���p��355����}h-%%�~��_������Qm���?��#�ܷo�����7ߠM�?�>��Tf���O����hM�W>�E7c�֮� ��e���{��}���q��>Y20r8��ŋ/\�@�)A��;!!���رc]����)�"����?��C*�������!�X=���$_�����u��5oޜ>��޽{3g΄�"�h�"/N8n���a�ӧOz�� _F���@-��Jl���JʾӧO>��E��H�F쫯��={6�C�?�8����:�}L�:�>r�λV�@k�>.�Ei��>�]�Gw�N�HK`�F�ȸq�ڷoOL ]�ϻw�Iٸq�$��D�f�[ + +"�LLL,�1>e"��w�� Ennn��C$�m��ɓ䩘��ɟ�9�C�������O?�IW�z�С�>hjj�������ȑ#���ի/^�(//G��DT=��"����z������x0��2���KKK�q����ͻ|�2c�{�γ�5(|h�L���(���ܸ�?���M�W�N�����}� +������|�18h�B�׺u�͛7� Aqq�ԩS��g��a��{ +�������M�zzz���Ȉ#`��?�G^�@Էo_b�nݺ%''d uuu�u����+� ���L� +v� Hޙ}XHjѢ���QQQ0h�}�q5�;q����pǒcnkk�hccc���2񊊊�4 �ݻw���G��}�*Ps�>��#E8�q"a$��C���ٝd�-��OL7oބ���Į���������z���>}��=�B}$v����C��������T�M�6�  ����a0544Z�l����bŊ���_���J�o���:xyy�9s�S�NS�L�z�*�h������֣GP)==]x�Q싏�Ƕ|||�Bt����ػw/���Ŋ h��6l9�YV-�����0�^WW�^�B��x0\����}D>|8<%%e���}L�zžW�����}g� ��3Ic��u�}��.y�-���B*&o&500���嚚�AHHa���)))�������ܹ�#���|a_� ӦM�@��:�Gt�ʕ�]���� V��"Ӫ0z�۷oҤ�O?��A�/���� ���}0w�O�N���}�]QQ������_bC*��`7<&�_��'�)�4�{����8��Z8~@.��Ȥ�$�jut����U�����2��X��Ic��w���ˮ���e%�SV���2�]�~�L�!߂H�7nM��KKK+=�����C���������\���Π���_ 4��}�V�*�r��`hh��Y�������D4QSS����aB]9�۷/�D��‹�6�={v���1c�4o�< ����w�ލ���S�0aF�ŷ :N���رc���}4�c� +�M�w�܁��޽;Ќ��R���.����%�N0�1���-�� (|����높���/���ߊW���qd���$��8�>"�$[�<�2���mۂ3g���ˣ��U�>�sd�~YXX1b������xU�C�Ν;s���Р���װaC===���_~�e�…@Dd������,PHۄÂ�JLL�ٳ'"�L�5k&''���J,�fnn�m�\�f Ɓ\T=��D��h�nЇmĿ$*:t�}� {�F����Y�g�m���������kP�:$gY`_�����1A�Q���6^�������.���93����Y�t��֭[---�������҉�·����|�6������bbb�'� r_ҍ���驣�#9G*����T�u������=s� Y��� �ٳ��,�2�5<���!!!�& �X�`z�h^���L___ ������Q4`��ڣ����c``@�y��-777z�Ã�E###�-l��� flۂ;憊����p��nb�I.bbz[I�}w}����_� +=��;�^��ѯ�����a�����#��/-�9{�[�-F]"�;��@:��G�!����%X.v���# �)�"��V�hUU;X�=_F��%�)� u��c����n�"5���VĖaE��lB,<�X�݄X�dCb��%�^31����>R@���a���'�߫R�:�w����������}sv���7�g���}�@X�fS_���vA�X~�[�QK�#� ��� ����}����I�s��Ɍ�ّ��}LLL��e)w|¯z��q��)�c�Y�65��=�QD�p�@�a_a�pr����gn$1�111}���QxY� `9-]K �lk�}��~����}�~9/^�?|1g�!�>&&��N�þ�)��snA�w�n���H �r� x���>����;W��}Br +�����I�%S�W�^� 9+;S��*Pk��*��-M�����u4��+#7>yv������50�}LLL�,Yc߫�@}�/y�Ȗt�M���>����v[�����2�G� �Z���������=7��iv�1�lI6�G x�+�[���X�된�N�¾.�ql�����Uhq8�7���o?,�u<;2�w��l���Qg��I��\�ׯGFF�;�믿nڴi�.]����;Fn7{����ѣɫ�?��&M����_VVVW�\��������;��������߿�<���?www�w�=z�h���ܧ��)//��_���JJJ�˟>}�%ZZZ��g�f�ڴi������޻wO8��޽{ѢEX]��_֭[����nݺթS�� ���Ћ��4�x��]]]t��>úӧOOHH w�9sFCC�C��z4k�,�B�����_|�ZC��K�Zϟ?/((���Ʀ��0aB||��۷��CM33�ƍ��>���o��vΜ9$������8���c/[���$�^�jz�M?~�,�_�����ݻw� �.899qo]<�|��ӧi��N�B0�bϜ�P˖-G�G������g�-�v�z��E�"6�=E�4�A�@@��=z���{J���4uuu�"�`�9BF����aaa:�n>`p���Òe�q� 9��?��$S���ȾN�q���7���9 4�}D�)��� Smw�_��Fo^�T���|�rd���#-Λ79�_�~���}���� +�=*:H�JJJȁ�};v4222����3�B؇<Ӯ];�H��7�u������+�f����vȁ@R�^�ڶm�`������!�y`���20�TUUE$���I^^��ɓ�}ݺu��ѡZXX 9#/Y�����r�G؇��ĺ���k֬�������s����0���9G8���;w�o߾��>�Ďþ@ �}H�+����#�������s�G�A_�[&B"�LLL� P��TŠ���R� f�6~8��>����n�㏽�i�&B"���N������&�vɠc�a���a���M���1�������`!��r�J �A�<� ��o0t��:�8�(p���ɼd�}�����"C%S�Vn�6@~ٗ�u�3��g�ߒ~�W&�3��>��=s������ĉ� �g�<�s��?�ZԬ�a�Fu�J�RUT��ڣ���]���UT"�؛P���j�{�DD���y�r~��I� �������}�=�{������s._�:-� +�(�k�D@�Ԥ3s```��:;n�8JZ�^��_~Y���p����O����<�{< +#!8<���q̾��3����r �1�/� +����Qಬ��B �W�����}�k�N� &O�w�K���RDa�q���P,�hÆ � A4///�1��r���� F���H�xp�����8y|�7�|s�� �����Y�����*a��p��E�v�2c��}�b ��x���`�X�\�r�U���aj֬Y��_�G�j]�B��m���xrb"�i��L�G��m�"F��,1&��͛���6�IӀbL���������r�uT��駟2�0s��߿���}�����S�O<[�`A�IHȞ��Å}�%hYEq����>�Ns-��}"�N�0�D�gڎQC �n����֓�>�z�}�����&�����:�������=&��%�g�}2p6���0'�+�V�����W�L��ٳ� a^W#�)R�_~��+e ��;w֬Y������^H��F�2a�'�|�p>��q���N|�k�D��}x����[W�p8/��-<�)*�֭;}�tIY���GD"�b� �F6��?۷o뭑��:�Q������e0�&"���}r�HV��K 'M��R�˖-�)��}ɓ'�rq�#a�ùT4��4 ��@ZPn�:\�1r�����+T�@sT�X��b��ma�/_��(� +�g�G� ���Z��;Q9�2��;vfΜ�{��BI��%�[�V-�D�La�k�%���里R0�*�_�FRN�6m�d���C�����8�F�z�mr<�:�{g��;J4��� �i<={���2��>��:�3�����o� ��իW�v��Ѝq����a��� �C���A׮] �B��9r�.\mذ!Z��\=������젹w�Z�@�F����˗/�}IA�(��Pcl�$��+W�ԩS�%�4�-�1�ֶa��%���>kF@��3f̸z�jYC����}6Ț �>v�4~b�D�ӧ�/��OJ�%��_j[:3Rϴ,�B�-3�3�Y�h�d�E��M��ZҧOO"��O�ƛK�GO�ٸێRMw|�ĿC�m�}^7%`��ݓ~V�9�K��@��u;>�@41g�����s��1i3$��L�o�g{�^�kF�/���3���M�{ t�3�>�����le˖%n�u�69��Ư�� +�a{��&s��=�-�[��p�+�ۑ7l��լX�ח#G�b�,��M�6�3 +g��a��� G=z�h��O`B��p���;wnٲe�F��<�B�V )�ҥK�y}}}ٟ=q�9K�,��p׭[' �ɱ�ζp}����u�2�` A�P{@���X n�扵 4��d-���ST�að�Ν�FhF�h�GEY�+��(���Tu��!����R$6`0�t��M��x ��Y�f�|��5k��g0�_?�b�ο�:� �j�I�����Ye�Gv���s��'�#-�o�>~��#yxx����u��睗@5��̲��t��� ˳eˆw�����N�0�쓗��E�]�v;v찱�f��W֦M%�Ê�E+W�d�ߴiS��t+V��۫W�N:x0�!P/��2D�5j�(Q��)d�a?!d3��%,X�;wn"�'M���O8�� bU�N�� +��b��^��s8� ���/�H�T�\'|��v���9Oꙝ���^�\�MV���"����*�\�rI Ў�\��_�h����v�I�TY7��a"5fCH�qȼy�����H�dɒ�K�S۴�����;�0����T�<�b��>Qؘ1c�ݓ}���');�Pn�߷d��d�����5IY�m�Mɨ�Z����<}z +�gx�c��q3��}�����o���0�� ����}���ZM�^�X�g�}�7.�C�D+8��`@@�\#b�ܼys���\�k8�<�C�U�V���@ +�b�ڳgP��Z�a$���ѣ��o2;_�x�E�eʔ�I�F㞞���a�ҥIĘ�� ���mn<�Y�~}�.]����=}�t�3uE%ێ*\�0�� �:�ҷS�L��k׮e�B��Lse����4����/��)�m������ѣ�Q�Ըqc���% �׶��-#Su��X��6 �=f�m�3���T�U�r���� �>�L�� �����y ���>3i�$PH��նo����}t�o���.p����q�>C���>�g`�e_=�����G�8g�|.}��l��'J�]?�Yd�J�R9���s�ss�����#I���5��g�S����.7���9�a�|���A��T>6z��SM8�A٧R��5�w�E��������R��zO�E�������].�����t�x;d�ՀY���}*���Ca�[nzl ���5�`ܱ/0�=�+�=��@�}w��Y?��O���"�l�� +����!���}*����b_�ǃ��M�@�?��r�/.�w��^ӯyN;;v�Gy �>��?nv�_1$�&`���t�;�'//8�ݕ��>�J��顳/���1�si���8g���zL;���G� ��/��G�8g���\��۷v_ �w��K٧R��j= +������{sܺ@�?����"��G�"��`_�30��׎8��8g���q9$pӅ�&���}+U�T���GǾ(�z���I��f��/�=��_�/�e���5����'"<tm�m^�� ��-����i���;�T*Փ�G�>�_�%P�%��@�|W�M=��C{ +���Ϸ��ͅ�l(迤�ԩ��}����O�<�W�Ǡ]��ܼ��L�R��z=���������q�=zΌs�:���� ��ø������G��oS�U;&��sZ<`��� ����.�?�w���?����lԨQ�V�<==Ϝ9cLݵkW=�Z�n��lAXX�̙36lhݡ~���&M�u�ֱc�ڵkg�k�޽mڴY�~�͆ݻw��$ҬY3Y5[��8~�x�N���Sl�)����.\إK��ܹ3&ɪ׮]���mG5m��:��LN+V��Z4)����ոq�~�i۶m�肂�ȷ[�n����O�>���ކ ַo���ϛ�CBB��#F�Nq�`���;6iҤk׮�f�2y��ٳg'N�ضm[�G�K�,�N����c+ ���߿���ާO�OyM�JQ��i�h����⓵m.� &X�gϞm�ܾ}�� ԀuRn���F +Hך3g�m�NkK��cƌ�6��ݻ7v�: R��4̐��+�ڴi��=��ښ����K��wҤqM�p�����F��" +�x�b:���?L�:�Z-�?cƌ:|��w���X^����h�ӧO�-�OALFt]c^� H� +�NoN���!C�Ў͛7��:tH� +�Z�t˜��豱ω?��n[��z̉s��0rA@�)���Ca�-[z�m=���2��Sb ���6~R��t ժUK�4���?�,Y2�&N�8w��7n�p� $`�KN��� |/_��,� +k%J�|�$�s���+�_F2_7Z����/;v�-�p�:XB�f p&K ����kߘͯ�w��L�+& j8���Y��Æ �I����ߗ)�@>�|ӦM;�|k7����e���>��ٙ����h$S��I�����/��bDOcH�9sf�$Kv�4W2����4�I�2%I���^�a���������2�Q�ҥeM�I�J�����V3�!p�|���駟^�p��\t�>ۨ�~5�}���d*�5�_�`A��,Ag�<�J�s�^P��ˋB�9�O�h��'� �(����"ڕ��'�"?n���n���z��{�I (+���K��}�N�������|VM�� ��/� +�����U�Z���Of����_�|�M� ܑ�'9��� ��e�q81���9�8[9�]�q.����7o��i����ʿ�� �_y���a��5����+k׮�(2��"�I�ǁ֑�U��%������{��W9�+���o�_�!Μ:^p�E%��֭K� + �F�e˖��bnY� +�%��IaʨQ�H�0Q:R&�` ($G*�V��Ba�e*6�=Ν;Wʅo/S� 3������8Rj^��ի�f͚�,��.\��Д-[��Ξ��T��5�t���IJ���p�5Ѯ[dįP���';�.P���!t��N����@���i;*�~+� D�+����π���V�Z�����첄�1�������o�U�8�-!4�%s��E�@p���bmYf�Q'��>�٤@���ҥK�w��2E95���z��#�}�n�ĉ�K�Zq�>'��n�ǣK��w�C�rr̿Y��.��y�����o����4|��:��KwwwΣ p���'�d˖ �����}D��aD�r�;& ��N�~��r,�#Y�;��_�^<<^BN�{�����7؍8�\_��O +U���yQ�ƍ@�/^���} 0�d�B^�Pc?�Q�nnn��x� �j�A��@�i��i�q�&� � �ȵ��˗3�9r$@aR���+W��9sb�I��>��E��t�֍��}�9t�P��f��F+�.�K +��̢�f;M_�R%����'Ga �����dEf�I'L���1��>�n��5�EE�@ ��4h��%#I�H�cƌa�b�hݺu�UƎK��Ldm�9��W��b�]�F^���3�#��>��.�����z�m�!�e[�L_��j����F<�m;��?��W�mذ�3Hܗ$I�J�(6 +k`_�L��v��p��/�Iz����T�B��L.={�d0�i�۩W�.��0{޼y�A������M�6r�̘1C.��&O�̯����P"��c���b4N���'��.e�> �z�G �N������؏[k֬�����'N)R�BQ�ķlق���Z�ocϬ(Y�f�؇7[��,YH���}����p��'u���⺉�#_�$�&�kwe��0F�>B~Z� �O.Ѯ�AI1���mΜ9�7��_Ⱥ���y(�L��09Z�B�,�Q�G.�-�� �zf<@\���>� Ѳ��h,�e�@y�>)5� ���u&Y��6m���_��������i#��z�M����|��R@µ�<\�!�,� �,�sV<���Ǵz +���|6��z��i�>��K8:�`^��Ź�={v�a_�֭ jp>��8pN�իW��>��x��Qʕ+�,�&���g=��… �ca�J�}1�hтq��#Q��^x�� pv ��?��� �(Y�d�/_>��ܒ��}l��R�H(B�+���o��Ո�����ut��><�؃�駟p�D��KDF�Uba��`�(ɭ7W�9���t���TE��*T�_�+��dqy��x�bkAȂd1G������d�e��Jg�X�ܶ�=��;�3��QXȂ=gΜ�e����Y������>�DW��41�-�θ�����ߖۯ|a(H�����L� �'Y`����~�~�zv�t�|y�)???ڂ�ѱcG�x�Mr��` +H�r�D���R|`�Ā����-�$`\��Ā��@��}|"/�n�8g��)�}�Fܗ���������;�����g+^���s�p\r{ݰ�9yQ3��@�XL�3��C �P��U�Re�Er����gذa���J��3Ğ�NU�TI���rǍ��%�GaIvY�`Epe��~���'E&��i [���Kr�����%�ڬ��CS�N�N�O�^�N�X�����O��3gΤ}� A.��F!�� �eX�3��6��(Q"���w����l�5O�E}J��Y/#�޽{blf�A�)�,�g �1Df�y�G�(e���g(�y�Kd0�>}zX�}�v�.����CCH��@�3��XE�qe�i,�Ih ��������w +bk,�4M�4:t�}k׮ +厰�+9��|�T��D�J�X���ĉɎڣ)+W�튐'���A�1j���7��_�2��'��w^� �G�?�'��D^�2z�g�}q��?�к,)cNNj��UN:a��s�� Gʙ.田/��}�>�����8�.P��}�$d1y�d��.X�@�l,��� �/N�"�3�-������\�t�˒E ���;v�,z�O�f�8նm[~�3�c'S��O>1�:���K����>R��? �= +��f͊ӆn�����@ZG)��P���Ϻ��aq� 4 � +4"�2&!�#r�6ŗℓY���>�w7`MI�~���o�.]��8���}��jժa2������^�:u�>J�C��#FXS#�"h����}/���ȑ#�.]�T�9q8/<�~�τc�?��Q�����"S�O����Q���~��\��4�d��4(=s„ 2db�u��qRHT�p��Q ݌"PZ�X�bhS?�Ft�P�Tir2ʚ�����h���7�-�'�,~�����>����C��3��g�} ��ww�]���V��PhӦ�\��~;�@�~��7ָ�d�e߾}D�[��G�s�T L�N+ زeKx�� &�m!7��СC�T ��3�w����u�)阧G�g��EpM�լY�Q��~._�ܴiSj���k��' Ǔ9J��>��;vX��zɎ6|��F�J!��������TY3:bDDy�����H��l��>��}�`�?�GűV�Q����ӎtTW� 8���)&�./�ЬtW &����'M��D�2Z�WS�~�fA___�Z`�-a��~�@v�Ȱa��� ��Z�zu���ٳ������.���iJSFr��Y���x�>�'�}C�ǂx��W��'bx�q�O>�n:Z�sS�Q�r _�t̳�>��my�_ލ�����%�#����f[z����5�?Ź 7�;�� ��AJ�(���`KjŊٲe�ի�قU� ��Y��̕+�-�<3�^�Al&Y�E��w�� �E���"�Y�fY��mH�p�ԩS81k�@Ay�d��75j%@ _rǏ��תU�RR���p #�����p>��) �(�X�]�V�i jǯ��$�0�P����%�ƥI���$Ef;�ȗ/�y/�]�T)r!bbgOF��v���p�r�lQ�t����Ԙ*Ã� &wQ��٭1�dDE�&�Gj� ���k�x�����iZP*��Ō3�[B1~%w�2�U�鴦�E�'O:�?c�={���P�PT�-5FMp�\����!]W�E��� є`���@��#)Cj�H���ƈ��hZ��I�v�9?����+��B�,[�,����JmP��ϯT�k�����=\�� +��=�/�)�U����30�s�|��)�8a_1�M�Ǯ/�n�R���K�A�9����g�ҥ i�̙ ;%��_�i P ���J.�L�0a�?�r����P�*�y�L�#k@dr�뒚قwed �ȅ���\�M�&חȋ�a޼y'N$ �c�������퍯d$(�X끈��7�@��eժU�W�~�����D �XO��� ��o�sg�f��{��m�;w��_ +����O�R��_ၷnm<4�?^��~ �sn� �����{�����Ϛ���k��gܲOy;8�֖���,07ﲯh��� s�R�T�'Ba�a�Z}(h���ƾ�p��)и&��}�?k��d�����?n��}��n#��߹x%��$a��/�O�q3����%�?��s���ӧO���y�y����S�N5�?�}˖-��mXX��������u�͛7�1$���k�eQ^����:ujʔ)���1ü���؞�&RsD-պx��I�&͛7o�޽�*4G-Z�(ڣL����k֬�ΣG�ZM�f͚e=j�ڵ�U��ܹ�Q>>>'N�?������۷oc�… �Ϻ�}ڴiT�mfE*��J���Y��*�Hׯ_/�� Y�>��>���C�M�0�&x��*�TW���������o�3�M����@�����'����wz����o�����J��As��� ��!�S��g�b��|��4iҼ��+���w�}g^�۴iS�l�̺�iӦ}��w{��y������~���ź�%�v��8�ڵ+C� E�F&� ���ٳ�/(�<0Y�d1���#��A�%J��O���[��c�dɒ3f��t���̙�C�p����� �QY�f�(�;d/X��,j`] w���Ba^ժUM����~���?$Ǘ_~9}��y��8p�kנR�…K�*e��c�~��jժٖ�����+�Ex�7(\��0x��d�O�& ��f�������ЦM��N��J�*Ѿ��R�s��ܾ��XФ��}& t_g�?�qލO�9��G$�5|,�E0X�r-� ��J��0ǎSg6lX�lٷ�~��N��1����YZ���uX�u�V2���v�*�o0�� c��)SR��n��#G�O+�?�>�d���\�*��>�~�R=*E>@q�Z��CA���7���E�e��s������q �1�/~�����h�m۶7�7� А�/*�[���X��ŷ�b��ꫯF;��7E�2�#P��Jba���w��BY�U�g]��`���AV�a�>�:�*6�,ʔ)�'O"8¥#F�J��n��\�� �cR#�ۼy�L���Q�Hݸq#� �|P�z3�F�s�^��T��Bl�2��'JT�xqk�(�*���c��FR���e��L�i��P��,E^ +�}�b�����>s 4�}]�ŏu.��'���n׎����ckt|��7k�,�c�����o�y�;�ݕ}8���syxx�ew�_0����O>�,S�L���?��\� ��}���"Y��_��&MJ4��?t萙qK���;�,X�@Y�|�_�%�1`���3f���_)&@�Pff0W�q�b��u�F�f̘����>� +���7��T�����k�]�vo���@AJ�,�yD�T8A��~��+����hѢD�/^l߾}��魫�TO��&`��頉[����-��@=W+�$�������悿g�}��hW`'ܫS�� /��i�&�聣G��w� �o��ͺ�*!�Ν;Q�:t(����}���\~��g��%L�0�E 6�'!u�Z�^|��ĉ�c�z��¾�{���dɒ��R�@��~��s�����Ӈ_'N�(L��k�,Y��K/}��WD�D��)�%>�}XkV�E|ǒ�d�wssK�& �O�XEl�l۲e Ղf�OW�q���;�ҥK�^*�2��'^TO�"�#��\ �c�]��Y �{�E�G}�^�3�[n'��s�S�'���kӧ��7��]R�H�g�aʗ/_� n_}��q�ߗ*U�^�z�%�;���}Æ ��hkܸ��ɓcb�<�AP�/�%"ؑ5X�}�E �:��޽{L��ʖ- S�}�3g&�5G� � _��� �d �0�L�2�&ؗ%K�C�ؓ={v�e�8͚5c�@R�������.J|���}d��F�2������D�"E�������">!�z��z`���?��*�v����{�<����ƀ��=�%��f�;�K��v{��=��#ry��".^�h6���͟?�H���?'8����;� �߶m[⾣G�/a_,�� ��^�����C7��oL�+V�Y4j���իV���b���m�f�d�R�J����c��G��{��X��)�¿d�x�b~��0�դIґ�z���1`˖-m�v��y����9�o�Pdz���͛7���D֙��2eJ"�Ν;��@W�1*�ZPˁ��_�1��}���>˪R=�" ��י� ����w �mm�\��}r ���������P�ݼy�_�~��O>�d���D������3����s��jՂ2�g��(��?�d��y„ �c��I�F�X5s���ɓ˻���G"nnn�ھ}{B$�]�vm�ܹ��gΜ��}��.đw E���%mݺ��k��g��͡��[�NV�r��X��P-��O[�hA�Jl�/��4� %��ŋS��c����c'���O He�6�h�ʕ+�:�2i�$ڴhѢ���~��Iw��,��5^���謇 ��g�zz�ۙ-5��-�̲��|:oO�@\��/���)+k#�g]��}"�׊����}'����O���}�~y�=��˗�� +����/���� 6�����EP5A������˕R���Q�ӧ���� �����U�Rla��~E N$$Ł� �z3�&M��R�J�/�>S4�;�Q�dM +����֣��-��y �� Q^ӦMm���3�xyy����R=��|&�����}�yn�o� �����q��l���}�1��1��V����{f��p��M�66nܸe˖��xȃ�j� +ʸ>)A�2{�� Գ�~���F����-[f.HOO�f͚�����n�ź�+�b�ĉ� ��O�>��)ґů���Y�:w�ܨQ��]��K �8 +ڎ���-8p A��߄�ؼq�F���K���ϟ�Ԓ\��@�ϟ��?�&���rE���۷o_�z�ӦM�k[ؔ�1ƘפIl�_Rik֬�b��@�s�()a2�'�1cƐ8��/&5lؐ!�-�#n۶������y���P�%Ы������o��_(�!��E�~���������e�J�R=��F�����%�����g�%�x��;�|b�����>�J�z�q'��� �3wyl�?�~#W����?�/|g��O�`*-�z�ƒ�=��R��T*��șЮ�����W��.������P�G�߾��� ���ܕ}*�J�dʹn���A�V �-�y7����s�>T��J ���o����>�J�zRy �����;���O'7z,� (|�[c�o��wv���O�R��TE�q�V���xL�@��v ���O��y�K��}E=7����Q*�J��W" ;}-d��x���0rE�����ه�#"��n9��ĿN����c���7��{ǭ���:d�8z�([�{^��������իW�ƃ^�t�( ��_�9�:=[nܸq��က�ˀ�{��q�±cǬSx�<`Xn]�����7�X_�c�Lm� �E�����X�%�`�Xr�� �ƺyQ��!#�u� �N�>m݁�)�- 3E S����/��$�Q�b=�?��l9� +7G�$���C._�̞�f�������;�Ђ�.Q!��c�͛7�����5@�6�m���1��ٳ�]d�'� ���q� ZE�O�:��^���B#�al���d�%�LZ��&�l�� .�� �bu%�j��I�L.l�8�9�Q��T����z�r��v�����v1��/�mS�Ǣ{���}Rc�jp��C�:�7�������mz��[����f�)RdϞ���~�3��L�2���W_}u֬Y�:v�ؗ_~���K/�3����y��&�1cƞ={Z� +��5n����^��T�R�˗o�ԩ�E�1>��k +�ҥ�ݻ�LWB�/Μ9�q�L�x������.�%44���=gΜ)S�$�,Y������߹s'����駟ąbI�R��+f�茼^�u[�T�P�: �d�)S&�2��ȑc�̙RR/\��- �"`j�V���R�N� s�����I��(�� ��*~��g4֘1c��\&g��0|�p�cI۶mU�T�Q��&�e���~��_~i[_>[�lu�ֵ�?�p��HH����u���>��jO� :u�d��"�t��e�H�}�֭�O[=�M�v�ܹ�����_��m��H�I�& ma�m�]�v$H�5#:U�Z�(�,>b*�Ή1 4�Y^�b�%J�9s�lY�j�5h� �B"_�7o�����t�ӦM����d͚�{��@Ȃ�4e�ɑ����/}#���ݻw�$<�H�A��|OE��k����K��>�͆�X�gt�N��s����G�:�B[�l�ɕ+W�.]8�ڷo_�P!�\�$�}ŋǟ�iӦ{����#�B�HN�:u갱s��xK�b�����;����$(ɲ��_ē�"������K�>=��СCÆ ae�D�Z�h!��ˤd���dʁ��W�lY��h���i�2Y���6���'S���� , ǎ;����;v�jp����p�B^&Lpļ�-yQ3���"�h��̹���ڭ[�&M��� �����ky��1� 0��g[�Z�dɒ�,Y�Y�f�˗�J�d�0 F����(`�w�^a.\�$I#���yE�,���t��U*�O-�I��+��V�Z���Y�&�!����O��Ȯs�}���6�aúu�`E`�g��S�H� K�{���{ X���xr�M�5� X|��32'#���^�<���N�l޼�PH1�Ѯ7�OD|��!G�r�w���/c5���e#� � "�=�O�>"& :H���>�K�&Y|8�i�E��Tf�i�&777ҔyG�}4�ɞ={he�?��u�WO����V�2�䍍�k׮ !�5���ٟJ�.�K�{yyqȐ!C(� �bj�?������.G�e�e~�G�?�<`�'��#:�QE������X9��>�b� +N�'�s��U�\���5�C�1:t(��k׮��a��VI�kܸ1)�tyk֬�-�ە��L��@o����B E�@�x��s8�=� \�܈��}T"�������^�#ӕ+W2��@@}ʕ+g���O� gN��o��\��X#_�u��eT�V�����._m, +�p�B�ׯ�;�ġ?5Iv�g�6;G�>R�$K5�Pj�֕�I�:z;v�_~!s���d]�z��@����\�Ǹ��O?��9r�A� l�e��}���a|Űd���dA0KAo�sv�:)֙T�������  �'M��RգSxHح '����v10rEx[��H��p_�����NK�(���%H� U�T�j�^NF��>F�r ��߿?�$d���|��gϞ�_9�q�`���m۶��p�BW{5j���*\�G�~d�w���ٷ{�nF�mڴ���ѲO�V@x�dɒ���W���ξ_|Ѭ�nH6��&F�%� ���ܹ���l�M� �8V��e�2*V�d��&p� `�`j��ӧO���V�<Gj֗'��-[R��ĉ�C��B� +�O��֛��� +(�q���ĉG�>�@���ϐ +S3dg��-�M#w��x�…kԨA�?ɝ�i�Õ}�0 ���� ��#�=�n �M��lذ��C�vFA��!aPD%X�G��������D���3Z�I��9sf�z�9�4�&{� �����4hM�a��y�~�������� 1;}L!��9&�Oǎq��"_�|��򴀰��jժfͅ�Ç�Q���H���ɞ�jՂw8a�Ǡ�SR��ۈ n�ތl�̙3m�)g7��y�͕}K�� ���”ѣG�_�<,�>BN,ڻ<Ѳ/K�,8j�4F��j6��}l7K*y Ö&� � +8��ܹs�]�V������?;@�I%SF X���@�eڅ� :x�3i �1�#̔�~�>�Ai�-Z`U�����44n\ls�7a�V�T� +Q�{�$I\�G�ET�lY� �aK�n����F��}�>t�P�E?�;ǂrOO�s��[1@�D>>>b�+�BBB���+j��tc�l��i��8�Ip��~����� �G1B ��W̟?��԰���d�F�c�gt~�.p��s8��3���Y�G"Mr�!���9\b����X���у,�ct�3f��S����n������?g;���ӵ�iE����W�VM�\W���4Na*��8��Tؘ�ȅv�2ٓL���!��ކ}�ѫI���aT��d ya ���c� +�bb���� �S�Ç�����@t��G�2�5�؞u��H���� )���U�S� `�րx� ̖�K��s�}F¾�5k�p2�֭�?������'�8v���7��řK.¾r��ɛG$�K�Ĕ�Hx6�ЩS��% +8��ߎ� �tve{2����_O��ٻw��@^�l�Z,���" �e'4f�&[�9��iYxM� 2��>�W�^ � ����0�z���gzU�S�]�|�%���%�ȏ�������h������� �W˖-�e˶f�y�D�y֯_�U�Z�n�b� +GtϺ �̳.0�G�������6|[p; 6�R� +#g�0h0����[o�E��T�^=���i�KaX����pS�r� +7���m۹sg������U�R.�޳g�h�q"�� NW٥Ks͓�.5о}{�[a��Va����}I*�C��@A�'�4�С�ƍa��`H�O���.�J�b7&�U�1&O� �j}����e`{օ�" +K(*�#j&G�-��b��C��,�1 -X�|��S��IM�*��K�4��R4jԈ +�� �I��'%O��q�Ο�ee�l߾���D�V +P�"F��D��KT#�3�2{�+1�[���9��>�slP�Np)�۳g�u�e'�~�������s��#G�X�� ����:�]�ʕ+�o��Y�>z�@[J �9���9sf]h8�q���C|��ǩ`<��j�v�x����;�˳�>ε +* 0�<��O& �ݻ7'>ޣm۶����%K�@�s�ܹ8(�[NF8�ڵk������5j�����cLj�؈/���\^�`�����d�)��Y��`l� �7ƽ�ns֬Y� d0cƌd�`�{�u�M�6Qpv����L^Ĥ���Tr�eYk���%���UN� �sD=�o2rD=Q���M�������liXƘc�sb�_~���6Z��@�L�C�]�v�.�M�2_ +�p�����Rܹs'�J�>}āc��۷/��*U��! N�K��Á,�Icؓ�mW�H�1� "Я��zذaO 2 9{���IӔ��@[�G�����S{���� �Ò˗/���+2 �Q ř={��|}}�"�0W��i�J�(��8��ȇ�bA͛�8꯿��E0�.M�SE�e� ���!!����>);� +1��>��Jz;݌�Ю]�U�Va����Xb�ҥ2.�8q"��s�4Icl7Tq���o�n?���������IWD�wfه���d g+��؊ә� x�E��c 4����9���/��5! �3a8m�!�����ȁ1͊����Κ�yQѺ�\�#_q�8v���h��Z"�K +���6��5#S�#�˰3��ay�a{����r�&!�������_̠8��ᔩ+i,�2�|iz��D��nl��Qd7�NJ�B�RX1����uOI�Zv����h��XH��-�Rp�&��h���v1^:�ԉ�ٗ�"A��uZ��d-�b���J���̌g;���E[���R�/������w���o�h�U�*�����#�Ís�Y�w�xW>F�q�VD��� ��&�T*�J�(� ̭��.�� ��}N �#n_���&��De�J�R��" x-����)q=� ��Z|$��Be�J�R����\ +Yt0��!�W�oG����O�R�T�J��_��/nn��>����D^�0A٧R�T�����jp�ր8�z�s�'�@G��x��w��??�u��:t���̅�:Jk׮=p��y��ʕ+7n\�OmݺU*۱c�mO��E�*�֌�ׯsf�ڵ�6I9^�t #7l�@ +�G��������v�!d˖-۷o�����:|��a�3��F.�8k֬������}���m۶����k׮�ݤ� �Νs}�k����l�\�]DRN��ׯ?}��-#��ݻw۞<<{�,���?��'O�����J�z"y�1,�Ε�[k�=��~�g,�y ��T� ?;���} +�ݻwoyS8}��o���7�|/�W�r���_�R�t�e�V�Ny� Ϝ+W����R�Jɜ`_�5�Y'�F��y���]��'O����|�M�Ȝ9sٲeqԲ�O�>9r�#�f�Z�R%X ��M�4���Q�W�>��� +*��!��hmڴ�DǺ� �/^�Zd��N�+x�C�˔)���I +�.�����O&��3�̙s�С64/]��� +,�y�f�iǎ3d�`�O*d�����{ﹹ�I���������oHu�(Qb�…��T&?�.]ڼQ΁�G�&G��ޡR��"�D�x^��o�E���G��42�˾B�'�E������K�H@Q�bń �����:u���_˗/�2eJܦ̉����/^y��ݻ�3fĈ�ʕK�4iݺu���-[�s.Y���f͚%�P�$H�[6�t�)Yl/�K߾}�%K��bÆ �/��ɓ��ѣ���K�$ @��A����ɓ�L5Fv�~��k��f���������);�,4!S,���,sp�x���)�P�D�5oޜ"C��s�REiӦ%S��P�M�6�wʀr�T��V�:j�(pӥK�x����uYU�0q��d������۰~�رR�Z5��2p����I��_� �ٹ��6k�lȐ!�k��WjL�mt8'O�fҧᤶ9p���/������IR�T�E��o� +�;<}�#'���ﮅᑗ@�,?�"�쨈����BTg����L����'L��燆� �>�7Y��ᜠ�~��x]�$�k׮]��þ)R���d}̘�G:��'oD��V��� hZ�je&�� >>>��p {��Hd�x�%K�nݺ���¾�>�Ȭ��cǎ��~���}�lْ={v�+V�k��p�ȑ5jP�2���ف�����f�0G:�< *d�H���fe߂ ����'N�W��.\���%�}B�L�"o�+�T��F��s��;4.ޱ�q�h`D��p���q.Y=�H�"�9!�,���ၷ��j���#u�Իv��5h��X���21���>�1$N�)1��h.S�L�֭s5�H�@��s��!���/��������� ��T�Ν����򓰏-�n���۶m#&�����;e�ӧa/�u�U�V�^�… 6�9��aI�2e�T�T���M�&���bgi֪U��6n�h-�L��Q]�vel �#����o�H٧R=er��p=x���ƾ�,�!�G���]�~"�J� �L�nct+_�|�l�p��>|l�(�r�rO +�+���>� ��t�Ҙ�W�reb�|�F_�8��ճM��� .�w�;w��1cF� fΜ &�̙CP���/a„ɒ%K�_�7o������L�hD�Ay��}K�,! +7:)@�]�6G�?d�P�����ƍ���_�.lJ�G|J�`��������t����S��J�ߎ�:�<�4����8������Tċ�?����O �duT�fMh"��(a_޼y�F�nժU� ��t����d2�c�۷/Z�A�\�r����9sF5�>��p^��_iҤ0`����-[���pȚ5k��ˇ����7ހ��HɫJ�*p���#9�[��QDv$ҪU+y�}�{�=�_�T)0�1c�+V��R�`�0@f闕O)��Ќ�}� �8��iӬ���z�"&��V �΋�@�8��>����>��T��:�N��sE���}xљ3g���9s�� � . �j�ɳ.D�a3����}2>���N��o׮]�@B!YQ�Z����,^�vFm >�رcGX@R�n�z ��5E��)J͚5�5B�5�=�P�dI��y�f�P��5��' �7o�˔)�ļ�)v`g>|8�9r�ԛ<&�(�"H�|!�t��ey(%�g]��d�"�|��I��>mI�*+�H�Α#�O8��S��nE���n94�_���?��s��� Cp�Њ膈�M�Sp}��*ad�Ң �s)�}�3�����>�����|��|��N�3ʗ/w0r̘1���*�)+V +$I��lٲ�2���-�(6T�X/�cg�/�^�^=�"�5jX�1�_XE�gϖ�A1@R��3g�#|�:�� d��̈IdM.�&M�����?Leg�I���e�fޠ��T�R�Aj��C Z`�E�}��p�� +ڶm��oe���ĉ)��O�z9��xk�Ѡ����S�>��+nܸ��￯T�n:xzz`��nݺ���G�����^���Tݺu��~��k׮֕1}}}�����#j��^�zU�Z��&�K�.�e= ڷo���Z�V-|�y/� 0�L�2��!#��hV�H�s��*Tp]8������!jӦM�!$[�J����[�Cg������{��̄ûw�&�$�3+��&Qpl�ӧ����m۶�gϞ�E�S�j�v��iv0iJ.�%��ѣ��_M��~Y��\]�d 9Κ5ˤ@�R�gӦM��*��W�%А��WC���>���o���zI�g��¦g�t���K�;K�s�)ے�W�^%��V̔uK��F��,��m�2�"���M(;[7b-��׵8��+�e��k5&$$�C����(>���o���}�jPP_�]0W�_�R�Xv�� #jj5��jK�-Xn��Mvf�u�9�J�t+�E��ۡ����g`�v��T*��iU�%Ћ7o-;4a��O�R�Tώ"���>t)d�����S�T*���������;��o��S�T*�Ӣ��BV��S��>�J�R=-r�7v�����c[P٧R�T��K�t^�\I٧R�T�gF��w�Yz�Id_``����4iR�R��;���Z_��:u��w�bŊ�ڵ3;�9s�Z�j_�S�~���S�n޼9bĈʕ+�ر�$�������͛7��;�p�߳gO�B� +z��o�>y���߿V�Z��5j�3Ƽ�~�…N�:a�,�:t�P�Y�b�)y���;�Z�t���_��jժ����fV��9s&e\�l��9��7�|c�S�΄ ����=����[�ha�����j��͒�c�~��G��q�Ƥf�d[�R��DE�G���/����`ߝ;wp��q�ԩ�{������[)R�������/��ӧO�D���`���5k�dɒ����>|���^�ؼy��Rɒ%�?~�ڵ� >��sŋ?t萐`��9rX'lqD�~�$I~�\d�)S��^{�5~*P�@�ܹ_y���\ +��Ɓ�z��%H�࣏>ڳg�dz��u���2c'y��,Y��O�����C�2e���6@3fxxx��޽{S�L�ꫯ�˗�|s��I"E�=z�()[�4��<�'O�+W.S!~��5k���ɓ)ّT���I���K�T*���K��BB���o9$þ�ݯL[��O��u�V��-[���ܹ�hh�ƍ�۷�n@J湂}��?�����֮][�D �a�a� �\����� ��&MJP&k���>x�r�JR�D0]�tI�G�$�9rd���o��f�1�*�n�Z2���Ə:K�*5g� +�f͚E|Gx˯6��uY]�|�C�w����W�^�=��}���=gQHH�R�|y*vܸqTז-[~��w�Tr�T*�-�z둗@o�=:�w�^�,�t����E�&M&L���n&��/�s��iӦ���$�}2d0+}����P"t0� Ѷz��9)�{�嗋+k���b�u�k�i۶-�@d��H�ʕ?��3�ʾ��-d=j�(����S �Mf21�Ȅl|we_˖-� +-I8p#+T�����Ϻ��љ3gn�ĉ���(A��N��R�TO�"���N���L��Wfm ��1�Q�N���� �\='�J�.�JV��o߾=`�pF�$�{���;u��c�d�ga�k���dɒB� +eϞ�`�����\ ����>l�,%K�|we� /����]�h�4iҬ[���ϰoѢE0�Y�f��.���}��`-S�L�j���Xؗ1c�6mژ +�>}��yO���z�r��M�0� Z٧R��JE^ �}���������py��D���UV0�����ȯ�� +�.\8r�ȬY�)R��5O����T�|���������m۶-��O�~����������o�3? ��?-^�x޼yM�6e˘1c�]�}�ׯ�0�x����o�̙�܏���"�8ꫯ�"G�6m�0t��� ��%K�x O��ڬ�J�W�Z�\�r%O�����o���=U*�Juo]�t ��룘��˗���;44�%L���6mZ6�L����?.w����.]�쏒��c�q ]�.]H�N�:��L.����nݺ���sD����/�u +ءgϞ/^t��>�Ԑ�x�"E��5kB@aߦM��0aWLX�����Kv���o�=l�0Y�/���+V� + ��C�4s��Y�dY�f��~*�J���~��Y�z�����ă�e�#Ϻ���+Y�n���Ǐ��K�$��~���D`M�bŀ��DѲ/C� �Ǐ'/�-Ha1 ?�]��v������#���>��a��ՁWҤI�}�[�|y��ᅴ˞��ʒ��q_�ʕ���O�޽{�իW�Fz��P�x�,�K�mڴ!}�☂P�J�R= +��kժ ʔ)�& +�-�d|�8�g]��}�駋/^� 6���}��C�d����� +��~ߩS�r�΍���z,�C@�H�"�=��yǁ,�-�-Z̟?�ܹ�7&�8p An,��lb�D�ϚRS@J �r��9k֬�a3���o��/_�!C�����G��V����O�>�J�z�"d�ׯ� +t D�V�K�vss3x�'�&L/^�R�T����DӥK��G:�)S�LЇ�)v��͘1#}��}N�ڨQ#�K�81q(�������{���?)u��ɛ5kv��9�j������F�Q�T)ꄣH�@����uz�J�R=�����={������㥭W����$1�끻v���O���`�v��m]���>�������ȑ#�&Z��X;?~Z�pt{�t�Yi|����œ�<����~z[�����'�_�#߸w����[�H���'j������;��Η����>��_z��v�Ï��]>�o�?85�C/�_�z���h5Zi�- Y��/|���}�+�+�S������ ���U��4�;����R�S������+�����~�i�����/�{�)��O?���>Ǥ�R�*o��3���r>D������f��F��U1g��)��>萣��Dm���U�`l�!&�b +�����A'�Vyȡ���t�kw7�Xφ�(�MH9jg��>�lLR�|4�`ݮ�7�������1�3��� �cc�t�G�ſQ&����G�1d�7z���S��M��!W�,�>�0Z�3�z��'EB��Ϯ�|�ţۛW��~�����gOn�?���Ջ'�~�ɿ�|���o�����/~�����__����E��)s�V�����C_|��'�_�T�����/~�~:���7�_<�}�������/�?}��D�W�~��������f�x�u���O~���{�W�ߦ��޸�w!�t�{�� ���H��_�;e�㷶�6����X��������|_c0)9�m�6''A>�����/W�}���������n7�>#���q�>#�Gh������o��듗�>���>{ձ=|��Փ�������ϖ��O�������_����>� +|���~������/��v׏�����添!� +m�W�_� 9 +�x����ሩ�総_�\K��^ܾ|�����˷ƪ��U3qLygpgie�Q�����Onv��~Z��c��W?�^�<�}vs�V#+M]��bSW��>���nA��z����k��wX��o?b���i�����`k�ܽz��v{����/8�/�aו��^=�Z��Z���� �������z���_�z��v��;��P�� C[���� WW�/K�P�>�e�ˠO��1T��_��G�[�Ƌ�_^<�{�����p�Q���L��哷۞��#, �V��V?��Y�w0�o�rϟ>������<�獏?�j~�1/�S�� ㏀27O~y����a�ۏ8��A#2.��H9x����e:{�����L�O�{��!�����=~N~�G(i�=����˾}���o������g�eO�_F�����?�Й˘�q�!�񧟄�g��}��w�'�B|��닇�|���~������wo�Ǐ-�ফ��k>����y�t�=�Ž��y�����j��W�gv�X����� �@�U��u��8�6��*B56$�7���W:F�\�6�"7�K�Ơm4ޑ�Xo��:�1�/h ���I�lv�_�����Cth�&�BH��0c��L�b�֭�b�t9b�V�L���b|��jd9��m�*�*>y�n��s|®��Y��w����h��"6p�>��g�ޖ�.�2���5��}�1c��,sʩ!������ڷ��{�"�V���1�V��kQ����r+�7��o"�ۀ��p<��� ��n?����E��[��v1�v��!�Z�Ct���@��L�C&������YL��N^ �n�o��_�d��O۵�,��*6���НkG����L)���>wS����=o��9E� :�;�JW*�+g�t�� &b06hA�U�$���'��)uh&�7i7�L����cj8.���0q���X�J�X�?v�r��m�{���L���iW���xÖ�眦*|��^m�C�6��w0]��.[��sg1��{���]�)�ݛ$�꿜#�S'l3�3�3 &���YL��g8�3f8u�0�=�`��ϝ��a��{[��Nƣb%��:.������7�����7_��m���ǰ����6 +t��a 0;��&�+{�����J���W�h�r�A���)���YRvI9��bԁ�������'�.i��x��@�Q��Ke��}�ͷ�}��=�����៳S�`�y3��+�Ʒ\��Fߛ����%&[���6W��+��HY ɽ�Ogb�!�= 1z��x���W��=A- �c +�j� �H���l�CH��@����uvA�}V:ج���>���!suR��B`Q{K���K9�F�4d_�J�qs�r{�MpN�d�+�����- �BPz���qF��A� ��lR�^G ��ǥ�脎��n�s�q����>Cӄ ��K���z�����~q� ��69C���Ƈ��qIa�1�mp!+��j��������b�������\�a蠳u�ZVA��Gg�x�j���7.[��2D�o��H�.Pe�-,-�|rT�̜eb4���%,xe{.�F�����5.��\&���U\)�싸=2�5�w��{*��"�8b0��&�}��0X�k��s�y�9]���.K2�(���>`#b.�:}���;�V��hd�"�|�b���90Pq����-�i�&t���|1��f[.Z�f���.,p��뭁 �߅�}�X�e���a�**c<�\�B�L�8�V�d16 ��7���߲<*l�q�/�bF�2�|+e9��MF�ìM�\���hʀ�NdE�{���M��*&�Y<��!,�Ɇ���"p���Kf��r2��'����{�٦���Z(�(� 77���@�(��r�T��u���'8�Q�0m��e��i� +3�"��hס���6�f�{#��C������a�D�)��A(�b ɕ��c���v�g0}9A�[���C��9"7Z �)�x'��M�0��������\8�r��m���0na���d�^F�MW[��%o�[L +�;r2z�Y� ����� �p�-�D�'lb�삧͈��6�(�ͤ;�縨V\��>G?`��W��*X>�k������8Y�?ȉ�8Z�B��<:�3^�v�id�u0�/ܐ�Z;n�@r� ��¿� +Ʒp�weGN �9a�:�b���������QC����`�>�R "e����Y�Fe���V0� +��jlw��U��B��pNڟX�D)�}C�u3\L|`��G��4�8�"��i�;��č��Bq��c/�cS��(�/�'#�q޺r���S4�������d�����]� +�Zg����o���8������|R�,Ĉ �����p��"� V�3�a�-�R�5 �:�٘ +@�%3J, ´2e1��g�]�s69I@��w�;��>���h�6 ���b�\�ț�%0{BD�*W��糸.?,��_N-���/��R���EL�7�~�P�ǔ ��G�␀`B\�}F�� א�8�h��� ��r��@j�4�(�)B�-SП��d��d=H�LY�Ŷ��� +��k%�o�cgI�a��2R۳�CЉ,�r6:`�AC�!\{��ؕ�x�L��Qi�.���ZI��<��!�"%�Ő輞�J[j3@��FL*t+��B���� ��f 2k8n�[!��T��¹�[�,�=��^�#���z��V���0$ `dcw+v@�,�]��4���{Źp7X7^N�{ +l%8�AS�|��y\ �8�V�>p��>zQ" ��%l���#;8�zT����З�E���+��;*Q$�zOS/���( ��\��������pA&��E�p1��[z/F�è�{&jO=��5����(���%,<�Z;��g�8�<� +0^�}Ȅ�p#L�2�X#�$�c��Т�b��r?�W�5^U�C�ǕjZg9�M;��y�X�8w� �bԪ�����SCg�[������{��/y�:n BSN��������H�" y�9�-O8d��S`�]E�[n�:���U�el�d:y����X���X� �c��Ft�đC%tF$7u��(�ҭJn�j*�5�S���k����1Gl��ᆶFK��gDu↾����r}��W�d�h�#�B:��R��!V+�mr9�Ȋ��W�WX�� +��F��H��{��j�W$�"V��ԐSYټK�dljpAf`sY��@�U/\-��i��D�!�^� �Aa4�ga�"���:[.�Q�s�f���������ӄ�5��/vQ�N��+5\+�����_����a��sy�HEK`��\O� ��fH��0O��r=G�?1�ę>N�Ȝ~�ب*�!jԘ�{�B���.)�2a��U�B���nl�- �E'�lUe4�{��a@��� �j�$aw��Eyǫ��4?�V1`���������4��[8�Λ-mFH2>�Y��sɺ����`j���4vvFn�``�㦦�3ܬi﹨�-�W��P�3���&g#%y,Nqщ��' {��.I+u +�.]C��<������O�P.�=��r�h����'(�o��?S����%,��S .Ca J�]P� nm���_� ��pP +B�e�`t�e�B �|K�&b$'&A�X�6�tf:p�����U&꺀@��8���E�����> �b�Ҍ�큃���PNf T�Y9a@gT{�W�"�(���.�n�u/`�Zv����"'�&���b1 X, bMp�q���c�-�<�D�F�A&\+�dp��}6뗆�/����x P0FL�ue��9�Q����N�����7��NDw�%��r�ջ����/Oo�̝qL͗���� 3����:Q=z��G�|����//�=����G����������������o���n�ji��իc���ꭜ��.�U??�����>���̖^���W� 5Z?O�ahH�2 b B��Q��I�`�(]rPQ^z�5�-���KT\�t3���z��)O������g�����o.z/�3b%;&��wL&�n��6��Ou\C/+��˿��{�����e��GZ憲Ps�YM7��_�Y}�7�6`����xc6mѸm�c��5\��O�]f 4��&̕L"_!!�Kٞ`ic�TU���78�M�5�2|%��%t�� �}�W�l�(�����a�; ^����3��}|c���� ug�Yw,͑O��{l���j8EF�#�i����h���&U����·�����8L��R_�N���3������7��Q��;z>i��! 4s|���x�,uRl���v8C6'HwKA�����,�� ������%b���\���Ч�aw9� +�L�e�rYqTbf?i�'��G~����ay#�,7EzH,���7��Lt�����>L)��c9C?���,�h-� �Xq�s��`�X�������2w�Q�Tq#a���^�Kx=o�DF8q�q�5T�Ԡ-�����\�'���n���� �r���AY��ݤ9X�@@��K@� |��U��A�+ݬ_�\�{�C�$b0��E���� ���%�ͧLQY� ��,EWm�,v��c�(��bw�u{�ry�q{�y�=gj�"����zM�g�9\�Vi���5ԝ��tú8k� +4���� S���ؚ��[�vی��J�/�n��z���*�Uq�� +�&�J�!�vU:�� jWe�Ҡ��e�� +��`��{� �����<����u�@���W�7�����1�?>�����ǎ�K+��_L��-�c���&�x��;������e�Z���Y?��]z�md{������B�8v�2��<�L ��bH�\p�d�lԈI~%�q��Ѩ�[����*�")� Ԫ�.o0�G_}�gm�6��`z��4~�鸆�0�1\��Fߛ�0�g M=��Fp��0�f>����Ƥ~o.���8��'�E"@#Ny�E2#�b(F��h(5@��;����t�Y�Gg���l�1%t�4P��q��$O&|eϫF\Ҿ&����>�z=A��#��&�L��M�ϘUab�!�QE�X0�=a.��D�Aa�������`�q�F��> +Χ�.�qh<����V +1:�+`�s�߸NrTڧZ5;c���I�������A4z ���ky����2�;��_t'eL6�����7]S+����_����O�5�/(��i �����a���>% ;��,&y��T�21���0�{�>̯����;�#[�)�����5���-��)��wL�A�d�D�d�d�/}�S�{����-�^�[ +�aF��� 8���o-�����S�ὢ&f��� �1�����7M`�o8.8 C#F��t����ҙ����;Y�i��f���|Ĺ�Q��PN/�([�9n�cԛ� m���k�p�d�<�m9̖㢼���b;�X`���)sK/^Ŕ��Pz���47<��&�, ��\Z9*��޲�܀Co9n���d/�Wg#�I  �r@-�f �:C�m �ҋ�Zw�����a2���R(���6�.c�AD�G+n֏©L� b�n+m�6ytQ>�8�9�䰛�.+��PǑ��+�윆ɷ#2-Gj9|���6��7��T��ΩL.�<0:���-S,r�����8ݪ���,:�fLˑ�y�Ay�l� _�6rQ����S�QT&�W2+E螃� +��L�Bf�af9xa�!��n��L%�\$2�"����b��i��b�k$����K�0�f0��0��܎cxwuL��ήl'F����ۘ���� ��C��3�e�1�ej���L�Ci�᡿͙pā�l���'0ϑ÷��.�XR�.�nD��f���X6"�c"@����8������b�w��+)i{d4�Ͷ l�7~�t�ɚ6m�P�:�G֜���C�C��#���reBp��}�t���-���v��G��m�!T C�����0��z��o�3A��9�ln;Re��T��DT�&�;�i=��a2u;ݑӌ�5�o��!u�l�k��5��p��+0�h�5f���_�l[�L�|��6s�r������$)�=�`�����(���>bix1���QW�L���'�ڮY�\�8UJ��� �{�/���� +�pg�����4 ��<�;k ��g{(Pz�t�MB\`s?e�W�h�f�v������pt#�z�\�X��Uԭ��*��%3X�s�N]E=��*�v�pֺc�/R?�R�,ힹO�`� ��,���^p�# |��?����=x���c#r ~�ȁ��"r`g�/v�ʼu��!��>Re��n��h�3�e0J*�͈��,���yMe)�g�V@���B߲�X�z� IL��6�R? +c?kq�o����_�_ |M�%j�I� 1��H��q�@�6�8�'\4/g�!g1 ?8��>�2:��,�7Cf���ߔ>`��7&w� #�����ߧ׿�z��[%K\~r�%���{��7߃,�Xz�VT�i6vx+1�sMa�p'e�K�e�a�[ :�q��1Y�V�@��4/��]��P�% �fِ�u���SvޅC���K�qW�z�Ro��G�)I��t�z)H۳AsgJ'�����qd�D��D���x|�����\���,�D�X^j(�l�sPX��LJ��Sl�p�k�c4�$K�5)ƻHa���R��@3 3H�a��N�EI��e�ψk��.�A��ڊ7c�i8��YX�)�T����ݫ��Ѵ{��r4'��o�e������y\�O~�"L�� ��k�q���H)�4g���5�+>�[�waQ �9�87��v$��>ͼ0, I(��2c:�O����<��t<� I���R��zӳ]V��§��||������{�N��`�Z켜��K����=�.7="O�y :Fy�a�����t�1%�ϿҊAH��a|�d ���_4y[qu������� ��[�r�_���L*�$ל��Y�FL�5[�`��E�|rP��5�����QV=s �k��� %[bO��g,kϜb��ke$R�B fE]�?ٕ�#:2s&–��6VW�>�K� � ��d�0X ��7,ć���� x$���@�1�ۘ�~�!�����bێ>�'z͠/ �ω��x�9�(�k�F�H��3I�d�[�������o��q��۷ː.F��w�F�?��.��������w��Ἁ�� +)�F5&G�J�1���T�^E���l�y�g��A�y���I +&0c�;V��YOj=&�]���<���d�3,��� ��Y��a�c��i����o��5��4���Ms� ��2S�O��4�ݝՃ����mN� mmg��Y���n�"�:�,U}뵢���4�d�� �r�e�%�X�*D���'�a-[o�Ux��2�x.>stream +p��k����e yO?P-b�x�0���C5L ��g����z��8�^�����_[���I)r�0�>��y�$Wx����TT�����d�&6���RyQY�|�E�Ř�`�ʦX�lR�M�NŅ�I��bx"he����c$�e���+119w�!�D�3�\e�,��9���3�0��#�~��Y��u�(����؀-�����|M��@ c�q�b�%�w$�����ÆR �1��0���Z&�<��C,_��Zxh�`� +��%5�e1J�sA�)QA�����k\V$Lq�XZ^ �t W,0�\�5$؜�� K-z1s&�~� ������(e����2�?ͥ��̢���K�6c$4X٪�q�YŞE �$�|ۆ��ʕ2쉵 ��! }�/�buL� ���D��4ݐa��sA� ]� �bB��<�A b�6GM�&€,�/������<�$��ì�G:���Z��\�<��Rb�D&�6��e�~y"H�LC/���sRΑ�(p0 iq��sq�zz�k3U��� �1ǎU��4�(��p(K��+K^Z'�I!���d�;Ou.=��Q�� 6���˴&�m�r��P��[JB?�DcI�Ѹ����e�m-�HomHiK�?���f ���^��b�)��/�I1G����5f��)Y�%11T��.E��RC���H<[l�aS�ΖŇ9����.a:��:xܯLH��|H�*� +O�\��\"���X��X R������V=�����zf�$˞f� +����zd�5��̶��>�zY�N����-٢f� y�I� ]�j�����c�B,���B�'f)�����<�vp�1���,�S�v�N�yW�Ĭ����e�X:�l�~��C�Tҥ35P�%�*�Q�l`��T�Tҷ�xX4Zs����=��m���[��!`\߮�X�'�0+qV\r�U'�R:yXM�{���>L�H�"[�BL���j*-�)��<�,Jea!Z�PRS�FwZ_�䊇�AY�&%�� �ir��PB��,1oиquqy�3�c0��:���e}���{R��_xР�ձ9���,�� �,4�+%ȶ�X�*�g��ȅ��2��|�W8𗕄R`�{�A`%��*h��� Y +��Rw����4L���]2� O���n���sh�a�s�-l����<ʎ���K +�r�<q�,�*�t`=_�¹S��$��<#��s,����ip�W�e���+My�w,�h��*���Sٺ��14&|�!MS������a Bf��94X�5� |(<��*����/$��1�J�䓇Eu���gy "��%M֘ ++�y��)MECV4ƈ|� �zڗ���_�2�u7�K,U�M�&>睦�2�*։���2�^�0�ڭ��p��q Vl8�."��&�p�:.,| +�%:|Lm(�Ǎʆ �v���l��� ��/ǗQ37Dj9�,{̊���L\x�t#N3��9<�E��P��e+d� �ovy���"��)S�W1�c�<�$6�@������!�o��Qj�i��HQr}�����OA�G��^�V�x�q�R������K)�Cd"VR�q��|��ʫv ڧ�fYc�Ef`�ml��l��,�T�e�h#U�q#h^���aI�&rL���;Yf>�� =�Y��qk�������%qQ^ҀT=�.���dc�"Z����0ݍc&�L5G�3JnhH8[^�̞�y�����v0+�4d��iƝ����������9��5�Ƃ�ï���ހL x6�G�Y�'�R�v`���|8�ZY��ۏ'�� ��IE��:��u��e��`�����M�o���~V{V ÅkYr�v +?�,��dO��3S�,A����!X'-��A�`� �\v�c�X ��?%�v������a,�N]^�;��S��Ȧ��̥lC�` +�]�0��-J�ԋ����X`����IA�lX^m��pN����� �ԥ�O|,BE� +���L^4�L�="[��n3O_~���Z ��X +�Y�Ye�� ����N��N�K~�ȏ�bT���C }Hx+����h$�qRN�} 4Rc�Y�<K�3@�OH[f�Ef��w�V�}F�hv�\6��0��m�� � i외�A��Xxg�S��G KZ���:����P�(���c+vK��H-])����/%S�_ Q&�(RN����n<Ҹ4�b��:�8;�`S�~B|V��N>O��U�, 3 !�n������� �ܯY�^��mk +]L����"�q�8��j+�P�6�Ũ�PM��e� S���k��``E��I6�eA� �!z%����@�ҷ��w���b[5&r9%���XLLNJ��E�W�Ył�,�6�(Ndú�ij +S��lX�KQOԢC��,���B�Wx �-� �� ��C�0�H<��8�Qߣa ?N�$c���.��%+ki��ߠ�W��cu��6�=�� *�a�*-ycM�H������ +��.`�0UTM�󱜵XL�T�5U�`+��&�“�i�cr)@���+#��F��p���.��@�5�R�D{EV3��4�$S�.�*� }� y-���H%6���� � q1B7׹�@�EF�c�–�-��K`$�� �q��4j�܄J�kB@�X�fT�Y:CXs�I�s��uR�\��=�r��ZG/EyT?:�K@�^��] O�x'�E^p�)+���-Z~�ȟ�r�xU1�n�gb�B@�w#� ��X +E� �P,%�B�������<�J.� �/ �<�p-�|U�1�D�{��€�,tBŗ n��<-Po{T�Ր�!�@Tv4U��� �� ��E���F�[��!(���]�R��� P*W-葍f��1*=`[��a]DS$-]4�S� �V̈́��t*� j����*������!l�P��k�DOwp� �9�Հ��LZy�X�Fo�� �0���bae��LCaS���%�v%�u�H֎��-vGK-�i!�t�+�St�a��:O=O'8����s�� �t�fL@#��+_}t�%i�"���F����y`yg(��X_�I� `�\]�A�F�w��Ś��2��;n�_�7�2�Z^�N�1�M ��Ȃ����O.%\}8��.� m+�6˿�Tٰ=P[���<����|�ny���!0�LGW~�E,��)$A�E�k���)^��(O*|��˲3K������fyv��"��s\xC3�@�|x�I�Nn�J;`Cyx��ej�y��ѺB\�w��D��hn.���0@��,Xұ��la�oMݬ�9��/���-)1����%���)/i,>�"�q�a�r�i��z� �p��d��]���{S��RO�i~�K�`��� +��]��F4D?s�˸4�^��LO��m�FI�k;���1#h��ycBee��T< x��"�R��#��`P��w�9 ��{nʴ/�h�g�~�U����8�(7�4��q�sbS�g5� +c5]-��<��\���F��pW*:��I��$��u���k /��\Bt����} d(�����i��'#��U) � +������J]�[A��T�=E\D۸_���ۛ���s�A�v'����À��%�\�C�W4*[V���Y�F�T�?�p9��A�$���A�����*e0ؔ���"�iV���sǹ%��={�;�S�EX�B +�2!"ur�����N�!�̓%/t�ɐ�����g�mz���2G,�+W�F�â�z����I�����c���ݷ�~]4����$�#��U�h 4u� ��Vx���aP�4�`:xL�G���YJ�rĿ � ���D�0��f<������9zsez,�������ʗ���Gqg ��vO��C`��v���=D���Ŗ���(�]�'�Ф�$�4q�0t��lwv�=GRn����/_�<�P�c?|\ƒ����?=���O%���LjO��c(�� +�=�{$Qt��c��3c �� +<�&�(���E�`�D)��@�L�Er��O3H�(�.I�K�ƒ���fw�B�6�R��qF��y\�O�(��w����Q +3JaF)�U��1-ơ���ϮI��,��+I��@�.0�_?�mCb�����l3��4�:K����&Ƈ��3(& ���T���3�ҽ��&�y��� ���$p�r��=��� C�L����$E/5ﬓ�|z�]X����;�:�Ѕp12��pE:�yz��|"���M��H��S��4��y,��r7#53� �S!��t�=�Gu��?eU����,��i��|=5� +�>�'�ĉ���B�<�ww�`�3*k�Ԡ���񕮟���{�������㋃�a���'�+�3�6gf���2Z ��V�8{-ׇ?{���d����}18{-�{��p�գ0:|ʆ�sYG��1�q�d���/v��1�5��1�fgO�w���+E��c-}���/���e����/.���Fm��`ԯ������|�=���S@?��:-ׇ?!�=�����;q Z��+W�9Ʊ���_\��~ ��.SEw���db�p����/^ti��D�<�bJ�l+%׼X&�f�XfZ��S^��5����1a����cZAq8��I�/F�`�"�=#�1��I,k���I�ȣ �i���Y���;�]���7���gh���Q1,�^��� Ck|`,[8nQ\���-�WFb����/� ���j��%�n0��+9;�>��{N2�Ԉ�Բ�x#��z��۝#.�`�;��T�0 f]�$O�������o��\�s�����T�1CC,c �x��V��N&��6�f+�A���0�8�92���4::�{:Q�Ɍ<>�x�K�������]� K`�:��0���D�� �#��� B��T~�qt �C(ړEc� ��6� ����o2�����x 2��~b�䖜�*2���wz�:P �A�1Ŕ)<ъgB����T�C'�j2Ȑ��9�1��a>�%>��\4ْ\ kD�6 5����T"^�������:ʹ�6Y����� XH���fq�<*{�)�yJ?e6� L��Y��]�V�<$���]$S��L�ʴ���$��!<�)�w�;�?:�2�:vU�?�%��㪈�@��=�z_q+�L7&�%�1�� � ���{����ص@=��%1�-�e��-CjŚY�CTf�A)DOW)�R,�t���¡��谞��a�W&57��;M��s$�{ +�c^�U�g>�0�uI��b��?;sH;��4mM�����z��^�LKP6q�&k8U��?m��tc?���e�l��C\ϖ7��r�Ś���  5�d�q�E�$�܃)hX.����ɶ��%�-�e� +[��~�,�ЕL���(_���BI���+4����.�-<6�d ����3 *b4T��!��� +J\�X2�ѫpu) +���S����l���Z��)_�.�`�'k����GSB�#�YJt�kd����F�`%��r�c.�ŗ7E]��k���ѯ5����COEJn��9J^H:�--A6������@"$�q?�D��N. g��9 �� �a���5�ibZ]E2gDF�E �!�#.�?n�&�p�p-f��'&����B1�azW�G ��>�)���B� &��Z6���l� �؈o��wwя�Y�c1���)c4bÀ�b�c�Tgt�q�٤W Q���Y��!BZ�X�E`�Oٯ�*y��E�؊�Z��K��-P��D鹕�6@�El��� a��i�P�#-�����`o�n��"��g� fVa���V�Ӕ�>�����Fͅ � , �x�R���۝~�J�]�(�yLf&k��� ��,���Xz@q{bЇ��gZaI��V���7�Ҩ�43�R�Q@��O`и��F���I���F�x� �R�5f[fZw�,xBIn��(���j��]A����F���E��V7xX�ET.z��k�-�幩�]��ᴱ������� @�8����� �c���'9��s��r�1 �J8��Ջ�+�[J'IS��<f�1��#(f� 1<0��_����f�X!ܦ���W�����0�o�iZ���նWMS�� +0zһq�$ZD3ڍ��lj�?Ӻۃb IVle�,���JMr I���_"��3i�\U�(���Y�8.ˇ-h�ܝ���� +W��q�9"iu�f��ۯ������h�+贞��2�}VI�e��㡲�^���QO0��4�A�v�x%� �$h�;ͤbq����)��� �n`t�'Ś�ࡅ.�{����漲�F((1�T�v��=�)Q�Z���(��M� �u<��R���žܵ +"�7�B���r�l� x< _^��R� � +j`}V�8�T��KĆ] +D���vZ�|ݰ P�ҏ���V��wKN�b����eb�#�L� ���|��A��b ����q@OL'`3�Y&�T4�pe�e� jpƎayb�,r�8 ��|�ы����옘l�I�1W��c�$�H��;E��A��v�Z�~S>��:�px��LSB�r�k����_`��< pSH��6�e�޷_�R%yI7�����C��ؤU�pPb����#�x�������4����|i�^DpPoτ�"�G@(Q�����)"k�z��X��.EVw�>[�����\Z��B`�ty��^G Dqͬ!�QUH�]��T��)���ɞB ���*��iQ�s��ob����Mۗ?B [f�Z��d�B�Jf٘��7@HZ��E�2^0I �΀7���c3BLbe�/���ha��j��f5�����V!ɉe�V�"��*�'ۃ"/7 P�x��я����=H҂ō0��������D��̋�=�)��tѰ�TU ����:�i�k^�������#� 1<<�{�^��]��j����'�ÎY�*�u+f�H����BIӣ�d:h�-J�\ g�!�xmm���c635B���� ��������g���Q����&�G�{ݝ+ �r�v@�l�P�����.gН ���K�B��5��,�ޣ Y���Cb�!q(ݞ�uD/��J�t�E�+ 0c��T�-�Zj�J�(P}ST���W��C�#�e�h�#D��T�,�P�6�Ads�����_60(2���ux#�!�%n˭�Xg0�m*W\�ٜ@A��}�ݞ�� b��T}l:�Ѣs��@PH{&����d5 HZ�>�t � h��Ra0Ü�()#�.���*|O�]P��9>� �yv%J��y�<�����AȲ<�w�]��h�mdɐXà:�.�]k�F�]́o�b�-NM׮��b��v�]�a���Qm��oS�Ӧ[,��W�.��3���l��%Y񷞐#�.V��br'_��Ε�E0��Av�U��b}P�l��s�!0�.W�Y�_��!��"v]���Ŗ�p� 3���*��U�� ����g�r��l����D�n�Xm;�.~1 +6w�V�ywќ�k}��G�]��=��ԏܻ��������+Iɟƛq��EE���w���zw�nj��9� >��ղ���j�F�V>��e�����������h�3����G�pi1� �.��>���샞P)��I� �OB�sxc����Ųe[��(�x0�@����*]x�./>@��e�`IJ'���B���d�G�W���$�Y�;n3���]%�:��YrDz.���A�� �>|�acYZ>�/z��u+z�D/n��e�!W�Օ�����]<�d�ڋ�E���8��y�L�d~r�,븉�&��֙0UwU����YA3#��q�ML%g + M���o�ftײ7o��R�D��9�)��=�yѶ +� VNn�r����g�e\��Imݲ�A��|�-jY:���^%7��{m�o%�"#E���K���S"\��=�b�/�ktc��=����Z5U�׶(6n �G�xp�8Hϣ���昹1��6� +F˶��kam� +|�M,j�;�꿖�z������hӡS�ضxH�'�%*��kA^So���ib�Y�N�U� �f���k��� ��+l�8F��^��{M7_��Q���G�5�hܖ��P�׈�Ud���c$-�b�^��rkY�W����x�� �����&,7Y�h!���;�/3��H/���2�jy��k�p�qS|��,f���y�%���u���{��>���X��{��2WR���ƃ��eUؼ�M�-�� +�5K+�����Ӥ_}���be��%;!p��~�_�JJ^Ut)��E����,/ӲE���QT���j���P����L̰iqڒ##�J�<�����bUw�>>���������C(�h�c,G���� ~`�E��]$��^�E�앣Hx�W2ջa�㓑�b�4��~"^̕MߛHx�Vf�,a.�q$�X+�᤟��k��}x /�J�r�'#��Z�$���O�Hx�Vj�yq]���e3�M8�(��J�'��Cŋ�k��V�]ꭕ�H���r ?�V"���n ��.~`�E����H���r �+�@���� �?C4|�@�@{�$ˑw�S���< +��-m߶v"����d���}���f|�c�@���ȕ��D |��P��u"�UYhvY�q<�"�q����U���GDr腭���G�uoC��>8vt>"`����%O��!��q��Ci_}�;�������M�5��]]�����\��-6��K�糚���s�#�Ġ� q���H���~2����)�W/o��X�4 .�:^���C�))�������LmkS��A�1A��N��S^=w�����b�K��fz�!r�hx_��R�Z,K�����a�ge }X�0��Z�v��1��.q�� ���(,H ���O�|�ɋ�W�N���h;�} ����$~� �ǃ���Q+~X~et>x7��!l܆ی�?���8?�Π�ph?J�U�p�4�Wk.���!��~غ�~vCvZ�ެ��F�'�"�6O�)�@�U��G%��O+�ơ���,/��Ut�?�)ZǛ(�v�+[&rP|@�ǓK�0�JB�f;Kn�|�"��s���� ���&jv�t^��6��+c���kϳ͆�Uf��)�,*/~+~B�ي�3d��tP�YB��ABc��m�ֱ �Ŭ���D�@I~л�� 3+�raT8堦�8 �O8�i�� ��c��i3�V #����Z�v�?m���? <��J0J���AM3�T���9�i�P�c(�vC�3�)�>e�uRftȃ{92���*N8�>}9�ű�'�8@c�6�jdӫ�Fj�C�$�2�C-lb�e>q�2]��9�ƥNɸ��ǁ��nm��f�@fC� @�A[`R�\�F6O7*�Jun�F��yk�F�X��1'�Ԅ�z=��x ��ȳM%�;��G*��s|�IM��(@^L"/5�t��u�������#b�3�R�SE�,�s^j���DԞ��K��{��\'�Ժ\�DF�;>�q��@'��Dx�����K��ý�Z/�ϸA��R���e�W���>� �XK��^jd <��>D~�� 2�j{/5�Up3+W��Nx�)V�R<9�.�^j�1ߎIc�8�x���l�7�y&W�b\��T�30a0����d8�I%qڪP�)�Z�Nj}S������e��hJ,L��nS��u��Ν��f$]x����)'5���FNj�!�3��}�ְ�pQ���%-y̸?��� �<�-r�CM&��<�s����̹9Z�Gj�vf����PkyNx� ��<�dg���x�q��Ҫ��m�y��iYC����������vG�������X9��I���Z'�.j�T�:�QɢB�k���靛� z�I�/J5O%���}��̰M�`�� oV�,~l[5��Yp��<�QS��i��j�f�(�~���Y���(+��k�2�P4�:����9��C;�Ɖ�fI������̹[GI�j�� �d�8hJ��j�:�&�s�:rP���CMNL�����G�>�P�C��0�(uq2��t 2��@�dX�5K�1SG'��:�8Ȱ6]� k#�5Ȱց�A��t��z�5H��a�A��( ��0� �ڨ�6�Z�� �F�k�`m�� �:�3H��B�A~���jn�Ew�P�]mԟ a8*���b.ɮF#1�����u���Վd� ȓ����j|� �׬\��9fr���a &Es��Z $���� +�;Ʒ�@�\��0` ɮ�M6�@�]�.X8�ve6W�jiR*^�a��<�@�]m�Tk��$-Z���@�u�: $�l�o��$�Z(��Ͽ����j�SA��$��J)�$�a I��Hɏ1�dW��v�(������H<|)�1WM� $�ն��FH���u�{̑J>��p�u�E��Y�A# $��XU�$P�o��Z��� ,����j��V/�^FHR�aR���,��jy�g�C $���TT�v�[�)�Z2�0��VCSº�Ca ɭ�-N%��o���|xb�rI���C;:0��j�V� ����dn��VA�An�N� r�u�q�[m���Fh�[��@��j��j̢��R���@�Z-Ii_U_�: TR�i�'Z^Q;�2H�6�@��j����j�Va�Aj���j�d���Vk�� �� 2� j����&Va�Ab�b��Tʱ,zQgw��z����"��)�vFv�Xp�ʇe�R6 ��d���3; Ҹ� � �[�i�:�%i��f� W��~A.a�#�g���E�,%���� (F +5.:�U��)��q WM�q�h!WM�fp�%�M �j"���[���~�]E%��:Gk�j��{�p�4nM�\5��*}WwgK7���o�$n��b7����fx�$q#��5�Y�oj�U��&�t����V6�o�nkoҾ!I�v��G�&��#�Us��z�of��J�m��o�$n��g�v%��� �VI�)5�[%��������T��m��u���q��ٜi�:�5����3`T\�o�Y<\>v� F��K��6:yF�������(��m�6�'twT���p� ��6�,��bST,�Y��[��I`걕s[�|�(�\>u�:��6�q���q�x�N���_B�_�j0���ʂ��,��G�+hڸ6Az#�E6G֎��w5���^���"?[�T<�6��o���4�MF�,Y�G��M��� +��c[�ޢ�$�+}mm��[�C���ڬ�FI b��o1,��M���)ڳ� �ṻ\����&�u���Q����\�F���:�$i��}`���.�}�;�E.fʬ%��x+����������;-�ѓ'j(��졟Q8�y�@ �Q+:���6���W�ɶx+O��+�\Lё�F'��(ٶ!�e��nV���5\؏n[�x�=� +}������3�9o��Mg��I���q� +��-pѭG�����0�e��;������V�mi��!��MŠ?�ά۵�-z�6�)��������[��M�.��{& ����pڒ*��h����z���-�� o���o��EZGGl����a-i�n��Z�u��X� ��k9>m5s:�ZN����K��N�9�n�;�HK��e[ ��Ve���=��Zέ+����k9��Mm�k���Cp%{�Ů3��JwX� l?-@b�Uo�Ζ�6�h⣦h��9?�k��Ǝ���K�)m���M�I�!�r�75�O�+[�"��u~��r ���I����!� ��Ot���h),���򐉾a@-ƥ��||[�F�Q�-6�yݏm���P(o�AO��͡O�jy#�6�����ƨuP <���& +\wP�F���'F���16����;^$��!��EZ�J�'����j�0����@-��L+�$�C-F�B-c���n�����ib�H3@Z.K":mUh�M����]��zh����hc(v�!���!1,���r�Ɂ�ӬK:|�dd��x��>t{����k;@Z�w�D��|=�Zdc�C돯h팳|7�<��(p�ʑ{��_J&����{�h# FX `��6�U��ֲ�J@]>@�v���ER�hs1� [�ҌRSA�7[�%-5�-�:B]|�|_Q��vKE�Q�g}ڄ*J��q�+�*੃����� 4��!� ��O�0��&� �c���6���iE\�Ç����[FM-�u�<��Q�\ qzh-�ԓ��H��'�*P@����c�0�´YvU��!��B�[�0� ��U�>43�NX��,�r +Xb7�Ƒ�nɐ���\�%�T,h������Op�q&�ӍfQb6y��ۡ�6�D��U�ľ)��c�1�.o7V��� ^N�:���['��T�[��U�=�LD�����<6�xw�����O�����0+�!�E��!�6��|Ns��� ���8�!V�Q���Ftn����^;���̪֍�?���EW���3��{��`�����~��b���W�/�n~�~u���~xz{�����+}���?�>��P�+��m`��)����?��?������w�����'�_���C�������n_ܻW����O�ɟ(#����zg��d��(����>��?~���Ë݃�����<�Ã������3����3�W~}����v������~�9���n���m���������g���_����ۇ�����E������������?���Փ�O�_>y�#�����o����G�}p���ﯾ����W�-����o��WL������������u���48%̎�^���?~��v����7��<�川��)����_��W����'~��fj� �c�Z[�����(�F{�*B�dX���qEʝ�9��Hӂҙ��t�?��B�Xh�ֳ!�y+1d)����o����׿���5'����@߽�M�p�� ��*&TW��ֻ���iu��v9d�I ��o��5��I��y��Ϯ�|�ţۛW��~|�����g�1�o�}��gB��哿ފ���_?{�����8]u��λ��^?�m���??y�����>�+N���W?�x���� ��/�Po��G�_?{�Q~n�ϱ O����O~����tQ�C�Ռ#_���/n���}�o�G����6��+�>?���Q���Q�n]z㐾�cH���?y����]X�V{�]���c��>���c�<`{��������'/��d+Ji�>{�k񿿾~����r����-��Ŀno�r��7�����:5����7��z��>���'�:�n� [r�� `_SI�Д�qU�zz��3kM04)[>C%q�� ��Z+Vp�0��:�� �.0��xy'a�a`Y#,��xKgd��lu���=Q��1Af9����ۄ90���K)���䕘��(^�'Y65D7�)~s�� p�F܏æ3\�T�Y�>�z����C)~��w���#?]� �7֍��!�>���v/-u�-A|^9/t��n�$�N�=���!���Rt�Ꮩ�=b�=|+X��!�ء��E�I x���N��z6�b����Ҕ�c"���jD���j���ȗ�*�9"�q��Q{�B�j0��22��Xf���n�=MG�������� Pă��3�s�Ӂê�\��K#Ef��=�D 6�b��b�+(�f7Z\�����������f +m:�3�\�Q[ݎӴw��w���m\-�~�l� ��hyn��W{���Dz�R�Gc��n�"����j�X�}T�tI$Y�,�i��m+=����?��\HƝؖ���2��v��c =^!:X�wtRF<�C7`�ίM{I��Ѱ����3n���� #m��E;��$C��!�bP��ے�)�=�N���\e��o2wl�5���i�rt�1�r�HK}��� "��Q���Y� ��� �e�|,-�5�pI8jM�i����N�o{g�#�q\����b� ���Q� ^��� hA1ف+?��;ù���!))�8L8��K�^j9�T[�,z�n)L�&��eB*Q?�J�yBkn3� 3l��V#Ð���y��̮��e���\���M�" |�t�/ I��Ի�Th��w��llq =%�HL����Ii�h�R��6�'z��9ovS�,�^�,X�c��:�4���M�v7�3���s]d�Ģ�ީ��&T�B^$w� (E���e_����2�H}��LjE/H�p��T��"�S� �l�����7H(�r�bB��F�E��o&q{P�5U#e��˨�����K`���>��Hju~H�K�JG^�F2�DUu2۴󽥏u6S|��$��[*Z����jq3,п�����P TCw@��'��i���G� 9�/����lJ���[u���$Q�̗k� +ŚxV��S��v�L0CI�4�ՔH#��T�>M�aN}�;��$�|�M�R��JZR�~����B"��mL��%t�Ao0��ر�Ó6tM��2;�N���%�z!Y� �g6��ZO(�0c����.8 ^]nht� |��ʹ��@ +�&�d�[B�����#m�g$i$��Ў� ba>dE�nXK��:F :�k)5���˓���#�}���8� +%�jp�*&��l��?3�bȖ +Bb�7� ���q����2b꼕��7� +n�=�R����!5p�/G�|���JK(>�i(M��6 � ++�����o��5���� =e�Av�Lu"��i������n��d�Ohw�J��rmj9�)8M� w�i%��� )8�b�ԕ�P�YnU[ +#9DՀӛ��r,d���(�臂��B�>Y�t+s9�dfz!�H�W)�O�����W�0%/P`�a�|��$F�gq�� ��>��B[�,z�vξ�9;,9��B�+�TO +4��xH���4q��M�0��[�O���w� ��BGG���<�47 +9kD ��.}�N �N���Y20BUt�úT���)Nw9/6�x� ԢK~�Fg�^t03��J8�r0�" ����8�ZT�v�i .���m|���!�tA������ª��E70���u|����c:s�b�@�>0Iο�%6h �|1�@�% ��-� /C;0}��R��F- +"��،w�ׂL�*�(�����:� C�m�';JD�-zw5�q��#���}Y[�гMO?k��U���\�"�p������Ϲ�>t ��I�4B��t2M��]G�u~� +��G����n����Y���(e�-C����4��2����`��7�93*bmmč��q-O�KU`5�/m�e��'�Bg)�f���66o���R�jMR8�D6d&?`s�P�es�"��:i��)m�U��5�p8Lk��`Gg��|��J�Te6Ʋ ���P�L��A�M( +0�7f��V�9��<�f�#�#�%�V��jhG�}ІԨ�>��4�_tl�ݡ %Q[H }������Y� 4� �  +xm��`���l��T�/��+GZ���DN��?�f�>'��闢|O菗�6�������s�)m�� ��[(�vo���HW�2�Z)_''����Ja~k��c�M"���7�����C� ӂ���� �ܻ�߶����6T�O�$}�y��r�~��rJٵ�:��L{7�H!�����Ơ��H��zA�tT�F~!nܻ�!8��� ";G�Hk}�ޱ� ���T 8/�����NtȒ�~2m+`D���'�V&��a��5 ��'௓� eg��u���S*�\@mF:�E�U0l���5���)�B +��AK�����R�e��r�?K��H�3Y��i�hF�:�6�y��Ʉ�.Z��.�p�"�n���Dsqsڏ�*閪��|��^`���g�击��u_4�yQ�Zܴ�S�Ƕro/��4�]qs�LK_�@7W Z��A�Y�V�9p��� ݮ�q�ލ����H��=��Z8�q��җ@�ڂ�;��otp�P��v ۘ�m�?Qw������Ѧnm��Q���T|^޷�ckX�nC�'���P����L�k���Q�!%�$[yz�Evd��U4��;� ��ȣ�^�t/�����*e9��5�� ݽzN���8q&ԋ����L�"�beOE�  Y���#k ���l�9��A�^-u�T�����L���3����6�6��'��W>)�&�zӭ�x�rZN��|YQ +5�e���f�d�h! ��S������ 2*����1�/x�C }�s�F:�W~��vP�}h3�t ӡg�w ++M&�g�=ns�ZI��bE3��q{7�. ��?���i�W�٢��і�"� "F�����6x�0nl�+)}��2Q�=�+z/[�M�致h��6퉿 +9rL*��7L~�~����{��S1wI��q9М�ĵ�����J^��;�?�~.n��K~64Mt�ǺMKt$�J������i���G�n6��ཕ�,�U&� ��暱��BXJ����j l��&��� �~kv/����!P���z�?h((o��1���1څ�f��BR���<���{����aB�"(E7^�ڡ��h(ݭ nq��uif�Q�р�xC�������b9@_�M�aa���� �&��(~xy�*:�V��5��L ב �ۉІ���?0t�P ��>��q^/���4�(Y��k� Ѡ|bm#��xN�TSھ�)Y��v��>�����;�PC �W����I�`&������A�q�M����vB⫇��D*ƦL�Y���߭b���q�~�yW֥&LL 2}!y:T}�>�'8�oV\C��b����D��4#�(��θ��$��W{�0Qe������6�J��T�Q�l`1����9�LTl���d`�D(�M:�F7�4�*����Qb��X}�-hO�U-�|��T�ʮ�|����Ӗ�D�Z$&��ͣ�n Ţa�t��SLBfQ��ڟ6i��H��E���|Y[���C�F��l��a���d�4��Z�B����9v�v�]�����id�ylґ���F��qQ�zz��� +�:�k�:?K�� ��Z�u�y���+jM���a� �IB�Yj� ��3��@:#$�ξAm��"��w�<��'X��7c�A��)V�����D&��gr�#,ɶ��"��l�M�g����̺)Lуm,焷C�й� +Ȭ�%K�M;��kw/$n�n�p��z]����,��L�vq4?�A� Ł�*ٍ�&Śh�����){�P%��m���?2��Ȟ�l�F� ��J7цd���� ^��olbG����/whW�|��JנOH������_�}�͞�h�bS���"M !�a�?e� aãY�]����Lcr�a����>Ӟ(����39:��H��/nR�zy��:TBE����T�Dk���o.�I(��6e7 ��P N��O��P��N�ŵ��4[�-*AFQ���Ӓ���r����� ¾pC0Wi�[�:6[��sW3�G�+p��ߊ��j]K[�{6�,�9�uS��K�3S����!n��s ���vR� 8��v�&8���5,�Ҩ'����/�+k�cn�y�P?��hD��mX�(��x۹=�_N��k�=T�3.�z*�V2~��u���&���a@��HV2�x�Ys;J����>�{�RwV�� ��"H��Z�u���fĜP�3���(a�i���=�Q���%w l���UC���x�]Zqyy˝r��i�{�eɘpl1 L2�|�ZMa%�+9Z=���35�8xK=�@]���á"��K�<��Va�'j�: k{]Vٴ�.,�-�P�Iܵ��Bz�3{�J�{�{ �7~��eL ��NL��&m�Wː��J�3�W�����OϺȐ�Nv��tu�|�� �䘇'�&����IZ}s{)tR� +B�}��޽��=��h� ���� ���H�����]��t�l�J�7�J=������#����I/Os1̦X�.���(����<0�3f�Zo�B���MŶ6�L�~�lm���ѯ��@�1C����TL��;�Q�Z.�]�d��L^�l��yL"�f���p&k��-�(m?��ZI�x�����4稹��_�g[f�����'��]3P�^�ͻ�D�iKP7����<�� !>}���w3�U͗�6P�,3[[:q���BQ"��ie��96����N�u�em�t#�wb#�I�;���W�C�G��4_�޲u��#�Z����?S�h >���X� �<>[y��F�!<Ý��U}#@�Z:�Pޙ�YipI1+�{�� ��ӞK��}�M��T���hwst^o�pa{���c��#�YKxH`��#�Ϸ[8�Y�Df�sos/��ٜ�Ӻ�&&l� :pͺ ?q�PwCs��&N��8�e3\8��3�roC3�T(?Yr�Sd�}$�K�z����98�����zӷWq��rtm:��X7��r�uAC�� ��X�.:���#���m̀��_B�.�����^���ir"��Dt2ڤ+5Rq��r�) J��ݬgӢ�/�E�wf&Y������ 3�2���8nv~&����wM�eG +��@��|V(���$��� ��]�@�66�D�M���/���7|[��-a���5v���ף�=�HT��(�PQC`�&y���i< +�f�p1xP����ֆ�S-�q��Iw�I$l�����3O��d_kY2C,���V�ܐt&�Ca� �;���l6�S�������ծ&$]�&L����=鎋tu����s��Z������� + O��Y��7� �s���B��o���H���!�f��x �L�{�vK��� %�&���ӱ�� �7�)�|��4гLMz����6ybd�͑-��vœkzh�%O'�&�i��<æ~; 挖^������s�����v�vY���P�mE�̛���.Yu�9_� +�S*+4���r�H��M��FgU�zqH �ܼb\/����<�7B��d1�8��g<8�\��~����R�#Z�����Y��r�o�S0� ����]6�R����>��c��@�����DQռ��7x����������o�������~�$�J�|�2|�2?��h��ڢ�����Ϥ1�>Rc�}�ƨ�n��@`���F����L� +ө���2 J)���2�ѹDFa���J��r@8X��酂�a*�_���H`t�(��F/f{�Q=�c�Ce���g�џ]`����,0�Y`����F������w�ܷ{���7��ˏ�𸣍_����#�3�:�w��O�}��$��CMhXl>qQh@&���i���ndĴ3�Z���!6�>�M��&���@>�n�~'I�L�s�X7G�J�~�߾������o\�'l!b�������L XjgS������Yz� �@ +B!��i�s�[{p�L�\@mȦ]�3�Xš��S3b6x@z�z���❙�� �t�k�=��h�d�M +[�?;������V �������`ӝ_�����i�$������j�4A�p)á� ��ϱ��@�f����7����E �x��נ�!2R�"ԼN����� �qSz����A4��H�w�-�.���_l�xVF�J&�mw��G���&�u0Ѵ�O�: �G`� �q�j�������x2{�w�ȩ�u>i���߼�����_=)���%#�S%$�������������������(��z��Fb�����|���]O�n]M���$2��ߐ������-o���Gf �@y;� q1�g ��_�.�|��斟3?{ �� �� �� ��3�4-a�7��?-�+k���8�U~��/߼��w���������B����~}`��w�S���Z�=4S�� +��T���OY/��y���0Qʀu��*@�D�hs����*�ޯ��*J� .� #�y@�o���=ۻ�����_��*��I��ۛ�C4�Fo��7�R��D��+ĀV���|�܌y��+nf�dR)�{���|F�a�5��@�qf0�v��y��n�L����tE�7#�0@��5F�hT�IqU�����t�6�h���Q&d��^�"�Ì�͟w�ڄ��iYږ�����'!o����4M&�M�'�<�4>5�mo�1��6�E0��w�?��\'� @�\S�J ������N��:�Q20�S�� �N<)���Z-�5�� �a�� �6e��k��uF�"�01͇�_�oF�Ϭ��Q�y1�Pt��. I�-�\Qہ�:atC���1�ul�=S�⍁zd䕐!n��M�N�Uo +-D}�R�ڹ�����B< �8X“��j�zm����+��D�63@: �m6͛ix�n�־(������hx��|������ɏ�B +�� +���O�3XZ^:&Z�奁����!<�����UvjS�z��@0Z�[�JU��5��x�H�)^9��S�Q �em0 +"����F��Yн�{��O5 ��� :�ׂ�:� CY6Xђ��N�X�fiM\���Y�X*�/d(�S�m��SK�JM��l����x��E/�v� iU����̚ ����& ����d���������E�8oѿMŅ72������V#JP3k��a�]� �y}��VV;d.О�zIP�� "^7�2�`�k/������2Ӵॻ��nF�Xxlr$5�@��p5��f鏆���3#� +�6���Uk����TG��(���y�l��fx�hBe���aA��@v����Ի��L��8T��@$����[��Ěg�I����j�D����4}�9Y^8t�����BwϬ��BC�i���I���@mMςmh���}�w��O�ô=�6����[��Gc�F���{F�=�:����*����Y��,��dj�I�q�h˜��� �>AY^�6�N�7l��؞4������>d�$֯��?����'͗�V8��)��}�ڰ��. ���AF��q:�����>�ǧo�L�.���su�t�MK�����~D�����k�8)�Z���m H�,rG���W�������E�������~քtC�0��x(o�%;=�ӀGз��S� +�95)�|h�R� +�3��@@V������~d�����i�5�r�r�O}��8޺ ����|����Iu��W�� +��Gճ�����W���_���t��J.�?=��� �˟'وN��u/���~�]�� ��~9"�M #E2�۷W�����\��{�+D� �O�_���o����^��?�|�~���on^��/�������ۿ}�|�on�����?�nᗿ�����?�/��X +endstream endobj 62 0 obj <>stream +H�lTKo�0��W�qs��3~��v!��n*��B�P�p�����>h�Hɼ�}3��Z�n ]�lH��x����&Y����&�4��).M�ҔP[���A� �A��A�,�A�<��Cfד2����j�ng���ZO�!��I�T|��c-Bb�L�4�j�M?��F���d�<�o���S�>��;�N=�.�W>�䄾f��9��6�> endobj 60 0 obj <>stream +H�lW�n���W�Q:L�ޏ�E� K6�\���i���A�5@��Ȭ$p�U�U�������~��>��x�o_��Ζ�ۥ�v)�]*o��ۥ�v��]o��;����;��w ��X�1!�cC|Lj���3�;v$����v�������a������c����˖���K&�y�yL�u��i�=��������6��_��G��Al�B�߅� ,đF)�A��ɣ�{L�I��:D ������ga2��42��r����]�`��]�`z�8��5�pHw@�~��(���>@,Xz*�.|L��އ<�ju1�G~�d���M�O���(0���6�k�,�A��{eu(+ߵB�s,�{�#ȷf_�nZY�b!��jT�}�<]o��(�7�V�+�3$t�X�:���� +�T��Xz��*��i[��� |b?U�C�QՇ=�<�։=�,ODT�����~���-�hU��@�B�R3C��n��5)�ڭp��Ա��B�@/HTD-��n��u�<��]�HVqqr�� �>h���o�MՋ����� T.�D_��<��l�H�.�/�P��8�ÑՀ��)� +����(IL�����_[b�P����!Hn�n�=7�WT��J���~26���ׅ����w�U `a� $�B�mq�G���0o-�1���'E�Fi�� �/� +�BE����y�:1qU�����`Dff�W�szg����V�4�����y��<����@�(��E&�u3�5�����k��c�� r�ҼRp=:�Lb�� +{�9D�fA;e}M���XL>�&!�2g��� �:K'CR!��5>G�쳛#(B{`����v��_Pr�Eڞb.ûd�fh��t6t�1�� �q��N�����P�~�bk˹d�x0�(��:5d,F��3,8op#l�pt��&������@����H#fq��" +7ve���&�Z� �ԉ��S�B��g�3 ��e��AKݝ�\�%�b��~B�bA��:��~�7C mNC��@4�44���X7� _YZ�]�� f���~����H�O`#��&M/�B��}>���~����6P!���l] �dI�Z[���Z6�sQ �{�֓�h �1�P�.?yo�ᑅxq�3�0��1�?� ?jH�q�J2"�0�u�^6k@©>�v��1s�lyySԠ(��t�Sx�=�.spq�A�p��Wآ�a�ݛ8@3itÕ�����K�0ùL?���Y�F�Wz�I!��M�� +r��b�R���%� �n ��*�Ddf7T�q�m�} s�otNB�qU#��g�Ɠ��l����� ��к�)d1$� M�F�/�V&"Z�0 ^��f�ƚ4�%D�jB�$�t���w��'F�XEHY 'Ӹ���4�X�ܖ#�� ��lNi!�A�@�M����B�0�0ቑ��,�H�-y䞝^ v��UE��%K��q4��!���8;��>'��zYa7��&@hER�2��;"��9�O�/�A���O8V�42BF��@�.�yj D�@���\���4}�co��9Z�B�yE�)_�j��o������ rIe!%�ξk( +i*��ːɐ��R�K_��) mt��Oj-)�uٚt �Z{�zzV�6������f� ����޳KU��n6Iv.h���r �!� FA�=d� �՟5c9�(ц]�(r��l`� +��RV �j}P�]ӌsĊ-�p���IbW�n�]���H�����]� 9��3�ռ��d�����u_�*�c$^�$�T���p#�(X��ԆJ0h��Bc|��N��Hp� +�Xm�T��۾�2h�x�/����_�I8!pE��*���h-\8}�Ќх���$V� +őVs87�~�:��p�&���LRT���O֫p ��E�����&A�hzk�(���]-���0����,ɒ����(� ������pH)����,�1$��H�h�~�'H1��8qWqZ�/�q��r��փ��t5�UԞ�c1�yW�����y ��ZU�t�᪣�� +҄t �{�> v��Ji"E�G�e��B���l�����xRp��Y�߈rU�n��|�����z�^oKd (�74�}$��W��8j�� �n"'g��a���&�sQ�2��[����{j�Nٸ83�g��5�@��c�IS(�l!'��-�;�N����ρ��M}:����^��@a���bf�Q[��PD�-��"�Mh�|h)���f7_>g ��L�� �F��b9@�n�6�Xq�]�/��X� +�d�GB�F��Y� ���2 �;l�U�R%Ңi��A�a�>��yA�l���� M�F�dX����A&���M7�Bb��/�W�6�������sn&�D1�ҤHə�<��T�|�`���� �Ɖq4M�q2%���|��dc��.@�O!<��, +���}J 6��P�gP��8VWœi�V�&܎5j�PgF�w�p� ��Q$`������"�.��0��\��/3M�j��?��IA��bD��91qo�R� ��}�D�J�Q )b�:c���� +���g��>>�o�������2��7��~�+�z�k��B�/t�I���`�d]� +endstream endobj 61 0 obj <> endobj 58 0 obj <>stream +H��WMoe� ݿ_����W�DJ�d:(Z4�q�EЕ�(<�.��{HJW��;SH��"yH�����_���_������+���W�����=�����_��W��Uy��^9��_9�r�p���•;�+�Wn�\#�{|�p�����w��_��������a?��÷'�6����5li�����t�����}�q����]{��y� +g�{��雛?�����n~�ׇ��k Ϭ���y�X!� +w��]؂T_������[̹��w�t ��|qKy 盇�|�=��N[�Qϡ��m���:�p���oO��)���#9�����Y���f��e\��M��hK~<ᨹ�'�fO'=2 F�11�]x�O�@�+�I@� E���t����bXW����(���?�榄��Rkb�Xu!|S�P�r�3�n��*�8�> _���=�Q���G�!r�gDڴ}K���xj`����jj�Q�4�B9�~���t\�����cM��8 +���r�;�I�<�W�q�N���K�pTZU��dv����O�ܔ��eqSz�M��O��I�쨲8�\8����σ�R틺���,�Ud'j����7���o³��ꬴ:+}��8cI�B@A�]@I[S$�5�(E��HS��� +��^R�U�НI�����qiH���gs��S+����G��oa7Sp��́q�G � ��dO��~�NV�$��r����E�#�21b[�8FK��̪�Qd�Pѱm��P��L��q�H)��FQ$)N7�[�pt�hMF�s3�}�%�ōI�,�?*�£����"b�Ꞟ� vE�hn ���{s��! )�iBJ^ E����x�G㳙!J��n���h���l;�<|��t@�9��a=�ʻ%J�8)To��8Fv.f3�s�bK��&%�}_�Q3E*�vk�#!� H"�8!�UO��ܲ��=) ��† S����R�[ "V3$و��k��]@xb''Jg��3-�]��0��Ͷ���ib�&�0"'y�q���: ���B=8��01.h$�<�8��R�B�j +��E�yB���``��D<}Q��繑ib^ԼB�zQ $���A�Hp Mf��0�0`�aD� ���j5�����"�P7���#`�C��@��hca0x/�3�0"�10zY<sp̃�j��z�G���i�0"��`a����&�\3��AD��m��"��'�VyP�R( .{��i%�g��{��Gp)�pd"-�8� ���������GKc�चDZ� ~�b\����?����@&~F5� ~*��)X �`���G�Āɛ��B ��/�,�:i0yٴ�(p�Vk�XX�;ؚ߽p��(��7�ʀv�͋�QY��#e=`��W�ִ{�-��6�<9k�b����[c�,5���9K��YK> o����9m��h \��:r�b�cw��Cj�a�p �%�^*�E#�~K5���<��\��R�[��� $y=�k=7�磜���˹W������~mWђl}�ʦ�.אm�嚲�|�JT�о���i��7�*�j��ՠZQk��h��j��Q�Х�S���*�j��̠륆�������5^�:�j��<5�V��렝]�����Kug��;�^roD�5�F�ͥ5u���d�/��U7&�̀�ُ�v1�=͏����9g;Ts@m�/��i�v���)��]&Ѯ}�x�u�E�D�<���R�~r�/��ݼ�>q�y������ܛ�w�w��?7��e�J�h}o�d\@��zFKd)�/�)t��pG�}��X�dP�g�?�3�/0\6�F��Ge�ͼ�eMY׾��!�)���R�叺܅T,}r5,Y;�QZ�=X�-��/�6��3\���hG���y7��������E3���z�u�b���1��|�Ka�L�]J��]d�"?_��~����%�C�a/��$������zܒl�@�Վ�����e�v��%�Ђ�b��z�E��% Ƽ J5���m��G,A�1{���..�!�|��L�^�,7n���ֿe�O~��q4�Ԙ�F�r?�|�7��&��`��V{��,��p��&+,~$K�~ _�f��_"��S����|a�m�o�_?�������ۏ�����O����7c�͘�ko���1}36ތ�4����W3� +endstream endobj 59 0 obj <> endobj 56 0 obj <>stream +H��Wˎ\7��Wܥ{Q�ER��]i$���. ���:��`<@��r�jw�7�&h�K�W�(���w�����lo��l����w�"y)җ"{)�/E�����K�x)���`�@N G��3��!������z*�>�|���C�~��t��M��O'�� ?ReW�`�^K����ջ��Oe/j��q+�� ��?�����;x���?��1����W����}�ִ�X`�{W�~��g��g*���w�����_�پ��P�z)���"�������O�_ "�1��������'����O�ܦ�Z�nR�.����c��x:K�u��ls�:����Z�Y����`%u�c;w +d ���ԁ��}���K� + �c�,�$ �|�m/2�4`��� g@�)w�0w}n�[딸��bN�}�I�� ���|��f}���Pb-%:rF��e&�ǔ�@�S�jL�� +U�T���B-u��*o���RK��.�HF��ڮ#N +�(��luw �W-v� +p��� �SP����t��o�bL��s���%:� �j��ȯ���N#���7A@�gH��Ŝ�w�&�J,f�C�aZX�87X�i`L4x��2 � j��cA����qw��Q����+��L��!�}�n� eZD�&I�i�C��`� � ���IG`[p��q�=$.qr�Kq{���p�s�w��oI�^�Eع��3$������yb�w�����$&nR#�1j$d�"���:wE0��z� F�| +�Wh��c#;{�^�MB�~1�3�a���{��w8���@H�0��� +��3+3 T�+��DUl?�|�p�rU���ݕ�����-�?��BXr/�����{0 ��{� ���$<�1�CT�Uv�p +�q%���&'r���#���s�;2 ��F�Ō ���&�8B��p��S;?t���`����pĬFX�d±M��N܇�g�O[8EoV �"qV�ey�m����c�n~r�X���w���Hxȹ`*�ޜS‡�6V�0_�E��z�Hr,���GfSC�AC~#=a�-K rmq�'HU��X�f +HᾁR3� e�a:]3�2S.�y��t "���>/�n��H�^r����K�1�lm��0��� �#����"[�� �H��*�k4��b�>15I��{�#�ϊ�#H�#H ���,�9��e�Y�t�L�,?�@@��Cb-5Z��b@I1O��y�̜�{%�)��}���&�"}�j�ڱ��f���-o��dC&����m���B2���L�MZF�~;�H������n�V� b�AJ��X#�������w:�\+��@1�J���-i~��Ԟ���Y����pBO�Ό r!��K|o��4v%���n�NA¶�Y�r9�Mb���m��R��K-* q���&��F:C�����˭�I���Q�Y�S�k�蔰R�/b �#;ѰS=;Q��� +�df��8�/f�M�K"}��� ������^j�E>ᾔ�\�m&��:b`��z������g+������$=�`X5F�` + L޳��\Qm���d,�̝�瞊H�{j��n!�%%�r�D�|��s��]�Vg +�b���ާ��e\d_M�6�+)} +r�3ظ���L�h0fP��'L'��f\R� +/ �(����hco�$�g�dc��A!c;�U�*�8 +�9��G��@�K��lB�@��P��L�O%B �@�/ �b�,�0�XX$ ���f����=�j$.4!�6������� +����fr�#h��6�<{�?(�8^����������A��K�|x��|�X��6��8�{La�:��e?��^D���76����w~ٿo/����e�{3�|�\��@&2=�ف��ځ���@6�ɾ;�&��祙 +endstream endobj 57 0 obj <> endobj 54 0 obj <>stream +H��UMo1��W����:���؆B��vQ�@[��E��y��7��Bh%�o�;��xf����͇��篷R�>`��)�'���FƔU"#��č�!����^l�^ky�S<j<0���7$�V&Y/����<��hes����Q�f�C*� ��:Έ�,�b~[~G�U)%���R��QT���� \R��uRAq���PQQW\��ً;q�"o�L�]���䗦�4ť)-M�s��;6['Ӊ�tB0�L'S���gy���$���6���5މ�l��n7y��r�'9>���z�.���>��Pw��V)x|8~��O�5�E�>�����n�"fU���2U.ؾ���yGJ�������zwv�}'�/ެ/W�Io�6���ծ����1���'bک�L�5/wE�cʓ�� aY����v��x%/>M������j;��W�g%+CT�����x/�W!>��������rJ*�� D�D�|�&F{e�=B�E>�y�@����b'=�dq��F��"��RÇ9���p� )���S�����$c/*<ʜ %�jps����E�x�G'�����29�qrj�bL!^yI�ź�Ʃ��]��:�v +�%לT��x��8�;T�6 ���F����O啋���@u�%(����YŐg��u�@��_OT� +��X㳜�����bCPi��S3��MhQ� �͸4ߜ ���6.|���15���l5l�GI .)�֯�j}~}`�>4����O{ٸ�3�MglSglSglSglSglSglSglSglS�χN��� +endstream endobj 55 0 obj <> endobj 49 0 obj <>stream +H��TMO�0 ��W�8=����M��P�VbYf"qX�i`@+f%���ˇ�#���W��/��]n���889݀�.x�*�K���_B� +KhXB��K()�j�+��"�H@E�"N�q �ӝ��v�M���|0>�ol���!��%�!��ɿ�k7��`g�/�6������15�U��7��7��9��:x)̉T(�dۚn���*�v�M���N�Z7>�*�u]h�*�;e�[s0� �f����LX�\�� �~dS2a��=�}J��By4$�znk`�"�}/�{ÆO��Tcs/��r�#�;;P�g�me�!��6���K���j����p�q;L2��+|���! �8(��W���cga�r)NRԻ7�d,�����@��]��K�s�\����߽����œ�E�q� �� y7� ��Ce��2�H��� #e��2�H�a� bRt�[W�l.= +endstream endobj 50 0 obj <> endobj 5 0 obj <>/Resources<>/ProcSet[/PDF/ImageC]/Properties<>/XObject<>>>/TrimBox[0.0 0.0 596.0 479.0]/Type/Page>> endobj 6 0 obj <>/Resources<>/Properties<>/XObject<>>>/TrimBox[0.0 0.0 251.333 153.46]/Type/Page>> endobj 7 0 obj <>/Resources<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 8 0 obj <>/Resources<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 9 0 obj <>/Resources<>/Font<>/ProcSet[/PDF/Text]/Properties<>>>/TrimBox[0.0 0.0 789.821 151.0]/Type/Page>> endobj 145 0 obj <>stream +H�lUMo�0��W���{��W|�nWP�� B��hj� +�>ω�l6�J��ؙyof�����ۻ��W�{�wxlG�^�hm2k�]����צ�6�kS�P�ѯ����@W$��cJ�h���<}���۾=)���m�NqͻGf"W�ᥭ��[͵��� k���WrE�w=y���w�}g�������������|��31��b:����Ǔ�(ֈ�@�{i�D�� �*���)%�FR0��O��C�_4v�x� ��}��n�ț���9ݜz�� +�)vh�lI��H� �͐>����(wQ*g#wV:y�2xa�Z)p�n=fB{��S�����s�L����5E����-�8^� W=��t��g>�W-��}Aһ�}���ζ�ռi��e8��U+�1� +`�f�22ReQ���$���\��]�/��*���(�À"��,/�D��Y�7-N'u�k�����*����Fw��l�I�_ZB�]� �č��ަ��Z��EȈ�X�����r9�%��@���'����Y[q H.�CnZnp��J�C��Y��[�L�X�e��[i���)���O-Z·����a�ॿ�s���زspa@��ํ&����`�Z8�����U:CC�f�d\/��>�N��m9��m��̘��KA3V�\�Y��U��<���5m��Q�:,ktY�Gv��I�+�NWf�� ;� ;� �ʰ�ʰ�ʰ�������s��Ă�� +endstream endobj 146 0 obj <> endobj 143 0 obj <>stream +H�ԗɎ$5���>v����馅@34%q@�fFhi�����aגU wTRe���>���Ϗ���cp_=�����G��6�mS�mJ�M����6�ۦv��7L�2�~�p@6<����a ��??.�_��ǿ���!8q�K�և���v׺]�;�.wϻ��K��^�~���o%c������v{���÷�8�� ��4Mm�~�ۋ����ǝdw����n�盇���b����y+>�߻R+��V�vX������M*�o*P2z~r�m����t����R��T�˫-�����+��Eݾ�Ա��+���pf����/�z��'���>��:���^3��1��c`�_�O����A��d��S:$>5qXK����o��$�st��(��dh�%-�ix�iC�\�Bux(}��#ҁI��/W6e�T��D�k(g��J7��T(V��8Ĩ&8�}����"ba�@?��_T�6�Cs%����� ��T "�LAD�;�+�'p����S�u�f��r�>�j= +�@�)0w�� +���خՈ5�}oG cq�7=����3q2��ll,\�Zu��C` ��aO��p4uJ�":,UO�q�g=��=ł�!N�5F�Q�R�VQƕ�6Z0�y��d�4�:�J2�!��φH`�J�fK|��nxK�� �q����Yթ� N��2�m��G �^W�2wAqM�&1���$HP<� f�A�<�VX���]��a㿠,�Z��D���'����/�ܼ�⮮����!��!���[�K���_L��n��~��0�°� +endstream endobj 144 0 obj <> endobj 141 0 obj <>stream +H�l�1O�0�w���!N��ľ��BE���b*��H-�s\"�/Q�ӳ��/~�7s�W�f7s0����,a)Q)���J�+���|)�Rbe�������W��z�v�n{���Է��_��t""v�7s0�hHw��B<��Mthɷ�� S�x2��4�';Y6V߲d�� +�� h��c�2Y�QΪ��D�5*�F%ب$�h��mR�Mڿ�d����R8H� ���rl͏�[�� +endstream endobj 142 0 obj <> endobj 137 0 obj <>stream +H�l�A +�@ E�9E.`3I:U�NՍE��'WV���v��o��?r,,CI�� �v��|�FdyDmD9�.�eD��� *���,�Ě�i;�����b���t�N�������������ۀ�y� +��+(@A0tApB< x�0�a�#}^G�� +endstream endobj 138 0 obj <> endobj 140 0 obj <>/Font<>/ProcSet[/PDF/Text]>>/Subtype/Form>>stream +BT +0.537 0.008 0.008 rg +/GS0 gs +/TT0 1 Tf +0 Tc 0 Tw 0 Ts 100 Tz 0 Tr 53.328 0 0 53.328 84.4165 108.1318 Tm +(O)Tj +0.013 Tc -0.013 Tw 0.865 0 Td +[(PE)-10 (N)]TJ +0.023 Tc -0.023 Tw -2.448 -0.881 Td +(SCIENCE)Tj +0 Tc 0 Tw 1.324 -0.881 Td +(MOOC)Tj +ET + +endstream endobj 147 0 obj <> endobj 139 0 obj <> endobj 133 0 obj <>stream +H�l�� +�0E������I��ڊ�("~�hW"���pQ�%��!�L�;����iߵd���L�DŽ��դ����q�'S�2s觻n���r��F�FM�B�b�R�r�J��� ` @A�� 4x(�P��C����`��tX�a�},4spd �&�8���-��mΑ�Zf��n���I�v_�� +endstream endobj 134 0 obj <> endobj 136 0 obj <>/Filter/FlateDecode/Height 479/Intent/RelativeColorimetric/Length 85224/Name/X/SMask 148 0 R/Subtype/Image/Type/XObject/Width 596>>stream +H���0����#�2$)�$uk����_��Q��4 +e��D��M����Lt�$9ʏ���m.�KN\��Ja�(Ih�aO��]���>���m��|�,XPRR��h�@��Z�z� ���V&8�N+�[y\fW�������X����r����p�So�722������z�4�\�G�aÆ��6�����}� ���:�}H���'����#�<*d��\Z�Y�������v���:�"���������*��}@__?((���S��I����#,U�� ۇ4�Ծ����wOo�.9U�A:~��Lj�����"1��U#�>��޽-[���IK`O����ꕒ���}`ѢE���3�8ڧ�����;66�귆�CEj��u�e���@����^WrE��^8BJ�_JJ��� �!���}}}ʬ���I:thbbB��}R���B�P�/ۇ4JA�� ����?=��7�� T�Q��d xJ>�"������� +*`hhhkk g������������暚���숈WWWkkk}}��^ngg���*�/��>8�;v,###S�{bb� |||�n�jff���C�Z���*�)�����yyy�J�Ψ������a�op��YYY��D"����i���I���)�%��g��]���ފk�4 3ĒY~�iD!==���T�n�dɒܹs���C P���/\.�������pē[�����֦�!��>h.dk�h� ���EEE!!!����� ���^�t ީ��2fmd���I�Rg��C5k�����5q���*d��Trū*1���#\���i Fp�۸q�ܭx�ҥIII����������`]]]�Pp*� +���KNե��� �?�򸨖��!_$��{B�Ĉ��K7O����Ƅ`ZZ�ʕ+�������9� ii����}ӦM���1�L���}����&�fۇ4J�����Qw���j�o�S�}�o&����7�����h���{��n����A�7,ԇ�f_�xQGG�4���^LL�L#�ҾO��{��q��666����;�C�LMMI��k��ݰ`�0���C�F����<�h�r*����zU%V3;�ѥ��bDm_BB������x����nݺ��~�/��}����yW�^���L�m>�����!!!,j�Μ9W<&�i�\�'=��������� 0����c�$!��[D_�T}||�mruu�e���Fsss�����l�9��}999�ya%MMM���C�`a;v�~���3 +� +���!��c���������l�Kb�T�$wT��H$�~��K���[KKK�fkddD�911L:�����q8��4��Jzk�V���� �*??Ŋ��A��n +>lR�F�K��n��P|'-�s�G5��WqM��ΥW�}|+���O(z�C�.�����ill��)`dž�����I��ђ�`P匿}𻎋�#M�v�ڶ�6ҝ�}p����������spp�����}H%�I"���sn]�]�?-"أ��bt�o���o����.�J*���|gH@�� �4G$��ƒvZ�3ϟ?�0��x���ޞ�����644P����>>�@���Ύ\��;w�\__߯�R�T�� �������%��Ixx8�Œ;&��D<������S$'9Bz��$L�x*�ʋ��^�=Oc_<�{|ӥ4V��%����fKT��#�$�w��Y�$e�Lcc���tUC������$m����j ���jccC����cxx�t3�}��~��?����}�BI)nx�򥋋����SSә>lR 1򁛗�I��T��w�~�{�.|�I�Uj�}��������-�&�?8�f��'Z�|��lѰ��XHRQQ��g?����'O�&��deeQwrZ���p"## d��/�e�T���i4���D5�=�!&j41F�1�F�� �33y�qy���:��Yh:D�l�<���y)ի�O+2���=��C�>]EWs�� ]�џ�D��l�l;B �E{xx�kVV�e6����lj�N��� �����.���166�xi��}�Z�����;�ҥK���qs"�!�ܧ�2��NJ�(Y�Mi��~��{�'�zo��>�T�� �����y����Ȧ���c���s�aM~��CcN@ ���}s���r�w�Bj���DRRR���Ύh�u��544���Z6oތ۴�>�W2� �b�r�,t�����h9���(@��/�����?Ԕ}��k�A% �#*9�Jb�rnb�6�ϣ1kK:�2�����s߾}$ �3�O��f�-��I����DFF��>[[[OOO��i�HNN�4 +Ai�Ν�f��-���`04 ����ׯ_�X����ԩS!�B6ԍD�C��}����>�>���.��J�� +���a~�������[�5r�oo$MH�A"���)�8F�Ѿ�̢��u����>m�4�E|����곹� 3i�$@�HX�8qB�R�=�a�>�[BBlq�9s�|�R7�a8��%��HY�Mi��O��d�.��� �TqY�|��zٓm��ߤ�3����a�9"E��%{bT����o�ر؞9f�ooob0`����e˖ặ��saa��� 0��ݻ��� �!�}�ƍ�x�boo��6������� �Bѻ! [ZZ��܇0 C��f@9;��k�`����'�T�J���˜䚧��TdU�wq�V�-�� ����^u��("<6L&����TW�^���i�U B���888�Vqss��u����L���3� g����۶m�{upiڌ�܇0 #��@yB��vI羶;GDw� �N�I��S�b��$̏䅼M#��zk��7N<�)j��4�v"F�޽�6m�����s�\K-�R�> qי<���Mo)��H�ŋ�h4�Xl��p��N�����Ņx��E���C8O�>�Y|�}� (� ��պO|����cϹ�d�� �M����X�A��]��͒�*��E�0�J��͛�Mu�ƍ +��"KTVVΘ17��� X "!q�E����_߻wO&��r���}�NR���sii)r�,Lt�Vr6]qE���O�A�R�T=�{Z�R�c� ]�=������̨���¤k��?�z�1PD=��U�u��ϳ�� 2�y�7�Yw��[�ryC� ��D��k`�1���$..nҤI�� +�s�ڵ����V�VGEE9::{���Kss����0aBHH�`,l��?>ߩ���3f��r�q_�ma�O�x��v +����"��jB%xg���g\_�1 �GNHV�۞Jd@�q���v��q��Bׯ_/4�Y�5RSSgϞM�D���#�"�'22r����}��0(�A��_j +;��_S��};�4�%�YG�B^PE���{�5��֦��`�k��^d@�!@+g޼yz��E�"""���My�`*>�������D� +��w�^P��������s�̙���#�!�����gj������!�G]h >�π�#���%����Y,a���$̏���~�''�5�k���w}��(� ��#F4J��J�B���Hv�ڕ��i\R�4,,�����<˗//**2n1�>�@����qww'�H�}-M_VN�Yߴ�UbS h��*�Jc�i���P��\v)�:�O[L�ӳ%�ߤ���έ��ru�� +71�༃c��W�^����f@�JUXXx��Q�Ap��Ϟ=��i ���Ǜ2e +rh�]��C�=���d� �:�jc�ޒ�������N�R�`%�x��u ��YG�镙��i��p�*f� �!y�#�����������e�ĉs��]�p!�q[[[�nj�dɒ��<�1 1�gV�\ �I\�"�ݠ������VG�%@�d��79ħ����>u!�������7����3�G�o � ���t�b��Ԫl��X~��� {s�W���w��[i[� �1��7E"����Μ9�R +�&�iӦ��\��؇��>Q*� +����ж#��>�5 �7?_��׏1:�A}��_��߶��ߐ��}�S%^ix��J�K�[�z�ҼyE ��g�Ȩɹ�*vC� ��\S��� �>�����)Є+**@X�?��!����D"�ͅ�g:B���ݝD"WG�%�՚����'�A� �'p�w �~�s�X +�{Ԑ�Su�K͡&B5+O��^�����+3x�m�U���5�"�F���1�����2�ij�¸��1����z��+��5F��U�{�'��T�rU�H�>� ���B[K��P�҂ +O��V�PPd������=L��z9_� z�:{�}�o}�����K�,���T��kii��� �‘��ا��L�6c߸��}̸���g�W�O�������q���5����o;���Su��4�$ ����䤆� ���Btx_�I̋��4�y# ��߭�T `�7�-�ƨ���D�7n�ܹS[[[8}�t}}}{{{&���ب�L��oD�P�iӦ Xc�8�}1��Z�/��#��E���=Obj�SIgW�'����/�{I�]jo:! �4�x�#�a j�H+Mu͍����_�3��Pcďۺ��T1�stjkk���CCCq8�����ŋ�q�����X�j�����������Ϩr:�&�sB���ߎ}|>_OOO[�ڵkא�q���|U�����>PSS����A.������9���44�}}A{���4��@Jvs����G@�i�ɺ뎯�{��wbq +:��Gty�� ؐK@@[���D6�Y�8x:�� +�&L��p샠3����,k;:��8��0�$옶�(4�'�x⫈�� |�B��R� +��y@�(�/��?����&�������|�b„ &ER�>� ���~��!im�|OS�ɴ�$�����zI�]jo�Pp�4�S�E�g���8��V����h��kH�~�`&%F��]G+6�c„ �+���/��O{v�o��c��tĚ��L�dD�'�¾^��|ɼR|i�wM��"��E":��`���%ɎY��y$M}�i��+1�a„ ӏ��� A��q 9��!��������y�@oO���}}O���Vp���`��pK���B�(�/��L����$��'�(U���#�}�"c„ �~R�}�b Bh9�=�vG�}샰�"O�I�w��!=�����_���s�^"=`�#K�)�@����\s�,R�#�K�ɤ0_��w��[���S___YYYRR��� �HTSS����M���x���X,.***..�x��mww� +��d��A���R�H����N݂����g�K�F1w���������p���R��� +�UUU *�����z��E���xЈ�'������t�#ԃ�A8�I��B-��D�}_ `%ó��<�A��{ީy ��8�0V$�'�zD3I��A@�$�}VX���S�J���1 <onn�~��իW�\�r���:::k֬ٸq��ݻ=<<������ԕ�Uii)�L���711Y�n����+ 5���֭[O�>�b�޼y�|[{������!B�6m"����)((ضm��Z���p�LH�t�a� p�\�����bgg7`M''�>����A��ӧ������p�k׮����_~���۰aÎ;���SRR��ڔ_3!!���X��maaE���K�}z6�@�?vȪ� ��A���iN�B�}_<�+���&��B�T<@�B�~����B�$��=Oj�l�����^!�HN�8���5A9͚5 P���7���y<4�ɓ'+�t��هW���|>�ԩV8z�(g�G>|�`�%O@I����ÇL�g����122�a@����{,�y�f婄�4$$F%7;o�<77������<�>,,l�̙�<�Z�xqvv6ƾ�X*�/�'�M��_�]t9jM���$�� J����B�}��{�"H����'`�M�2�G��hnij0�,�_9t��k���ܛ[����������# .T��L�4����#�Hy�"OJ"��2��TSS�'����Y�.��L������ܬ�y"��uww�������\�RCCc�[�;w�cDŽB!la8 a�ä�F�>�g�\�9;���gC���ED���`�<≕1��)��t��o���Ӟ���t�[,$&���}��,�x&'FPW֣t���-������r0#F��ӧoٲ%""�����I���8����G��R`<�?>s�.�&�:���`�����(�W����J��`3��/]���յ��j�D�0��ѳ"�+��q�L��d$����m� �� (�����|_E]��^�7���>�S�����e�~���[H�r����.?cW�6�l������X��~H-Z����TqR�^^^s��QKƉ'jkk3�L���\c�} 8�����޴Q������iii�e�`` ����c�O����cG���S& �����Ǟ��N-��#`�E����U�z�]/a��P��<�>��[8��ݽ����?@���b�k.�7��!����$U4u���O 0����Y-= +.���[qƳgςIT��Ҍ3f� +~@�T���ϟO�P�O]샭M�` 4m�t��211����5S�}����f߾}�g!�W�0K^ ��W�! E\.w���}0ό洗-[E����g�5��*���7�����ldo�?�F�����ҹ;�쀀����A�޾�z#��_�"�≕1��)!����q�Ҟ�Dt�� � +yDzB�$��3M�=�{OPW���a�~h��Ɛ� +��VVV`����x�����������ٳ�|d�…0�+HW__�?��4��,���VR��^�U*�,�q����[�U�pKil�AA" ;ڈ���&�FEYE����(�2��:tL򒐄�l���| �}��w������� #�� @o޼ ���˃� �G�ndd���!�[ZZB�G��\طu��+W�ܒUaaa---B���>X?��Ȉ��*�����8p@�b�XXXxzz�p�������&%%�������R�Ǐx ��SQQ177 �y�I$�˗/����أO��Hӎi�q�e7�}y_�f����>N����I�p��q�g{(��0� �ђ�!�rC���_a�Ȉ����P�VB�b���j&� +ߒ�\I2��%ྒ�����WX^�A��G��;|�f�Ν���===`Ӡ@�������pWWWuuuhh���!�)�SЫ���1�ҍ����U�JKK{{{a���n��t(�D��͛7%��ʋ}�z�v�^T6��>���vbb��s��>&�����I�`ϡo���a����vtt�������ex\__���^����nȥK��)�"�}�0����ljf#��-��k�O��8�q55�t��#3�;�G�v�*��ݝ�!!�o�'_`E��'`O�_�326��Ъ��ŏ?�o�o�1'��0�9s��������UnÆ uuu���0���Zth���#��z +�ط�˗ ���}VV��������;����H����\�R�sDK6��9UWW����� �����x�}\KK��� +=R�>��}l �=��Z�� ��~yCaO�b����yeD���� )>���m�m�%p��dG��EV&�:s0�����γ��8�f��׉�~��$�?~����.t��K^y�t_���]]]����#022B�b�w�^�U��www���ijjB����:�a����ZI�>N�z�jtR��������T/P+++333SQQY�lY\\��e+ا�4�c!a!H� �܅�}���ȧ+1�� _O�w�n�7��'�� ר�����hXz��}���D�A8�AO��iqla{)f��讼�Dv�$��@�2|hKa�A��Q��B�())����v����6%�����/_��&� ���M[[�l뇂 �vvv~�ꕨ1_��#�����ó�)-�`�}���3*++C_���/���<-������\�c�S�˾2 H�?�|��W>��O�q�=1s�1`��%�E]"]8cw���κE�DŽc����� �0�\d�=x�7m{.T��ϥUcF���*�5dZ�x�g�|�,�nw�؟�fgg �����"Op�֮]�Nd``?ɜ ��FGGŌY��ږ������cp%�`SRR-Z�N�yk�ǝN�H� +�)$��O��#-�@����[-��d ���$6�K�O��$��O�L#������o�FV��:s04�ya�W�� �=x= ����Vڔ�K�Œ�w:JNR���%�I~��~j|���z̫JKK������ظ��O^k��� �!fN�� ���ٸq��`�Qtt4�-���\)ا�(�M��0����ȋ!M;x �d��m� (%�9�pn새 $E���c�l�����Y��̈�v2�07��̸�NJ:3�C@�H��_CM��Tcf�:˃�3��v�_��owA�m1��jtb�s_4>}����� ]Z9�����"T����Қ�����!0��u�Z[[��"b�p�W[[��ٹ}�v��p:���QsJ�>8ߘ��vAv��r�R�O!q��z@&�vG� ��j��H�.f����;R�o������'�� �w6����0����\g�� �v�|��bl�lj�'� ����¹2��u/2��D*UUU��������?z��մB PQQ����v|�N`A�~��_�d �htttEu&������}�^^^�����J�>�$aߌ쿎��!-���������u��Β�ey�9uX��@���9�m{���nN�;7"��p� H�՗��$ +��}]i|Q�CJ��i������� +Bx=������b֯�>����N��N�@@.==���ϓ��������Kn`�������`�����(%���gaaA�R�e��ؘ��gl5�[�t)�\�P����� ����x|SS�$�b�X{���g͚5 +�������k۶mۥ׎;ĬP<�@L&���UUUU` t,�������9%g���Ж-[Fjkkz��j$�}�)�Z�J___��vpp�`� +�KR�o`"�h���~��4�oq��L�ฐHBb f��[4�TA����' (n��OY��R@)0��.@,�@ +B�]�V +̳̀dޑkj{oaJ�w�������m����|~�wU�@=C�;��>-&��p��/)�L��P��ՙtʍt�7�IfL?�%)�����ߒ/��˚�������n�#�'�;�� +��u +��_ܗ + �-[6{��`Ŋ� at���+�oߎ����F |_�i]`���O탍�D /�Í�������^s��YZZ�VZYY���j�߳+��}M�Icll �;7�\�l�Cz�mR{EܛG^�ʽ +��*wu��+岘��d@ا�`�ʙd����3'H�[�sq��&+����?�A 5�yYR��RNW�3|Lx� ڇTKK������٬F �޽{���r��uvvn޼u���]}}=>ϥ��1���^�t)��MMM�t:�q4��������r�������ٕ�>Cim߫rz#/������6T�[9J�J�y\�� ��C���B �/�LßH$nݺ���HÙ~���"�������mۆZlcc#���9/�� +D���D-666vvv����J �dii�Ziee%�g�ٳ.�}��j�>7(�R�?E��+��p���0Z�ʱ�*�A�$����X8�7�����T �[�_�C���`�r�M +BCC���5;���suuEMW�D�g��Jl6�`R��;��ЀE +N)7n܀� �Rs����֬Y�Ziff& +���ٕ�>�+��� �� �����9�ɀ�C�� C����8�P��>���fҤ��sw7��Aމ q/�q寛I�Kɾ��|#��INq_w���OQ��㍍�����!eL7yLLL�T* 4ŵ0�a����LMM�Ӕ����>����f�U[[+��g�C�탚����������hѢ��l�4�oddd׮]��������D�} ,8}�4�����f��PI ��+�.��g��Կ���hN���_/���b�A� 8T�����Z����T���}дP��@����ʟ� ҝ�� +zlG9�=��������*�BsAt����2؇*� �D��aggAC-6l�Ŋ�`��� +a��s>�+����߽{��;��}P�WHH���M�6 �������:����������o�…׮]�����Ps^��9~�d�Tҡ����������&^|O�?�*�@�p�ю��N�_��:%���"��8*��q�D ��_��� �#����i���C����@ ���(��P,��=���e�V�jmm��n�Ͼ?�š�� v���988��}�O\\v��l�oY���`�>�\6�Q��>E�J�(���}B��I7�}��$���4T�ɀ��ۆK~(��t�\I ���{3g���O! �:��Y�IJ@r���bj����..���Ӄ�$_�ݝ~�Z�]�^�~]m�[���Î_(__߉� �J� v����}]ߧ^����<���H��d����[ZZbo�B�(�G]��>}���ѧ��p^�G�p��� Z� ��,���S��Q�J��凁?�>�!ޏ=���AE�)�هtr���TO��*��p%E]���vpp�ٓ!I �OP �ʅfVLE����^��C�\��پ��b���XԎ9�}*ľ��z�+i�^�����ʻ������p��s7�r��z +�e�[�3��I��>�i����d7��O'Qb;� �Ь���`����iW*�?`ߔ}�]dd$�� �����{[[[��=x�}�w��~0]��ܤR�v����dfdd̰F�탒��w��n�ښ�>���޵kת����ottT�G�S ��x���)� ��S��ɪ��r�K��$�T� ��-,h'?N�i_=��p�jG� )w�o�r�-��Sڕ�$���}���ȟ��=�ǎ��w�@��_���‘?nTO^�$�4�G9J΋�������߻}0� p)�d����H����O���Q��<==��@�����.�?��� |�t�q�o���r�J�� +ellL�J��������nz�� ��~��� R܉��OOO�����m�������^�����?j��ӛB٧�q��y���v�9��-��j�>#�>=�)�R�}�@�ߔ3��xkG�K���/Ӡ��,��` +���� � ��FѶ��dW�V�fAtl�(�@���H8 4�6`YP@��*j� H3�<;���$�@�u>P�wϹ�����?�r�� ��"� +J BCN�<ܨN��� AH��I�-| �y�=�ԉ��5��сUʃ��)x���}@��/B_�4]]]���P�H�'�˗/�����X +���'�bXX��������ٸq#��z���� ����w�^ Ԗ,Y�d2�������|���@�]HK�@pz������S\߉���9lϖcbbLMM��2�����CE�i�7���}X�GCCæM��`���0 �Ȉ������eS̰x����Z�������Nԙ�{������h簥��b�"����S�3�}p�o߾���f"�f߸�B���+] �ɢE�����B�R� �s8�� �Hg066.//��m��IR��OL����I�'�Œʰ'W��o�P�T�}FPSڱn�Y�ߥ!��3��q���$�}���{��:��`�rQ���  `֭�{!�̀����߿����^��~%샆c9tW}��#@011�����ӧUUU0�#bggg``�t՜9s� ��W[[ۖ-[�1�׬Y���-�l�X��˅�@�T�p�֭���r5�>�2��F5��ܬ�7��>L"�hǎr��Ӥ���[�NEx6o�L"��|>|p;555�==<H�|��!##���&�ra�)� *�hR��`n���X�~���1�[���x.r����]�R����+V�sڮ��CCC�P+���}�'T�S��}�Q�`�Âq�,�D�,e�uɾ���_�K:%ʰ�6��9߈x[�$8�� � F��� +J�@��C;�$C@W�GwJZxdc:���-��� ���^)�� |(�M#���{�a///< �)h8l62�. ]������Lq +���� ��˕���ׂ ���i�}��&!!a�ҥm���߆P(ܽ{��.�XZZ�IQ���}�kϞ=x^V+ +'�$�8�R�K�cJ�'!�o�їD��S゚�������l����� XD��{�3�Tg{�?Y�'��ډ���ɾ�|a��b& ����zzz4 \��З�Ν�f]pvvv�=� ���Mt�8�7.�0:;;���B�қ7o�Ҳo&�sهy�BjN��h�="G�4������P�a�:!��]o�v � z�7ֱ�ɋ���?���N���dx�Y���G y7���}�5���f�����ٳ��Ɗ��#��;wb&u|�� + +lmm�-[6�� .���HIIy����*��}��gϞ���)�~�aJNN677���ܗ��с3�����Wq�Z��$M�}�_\^-�ߡ�)?0(V2�c:�ATQ�C���"�>ٻ=ݪ7�D`+�pn�����Q�?��ܜe��“����h� ����w#�ӝ}������2 `��Lٷo_ll�H$�Z]�����999����W���2V�\������ T��U�O%%%�����\�a��p�9�sD����U�\˾��)��1��M+ARs�$0d �UT�#��J���&�} �0�|u�ї|S����=�&@��'>�B0���ш�t��d��x9��˛R��ײ��z������d���O������o�RSS300�&Ф����x���?~|͚5������'������;�Eg���@a�����r9�ƾq1O�bp�w��qtt�Á�K��]����C tuuA!<�e�W�ѱ�w#�c����a�$�"S8�1t[��^H�t�� a��'����s�cO6��ߕ�Y���z�~�mR\B� ���8Du����w��������Ͽ(g���������B�Rnnnyyykk+��/]:-`�������������75���*hxxXE���(ףi����f����А�&�>�S@+'�����S]] WT\\\[[ ���`�'��ӆ)NS/;k52�����Z� K�t��'=O�����0��Y�� ��F��^�#���$�lit@U���?��HP�3}{�s3�L'��ѳ�47g�9ۃ$s�"j���Rp�O+�����p���6�� t��~�)}��S�DS�x���P�YA�,Wy���w��D��5�>q>b�U'�I؇��y!�3�*B$� ���[��?�����!��a�ֲO+��� +����� °~��Μ�Ԇ����}�1�P��D �朊�ZGI@�h'r}��f�cߧ(c�5q�cP��r��#.���b�QB x �t:(C@WG!E�� �רԲO+�����?�u�D��|� K/J��JK��(eX�*e�t����)T�rW9䔄$���$D�@B�[TD�KE� A�TTP��W�d�I�n?���H���OO�GA��Ceq;/������+���Ϟ��W)�>a��rQ� ��+Rh|A6��S��D�d��߄�'�5���T#P�>��\k�� d*�2|QoW� �Ӆ�N��I�� �Q�e>�eݻv+ۡ��?����U)�h���_�*�'�����{6�����`H$���$/ZI"[�8q�}ᯌ�q+��i��x�k��y�����FRr�@ >̣N��� ��aE?u��Mn��� �#���x-�^�P|3��r�8P�/�u���_��Yx��� X�K9�AL�"�`wsr.�Y��L�,?�~W`:����L��+���O��?��t�R鎶t�_��d̾�6===����aa0�=��������� � � <���2inn��������-,,���,--:D rrr:::&�)�����˂�x(kkk���9::����x������>E���ݻ��r8�cǎ��ښ��8p�������|�@�P� ���'Z���2����²)�.h�����Q�}"/�N��GHFB��!1�%�(�U���*�>��s|�nЖ�+�[�u+�l��Os�ev@�q����Y ���;-�N��є�����ئ������Ǐ��7/--�={��[�7�����y��yaa����ʕ+%�/������n@@@UU8��ӧ�?���ꪫ�������WSS�2���ʒ%K\]]���A� 9��|=[𮱱166>GMMM��j�*//���ⶶ68CQ�2� ���ՠ���E,üpq� ��}k�*_4���K � W1��D�!�πH�KJ9I/�%v@���e�����=��Q�?X�S� +���0��; �J�a��X���ʜ~�'� �ܘ���]E�}ZZZeee�ś7o�I�������Ç(�Ϙ1C�zC2w�\�Z؋/������NKKspp���� @����#""�����{zz@1oo�5k�L�4iL3�޽��f?y��o��h�p1�^�I�_@�okN��ej`Y���r��W�����=Q�=�hD ��?�"~""8rf(R�1��)�>a�8s�>i��]�}@ak���ۙM=��*U@� >�8���Œ�lO�<uC@���n"�>H�b<�ë5//OCCC��c��ȥ��7V��3k�,p���@����_RRbbb2o�x9;99�_�m������˗������>���P�� y� 3}�tx�Ǐ��,mmmP�&p��͛ ���2�۷o��ⴵ�e�y����c{�쩭���`,�Ѿ�{E��+��i��֜P�}¥/ �'ť򮖏V�o�_������C +`�5%3�:�D��{�^��%k�xl����I��ѽ�1������!�W{�ŁA:�ѳd̾o:�타\��|����p��,���kjj)�q0z�ڵ666`���p8t:������j����#]8u��}��=}�t��A���$嬀$��ۛB�� (�������nٲe�̙#]���%�… ��� {.\�P�V�-�.y�� A���\.����������� a����1}�X���>�������9�"�`�O +2J"G��WH�o���_��#L�>� � ����|d�V�L����g�Y�G�����-Yz�\��0�2� +H?�?�X�����(�}�tF��qss�������Io(��w����7��Y�ӦM��A����ׯ_���� @U?| �iii���244�`x�]�vuttH��իW>>>jjj#=Ⱥu��l6T����������� 0 �9hj����kL+�rx:ء��^�Q���-X�`��.]J�њ�����@I�9�Up0|а�������_;gΜ��l��ɗ1�'0�6��(A������&���D'�>Q<� �J��7H�����TBԔg�_��6ϲ9sk{��H@X�3����0JL �d)�d�dX�a�}'�>��7o�ug���K�Y�}���lmm%����� 77|u� 99YGGG�����Gb����%�&�pŊ���Ln߾������%�t����f�������p/^,� �9����NUCC�L&�ײ<���>Xȭ+�3�S��Q։�k[���M�_Ο"����DY�T�N�Ⱦ�� �B��~y5��q� +��XGQQ`(�?���RT�z���rE� D���*uUH!B5�$t���RT@3�خ�.���"J���{�N�,����߼�@�z����'���B/Z��T(��N�Nu$����!�__\�@hu9�D3Z�<�'d��<��۷�A�i�-..�� |Ff��gΜ9ā����2�$�p[��B)L�;�*�_e:�8�o�n$�d94��@�6�3*�?�J�p�!H�o"E&�6mڤ���|�s�N �}�t,{hoooii� d�(�}P� +���M��><�D�~Ba�� ��a%<���cD�� <��bw0/�Ӏ��d�U)���������?Z���McD��##d�������X��PCC�&�b"�õ��Y,��ŋ�d߃.\HDX�د�Ǐ�[��H�իW777K��B��ֈ�BV�ZUWW7�=ttt�����hѢ��r|���d���ב�� �6�m�.�{{?44~�����XP䓟K�M6�����NE��-N�U9�{q3�c��؛��/�[f +�~�X�S&��&��Y�E�P��vθ&�.��W�#�7�"�}...׮]����,sww�G�jjj�,Y�����t��u9��@d��������j6��/]�4{�ld~xF�H��?���ttt�>0 +`4v����=wC�WQQ9~��������Ғx@���󹻧+5�e�kgdڧ���;+�}X�[�E�F�ܝ*LN�L)�}�\�g�Ѓ��/8��2ö2�]�7cy�c&�7������p�/���ּ,2.MtMc�'��H��>WW׶�6[[["z$��•��� 8~�Y�nݺUYY)'�@�LMM��UUUi4�<�)O�)�v�"2�������_�9u�T��By������-B�Ê+���>�����H�iӦݸqC!{ �N�ck[GDj�F��ov6m��c���k7?uv w� +g��]�27�9+Et)�Vf�b���Q��qSn��}b��n��6��-�b�����݊�'���7XU���w���!�#�ea��TF�� H�o"e8���TTT f4iҤ'N��=�lp9��ihh ����`ª�*y��ܹsG[[ٕ���{���xD�344����֞�GGG�ĐN����)j�QMMMd.����Q$$$�"�����2�|��{��D�����j�ط���ܨ$'�a���ϵ�M6�Hٷ^ແ�k�Ic��\N���cg��0;}p���P�!� 5e�PWh����ʾ�:Q�q��y�/���������gxt.�>223���wwwG.^�Zuu�p��{�n�޽�lzzzuuu�*'��첳��ص��nmmU�?|�p����nuttJJJ:`^�Vز�2 +�3�J�!��� +@���i��x|/��i>���]�JuFg�qЯ����V.�+��W��fVEy����o�dV�|��i�W�@��P��$�;a+d��0C�e������U��dz�L>�Xl���:@��(0=�B�M�œ�##��� ������[gΜYZZ��������������Xg9���h�{�pq�܏?*�ٻ�������f̘ ���"� 133S,!���t�B۶m�shhhX�l�d``��ب�=��F�>�� �zl�nϦMvC�Ѷ=�lg�����(�}X��X[�n� +�H����W��J�������X&L�)#�z�ݷ4N����p��3�����[��9:�G�� ���̺r办��������ֆ�z���õ��tvv�%�>r���۷G�A��������1`�L&SMM �дi�X,V___EE��� �J�NF��.\�Y�B���{jjj"M���Ф�=���> �Qmޡ-� �����];�S?����C���,�����H��i(���o��1#(��T� T�D?�d��S��,�Ͼ�b���S�N_�Q�����d9�������Kk��'9�9�@�})#���ں�~�DK�.�~�:�D�c�Z�8�W^��aVoˊ���n˭u*��s��•�έӵF/�#�)2D3�`h� 321�����8�"!�N�x)9�^n�9���.��c4m{�4���n��z�(����{�^k� 1((HSS=���=m2��d2b탞ccc�e<���A{�����5k������uvv�U���7"��t�?� �+<۷�(�\ ��4���$%�"|r��2i�*)3(3�0-�H���{+`,�/V� Pi��c'�?�Sh�� x���=�{�ϣ ��Ƽ��㏲�S������'���O`�r���&sss�:���===�9��[�geee�N__���x��� �}W�^�cN�������͛&&&���G�͛7si�ҥ�ֈ�*e�i�2v�S���f� -`��[�������%��2�S�$��: �Q"��9��b��4�B��j�2f�p�9s��X��" �C��$?ޒ��q����k��,.�$+e\R�}J�A��������0s����}S�N�hUUU�� >�3�}������虺������&_]]] 8�/��B  @344Ĝ���h&����B<����涶6+++����*e��}#&��>����Vl��K���\�{y\I����i ����*$ ��r��o�bD�R�)�S�G���H͟-'�+��QB~��T#�~|vM�/6� ������7^u�(�>�R�>��� +� =�c0���p�ڵk3g�D_�6m\�,R]]�I1c�w%''c8PWW���� <{KK �즦�����v���-,,0W��\N`P\.f���@����䄹���s��b{�JY�O�x�F�"*쓻К6����Q��4ɲO1bA0�����N9�^�(�����l?N�/'œ����\{v�k�,�Y* ����7N2����R���EK�Al�yYݩ͙��}�a�h��ׯ}||0� �z��t:��ܹs++�oME����g�b��Z�vmkk+�g�}���E�0�XZZ޸q��p*tpp�\���*((���~<�h�)���̄������:<888HTT�K����XF�ր��[���.4���7�Tv���4�#�Eb�G@����Di�(������y;2�p���1v�QЎ��U� )$?Ve�������H�;�Z�I��� KE���_�>{�l�Loo�/�t�?B�(;;;1+�h�ݻw,X�� �eii)Q�RSS�O���������a�������{�z�g�����A__���N1����f���9�?�Rָ��O���h��|���>Ȁ ޞ���S�g����f�b0�V�e�p�4�&3�S���$� +�c������nH@t�9�r���$���+��!��U`�����-�����رc|��3MLL���455�466���&�����ˍ7b�����$rp�렂��[@�b����Mmm-!ѯ������������s�>}jff�B�����|��>�i�ߓ��B�=��,G�?��s��o�g���<��% )\$9I،D}�־Œ�ř!��]?p�Yɹ*���#� ;œ����7E��,���@�߲w��bC��9��� �>}��;�� ���ގ�Ku�����t�������݁���||���҂h�X_ʬ��1s�.&����?���������Ǭ������g�Ǿ{�n��S�,_��(��B�D�;�4Jj%�:AX㾯��s�5mX_@�]=M�}Jc�[����'y+`zX0��A��O)`�/�o�?�=x�0&�<�T�ng�P�}楺}������� 2`qq��d��z�왕�~}KK˦���z���055�/N��z{{�3{zz�t:�`CCÒ����������_|�…�����Ν322�L" ƫW�&�U���}��CC¾������|�3:6mr-�G!� ����c�R�H$�]zt�$�C �I��� �A�S +(���c�؍B0n +{oą����Ju�� ������K�rss���s�q���� +�b�,v�����<��w566:99�㪁�AQQf��˗�Q����Ν;�&��f��������cD���S�7���/�&�@�744@x��)>ɚ�}ãN���i��Q�եf����c�����2r�S�x��~"�+񑿠������3)�8+yL��O�_�/G������ F�0K�� eU���� ����(++{�^��������|���� �W.�{xx1�5���}}}���1���C�����kkk{�ҥ�=�?������(===���/���İ[���I0$P�/q�=�2����fɒ%�;ş���6�S���O +m��%���?���t�}g���՗H�/A�.L�.%�kQ���h�@�5i� q�B���9��I�Ȏ�gG�|'`�j��3� �p��������x��2|*�,�w�ھ};�e�׾�����|�-��1c���sMM���V+//_�j>(kk몪�1Wkmm���@t+������:�����_"-��3ǽx�b�Ν�[� �Ux_��ꏢ����dΙ3^l���P__����_�x��H���������1��O9$��ܠ����i� ��}cILxLn��3C���!��u�D���$0e���C�; ��8�/aCTS���2U�ދ+T��7a�L��2U��{��x�ZA,X����ӎw_8����!�Hs�̑y]]]�#�KJJ�.]*�\p�޽���CCCrf W�Z%�:`|9@�o�G��2υ �B@m�W߾} �fee}�,NNN�?�����\S����ky�.9��w�{޼���*��\|K�o������7=�2Idr�p��)o�06ÇG�I�2��/#t=?ŸNf�&��S�}�J����Pm�12c�G�0C0���# +��xnn.�#} +4�]�v��,�^ +��������?�g�ƍ4 :{;��0!H����{��S(�իW.2����800 g ��fϟ?"��k�����;K��}(�w�!-G��w���>�V�mw��zЏ�J$\(И}�"، 6mO2т1��%��P��\��7R$�@��H�Q�?�bM�yE_ć�s1���8 +��:��nnnқ0T������a9�R�>�6___���I.\;�i߾}��Ν;����D__?&&�҄3�b,k��f���&&&������0�AWWW++�e˖ikk�w� ���.`]$�\�R� �*`H��$����������`69g��D���{28�T��1�Fb�6�T�(jz��J� W�}hL'�=����Mb��mM�ۯ�f�LXUe�D��l�cJ��h�V�����4b�b}���F�?�l���?�zhE2f�E�ދ�_vv�̙3%�X[[���˿���A����XI�Q�̝;�J�N�]�;��d``�� ����-1!|��_�uQYY�|�rO$ +���&�>��}9�]�7@�ۚ~}��.��皺�x5�n��W�}���{��67��}k��� SH³rT�}s��iQ��f���CS�\\��Y��IV��$w�b3V@Ǝ���WY!*��b�A޼y�����!����K� �P�d�(JJ��3f�066� ��$��/--ݼy��7�L�6 �jDD�m������I~��Ltuu�l�R\\ �Nu�O?�}�[���v�U���O�V�QnL��񗻗�z_��S��� �`e \�^�f���B,9��`���P�I +��d�?���� �vpB��ѩ'��&P%��0��錐�F H�D�wGD��q0��(��S J��^LOmm-�ghh����bŊ��0�z2U���Y��񦦦ZZZ��l�ٳ����� �� �'O��x�M�6�� + �ùt:���G����Gb��⯶��֮�]��.�Ʃ����D�Ґ}Yq��J�����7���)���\E�'��9�]�l�q�i���5�a�}k҃�҃��v1��� �����G���3������;���T1#��2���{/��ٳg���~~~��͛҆�x�bWSS���K%7|(����d2���hJ3�z^^^������@��3���%&&�[�n�����Y,Vkk��А2|�c��j6mI�^'�,����P�_kξ�H8,�����m-b�$�;:g�)c +�Z�t@.ła*Y#�]�� ��Q�$PfT�}��������{ѢEzzz:::��1���uuuaBsss*���٩�����-�̀�8� }||*++���U;������Rپ��mk�u3�_p}r�ѝ+͏�rE��od�*B�����f��[��k�j��_*=9Ac��+�M��ClSC�J�k+/�D���׌}���"Lg����هeʁm�ƍ?�4%+�����I�H$uww+yY������� +tc2����D"�D"%%%���C�{���J�?����PPP���( +��`0233��ʺ��4 ΫW��ܹs��%���W����T�R_�~��>�ȱ�[�ɺ� ��e����h/�>����}�I������U=���*�����ή��`ҡj̾��:s<����������i8��pO�9N֌}Y� b�7Bێ�[� + rƎ ���æT1��`��F�}�j:C~m�\�ط�S�O�/f�KL��>��r��@���1�9���0Cc�E�hxZ\-�$�O 5�@��� ��;FV�}3��/�l�H�'Оp6(r��a�a���K̈́�I$_kߓ_�9� +�o9��?h�e��w3 ʸ�������A���c���v��nkW~��4!!�E�fkľ�S�F'z'ᷤ�-�V^HTt � T�}��D�7i��� ��Â��<��]�Έ�w����P��eB�98�B���To���~�?5��q�:��/u��N[�Cg�ʺ�.AtU�Ut�*+^� &@(��{0���"�RDPE�pYP�����9�}� �p�����9�<�ɼ�}���B>|ch-���qr���Gqx�۹.������o��AC 氼����É�.?̃˸ȼ(�l�,���Ƨ��]�n�2iYd�E�/��얁�ڗ����A��-vH��*��ؾ���z�D�ˮ>'ܾ^�}��C��W����k\�هw$;�7v��g �}^ĉ�,*K`3��� O�ey�}d�EY����Yl�_����5U� [LDp7Gt7�\�{]c3���>3��kO���>���[�=��X�ř+g�5��(v‰ �6<�!����W����� `�m,�O������+E�z=� �}�e +�f0�q�Bև��'(�-c�Un��,���Ռ��fж��;����龯w�jd��.����r����>�C�X�ّg��T@�̧�sh�.���l�I������ࠏ�Occ�zd��H$~~~��!�Cj5��X,,yׄT^�Vn��s'�(-�b0׶�A����ZOzu���%��^�'�+A�������N�dǟ���ͥ�X�3w>u;/�X#�[�� iY�,Te0��/jdT�� �!����1 �Wa&y�Zf��4�b���M��vttlذ���d�Sa(��妉T��� �#� ����� C,�bz-2��xoT��`G�8"�I^ +K&ޠZ5��MMM7n�G�L�A�Z=���g�M�a��+�袖��� +Je�~����Fd��vH=�\8a\+�L�4�u&�������H����ϕ�i�d�4=����w&ña��͉w���Uڇ7�{M�m��P�G�؅#j�-���>s� ��<�~��7��G���Ay���O�g�d��>�� >����7Y��0��Q�#�#k�������V��o�I���R����Q�/-2�Q|W�x�=_߈N�1��� ��d����>���Ӯ�Pe O����ʢ��Q��b��|�=U\�̠Ne�z,��v��� �G�0�A�͐G1t=����?�_8�o�%�a�}� +���{4� vJ�����2��a`~w-�c93�Ƶ `<�M`0�k_��<�� +�A�3z�C�G�͙;E!�A��ɟ��2�{ja���������p�ž���Р&.k�<�S�ܻ�]��Z�� �<�Y��8�^}����R4��e� +�L، C �3yM�����R��)��ϩb"� `X��in�Ė�]8k���-�_�u�v;:Ԅ���Ndg��>���T�ޜ�[���zD�d�1��8�G@����U���kjK��p�2�FdiMj�N� ����#�sb�G)� ��H����Gʧ� +��A���N*F�Զ<�� +¥���v>^-�Ϝ¦U�U��Я�ч�)��hE��<��]{�GT���,F�+9��<���T��#�dtc������U�᪥��%�,�Re!n����⸘�{�k�M���dY�����K�̎��=7i����\po�n����p�~���ō�n�>��u��j�B4q`�����w.��@�O�X#��}]��]�W�\:k���y1�wk���{j��<��m~[+���D� �8����;T=����.��׷c�<�:�)ݯ1 .m��l�N�k��Z��PY5:����a����< �ÛMī�Ʃ&��)[�V��w��Jg� ��F�O� �.םD�z(O<�WDg��x��}8�d���h��gD� +��xaǓɴ�5珴��U�'���� Ŭ��˩���i�O�#�T�f5Hw����)�T����=�>��b��a�ҫx-c�Ѡ�_!�� ���YC����8��kQ��*�N�^u㧂 ��bY�uI���.ا.N��ŧ���W�4D�F��Z���#s)oQS��r����, +M|��5��Ŷ���hf�wT�݁ �>���v�'�%��|���+۷�����iGg���oZګ� 'B��ҙ��9c�Ҳ���COfPӻ>��- Ѣ����܌���;�J��RP]s��KBn��DN�߀� RĿ�r��d��E�li�=��B�k?r��?�P����۳(��Z�i3��,z��Д��ܨ�y  h����������{�g��/�Y���| �L�o�'y�q�P�{ E�]���U9����D��B��ʙު���暕�S$|�̠A���c� �T�yݎ�����$�j*�7wi�}�A��zJ�u��s�I� ���V�4��W�/�9 nq�>��z�����/� cJG�vFO�\ ���q�u/ 5���h�����@ Ð���} ��.�� ӫ?5@1���Ċ�$V�.��'���rGQ�̀kc�f�c/���:�3��i�-N~*3;-*3��Y;,9�@ݎO% +��2h���k�qk�I��Ze�}�a�N��H�8����[�b�j�jk_���bj��kVQGű�TnA�#ȩ�� r߈Q�[PB8I��sw��_�؄�a���*O}� +O?���|��f�i]��iMnj!�3z ��}z�� vg��L��a�=�'�(R����0��hx*���A��y��Zڧ{% +q��&Yn��j�,q���f��vf��e��^0�>q���. �MF��D@�4�'I��-��zbA�v�!�95'�*RؕdJ'Z%j9SJ�[쫼j������FP,�SKZci<�W���S�3���5���nOn�W_�z����W6� �#<�lK�i�y|�>�hƫ#��ʋl������ʯ�/8 ��Д/���lV�كx)�r��h���>29��W5�)u�h�1�����sh������k��㋳LJ8�������<_�;�ѿ������cr�>z���6��g�h��r Z����%����t�m�pw�R��i��H�d�Z5�o!����+�,��}��3�#E��R��u��F�;����!��a������� r��9����'ˎ��V���>ѭSʶ* ��IӃ0�#ʮG��N�v��8�yqRH)��R����:�v��x�~�l�.ivj\�_O:�Q�B�ՙ�ʌ�T���R>,;�4ꍔ֐�]�Hl���u��5S�g�ށ�>iT6�Ϲ����y����U�+,ࢀ��r�$V܈���h�k_��F2g��6�냫��د��|��b&����44鉆v!֖e��6h� J =nz�綏,��9���5���3�O��{�S)?�����Q��k�.��{v.�go7�g7�{����C�%n�}��~w��R�w>�7+��i��n>����g�� �}_g���Ϧ�m�t� LI�'��~�g�C���ީ�'�����*@I�k�R8 H��@���yx!K��G� ����O;5,�p�2B��m�D�O�k�I؇����@O��D!'AL��'�k'��S2z� �יڇ�J��R,�$�R�~�ε�,��c2c�wjc�k�p����\(}�z lw�ղ�g�i�E8s��^��>�`�)�* ��z��oj�O1#�'�V�N�`Sp",�m�����g�3�u>��7�x�� ����>F�g,�7�0��OGsw�� �ڶ�B?C(��gm�}$��7�[�C�3�⏱��g�X��s�g�5ģ�����������ٱw�lts�C�����Q�� ?�v�?+�ے�����u�j�վ/2�>��,U�����둴O�Ө,�@dQ�P���]��Q�>�g�C����C5��f_����>$Š�a��U�����a3z�\�� +b��C�yRQ���Wu?���-t#�>�C��wej�D�L�����b~߭Ӿk]��s��p�}k��m�;*<��c��ڙ*V�I�~h�)�h���x{�%��r�u��ZeT�� +�}����U�#߾���]�Wk}쫼��9���������E��t�� �>��д/އX�Wt@�#�>|67�>Ha~ebFn=�9��ﴲ߭�����i������e���/ݏs�9�����_;����~�"�~F�����|R�#`k��r�ܖ�{6.(�?�Wh��:�9V�� g�\#��;\�V.�w�Z7+���T�0B+4������qX�|t^N�t�-,�� x����S����3 ��7�7B5�y��k�l`���9�4��,ٗ��ùT@:S��ź�Yq��}�f�.?�������oa��О�^.��!@G8�P���u���~�0Bſ'�^��(i_x_��X�zM�@�}�W�S�7�P�}���-�@'du��w0��ض�2,�x��FÚ��+�h�^�ؿ0z�%Ʃ1xT�۰o����I�qXY����HP*��6Ӿ�Y1�,����I�Wf��[�7� ��ڝCu�Ͱ������rdhϮa�}d�2O:��%����mUO�����I�>��u�oD�/�j�u�s��>Z�"�T$��H/�I����W�Yd��>���H��Q�{��o����}������T,�I�&�]�:��҃5�~��s�A$�K���(��X�g�,q�ٵ�gSq�pC��H2��fz�/ȅA�Ch�K�z3����/���v̱���G��M�ߛ����ۢ+3�Ds[��gf� c�ǛB�b �w7�?�=����<�A�s�󅳜�ok +��&�*1�t-��M���,��t4���k� +ٶ����FF�̦�G&���s?�6���+ +`��ʜ�s��7�>H�gK����C�ld|����3�Ѵ;�f%�wB�����~�n���/�fhhޕ��g��:�94�s��.� �Z�Ix��,�����u��!U�֥�!� +��G��E�p���kyH۩PW��}��I�n��顢<giF_!����l����B��s�?�u���q�����Yǝݝn����gW�n[!� ��"�U�*�A������ +� �#�!Xy�$*��P�%�TAQ�Ox��#�J[x���{�d�y�3��k>3woO^N�',��֫���$b͓�g�kvvZ�0Y�{ڣ ϋ�a����憶f�^Ԯ�IT��_ZgY��P�Uq���?,����������g���0��?;6$17���g5�-�@UJVI37�4� +���+�Ϫ������͍��y�>�b����E��d��C��SOw��c�ޚ�(��b��hfz�K�Bz�}� ����t�k��=��{��RN�L)��Qf!���ޏ(\�����MY���{IB<��.?�ް�?JS��y��[�3N�`$�0W�>B@����3Z���C��}ow�}�ׯѾ_B�z6l �{#��>|�jz=4* ¾ ��qT�6��F��!S-e.�}������~)&@lZ�^��ŗ� *�+;}���P�� �h�07�2x�;��j�w'��c(���Q̽��&�g7�����A���###����􋼤���������7�"���h��_3��di��izO���컏ꍣzߣ���TY/��� ��/�Nʤň֑�����w�$ZK�}����t��;��Y���YfmU���\��؇��h,�1)�S!����Z�}��#ɚA��� ��>��4o�22����G&�e}�� ��S� +���� �:�O�_s٥03],��e��#! єه]@�/�i%���A�`0ޏ�O'ࣁ��-^���8��B8��*���e�Y?5��KI7��{/�p޾��2b� I(�O'`��K�v�c��*#�:g�8�9V�F�}����C��}0 ����`jP�jMS���1% vj�X��*i��*��>�8�wR�A���B�}���� RA�k�[^���9X��,��Vu����]r�K�_"�>h [�~�}B�T�&�w��at�8��qT�jCr��-��ه@&_螐����>��5@�>���.�`J��'ܾvqR��t,�?��@_ji^�� +�Y�>h ���e_s��I�z��j�l�#o���}�T�&�����oa&����B3ւ�Ul1ynYQ�S _�W�>ݕ�t��Uuv�h�4=���a��ˈ +}���A�`0�2�<���rW5jFހjԸCz�r����J��`.ύ#� �� Hڀ�@�H��*H�}Z�N�ww��Yw��hzr3�����j��A�`0� +�l�iO�&��!���F�j�JA.�@�}�ѹ�3�$�����Հ<�� �Щ�O{��*����Qnn��� x��*��=�À�A�`0�қ��h~�#�i>1SzP�1Ā��Jj�v�� ��2��Z�f�f?�� �;Ȏ�:*훿+>+<� V��S�7��$��G�A��}0��>|4^���K�:K��znfv)O-�}� ��� �j�l� �l�wuSw�c�8Z�S@�}����&���"m@l�~2� *��r������nN2ks�!��`��~���Ɔ�1�B�}�> ����?�=��ǁ�m�m8p�?Q٢y����}�k� +n��C��qD��O-�x�ƨ�Ļ,���s�2��;�Mr K��b?��� ۃ�G?%�Ҿ����{��Ana���F[�\����lB��}0lQ�����n�-|��e ��M)��4�W<���iO*l�&u�~��%n߽��؍J>�6m*�+�Qiv!\�/'�.T�N; �[�}�J d0*�G��O{��W~��a�-6;���(T��lROcDh��������T���M{����݇�Z��V��RQZANQ~N15��G��<9)@� �7�;�߰�W�� ����k�ه����nN�&�y�� �0�]r�� >������t�Mf5M$��ڱ����܀w�D]��؃��>��7U��5[G%� 8R�yk�WY~J~�E���������}�q���3+ˋ�M���Ҿ�+��w� ��V@�D-�3��+�'����`o_�}S�&��8���I�,����3���)e�"����9�Z�n/=s�b�wuz���I�7*��vն���\����('�24}-[D؇ݻ���X|:�^E���>|V���N��>avm�5��ɾG#��>h �+�l�T2���Pبf߲f�4�&�i@�-c�_1���,�������������>*�D��-AL�o��U���p�B���a��`���" (Z���g&g�}%T�7�ߕc��:�X��t�a�A�k7�U����_���A�`���E�-\&�@p}訰׎��Gހ}n�?Զ�ܟ�6�������2�;���������Z�Y*:_�SD�} Ď��N������6�����S�w��.U��Yw���tvvv��N=���Y��`�*�$� �|��� WB�L� ��B8T�U��E ����ڕc������/~ �Z��3������<��W,�3,R��j�������ݻ���E�d��mT��a�샠7�_�o$�nHy�����'Zl��?`����o�i�r��J����V�%�3�?��w�|:8_����}�p +����x���z���G\�K�D���,��Q?`�ݗ�;֭6�q��L�!�샠7��GnXuu(��XQ�N���G�}GDܩ�[�-g߸��%����,��#7P���|�F�����GM�K�����Lk����rօɕ".��q�p�.a���w�m�д�3�����>����^l߈ʴጺ�� =G���z�ؼ.�c�й�'��G�R,d߸��ʎ�V�!@����u_�um�k�B*NKg̾q�%�q�{�z������OQ$��0h_��~r%,��=[�|��}��2�� ���(�L�y���}AF�с�-��7 h9�Lk(i+���z �i�@���ꅷ�l�u�2E�����e�<�K�S����[���!�I���m���>z�{y�� &]x�wg�x�>[�ю���ߟ��-�h�����ԷS��� �zU�F2�QV�=����ټ�w�?[V��~h���']���g�>z�7Kwt�?��^4P�/Z���Յ���J�8M��}��x�x���z��/�J��B��xA< ׻�� ��^۾�a�ǩ����}!F;�qÄ��8 ��G�F{����z��6�{���v�����=U��+�R��I�����0\r����0͸}c����qQA�~�?�� h�MѾ���A������e�Q�����p�>�E���Xξ�sy���8W �;*֐O�yw�W\-ܙ�c�δ}���1�q�GX6m�YH�a�s�$'���>���)6u��2��U�GRe� �hϞ$�-�+���2Z@�� x:���]ݕ��$wG�i�& .=�����q�Z�2�X!�9� �EA�����+ ��A4Ŧ�>j�9}����!��׶��� �'C�ho���3�w9��X�����U��+�R��ǧ'1i��!(���F�&E�Ej������_��}M��o(S7��}���{,�u��$؝�pȌ4&[�>���q�N�����^D�GRxW����=i��L���x(*E�A ��Y� �~ T��E��`�A�46����SiMJ�w܄F[����N�6�C�܂�Q`m�/Q�������^�Y��T�7-e�>j��~(�%�M�o��M����}`A��L�G�B�/��� +2:���ݲ=b��j�Ĉ�$��B����)oc�Sw�2�q�����IA�x��<�b���Hme���6��1�i�'��A4�fԾ!�n0C��L�͉nu0nxj��W��W�G �E\J��}���q������^D��Wiݫ_�^�y��E.�E����?�vF�H����6�~$T��"R�>%`�A��i��T���� +��8eס0����`oZ�o�3�8�@�)-e����=�s��{���}}�����?i.^]��ä}�BP�/�lB�?�l+3� �~:'04����}�^ �7���H����\�=��a��1YD4N�K!����3l߸��4�ړ��9�&�� 8fz���Ey���#�l��|��j���TGq�~���>���I�L�)s�OP����>����T1q.��2b�igs�^ֺ4��Xj.����lW�Lf�!���(g���ȧ��1���f~�%�y�}`A��� e�|��kO�4��B�F��� ��|� M�/�c̾j�_��S�u�Z�;e��UZ��uW,�]��Du02&�I����$�H>e�����r��O����>�����"�u��jI�l��n�� [} v�'��@��M��}*_rUY^�.�D�G�G~�=�%7 ק�y�����/E���)����7G�# +���>�^>K��*_K��&;�љcp���%R�ĹT�d�>z�5��E�II�h�YQ�ޗ��c�Rf�$��!��P��1�>���W��������"Y�A�a#k���E��m���x[(m�͝�����CQ}� ��7.��l���-�H�G�^�2"�A���������G�Cj�� �5���7�`���q�f��>�p�O}\P�I~}��B�� X���������������I���;P��ڞ�rTF�q�F�Y�J !,ʖ�%�Zu�N6�ѣ`�8���%�h��qtNO�?��ˍ��8��&7���>����G�߀����uf4�+�N��B!^���n�����R�O@־T�L�tF�M_(�������;�ڡ�p��4��;j�zl���Y�� �`B��B�>�� ����s'�r�y�3���NKw�n:g��iG��g��r�$�~�"������K�rQSe��>؇ +�B�>v.7�-���i٬}��$J�tm+�x�>r:"�X�����7n�>���/o��^g��Q�`B( e�8��.͋˼�r�?F�d)-��_�}�u7`���ݝ�l�j��ݯ��|���n���QV���WU�>؇ +�B�>���[�-���z3��~�Ӆ:���}��f9W[���l�����r��+��ɞ}��Sau�>؇ +��ž�ڗ+�-�9��K��IҙCJ����n��>w[�P�j�S�7b������ݟ��1X�@��B�F�=�:�����R�}�ޛ,��A�����7�d���eM�|�|`�����qkW�����}�!�K�h�es�,�㌜5��K��gr��R���?��F����S_F� ��k�d���jmv+��>�/��} �uK%�'���}�5&Ig+iC�SƟ}� ���sr�z�k�v��b0�^����E��!�x)|��\,�?>������Xc���p�M9�}��kgY�����Ij�Xa�C�p��'��vAc�=��&e��/�R$�yY���#=�<����\���h���tGM��45�X�+a�C-a���R�c^Z4��������$���5#�[ڀ���P��a�����8�^������n;|���`��>؇ +BB�o��� ��"��W���L�>���8y\6V���dξ��nW�wy>����7+��+j�Ɵ�7�����>؇ +tB������O�����t�d\�m�o `��� x�C��ϫ۟���歀�h9 +�`B(� �>��J�Ӆ�}J�>v>:"�o�m׏\]G���Ǟ�A�W���~�� �j���>�P��},sf���B��{�3��,�<�sj��/��i\�Y����C/n�}�!�n��1k��m^�ѳW�[3�S$�H��G��T�M�'� ��!�����{��f���F@�Hn�@�K�hG��n�z�I9ߟ�`B(hE�} &�7�L��{�3�l�ɤHqߞ3ҕ/H�n�D��!��"�>�?�}�`��=���/��+רFu��'��' �}�H�oA�uFzƓ"c��ۀSi��꡿i�m�`���m���T����"�>vz����\����Olj��}W�q7o��p���}�H���ޜ2�A g+��=�#�/��goe��} +�]S��`�C("�}�h��(Β����G p�X�� ;���)Ŧ:��� +��>�"-طF� }LG%IY�|Ƌ�̼�и�tڀ�h_�Rl����m�:���>���{�eV����Ȣ�ū�Ŋ�R�cJ�pM�����}̑h�����!$�`߻��ltq���B� ����%�gU�w� w�X�k�`�CH���u�+�bϓo�eS �5Ɗ&��޵k��:!������}�!�6����l%;K����5�=�#�R ��[u��݀+�>؇�0�}���W\� @I!�*[3�D��<��e����4�U��V�>؇���}�ڷ� ����p"C:\�%<��3���K`�CHH������WT�QSiJ*^�/�T��1u�:���# ��71�}�`BB �}�}> ,���ګ�[��bOO��eC��³���r�jcg��9؇ +�`�ϰ�.��UxrM�1'f�c�#p4W&<��; +�e՘{�������>�P(�~�}~G�K�4�t��?rcj�ڧ�1_�U_�:{�����!���_h����I�T�8B�c�*7���!�B9��K�c�_q��!E��3� Z��_�X�z����!>����}%哇�d{:s��,<��]o��+��/ �C�r�/|����_�����n�n�����B�� k�~m������̙��_���:W �C�r�/��������h�@>��ɳ}o`��qEi�}�� � þ�B�7�����n��&�!�B4�'$�|�������M�:>�:aB(�}���33�0��ٜ1��x��<�\�B���}���'��>�w� +O�y���B!��}� H����G�[�Qm���� �C'�'l�V��d��>rdN�'���!������t�s>�c�#7�jx�\�B��"�>����kwT�i���!���8��[��Y����*��W�� 8�j��}���"о���Ř�>��p��T��X�����m�G-e��`B(8���/6�/[a�t�p����܀� ��m��N��\Ѱ!�`_D�w�~�l�X��<_c�w�x��?���u���Fr��"�C���:�i2M�nb���♘h̨�Yu�5��q4�����l�1q9�`D���C(�.m�Ж,�"��� �"�XpE�\A�� ���|��q�f�y�ybJ����5����S�>��Q�eJ�n������^��^��$�[U���1��S�щ,���`0����}0z�}i��g��A^!=Л�}���pcԜ�v\,3��`0}��B��/Sf�Q0"�3���Oо� +�4NB2;6���a0�� +�g)���d�wy3��>��{]�=C�[��PŎKdI%L1ڇ�`~�>��/[�ˇ�c��|����wa�& ㏱G����;>���a0������|U�������y챥��<��Q0�%�1��> ��Y�}�L�+��'w��B8��C�x���K��+XR)S��a0���ϒ����$o���.�?�C�<&P +"h-��ǒ�} ����p���)����Y�T��0)�Q�#�ʃ�,�f��Pΐ9؈�> ��A��}T�Ї�}=�y�S�D��C��"���I�?���� +�> �A��}��^�܃>��{��c�� ���|'޳K��o��<~�&��`0��7��ƅ<�7��}�]�������-F[��Uh>���> fp���Mf)���4ڇ�}TT� ��C�y�ӆȟ�In�V<�E�O@���A��}Ø�c� +?O7�1����h��pp�=I�`Z����:�}Mڇ�`��7���J�Tp�9��Y�����Kq�J<�\GҾ^��Zz|� G�0�iоOþ\�C���O������$ר��<��A��i�R<�EVh��> C��d� +`����(y��6r���s��� 2_��Q�A/�� �)�> fp��#���l-�>���^�Y�m���� X� ��� �0C��}/ `��^/�Ԇ�}� ��'�]=�*�eg!��s��Nw̖�}�bqq��H#6$�������Z������a0�6h_���� e�N�o��Gu�,w;�Q�����# +C�/A�`Z�����Z�a���A���>����K'u�I?��~���?(����ZW�����ؐj�c���Y�(t�;dE�0�����;�;NS��>���fK�>Z@�d�����E�$<�~�A��ߝ+�W�> f����2e���c����Rf� A�^ �a�������L��~�����������`Cо~���:��4�}0�;�ܲ��ie�r"l�K5ц�J@����BgW��Yͥ����> ����������hv�������p�g��H�G���HCr`�™�}����g���~4f?���`>�}f����Ig��h]�B�<�;��}���d>�ZM�>:��/ 5��(�h���3�}�:��Nꨳ��E0� B���x��f�/����u��?3��p%�tvu��^����(0��/gC�U���W���a�>3����Q��상'`��ۗ�e]���fD��ܓjr�A���[ڟ��јSi�l�G��i+��wv�������w�R� ׯ_okk�_�}�6�$��񰠳�*:::Μ9��E._� ���}�X|��yxLmJJJbbb=z���EFF�;w��777����� ����t��k`�7nDEE�Φ[�%���D���j��ϟ_�xQ&���p�`TZ[[���% ,���555� ��[����%ӳ�WPP���O�>���X_Ttttii��7P\\ O޼y�O� ��}�� 3-�>�?J@��p��՟�2�~��~I�X��Nо�zot]�]Mm-�5w���מ>��8�ZZZ<<<�̙3v��#F�=z�̙MMMԂ����&�����������<#G��p8p�Z�vƌs�΅�=����ٳg�Ao=Lcc#l8jԨU�V=����ƍ鏀s�7nɒ%'O��=ʅ ��؀��[@�M�6�X����s�ڵ���O�4 6�ߞ��JTQQ�}��ɓ'S &L��l�2��^���������g������`����{��Ǐ�|�r���ݻ ,������a֭[�z�j���A�H�G �)u,M[O�>z)�KDھS��S%*�U1�iJ(����j��Y�zp���''��C��5P��Jd� |۶m����F�P�� ��7��ʕ+�(t`�޽{�1�)5 ՘��L�8qȐ!����@WԢ�S�^�p�G%�Y�f�'�I�zo_�v-����_�z5 `��� �F����ڹs�i�����KS�հ���?<���o߾��,������3z. ~�Νŋ�p9999�=O�8��޽� �'�ĥK�N�>�.�@@��*�6l�/C%����E����^OOO86�����C*��~~~�Lc\�>����e�e� U��-�>�r���p��}�$_Ie��' �g��{>t:���ر�����!���Ǐ�^Pv�5���D"XO-P*�𪏏�� #??���)��g� Ѓ|}}����>x��:&�ˡ0�ٳ�^��7o0GJlAe�m� �f�)S��ڜ��A7h^P`�y�I`͋/�ς� BA--,,tqqm�\nKK �P����1,, �����cX[[�?��A�mʔ)�*�9\;|������ݚ5k�n�Z__j�U���U`h�>��o �-�;��{U��� qy�ү�ҊU �ia�l��>��:���&����S**��D��QP���Z�FѪ�hZR�0,��*��-*��슢�H �,���� h�Y�w��w� ���2������{��|�;x~�wĽ r��]"�hiim۶ ��ct��m��E�j�������I؇����Ĥ���}�̗���۷oA+��إ���a��G����gff>{�l������d�dddt��a8S`h���VVV׈�-l�tuu]�fͲe�`WG�@@�H���sssþ�-X׮] J��'"���@�7((����Fbb"�ڌ~�bا"���ݥx��l�67H����W#���^vU&��'U����aQ[[۾}��e�"ׅ+������ ࠳�vKK ��ɓ�9\�`߄� +��(��ā��|� L2�0���kjj&''㽸�x֬Y ��(�����U�ܭ[�09�9Ÿ⽴�TGG.U�iii�&M�Y�b4$�۟2eJTTx��!���-���0�СC��Spill Fkhh�ܹS��~�bاj���.�p�� es?�ǎ}�;���ǹ]q����G<`BaJdf�߮��{����71�?������e�X0pDMM ��b�@.t999�'������6���,@�ξ��^���検��:����������޾���jA(�)///���شix�����pRSSA7�$�>}znn�\p:�����|�����ٌ3�؇L�ݦ�4gΜ�s��_�M��P��C#� �����hA�{��]�z�ŋ����?�b �*�����Qӟ>} +�͞=�޽{vvvӦMC�45@*���@������ se–�I�ptt����xzp��� -� ���b����7����İO5�G#`�� +���z)8'Rx?(6�}A��#0�$�rv�-,,�Xdd$���� 4ĺ0e$O ��͛����PP���6l�������|�������yIII�ׯwpp�궶���Q�%&&�F�����b������� 'x�� ԇ$�F@�֭ ��#e4Fb�'����%^�b�L������P@� +�//�����~-���T� ?�q�Ǯ��������;v̜9& V v,�C����/���Ν�C��8À�4��ɱ�����cF޸qC[[��b�ST;,�'򉉉b�>555 ��-B��ҳ�����C>���w�d��W���D��#�>��� ��'ld�ĉXI��H���T__�رc��0K �*�x���Z�x15�*F�1���þ�(�䂀�����es�ٜ@���< �($.��4ǔ}B��ө�\�Ӿ�qɾ!����S�N��������R�EEE���999�˟���d<� 4�&eee�x����ť���j��ܹ����Ѓ#Byy��ӧ�\�"�J�*���Spppaa!,��F�ΰ x1�m�]�Kb!L �t�O�b-���?���[�0�H$�ܰooo�N�O̍�����tuu��������G ���joo��q0R}1�����Y�"��{��`�~����g�_N��~pl��%��9��/Wm�w뚨�a�8d#F��J �>O@/�*���@�� +죞�H�m�%%��P<�̬ʁ����G �1b� 1��B{re���%����G��?^E�G<�(4��ߓ���>�U�o~�[��-Jb�Lj#E1�� +��+fq�����m@�,��7� ���U�}0�����^T.��d��~���yO^&�>F�ɉa�W���x���>u��3��g|�Dl���"�?#��� ��2�G�t�=/�3� �1�b��s ��{r�T�}��$jr�g8|�����Q�>�t���A赈�ag6Z*�}D`�GY�?\�}�����#F��(�}?��x��,�����!�w�:�|N|��cƾsւ�-!aι�d� ����>�@�� ?�/ܵ��Ձ�����/RooommmAA�X,njj��zzzИ7,�766�ޚ��<UWW�K*��x��/,,lnn�K���M1Nii���ﻻ���ʨF$���˾�>jngggIIIEE=m,..FzT VoooG������*��#�---� ]�K �bV�=B���:�� =Bkk�D"��u�o����;::�H��F�D���� w@������x��;R�����,G�ǁ���c�$����#�![��"a*,��1����8D��8ϟ?����'��~|�0}0��j�D��2�K4�I�͛7%2�b�Gİ�ЋW��)T��} �i�A�'��Ws?��8���eߐ�tv���t����}�>�3�B�rUA�T��쌌�tuu������|||pU�^�-����������� �,k��Ј��w�`.�> +�HLL�Biiir9�c<�ĉ#F�>}Z#�+W��$�طo����Idž��GL���_2�={�Ȑ����n��VVV��w4�����ѣ׭[�`���=n�8�����9 0���D�����Vd]J4r����D~"˼�e˖�^�vM��,���H���H����M����`�@ ���( ��p��&�\�BT���N��P˗/G��.ϑB�{�8$I��СC���[9��ÇC��*..&�˗/���LMMW�Z�a��DEE�i������:&&�������]@����E�=}�TJ��p�D]G���P�߿��p�l��㣇��p�v�ܹy�f�211IMM��� ki_0r;u���� ?~<�KKK����ʾ��K������TP�3A��������<�ϏOs^��6|w�^i���C����_�:�+��d�P�$��"8��۷oa��w�@MM����eC�F�h�� �����P������`&N�<��]�v�V���>%�!C����CA"��~��ŋ�O6�E�!;�B���v)�� {"fh�᜜HGA$�A�K�.aF�?��I�idd��dF���>&:���62��$'�K��H�������d��~�Ν;@$$$D�䫪�8wV�bk���f�5&���Ǝ����~�co���I���Zq:p���2C:�W�^�``�N)xR��q()WWJ����.��磟}��}�U�?�o}�N�O��ܬH�V����%����:�N����ts���aÆ!z��@���>|��>F[[�`Mkk+���:����߫�`���~�����ݻ?�>�=~�X�AK��G�d �Y�o��C]��c������� ,�w�^@@/ +S#��3�j�߸qC=>���d�Ed���>p����իɟ�5���҂���𨭭����#�� �G|��ĉ��N�� +��b9Δǚ��lll�L�r�����F�A$��%���o���m��9���Oe�n��������U<�=8A8�}� ݞ={ ���Ν�*�`o�>����W��nƄ�O�^9��S} �EJ�0�}�>>?=S�9�፼~������t�3g�h�?R�WYY������Ы�� f�&dp\�8Ge'��t�V�Q�ا��:�Sv�>���ߺu+m��˗{f�����SS���TT���� ����h=�.@�Y�fi�7�9�%� ��3..����I���+O�<�q;v m/,,$�HI���G|�|/\���d'�5���=z���ԟJ��r���������G43��E�i�>�}X�e˖M�4�R�s�<���r_�l���\0������2��pECCC����}��}|nm�P�~Sٙ��S.����w&(v��9��ϏOs^��~_�<��>��w�^:���\ �@�����t���;�o4�aaa�r��\�z���&��� I(�`_bb�w��e^G�c������cpp����*�a`���'& �}vvv� ъ�`"�� h���7 ����@�׮]�����D�3f@.�@��㱩S�N�6 +hD~��(�I�}��ׯ_�rR���}$Lc�����UUU�q��}�������344�����o��͗.]��֦�dyJ'�$�hnn�t/^̩�jjj���),���Z�QR16EG��-\�P��=z�[��>~�vI,00���oܸ���C&�qYK�bp�����R�A�\���ggg'n�[���hߎ~�} ���a3x�w��'u�}��Ͼ+��t�}�b�T�>(;��> ��(�؋FZ�7�����I��~{�ܹ�ܨ���c+�*i�/@ �����0 �!��^�죁g`B�O����&L@���A����D��daa�oœnڴ @��剜�@l�I(���r����Z5�$�m~~�x�g���+V���a��^��SPP@>��N�@ygΜ�]����xb_�@��# ^�%��!9&WWW����F�"NV�O�ƠD�B<�f���H� kkk===�~��Mu��-`�H�����i@�`�x����x1� ��*��$��Y����'@���t&� �@�����Ҿ`����":` +F3335H�#��}_�}eś�7¸�<��d�{+}�7����I�㯲G7���٧T�XHH�h�a�R%)�‹N�QT�1Mر��&�#��8)B/ه:��ף{FFF����}0.""߁��/�� �?r�ȍ|8 �*��O۷o���p(i?���}I�S��/٧�����>?����\ܚ����>`����~��T�dؠ�]۸K�c�¸�3ǨǸ`Z��m��h�a���(:�%67�AP�Fe�DDq��Ǣ�3 �*""�|P�r�����{#��p�:uΩz��@S����̾�V�Q�;t�@��)�.Q@xC���V��>*���� ��*gee��Bv��M-R�OHBB��m�&�-Y�D�݁`�[�x1�8I���֭�4��ͥ��ԩ���� + +����w�6���a�%h`�,�f�#�S�-!���ɉN�V��c��!^���YZZR���ͱ?r�H`'Уd���l�2t�����O�W�mr�. ��c �!�nݒ��%�X1���|�r�%���c�*w��C������x�W`������`�K �"Q�Y���b���r� L�pk��(�{2.HW�6lz{{cpǎ��-�ҥKI��� �w�ƀ�69�����ܹsY�hQ���? ���P���#_1��S�왧��})a���¾��;\b�\�#� �c_ �����u��KӾk��ChR��s���͙3�NHIp�_�b����@��xxx�0�H��ŋmll�,��Y\]]Y�FI��|C;p��|)�r��P#88��U�bhgggΜ�D???�K�t:�3 +�c�����hѢ���&f�_�eD}&p�̕+W� �@t>>>�gφJb��F�-����&Z!���� ��q�|'FF�8w�t�@kѨr���I��ҥ��l1�#�����X�`���U�V�III�8��%� ���)[ B=􉕕�J?�ݻ���8o�<�Ji�Z��֭[7m��5F�VTT����ircij(��Xfuv��C"8`$@�J�����h*E�i��ճ�@�Ӹ}��a��_j�������N�������&�0�s�m�uZu֒���سy�}ͭ� +_ZZJ=��(�Bad�R�,�R���` 555%%%/_�T�n���~M +��az}��_9P�͑R�{�UfѢ�y�G ��oB����WC���f-�8�*q�\����h���^�‚�t�P��F�bMo��@>�z�Q�.]��a�<�m�!B�= ʠD��G&G)8�����6sK�"��TO�I��_q g8 JM�!X>)���&S�q|3Y� ��٧G�\��OI���?����B�Sn�A�����k̙�P�������/`Y�d�*���Y���O���C�Ѷ4����C�L sH �O 2��'�kt�6�g�0_O����pIS]���MM]V�og U����*�P��$ 4:'7�O<4�)�����o�g_�'&f_T�{��� ���66�>��|�Q��<�}����'�vȾ� �g�����ky"}eI���>��9�S� @��ijjz�������T����*]�'���s��8�}QE s��X�%[$��6 ��'���(m����'����W]I�<�}����_�v�>����N�f\�=M��I�����|�ݽM�}��sL��픰�L�}-'����iUF����|=��`��~� ��TQE��GQ�'�Ǔ�qϥ�S=�'c_�n�~�����Q��$ا���g�A�p�Ia��W����'��]���iw%��o� 8y����_��Qy��]hh������{�����r�5//��[�nI����k׮��������333����B�9r������˗/�vv6��~ZZ���_ee�\����h�\\��Ǐ�Va���dooo<��>:]Dd�[aa!��+�cbb�z�ⅿ���'JJJx}��)����,KX��%�UuuuPPPbb�˗/#""�%�P~��-��Y��իW�ϝ;w���ϟ;v,%%��WL4��N��\,��x/�"(�b �z���+W��y�F9�|���G��������z��ܹs!!!�L�L=g𳦦���y�M�lii)��k1�Y� EFF޽{��Ła�M|��a��De����Ξy�q٧� ��.�>&��_ZD8�l ��7�5�6��o�� +�¢�w8�OM+�zEոB��:uj��ҭ[�o��f�̙�lR����~�O�ػwo�>}Μ9�kRRR�0�o߾�ׯ�ض9��͙3g„ � ^)J S�L�ݻ7ӻw�Δ� ���+���q�z�s�N��={vqq��߲eˠA���b�*�`���ٵkW�1bľ}�����hƌ(���UUU)���t��o�R�;t�)6lXNN��Fh �x�CFF#7o�>|���%^-X������G.!��V.��"ެ�뉀�"��k,�>��F%�"���.�T�TWf�ן�I�z}0�oК` +��H�xB ��"�f�/ @����h4'??�`��V$n֬Y衬�٢��-��Oi�}Ç�����˗/�{J��'///4i�����B���GBs"���teD*� ��yFpLd���LZTZ�>�0�3�C����#���d߻w��0`&�K�V`|ii)�9t�.��={�l�Rp��NNN�A���bvY�f ��\�|����� ;�\4I`M�n� WWWe�8Ӛ�e��/��i�8hb'�d��ӓ�1��}�1ޒ1塓[��3|�}˗/W���Y��{�,"޲��9�y��``��T��S�\��}=�)S�� ����d�h^��222�%x������!�O�ݾ}{�ȑ4S�)��9c� Z��.����.���ɓ'� $mhhP��fYj�����ƩS�&''�"�Q�9]D�}�����C$��L�6-<<����Ñ�K����(�ggg�{��\�ѿ����9z�hX����~���X�n�a.�޾};��+Wx�YQQ!DGG�#��G�###�2r����C�H�={(��ΝnȌxEfO�D�dj*����� �"z7�9�u֯_O���d�9x� %�͛7e��4�>�L�#����� �Y��'�#�X�}���gR ��0���\+�LM�> �M�2e�9��[C3Q!/}�Ќ1")) !5�O�^0ڠ`_aa����Ǐ/�w�>�O�<���]xE����Q�@���S�p���1�9]D����� ��G���4Ymmm���`�={V�>ȢZD�N���`:!��ؐ1�/�����(�777�ɑ#G8��+W�ر�� 4�1泏톘�؇�����$�̟?�Hpp0�޸q��;�b!(������.�yyy��K^^�������U�=�v��`�*2��} ���L��B��a����ң�������G�� +�����׭�S�`ʘ1ch����ӧSW;::��*33��������RW#�����{�.�JQ�k׮q�ơ��O�Q��>__�ϟ?��'NB��L�d> q��<"\�}���3g�D�YD�fll� ����֢�k׮�s�-�N��\�lh�s����������:A���iri�BBB<<<�U��;w��^a����@�-Z����~�+++�-��v�s� �I^ѯm۶���]D��>OOϘ.�0�]L�'���5;;{˖-�qvv�橩�,��jIW�SL�����"�@'kP��tqd�dWWWq���H���KP�+�BV���s����+**�ui���>s?'N6�H|oq���iR2�����0XؗS]y��b�>#��>i�)�5 #22=�t��$������U�����SП����}�h2���� +R�M�6}������ޞ*�JE� ����?U��}������-�H�۷o�O�177W)�^���:t(:�D~�ĉ@ă ����ʈT�I�1�͛7��,��*����D�3R��`D�� K��}�$����~�@�@��m��zf޼yiiib���ιx�*^���t.��$�4�n�JZ�?ϟ?��X�bŋ/�x +'�Q�W����/���u�tpp���z�TAA���eP��a#� ������YĬ��W�ǧ!"�.BcA��O�.󯚫���B�����b}� +� ]LqqqA�@R�H QQ/\��%%%!��>}� \pssC� &��!��O�d{�fh/���G�E_@�D�\%h�gΣ���C� H"����"�<�0��Ç�W�𪫫C'Y�>�N��'�߸q#�{�n&�p�����fΜ9MMM����GU@J� X��}8@_�Dv��1qʰ�������Ǐ��?�}�Kb9S�Z�H���}ᰏ� C���P������#.38f��ŋ��ke��ž�Ӗe���"�X�}��s��]�&�jR��$6����B���e�?�lL���'����uvv;v,MШQ���%K�!2�bP��F��F�����j� +�1K�e��͛�����N���A�� �R�[(;A�'Xp��N�:�b����)))�#�g0 +uqq���#����:t����{��D :tU�#ڈ�O�4����i:M�`$R��d�Xl�K�*�A�+a������*�a��6l �bF�I Q@��э�<��K���cd}}=Ga �9,;;�������rE���������2W�>V`�2K@��Q�>&644pu9/�>���pdl�lġ��Çɧ��(�Ĕ�sF=%s�ka�s��S鍱Z��O��D�ӈx˲o�溝&7==�:-��Nj�-�>�)O�Hٜ����ܺ3Z��D��šWR�L��˗���H```ZZ + �+4�����茐)��� +�ȷG�e����3&**���M,�.Q�C1�G4��ի����#&v��=��~���`v�c�j�������� + +���?}�4M�%����*G��{�}آ�����$"S�D��"�|��� ���|P�p��q� �F숰<#��� �L#���S�ǭ�hش��]>,++����u������0jJ�]8;<��S ���K�xy(�R�Uaaall,)��#9�s�y�0��*��D����Z����D��q4999�K:� ��D��ϸQPP�l�vx��c�cz� �)�`_ �d�����m������ u˲/#=�qv�/Y�V�N�����2�)�YZ��s�S���?'��f5� Z������������N�W[�}O�� ��t� a_�G�nS��Z~}_�k{�ݟ�}V�/��TS���2S���R�aj��A^ae��9s̼�9$�򾺺��y�E馻�t�e�J������&�.��m�j���ͱ���?���g=������_N�~�~F����ًrp��Y_/�����=�%�#�{v�2��"�F4�W���;�֗�H�ڃc'N�X�Oʾ^��ڻR6<a_��i'���>��\/�7.�/���z���4��L7ҵ!�8q�Į~^����I]r����@��u"-�,9؈E�1��~�;/������!�c'N�X��ξ~:�Q�N����N>�{�i�Ej�٤�o!���} �ܭR�͓l�g_��?W�"�j�}�8q����%��-��_� �u�eE�� L;�6f�}����θ�g�e��g�}ѝ���������Cw4��� �8q�u4���k�yP�oPV�����/3�"-�,9� ��#0�Ӫ��D�}�=����nJ�k���G��j|�W?�?�7 +�����������>���g^uuuy{{����۱�� L@BB­���ǫ�ϟ{yy���!?�-33I�����!99y`��O�fee������ +���b������f�RSS��'O�tv�w���顡�---‡-//���/..^,,,�"� |v��LJ�(����<�������$\gnn.�"���ۅO�#��� 7㯀?A[[�/N��c_?A@['��' �G!&첏\��{K6�F� �a�ҟ�����v�j-e :BIڶ���&M���=z����ĉy<^cc#�b;;;K +4f�0u�Ԡ����dغu�������!988����������Nii��񉑑0޿�dϞ=�;w4A=...L�p$����������XZZ�sMMM�ɉ�Ǐ�7o�۷�,GFFN�<@gqLGGG4!55�Y1w�څ#��Aa��F�BC𨬬 �!xڴieee$aNNΪU�HW�===`H%��:,^�r��5X�r��m۶ ���o�He_��nQWnP6"���nL���t�� ��#G�.6lprr +�x���իeeeϜ9H؇G��޽{�) ���ʒ%K���WUUuuu�'�����>�a򛘘744��O�~��U�<���������6�FQ��@V^�~���9998,���d߳g�f͚%//�㧤��C��/þ^�p���/���+**�![\\܌3�Νkhhx����.�p�[�����8�:*R�'��� �`?p���7o�"&|yy9H��M�R$%%1_yzz��a��-���Vqq���`����8������{����b�ӧO�N�������Ǐw��KU[[�-�)�%##cccp�uuu$r8�koo?t�������H�����x���v�߿'+�þ�/_����X�h�ptl„ �w�&��uxDr04++ 1�8 S#�}p�.Gw�։u� �q�)��s�R���r�y⥄h�}Cټy���"15�:��|>L����@�A111p+>>>�}��<(�1>��IJJ�8q)����f*�<���ȃbbccǎ � ����URR���D6<��k֬��155E���p�s8�+((��;VYY�v�Z88����}(nF����m�Ν��Eh݊+���$$$���_]]ͱ��05���?�ޕ���>�����`�$����>��^$�2�mDq���0l1{���1ɿx�������Ʌ��� ��/ �̙� p�;���}�����OJJ��͛�$8�~�z�!�'..�WLMMMb����utt����|�s��Yfw<"9|��� ���Q*�����= ��C iܸq�.�CBB`�p��!�WRR�##���s`�PmDDi�����Ql ����ı��p� ��x�k��딕#��# `Z�II�o�< �U~���H�}_����h�#�}PNN��� +\a�������~!�h��>{�����;�)00������}���0�&&&�������}���ӧ�<���$jsuu���^�z5�|}}�w�ޑ#�1�&aaaiii ˖-���`싊��^�"�ioo�-=z��N�FGG#-\-h�n�:p���>�, �������>#vA=hW||<þ7����qqq����8}�~)���ѝ�s��\w�_�l|���]�1����r��}��(����úp�BKK�𺩩)@���G؇x=CCÓ'O���&''|�d��ϰ�[���۷R �`�SVV����j�555jjj�����9� +�:�����������,�ddd$$$� ���s�xBv�H"��勊�\�7~�xl4�O���1I��+++�?UWWǾ̏ +��7���������QUU]�x1�ñ��w�d_?�]E�}�c6�&���8%t?��c蜙`-0���jkkap���)�[�����4�d���"�KJJJ�j�!C^^^�����0ۅ��-�}�V��p���V���� ��`�����nM���`B]���� ����bLΎ����ʍ7JKK{{{744455EDD��mٲg������ >�۴i��C�M=�C��|�oll��[�t)ЌM s��i�.�c���/˾^�<�K�)+G`�\Y�{ܺs���� pM���Ymi1�zԌg+�ම33��1���zC;T$!!r"�sN��ɉ��۸�KC)i"B���H��K��X����r��>i����Y{�o}�[+�o}�-��O<��C[�s��U�'D1B��ʣ(�Zi׮�������~������ +�����{Æ U�������Wy%x�9�����[/m�w���=/r{L�9Zg�l��e՟�l�����_e"�����޽{�������&>>��. +.�=z������M&ӊ:���O��IIIrgR�N�[�z5ز�c�X�� �C�NZ�v͚5����%999rD�#TPP���7)��l�t�0$##�lݺu...�"33^� legg�E)%%e����������AAA0Qt���`FWWW[̎���������l]eҀ� +O�n2=��L��Jկ���Wf�,5D�-̊��1�|C��:����g ��'��oW�}��K�O=��]�}v���'���UUU�A=���I�OH5WUG�� +���bG�߲C�� /���"��uX���P�t����E����c�!�.!��v����v +;?�DvѦ�vժT=�^<��KB�W��g+ ���f�<$W���h0Ej�&e�7���!q�bݜ�w��.��/�>U�T� ��d魗��j@M���|���'��� �x�Cd�����h��]z����9�Q�T��R���JY����y]g� 4+��� ��j%7��'0,lg�!\a�e쯭��Su�0���ޟU��R�����'׀�T�B�́҆�/�h�� 5��R�{|r�ݻt�J�?�T��R����q�'�W�����S�z���g����&?G`�x"�����IY�=8<�jK��e�N�:��O�*U�H�>��RC� ]��@������^Z����' +�m�Qݼ�S�}�������FN���V�}�T��r4��,1D�k#��f�6:�x��b�4�;l�W��� +�OVWT_J��iϾ���œ�|�U�R���rL���������j@M���OڠQ�}}|b�x�u��m��3̞�͊�OV�TJ�璣N �Q7�dY������'������ƍ׺u�_~�O�>�֭��ʺw�***>�裗^z�aÆ�5jڴi�޽�l�r��5�fdd��Ջ��lԹs�cǎUWW��W���ZpppMM�����j�ԩC��q���IMM�֭�[o�%I�m��;whY�jջ�۬Y�6m�L�2%!!����A�++>�駟2�%t��uٲe�N�‡���={6h�@�/�*8��˗/�Y��%���+��4iR||�f�s��d�}a�111�eee�F�b ~~~�!�t��%�+�H�!C��F��۷o��_|!�}����7%%�����r~>��Ü���� X�ٮ�n�7�xc̘1���������t-Z��۷�+W�L�N�g����...�k�߿ݵ�߿�W�X!$�8@���Enݺe�Z �x!8,������}�)5D^�E\ W|r �*yi�������ޱ{��B�rC��� +�O����FNQZ\F�����("R�g޼yd�#F�6l�0c� rڠA�~��:���ŋ���tH�-" +������������a�`y�}����� :��v�ږ-[���˗cA��A���۵k7k�,|�/Æ #͂<��j����d����:��I6�����>;��G���r��;��G�}��M�%���;www|�n@����h2�)�|&� �C�>}����v"�����;v�$�}���6� �-�vE�����5���I���۞I֎�},�sB#�~��p���8�W��Ҷm[BG7��pn\c��l/�g�(�C�ڈ�ʃ��Qn3J[�$W����]��,�Y��R�Ve�w�a��*��u���ί�H>>>�2@vv6�����B��ܱcG�B������O/^$#�G�`仺,�>q-ǔȺOg�v'''�J.��¸��d3��4>���g��ڵ�����ɓ�G�:~��H���)L�qx�c��ׯ�MGN#Qw�Ёb�Ra?y�P��^H� /���� -T +�簰0����.j%. YYY`��J!c[d �1� k�(**�LN��+�$gV7\��N�8�� Y�#�}�a�C����ĭ�1�� .���@{'�X�yV4v�X��,��$&&b\܎�Ν;K�y�'&���dќ9s� :�ٳ�P�эS���^�u9h��ǡ�U�>�^�L"��5q�X^^NeG��z=+�={v�V� �ᬗ�O��L��}���|A�i��Y7��wK�C<@�18��yڬ�Vd ����e�N�2Ͻ�|>l�'�~��7��'�w�K��T/Ri��Ta�qܢ��S��4 Y���e�^c�,�8�{�,F߸�5��D0�"ġ��hS�Nj4E�8ITP�:е�d�+����FG�8D�9�)<`� �P� 5��Y�Y/�{��S��-�+O��t]�����������K�.�P)����n�cz�y���[��� �{�G�5 商 ������^K��Ǵ~A��o�N�)9O$�P��B`���} A2���1-r��7�J�an�}ak��ǴݢB����8���Y�w�]w�"�ό�{�ڠ�g_�� �����[���իt���D��}������H6o��'w�Xv�q���w\6�"�l�tZ LW��֯��ƙ�>�oR�G+j��������fU��S����w��}U�fS���v�/>\���[�x"@ +`1�>��gϞ���믿/�����:���[\\L��n����Ƒ#G�W������ǼX���+�]����'�3'��'M���62���G{���;3���ܢl�E�/��2��G���g$�Q�*`��9�Dx1@�^?�>�9a���;�0�dĈ�=�@�C# 0q"N]g���;6jjjB���%�\������@�[2���K�#LიG6�]�E)��2��Z�*�|ŽyK:�����������1@L����СC��䔾Ϸ�t�R`}�g�ڵ�]]�cqv�����^ +�L�O��;j�(&18S����:-?f���C]����)�h�+���k׮>��ӧ�9��3�}~'S�'*���������J��'����~'+M��W�H�s�}(F3 {��))����jF8 ��裏�=���ӦMÎ�>&e�ĉ/��I�&1n�j�}}������6n��&� ���0� +�ށ����ro�lv +�2h� YP�65*�I�Bf,^�8z/�RWWGo)'2��M(&����� �����'ا&���xgذaݻwgj�����_u&ź �Y��_u�Uw�q� ��nݺ(}ƭ�����u�(++3+�O���sυ)`���ܹ{�-�������+W���Vd�$XT1W��Y7�t��_��Ç�n�Y�}����}��������w�}p�S�y��yF����Ԓa_�.]B���2�B^}�U��{�M�ܭ���>����lg�ˡC���:4?˗/�J����&�egz��3�}~�*~�v_+��l3y���;<��8a{`_0�'���*�U��N��͉4�Z��4�2p�-�#����ϧ:m-1�Y�`��d`�?~����gE��������>���˻�� �[� eN�7���q��ab"/h� =ޒ�c`�<�W�# s-�JS���v�ej�ŖVk��{�ףG�d�֭���N/���`X�O��z�����袋By�W�*,�<�}���+�|!�|p���f��KA��}��&�/&�R)0R�Pj��R۶mk� IH�jl:Gf�gΜ���š���w�wrm�d�|`�~�}�EĎ�n�)�B7�� =��%%%-��w�y�h�Eg�A������~�ر�ܱ/���ulG>���h��&�z�7a„�/����0;ӳo���C�ˊ��e�;����ߎ�ߧ��J/����~��.h�~c��B�\fA����l��H��/?}�t��>�A�����@� +�}��w�A����u���6 ���U�V��o��7��Y\��M�R���"�Y�EZ�z�j �رc-�t������?��C֡`�a���z����p�#F��߿�c� c`��Y�'c��\�dI���9�[�_�~��-��l�2 fT�n�޽{SZ���D��g`���뢕�y�����쳚:{8//>��. l�����sFӦM��2d� U�>�Y�}N�1��� %�+�����M^Ќ�*�@�*�)�/~s�ʕ�� �}��ޫ�mw���{����~��6:��x���� +++ӡ��}r��r׷@y�}N�y�O�s�?�#���j�v�С�UE�췜}�,`_D��+��Y^���oǶy~魕�e醢�� +ڝ}~��'��~6ꉃ�r䜣�#5lI�8q"�Bp�3x��a�<�a~ꩧ���}��������E_�f i-,,)��k��ݠ@K���":yR���/�$�_|���O���I�Ѝ�R)PK�=�XbH�'�e@��&pSx2n�ɓ'��#k�-R��EEE���o�]��f�-Id���k����j2lذ��rL 5I�~*�F�@�9m:F�\vR#�XA���������nz"xW�>�lذ��_8p���w}}}��Z)rbV�>}���t�H���"��u�(�V�G��V}Af�;7d������{���N�_���f9��rh����c��)�^_K|�'&Bc�&�h�KI��φ?��`V !��З�q����7���ؗ�?�1� �b9��ؗ���155����'�wDk��DM~�,��?��!\1�T�ﱁϑ��D@��K��m;�lq<��0Lg9����X��4���1Vy�w!#5�i��&?�t���Oo���)�c�i���'{�D��C���B�����k,�v8�z@�7W���4�i��&k��K֧?V}�U���}�I2�=��ĸy:�>0Lo9�w�����)RS��VB�d�f�������DM4�t�y��L���Ŵ/�>�f����3����K���a��5u� �|��r�y��`��v�+g�}�S��295�DM~��l� �Y���QŴ�����{�����+�I�>81��C�[�N��9�p� Iy��⳸��#� ��>�&�ĬIKM4�D��*Y��d}d�.Ҫ3+� �pD1�v8���g�K2�=��Ĺy:�^������_���� ؇|H������6�9���ݻ�'O.Y�d̘1�'O6�>��^�re�J�L�€��>��~��ݣG�V1b���כ7o����O��~�z֗��_�>u����`;�^��{�Ȅ 6m�� +?~�kLL̜9s������ݻ|MII9|�����޼y����3޿|�r�ʕv�Ə.��Mooo&�:uJm+,^�X�;v�+Ο?���9&99� .d�̙3����%��a��'O����_��î��Xe�wEQ���gϞ=nܸ �ٳ����ҼG�yzzN�6 ?/Z���ѣo߾�sTGSȌ3��n�ʚW��Yk֬��"���I6�5w�\�[�k׮Y�f�5�1�!��͛�������g#R�+�9r$���L�'��y�ڵ�q�ĉ���۷�a�N~͟�"Y�>�"��fEwF1�s8�d�ݽ�8�x���滦�>��j��}i�c��K����ϟ?gΜ +�o�**J.Hq��޽{3Q�q��Pm����.^�H,p����8Q��T��L�2l_��W&��S��~�v�^�z%�SH�W�΀��i �[��[��lȦ���dɒ�R�1�W��~a�՞�lGT�I��Y��׷D��|(��Pg�2�+AժUEb�,��4.W�.��p�.\E"�I?�nܸA4�򇸋 ϠA��HJ�x�B�I�:ujӦ i�6� �;�U�C����|C-Z�����޻wOL�۷/Jpe��7o.� XB2cER�9/(f2�ؔ8�|bG�˗��|�o����{dj�f�d%��������*ƽ����=� �ュ^Q����q<�"cu!7<������=`���i��A������?""����ݺuU�T�����`8Whzz�;8q�V�mc���8˕+W ��o��2+���rI.U�z�A��ҝQ˃X��={�Y(�R"�$��D�M[-��%�� �.]������}����R )��x�JN�6���n�]�v��be�v��u�ԡ$R��Ia�o��ƚׯ_L쎕i.x +шÉ�E��`ɋ��lٲ�߿_��ޥK(�:b���W���)�x^(11��<��̺v�Z�V�� ��4i���c'N#=�l��d)�wQ�ijD�|M�JմiSԡ�Y�f�c��6�� u��e����!o�H!|E>��;G��!C��� �� (���-���)�Z�j�*@�ޖ���͛ 6�!�|Ù�3f ����|��&ק4[r���nǎ�_���0v2�7d����`����Ęo8� n�0�[�n�����N��� ��&��o��e)�h �*!�����щЬQ7FG=���'6U������رc�,@iBB�P����Q�Z5t �J���� ߺ����,�J>z�h�F��b�7n��X�M1�n@�2�ĉ\96m�P� +�w��i�W�~}�����'�s��:�� ��;��������3g��5�@=���X���lS"�}����Ď}b���111�>����zxx0�k�#'� � �"@ Ǯ^�Z��M�;ͬ���~�}B�����l۶��&T�ohi����K���hug���O�0^�}w��3�/���u�m:��wXc�ZV�ZEm��ٽ���R��ܹ��}!!!�P@ ���y�RU��Yhk`_�J�,X�u�SI��~?��0���K�}�/�e���\�9攝�ÇSr)J\�8Dq죯�:u���%�v��i���s�N�ro�>Oς+$D2���@�J�m�^���.�*-�a0���Y(�8-#�PM��~�ڄ �bcc[�nͦ����f�S���R��g�g�s�#�؇n�@m��%4����B�(�����mh���q*��|@�*���)ݐE�E��E|��A�*� +>��E�҂hm�4j�������6f����T�Xjաk��:}Q��lg������sb)]��M�����뾮��}����(�t���I"���,��>�K�D�w�}����ۅ��}��(((Уm߾= �J� �a�w��.]���>-�eec��.�Rd����GyD}"��*���,�*��g2�LS���p6�x[K�)S�(~����W^^.�b���q�F�3Y�q����q@��O�v�!�R<Ҽvws��沶�_�}U��/��S{`_�w�\ÉMj6�ľ�K�.]��p���^�z�J�}3f���w���� /n�r�tj}��A�L �(���zW���V��-�#V:P�蘦N��_�l������?مnw���,^��u��ќ��B�O~�}���V�Ν�R �)�=z4� ٩��!���$a}^�`�����#s4�ʷ�Ͼ�E��"��/��߻���:t���?}���C�uV�@J�}�#?o&LHf��+e�3g�`7���ǏG 4zg�G��,�iÇg��R�'�۶m�"G�'�֭[��°$d�[WW�>�-, �޲��p��� גi���|�n��u��\I�sXdaϥK�� �y������t�Q�e��$���B����V�g����z��-�4�>(EDSfO����� ��Ucc�ɮ��R�ǎ ��ꫯrQ�)�}.� +�����j��E�\�ؼ��?lk�%=��J6|��;���>�.��;Q�}�� ��>�H�ܚ����;p�\g"��?� ��*jյ���ؗ�T +�����5�Ըz��Z�96%͉���K �}���[NDW��\�|r�x�,]�ԋ�T x�cF��!;F�A!=�đܵk�F�?��a-��e`�#�/m?;�?u|����@�ȉ��� +�v��������2��H��>c�ƍ|�y����� 1kI�~�i�\]]-� ���#kE� �+i]�j���,�$���<�W�?���u�[������Ϛ�߼y��b6�=��A�E��n����ɐ U�… �3��cK��޽;֜8q�ك}؝���� ��A�d���g_,�ljjB"i`eŃ���Ar�%�>���9s�wܸqX)���t�8_s�ɲ�K��������=�9rd�}~ʹ�\��;祥��Ν����v¾�6�%��Kk�|_�|��ʚ��+�������K�Η�>�r�{�}�8=`����Q��R+2�K�#,��EH���S�>3us��'��Y��'T���>-�-���ȑ�;w� � ����+����CT�f4��@p�ϋ!,��\$=���-�`R�իWcJ��g6dǬY�<��!�6�!C���dMJ���#���) �5�}V�裏(\�p�{�1��ndp���Z�����; �U_ňK>�j|[e���O�4�.JA�TM��=z4dH�Ĕ�Ꭹ��߿��+W2Y� �����r��СC%O���j�A�_z�%60���'���^{�3�ir@BBҲe�ҫi�4M¤� �u��u����A���zq�� ����˭���/��b����g��Iy�3 �*C$*KV�X!C&N�h�$X���_:J��̬����Ʉ���"��L�}�i�"ʃv�L��=��CJ�� �t�a)�8�Kz�z���}DzG���i' ��������}��5�|�����^��{�}�D�:u��裏.X���?޵k���Y�ѝ +���|K�����?�uu��(̜9�R��n�;wnL& *�^x!����.gΜ�̘1���;�u�Foܸ�{���x�a��^��l~�2.^�د_?G�`žk׮��}B9P��у�����Ma�M����@La�� +�M��w�/L��X<���X�,YB9PԈ��c�'L�`S��ٳ'�g骽F�%��n�;v,�qi��ׯO�2��u�T��Hzx]�KKK�e�X0ا����O�!�r�%b|x'(\� �U��/郜?�l�:R9����lGN�c�O +��*F�τt�菼��4�����Q%j>�-Z�X�c +���a�� �։ݓO>�~�zHRDiWU ��?����A�ȑ#n�hl�-(�'O���e;Y�t)�z����u̘1�a۶m�Ǐ��ӧO�]��-\�P(�3��[j��}�+����K��\���������J6~������m�0���W��YTW��^d_&�?���1Z�&���1ȣ�6o޼V���o(\;}s��� X��d>�<��3J_␳�޽{{����o&���!�[O?��铳���2+z��l�,��⩧�"_V� �y����h�ŷnݚ.�Cm,u���/�����;��9s�D�N͊��T�`_��1�O_|�E'U�'�kd����)� ��a���СCAp �!q�ڋxj1�+^�^6(ZZ .Z,��sd��Daa��ď���瞳���d��X�SzA��ɓ'3 ��Pz�#�pj�FyP[[�)o��� &��b#��&�Y �ئ��?ms(���?/"�R]lٲ%��<�������ٳ��c����� +t�_�s�ԩ���p����� ���&|(u�\R���_W % ��`]�T}��H�F���Tt�1���$�Aa�2d��p�v��Æ �޽;��o���rE�í��o?��筿��q�����ds��emϾ�5���T�ye.ۄ}�˛��n���@��[տ�ٗ� +������������S�S��@���O��o+**���طoߝ;w��[�W(�]� Q� յZ� uU٢�]t>��i�&B�N{iv��QYY� �a*��5�u��� + � sVǪ�!� +i�������4�����0,9��~$�&�X�"z��t��usx&Q�d(?������&< +����X�tz j���{���ի�FV��K��9�Ex�����Рw�V��1��������&���$�{�<��1��7�Ǧ4����|�NpGI6���S���m��ԉy s������c���C+H�jX�.����d���@ ���~�x��"��x>����3�+�6�7�eյ�q� ��}��I��j[w#�ZM5n�����/M��G��>��%Omc�ofT���"X� U���� ��f�G�s32ԇM��z���˹����{�}�7--͹�y���👉�)���Z�ڱ�l&�A>���`p�p����?Q��v����T�~e�ω�ђ.B��9�dx���� �o�&j���(c�-/�r.�D�� �ߌ}�T�R��Ua���嘂��>'F=1���t2v)�}�ߵ��X�Y���%�C*�T�R��&ڋ͖8���o�}��>c%��OX���b}@��߸�W ���Z��MAa*�T�R�����ڬw�JϤ+�}xQ҅K��:�l t��~I�P��G���Y�Ǘ���S�J��������M�W��>��$���� 0~�>q⊫�}O>��f��39��O�*U�j.�d�\�0$(�}N����J��A����?�p}§IV$�k�oq��e���K-�WM�!%�CO�o�BS@h-�(U�T��K��j�fZΤ����>���tuO�J��N��u�Ooo�x'U��Pc��0�=��v�-7�r>�d�Q���)K%m��aI���R���>}���/ ..����F�6''���{�966�d2��j�&$$�ݻw���3&&�W�$�K�x��u�222��"���L9H`` i�����$?~|��"������'O�߿�ȑ#)))�E�:q�D�����v�ٳg��`08��9�:w����f��u��Q__�cǎ�F��^^^N���$ w�����l��M�&OD&�y���s2B��^�p�yE@���"��!���������s U��H�ޡ��F���D��O6�%� �w����޿7Z�5ŷ�7���>!���j�2���Zg�9�^b���SsQ)�cƌ�ѣG�v�:t�������߸qCt����ӧ��C�۷�߿���닋��f��~K����s͚5��ڵk]�t9r$0�� !Zpp���G��߫W/9�H�8yyyt���ǏwZ���@�-,����ڵ+�w�����__�z5�dԔ)S���ݻ7���!��aØ�モQ�����(қ7o��-pp�Ν��3�m۶s��o����͛����҈#&N�x��m9 ����o���2��ب�3g����ݻ���|g �~���xťB�4i�������d:���-_��M�6s�̡�rZT��M�K�-Q���Jc��%��:3�O��"G��~O��0����>� +.��E������&P�)��,***<<|��nnn������I�A�Y�fa��Ydd�{�G��[�n��s͡����w�R��\�ҢE�z��M�>��Çb�48�*�G����_!117y��֭[k�ZD�~�mJ�5']�~��RC� ����$��?�95���5j(�BV%�����OÆ I� �E +ݹsJ~��b�3g�-[ָq�%K��6nٲ�� 4�������[ �E�؁>'($�W_}uƌ�\�`X�����L�n��W �[�n����ݠU�V�!����͛b��+П?��Ϻ�U�Ň��c�J� 'ٖ]T�n�4� ��h�%]��?�� ��35wR�}�TZZ%�r��e��R�!4��QE�>��C��":P`��p��:v옚����۲eKlQӦM�>��3�G��ݻ�F +Y���`�`�ܹs]g�/$�@sE�Ȫ���Q%n`��U�|��:u�o����.���O�&�}��gD-�1f��a�bbb^y�ٳg 9H�x�"� z>/�= y7~��\���O(Z�?�����=+7n��y$ ��gddȍD��o��:�(�7ae���J�������<=�|4Ii�s0VҞ�t'%��:g��Y�5EF�!��J|u|��q�k��ʓ���I���S�]�G�T~�><���w�C���}�^�1v�X����O �9��n�ҥ�(�>R +w�f��?ؼys�(u>==�FA����#$$D ����z�*i<�4}���]�vݶm�(,*//O�ue� �@�v�Z�(��,�>� +�R���dk�>�V�\��{��\N�B<==Igʆ��� �':���W��ѣGϚ5+77wժU�;w�W>��P�J1zL@syY��o����0���][��>���v�/=�(���\���x��F�EGGWa'p׮]�n˖-�mJ1�pv�‚%&&>r�o���� bp�����,�6mb^�� 4wҲe�x%H�hѢf͚5i҄?��#A������J iѢv��[�I�~��ݻw��q�F���� +�Taن�����Ϝ9��L8�jwU��l ���dRC��|�FӮ];6��� +oK ���Ʋ-�!.��cC�Zm�N�N�:�2'[n,�U�� +{��n��x�*�}N�$��%}���c��?K��V����z�}B>>>p- �J;�*ݲe���d�>4dȐ�C�B�8��h4R�a_�֭7lذ�!��pO�};v��2h�������=�Q�)�L%yhܸ1f�x+�G���e��CJJ�֭[���nڴi0E��gϞ[y�O@�.\�U��b�Hl�ԩ�M��W�^�Iɧo߾��?y��+Vpg �l��%�z`�J�xf�j�>����$ ��t�֍A�\$F�٥K��gϊ�]�G8>.!x=�p�a���jV�ꯥ�2k��CRL�8���Ԟ��OJ�CJ`?[��Ȩ�H�"��ҳ�2`�LDnn��h�Z�;�S�4i�H����rQ�?��s|��`^�}"55�5�3� + + +��� �FL��=����2����p*K����X�qMp��d�--�1USV�P܍�F4�5�ˤ�JF ��v��PGTpD�e\�A�۸ в���n����;/1�����W]]���{ι��w�w����3g�:::Z �۪�4��������Çc�b�������,̮Y��a� +��4""���} f���c�n��A[�C8�|M�233Y�`ߠA�x=�w �?�})))�����W�\i׮]ݺu�W�Q�Ft�8*�x+ ZP�D���k�+���n�:����KWm ����f���6�}���%�b ���qg�Z���n&u��_��2>lذ�g��;��z�����Tr��ɓ'C�3cd����ǔ;w�P�_~�e��'���� �DҠA�e˖ lU�>��%Kh��{���޽{S��V�Sؗ��޵kW��gϞVj����t�…��w�ܹЇ�.\�ѣ�$W��Xia���?�7o�*�9|f�9�N|���� �F�xg��_~�����4�����U���?�u۶mrʁ��!C��Sz���$�#�#<ٶ��W�OA�|J1�*�f������ ���)�<��e��S��� �ի�7����ɓb�`ߔ)S(����}L�i\)x7u�T=5��+w��iF>�};w�W��ڶm�H�>5V�N L_�x�{��EP���^lݰa6����պuk���D'D�6a��E�}ds�%�'���@OCޤ����7�����ɥa���Fbq��N>A�z8���,S�>������͞=���P=��Lٽ{�:�t�vq�ݎ���,��$�骷�O��?Jp�b:� �{�> ,I�T�����+����/_��5k����=����e��PF�|�hR:4}���*M�6mǎ��=�ЩS��/���4g��٬a��Ktt���_�ٿ?��]�v���3g�`|���̜9s�ʕ�R4z���Y�1<<|˖-4������6�K� +h�$�333A!Y���1�7,,l����U���pp�o��[�.''GZ&�E��\2���}������3�I;w�c�jF��� V���_�| b���KH~~~�h�;����g��ݻWn�.]�y�w���n;��m��h�V�a��>OF����޳���_�� �>]�t����v�r����x%�b+>1J�i�� �s]{��=e�9-���O�.]�j�<.w��,{� �1�{ا�(A��_J�_�>�����J�� �',������ӥK��Z*���zdsƦ��~��IWMhV ъ1L1y�Ș�ԕ�Ut49à�O�.]�j�(�2�+�Vu�W㼫��W��t���'�����[��&<��٧K�.]�E���V���u���yW /)�c����>�}�1�'S)9r�ap��>]�t骭����J��T�P1�U�a?ύ}BenwRVѿ�3 U�7�tٜ��o�.]�t�z���8��z-�7�����7��=�G6gLZ��ki�}0^�s�%���ܻ��n�}�vr��޽��y�ݲ��$�P�n�*,,�n^^��h�Xrrr�D�˅)y�Ν;��ȻR\)**������KKKշ�{��}i�޽{���A"'����|"��bII � iA3��p���BAA���HRRR��v�#���$G�]'倴�4���4��K�)A|vv��(B��� �T'933S���%�r�R'Vynn.#��J0D�՛�o����u������^�ڮ]��jذa�N��o�.�8�#F4���x���P��ڵ���_�7n�`����Wm�M�6�W���l�5o֬Y-Z���k��֯_���A�6l��B�V�֬YCA.�HoDDD��탃��A�֞={ڶm)�8�N����;�4j� :tX�~=(�L�~�:�5�6n�(J(� >|�С���Z�l�IΘ1c(_r�p��[oI�DңG����R�5yhّ֬#G�+X�y�f��ĉű�NXcG�6m�^@O��ŋc��+�8T�'Of99��q8 f����ǎ��Z�Vy%::��ںu+��͹s�v�ޝ5�Y�~�-{$N��o��j�*�@p�s�Y��o�G����l�3nܸ�!��_Kn��Ĝj;x��yW�Y�tT1��,��J]�pK?���<�bbbxjz��b� +�%K� 8�"�x�`���/�dѢE�*�v����D�H�O?���TKL1���w�ҥs��f�^�^=*9�˔��={�n�\.]����V֩Sg޼y�S���T�nݺ �L����#G�D0����͛7߹s�\ E��ɓ'@��6mN�Ca� ,��e������@���� +|Qx���}��q�H��} ��Ef��J%��&�$$���_|���O��M��ѣ��}����ڴi}���O>y��W?��9s�=���d��X�|��������>q�Dݺu)�P�U�D��X�ʕ+Eb)�d�R�eB��[�;i�$R�?��-����8�+�[��4�F���RL�Q�D��a0���EE�(�[q�D� 3L" ��Qp��C0$rQd uH�F:E�<��_���9s���g��3������k��kM�4I9$>]]]8��lY#�&���o,bZ[[1B�4&���OX̩,� ���|p.Iɥ�ĉE��~�i�^�a]]���ː!C,a�9s�@�ՕarFNⅪ��m:q)�`����2gРA�&��S�^����Q-]�G�y�E)9��[ι�=�ò���[����_���СC]ώ��0�� �I{{�3�����>\z�4J#\���~�`��{����.y���n������� +�����}�r �)�����jVAZix�OD0K��>Z�~����rm�)�uǎk�B&����F���R�@%���T���o۶�e�#�J�7�w�����fP�+� wE� +˳�3t�o�������Z�|9 /Bg��F��^en!*Z�1�͝[) J�Igg'�j(b�40m3o^��4�N�W$W��y���E-r)���I4� �z��*اc�����>W\q�ii��N����ƗB&����… � @?�}Bc�e�������fɊx�_SVa�~'�o��&dD�|�܍�7��N����'��ͳ�(�:�w�yG(����U}��xt�G�WR�P�}Bdۣz���%��&���Wȴ�޺�zM����I֤�����1c�P�"��i容暳�>�$Vf�!�4��g�}�*k׮Mo9/�'L�:�"�J٧����K%�Ν;o��f�M~/e�UNG}�,Y�t�%4�6�޸����|w���˳��e˖i�<��W_}U ��;�v`����]{7v� D��E-� `�}9 �)�i��FtÆ 1b� � 4��O<)3׍�E5�S�Ƈ���O��!�������/����>��-}�]w�eη�z�ԟ[n��LQ��쳄�Ϣ[�n�̾>�@~�w��uY�q�7�Q��;��۳gO���w�Ǟ�opc�|͟z�"Ѿ���ٳS�w�G��3�Z�ɷz7]��2�uL�%��OH�&%��r�)���<�CHC�}3f̰��g׮]b���q�ƩOdEٸ�,7t��?���}��e٧���_I��"c�|| +��9hX�b�y���/�8q�#��V���8J��e�iǼ}��7����/ �)-1�o:/�6m���� �Uڈ�H��3�I��_!��H��Ͳ��Ugg�i��v�9��s�Lͦ�.��r�555�<�#;�Yb�ab;u�T ���N3}���B������}�;�6��a7@����ڃ���Ă}���{�%�`q���+W��6��~�?�63�m���������㏧H>T9�M#'O��wD���dL��@%�-�V��k��VtO�n�c�����Ϸ�EE�~Ly���[ʣa �ia+Rڰ��;��� �8@3Ш޽ ��>�k� Pp-��X�K&�>}:��w�y---1�� +xm��" +~7�vT <$��4W:��ر4�YPc��f�B?�4�q��~;���������o�q ���e�]6���>�cJ٧���5J�c ݍ'=���JY�Y�����߾�򆆆����Ɏ�x�p��}���g��ϥ�-'��S�B&w�_|����ۅԟ �}2J�����V�ғ&��4lS��蹇�p&�1CY�1��d���mcN��5�?��s�\� +YǪ��������˗/Ϸ�U��M��������;�n[{��mXQe_ނ}.�ʜ8�>����>Of鷆�N� >��+����GE%�4��R�{����N8ᩧ���WX��� �&#f����e�g�}"&���o׭[G�|����n= $�mmm�w��� T����e�9�>l\��Ie�͝[) ��o܆<1Q�S6�\�:8\(�fe�M������Y�Q�D�\J�A��cdfÇח�Iɲ�%gp���Lȥŋ㚷�#��%BQd�>��y���M#f�R��l�ۦZ�`�o��6�y���s��_~����ޥ�L6lX�Hk6x��n�!��R���o3��&�/APT1����H��@�S3�}}������ "���UbNS��g��'��?j+�bʔ)��J;H�}��{��{��j���.j�mZYe_�`ߤI��immŎGyDQZ����իW�+3�>qm�}�G�V-Ǵ$��t� ��)�O>�d���O��~�u.e��*a��� <@v�n���i����fe����K.�jQ���}�?�8p��'�$�� ���5kրo{{{�tcǎp��>��SO=����7A���?��Fߧ�P`�l�2%�0����� +��Y�A#������o� �q�3�R�-Y�[5P!�x��RB��J�g�:ꨙ3gz�wgE�oӎl��џ�O��z�>��4��/=l0:2Pf�i$��.3P+��W�(��K/i��1�'8�ZO1͠���Q��j�D��CH:Y�vv���˳�A̙3G�H���ō &8ց��j��L�ݽ}۱���H����>�ӟ������U���}��s�f̘q�glذ�Q�� 2u�ԙ�v�w4772����חN�s���OL���s7:8�� ٙ6m�u�]�r&���쫩��"�jkkM���e�… ��Li&O��s2��Ԕ;{�l�� y?~�}!��?L�J�D����H��x@4>�<"p��wko�}���9�E�-d� ��}��d�f������^( h�@ �X�]�d�ɥ�J��t�Rį��� C�e@��!F_�Lb�Z��_[[����/�-���wc�l�k� ����JDc//F1c�!v�ڱ����<� FAĊ���DT��ć�P���`d�=��׈�� !0�fͷ��^�,�Ģ���Z����s�G�� ���]�p�ҍ7����e�%|rp\B�X�r% ����� =� ���ꖍ6��Z������絲�R�����@�e�I�DqλKZ2KE�����*��UV� /^�\ +�KHH@�9u�xزe �x��7�ɏ���ʴ�4u�h�Ծ��D��C777wtt�� +�:+��[�;�Ɩ�k>4�033��C�𣠔U�%�&�|����9���D��k����?���͞=���KTB@M� tss�çzlܸ��SX[[S@� �@�U�vKqsvv�4��CE��:99QI�8==��Aj)�?�rPP�켨�d�)���c\�@��pX�j���+WxKh\�t�M����% ..N���ϏZ��Q6Q4: Q$1����[qwwVȇ���666�,yj/�C GdiO�g�`�rك��7� :�9s�x{{C��A����>D|�(� ��(Ҙ��D��cU;G5h�`����FD�U�A"^5�s��U� �޽���k��D�����v��� sB������+Ic��נAC�߯Rn��ו����K�O2��+��J�Z�7Ɛ��w�������6l�0SS��ݻ���o�ܹ腘�,/\��t�֭o߾�/~�� �T��Ç��)lll��ә�3gޮ^�*�B��M��i�&�B ����Ϗ7�w���033�9s&�E����>d�A�O�>h��Ps��Մ@mW�D)&O�<{�����ځچ ��p����8�,--�!C��dgg ����-4����������ѣGLI={�:t���988��1c�<|�P���͛{��a�O����A��8qB�\TT���;jԨ^�z�tYYY]�~]�<�dj���/^�#,� @4�y�A� +�Y�~}A��� ������|�B�p�d �h�������u:���7a��1�'::z��t��DDD��ߟ>�֭[��0HKKsrr"��O�fP�4 �.]��<�{�N� s�!� �?^��:t�BFw)��D�GV_�|)��o�*'N��*k��%���燁�}4��@X�]T�Bw��i_euQR*��3�PRI�:dddL�2eҤI����qj˖-�5:u���H�P��;wv��!..Nh߲e��k@5F�DAF���tp( } +��9���355 �M�N���?���F$i۷o�������,��T�1"%%EL �c�����###�I��cǎ���;m/�=��F���h�… srr����0�������~u��^�xQ8���\��h�"�ݿ_|Μ9êm۶�6�Gk9v�X�[6ҴO����������{�����} �CZ�M�6�VK��LLL��H����nvvv}���� ��ƶ�����Pf+���^ =�:u*�=88�s����H��?��<�Ʒ��a���l���є(�e�1Ȓ%E�4�>vMB���`�R��U"��eb���L�%)I)��T�sz?���;α��1r�8�t/�}]���y��Ib��8^^^db�����ᝯ��?`�ɓ'۴i &���1e�-��j׮mfff^-�ΝK�L�<��� R2h���tc�w��%�ْ�Q@�2e +�X�!�R��W\\��#Fdgg\���!U�����C�Q��%�'%U#�-�(�����)�}?�Ǐ��O�N�������þ�7�iaP5j�4i4�`�`�����O��]�vM��2SIO� :vjj����� �q�999������s�j֬��͛�}���g\�r��u��ڵkU�l��%���{˂���cc�� X9��|�)n�z�N�����NL�U+44�l���Cɼ��3g֯_+����S�TNNN4(����{bfa�`���e˖����f��>)�)]�Xv-Cu(Y���z�捻���tJ'�q�z��]�p�NK�nРA߾}�5ÇǼDDDx �-^�����aC���i��cǎ��۶mtؿ�0�B�@�Ѱ���;C���RXX8o���� )**��ԩ�1��ݻ��PF@�rݺu���ŋ����`p��尀Peee?�>��4P���4g�X rss �G�������`%>>�3G�)����gΜ�tm۶�Wp�,&�Ν;A��ݻŽ�>}���+���qÇ�(�>)�?A���>�]MW�'K�}QtŸ���K��3�� ���`��U�V����t�tډ'�\ӦM{�� �+W����>N(!!*��A�A(����ݡ l + ���g�Y �,YB��N�<�ގ��Pl���Æ Sr` ^  ��s#�������>V�Z�E�={�����ƍ۴i�(����4������L�ݿk���h4�4�#% +OJJ"�����.Z����+�x���HIIQ(1�)�b�V�^����[�[�n�K@\�t�'FEE)x�\/�ܺu�{�(���~{� tj�&�P}��*D��ˢ�)�#H�Uշ��a\�q���� �s�X��w�R�PUݱ 9H�}�����a���r��.��AЄ�X�?H��ǯq9,f�8ƻ�b�U(ɨ�j����,�3NJ �TU�Tp�}RRRRR�>�CyEJn��I�IIIII�9�i��E�����������ٳg}||ƌ�|�����?~|�'�=z���ʂ����k„ YYY%%%�v�;v�ݻw�PEEE���s��U�T� ����Y�F����z����T�V�lRR��ɓ�YOOϠ��ׯ_����V�XAV���رckbcc�8k�ƍ䙛�+F�z��������݉\XX(�"##�����<�x�b���ʹS�N���(..6^����t޼y2jԨ��xqnFF�ڵk��Y�f����������~�tZ�&�P�� +K�-�WYYI[�ׯ_ӦM���{��ѹs� ���=z�H�g���u��e��ё;v433۰at{��i�&M�koo��I���ϟ?��݌3jժ��䔖�&H������B!�1t�PSSS��)�ڵ���:v�9��V�Z�lْ��={v�֭y��<'&&VU���Ɓ?///�>}��}�.O�)=�b�>g��'n����޹s�u��i˖-)))�����%K�@7 �����G�>w���7n�4h�y�`ֆ��I1�F���A��2_�����^�J���lHS~~�`FI����޽{۷o�lٲ���*-\�P��> ������@@����2k�>�����F�Ň����a`�R�`��bs��V����#Gr��n߾�o�>\�b9����~k� ]������F�GX;��yA5�U�:��,|||j׮Bs�_������EXXV��i�&�],8x� P":��I|���A���_v!Q�q.+�,�2B��(/"������LBʐ "��"I�0oZ� ���ěR�3J �P*B�2�>`Z��ʴE�fgFw�����m� ~�����y�ǟq�̙3�!���>"g{{�yC?� +F6�C��|<����q�F�I�&���ϟ��`�>��dR-�@�^���P���糳����n���̤�0�#u:�Nہ��n��d9US����? ���S]���j��e����}FOOϪU�P��*..�`�`�qQ�|�'N�3�q�ҥK>��(w�� ��oΜ9�n�Z�z��%K;_�>��P���>z�,6l@<��o���k׮�5k����I�����qb^^_1晄q�"�%$$dgg�p�͛7����:���j�譸��scUFFFee�����?%�lß|��}���ѽ�Ap+W��D��6D�‰�B�}܇DN�:��n޼y���E����">���\�+FHII)//��i\��������}��񥥥T�Pǎ�?>�2��_ 544\�vm߾}��p�k�t_ss3j�A�֭{��Ѷmی��^�����g��>Vmݺ��Խ|��qhUU�øoڴi�?�b�C�Y���;v,_�<::ן;w�X���B!�����1.hkk� 566���=��zq_DDw~ll,/g̘�i�&�244dܗ���_�Fa�@��0�+**b���\&��>� +�%p�޽����Q㾨���h� G���� ��>R�����ӧOߵk4�{��&v����y&6�����3u���N�>���G�L �>|w��m�@z{{ms��.\�011��ݻ�~B��@^%%%&L@R�{��E�ZF:\˸o��ل,|���C����or�q���ۃ����&���p ���t�ܹs/]�D �/����q��{������g]]��} ��Ν;��ԩS����e� ^VV���Lu�ɢ��X�/++�����>|���fU��Y��S�Ȼw�ȳ��)ZPP������P!��\���ٸ`���\�d�@l�N�n7�a%c�}�ׯohhh����m��g&�x�ćj�s��ŋ���C���)!!���»�pw�ʕ��x�},'��ٳ�r����P\_XX�%�G�����|uttt^^ޛ7op� >())!��9s&--�3a;;��� +����pB!��������3�����-�S�Q�s�L ]���Ӷ�a������!�����k>a,�h�>�{���S��>t���N'�"���?><E��� �IQ��:�%��'Ox���c�޾}K����<�Q��|̴}8��Zj�������y`u���v�������B!�B!�B!�B!�B!�B!�B!�B!�B!�B!�B���� [p ~ +endstream endobj 148 0 obj <>/Filter/FlateDecode/Height 479/Intent/RelativeColorimetric/Length 302/Name/X/Subtype/Image/Type/XObject/Width 596>>stream +H���1 ��g ���X� +endstream endobj 135 0 obj <> endobj 43 0 obj [42 0 R 41 0 R 16 0 R 40 0 R 15 0 R 39 0 R 38 0 R 37 0 R 36 0 R 35 0 R 34 0 R 33 0 R 32 0 R 31 0 R 30 0 R 29 0 R 28 0 R 27 0 R 26 0 R 25 0 R 24 0 R 23 0 R 22 0 R 21 0 R 20 0 R 19 0 R 18 0 R 14 0 R 17 0 R] endobj 149 0 obj <> endobj xref +0 150 +0000000000 65535 f +0000000016 00000 n +0000000544 00000 n +0000034243 00000 n +0000000000 00000 f +0000237117 00000 n +0000237858 00000 n +0000238582 00000 n +0000239320 00000 n +0000240053 00000 n +0000034500 00000 n +0000035232 00000 n +0000035964 00000 n +0000036696 00000 n +0000040633 00000 n +0000041923 00000 n +0000042067 00000 n +0000040556 00000 n +0000041530 00000 n +0000042287 00000 n +0000042379 00000 n +0000042475 00000 n +0000042556 00000 n +0000042652 00000 n +0000042754 00000 n +0000042838 00000 n +0000040705 00000 n +0000040787 00000 n +0000040866 00000 n +0000040945 00000 n +0000041030 00000 n +0000041107 00000 n +0000041208 00000 n +0000041290 00000 n +0000041376 00000 n +0000041451 00000 n +0000041607 00000 n +0000041688 00000 n +0000041778 00000 n +0000041850 00000 n +0000041993 00000 n +0000042141 00000 n +0000042215 00000 n +0000331101 00000 n +0000034303 00000 n +0000034392 00000 n +0000037428 00000 n +0000038160 00000 n +0000038891 00000 n +0000236516 00000 n +0000237052 00000 n +0000056853 00000 n +0000056596 00000 n +0000046330 00000 n +0000235603 00000 n +0000236451 00000 n +0000233535 00000 n +0000235538 00000 n +0000228859 00000 n +0000233470 00000 n +0000225287 00000 n +0000228794 00000 n +0000224578 00000 n +0000225222 00000 n +0000039649 00000 n +0000040491 00000 n +0000056709 00000 n +0000046214 00000 n +0000046245 00000 n +0000046098 00000 n +0000046129 00000 n +0000045982 00000 n +0000046013 00000 n +0000045866 00000 n +0000045897 00000 n +0000045750 00000 n +0000045781 00000 n +0000045634 00000 n +0000045665 00000 n +0000045518 00000 n +0000045549 00000 n +0000045402 00000 n +0000045433 00000 n +0000045286 00000 n +0000045317 00000 n +0000045170 00000 n +0000045201 00000 n +0000045054 00000 n +0000045085 00000 n +0000044938 00000 n +0000044969 00000 n +0000044822 00000 n +0000044853 00000 n +0000044706 00000 n +0000044737 00000 n +0000044590 00000 n +0000044621 00000 n +0000044474 00000 n +0000044505 00000 n +0000044357 00000 n +0000044388 00000 n +0000044239 00000 n +0000044271 00000 n +0000044121 00000 n +0000044153 00000 n +0000044003 00000 n +0000044035 00000 n +0000043885 00000 n +0000043917 00000 n +0000043767 00000 n +0000043799 00000 n +0000043649 00000 n +0000043681 00000 n +0000043531 00000 n +0000043563 00000 n +0000043413 00000 n +0000043445 00000 n +0000043295 00000 n +0000043327 00000 n +0000043177 00000 n +0000043209 00000 n +0000043059 00000 n +0000043091 00000 n +0000042941 00000 n +0000042973 00000 n +0000046658 00000 n +0000046934 00000 n +0000056928 00000 n +0000057152 00000 n +0000059510 00000 n +0000067393 00000 n +0000132983 00000 n +0000198573 00000 n +0000244615 00000 n +0000244879 00000 n +0000330988 00000 n +0000244945 00000 n +0000243657 00000 n +0000243890 00000 n +0000244492 00000 n +0000243956 00000 n +0000243131 00000 n +0000243591 00000 n +0000241680 00000 n +0000243065 00000 n +0000240786 00000 n +0000241614 00000 n +0000244428 00000 n +0000330435 00000 n +0000331322 00000 n +trailer +<]>> +startxref +331524 +%%EOF diff --git a/module-logos/open-access-research-papers.png b/module-logos/open-access-research-papers.png new file mode 100644 index 0000000..76b83e3 Binary files /dev/null and b/module-logos/open-access-research-papers.png differ diff --git a/module-logos/open-advocacy.png b/module-logos/open-advocacy.png new file mode 100644 index 0000000..0982ae7 Binary files /dev/null and b/module-logos/open-advocacy.png differ diff --git a/module-logos/open-collaboration.png b/module-logos/open-collaboration.png new file mode 100644 index 0000000..da53053 Binary files /dev/null and b/module-logos/open-collaboration.png differ diff --git a/module-logos/open-educational-resources.png b/module-logos/open-educational-resources.png new file mode 100644 index 0000000..3c5e446 Binary files /dev/null and b/module-logos/open-educational-resources.png differ diff --git a/module-logos/open-evaluation.png b/module-logos/open-evaluation.png new file mode 100644 index 0000000..badbbaf Binary files /dev/null and b/module-logos/open-evaluation.png differ diff --git a/module-logos/open-principles.png b/module-logos/open-principles.png new file mode 100644 index 0000000..46bc360 Binary files /dev/null and b/module-logos/open-principles.png differ diff --git a/module-logos/open-research-data.png b/module-logos/open-research-data.png new file mode 100644 index 0000000..08c5513 Binary files /dev/null and b/module-logos/open-research-data.png differ diff --git a/module-logos/open-research-software.png b/module-logos/open-research-software.png new file mode 100644 index 0000000..31e7f4b Binary files /dev/null and b/module-logos/open-research-software.png differ diff --git a/module-logos/open-research-software_white.png b/module-logos/open-research-software_white.png new file mode 100644 index 0000000..9d80bdb Binary files /dev/null and b/module-logos/open-research-software_white.png differ diff --git a/module-logos/public-engagement-w-science.png b/module-logos/public-engagement-w-science.png new file mode 100644 index 0000000..e3e4195 Binary files /dev/null and b/module-logos/public-engagement-w-science.png differ diff --git a/module-logos/readme.md b/module-logos/readme.md new file mode 100644 index 0000000..9c41f14 --- /dev/null +++ b/module-logos/readme.md @@ -0,0 +1,8 @@ +# OS MOOC Module Logos +* Designed by: Mike Morrison +* Email: mikeamorrison@gmail.com +* Twitter: @mikemorrison + +_Please feel free to contact me with questions/alternate version requests!_ + +## Description of each logo file... diff --git a/module-logos/reproducible-research.png b/module-logos/reproducible-research.png new file mode 100644 index 0000000..aa27df6 Binary files /dev/null and b/module-logos/reproducible-research.png differ diff --git a/module-logos/svg/Readme.MD b/module-logos/svg/Readme.MD new file mode 100644 index 0000000..d5cff78 --- /dev/null +++ b/module-logos/svg/Readme.MD @@ -0,0 +1,6 @@ +# SVG Module Logos +` +by: Mike Morrison | @mikemorrison | mikeamorrison@gmail.com +` + +Universal vector format for use in Inkscape, etc. diff --git a/module-logos/svg/_os-mooc-module-logos.svg b/module-logos/svg/_os-mooc-module-logos.svg new file mode 100644 index 0000000..9cc2439 --- /dev/null +++ b/module-logos/svg/_os-mooc-module-logos.svg @@ -0,0 +1,392 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Opensciencemooc + + + + + + Openprinciples + + + + + + + + + + + + + + + + + + + + + + + Opencollaboration + + + + + + + + + + + + + + COLLABORATION + + + REPRODUCIBLE RESEARCH + + + + & DATA ANALYSIS + + + + + OPENRESEARCH DATA + + + + + + + + + openresearch software + + + + & open source + + + + + + + open ACCESSTO RESEARCH PAPERS + + + + + + + + + + + + + + openevaluation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + PUBLIC ENGAGEMENTWITH SCIENCE + + + + + + + + + open educational resources + + + + + + + + + + open ADVOCACY + + + + + + + + + + + + + + + + + OPEN + + + + + + + + + + + diff --git a/module-logos/svg/open-access-research-papers.svg b/module-logos/svg/open-access-research-papers.svg new file mode 100644 index 0000000..7dab838 --- /dev/null +++ b/module-logos/svg/open-access-research-papers.svg @@ -0,0 +1 @@ +open-access-research-papersOPEN ACCESSTO RESEARCH PAPERS \ No newline at end of file diff --git a/module-logos/svg/open-advocacy.svg b/module-logos/svg/open-advocacy.svg new file mode 100644 index 0000000..bd43621 --- /dev/null +++ b/module-logos/svg/open-advocacy.svg @@ -0,0 +1 @@ +open-advocacyOPEN ADVOCACY \ No newline at end of file diff --git a/module-logos/svg/open-collaboration.svg b/module-logos/svg/open-collaboration.svg new file mode 100644 index 0000000..e9e3476 --- /dev/null +++ b/module-logos/svg/open-collaboration.svg @@ -0,0 +1 @@ +open-collaborationOPENCOLLABORATIONCOLLABORATION \ No newline at end of file diff --git a/module-logos/svg/open-educational-resources.svg b/module-logos/svg/open-educational-resources.svg new file mode 100644 index 0000000..bbceb4d --- /dev/null +++ b/module-logos/svg/open-educational-resources.svg @@ -0,0 +1 @@ +open-educational-resourcesOPEN EDUCATIONAL RESOURCES \ No newline at end of file diff --git a/module-logos/svg/open-evaluation.svg b/module-logos/svg/open-evaluation.svg new file mode 100644 index 0000000..0363478 --- /dev/null +++ b/module-logos/svg/open-evaluation.svg @@ -0,0 +1 @@ +open-evaluationOPENEVALUATION \ No newline at end of file diff --git a/module-logos/svg/open-principles.svg b/module-logos/svg/open-principles.svg new file mode 100644 index 0000000..42458f8 --- /dev/null +++ b/module-logos/svg/open-principles.svg @@ -0,0 +1 @@ +open-principlesOPENPRINCIPLES \ No newline at end of file diff --git a/module-logos/svg/open-research-data.svg b/module-logos/svg/open-research-data.svg new file mode 100644 index 0000000..83f56f0 --- /dev/null +++ b/module-logos/svg/open-research-data.svg @@ -0,0 +1 @@ +open-research-dataOPENRESEARCH DATA \ No newline at end of file diff --git a/module-logos/svg/open-research-software.svg b/module-logos/svg/open-research-software.svg new file mode 100644 index 0000000..e1419e2 --- /dev/null +++ b/module-logos/svg/open-research-software.svg @@ -0,0 +1 @@ +open-research-softwareOPENRESEARCH SOFTWARE& OPEN SOURCE \ No newline at end of file diff --git a/module-logos/svg/public-engagement-w-science.svg b/module-logos/svg/public-engagement-w-science.svg new file mode 100644 index 0000000..c1ed61a --- /dev/null +++ b/module-logos/svg/public-engagement-w-science.svg @@ -0,0 +1 @@ +public-engagement-w-sciencePUBLIC ENGAGEMENTWITH SCIENCE \ No newline at end of file diff --git a/module-logos/svg/reproducible-research.svg b/module-logos/svg/reproducible-research.svg new file mode 100644 index 0000000..7aa0b91 --- /dev/null +++ b/module-logos/svg/reproducible-research.svg @@ -0,0 +1 @@ +reproducible-researchREPRODUCIBLE RESEARCH & DATA ANALYSIS \ No newline at end of file diff --git a/objectives2020.md b/objectives2020.md new file mode 100644 index 0000000..b5ff377 --- /dev/null +++ b/objectives2020.md @@ -0,0 +1,19 @@ +# Objectives for the OSMOOC in 2020 + +## courses +- find and activate new platform // add Module 1 and 5 +- finalize Module 6 (Open Access) +- creating Module 4 (Open Data) +- find volunteers to lead the creation on all other modules + +## community +- partnership building across other OS-MOOCs +- local OSMOOC chapters across the world +- collaborate with other OS initiatives & orgs +- develop strategic partnerships with partners + +## sustainability +- reassess Roadmap (see main page) +- design OSMOOC offline workshops (raise fee to cover expenses) +- crowdfunding via OpenCollective - https://opencollective.com/ +- sponsorship packages for the final design & marketing of each module diff --git a/production_files/Open Science MOOC.md b/production_files/Open Science MOOC.md new file mode 100644 index 0000000..d3cdf86 --- /dev/null +++ b/production_files/Open Science MOOC.md @@ -0,0 +1,2158 @@ +Open Science MOOC + +* Status: **In preparation **(scroll down for module structure/resources - it helps if you select to view the document outline). + +* If you wish to contribute as a volunteer on any aspect of this project, please see the open list of [Potential Contributors](#heading=h.bds5ysnp6zsj). + +* [Primary website](https://opensciencemooc.eu/), including details of the Steering Committee, Code of Conduct, and Terms of Service, as well as all resources mentioned here. + +* All development content is currently managed via [Google Drive](https://drive.google.com/drive/folders/0B8feXr2D75GWTnduS3M4ZlRSZ3M). This includes copies of all papers/reports referenced, which are themselves checked to be Open Access for re-use. + +* [GitHub](https://github.com/OpenScienceMOOC) organisation (one repo per module). Module development will primarily be here in the next phase. + +* Twitter: [@OpenSci_MOOC](https://twitter.com/OpenSci_MOOC). + +* [Etherpad](https://public.etherpad-mozilla.org/p/OpenSci_MOOC) for meeting notes. + +Internal communications for this project are primarily via Slack. Please use this [link](https://openmooc-ers-slackin.herokuapp.com/) to join or [contact us](mailto:jon.tennant.2@gmail.com) for details. + +![image alt text](image_0.jpg) + +# Rationale + +Research is getting a global makeover, in part thanks to the power of the internet and the tools it provides for us, and in part due to a growing call for accountability (e.g., reproducibility and data provenance) in research. Global policies are emerging at different levels that include some aspect of ‘Open Research’, ‘Open Scholarship’, or ‘Open Science’, and inclusive of all research disciplines. But our universities are often letting us down, and they are not teaching us the knowledge, tools and skills we need to do research effectively in the 21st century. + +Open Science is about increased rigour, accountability, reproducibility for research. It is based on the principles of inclusion, fairness, equity, and sharing. Open Science can be viewed as research simply done properly, and it extends across the Life and Physical Sciences, Engineering, and Mathematics, to Social Science and Humanities. + +This MOOC is designed to help equip students and researchers with the skills they need to excel in a modern research environment. It brings together the efforts and resources of hundreds of researchers and practitioners who have all dedicated their time and experience to create a community platform to help propel research forward. + +The content of this MOOC is distilled into 10 core modules that comprise some of the most important current topics in Open Science. Each module will comprise a complete range of resources including videos, research articles, dummy datasets and code, as well as ‘homework’ tasks to complete as individuals. The MOOC will be hosted through OpenEdX, an open source provider. We expect that in the future different systems of certification will be developed, including completion badges. We also intend to build a forum for the open discussion of the MOOC and any relevant topics. + +**_Disclaimer: _**We, the contributors, are fully aware that there is no magic "one size fits all solution" when it comes to implementing Open Science philosophies and best practices, especially when covering all research disciplines. However, we also believe that we should not limit ourselves from the onset, just because pragmatic solutions may not exist today for certain disciplines. We aim to set a highly inclusive standard, fully accepting the risk that for some disciplines this strategy may not be fully appropriate. Through that failure, we hope that you, the course users, will join us as course contributors and help us co-create bespoke solutions to your discipline based on principles of transparency, provenance, reproducibility and reuse of knowledge. + + ![image alt text](image_1.png) + +**_Conceptual model of the Open Science MOOC (_****_to be redrawn to match final 10 chapters_****_)_** + +# Proposed Structure + +A 10-module course with each one focussing on making each step or aspects of the research lifecycle more transparent and accountable, collaborative and efficient, and reusable and easier to build upon. + +An introduction to open principles as an initial course, with several modules (chapters) to follow with more detailed discussions of different aspects of the open science movement. Each module is a stepping stone to a more transparent and accountable, collaborative and efficient, re-usable and easier research lifecycle. + +[[TOC]] + +# How to contribute to this document + +* This document can be freely edited by anyone. + +* Please feel free to add any information or links that you see fit. + +* If you are unsure about something, please leave a comment. + +* Please be courteous, and do not delete the contributions of others. + +* For any images, data, or other source material, please add to the relevant folders on [Google Drive](https://drive.google.com/open?id=0B8feXr2D75GWTnduS3M4ZlRSZ3M). + +* Please do not share any copyrighted content or content which you do not have permission to re-use. + +* Please be as explicit and detailed as possible regarding source material. + +* Please avoid any large-scale changes to the structure at the present. + +### How to volunteer for developing the MOOC itself + +[FULL LIST MAINTAINED HERE](https://docs.google.com/spreadsheets/d/1FEbnxBkmIQFDEViTgZDn0AJ0MSK6eDSBEgZuCkW_B5s/edit?usp=sharing) + +Please note that anyone is free to add themselves to this list and contribute as they see fit. Useful data: + +* Name + +* Affiliation (e.g., University, independent, NGO etc.) + +* Twitter handle + +* Email address + +* Country of work + +# Modules + +## 1: Open Principles + +**_Rationale (_****_3 lines max_****_)_**: + +*To innovate in a field frequently implies moving against prevailing trends and cultural inertia. Open Science is no different. No matter how convinced you are, you will come across resistance from peers and colleagues, and the best defence is strong personal conviction that what you are doing may not be perfect now, but is the right decision in the long run. This module will introduce the guiding principles of the ‘open movement’, the different actors involved, and the impact that they are having.* + +**_Learning Objectives (1-3 specific)_**: + +*LO1a: Understand the ethical, **legal**, social, economic, and research impact arguments for and against Open Science (knowledge).* + +*LO1b: Set up a personal profile for defining your impact: measure the social and academic attention on the full range of research processes and outputs** (tasks).* + +### Key components: + +* What is "Open Science", and why should we care. + +* History of Open Science and Open Cultures. + +* Differences and commonalities in understanding and interpretation of the term. + +* Communities and diversity, inclusivity, fairness, equity, social behaviour, accountability, ethics and responsibility. + +* Open Science on a global scale. + +* How Open Science influences your career now and the future of research evaluation. + +* Open licensing, copyright, and speaking ‘legalise’. + +* The different dimensions of Open Science (e.g., Open Access, Open Data, Open Peer Review). + +* What are some of the barriers to Open Science, and why. + +* Open science and reproducible research: 2 sides of the same coin? + +* Open science in daily work: design your workflow with sharing in mind and invest time early. + +### Who to involve: + +* Individuals: Erin McKiernan, Michael Eisen, Katja Mayer, Steven Hill, Cameron Neylon, Peter Kraker, Bianca Kramer, Jeroen Bosman, Ahmed Ogunlaja, Stephanie Wright. + +* Organisations: [Right to Research Coalition](http://righttoresearch.org/) (R2RC) and the [Scholarly Publishing and Academic Resources Coalition](https://sparcopen.org/) (SPARC), including [SPARC EU](https://sparceurope.org/), [OpenCon](http://www.opencon2017.org/) community and regional groups, [HEFCE](http://www.hefce.ac.uk/), [NESTA](https://www.nesta.org.uk/), [Mozilla Science Lab](https://science.mozilla.org/) (and the Open Leadership Cohort), [Global Open Science Hardware (GOScH)](http://openhardware.science) Community. [Creative Commons](https://creativecommons.org/) [Center for Open Science](https://cos.io/) (plus their [ambassadors](https://cos.io/our-communities/become-ambassador/) cohort), [OCSD](https://ocsdnet.org/) network, ORCID. + +* Other: [The Force11 Scholarly Commons Working Group](https://www.force11.org/group/scholarly-commons-working-group). + +### Key resources: + +**Tools** + +* [The Open Science Training Initiative](http://www.opensciencetraining.com/content.php). + +* [Why Open Research](http://whyopenresearch.org/), Erin McKiernan. + +* [Transparent and Open Social Science MOOC](http://www.bitss.org/events/mooc-transparent-and-open-social-science/), Berkeley Initiative for Transparency in the Social Sciences (BITSS). + +* [Rainbow of Open Science practices](https://zenodo.org/record/1147025#.Wnib8yXwaM8) (Kramer and Bosman, 2018). + +* [Creative Commons license chooser](https://creativecommons.org/choose/). + + * [Open Content](https://meta.wikimedia.org/wiki/Open_Content_-_A_Practical_Guide_to_Using_Creative_Commons_Licences/The_Creative_Commons_licencing_scheme) - A practical guide to using Creative Commons licenses/the Creative Commons licensing scheme. + +* [Open Research Glossary](http://www.righttoresearch.org/resources/OpenResearchGlossary/index.shtml), Right to Research Coalition. + +* [Scholarly Communications super-collection](https://www.scienceopen.com/search#collection/69988c7e-1855-4007-ba94-caa4c4638b1f), ScienceOpen. + +**Research Articles and Reports** + +* [Open science is a research accelerator](https://www.nature.com/articles/nchem.1149) (Woelfle et al., 2011). + +* [ORCID: A system to uniquely identify researchers](http://onlinelibrary.wiley.com/doi/10.1087/20120404/epdf) (Haak et al., 2012). + +* [The Conundrum of Sharing Research Data](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2073876) (Borgman, 2012) + +* [Open Science: The Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing](https://link.springer.com/book/10.1007/978-3-319-00026-8) (Bartling and Friesike, 2014). + +* [Open Science: one term, five schools of thought](https://link.springer.com/chapter/10.1007%2F978-3-319-00026-8_2) (Fecher and Friesike, 2014). + +* [From Open Science to Open Innovation](https://www.fosteropenscience.eu/sites/default/files/pdf/1798.pdf) (Chesbrough, 2015). + +* [Winning Research Grants with Open Science](http://doi.org/10.5281/zenodo.12247) (Grigorov et al., 2015). + +* [Promoting transparency in social science research](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4103621/pdf/nihms605501.pdf) (Miguel et al., 2014). + +* [Promoting an open research culture](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4550299/pdf/nihms-714651.pdf) (Nosek et al., 2015). + +* [When will ‘open science’ become simply ‘science’?](https://genomebiology.biomedcentral.com/articles/10.1186/s13059-015-0669-2) (Watson, 2015). + +* [How does one "open" science? Questions of value in biological research](http://journals.sagepub.com/doi/10.1177/0162243916672071) (Levin and Leonelli, 2016). + +* [Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights](https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/2016_0504_data_discrimination.pdf) (White House, 2016). + +* [Providing researchers with the skills and competencies they need to practice Open Science: Open Science Skills Working Group Report](https://ec.europa.eu/research/openscience/pdf/os_skills_wgreport_final.pdf#view=fit&pagemode=none) (European Commission, 2017). + +* [Do you speak open science? Resources and tips to learn the language](https://peerj.com/preprints/2689/) (Masuzzo and Martens, 2017). + +* [Early-career researchers’ perceptions of the prevalence of questionable research practices, potential causes, and Open Science](https://econtent.hogrefe.com/doi/full/10.1027/1864-9335/a000324) (Stürmer et al., 2017). + +* [Making Science Transparent By Default; Introducing the TOP Statement](https://osf.io/sm78t/) (Aalbersberg et al., 2018). + +* [Defining success in Open Science](https://mniopenresearch.org/articles/2-2/v1) (Ali-Khan et al., 2018). + +* [Open Science is liberating and can foster creativity](https://osf.io/edhym/) (Frankenhuis and Nettle, 2018). + +* [Open Educational Science](https://osf.io/preprints/socarxiv/d9bme) (van der See and Reich, 2018). + +**Key posts** + +* [Anatomy of an Open Science Paper](https://intermolecular.wordpress.com/2014/12/15/anatomy-of-an-open-science-paper/), Matt Todd. + +* [Changing hiring practices towards research transparency: The first open science statement in a professorship advertisement](http://www.nicebread.de/open-science-hiring-practices/), Felix Schoenbrodt. + +* The [SV-POW Tutorials](https://svpow.com/tutorials/) include a helpful [definition of Open Access](https://svpow.com/2012/11/15/tutorial-19a-open-access-definitions-and-clarifications-part-1-what-actually-is-open-access/) and clarifications on licensing, copyright and related topics (Tutorial 19). + +* [Science 2.0 repositories: Time for a change in scholarly communication](http://www.dlib.org/dlib/january15/assante/01assante.html) (Assante et al., 2015). + +* [Power and inequality in Open Science discourses](https://medium.com/@denalbz/power-and-inequality-in-open-science-discourses-9d425b0c2b63), Denisse Albornoz. + +* [Clash of cultures: Why all science isn’t open science](https://danielskatzblog.wordpress.com/2016/10/25/clash-of-cultures-why-all-science-isnt-open-science/), Daniel Katz. + +* [Preliminary Findings: Rent Seeking by Elsevier](http://knowledgegap.org/index.php/sub-projects/rent-seeking-and-financialization-of-the-academic-publishing-industry/preliminary-findings/), Alejandro Posada. + +* [Reassessing the ‘Digital Commons’](http://ivory.idyll.org/blog/2017-digital-commons-funding.html), Marianne Corvellec and Jeanne Corvellec. + +* [A list of publicly available grant proposals in the biological sciences](https://jabberwocky.weecology.org/2012/08/10/a-list-of-publicly-available-grant-proposals-in-the-biological-sciences/) (Jabberwocky). + +* [Open Science and its Discontents](http://ronininstitute.org/open-science-and-its-discontents/) Alex Lancaster + +**Other** + +* [The Open Science Manifesto](https://ocsdnet.org/manifesto/open-science-manifesto/), OCSD Net. + +* [The Open Definition](http://opendefinition.org), The Open Knowledge Foundation. + +* [Vienna Principles](http://viennaprinciples.org/): A vision for scholarly communication. + +* [Summer School Open Science course](https://www.utrechtsummerschool.nl/courses/science/open-science-and-scholarship-changing-your-research-workflow), Utrecht University. + +* [A short lecture on Open Licensing](https://figshare.com/articles/A_short_lecture_on_Open_Licensing/4516892/1) (Lorena, 2017; presenter notes also on [SpeakerDeck](https://speakerdeck.com/labarba/a-short-lecture-on-open-licensing). [Bonus interview](http://rtalbert.org/blog/2015/interview-lorena-barba)). + +* [The Panton Principles of Open Science](https://en.wikipedia.org/wiki/Panton_Principles). + +* [List of advocacy organisations for Open Access](http://oad.simmons.edu/oadwiki/Advocacy_organizations_for_OA), Open Access Directory. + +* [OpenCitations](http://opencitations.net/) and the [Initiative for Open Citations](https://i4oc.org/) (I4OC). + +* [The Knowledge Gap: Geopolitics of Academic Production](http://knowledgegap.org/). + +* [The Open Archives Initiative](https://www.openarchives.org/) - Standards for Web content interoperability. + +* Force11 [Scholarly Commons Working Group charters database](https://docs.google.com/spreadsheets/d/1-aRXFiRg-VL9hpLpxoJqX6-OC-A0R2oCogHfIx52Nug/edit#gid=956616118). + +* [Open Science 101](https://github.com/OKScienceDE/Open_Science_101), Konrad Förstner (CC0). + +* [101 Innovations in Scholarly Communication](https://101innovations.wordpress.com/), Bianca Kramer and Jeroen Bosman ([interactive version](https://bmkramer.databox.me/Public/Wheel_of_Open_Science/)). + +* [Open Science 101](https://github.com/OKScienceDE/Open_Science_101) (OK Science Deutschland). + +* [European Open Science Cloud](http://ec.europa.eu/research/openscience/index.cfm?pg=open-science-cloud) (EOSC). + +* [Open Research 2015](https://courses.p2pu.org/en/courses/3230/open-research-2015/) (P2PU). + + * [The accompanying textbook](https://openresearch.pressbooks.com/) (Pitt et al., 2016). + +### Tasks: + +* Get an [ORCID](orcid.org) account, and fill out your profile. This is a unique identifier for you as a researcher. + +* Get a [Publons](https://publons.com/home/) account, integrate with your new ORCID, and valorise your reviewer effort! + +* Get an [ImpactStory](http://impactstory.org/) account, and integrate with ORCID, showcase your output (not just publications)! + +* Write a summary about Open Access efforts either on your research discipline and/or in your country. If you have a website or blog, post it there. + + * Were the data for this easy to acquire? Which sources did you use? + +* Look at the status of Open Science in your research group or lab. Make a note of who is doing what. What could be improved? + +* Define clearly what Open Science means to you. Have a conversation about it with a colleague. Then, find someone from a different country, and have another conversation about Open Science. + +* Find out the policies are in your department or institute regarding: + + * Career progression and assessment. + + * Publishing and Open Access. + + * Data sharing. + + * Intellectual Property (IP). + +* Identify any disciplinary repositories either for research articles or data. + +### Learning outcomes: + +1. The researcher will be able to describe the ethical, legal, social, economic, and research impact arguments for and against Open Science. + +2. After deciding which platforms/tools/services are most useful for themselves and their community, the researcher will develop a personal profile for showcasing their research profile and outputs. + +3. After reflecting on the status of Open Science within their research group or lab, the researcher will devise concrete ways to locally improve open practices. + +4. Using the guidelines published by their research laboratories, departments, or institutes, researchers will identify the policies for career progression and assessment, publishing and open access, data sharing, and intellectual property. + +5. Researchers will collaborate with colleagues and international peers to develop a shared definition of Open Science. + +## 2: Open Collaboration + +**_Rationale (_****_3 lines max_****_)_**: + +*Research is becoming an increasingly collaborative process, driven by a combination of technical opportunities and more complex research questions. **Virtual Research Environments (VRE) are the way of the future in collaboration across continents, time zones and disciplines. While the definition of a VRE may be up for grabs, they provide powerful examples of high-performing modern research tools. In this module you will develop an understanding of collaborative platforms that work today, and how they can greatly enhance your research workflows.* + +**_Learning Objectives (_****_specific_****_): _** + +*LO2a: Learn what major types of collaborative platforms are available and what the use cases for each might be (knowledge).* + +*LO2b: Be able to use a variety of collaborative research platforms (tasks).* + +### Key components: + +* Principles of collaborative research. + +* Documentation as conversation and collaboration. + +* Version control, organising repositories, and project management ([GitHub](http://github.com/), [Git](https://git-scm.com/), [Zenodo](https://zenodo.org/)). + +* Virtual Research Environments (VRE) on the horizon (e.g., EU projects on VREs). + +* Website and content management. + +* Collaborative writing platforms. + +* Collective annotation services. + +* Community spaces and communication tools. + +### Who to involve: + +* Individuals: Andy Byers, Anna Krystalli, Julien Colomb, Rutger Vos. + +* Organisations: Center for Open Science (COS), Overleaf, PaperHive, Hypothesis, Authorea, protocols.io (Anjuli Manche), [European Grid Infrastructure](https://www.egi.eu/) ([Tiziana Ferrari](mailto:tiziana.ferrari@egi.eu)), [Research Ideas and Outcomes](https://riojournal.com/) (RIO). + +* Other: EU projects on VREs: [BlueBridgeVRE](https://www.bluebridge-vres.eu/news/new-bluebridge-vre-supporting-scientific-training-stock-assessment) (Donatella Castelli), [VRE4EIC](http://www.vre4eic.eu/), H2020 EVEREST, + +### Key resources: + +**Tools** + +* [Mozilla Science study group lessons](https://github.com/mozillascience/studyGroupLessons/issues). + + * [Working Open Workshop](https://github.com/mozillascience/working-open-workshop), Mozilla Science Lab. + +* [Overleaf](https://www.overleaf.com/), [Authorea](https://www.authorea.com/product), [PaperHive](https://paperhive.org/), [Figshare](https://figshare.com/), [ScienceOpen](https://www.scienceopen.com/), [Hypothes.is](https://web.hypothes.is/), [Protocols.io](https://www.protocols.io/). + +* [Open Science Framework](https://osf.io/) (OSF), [Center for Open Science](https://cos.io/) (COS). + +* [ScholarlyHub](https://www.scholarlyhub.org/), [Academia.edu](https://www.academia.edu/), [ResearchGate](https://www.researchgate.net/), [Humanities Commons](https://hcommons.org/). + +* Discipline-specific platform examples: + + * [NMR Lipids Project](http://nmrlipids.blogspot.de/): Collaboration platform to find a lipid force field that matches NMR order parameters. [Synapse](http://synapse.org): open source, free to use, goals of enabling open science and collaborative research in real-time, integrates with R/Python via analytic clients, and has advanced data access controls for sharing human subjects data. + + * [HackMD](https://hackmd.io/): can be used to clean up other collaboratively written documents before conversion to markdown files and archived in GitHub. + + * [Leishmaniasis Virtual Laboratory](http://lvl.i3m.upv.es/#home). + + * [The Open Source Malaria project](http://opensourcemalaria.org/). + + * [Benchling](https://benchling.com/). + + * [Tools for open geospatial science course](https://ncsu-geoforall-lab.github.io/open-science-course/) (NCSU). + +* [Google docs](https://docs.google.com/document/u/0/) (and sheets and slides). + +* [CRediT](http://docs.casrai.org/CRediT), defining contributor roles in research outputs (CASRAI). + +* [Quartzy](https://www.quartzy.com/), a lab management platform. + +**Research Articles and Reports** + +* [Virtual research environment collaborative landscape study](https://www.webarchive.org.uk/wayback/archive/20140615234259/http://www.jisc.ac.uk/media/documents/publications/vrelandscapereport.pdf) (Carusi and Reimer, 2010; JISC) + +* [Virtual research environments: An overview and research agenda](https://www.jstage.jst.go.jp/article/dsj/12/0/12_GRDI-013/_article/-char/ja/) (Candela et al., 2013). + +* [Creating and maintaining high-performance collaborative research teams: the importance of diversity and interpersonal skills](http://onlinelibrary.wiley.com/doi/10.1890/130001/full?wol1URL=/doi/10.1890/130001/full®ionCode=DE-BE&identityKey=7b1fdb0f-b092-4ce4-8fe6-14c943f803f6) (Cheruvelil et al., 2014). + +* [Collaborative research and development (R&D) for climate technology transfer and uptake in developing countries: Towards a needs driven approach](http://sro.sussex.ac.uk/49408/1/RD_paper_Ockwell.pdf) (Ockwell et al., 2014). + +* [Multinational teams and diseconomies of scale in collaborative research](http://advances.sciencemag.org/content/1/8/e1500211) (Hsiehchen et al., 2015). + +* [Ten simple rules for establishing international research collaborations](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004311) (de Grijs, 2015). + +* [Collaborative research and the co-production of knowledge for practice: an illustrative case study](https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0383-9) (Heaton et al., 2016). + +* [Current reflections on collaborative and engaged research](http://jrp.icaap.org/index.php/jrp/article/view/518) (Rodrigues et al., 2017). + +**Key Posts** + +* [Six new projects on e-Infrastructure for virtual research environments](https://ec.europa.eu/programmes/horizon2020/en/news/six-new-projects-e-infrastructures-virtual-research-environments), European Commission (2015). + +* [Baby steps for the open-curious](https://practicaldatamanagement.wordpress.com/2014/10/23/baby-steps-for-the-open-curious/), Christie Bahlai. + +**Other** + +* [Friendly Git](https://github.com/KirstieJane/friendly-github-intro)[Hub Intro](https://github.com/KirstieJane/friendly-github-intro), Kirstie Jane. + +* [Open Science ASAP](http://openscienceasap.org/). + +* OSF [YouTube archive](https://www.youtube.com/channel/UCGPlVf8FsQ23BehDLFrQa-g), [OSF webinar](https://osf.io/qpxv2/) series, and [OSF Help Guide](http://help.osf.io/). + +* [Free & Interactive Online Introduction to LaTeX](https://www.overleaf.com/latex/learn/free-online-introduction-to-latex-part-1#.Wo6jbYPwaM9), Overleaf. + +### Tasks: + +* Create a GitHub account, and create your first repo with a license and readme file. Use your new account to login to Zenodo and preserve any GitHub repos. Connect with your ORCID profile too. + + * Push different data formats in GitHub (e.g. CSV, SVG) and see what happens. + + * Find a GitHub repo that a colleague has created. Make changes or comments collaboratively, and inspect the differences. + + * Tag the releases and backup to Zenodo. + +* Create an OSF collaborative environment from data to publication. + + * Connect your OSF project to GitHub. + + * Upload any raw code, images, data, tables to project. + + * Obtain a DOI and ARK identifier for your project. + +* Use PaperHive, PubPeer, or Hypothes.is to comment on (annotate) any research article of your choice. + + * Consider doing this as part of a regular preprint journal club. + +* Create a ScienceOpen/Figshare collection on your favourite research topic. + +* Begin a new article draft using an Overleaf template. Clone this article with GitHub. + +### Learning outcomes: + +1. The researcher will become familiar with the range of options available to you to aid greater collaborative research. + +2. After deciding what works optimally for their workflow, the researcher will be able to use collaborative tools such as GitHub and the Open Science Framework for increased collaboration for the research process, writing/authoring, and sharing your research outputs. + +3. The researcher will be able to collaborate with colleagues to annotate preprints or other published articles, and share this discussion with the original authors and wider research community. + +## 3: Reproducible Research and Data Analysis + +**_Rationale (_****_3 lines max_****_)_**: + +*Reproducible research is at the heart of science.** There has been an increased need and willingness to open and share research from the data **collection **right through to the interpretations of results. This has come with its own set of challenges, which include designing workflows that can be adopted by collaborators in a way that does not compromise the integrity of their contribution. This module will introduce the necessary tools required for transparent reporting which is reproducible and readable.* + +**_Learning Objectives (_****_specific_****_)_**: + +*LO3a: Learn about the nature of reproducible research, workflow design, data management and manipulation, dynamic reporting, what the key requirements are, and which resources are available to support these (knowledge)**.* + +*LO3b: Be able to use available resources to create a workflow for reproducible research (task).* + +### Key components: + +* Factors that affect reproducibility of research. + +* Principles of reproducibility, and integrity and ethics in research. + +* What is the ‘reproducibility crisis’, and meta-analyses of reproducibility. + +* Open materials, reagents and hardware, including resources, repositories and standards. + +* Electronic lab notebooks. + +* Data analysis documentation and open research workflows. + +* Living figures, turning scripts into reproducible documents, and Markdown. + +* Pre-registration and prevention of p-hacking/HARK-ing (Hypothesising After Results are Known). + +* Reproducible analysis environments (virtualization). + +* What are the computing options and environments that allow collaborative and reproducible set up. + +### Who to involve: + +* Individuals: Andy Byers, Anna Krystalli, Julien Colomb, Rutger Vos, Brian Nosek, Lorena Barba, Karl Broman, Victoria Stodden, John Ioannidis, Chris Chambers. + +* Organisations: [FOSTER](https://www.fosteropenscience.eu/), Center for Open Science, [COPE](https://publicationethics.org/), [Protocols.io](https://www.protocols.io/), ROpenSci, [Addgene](addgene.org), [BITSS](https://www.bitss.org/), [Project TIER](https://www.projecttier.org/). + +* Other: [GOSH](http://openhardware.science/) Community, Software and Data Carpentry communities. + +### Key resources: + +**Tools** + +* Open Science Framework (COS). + + * [Reproducibility Project: Cancer Biology](https://osf.io/e81xl/wiki/home/). + + * [Reproducibility Project: Psychological Science](https://osf.io/ezcuj/wiki/home/). + + * [Registered Reports](https://cos.io/rr/). + +* Existing reproducible research workshops/practical resources: + + * [Reproducible Research Workshop](https://docs.google.com/presentation/d/1QfJVeim797fLBBAE003W6cqtahkGdJqa2GSsmm-t89s/edit?usp=sharing) (CC-BY, April Clyburne-Sherin & Courtney Soderberg). + + * [Initial steps towards reproducible research](http://kbroman.org/steps2rr/) (Karl Broman). + + * [The Open Science and Reproducible Research course](https://github.com/cbahlai/OSRR_course) (CC-BY, Christie Bahlai). + + * [Reproducibility Workshop](https://codeocean.com/workshop/caltech): Best practices and easy steps to save time for yourself and other researchers ([Code Ocean](https://codeocean.com/)). + + * [Reproducibility in Science: A guide to enhancing reproducibility in scientific results and writing](http://ropensci.github.io/reproducibility-guide/), ROpenSci. + + * [Reproducible Research using Jupyter Notebooks workshop](https://reproducible-science-curriculum.github.io/workshop-RR-Jupyter/) (Data Carpentry). + + * [R markdown workshop](https://github.com/libscie/rmarkdown-workshop) (Liberate Science). + + * [rrtools: Tools for Writing Reproducible Research in R](https://github.com/benmarwick/rrtools) (Ben Marwick). + +* [ReproZip](https://reprozip.org), an open source tool for full computational reproducibility. + +* [Software Carpentry](https://software-carpentry.org/lessons/) and [Data Carpentry](http://www.datacarpentry.org/lessons/) lessons. + +* [Jupyter notebooks](http://jupyter.org/) (and [JupyterLab](https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f039b8906)), [R Markdown](https://www.rstudio.com/resources/webinars/getting-started-with-r-markdown/), [Stencila](https://stenci.la/). + +* Virtual Machines, [Docker](https://www.docker.com/), [Vagrant](https://www.vagrantup.com/), [Binder](https://github.com/jupyterhub/binderhub)[Hub](https://github.com/jupyterhub/binderhub), [nteract.io](https://nteract.io/). + + * [Binder Documentation](https://mybinder.readthedocs.io/en/latest/), for creating custom computing environments that can be shared and used by multiple remote users. + +* [Statcheck](http://statcheck.io/), GRIM. + +* [Scienceroot](https://www.scienceroot.com/), the first blockchain-based scientific ecosystem. + +* Online repositories for open hardware: [PLOS open source toolkit channel](https://channels.plos.org/open-source-toolkit); [Open Neuroscience](http://www.openeuroscience.com); [Open Plant Science](http://openplant.science/); [Appropedia](http://www.appropedia.org/Welcome_to_Appropedia); [DocuBricks](http://www.docubricks.com/); [Hackaday.io](https://hackaday.io/submissions/prize2016_citizen/list). + +* [Bio-protocol](https://bio-protocol.org/Default.aspx), a peer reviewed protocol journal. + +* [BMJ Open Science](http://openscience.bmj.com/), a new journal that aims to improve the transparency, integrity and reproducibility of biomedical research. + +* [Evernote](https://evernote.com/), [Labguru](https://www.labguru.com/), [sciNote](https://scinote.net/). + +* [AsPredicted](https://aspredicted.org/). + +* The [Sci-Gaia Open Science Platform](http://www.sci-gaia.eu/osp/). + +* [Improving your statistical inferences](https://www.coursera.org/learn/statistical-inferences), Daniel Lakens. + + * [Open Stats Lab](https://sites.trinity.edu/osl/data-sets-and-activities), Kevin McIntyre. + +* [R for Data Science](http://r4ds.had.co.nz/). + + * [R tutorial: Introduction to cleaning data with R](https://www.youtube.com/watch?v=6PVMJE3HBN0) (DataCamp). + +* [Nextflow](https://www.nextflow.io/), open source tool than enables reproducible and portable computational workflows across cloud and clusters. + +**Research Articles and Reports** + +* [Reproducibility, Virtual Appliances, and Cloud Computing](https://osf.io/sp2vg/) (Howe, 2012). + +* [The Ironic Effect of Significant Results on the Credibility of Multiple-Study Articles](http://datacolada.org/wp-content/uploads/2014/06/3644-Schimmack-PM-2012-the-ironic-effect-of-significant-results-on-the-credibilit-of-multiple-study-articles.pdf) (Schimmack, 2012). + +* [Power failure: why small sample size undermines the reliability of neuroscience](https://www.nature.com/articles/nrn3475) (Button et al., 2013). + +* [Git can facilitate greater reproducibility and increased transparency in science](https://scfbm.biomedcentral.com/articles/10.1186/1751-0473-8-7) (Ram, 2013). + +* [Ten simple rules for reproducible computational research](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003285) (Sandve et al., 2013). + +* [Investigating Variation in Replicability: A "Many Labs" Replication Project](https://econtent.hogrefe.com/doi/abs/10.1027/1864-9335/a000178) (Klein et al., 2014). + +* [An introduction to Docker for reproducible research](https://arxiv.org/pdf/1410.0846.pdf) (Boettiger, 2015). + +* [Opinion: Reproducible research can still be wrong: Adopting a prevention approach](http://www.pnas.org/content/112/6/1645) (Leek and Peng, 2015). + +* [Replicability vs. reproducibility - or is it the other way around?](http://languagelog.ldc.upenn.edu/nll/?p=21956) (Liberman, 2015). + +* [The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology](https://peerj.com/preprints/2064v1/) (Brown and Heathers, 2016). + +* [What does research reproducibility mean?](http://stm.sciencemag.org/content/8/341/341ps12) (Goodman et al., 2016). + +* [Tools and techniques for computational reproducibility](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4940747/) (Piccolo and Frampton, 2016). + +* [Transparency, Reproducibility, and the Credibility of Economics Research](https://osf.io/preprints/bitss/9a3rw/) (Christensen and Miguel, 2017). + +* [A trust approach for sharing research reagents](http://stm.sciencemag.org/content/9/392/eaai9055.full?ijkey=uMGKxsCEiOb5s&keytype=ref&siteid=scitransmed) (Edwards et al., 2017). + +* [Estimating the Reproducibility of Psychological Science](https://osf.io/447b3/) (Nosek et al., 2017). + +* [Digital Open Science – Teaching digital tools for reproducible and transparent research](https://osf.io/fbvep/) (Toelch and Ostwald, 2017). + +* [Terminologies for reproducible research](https://arxiv.org/abs/1802.03311) (Barba, 2018). + +* [An introduction to statistical and data sciences via R](https://ismayc.github.io/moderndiver-book/) (Ismay and Kim, 2018). + +* [The practice of reproducible research: case studies and lessons from the data-intensive sciences](https://www.practicereproducibleresearch.org/) (Kitzes et al., 2018). + +* [bookdown: Authoring Books and Technical Documents with R Markdown](https://bookdown.org/yihui/bookdown/) (Xie, 2018). + +* [Our path to better science in less time using open data science tools](https://www.nature.com/articles/s41559-017-0160) (Lowndes et al. 2017). + +* [Haves and Have nots must find a better way: The case for Open Scientific Hardware](https://osf.io/uh682/) (Chagas, 2018). + +* [Computational Reproducibility via Containers in Social Psychology](https://psyarxiv.com/mf82t) (Green and Clyburne-Sherin, 2018). + +* Recommendations for open data science (Melissa Gymrek and Yossi Farjoun, 2016) + +**Key Posts** + +* Data hygiene and data provenance. + + * [A Data Cleaner’s Cookbook](https://www.polydesmida.info/cookbook/). + + * [Storify by Dawn Bazely](https://storify.com/dawnbazely/data-hygeine-dataprovenance-harvardforest). + +* [Failure is moving science forward](https://fivethirtyeight.com/features/failure-is-moving-science-forward/), Christie Aschwanden. + +* [5 keys to building open hardware](https://opensource.com/article/18/2/5-steps-creating-successful-open-hardware), Joshua Pearce. + +* [How to make replication the norm](https://www.nature.com/articles/d41586-018-02108-9) (Gertler et al., 2018). + +* [Reproducibility PI Manifesto](http://lorenabarba.com/gallery/reproducibility-pi-manifesto/), Lorena Barba. + + * [How to run a lab for reproducible research](https://figshare.com/articles/How_to_run_a_lab_for_reproducible_research/4676170/1), Lorena Barba. + + * [Essential skills for reproducible research computing](https://barbagroup.github.io/essential_skills_RRC/) (Barba et al., 2017). + +**Other** + +* Institutions, projects, and companies using or providing open hardware/materials: CERN’s [Open Hardware Repository](http://www.ohwr.org/) and Open Hardware License; [UFRGS Centro de Tecnologia Acadêmica](http://cta.if.ufrgs.br) (CTA); [Michigan Tech Open Sustainability Technology research group](http://www.mse.mtu.edu/~pearce/Index.html); [Open Plant](https://www.openplant.org/); [Trend in Africa](www.trendinafrica.org); [Open Lab Tools Cambridge University](http://openlabtools.eng.cam.ac.uk/index.php); [PhotosynQ](https://photosynq.org/); [PublicLab](https://publiclab.org/); [BackyardBrains](https://backyardbrains.com/); [OpenPCR](http://www.openpcr.org); [OpenROV](https://www.openrov.com/); [Prometheus Science](http://www.prometheus-science.com); [senseBox](https://sensebox.de/en/), [Addgene](addgene.org) + +* [Definition of Open Reproducible Research](https://www.fosteropenscience.eu/taxonomy/term/103), FOSTER. + +* [Global Open Science Hardware Roadmap](http://openhardware.science/global-open-science-hardware-roadmap/), GOSH. + +* [Open and Reproducible Science syllabus](https://osf.io/qbm89/) (Campbell, 2018). + +* [EQUATOR network](http://www.equator-network.org/) (Enhancing the QUAlity and Transparency Of health Research). + +* [Knitr](https://yihui.name/knitr/): Elegant, flexible, and fast dynamic report generation with R (Yihui Xie). + + * [Using Sweave and knitr](https://support.rstudio.com/hc/en-us/articles/200552056-Using-Sweave-and-knitr) (RStudio Support). + +* [SoS](http://vatlab.github.io/SoS/), multi-language notebook (based on Jupyter Notebook) and workflow system for cost-effective reproducible analysis. + + * Introduction to [SoS Notebook](https://vatlab.github.io/blog/post/sos-notebook/), [SoS Workflow Engine](https://vatlab.github.io/blog/post/sos-workflow-engine/), and [the power of backing a polyglot notebook with a workflow engine](https://vatlab.github.io/blog/post/power-of-sos-plus-sos-notebook/). + +### Tasks: + +* Find a core data set that is used throughout the examples. + + * If possible, the dataset should have a diverse set of formats and styles for different types of analysis. + +* Designing a reproducible research workflow. + + * Create a flowchart of options to help get you started Check if your collaborators, colleagues or supervisors are using the same tools. + + * This can be created as a Google doc and shared for collaboration. + + * Use validated, standardized reagents where possible. + + * Use an electronic lab notebook and best practices for recording protocols and actual steps, reagents used. + +* How well annotated are your code scripts? As a general rule of thumb, try and include one comment for every three lines of code. Bear in mind, the primary audience is future you and other people less familiar with your code. + +* Posting raw and cleaned data files. + + * Post your data (raw and/or treated) online in a non-proprietary format. + + * Make sure it is in a place where you can get a unique identifier for it. + +* Write a study plan or protocol. + + * Preregister your study design using [AsPredicted](https://aspredicted.org/), [OSF](https://osf.io/), or [Registered Reports](https://cos.io/rr/). + + * For clinical trials use [Clinicaltrials.gov](https://clinicaltrials.gov/). + +* Set up a reproducible project using an electronic lab notebook to help organise and track your research. + + * Track changes as your research develops using a version control system such as GitHub. + + * Document everything done by creating a README file. + + * Make sure to select an appropriate license for your repo. + + * Convert the notebook into a standard research manuscript. + + * In this manuscript, include all necessary code to reproduce any figures and tables in their respective captions. + +### Learning outcomes: + +1. Researchers will be able to describe the key factors that affect the reproducibility of research, including workflow design, data management, and reporting. + +2. The researcher will be able to use a range of resources to create and implement a workflow for reproducible research, including using lab notebooks and tools for sharing code and data. + +## 4: Open Research Data + +**_Rationale (_****_3 lines max_****_)_**: + +*Open research data refers to the publishing the data underpinning scientific research results so that they have no restrictions on their access. Openly sharing data opens it up to inspection and re-use, forms the basis for research verification and reproducibility, and opens up a path to broader collaboration. In this module, you will gain insight into the importance of data sharing for reproducible research and how to curate and share your own research data.* + +**_Learning Objectives (_****_specific_****_)_**: + +*LO4a: Learn the characteristics of open data, understand the advantages and disadvantages (alternatively, arguments for and against) open data (knowledge).* + +*LO4b: Be able to turn a closed data set made for personal use into an open data set made for maximised accessibility, transparency, and re-use (task). * + +### Key components: + +* What is open data. + +* [FAIR Principles](https://www.force11.org/group/fairgroup/fairprinciples) and data infrastructure. + +* Pros and cons of sharing data openly. + + * Sensitive data and anonymisation. + +* [Peer Reviewers Openness initiative](https://opennessinitiative.org/the-initiative/). + +* Data management workflows, data literacy, and data stewardship: + + * Data management plans. + + * Raw and primary data. + + * Tidy data. + + * Computer and human readability. + + * Interoperability: from vocabulary to ontologies. + +* Metadata: + + * Basic scheme for data publishing. + + * Additional information for data. + + * Folder organisation. + +* Data publishing (discipline-specific and generic databases) and data journals. + +* Sensitive data: privacy, de-identification/anonymization, mediated access. + +* Data citation. + +* Version control and data. + +### Who to involve: + +* Individuals:, Ross Mounce, Stephane Pesant, Julien Colomb, Rutger Vos, Eva Mendez, Brianna Marshall, Barend Mons, Hadley Beeman, Fiona Murphy, Peter Murray-Rust, Kate LeMay. + +* Organisations: The [Open Data Institute](https://theodi.org/), [Open Knowledge International](https://okfn.org/), Figshare, [EIFLNet](http://www.eifl.net/), [UK Anonymisation network](http://ukanon.net/), [NISO](http://www.niso.org/), [Australian National Data Service](https://www.ands.org.au/), [DataCite](https://www.datacite.org/), [Figshare](https://figshare.com/browse). + +* Other: Data management librarians from [OpenCon ](http://www.opencon2017.org/)and [Research Data Access and Preservation](https://www.asist.org/rdap/) (RDAP) communities, people from the [PRO initiative](https://opennessinitiative.org/contact-us/), RDA [Privacy Interests of Research Data sets Interest Group](https://www.rd-alliance.org/groups/rdaniso-privacy-implications-research-data-sets-wg.html). + +### Key resources: + +**Tools** + +* [Re3data](https://www.re3data.org/) (Registry of Research Data Repositories). + +* [Data.gov](https://www.data.gov/), comprises data, tools, and resources to conduct research, develop web and mobile applications and design data visualizations. + +* [World Bank Open Data](https://data.worldbank.org/). + +* Generic databases/repositories: [Zenodo](https://zenodo.org/), [Figshare](https://figshare.com/), [Dryad](http://datadryad.org/), [Pangaea.de](https://pangaea.de/), [Mendeley Data](https://data.mendeley.com/), [Datahub.io](http://datahub.io/), [Harvard Dataverse](https://dataverse.harvard.edu/), [data.opendatasoft.com](http://data.opendatasoft.com) (+10,000 open datasets). + +* Discipline-specific databases/repositories: + + * GenBank (see also [GenBank](https://academic.oup.com/nar/article/41/D1/D36/1068219), Benson et al., 2012). + + * [UniProt: A hub for protein information](https://academic.oup.com/nar/article/43/D1/D204/2439939), The UniPort Consortium. + + * [The SIMBAD astronomical database](https://aas.aanda.org/articles/aas/abs/2000/07/ds1821/ds1821.html) (Wenger et al., 2000). + + * [CiteAb](https://www.citeab.com/), an antibody search engine. + + * [ICLAC](http://iclac.org/), the International Cell Line Authentication Committee. + + * [SEEK: a systems biology data and model management platform](https://bmcsystbiol.biomedcentral.com/articles/10.1186/s12918-015-0174-y) (Wolstencroft et al., 2015). + + * [openBIS: a flexible framework for managing and analyzing complex data in biology research](https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-12-468) (Bauch et al., 2011). + + * [Datastro.eu](http://www.datastro.eu): an open data portal build with the OpenDataSoft platform, with data about astronomy (e.g., all Apollo program pictures, light pollution maps, NASA and Minor Planet Center data, asteroids orbits, exoplanet catalog, Messier catalog, sunspots reports, constellations list). + +* [Open Data Training](https://github.com/mozillascience/open-data-training) and [Open Data Primers](https://github.com/mozillascience/open-data-primers), Mozilla Science Lab. + +* [Open Data Workshop SSEAC Usyd](https://osf.io/s76gu/) - Institut Teknologi Bandung. + +* [Open Data Essentials](http://accelerate.theodi.org/), Open Data Institute (ODI). + +* [DMPonline](https://dmponline.dcc.ac.uk): Tool for creating, reviewing, and sharing data management plans. + +* [Open Science, Open Data, Open Source](https://pfern.github.io/OSODOS/gitbook/) (Fernandes and Vos, 2017). + +* [Scientific Data](https://www.nature.com/sdata/) and the [Data Science Journal](http://www.codata.org/publications/data-science-journal). + +* [Expert tour guide on Data Management](https://www.cessda.eu/Research-Infrastructure/Training/Expert-tour-guide-on-Data-Management), Consortium of European Social Science Data Archives. + +* [DataCite](https://www.datacite.org/), a leading global provider of DOIs for research data. + +* [CKAN](https://github.com/ckan/ckan), an open source data management system (DMS) for powering data hubs and data portals. + +**Research Articles and Reports** + +* [Research Objects: Towards Exchange and Reuse of Digital Knowledge](http://precedings.nature.com/documents/4626/version/1) (Bechhofer et al., 2010). + +* [The Enduring Value of Social Science Research: The Use and Reuse of Primary Research Data](https://deepblue.lib.umich.edu/handle/2027.42/78307) (Pienta et al., 2010). + +* [The data paper: a mechanism to incentivize data publishing in biodiversity science](https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-12-S15-S2) (Chavan and Penev, 2011). + +* [The Dataverse Network: An Open-Source Application for Sharing, Discovering and Preserving Data](http://www.dlib.org/dlib/january11/crosas/01crosas.html) (Crosas, 2011). + +* [Data sharing in neuroimaging research](https://www.frontiersin.org/articles/10.3389/fninf.2012.00009/full) (Poline et al., 2012). + +* [Toward interoperable bioscience data](https://www.nature.com/articles/ng.1054) (Sansone et al., 2012). + +* [Making data sharing count: a publication-based solution](https://www.frontiersin.org/articles/10.3389/fnins.2013.00009/full) (Gorgolewski et al., 2013). + +* [EUDAT: A New Cross-Disciplinary Data Infrastructure for Science](http://www.ijdc.net/article/view/8.1.279) (Lecarpentier et al., 2013). + +* [Data reuse and the open data citation advantage](https://peerj.com/articles/175/) (Piwowar and Vision, 2013). + +* [Nine simple ways to make it easier to (re)use your data](https://peerj.com/preprints/7/) (White et al., 2013). + +* [The data sharing advantage in astrophysics](https://arxiv.org/abs/1511.02512) (Dorch et al., 2015). + +* [What Drives Academic Data Sharing?](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0118053), (Fecher et al., 2015). + +* [From Peer-Reviewed to Peer-Reproduced in Scholarly Publishing: The Complementary Roles of Data Models and Workflows in Bioinformatics](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0127612) (González-Beltrán et al., 2015). + +* [Making data count](https://www.nature.com/articles/sdata201539) (Kratz and Strasser, 2015). + +* [The center for expanded data annotation and retrieval](https://academic.oup.com/jamia/article/22/6/1148/2357598) (Musen et al., 2015). + +* [Public Data Archiving in Ecology and Evolution: How Well Are We Doing?](http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002295) (Roche et al., 2015). + +* [Achieving human and machine accessibility of cited data in scholarly publications](https://peerj.com/articles/cs-1/) (Starr et al., 2015). + +* [The State of Open Data Report](https://figshare.com/articles/The_State_of_Open_Data_Report/4036398) (Treadway et al., 2016). + +* [The FAIR Guiding Principles for scientific data management and stewardship](https://www.nature.com/articles/sdata201618) (Wilkinson et al., 2016). + +* [Towards coordinated international support of core data resources for the life sciences](https://www.biorxiv.org/content/early/2017/04/27/110825) (Anderson et al., 2017). + +* [A reputation economy: how individual reward considerations trump systemic arguments for open access to data](https://www.nature.com/articles/palcomms201751), (Fecher et al., 2017). + +* [Cloudy, increasingly FAIR; revisiting the FAIR Data guiding principles for the European Open Science Cloud](https://content.iospress.com/articles/information-services-and-use/isu824) (Mons et al., 2017). + +* [Code of practice for research data usage metrics release 1](https://peerj.com/preprints/26505/) (Fenner et al., 2018). + +**Key posts** + +* [Primer on Data Management: What you always wanted to know](https://www.dataone.org/sites/all/documents/DataONE_BP_Primer_020212.pdf), DataOne. + +* Data Citation Synthesis Group: [Joint declaration of data citation principles](https://www.force11.org/datacitationprinciples), FORCE 11. + +**Other** + +* [FAIR sharing](https://fairsharing.org/): A curated, informative and educational resource on data and metadata standards, inter-related to databases and data policies. + +* Australian National Data Service [Guides](http://www.ands.org.au/guides) and [Sensitive Data Resources ](http://www.ands.org.au/working-with-data/sensitive-data). + +* The [Open Data Institute](https://theodi.org/) (ODI). + +* The [Digital Curation Centre](http://www.dcc.ac.uk/) (DCC). + +* [RDA Metadata Standards Directory Working Group](http://rd-alliance.github.io/metadata-directory/). + +* [Data Archiving and Network Services](https://dans.knaw.nl/nl) (DANS). + +* [How to create a data organisation dictionary](http://kbroman.org/dataorg/pages/dictionary.html), Karl Broman. + +* Data Curation Centre: [How to License Research Data](http://www.dcc.ac.uk/resources/how-guides/license-research-data). + +* [What is Open Data?](http://opendatahandbook.org/guide/en/what-is-open-data/), Open Data Handbook. + +* [How to select a repository?](https://www.openaire.eu/opendatapilot-repository), OpenAIRE. + +* [Developing Open Data policies](https://www.fosteropenscience.eu/sites/default/files/pdf/2276.pdf), FOSTER. + +* [Data Packaging Guide](https://github.com/saverkamp/beyond-open-data/blob/master/DataGuide.md) (Shawn Averkamp, Ashley Blewer, Matt Miller). + +* [Frictional Data](https://frictionlessdata.io/), specifications and software for the publication, transport and consumption of data. + +* [Metadata 2020](http://www.metadata2020.org/), a collaboration that advocates richer, connected, and re-usable open metadata for all research outputs. + +* [What is open data?](https://www.opendatasoft.com/2017/02/16/what-is-open-data/) (OpenDataSoft). + +* [Nope, HTML is not Open Data](https://www.opendatasoft.com/2016/05/23/nope-html-is-not-open-data/) (OpenDataSoft) + +* [What is metadata and why is it as important as the data itself?](https://www.opendatasoft.com/2016/08/25/what-is-metadata-and-why-is-it-important-data/) (OpenDataSoft). + +* [What is a Smart City? A Comprehensive Introduction](https://www.opendatasoft.com/2017/07/18/what-is-a-smart-city-a-comprehensive-introduction/) (OpenDataSoft). + +* [Open Data as Terraces](https://www.opendatasoft.com/2016/12/27/open-data-terraces/) (OpenDataSoft). + +* [Author Reagent Table: A proposal](https://f1000research.com/posters/6-449) (Crosby et al., 2017). + +### Tasks: + +* Find a core data set that is used throughout the examples. + + * If possible, the dataset should have a diverse set of formats and styles for different types of analysis + +* Metadata: add minimal context for data interpretation and re-use. + + * Think about your target audience, the delivery format, file names, and general accessibility. + + * Upload some of your data to a public repository. + + * Make sure it conforms to the FAIR principles. + +* Search for data that might be of use to you in your research. + + * Does it meet FAIR requirements? + +### Learning outcomes: + +1. The researcher will be able to define the characteristics of open data, the advantages and disadvantages associated with sharing different types of data openly, and the FAIR principles. + +2. Researchers will be able to share their research data openly to a relevant public repository in a way that conforms to the FAIR principles. + +3. The researcher will be able to locate and re-use datasets for their research from relevant disciplinary repositories. + +## 5: Open Research Software and Open Source + +**_Rationale (_****_3 lines max_****_)_**: + +*Software and technology underpin modern** science.** There is an increasing demand for more sophisticated open source software, matched by an increasing willingness for researchers to openly collaborate on new tools. These developments come with a specific ethical, legal and economic challenges that impact upon research workflows. This module will introduce the necessary tools required for transforming software into something that can be openly accessed and re-used by others.* + +**_Learning Objectives (1-3 _****_specific_****_)_**: + +*LO5a: Learn the characteristics of open software; understand the **ethical, legal, economic, and research impact arguments for and against open software**, and further understand the quality requirements of open code (knowledge).* + +*LO5b: Be able to turn code made for personal use into open code which is accessible by others (task).* + +*LO5c: Use software (tools) that utilizes open content (task).* + +### Key components: + +* Principles of open source software. + +* What open, collaborative platforms, with version control, exist. + +* GitHub and Zenodo plug-in for code archiving. + +* How to document and publish code. + +* Open Source licensing. + +* Tools for better open research (e.g., RStudio). + +* Community codes, governance, and contributions. + +* How to access and start working on general computing platforms (e.g., GCE, AWS, OpenStack and more specific - Galaxy, InsideDNA.) + +* Differences in setting up accounts/storage/computing on different platforms. + +* Comparison in terms of collaboration and openness with clusters/in-house servers. + +### Who to involve: + +* Individuals: Paola Masuzzo, Naomi Penfold, Titus Brown, René Bernard, Daniel Katz, Neil Chue Hong, Heidi Seibold, Anna Kostikova. + +* Organisations: Sustainable Software Institute, COKO Foundation, [Free Software Foundation](https://www.fsf.org/licensing/). + +* Other: [WSSSPE community](http://wssspe.researchcomputing.org.uk), Editors of software peer-reviewed journals ([Open Research Software](http://openresearchsoftware.metajnl.com/), [JOSS](http://joss.theoj.org)), Testimonial of scientists who just published code explaining why they went through the trouble and of scientists who already use cloud computing. Explanations about large initiatives (e.g. TCGA) moving their data into cloud and why it has huge impact. + +### Key resources: + +**Tools** + +* [Journal of Open Research Software](https://openresearchsoftware.metajnl.com/) and the [Journal of Open Source Software](https://joss.theoj.org/). + +* [InsideDNA](https://insidedna.me/), [Galaxy](https://galaxyproject.org/) - Reproducible Research environments. + +* [Google Compute Engine](https://cloud.google.com/compute/) (GCE), [Amazon Web Services](https://aws.amazon.com/) (AWS) - Cloud-based software environments. + +* [Software Citation Tools](https://github.com/mozillascience/software-citation-tools), Mozilla Science Lab. + +* [Open Science, Open Data, Open Source](https://pfern.github.io/OSODOS/gitbook/) (Fernandes and Vos, 2017). + +* [Choose an open source license](https://choosealicense.com/). + +**Research Articles and Reports** + +* [The Future of Research in Free/Open Source Software Development](http://www.ics.uci.edu/~wscacchi/Papers/New/FoSER-Scacchi-2010.pdf) (Scacchi, 2010). + +* [The Scientific Method in Practice: Reproducibility in the Computational Sciences](http://datascienceassn.org/sites/default/files/The%20Scientific%20Method%20in%20Practice%20-%20Reproducibility%20in%20the%20Computational%20Sciences.pdf) (Stodden, 2010). + +* [The case for open computer programs](https://www.nature.com/articles/nature10836) (Ince et al., 2012). + +* [Code Sharing Is Associated with Research Impact in Image Processing](https://infoscience.epfl.ch/record/206184/files/Vandewalle12.pdf) (Vandewalle, 2012). + +* [Current issues and research trends on open-source software communities](https://idus.us.es/xmlui/bitstream/handle/11441/32245/Current%20issues%20and%20research%20trends.pdf?sequence=1) (Martinez-Torres and Diaz-Fernandez, 2013). + +* [Ten simple rules for reproducible computational research](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003285) (Sandve et al., 2013). + +* [Practices in source code sharing in astrophysics](https://arxiv.org/abs/1304.6780) (Shamir et al., 2013). + +* [A systematic literature review on the barriers faced by newcomers to open source software projects](http://igor.pro.br/publica/papers/IST_SysReview_PrePrint.pdf) (Steinmacher et al., 2014). + +* [Knowledge sharing in open source software communities: motivations and management](https://pdfs.semanticscholar.org/f2a2/c5129cf5656af7acc7ffaf84c9c9bafe72c5.pdf) (Iskoujina and Roberts, 2015). + +* [An open source pharma roadmap](http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002276) (Balasegaram et al., 2017). + +* [An introduction to Rocker: Docker containers for R](https://arxiv.org/abs/1710.03675) (Boettiger and Eddelbuettel, 2017). + +* [Upon the Shoulders of Giants: Open-Source Hardware and Software in Analytical Chemistry](https://pubs.acs.org/doi/abs/10.1021/acs.analchem.7b00485) (Dryden et al., 2017). + +* [Four simple recommendations to encourage best practices in research software](https://f1000research.com/articles/6-876/v1) (Jiménez et al., 2017). + +* [Perspectives on Reproducibility and Sustainability of Open-Source Scientific Software from Seven Years of the Dedalus Project](https://arxiv.org/abs/1801.08200) (Oishi et al., 2018). + +* [Good enough practices in scientific computing](https://doi.org/10.1371/journal.pcbi.1005510) (Wilson et al. ,2017). + +**Key Posts** + +* [A breakdown of FOSS [Free and Open Source Software] for students and researchers in academia](https://opensource.com/education/15/11/foss-academic-research), Lois Donnelly. + +* [Publish your computer code: it is good enough](http://www.nature.com/news/2010/101013/full/467753a.html), Nick Barnes. + +* [Making your code citable](https://guides.github.com/activities/citable-code/), GitHub Guides. + +* [FLOSS and FOSS](https://www.gnu.org/philosophy/floss-and-foss.en.html), Richard Stallman. + +**Other** + +* [The Sustainable Software Institute](https://www.software.ac.uk/). + +* [The Science Code Manifesto](http://sciencecodemanifesto.org/). + +* [Software Carpentry](https://software-carpentry.org/). + +* Github’s [Open Source Guide](https://opensource.guide/). + +* [Software Citation Principles](https://doi.org/10.7717/peerj-cs.86). + +* Open Source Initiative: [Licenses and Standards](https://opensource.org/licenses). + +* [Arduino](https://www.arduino.cc/), an open source electronics platform based on easy-to-use hardware and software. + +### Tasks: + +* Set up a GitHub account, if you haven’t already. + + * Share some of your code in a new repo. + + * Track changes as your research develops using version control. + + * Document everything done by creating a README file. + + * Make sure to select an appropriate license for your repo. + + * Archive your versioned code in Zenodo. + +* Explore minimum requirements to publish research code. + +* Set up accounts in different platforms and see how to upload and share data and software. + +* Create a Docker ID. + + * Learn how to bring Docker containers into the cloud and execute analyses with them. + +### Learning outcomes: + +1. The researcher will be able to define the characteristics of open source research software, and the ethical, legal, economic and research impact arguments for and against it. + +2. Based on community standards, researchers will be able to describe the quality requirements of sharing and re-using open code. + +3. The researcher will be able to use a range of research tools that utilise open source software. + +4. Individual researchers will be able to transform code designed for their personal use into code that is accessible and re-usable by others. + +## 6: Open Access to Research Papers + +**_Rationale (_****_3 lines max_****_)_**: + +*Making scholarly research outputs openly available is easy, legal, and has demonstrable benefits to authors, making it a good beginning step for a researcher just beginning to explore the open world. There is a set of knowledge required to navigate the Open Access landscape, involving copyright, article status, repositories, and economics. This module will introduce key concepts and tools that can help a researcher make their work openly available and maximize the benefits to themselves and others. * + +**_Learning Objectives (1-3 _****_specific_****_)_**: + +*LO6a: Understand the allowances for self-archiving in publishing contracts, including issues to do with copyright, licensing, article versions, availability, embargoes, and the types of outlets for self-archiving (knowledge).* + +*LO6b: Gain an understanding of the history of scholarly publishing, and be able to articulate benefits of Open Access in terms of impact on society and our knowledge economy (knowledge).* + +*LO6c: Develop a personal infrastructure for self-archiving (task).* + +### Key components: + +* Sharing research findings with international academic and non-academic communities without paywall and other usage restrictions. + +* Personal academic impact and advantages of Open Access (e.g., increased citation counts, visibility, readership). + +* Global, national, funder, and institutional policies and mandates. + +* Pre-prints, post-prints, and versions of record (VOR). + +* Different ‘types’ of Open Access: gold, green, [diamond/platinum](https://www.google.com/url?q=https://openscience.uni-bielefeld.de/1013/what-is-platinum-open-access&sa=D&ust=1496297601957000&usg=AFQjCNFCmRz4Ub5JSo_23w3OWvdz5B-O8Q), black. + +* The cost and economics of Open Access. + +* Open Access platforms. + +* Institutional and subject repositories . + +* Scholarly Collaboration Networks (e.g., ResearchGate, Academia.edu). + +* Open Access monographs and books. + +* Pre-registration. + +### Who to involve: + +* Individuals: Lauren Collister, Martin Paul Eve, Chris Chambers, Jessica Polka, Mark Patterson, Pablo Dorta-González, Ahmed Ogunlaja, Ricardo Hartley, Dasapta Erwin Irawan, Bjoern Brembs, Erin McKiernan, Anna Sharman. + +* Organisations: [DOAJ,](https://doaj.org/) [SPARC](https://sparcopen.org), [Open Library of Humanities](https://www.openlibhums.org/), [ASAPbio](http://asapbio.org/), Open Access advocacy groups, including local initiatives on the country and institute level. + +* Other: SHERPA/RoMEO, [Open Access Directory](http://oad.simmons.edu/oadwiki/Main_Page). + +### Key resources: + +**Tools** + +* [Think, Check, Submit](http://thinkchecksubmit.org/) and [Cofactor journal selector tool](http://cofactorscience.com/journal-selector). + +* [SPARC Author Addendum](https://sparcopen.org/our-work/author-rights/brochure-html/) and the [Termination of Transfer tool](https://rightsback.org/), by Authors Alliance and Creative Commons. + +* [Open Access Journal Whitelist](http://s-quest.bihealth.org:3838/OAWhitelist/), QUEST Center. Contains biomedical open access journals that are listed on the Directory of Open Access Journals (DOAJ) and Pubmed Central. + +* [Unpaywall](https://oadoi.org/), [Open Access Button](https://openaccessbutton.org/). + +* [APCDOI](https://github.com/ryregier/APCDOI), a program for determining how many DOIs are ‘gold’ or ‘hybrid’ Open Access and how much was spent on the article processing charge (APC) for these (Ryan Regier). + +* [Open Science Framework](https://osf.io/preprints/) preprints and [PrePubMed](http://www.prepubmed.org/). Other repositories including: + + * [SSRN](https://ssrn.com/en/index.cfm?) (Social Sciences Research Network). + + * [ChemRxiv](https://chemrxiv.org/) (Chemistry). + + * [ESSOAr](https://www.essoar.org/) (Earth Sciences). + + * [Cogprints](http://www.cogprints.org/) (Psychology, Neuroscience and Linguistics). + +* [CORE](https://core.ac.uk/), an aggregator of 125 million Open Access articles. + +* [Social Science Open Access Repository](https://www.gesis.org/ssoar/home/) (SSOAR). + +* [LOADB](http://www.loadb.org/), Listing of Open Access Databases. + +* [REDALYC](http://www.redalyc.org/home.oa), network of scientific journals of Latin America and the Caribbean, Spain and Portugal. + + * [LA Referencia](http://www.lareferencia.info/joomla/en/). + +* Language-specific servers: + + * [Aribixiv](https://arabixiv.org/). + + * [Frenxiv](https://frenxiv.org/). + + * [INA-Rxiv](https://osf.io/preprints/inarxiv/). + +* [SHERPA/RoMEO](http://www.sherpa.ac.uk/romeo/index.php) - Publisher Copyright Policies and Self-Archiving. + + * [DULCINEA](http://www.accesoabierto.net/dulcinea/) (for Spanish journals). + + * [Héloïse](https://heloise.ccsd.cnrs.fr/) (for French journals). + +* [SHERPA Juliet](http://v2.sherpa.ac.uk/juliet/) - Research Funders’ Open Access policies. + +* [Wellcome Open Research](https://wellcomeopenresearch.org/) and [Gates Open Research](https://gatesopenresearch.org/). + +* Open Access publishers: + + * [MDPI](http://www.mdpi.com/), [F1000](https://f1000research.com/), [Hindawi](https://www.hindawi.com/), [Cogent OA](https://www.cogentoa.com/), [Open Library of Humanities](https://olh.openlibhums.org/), [BioMed Central](https://www.biomedcentral.com/journals), [eLIFE](https://elifesciences.org/), [Frontiers](https://www.frontiersin.org/), [PLOS](https://www.plos.org/). + +* [Open Knowledge Maps](https://openknowledgemaps.org/). + +* [PASTEUR4OA](http://www.pasteur4oa.eu/), Open Access Policy Alignment Strategies for European Union Research. + +* [The Publishing Trap](https://copyrightliteracy.org/resources/the-publishing-trap/) board game, to help researchers understand how money, intellectual property rights, and both open and closed publishing models affect the dissemination and impact of their work (UK Copyright Literacy). + +* [Mathoverflow](https://mathoverflow.net/) and [PhysicsOverflow](https://physicsoverflow.org/). + +* [PubPub](https://www.pubpub.org/), collaborative community publishing. + +* [JSTOR](http://www.jstor.org/open/?cid=SOC_JSTOR), a portal for open content. + +* [Dimensions](https://www.dimensions.ai/), for information on grants, publications, citations, clinical trials and patents. + +* Preprint recommendation services: + + * [preLights](https://prelights.biologists.com/). + + * [Peer Community in](https://peercommunityin.org/). + +* Overlay journals: + + * [Open Journals](http://www.theoj.org/). + + * [Discrete Analysis](http://discreteanalysisjournal.com/). + + * [biOverlay](https://www.bioverlay.org/post/welcome/). + +**Research Articles and Reports** + +* [The Nine Flavours of Open Access Scholarly Publishing](http://www.jpgmonline.com/article.asp?issn=0022-3859;year=2003;volume=49;issue=3;spage=263;epage=267;aulast=Willinsky) (Willinsky, 2003). + +* [The Development of Open Access Journal Publishing from 1993 to 2009](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0020961) (Laakso et al., 2011). + +* [A Study of Open Access Journals Using Article Processing Charges](http://www.openaccesspublishing.org/apc2/preprint.pdf) (Solomon and Björk, 2012). + +* [Open Access (the book)](https://cyber.harvard.edu/hoap/Open_Access_(the_book)) (Suber, 2012). + +* [Anatomy of Green Open Access](http://www.openaccesspublishing.org/apc8/Personal%20VersionGreenOa.pdf) (Björk et al., 2013). + +* [The case for open preprints in biology](http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001563) (Desjardins-Proulx et al., 2013). + +* [arXiv e-prints and the journal of record: An analysis of roles and relationships](https://arxiv.org/abs/1306.3261) (Larivière et al., 2013) + +* [Proportion of Open Access Papers Published in Peer-Reviewed Journals at the European and World Levels—1996–2013](http://science-metrix.com/sites/default/files/science-metrix/publications/d_1.8_sm_ec_dg-rtd_proportion_oa_1996-2013_v11p.pdf) (European Commission, 2014). + +* [Disrupting the subscription journals’ business model for the necessary large-scale transformation to open access](http://pubman.mpdl.mpg.de/pubman/faces/viewItemOverviewPage.jsp?itemId=escidoc:2148961) (Schimmer et al., 2015). + +* [Hybrid open access—A longitudinal study](https://www.sciencedirect.com/science/article/pii/S1751157716301523?via%3Dihub) (Laakso and Bjork, 2016). + +* [Point of View: How open science helps researchers succeed](https://elifesciences.org/articles/16800) (McKiernan et al., 2016). + +* [Converting scholarly journals to Open Access: A review of approaches and experiences](https://dash.harvard.edu/handle/1/27803834) (Solomon et al., 2016). + +* [The academic, economic and societal impacts of Open Access: an evidence-based review](https://f1000research.com/articles/5-632/v3) (Tennant et al., 2016). + +* [Open Access policies and Science Europe: State of play](https://content.iospress.com/articles/information-services-and-use/isu839) (Crowfoot, 2017). + +* [Gold Open Access Publishing in Mega-Journals: Developing Countries Pay the Price of Western Premium Academic Output](https://research.vu.nl/ws/files/41184625/2017ellerscrowther_harvey_Gold_Open_Access_Publishing_in_Mega_Journals.pdf) (Ellers et al., 2017). + +* [Looking into Pandora's Box: The Content of Sci-Hub and its Usage](https://f1000research.com/articles/6-541/v1) (Greshake, 2017). + +* [On the origin of nonequivalent states: How we can talk about preprints](https://f1000research.com/articles/6-608/v1) (Neylon et al., 2017). + +* [Open Access and OER in Latin America: A survey of the policy landscape in Chile, Colombia and Uruguay](https://zenodo.org/record/1094840#.WoAX4SXwaM9) (Toledo, 2017). + +* [Research: Sci-Hub provides access to nearly all scholarly literature](https://elifesciences.org/articles/32822) (Himmelstein et al., 2018). + +* [Converting the Literature of a Scientific Field to Open Access Through Global Collaboration: the Experience of SCOAP3 in Particle Physics](https://www.preprints.org/manuscript/201802.0149/v1) (Kohls and Mele, 2018). + +* [Open Access Initiatives and Networking in the Global South](https://zenodo.org/record/1176573#.Wpf28WrwaM_) (Kuchma, 2018). + +* [The State of OA: A large-scale analysis of the prevalence and impact of Open Access articles](https://peerj.com/articles/4375/) (Piwowar et al., 2018). + +* [Authorial and institutional stratification in open access publishing: the case of global health research](https://peerj.com/articles/4269/) (Siler et al., 2018). + +**Key posts** + +* [Good practices for university open-access policies](https://cyber.harvard.edu/hoap/Good_practices_for_university_open-access_policies), Harvard University, 2017. + +* [Open Access Policy concerning UNESCO publications](http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/ERI/pdf/oa_policy_rev2.pdf), UNESCO. + +* [A genealogy of open access: negotiations between openness and access to research](http://journals.openedition.org/rfsic/3220), Samuel Moore. + +* [Open access and development: Research findings](http://blogs.worldbank.org/voices/open-order-end-extreme-poverty-open-access-and-development-research-findings), Elisa Liberatori Prati. + +* [DOAJ APC information as of Jan 31, 2018](https://sustainingknowledgecommons.org/2018/02/06/doaj-apc-information-as-of-jan-31-2018/), Heather Morrison. + +* [Open access policies and mandates around the globe](https://www.editage.com/insights/open-access-policies-and-mandates-around-the-globe?utm_source=twitter&utm_medium=referral&utm_campaign=OpenAccessWeek), Jayashree Rajagopalan. + +**Other** + +* [OpenDOAR](http://www.opendoar.org/) (Directory of Open Access Repositories) and the [Registry of Open Access Repository Mandates and Policies](http://roarmap.eprints.org/) (ROARMAP). + +* [UK Scholarly Communications Licence and model policy](http://ukscl.ac.uk/) (UKSCL). + +* [Directory of Open Access Books](http://www.doab.org/) (DOAB). + +* [Knowledge Unlatched](http://www.knowledgeunlatched.org/). + +* [Compact on Open-Access Publishing Equity](http://www.oacompact.org/). + +* [Quality Open Access Market](https://www.qoam.eu/). + +* [Scientific Electronic Library Online](http://www.scielo.org/php/index.php?lang=en), SciELO. + +* [SCOAP3](https://scoap3.org/) - Sponsoring Consortium for Open Access Publishing in Particle Physics. + +* [SPARC article and data sharing requirements by federal agency](http://researchsharing.sparcopen.org/). + +* [Open APC initiative](https://treemaps.intact-project.org/), information on fees paid for OA journal articles by universities and research institutions under an Open Database License. + +* [Free Journal Network](http://freejournals.org/). + +* [Information Note: Towards a Horizon 2020 platform for Open Access](https://ec.europa.eu/research/openscience/pdf/information_note_platform_public.pdf), European Commission. + +* [Declarations in Support of Open Access](http://oad.simmons.edu/oadwiki/Declarations_in_support_of_OA) - the Open Access Directory. + + * The [Budapest Open Access Initiative](http://www.budapestopenaccessinitiative.org/). + +* [HRCAK](https://hrcak.srce.hr/?lang=en) Repository of the Croatian OA journals. + +### Tasks: + +* Get an overview of the relevant journals and publishing outlets in your research discipline. + + * Which ones have Open Access options. + + * How much do they each charge for Open Access. + + * What funds are available to you to cover these (where relevant). + +* Preferably, find out which diamond/platinum OA journals (i.e., those which do not charge APCs) with high-quality editorial policies exist in your field. + + * Check the description of the peer review process. + + * Check which other additional services are offered by the journal (e.g., XML conversion, publication of updated versions, text and data mining allowances). + + * Check journal ‘whitelists’ (e.g., [The Norwegian Register for Scientific Journals, Series and Publishers](https://dbh.nsd.uib.no/publiseringskanaler/Forside.action?request_locale=en)). + +* Draft a summary statement/report outlining the pros and cons of these outlets (e.g., editorial quality, OA policies). + + * What do your colleagues think about the credibility, advantages, and disadvantages of these outlets? + + * How does this compare to your views? + +* Simple exercises on average "cost of a paper"; for example the average institute budget/publication output, or your last research grant/papers out compared to the average ‘gold Open Access’ cost in that discipline. + +* Find out if you are eligible for funds to pay for article-processing charges APCs. + + * Is the policy from your funder or institute? + + * What are the conditions? + +* Find a way to make all of your research papers legally freely available. + + * Use SHERPA/RoMEO to detangle the legalese in publishing contracts. + + * Check with [Dissem.in](https://dissem.in/) which of your papers can be made Open Access via self-archiving. + + * Self-archive one paper (can be previously published) or share a pre-print to an archive. + + * Make sure to identify and include all relevant metadata (e.g. publisher requires citation with a URL to the final published version). + +* Check ImpactStory to see the impact of your research outputs. + + * What can be improved? + + * What happens to your Open Access score when you self-archive your papers? + +* Request an article using the OA Button. + + * Perhaps consider a Green OA advocacy volunteer effort for your discipline or a favorite journal in the spirit of the [Library Pipeline Green OA Working Group](https://www.librarypipeline.org/lis-publications/green-open-access/ ). + +* Look for a local OA journal at your university or in your region. + + * Is there a preprint server for your research discipline? + +### Learning outcomes: + +1. The researcher will become familiar with the history of scholarly publishing, and development of the present Open Access landscape. + +2. The researcher will gain a multi-stakeholder insight into Open Access, and be able to convey a balanced overview of the perceived advantages and disadvantages associated with Open Access publishing. + +3. The researcher will be able to describe some of the complexities of the current the Open Access landscape, including allowances for self-archiving and embargoes, copyright transfer, and publishing contracts. + +4. Based on community-specific practices, the researcher will be able to use the different types of outlets (repositories) available for self-archiving, as well as the range of Open Access journal types available to them. + +5. Each researcher will able to make all of their own research papers Open Access through a combination of journals and development of a personal self-archiving protocol. + +6. Researchers will be able to describe the current ebb and flow in the debates around preprints, and be able to locate and use relevant disciplinary preprint platforms. + +7. Researchers will be able to use services like ImpactStory to track the proportion of their research that is Open Access. + +## 7: Open Evaluation + +**_Rationale (_****_3 lines max_****_)_**: + +*Concurrent with broader developments in Open Science and increased transparency in research, Open Peer Review is a complex, and rapidly evolving topic. Alongside this, more diverse criteria of research evaluation beyond traditional methods are emerging, and with these come a range of practical, ethical, and social factors to consider. This module will provide insight into current developments in Open Peer Review and research evaluation.* + +**_Learning Objectives (_****_specific_****_)_**: + +*LO7a: To understand the history of peer review, and place current developments in Open Peer Review in that context (knowledge).* + +*LO7b: To gain insight into the process of responsible research evaluation, and the role that peer review and traditional and next-generation metrics play in this (knowledge).* + +*LO7c: To be able to identify and apply a range of metrics to demonstrate the broader impact of your research outputs (tasks).* + +### Key components: + +* Fundamentals of good peer review. + +* History of peer review and scholarly publishing. + +* Types of open peer review and new models. + +* Pros and cons associated with different types of open peer review, including post-publication peer review, commenting and annotation. + +* Issues with traditional methods of research assessment and evaluation. + +* The San Francisco Declaration on Research Assessment ([DORA](http://www.ascb.org/dora/)), Leiden Manifesto, and Metric Tide reports. + +* Next generation metrics (aka altmetrics), responsible metrics use and peer review. + +* Role of metrics in research evaluation, funding, promotion, signalling and reporting. + +* Differentiating between impact and attention. + +### Who to involve: + +* Individuals: Nikolaus Kriegeskorte, Irene Hames, Tony Ross-Hellauer, Peter Kraker, Michael Markie, Sabina Alam, Elizabeth Gadd, William Gunn. + +* Organisations: OpenAIRE, ScienceOpen, [Publons](https://publons.com/home/), [PubPeer](https://pubpeer.com/), [OpenUP](http://openup-h2020.eu/contact/), Altmetric, ImpactStory, BioMed Central, Frontiers, eLife, [PEERE](http://www.peere.org/). + +* Other: Editorial staff at journals offering traditional peer review. + +### Key resources: + +**Tools** + +* [Peer review report template](https://www.authorea.com/templates/peer_review_report_template), Authorea. + +* [Eigenfactor](http://www.eigenfactor.org/index.php) project. + +* [Publons Academy](https://publons.com/community/academy). + +* [Metrics Toolkit](http://www.metrics-toolkit.org/). + +* [Open Badges](https://openbadges.org/). + +**Research Articles and Reports** + +* [Why the impact factor of journals should not be used for evaluating research](http://europepmc.org/backend/ptpmcrender.fcgi?accid=PMC2126010&blobtype=pdf) (Seglen, 1997). + +* [Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial](http://www.bmj.com/content/318/7175/23) (van Rooyen et al., 1999). + +* [A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0014331) (Bornmann et al., 2010). + +* [Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial](http://www.bmj.com/content/341/bmj.c5729) (van Rooyen et al., 2010). + +* [Open peer review: A randomised controlled trial](https://www.cambridge.org/core/journals/the-british-journal-of-psychiatry/article/open-peer-review-a-randomised-controlled-trial/1F81447FC67B3BAFDCCCCE82B6C7A187) (Walsh et al., 2010). + +* [Deep impact: unintended consequences of journal rank](https://www.frontiersin.org/articles/10.3389/fnhum.2013.00291/full) (Brembs et al., 2013). + +* [Excellence by Nonsense: The Competition for Publications in Modern Science](https://link.springer.com/chapter/10.1007/978-3-319-00026-8_3) (Binswanger, 2014). + +* [Attention! A study of open access vs non-open access articles](https://figshare.com/articles/Attention_A_study_of_open_access_vs_non_open_access_articles/1213690) (Adie, 2014). + +* [Publishing: Credit where credit is due](http://www.nature.com/news/publishing-credit-where-credit-is-due-1.15033) (Allen et al., 2014). + +* [The Metric Tide](https://responsiblemetrics.org/the-metric-tide/) report (Wilsdon et al., 2015). + +* [Grand challenges in altmetrics: heterogeneity, data quality and dependencies](https://arxiv.org/abs/1603.04939) (Haustein, 2016). + +* [Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency](http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002456) (Kidwell et al., 2016). + +* [A framework to monitor open science trends in the EU](https://www.oecd.org/sti/063%20-%20OECD%20Blue%20Sky%202016_Open%20Science.pdf) (Smith et al., 2016). + +* [Peer Review Survey 2015: Key Findings](http://publishingresearchconsortium.com/index.php/134-news-main-menu/prc-peer-review-survey-2015-key-findings/172-peer-review-survey-2015-key-findings) (Mark Ware Consulting, 2016). + +* [Point of View: How open science helps researchers succeed](https://elifesciences.org/articles/16800) (McKiernan et al., 2016). + +* [Peer Review Quality and Transparency of the Peer-Review Process in Open Access and Subscription Journals](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0147913) (Wicherts, 2016). + +* [Next-generation metrics: Responsible metrics and evaluation for open science](https://ec.europa.eu/research/openscience/pdf/report.pdf) (European Commission, 2017). + +* [Evaluation of Research Careers fully acknowledging Open Science Practices: Rewards, incentives and/or recognition for researchers practicing Open Science](https://ec.europa.eu/research/openscience/pdf/os_rewards_wgreport_final.pdf) (European Commission, 2017). + +* [Research: Gender bias in scholarly peer review](https://elifesciences.org/articles/21718) (Helmer et al., 2017). + +* ["Excellence R Us": university research and the fetishisation of excellence](https://www.nature.com/articles/palcomms2016105) (Moore et al., 2017). + +* [Metrics for openness](https://researchcommons.waikato.ac.nz/handle/10289/10842) (Nichols and Twidale, 2017). + +* [What is open peer review? A systematic review](https://f1000research.com/articles/6-588/v2) (Ross-Hellauer, 2017). + +* [Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers](http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0189311) (Ross-Hellauer et al., 2017). + +* [A multi-disciplinary perspective on emergent and future innovations in peer review](https://f1000research.com/articles/6-1151/v3) (Tennant et al., 2017). + +* [Reviewer bias in single- versus double-blind peer review](http://www.pnas.org/content/114/48/12708) (Tomkins et al., 2017). + +* [Prestigious science journals struggle to reach even average reliability](https://www.frontiersin.org/articles/10.3389/fnhum.2018.00037/full) (Brembs, 2018). + +* [Making research evaluation more transparent: Aligning research philosophy, institutional values, and reporting](https://psyarxiv.com/48qux/) (Dougherty et al., 2018). + +* [Research excellence indicators: time to reimagine the ‘making of’?](https://academic.oup.com/spp/advance-article/doi/10.1093/scipol/scy007/4858431) (Ferretti et al., 2018). + +* [The Journal Impact Factor: A brief history, critique, and discussion of adverse effects](https://arxiv.org/abs/1801.08992) (Lariviere and Sugimoto, 2018). + +**Key posts** + +* [Six essential reads on peer review](http://asapbio.org/six-essential-reads-on-peer-review), ASAPbio. + +* [Peer reviews are open for registering at Crossref](https://www.crossref.org/blog/peer-reviews-are-open-for-registering-at-crossref/), Jennifer Lin. + +* [Why we don’t sign our peer reviews](http://www.molecularecologist.com/2014/04/why-we-dont-sign/), Jeremy Yoder. + +* [The Fractured Logic of Blinded Peer Review in Journals](http://blogs.plos.org/absolutely-maybe/2017/10/31/the-fractured-logic-of-blinded-peer-review-in-journals/), Hilda Bastian. + +* [The peer review process: challenges and progress](https://www.editage.com/insights/the-peer-review-process-challenges-and-progress), Irene Hames. + +* [Responsible metrics: Where it’s at?](https://thebibliomagician.wordpress.com/2018/02/16/responsible-metrics-where-its-at/), Lizzie Gadd. + +* [Goodhart’s Law and why measurement is hard](https://www.ribbonfarm.com/2016/06/09/goodharts-law-and-why-measurement-is-hard/), David Manheim. + +* [Academe’s prestige problem: We’re all complicit in perpetuating a rigged system](https://www.chronicle.com/article/Academe-s-Prestige-Problem/241432), Maximillian Alvarez. + +* [Let’s move beyond the rhetoric: it’s time to change how we judge research](https://www.nature.com/articles/d41586-018-01642-w), Stephen Curry. + +* [Blockchain offers a true route to a scholarly commons](https://www.researchresearch.com/news/article/?articleId=1373351), Lambert Heller. + +**Other** + +* [Metrics and Research Assessment](https://www.scienceopen.com/search#collection/78c15291-27e3-493a-99ec-7e5a00387745), ScienceOpen collection. + +* [The Open Access Citation Advantage](https://www.scienceopen.com/search#collection/996823e0-8104-4490-b26a-f2f733f810fb), ScienceOpen collection. + +* [Citation Behaviour and Practice](https://www.scienceopen.com/search#collection/2d601af5-aa90-4c63-9a11-85f2dc768868), ScienceOpen collection. + +* [Scholarly Publication Practices and Impact Factor Calculation and Manipulation](https://www.scienceopen.com/search#collection/e4870106-eea5-4ba3-88cf-e769c7d49ebe), ScienceOpen collection. + +* [Peer Review in the Age of Open Science](https://www.slideshare.net/OpenAIRE_eu/peer-review-in-the-age-of-open-science), Tony Ross-Hellauer, 2017. + +* [The San Francisco Declaration on Research Assessment](http://www.ascb.org/dora/) (DORA) and [Leiden Manifesto](http://www.leidenmanifesto.org/). + +* The [Humane Metrics Initiative](http://humetricshss.org/about/). + +* [Open Research Badges](https://openresearchbadges.org/). + +* [OpenUpHub must reads](https://www.openuphub.eu/review/must-reads). + +* [NISO Alternative Assessment Metrics (Altmetrics) Initiative](http://www.niso.org/standards-committees/altmetrics). + +* [Snowball Metrics](https://www.snowballmetrics.com/), standardized research metrics. + +### Tasks: + +* Perform one open peer review on a paper of your choice at ScienceOpen, and get a DOI for it. + +* Integrate one peer review (pre- or post-publication) experience into Publons. + +* Use Publons journal list to check open peer review policies of journal(s) in your discipline. + +* Sign DORA in either a personal or business-level capacity. + +* Define your impact. + + * Write a personal impact statement about your research (actual or predicted). Avoid using journal titles or the journal impact factor. + + * Discover the Altmetric scores for your published items using their [bookmarklet](https://www.altmetric.com/products/free-tools/bookmarklet/). + +* Track your research impact by integrating your ORCID profile with either ScienceOpen or ImpactStory (or both). + +* Do you have a personal website? If not, now is a good time to design one and make all of the above information part of your digital profile. + +* Find out what your research department or institutes research evaluation criteria are. Have a discussion about them with your research colleagues. + + * Find out who wrote them, and ask them what evidence they used to support the criteria. + +### Learning outcomes: + +1. The researcher will be able to describe the history of peer review in the context of scholarly publishing, the criticisms levied against ‘traditional’ peer review, and the ongoing developments with Open Peer Review. + +2. The researcher will be able to use a range of post-publication review, commenting, and annotation services. + +3. The researcher will be able to describe the issues associated with the use of ‘traditional’ metrics in research evaluation, and the role that peer evaluation and ‘next-generation’ metrics (or ‘altmetrics’) play in this. + +4. The researcher will be able to use a range of services to build and demonstrate their personal research impact profile, both quantitatively and qualitatively. + +5. The researcher will become familiar with the relevant criteria for research evaluation to them, and be able to have a critical discussion about them with their colleagues and those who drafted them. + +## 8: Public Engagement with Science + +**_Rationale (_****_3 lines max_****_)_**: + +*Citizen science describes the method of engaging people outside of academia within the research process itself. **Science communication is often seen as a unidirectional process from scientist to non-scientists, but with careful and strategic engagement they can both be so much more. For this, a deeper understanding of the basics of communication and engagement and existing structures are needed, as well as the capabilities of newer channels like social media. This module will teach effective techniques for communicating your research with a wider non-academic audience, as well as engaging them with the process itself.* + +**_Learning Objectives (_****_specific_****_)_**: + +*LO8a: Understand the basic concepts and the viewpoints of different stakeholders in science communication (knowledge).* + +*LO8b: Understand the different target groups/audiences and communication channels, who to involve in what kind of communication, and how to do it strategically and with which tools (knowledge).* + +*LO8c: Develop either a citizen science program to empower non-academics interested in your research field, or a personal communication strategy to bring your research to a wider audience (tasks). * + +### Key components: + +* Basics and principles of science communication, public outreach and engagement, and their relationship to Open Science. + +* Different stakeholders and audiences in public engagement and science communication, and how to shape messages for each of them. + +* Press releases and interacting with the media. + +* Different forms of social media: + + * How and why to blog about your research. + + * Using Twitter for outreach, conferences and networking. + + * How to use video and audio for outreach. + +* How to connect with citizen science initiatives, public advocacy groups, and patient organizations in your research area. + +* How to take your research to the stage (e.g., FameLab, Science Slam, Cosy Science). + +* When sh*t hits the fan - basics in crisis communication. + +### Who to involve: + +* Individuals: Dawn Bazely, Melanie Smallman, Lou Woodley, Caren Cooper, Shannon Dosemagen, Muki Hakley, Karen James, Elodie Chabrol, Andre Lampe. + +* Organisations: [Public Labs](http://publiclabs.org), [European Citizen Science Association](https://ecsa.citizen-science.net/), + +* Other: AAAS + +### Key resources: + +**Tools** + +* [The ‘Up-Goer’ five text editor](http://splasho.com/upgoer5/) (XKCD). + +* [Open Science School](http://openscienceschool.org/). + +**Research Articles and Reports** + +* [Towards an Analytical Framework of Science Communication Models](http://doras.dcu.ie/3629/1/framework_science_comm_models.pdf) (Trench, 2008). + +* [An introduction to social media for scientists](http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001535) (Bik and Goldstein, 2013). + +* [Ten simple rules of live tweeting at scientific conferences](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003789) (Ekins and Perlstein, 2014). + +* [Crowd science: The organization of scientific research in open collaborative projects](https://www.sciencedirect.com/science/article/pii/S0048733313001212) (Franzoni and Sauermann, 2014). + +* [Why did the proton cross the road? Humour and science communication](http://bura.brunel.ac.uk/handle/2438/10338) (Reisch, 2014). + +* [Science communication as political communication](http://www.pnas.org/content/111/Supplement_4/13585.short) (Scheufele, 2014). + +* [Why should we promote public engagement with science?](http://journals.sagepub.com/doi/abs/10.1177/0963662513518154) (Stilgoe et al., 2014). + +* [Bridging science education and science communication research](http://onlinelibrary.wiley.com/doi/10.1002/tea.21202/full) (Baram-Tsabari and Osborne, 2015). + +* [Opinion: Lay summaries needed to enhance science communication](http://www.pnas.org/content/112/12/3585.short) (Kuehne and Olden, 2015). + +* [Identifying what matters: Science education, science communication and democracy](http://onlinelibrary.wiley.com/doi/10.1002/tea.21201/full) (Lewenstein, 2015). + +* [Best practices for managing intellectual property rights in citizen science: A guide for researchers and citizen scientists](https://www.wilsoncenter.org/sites/default/files/research_brief_guide_for_researchers.pdf) (Scassa and Chung, 2015). + +* [Global change and local solutions: Tapping the unrealized potential of citizen science for biodiversity research](https://www.sciencedirect.com/science/article/pii/S0006320714004029) (Theobald et al., 2015). + +* [Emerging problems of data quality in citizen science](http://onlinelibrary.wiley.com/doi/10.1111/cobi.12706/full) (Lukyanenko et al., 2016). + +* [Youth-focused citizen science: Examining the role of environmental science learning and agency for conservation](https://www.sciencedirect.com/science/article/pii/S0006320716302051) (Ballard et al., 2017). + +* [Contribution of citizen science towards international biodiversity monitorin](https://www.sciencedirect.com/science/article/pii/S0006320716303639)g (Chandler et al., 2017). + +* [Citizen Science Terminology Matters: Exploring Key Terms](https://theoryandpractice.citizenscienceassociation.org/article/10.5334/cstp.96/) (Eitzel et al., 2017). + +* [Leveraging the power of place in citizen science for effective conservation decision making](https://www.sciencedirect.com/science/article/pii/S0006320716302841) (Newman et al., 2017). + +* [Austrian Citizen Science Conference 2017: Expanding Horizons](https://www.researchgate.net/profile/Susanne_Hecker/publication/322721621_Storytelling_in_Citizen_Science_-_Potential_for_Science_Communication_and_Practical_Guideline/links/5a6b28eaa6fdcc2aedee7713/Storytelling-in-Citizen-Science-Potential-for-Science-Communication-and-Practical-Guideline.pdf#page=26). + +* [Setting up crowd science projects](http://journals.sagepub.com/doi/full/10.1177/0963662516678514) (Scheliga et al., 2016). + +**Key posts:** + +* [Public-friendly Open Science](https://www.authorea.com/users/2/articles/50890-public-friendly-open-science/_show_article), Matteo Cantiello. + +* [Guidelines for good science Public Relations](https://www.wissenschaft-im-dialog.de/fileadmin/user_upload/Ueber_uns/Gut_Siggen/Dokumente/Guidelines_for_good_science_PR_final.pdf), Wissenschaft im Dialog and the Bundesverband Hochschulkommunikation. + +* [Effective communication, better science](https://blogs.scientificamerican.com/guest-blog/effective-communication-better-science/), Mónica I. Felíu-Mójer. + +* [Five strategies for identifying an audience](http://www.cdnsciencepub.com/blog/five-strategies-for-identifying-an-audience.aspx), Lisa Willemse. + +* [Policy: The art of science advice to government](https://www.nature.com/news/policy-the-art-of-science-advice-to-government-1.14838), Peter Gluckman. + +* [Guidance: Principles of scientific advice to government](https://www.gov.uk/government/publications/scientific-advice-to-government-principles/principles-of-scientific-advice-to-government), Government Office for Science (UK). + +* [Getting started with science communication](https://www.altmetric.com/blog/getting-started-with-science-communication/), Sam Illingworth. + +**Other:** + +* [European Citizen Science Association](http://ecsa.citizen-science.net/) (ECSA). + + * [10 Principles of Citizen Science](https://ecsa.citizen-science.net/engage-us/10-principles-citizen-science). + +* [Alan Alda Center for Communicating Science](http://www.centerforcommunicatingscience.org). + +* [Open Science Monitor, Citizen Science](https://ec.europa.eu/research/openscience/index.cfm?pg=citizen§ion=monitor) (European Commission). + +* [Cientópolis](https://www.cientopolis.org) (Citizen science and open science platform for Latin America). + +* [AAAS Community Engagement Fellows Program](https://www.aaas.org/cefp/about). + +* [Citizen Science & Open Science: Synergies & Future Areas of Work](https://ecsa.citizen-science.net/sites/default/files/ditos-policybrief3-20180208-citizen_science_and_open_science_synergies_and_future_areas_of_work.pdf), Doing it Together Science (ECSA). + +* [How to effectively communication science and scientific research to a broad audience](http://www.cyto.purdue.edu/archive/Education/How_to_effectively_communicate_science.pdf) (Robinson, 2009). + +* Collection: [Plain language summaries of research](https://elifesciences.org/collections/9e8f4a49/plain-language-summaries-of-research) (Rodgers et al., 2017). + +* Crowdfunding platforms: + + * [Kickstarter](https://www.kickstarter.com/). + + * [Experiment](https://experiment.com/). + +### Tasks: + +* Search and make a short list of your institution’s people involved in outreach, PR etc. Do you know everybody important for what you do? + +* Read a press release from your institution. + + * How does it compare to the research article itself? + +* Write a blog post summarising a selection of your research papers to date. + + * Start a blog and post them! + + * If possible, connect this to your main website. + +* Respond to discussions on (social) media about your topic of research. + + * Use hashtags to find relevant conversations. + + * Who is popular in your field? What do you notice about their style of engagement? + +* Identify relevant citizen science initiatives on social media. + + * Add them to your contacts. + + * Reach out to them and open a conversation on how you could mutually benefit from shared research. + +* How would you communicate with protesters in front of your institute? + +* What policy-level consultations are open at the moment at a national level? + + * Are any of them in a discipline or topic related to yours? + + * If so, draft a short response based on your understanding of the relevant research. + +### Learning outcomes: + +1. The researcher will be able to identify and describe some of the major different types of audience and stakeholder involved in science communication, what their needs and viewpoints are, and the importance of citizen science and public engagement with science. + +2. By working either individually or their research group, each researcher will be able to use a range of communication channels, including social media, to strategically engage different types of audience with their research. + +3. If there are relevant policy-related issues to their discipline, the researcher will be able to engage with them through available channels and make sure that their research field is appropriately represented. + +4. Each researcher will be able to identify relevant press/communication contacts at their institute, and be able to convey to them why their research is of importance for wider dissemination. + +5. The researcher will be able to write a blog post or non-specialist summary about either their own research or research that they are familiar with, and communicate this to wider non-academic audiences. + +## 9: Open Educational Resources + +**_Rationale (_****_3 lines max_****_)_**: + +*Open Educational Resources (OERs) are freely accessible, openly licensed materials for teaching and learning, and represent a paradigm shift compared to traditional methods of education. They are intrinsically related to developments in Open Science, due to the wider implications of access to knowledge in education in our global societies. This module will provide an understanding of the motivations behind OERs and how to develop your own.* + +**_Learning Objectives (_****_specific_****_)_**: + +*LO9a: Understand the driving forces and motivation behind the OER movement (knowledge).* + +*LO9b: Be able to openly license your research to enable educational re-use, or create your own educational resources (task).* + +### Key components: + +* Definition and scope of Open Educational Resources (OER), including aspects of resource licensing and re-use. + +* The ‘five Rs’ of OER: Retain, Re-use, Revise, Remix, Redistribute. + +* Motivations behind OER movement, including lower costs and increasing accessibility to education, and the role of institutional/organizational support. + +* OER repositories (national and others), and some of the major OER initiatives. + +* Principles of open pedagogy/andragogy. + +* The impact of OER on sustainable development, economic growth, social inclusion, and environmental conservation. + +* How OER can influence policy development at national and institutional levels through capacity building and social mobility. + +### Who to involve: + +* Individuals: Rajiv Jhangiani, Beck Pitt, Nicole Allen, Dawn Bazely. + +* Organisations: [United Nations Educational, Scientific and Cultural Organization](https://en.unesco.org/) (UNESCO), [OER Commons](https://www.oercommons.org/), [Organization for Economic Co-operation and Development](http://www.oecd.org/) (OECD), [Commonwealth of Learning](https://www.col.org/), SPARC, [Wikimedia](https://wikimedia.de/)[ Deutschland](https://wikimedia.de/), [Núcleo REA](http://www.nucleorea.ei.udelar.edu.uy/) (Recursos Educativos Abiertos, Uruguay); [Go_GN](https://go-gn.net/) (Global OER Graduate Network); [Opening Up Slovenia](http://www.ouslovenia.net/); [OER Info](https://open-educational-resources.de/) (Germany); the [Open Education Working Group](https://education.okfn.org/) (OKI), [Polish Coalition for Open Education](http://koed.org.pl/pl/english/) (KOED). + +* Other: Lots and lots of librarians. + +### Key resources: + +**Tools** + +* [Open Education Handbook](https://en.wikibooks.org/wiki/Open_Education_Handbook) (CC BY 4.0). + +* [Open Education Handsheet](https://sparcopen.org/wp-content/uploads/2017/04/Open-Education-Fact-Sheet_SPARC.11.10-2.pdf), SPARC. + +* [OER World Map](https://oerworldmap.org/). + +* [OER Commons](https://www.oercommons.org/), a public digital library of Open Educational Resources. + +* [OER Hub](http://oerhub.net/), researching the impact of Open Educational Resources. + +* [Temoa](http://www.temoa.info/), Open Educational Resources Portal. + +* Open education course directories: + + * [NovoEd](https://course.novoed.com/courses/), [Class Central](https://www.class-central.com/), [OpenLearn](http://www.open.edu/openlearn/), [OpenHPI](https://open.hpi.de/?locale=en), [OpenMed](http://openmedproject.eu/), [Open2Study](https://www.open2study.com/), [OpenClassrooms](https://openclassrooms.com/). + +**Research Articles and Reports** + +* [Open Educational Resources: Opportunities and challenges](http://www.oecd.org/education/ceri/37351085.pdf) (Hylén, 2005). + +* [Models for sustainable Open Educational Resources](https://www.learntechlib.org/p/44796/) (Downes, 2007). + +* [Giving knowledge for free: The emergence of Open Educational Resources](http://www.oecd.org/education/ceri/38654317.pdf) (OECD, 2007). + +* [Open content and Open Educational Resources: Enabling universal education](http://www.irrodl.org/index.php/irrodl/article/view/469) (Caswell et al., 2008). + +* [Linking open course wares and open education resources: creating an effective search and recommendation system](https://www.sciencedirect.com/science/article/pii/S187705091000325X) (Shelton et al., 2010). + +* Evaluating Open Educational Resources: Lessons learned (DeVries, 2013). + +* [Open Educational Resources](https://www.sciencedirect.com/science/article/pii/S1877042813032862) (Marcus-Quinn and Diggins, 2013). + +* [State of the art review of quality issues related to Open Educational Resources (OER)](http://www.pedocs.de/frontdoor.php?source_opus=9101) (Camilleri et al., 2014). + +* [Open Data as Open Educational Resources: Case studies of emerging practice](https://education.okfn.org/files/2015/11/Book-Open-Data-as-Open-Educational-Resources1.pdf) (Atenas and Havemann, 2015). + +* [Open data as Open Educational Resources: Towards transversal skills and global citizenship](http://www.openpraxis.org/~openprax/index.php/OpenPraxis/article/view/233) (Atenas et al., 2015). + +* [Multimedia resources as examples of polymorphic educational hypertexts in the post-literacy era](https://www.sciencedirect.com/science/article/pii/S1877042815060346) (Goodova et al., 2015). + +* [The Power of the Three Words and One Acronym: OER vs OER: Subtitle: I’m not an Ogre of the Enchanted Realm (of cyberspace). I’m an Omnipresent Educational Rescuer (because I use the OER!)](https://www.sciencedirect.com/science/article/pii/S1877042815027299), (Holotescu et al., 2015). + +* [Open Educational Resources development model for an inquiring cultural skill of Higher Education students](https://www.sciencedirect.com/science/article/pii/S1877042815009246) (Kaosaiyaporn et al., 2015). + +* [The use of Open Educational Resources in online learning: A study of students’ perception](https://eric.ed.gov/?id=EJ1092848) (Meirani, 2015). + +* T[he global information educational resources: Methodological issues](https://www.sciencedirect.com/science/article/pii/S1877042815028645) (Nail et al., 2015). + +* [From vision to action - A strategic planning process model for Open Educational Resources](https://www.sciencedirect.com/science/article/pii/S1877042815011623) (Shu-Hsiang et al., 2015). + +* [Open Educational Resources: American ideals, global questions](http://ger.mercy.edu/index.php/ger/article/view/128) (Weiland, 2015). + +* [Not all rubrics are equal: A review of rubrics for evaluating the quality of Open Educational Resources](http://www.irrodl.org/index.php/irrodl/article/view/2389) (Yuan and Recker, 2015). + +* [A Basic Guide to Open Educational Resources](http://oasis.col.org/handle/11599/36) (Commonwealth of Learning, 2015). + +* [MOOCs as disruptive technologies: Strategies for enhancing the learner experience and quality of MOOCs](http://www.um.es/ead/red/50/conole.pdf) (Conole, 2016). + +* [Use of Open Educational Resources: How, why and why not?](https://eric.ed.gov/?id=EJ1111136) (Islim et al., 2016). + +* [Open Educational Resources: Policy, costs and transformation](http://unesdoc.unesco.org/images/0024/002443/244365e.pdf) (Miao et al., 2016). + +* [OER in and as MOOCs](https://zenodo.org/record/161287#.WowNvoPwaM_) (Czerniewicz et al., 2017). + +* [Policy Approaches to Open Education - Case Studies from 28 EU Member States (OpenEdu Policies)](https://ec.europa.eu/jrc/en/publication/policy-approaches-open-education-case-studies-28-eu-member-states-openedu-policies), (European Commission, 2017). + +* [Open Educational Resources as a diver for manufacturing-related education for learning of sustainable development](https://www.sciencedirect.com/science/article/pii/S2351978917300148) (Roeder et al., 2017). + +* [Open pathways to student success: Academic library partnerships for Open Educational Resource and affordable content creation adoption](https://www.sciencedirect.com/science/article/pii/S0099133316301409) (Salem Jr., 2017). + +**Key posts** + +* [Foundations for OER Strategy Development](http://www.oerstrategy.org/home/read-the-doc/) (Allen et al., 2015). + +* [What are Open Educational Resources](http://www.unesco.org/new/en/communication-and-information/access-to-knowledge/open-educational-resources/what-are-open-educational-resources-oers/) (OERs)?, UNESCO. + +* [Open Educational Resources and Creative Commons](https://blog.eliademy.com/2015/03/16/oer/#.WoteXoPwaM8), Eliademy. + +**Other** + +* [Introduction to OER](https://pitt.libguides.com/openeducation), University of Pittsburgh. + +* [BC Campus OpenEd](https://open.bccampus.ca/). + +* [Wikimedia](https://www.wikimedia.org/), a global movements whose mission is to bring free educational content to the world. + +* [Discovering Open Educational Resources (OER): Home](http://guides.temple.edu/OER), Temple University. + +* [The Cape Town Open Education Declaration](http://www.capetowndeclaration.org/read-the-declaration): Unlocking the promise of Open Educational Resources. + +* The [2012 Paris OER Declaration](http://www.unesco.org/fileadmin/MULTIMEDIA/HQ/CI/WPFD2009/English_Declaration.html). + +* [Open Educational Resources](https://www.hewlett.org/strategy/open-educational-resources/), Hewlett Foundation. + +* [Open Educational Resources](https://web.archive.org/web/20130503135718/http://www.col.org/resources/crsMaterials/Pages/OCW-OER.aspx), Commonwealth of Learning. + + * [Global Report 2017](http://oasis.col.org/handle/11599/2788). + +* [Definition of OER](http://www.opencontent.org/definition/), OpenContent. + +* [MERLOT](https://www.merlot.org/merlot/index.htm), Multimedia Educational Resource for Learning and Online Teaching. + +* Open textbook initiatives: + + * [Unglue it](https://unglue.it/), [FreeBooks4Doctors](http://www.freebooks4doctors.com/), [InTech Open](https://www.intechopen.com/), [Bookboon](http://bookboon.com/), [BC Campus OpenEd](https://open.bccampus.ca/find-open-textbooks/), [E-books Directory](http://www.e-booksdirectory.com/), [Directory of Open Access Books](https://www.doabooks.org/), [Wikibooks](https://en.wikibooks.org/wiki/Main_Page/), [UCL Press books](http://www.ucl.ac.uk/ucl-press/browse-books/), [Saylor Academy Open Textbooks](https://www.saylor.org/books/), [Potto Project](http://www.potto.org/), [Open Library](https://openlibrary.org/) (Internet Archive), [Openstax](https://openstax.org/), [Open Textbooks](https://textbooks.opensuny.org/) (SUNY), [Open Textbook Library](http://www.open.umn.edu/opentextbooks/). + + * [LibreTexts](http://www.libretexts.org/). + + * [Biology](https://bio.libretexts.org/). + + * [Chemistry](https://chem.libretexts.org/). + + * [Geosciences](https://geo.libretexts.org/). + + * [Mathematics](https://math.libretexts.org/). + +### Tasks: + +* Create a Wikipedia account. + + * Integrate one or more of your research articles (or someone else’s) into Wikipedia. + + * Make sure to link to an Open Access version if possible. + +* Make some of your research outputs or teaching materials openly available. + + * Remember to choose an appropriate repository. + + * Make sure the content is openly licensed and granted a DOI. + +### Learning outcomes: + +1. The researcher will be able to convey the motivations behind the OER movement, and the relationship that this has with Open Science. + +2. The researcher will be able to identify and implement the steps to either prepare content for educational re-use purposes, or be able to design their own OER. + +3. The researcher will be able to either identify relevant places where their research can be integrated into Wikipedia, or integrate it themselves if they are a user. + +## 10: Open Advocacy + +**_Rationale (_****_3 lines max_****_)_**: + +*Now that you are an expert at applying Open Science at each step of **your r**esearch lifecycle, here are some basics on becoming a pro-active ambassador for open scholarship in any discipline. This module will teach you how to effectively engage researchers and other stakeholders in scholarly communication with the various aspects of Open Science.* + +**_Learning Objectives (_****_specific_****_)_**: + +*LO10a: To understand the needs of different stakeholder groups in scholarly communication, and the impact that Open Science can have on them.* + +*LO10b: To be able to translate your knowledge into an effective program or tool for external engagement.* + +### Key components: + +* What does it mean to be an ‘advocate’ for Open Science. + +* Advocating for your own rights as an author. + +* The basic steps for achieving local culture change (e.g., [Kotter’s 8-step change model of management](https://study.com/academy/lesson/kotters-8-step-change-model-of-management.html)). + +* Advocating to your peers, including writing letters and articles advocating for Open Science. + +* Talking to journal editors - catalysing the Open Access conversation within your field. + +* Talking to policymakers about Open Science. + +* Building or joining an Open Science community. + +* Effective leadership and training in Open Science, and empowering others to make change. + +### Who to involve: + +* Individuals: Josh Bolick (and colleagues, see the rebuttal article below), Johan Rooryck, April Clyburne-Sherin, Nick Shockey, Joseph McArthur, Heather Joseph, Nicole Allen, Erin McKiernan. + +* Organisations: R2RC, SPARC, Creative Commons, [IGDORE](https://igdore.org/). + +* Other: Country specific advocacy groups (e.g., [AOASG](https://aoasg.org.au/), Open Access Nigeria), OpenCon. + +### Key resources: + +**Tools** + +* [Why Open Research?](http://whyopenresearch.org/)** **(Erin Mckiernan). + +* [Open Speakers Database](https://openspeakers.org/), a crowdsourced database of regional experts on Open Access, Open Education and Open Data. + +* [Women Working in Openness database](https://fyoaw.vickysteeves.com/), Vicky Steeves. + +* [Starting Open Projects From Scratch](https://docs.google.com/document/d/1qSXBZa3-uBKdkFCkukt5lxRsYoREWNYf0_2OpOnh3mQ/edit?usp=sharing) (CC0, Crowdsourced by OpenCon attendees). + +* [Open Research Advocacy Train-the-Trainer](https://docs.google.com/document/d/1IYIqlFT7QSz4DPWcwsQSIzGRLk-T_SBejxqypZRCmi0/edit?usp=sharing) (CC0, by April Clyburne-Sherin). + +* [Train the Trainer](https://github.com/TrainTheTrainer/EXCELERATE-TtT) workshop, Allegra Via and Patricia Palagi. + +* Leiden University Centre for Innovation [toolkit](https://www.centre4innovation.org/innovation-works/use-our-toolkit). + +* [Open Science Leadership Workshop](https://github.com/mozillascience/open-science-leadership-workshop), Mozilla Science Lab. + +* [Advocating for transparency policies - a toolkit for researchers, staff, and librarians ](https://github.com/AllTrialsUSA/FSCI2017/blob/master/Transparency-advocacy-toolkit.md)(FSCI2017). + +* [Advocating Open Access - a toolkit for librarians and research support staff](http://blogs.ucl.ac.uk/open-access/files/2015/06/Advocacy-toolkit.pdf) (UCL). + +* [Making an Impact with Open Science](https://ocw.tudelft.nl/courses/making-impact-open-science/), TU Delft course. + +* [SPARC author addendum](https://sparcopen.org/our-work/author-rights/ ), to help advocate for your own rights as an author with a scholarly journal. + +* [Open Science course](https://github.com/punkish/open-science-course), Puneet Kishoor (CC0). + +**Research Articles and Reports** + +* [Open letter to the American Association for the Advancement of Science](https://thewinnower.com/papers/45-open-letter-to-the-american-association-for-the-advancement-of-science) (Tennant et al., 2014). + +* [When will ‘open science’ simply become ‘science’?](https://genomebiology.biomedcentral.com/articles/10.1186/s13059-015-0669-2) (Watson, 2015). + +* [How open science helps researchers succeed](https://elifesciences.org/articles/16800) (McKiernan et al., 2016). + +* [How open access is crucial to the future of science](https://kuscholarworks.ku.edu/handle/1808/22672?utm_content=buffer40244&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer) (Bolick et al., 2017). + +* [Effective Practices and Strategies for Open Access Outreach: A Qualitative Study](https://jlsc-pub.org/articles/abstract/10.7710/2162-3309.2216/) (Dawson, 2018). + +**Key posts** + +* [An early career researcher’s view on modern and open scholarship](https://lgatto.github.io/EPFL-open-science/), Laurent Gatto. + +* [Lingua is dead. Long live Glossa!](http://languagelog.ldc.upenn.edu/nll/?p=22162), Eric Baković. + +* [The Open Access movement and activism for the knowledge commons](http://www.asanet.org/sites/default/files/savvy/footnotes/mayjun14/asaforum_0514.html), Jackie Smith. + +* [Defending the global knowledge commons](https://www.opendemocracy.net/jackie-smith/defending-global-knowledge-commons), Jackie Smith. + +**Other** + +* [Ask ](https://ask-open-science.org/)[Open Science](https://ask-open-science.org/). + +* [IGDORE](https://igdore.org/), Institute for Globally Distributed Open Research and Education. + +* [Berlin Open Science and Reproducible Research Meetup](https://www.meetup.com/Berlin-Open-Science-Meetup/). + +* National Open Science Plans: + + * [National Plan Open Science](https://www.openscience.nl/) (Netherlands). + + * [Ciência Aberta](http://www.ciencia-aberta.pt/) (Portugal). + + * [Open Science and Research](https://openscience.fi/) (Finland). + +* [Open science guides: How universities promote putting it into practice](https://www.zbw-mediatalk.eu/en/2017/08/anleitung-fur-open-science-wie-universitaten-die-umsetzung-in-der-praxis-fordern/), ZBW Mediatalks. + +* [Challenges and strategies for the success of Open Science](https://www.fosteropenscience.eu/content/challenges-and-strategies-success-open-science), FOSTER. + +* [Open science and its advocacy](https://www.fosteropenscience.eu/content/open-science-and-its-advocacy) (FOSTER). + +* [FOSTER Plus: Supporting the practical adoption of Open Science](http://libereurope.eu/our-activities/projects/fosterplus/) (LIBER). + +* [Strengthen Advocacy](http://sites.path.org/advocacyandpolicy/how-we-do-it/strengthen-advocacy-capacity/) Capacity (PATH). + +* [OpenCon blog posts](http://www.opencon2017.org/resources_misc?utm_campaign=feb_march_cc&utm_medium=email&utm_source=righttoresearch) from participants. + +* [Geo for All](https://www.osgeo.org/initiatives/geo-for-all/in-your-classroom/) teaching resources. + +* [Retraction Watch](http://retractionwatch.com/). + +* Tag libraries from the [Open Access Tracking Project](https://cyber.harvard.edu/hoap/Open_Access_Tracking_Project) (OATP): + + * [Oa.best_practices](http://tagteam.harvard.edu/hubs/oatp/tag/oa.best_practices); [oa.data](http://tagteam.harvard.edu/hubs/oatp/tag/oa.data); [oa.ecr](http://tagteam.harvard.edu/hubs/oatp/tag/oa.ecr); [oa.incentives](http://tagteam.harvard.edu/hubs/oatp/tag/oa.incentives); [oa.obstacles](http://tagteam.harvard.edu/hubs/oatp/tag/oa.obstacles); [oa.open_science](http://tagteam.harvard.edu/hubs/oatp/tag/oa.open_science); [oa.policies](http://tagteam.harvard.edu/hubs/oatp/tag/oa.policies); [oa.reproducibility](http://tagteam.harvard.edu/hubs/oatp/tag/oa.reproducibility); [oa.south](http://tagteam.harvard.edu/hubs/oatp/tag/oa.south); [oa.stem](http://tagteam.harvard.edu/hubs/oatp/tag/oa.stem). + + * Each one grows in real time; each one is available in HTML, RSS, Atom, and JSONP; each one is open to additions from anyone, and the project welcomes volunteer taggers; there are similar feeds on disciplines (e.g., oa.anthropology, oa.biology, oa.chemistry) and countries and regions (e.g., oa.africa, oa.brazil, oa.china); these are just a few of hundreds of OATP feeds. + +### Tasks: + +* Write a letter to your local political representative about why you think research is important. + + * What do politicians care about? Use this context to empathise with them and deliver your message effectively. + +* Send an email to an editor (or editorial board) of a ‘closed access’ journal in your field to start the OA conversation. + + * Is the journal OA already (or ‘hybrid’)? How much are APCs for it? Are there any cheaper alternatives they might not be aware of? + +* Does your research institute have a magazine, forum, or newsletter? Write a letter/post for it in support of Open Science. + +* Draft your own email template reply for requests to peer review about how you only review for OA journals. + + * Re-use/base it on ones out there already. What has worked or not worked so well in the past? + +* Outline opportunity costs for your university administrators on role of Open Science in hiring, tenure and promotion guidelines. + + * Outline concrete solutions and benefits Open Science can deliver for current headaches university administrators may struggle with. + +* Find your local Open Science advocacy group and volunteer for them! + + * Does one not exist yet? Why not start one! Local groups are a great way to meet like-minded individuals and work together. + + * Having a webpage like Meetup can help keep people engaged and aware of meetings. + +### Learning outcomes: + +1. The researcher will gain an appreciation of, and be able to identify, the diversity of different communities and stakeholders in scholarly communication, and the potential impact that Open Science can have on them. + +2. The researcher will become an effective leader in Open Science, and use their skills and knowledge to empower others. + +3. By working either alone or with like-minded colleagues, the researcher will either join or establish a local open science advocacy group or meetup, and identify concrete action steps that they can take together. + +4. The research will prepare an open science statement to distribute to administrative staff at their research institute, as well as any other relevant local stakeholders. + +5. Together with like-minded colleagues, the researcher will start the ‘open access conversation’ with the editorial board of a relevant journal in their field. + +# Communications Strategy + +Open Science MOOC platform: **[https://opensciencemooc.eu**/](https://opensciencemooc.eu/) + +Two-phase release strategy: + +(1) Open review of final draft by Graduate Schools and professional societies aiming for a good discipline coverage, and use the review process as part of the promotion; + +(2) Full public release and promotion as below. + +* Mailing lists: + + * Open Science. + + * And any regional variants. + + * OpenCon. + + * Open Access. + + * Psci Comm. + +* Social Media + + * Twitter blitz: + + * Dedicated Twitter handle ([@OpenSci_MOOC](https://twitter.com/OpenSci_MOOC/)). + + * Thunderclap. + + * Dedicated Facebook page. + + * Dedicated Slack/Gitter channel for all participants ([contact](mailto:jon.tennant.2@gmail.com) to join). + + * Suggestions for a [communication plan](https://docs.google.com/presentation/d/1Qp9nZDGLyVxa77mjMNp-jbAfiovWBk2guoYKhA65FTU/edit?usp=sharing), and a[ template](https://docs.google.com/document/d/1Y159-WI2jLdW1Uby67o0k3HztWufgUDSxMpXtkBp21I/edit?usp=sharing) from Leiden University. + +* Non-social media + + * University mailing lists. + + * Assets: EURAXESS, EURODOC. + + * National Open Science contact points (OpenAIRE NOADs = [National Open Access Desks ](https://www.openaire.eu/contact-noads)). + + * National science communication organizations. + + * [Etherpad](https://public.etherpad-mozilla.org/p/OpenSci_MOOC) for collaborative note taking. + +## Messaging + +For students: + +* These skills will get you a job both inside and outside of academia. + +* These skills will save you time during your research. + +* These skills will make your output of better quality. + +* These skills make peer review of your papers more efficient. + +For lab-heads and institutions: + +* Make collaboration in the workplace better and faster. + +* Continuing the work of alumni will be easier (information transfer, patenting). + +* Openness often implies more citations and attention for research. + +* Encourages integration of science within society, increases societal impact of research. + +For policymakers: + +* Good for researchers as it teaches them core competencies and transferable skills to be used outside of academia. + +* Contributes to innovation and economic growth. + +* Contributes to a healthier society. + +* Helps them meet policy objectives. + +For librarians: + +* Can be integrated with graduate school training programs at zero cost to them. + +* Helps train researchers/students in tasks that will ultimately make their jobs easier. + +* Will create a new knowledge pool that they can draw upon if needed. + +For publishers: + +* ... + diff --git a/reading_material/1 - Open Principles/Aalbersberg et al., 2018.docx b/reading_material/1 - Open Principles/Aalbersberg et al., 2018.docx new file mode 100644 index 0000000..0156ee2 Binary files /dev/null and b/reading_material/1 - Open Principles/Aalbersberg et al., 2018.docx differ diff --git a/reading_material/1 - Open Principles/Aalbersberg et al., 2018.pdf b/reading_material/1 - Open Principles/Aalbersberg et al., 2018.pdf new file mode 100644 index 0000000..f63cce6 Binary files /dev/null and b/reading_material/1 - Open Principles/Aalbersberg et al., 2018.pdf differ diff --git a/reading_material/1 - Open Principles/Ali-Khan et al., 2018.pdf b/reading_material/1 - Open Principles/Ali-Khan et al., 2018.pdf new file mode 100644 index 0000000..eccba89 Binary files /dev/null and b/reading_material/1 - Open Principles/Ali-Khan et al., 2018.pdf differ diff --git a/reading_material/1 - Open Principles/Bartling and Friesike, 2014.pdf b/reading_material/1 - Open Principles/Bartling and Friesike, 2014.pdf new file mode 100644 index 0000000..5ec6f38 Binary files /dev/null and b/reading_material/1 - Open Principles/Bartling and Friesike, 2014.pdf differ diff --git a/reading_material/1 - Open Principles/Borgman, 2012.pdf b/reading_material/1 - Open Principles/Borgman, 2012.pdf new file mode 100644 index 0000000..ec3ef38 Binary files /dev/null and b/reading_material/1 - Open Principles/Borgman, 2012.pdf differ diff --git a/reading_material/1 - Open Principles/Chesborough, 2015.pdf b/reading_material/1 - Open Principles/Chesborough, 2015.pdf new file mode 100644 index 0000000..0697e3a Binary files /dev/null and b/reading_material/1 - Open Principles/Chesborough, 2015.pdf differ diff --git a/reading_material/1 - Open Principles/European Commission, 2017.pdf b/reading_material/1 - Open Principles/European Commission, 2017.pdf new file mode 100644 index 0000000..7b32c6d Binary files /dev/null and b/reading_material/1 - Open Principles/European Commission, 2017.pdf differ diff --git a/reading_material/1 - Open Principles/Fecher and Friesike, 2014.pdf b/reading_material/1 - Open Principles/Fecher and Friesike, 2014.pdf new file mode 100644 index 0000000..475e6fe Binary files /dev/null and b/reading_material/1 - Open Principles/Fecher and Friesike, 2014.pdf differ diff --git a/reading_material/1 - Open Principles/Frankenhuis and Nettle, 2018.pdf b/reading_material/1 - Open Principles/Frankenhuis and Nettle, 2018.pdf new file mode 100644 index 0000000..bb85957 Binary files /dev/null and b/reading_material/1 - Open Principles/Frankenhuis and Nettle, 2018.pdf differ diff --git a/reading_material/1 - Open Principles/Grigorov et al., 2015.pdf b/reading_material/1 - Open Principles/Grigorov et al., 2015.pdf new file mode 100644 index 0000000..2edf6d3 Binary files /dev/null and b/reading_material/1 - Open Principles/Grigorov et al., 2015.pdf differ diff --git a/reading_material/1 - Open Principles/Haak et al., 2012.pdf b/reading_material/1 - Open Principles/Haak et al., 2012.pdf new file mode 100644 index 0000000..04201cd Binary files /dev/null and b/reading_material/1 - Open Principles/Haak et al., 2012.pdf differ diff --git a/reading_material/1 - Open Principles/Kramer and Bosman, 2018..pdf b/reading_material/1 - Open Principles/Kramer and Bosman, 2018..pdf new file mode 100644 index 0000000..6f672f6 Binary files /dev/null and b/reading_material/1 - Open Principles/Kramer and Bosman, 2018..pdf differ diff --git a/reading_material/1 - Open Principles/LERU, 2018.pdf b/reading_material/1 - Open Principles/LERU, 2018.pdf new file mode 100644 index 0000000..0406b00 Binary files /dev/null and b/reading_material/1 - Open Principles/LERU, 2018.pdf differ diff --git a/reading_material/1 - Open Principles/Levin and Leonelli, 2016.pdf b/reading_material/1 - Open Principles/Levin and Leonelli, 2016.pdf new file mode 100644 index 0000000..5096393 Binary files /dev/null and b/reading_material/1 - Open Principles/Levin and Leonelli, 2016.pdf differ diff --git a/reading_material/1 - Open Principles/Masuzzo and Martens, 2017.pdf b/reading_material/1 - Open Principles/Masuzzo and Martens, 2017.pdf new file mode 100644 index 0000000..6d9cdfd Binary files /dev/null and b/reading_material/1 - Open Principles/Masuzzo and Martens, 2017.pdf differ diff --git a/reading_material/1 - Open Principles/Miguel et al., 2014.pdf b/reading_material/1 - Open Principles/Miguel et al., 2014.pdf new file mode 100644 index 0000000..2d80256 Binary files /dev/null and b/reading_material/1 - Open Principles/Miguel et al., 2014.pdf differ diff --git a/reading_material/1 - Open Principles/Nosek et al., 2015.pdf b/reading_material/1 - Open Principles/Nosek et al., 2015.pdf new file mode 100644 index 0000000..715f100 Binary files /dev/null and b/reading_material/1 - Open Principles/Nosek et al., 2015.pdf differ diff --git "a/reading_material/1 - Open Principles/St\303\274rmer et al., 2017.pdf" "b/reading_material/1 - Open Principles/St\303\274rmer et al., 2017.pdf" new file mode 100644 index 0000000..c493a07 Binary files /dev/null and "b/reading_material/1 - Open Principles/St\303\274rmer et al., 2017.pdf" differ diff --git a/reading_material/1 - Open Principles/Watson, 2015.pdf b/reading_material/1 - Open Principles/Watson, 2015.pdf new file mode 100644 index 0000000..10d94bc Binary files /dev/null and b/reading_material/1 - Open Principles/Watson, 2015.pdf differ diff --git a/reading_material/1 - Open Principles/White House, 2016.pdf b/reading_material/1 - Open Principles/White House, 2016.pdf new file mode 100644 index 0000000..eb62a64 Binary files /dev/null and b/reading_material/1 - Open Principles/White House, 2016.pdf differ diff --git a/reading_material/1 - Open Principles/Woelfle et al., 2011.pdf b/reading_material/1 - Open Principles/Woelfle et al., 2011.pdf new file mode 100644 index 0000000..7e0c428 Binary files /dev/null and b/reading_material/1 - Open Principles/Woelfle et al., 2011.pdf differ diff --git a/reading_material/1 - Open Principles/van der See and Reich, 2018.docx b/reading_material/1 - Open Principles/van der See and Reich, 2018.docx new file mode 100644 index 0000000..548959b Binary files /dev/null and b/reading_material/1 - Open Principles/van der See and Reich, 2018.docx differ diff --git a/reading_material/1 - Open Principles/van der See and Reich, 2018.pdf b/reading_material/1 - Open Principles/van der See and Reich, 2018.pdf new file mode 100644 index 0000000..58d3501 Binary files /dev/null and b/reading_material/1 - Open Principles/van der See and Reich, 2018.pdf differ diff --git a/reading_material/10 - Open Advocacy/Bolick et al., 2017.pdf b/reading_material/10 - Open Advocacy/Bolick et al., 2017.pdf new file mode 100644 index 0000000..b981af0 Binary files /dev/null and b/reading_material/10 - Open Advocacy/Bolick et al., 2017.pdf differ diff --git a/reading_material/10 - Open Advocacy/Dawson, 2018.pdf b/reading_material/10 - Open Advocacy/Dawson, 2018.pdf new file mode 100644 index 0000000..6a0c926 Binary files /dev/null and b/reading_material/10 - Open Advocacy/Dawson, 2018.pdf differ diff --git a/reading_material/10 - Open Advocacy/McKiernan et al., 2016.pdf b/reading_material/10 - Open Advocacy/McKiernan et al., 2016.pdf new file mode 100644 index 0000000..96e7c52 Binary files /dev/null and b/reading_material/10 - Open Advocacy/McKiernan et al., 2016.pdf differ diff --git a/reading_material/10 - Open Advocacy/Tennant et al., 2014.pdf b/reading_material/10 - Open Advocacy/Tennant et al., 2014.pdf new file mode 100644 index 0000000..3617ae2 Binary files /dev/null and b/reading_material/10 - Open Advocacy/Tennant et al., 2014.pdf differ diff --git a/reading_material/10 - Open Advocacy/Watson, 2015.pdf b/reading_material/10 - Open Advocacy/Watson, 2015.pdf new file mode 100644 index 0000000..10d94bc Binary files /dev/null and b/reading_material/10 - Open Advocacy/Watson, 2015.pdf differ diff --git a/reading_material/2 - Open Collaboration/Candela et al., 2013.pdf b/reading_material/2 - Open Collaboration/Candela et al., 2013.pdf new file mode 100644 index 0000000..2a910d4 Binary files /dev/null and b/reading_material/2 - Open Collaboration/Candela et al., 2013.pdf differ diff --git a/reading_material/2 - Open Collaboration/Carusi et al., 2010.pdf b/reading_material/2 - Open Collaboration/Carusi et al., 2010.pdf new file mode 100644 index 0000000..d84afa4 Binary files /dev/null and b/reading_material/2 - Open Collaboration/Carusi et al., 2010.pdf differ diff --git a/reading_material/2 - Open Collaboration/Cheruvelil et al., 2014.pdf b/reading_material/2 - Open Collaboration/Cheruvelil et al., 2014.pdf new file mode 100644 index 0000000..b9d52d2 Binary files /dev/null and b/reading_material/2 - Open Collaboration/Cheruvelil et al., 2014.pdf differ diff --git a/reading_material/2 - Open Collaboration/Heaton et al., 2016.pdf b/reading_material/2 - Open Collaboration/Heaton et al., 2016.pdf new file mode 100644 index 0000000..02d4e2a Binary files /dev/null and b/reading_material/2 - Open Collaboration/Heaton et al., 2016.pdf differ diff --git a/reading_material/2 - Open Collaboration/Hsiehchen et al., 2015.pdf b/reading_material/2 - Open Collaboration/Hsiehchen et al., 2015.pdf new file mode 100644 index 0000000..42a5270 Binary files /dev/null and b/reading_material/2 - Open Collaboration/Hsiehchen et al., 2015.pdf differ diff --git a/reading_material/2 - Open Collaboration/Ockwell et al., 2014.pdf b/reading_material/2 - Open Collaboration/Ockwell et al., 2014.pdf new file mode 100644 index 0000000..ebd2fc6 Binary files /dev/null and b/reading_material/2 - Open Collaboration/Ockwell et al., 2014.pdf differ diff --git a/reading_material/2 - Open Collaboration/Rodrigues et al., 2017.pdf b/reading_material/2 - Open Collaboration/Rodrigues et al., 2017.pdf new file mode 100644 index 0000000..a3894eb Binary files /dev/null and b/reading_material/2 - Open Collaboration/Rodrigues et al., 2017.pdf differ diff --git a/reading_material/2 - Open Collaboration/de Grijs, 2015.PDF b/reading_material/2 - Open Collaboration/de Grijs, 2015.PDF new file mode 100644 index 0000000..00015b7 Binary files /dev/null and b/reading_material/2 - Open Collaboration/de Grijs, 2015.PDF differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/.gitignore b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/.gitignore new file mode 100644 index 0000000..a0322db --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/.gitignore @@ -0,0 +1,220 @@ +## Core latex/pdflatex auxiliary files: +*.aux +*.lof +*.log +*.lot +*.fls +*.out +*.toc +*.fmt +*.fot +*.cb +*.cb2 + +## Intermediate documents: +*.dvi +*-converted-to.* +# these rules might exclude image files for figures etc. +# *.ps +# *.eps +# *.pdf + +## Generated if empty string is given at "Please type another file name for output:" +.pdf + +## Bibliography auxiliary files (bibtex/biblatex/biber): +*.bbl +*.bcf +*.blg +*-blx.aux +*-blx.bib +*.run.xml + +## Build tool auxiliary files: +*.fdb_latexmk +*.synctex +*.synctex(busy) +*.synctex.gz +*.synctex.gz(busy) +*.pdfsync + +## Auxiliary and intermediate files from other packages: +# algorithms +*.alg +*.loa + +# achemso +acs-*.bib + +# amsthm +*.thm + +# beamer +*.nav +*.pre +*.snm +*.vrb + +# changes +*.soc + +# cprotect +*.cpt + +# elsarticle (documentclass of Elsevier journals) +*.spl + +# endnotes +*.ent + +# fixme +*.lox + +# feynmf/feynmp +*.mf +*.mp +*.t[1-9] +*.t[1-9][0-9] +*.tfm + +#(r)(e)ledmac/(r)(e)ledpar +*.end +*.?end +*.[1-9] +*.[1-9][0-9] +*.[1-9][0-9][0-9] +*.[1-9]R +*.[1-9][0-9]R +*.[1-9][0-9][0-9]R +*.eledsec[1-9] +*.eledsec[1-9]R +*.eledsec[1-9][0-9] +*.eledsec[1-9][0-9]R +*.eledsec[1-9][0-9][0-9] +*.eledsec[1-9][0-9][0-9]R + +# glossaries +*.acn +*.acr +*.glg +*.glo +*.gls +*.glsdefs + +# gnuplottex +*-gnuplottex-* + +# gregoriotex +*.gaux +*.gtex + +# hyperref +*.brf + +# knitr +*-concordance.tex +# TODO Comment the next line if you want to keep your tikz graphics files +*.tikz +*-tikzDictionary + +# listings +*.lol + +# makeidx +*.idx +*.ilg +*.ind +*.ist + +# minitoc +*.maf +*.mlf +*.mlt +*.mtc[0-9]* +*.slf[0-9]* +*.slt[0-9]* +*.stc[0-9]* + +# minted +_minted* +*.pyg + +# morewrites +*.mw + +# nomencl +*.nlo + +# pax +*.pax + +# pdfpcnotes +*.pdfpc + +# sagetex +*.sagetex.sage +*.sagetex.py +*.sagetex.scmd + +# scrwfile +*.wrt + +# sympy +*.sout +*.sympy +sympy-plots-for-*.tex/ + +# pdfcomment +*.upa +*.upb + +# pythontex +*.pytxcode +pythontex-files-*/ + +# thmtools +*.loe + +# TikZ & PGF +*.dpth +*.md5 +*.auxlock + +# todonotes +*.tdo + +# easy-todo +*.lod + +# xindy +*.xdy + +# xypic precompiled matrices +*.xyc + +# endfloat +*.ttt +*.fff + +# Latexian +TSWLatexianTemp* + +## Editors: +# WinEdt +*.bak +*.sav + +# Texpad +.texpadtmp + +# Kile +*.backup + +# KBibTeX +*~[0-9]* + +# auto folder when using emacs and auctex +/auto/* + +# expex forward references with \gathertags +*-tags.tex diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/README.md b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/README.md new file mode 100644 index 0000000..abc4d42 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/README.md @@ -0,0 +1,36 @@ +**Documentos Dinámicos con R Markdown** + +Fernando Hoces de la Guardia + +Berkeley Initiative for Transparency in the Social Sciences + +* Breve introducción ocupando la presentación de [Thomas Leeper](http://thomasleeper.com) de RT2 London. [Presntación aquí](https://osf.io/4ah58/). + +* Para el ejercicio practico vamos a seguir el taller de Garret Christensen [en ingles más abajo]. + * Primero deben descargar y descomprimir [este archivo](https://github.com/fhoces/BITSS_SPANISH/blob/master/3-Rmarkdown/3-Rmarkdown.zip). + +* Si el tiempo alcanza podemos ver el ejemplo de mi disertación [aquí](https://rpubs.com/fhoces/dd_cbo_mw). +----- + +This directory contains several files designed to familiarize you with the basics of dynamic documents in R Studio using R Markdown and KnitR. The most important files are: + +(Don't worry about any Rproj or Rhistory files you may see. That's just R Studio being friendly and keeping track of everything in the background for you if you organize the directory as an R Project.) + +1. RMarkdownHTMLExample.Rmd is an R Markdown file for use in R. When successfully run, it will load data (the WASHpublic_mock.dta file) and run three basic regressions. It uses the stargazer package to create nicely formatted HTML regression tables in outputR.html. That table is included in the final combined output: RMarkdownHTMLExample.html. + +2. RMarkdownPDFExample.Rmd is similar to the above but makes PDF output. When successfully run, it will load data (the WASHpublic_mock.dta file) and run three basic regressions. This uses the stargazer package to create nicely formatted LaTeX regression tables (outputR.tex). The combined output file is RMarkdownPDFExample.pdf. + +3. Tons of people use Word. So there's an option for that, too. + +4. Rmarkdown_slides.Rmd generates HTML slides. + +5. Rmarkdown_advanced.Rmd is a bit more advanced, with citations, footnotes, and a detailed YAML header. + +6. my_bib.bib is a simple bibliography to be used with Rmarkdown_advanced.Rmd. + +7. WASHpublic_mock.dta is a Stata data file that gets used for demo regressions. + + +# Step One + +That's it. There's one step. Open a .Rmd file and click Knit. It's beautiful. You don't even have to worry about filepaths, and version control is built right in. diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownHTMLExample.Rmd b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownHTMLExample.Rmd new file mode 100644 index 0000000..31b2531 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownHTMLExample.Rmd @@ -0,0 +1,117 @@ +--- +title: "Short HTML example of R Markdown" +author: "Garret Christensen" +date: "`r format(Sys.time(), '%d %B, %Y')`" +output: html_document +--- + +ONE CLICK! Knit HTML and you're done. +=============================================================== + +1. Remember how we had to set the directory in Stata? R Projects to the rescue! And it's got version control built right in. Look under Tools>Version Control. (OMG someone build this for Stata plz.) + +(If you're doing filepaths by yourself in R, and you're on a Windows machine and you copy-pasted the filepath, change slash directions. Ugh.) + +2. Install/Load packages. You only have to install a package once, but you have to load the library every time you want to use it. You'll see that a bunch of ugly commands and output gets displayed. You probably wouldn't really want that in your actual paper. + +```{r, echo=TRUE, message=FALSE, results='hide'} +# Loading required libraries +list.of.packages <- c("foreign", "stargazer", "sandwich", "haven") + +# Chicking if library installed, installed if not. +new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])] +if(length(new.packages)) install.packages(new.packages, repos= "http://cran.cnr.berkeley.edu/") + +# Calling libraries +lapply(list.of.packages, require, character.only = TRUE) +``` + +Which is why you can turn off code (with echo=FALSE) and/or results (with results='hide'). You're not going to see anything from this chunk here: + + +```{r, echo=FALSE, results='hide'} +# Permanently set echo OFF! +knitr::opts_chunk$set(echo=FALSE) +pi<-3.14159 +print(pi) +``` + +This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see . + +When you click the **Knit** button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this: + +```{r} +summary(cars) +``` + +Loading the data +---------------- +You can bring in the Stata data directly with the 'foreign' package. + +```{r, echo = FALSE } +WASHB<-haven::read_dta("WASHBpublic_mock.dta") +``` + +Running Analysis +----------------- +I ran some regression analysis. The results are good. +```{r, echo = FALSE} +#Model 1=simple model +model1<-lm(free_chl_yn~treatw, data=WASHB) +#Model 2=with hetero-robust se's--a bit more complicated than Stata +cov <- vcovHC(model1, type = "HC") +robust.se <- sqrt(diag(cov)) +#Model 3=add baseline control variables +model3<-lm(free_chl_yn~treatw+kiswahili+english, data=WASHB) +``` + +Graphics +======== +Graphics can be easily inlaid. +Here, I'll make a map of the number of US military recruits to a 16 year period by county. + +```{r, echo=FALSE} +plot(haz06~waz06, data=WASHB) +``` + +Equations +========== +Equations written with LaTeX syntax works, so you can write short reports all in one file. +$$ \frac{dN}{dt} = r * N * (1 - \frac{N}{K})$$ + +Refer to Values +=============== +You can refer to values calculated in R by just surrounding "r" and the code with single accent marks. +For example, the mean frequency is `r mean(WASHB$treatw, na.rm=TRUE)`. + + +Simple Output +============== +You can just use built in R functionality. + +```{r} +summary(model1) +summary(model3) +``` + +Fancier Output +============== +Markdown is designed to be simple and also readable by humans in marked-up form. Like I said, mark*down*, not mark*up*. But you can still get really nicely formatted regression output with a couple of R packages, xtable or stargazer. (Very similar to estout or outreg2 in Stata.) + +Stargazer has three types of output (text, html, and LaTeX). This file does HTM. For the LaTeX(PDF) version see the file (RMarkdownPDFExample.Rmd). There are very few differences--you pretty much just have to tell stargazer what to expect with the 'type' option and you're done. + +RMarkdown-->HTML +--------------- +```{r, results='asis'} +stargazer(model1, model1, model3, se=list(NULL, robust.se, NULL), type="html", out="outputR.html", title="Made Automatically in R", align=TRUE, column.labels=c("default","robust", "controls")) +``` + +Everything All in One Place? +===================== +You can do [citations](http://rmarkdown.rstudio.com/authoring_bibliographies_and_citations.html). Plots, graphs, and citations, what else do you need for a research paper? + +Knitr is smart! It doesn't re-run code that hasn't changed, meaning it can be fast. You don't need to worry about re-running your whole paper every time you change a single line of code. + +Knitr's predecessor is called Sweave, which is a direct combo of LaTeX and R. So you might want to try that out if you're already a LaTeX pro, but I'm pretty sure Knitr is better. + +Send your output to .tex files, include those in your master paper file. diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownHTMLExample.html b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownHTMLExample.html new file mode 100644 index 0000000..cf324c9 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownHTMLExample.html @@ -0,0 +1,740 @@ + + + + + + + + + + + + + + +Short HTML example of R Markdown + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + +
+

ONE CLICK! Knit HTML and you’re done.

+
    +
  1. Remember how we had to set the directory in Stata? R Projects to the rescue! And it’s got version control built right in. Look under Tools>Version Control. (OMG someone build this for Stata plz.)
  2. +
+

(If you’re doing filepaths by yourself in R, and you’re on a Windows machine and you copy-pasted the filepath, change slash directions. Ugh.)

+
    +
  1. Install/Load packages. You only have to install a package once, but you have to load the library every time you want to use it. You’ll see that a bunch of ugly commands and output gets displayed. You probably wouldn’t really want that in your actual paper.
  2. +
+
# Loading required libraries
+list.of.packages <- c("foreign", "stargazer", "sandwich", "haven")
+
+# Chicking if library installed, installed if not. 
+new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])]
+if(length(new.packages)) install.packages(new.packages, repos= "http://cran.cnr.berkeley.edu/") 
+
+# Calling libraries 
+lapply(list.of.packages, require, character.only = TRUE)
+
## Warning: package 'sandwich' was built under R version 3.4.3
+
## Warning: package 'haven' was built under R version 3.4.3
+

Which is why you can turn off code (with echo=FALSE) and/or results (with results=‘hide’). You’re not going to see anything from this chunk here:

+ +

This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.

+

When you click the Knit button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:

+
##      speed           dist       
+##  Min.   : 4.0   Min.   :  2.00  
+##  1st Qu.:12.0   1st Qu.: 26.00  
+##  Median :15.0   Median : 36.00  
+##  Mean   :15.4   Mean   : 42.98  
+##  3rd Qu.:19.0   3rd Qu.: 56.00  
+##  Max.   :25.0   Max.   :120.00
+
+

Loading the data

+

You can bring in the Stata data directly with the ‘foreign’ package.

+
+
+

Running Analysis

+

I ran some regression analysis. The results are good.

+
+
+
+

Graphics

+

Graphics can be easily inlaid. Here, I’ll make a map of the number of US military recruits to a 16 year period by county.

+

+
+
+

Equations

+

Equations written with LaTeX syntax works, so you can write short reports all in one file. \[ \frac{dN}{dt} = r * N * (1 - \frac{N}{K})\]

+
+
+

Refer to Values

+

You can refer to values calculated in R by just surrounding “r” and the code with single accent marks. For example, the mean frequency is 0.4822888.

+
+
+

Simple Output

+

You can just use built in R functionality.

+
## 
+## Call:
+## lm(formula = free_chl_yn ~ treatw, data = WASHB)
+## 
+## Residuals:
+##      Min       1Q   Median       3Q      Max 
+## -0.37692 -0.37692 -0.01299 -0.01299  0.98701 
+## 
+## Coefficients:
+##             Estimate Std. Error t value Pr(>|t|)    
+## (Intercept)  0.01299    0.02736   0.475    0.635    
+## treatw       0.36394    0.04044   9.000   <2e-16 ***
+## ---
+## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
+## 
+## Residual standard error: 0.3395 on 282 degrees of freedom
+##   (83 observations deleted due to missingness)
+## Multiple R-squared:  0.2231, Adjusted R-squared:  0.2204 
+## F-statistic:    81 on 1 and 282 DF,  p-value: < 2.2e-16
+
## 
+## Call:
+## lm(formula = free_chl_yn ~ treatw + kiswahili + english, data = WASHB)
+## 
+## Residuals:
+##      Min       1Q   Median       3Q      Max 
+## -0.39602 -0.35122 -0.02021  0.00334  0.97979 
+## 
+## Coefficients:
+##              Estimate Std. Error t value Pr(>|t|)    
+## (Intercept) -0.003344   0.052899  -0.063    0.950    
+## treatw       0.365188   0.040614   8.992   <2e-16 ***
+## kiswahili   -0.010624   0.075772  -0.140    0.889    
+## english      0.034176   0.063663   0.537    0.592    
+## ---
+## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
+## 
+## Residual standard error: 0.3405 on 280 degrees of freedom
+##   (83 observations deleted due to missingness)
+## Multiple R-squared:  0.2243, Adjusted R-squared:  0.216 
+## F-statistic: 26.98 on 3 and 280 DF,  p-value: 2.33e-15
+
+
+

Fancier Output

+

Markdown is designed to be simple and also readable by humans in marked-up form. Like I said, markdown, not markup. But you can still get really nicely formatted regression output with a couple of R packages, xtable or stargazer. (Very similar to estout or outreg2 in Stata.)

+

Stargazer has three types of output (text, html, and LaTeX). This file does HTM. For the LaTeX(PDF) version see the file (RMarkdownPDFExample.Rmd). There are very few differences–you pretty much just have to tell stargazer what to expect with the ‘type’ option and you’re done.

+
+

RMarkdown–>HTML

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+Made Automatically in R +
+
+ +Dependent variable: +
+ +
+ +free_chl_yn +
+ +default + +robust + +controls +
+ +(1) + +(2) + +(3) +
+
+treatw + +0.364*** + +0.364*** + +0.365*** +
+ +(0.040) + +(0.043) + +(0.041) +
+ + + +
+kiswahili + + + +-0.011 +
+ + + +(0.076) +
+ + + +
+english + + + +0.034 +
+ + + +(0.064) +
+ + + +
+Constant + +0.013 + +0.013 + +-0.003 +
+ +(0.027) + +(0.009) + +(0.053) +
+ + + +
+
+Observations + +284 + +284 + +284 +
+R2 + +0.223 + +0.223 + +0.224 +
+Adjusted R2 + +0.220 + +0.220 + +0.216 +
+Residual Std. Error + +0.340 (df = 282) + +0.340 (df = 282) + +0.340 (df = 280) +
+F Statistic + +81.002*** (df = 1; 282) + +81.002*** (df = 1; 282) + +26.982*** (df = 3; 280) +
+
+Note: + +p<0.1; p<0.05; p<0.01 +
+
+
+
+

Everything All in One Place?

+

You can do citations. Plots, graphs, and citations, what else do you need for a research paper?

+

Knitr is smart! It doesn’t re-run code that hasn’t changed, meaning it can be fast. You don’t need to worry about re-running your whole paper every time you change a single line of code.

+

Knitr’s predecessor is called Sweave, which is a direct combo of LaTeX and R. So you might want to try that out if you’re already a LaTeX pro, but I’m pretty sure Knitr is better.

+

Send your output to .tex files, include those in your master paper file.

+
+ + + + +
+ + + + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownPDFExample.Rmd b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownPDFExample.Rmd new file mode 100644 index 0000000..21d6985 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownPDFExample.Rmd @@ -0,0 +1,115 @@ +--- +title: "Short PDF example of R Markdown" +author: "Garret Christensen" +date: "`r format(Sys.time(), '%d %B, %Y')`" +output: pdf_document +--- + +## One Click! Knit PDF and you're done. + +You only have to install a package once, but you have to load the library every time you want to use it. You'll see that a bunch of ugly commands and output gets displayed. You probably wouldn't really want that in your actual paper. + +```{r, echo=FALSE, warning=FALSE, results='hide', message=FALSE} +# Loading required libraries +list.of.packages <- c("foreign", "stargazer", "sandwich", "haven") + +# Chicking if library installed, installed if not. +new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])] +if(length(new.packages)) install.packages(new.packages, repos= "http://cran.cnr.berkeley.edu/") + +# Calling libraries +lapply(list.of.packages, require, character.only = TRUE) +``` + +Which is why you can turn off code (with echo=FALSE) and/or results (with results='hide'). You're not going to see anything from this chunk here: + + +```{r, echo=FALSE, results='asis'} +# Permanently set echo OFF! +knitr::opts_chunk$set(echo=FALSE) +pi<-3.14159 +print(pi) +``` + +This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see . + +When you click the **Knit** button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this: + +```{r} +summary(cars) +``` + +Loading the data +---------------- +You can bring in the Stata data directly with the 'foreign' package. However, as far as I know, this package only works through Stata version 12 data. I might be wrong on that. + +```{r, echo = FALSE, warning=FALSE } +WASHB<-haven::read_dta("WASHBpublic_mock.dta") +``` + +Running Analysis +----------------- +I ran some regression analysis. The results are very, very, very good. +```{r, echo = FALSE} +#Model 1=simple model +model1<-lm(free_chl_yn~treatw, data=WASHB) +#Model 2=with hetero-robust se's--a bit more complicated than Stata +cov <- vcovHC(model1, type = "HC") +robust.se <- sqrt(diag(cov)) +#Model 3=add baseline control variables +model3<-lm(free_chl_yn~treatw+kiswahili+english, data=WASHB) +``` + +Graphics +======== +Graphics can be easily inlaid. +Here, I'll make a scatter plot of the WAZ vs HAZ for the children in the wASHB dataset. + +```{r, echo=FALSE} +plot(haz06~waz06, data=WASHB) +``` + +Equations +========== +Equations written with LaTeX syntax works, so you can write short reports all in one file. +$$ \frac{dN}{dt} = r * N * (1 - \frac{N}{K})$$ + +Refer to Values +=============== +You can refer to values calculated in R by just surrounding "r" and the code with single accent marks. +For example, the mean frequency is `r mean(WASHB$treatw, na.rm=TRUE)`. + +The mean frequency rounded to two decimal place is `r round(mean(WASHB$treatw, na.rm=TRUE), digits=2)`. + +Simple Output +============== +You can just use built in R functionality. + + + +Fancier Output +============== +Markdown is designed to be simple and also readable by humans in marked-up form. Like I said, mark*down*, not mark*up*. But you can still get really nicely formatted regression output with a couple of R packages, xtable or stargazer. (Very similar to estout or outreg2 in Stata.) + +Stargazer has three types of output (text, html, and LaTeX). + +TeX->PDF +--------------- +When we Knit a Markdown as a PDF, it actually makes that PDF using LaTeX. (See [here](http://rmarkdown.rstudio.com/pdf_document_format.html).) So you can use the .tex output option from stargazer and get nice PDF documentation. + +```{r, results='asis'} +stargazer(model1, model1, model3, se=list(NULL, robust.se, NULL), title="Made Automatically in R", out="outputR.tex", header=FALSE) +``` + + +Everything All in One Place? +===================== +You can do [citations](http://rmarkdown.rstudio.com/authoring_bibliographies_and_citations.html). Plots, graphs, and citations, what else do you need for a research paper? + +You could also maybe try Sweave (direct combo of LaTeX and R). Or, just like in Stata, you could send your output to .tex files, and include those in your master paper file. diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownPDFExample.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownPDFExample.pdf new file mode 100644 index 0000000..cb87778 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownPDFExample.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownWordExample.Rmd b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownWordExample.Rmd new file mode 100644 index 0000000..7092e49 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownWordExample.Rmd @@ -0,0 +1,22 @@ +--- +title: "I'm just learning Markdown" +author: "Garret Christensen" +date: "April 28, 2016" +output: word_document +--- + +This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see . + +When you click the **Knit** button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this: + +```{r} +summary(cars) +``` + +You can also embed plots, for example: + +```{r, echo=FALSE} +plot(cars) +``` + +Note that the `echo = FALSE` parameter was added to the code chunk to prevent printing of the R code that generated the plot. diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownWordExample.docx b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownWordExample.docx new file mode 100644 index 0000000..9d21dfa Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/RMarkdownWordExample.docx differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.Rmd b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.Rmd new file mode 100644 index 0000000..c416ec3 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.Rmd @@ -0,0 +1,206 @@ +--- +title: "R Markdown Demo PDF" +author: "JPT" +output: + html_document: + df_print: paged + pdf_document: + citation_package: natbib + fig_caption: yes + keep_tex: yes + number_sections: yes +biblio-style: apsr +fontfamily: mathpazo +fontsize: 11pt +geometry: margin=1in +header-includes: +- \usepackage[english]{babel} +- \usepackage{color} +- \usepackage{float} +- \usepackage{hyperref} +- \hypersetup{colorlinks, linkcolor=, urlcolor=blue, citecolor=blue} +- \setlength\parindent{0pt} +- \setlength\parskip{0.12in} +- \frenchspacing +bibliography: my_bib.bib +spacing: single +subtitle: Based on 'RMarkdownPDFExample.Rmd' by Garret Christensen +abstract: If I were writing an article and had an abstract, it would go here! +--- + +```{r setup, include=FALSE} +knitr::opts_chunk$set(echo = TRUE) # change to echo=FALSE to turn echo permanently off +# Loading required libraries +list.of.packages <- c("foreign", "stargazer", "sandwich", "haven", "ggplot2") + +# Chicking if library installed, installed if not. +new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])] +if(length(new.packages)) install.packages(new.packages, repos= "http://cran.cnr.berkeley.edu/") + +# Calling libraries +lapply(list.of.packages, require, character.only = TRUE) +``` + + + + + +# What is R Markdown? +This is an R Markdown document. Markdown is a simple formatting syntax for reports with embedded R code that can be exported as an html, pdf, MS Word, ODT, RTF, or markdown document; or as an html or pdf-based (Beamer) slide show. + +Essentially, you write a document---like this one---in RStudio using Markdown syntax. Then you embed chunks of R code in the document, like this: + +```{r iris_tab} +summary(iris) +``` + +When you click the **Knit** button, a document (e.g., HTML, PDF) will be generated that includes the content you've typed as well as the output of any embedded R code chunks within the document. + +This means that your code, analysis, and output are all in the same place! You never have to copy-and-paste a table or figure again! If you change your code and get and estimated effect size of 0.3 instead of 0.5, you don't need to scour your results section or use find and replace to change this result. + +For more details on using R Markdown see . Also check out [this tutorial](http://rmarkdown.rstudio.com/lesson-1.html) and [this cheatsheet](https://www.rstudio.com/wp-content/uploads/2016/03/rmarkdown-cheatsheet-2.0.pdf). + +# Getting Started + +## Installing and Loading +To use R Markdown, you need R and RStudio installed. Let's do that now: + +1. Download and install [R](https://cran.rstudio.com/) +2. Download and install [RStudio](https://www.rstudio.com/products/rstudio/download/)---an "integrated development environment"" or IDE for R + +Once you've got **RStudio open**, then + +3. Install the R Markdown package by typing `install.packages("rmarkdown")` into the console +4. Open a new .Rmd document [File > New File > R Markdown ...] + +## Basic Syntax +The content of an .rdm file is a mixture of different types of syntax and code, including: + +- An (optional) YAML header at the beginning surrounded by "`---`"---this header gives basic document metadata and sets key style and other options, as desired +- Text using Markdown formatting---like this! +- R code chunks, which are the same bits of code you would write in a .R script file +- \LaTeX\ syntax---enclose text in "`$`" for inline equations, e.g., $\hat \beta = (X'X)^{-1}X'y$ or "`$$`" for displayed equations: $$\hat \beta = (X'X)^{-1}X'y $$ + +## Basic Options + +Within the R code cunks, you can se the following options: + +- `echo=FALSE`---prevents R source code from displaying +- `eval=FALSE`---prevents Knitr from evaluating the R code +- `results='hide'`---hides the results of the code +- `include=FALSE`---Knitr will run the code but not include in the final doc +- `warning=FALSE`---turns off warnings +- `message=FALSE`---turns off messages + + +# Analysis Example + +Let's first begin by clearing our workspace and setting our working directory: + +```{r, echo=TRUE, message=FALSE} +rm(list = ls()) # clear workspace; always a good idea when starting +#setwd ("~/Documents/RA/India_BITSS/rmarkdown") # change your working directory +``` + +Then, let's load our packages: + +```{r, echo=TRUE, message=FALSE, eval=FALSE} +need <- c("foreign", "sandwich", "ggplot2", "stargazer") # list packages you need +have <- need %in% rownames(installed.packages()) # see which are already +if(any(!have)) install.packages(need[!have]) # installs the missing ones +invisible(lapply(need, library, character.only=T)) # then loads them all +``` + +Load the data: + +```{r, echo = T, message = F, warning=F} +washb <- haven::read_dta("WASHBpublic_mock.dta") +``` + +Run the models: + +```{r, echo = T, message = F} +model1 <- lm(free_chl_yn ~ treatw, data = washb) +model2 <- lm(free_chl_yn ~ treatw + kiswahili + english, data = washb) +robust.se1 <- sqrt(diag(vcovHC(model1, type = "HC"))) +robust.se2 <- sqrt(diag(vcovHC(model2, type = "HC"))) +``` + +And make our table: + +```{r reg, echo=T, results='asis'} +stargazer(model1, model1, se=list(robust.se1, robust.se2), + title="Made Automatically in R", + out="outputR.tex", header=FALSE) +``` + +Note that we needed the `results = 'asis'` option to get the table to output correctly, otherwise we would have gotten the copy-and-paste \LaTeX\ output like in R. + +## Referring to values +You can refer to values calculated in R by just surrounding "r" and the code with single accent marks. For example, the mean frequency is `r mean(washb$treatw, na.rm = TRUE)`. The mean frequency rounded to two decimal place is `r round(mean(washb$treatw, na.rm = TRUE), digits = 2)`. + +# Figures/Plots + +For figures generated in R, you can code them directly (here, the data comes from the `iris` dataset, which comes pre-loaded in R: + +```{r, echo=FALSE} +plot(haz06 ~ waz06, data = washb) +``` + +Note that the `echo = FALSE` parameter was added to the code chunk to prevent printing of the R code that generated the plot. + +For external files that you want to include, use `image: ![](de-identification_indirect.png)`. Or you can use \LaTeX\ syntax if you want advanced formatting capacity, e.g., + +\begin{figure}[H] + \centering + \caption{Options for De-Identifying Data} + \includegraphics[width=\textwidth]{de-identification_indirect.png} +\end{figure} + +# Basic Formatting in Markdown + +## Headers +Make yourself a header of different levels using `#` for level 1, ``##`` for level 2 etc. + +## Typeface +Surround words in `*` for *italics*, and `**` for **bold**. + +## Punctuation +Use "`---`"" to get an em-dash (---) and "`--`"" to get an en-dash (--). Use normal quotation marks ("", or ''), unlike in \LaTeX\. + +## Lists +Make a numbered list using "`1.`", and a bulleted list using "`-`": + +1. item 1 +1. item 2 +1. item 3 + +- item a +- item b +- item c + +## Hyperlinks +Rmarkdown will automatically format a copy-and-pasted URL as a hyperlink (e.g., http://rmarkdown.rstudio.com). If you want to add a link to a particular word, type "`[Rstudio](http://rmarkdown.rstudio.com)`" to get [Rstudio](http://rmarkdown.rstudio.com). + +## Commenting +Commenting can get a little tricky to remember, as notation is different in R (`#`), \LaTeX\ (`%`), and markdown (``). In the preamble and code snippets, comment using `#` as in R (see above). In the rest of the document, comment by surrounding text with ``. + +# Footnotes and Citations + +## Footnotes +Add a footnote using `^[I am a footnote]`.^[I am a footnote.] + +## Citations +Here's how you add a citation from your BibTex library (formatted in author-date style, using the `natbib` package loaded in the preamble): + +- For a single source, use `[@citekey]`, e.g., "`[@banerjee2010pitfalls]`" gives us [@banerjee2010pitfalls]. +- For multiple sources, use `[@citekey1; @citekey2; etc.]`, e.g., "`[@banerjee2010pitfalls; @easterly2001elusive]`" gives us [@banerjee2010pitfalls; @easterly2001elusive]. +- For an in-text citation, use `@citekey` with no brackets, e.g., "`@banerjee2010pitfalls`" gives us @banerjee2010pitfalls. + +For more, see http://rmarkdown.rstudio.com/authoring_bibliographies_and_citations.html. + +And the bibliography/references will automatically show up with the sources you've added using cite keys; no need to copy and paste citations or double check to make sure you've included/removed sources as necessary!... + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.html b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.html new file mode 100644 index 0000000..40b0254 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.html @@ -0,0 +1,1796 @@ + + + + + + + + + + + + + + +R Markdown Demo PDF + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + +
+

What is R Markdown?

+

This is an R Markdown document. Markdown is a simple formatting syntax for reports with embedded R code that can be exported as an html, pdf, MS Word, ODT, RTF, or markdown document; or as an html or pdf-based (Beamer) slide show.

+

Essentially, you write a document—like this one—in RStudio using Markdown syntax. Then you embed chunks of R code in the document, like this:

+
summary(iris)
+
##   Sepal.Length    Sepal.Width     Petal.Length    Petal.Width   
+##  Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100  
+##  1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300  
+##  Median :5.800   Median :3.000   Median :4.350   Median :1.300  
+##  Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199  
+##  3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800  
+##  Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500  
+##        Species  
+##  setosa    :50  
+##  versicolor:50  
+##  virginica :50  
+##                 
+##                 
+## 
+

When you click the Knit button, a document (e.g., HTML, PDF) will be generated that includes the content you’ve typed as well as the output of any embedded R code chunks within the document.

+

This means that your code, analysis, and output are all in the same place! You never have to copy-and-paste a table or figure again! If you change your code and get and estimated effect size of 0.3 instead of 0.5, you don’t need to scour your results section or use find and replace to change this result.

+

For more details on using R Markdown see http://rmarkdown.rstudio.com. Also check out this tutorial and this cheatsheet.

+
+
+

Getting Started

+
+

Installing and Loading

+

To use R Markdown, you need R and RStudio installed. Let’s do that now:

+
    +
  1. Download and install R
  2. +
  3. Download and install RStudio—an “integrated development environment”" or IDE for R
  4. +
+

Once you’ve got RStudio open, then

+
    +
  1. Install the R Markdown package by typing install.packages("rmarkdown") into the console
  2. +
  3. Open a new .Rmd document [File > New File > R Markdown …]
  4. +
+
+
+

Basic Syntax

+

The content of an .rdm file is a mixture of different types of syntax and code, including:

+
    +
  • An (optional) YAML header at the beginning surrounded by “---”—this header gives basic document metadata and sets key style and other options, as desired
  • +
  • Text using Markdown formatting—like this!
  • +
  • R code chunks, which are the same bits of code you would write in a .R script file
  • +
  •  syntax—enclose text in “$” for inline equations, e.g., \(\hat \beta = (X'X)^{-1}X'y\) or “$$” for displayed equations: \[\hat \beta = (X'X)^{-1}X'y \]
  • +
+
+
+

Basic Options

+

Within the R code cunks, you can se the following options:

+
    +
  • echo=FALSE—prevents R source code from displaying
  • +
  • eval=FALSE—prevents Knitr from evaluating the R code
  • +
  • results='hide'—hides the results of the code
  • +
  • include=FALSE—Knitr will run the code but not include in the final doc
  • +
  • warning=FALSE—turns off warnings
  • +
  • message=FALSE—turns off messages
  • +
+
+
+
+

Analysis Example

+

Let’s first begin by clearing our workspace and setting our working directory:

+
rm(list = ls()) # clear workspace; always a good idea when starting
+#setwd ("~/Documents/RA/India_BITSS/rmarkdown") # change your working directory
+

Then, let’s load our packages:

+
need <- c("foreign", "sandwich", "ggplot2", "stargazer") # list packages you need
+have <- need %in% rownames(installed.packages()) # see which are already
+if(any(!have)) install.packages(need[!have]) # installs the missing ones 
+invisible(lapply(need, library, character.only=T)) # then loads them all
+

Load the data:

+
washb <- haven::read_dta("WASHBpublic_mock.dta")
+

Run the models:

+
model1 <- lm(free_chl_yn ~ treatw, data = washb) 
+model2 <- lm(free_chl_yn ~ treatw + kiswahili + english, data = washb)
+robust.se1 <- sqrt(diag(vcovHC(model1, type = "HC")))
+robust.se2 <- sqrt(diag(vcovHC(model2, type = "HC")))
+

And make our table:

+
stargazer(model1, model1, se=list(robust.se1, robust.se2), 
+          title="Made Automatically in R", 
+          out="outputR.tex", header=FALSE)
+ +

Note that we needed the results = 'asis' option to get the table to output correctly, otherwise we would have gotten the copy-and-paste  output like in R.

+
+

Referring to values

+

You can refer to values calculated in R by just surrounding “r” and the code with single accent marks. For example, the mean frequency is 0.4822888. The mean frequency rounded to two decimal place is 0.48.

+
+
+
+

Figures/Plots

+

For figures generated in R, you can code them directly (here, the data comes from the iris dataset, which comes pre-loaded in R:

+

+

Note that the echo = FALSE parameter was added to the code chunk to prevent printing of the R code that generated the plot.

+

For external files that you want to include, use image: ![](de-identification_indirect.png). Or you can use  syntax if you want advanced formatting capacity, e.g.,

+ +
+
+

Basic Formatting in Markdown

+
+

Headers

+

Make yourself a header of different levels using # for level 1, ## for level 2 etc.

+
+
+

Typeface

+

Surround words in * for italics, and ** for bold.

+
+
+

Punctuation

+

Use “---”" to get an em-dash (—) and “--”" to get an en-dash (–). Use normal quotation marks (“”, or ’’), unlike in .

+
+
+

Lists

+

Make a numbered list using “1.”, and a bulleted list using “-”:

+
    +
  1. item 1
  2. +
  3. item 2
  4. +
  5. item 3
  6. +
+
    +
  • item a
  • +
  • item b
  • +
  • item c
  • +
+
+ +
+

Commenting

+

Commenting can get a little tricky to remember, as notation is different in R (#),  (%), and markdown (<!-- -->). In the preamble and code snippets, comment using # as in R (see above). In the rest of the document, comment by surrounding text with <!-- -->.

+
+
+
+

Footnotes and Citations

+
+

Footnotes

+

Add a footnote using ^[I am a footnote].1

+
+
+

Citations

+

Here’s how you add a citation from your BibTex library (formatted in author-date style, using the natbib package loaded in the preamble):

+
    +
  • For a single source, use [@citekey], e.g., “[@banerjee2010pitfalls]” gives us (Banerjee et al. 2010).
  • +
  • For multiple sources, use [@citekey1; @citekey2; etc.], e.g., “[@banerjee2010pitfalls; @easterly2001elusive]” gives us (Banerjee et al. 2010; Easterly 2001).
  • +
  • For an in-text citation, use @citekey with no brackets, e.g., “@banerjee2010pitfalls” gives us Banerjee et al. (2010).
  • +
+

For more, see http://rmarkdown.rstudio.com/authoring_bibliographies_and_citations.html.

+

And the bibliography/references will automatically show up with the sources you’ve added using cite keys; no need to copy and paste citations or double check to make sure you’ve included/removed sources as necessary!…

+
+
+

Banerjee, Abhijit V, Rukmini Banerji, Esther Duflo, Rachel Glennerster, and Stuti Khemani. 2010. “Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India.” American Economic Journal: Economic Policy. JSTOR, 1–30.

+
+
+

Easterly, William Russell. 2001. The Elusive Quest for Growth: Economists’ Adventures and Misadventures in the Tropics. MIT press.

+
+
+
+
+
+
+
    +
  1. I am a footnote.

  2. +
+
+ + + + +
+ + + + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.pdf new file mode 100644 index 0000000..b7a7e94 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.tex b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.tex new file mode 100644 index 0000000..01eb0d1 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_advanced.tex @@ -0,0 +1,501 @@ +\documentclass[11pt,]{article} +\usepackage[]{mathpazo} +\usepackage{amssymb,amsmath} +\usepackage{ifxetex,ifluatex} +\usepackage{fixltx2e} % provides \textsubscript +\ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\else % if luatex or xelatex + \ifxetex + \usepackage{mathspec} + \else + \usepackage{fontspec} + \fi + \defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase} +\fi +% use upquote if available, for straight quotes in verbatim environments +\IfFileExists{upquote.sty}{\usepackage{upquote}}{} +% use microtype if available +\IfFileExists{microtype.sty}{% +\usepackage{microtype} +\UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts +}{} +\usepackage[margin=1in]{geometry} +\usepackage{hyperref} +\hypersetup{unicode=true, + pdftitle={R Markdown Demo PDF}, + pdfauthor={Julia Clark}, + pdfborder={0 0 0}, + breaklinks=true} +\urlstyle{same} % don't use monospace font for urls +\usepackage{natbib} +\bibliographystyle{apsr} +\usepackage{color} +\usepackage{fancyvrb} +\newcommand{\VerbBar}{|} +\newcommand{\VERB}{\Verb[commandchars=\\\{\}]} +\DefineVerbatimEnvironment{Highlighting}{Verbatim}{commandchars=\\\{\}} +% Add ',fontsize=\small' for more characters per line +\usepackage{framed} +\definecolor{shadecolor}{RGB}{248,248,248} +\newenvironment{Shaded}{\begin{snugshade}}{\end{snugshade}} +\newcommand{\KeywordTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{\textbf{#1}}} +\newcommand{\DataTypeTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{#1}} +\newcommand{\DecValTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}} +\newcommand{\BaseNTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}} +\newcommand{\FloatTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}} +\newcommand{\ConstantTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}} +\newcommand{\CharTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}} +\newcommand{\SpecialCharTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}} +\newcommand{\StringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}} +\newcommand{\VerbatimStringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}} +\newcommand{\SpecialStringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}} +\newcommand{\ImportTok}[1]{#1} +\newcommand{\CommentTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textit{#1}}} +\newcommand{\DocumentationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}} +\newcommand{\AnnotationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}} +\newcommand{\CommentVarTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}} +\newcommand{\OtherTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{#1}} +\newcommand{\FunctionTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}} +\newcommand{\VariableTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}} +\newcommand{\ControlFlowTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{\textbf{#1}}} +\newcommand{\OperatorTok}[1]{\textcolor[rgb]{0.81,0.36,0.00}{\textbf{#1}}} +\newcommand{\BuiltInTok}[1]{#1} +\newcommand{\ExtensionTok}[1]{#1} +\newcommand{\PreprocessorTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textit{#1}}} +\newcommand{\AttributeTok}[1]{\textcolor[rgb]{0.77,0.63,0.00}{#1}} +\newcommand{\RegionMarkerTok}[1]{#1} +\newcommand{\InformationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}} +\newcommand{\WarningTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}} +\newcommand{\AlertTok}[1]{\textcolor[rgb]{0.94,0.16,0.16}{#1}} +\newcommand{\ErrorTok}[1]{\textcolor[rgb]{0.64,0.00,0.00}{\textbf{#1}}} +\newcommand{\NormalTok}[1]{#1} +\usepackage{graphicx,grffile} +\makeatletter +\def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi} +\def\maxheight{\ifdim\Gin@nat@height>\textheight\textheight\else\Gin@nat@height\fi} +\makeatother +% Scale images if necessary, so that they will not overflow the page +% margins by default, and it is still possible to overwrite the defaults +% using explicit options in \includegraphics[width, height, ...]{} +\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio} +\IfFileExists{parskip.sty}{% +\usepackage{parskip} +}{% else +\setlength{\parindent}{0pt} +\setlength{\parskip}{6pt plus 2pt minus 1pt} +} +\setlength{\emergencystretch}{3em} % prevent overfull lines +\providecommand{\tightlist}{% + \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}} +\setcounter{secnumdepth}{5} +% Redefines (sub)paragraphs to behave more like sections +\ifx\paragraph\undefined\else +\let\oldparagraph\paragraph +\renewcommand{\paragraph}[1]{\oldparagraph{#1}\mbox{}} +\fi +\ifx\subparagraph\undefined\else +\let\oldsubparagraph\subparagraph +\renewcommand{\subparagraph}[1]{\oldsubparagraph{#1}\mbox{}} +\fi + +%%% Use protect on footnotes to avoid problems with footnotes in titles +\let\rmarkdownfootnote\footnote% +\def\footnote{\protect\rmarkdownfootnote} + +%%% Change title format to be more compact +\usepackage{titling} + +% Create subtitle command for use in maketitle +\newcommand{\subtitle}[1]{ + \posttitle{ + \begin{center}\large#1\end{center} + } +} + +\setlength{\droptitle}{-2em} + \title{R Markdown Demo PDF} + \pretitle{\vspace{\droptitle}\centering\huge} + \posttitle{\par} +\subtitle{Based on `RMarkdownPDFExample.Rmd' by Garret Christensen} + \author{Julia Clark} + \preauthor{\centering\large\emph} + \postauthor{\par} + \predate{\centering\large\emph} + \postdate{\par} + \date{03 December, 2017} + +\usepackage[english]{babel} +\usepackage{color} +\usepackage{float} +\usepackage{hyperref} +\hypersetup{colorlinks, linkcolor=, urlcolor=blue, citecolor=blue} +\setlength\parindent{0pt} +\setlength\parskip{0.12in} +\frenchspacing + +\begin{document} +\maketitle +\begin{abstract} +If I were writing an article and had an abstract, it would go here! +\end{abstract} + +\hypertarget{what-is-r-markdown}{% +\section{What is R Markdown?}\label{what-is-r-markdown}} + +This is an R Markdown document. Markdown is a simple formatting syntax +for reports with embedded R code that can be exported as an html, pdf, +MS Word, ODT, RTF, or markdown document; or as an html or pdf-based +(Beamer) slide show. + +Essentially, you write a document---like this one---in RStudio using +Markdown syntax. Then you embed chunks of R code in the document, like +this: + +\begin{Shaded} +\begin{Highlighting}[] +\KeywordTok{summary}\NormalTok{(iris)} +\end{Highlighting} +\end{Shaded} + +\begin{verbatim} +## Sepal.Length Sepal.Width Petal.Length Petal.Width +## Min. :4.300 Min. :2.000 Min. :1.000 Min. :0.100 +## 1st Qu.:5.100 1st Qu.:2.800 1st Qu.:1.600 1st Qu.:0.300 +## Median :5.800 Median :3.000 Median :4.350 Median :1.300 +## Mean :5.843 Mean :3.057 Mean :3.758 Mean :1.199 +## 3rd Qu.:6.400 3rd Qu.:3.300 3rd Qu.:5.100 3rd Qu.:1.800 +## Max. :7.900 Max. :4.400 Max. :6.900 Max. :2.500 +## Species +## setosa :50 +## versicolor:50 +## virginica :50 +## +## +## +\end{verbatim} + +When you click the \textbf{Knit} button, a document (e.g., HTML, PDF) +will be generated that includes the content you've typed as well as the +output of any embedded R code chunks within the document. + +This means that your code, analysis, and output are all in the same +place! You never have to copy-and-paste a table or figure again! If you +change your code and get and estimated effect size of 0.3 instead of +0.5, you don't need to scour your results section or use find and +replace to change this result. + +For more details on using R Markdown see +\url{http://rmarkdown.rstudio.com}. Also check out +\href{http://rmarkdown.rstudio.com/lesson-1.html}{this tutorial} and +\href{https://www.rstudio.com/wp-content/uploads/2016/03/rmarkdown-cheatsheet-2.0.pdf}{this +cheatsheet}. + +\hypertarget{getting-started}{% +\section{Getting Started}\label{getting-started}} + +\hypertarget{installing-and-loading}{% +\subsection{Installing and Loading}\label{installing-and-loading}} + +To use R Markdown, you need R and RStudio installed. Let's do that now: + +\begin{enumerate} +\def\labelenumi{\arabic{enumi}.} +\tightlist +\item + Download and install \href{https://cran.rstudio.com/}{R} +\item + Download and install + \href{https://www.rstudio.com/products/rstudio/download/}{RStudio}---an + ``integrated development environment''" or IDE for R +\end{enumerate} + +Once you've got \textbf{RStudio open}, then + +\begin{enumerate} +\def\labelenumi{\arabic{enumi}.} +\setcounter{enumi}{2} +\tightlist +\item + Install the R Markdown package by typing + \texttt{install.packages("rmarkdown")} into the console +\item + Open a new .Rmd document {[}File \textgreater{} New File + \textgreater{} R Markdown \ldots{}{]} +\end{enumerate} + +\hypertarget{basic-syntax}{% +\subsection{Basic Syntax}\label{basic-syntax}} + +The content of an .rdm file is a mixture of different types of syntax +and code, including: + +\begin{itemize} +\tightlist +\item + An (optional) YAML header at the beginning surrounded by + ``\texttt{-\/-\/-}''---this header gives basic document metadata and + sets key style and other options, as desired +\item + Text using Markdown formatting---like this! +\item + R code chunks, which are the same bits of code you would write in a .R + script file +\item + \LaTeX~syntax---enclose text in ``\texttt{\$}'' for inline equations, + e.g., \(\hat \beta = (X'X)^{-1}X'y\) or ``\texttt{\$\$}'' for + displayed equations: \[\hat \beta = (X'X)^{-1}X'y \] +\end{itemize} + +\hypertarget{basic-options}{% +\subsection{Basic Options}\label{basic-options}} + +Within the R code cunks, you can se the following options: + +\begin{itemize} +\tightlist +\item + \texttt{echo=FALSE}---prevents R source code from displaying +\item + \texttt{eval=FALSE}---prevents Knitr from evaluating the R code +\item + \texttt{results=\textquotesingle{}hide\textquotesingle{}}---hides the + results of the code +\item + \texttt{include=FALSE}---Knitr will run the code but not include in + the final doc +\item + \texttt{warning=FALSE}---turns off warnings +\item + \texttt{message=FALSE}---turns off messages +\end{itemize} + +\hypertarget{analysis-example}{% +\section{Analysis Example}\label{analysis-example}} + +Let's first begin by clearing our workspace and setting our working +directory: + +\begin{Shaded} +\begin{Highlighting}[] +\KeywordTok{rm}\NormalTok{(}\DataTypeTok{list =} \KeywordTok{ls}\NormalTok{()) }\CommentTok{# clear workspace; always a good idea when starting} +\CommentTok{#setwd ("~/Documents/RA/India_BITSS/rmarkdown") # change your working directory} +\end{Highlighting} +\end{Shaded} + +Then, let's load our packages: + +\begin{Shaded} +\begin{Highlighting}[] +\NormalTok{need <-}\StringTok{ }\KeywordTok{c}\NormalTok{(}\StringTok{"foreign"}\NormalTok{, }\StringTok{"sandwich"}\NormalTok{, }\StringTok{"ggplot2"}\NormalTok{, }\StringTok{"stargazer"}\NormalTok{) }\CommentTok{# list packages you need} +\NormalTok{have <-}\StringTok{ }\NormalTok{need }\OperatorTok{%in%}\StringTok{ }\KeywordTok{rownames}\NormalTok{(}\KeywordTok{installed.packages}\NormalTok{()) }\CommentTok{# see which are already} +\ControlFlowTok{if}\NormalTok{(}\KeywordTok{any}\NormalTok{(}\OperatorTok{!}\NormalTok{have)) }\KeywordTok{install.packages}\NormalTok{(need[}\OperatorTok{!}\NormalTok{have]) }\CommentTok{# installs the missing ones } +\KeywordTok{invisible}\NormalTok{(}\KeywordTok{lapply}\NormalTok{(need, library, }\DataTypeTok{character.only=}\NormalTok{T)) }\CommentTok{# then loads them all} +\end{Highlighting} +\end{Shaded} + +Load the data: + +\begin{Shaded} +\begin{Highlighting}[] +\NormalTok{washb <-}\StringTok{ }\NormalTok{haven}\OperatorTok{::}\KeywordTok{read_dta}\NormalTok{(}\StringTok{"WASHBpublic_mock.dta"}\NormalTok{)} +\end{Highlighting} +\end{Shaded} + +Run the models: + +\begin{Shaded} +\begin{Highlighting}[] +\NormalTok{model1 <-}\StringTok{ }\KeywordTok{lm}\NormalTok{(free_chl_yn }\OperatorTok{~}\StringTok{ }\NormalTok{treatw, }\DataTypeTok{data =}\NormalTok{ washb) } +\NormalTok{model2 <-}\StringTok{ }\KeywordTok{lm}\NormalTok{(free_chl_yn }\OperatorTok{~}\StringTok{ }\NormalTok{treatw }\OperatorTok{+}\StringTok{ }\NormalTok{kiswahili }\OperatorTok{+}\StringTok{ }\NormalTok{english, }\DataTypeTok{data =}\NormalTok{ washb)} +\NormalTok{robust.se1 <-}\StringTok{ }\KeywordTok{sqrt}\NormalTok{(}\KeywordTok{diag}\NormalTok{(}\KeywordTok{vcovHC}\NormalTok{(model1, }\DataTypeTok{type =} \StringTok{"HC"}\NormalTok{)))} +\NormalTok{robust.se2 <-}\StringTok{ }\KeywordTok{sqrt}\NormalTok{(}\KeywordTok{diag}\NormalTok{(}\KeywordTok{vcovHC}\NormalTok{(model2, }\DataTypeTok{type =} \StringTok{"HC"}\NormalTok{)))} +\end{Highlighting} +\end{Shaded} + +And make our table: + +\begin{Shaded} +\begin{Highlighting}[] +\KeywordTok{stargazer}\NormalTok{(model1, model1, }\DataTypeTok{se=}\KeywordTok{list}\NormalTok{(robust.se1, robust.se2), } + \DataTypeTok{title=}\StringTok{"Made Automatically in R"}\NormalTok{, } + \DataTypeTok{out=}\StringTok{"outputR.tex"}\NormalTok{, }\DataTypeTok{header=}\OtherTok{FALSE}\NormalTok{)} +\end{Highlighting} +\end{Shaded} + +\begin{table}[!htbp] \centering + \caption{Made Automatically in R} + \label{} +\begin{tabular}{@{\extracolsep{5pt}}lcc} +\\[-1.8ex]\hline +\hline \\[-1.8ex] + & \multicolumn{2}{c}{\textit{Dependent variable:}} \\ +\cline{2-3} +\\[-1.8ex] & \multicolumn{2}{c}{free\_chl\_yn} \\ +\\[-1.8ex] & (1) & (2)\\ +\hline \\[-1.8ex] + treatw & 0.364$^{***}$ & 0.364$^{***}$ \\ + & (0.043) & (0.043) \\ + & & \\ + Constant & 0.013 & 0.013 \\ + & (0.009) & (0.045) \\ + & & \\ +\hline \\[-1.8ex] +Observations & 284 & 284 \\ +R$^{2}$ & 0.223 & 0.223 \\ +Adjusted R$^{2}$ & 0.220 & 0.220 \\ +Residual Std. Error (df = 282) & 0.340 & 0.340 \\ +F Statistic (df = 1; 282) & 81.002$^{***}$ & 81.002$^{***}$ \\ +\hline +\hline \\[-1.8ex] +\textit{Note:} & \multicolumn{2}{r}{$^{*}$p$<$0.1; $^{**}$p$<$0.05; $^{***}$p$<$0.01} \\ +\end{tabular} +\end{table} + +Note that we needed the +\texttt{results\ =\ \textquotesingle{}asis\textquotesingle{}} option to +get the table to output correctly, otherwise we would have gotten the +copy-and-paste \LaTeX~output like in R. + +\hypertarget{referring-to-values}{% +\subsection{Referring to values}\label{referring-to-values}} + +You can refer to values calculated in R by just surrounding ``r'' and +the code with single accent marks. For example, the mean frequency is +0.4822888. The mean frequency rounded to two decimal place is 0.48. + +\hypertarget{figuresplots}{% +\section{Figures/Plots}\label{figuresplots}} + +For figures generated in R, you can code them directly (here, the data +comes from the \texttt{iris} dataset, which comes pre-loaded in R: + +\includegraphics{Rmarkdown_advanced_files/figure-latex/unnamed-chunk-5-1.pdf} + +Note that the \texttt{echo\ =\ FALSE} parameter was added to the code +chunk to prevent printing of the R code that generated the plot. + +For external files that you want to include, use +\texttt{image:\ !{[}{]}(de-identification\_indirect.png)}. Or you can +use \LaTeX~syntax if you want advanced formatting capacity, e.g., + +\begin{figure}[H] + \centering + \caption{Options for De-Identifying Data} + \includegraphics[width=\textwidth]{de-identification_indirect.png} +\end{figure} + +\hypertarget{basic-formatting-in-markdown}{% +\section{Basic Formatting in +Markdown}\label{basic-formatting-in-markdown}} + +\hypertarget{headers}{% +\subsection{Headers}\label{headers}} + +Make yourself a header of different levels using \texttt{\#} for level +1, \texttt{\#\#} for level 2 etc. + +\hypertarget{typeface}{% +\subsection{Typeface}\label{typeface}} + +Surround words in \texttt{*} for \emph{italics}, and \texttt{**} for +\textbf{bold}. + +\hypertarget{punctuation}{% +\subsection{Punctuation}\label{punctuation}} + +Use ``\texttt{-\/-\/-}''" to get an em-dash (---) and ``\texttt{-\/-}''" +to get an en-dash (--). Use normal quotation marks (``'', or ''), unlike +in \LaTeX. + +\hypertarget{lists}{% +\subsection{Lists}\label{lists}} + +Make a numbered list using ``\texttt{1.}'', and a bulleted list using +``\texttt{-}'': + +\begin{enumerate} +\def\labelenumi{\arabic{enumi}.} +\tightlist +\item + item 1 +\item + item 2 +\item + item 3 +\end{enumerate} + +\begin{itemize} +\tightlist +\item + item a +\item + item b +\item + item c +\end{itemize} + +\hypertarget{hyperlinks}{% +\subsection{Hyperlinks}\label{hyperlinks}} + +Rmarkdown will automatically format a copy-and-pasted URL as a hyperlink +(e.g., \url{http://rmarkdown.rstudio.com}). If you want to add a link to +a particular word, type +``\texttt{{[}Rstudio{]}(http://rmarkdown.rstudio.com)}'' to get +\href{http://rmarkdown.rstudio.com}{Rstudio}. + +\hypertarget{commenting}{% +\subsection{Commenting}\label{commenting}} + +Commenting can get a little tricky to remember, as notation is different +in R (\texttt{\#}), \LaTeX~(\texttt{\%}), and markdown +(\texttt{\textless{}!-\/-\ -\/-\textgreater{}}). In the preamble and +code snippets, comment using \texttt{\#} as in R (see above). In the +rest of the document, comment by surrounding text with +\texttt{\textless{}!-\/-\ -\/-\textgreater{}}. + +\hypertarget{footnotes-and-citations}{% +\section{Footnotes and Citations}\label{footnotes-and-citations}} + +\hypertarget{footnotes}{% +\subsection{Footnotes}\label{footnotes}} + +Add a footnote using \texttt{\^{}{[}I\ am\ a\ footnote{]}}.\footnote{I + am a footnote.} + +\hypertarget{citations}{% +\subsection{Citations}\label{citations}} + +Here's how you add a citation from your BibTex library (formatted in +author-date style, using the \texttt{natbib} package loaded in the +preamble): + +\begin{itemize} +\tightlist +\item + For a single source, use \texttt{{[}@citekey{]}}, e.g., + ``\texttt{{[}@banerjee2010pitfalls{]}}'' gives us + \citep{banerjee2010pitfalls}. +\item + For multiple sources, use \texttt{{[}@citekey1;\ @citekey2;\ etc.{]}}, + e.g., ``\texttt{{[}@banerjee2010pitfalls;\ @easterly2001elusive{]}}'' + gives us \citep{banerjee2010pitfalls, easterly2001elusive}. +\item + For an in-text citation, use \texttt{@citekey} with no brackets, e.g., + ``\texttt{@banerjee2010pitfalls}'' gives us + \citet{banerjee2010pitfalls}. +\end{itemize} + +For more, see +\url{http://rmarkdown.rstudio.com/authoring_bibliographies_and_citations.html}. + +And the bibliography/references will automatically show up with the +sources you've added using cite keys; no need to copy and paste +citations or double check to make sure you've included/removed sources +as necessary!\ldots{} + +\bibliography{my\_bib.bib} + + +\end{document} diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_slides.Rmd b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_slides.Rmd new file mode 100644 index 0000000..4bd6046 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_slides.Rmd @@ -0,0 +1,83 @@ +--- +title: "HTML Slides" +author: "Julia Clark" +date: "March 16, 2017" +output: ioslides_presentation +--- + +```{r setup, include=FALSE} +knitr::opts_chunk$set(echo = FALSE) +knitr::opts_chunk$set(echo = TRUE) # change to echo=FALSE to turn echo permanently off +# Loading required libraries +list.of.packages <- c("foreign", "stargazer", "sandwich", "haven", "ggplot2", "plotly") + +# Chicking if library installed, installed if not. +new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])] +if(length(new.packages)) install.packages(new.packages, repos= "http://cran.cnr.berkeley.edu/") + +# Calling libraries +lapply(list.of.packages, require, character.only = TRUE) +``` + +## HTML Slides in RMarkdown + +You can also use R Markdown to make slides. These can be in pdf (beamer) format, or in HTML, as in this example. The principles are the same; make new slides and add content in Markdown and R.Then click "knit" and open the resulting file in any web browser. + +## Slide with Bullets + +- Bullet 1 +- Bullet 2 +- Bullet 3 + +## Slide with R Output + +```{r cars, echo = TRUE} +summary(cars) +``` + +## Slide with Plotly Histogram +Since we're doing HTML and now PDF, we can make some cool interactive graphs using `plotly`: + +```{r, echo=FALSE, message=FALSE} +# +# need <- c("plotly") # list packages you need +# have <- need %in% rownames(installed.packages()) # see which are already +# if(any(!have)) install.packages(need[!have]) # installs the missing ones +# invisible(lapply(need, library, character.only=T)) # then loads them all + +plot_ly(x = rnorm(500), type = "histogram") + +``` + +## Percent College Graduates in US Midwestern States +```{r, echo = FALSE, message = FALSE} + +plot_ly(midwest, x = ~percollege, color = ~state, type = "box") + +``` + +## US Unemployment Rate Over Time +```{r, echo = FALSE, message = FALSE} + +plot_ly(economics, x = ~date, y = ~unemploy / pop) + +``` + +## Iris Data +```{r, echo = FALSE, message = FALSE} + +plot_ly(data = iris, x = ~Sepal.Length, y = ~Petal.Length, color = ~Species) + +``` + +## A Choropleth Map +```{r, echo = FALSE, message = FALSE} + +plot_ly(type = 'choropleth', + locations = c('AZ', 'CA', 'VT'), + locationmode = 'USA-states', + colorscale = 'Viridis', + z = c(10, 20, 40)) %>% layout(geo=list(scope = 'usa')) + +``` + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_slides.html b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_slides.html new file mode 100644 index 0000000..98845f3 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/Rmarkdown_slides.html @@ -0,0 +1,193 @@ + + + + HTML Slides + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+

+

+

+

March 16, 2017

+
+
+ +

HTML Slides in RMarkdown

+ +

You can also use R Markdown to make slides. These can be in pdf (beamer) format, or in HTML, as in this example. The principles are the same; make new slides and add content in Markdown and R.Then click "knit" and open the resulting file in any web browser.

+ +

Slide with Bullets

+ +
    +
  • Bullet 1
  • +
  • Bullet 2
  • +
  • Bullet 3
  • +
+ +

Slide with R Output

+ +
summary(cars)
+ +
##      speed           dist       
+##  Min.   : 4.0   Min.   :  2.00  
+##  1st Qu.:12.0   1st Qu.: 26.00  
+##  Median :15.0   Median : 36.00  
+##  Mean   :15.4   Mean   : 42.98  
+##  3rd Qu.:19.0   3rd Qu.: 56.00  
+##  Max.   :25.0   Max.   :120.00
+ +

Slide with Plotly Histogram

+ +

Since we're doing HTML and now PDF, we can make some cool interactive graphs using plotly:

+ +
+ + +

Percent College Graduates in US Midwestern States

+ +
+ + +

US Unemployment Rate Over Time

+ +
## Warning in format.POSIXlt(as.POSIXlt(x), ...): unknown timezone 'default/
+## America/La_Paz'
+ +
+ + +

Iris Data

+ +
+ + +

A Choropleth Map

+ +
+
+ + + + +
+ + + + + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/WASHBpublic_mock.dta b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/WASHBpublic_mock.dta new file mode 100644 index 0000000..13a1b10 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/WASHBpublic_mock.dta differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/de-identification_indirect.png b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/de-identification_indirect.png new file mode 100644 index 0000000..8f91fcc Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/de-identification_indirect.png differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/my_bib.bib b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/my_bib.bib new file mode 100644 index 0000000..08e01f3 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/my_bib.bib @@ -0,0 +1,53 @@ +%% This BibTeX bibliography file was created using BibDesk. +%% http://bibdesk.sourceforge.net/ + +%% Created for Julia Clark at 2017-03-17 12:33:42 +0530 +%% Saved with string encoding Unicode (UTF-8) + +@article{banerjee2010pitfalls, + Author = {Banerjee, Abhijit V and Banerji, Rukmini and Duflo, Esther and Glennerster, Rachel and Khemani, Stuti}, + Date-Added = {2015-05-08 17:32:09 +0000}, + Date-Modified = {2015-05-08 17:32:09 +0000}, + Journal = {American Economic Journal: Economic Policy}, + Pages = {1--30}, + Publisher = {JSTOR}, + Title = {Pitfalls of Participatory Programs: Evidence from a randomized evaluation in education in India}, + Year = {2010}} + +@book{easterly2001elusive, + Author = {Easterly, William Russell}, + Date-Added = {2015-01-15 20:12:00 +0000}, + Date-Modified = {2015-01-15 20:13:01 +0000}, + Keywords = {231e, development}, + Publisher = {MIT press}, + Title = {The Elusive Quest for Growth: Economists' Adventures and Misadventures in the Tropics}, + Year = {2001}} + + +@comment{BibDesk Smart Groups{ + + + + + + conditions + + + comparison + 0 + key + Date-Added + value + + version + 1 + + + conjunction + 0 + group name + Added Today + + + +}} diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/outputR.html b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/outputR.html new file mode 100644 index 0000000..101f95b --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/outputR.html @@ -0,0 +1,26 @@ + + + + + + + + + + + + + + + + + + + + + + + + + +
Made Automatically in R
Dependent variable:
free_chl_yn
defaultrobustcontrols
(1)(2)(3)
treatw0.364***0.364***0.365***
(0.040)(0.043)(0.041)
kiswahili-0.011
(0.076)
english0.034
(0.064)
Constant0.0130.013-0.003
(0.027)(0.009)(0.053)
Observations284284284
R20.2230.2230.224
Adjusted R20.2200.2200.216
Residual Std. Error0.340 (df = 282)0.340 (df = 282)0.340 (df = 280)
F Statistic81.002*** (df = 1; 282)81.002*** (df = 1; 282)26.982*** (df = 3; 280)
Note:*p<0.1; **p<0.05; ***p<0.01
diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/outputR.tex b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/outputR.tex new file mode 100644 index 0000000..77e72be --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/outputR.tex @@ -0,0 +1,29 @@ + +\begin{table}[!htbp] \centering + \caption{Made Automatically in R} + \label{} +\begin{tabular}{@{\extracolsep{5pt}}lcc} +\\[-1.8ex]\hline +\hline \\[-1.8ex] + & \multicolumn{2}{c}{\textit{Dependent variable:}} \\ +\cline{2-3} +\\[-1.8ex] & \multicolumn{2}{c}{free\_chl\_yn} \\ +\\[-1.8ex] & (1) & (2)\\ +\hline \\[-1.8ex] + treatw & 0.364$^{***}$ & 0.364$^{***}$ \\ + & (0.043) & (0.043) \\ + & & \\ + Constant & 0.013 & 0.013 \\ + & (0.009) & (0.045) \\ + & & \\ +\hline \\[-1.8ex] +Observations & 284 & 284 \\ +R$^{2}$ & 0.223 & 0.223 \\ +Adjusted R$^{2}$ & 0.220 & 0.220 \\ +Residual Std. Error (df = 282) & 0.340 & 0.340 \\ +F Statistic (df = 1; 282) & 81.002$^{***}$ & 81.002$^{***}$ \\ +\hline +\hline \\[-1.8ex] +\textit{Note:} & \multicolumn{2}{r}{$^{*}$p$<$0.1; $^{**}$p$<$0.05; $^{***}$p$<$0.01} \\ +\end{tabular} +\end{table} diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/r_simple_intro.R b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/r_simple_intro.R new file mode 100644 index 0000000..c39afd0 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/3-Rmarkdown/r_simple_intro.R @@ -0,0 +1,163 @@ +######## INFO ######## + +# Purpose: Super basic and abbreviated intro to R +# Created: 17 March by Julia Clark + +##### What is R? + +# R is not a statistical software package. It is an "object oriented" programming language, based on another language called 'S' and similar to Python. Using base functionality in R as well as lots of user-created packages you can load, R can do statistical analysis, awesome data visualization, scrape the web, run simulations, etc., etc.). This means that it is much more versitile than Stata, but has a steeper learning curve, and you have to manually program some tasks that are canned functions in purpose-build statistical software like Stata. + +# We're working in RStudio, which is an 'integrated development environment' or IDE to run R code, which you could also execute directly in R, or in the command line (shell), or in a number of text editors. But RStudio is easiest to learn for beginners. + +##### Executing code + +# Right now, your reading an R script, which saves with the extension '.R'. Like Stata's do files, we use scripts to write and save code for reproducibility. To run part of your code, select it and press 'command + return' (Mac) or 'control + return' (PC), or click the 'Run' button at the top of this script. Try that now with the following command: + + print('Hello, World!') + +# You can always type commmands directly into the 'console'. Try copying and pasting the above command directly into the console and run it by clicking 'return'. + +######## BASICS ######## + +# A simplistic way of thinking about R is that you can give it two kinds of commands. The first kind (like the above "print('Hello, World!'") are just telling it to execute something and print the result in the console. + +# The second type of command tells it to ASSIGN something to an 'object'. We do this by using the '<-' operator. For example, put your name between the quotations below, and execute: + + my_name <- "Julia" # Creates an object called "my_name" with the value "Julia" + +# If we want to see the contents of that object, we can either command ... + + print(my_name) + +# Or simply type the name of the object + + my_name + +# We can then use this object in a variety of ways. Let's create a few more ... + + greeting <- "Hello" + punctuation <- "!" + greeting + punctuation + +# Now, let's put them together in a new object called 'phrase' using the 'c()' or concantinate function that groups things together: + + phrase <- c(greeting, my_name, punctuation) + phrase + +# So now have a few objects. You can see them by running the command 'ls()' to list all the object you have stored, or by looking at the 'Environment' window in Rstudio. + + ls() + +# Each of these objects is a 'vector' (one row of data) of a different length. Because the type of data they're storing is text, they are 'character' vectors. We can tell that by looking at each object using the 'class()' command, or the 'str()' (structure) command: + + class(greeting) # the 'chr' in the console tells me that this is a character + str(phrase) + +# As you can see, 'greeting' is a character vector of length 1, while 'phrase' is a character vector of length 3. I'm not happy with the 'phrase' object, because I wanted all the words to appear as a single phrase, not as distinct components of a vector. Unlike Stata, R let's me write over an object (as supposed to dropping a variable and then recreating it). So let's do + + phrase <- paste(greeting, my_name, punctuation, # listing character objects to paste together + sep= " ") # telling it to separate them with a space + phrase + +# Let's change the values of our other objects and try to make a new phrase: + + greeting <- "Goodbye" + punctuation <- "?" + phrase + +# Well that didn't work. Why? Because 'phrase' was already defined as an object made up of the old greating and punctuation, and we didn't redefine it. Instead, let's make a function called 'phrase_fun', where these three arguments, named 'greeting', 'my_name', and 'punctuation': + + phrase_fun <- function(greeting, my_name, punctuation){ + paste(greeting, my_name, punctuation, sep= " ") + } + +# Now, we can call the function and enter any values we want for 'greeting', 'my_name', and 'punctuation': + + phrase_fun(greeting = "Goodbye", my_name = "Julia", punctuation = ":(") + phrase_fun(greeting = "What's up", my_name = "Delhi", punctuation = "?") + +# R can also store lots of other types of data like numbers, which we obviously need for stats, and in addition to vector, you can create objects that are matrices, dataframes, and lists. For example: + + x <- rnorm(100) # x values are 100 draws from the normal distribution + x + +# Let's make another vector y, which is a function of x, plus random noise: + + y <- x + rnorm(100) + y + +# In this case, R is adding each value of the x vector to the ordered value of the rnorm values + +# Now let's make a dataframe: + + dat <- data.frame(y, x) + dat + +# Run a simple regression and print a summary of the results + + reg1 <- lm(y ~ x, data=dat) + summary(reg1) + +# It's significant, obviously. + +######## EXAMPLE DATA ######## + +# It's great that you can make fake data in R, but let's try some real data. Let's begin by clearning our workspace and setting our working directory: + + rm(list=ls()) + setwd ("~/Documents/RA/India_BITSS/rmarkdown") + +# Then, we need to load the packages that we'll use for this analysis, including 'foreign' (to read Stata .dta files), 'sandwich' to help with robust SEs, 'ggplot2' (for cool graphs), and 'stargazer' (to format nice regression and summary tables). + +# These packages only need to be installed once on your system, using the 'install.packages()' command, but they need to be loaded using 'library()' for each session. This can be a bit of a pain, since you may not remember what you have installed previously and it's annoying to have a separate 'library()' command for each package. But, if you omit 'install.packages()' in your code because you already have something loaded, someone else trying to run your code that hasn't already installed the package will get an error. So, here's a nice bit of code to deal with this: + + need <- c("foreign", "sandwich", "ggplot2", "stargazer") # list packages you need + have <- need %in% rownames(installed.packages()) # see which are already installed + if(any(!have)) install.packages(need[!have]) # installs the missing ones + invisible(lapply(need, library, character.only=T)) # then loads them all + +# Now we can load our data + + washb <- read.dta("WASHBpublic_mock.dta") + +# Let's have a look at it + head(washb) # look at first few rows + str(washb) # look at structure + plot(haz06 ~ waz06, data = washb) # simple scatter plot + hist(washb$waz06) # simple histogram + +# And run Garret's models + + model1 <- lm(free_chl_yn ~ treatw, data = washb) # basic model, saved to an object + summary(model1) + + model2 <- lm(free_chl_yn ~ treatw + kiswahili + english, data = washb) # adds baseline controls + summary(model2) + + robust.se1 <- sqrt(diag(vcovHC(model1, type = "HC"))) + robust.se2 <- sqrt(diag(vcovHC(model2, type = "HC"))) + +# Let's make a nice table of this in Stargazer, which is analogous to estout in Stata: + + stargazer(model1, model1, + se=list(robust.se1, robust.se2), + title="Made Automatically in R", + out="outputR.tex", header=FALSE) + +# You can copy and paste the output in the console into LaTeX, or you can write it as an object in a .tex., .txt, or .html format) using the "out" option. + +# Or better yet, we can just do this all in RMarkdown! + +##### Next steps + +# There are many, many online resources and courses for learning R if you're interested, and a lot of them are free. One awesome resource are the free 'Swirl' tutorials that you can right in your RStudio console: + + install.packages("swirl") + library("swirl") + swirl() + +# Type 'esc' then 'bye()' to exit swirl + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/BITSS workshop.rmd b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/BITSS workshop.rmd new file mode 100644 index 0000000..83023bc --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/BITSS workshop.rmd @@ -0,0 +1,32 @@ +--- +title: "BITSS workshop" +author: "JPT" +date: "December 3, 2017" +output: html_document +--- + +```{r setup, include=FALSE} +knitr::opts_chunk$set(echo = TRUE) +``` + +## R Markdown + +This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see . + +When you click the **Knit** button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this: + +## This begins a piece of code - three apostrophes + +```{r cars} +summary(cars) +``` + +## Including Plots + +You can also embed plots, for example: + +```{r pressure, echo=FALSE} +plot(pressure) +``` + +Note that the `echo = FALSE` parameter was added to the code chunk to prevent printing of the R code that generated the plot. diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/BITSS_workshop.html b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/BITSS_workshop.html new file mode 100644 index 0000000..d5e1109 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/BITSS_workshop.html @@ -0,0 +1,362 @@ + + + + + + + + + + + + + + +BITSS workshop + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + +
+

R Markdown

+

This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.

+

When you click the Knit button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:

+
summary(cars)
+
##      speed           dist       
+##  Min.   : 4.0   Min.   :  2.00  
+##  1st Qu.:12.0   1st Qu.: 26.00  
+##  Median :15.0   Median : 36.00  
+##  Mean   :15.4   Mean   : 42.98  
+##  3rd Qu.:19.0   3rd Qu.: 56.00  
+##  Max.   :25.0   Max.   :120.00
+
+
+

Including Plots

+

You can also embed plots, for example:

+

+

Note that the echo = FALSE parameter was added to the code chunk to prevent printing of the R code that generated the plot.

+
+ + + + +
+ + + + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/BibASCII.bib b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/BibASCII.bib new file mode 100644 index 0000000..144a43b --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/BibASCII.bib @@ -0,0 +1,17771 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Annual2017/BibASCII.bib at master · BITSS/Annual2017 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Skip to content +
+ + + + +
+
+
+
+ Help save net neutrality! + A free, open internet is once again at stake—and we need your help. +
+
+ Learn more + +
+
+
+ + + + + + + + + + +
+ +
+ +
+
+ + + +
+
+
+ + + + + + + + +
+
+ +
    +
  • +
    + +
    + + + +
    +
    +
    + + Notifications +
    + + + +
    +
    +
    +
    +
  • + +
  • + +
    +
    + + + +
    +
    + + + +
    + +
  • + +
  • + + + Fork + + + + + +
  • +
+ +

+ + /Annual2017 + +

+ +
+ + + + +
+ +
+
+ + + Permalink + + + +
+ +
+ + +
+ +
+
+ + Switch branches/tags +
+ +
+
+ +
+
+ +
+
+ + + +
+
+ + +
+ +
Nothing to show
+
+ +
+
+
+ +
+ + Find file + + +
+ +
+ + + +
+ Fetching contributors… +
+ +
+ + Cannot retrieve contributors at this time +
+
+ +
+
+
+ +
+ Raw + Blame + History +
+ + + + + +
+ +
+ +
+ +
+ 4093 lines (3450 sloc) + + 311 KB +
+
+ + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
% This file was created with JabRef 2.10.
% Encoding: Cp1252
+
+
@Article{registration_2010,
Title = {The Registration of Observational Studies When Metaphors Go Bad:},
Author = {{}},
Journal = {Epidemiology},
Year = {2010},
+
Month = jul,
Pages = {1},
+
Doi = {10.1097/EDE.0b013e3181eafbcf},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {The Registration of Observational StudiesWhen Metaphors Go Bad},
Url = {http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001648-900000000-99655},
Urldate = {2014-10-28}
}
+
@Article{acemoglu_colonial_2012,
Title = {The Colonial Origins of Comparative Development: An Empirical Investigation: Reply},
Author = {Acemoglu, Daron and Johnson, Simon and Robinson, James A.},
Journal = {The American Economic Review},
Year = {2012},
+
Month = oct,
Number = {6},
Pages = {3077--3110},
Volume = {102},
+
Abstract = {Acemoglu, Johnson, and Robinson (2001) established that economic institutions today are correlated with expected mortality of European colonialists. David Albouy argues this relationship is not robust. He drops all data from Latin America and much of the data from Africa, making up almost 60 percent of our sample, despite much information on the mortality of Europeans in those places during the colonial period. He also includes a "campaign" dummy that is coded inconsistently; even modest corrections undermine his claims. We also show that limiting the effect of outliers strengthens our results, making them robust to even extreme versions of Albouy's critiques.},
Doi = {10.1257/aer.102.6.3077},
Shorttitle = {The Colonial Origins of Comparative Development}
}
+
@Article{acemoglu_colonial_2001,
Title = {The Colonial Origins of Comparative Development: An Empirical Investigation},
Author = {Acemoglu, Daron and Johnson, Simon and Robinson, James A},
Journal = {American Economic Review},
Year = {2001},
+
Month = dec,
Number = {5},
Pages = {1369--1401},
Volume = {91},
+
Doi = {10.1257/aer.91.5.1369},
File = {AER (91,5) p. 1369 - The Colonial Origins of Comparative Development\: An Empirical Investigation:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\K96I5TIP\\articles.html:text/html},
ISSN = {0002-8282},
Language = {en},
Shorttitle = {The Colonial Origins of Comparative Development},
Url = {https://www-aeaweb-org.proxy.swarthmore.edu/articles.php?f=s&doi=10.1257/aer.91.5.1369},
Urldate = {2014-08-20}
}
+
@Article{alatas_targeting_2012,
Title = {Targeting the Poor: Evidence from a Field Experiment in Indonesia},
Author = {Alatas, Vivi and Banerjee, Abhijit and Hanna, Rema and Olken, Benjamin A and Tobias, Julia},
Journal = {American Economic Review},
Year = {2012},
+
Month = jun,
Number = {4},
Pages = {1206--1240},
Volume = {102},
+
Doi = {10.1257/aer.102.4.1206},
File = {AER (102,4) p. 1206 - Targeting the Poor\: Evidence from a Field Experiment in Indonesia:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\9DWN64BK\\articles.html:text/html},
ISSN = {0002-8282},
Language = {en},
Shorttitle = {Targeting the Poor},
Url = {https://www.aeaweb.org/articles.php?doi=10.1257/aer.102.4.1206&fnd=s},
Urldate = {2014-10-01}
}
+
@Article{albouy_colonial_2012,
Title = {The Colonial Origins of Comparative Development: An Empirical Investigation: Comment},
Author = {Albouy, David Y.},
Journal = {The American Economic Review},
Year = {2012},
+
Month = oct,
Number = {6},
Pages = {3059--3076},
Volume = {102},
+
Abstract = {Acemoglu, Johnson, and Robinson's (2001) seminal article argues property-rights institutions powerfully affect national income, using estimated mortality rates of early European settlers to instrument capital expropriation risk. However, 36 of the 64 countries in the sample are assigned mortality rates from other countries, often based on mistaken or conflicting evidence. Also, incomparable mortality rates from populations of laborers, bishops, and soldiersoften on campaignare combined in a manner that favors the hypothesis. When these data issues are controlled for, the relationship between mortality and expropriation risk lacks robustness, and instrumental-variable estimates become unreliable, often with infinite confidence intervals.},
Doi = {10.1257/aer.102.6.3059},
Shorttitle = {The Colonial Origins of Comparative Development}
}
+
@Article{almeida_impact_2012,
Title = {The Impact of Vocational Training for the Unemployed in Turkey: Pre-Analysis Plan},
Author = {Almeida, Rita and Hirshleifer, Sarojini and McKenzie, David and Ridao-Cano, Cristobal and Yener, Ahmed Levent},
Journal = {Poverty Action Lab Hypothesis Registry},
Year = {2012},
+
Month = feb,
+
Url = {http://www.povertyactionlab.org/sites/default/files/documents/ISKURIE_AnalysisPlan_v4.pdf}
}
+
@Article{almond_is_2006,
Title = {Is the 1918 Influenza Pandemic Over? Long Term Effects of In Utero Influenza Exposure in the Post1940 U.S. Population},
Author = {Almond, Douglas},
Journal = {Journal of Political Economy},
Year = {2006},
+
Month = aug,
Number = {4},
Pages = {672--712},
Volume = {114},
+
Abstract = {This paper uses the 1918 influenza pandemic as a natural experiment for testing the fetal origins hypothesis. The pandemic arrived unexpectedly in the fall of 1918 and had largely subsided by January 1919, generating sharp predictions for long?term effects. Data from the 196080 decennial U.S. Census indicate that cohorts in utero during the pandemic displayed reduced educational attainment, increased rates of physical disability, lower income, lower socioeconomic status, and higher transfer payments compared with other birth cohorts. These results indicate that investments in fetal health can increase human capital.},
Copyright = {Copyright 2006 The University of Chicago Press},
Doi = {10.1086/jpe.2006.114.issue-4},
ISSN = {0022-3808},
Shorttitle = {Is the 1918 Influenza Pandemic Over?},
Url = {http://www.jstor.org/stable/10.1086/507154},
Urldate = {2014-08-21}
}
+
@Article{anderson_fwer,
Title = {Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects},
Author = {Anderson, Michael L.},
Journal = {Journal of the American Statistical Association},
Year = {2008},
Number = {484},
Pages = {1481-1495},
Volume = {103},
+
Doi = {10.1198/016214508000000841},
Eprint = {
http://dx.doi.org/10.1198/016214508000000841
+
},
Url = {
http://dx.doi.org/10.1198/016214508000000841
+
}
}
+
@Article{anderson_registration_2013,
Title = {Registration and Replication: A Comment},
Author = {Anderson, Richard G.},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {38--39},
Volume = {21},
+
Doi = {10.1093/pan/mps034},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2BVUD8MF\\Anderson - 2013 - Registration and Replication A Comment.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\VCMHDK68\\38.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Shorttitle = {Registration and Replication},
Url = {http://pan.oxfordjournals.org/content/21/1/38},
Urldate = {2014-08-28}
}
+
@Article{anderson_replication_1994,
Title = {Replication and scientific standards in applied economics a decade after the Journal of Money, Credit and Banking project},
Author = {Anderson, Richard G. and Dewald, William G.},
Journal = {Federal Reserve Bank of St. Louis Review},
Year = {1994},
Number = {Nov},
Pages = {79--83},
+
File = {RePEc Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NKMSHAB8\\y_3a1994_3ai_3anov_3ap_3a79-83.html:text/html},
Keywords = {Research},
Url = {http://econpapers.repec.org/article/fipfedlrv/y_3a1994_3ai_3anov_3ap_3a79-83.htm},
Urldate = {2014-08-20}
}
+
@Article{anderson_role_2008,
Title = {The role of data/code archives in the future of economic research},
Author = {Anderson, Richard G. and Greene, William H. and McCullough, B. D. and Vinod, H. D.},
Journal = {Journal of Economic Methodology},
Year = {2008},
Number = {1},
Pages = {99--119},
Volume = {15},
+
Abstract = {This essay examines the role of data and program?code archives in making economic research replicable. Replication of published results is recognized as an essential part of the scientific method. Yet, historically, both the demand for and supply of replicable results in economics has been minimal. Respect for the scientific method is not sufficient to motivate either economists or editors of professional journals to ensure the replicability of published results. We enumerate the costs and benefits of mandatory data and code archives, and argue that the benefits far exceed the costs. Progress has been made since the gloomy assessment of Dewald, Thursby and Anderson some 20 years ago in the American Economic Review, but much remains to be done before empirical economics ceases to be a dismal science when judged by the replicability of its published results.},
Doi = {10.1080/13501780801915574},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NNWSITCD\\Anderson et al. - 2008 - The role of datacode archives in the future of ec.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NT9DFTIR\\13501780801915574.html:text/html},
ISSN = {1350-178X},
Url = {http://dx.doi.org/10.1080/13501780801915574},
Urldate = {2014-08-20}
}
+
@Article{andreoli-versbach_open_????,
Title = {Open access to data: An ideal professed but not practised},
Author = {Andreoli-Versbach, Patrick and Mueller-Langer, Frank},
Journal = {Research Policy},
+
Abstract = {Data-sharing is an essential tool for replication, validation and extension of empirical results. Using a hand-collected data set describing the data-sharing behaviour of 488 randomly selected empirical researchers, we provide evidence that most researchers in economics and management do not share their data voluntarily. We derive testable hypotheses based on the theoretical literature on information-sharing and relate data-sharing to observable characteristics of researchers. We find empirical support for the hypotheses that voluntary data-sharing significantly increases with (a) academic tenure, (b) the quality of researchers, (c) the share of published articles subject to a mandatory data-disclosure policy of journals, and (d) personal attitudes towards open science? principles. On the basis of our empirical evidence, we discuss a set of policy recommendations.},
Doi = {10.1016/j.respol.2014.04.008},
File = {ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\ETGVNNZQ\\S0048733314000699.html:text/html},
ISSN = {0048-7333},
Keywords = {Mandatory data-disclosure, Open science, Voluntary data-sharing},
Shorttitle = {Open access to data},
Url = {http://www.sciencedirect.com/science/article/pii/S0048733314000699},
Urldate = {2014-08-29}
}
+
@Article{angrist_credibility_2010,
Title = {The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics},
Author = {Angrist, Joshua D and Pischke, Jrn-Steffen},
Journal = {Journal of Economic Perspectives},
Year = {2010},
+
Month = may,
Number = {2},
Pages = {3--30},
Volume = {24},
+
Doi = {10.1257/jep.24.2.3},
File = {JEP (24,2) p. 3 - The Credibility Revolution in Empirical Economics\: How Better Research Design Is Taking the Con out of Econometrics:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NNXZ8AAF\\articles.html:text/html},
ISSN = {0895-3309},
Language = {en},
Shorttitle = {The Credibility Revolution in Empirical Economics},
Url = {https://www-aeaweb-org.proxy.swarthmore.edu/articles.php?doi=10.1257/jep.24.2.3},
Urldate = {2014-09-08}
}
+
@Book{angrist2008mostly,
Title = {Mostly harmless econometrics: An empiricist's companion},
Author = {Angrist, Joshua D and Pischke, J{\"o}rn-Steffen},
Publisher = {Princeton university press},
Year = {2008}
}
+
@Article{antonovics_does_2005,
Title = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation? Comment},
Author = {Antonovics, Kate L. and Goldberger, Arthur S.},
Journal = {The American Economic Review},
Year = {2005},
+
Month = dec,
Number = {5},
Pages = {1738--1744},
Volume = {95},
+
Copyright = {Copyright 2005 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\95TJCXRX\\Antonovics and Goldberger - 2005 - Does Increasing Women's Schooling Raise the School.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation?},
Url = {http://www.jstor.org/stable/4132776},
Urldate = {2014-09-23}
}
+
@Article{ashenfelter1999review,
Title = {A review of estimates of the schooling/earnings relationship, with tests for publication bias},
Author = {Ashenfelter, Orley and Harmon, Colm and Oosterbeek, Hessel},
Journal = {Labour economics},
Year = {1999},
Number = {4},
Pages = {453--470},
Volume = {6},
+
Publisher = {Elsevier}
}
+
@Article{ashton_national_2010,
Title = {Do national soccer results really impact on the stock market?},
Author = {Ashton, J. K. and Gerrard, B. and Hudson, R.},
Journal = {Applied Economics},
Year = {2010},
+
Month = nov,
Number = {26},
Pages = {3709--3717},
Volume = {43},
+
Abstract = {This study is a response to Klein et al. (2008), which was highly critical of earlier work by Ashton et al. (2003). This work considering the link between international soccer results and stock market returns was challenged by Klein et al. (2008), who reject the presence and importance of this link. In response, this work provides a reassessment of the link between international soccer results and stock market returns within Ashton et al. (2003). This new analysis extends the original work by using a larger dataset, employing an extended range of tests and allowing for outliers. It is reported that, contrary to the findings of Klein et al. (2008), the link between international soccer results and stock market prices does indeed exist particularly within the sample period 19842002 used by Ashton et al. (2003). After extending the dataset to include observations from 2002 until 2009, it is reported that the effect on stock market returns has declined in importance over this period, particularly the impact of wins.},
Doi = {10.1080/00036841003689762},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\45JFF45V\\Ashton et al. - 2010 - Do national soccer results really impact on the st.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CD2H9AH5\\00036841003689762.html:text/html},
ISSN = {0003-6846},
Url = {http://dx.doi.org/10.1080/00036841003689762},
Urldate = {2014-08-28}
}
+
@Article{ashton_economic_2003,
Title = {Economic impact of national sporting success: evidence from the London stock exchange},
Author = {Ashton, J. K. and Gerrard, B. and Hudson, R.},
Journal = {Applied Economics Letters},
Year = {2003},
+
Month = oct,
Number = {12},
Pages = {783--785},
Volume = {10},
+
Abstract = {In this article strong association is reported between the performance of the England football team and subsequent daily changes in the {FTSE} 100 index, representing the price of shares in the 100 largest companies traded on the London stock exchange.},
Doi = {10.1080/1350485032000126712},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\N86J838Z\\Ashton et al. - 2003 - Economic impact of national sporting success evid.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\VEH27K34\\1350485032000126712.html:text/html},
ISSN = {1350-4851},
Shorttitle = {Economic impact of national sporting success},
Url = {http://dx.doi.org/10.1080/1350485032000126712},
Urldate = {2014-08-28}
}
+
@Article{auspurg_what_2011,
Title = {What Fuels Publication Bias? Theoretical and Empirical Analyses of Risk Factors Using the Caliper Test},
Author = {Auspurg, Katrin and Hinz, Thomas},
Journal = {Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik)},
Year = {2011},
Number = {5-6},
Pages = {636--660},
Volume = {231},
+
Abstract = {Significance tests were originally developed to enable more objective evaluations of research results. Yet the strong orientation towards statistical significance encourages biased results, a phenomenon termed publication bias?. Publication bias occurs whenever the likelihood or time-lag of publication, or the prominence, language, impact factor of journal space or the citation rate of studies depend on the direction and significance of research findings. Although there is much evidence concerning the existence of publication bias in all scientific disciplines and although its detrimental consequences for the progress of the sciences have been known for a long time, all attempts to eliminate the bias have failed. The present article reviews the history and logic of significance testing, the state of research on publication bias, and existing practical recommendations. After demonstrating that more systematical research on the risk factors of publication bias is needed, the paper suggests two new directions for publication bias research. First, a more comprehensive theoretical model based on theories of rational choice and economics as well as on the sociology of science is sketched out. Publication bias is recognized as the outcome of a social dilemma that cannot be overcome by moral pleas alone. Second, detection methods for publication bias going beyond meta-analysis, ones that are more suitable for testing causal hypotheses, are discussed. In particular, the caliper test? seems well-suited for conducting theoretically motivated comparisons across heterogeneous research fields like sociology. Its potential is demonstrated by testing hypotheses on (a) the relevance of explicitly vs. implicitly stated research propositions and on (b) the relevance of the number of authors on incidence rates of publication bias in 50 papers published in leading German sociology journals.},
File = {RePEc Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\XU8VSDT7\\v231y2011i5-6p636-660.html:text/html},
Keywords = {caliper test, Publication bias, rational choice, Significance testing, sociology of science.},
Shorttitle = {What Fuels Publication Bias?},
Url = {http://ideas.repec.org/a/jns/jbstat/v231y2011i5-6p636-660.html},
Urldate = {2014-08-19}
}
+
@Article{ayres_latest_2003,
Title = {The Latest Misfires in Support of the "More Guns, Less Crime" Hypothesis},
Author = {Ayres, Ian and Donohue III, John J.},
Journal = {Stanford Law Review},
Year = {2003},
+
Month = apr,
Number = {4},
Pages = {1371--1398},
Volume = {55},
+
Copyright = {Copyright 2003 Stanford Law Review},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NJN8ETR2\\Ayres and III - 2003 - The Latest Misfires in Support of the More Guns, .pdf:application/pdf},
ISSN = {0038-9765},
Url = {http://www.jstor.org/stable/1229605},
Urldate = {2014-08-21}
}
+
@Article{ayres_shooting_2003,
Title = {Shooting down the "More Guns, Less Crime" Hypothesis},
Author = {Ayres, Ian and Donohue III, John J.},
Journal = {Stanford Law Review},
Year = {2003},
+
Month = apr,
Number = {4},
Pages = {1193--1312},
Volume = {55},
+
Copyright = {Copyright 2003 Stanford Law Review},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\PXQNJN4Q\\Ayres and III - 2003 - Shooting down the More Guns, Less Crime Hypothes.pdf:application/pdf},
ISSN = {0038-9765},
Url = {http://www.jstor.org/stable/1229603},
Urldate = {2014-08-21}
}
+
@Article{baicker_katherine_oregon_2014,
Title = {The Oregon Health Insurance Experiment: Evidence from Criminal Charges Data, Analysis Plan},
Author = {Baicker, Katherine and Finkelstein, Amy and Taubman, Sarah},
Year = {2014},
+
Month = apr,
+
Series = {{NBER} Working Paper},
Url = {http://www.nber.org/oregon/files/oregon_hie_crime_analysis_plan.pdf}
}
+
@Article{doi:10.1056/NEJMsa1212321,
Title = {The Oregon Experiment Effects of Medicaid on Clinical Outcomes},
Author = {Baicker, Katherine and Taubman, Sarah L. and Allen, Heidi L. and Bernstein, Mira and Gruber, Jonathan H. and Newhouse, Joseph P. and Schneider, Eric C. and Wright, Bill J. and Zaslavsky, Alan M. and Finkelstein, Amy N.},
Journal = {New England Journal of Medicine},
Year = {2013},
Note = {PMID: 23635051},
Number = {18},
Pages = {1713-1722},
Volume = {368},
+
Doi = {10.1056/NEJMsa1212321},
Eprint = {http://www.nejm.org/doi/pdf/10.1056/NEJMsa1212321},
Url = {http://www.nejm.org/doi/full/10.1056/NEJMsa1212321}
}
+
@Article{barrick1991big5,
Title = {The big five personality dimensions and job performance: a meta-analysis},
Author = {Barrick, Murray R and Mount, Michael K},
Journal = {Personnel psychology},
Year = {1991},
Number = {1},
Pages = {1--26},
Volume = {44},
+
Publisher = {Wiley Online Library}
}
+
@Article{begg_c_improving_1996,
Title = {Improving the quality of reporting of randomized controlled trials: The consort statement},
Author = {{Begg C} and {Cho M} and {Eastwood S} and {et al}},
Journal = {{JAMA}},
Year = {1996},
+
Month = aug,
Number = {8},
Pages = {637--639},
Volume = {276},
+
Abstract = {{THE} {RANDOMIZED} controlled trial ({RCT}), more than any other methodology, can have a powerful and immediate impact on patient care. Ideally, the report of such an evaluation needs to convey to the reader relevant information concerning the design, conduct, analysis, and generalizability of the trial. This information should provide the reader with the ability to make informed judgments regarding the internal and external validity of the trial. Accurate and complete reporting also benefits editors and reviewers in their deliberations regarding submitted manuscripts. For {RCTs} to ultimately benefit patients, the published report should be of the highest possible standard.For editorial comment see p 649.
Evidence produced repeatedly over the last 30 years indicates a wide chasm between what a trial should report and what is actually published in the literature. In a review of 71 {RCTs} with negative results published between 1960 and 1975, the authors reported that the vast},
Doi = {10.1001/jama.1996.03540080059030},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HCK2H7WC\\Begg C et al. - 1996 - Improving the quality of reporting of randomized c.pdf:application/pdf},
ISSN = {0098-7484},
Shorttitle = {Improving the quality of reporting of randomized controlled trials},
Url = {http://dx.doi.org/10.1001/jama.1996.03540080059030},
Urldate = {2014-08-29}
}
+
@Article{behrman_does_2005,
Title = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation? Reply},
Author = {Behrman, Jere R. and Rosenzweig, Mark R.},
Journal = {The American Economic Review},
Year = {2005},
+
Month = dec,
Number = {5},
Pages = {1745--1751},
Volume = {95},
+
Copyright = {Copyright 2005 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\7ZJXU3S3\\Behrman and Rosenzweig - 2005 - Does Increasing Women's Schooling Raise the School.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation?},
Url = {http://www.jstor.org/stable/4132777},
Urldate = {2014-09-23}
}
+
@Article{behrman_does_2002,
Title = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation?},
Author = {Behrman, Jere R. and Rosenzweig, Mark R.},
Journal = {The American Economic Review},
Year = {2002},
+
Month = mar,
Number = {1},
Pages = {323--334},
Volume = {92},
+
Copyright = {Copyright 2002 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\Z9FUB53C\\Behrman and Rosenzweig - 2002 - Does Increasing Women's Schooling Raise the School.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/3083336},
Urldate = {2014-09-23}
}
+
@Article{benjamini1995controlling,
Title = {Controlling the false discovery rate: a practical and powerful approach to multiple testing},
Author = {Benjamini, Yoav and Hochberg, Yosef},
Journal = {Journal of the Royal Statistical Society. Series B (Methodological)},
Year = {1995},
Pages = {289--300},
+
Publisher = {JSTOR}
}
+
@Article{benjamini2006adaptive,
Title = {Adaptive linear step-up procedures that control the false discovery rate},
Author = {Benjamini, Yoav and Krieger, Abba M and Yekutieli, Daniel},
Journal = {Biometrika},
Year = {2006},
Number = {3},
Pages = {491--507},
Volume = {93},
+
Publisher = {Biometrika Trust}
}
+
@Article{Bernanke2004,
Title = {Editorial Statement},
Author = {Ben S. Bernanke},
Journal = {The American Economic Review},
Year = {2004},
Number = {1},
Pages = {404-404},
Volume = {94},
+
ISSN = {00028282},
Publisher = {American Economic Association},
Url = {http://www.jstor.org/stable/3592790}
}
+
@Article{bhattacharjee_diederik_2013,
Title = {Diederik Stapel Audacious Academic Fraud},
Author = {Bhattacharjee, Yudhijit},
Journal = {The New York Times},
Year = {2013},
+
Month = apr,
+
Abstract = {Diederik Stapel, a Dutch social psychologist, perpetrated an audacious academic fraud by making up studies that told the world what it wanted to hear about human nature.},
Chapter = {Magazine},
File = {New York Times Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KVN625TA\\diederik-stapels-audacious-academic-fraud.html:text/html},
ISSN = {0362-4331},
Keywords = {Frauds and Swindling, Netherlands, Psychology and Psychologists, Research, Stapel, Diederik, Tilburg University},
Url = {http://www.nytimes.com/2013/04/28/magazine/diederik-stapels-audacious-academic-fraud.html},
Urldate = {2014-09-12}
}
+
@Article{bloom1995minimum,
Title = {Minimum detectable effects a simple way to report the statistical power of experimental designs},
Author = {Bloom, Howard S},
Journal = {Evaluation review},
Year = {1995},
Number = {5},
Pages = {547--556},
Volume = {19},
+
Publisher = {Sage Publications}
}
+
@Book{broad_betrayers_1983,
Title = {Betrayers of the truth},
Author = {Broad, William and Broad, William J. and Wade, Nicholas},
Publisher = {Simon and Schuster},
Year = {1983},
Month = jan,
+
Abstract = {"Fraud and deceit in the halls of science"--Cover subtitle.},
ISBN = {9780671447694},
Keywords = {Biography \& Autobiography / Science \& Technology, Deception, Fraud in science, Reference / Research, science, Science / Research \& Methodology},
Language = {en}
}
+
@TechReport{brodeur_star_2012,
Title = {Star Wars: The Empirics Strike Back},
Author = {Brodeur, Abel and Le, Mathias and Sangnier, Marc and Zylberberg, Yanos},
Institution = {Social Science Research Network},
Year = {2012},
+
Address = {Rochester, {NY}},
Month = jun,
Number = {{ID} 2089580},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {Journals favor rejections of the null hypothesis. This selection upon results may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the {AER}, {JPE} and {QJE}, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a camel shape with abundant p-values above .25, a valley between .25 and .10 and a bump slightly under .05. Missing tests are those which would have been accepted but close to being rejected (p-values between .25 and .10). We show that this pattern corresponds to a shift in the distribution of p-values: between 10\% and 20\% of marginally rejected tests are misallocated. Our interpretation is that researchers might be tempted to inflate the value of their tests by choosing the specification that provides the highest statistics. Note that Inflation is larger in articles where stars are used in order to highlight statistical significance and lower in articles with theoretical models.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\64W563JE\\papers.html:text/html},
Keywords = {distorting incentives, Hypothesis testing, research in economics, selection bias},
Shorttitle = {Star Wars},
Url = {http://papers.ssrn.com/abstract=2089580},
Urldate = {2014-08-21}
}
+
@Article{bronars_criminal_1998,
Title = {Criminal Deterrence, Geographic Spillovers, and the Right to Carry Concealed Handguns},
Author = {Bronars, Stephen G. and Lott, Jr., John R.},
Journal = {The American Economic Review},
Year = {1998},
+
Month = may,
Number = {2},
Pages = {475--479},
Volume = {88},
+
Copyright = {Copyright 1998 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\AR4F3WXQ\\Bronars and Lott - 1998 - Criminal Deterrence, Geographic Spillovers, and th.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/116970},
Urldate = {2014-08-21}
}
+
@Article{broockman2015irreg,
Title = {Irregularities in LaCour (2014)},
Author = {Broockman, David and Kalla, Joshua and Aronow, Peter},
Year = {2015},
+
Month = {May},
+
Url = {http://stanford.edu/~dbroock/broockman_kalla_aronow_lg_irregularities.pdf}
}
+
@Article{brown_1918_2010,
Title = {The 1918 U.S. Influenza Pandemic as a Natural Experiment, Revisited},
Author = {Brown, Ryan},
Year = {2010},
+
Month = oct,
+
Abstract = {Douglas Almond's use of the 1918 U.S. in uenza pandemic as a
natural experiment led to the seminal works on the subject of in utero
health's impact on later life outcomes. The identication strength and
clarity of his work, though, is driven by the inherent natural experi-
ment supposition of random assignment. By using data from the 1920
and 1930 U.S. census, this study investigates this keystone assump-
tion and shows that the families of the {\textbackslash}treatment" cohort used by
Douglas Almond were signicantly less literate and economically pros-
perous than the families of the {\textbackslash}control" group. The rst order eect
of parent's education and wealth on a child's long-run {SES} and health
conditions eliminates Almond's ability to make direct inference on fe
tal health's impact on one's long-term wellbeing using the 1918 U.S.
in uenza pandemic.},
Url = {http://mitsloan.mit.edu/neudc/papers/paper_276.pdf}
}
+
@Article{bruhn_pursuit_2009,
Title = {In Pursuit of Balance: Randomization in Practice in Development Field Experiments},
Author = {Bruhn, Miriam and McKenzie, David},
Journal = {American Economic Journal: Applied Economics},
Year = {2009},
+
Month = oct,
Number = {4},
Pages = {200--232},
Volume = {1},
+
Abstract = {We present new evidence on the randomization methods used in existing experiments, and new simulations comparing these methods. We find that many papers do not describe the randomization in detail, implying that better reporting is needed. Our simulations suggest that in samples of 300 or more, the different methods perform similarly. However, for very persistent outcome variables, and in smaller samples, pair-wise matching and stratification perform best and appear to dominate the rerandomization methods commonly used in practice. The simulations also point to specific recommendations for which variables to balance on, and for which controls to include in the ex post analysis.},
Doi = {10.1257/app.1.4.200},
Shorttitle = {In Pursuit of Balance}
}
+
@Article{burnside_aid_2004,
Title = {Aid, Policies, and Growth: Reply},
Author = {Burnside, Craig and Dollar, David},
Journal = {The American Economic Review},
Year = {2004},
+
Month = jun,
Number = {3},
Pages = {781--784},
Volume = {94},
+
Copyright = {Copyright 2004 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JGFHQF96\\Burnside and Dollar - 2004 - Aid, Policies, and Growth Reply.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Aid, Policies, and Growth},
Url = {http://www.jstor.org/stable/3592955},
Urldate = {2014-08-20}
}
+
@Article{burnside_aid_2000,
Title = {Aid, Policies, and Growth},
Author = {Burnside, Craig and Dollar, David},
Journal = {The American Economic Review},
Year = {2000},
+
Month = sep,
Number = {4},
Pages = {847--868},
Volume = {90},
+
Abstract = {This paper uses a new database on foreign aid to examine the relationships among foreign aid, economic policies, and growth of per capita {GDP}. We find that aid has a positive impact on growth in developing countries with good fiscal, monetary, and trade policies but has little effect in the presence of poor policies. Good policies are ones that are themselves important for growth. The quality of policy has only a small impact on the allocation of aid. Our results suggest that aid would be more effective if it were more systematically conditioned on good policy.},
Copyright = {Copyright 2000 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2Q463KRC\\Burnside and Dollar - 2000 - Aid, Policies, and Growth.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/117311},
Urldate = {2014-08-20}
}
+
@Article{button_power_2013,
Title = {Power failure: why small sample size undermines the reliability of neuroscience},
Author = {Button, Katherine S and Ioannidis, John PA and Mokrysz, Claire and Nosek, Brian A and Flint, Jonathan and Robinson, Emma SJ and Munaf{\`o}, Marcus R},
Journal = {Nature Reviews Neuroscience},
Year = {2013},
Number = {5},
Pages = {365--376},
Volume = {14},
+
Publisher = {Nature Publishing Group}
}
+
@Article{camfield_three_2013,
Title = {Three R's of Econometrics: Repetition, Reproduction and Replication},
Author = {Camfield, Laura and Palmer-Jones, Richard},
Journal = {Journal of Development Studies},
Year = {2013},
Number = {12},
Pages = {1607--1614},
Volume = {49},
+
Abstract = {Development economics has become increasingly quantified in recent years, reflecting the aspirations of economists to practise hard science. We argue that standard applied econometric methodology lacks one key feature of the claim of science to be scientific, namely replication as part of independent confirmation of findings. Replication plays a large role in understanding the confidence we can place in the quantitative studies on which much policy advice rests, which is particularly salient in a {UK} context given the emphasis placed on evidence-based policy-making by the {UK} Department for International Development.},
Doi = {10.1080/00220388.2013.807504},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\B3TF5BD3\\Camfield and Palmer-Jones - 2013 - Three Rs of Econometrics Repetition, Reproducti.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\XHAG6H2I\\00220388.2013.html:text/html},
ISSN = {0022-0388},
Shorttitle = {Three Rs of Econometrics},
Url = {http://dx.doi.org/10.1080/00220388.2013.807504},
Urldate = {2014-08-21}
}
+
@Article{CanadianMinWage2006,
Title = {Minimum Wage Impacts from a Prespecified Research Design: Canada 19811997},
Author = {CAMPOLIETI, MICHELE and GUNDERSON, MORLEY and RIDDELL, CHRIS},
Journal = {Industrial Relations: A Journal of Economy and Society},
Year = {2006},
Number = {2},
Pages = {195--216},
Volume = {45},
+
Doi = {10.1111/j.1468-232X.2006.00424.x},
ISSN = {1468-232X},
Publisher = {Blackwell Publishing Inc},
Url = {http://dx.doi.org/10.1111/j.1468-232X.2006.00424.x}
}
+
@Article{card_minimum_2000,
Title = {Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Reply},
Author = {Card, David and Krueger, Alan B.},
Journal = {The American Economic Review},
Year = {2000},
+
Month = dec,
Number = {5},
Pages = {1397--1420},
Volume = {90},
+
Copyright = {Copyright 2000 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8UMCAMVW\\Card and Krueger - 2000 - Minimum Wages and Employment A Case Study of the .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Minimum Wages and Employment},
Url = {http://www.jstor.org/stable/2677856},
Urldate = {2014-08-20}
}
+
@Article{card1995time,
Title = {Time-series minimum-wage studies: a meta-analysis},
Author = {Card, David and Krueger, Alan B},
Journal = {The American Economic Review},
Year = {1995},
Pages = {238--243},
+
Publisher = {JSTOR}
}
+
@Article{card_minimum_1994,
Title = {Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania},
Author = {Card, David and Krueger, Alan B.},
Journal = {The American Economic Review},
Year = {1994},
+
Month = sep,
Number = {4},
Pages = {772--793},
Volume = {84},
+
Abstract = {On April 1, 1992, New Jersey's minimum wage rose from \$4.25 to \$5.05 per hour. To evaluate the impact of the law we surveyed 410 fast-food restaurants in New Jersey and eastern Pennsylvania before and after the rise. Comparisons of employment growth at stores in New Jersey and Pennsylvania (where the minimum wage was constant) provide simple estimates of the effect of the higher minimum wage. We also compare employment changes at stores in New Jersey that were initially paying high wages (above \$5) to the changes at lower-wage stores. We find no indication that the rise in the minimum wage reduced employment.},
Copyright = {Copyright 1994 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\3GJU4UC9\\Card and Krueger - 1994 - Minimum Wages and Employment A Case Study of the .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Minimum Wages and Employment},
Url = {http://www.jstor.org/stable/2118030},
Urldate = {2014-08-20}
}
+
@Article{Card1992minwage,
Title = {Using Regional Variation in Wages to Measure the Effects of the Federal Minimum Wage},
Author = {Card, David AND Alan B Krueger},
Journal = {Industrial \& Labor Relations Review},
Year = {1992},
+
Month = {October},
Number = {1},
Pages = {22-37},
Volume = {46},
+
Owner = {garret},
Timestamp = {2015.07.03}
}
+
@Article{carey_noted_2011,
Title = {Noted Dutch Psychologist, Stapel, Accused of Research Fraud},
Author = {Carey, Benedict},
Journal = {The New York Times},
Year = {2011},
+
Month = nov,
+
Abstract = {A Dutch scholar was found to have falsified findings in dozens of papers, in a field that critics say is vulnerable to such abuses.},
Chapter = {Health / Research},
File = {New York Times Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HJIS8BZD\\noted-dutch-psychologist-stapel-accused-of-research-fraud.html:text/html},
ISSN = {0362-4331},
Keywords = {Falsification of Data, Frauds and Swindling, Psychology and Psychologists, Research, Stapel, Diederik},
Url = {http://www.nytimes.com/2011/11/03/health/research/noted-dutch-psychologist-stapel-accused-of-research-fraud.html},
Urldate = {2014-09-12}
}
+
@Article{carpenter_psychologys_2012,
Title = {Psychology's Bold Initiative},
Author = {Carpenter, Siri},
Journal = {Science},
Year = {2012},
+
Month = mar,
Number = {6076},
Pages = {1558--1561},
Volume = {335},
+
Abstract = {Some psychology researchers argue that a scientific culture that too heavily favors new and counterintuitive ideas over the confirmation of existing results has led to too many findings that are striking for their novelty and published in respected journalsbut are nonetheless false. A new online site ({PsychFileDrawer}.org) lets psychologists post results of replications of experimentswhether they succeed or failthat would ordinarily never leave their file drawer because most journals decline to publish straightforward replication studies. And a group of more than 50 academic psychologists, which calls itself the Open Science Collaboration, has begun an unprecedented, large-scale project to systematically replicate psychological experiments recently published in leading journals. Some researchers are optimistic that many published findings will be replicated. But others are concerned that if the project confirms few studies, it could unfairly indict psychology. Indeed, the prospect of exposing psychology's foibles has upset some scientists.},
Doi = {10.1126/science.335.6076.1558},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6E47UU9Q\\Carpenter - 2012 - Psychology's Bold Initiative.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WNTFGWVE\\1558.html:text/html},
ISSN = {0036-8075, 1095-9203},
Language = {en},
Pmid = {22461583},
Url = {http://www.sciencemag.org/content/335/6076/1558},
Urldate = {2014-09-08}
}
+
@Article{casey_reshaping_2012,
Title = {Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan*},
Author = {Casey, Katherine and Glennerster, Rachel and Miguel, Edward},
Journal = {The Quarterly Journal of Economics},
Year = {2012},
+
Month = nov,
Number = {4},
Pages = {1755--1812},
Volume = {127},
+
Abstract = {Despite their importance, there is limited evidence on how institutions can be strengthened. Evaluating the effects of specific reforms is complicated by the lack of exogenous variation in institutions, the difficulty of measuring institutional performance, and the temptation to cherry pick? estimates from among the large number of indicators required to capture this multifaceted subject. We evaluate one attempt to make local institutions more democratic and egalitarian by imposing participation requirements for marginalized groups (including women) and test for learning-by-doing effects. We exploit the random assignment of a governance program in Sierra Leone, develop innovative real-world outcome measures, and use a preanalysis plan ({PAP}) to bind our hands against data mining. The intervention studied is a community-driven development? program, which has become a popular strategy for foreign aid donors. We find positive short-run effects on local public goods and economic outcomes, but no evidence for sustained impacts on collective action, decision making, or the involvement of marginalized groups, suggesting that the intervention did not durably reshape local institutions. We discuss the practical trade-offs faced in implementing a {PAP} and show how in its absence we could have generated two divergent, equally erroneous interpretations of program impacts on institutions. {JEL} Codes: F35, H41, O4},
Doi = {10.1093/qje/qje027},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\MHIPH78X\\1755.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {Reshaping Institutions},
Url = {http://qje.oxfordjournals.org/content/127/4/1755},
Urldate = {2014-08-26}
}
+
@Article{chambers2014instead,
Title = {Instead of" playing the game" it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond},
Author = {Chambers, Christopher D and Feredoes, Eva and Muthukumaraswamy, Suresh Daniel and Etchells, Peter},
Journal = {AIMS Neuroscience},
Year = {2014},
Number = {1},
Pages = {4--17},
Volume = {1},
+
Publisher = {AIMS Press}
}
+
@Article{chan_a_empirical_2004,
Title = {Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles},
Author = {{Chan A} and {Hrbjartsson A} and {Haahr MT} and {Gtzsche PC} and {Altman DG}},
Journal = {{JAMA}},
Year = {2004},
+
Month = may,
Number = {20},
Pages = {2457--2465},
Volume = {291},
+
Abstract = {ContextSelective reporting of outcomes within published studies based on the
nature or direction of their results has been widely suspected, but direct
evidence of such bias is currently limited to case reports.ObjectiveTo study empirically the extent and nature of outcome reporting bias
in a cohort of randomized trials.DesignCohort study using protocols and published reports of randomized trials
approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg,
Denmark, in 1994-1995. The number and characteristics of reported and unreported
trial outcomes were recorded from protocols, journal articles, and a survey
of trialists. An outcome was considered incompletely reported if insufficient
data were presented in the published articles for meta-analysis. Odds ratios
relating the completeness of outcome reporting to statistical significance
were calculated for each trial and then pooled to provide an overall estimate
of bias. Protocols and published articles were also compared to identify discrepancies
in primary outcomes.Main Outcome MeasuresCompleteness of reporting of efficacy and harm outcomes and of statistically
significant vs nonsignificant outcomes; consistency between primary outcomes
defined in the most recent protocols and those defined in published articles.ResultsOne hundred two trials with 122 published journal articles and 3736
outcomes were identified. Overall, 50\% of efficacy and 65\% of harm outcomes
per trial were incompletely reported. Statistically significant outcomes had
a higher odds of being fully reported compared with nonsignificant outcomes
for both efficacy (pooled odds ratio, 2.4; 95\% confidence interval [{CI}], 1.4-4.0)
and harm (pooled odds ratio, 4.7; 95\% {CI}, 1.8-12.0) data. In comparing published
articles with protocols, 62\% of trials had at least 1 primary outcome that
was changed, introduced, or omitted. Eighty-six percent of survey responders
(42/49) denied the existence of unreported outcomes despite clear evidence
to the contrary.ConclusionsThe reporting of trial outcomes is not only frequently incomplete but
also biased and inconsistent with protocols. Published articles, as well as
reviews that incorporate them, may therefore be unreliable and overestimate
the benefits of an intervention. To ensure transparency, planned trials should
be registered and protocols should be made publicly available prior to trial
completion.},
Doi = {10.1001/jama.291.20.2457},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\AFKZWBJ7\\Chan A et al. - 2004 - Empirical evidence for selective reporting of outc.pdf:application/pdf},
ISSN = {0098-7484},
Shorttitle = {Empirical evidence for selective reporting of outcomes in randomized trials},
Url = {http://dx.doi.org/10.1001/jama.291.20.2457},
Urldate = {2014-10-09}
}
+
@Article{chan_spirit_2013,
Title = {{SPIRIT} 2013 explanation and elaboration: guidance for protocols of clinical trials},
Author = {Chan, A.-W. and Tetzlaff, J. M. and Gotzsche, P. C. and Altman, D. G. and Mann, H. and Berlin, J. A. and Dickersin, K. and Hrobjartsson, A. and Schulz, K. F. and Parulekar, W. R. and Krleza-Jeric, K. and Laupacis, A. and Moher, D.},
Journal = {{BMJ}},
Year = {2013},
+
Month = jan,
Number = {jan08 15},
Pages = {e7586--e7586},
Volume = {346},
+
Doi = {10.1136/bmj.e7586},
ISSN = {1756-1833},
Language = {en},
Shorttitle = {{SPIRIT} 2013 explanation and elaboration},
Url = {http://www.bmj.com/cgi/doi/10.1136/bmj.e7586},
Urldate = {2014-10-09}
}
+
@Article{chemin_response_2012,
Title = {Response to High Noon for Microfinance Impact Evaluations},
Author = {Chemin, Matthieu},
Journal = {Journal of Development Studies},
Year = {2012},
Number = {12},
Pages = {1881--1885},
Volume = {48},
+
Abstract = {Abstract Duvendack and Palmer-Jones are critical of analysis and conclusions in Chemin (2008) because they are unable to replicate my results. This response identifies key differences between the two papers, especially regarding the sample and measurement of variables, which imply that Duvendack and Paler-Jones should not be considered as either a replication or a criticism of my work.},
Doi = {10.1080/00220388.2012.727561},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\48233AME\\Chemin - 2012 - Response to High Noon for Microfinance Impact Eva.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WG76XX86\\00220388.2012.html:text/html},
ISSN = {0022-0388},
Url = {http://dx.doi.org/10.1080/00220388.2012.727561},
Urldate = {2014-08-21}
}
+
@Article{chemin_benefits_2008,
Title = {The Benefits and Costs of Microfinance: Evidence from Bangladesh},
Author = {Chemin, Matthieu},
Journal = {Journal of Development Studies},
Year = {2008},
Number = {4},
Pages = {463--484},
Volume = {44},
+
Abstract = {Abstract Using the latest developments from the evaluation literature, namely the technique of matching, this paper shows a positive, but lower than previously thought, effect of microfinance on expenditure per capita, supply of labour, and level of school enrolment for boys and girls. For instance, participants spend 3 per cent more on average than non-participants in control villages. This paper also takes into account repayment delays to calculate the cost of credit provision. It shows how a better investigation at the individual level of the benefits brought and the cost borne could help microfinance institutions to better select their customers. {JEL} Classification : C14, D10, G21, I38, O12, O16},
Doi = {10.1080/00220380701846735},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\A2VH4E68\\Chemin - 2008 - The Benefits and Costs of Microfinance Evidence f.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\B967ER6G\\00220380701846735.html:text/html},
ISSN = {0022-0388},
Shorttitle = {The Benefits and Costs of Microfinance},
Url = {http://dx.doi.org/10.1080/00220380701846735},
Urldate = {2014-08-21}
}
+
@Article{ciccone_economic_2011,
Title = {Economic Shocks and Civil Conflict: A Comment},
Author = {Ciccone, Antonio},
Journal = {American Economic Journal: Applied Economics},
Year = {2011},
+
Month = oct,
Number = {4},
Pages = {215--227},
Volume = {3},
+
Abstract = {Edward Miguel, Shanker Satyanath, and Ernest Sergenti (2004), henceforth {MSS}, argue that lower rainfall levels and negative rainfall shocks increase conflict risk in sub-Saharan Africa. This conclusion rests on their finding of a negative correlation between conflict in t and rainfall growth between t 1 and t 2. I show that this finding is driven by a (counterintuitive) positive correlation between conflict in t and rainfall levels in t 2. If lower rainfall levels or negative rainfall shocks increased conflict, {MSS}'s finding should have been due to a negative correlation between conflict in t and rainfall levels in t 1. In the latest data, conflict is unrelated to rainfall.},
Doi = {10.1257/app.3.4.215},
Shorttitle = {Economic Shocks and Civil Conflict}
}
+
@Article{ClemensRepTax,
Title = {The Meaning of Failed Replications: A Review and Proposal},
Author = {Michael Clemens},
Journal = {Journal of Economic Surveys},
Year = {2016},
+
Owner = {garret},
Timestamp = {2016.01.20}
}
+
@Article{clemens_replication,
Title = {The Meaning of Failed Replications: A Review and Proposal},
Author = {Michael Clemens},
Journal = {CGD Working Paper 399.},
Year = {2015},
+
Month = {April},
Number = {399},
+
Owner = {garret},
Timestamp = {2015.07.13},
Url = {http://www.cgdev.org/sites/default/files/CGD-Working-Paper-399-Clemens-Meaning-Failed-Replications.pdf}
}
+
@Article{collaboration_open_2012,
Title = {An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science},
Author = {Collaboration, Open Science},
Journal = {Perspectives on Psychological Science},
Year = {2012},
+
Month = nov,
Number = {6},
Pages = {657--660},
Volume = {7},
+
Abstract = {Reproducibility is a defining feature of science. However, because of strong incentives for innovation and weak incentives for confirmation, direct replication is rarely practiced or published. The Reproducibility Project is an open, large-scale, collaborative effort to systematically examine the rate and predictors of reproducibility in psychological science. So far, 72 volunteer researchers from 41 institutions have organized to openly and transparently replicate studies published in three prominent psychological journals in 2008. Multiple methods will be used to evaluate the findings, calculate an empirical rate of replication, and investigate factors that predict reproducibility. Whatever the result, a better understanding of reproducibility will ultimately improve confidence in scientific methodology and findings.},
Doi = {10.1177/1745691612462588},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JFFM4EBN\\Collaboration - 2012 - An Open, Large-Scale, Collaborative Effort to Esti.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KVKQRFJ8\\657.html:text/html},
ISSN = {1745-6916, 1745-6924},
Keywords = {methodology, open, psychological science, replication, reproducibility},
Language = {en},
Url = {http://pps.sagepub.com/content/7/6/657},
Urldate = {2014-09-08}
}
+
@Book{openmaximizing,
Title = {Maximizing the Reproducibility of Your Research},
Author = {Open Science Collaboration and others},
Editor = {Lilienfield, S.O. and Waldman, I.D.},
Publisher = {Wiley},
Year = {2014},
+
Address = {New York, NY},
+
Journal = {Psychological science under scrutiny: Recent challenges and proposed solutions.},
Url = {https://osf.io/nte3j/}
}
+
@Article{cyranoski_cloning_2014,
Title = {Cloning comeback},
Author = {Cyranoski, David},
Journal = {Nature},
Year = {2014},
+
Month = jan,
Number = {7484},
Pages = {468--471},
Volume = {505},
+
Doi = {10.1038/505468a},
File = {Cloning comeback \: Nature News & Comment:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\PCNSJG73\\cloning-comeback-1.html:text/html},
ISSN = {0028-0836, 1476-4687},
Url = {http://www.nature.com/news/cloning-comeback-1.14504},
Urldate = {2014-09-12}
}
+
@Article{dahl_rasmussen_walking_2011,
Title = {Walking the talk: the need for a trial registry for development interventions},
Author = {Dahl Rasmussen, Ole and Malchow-Mller, Nikolaj and Barnebeck Andersen, Thomas},
Journal = {Journal of Development Effectiveness},
Year = {2011},
+
Month = sep,
Number = {4},
Pages = {502--519},
Volume = {3},
+
Abstract = {Recent advances in the use of randomised control trials to evaluate the effect of development interventions promise to enhance our knowledge of what works and why. A core argument supporting randomised studies is the claim that they have high internal validity. The authors argue that this claim is weak as long as a trial registry of development interventions is not in place. Without a trial registry, the possibilities for data mining, created by analyses of multiple outcomes and subgroups, undermine internal validity. Drawing on experience from evidence-based medicine and recent examples from microfinance, they argue that a trial registry would also enhance external validity and foster innovative research.},
Doi = {10.1080/19439342.2011.605160},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\I3NG7FSN\\Dahl Rasmussen et al. - 2011 - Walking the talk the need for a trial registry fo.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2RWVI6V9\\19439342.2011.html:text/html},
ISSN = {1943-9342},
Shorttitle = {Walking the talk},
Url = {http://dx.doi.org/10.1080/19439342.2011.605160},
Urldate = {2014-08-27}
}
+
@Article{dal-re_making_2014,
Title = {Making Prospective Registration of Observational Research a Reality},
Author = {Dal-Re, Rafael and Ioannidis, John P. and Bracken, Michael B. and Buffler, Patricia A. and Chan, An-Wen and Franco, Eduardo L. and Vecchia, Carlo La and Weiderpass, Elisabete},
Journal = {Science Translational Medicine},
Year = {2014},
+
Month = feb,
Number = {224},
Pages = {224cm1--224cm1},
Volume = {6},
+
Abstract = {{AbstractBack} to Top
The vast majority of health-related observational studies are not prospectively registered and the advantages of registration have not been fully appreciated. Nonetheless, international standards require approval of study protocols by an independent ethics committee before the study can begin. We suggest that there is an ethical and scientific imperative to publicly preregister key information from newly approved protocols, which should be required by funders. Ultimately, more complete information may be publicly available by disclosing protocols, analysis plans, data sets, and raw data.},
Doi = {10.1126/scitranslmed.3007513},
ISSN = {1946-6234, 1946-6242},
Language = {en},
Pmid = {24553383},
Url = {http://stm.sciencemag.org/content/6/224/224cm1},
Urldate = {2014-10-28}
}
+
@Article{DeAngelis2004,
Title = {Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors},
Author = {De Angelis, Catherine AND Jeffrey M. Drazen AND Frank A. Frizelle AND Charlotte Haug AND John Hoey AND Richard Horton AND Sheldon Kotzin},
Journal = {New England Journal of Medicine},
Year = {2004},
Note = {doi:10.1056/NEJMe048225},
Number = {12},
Pages = {1250-1251},
Volume = {351},
+
Owner = {garret},
Timestamp = {2014.12.02}
}
+
@Misc{deaton_letter_2012,
Title = {Letter from America - Royal Economic Society},
+
Author = {Deaton, Angus},
Month = oct,
Year = {2012},
+
Abstract = {Angus Deaton reports on the creeping regulation affecting academic research in {US} universities.},
File = {Letter from America - Royal Economic Society:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\3EU96EGG\\article1Oct12Correspondence.html:text/html},
Journal = {Your wolf is interfering with my t-value!},
Url = {http://www.res.org.uk/view/article1Oct12Correspondence.html},
Urldate = {2014-10-23}
}
+
@Article{Deere1995minwage,
Title = {Employment and the 1990-1991 Minimum-Wage Hike},
Author = {Deere, Donald AND Kevin M. Murphy and Finis Welch. 1995},
Journal = {American Economic Review Papers and Proceedings},
Year = {1995},
+
Month = {May},
Number = {2},
Pages = {232-237},
Volume = {85},
+
Owner = {garret},
Timestamp = {2015.07.03}
}
+
@Article{delong_are_1992,
Title = {Are all Economic Hypotheses False?},
Author = {DeLong, J. Bradford and Lang, Kevin},
Journal = {Journal of Political Economy},
Year = {1992},
+
Month = dec,
Number = {6},
Pages = {1257--1272},
Volume = {100},
+
Abstract = {We develop an estimator that allows us to calculate an upper bound to the fraction of unrejected null hypotheses tested in economics journal articles that are in fact true. Our point estimate is that none of the unrejected nulls in our sample is true. We reject the hypothesis that more than one-third are true. We consider three explanations for this finding: that all null hypotheses are mere approximations, that data-mining biases reported standard errors downward, and that journals tend to publish papers that fail to reject their null hypotheses only when the null hypotheses are likely to be false. While all these explanations are important, the last seems best able to explain our findings.},
Copyright = {Copyright 1992 The University of Chicago Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FUXR964Z\\Long and Lang - 1992 - Are all Economic Hypotheses False.pdf:application/pdf},
ISSN = {0022-3808},
Url = {http://www.jstor.org/stable/2138833},
Urldate = {2014-08-19}
}
+
@Article{denton_data_1985,
Title = {Data Mining as an Industry},
Author = {Denton, Frank T.},
Journal = {The Review of Economics and Statistics},
Year = {1985},
+
Month = feb,
Number = {1},
Pages = {124--127},
Volume = {67},
+
Abstract = {"Data mining" by an individual investigator can distort the probabilities in conventional significance tests. This paper argues that the same effect can occur when a given data set is used by more than one investigator, even if no individual investigator engages in data mining. A problem of publication selection bias is recalled and note is taken of its implications for the interpretation of published test results when there is collective data mining. Some illustrative calculations of probabilities associated with collective data mining are provided.},
Copyright = {Copyright 1985 The {MIT} Press},
Doi = {10.2307/1928442},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BWD85GFD\\Denton - 1985 - Data Mining as an Industry.pdf:application/pdf},
ISSN = {0034-6535},
Url = {http://www.jstor.org/stable/1928442},
Urldate = {2014-08-19}
}
+
@Article{dewald_replication_1986,
Title = {Replication in Empirical Economics: The Journal of Money, Credit and Banking Project},
Author = {Dewald, William G. and Thursby, Jerry G. and Anderson, Richard G.},
Journal = {The American Economic Review},
Year = {1986},
+
Month = sep,
Number = {4},
Pages = {587--603},
Volume = {76},
+
Abstract = {This paper examines the role of replication in empirical economic research. It presents the findings of a two-year study that collected programs and data from authors and attempted to replicate their published results. Our research provides new and important information about the extent and causes of failures to replicate published results in economics. Our findings suggest that inadvertent errors in published empirical articles are a commonplace rather thana rare occurrence.},
Copyright = {Copyright 1986 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\TT6PZMZX\\Dewald et al. - 1986 - Replication in Empirical Economics The Journal of.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Replication in Empirical Economics},
Url = {http://www.jstor.org/stable/1806061},
Urldate = {2014-08-20}
}
+
@Article{dezhbakhsh_lives_1998,
Title = {Lives Saved or Lives Lost? The Effects of Concealed-Handgun Laws on Crime},
Author = {Dezhbakhsh, Hashem and Rubin, Paul H.},
Journal = {The American Economic Review},
Year = {1998},
+
Month = may,
Number = {2},
Pages = {468--474},
Volume = {88},
+
Copyright = {Copyright 1998 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FWNVERZE\\Dezhbakhsh and Rubin - 1998 - Lives Saved or Lives Lost The Effects of Conceale.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Lives Saved or Lives Lost?},
Url = {http://www.jstor.org/stable/116969},
Urldate = {2014-08-21}
}
+
@Article{donohue_iii_final_2003,
Title = {The Final Bullet in the Body of the More Guns, Less Crime Hypothesis},
Author = {Donohue III, John J.},
Journal = {Criminology \& Public Policy},
Year = {2003},
+
Month = jul,
Number = {3},
Pages = {397--410},
Volume = {2},
+
Doi = {10.1111/j.1745-9133.2003.tb00003.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\48D2B47B\\Iii - 2003 - The Final Bullet in the Body of the More Guns, Les.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WFSSXZI6\\abstract.html:text/html},
ISSN = {1745-9133},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1745-9133.2003.tb00003.x/abstract},
Urldate = {2014-08-21}
}
+
@Article{donohue_iii_further_2004,
Title = {Further Evidence That Legalized Abortion Lowered Crime: A Reply to Joyce},
Author = {Donohue III, John J. and Levitt, Steven D.},
Journal = {The Journal of Human Resources},
Year = {2004},
+
Month = jan,
Number = {1},
Pages = {29--49},
Volume = {39},
+
Abstract = {Donohue and Levitt (2001) suggest there is a causal link between legalized abortion and reductions in crime almost two decades later when the cohorts exposed to legalized abortion reach their peak crime years. Joyce (2003) examines crime committed in the period 1985-90 for the cohorts born immediately before and after abortion legalization. He finds little impact of legalized abortion. In this paper, we demonstrate that Joyce's failure to uncover a negative relationship between abortion and crime is a consequence of his decision to focus almost exclusively on one nonrepresentative six-year period during the peak of the crack epidemic. We provide empirical evidence that the crack-cocaine epidemic hit the high-abortion early-legalizing states earlier and more severely than other states. When we simply replicate his analyses, but extend the sample to cover the entire lives of these exact same cohorts, abortion is just as negatively related to crime as in our original analysis. Joyce's results appear to be purely an artifact of omitted variable bias due to focusing on the peak crack years without including adequate controls for crack.},
Copyright = {Copyright 2004 The Board of Regents of the University of Wisconsin System},
Doi = {10.2307/3559004},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\UFCIIPCZ\\III and Levitt - 2004 - Further Evidence That Legalized Abortion Lowered C.pdf:application/pdf},
ISSN = {0022-166X},
Shorttitle = {Further Evidence That Legalized Abortion Lowered Crime},
Url = {http://www.jstor.org/stable/3559004},
Urldate = {2014-08-21}
}
+
@Article{donohue_iii_impact_2001,
Title = {The Impact of Legalized Abortion on Crime},
Author = {Donohue III, John J. and Levitt, Steven D.},
Journal = {The Quarterly Journal of Economics},
Year = {2001},
+
Month = may,
Number = {2},
Pages = {379--420},
Volume = {116},
+
Abstract = {We offer evidence that legalized abortion has contributed significantly to recent crime reductions. Crime began to fall roughly eighteen years after abortion legalization. The five states that allowed abortion in 1970 experienced declines earlier than the rest of the nation, which legalized in 1973 with Roe v. Wade. States with high abortion rates in the 1970s and 1980s experienced greater crime reductions in the 1990s. In high abortion states, only arrests of those born after abortion legalization fall relative to low abortion states. Legalized abortion appears to account for as much as 50 percent of the recent drop in crime.},
Copyright = {Copyright 2001 Oxford University Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WSG9VI76\\III and Levitt - 2001 - The Impact of Legalized Abortion on Crime.pdf:application/pdf},
ISSN = {0033-5533},
Url = {http://www.jstor.org/stable/2696468},
Urldate = {2014-08-21}
}
+
@Article{donohue_measurement_2008,
Title = {Measurement Error, Legalized Abortion, and the Decline in Crime: A Response to Foote and Goetz},
Author = {Donohue, John J. and Levitt, Steven D.},
Journal = {The Quarterly Journal of Economics},
Year = {2008},
+
Month = feb,
Number = {1},
Pages = {425--440},
Volume = {123},
+
Abstract = {We are grateful to Foote and Goetz for noting that the final table of Donohue and Levitt (Quarterly Journal of Economics, 116 (2001), 379420) inadvertently omitted state-year interactions. Correcting our mistake does not alter the sign or statistical significance of our estimates, although it does reduce their magnitude. Using a more carefully constructed measure of abortion that better links birth cohorts to abortion exposure (by using abortion data by state of residence rather than of occurrence, by adjusting for cross-state mobility, and by more precisely estimating birth years from age of arrest data), we present new evidence that abortion legalization reduces crime through both a cohort-size and a selection effect.},
Doi = {10.1162/qjec.2008.123.1.425},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\5ECJ5PZM\\Donohue and Levitt - 2008 - Measurement Error, Legalized Abortion, and the Dec.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BSQKM7CG\\425.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {Measurement Error, Legalized Abortion, and the Decline in Crime},
Url = {http://qje.oxfordjournals.org/content/123/1/425},
Urldate = {2014-08-21}
}
+
@Article{donohue_uses_2005,
Title = {Uses and Abuses of Empirical Evidence in the Death Penalty Debate},
Author = {Donohue, John J. and Wolfers, Justin},
Journal = {Stanford Law Review},
Year = {2005},
+
Month = dec,
Number = {3},
Pages = {791--845},
Volume = {58},
+
Copyright = {Copyright 2005 Stanford Law Review},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\IT65BFI2\\Donohue and Wolfers - 2005 - Uses and Abuses of Empirical Evidence in the Death.pdf:application/pdf},
ISSN = {0038-9765},
Url = {http://www.jstor.org/stable/40040281},
Urldate = {2014-08-20}
}
+
@Article{doucouliagos_publication_2005,
Title = {Publication Bias in the Economic Freedom and Economic Growth Literature},
Author = {Doucouliagos, Chris},
Journal = {Journal of Economic Surveys},
Year = {2005},
+
Month = jul,
Number = {3},
Pages = {367--387},
Volume = {19},
+
Abstract = {Abstract. The impact of institutions on economic performance has attracted significant attention from researchers, as well as from policy reformers. A rapidly growing area in this literature is the impact of economic freedom on economic growth. The aim of this paper was to explore publication bias in this literature by means of traditional funnel plots, meta-significance testing, as well as by bootstrapping these meta-significance tests. When all the available estimates are combined and averaged, there seems to be evidence of a genuine and positive economic freedom economic growth effect. However, it is also shown that the economic freedom economic growth literature is tainted strongly with publication bias. The existence of publication bias makes it difficult to identify the magnitude of the genuine effect of economic freedom on economic growth. The paper explores the differences between aggregate and disaggregate measures of economic freedom and shows that selection effects are stronger when aggregate measures are used.},
Doi = {10.1111/j.0950-0804.2005.00252.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\66ND5K27\\Doucouliagos - 2005 - Publication Bias in the Economic Freedom and Econo.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CHZTX6WD\\abstract.html:text/html},
ISSN = {1467-6419},
Keywords = {Economic freedom, Economic growth, Publication bias, Recursive meta-significance},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.0950-0804.2005.00252.x/abstract},
Urldate = {2014-08-19}
}
+
@Article{doucouliagos_are_2013,
Title = {Are All Economic Facts Greatly Exaggerated? Theory Competition and Selectivity},
Author = {Doucouliagos, Chris and Stanley, T.d.},
Journal = {Journal of Economic Surveys},
Year = {2013},
+
Month = apr,
Number = {2},
Pages = {316--339},
Volume = {27},
+
Abstract = {Abstract..There is growing concern and mounting evidence of selectivity in empirical economics. Most empirical economic literatures have a truncated distribution of results. The aim of this paper is to explore the link between publication selectivity and theory contests. This link is confirmed through the analysis of 87 distinct empirical economics literatures, involving more than three and a half thousand separate empirical studies, using objective measures of both selectivity and contests. Our metameta-analysis shows that publication selection is widespread, but not universal. It distorts scientific inference with potentially adverse effects on policy making, but competition and debate between rival theories reduces this selectivity and thereby improves economic inference.},
Copyright = { 2011 Blackwell Publishing Ltd},
Doi = {10.1111/j.1467-6419.2011.00706.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NBSGU4V9\\Doucouliagos and Stanley - 2013 - Are All Economic Facts Greatly Exaggerated Theory.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\7T89ATRK\\full.html:text/html},
ISSN = {1467-6419},
Keywords = {Contested theory, Empirical economics, Meta-analysis, Publication selectivity},
Language = {en},
Shorttitle = {Are All Economic Facts Greatly Exaggerated?},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1467-6419.2011.00706.x/abstract},
Urldate = {2014-08-20}
}
+
@Article{doucouliagos2013all,
Title = {Are all economic facts greatly exaggerated? Theory competition and selectivity},
Author = {Doucouliagos, Chris and Stanley, TD},
Journal = {Journal of Economic Surveys},
Year = {2013},
Number = {2},
Pages = {316--339},
Volume = {27},
+
Publisher = {Wiley Online Library}
}
+
@Article{randomizationtoolkit,
Title = {Using randomization in development economics research: A toolkit},
Author = {Duflo, Esther and Glennerster, Rachel and Kremer, Michael},
Journal = {Handbook of development economics},
Year = {2007},
Pages = {3895--3962},
Volume = {4},
+
Publisher = {Elsevier}
}
+
@Article{duggan_more_2001,
Title = {More Guns, More Crime},
Author = {Duggan, Mark},
Journal = {Journal of Political Economy},
Year = {2001},
+
Month = oct,
Number = {5},
Pages = {1086--1114},
Volume = {109},
+
Abstract = {This paper examines the relationship between gun ownership and crime. Previous research has suffered from a lack of reliable data on gun ownership. I exploit a unique data set to reliably estimate annual rates of gun ownership at both the state and the county levels during the past two decades. My findings demonstrate that changes in gun ownership are significantly positively related to changes in the homicide rate, with this relationship driven almost entirely by an impact of gun ownership on murders in which a gun is used. The effect of gun ownership on all other crime categories is much less marked. Recent reductions in the fraction of households owning a gun can explain one?third of the differential decline in gun homicides relative to nongun homicides since 1993.},
Copyright = {Copyright 2001 The University of Chicago Press},
Doi = {10.1086/322833},
ISSN = {0022-3808},
Url = {http://www.jstor.org/stable/10.1086/322833},
Urldate = {2014-08-21}
}
+
@Article{duvendack_replication_2013,
Title = {Replication of quantitative work in development studies: Experiences and suggestions},
Author = {Duvendack, Maren and Palmer-Jones, Richard},
Journal = {Progress in Development Studies},
Year = {2013},
+
Month = oct,
Number = {4},
Pages = {307--322},
Volume = {13},
+
Abstract = {There is a growing demand for replications of authoritative works in development studies, which reflects recent trends in other social sciences as well as challenges to important quantitative works in development studies where replications have made contested contributions to understanding. At the same time, there is a strong trend within development towards adoption of medical models of evidence-based policy to find out what policies and interventions work. Replication is a key practice of medical (and natural science) research and was advocated frequently over several decades without success. This article addresses the incentives for replication going beyond a narrow focus on extrinsic rewards, reviews some significant examples, discusses behaviour during replication and draws lessons for replicators and replicatees.},
Doi = {10.1177/1464993413490480},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BQ74ZZN5\\Duvendack and Palmer-Jones - 2013 - Replication of quantitative work in development st.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EXGEMR4N\\307.html:text/html},
ISSN = {1464-9934, 1477-027X},
Keywords = {economic methodology, incentives, professionalism, replication, research ethics, sociology of economics},
Language = {en},
Shorttitle = {Replication of quantitative work in development studies},
Url = {http://pdj.sagepub.com/content/13/4/307},
Urldate = {2014-08-21}
}
+
@Article{duvendack_high_2012,
Title = {High Noon for Microfinance Impact Evaluations: Re-investigating the Evidence from Bangladesh},
Author = {Duvendack, Maren and Palmer-Jones, Richard},
Journal = {Journal of Development Studies},
Year = {2012},
Number = {12},
Pages = {1864--1880},
Volume = {48},
+
Abstract = {Abstract Recently, microfinance has come under increasing criticism raising questions of the validity of iconic studies which have justified it, such as Pitt and Khandker. Chemin applied propensity score matching to the Pitt and Khandker data, finding different impacts, but does not disaggregate by gender of borrower. We first replicate Chemin and extend his analysis in two ways. We test the robustness of propensity score matching results to selection on unobservables using sensitivity analysis, and we investigate propensity score matching estimates of impacts by gender of borrowers. The mainly insignificant impacts of microfinance differ greatly by gender of borrower, but are all vulnerable to selection on unobservables. We are therefore not convinced that the relationships between microfinance and outcomes are causal with these data.},
Doi = {10.1080/00220388.2011.646989},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\MNR6R6A3\\Duvendack and Palmer-Jones - 2012 - High Noon for Microfinance Impact Evaluations Re-.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\E2ABQMHH\\00220388.2011.html:text/html},
ISSN = {0022-0388},
Shorttitle = {High Noon for Microfinance Impact Evaluations},
Url = {http://dx.doi.org/10.1080/00220388.2011.646989},
Urldate = {2014-08-21}
}
+
@Article{duvendack_response_2012,
Title = {Response to Chemin and to Pitt},
Author = {Duvendack, Maren and Palmer-Jones, Richard},
Journal = {Journal of Development Studies},
Year = {2012},
Number = {12},
Pages = {1892--1897},
Volume = {48},
+
Abstract = {Abstract We reply to the discussion and criticisms of Matthieu Chemin ({MC}) and Mark Pitt ({MP}) to our paper ((Duvendack and Palmer-Jones ({DPJ})) (all this issue). {MC} clarifies many issues which now make replication pure probable (but not yet achieved), and {MP} identifies a number of problems with {DPJ} (some of which are shared with Chemin, 2008). Chemin (2008) made at least one crucial undocumented and unrealistic assumption, and did not document many of his variable constructions. {MP} correctly identifies inappropriate members of control groups, and other problems, but his claim that his propensity score matching ({PSM}) results provide support for Pitt and Khandker's (1998 {PnK}) most important claim is misleading as it is not robust. We see no reason to change our conclusion that {PnK} is limited as an evaluation of microfinance by a weak research design which cannot be convincingly mitigated by the sophisticated methods used in {PnK}, or by {PSM}.},
Doi = {10.1080/00220388.2012.747781},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DH9ZEJI2\\Duvendack and Palmer-Jones - 2012 - Response to Chemin and to Pitt.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\4IWADWAJ\\00220388.2012.html:text/html},
ISSN = {0022-0388},
Url = {http://dx.doi.org/10.1080/00220388.2012.747781},
Urldate = {2014-08-21}
}
+
@Article{easterly_aid_2004,
Title = {Aid, Policies, and Growth: Comment},
Author = {Easterly, William and Levine, Ross and Roodman, David},
Journal = {The American Economic Review},
Year = {2004},
+
Month = jun,
Number = {3},
Pages = {774--780},
Volume = {94},
+
Copyright = {Copyright 2004 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JN48ADJB\\Easterly et al. - 2004 - Aid, Policies, and Growth Comment.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Aid, Policies, and Growth},
Url = {http://www.jstor.org/stable/3592954},
Urldate = {2014-08-20}
}
+
@TechReport{ebenstein_fertility_2007,
Title = {Fertility Choices and Sex Selection in Asia: Analysis and Policy},
Author = {Ebenstein, Avraham Y.},
Institution = {Social Science Research Network},
Year = {2007},
+
Address = {Rochester, {NY}},
Month = feb,
Number = {{ID} 965551},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {High sex ratios in China and India have historically concerned researchers (Sen 1990) and their recent increase has alarmed policymakers worldwide. This paper identifies sex selection via infanticide and abortion as the principal explanation for the sex ratio distortion, and rules out competing explanations such as biology (Oster 2005) or differential mortality rates. Consistent with recent work (Jha et al. 2006), I find that the sex ratio of first-order births is close to the natural rate and steeply rising following the birth of low-order daughters, indicating that mothers are practicing pre-natal sex selection or immediate infanticide. Sex ratios are found to be higher among those anticipating lower fertility, such as those under stricter government fertility limits. I present a model of a mother's fertility choice when she has access to a sex-selection technology and faces a mandated fertility limit. By exploiting variation in fines levied in China for unsanctioned births, I demonstrate that higher fine regimes discourage fertility but are associated with higher sex ratios among those who choose to have an additional child. I then estimate a structural model of parental preferences using China's 2000 census data that indicates that a son is worth 2.90 years of income more than a daughter, and the premium is highest among less educated mothers and rural families. I conclude with a set of simulations to model the effect on sex ratios and total fertility of a proposed subsidy to families who fail to have a son, and find that such a policy would reduce sex ratios and lower overall fertility.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DZ47VIV6\\papers.html:text/html},
Keywords = {China, missing girls, sex ratio, sex-selective abortion},
Shorttitle = {Fertility Choices and Sex Selection in Asia},
Url = {http://papers.ssrn.com/abstract=965551},
Urldate = {2014-08-20}
}
+
@Article{von_elm_strengthening_2007,
Title = {The Strengthening the Reporting of Observational Studies in Epidemiology ({STROBE}) Statement: Guidelines for reporting observational studies},
Author = {von Elm, Erik and Altman, Douglas G. and Egger, Matthias and Pocock, Stuart J. and Gtzsche, Peter C. and Vandenbroucke, Jan P.},
Journal = {Preventive Medicine},
Year = {2007},
+
Month = oct,
Number = {4},
Pages = {247--251},
Volume = {45},
+
Abstract = {Much biomedical research is observational. The reporting of such research is often inadequate, which hampers the assessment of its strengths and weaknesses and of a study's generalisability. The Strengthening the Reporting of Observational Studies in Epidemiology ({STROBE}) Initiative developed recommendations on what should be included in an accurate and complete report of an observational study. We defined the scope of the recommendations to cover three main study designs: cohort, casecontrol and cross-sectional studies. We convened a 2-day workshop in September 2004, with methodologists, researchers, and journal editors to draft a checklist of items. This list was subsequently revised during several meetings of the coordinating group and in e-mail discussions with the larger group of {STROBE} contributors, taking into account empirical evidence and methodological considerations. The workshop and the subsequent iterative process of consultation and revision resulted in a checklist of 22 items (the {STROBE} Statement) that relate to the title, abstract, introduction, methods, results, and discussion sections of articles. 18 items are common to all three study designs and four are specific for cohort, casecontrol, or cross-sectional studies. A detailed Explanation and Elaboration document is published separately and is freely available on the websites of {PLoS} Medicine, Annals of Internal Medicine and Epidemiology. We hope that the {STROBE} Statement will contribute to improving the quality of reporting of observational studies.},
Doi = {10.1016/j.ypmed.2007.08.012},
File = {ScienceDirect Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JCN73RKN\\von Elm et al. - 2007 - The Strengthening the Reporting of Observational S.pdf:application/pdf;ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\E7UD734I\\S009174350700374X.html:text/html},
ISSN = {0091-7435},
Keywords = {Epidemiology, Observational studies, Quality of reporting, Reporting guidelines},
Shorttitle = {The Strengthening the Reporting of Observational Studies in Epidemiology ({STROBE}) Statement},
Url = {http://www.sciencedirect.com/science/article/pii/S009174350700374X},
Urldate = {2014-08-29}
}
+
@Article{fanelli_negative_2012,
Title = {Negative results are disappearing from most disciplines and countries},
Author = {Fanelli, Daniele},
Journal = {Scientometrics},
Year = {2012},
+
Month = mar,
Note = {{WOS}:000300325800009},
Number = {3},
Pages = {891--904},
Volume = {90},
+
Abstract = {Concerns that the growing competition for funding and citations might distort science are frequently discussed, but have not been verified directly. Of the hypothesized problems, perhaps the most worrying is a worsening of positive-outcome bias. A system that disfavours negative results not only distorts the scientific literature directly, but might also discourage high-risk projects and pressure scientists to fabricate and falsify their data. This study analysed over 4,600 papers published in all disciplines between 1990 and 2007, measuring the frequency of papers that, having declared to have "tested" a hypothesis, reported a positive support for it. The overall frequency of positive supports has grown by over 22\% between 1990 and 2007, with significant differences between disciplines and countries. The increase was stronger in the social and some biomedical disciplines. The United States had published, over the years, significantly fewer positive results than Asian countries (and particularly Japan) but more than European countries (and in particular the United Kingdom). Methodological artefacts cannot explain away these patterns, which support the hypotheses that research is becoming less pioneering and/or that the objectivity with which results are produced and published is decreasing.},
Doi = {10.1007/s11192-011-0494-7},
ISSN = {0138-9130},
Keywords = {animal behavior, behavioral ecology, Bias, citation, Competition, journals, Misconduct, publication, Publication bias, publish, Publish or perish, replication research, Research evaluation, science, scientific publication, statistical power},
Language = {English}
}
+
@Article{feldstein_social_1974,
Title = {Social Security, Induced Retirement, and Aggregate Capital Accumulation},
Author = {Feldstein, Martin},
Journal = {Journal of Political Economy},
Year = {1974},
+
Month = sep,
Number = {5},
Pages = {905--926},
Volume = {82},
+
Copyright = {Copyright 1974 The University of Chicago Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DA5K2B3P\\Feldstein - 1974 - Social Security, Induced Retirement, and Aggregate.pdf:application/pdf},
ISSN = {0022-3808},
Url = {http://www.jstor.org/stable/1829174},
Urldate = {2014-09-23}
}
+
@Article{feldstein_social_1982,
Title = {Social Security and Private Saving: Reply},
Author = {Feldstein, Martin S.},
Journal = {Journal of Political Economy},
Year = {1982},
+
Month = jun,
Number = {3},
Pages = {630--642},
Volume = {90},
+
Copyright = {Copyright 1982 The University of Chicago Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\F2SBK6IT\\Feldstein - 1982 - Social Security and Private Saving Reply.pdf:application/pdf},
ISSN = {0022-3808},
Shorttitle = {Social Security and Private Saving},
Url = {http://www.jstor.org/stable/1831374},
Urldate = {2014-09-23}
}
+
@Article{FindleyCPS,
Title = {Introduction: Special Issue on Research Transparency in the Social Sciences},
Author = {Findley, Michael AND Nathan M. Jensen AND Edmund J. Malesky AND Thomas B. Pepinsky},
Journal = {Comparative Political Studies},
Year = {2016},
+
Owner = {garret},
Timestamp = {2016.01.20}
}
+
@Article{finkelstein_oregon_2012,
Title = {The Oregon Health Insurance Experiment: Evidence from the First Year*},
Author = {Finkelstein, Amy and Taubman, Sarah and Wright, Bill and Bernstein, Mira and Gruber, Jonathan and Newhouse, Joseph P. and Allen, Heidi and Baicker, Katherine},
Journal = {The Quarterly Journal of Economics},
Year = {2012},
+
Month = aug,
Number = {3},
Pages = {1057--1106},
Volume = {127},
+
Abstract = {In 2008, a group of uninsured low-income adults in Oregon was selected by lottery to be given the chance to apply for Medicaid. This lottery provides an opportunity to gauge the effects of expanding access to public health insurance on the health care use, financial strain, and health of low-income adults using a randomized controlled design. In the year after random assignment, the treatment group selected by the lottery was about 25 percentage points more likely to have insurance than the control group that was not selected. We find that in this first year, the treatment group had substantively and statistically significantly higher health care utilization (including primary and preventive care as well as hospitalizations), lower out-of-pocket medical expenditures and medical debt (including fewer bills sent to collection), and better self-reported physical and mental health than the control group. {JEL} Codes: H51, H75, I1.},
Doi = {10.1093/qje/qjs020},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\IM4CS48H\\1057.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {The Oregon Health Insurance Experiment},
Url = {http://qje.oxfordjournals.org/content/127/3/1057},
Urldate = {2014-08-26}
}
+
@Article{food1998guidance,
Title = {Guidance for industry: E9 statistical principles for clinical trials},
Author = {Food{\ }and{\ }Drug{\ }Administration},
Journal = {Food and Drug Administration: Rockville, Maryland, USA},
Year = {1998},
+
Url = {http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm073137.pdf}
}
+
@Article{foote_impact_2008,
Title = {The Impact of Legalized Abortion on Crime: Comment},
Author = {Foote, Christopher L. and Goetz, Christopher F.},
Journal = {The Quarterly Journal of Economics},
Year = {2008},
+
Month = feb,
Number = {1},
Pages = {407--423},
Volume = {123},
+
Abstract = {This comment makes three observations about Donohue and Levitt's paper on abortion and crime (Quarterly Journal of Economics 119(1) (2001), 249275). First, there is a coding mistake in the concluding regressions, which identify abortion's effect on crime by comparing the experiences of different age cohorts within the same state and year. Second, correcting this error and using a more appropriate per capita specification for the crime variable generates much weaker results. Third, earlier tests in the paper, which exploit cross-state rather than within-state variation, are not robust to allowing for differential state trends based on statewide crime rates that predate the period when abortion could have had a causal effect on crime.},
Doi = {10.1162/qjec.2008.123.1.407},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KQR74GVH\\Foote and Goetz - 2008 - The Impact of Legalized Abortion on Crime Comment.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\55XP26KR\\407.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {The Impact of Legalized Abortion on Crime},
Url = {http://qje.oxfordjournals.org/content/123/1/407},
Urldate = {2014-08-21}
}
+
@Article{foote_impact_2008-1,
Title = {The Impact of Legalized Abortion on Crime: Comment},
Author = {Foote, Christopher L. and Goetz, Christopher F.},
Journal = {The Quarterly Journal of Economics},
Year = {2008},
+
Month = feb,
Number = {1},
Pages = {407--423},
Volume = {123},
+
Abstract = {This comment makes three observations about Donohue and Levitt's paper on abortion and crime (Quarterly Journal of Economics 119(1) (2001), 249275). First, there is a coding mistake in the concluding regressions, which identify abortion's effect on crime by comparing the experiences of different age cohorts within the same state and year. Second, correcting this error and using a more appropriate per capita specification for the crime variable generates much weaker results. Third, earlier tests in the paper, which exploit cross-state rather than within-state variation, are not robust to allowing for differential state trends based on statewide crime rates that predate the period when abortion could have had a causal effect on crime.},
Doi = {10.1162/qjec.2008.123.1.407},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\92B6S3HB\\Foote and Goetz - 2008 - The Impact of Legalized Abortion on Crime Comment.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\UT77Z9WK\\407.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {The Impact of Legalized Abortion on Crime},
Url = {http://qje.oxfordjournals.org/content/123/1/407},
Urldate = {2014-08-21}
}
+
@Article{franco_publication_2014,
Title = {Publication bias in the social sciences: Unlocking the file drawer},
Author = {Franco, Annie and Malhotra, Neil and Simonovits, Gabor},
Journal = {Science},
Year = {2014},
+
Month = sep,
Number = {6203},
Pages = {1502--1505},
Volume = {345},
+
Abstract = {We studied publication bias in the social sciences by analyzing a known population of conducted studies221 in totalin which there is a full accounting of what is published and unpublished. We leveraged Time-sharing Experiments in the Social Sciences ({TESS}), a National Science Foundationsponsored program in which researchers propose survey-based experiments to be run on representative samples of American adults. Because {TESS} proposals undergo rigorous peer review, the studies in the sample all exceed a substantial quality threshold. Strong results are 40 percentage points more likely to be published than are null results and 60 percentage points more likely to be written up. We provide direct evidence of publication bias and identify the stage of research production at which publication bias occurs: Authors do not write up and submit null findings.
The file drawer is full. Should we worry?
Experiments that produce null results face a higher barrier to publication than those that yield statistically significant differences. Whether this is a problem depends on how many null but otherwise valid results might be trapped in the file drawer. Franco et al. use a Time-sharing Experiments in the Social Sciences archive of nearly 250 peer-reviewed proposals of social science experiments conducted on nationally representative samples. They find that only 10 out of 48 null results were published, whereas 56 out of 91 studies with strongly significant results made it into a journal.
Science, this issue p. 1502},
Doi = {10.1126/science.1255484},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JS8TRIMD\\1502.html:text/html},
ISSN = {0036-8075, 1095-9203},
Language = {en},
Pmid = {25170047},
Shorttitle = {Publication bias in the social sciences},
Url = {http://www.sciencemag.org/content/345/6203/1502},
Urldate = {2014-10-15}
}
+
@Article{fuess_replication_1996,
Title = {On Replication in Business and Economics Research: The {QJBE} Case},
Author = {Fuess, Jr., Scott M.},
Journal = {Quarterly Journal of Business and Economics},
Year = {1996},
+
Month = apr,
Number = {2},
Pages = {3--13},
Volume = {35},
+
Abstract = {In 1984 the Quarterly Journal of Business and Economics ({QJBE}) became the first journal to give priority to publishing econometric replications. This study is a retrospective on the research published in the {QJBE} under this policy. Over the 1984-1994 period replications occupied more than 21 percent of the journal's contents. Further, the acceptance rate for replications has been higher than for other studies. The journal also devoted more than 5 percent of its pages to reviews of empirical literature. Replication articles have focused on financial economics and industrial organization; they tend to use publicly available data sets and not to contradict entirely the findings of earlier studies. Econometric replications have been more heavily represented in the {QJBE} than in other outlets, but they have not dominated the contents of the journal. This finding suggests that editorial policy can bring forth a supply of replication research, but other active measures are necessary before such studies become more common.},
Copyright = {Copyright 1996 University of Nebraska-Lincoln College of Business Administration},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CQV6HM3S\\Fuess - 1996 - On Replication in Business and Economics Research.pdf:application/pdf},
ISSN = {0747-5535},
Shorttitle = {On Replication in Business and Economics Research},
Url = {http://www.jstor.org/stable/40473179},
Urldate = {2014-10-01}
}
+
@Book{gandrud2013reproducible,
Title = {Reproducible Research with R and R Studio},
Author = {Gandrud, Christopher},
Publisher = {CRC Press},
Year = {2013}
}
+
@Article{gelman_preregistration_2013,
Title = {Preregistration of Studies and Mock Reports},
Author = {Gelman, Andrew},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {40--41},
Volume = {21},
+
Doi = {10.1093/pan/mps032},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CB399XI9\\Gelman - 2013 - Preregistration of Studies and Mock Reports.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\TZ2UKXAK\\40.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Url = {http://pan.oxfordjournals.org/content/21/1/40},
Urldate = {2014-08-28}
}
+
@Article{gelman_garden_2013,
Title = {The garden of forking paths: Why multiple comparisons can be a problem, even when there is no fishing expeditionor p-hacking and the research hypothesis was posited ahead of time.},
Author = {Gelman, Andrew and Loken, Eric},
Year = {2013},
+
Month = nov,
+
Url = {http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf}
}
+
@Article{GentzkowShapiro,
Title = {Code and Data for the Social Sciences: A Practioner's Guide},
Author = {Matthew Gentzkow and Jesse Shapiro},
Year = {2014},
+
Month = {March},
+
Owner = {garret},
Timestamp = {2015.07.13},
Url = {https://www.brown.edu/Research/Shapiro/pdfs/CodeAndData.pdf}
}
+
@TechReport{gerber_reporting_????,
Title = {Reporting Guidelines for Experimental Research: A Report from the Experimental Research Section Standards Committee},
Author = {Gerber, Alan and Arceneaux, Kevin and Boudreau, Cheryl and Dowling, Conor and Hillygus, Sunshine and Palfrey, Thomas and Biggers, Daniel R. and Hendry, David J.},
Year = {2014},
+
Url = {http://www.davidhendry.net/research-supplemental/gerberetal2014-reportingstandards/gerberetal2014-reportingstandards&appendix1.pdf}
}
+
@Article{gerber_testing_2001,
Title = {Testing for Publication Bias in Political Science},
Author = {Gerber, Alan S. and Green, Donald P. and Nickerson, David},
Journal = {Political Analysis},
Year = {2001},
+
Month = jan,
Number = {4},
Pages = {385--392},
Volume = {9},
+
Abstract = {If the publication decisions of journals are a function of the statistical significance of research findings, the published literature may suffer from publication bias.? This paper describes a method for detecting publication bias. We point out that to achieve statistical significance, the effect size must be larger in small samples. If publications tend to be biased against statistically insignificant results, we should observe that the effect size diminishes as sample sizes increase. This proposition is tested and confirmed using the experimental literature on voter mobilization.},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\XK39RAGF\\Gerber et al. - 2001 - Testing for Publication Bias in Political Science.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KJXWSKA4\\385.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Url = {http://pan.oxfordjournals.org/content/9/4/385},
Urldate = {2014-08-28}
}
+
@Book{gertler_impact_2011,
Title = {Impact Evaluation in Practice},
Author = {Gertler, Paul and Martinez, Sebastian and Premand, Patrick and Rawlings, Laura B. and Vermeersch, Christel M. J.},
Publisher = {World Bank Publications},
Year = {2011},
+
Abstract = {This book offers an accessible introduction to the topic of impact evaluation and its practice in development. While the book is geared principally towards development practitioners and policymakers designing prospective impact evaluations, we trust that it will be a valuable resource for students and others interested in using impact evaluation. Prospective impact evaluations should be used selectively to assess whether or not a program has achieved its intended results, or to test alternatives for achieving those results. We consider that more and better impact evaluation will help strengthen the evidence base for development policies and programs around the world. If governments and development practitioners can make policy decisions based on evidence - including evidence generated through impact evaluation - our hope is that development resources will be spent more effectively, and ultimately have a greater impact on reducing poverty and improving people's lives. The three chapters in this handbook provide a non-technical introduction to impact evaluations, including Why Evaluate? in Chapter 1, How to Evaluate? in Chapter 2 and How to Implement Impact Evaluations? in Chapter 3. These elements are the basic 'tools' needed in order to successfully carry out an impact evaluation. From a methodological standpoint our approach to impact evaluation is largely pragmatic: we think that the most appropriate methods should be identified to fit the operational context, and not the other way around. This is best achieved at the outset of the program, through the design of prospective impact evaluation that can be built into the project's implementation. We argue that gaining consensus between key stakeholders and identifying an evaluation design that fits the political and operational context is as important as the method itself. We also believe strongly that impact evaluations should be upfront about their limitations and caveats. Finally, we strongly encourage policymakers and program managers to consider impact evaluations in a logical framework that clearly sets out the causal pathways by which the program works to produce outputs and influence final outcomes, and to combine impact evaluations with monitoring and selected complementary evaluation approach to gain a full picture of performance.This book builds on a core set of teaching materials developed for the Turning Promises to Evidence? workshops organized by the office of the Chief Economist for Human Development ({HDNCE}) in partnership with regional units and the Development Economics Research Group ({DECRG}) at the World Bank.},
ISBN = {9780821385418},
Keywords = {Business \& Economics / Development / Economic Development, Education / Evaluation \& Assessment, Political Science / Public Policy / Economic Policy},
Language = {en}
}
+
@Article{gherghina_data_2013,
Title = {Data Availability in Political Science Journals},
Author = {Gherghina, Sergiu and Katsanidou, Alexia},
Journal = {European Political Science},
Year = {2013},
+
Month = sep,
Number = {3},
Pages = {333--349},
Volume = {12},
+
Abstract = {A characteristic of recent decades of scholarly work in the social sciences has been the increased amounts of empirical research. Access and availability of data are prerequisites for further research, replication work, and scientific development. As international peer-reviewed journals have gradually become the central forum for research debate, moves towards data sharing are dependent upon the policies of journals regarding data availability. This article examines contemporary data availability policies in political science and investigates the extent to which journals adopt such policies and their content. It also identifies a few factors associated with the existence of such policies.},
Doi = {10.1057/eps.2013.8},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NTP7MNDR\\Gherghina and Katsanidou - 2013 - Data Availability in Political Science Journals.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FQKE2CNP\\eps20138a.html:text/html},
ISSN = {1680-4333},
Keywords = {data availability, political science, replication policy},
Language = {en},
Url = {http://www.palgrave-journals.com/eps/journal/v12/n3/full/eps20138a.html},
Urldate = {2014-08-28}
}
+
@TechReport{glandon_report_2010,
Title = {Report on the American Economic Review Data Availability Compliance Project},
Author = {Glandon, Philip},
Institution = {Vanderbilt University},
Year = {2010},
Month = nov,
+
Url = {https://aeaweb.org/aer/2011_Data_Compliance_Report.pdf}
}
+
@Book{glennerster_running_2013,
Title = {Running Randomized Evaluations: A Practical Guide},
Author = {Glennerster, Rachel and Takavarasha, Kudzai},
Publisher = {Princeton University Press},
Year = {2013},
Month = nov,
+
Abstract = {This book provides a comprehensive yet accessible guide to running randomized impact evaluations of social programs. Drawing on the experience of researchers at the Abdul Latif Jameel Poverty Action Lab, which has run hundreds of such evaluations in dozens of countries throughout the world, it offers practical insights on how to use this powerful technique, especially in resource-poor environments. This step-by-step guide explains why and when randomized evaluations are useful, in what situations they should be used, and how to prioritize different evaluation opportunities. It shows how to design and analyze studies that answer important questions while respecting the constraints of those working on and benefiting from the program being evaluated. The book gives concrete tips on issues such as improving the quality of a study despite tight budget constraints, and demonstrates how the results of randomized impact evaluations can inform policy. With its self-contained modules, this one-of-a-kind guide is easy to navigate. It also includes invaluable references and a checklist of the common pitfalls to avoid. Provides the most up-to-date guide to running randomized evaluations of social programs, especially in developing countries Offers practical tips on how to complete high-quality studies in even the most challenging environments Self-contained modules allow for easy reference and flexible teaching and learning Comprehensive yet nontechnical},
ISBN = {9781400848447},
Keywords = {Business \& Economics / Development / Economic Development, Political Science / History \& Theory, Political Science / Public Policy / General},
Language = {en},
Shorttitle = {Running Randomized Evaluations}
}
+
@Article{greenwald2009understanding,
Title = {Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity.},
Author = {Greenwald, Anthony G and Poehlman, T Andrew and Uhlmann, Eric Luis and Banaji, Mahzarin R},
Journal = {Journal of personality and social psychology},
Year = {2009},
Number = {1},
Pages = {17},
Volume = {97},
+
Publisher = {American Psychological Association}
}
+
@Article{gupta_cultural_2006,
Title = {Cultural versus Biological Factors in Explaining Asia's Missing Women : Response to Oster},
Author = {Gupta, Monica Das},
Journal = {Population and Development Review},
Year = {2006},
+
Month = jun,
Number = {2},
Pages = {328--332},
Volume = {32},
+
Doi = {10.1111/j.1728-4457.2006.00121.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\QQ2J68BZ\\Gupta - 2006 - Cultural versus Biological Factors in Explaining A.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6STTEKME\\abstract.html:text/html},
ISSN = {1728-4457},
Language = {en},
Shorttitle = {Cultural versus Biological Factors in Explaining Asia's Missing Women?},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2006.00121.x/abstract},
Urldate = {2014-08-20}
}
+
@Article{gupta_explaining_2005,
Title = {Explaining Asia's Missing Women?: A New Look at the Data},
Author = {Gupta, Monica Das},
Journal = {Population and Development Review},
Year = {2005},
+
Month = sep,
Number = {3},
Pages = {529--535},
Volume = {31},
+
Abstract = {The fact that millions of females are missing? in East Asia and South Asia has been attributed to cultural factors that support strong son preference in these countries. A widely disseminated paper by Emily Oster argues that a large part of this phenomenon can be attributed to excessively masculine sex ratios at birth resulting from maternal infection with hepatitis B. If her thesis is true, current policies to address this problem would need to be reframed to include biological factors in addition to cultural factors. The data show, however, that whether or not females go missing? is determined by the existing sex composition of the family into which they are conceived. Girls with no older sisters have similar chances of survival as boys. However, girls conceived in families that already have a daughter experience steeply higher probabilities of being aborted or of dying in early childhood. This indicates that cultural factors still provide the overwhelming explanation for the missing? females.},
Doi = {10.1111/j.1728-4457.2005.00082.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8CC7PZXJ\\Gupta - 2005 - Explaining Asia's Missing Women? A New Look at t.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WBSC9R49\\abstract.html:text/html},
ISSN = {1728-4457},
Language = {en},
Shorttitle = {Explaining Asia's Missing Women?},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2005.00082.x/abstract},
Urldate = {2014-08-20}
}
+
@Article{hamermesh_viewpoint:_2007,
Title = {Viewpoint: Replication in economics},
Author = {Hamermesh, Daniel S.},
Journal = {Canadian Journal of Economics/Revue canadienne d'economique},
Year = {2007},
+
Month = aug,
Number = {3},
Pages = {715--733},
Volume = {40},
+
Abstract = {Abstract. This examination of the role and potential for replication in economics points out the paucity of both pure replication checking on others' published papers using their data and scientific replication using data representing different populations in one's own work or in a comment. Several controversies in empirical economics are used to illustrate how and how not to behave when replicating others' work. The incentives for replication are examined, and proposals aimed at journal editors and authors are advanced that might stimulate an activity that most economists applaud but few perform.},
Doi = {10.1111/j.1365-2966.2007.00428.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\R8X563MJ\\Hamermesh - 2007 - Viewpoint Replication in economics.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\MZQCI34R\\full.html:text/html},
ISSN = {1540-5982},
Keywords = {A14, B41, C59},
Language = {en},
Shorttitle = {Viewpoint},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2966.2007.00428.x/abstract},
Urldate = {2014-08-21}
}
+
@Article{hartung_reporting_2014,
Title = {Reporting Discrepancies Between the {ClinicalTrials}.gov Results Database and Peer-Reviewed {PublicationsDiscrepancies} Between {ClinicalTrials}.gov and Peer-Reviewed Publications},
Author = {Hartung, Daniel M. and Zarin, Deborah A. and Guise, Jeanne-Marie and McDonagh, Marian and Paynter, Robin and Helfand, Mark},
Journal = {Annals of Internal Medicine},
Year = {2014},
+
Month = apr,
Number = {7},
Pages = {477--483},
Volume = {160},
+
Abstract = {Background: {ClinicalTrials}.gov requires reporting of result summaries for many drug and device trials.Purpose: To evaluate the consistency of reporting of trials that are registered in the {ClinicalTrials}.gov results database and published in the literature.Data Sources: {ClinicalTrials}.gov results database and matched publications identified through {ClinicalTrials}.gov and a manual search of 2 electronic databases.Study Selection: 10\% random sample of phase 3 or 4 trials with results in the {ClinicalTrials}.gov results database, completed before 1 January 2009, with 2 or more groups.Data Extraction: One reviewer extracted data about trial design and results from the results database and matching publications. A subsample was independently verified.Data Synthesis: Of 110 trials with results, most were industry-sponsored, parallel-design drug studies. The most common inconsistency was the number of secondary outcome measures reported (80\%). Sixteen trials (15\%) reported the primary outcome description inconsistently, and 22 (20\%) reported the primary outcome value inconsistently. Thirty-eight trials inconsistently reported the number of individuals with a serious adverse event ({SAE}); of these, 33 (87\%) reported more {SAEs} in {ClinicalTrials}.gov. Among the 84 trials that reported {SAEs} in {ClinicalTrials}.gov, 11 publications did not mention {SAEs}, 5 reported them as zero or not occurring, and 21 reported a different number of {SAEs}. Among 29 trials that reported deaths in {ClinicalTrials}.gov, 28\% differed from the matched publication.Limitation: Small sample that included earliest results posted to the database.Conclusion: Reporting discrepancies between the {ClinicalTrials}.gov results database and matching publications are common. Which source contains the more accurate account of results is unclear, although {ClinicalTrials}.gov may provide a more comprehensive description of adverse events than the publication.Primary Funding Source: Agency for Healthcare Research and Quality.},
Doi = {10.7326/M13-0480},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NET5TC3N\\Hartung et al. - 2014 - Reporting Discrepancies Between the ClinicalTrials.pdf:application/pdf},
ISSN = {0003-4819},
Url = {http://dx.doi.org/10.7326/M13-0480},
Urldate = {2014-09-18}
}
+
@TechReport{harvey_and_2014,
Title = {and the Cross-Section of Expected Returns},
Author = {Harvey, Campbell R. and Liu, Yan and Zhu, Heqing},
Institution = {Social Science Research Network},
Year = {2014},
+
Address = {Rochester, {NY}},
Month = aug,
Number = {{ID} 2249314},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {Hundreds of papers and hundreds of factors attempt to explain the cross-section of expected returns. Given this extensive data mining, it does not make any economic or statistical sense to use the usual significance criteria for a newly discovered factor, e.g., a t-ratio greater than 2.0. However, what hurdle should be used for current research? Our paper introduces a multiple testing framework and provides a time series of historical significance cutoffs from the first empirical tests in 1967 to today. Our new method allows for correlation among the tests as well as publication bias. We also project forward 20 years assuming the rate of factor production remains similar to the experience of the last few years. The estimation of our model suggests that today a newly discovered factor needs to clear a much higher hurdle, with a t-ratio greater than 3.0. Echoing a recent disturbing conclusion in the medical literature, we argue that most claimed research findings in financial economics are likely false. Our key results are summarized:},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\5CRDDEIH\\papers.html:text/html},
Keywords = {3-factor model, Beta, Bonferroni, Factor zoo, {HML}, Idiosyncratic volatility, Liquidity, Momentum, Multiple tests, Risk factors, Skewness, {SMB}, Volatility},
Url = {http://papers.ssrn.com/abstract=2249314},
Urldate = {2014-08-21}
}
+
@Article{NosekPreRegistered,
Title = {In Search of an Association between Conception Risk and Prejudice},
Author = {Hawkins, Carlee Beth and Cailey E. Fitzgerald and Brian A. Nosek},
Journal = {Psychological Science},
Year = {2015},
+
Month = {February},
Number = {2},
Pages = {249-252},
Volume = {26},
+
Doi = {10.1177/0956797614553121},
Owner = {garret},
Timestamp = {2014.11.19},
Url = {http://pss.sagepub.com/content/26/2/249}
}
+
@Article{heffetz_privacy_2014,
Title = {Privacy and Data-Based Research},
Author = {Heffetz, Ori and Ligett, Katrina},
Journal = {Journal of Economic Perspectives},
Year = {2014},
+
Month = may,
Number = {2},
Pages = {75--98},
Volume = {28},
+
Doi = {10.1257/jep.28.2.75},
File = {JEP (28,2) p. 75 - Privacy and Data-Based Research:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\H3W29M6H\\articles.html:text/html},
ISSN = {0895-3309},
Language = {en},
Url = {https://www-aeaweb-org.proxy.swarthmore.edu/articles.php?doi=10.1257/jep.28.2.75},
Urldate = {2014-09-12}
}
+
@Article{helland_using_2004,
Title = {Using Placebo Laws to Test "More Guns, Less Crime"},
Author = {Helland, Eric and Tabarrok, Alexander},
Journal = {Advances in Economic Analysis \& Policy},
Year = {2004},
+
Month = jan,
Number = {1},
Volume = {4},
+
Abstract = {{AbstractWe} reexamine Mustard and Lotts controversial study on the effect of shall-issue? gun laws on crime using an empirical standard error function randomly generated from placebo? laws. We find that the effect of shall-issue laws on crime is much less well-estimated than the Mustard and Lott (1997) and Lott (2000) results suggest. We also find, however, that the cross equation restrictions implied by the Lott-Mustard theory are supported. A boomlet has occurred in recent years in the use of quasi-natural experiments to answer important questions of public policy. The intuitive power of this approach, however, has sometimes diverted attention from the statistical assumptions that must be made, particularly regarding standard errors. Failing to take into account serial correlation and grouped data can dramatically reduce standard errors suggesting greater certainty in effects than is actually the case. We find that the placebo law technique (Bertrand, Duflo and Mullainathan 2002) is a useful addition to the econometricians toolkit.},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2FFS2ITD\\Helland and Tabarrok - 2004 - Using Placebo Laws to Test More Guns, Less Crime.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\R43HS2GK\\bejeap.2004.4.1.1182.html:text/html},
Url = {http://www.degruyter.com/view/j/bejeap.2004.4.issue-1/bejeap.2004.4.1.1182/bejeap.2004.4.1.1182.xml},
Urldate = {2014-08-21}
}
+
@Article{herndon_does_2014,
Title = {Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff},
Author = {Herndon, Thomas and Ash, Michael and Pollin, Robert},
Journal = {Cambridge Journal of Economics},
Year = {2014},
+
Month = mar,
Number = {2},
Pages = {257--279},
Volume = {38},
+
Abstract = {We replicate Reinhart and Rogoff (2010A and 2010B) and find that selective exclusion of available data, coding errors and inappropriate weighting of summary statistics lead to serious miscalculations that inaccurately represent the relationship between public debt and {GDP} growth among 20 advanced economies. Over 19462009, countries with public debt/{GDP} ratios above 90\% averaged 2.2\% real annual {GDP} growth, not 0.1\% as published. The published results for (i) median {GDP} growth rates for the 19462009 period and (ii) mean and median {GDP} growth figures over 17902009 are all distorted by similar methodological errors, although the magnitudes of the distortions are somewhat smaller than with the mean figures for 19462009. Contrary to Reinhart and Rogoffs broader contentions, both mean and median {GDP} growth when public debt levels exceed 90\% of {GDP} are not dramatically different from when the public debt/{GDP} ratios are lower. The relationship between public debt and {GDP} growth varies significantly by period and country. Our overall evidence refutes {RR}s claim that public debt/{GDP} ratios above 90\% consistently reduce a countrys {GDP} growth.},
Doi = {10.1093/cje/bet075},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\J9SBWB55\\Herndon et al. - 2014 - Does high public debt consistently stifle economic.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JC879UJJ\\257.html:text/html},
ISSN = {0309-166X, 1464-3545},
Keywords = {Austerity, E60, E62, E65, Public debt},
Language = {en},
Shorttitle = {Does high public debt consistently stifle economic growth?},
Url = {http://cje.oxfordjournals.org/content/38/2/257},
Urldate = {2014-08-20}
}
+
@Article{hines2014sorting,
Title = {Sorting Out the FACS: A Devil in the Details},
Author = {Hines, William C and Su, Ying and Kuhn, Irene and Polyak, Kornelia and Bissell, Mina J},
Journal = {Cell reports},
Year = {2014},
Number = {5},
Pages = {779--781},
Volume = {6},
+
Publisher = {Elsevier Science}
}
+
@Article{vocationalTurkey,
Title = {The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey},
Author = {Hirshleifer, Sarojini and McKenzie, David and Almeida, Rita and Ridao-Cano, Cristobal},
Journal = {The Economic Journal},
Year = {2014},
Pages = {n/a--n/a},
+
Doi = {10.1111/ecoj.12211},
ISSN = {1468-0297},
Keywords = {Vocational training, Active Labour Market Programs, Randomized Experiment, Private Provision},
Url = {http://dx.doi.org/10.1111/ecoj.12211}
}
+
@Article{holm_multipletesting,
Title = {A Simple Sequentially Rejective Multiple Test Procedure},
Author = {Holm, Sture},
Journal = {Scandinavian Journal of Statistics},
Year = {1979},
Number = {2},
Pages = {pp. 65-70},
Volume = {6},
+
Abstract = {This paper presents a simple and widely applicable multiple test procedure of the sequentially rejective type, i.e. hypotheses are rejected one at a time until no further rejections can be done. It is shown that the test has a prescribed level of significance protection against error of the first kind for any combination of true hypotheses. The power properties of the test and a number of possible applications are also discussed.},
Copyright = {Copyright 1979 Board of the Foundation of the Scandinavian Journal of Statistics},
ISSN = {03036898},
Jstor_articletype = {research-article},
Jstor_formatteddate = {1979},
Language = {English},
Publisher = {Wiley on behalf of Board of the Foundation of the Scandinavian Journal of Statistics},
Url = {http://www.jstor.org/stable/4615733}
}
+
@Article{hoxby_does_2007,
Title = {Does Competition among Public Schools Benefit Students and Taxpayers? Reply},
Author = {Hoxby, Caroline M.},
Journal = {The American Economic Review},
Year = {2007},
+
Month = dec,
Number = {5},
Pages = {2038--2055},
Volume = {97},
+
Copyright = {Copyright 2007 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2SKTX83D\\Hoxby - 2007 - Does Competition among Public Schools Benefit Stud.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Does Competition among Public Schools Benefit Students and Taxpayers?},
Url = {http://www.jstor.org/stable/30034600},
Urldate = {2014-08-20}
}
+
@Article{hoxby_does_2000,
Title = {Does Competition among Public Schools Benefit Students and Taxpayers?},
Author = {Hoxby, Caroline M.},
Journal = {The American Economic Review},
Year = {2000},
+
Month = dec,
Number = {5},
Pages = {1209--1238},
Volume = {90},
+
Abstract = {Tiebout choice among districts is the most powerful market force in American public education. Naive estimates of its effects are biased by endogenous district formation. I derive instruments from the natural boundaries in a metropolitan area. My results suggest that metropolitan areas with greater Tiebout choice have more productive public schools and less private schooling. Little of the effect of Tiebout choice works through its effect on household sorting. This finding may be explained by another finding: students are equally segregated by school in metropolitan areas with greater and lesser degrees of Tiebout choice among districts.},
Copyright = {Copyright 2000 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SWX7PN98\\Hoxby - 2000 - Does Competition among Public Schools Benefit Stud.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/2677848},
Urldate = {2014-08-20}
}
+
@Article{hsiang2013climate,
Title = {Quantifying the influence of climate on human conflict},
Author = {Hsiang, Solomon M and Burke, Marshall and Miguel, Edward},
Journal = {Science},
Year = {2013},
Number = {6151},
Pages = {1235367},
Volume = {341},
+
Publisher = {American Association for the Advancement of Science}
}
+
@Article{humphreys_fishing_2013,
Title = {Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration},
Author = {Humphreys, Macartan and Sierra, Raul Sanchez de la and Windt, Peter van der},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {1--20},
Volume = {21},
+
Abstract = {Social scientists generally enjoy substantial latitude in selecting measures and models for hypothesis testing. Coupled with publication and related biases, this latitude raises the concern that researchers may intentionally or unintentionally select models that yield positive findings, leading to an unreliable body of published research. To combat this fishing? problem in medical studies, leading journals now require preregistration of designs that emphasize the prior identification of dependent and independent variables. However, we demonstrate here that even with this level of advanced specification, the scope for fishing is considerable when there is latitude over selection of covariates, subgroups, and other elements of an analysis plan. These concerns could be addressed through the use of a form of comprehensive registration. We experiment with such an approach in the context of an ongoing field experiment for which we drafted a complete mock report? of findings using fake data on treatment assignment. We describe the advantages and disadvantages of this form of registration and propose that a comprehensive but nonbinding approach be adopted as a first step to combat fishing by social scientists. Likely effects of comprehensive but nonbinding registration are discussed, the principal advantage being communication rather than commitment, in particular that it generates a clear distinction between exploratory analyses and genuine tests.},
Doi = {10.1093/pan/mps021},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\QVHCIGGE\\Humphreys et al. - 2013 - Fishing, Commitment, and Communication A Proposal.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BZWC5VG7\\1.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Shorttitle = {Fishing, Commitment, and Communication},
Url = {http://pan.oxfordjournals.org/content/21/1/1},
Urldate = {2014-08-27}
}
+
@Article{hunter_desperate_2001,
Title = {The Desperate Need for Replications},
Author = {Hunter, John},
Journal = {Journal of Consumer Research},
Year = {2001},
+
Month = jun,
Number = {1},
Pages = {149--158},
Volume = {28},
+
Abstract = {An overemphasis on creativity for evaluating research has lead to a serious devaluation of replication studies. However, we need a total sample size of \$N=153,669\$ to estimate a causal effect to two digits, which is quite rare for a single study. The only way to get accurate estimation is to average across replications. If the average sample size were as high as \$N=200\$ , we would need over 700 replication studies. Scientific replications are more problematic than pure statistical replications, and so we need even more replications to achieve reasonable accuracy.},
Copyright = {Copyright 2001 Journal of Consumer Research, Inc.},
Doi = {10.1086/jcr.2001.28.issue-1},
ISSN = {0093-5301},
Url = {http://www.jstor.org/stable/10.1086/321953},
Urldate = {2014-09-23}
}
+
@Article{husereau_consolidated_2013,
Title = {Consolidated Health Economic Evaluation Reporting Standards ({CHEERS}) statement},
Author = {Husereau, Don and Drummond, Michael and Petrou, Stavros and Carswell, Chris and Moher, David and Greenberg, Dan and Augustovski, Federico and Briggs, Andrew H. and Mauskopf, Josephine and Loder, Elizabeth and {CHEERS Task Force}},
Journal = {Value in Health: The Journal of the International Society for Pharmacoeconomics and Outcomes Research},
Year = {2013},
+
Month = apr,
Number = {2},
Pages = {e1--5},
Volume = {16},
+
Abstract = {Economic evaluations of health interventions pose a particular challenge for reporting. There is also a need to consolidate and update existing guidelines and promote their use in a user friendly manner. The Consolidated Health Economic Evaluation Reporting Standards ({CHEERS}) statement is an attempt to consolidate and update previous health economic evaluation guidelines efforts into one current, useful reporting guidance. The primary audiences for the {CHEERS} statement are researchers reporting economic evaluations and the editors and peer reviewers assessing them for publication. The need for new reporting guidance was identified by a survey of medical editors. A list of possible items based on a systematic review was created. A two round, modified Delphi panel consisting of representatives from academia, clinical practice, industry, government, and the editorial community was conducted. Out of 44 candidate items, 24 items and accompanying recommendations were developed. The recommendations are contained in a user friendly, 24 item checklist. A copy of the statement, accompanying checklist, and this report can be found on the {ISPOR} Health Economic Evaluations Publication Guidelines Task Force website: (www.ispor.org/{TaskForces}/{EconomicPubGuidelines}.asp). We hope {CHEERS} will lead to better reporting, and ultimately, better health decisions. To facilitate dissemination and uptake, the {CHEERS} statement is being co-published across 10 health economics and medical journals. We encourage other journals and groups, to endorse {CHEERS}. The author team plans to review the checklist for an update in five years.},
Doi = {10.1016/j.jval.2013.02.010},
ISSN = {1524-4733},
Keywords = {Biomedical Research, Checklist, Cost-Benefit Analysis, Delphi Technique, Economics, Medical, Evaluation Studies as Topic, Guidelines as Topic, Health Policy, Humans, Journalism, Medical, Peer Review, Research, Resource Allocation},
Language = {eng},
Pmid = {23538200}
}
+
@Article{ioannidis_effectiveness_2008,
Title = {Effectiveness of antidepressants: an evidence myth constructed from a thousand randomized trials?},
Author = {Ioannidis, John PA},
Journal = {Philosophy, Ethics, and Humanities in Medicine},
Year = {2008},
+
Month = may,
Number = {1},
Pages = {14},
Volume = {3},
+
Abstract = {Antidepressants, in particular newer agents, are among the most widely prescribed medications worldwide with annual sales of billions of dollars. The introduction of these agents in the market has passed through seemingly strict regulatory control. Over a thousand randomized trials have been conducted with antidepressants. Statistically significant benefits have been repeatedly demonstrated and the medical literature is flooded with several hundreds of "positive" trials (both pre-approval and post-approval). However, two recent meta-analyses question this picture. The first meta-analysis used data that were submitted to {FDA} for the approval of 12 antidepressant drugs. While only half of these trials had formally significant effectiveness, published reports almost ubiquitously claimed significant results. "Negative" trials were either left unpublished or were distorted to present "positive" results. The average benefit of these drugs based on the {FDA} data was of small magnitude, while the published literature suggested larger benefits. A second meta-analysis using also {FDA}-submitted data examined the relationship between treatment effect and baseline severity of depression. Drug-placebo differences increased with increasing baseline severity and the difference became large enough to be clinically important only in the very small minority of patient populations with severe major depression. In severe major depression, antidepressants did not become more effective, simply placebo lost effectiveness. These data suggest that antidepressants may be less effective than their wide marketing suggests. Short-term benefits are small and long-term balance of benefits and harms is understudied. I discuss how the use of many small randomized trials with clinically non-relevant outcomes, improper interpretation of statistical significance, manipulated study design, biased selection of study populations, short follow-up, and selective and distorted reporting of results has built and nourished a seemingly evidence-based myth on antidepressant effectiveness and how higher evidence standards, with very large long-term trials and careful prospective meta-analyses of individual-level data may reach closer to the truth and clinically useful evidence.
{PMID}: 18505564},
Copyright = {2008 Ioannidis; licensee {BioMed} Central Ltd.},
Doi = {10.1186/1747-5341-3-14},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\S5SJ29NE\\Ioannidis - 2008 - Effectiveness of antidepressants an evidence myth.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HMHTZPTG\\abstract.html:text/html},
ISSN = {1747-5341},
Language = {en},
Pmid = {18505564},
Shorttitle = {Effectiveness of antidepressants},
Url = {http://www.peh-med.com/content/3/1/14/abstract},
Urldate = {2014-10-09}
}
+
@Article{ioannidis_why_2005,
Title = {Why Most Published Research Findings Are False},
Author = {Ioannidis, John P. A.},
Journal = {{PLoS} Med},
Year = {2005},
+
Month = aug,
Number = {8},
Pages = {e124},
Volume = {2},
+
Abstract = {Published research findings are sometimes refuted by subsequent evidence, says Ioannidis, with ensuing confusion and disappointment.},
Doi = {10.1371/journal.pmed.0020124},
File = {PLoS Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2V44N3ZE\\Ioannidis - 2005 - Why Most Published Research Findings Are False.pdf:application/pdf;PLoS Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NMZ3G7BJ\\journal.pmed.html:text/html},
Url = {http://dx.doi.org/10.1371/journal.pmed.0020124},
Urldate = {2014-08-19}
}
+
@Article{ioannidis_why_2005-1,
Title = {Why Most Published Research Findings Are False},
Author = {Ioannidis, John P. A.},
Journal = {{PLoS} Med},
Year = {2005},
+
Month = aug,
Number = {8},
Pages = {e124},
Volume = {2},
+
Abstract = {Published research findings are sometimes refuted by subsequent evidence, says Ioannidis, with ensuing confusion and disappointment.},
Doi = {10.1371/journal.pmed.0020124},
File = {PLoS Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SXACAGP3\\Ioannidis - 2005 - Why Most Published Research Findings Are False.pdf:application/pdf;PLoS Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CEMBCCE3\\journal.pmed.html:text/html},
Url = {http://dx.doi.org/10.1371/journal.pmed.0020124},
Urldate = {2014-08-28}
}
+
@Article{isis-2_second_international_study_of_infarct_survival_collaborative_group_randomised_1988,
Title = {{RANDOMISED} {TRIAL} {OF} {INTRAVENOUS} {STREPTOKINASE}, {ORAL} {ASPIRIN}, {BOTH}, {OR} {NEITHER} {AMONG} 17 187 {CASES} {OF} {SUSPECTED} {ACUTE} {MYOCARDIAL} {INFARCTION}: {ISIS}-2},
Author = {{ISIS-2 (SECOND INTERNATIONAL STUDY OF INFARCT SURVIVAL) COLLABORATIVE GROUP}},
Journal = {The Lancet},
Year = {1988},
+
Month = aug,
Number = {8607},
Pages = {349--360},
Volume = {332},
+
Abstract = {17 187 patients entering 417 hospitals up to 24 hours (median 5 hours) after the onset of suspected acute myocardial infarction were randomised, with placebo control, between: (i) a 1-hour intravenous infusion of 15 {MU} of streptokinase; (ii) one month of 160 mg/day enteric-coated aspirin; (iii) both active treatments; or (iv) neither. Streptokinase alone and aspirin alone each produced a highly significant reduction in 5-week vascular mortality: 791/8592 (92\%) among patients allocated streptokinase infusion vs 1029/8595 (120\%) among those allocated placebo infusion (odds reduction: 25\% {SD} 4; 2p\&lt;000001); 804/8587 (94\%) vascular deaths among patients allocated aspirin tablets vs 1016/8600 (118\%) among those allocated placebo tablets (odds reduction: 23\% {SD} 4; 2p \&lt;000001). The combination of streptokinase and aspirin was significantly (2p\&lt;00001) better than either agent alone. Their separate effects on vascular deaths appeared to be additive: 343/4292 (80\%) among patients allocated both active agents vs 568/4300 (132\%) among those allocated neither (odds reduction: 42\% {SD} 5; 95\% confidence limits 34-50\%). There was evidence of benefit from each agent even for patients treated late after pain onset (odds reductions at 0-4, 5-12, and 13-24 hours: 35\% {SD} 6, 16\% {SD} 7, and 21\% {SD} 12 for streptokinase alone; 25\% {SD} 7, 21\% {SD} 7, and 21\% {SD} 12 for aspirin alone; and 53\% {SD} 8, 32\% {SD} 9, and 38\% {SD} 15 for the combination of streptokinase and aspirin). Streptokinase was associated with an excess of bleeds requiring transfusion (05\% vs 02\%) and of confirmed cerebral haemorrhage (01\% vs 00\%), but with fewer other strokes (06\% vs 08\%). These "other" strokes may have included a few undiagnosed cerebral haemorrhages, but still there was no increase in total strokes (07\% streptokinase vs 08\% placebo infusion). Aspirin significantly reduced non-fatal reinfarction (10\% vs 20\%) and non-fatal stroke (03\% vs 06\%), and was not associated with any significant increase in cerebral haemorrhage or in bleeds requiring transfusion. An excess of non-fatal reinfarction was reported when streptokinase was used alone, but this appeared to be entirely avoided by the addition of aspirin. Those allocated the combination of streptokinase and aspirin had significantly fewer deaths (80\% vs 132\%) than those allocated neither. The differences in vascular and in all-cause mortality produced by streptokinase and by aspirin remain highly significant (2p\&lt;0001 for each) after the median of 15 months of follow-up thus far available.},
Doi = {10.1016/S0140-6736(88)92833-4},
File = {ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NX7SNPHI\\S0140673688928334.html:text/html},
ISSN = {0140-6736},
Series = {Originally published as Volume 2, Issue 8607},
Shorttitle = {{RANDOMISED} {TRIAL} {OF} {INTRAVENOUS} {STREPTOKINASE}, {ORAL} {ASPIRIN}, {BOTH}, {OR} {NEITHER} {AMONG} 17 187 {CASES} {OF} {SUSPECTED} {ACUTE} {MYOCARDIAL} {INFARCTION}},
Url = {http://www.sciencedirect.com/science/article/pii/S0140673688928334},
Urldate = {2014-08-28}
}
+
@Article{johnr.lott_crime_1997,
Title = {Crime, Deterrence, and Right to Carry Concealed Handguns},
Author = {JohnR.Lott, Jr., By and Mustard, DavidB.},
Journal = {The Journal of Legal Studies},
Year = {1997},
+
Month = jan,
Number = {1},
Pages = {1--68},
Volume = {26},
+
Abstract = {Abstract Using cross?sectional time?series data for U.S. counties from 1977 to 1992, we find that allowing citizens to carry concealed weapons deters violent crimes, without increasing accidental deaths. If those states without right?to?carry concealed gun provisions had adopted them in 1992, county? and state?level data indicate that approximately 1,500 murders would have been avoided yearly. Similarly, we predict that rapes would have declined by over 4,000, robbery by over 11,000, and aggravated assaults by over 60,000. We also find criminals substituting into property crimes involving stealth, where the probability of contact between the criminal and the victim is minimal. Further, higher arrest and conviction rates consistently reduce crime. The estimated annual gain from all remaining states adopting these laws was at least \$5.74 billion in 1992. The annual social benefit from an additional concealed handgun permit is as high as \$5,000.},
Copyright = {Copyright 1997 The University of Chicago},
Doi = {10.1086/467988},
ISSN = {0047-2530},
Url = {http://www.jstor.org/stable/10.1086/467988},
Urldate = {2014-08-21}
}
+
@Article{johnson_harvard_2012,
Title = {Harvard professor who resigned fabricated, manipulated data, {US} says - The Boston Globe},
Author = {Johnson, Carolyn Y.},
Journal = {{BostonGlobe}.com},
Year = {2012},
+
Month = sep,
+
Abstract = {Marc Hauser, a prolific scientist and popular psychology professor who resigned last summer from Harvard University, had fabricated data, manipulated results in multiple experiments, and incorrectly described how experiments were conducted, according to the findings of a federal research oversight agency posted online Wednesday.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\ZC9UU8TT\\story.html:text/html},
Keywords = {02138 (Magazine), Boston Globe, fraud, harvard, Harvard University, hauser, Health and Human Services Department, {LinkedIn}, {McMaster} University, monkeys, National Institutes of Health, New York Times, {NIH}, Office of Research Integrity, Public Health Service, research misconduct, Twitter},
Url = {https://www.bostonglobe.com/news/science/2012/09/05/harvard-professor-who-resigned-fabricated-manipulated-data-says/6gDVkzPNxv1ZDkh4wVnKhO/story.html},
Urldate = {2014-09-12}
}
+
@Article{joyce_did_2004,
Title = {Did Legalized Abortion Lower Crime?},
Author = {Joyce, Ted},
Journal = {The Journal of Human Resources},
Year = {2004},
+
Month = jan,
Number = {1},
Pages = {1--28},
Volume = {39},
+
Abstract = {In this paper I compare changes in homicide and arrest rates among cohorts born before and after the legalization of abortion to changes in crime in the same years among similar cohorts who were unexposed to legalized abortion. I find little consistent evidence that the legalization of abortion in selected states around 1970, and then in the remaining states following Roe v. Wade, had an effect on recent crime rates. I conclude that the dramatic association as reported in a recent study is most likely the result of unmeasured period effects such as changes in crack cocaine use.},
Copyright = {Copyright 2004 The Board of Regents of the University of Wisconsin System},
Doi = {10.2307/3559003},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BHQI74T4\\Joyce - 2004 - Did Legalized Abortion Lower Crime.pdf:application/pdf},
ISSN = {0022-166X},
Url = {http://www.jstor.org/stable/3559003},
Urldate = {2014-08-21}
}
+
@Book{jr_more_2010,
Title = {More Guns, Less Crime: Understanding Crime and Gun Control Laws, Third Edition},
Author = {Jr, John R. Lott},
Publisher = {University Of Chicago Press},
Year = {2010},
+
Address = {Chicago},
Edition = {Third Edition edition},
Month = may,
+
Abstract = {John R. Lott, Jr., is the author five books, including Freedomnomics and Are Predatory Commitments Credible? Who Should the Courts Believe? , the latter also published by the University of Chicago Press.},
ISBN = {9780226493664},
Language = {English},
Shorttitle = {More Guns, Less Crime}
}
+
@Article{kane_why_1984,
Title = {{WHY} {JOURNAL} {EDITORS} {SHOULD} {ENCOURAGE} {THE} {REPLICATION} {OF} {APPLIED} {ECONOMETRIC} {RESEARCH}},
Author = {Kane, Edward J.},
Journal = {Quarterly Journal of Business and Economics},
Year = {1984},
+
Month = jan,
Number = {1},
Pages = {3--8},
Volume = {23},
+
Copyright = {Copyright 1984 University of Nebraska-Lincoln College of Business Administration},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WWHAQMXS\\Kane - 1984 - WHY JOURNAL EDITORS SHOULD ENCOURAGE THE REPLICATI.pdf:application/pdf},
ISSN = {0747-5535},
Url = {http://www.jstor.org/stable/23526567},
Urldate = {2014-08-28}
}
+
@TechReport{katz_elizabeth_pre-analysis_2013,
Title = {Pre-Analysis Plan: {TOMS} Shoes Impact Study},
Author = {Katz, Elizabeth and Janet, Brendan and Wydick, Bruce and Gutierrez, Felipe},
Year = {2013},
Month = feb,
+
Url = {http://www.povertyactionlab.org/sites/default/files/documents/Pre-Analysis%20Plan_Wydick_2-12-13.pdf}
}
+
@Article{king_replication_1995,
Title = {Replication, Replication},
Author = {King, Gary},
Journal = {{PS}: Political Science \& Politics},
Year = {1995},
Number = {03},
Pages = {444--452},
Volume = {28},
+
Doi = {10.2307/420301},
File = {Cambridge Journals Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\PICSS99J\\displayAbstract.html:text/html}
}
+
@Book{kirchkamp_workflow_????,
Title = {Workflow of Statistical Data Analysis},
Author = {Kirchkamp, Oliver},
+
Url = {http://www.kirchkamp.de/oekonometrie/pdf/wf-screen2.pdf}
}
+
@Article{klein_reconsidering_2009,
Title = {Reconsidering the impact of national soccer results on the {FTSE} 100},
Author = {Klein, Christian and Zwergel, Bernhard and Henning Fock, J.},
Journal = {Applied Economics},
Year = {2009},
+
Month = nov,
Number = {25},
Pages = {3287--3294},
Volume = {41},
+
Abstract = {In past decades, many empirical studies revealed return anomalies in many different asset classes and markets. Very recent publications have, however, even found evidence that stock markets react to the results of soccer matches. In this article, we argue that such empirical studies should be analysed carefully; we thus endorse the use of replication studies to verify results. Consequently, by rebuilding the study of Ashton et al. (2003), we are able to detect mistakes in the empirical set-up. Based on these findings, we demonstrate how even minor flaws can have a crucial influence on the results of such studies and point out pitfalls that are frequently encountered. We furthermore emphasize the importance of robustness checks to validate the results of empirical studies.},
Doi = {10.1080/00036840802112471},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NQ5RSRPD\\Klein et al. - 2009 - Reconsidering the impact of national soccer result.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SZW2TZ25\\00036840802112471.html:text/html},
ISSN = {0003-6846},
Url = {http://dx.doi.org/10.1080/00036840802112471},
Urldate = {2014-08-28}
}
+
@Article{many_labs,
Title = {Investigating Variation in Replicability},
Author = {Klein, Richard A and Ratliff, Kate A and Vianello, Michelangelo and Adams Jr, Reginald B and Bahn{\'\i}k, {\v{S}}t{\v{e}}p{\'a}n and Bernstein, Michael J and Bocian, Konrad and Brandt, Mark J and Brooks, Beach and Brumbaugh, Claudia Chloe and others},
Journal = {Social Psychology},
Year = {2014},
Number = {3},
Pages = {142--152},
Volume = {45},
+
Publisher = {Hogrefe \& Huber}
}
+
@Article{kling2007experimental,
Title = {Experimental analysis of neighborhood effects},
Author = {Kling, Jeffrey R and Liebman, Jeffrey B and Katz, Lawrence F},
Journal = {Econometrica},
Year = {2007},
Number = {1},
Pages = {83--119},
Volume = {75},
+
Publisher = {Wiley Online Library}
}
+
@Book{knuth_literate_1992,
Title = {Literate Programming},
Author = {Knuth, Donald Ervin},
Publisher = {Center for the Study of Language and Information},
Year = {1992},
Month = jan,
+
Abstract = {Literate programming is a programming methodology that combines a programming language with a documentation language, making programs more robust, more portable, and more easily maintained than programs written only in a high-level language. Computer programmers already know both kinds of languages; they need only learn a few conventions about alternating between languages to create programs that are works of literature. A literate programmer is an essayist who writes programs for humans to understand, instead of primarily writing instructions for machines to follow. When programs are written in the recommended style they can be transformed into documents by a document compiler and into efficient code by an algebraic compiler. This anthology of essays from the inventor of literate programming includes Knuth's early papers on related topics such as structured programming, as well as the Computer Journal article that launched literate programming itself.},
ISBN = {9780937073810},
Keywords = {Computers / Programming / General, Computers / Programming / Object Oriented},
Language = {en}
}
+
@Article{knuth_literate_1984,
Title = {Literate Programming},
Author = {Knuth, D. E.},
Journal = {The Computer Journal},
Year = {1984},
+
Month = jan,
Number = {2},
Pages = {97--111},
Volume = {27},
+
Abstract = {The author and his associates have been experimenting for the past several years with a programming language and documentation system called {WEB}. This paper presents {WEB} by example, and discusses why the new system appears to be an improvement over previous ones.},
Doi = {10.1093/comjnl/27.2.97},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\53H8RZZ3\\97.html:text/html},
ISSN = {0010-4620, 1460-2067},
Language = {en},
Url = {http://comjnl.oxfordjournals.org/content/27/2/97},
Urldate = {2014-09-25}
}
+
@Article{koenker_reproducible_2009,
Title = {On reproducible econometric research},
Author = {Koenker, Roger and Zeileis, Achim},
Journal = {Journal of Applied Econometrics},
Year = {2009},
+
Month = aug,
Number = {5},
Pages = {833--847},
Volume = {24},
+
Abstract = {Recent software developments are reviewed from the vantage point of reproducible econometric research. We argue that the emergence of new tools, particularly in the open-source community, have greatly eased the burden of documenting and archiving both empirical and simulation work in econometrics. Some of these tools are highlighted in the discussion of two small replication exercises. Copyright 2009 John Wiley \& Sons, Ltd.},
Copyright = {Copyright 2009 John Wiley \& Sons, Ltd.},
Doi = {10.1002/jae.1083},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\Q8JDP4K8\\Koenker and Zeileis - 2009 - On reproducible econometric research.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2TH4N2D2\\full.html:text/html},
ISSN = {1099-1255},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1002/jae.1083/abstract},
Urldate = {2014-08-20}
}
+
@Article{koole_rewarding_2012,
Title = {Rewarding Replications A Sure and Simple Way to Improve Psychological Science},
Author = {Koole, Sander L. and Lakens, Danil},
Journal = {Perspectives on Psychological Science},
Year = {2012},
+
Month = nov,
Number = {6},
Pages = {608--614},
Volume = {7},
+
Abstract = {Although replications are vital to scientific progress, psychologists rarely engage in systematic replication efforts. In this article, we consider psychologists narrative approach to scientific publications as an underlying reason for this neglect and propose an incentive structure for replications within psychology. First, researchers need accessible outlets for publishing replications. To accomplish this, psychology journals could publish replication reports in files that are electronically linked to reports of the original research. Second, replications should get cited. This can be achieved by cociting replications along with original research reports. Third, replications should become a valued collaborative effort. This can be realized by incorporating replications in teaching programs and by stimulating adversarial collaborations. The proposed incentive structure for replications can be developed in a relatively simple and cost-effective manner. By promoting replications, this incentive structure may greatly enhance the dependability of psychologys knowledge base.},
Doi = {10.1177/1745691612462586},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WXEQAJJP\\Koole and Lakens - 2012 - Rewarding Replications A Sure and Simple Way to Im.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\X672EUCA\\608.html:text/html},
ISSN = {1745-6916, 1745-6924},
Keywords = {philosophy of science, Publication bias, replication, scientific fraud, selective reporting},
Language = {en},
Url = {http://pps.sagepub.com/content/7/6/608},
Urldate = {2014-09-08}
}
+
@Article{kremer2009incentives,
Title = {Incentives to learn},
Author = {Kremer, Michael and Miguel, Edward and Thornton, Rebecca},
Journal = {The Review of Economics and Statistics},
Year = {2009},
Number = {3},
Pages = {437--456},
Volume = {91},
+
Publisher = {MIT Press}
}
+
@Article{laine_clinical_2007,
Title = {Clinical Trial Registration Looking Back and Moving Ahead},
Author = {Laine, Christine and Horton, Richard and DeAngelis, Catherine D. and Drazen, Jeffrey M. and Frizelle, Frank A. and Godlee, Fiona and Haug, Charlotte and Hbert, Paul C. and Kotzin, Sheldon and Marusic, Ana and Sahni, Peush and Schroeder, Torben V. and Sox, Harold C. and Weyden, Martin B. Van Der and Verheugt, Freek W.A.},
Journal = {New England Journal of Medicine},
Year = {2007},
+
Month = jun,
Number = {26},
Pages = {2734--2736},
Volume = {356},
+
Abstract = {In 2005, the International Committee of Medical Journal Editors ({ICMJE}) initiated a policy requiring investigators to deposit information about trial design into an accepted clinical trials registry before the onset of patient enrollment.1 This policy aimed to ensure that information about the existence and design of clinically directive trials was publicly available, an ideal that leaders in evidence-based medicine have advocated for decades.2 The policy precipitated much angst among research investigators and sponsors, who feared that registration would be burdensome and would stifle competition. Yet, the response to this policy has been overwhelming. The {ICMJE} promised to reevaluate the policy . . .},
Doi = {10.1056/NEJMe078110},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\D5BXWI59\\Laine et al. - 2007 - Clinical Trial Registration Looking Back and Mov.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DUFG5DV3\\NEJMe078110.html:text/html},
ISSN = {0028-4793},
Pmid = {17548427},
Url = {http://www.nejm.org/doi/full/10.1056/NEJMe078110},
Urldate = {2014-08-28}
}
+
@Article{laitin_fisheries_2013,
Title = {Fisheries Management},
Author = {Laitin, David D.},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {42--47},
Volume = {21},
+
Doi = {10.1093/pan/mps033},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\7VZ62N93\\Laitin - 2013 - Fisheries Management.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\S8QFGJMS\\42.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Url = {http://pan.oxfordjournals.org/content/21/1/42},
Urldate = {2014-08-28}
}
+
@Article{laitin_fisheries_2013-1,
Title = {Fisheries Management},
Author = {Laitin, David D.},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {42--47},
Volume = {21},
+
Doi = {10.1093/pan/mps033},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\U7GPDX2C\\Laitin - 2013 - Fisheries Management.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\67M78Z9X\\42.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Url = {http://pan.oxfordjournals.org/content/21/1/42},
Urldate = {2014-08-28}
}
+
@Article{lalonde_evaluating_1986,
Title = {Evaluating the Econometric Evaluations of Training Programs with Experimental Data},
Author = {LaLonde, Robert J.},
Journal = {The American Economic Review},
Year = {1986},
+
Month = sep,
Number = {4},
Pages = {604--620},
Volume = {76},
+
Abstract = {This paper compares the effect on trainee earnings of an employment program that was run as a field experiment where participants were randomly assigned to treatment and control groups with the estimates that would have been produced by an econometrician. This comparison shows that many of the econometric procedures do not replicate the experimentally determined results, and it suggests that researchers should be aware of the potential for specification errors in other nonexperimental evaluations.},
Copyright = {Copyright 1986 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6WQN3FIU\\LaLonde - 1986 - Evaluating the Econometric Evaluations of Training.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/1806062},
Urldate = {2014-09-08}
}
+
@Article{lash_preregistration_2010,
Title = {Preregistration of Study Protocols Is Unlikely to Improve the Yield From Our Science, But Other Strategies Might:},
Author = {Lash, Timothy L.},
Journal = {Epidemiology},
Year = {2010},
+
Month = sep,
Number = {5},
Pages = {612--613},
Volume = {21},
+
Doi = {10.1097/EDE.0b013e3181e9bba6},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {Preregistration of Study Protocols Is Unlikely to Improve the Yield From Our Science, But Other Strategies Might},
Url = {http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001648-201009000-00011},
Urldate = {2014-10-28}
}
+
@Article{leamer_lets_1983,
Title = {Let's Take the Con Out of Econometrics},
Author = {Leamer, Edward E.},
Journal = {The American Economic Review},
Year = {1983},
+
Month = mar,
Number = {1},
Pages = {31--43},
Volume = {73},
+
Copyright = {Copyright 1983 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DK5VRK58\\Leamer - 1983 - Let's Take the Con Out of Econometrics.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/1803924},
Urldate = {2014-09-08}
}
+
@Article{leimer_social_1982,
Title = {Social Security and Private Saving: New Time-Series Evidence},
Author = {Leimer, Dean R. and Lesnoy, Selig D.},
Journal = {Journal of Political Economy},
Year = {1982},
+
Month = jun,
Number = {3},
Pages = {606--629},
Volume = {90},
+
Copyright = {Copyright 1982 The University of Chicago Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KSMSTFMK\\Leimer and Lesnoy - 1982 - Social Security and Private Saving New Time-Serie.pdf:application/pdf},
ISSN = {0022-3808},
Shorttitle = {Social Security and Private Saving},
Url = {http://www.jstor.org/stable/1831373},
Urldate = {2014-09-23}
}
+
@Article{Levine2001editorial,
Title = {Editor's Introduction to The Unemployment Effects of Minimum Wages: Evidence from a Prespecified Research Design},
Author = {Levine, David I.},
Journal = {Industrial Relations: A Journal of Economy and Society},
Year = {2001},
Number = {2},
Pages = {161--162},
Volume = {40},
+
Doi = {10.1111/0019-8676.00204},
ISSN = {1468-232X},
Publisher = {Blackwell Publishers Inc.},
Url = {http://dx.doi.org/10.1111/0019-8676.00204}
}
+
@Article{levitt_using_2002,
Title = {Using Electoral Cycles in Police Hiring to Estimate the Effects of Police on Crime: Reply},
Author = {Levitt, Steven D.},
Journal = {The American Economic Review},
Year = {2002},
+
Month = sep,
Number = {4},
Pages = {1244--1250},
Volume = {92},
+
Copyright = {Copyright 2002 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SJP8QQBP\\Levitt - 2002 - Using Electoral Cycles in Police Hiring to Estimat.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Using Electoral Cycles in Police Hiring to Estimate the Effects of Police on Crime},
Url = {http://www.jstor.org/stable/3083312},
Urldate = {2014-08-20}
}
+
@Article{levitt_using_1997,
Title = {Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime},
Author = {Levitt, Steven D.},
Journal = {The American Economic Review},
Year = {1997},
+
Month = jun,
Number = {3},
Pages = {270--290},
Volume = {87},
+
Abstract = {Previous empirical studies have uncovered little evidence that police reduce crime, possibly due to simultaneity problems. This paper uses the timing of mayoral and gubernatorial elections as an instrumental variable to identify a causal effect of police on crime. Increases in the size of police forces are shown to be disproportionately concentrated in mayoral and gubernatorial election years. Increases in police are shown to substantially reduce violent crime, but have a smaller impact on property crime. The null hypothesis that the marginal social benefit of reduced crime equals the costs of hiring additional police cannot be rejected.},
Copyright = {Copyright 1997 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\J7GXNZH9\\Levitt - 1997 - Using Electoral Cycles in Police Hiring to Estimat.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/2951346},
Urldate = {2014-08-20}
}
+
@Article{lin_can_2008,
Title = {Can Hepatitis B Mothers Account for the Number of Missing Women? Evidence from Three Million Newborns in Taiwan},
Author = {Lin, Ming-Jen and Luoh, Ming-Ching},
Journal = {The American Economic Review},
Year = {2008},
+
Month = dec,
Number = {5},
Pages = {2259--2273},
Volume = {98},
+
Copyright = {Copyright 2008 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EEWVARXU\\Lin and Luoh - 2008 - Can Hepatitis B Mothers Account for the Number of .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Can Hepatitis B Mothers Account for the Number of Missing Women?},
Url = {http://www.jstor.org/stable/29730171},
Urldate = {2014-08-20}
}
+
@Article{loder_registration_2010,
Title = {Registration of observational studies: The next step towards research transparency},
Author = {Loder, E. and Groves, T. and MacAuley, D.},
Journal = {{BMJ}},
Year = {2010},
+
Month = feb,
Number = {feb18 2},
Pages = {c950--c950},
Volume = {340},
+
Doi = {10.1136/bmj.c950},
ISSN = {0959-8138, 1468-5833},
Language = {en},
Url = {http://www.bmj.com/cgi/doi/10.1136/bmj.c950},
Urldate = {2014-10-28}
}
+
@Book{long_workflow_2008,
Title = {The Workflow of Data Analysis Using Stata},
Author = {Long, J. Scott},
Publisher = {Stata Press},
Year = {2008},
Month = dec,
+
Abstract = {The Workflow of Data Analysis Using Stata, by J. Scott Long, is an essential productivity tool for data analysts. Long presents lessons gained from his experience and demonstrates how to design and implement efficient workflows for both one-person projects and team projects. After introducing workflows and explaining how a better workflow can make it easier to work with data, Long describes planning, organizing, and documenting your work. He then introduces how to write and debug Stata do-files and how to use local and global macros. After a discussion of conventions that greatly simplify data analysis the author covers cleaning, analyzing, and protecting data.},
ISBN = {9781597180474},
Keywords = {Computers / Mathematical \& Statistical Software, Mathematics / Probability \& Statistics / General},
Language = {en}
}
+
@Article{loomis_journal_2011,
Title = {Journal Requirements to Register Observational Studies: {OEM}'s Policy},
Author = {Loomis, Dana},
Journal = {Occupational and Environmental Medicine},
Year = {2011},
+
Month = feb,
Number = {2},
Pages = {83--84},
Volume = {68},
+
Doi = {10.1136/oem.2010.064477},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DDGUADAD\\Loomis - 2011 - Journal Requirements to Register Observational Stu.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HCCBMA8X\\83.html:text/html},
ISSN = {, 1470-7926},
Keywords = {Epidemiology, health services research, hygiene / occupational hygiene, occupational health practice, toxicology},
Language = {en},
Pmid = {21242278},
Shorttitle = {Journal Requirements to Register Observational Studies},
Url = {http://oem.bmj.com/content/68/2/83},
Urldate = {2014-10-28}
}
+
@Article{lott_abortion_2007,
Title = {Abortion and Crime: Unwanted Children and Out-of-Wedlock Births},
Author = {Lott, John R. and Whitley, John},
Journal = {Economic Inquiry},
Year = {2007},
+
Month = apr,
Number = {2},
Pages = {304--324},
Volume = {45},
+
Abstract = {Legalizing abortion can either increase or decrease investments in childrens human capital. This article finds that abortion increases the number of out-of-wedlock births. Using data that more directly links the criminal with age when the crime was committed, not age when arrested, and fixing the assumption in previous research that no abortions took place prior to the Roe v. Wade decision in the 45 states affected by that decision, we find consistent significant evidence that legalizing abortions increased murders by over 7\%. Linear estimates indicate that legalization increased total annual victimization costs by at least \$3.2 billion. ({JEL} K42, K14, J24)},
Doi = {10.1111/j.1465-7295.2006.00040.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6QJ76BZ7\\Lott and Whitley - 2007 - Abortion and Crime Unwanted Children and Out-of-W.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\MJXAZWTD\\full.html:text/html},
ISSN = {1465-7295},
Language = {en},
Shorttitle = {Abortion and Crime},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1465-7295.2006.00040.x/abstract},
Urldate = {2014-08-21}
}
+
@Article{marwick_how,
Title = {How computers broke science and what we can do to fix it},
Author = {Marwick, Ben},
Journal = {The Conversation},
Year = {2015},
+
Month = {November},
+
Abstract = {Virtually every researcher relies on computers to collect or analyze data. But when computers are opaque black boxes that manipulate data, it's impossible to replicate studies a core value for science.},
Day = {9},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\RVZCJW3H\\how-computers-broke-science-and-what-we-can-do-to-fix-it-49938.html:text/html},
Url = {http://theconversation.com/how-computers-broke-science-and-what-we-can-do-to-fix-it-49938},
Urldate = {2016-02-02}
}
+
@Article{mathieu_s_comparison_2009,
Title = {{COmparison} of registered and published primary outcomes in randomized controlled trials},
Author = {{Mathieu S} and {Boutron I} and {Moher D} and {Altman DG} and {Ravaud P}},
Journal = {{JAMA}},
Year = {2009},
+
Month = sep,
Number = {9},
Pages = {977--984},
Volume = {302},
+
Abstract = {ContextAs of 2005, the International Committee of Medical Journal Editors required investigators to register their trials prior to participant enrollment as a precondition for publishing the trial's findings in member journals.ObjectiveTo assess the proportion of registered trials with results recently published in journals with high impact factors; to compare the primary outcomes specified in trial registries with those reported in the published articles; and to determine whether primary outcome reporting bias favored significant outcomes.Data Sources and Study Selection{MEDLINE} via {PubMed} was searched for reports of randomized controlled trials ({RCTs}) in 3 medical areas (cardiology, rheumatology, and gastroenterology) indexed in 2008 in the 10 general medical journals and specialty journals with the highest impact factors.Data ExtractionFor each included article, we obtained the trial registration information using a standardized data extraction form.ResultsOf the 323 included trials, 147 (45.5\%) were adequately registered (ie, registered before the end of the trial, with the primary outcome clearly specified). Trial registration was lacking for 89 published reports (27.6\%), 45 trials (13.9\%) were registered after the completion of the study, 39 (12\%) were registered with no or an unclear description of the primary outcome, and 3 (0.9\%) were registered after the completion of the study and had an unclear description of the primary outcome. Among articles with trials adequately registered, 31\% (46 of 147) showed some evidence of discrepancies between the outcomes registered and the outcomes published. The influence of these discrepancies could be assessed in only half of them and in these statistically significant results were favored in 82.6\% (19 of 23).ConclusionComparison of the primary outcomes of {RCTs} registered with their subsequent publication indicated that selective outcome reporting is prevalent.},
Doi = {10.1001/jama.2009.1242},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HH9WTG2H\\Mathieu S et al. - 2009 - COmparison of registered and published primary out.pdf:application/pdf},
ISSN = {0098-7484},
Url = {http://dx.doi.org/10.1001/jama.2009.1242},
Urldate = {2014-10-09}
}
+
@Article{mccrary_using_2002,
Title = {Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime: Comment},
Author = {McCrary, Justin},
Journal = {The American Economic Review},
Year = {2002},
+
Month = sep,
Number = {4},
Pages = {1236--1243},
Volume = {92},
+
Copyright = {Copyright 2002 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8PXGEEEX\\McCrary - 2002 - Using Electoral Cycles in Police Hiring to Estimat.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime},
Url = {http://www.jstor.org/stable/3083311},
Urldate = {2014-08-20}
}
+
@Article{mccullough_economics_2008,
Title = {Do economics journal archives promote replicable research?},
Author = {McCullough, B.d. and McGeary, Kerry Anne and Harrison, Teresa D.},
Journal = {Canadian Journal of Economics/Revue canadienn economique},
Year = {2008},
+
Month = nov,
Number = {4},
Pages = {1406--1420},
Volume = {41},
+
Abstract = {Abstract. All the long-standing archives at economics journals do not facilitate the reproduction of published results. The data-only archives at Journal of Business and Economic Statistics and Economic Journal fail in part because most authors do not contribute data. Results published in the {FRB} St. Louis Review can rarely be reproduced using the data+code in the journal archive. Recently created archives at top journals should avoid the mistakes of their predecessors. We categorize reasons for archives' failures and identify successful policies.},
Copyright = { Canadian Economics Association},
Doi = {10.1111/j.1540-5982.2008.00509.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JUWCUUJN\\McCullough et al. - 2008 - Do economics journal archives promote replicable r.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6V8H3493\\full.html:text/html},
ISSN = {1540-5982},
Keywords = {B40, C80},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1540-5982.2008.00509.x/abstract},
Urldate = {2014-08-20}
}
+
@Misc{mccullough_open_2009,
Title = {Open Access Economics Journals and the Market for Reproducible Economic Research},
+
Author = {McCullough, B. D.},
Month = mar,
Year = {2009},
+
Abstract = {Most economics journals take no substantive measures to ensure that the results they publish are replicable. To make the data and code available so that published results can be checked requires an archive. Top economics journals have been adopting mandatory data+code archives in the past few years. The movement toward mandatory data+code archives has yet to reach the open access journals. This is paradoxical; given their emphasis on making articles readily available, one would think that open access journals also would want to make data and code readily available. Open access economics journals should adopt mandatory data+code archives en masse. Doing so will give them a competitive advantage with respect to traditional economics journals.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FG55T8WN\\documentSummary\;dn=775889276720076\;res=IELBUS.html:text/html},
Keywords = {abstract and citation, Architecture Building and Design, Arts and Entertainment, Asian Resources, bibliographic databases, books, Business and Management, {CD} {ROMs}, Communications, Crime and Justice, Directory and Reference Resources, Early Childhood Resources, Education and Research, Engineering and Technology, Environment and Natural Resources, e-resources, e-titles, full text databases, genealogy, Genealogy (Family History), Health (Medicine), History and Heritage, Hospitality and Tourism, Indigenous Peoples, Information Management and Technology, journals, Law, Literature, monographs, Newspapers, New Zealand Resources, online resources, papers, print publishing, Public Affairs, Public Relations, reports, Science and Technology, Secondary School Resources, Social Sciences and Community Issues, Sports, Trade Skills},
Url = {http://search.informit.com.au/documentSummary;dn=775889276720076;res=IELBUS},
Urldate = {2014-08-28}
}
+
@Article{mccullough_got_2007,
Title = {Got Replicability? The Journal of Money, Credit and Banking Archive},
Author = {McCullough, B. D.},
Journal = {Econ Journal Watch},
Year = {2007},
+
Month = sep,
Number = {3},
Pages = {326--337},
Volume = {4},
+
Abstract = {In a paper published in the Journal of Money, Credit and Banking, {McGeary}, Harrison, and I showed that the {JMCB}s data+code archive of generally did not support the replication of the journals published results. We recommended several procedures for ensuring that the archived data and code would reproduce the published results. The {JMCB} Editors recently adopted a few new procedures, ignoring most of the recommendations. This paper checks to see whether the new procedures are working. They are not.},
Url = {http://econjwatch.org/articles/got-replicability-the-journal-of-money-credit-and-banking-archive?ref=section-archive}
}
+
@Article{mccullough_lessons_2006,
Title = {Lessons from the {JMCB} Archive},
Author = {McCullough, B. D. and McGeary, Kerry Anne and Harrison, Teresa D.},
Journal = {Journal of Money, Credit, and Banking},
Year = {2006},
Note = {{\textless}p{\textgreater}Volume 38, Number 4, June 2006{\textless}/p{\textgreater}},
Number = {4},
Pages = {1093--1107},
Volume = {38},
+
Abstract = {Abstract We examine the online archive of the Journal of Money, Credit, and Banking, in which an author is required to deposit the data and code that replicate the results of his paper. We find that most authors do not fulfill this requirement. Of more than 150 empirical articles, fewer than 15 could be replicated. Despite all this, there is no doubt that a data/code archive is more conducive to replicable research than the alternatives. We make recommendations to improve the functioning of the archive.},
Doi = {10.1353/mcb.2006.0061},
File = {Project MUSE Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\P785GVJX\\McCullough et al. - 2006 - Lessons from the JMCB Archive.pdf:application/pdf;Project MUSE Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BEB9MB9B\\38.4mccullough.html:text/html},
ISSN = {1538-4616},
Url = {http://muse.jhu.edu/journals/journal_of_money_credit_and_banking/v038/38.4mccullough.pdf},
Urldate = {2014-08-20}
}
+
@Article{mccullough_verifying_2003,
Title = {Verifying the Solution from a Nonlinear Solver: A Case Study},
Author = {McCullough, B. D. and Vinod, H. D.},
Journal = {The American Economic Review},
Year = {2003},
+
Month = jun,
Number = {3},
Pages = {873--892},
Volume = {93},
+
Copyright = {Copyright 2003 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8NB4E2JQ\\McCullough and Vinod - 2003 - Verifying the Solution from a Nonlinear Solver A .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Verifying the Solution from a Nonlinear Solver},
Url = {http://www.jstor.org/stable/3132121},
Urldate = {2014-08-20}
}
+
@Article{mckenzie2012caseformoreT,
Title = {Beyond baseline and follow-up: The case for more T in experiments},
Author = {McKenzie, David},
Journal = {Journal of Development Economics},
Year = {2012},
Number = {2},
Pages = {210--221},
Volume = {99},
+
Publisher = {Elsevier}
}
+
@Article{mcshane_2014,
Title = {You Cannot Step Into the Same River Twice When Power Analyses Are Optimistic},
Author = {McShane, Blakeley B and B{\"o}ckenholt, Ulf},
Journal = {Perspectives on Psychological Science},
Year = {2014},
Number = {6},
Pages = {612--625},
Volume = {9},
+
Publisher = {SAGE Publications}
}
+
@Book{merton1973sociology,
Title = {The sociology of science: Theoretical and empirical investigations},
Author = {Merton, Robert K},
Publisher = {University of Chicago press},
Year = {1973}
}
+
@Article{miguel_promoting_2014,
Title = {Promoting Transparency in Social Science Research},
Author = {Miguel, E. and Camerer, C. and Casey, K. and Cohen, J. and Esterling, K. M. and Gerber, A. and Glennerster, R. and Green, D. P. and Humphreys, M. and Imbens, G. and Laitin, D. and Madon, T. and Nelson, L. and Nosek, B. A. and Petersen, M. and Sedlmayr, R. and Simmons, J. P. and Simonsohn, U. and Laan, M. Van der},
Journal = {Science},
Year = {2014},
+
Month = jan,
Number = {6166},
Pages = {30--31},
Volume = {343},
+
Doi = {10.1126/science.1245317},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\QXKPAMAF\\Miguel et al. - 2014 - Promoting Transparency in Social Science Research.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WJ3W282B\\30.html:text/html},
ISSN = {0036-8075, 1095-9203},
Language = {en},
Pmid = {24385620},
Url = {http://www.sciencemag.org/content/343/6166/30},
Urldate = {2014-08-28}
}
+
@Article{miguel_re-examining_2011,
Title = {Re-examining Economic Shocks and Civil Conflict},
Author = {Miguel, Edward and Satyanath, Shanker},
Journal = {American Economic Journal: Applied Economics},
Year = {2011},
+
Month = oct,
Number = {4},
Pages = {228--232},
Volume = {3},
+
Abstract = {Miguel, Satyanath, and Ernest Sergenti (2004), henceforth {MSS}, show that economic growth is negatively related to civil conflict in Africa, using annual rainfall variation as an {IV} for growth. Antonio Ciccone (2011) argues that thanks to rainfall's mean-reverting nature, rainfall levels are preferable to annual changes. We make three points. First, {MSS}'s findings hold using rainfall levels as instruments. Second, Ciccone (2011) does not provide theoretical justification for preferring rainfall levels. Third, the first-stage relationship between rainfall and growth is weaker after 2000, suggesting that alternative instruments are needed when studying recent conflicts. We highlight the accumulating microeconomic evidence that adverse economic shocks lead to political violence.},
Doi = {10.1257/app.3.4.228}
}
+
@Article{miguel_economic_2004,
Title = {Economic Shocks and Civil Conflict: An Instrumental Variables Approach},
Author = {Miguel, Edward and Satyanath, Shanker and Sergenti, Ernest},
Journal = {Journal of Political Economy},
Year = {2004},
+
Month = aug,
Number = {4},
Pages = {725--753},
Volume = {112},
+
Abstract = {Estimating the impact of economic conditions on the likelihood of civil conflict is difficult because of endogeneity and omitted variable bias. We use rainfall variation as an instrumental variable for economic growth in 41 African countries during 198199. Growth is strongly negatively related to civil conflict: a negative growth shock of five percentage points increases the likelihood of conflict by one?half the following year. We attempt to rule out other channels through which rainfall may affect conflict. Surprisingly, the impact of growth shocks on conflict is not significantly different in richer, more democratic, or more ethnically diverse countries.},
Copyright = {Copyright 2004 The University of Chicago Press},
Doi = {10.1086/jpe.2004.112.issue-4},
ISSN = {0022-3808},
Shorttitle = {Economic Shocks and Civil Conflict},
Url = {http://www.jstor.org/stable/10.1086/421174},
Urldate = {2014-08-21}
}
+
@Article{mittelstaedt_econometric_1984,
Title = {{ECONOMETRIC} {REPLICATION}: {LESSONS} {FROM} {THE} {EXPERIMENTAL} {SCIENCES}},
Author = {Mittelstaedt, Robert A. and Zorn, Thomas S.},
Journal = {Quarterly Journal of Business and Economics},
Year = {1984},
+
Month = jan,
Number = {1},
Pages = {9--15},
Volume = {23},
+
Copyright = {Copyright 1984 University of Nebraska-Lincoln College of Business Administration},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\PFHSVF37\\Mittelstaedt and Zorn - 1984 - ECONOMETRIC REPLICATION LESSONS FROM THE EXPERIME.pdf:application/pdf},
ISSN = {0747-5535},
Shorttitle = {{ECONOMETRIC} {REPLICATION}},
Url = {http://www.jstor.org/stable/23526568},
Urldate = {2014-08-28}
}
+
@Article{moher_d_use_2001,
Title = {Use of the consort statement and quality of reports of randomized trials: A comparative before-and-after evaluation},
Author = {{Moher D} and {Jones A} and {Lepage L} and {for the CONSORT Group}},
Journal = {{JAMA}},
Year = {2001},
+
Month = apr,
Number = {15},
Pages = {1992--1995},
Volume = {285},
+
Abstract = {ContextThe Consolidated Standards for Reporting of Trials ({CONSORT}) statement
was developed to help improve the quality of reports of randomized controlled
trials ({RCTs}). To date, a paucity of data exists regarding whether it has
achieved this goal.ObjectiveTo determine whether use of the {CONSORT} statement is associated with
improvement in the quality of reports of {RCTs}.Design and SettingComparative before-and-after evaluation in which reports of {RCTs} published
in 1994 (pre-{CONSORT}) were compared with {RCT} reports from the same journals
published in 1998 (post-{CONSORT}). We included 211 reports from {BMJ}, {JAMA}, and The Lancet (journals that
adopted {CONSORT}) as well as The New England Journal of Medicine (a journal that did not adopt {CONSORT} and was used as a comparator).Main Outcome MeasuresNumber of {CONSORT} items included in a report, frequency of unclear reporting
of allocation concealment, and overall trial quality score based on the Jadad
scale, a 5-point quality assessment instrument.ResultsCompared with 1994, the number of {CONSORT} checklist items in reports
of {RCTs} increased in all 4 journals in 1998, and this increase was statistically
significant for the 3 adopter journals (pre-{CONSORT}, 23.4; mean change, 3.7;
95\% confidence interval [{CI}], 2.1-5.3). The frequency of unclear reporting
of allocation concealment decreased for each of the 4 journals, and this change
was statistically significant for adopters (pre-{CONSORT}, 61\%; mean change, 22\%;
95\% {CI}, 38\% to 6\%). Similarly, 3 of the 4 journals showed an
improvement in the quality score for reports of {RCTs}, and this increase was
statistically significant for adopter journals overall (pre-{CONSORT}, 2.7;
mean change, 0.4; 95\% {CI}, 0.1-0.8).ConclusionUse of the {CONSORT} statement is associated with improvements in the
quality of reports of {RCTs}.},
Doi = {10.1001/jama.285.15.1992},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\94CAFTPU\\Moher D et al. - 2001 - Use of the consort statement and quality of report.pdf:application/pdf},
ISSN = {0098-7484},
Shorttitle = {Use of the consort statement and quality of reports of randomized trials},
Url = {http://dx.doi.org/10.1001/jama.285.15.1992},
Urldate = {2014-08-29}
}
+
@Article{moher_consort_2001,
Title = {The {CONSORT} statement: revised recommendations for improving the quality of reports of parallel group randomized trials},
Author = {Moher, David and Schulz, Kenneth F. and Altman, Douglas G.},
Journal = {{BMC} Medical Research Methodology},
Year = {2001},
+
Month = apr,
Number = {1},
Pages = {2},
Volume = {1},
+
Abstract = {To comprehend the results of a randomized controlled trial ({RCT}), readers must understand its design, conduct, analysis and interpretation. That goal can only be achieved through complete transparency from authors. Despite several decades of educational efforts, the reporting of {RCTs} needs improvement. Investigators and editors developed the original {CONSORT} (Consolidated Standards of Reporting Trials) statement to help authors improve reporting by using a checklist and flow diagram. The revised {CONSORT} statement presented in this paper incorporates new evidence and addresses some criticisms of the original statement.
{PMID}: 11336663},
Copyright = {2001 Moher et al; licensee {BioMed} Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original {URL}.},
Doi = {10.1186/1471-2288-1-2},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JF4V4KKT\\Moher et al. - 2001 - The CONSORT statement revised recommendations for.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2HHPQ9N4\\2.html:text/html},
ISSN = {1471-2288},
Language = {en},
Pmid = {11336663},
Shorttitle = {The {CONSORT} statement},
Url = {http://www.biomedcentral.com/1471-2288/1/2/abstract},
Urldate = {2014-08-29}
}
+
@Article{monogan_case_2013,
Title = {A Case for Registering Studies of Political Outcomes: An Application in the 2010 House Elections},
Author = {Monogan, James E.},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {21--37},
Volume = {21},
+
Abstract = {This article makes the case for the systematic registration of political studies. By proposing a research design before an outcome variable is observed, a researcher commits him- or herself to a theoretically motivated method for studying the object of interest. Further, study registration prompts peers of the discipline to evaluate a studys quality on its own merits, reducing norms to accept significant results and reject null findings. To advance this idea, the Political Science Registered Studies Dataverse (http://dvn.iq.harvard.edu/dvn/dv/registration) has been created, in which scholars may create a permanent record of a research design before completing a study. This article also illustrates the method of registration through a study of the impact of the immigration issue in the 2010 election for the U.S. House of Representatives. Prior to the election, a design for this study was posted on the Society for Political Methodology website (http://polmeth.wustl.edu/{mediaDetail}.php?{docId}=1258). After the votes were counted, the study was completed in accord with the design. The treatment effect in this theoretically specified design was indiscernible, but a specification search could yield a significant result. Hence, this article illustrates the argument for study registration through a case in which the result could easily be manipulated.},
Doi = {10.1093/pan/mps022},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6X4Z7VZ3\\Monogan - 2013 - A Case for Registering Studies of Political Outcom.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WAR3Z6N2\\21.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Shorttitle = {A Case for Registering Studies of Political Outcomes},
Url = {http://pan.oxfordjournals.org/content/21/1/21},
Urldate = {2014-08-27}
}
+
@Article{montgomery2013protocol,
Title = {Protocol for CONSORT-SPI: an extension for social and psychological interventions},
Author = {Montgomery, Paul and Grant, Sean and Hopewell, Sally and Macdonald, Geraldine and Moher, David and Michie, Susan and Mayo-Wilson, Evan},
Journal = {Implement Sci},
Year = {2013},
Number = {1},
Pages = {99},
Volume = {8}
}
+
@TechReport{mueller-langer_open_2014,
Title = {Open Access to Research Data: Strategic Delay and the Ambiguous Welfare Effects of Mandatory Data Disclosure},
Author = {Mueller-Langer, Frank and Versbach, Andreoli and Patrick},
Institution = {Social Science Research Network},
Year = {2014},
+
Address = {Rochester, {NY}},
Month = jun,
Number = {{ID} 2458362},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {Mandatory data disclosure is an essential feature for credible empirical work but comes at a cost: First, authors might invest less in data generation if they are not the full residual claimants of their data after their first publication. Second, authors might "strategically delay" the time of submission of papers in order to fully exploit their data in subsequent research. We analyze a three-stage model of publication and data disclosure. We derive exact conditions for positive welfare effects of mandatory data disclosure. However, we find that the transition to mandatory data disclosure has negative welfare properties if authors delay strategically.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\GK7J4693\\Papers.html:text/html},
Keywords = {data disclosure policy, strategic delay, welfare effects},
Shorttitle = {Open Access to Research Data},
Url = {http://papers.ssrn.com/abstract=2458362},
Urldate = {2014-08-29}
}
+
@Book{national_research_council_firearms_2004,
Title = {Firearms and Violence: A Critical Review},
Author = {{National Research Council}},
Publisher = {The National Academies Press},
Year = {2004},
+
Address = {Washington {DC}},
+
Abstract = {The Committee to Improve Research Information and Data on Firearms, which produced this book, was charged with providing an assessment of the strengths and limitations of the existing research and data on gun violence. One theme evident throughout the report is the relative absence of credible data that are central to answering the most basic questions about the link between firearms and violence, such as the effects of firearms on violence and the impacts of various violence-control policies. The inadequacy of data on gun ownership and use is among the most critical barriers to better understanding gun violence. The committee recommends a research effort to determine whether or not these kinds of data can be accurately collected with minimal risk to legitimate privacy concerns. Further, the committee recommends that for research purposes appropriate access be given to data maintained by regulatory and law enforcement agencies, including the trace data maintained by the Bureau of Alcohol, Tobacco, and Firearms; registration data maintained by the Federal Bureau of Investigation and State agencies; and firearms manufacturing and sales data. Further, there are difficult methodological issues regarding how various datasets might be used to address the complex causal issues of interest. The committee recommends that a methodological research program be established to address these problems. Regarding the focus of research on firearms and violence, the committee recommends further individual-level studies of the link between firearms and both lethal and nonlethal suicidal behavior. Other research recommendations pertain to deterrence and defense and interventions to reduce violence and suicide. Chapter references and tables and a subject index},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\5GX8AFGP\\abstract.html:text/html},
Url = {http://www.nap.edu/catalog.php?record_id=10881},
Urldate = {2014-08-21}
}
+
@Article{neumark_employment_2001,
Title = {The Employment Effects of Minimum Wages: Evidence from a Prespecified Research Design},
Author = {Neumark, David},
Journal = {Industrial Relations: A Journal of Economy and Society},
Year = {2001},
+
Month = jan,
Number = {1},
Pages = {121--144},
Volume = {40},
+
Abstract = {This article presentsevidence on the employment effects of recent minimum wage increases from a prespecified research design that entailed committing to a detailed set of statistical analyses prior to going to? the data. The limited data to which the prespecified research design can be applied may preclude finding many significant effects. Nonetheless, the evidence is most consistent with disemployment effects of minimum wages for younger, less-skilled workers.},
Copyright = {2001 The Regents ofthe University of California},
Doi = {10.1111/0019-8676.00199},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\IQIM6D83\\abstract\;jsessionid=6B03069B6744E8116003AD9F7EE2C4E7.html:text/html},
ISSN = {1468-232X},
Language = {en},
Shorttitle = {The Employment Effects of Minimum Wages},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/0019-8676.00199/abstract},
Urldate = {2014-08-26}
}
+
@Article{Neumark_2014_bathwater,
Title = {Revisiting the Minimum WageEmployment Debate: Throwing Out the Baby with the Bathwater?},
Author = {Neumark, David AND JM Ian Salas AND William Wascher},
Journal = {Industrial \& Labor Relations Review},
Year = {2014},
Number = {3},
Pages = {608-648},
Volume = {67},
+
Owner = {garret},
Timestamp = {2015.07.03}
}
+
@Article{neumark_minimum_2000,
Title = {Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Comment},
Author = {Neumark, David and Wascher, William},
Journal = {The American Economic Review},
Year = {2000},
+
Month = dec,
Number = {5},
Pages = {1362--1396},
Volume = {90},
+
Copyright = {Copyright 2000 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\AIESKN8D\\Neumark and Wascher - 2000 - Minimum Wages and Employment A Case Study of the .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Minimum Wages and Employment},
Url = {http://www.jstor.org/stable/2677855},
Urldate = {2014-08-20}
}
+
@Article{nosek2014registered,
Title = {Registered Reports},
Author = {Nosek, Brian A and Lakens, Dani{\"e}l},
Journal = {Social Psychology},
Year = {2014},
Number = {3},
Pages = {137--141},
Volume = {45},
+
Publisher = {Hogrefe \& Huber}
}
+
@Article{nosek_scientific_2012,
Title = {Scientific Utopia {II}. Restructuring Incentives and Practices to Promote Truth Over Publishability},
Author = {Nosek, Brian A. and Spies, Jeffrey R. and Motyl, Matt},
Journal = {Perspectives on Psychological Science},
Year = {2012},
+
Month = nov,
Number = {6},
Pages = {615--631},
Volume = {7},
+
Abstract = {An academic scientists professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, disciplinary incentives encourage design, analysis, and reporting decisions that elicit positive results and ignore negative results. Prior reports demonstrate how these incentives inflate the rate of false effects in published science. When incentives favor novelty over replication, false results persist in the literature unchallenged, reducing efficiency in knowledge accumulation. Previous suggestions to address this problem are unlikely to be effective. For example, a journal of negative results publishes otherwise unpublishable reports. This enshrines the low status of the journal and its content. The persistence of false findings can be meliorated with strategies that make the fundamental but abstract accuracy motivegetting it rightcompetitive with the more tangible and concrete incentivegetting it published. This article develops strategies for improving scientific practices and knowledge accumulation that account for ordinary human motivations and biases.},
Doi = {10.1177/1745691612459058},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\33HKNQZV\\Nosek et al. - 2012 - Scientific Utopia II. Restructuring Incentives and.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\GUZZU54I\\615.html:text/html},
ISSN = {1745-6916, 1745-6924},
Keywords = {false positives, incentives, methodology, motivated reasoning, replication},
Language = {en},
Url = {http://pps.sagepub.com/content/7/6/615},
Urldate = {2014-08-11}
}
+
@Article{obrien1984procedures,
Title = {Procedures for comparing samples with multiple endpoints},
Author = {O{'}Brien, Peter C},
Journal = {Biometrics},
Year = {1984},
Pages = {1079--1087},
+
Publisher = {JSTOR}
}
+
@Article{OlkenPAP,
Title = {Promises and Perils of Pre-Analysis Plans},
Author = {Olken, Benjamin A},
Journal = {The Journal of Economic Perspectives},
Year = {2015},
Number = {3},
Pages = {61--80},
Volume = {29},
+
Publisher = {American Economic Association}
}
+
@TechReport{olken_targeting_2009,
Title = {Targeting Analysis Protocol},
Author = {Olken, Benjamin A.},
Year = {2009},
Month = mar,
+
Url = {https://www.povertyactionlab.org/sites/default/files/documents/090318a%20Targeting%20Analysis%20Protocol.pdf}
}
+
@Article{olken_should_2014,
Title = {Should Aid Reward Performance? Evidence from a Field Experiment on Health and Education in Indonesia},
Author = {Olken, Benjamin A. and Onishi, Junko and Wong, Susan},
Journal = {American Economic Journal: Applied Economics},
Year = {2014},
Number = {4},
Pages = {1-34},
Volume = {6},
+
Doi = {10.1257/app.6.4.1},
Url = {http://www.aeaweb.org/articles.php?doi=10.1257/app.6.4.1}
}
+
@TechReport{olken_generasi_2010,
Title = {Generasi Analysis Plan: Wave {III}},
Author = {Olken, Benjamin A. and Onishi, Junko and Wong, Susan},
Year = {2010},
Month = jan,
+
Url = {http://www.povertyactionlab.org/sites/default/files/documents/100122_Generasi_AnalysisPlan_Wave_III_CLEAN.pdf}
}
+
@TechReport{olken_indonesias_2010,
Title = {Indonesia's {PNPM} Generasi program : interim impact evaluation report},
Author = {Olken, Benjamin A. and Onishi, Junko and Wong, Susan},
Institution = {The World Bank},
Year = {2010},
Month = jan,
Number = {59567},
+
Abstract = {This document describes the findings from the interim evaluation survey conducted between October and December 2007, after 15 to 18 months of Generasi implementation in 129 treatment sub districts. Since one full year's project cycle had been completed, authors refer to this survey as the one-year interim evaluation survey. A final evaluation survey is planned for October-December 2009, after the program will have been in operation for 27-30 months. The Generasi project is focused on 12 indicators of maternal and child health behavior and educational behavior. These indicators were chosen by the government of Indonesia to be as similar as possible to the conditions for the individual household conditional cash transfer program being piloted at the same time as Generasi. They are in the same spirit as the conditions used by conditional cash transfer programs in other countries, such as Progresa in Mexico. These 12 indicators respond to those seeking health and educational services that are within the direct control of villagers-such as the number of children who receive immunization, prenatal and postnatal care, and the number of children enrolled and attending school-rather than long-term outcomes, such as test scores or infant mortality. This study provides strong evidence that in this context, community incentives work and are more effective for focusing impacts on the poorest quintiles and increasing providers' efforts. The policy implications are that poverty programs may wish to experiment more with embedding incentives into their designs. The Generasi project is implemented by the government of Indonesia's Ministry of Home Affairs, and is funded in part with loans from the World Bank and grants from the Netherlands Embassy.},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WRN822RU\\Olken et al. - 2010 - Indonesia's PNPM Generasi program interim impact.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DDJ8JRIR\\indonesias-pnpm-generasi-program-interim-impact-evaluation-report.html:text/html},
Keywords = {access to health services, achievement level, Acute respiratory infection, acute respiratory infections, aged, age groups},
Language = {en},
Pages = {1--154},
Shorttitle = {Indonesia's {PNPM} Generasi program},
Url = {http://documents.worldbank.org/curated/en/2010/01/13763479/indonesias-pnpm-generasi-program-interim-impact-evaluation-report},
Urldate = {2014-08-27}
}
+
@TechReport{olken_generasi_2009,
Title = {Generasi Analysis Plan},
Author = {Olken, Benjamin A. and Onishi, Junko and Wong, Susan},
Year = {2009},
Month = apr,
+
Url = {http://www.povertyactionlab.org/sites/default/files/documents/090408_Generasi_Analysis_Plan_CLEAN.pdf}
}
+
@Article{RPP,
Title = {Estimating the reproducibility of psychological science},
Author = {{Open Science Collaboration} and others},
Journal = {Science},
Year = {2015},
Number = {6251},
Pages = {aac4716},
Volume = {349},
+
Publisher = {American Association for the Advancement of Science}
}
+
@Article{oster_explaining_2006,
Title = {On Explaining Asia's Missing Women: Comment on Das Gupta},
Author = {Oster, Emily},
Journal = {Population and Development Review},
Year = {2006},
+
Month = jun,
Number = {2},
Pages = {323--327},
Volume = {32},
+
Doi = {10.1111/j.1728-4457.2006.00120.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EAZPQVE3\\Oster - 2006 - On Explaining Asia's Missing Women? Comment on D.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8TABKTFM\\abstract.html:text/html},
ISSN = {1728-4457},
Language = {en},
Shorttitle = {On Explaining Asia's Missing Women?},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2006.00120.x/abstract},
Urldate = {2014-08-20}
}
+
@Article{oster_hepatitis_2005,
Title = {Hepatitis B and the Case of the Missing Women},
Author = {Oster, Emily},
Journal = {Journal of Political Economy},
Year = {2005},
+
Month = dec,
Number = {6},
Pages = {1163--1216},
Volume = {113},
+
Abstract = {In many Asian countries the ratio of male to female population is higher than in the West: as high as 1.07 in China and India, and even higher in Pakistan. A number of authors (most notably Amartya Sen) have suggested that this imbalance reflects excess female mortality and have argued that as many as 100 million women are missing.? This paper proposes an explanation for some of the observed overrepresentation of men: the hepatitis B virus. I present new evidence, consistent with an existing scientific literature, that carriers of the hepatitis B virus have offspring sex ratios around 1.50 boys for each girl. This evidence includes both cross?country analyses and a natural experiment based on recent vaccination campaigns. Hepatitis B is common in many Asian countries, especially China, where some 1015 percent of the population is infected. Using data on prevalence of the virus by country and estimates of the effect of hepatitis on the sex ratio, I argue that hepatitis B can account for about 45 percent of the missing women?: around 75 percent in China, between 20 and 50 percent in Egypt and western Asia, and under 20 percent in India, Bangladesh, Pakistan, and Nepal.},
Copyright = {Copyright 2005 The University of Chicago Press},
Doi = {10.1086/498588},
ISSN = {0022-3808},
Url = {http://www.jstor.org/stable/10.1086/498588},
Urldate = {2014-08-20}
}
+
@Article{oster_hepatitis_2010,
Title = {Hepatitis B does not explain male-biased sex ratios in China},
Author = {Oster, Emily and Chen, Gang and Yu, Xinsen and Lin, Wenyao},
Journal = {Economics Letters},
Year = {2010},
+
Month = may,
Number = {2},
Pages = {142--144},
Volume = {107},
+
Abstract = {Oster (2005) argued that parents with Hepatitis B ({HBV}) have more sons, which explained Asia's missing women?. Lin and Luoh (2008) show no relationship between gender and mother's {HBV}. We test for a relationship between paternal {HBV} and son share and find none.},
Doi = {10.1016/j.econlet.2010.01.007},
File = {ScienceDirect Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\VUMXQRZT\\Oster et al. - 2010 - Hepatitis B does not explain male-biased sex ratio.pdf:application/pdf;ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DC7KUXQN\\S016517651000008X.html:text/html},
ISSN = {0165-1765},
Keywords = {China, Hepatitis B, Missing women},
Url = {http://www.sciencedirect.com/science/article/pii/S016517651000008X},
Urldate = {2014-08-20}
}
+
@Article{pearce_registration_2011,
Title = {Registration of protocols for observational research is unnecessary and would do more harm than good},
Author = {Pearce, Neil},
Journal = {Occupational and Environmental Medicine},
Year = {2011},
+
Month = feb,
Number = {2},
Pages = {86--88},
Volume = {68},
+
Doi = {10.1136/oem.2010.058917},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\A33K75TG\\Pearce - 2011 - Registration of protocols for observational resear.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\94WISPPT\\86.html:text/html},
ISSN = {, 1470-7926},
Keywords = {Epidemiology},
Language = {en},
Pmid = {21118848},
Url = {http://oem.bmj.com/content/68/2/86},
Urldate = {2014-10-28}
}
+
@Article{peng_reproducible_2011,
Title = {Reproducible Research in Computational},
Author = {Peng, Roger D.},
Journal = {Science},
Year = {2011},
+
Month = dec,
Number = {6060},
Pages = {1226--1227},
Volume = {334},
+
Abstract = {Computational science has led to exciting new developments, but the nature of the work has exposed limitations in our ability to evaluate published findings. Reproducibility has the potential to serve as a minimum standard for judging scientific claims when full independent replication of a study is not possible.},
Doi = {10.1126/science.1213847},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EZ7CXGIF\\Peng - 2011 - Reproducible Research in Computational.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EUSFBQAF\\1226.html:text/html},
ISSN = {0036-8075, 1095-9203},
Language = {en},
Pmid = {22144613},
Url = {http://www.sciencemag.org/content/334/6060/1226},
Urldate = {2014-09-24}
}
+
@Article{perugini2014safeguard,
Title = {Safeguard power as a protection against imprecise power estimates},
Author = {Perugini, Marco and Gallucci, Marcello and Costantini, Giulio},
Journal = {Perspectives on Psychological Science},
Year = {2014},
Number = {3},
Pages = {319--332},
Volume = {9},
+
Publisher = {SAGE Publications}
}
+
@Article{pitt_gunfight_2012,
Title = {Gunfight at the Not {OK} Corral: Reply to High Noon for Microfinance},
Author = {Pitt, Mark M.},
Journal = {Journal of Development Studies},
Year = {2012},
Number = {12},
Pages = {1886--1891},
Volume = {48},
+
Abstract = {Abstract Duvendack and Palmer-Jones claim to replicate Chemin (2008) and Pitt and Khandker (1998) but obtain different results and hence challenge the two papers' estimates of the impact of microfinance in Bangladesh. This response details a number of reasons to demonstrate that Duvendack and Palmer-Jones is not a replication so their results provide no evidence about the validity of either of the earlier papers or on the effectiveness of microfinance.},
Doi = {10.1080/00220388.2012.727563},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\5W45VKWU\\Pitt - 2012 - Gunfight at the Not OK Corral Reply to High Noon.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\TRD2QCD7\\00220388.2012.html:text/html},
ISSN = {0022-0388},
Shorttitle = {Gunfight at the Not {OK} Corral},
Url = {http://dx.doi.org/10.1080/00220388.2012.727563},
Urldate = {2014-08-21}
}
+
@Article{plassmann_confirming_2003,
Title = {Confirming "More Guns, Less Crime"},
Author = {Plassmann, Florenz and Whitley, John},
Journal = {Stanford Law Review},
Year = {2003},
+
Month = apr,
Number = {4},
Pages = {1313--1369},
Volume = {55},
+
Abstract = {Analyzing county-level data for the entire United States from 1977 to 2000, we find annual reductions in murder rates between 1.5\% and 2.3\% for each additional year that a right-to-carry law is in effect. For the first five years that such a law is in effect, the total benefit from reduced crimes usually ranges between approximately \$2 billion and \$3 billion per year. Ayres and Donohue have simply misread their own results. Their own most general specification that breaks down the impact of the law on a year-by-year basis shows large crime-reducing benefits. Virtually none of their claims that their county-level hybrid model implies initial significant increases in crime are correct. Overall, the vast majority of their estimates-based on data up to 1997-actually demonstrate that right-to-carry laws produce substantial crime-reducing benefits. We show that their models also do an extremely poor job of predicting the changes in crime rates after 1997.},
Copyright = {Copyright 2003 Stanford Law Review},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\3C2EH4Z8\\Plassmann and Whitley - 2003 - Confirming More Guns, Less Crime.pdf:application/pdf},
ISSN = {0038-9765},
Url = {http://www.jstor.org/stable/1229604},
Urldate = {2014-08-21}
}
+
@Article{resultscompliance,
Title = {Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study},
Author = {Prayle, Andrew P and Hurley, Matthew N and Smyth, Alan R},
Journal = {BMJ},
Year = {2012},
Volume = {344},
+
Abstract = {Andrew P Prayle, NIHR doctoral research fellow, Matthew N Hurley, Wellcome Trust paediatric clinical research fellow, Alan R Smyth, professor of child health1University of Nottingham, Division of Child Health, School of Clinical Sciences, Queens Medical Centre, Nottingham NG7 2UH, UKCorrespondence to: A P Prayle andrew.prayle{at}nottingham.ac.ukAccepted 4 November 2011AbstractObjective To examine compliance with mandatory reporting of summary clinical trial results (within one year of completion of trial) on ClinicalTrials.gov for studies that fall under the recent Food and Drug Administration Amendments Act (FDAAA) legislation.Design Registry based study of clinical trial summaries. Data sources ClinicalTrials.gov, searched on 19 January 2011, with cross referencing with Drugs@FDA to determine for which trials mandatory reporting was required within one year.Selection criteria Studies registered on ClinicalTrials.gov with US sites which completed between 1 January and 31 December 2009.Main outcome measure Proportion of trials for which results had been reported.Results The ClinicalTrials.gov registry contained 83 579 entries for interventional trials, of which 5642 were completed within the timescale of interest. We identified trials as falling within the mandatory reporting rules if they were covered by the FDAAA (trials of a drug, device, or biological agent, which have at least one US site, and are of phase II or later) and if they investigated a drug that already had approval from the Food and Drug Administration. Of these, 163/738 (22\%) had reported results within one year of completion of the trial compared with 76/727 (10\%) trials that were not subject to mandatory reporting (95\% confidence interval for the difference in proportions 7.8\% to 15.5\%; ?2 test, P=2.6{\texttimes}10-9). Later phase trials were more likely to report results (P=4.4{\texttimes}10-11), as were industry funded trials (P=2.2{\texttimes}10-16).Conclusion Most trials subject to mandatory reporting did not report results within a year of completion.FootnotesContributors: APP, MNH, and ARS designed the study. APP wrote the script to webscrape additional data from ClinicalTrials.gov. APP categorised the trials, MNH cross checked a subset, and ARS arbitrated. All authors had full access to the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. All authors wrote the manuscript. APP and MNH contributed equally to this work. ARS is the guarantor.Funding: The study was not externally funded.Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that APP is supported by a National Institute for Health Research fellowship (DRF-2009-02-112) and MNH is supported by a Wellcome Trust fellowship (WT092295AIA). ARS declares relevant activities outside the submitted work of membership of a REMPEX steering committee, consultancies for Novartis and Biocontrol, and a lecture paid for by Chiesi Pharma. ARS has registered trials on ClinicalTrials.gov and other registries.Ethical approval: Not needed.Data sharing: Technical appendix, statistical code, and dataset available from the corresponding author (andrew.prayle{at}nottingham.ac.uk) or at http://dx.doi.org/10.5061/dryad.j512f21p. Participants{\textquoteright} consent for data sharing not required.This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.},
Doi = {10.1136/bmj.d7373},
ISBN = {1468-5833},
ISSN = {0959-8138},
Publisher = {BMJ Publishing Group Ltd}
}
+
@Article{reinhart_growth_2010,
Title = {Growth in a Time of Debt},
Author = {Reinhart, Carmen M. and Rogoff, Kenneth S.},
Journal = {The American Economic Review},
Year = {2010},
+
Month = may,
Number = {2},
Pages = {573--578},
Volume = {100},
+
Copyright = {Copyright 2010 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HZU3ZQRI\\Reinhart and Rogoff - 2010 - Growth in a Time of Debt.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/27805061},
Urldate = {2014-08-20}
}
+
@Article{rosenthal1979file,
Title = {The file drawer problem and tolerance for null results.},
Author = {Rosenthal, Robert},
Journal = {Psychological bulletin},
Year = {1979},
Number = {3},
Pages = {638},
Volume = {86},
+
Publisher = {American Psychological Association}
}
+
@Article{rothstein_does_2007,
Title = {Does Competition among Public Schools Benefit Students and Taxpayers? Comment},
Author = {Rothstein, Jesse},
Journal = {The American Economic Review},
Year = {2007},
+
Month = dec,
Number = {5},
Pages = {2026--2037},
Volume = {97},
+
Copyright = {Copyright 2007 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BRTERQBH\\Rothstein - 2007 - Does Competition among Public Schools Benefit Stud.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Does Competition among Public Schools Benefit Students and Taxpayers?},
Url = {http://www.jstor.org/stable/30034599},
Urldate = {2014-08-20}
}
+
@Article{rushton_should_2011,
Title = {Should protocols for observational research be registered?},
Author = {Rushton, L.},
Journal = {Occupational and Environmental Medicine},
Year = {2011},
+
Month = feb,
Number = {2},
Pages = {84--86},
Volume = {68},
+
Doi = {10.1136/oem.2010.056846},
ISSN = {, 1470-7926},
Keywords = {Epidemiology},
Language = {en},
Pmid = {21123807},
Url = {http://oem.bmj.com/content/68/2/84},
Urldate = {2014-10-28}
}
+
@Article{sala-i-martin_i_1997,
Title = {I Just Ran Two Million Regressions},
Author = {Sala-I-Martin, Xavier X.},
Journal = {The American Economic Review},
Year = {1997},
+
Month = may,
Number = {2},
Pages = {178--183},
Volume = {87},
+
Copyright = {Copyright 1997 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\7F85Z3X2\\Sala-I-Martin - 1997 - I Just Ran Two Million Regressions.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/2950909},
Urldate = {2014-08-28}
}
+
@Article{samet_register_2010,
Title = {To Register or Not To Register:},
Author = {Samet, Jonathan M.},
Journal = {Epidemiology},
Year = {2010},
+
Month = sep,
Number = {5},
Pages = {610--611},
Volume = {21},
+
Doi = {10.1097/EDE.0b013e3181e9be54},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {To Register or Not To Register},
Url = {http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001648-201009000-00010},
Urldate = {2014-10-28}
}
+
@Article{saquib_practices_2013,
Title = {Practices and impact of primary outcome adjustment in randomized controlled trials: meta-epidemiologic study},
Author = {Saquib, N. and Saquib, J. and Ioannidis, J. P. A.},
Journal = {{BMJ}},
Year = {2013},
+
Month = jul,
Number = {jul12 2},
Pages = {f4313--f4313},
Volume = {347},
+
Doi = {10.1136/bmj.f4313},
File = {Practices and impact of primary outcome adjustment in randomized controlled trials\: meta-epidemiologic study | The BMJ:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\Q5M2NPIE\\bmj.html:text/html},
ISSN = {1756-1833},
Language = {en},
Shorttitle = {Practices and impact of primary outcome adjustment in randomized controlled trials},
Url = {http://www.bmj.com/content/347/bmj.f4313},
Urldate = {2014-10-09}
}
+
@Article{sarsons_rainfall_2011,
Title = {Rainfall and Conflict},
Author = {Sarsons, Heather},
Year = {2011},
+
Month = oct,
+
Abstract = {Starting with Miguel, Satyanath, and Sergenti (2004), a large literature has used rainfall variation as an instrument to study the impacts of income shocks on civil war and
conict. These studies argue that in agriculturally-dependent regions, negative rain shocks lower income levels, which in turn incites violence. This identifcation strategy
relies on the assumption that rainfall shocks affect conflict only through their impacts on
income. I evaluate this exclusion restriction by identifying districts that are downstream from dams in India. In downstream districts, income is much less sensitive to rainfall fluctuations. However, rain shocks remain equally strong predictors of riot incidence in these districts. These results suggest that rainfall affects rioting through a channel other than income and cast doubt on the conclusion that income shocks incite riots.},
Url = {http://www.econ.yale.edu/conference/neudc11/papers/paper_199.pdf}
}
+
@Article{schulz_consort_2010,
Title = {{CONSORT} 2010 Statement: updated guidelines for reporting parallel group randomised trials},
Author = {Schulz, Kenneth F. and Altman, Douglas G. and Moher, David and \$author.lastName, \$author firstName},
Journal = {{BMC} Medicine},
Year = {2010},
+
Month = mar,
Number = {1},
Pages = {18},
Volume = {8},
+
Abstract = {The {CONSORT} statement is used worldwide to improve the reporting of randomised controlled trials. Kenneth Schulz and colleagues describe the latest version, {CONSORT} 2010, which updates the reporting guideline based on new methodological evidence and accumulating experience.
{PMID}: 20334633},
Copyright = {2010 Schulz et al; licensee {BioMed} Central Ltd.},
Doi = {10.1186/1741-7015-8-18},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\J65R5ZR7\\Schulz et al. - 2010 - CONSORT 2010 Statement updated guidelines for rep.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\UNG8BJ48\\18.html:text/html},
ISSN = {1741-7015},
Language = {en},
Pmid = {20334633},
Shorttitle = {{CONSORT} 2010 Statement},
Url = {http://www.biomedcentral.com/1741-7015/8/18/abstract},
Urldate = {2014-08-29}
}
+
@Article{schulz_multiplicity_2005,
Title = {Multiplicity in randomised trials {II}: subgroup and interim analyses},
Author = {Schulz, Kenneth F and Grimes, David A},
Journal = {The Lancet},
Year = {2005},
+
Month = may,
Number = {9471},
Pages = {1657--1661},
Volume = {365},
+
Abstract = {Summary
Subgroup analyses can pose serious multiplicity concerns. By testing enough subgroups, a false-positive result will probably emerge by chance alone. Investigators might undertake many analyses but only report the significant effects, distorting the medical literature. In general, we discourage subgroup analyses. However, if they are necessary, researchers should do statistical tests of interaction, rather than analyse every separate subgroup. Investigators cannot avoid interim analyses when data monitoring is indicated. However, repeatedly testing at every interim raises multiplicity concerns, and not accounting for multiplicity escalates the false-positive error. Statistical stopping methods must be used. The O'Brien-Fleming and Peto group sequential stopping methods are easily implemented and preserve the intended level and power. Both adopt stringent criteria (low nominal p values) during the interim analyses. Implementing a trial under these stopping rules resembles a conventional trial, with the exception that it can be terminated early should a treatment prove greatly superior. Investigators and readers, however, need to grasp that the estimated treatment effects are prone to exaggeration, a random high, with early stopping.},
Doi = {10.1016/S0140-6736(05)66516-6},
File = {ScienceDirect Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\K5A5SQTH\\Schulz and Grimes - 2005 - Multiplicity in randomised trials II subgroup and.pdf:application/pdf;ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6QEUHN8R\\S0140673605665166.html:text/html},
ISSN = {0140-6736},
Shorttitle = {Multiplicity in randomised trials {II}},
Url = {http://www.sciencedirect.com/science/article/pii/S0140673605665166},
Urldate = {2014-08-28}
}
+
@Article{schulz_allocation_2002,
Title = {Allocation concealment in randomised trials: defending against deciphering},
Author = {Schulz, Kenneth F and Grimes, David A},
Journal = {The Lancet},
Year = {2002},
+
Month = feb,
Number = {9306},
Pages = {614--618},
Volume = {359},
+
Doi = {10.1016/S0140-6736(02)07750-4},
File = {Allocation concealment in randomised trials\: defending against deciphering \: The Lancet:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CVFTHIJQ\\fulltext.html:text/html},
ISSN = {01406736},
Language = {en},
Shorttitle = {Allocation concealment in randomised trials},
Url = {http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(02)07750-4/fulltext},
Urldate = {2014-09-17}
}
+
@Article{simmons_false-positive_2011,
Title = {False-Positive Psychology Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant},
Author = {Simmons, Joseph P. and Nelson, Leif D. and Simonsohn, Uri},
Journal = {Psychological Science},
Year = {2011},
+
Month = nov,
Number = {11},
Pages = {1359--1366},
Volume = {22},
+
Abstract = {In this article, we accomplish two things. First, we show that despite empirical psychologists nominal endorsement of a low rate of false-positive findings ( .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.},
Doi = {10.1177/0956797611417632},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HT3VR6SV\\Simmons et al. - 2011 - False-Positive Psychology Undisclosed Flexibility .pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DX68RTPX\\1359.html:text/html},
ISSN = {0956-7976, 1467-9280},
Keywords = {disclosure, methodology, motivated reasoning, publication},
Language = {en},
Pmid = {22006061},
Url = {http://pss.sagepub.com/content/22/11/1359},
Urldate = {2014-08-28}
}
+
@Article{simonsohn2013just,
Title = {Just post it: the lesson from two cases of fabricated data detected by statistics alone.},
Author = {Simonsohn, Uri},
Journal = {Psychological science},
Year = {2013},
Number = {10},
Pages = {1875},
Volume = {24}
}
+
@Article{simonsohn2014p,
Title = {P-curve: A key to the file-drawer.},
Author = {Simonsohn, Uri and Nelson, Leif D and Simmons, Joseph P},
Journal = {Journal of Experimental Psychology: General},
Year = {2014},
Number = {2},
Pages = {534},
Volume = {143},
+
Publisher = {American Psychological Association}
}
+
@TechReport{simonsohn_p-curve:_2013,
Title = {P-Curve: A Key to the File Drawer},
Author = {Simonsohn, Uri and Nelson, Leif D. and Simmons, Joseph P.},
Institution = {Social Science Research Network},
Year = {2013},
+
Address = {Rochester, {NY}},
Month = apr,
Number = {{ID} 2256237},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that work?, readers must ask, Are these effects true, or do they merely reflect selective reporting?? We introduce p-curve as a way to answer this question. P-curve is the distribution of statistically significant p-values for a set of studies (ps {\textless} .05). Because only true effects are expected to generate right-skewed p-curves containing more low (.01s) than high (.04s) significant p-values only right-skewed p-curves are diagnostic of evidential value. By telling us whether we can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\RI9CPIUI\\papers.html:text/html},
Keywords = {decision making, file drawer, judgment, p-hacking, Publication bias, science, statistics},
Shorttitle = {P-Curve},
Url = {http://papers.ssrn.com/abstract=2256237},
Urldate = {2014-09-04}
}
+
@Article{siskind_minimum_1977,
Title = {Minimum Wage Legislation in the United States: Comment},
Author = {Siskind, Frederic B.},
Journal = {Economic Inquiry},
Year = {1977},
+
Month = jan,
Number = {1},
Pages = {135--138},
Volume = {15},
+
Doi = {10.1111/j.1465-7295.1977.tb00457.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\RJK53NXB\\Siskind - 1977 - Minimum Wage Legislation in the United States Com.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\URRDFTHJ\\abstract.html:text/html},
ISSN = {1465-7295},
Language = {en},
Shorttitle = {Minimum Wage Legislation in the United States},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1465-7295.1977.tb00457.x/abstract},
Urldate = {2014-10-01}
}
+
@Article{sorge2014olfactory,
Title = {Olfactory exposure to males, including men, causes stress and related analgesia in rodents},
Author = {Sorge, Robert E and Martin, Loren J and Isbester, Kelsey A and Sotocinal, Susana G and Rosen, Sarah and Tuttle, Alexander H and Wieskopf, Jeffrey S and Acland, Erinn L and Dokova, Anastassia and Kadoura, Basil and others},
Journal = {Nature methods},
Year = {2014},
+
Publisher = {Nature Publishing Group}
}
+
@Article{stanley_beyond_2005,
Title = {Beyond Publication Bias},
Author = {Stanley, T. D.},
Journal = {Journal of Economic Surveys},
Year = {2005},
+
Month = jul,
Number = {3},
Pages = {309--345},
Volume = {19},
+
Abstract = {Abstract. This review considers several meta-regression and graphical methods that can differentiate genuine empirical effect from publication bias. Publication selection exists when editors, reviewers, or researchers have a preference for statistically significant results. Because all areas of empirical research are susceptible to publication selection, any average or tally of significant/insignificant studies is likely to be biased and potentially misleading. Meta-regression analysis can see through the murk of random sampling error and selected misspecification bias to identify the underlying statistical structures that characterize genuine empirical effect. Meta-significance testing and precision-effect testing ({PET}) are offered as a means to identify empirical effect beyond publication bias and are applied to four areas of empirical economics research minimum wage effects, union-productivity effects, price elasticities, and tests of the natural rate hypothesis.},
Doi = {10.1111/j.0950-0804.2005.00250.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\GBSEMN2E\\Stanley - 2005 - Beyond Publication Bias.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\M72Q5RS2\\full.html:text/html},
ISSN = {1467-6419},
Keywords = {Elasticities, Funnel plot, Meta-Regression Analysis, Minimum wage, Natural rate hypothesis, Publication bias, Union-productivity},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.0950-0804.2005.00250.x/abstract},
Urldate = {2014-08-19}
}
+
@Article{stanley2005beyond,
Title = {Beyond publication bias},
Author = {Stanley, Tom D},
Journal = {Journal of Economic Surveys},
Year = {2005},
Number = {3},
Pages = {309--345},
Volume = {19},
+
Publisher = {Wiley Online Library}
}
+
@Misc{sven_vlaeminck_research_????,
Title = {Research Data Management in Economic Journals},
+
Author = {{Sven Vlaeminck}},
+
Abstract = {The Open Economics Working Group of the Open Knowledge Foundation Identifying best practice as well as legal, regulatory and technical standards for open economic data},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\RPVJZQN8\\data-policies-of-economic-journals.html:text/html},
Journal = {Data Policies of Economics Journals},
Url = {http://openeconomics.net/resources/data-policies-of-economic-journals/},
Urldate = {2014-08-29}
}
+
@Article{ICMJEData,
Title = {Sharing Clinical Trial Data: A Proposal From the International Committee of Medical Journal EditorsSharing Clinical Trial Data},
Author = {Taichman, Darren B. and Backus, Joyce and Baethge, Christopher and Bauchner, Howard and de Leeuw, Peter W. and Drazen, Jeffrey M. and Fletcher, John and Frizelle, Frank A. and Groves, Trish and Haileamlak, Abraham and James, Astrid and Laine, Christine and Peiperl, Larry and Pinborg, Anja and Sahni, Peush and Wu, Sinan},
Journal = {Annals of Internal Medicine},
Year = {2016},
+
Doi = {10.7326/M15-2928},
Url = { + http://dx.doi.org/10.7326/M15-2928}
}
+
@Article{takkouche_meta-analysis_2010,
Title = {Meta-analysis Protocol Registration: Sed quis custodiet ipsos custodes? [But Who Will Guard the Guardians?]},
Author = {Takkouche, Bahi and Norman, Guy},
Journal = {Epidemiology},
Year = {2010},
+
Month = sep,
Number = {5},
Pages = {614--615},
Volume = {21},
+
Doi = {10.1097/EDE.0b013e3181e9bbbd},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {Meta-analysis Protocol Registration},
Url = {http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001648-201009000-00012},
Urldate = {2014-10-28}
}
+
@Article{taubman_oregon_2013,
Title = {{THE} {OREGON} {HEALTH} {INSURANCE} {EXPERIMENT}: {EVIDENCE} {FROM} {EMERGENCY} {DEPARTMENT} {DATA} Analysis Plan},
Author = {Taubman, Sarah and Allen, Heidi and Baicker, Katherine and Wright, Bill and Finkelstein, Amy},
Year = {2013},
+
Month = mar,
+
Series = {{NBER} Working Paper},
Url = {http://www.nber.org/oregon/files/ED%20Analysis%20Plan.pdf}
}
+
@Article{Taubman17012014,
Title = {Medicaid Increases Emergency-Department Use: Evidence from Oregon's Health Insurance Experiment},
Author = {Taubman, Sarah L. and Allen, Heidi L. and Wright, Bill J. and Baicker, Katherine and Finkelstein, Amy N.},
Journal = {Science},
Year = {2014},
Number = {6168},
Pages = {263-268},
Volume = {343},
+
Abstract = {In 2008, Oregon initiated a limited expansion of a Medicaid program for uninsured, low-income adults, drawing names from a waiting list by lottery. This lottery created a rare opportunity to study the effects of Medicaid coverage by using a randomized controlled design. By using the randomization provided by the lottery and emergency-department records from Portland-area hospitals, we studied the emergency department use of about 25,000 lottery participants over about 18 months after the lottery. We found that Medicaid coverage significantly increases overall emergency use by 0.41 visits per person, or 40% relative to an average of 1.02 visits per person in the control group. We found increases in emergency-department visits across a broad range of types of visits, conditions, and subgroups, including increases in visits for conditions that may be most readily treatable in primary care settings.},
Doi = {10.1126/science.1246183},
Eprint = {http://www.sciencemag.org/content/343/6168/263.full.pdf},
Url = {http://www.sciencemag.org/content/343/6168/263.abstract}
}
+
@Article{the_lancet_should_2010,
Title = {Should protocols for observational research be registered?},
Author = {{The Lancet}},
Journal = {The Lancet},
Year = {2010},
+
Month = jan,
Number = {9712},
Pages = {348},
Volume = {375},
+
Doi = {10.1016/S0140-6736(10)60148-1},
ISSN = {01406736},
Language = {en},
Url = {http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(10)60148-1/fulltext},
Urldate = {2014-10-28}
}
+
@Article{turner_selective_2008,
Title = {Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy},
Author = {Turner, Erick H. and Matthews, Annette M. and Linardatos, Eftihia and Tell, Robert A. and Rosenthal, Robert},
Journal = {New England Journal of Medicine},
Year = {2008},
+
Month = jan,
Number = {3},
Pages = {252--260},
Volume = {358},
+
Abstract = {Medical decisions are based on an understanding of publicly reported clinical trials.1,2 If the evidence base is biased, then decisions based on this evidence may not be the optimal decisions. For example, selective publication of clinical trials, and the outcomes within those trials, can lead to unrealistic estimates of drug effectiveness and alter the apparent riskbenefit ratio.3,4 Attempts to study selective publication are complicated by the unavailability of data from unpublished trials. Researchers have found evidence for selective publication by comparing the results of published trials with information from surveys of authors,5 registries,6 institutional review boards,7,8 and . . .},
Doi = {10.1056/NEJMsa065779},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\K8IGA3UZ\\Turner et al. - 2008 - Selective Publication of Antidepressant Trials and.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\S6KX8JVP\\nejmsa065779.html:text/html},
ISSN = {0028-4793},
Pmid = {18199864},
Url = {http://www.nejm.org/doi/full/10.1056/NEJMsa065779},
Urldate = {2014-10-07}
}
+
@Article{van2011targeted,
Title = {Targeted learning},
Author = {Van der Laan, Mark J and Petersen, Maya L},
Journal = {Ensemble Machine Learning},
Year = {2011},
Pages = {117--156},
+
Publisher = {Springer}
}
+
@Article{strobestatement2007,
Title = {The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies},
Author = {Von Elm, Erik and Altman, Douglas G and Egger, Matthias and Pocock, Stuart J and G{\o}tzsche, Peter C and Vandenbroucke, Jan P and Strobe Initiative and others},
Journal = {Preventive medicine},
Year = {2007},
Number = {4},
Pages = {247--251},
Volume = {45},
+
Publisher = {Elsevier}
}
+
@Article{weiss_identification_2011,
Title = {The Identification and Prevention of Publication Bias in the Social Sciences and Economics},
Author = {Weiss, Bernd and Wagner, Michael},
Journal = {Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik)},
Year = {2011},
Number = {5-6},
Pages = {661--684},
Volume = {231},
+
Abstract = {Systematic research reviews have become essential in all empirical sciences. However, the validity of research syntheses is threatened by the fact that not all studies on a given topic can be summarized. Research reviews may suffer from missing data, and this is especially crucial in those cases where the selectivity of studies and their findings affects the summarized result. So-called publication bias is a type of missing data and a phenomenon that jeopardizes the validity of systematic or quantitative, as well as narrative, reviews. Publication bias exists if the preparation, submission or publication of research findings depend on characteristics of just these research results, e. g. their direction or statistical significance. This article describes methods to identify publication bias in the context of meta-analysis. It also reviews empirical studies on the prevalence of publication bias, especially in the social and economic sciences, where publication bias also seems to be prevalent. Several proposals to prevent publication bias are discussed.},
File = {RePEc Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\XDK5FC44\\v231y2011i5-6p661-684.html:text/html},
Keywords = {meta analysis, Publication bias, Systematic review},
Url = {http://ideas.repec.org/a/jns/jbstat/v231y2011i5-6p661-684.html},
Urldate = {2014-08-20}
}
+
@Article{welch_minimum_1977,
Title = {Minimum Wage Legislation in the United States: Reply},
Author = {Welch, Finis},
Journal = {Economic Inquiry},
Year = {1977},
+
Month = jan,
Number = {1},
Pages = {139--142},
Volume = {15},
+
Doi = {10.1111/j.1465-7295.1977.tb00458.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\QFPT7Z3T\\Welch - 1977 - Minimum Wage Legislation in the United States Rep.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SIEIJ4D5\\abstract.html:text/html},
ISSN = {1465-7295},
Language = {en},
Shorttitle = {Minimum Wage Legislation in the United States},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1465-7295.1977.tb00458.x/abstract},
Urldate = {2014-10-01}
}
+
@Article{welch_minimum_1974,
Title = {Minimum Wage Legislation in the United States*},
Author = {Welch, Finis},
Journal = {Economic Inquiry},
Year = {1974},
+
Month = sep,
Number = {3},
Pages = {285--318},
Volume = {12},
+
Doi = {10.1111/j.1465-7295.1974.tb00401.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\ZVZUK3WG\\Welch - 1974 - Minimum Wage Legislation in the United States.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\46BQDM3X\\abstract.html:text/html},
ISSN = {1465-7295},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1465-7295.1974.tb00401.x/abstract},
Urldate = {2014-10-01}
}
+
@Book{westfall_young_multiple,
Title = {Resampling-Based Multiple Testing},
Author = {Peter H. Westfall and S. Stanley Young},
Publisher = {Wiley},
Year = {1993},
+
Owner = {garret},
Timestamp = {2014.11.20}
}
+
@Article{williams_registration_2010,
Title = {Registration of observational studies: Is it time?},
Author = {Williams, Rebecca J. and Tse, Tony and Harlan, William R. and Zarin, Deborah A.},
Journal = {Canadian Medical Association Journal},
Year = {2010},
+
Month = oct,
Number = {15},
Pages = {1638--1642},
Volume = {182},
+
Doi = {10.1503/cmaj.092252},
ISSN = {0820-3946, 1488-2329},
Language = {en},
Pmid = {20643833},
Shorttitle = {Registration of observational studies},
Url = {http://www.cmaj.ca/content/182/15/1638},
Urldate = {2014-10-28}
}
+
@Misc{wohlrabe_open_2014,
Title = {Do open access articles in economics have a citation advantage?},
+
Author = {Wohlrabe, Klaus and Birkmeier, Daniel},
Month = jun,
Year = {2014},
+
Abstract = {We investigate whether articles in economics that are freely available on the web have a citation advantage over articles with a gated access. Our sample consists of articles from 2005 from 13 economic journals (including the top five journals). In addition to standard mean comparisons we also use a negative-binomial regression model with several covariates to control for potential selection effects and quality bias. Using citation data from three different databases (Web of Science, {RePEc} and Google Scholar) we show that articles that are freely available on the internet have indeed a significantly higher citation count.},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\D7HWA4F9\\Wohlrabe and Birkmeier - 2014 - Do open access articles in economics have a citati.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KUGBM567\\56842.html:text/html},
Keywords = {A12 - Relation of Economics to Other Disciplines, A14 - Sociology of Economics},
Language = {en},
Type = {{MPRA} Paper},
Url = {http://mpra.ub.uni-muenchen.de/56842/},
Urldate = {2014-08-20}
}
+
@Article{wydick_-kind_2014,
Title = {Do in-kind transfers damage local markets? The case of {TOMS} shoe donations in El Salvador},
Author = {Wydick, Bruce and Katz, Elizabeth and Janet, Brendan},
Journal = {Journal of Development Effectiveness},
Year = {2014},
+
Month = may,
Number = {3},
Pages = {249--267},
Volume = {6},
+
Abstract = {We carry out a cluster randomised trial among 979 households in rural El Salvador to test whether shoe donations exhibit negative impacts on local shoe markets. Households in half of the communities were given a pair of childrens shoes at baseline (treatment communities), while all households were given coupons that could be used for shoe purchases at a local shoe store. Although point estimates on coupon redemption and difference-in-difference estimations indicate shoe purchases to be slightly lower among households receiving the donated shoes, we find no statistically significant difference in market shoe purchases between treatment and control households.},
Doi = {10.1080/19439342.2014.919012},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FHPGCJRP\\19439342.2014.html:text/html},
ISSN = {1943-9342},
Shorttitle = {Do in-kind transfers damage local markets?},
Url = {http://dx.doi.org/10.1080/19439342.2014.919012},
Urldate = {2014-08-27}
}
+
@InCollection{xie_knitr:_2014,
Title = {knitr: A Comprehensive Tool for Reproducible Research in R},
Author = {Xie, Yihui},
Booktitle = {Implementing Reproducible Research},
Publisher = {{CRC} Press},
Year = {2014},
Month = apr,
Pages = {3--32},
+
Abstract = {In computational science, reproducibility requires that researchers make code and data available to others so that the data can be analyzed in a similar manner as in the original publication. Code must be available to be distributed, data must be accessible in a readable format, and a platform must be available for widely distributing the data and code. In addition, both data and code need to be licensed permissively enough so that others can reproduce the work without a substantial legal burden. Implementing Reproducible Research covers many of the elements necessary for conducting and distributing reproducible research. It explains how to accurately reproduce a scientific result. Divided into three parts, the book discusses the tools, practices, and dissemination platforms for ensuring reproducibility in computational science. It describes: Computational tools, such as Sweave, knitr, {VisTrails}, Sumatra, {CDE}, and the Declaratron system Open source practices, good programming practices, trends in open science, and the role of cloud computing in reproducible research Software and methodological platforms, including open source software packages, {RunMyCode} platform, and open access journals Each part presents contributions from leaders who have developed software and other products that have advanced the field. Supplementary material is available at www.{ImplementingRR}.org.},
ISBN = {9781466561595},
Keywords = {Computers / Mathematical \& Statistical Software, Mathematics / Probability \& Statistics / General, Science / Life Sciences / Biology},
Language = {en}
}
+
@Book{xie_dynamic_2013,
Title = {Dynamic Documents with R and knitr},
Author = {Xie, Yihui},
Publisher = {{CRC} Press},
Year = {2013},
Month = jul,
+
Abstract = {The cut-and-paste approach to writing statistical reports is not only tedious and laborious, but also can be harmful to scientific research, because it is inconvenient to reproduce the results. Dynamic Documents with R and knitr introduces a new approach via dynamic documents, i.e. integrating computing directly with reporting. A comprehensive guide to the R package knitr, the book covers examples, document editors, basic usage, detailed explanations of a wide range of options, tricks and solutions, extensions, and complete applications of this package. The book provides an overview of dynamic documents, introducing the idea of literate programming. It then explains the importance of dynamic documents to scientific research and its impact on reproducible research. Building on this, the author covers basic concepts, common text editors that support knitr, and the syntax for different document formats such as {LaTeX}, {HTML}, and Markdown before going on to discuss core functionality, how to control text and graphics output, caching mechanisms that can reduce computation time, and reuse of source code. He then explores advanced topics such as chunk hooks, integrating other languages such as Python and awk into one report in the knitr framework, and useful tricks that make it easier to write documents with knitr. Discussions of how to publish reports in a variety of formats, applications, and other tools complete the coverage. Suitable for both beginners and advanced users, this book shows you how to write reports in simple languages such as Markdown. The reports range from homework, projects, exams, books, blogs, and web pages to any documents related to statistical graphics, computing, and data analysis. While familiarity with {LaTeX} and {HTML} is helpful, the book requires no prior experience with advanced programs or languages. For beginners, the text provides enough features to get started on basic applications. For power users, the last several chapters enable an understanding of the extensibility of the knitr package.},
ISBN = {9781482203530},
Keywords = {Computers / Mathematical \& Statistical Software, Mathematics / Probability \& Statistics / General, Science / Life Sciences / Biology},
Language = {en}
}
+
@Article{_registration_2010,
Title = {The Registration of Observational Studies When Metaphors Go Bad:},
Journal = {Epidemiology},
Year = {2010},
+
Month = jul,
Pages = {1},
+
Doi = {10.1097/EDE.0b013e3181eafbcf},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {The Registration of Observational StudiesWhen Metaphors Go Bad},
Url = {http://journals.lww.com/epidem/Fulltext/2010/09000/The_Registration_of_Observational_Studies_When.9.aspx},
Urldate = {2014-10-28}
}
+ + + +
+ +
+ + + + +
+ +
+ +
+
+ +
+ + + + + + +
+ + + You can't perform that action at this time. +
+ + + + + + + + + + +
+ + You signed in with another tab or window. Reload to refresh your session. + You signed out in another tab or window. Reload to refresh your session. +
+ + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Example 1.Rmd b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Example 1.Rmd new file mode 100644 index 0000000..8822cde --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Example 1.Rmd @@ -0,0 +1,65 @@ +--- +title: "Example 1 - Birthday Problem" +author: "JPT" +date: "December 5, 2017" +output: + pdf_document: default + html_document: default +--- + +```{r setup, include=FALSE} +knitr::opts_chunk$set(echo = TRUE) +set.seed = 1234 +n.pers = 23 +``` + +# The Birthday Problem + +The birthday problem - the probability that at least two people in the room have an identical birth date. + +Is it something like $\frac{1}{365} \times N =$ `r round(1/365 * n.pers, 3)`? + +Code for this: + +\begin{align} + 1 - \bar p(n) &= 1 \times \left(1-\frac{1}{365}\right) + \times \left(1-\frac{2}{365}\right) \times \cdots \times + \left(1-\frac{n-1}{365}\right) \nonumber \\ + &= \frac{ 365 \times 364 \times \cdots \times + (365-n+1) }{ 365^n } \nonumber \\ + &= \frac{ 365! }{ 365^n (365-n)!} = + \frac{n!\cdot\binom{365}{n}}{365^n}\\ +p(n= `r n.pers`) &= `r + round(1 - factorial(n.pers) * + choose(365,n.pers)/ 365^n.pers, 3)`\nonumber +\end{align} + + +# Simulate this stuff + + 1 - Simulate 10,000 rooms with $n = `r n.pers`$ random birthdays, and store the results in matrix where each row represents a room. + 2 - For each room (row) compute the number of unique birthdays. + 3 - Compute the average number of times a room has `r n.pers` unique birthdays, across 10,000 simulations, and report the complement. + +```{r birthday prob, eval=TRUE, echo=TRUE} +birthday.prob = function(n.pers, n.sims) { + # simulate birthdays + birthdays = matrix(round(runif(n.pers * n.sims, 1, 365)), + nrow = n.sims, ncol = n.pers) + # for each room (row) get unique birthdays + unique.birthdays = apply(birthdays, 1, unique) + # Indicator with 1 if all are unique birthdays + all.different = (lapply(unique.birthdays, length) == n.pers) + # Compute average time all have different birthdays + result = 1 - mean(all.different) +return(result) +} +n.pers.param = n.pers +n.sims.param = 1e4 +birthday.prob(n.pers.param,n.sims.param) +``` +## Results + +- Many people originally think of a prob ~ $\frac{1}{365} \times N =$ `r round(1/365 * n.pers.param, 3)` +- However the true probability is of $p(n= `r n.pers.param`) = `r round(1 - factorial(n.pers.param) * choose(365,n.pers.param)/ 365^n.pers.param, 3)`$ +- And the simulated probability is of `r birthday.prob(n.pers = n.pers.param, n.sims = n.sims.param)` diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Example_1.html b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Example_1.html new file mode 100644 index 0000000..c6cb76b --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Example_1.html @@ -0,0 +1,389 @@ + + + + + + + + + + + + + + +Example 1 - Birthday Problem + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + + + + + +
+

The Birthday Problem

+

The birthday problem - the probability that at least two people in the room have an identical birth date.

+

Is it something like \(\frac{1}{365} \times N =\) 0.063?

+

Code for this: https://goo.gl/cf3w1Y

+\[\begin{align} + 1 - \bar p(n) &= 1 \times \left(1-\frac{1}{365}\right) + \times \left(1-\frac{2}{365}\right) \times \cdots \times + \left(1-\frac{n-1}{365}\right) \nonumber \\ + &= \frac{ 365 \times 364 \times \cdots \times + (365-n+1) }{ 365^n } \nonumber \\ + &= \frac{ 365! }{ 365^n (365-n)!} = + \frac{n!\cdot\binom{365}{n}}{365^n}\\ +p(n= 23) &= 0.507\nonumber +\end{align}\] +
+
+

Simulate this stuff

+

1 - Simulate 10,000 rooms with \(n = 23\) random birthdays, and store the results in matrix where each row represents a room.
+2 - For each room (row) compute the number of unique birthdays.
+3 - Compute the average number of times a room has 23 unique birthdays, across 10,000 simulations, and report the complement.

+
birthday.prob = function(n.pers, n.sims) {
+  # simulate birthdays
+  birthdays = matrix(round(runif(n.pers * n.sims, 1, 365)), 
+                      nrow = n.sims, ncol = n.pers)
+  # for each room (row) get unique birthdays
+  unique.birthdays = apply(birthdays, 1, unique)
+  # Indicator with 1 if all are unique birthdays
+  all.different = (lapply(unique.birthdays, length) == n.pers)
+  # Compute average time all have different birthdays 
+  result = 1 - mean(all.different)
+return(result)
+}
+n.pers.param = n.pers
+n.sims.param = 1e4
+birthday.prob(n.pers.param,n.sims.param)
+
## [1] 0.502
+
+

Results

+
    +
  • Many people originally think of a prob ~ \(\frac{1}{365} \times N =\) 0.063
  • +
  • However the true probability is of \(p(n= 23) = 0.507\)
  • +
  • And the simulated probability is of 0.5075
  • +
+
+
+ + + + +
+ + + + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Example_1.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Example_1.pdf new file mode 100644 index 0000000..548707e Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Example_1.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/LICENSE b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/LICENSE new file mode 100644 index 0000000..6762e2c --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2017 Berkeley Initiative for Transparency in the Social Sciences + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Manual.tex b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Manual.tex new file mode 100644 index 0000000..a363036 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/Manual.tex @@ -0,0 +1,6142 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Annual2017/Manual.tex at master · BITSS/Annual2017 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Skip to content +
+ + + + +
+
+
+
+ Help save net neutrality! + A free, open internet is once again at stake—and we need your help. +
+
+ Learn more + +
+
+
+ + + + + + + + + + +
+ +
+ +
+
+ + + +
+
+
+ + + + + + + + +
+
+ +
    +
  • +
    + +
    + + + +
    +
    +
    + + Notifications +
    + + + +
    +
    +
    +
    +
  • + +
  • + +
    +
    + + + +
    +
    + + + +
    + +
  • + +
  • + + + Fork + + + + + +
  • +
+ +

+ + /Annual2017 + +

+ +
+ + + + +
+ +
+
+ + + Permalink + + + +
+ +
+ + +
+ +
+
+ + Switch branches/tags +
+ +
+
+ +
+
+ +
+
+ + + +
+
+ + +
+ +
Nothing to show
+
+ +
+
+
+ +
+ + Find file + + +
+ +
+ + + +
+ Fetching contributors… +
+ +
+ + Cannot retrieve contributors at this time +
+
+ +
+
+
+ +
+ Raw + Blame + History +
+ + + + + +
+ +
+ +
+ +
+ 1292 lines (1066 sloc) + + 113 KB +
+
+ + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
\documentclass[12pt] {article}
\usepackage{graphicx}
\usepackage{natbib}
\usepackage{fullpage}
\usepackage{amsmath}
\PassOptionsToPackage{hyphens}{url}\usepackage{hyperref}
\usepackage{pslatex}
\renewcommand{\baselinestretch}{2}
\usepackage{setspace}
\usepackage[nottoc]{tocbibind}
\usepackage{authblk}
\usepackage{rotating}
\usepackage{adjustbox}
+
+
\begin{document}
\title{Manual of Best Practices in Transparent Social Science Research}
+
\author{Garret Christensen\footnote{UC Berkeley, Berkeley Initiative for Transparency in the Social Sciences. Please send correspondence to garret@berkeley.edu, or find the version controlled history of this document on \href{https://github.com/garretchristensen/BestPracticesManual}{github}. I gratefully acknowledge helpful comments from Jennifer Sturdy, Alex Wais, and Courtney Soderberg, as well as participants in BITSS seminars and workshops. Funding for this manuscript was provided by an anonymous donor who play no role in writing or reviewing the manuscript, nor in the decision to publish.}}
%\affil[1]{UC Berkeley, Berkeley Initiative for Transparency in the Social Sciences}
\date{\today}
\maketitle
+
\begin{center}
Comments and suggestions are strongly encouraged. Please send correspondence to garret@berkeley.edu, or find the latest version of the manual on \href{https://github.com/garretchristensen/BestPracticesManual}{github}.
\end{center}
+
\newpage
\tableofcontents
+
\newpage
\section{Introduction}\label{introduction}
+
Scientific claims should be subject to scrutiny by other researchers and
the public at large. An important requirement for such scrutiny is that
researchers make their claims transparent in a way that other
researchers are able to use easily available resources to form a
complete understanding of the methods that were used by the original. In
the social sciences, especially given the personal computing and
Internet revolutions and the wide availability of data and processing
power, it is essential that data, code, and analyses be transparent.
+
This manual is intended to be a source mainly for empirical social
science researchers who desire to make their own research transparent
to, and reproducible by, others. The entire process of research, from
hypothesis generation to publication, is covered. Although norms differ
across disciplines, we attempt to bring a broad view of the empirical
social sciences to these recommendations, and hope that students and
researchers in any social science field may tailor these recommendations
to best fit their field.
+
The manual is laid out as follows: in section~\ref{ethical-research} we first discuss the motivation for this document: the
desire to do ethical research. A major component of ethical social
science research is treating research subjects appropriately. This is
mandated by federal law and overseen by Institutional Review Boards
(IRBs), and should be taken seriously by researchers. But just as
treating subjects fairly is ethical, we believe that transparent,
reproducible research is also a major part of ethical research.
+
In section~\ref{study-design} we discuss study design, including how to power studies
appropriately.
+
In section~\ref{registration} we discuss one of the major problems in non-transparent
research, specifically publication bias. We also discuss how this
problem can be resolved through the practice of registration.
Publication bias stems from the fact that published results are
overwhelmingly statistically significant. But without knowing how many
tests were run (the number of unpublished results), it is impossible to know whether these significant
results are meaningful, or whether they are the 5\% of tests that we
would expect to appear significant due to random sampling, even with no
true effect. By publicly registering all studies, we can have a better
idea of just how many tests have been run.
+
In section~\ref{rdof} we discuss researcher degrees of freedom and pre-analysis
plans; In addition to registering trials, researchers can also
specify their outcomes of interest and their exact methods of analysis
to bind their hands during the analysis phase by writing a Pre-Analysis
Plan (PAP). This is a relatively new idea in the social sciences, so
there is not yet a consensus on when a PAP should be required, what the
ideal level of detail is, and how much it should constrain a
researcher's hands in the actual analysis, but by pre-specifying analyses,
researchers can distinguish between confirmatory and exploratory
analysis. We do not necessarily place higher intrinsic value on one or
the other, but making the distinction clear is key for appropriate
interpretation.
+
In section~\ref{replication-and-reproducibility} we discuss workflow and materials sharing, with an eye on
making research replicable by others. Researchers should make their code
and data publicly available so that others may repeat and verify their
analysis. Making data available incentivizes researchers to make their
work accurate in the first place, and makes replication easier for
others, improving the scientific process, but also raises the concern of
differential privacy, since steps should be taken to prevent
identification of individuals in the data. We also discuss the issue of
reporting standards: a standardized list of things that authors should
report to help make their work reproducible.
+
Section~\ref{conclusion} concludes and presents a vision for moving forward.
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Ethical Research}\label{ethical-research}
+
Making one's research transparent and reproducible is a key component of ethical research. Not engaging in fraud is an obvious component of this.
+
\subsection{Fraud}\label{fraud}
+
While most of us are likely to presume that we ourselves would not
conduct outright fraud, fraud does indeed occur. From making up fake
data to creating bogus e-mail addresses so one could do one's own peer
review, the \href{http://www.retractionwatch.com}{Retraction Watch blog}
documents a distressingly large amount of deliberate fraud in research.
Although the blog tends to specialize in the life sciences, and there is significantly less money involved in the social sciences than in medical and pharmaceutical research, there is no reason to believe that social science researchers are inherently
more benevolent. Uri Simonsohn, Leif Nelson, and Joseph Simmons used statistical methods to detect fraud in the research of a pair of prominent social psychologists \citep{simonsohn2013just}.
+
Another source for information on fraud is part of the US Department of Health and Human Services
, the Office of Research Integrity (ORI), which works to promote research
integrity and document misconduct, especially when it involves federally
funded research. The \href{http://ori.hhs.gov/case_summary}{misconduct case summaries of the ORI}, and the stories of Diederik Stapel \citep{carey_noted_2011, bhattacharjee_diederik_2013} Hwang Woo-Suk \citep{cyranoski_cloning_2014}, Marc Hauser \citep{johnson_harvard_2012} and Michael LaCour \citep{broockman2015irreg} should be sobering warnings to us all.
+
\subsection{Unintentional Bias}\label{unintentional-bias}
+
Perhaps in addition to the obvious need to avoid deliberate fraud and
protect our human subjects is the need to avoid subconsciously biasing
our own results.
+
\cite{nosek_scientific_2012} summarize some of the evidence on this
subject, concluding that there are many circumstances common to academia
and the publishing paradigm that cause researchers to frequently use
motivated reasoning:
+
\begin{quote}
Because we have directional goals for success, we are likely to bring to
bear motivated reasoning to justify research decisions in the name of
accuracy, when they are actually in service of career advancement
(Fanelli, 2010a). Motivated reasoning is particularly influential when
the situation is complex, the available information is ambiguous, and
legitimate reasons can be generated for multiple courses of action
(Bersoff, 1999; Boiney, Kennedy, \& Nye, 1997; Kunda, 1990).
+
Motivated reasoning can occur without intention. We are more likely to
be convinced that our hypothesis is true, accepting uncritically when it
is confirmed and crutinizing heavily when it is not (Bastardi, Uhlmann,
\& Ross, 2011; Ditto \& Lopez, 1992; Lord, Ross, \& Lepper, 1979;
Pyszczynski \& Greenberg, 1987; Trope \& Bassok, 1982). With flexible
analysis options, we are more likely to find the one that produces a
more publishable pattern of results to be more reasonable and defensible
than others (Simmons et al., 2011; Wagenmakers, Wetzels, Borsboom, \&
van der Maas, 2011). Once we obtain an unexpected result, we are likely
to reconstruct our histories and perceive the outcome as something that
we could have, even did, anticipate all along---converting a discovery
into a confirmatory result (Fischoff, 1977; Fischoff \& Beyth, 1975).
And even if we resist those reasoning biases in the moment, after a few
months, we might simply forget the details, whether we had hypothesized
the moderator, had good justification for one set of exclusion criteria
compared with another, and had really thought that the one dependent
variable that showed a significant effect was the key outcome. Instead,
we might remember the gist of what the study was and what we found
(Reyna \& Brainerd, 1995). Forgetting the details provides an
opportunity for reimagining the study purpose and results to recall and
understand them in their best (i.e., most publishable) light. The reader
may, as we do, recall personal examples of such motivated
decisions---they are entirely ordinary products of human cognition.
\end{quote}
+
\subsection{Institutional Review
Boards}\label{institutional-review-boards}
+
In addition to fraud, a major ethical concern relates to our human
subjects.
+
\subsubsection{History}\label{history}
+
World history is rife with examples of atrocities conducted in the name
of research. Some of these have resulted in major changes in regulations
related to research.
+
\paragraph{Nuremberg}\label{nuremberg}
+
Nazi German doctors conducted horrible experiments on subjects during
World War II. The ``Doctor's Trial'' (USA v. Karl Brandt, et al.) tried
23 defendants, and the verdict included ten principles regarding voluntary consent, societal benefits from the research, minimizing risk, etc.
which although never entered as formal regulations in either Germany or
the USA, became widely accepted.
+
%\begin{enumerate}
%\def\labelenumi{\arabic{enumi}.}
%\item
% The voluntary consent of the human subject is absolutely essential.
+
% This means that the person involved should have legal capacity to give consent; should be so situated as to be able to exercise free power of choice, without the intervention of any element of force, fraud, deceit, duress, over-reaching, or other ulterior form of constraint or coercion; and should have sufficient knowledge and comprehension of the elements of the subject matter involved as to enable him to make an understanding and enlightened decision. This latter element requires that before the acceptance of an affirmative decision by the experimental subject there should be made known to him the nature, duration, and purpose of the experiment; the method and means by which it is to be conducted; all inconveniences and hazards reasonably to be expected; and the effects upon his health or person which may possibly come from his participation in the experiment.
+
% The duty and responsibility for ascertaining the quality of the consent rests upon each individual who initiates, directs or engages in the experiment. It is a personal duty and responsibility which may not be delegated to another with impunity.
%\item
% The experiment should be such as to yield fruitful results for the good of society, unprocurable by other methods or means of study, and not random and unnecessary in nature.
%\item
% The experiment should be so designed and based on the results of animal experimentation and a knowledge of the natural history of the disease or other problem under study that the anticipated results will justify the performance of the experiment.
%\item
% The experiment should be so conducted as to avoid all unnecessary
%physical and mental suffering and injury.
%\item
% No experiment should be conducted where there is an a priori reason to believe that death or disabling injury will occur; except, perhaps, in those experiments where the experimental physicians also serve as subjects.
%\item
% The degree of risk to be taken should never exceed that determined by the humanitarian importance of the problem to be solved by the experiment.
%\item
% Proper preparations should be made and adequate facilities provided to protect the experimental subject against even remote possibilities of injury, disability, or death.
%\item
% The experiment should be conducted only by scientifically qualified persons. The highest degree of skill and care should be required through all stages of the experiment of those who conduct or engage in the experiment.
%\item
% During the course of the experiment the human subject should be at liberty to bring the experiment to an end if he has reached the physical or mental state where continuation of the experiment seems to him to be impossible.
%\item
% During the course of the experiment the scientist in charge must be prepared to terminate the experiment at any stage, if he has probably cause to believe, in the exercise of the good faith, superior skill and careful judgment required of him that a continuation of the experiment is likely to result in injury, disability, or death to the experimental subject.
%\end{enumerate}
+
\paragraph{Tuskegee and US
codification}\label{tuskegee-and-us-codification}
+
In 1972 whistleblower Peter Buxton revealed to the Associated Press that
the US Public Health Service was conducting a 40-year experiment on poor
Alabama sharecroppers in which it did not treat those who had syphilis
for the disease despite the discovery and verification of penicillin as
an effective treatment, and prevented sufferers from obtaining
treatment elsewhere. As a result, the National Commission for the
Protection of Human Subjects of Biomedical and Behavioral Research was
formed by law in 1974, and released the Belmont Report in 1979. \href{http://www.hhs.gov/ohrp/humansubjects/guidance/belmont.html}{The
Belmont Report} contains three basic ethical principles, and three applications:
+
\begin{itemize}
\item
Ethical Principles
+
\begin{itemize}
\item
Respect for Persons: ``Respect for persons incorporates at least two
ethical convictions: first, that individuals should be treated as
autonomous agents, and second, that persons with diminished autonomy
are entitled to protection.''
\item
Beneficence: ``Two general rules have been formulated as
complementary expressions of beneficent actions in this sense:
\textbf{(1)} do not harm and \textbf{(2)} maximize possible benefits
and minimize possible harms.''
\item
Justice: ``An injustice occurs when some benefit to which a person
is entitled is denied without good reason or when some burden is
imposed unduly. Another way of conceiving the principle of justice
is that equals ought to be treated equally.''
\end{itemize}
\item
Applications
+
\begin{itemize}
\item
Informed Consent: ``Respect for persons requires that subjects, to
the degree that they are capable, be given the opportunity to choose
what shall or shall not happen to them. This opportunity is provided
when adequate standards for informed consent are satisfied.''
\item
Assessment of Risks and Benefits: ``It is commonly said that
benefits and risks must be `balanced' and shown to be `in a
favorable ratio.' The metaphorical character of these terms draws
attention to the difficulty of making precise judgments. Only on
rare occasions will quantitative techniques be available for the
scrutiny of research protocols. However, the idea of systematic,
nonarbitrary analysis of risks and benefits should be emulated
insofar as possible."
\item
Selection of Subjects: ``Individual justice in the selection of
subjects would require that researchers exhibit fairness: thus, they
should not offer potentially beneficial research only to some
patients who are in their favor or select only `undesirable' persons
for risky research. Social justice requires that distinction be
drawn between classes of subjects that ought, and ought not, to
participate in any particular kind of research, based on the ability
of members of that class to bear burdens and on the appropriateness
of placing further burdens on already burdened persons."
\end{itemize}
\end{itemize}
+
In 1981 the Department of Health and Human Services and the Food and
Drug Administration adopted regulations in line with the Belmont report,
and 15 federal agencies adopted these regulations (45 CFR part 46) as
the ``\href{http://www.hhs.gov/ohrp/index.html}{Common Rule}'' in 1991.
+
In practice, this means that researchers who receive funding from the US
government, or who work at institutions that receive federal funding should have their research approved by an Institutional Review Board (IRB). IRB are a decentralized approval
body set up by each research organization itself, consisting of at least
five members, a mix of men and women, scientists and non-scientists, and
at least one member not affiliated with the institution. Since IRBs and
the approval process are decentralized, the exact process varies from
institution to institution, but one example can be seen at
\url{http://cphs.berkeley.edu}.
+
When conducting research internationally, researchers should give their
human subjects the same protections as those inside the US. Laws in
developing countries may not be as well-defined or enforced, but
researchers should still register with their US institution's IRB, and
obtain approval from the host country government. A list of laws and
regulations that cover research in 107 foreign countries is available
from the \href{http://www.hhs.gov/ohrp/international/intlcompilation/2014intlcomp.pdf.pdf}
{Office for Human Research Protections}.
+
Another key resource for researchers and research conducted outside the
US is the \href{http://www.wma.net/en/30publications/10policies/b3/index.html}{Declaration of Helsinki} by the World Medical Association
(WMA). Originally adopted by the WMA in 1964, the document has significantly
influenced the laws and regulations adopted to govern research
worldwide.
+
Lest one think that ethical concerns are limited to monsters of bygone
eras, we refer readers to a dilemma caused by a Montana state election experiment by
researchers from \href{http://www.washingtonpost.com/blogs/monkey-cage/wp/2014/11/03/ethics-and-research-in-comparative-politics/}{Stanford and Dartmouth} in 2014, who sent 100,000 people official-looking election flyers bearing the state seal weeks before the election.
+
\subsubsection{Training}\label{training}
+
A large number of universities participate in the \href{https://www.citiprogram.org/}{Collaborative Institutional Training Initiative} at the University of Miami (CITI). Completing their course on Human
Subjects Research is often a requirement of being included on a research
proposal. For anyone at an institution not affiliated with CITI, the NIH maintains an \href{https://phrp.nihtraining.com/users/login.php}{online training course} that is free and open to the public.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Study Design}\label{study-design}
There are many issues involving study design that are somewhat related to transparent research. For randomized trials, we refer readers to two excellent resources, \cite{randomizationtoolkit} and \cite{glennerster_running_2013}, which cover many aspects of how to design and implement an excellent field trial. For studies that are not necessarily randomized trials, we recommend \cite{gertler_impact_2011} or \cite{angrist2008mostly} for those with more statistics training. Here we briefly discuss one aspect of study design especially relevant for reproducible research: determining sample size through power calculations.
\subsection{Power Analysis}\label{Conducting Power Analysis}
%By CS
Given that researchers are working within a null hypothesis testing framework, the ‘power’
of a study, the probability of rejecting the null hypothesis when it is false, is extremely
important. Though 80\% power is held as a lower bound for acceptable power in many disciplines
such as medical research, the actual power of studies can be much lower. For example, research
by \cite{button_power_2013} found that the median power in neuroscience was 21\%. This means that
if a study investigating a true effect were run 100 times, 79 studies would fail to reach significance,
meaning that the chance of a false negative is extremely high.
+
Though false negatives are perhaps the most often discussed issue with low powered studies, they are not the only issue. A second issue is that low powered studies actually decrease the likelihood of a true positive \citep{button_power_2013}.
This means that when a low powered studies does find an effect, the lower the likelihood that this effect is true in
the population; it is relatively more likely that it is a false positive. A third issue with low powered studies relates to effect
sizes. Small, low powered studies that reach statistical significance will over-estimate the true effect size in the population \citep{button_power_2013}. These inflated effect size estimates will make it difficult for others to properly power future
studies, and the inaccuracy of the estimate may also be problematic for making decisions based on scientific results.
An obvious way to help mitigate the problems of underpowered studies is to run studies with higher power. However, this
process is not always that straightforward. As previously mentioned, effect sizes from published studies are often inflated,
and so may not provide the most accurate estimate. Meta-analyses can provide better information, but they also often suffer
from publication bias and thus inflated effect sizes. Additionally, effect sizes may be highly heterogeneous between studies,
even studies with the same materials and methodologies \citep{many_labs}. Thus, using a single point estimate of an effect size
from published literature may lead to inaccurate power calculations.
+
To combat these problems, alternatives to the standard power analysis have been suggested. For example, \cite{perugini2014safeguard}
have suggested a technique called `safeguard power' which takes into account the uncertainty surrounding effect size estimates
when conducting power analyses. Specifically, they suggest basing power calculations off of the effect size corresponding to the lower bound of a 60\% confidence interval around the point estimate from published literature. Another approach by \cite{mcshane_2014} takes into account the between study variation in effect sizes when conducting power analyses to give a more conservative estimate of the number of participants needed to reach a given level of statistical power. Yet another approach, which is also feasible when an effect size estimate from the previous literature is not available, is to determine the smallest effect size you wish to be able to detect, and power your study to find this effect size \citep{bloom1995minimum}.
+
%GSC
Additionally, researchers may be able to help one another as well as decrease false-positives by publishing or clearly making available their power calculations. If a reader of a paper saw a clearly stated ``this trial was powered to detect an effect size of X,'' this would help to put the study into the appropriate statistical context. The parametric assumptions involved (such as the intra-cluster correlation) could also be publicly posted in study protocols to help other researchers learn reasonable assumptions for their own studies.
\subsection {Practical Considerations}
The impact of most of these calculations will in many cases mean a larger sample size than some disciplines are used to working with. There are several ways to mitigate the impact of this.
+
In psychology, a few of these possible solutions are presented in \cite{openmaximizing}. Researchers can join crowd-sourced projects where multiple labs share protocols and produce the same experiment, as was done with 13 psychological effects and 36 samples and settings in \cite{many_labs}. The \href{https://osf.io/wfc6u/}{Collaborative Replications and Education Project} (CREP) tracks findings that can be relatively easily replicated in a teaching setting.
+
Economists have also discusses how project budgets should play into study design, by maximizing power subject to a budget constraint \cite{randomizationtoolkit}. When data collection is the main cost, the proportion of treatment and control should each be one half, but if treatment is expensive, power can be maximized subject to the project budget using the rule: ``the ratio of subjects in the treatment group to those in the comparison should be proportional to the inverse of the square root of their cots.'' Another important contribution to this are of power maximization subject to a budget constraint is that power depends on the number of rounds of surveying, and also the autocorrelation of measurements over time. \cite{mckenzie2012caseformoreT} shows that baseline data is of most power when autocorrelation is high, and repeated measurements post-treatment of most power when autocorrelation is low.
+
+
+
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Registration}\label{registration}
+
One of the problems brought into focus recently is publication bias.
Publication bias is the selective publication of only significant
results. Thankfully, there are tools available for researchers to combat
these problems.
+
\subsection{Publication Bias}\label{publication-bias}
+
One of the primary drivers of the recent move towards transparency is
increased awareness of publication bias. Numerous papers use collections
of published papers to show that the proportion of significant results
are extremely unlikely to come from any true population distribution
\citep{delong_are_1992, gerber_testing_2001, ioannidis_why_2005}. By examining the publication rates of null results and
significant results from a large set of NSF-funded studies, \cite{franco_publication_2014} show that the selective publication of only
significant results may stem from the fact that social science
researchers largely fail to write up and submit results from studies
resulting in null findings, citing lack of interest or fear of
rejection. This idea of rejecting, or not even submitting for review, papers with null-results, is commonly referred to as the ``file drawer problem'' \citep{rosenthal1979file}. In fact, the percentage of null findings published in
journals appears to have been decreasing over time, across all
disciplines \citep{fanelli_negative_2012}. It seems unlikely that this would
be an accurate reflection of the state of the universe, unless the hypotheses that scientists are testing are systematically changing over time. If journals only
publish statistically significant results, we have no idea how many of
those significant results are evidence of real effects, and which are
the 5\% of random draws that we should expect to show a significant
result with a true zero effect. One way to combat this problem is to
require registration of all studies undertaken. Ideally we then search
the registry for studies of X on Y. If numerous studies show an
effect, we have confidence the effect is real. If 5\% of studies show a
significant effect, we give these outlier studies less credence.
+
\subsection{Trial Registration}\label{trial-registration}
+
A basic definition of registration is to publicly declare \emph{all}
research that one plans on conducting. Ideally this is done in a public
registry designed to accept registrations in the given research discipline,
and ideally the registration takes place before data collection begins.
+
Registration of randomized trials has achieved wide adoption in medicine, but is still
relatively new to the social sciences. After congress passed a law in
1997 requiring the creation of a registry for FDA-regulated trials, and
the NIH created clinicaltrials.gov in 2000, The International Committee
of Medical Journal Editors (ICMJE), a collection of editors of top
medical journals, instituted a policy of publishing only registered
trials in 2005 \citep{DeAngelis2004}, and the policy has spread to
other journals and been generally accepted by researchers \citep{laine_clinical_2007}. Several other countries have their own national trial registries, and the World Health Organization created the \href{http://www.who.int/ictrp/about/en/}{International Clinical Trials Registry Platform (ICTRP)} in 2007 to automatically collect all this information in one place.
+
A profound example of the benefit of trial registries is detailed in
\cite{turner_selective_2008}, which details the publication rates of studies
related to FDA-approved antidepressants. (See also \cite{ioannidis_effectiveness_2008}.)
The outcome is perhaps what the most hardened cynic would expect:
essentially all the trials with positive outcomes were published, a
50/50 mix of questionable-outcome studies were published, and a majority of the
negative-outcome studies were unpublished a minimum of four years after the
study was completed. The figure below shows the drastically different
rates of publication, and a large amount of publication bias.
\begin{center}
\includegraphics{TurnerFigure1.PNG}
+
Panel A of Figure 1 from \cite{turner_selective_2008}
\end{center}
Of course for this sort of exercise to be possible, unless a reader
merely assumes that a registered trial without an associated published
paper produced a null result (as in \cite{rosenthal1979file}), it requires that the registration site
itself obtain outcomes of trials. \href{http://www.clinicaltrials.gov}{ClinicalTrials.gov} is the only
publicly available trial registry that requires such reporting of
results, and only for certain FDA trials.\footnote{\cite{resultscompliance} finds that compliance with results reporting even among those required was fairly low (22\%). HHS and NIH took steps in November 2014 to expand the amount of results reporting required. See \url{http://www.nih.gov/news/health/nov2014/od-19.htm}} \cite{hartung_reporting_2014} raises
concerns about discrepancies between reporting of outcomes in published
papers and in the \href{http://www.clinicaltrials.gov}{ClinicalTrials.gov} database; as many as 20\% of
studies had discrepancies in primary outcomes and as many as 33\% had
discrepancies in reporting of adverse events, so there is definitely room for improvement.
+
Even with dramatic growth in medical trial registration, problems
remain. Not all journals have adopted the ICMJE policy, and complete
enforcement is elusive. \cite{mathieu_s_comparison_2009} looked at trials related
to three medical conditions and found that only 46\% of studies were
registered before the end of the trial with primary outcomes clearly
specified. Even among those adequately registered, 31\% showed some
discrepancies between registered and published outcomes, with bias in
favor of statistically significant definitions.
+
Almost all registration efforts have thus far been limited to randomized
control trials, as opposed to observational data. We believe
that registering all types of analysis should be accepted and encouraged, though there are definitely concerns to registering observational work---not least of which is the inability to verify that registration preceded analysis. See \cite{dal-re_making_2014} for a recent discussion of the pros and cons.
\subsection{Social Science Registries}\label{social-science-registries}
+
Registries in the social sciences are newer but are growing ever more
popular. A brief overview of the major trial registries is shown in Table~\ref{RegTable}. The Abdul Latif Jameel Poverty Action Lab began hosting a
\href{http://www.povertyactionlab.org/hypothesis-registry}{hypothesis registry}
in 2009, which was superseded by the American Economic Association's launch of its own \href{http://socialscienceregistry.org}{registry} for randomized trials in May 2013, which had accumulated 260 studies in 59 countries by
October 2014. The International Initiative for Impact Evaluation (3ie)
launched its own registry for evaluations of development programs, the
\href{http://ridie.3ieimpact.org}{Registry for International Development Impact Evaluations} (RIDIE) in September 2013, which had
approximately 30 evaluations registered in its first year.
+
+
+
\begin{sidewaystable}[]
\centering
\caption{Registries in Medicine and the Social Sciences}
\label{RegTable}
\begin{adjustbox}{max width=\textwidth}
\begin{tabular}{llllll}
{\bf Registry} & {\bf Sponsor} & {\bf Year Started} & {\bf Study Design} & {\bf Field} & {\bf Studies Registered} \\
ClinicalTrials.gov & National Institutes of Health & 2000 & RCT & Medicine & 196,000+ \\
International Clinical Trials Registry Platform & World Health Organization & 2007 & RCT & Medicine & 284,000+ \\
AEA RCT Registry (SocialScienceRegistry.org) & American Economic Association & 2013 & RCT & Economics & 430+ \\
Registry for International Development Impact Evaluations & 3ie & 2013 & Any & Developing country impact evaluation & 65+ \\
EGAP Design Registry & Experiments in Governance and Politics & 2012 & Any & Political Science & 185+ \\
Open Science Framework & Center for Open Science & 2013 & Any & Any & 700+
\end{tabular}
\end{adjustbox}
\end{sidewaystable}
+
+
In political science, \href{http://e-gap.org/design-registration}{EGAP: Experiments in Governance and Politics} has
created a registry as ``an unsupervised stopgap function to store
designs until the creation of a general registry for social science
research. The EGAP registry focuses on designs for experiments and
observational studies in governance and politics.'' EGAP's registry had 93
designs registered as of October 2014.\footnote{Earlier less-widely
adopted attempts to create registries in political science are the
Political Science Registered Studies Dataverse (PSRSD,
\href{../customXml/item1.xml}{http://spia.uga.edu/faculty\_pages/monogan/registration.php})
and the PAP Registry of the Experimental Research section of the
American Political Science Association
(\href{numbering.xml}{http://ps-experiments.ucr.edu/browser}).}
+
Another location for registrations is the \href{http://osf.io}{Open Science Framework} (OSF), created by the \href{http://centerforopenscience.org/}{Center for Open Science}. The OSF serves as a broad research management tool that encourages and facilitates transparency (see \cite{nosek_scientific_2012}.) Registrations are simply unalterable snapshots of research frozen in time, with a persistent URL and timestamp. Researchers can upload their data, code, hypotheses, etc. to the OSF, register it, and then share the resulting URL as proof of registration. OSF registrations can be relatively free-form, but templates exist to conform to standards in different disciplines. Psychology registrations are presently the most numerous on the OSF \footnote{See \url{https://osf.io/explore/activity/\#newPublicRegistrations}.}
+
\subsection{Meta-Analysis Research}
Another method of detecting and dealing with publication bias is to conduct meta-analysis. This method of research collects all published findings on a given topic, analyzes the results collectively, and can detect, and attempt to adjust for, publication bias in the literature. A handful of organizations specialize in producing these systematic reviews, including the \href{http://www.cochrane.org}{Cochrane Collaboration} for health studies and the \href{http://www.campbellcollaboration.org/}{Campbell Collaboration} for crime \& justice, education, international development, and social welfare research, and the \href{http://www.3ie.org}{International Initiative for Impact Evalution (3ie)} for social and economic interventions in low- and middle- income countries. The US government supports this type of analysis: the Department of Education's Institute of Education Sciences maintains the \href{http://ies.ed.gov/ncee/wwc/}{What Works Clearinghouse}, and the Department of Labor maintains the \href{http://clear.dol.gov/}{Clearinghouse for Labor and Evaluation Research (CLEAR)}, which serve to collect and grade the evidentiary value of research on education and labor, respectively. (The synthesis methods in the government clearinghouses is not quite as formally statistical in nature as the previously mentioned Collaborations.)
+
Although quite common in medical research, the tool is not widely used in some parts of the social sciences. But even in economics, where many graduate students are unfamiliar with the technqiue, important papers exist that have quantitatively synthesized bodies of literature. The unemployment effects of the minimum wage were meta-analyzed in \cite{card1995time}, and the returns to education in \cite{ashenfelter1999review}. A meta-analysis of 87 meta-analyses in economics shows that publication bias is widespread, but not universal. A helpful resource, which includes meta-analysis datasets for economics researchers interested in conducting a meta-analysis is available \href{http://www.deakin.edu.au/business/economics/research/meta-analysis}{here}. Also see \cite{stanley2005beyond}, which helpfully describes the tools of meta-analysis, and is part of a special issue of \textit{The Journal of Economic Surveys} dedicated to meta-analysis. Sol Hsiang, lead author of a prominent meta-analysis of 60 studies measuring the effect of climate on human conflict \citep{hsiang2013climate}, has also developed the \href{http://dmas.berkeley.edu}{Distributed Meta-Analysis System}, an online tool to crowdsource and simplify meta-analysis.
+
Prominent examples in psychology include meta-analyses of work on the Big 5 Personality Test and job performance \citep{barrick1991big5} and predictive validity of the Implicit Association Test \citep{greenwald2009understanding}. \cite{simonsohn2014p} also developed a meta-analysis tool that researchers can use to compare the uniform distribution of p-values under the null to the left-skewed distributions observed in research that has been gamed or selectively reported.
+
%Whoops! This was a duplicate.
%\subsection{Design-Based Review and Publication}
%Another proposed method of reducing publication bias is by altering the time at which peer review takes place. The standard method is for peers to review a completed paper, with results. Even with editorial statements in place encouraging reviewers to not be prejudiced against null results, the odds are stacked against null-result papers. The new method, refered to as `registered reports'\footnote{We consider the name `registered reports' unfortunate, since it can be easily confused with trial registrations, and we prefer the terms `design-based review' or `design-based publication.'} has reviewers evaluate a paper before the data is collected and analyzed. That is, a researcher submits a detailed protocol and analysis plan, and reviewers judge based on the importance of the question being asked and the appropriateness of the methods being used. The protocol is then rejected or given in-principle acceptance. If accepted in-principle, the author conducts the experiment and the analysis as planned, and submits the final result. If the author followed through on his promise, the paper is published, regardless of the statistical significance of the article.
+
%This method of review and publication is championed by Chris Chambers at Cardiff University. Nearly 20 journals have adopted this method to one degree or another. More information can be found at the project's \href{https://osf.io/8mpji/wiki/home/}{Open Science Framework page}. It is too early to do a sophisticated analysis of the effects on publication, but we believe it to be a promising development.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Researcher Degrees of
Freedom}\label{rdof}
+
Though registration helps solve the problem of publication bias, it does
not solve the problem of fishing for statistical significance within a
given study. This problem with research is known as data mining: the manipulation or repeated searching through statistical or regression
models unknowingly (or deliberately) until significance is obtained.
\cite{simmons_false-positive_2011} refer to this as
``researcher degrees of freedom,'' and it has also been referred to as
``fishing,'' ``p-hacking,'' or ``specification searching'' \citep{humphreys_fishing_2013}. The problem has many names because it can take many shapes. Using flexibility around when to stop
collecting data, excluding certain observations, combining and comparing
certain conditions, including certain control variables, and combining
or transforming certain measures, they ``prove'' that listening to the
Beatles' song ``When I'm Sixty-Four'' made listeners a year and a half
younger. The extent and ease of this ``fishing'' is also described in
\cite{humphreys_fishing_2013} who use simulations to show that multiplicity of outcome measures, multiplicity of heterogeneous treatment effects (sub-group analyses), and multiplicity of cut-points for turning a continuous outcome variable into a binary outcome, can all be used to virtually guarantee a false positive, even with large sample sizes. They also find that selective adding of covariates can produce false positives with small samples, though they do find little room to produce false positives through arbitrary selection of model for binary outcomes (linear, logit, or probit) regardless of sample size. \cite{gelman_garden_2013} agree that
``{[}a{]} dataset can be analyzed in so many different ways (with the
choices being not just what statistical test to perform but also
decisions on what data to exclude or exclude {[}sic{]}, what measures to
study, what interactions to consider, etc.), that very little
information is provided by the statement that a study came up with a
\emph{p}\textless{}.05 result.'' However, they also conclude that:
+
\begin{quote}
the term ``fishing" was unfortunate, in that it invokes an image of a
researcher trying out comparison after comparison, throwing the line
into the lake repeatedly until a fish is snagged. We have no reason to
think that researchers regularly do that. We think the real story is
that researchers can perform a reasonable analysis given their
assumptions and their data, but had the data turned out differently,
they could have done other analyses that were just as reasonable in
those circumstances.
+
We regret the spread of the terms ``fishing" and ``p-hacking'' (and even
``researcher degrees of freedom'') for two reasons: first, because when
such terms are used to describe a study, there is the misleading
implication that researchers were consciously trying out many different
analyses on a single data set; and, second, because it can lead
researchers who know they did not tryout many different analyses to
mistakenly think they are not so strongly subject to problems of
researcher degrees of freedom."
\end{quote}
+
In other words, the problem may be even worse than you think. What can be
done to solve it? We believe part of the answer lies in detailed
pre-analysis plans, described below.
+
\subsection{Pre-Analysis
Plans}\label{pre-analysis-plans}
+
While registration of studies can help to reduce publication bias or the file drawer problem,
A pre-analysis plan (PAP), a detailed outline of the analyses that will be conducted in a study, can be used to reduce researcher degrees of freedom. Registration is now the norm in medicine for randomized trials, and these often include (or link to) prospective statistical analysis plans as part of the project protocol. Official guidance from the US Food and Drug Administration's Center for Drug Evaluation and Research (CDER) from 1998 describes what should be included in a statistical analysis plan and discusses eight items related to data analysis that should be considered: pre-specification of the analysis; analysis sets; missing values and outliers; data transformation; estimation, confidence intervals, and hypothesis testings; adjustment of significance and confidence levels; subgroups, interactions, and covariates; and integrity of data and computer software validity \citep{food1998guidance}. %Sometimes this is because the medical researcher intends to do very little, if any, structural or formal modeling. But even if a researcher intends only to compare unadjusted means and bootstrap for standard errors, this should be explicitly stated. In the social sciences, this simple comparison is often not the end goal of a randomized trial, so there may be even more value in pre-specification of intended analyses.
This is an excellent start, and in the section below we discuss adapting these ideas for a pre-analysis plan in the social sciences.
+
A pre-analysis plan (PAP) contains a specification of the outcomes of the study (sometimes referred to as endpoints in the medical literature), as well as a specification of the methods that will be used to analyze the outcomes. By describing the
method(s) of analysis ahead of time, and to some degree tying the hands
of the researcher, we reduce the ability to data mine. Though one
example of this exists in economics from 2001 \citep{neumark_employment_2001}, the idea
is still quite new to the social sciences. The level of detail varies
widely, and the research community is still constructing norms for
incorporating these documents into final analyses and papers.
+
\paragraph{What to Include}
Suggestions have been made for the detailed contents of these documents.
\cite{glennerster_running_2013} suggest including the following:
+
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
the main outcome measures,
\item
which outcome measures are primary and which are secondary,
\item
the precise composition of any families that will be used for mean
effects analysis,
\item
the subgroups that will be analyzed,
\item
the direction of expected impact if we want to use a one-sided test,
and
\item
the primary specification to be used for the analysis.
\end{enumerate}
David McKenzie of the World Bank Research Group proposed a list of ten
items that should be included in a PAP, reproduced below from the \href{http://blogs.worldbank.org/impactevaluations/a-pre-analysis-plan-checklist}{World Bank Development Impact Blog}.
+
\begin{enumerate}
\item
Description of the sample to be used in the study
\item
Key data sources
\item
Hypotheses to be tested throughout the causal chain
\item
Specify how variables will be constructed
\item
Specify the treatment effect equation to be estimated
\item
What is the plan for how to deal with multiple outcomes and multiple
hypothesis testing?
\item
Procedures to be used for addressing survey attrition
\item
How will the study deal with outcomes with limited variation?
\item
If you are going to be testing a model, include the model
\item
Remember to archive it
\end{enumerate}
+
In their article on researcher degrees of freedom, Simmons, Nelson, and
Simonsohn (2011) suggest the following requirements for authors:
+
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
Authors must decide the rule for terminating data collection before
data collection begins and report this rule in the article.
\item
Authors must collect at least 20 observations per cell or else provide
a compelling cost-of-data-collection justification.
\item
Authors must list all variables collected in a study.
\item
Authors must report all experimental conditions, including failed
manipulations.
\item
If observations are eliminated, authors must also report what the
statistical results are if those observations are included.
\item
If an analysis includes a covariate, authors must report the
statistical results of the analysis without the covariate.
\end{enumerate}
+
\paragraph{Expecting the Unexpected}
\cite{glennerster_running_2013} also mention the ``tension between
the benefits of the credibility that comes from tying ones hands versus
the benefits of flexibility to respond to unforeseen events and
results.'' Writing a PAP can lend extra credibility to research by making it of a confirmatory nature as opposed to an exploratory nature.
Both types of research are absolutely valuable, but knowing the distinction is important. If some sort of restriction on the data, be it a specific functional form, exclusion of outliers, or an interaction term (subgroup analysis) that turns a null effect for the population into a significant effect for some subgroup, is specified ahead of time based on theory or previous research, this can be considered confirmatory research.
+
Some would say this is of more value than the exploratory research approach of simply running 20 sub-group analyses and finding that one or two are significant. This may be an estimate of a true effect, but should be labeled as exploratory, and future researchers could attempt to confirm this finding by addressing the question of the sub-group specifically. The potential downside to pre-stating hypotheses and analysis plans is that no matter how carefully researchers plan ahead, something truly unexpected can occur. (An example discussed at a recent conference was subjects showing up for an experiment stoned. One example from a field experiment involved fatalities from a lightning strike at a school that was part of a competitive girls scholarship program \citep{kremer2009incentives}. ) This is why, even though we may use the phrase ``bind our hands,'' we advocate that researchers not be punished for conducting research outside the analysis plan. We simply recommend that researchers clearly delineate which analysis was included in the analysis plan, and which was not, so that readers can know what is confirmatory and what is exploratory.
+
+
\paragraph{When to Write}
There is some question as to when one should write one's pre-analysis plan. ``Before you begin to analyze your data'' seems like the obvious answer, but this should be precisely defined. One could write the PAP before any baseline survey takes place, after any intervention but before endline, or after endline but before analysis has begun. \cite{glennerster_running_2013} and \cite{OlkenPAP} have an informative discussion of the relative values of PAP timing. If one writes the PAP before the baseline, this is in some sense the purest, most free from accusations of p-hacking, but one could also miss valuable information. For example, suppose in baseline one learns that the intended outcome question is phrased poorly and elicits high rates of non-response, or that there is very little variation in the answers to a survey question. If the PAP was written after baseline, one could have accounted for this, but at the same time, researchers would also be free to change the scope of their analysis--for example, in the baseline survey of a field experiment designed to increase wages revealed that few of the subjects worked outside the home, the researcher could change the focus of the analysis. This is not necessarily wrong, but it does change the nature of the analysis somewhat.
+
PAPs could also be written after endline data has been collected but before the investigators have begun to analyze the data. Some have suggested that one could even look at baseline data from the control group only before writing the PAP. We find this problematic, however, since a researcher could learn that the control group had a particularly low or high value of a certain outcome variable, and then choose to include or not include this variable in the analysis as a result. The original research design could have been intended to analyze the increase in secondary school attendance, but looking at the control group, the researcher sees that the control group had a very high rate of attendance, making a significant difference between control and treatment (the treatment effect) unlikely. Learning this after the experiment has concluded and searching for things that might be easily different between treatment and control is more exploratory than confirmatory. An alternative proposal discussed in \cite{OlkenPAP} is to remove the treatment status variable from the dataset before looking at the data, which seems to alleviate some of the concerns. However, one could still search for sub-group analyses at this stage. If you parse the outcome data by gender, and males and females have a similar distribution, to find a differential treatment effect by gender would seem unlikely. If male and female had wildly different outcomes, it would seem like a significant interaction is more likely. This is more exploratory than confirmatory.
+
+
\subsubsection{Examples}\label{examples}
Examples of pre-analysis plans in the social sciences are relatively rare, but several examples of good published papers resulting from studies with PAP exist. Several of the items below come from the \href{http://www.povertyactionlab.org/Hypothesis-Registry}{J-PAL Hypothesis Registry}; we highlight those that have publicly available final papers.
+
+
\begin{itemize}
\item
\cite{casey_reshaping_2012} includes evidence from a large-scale field experiment on community driven development projects in Sierra Leone. The analysis finds no significant benefits. Given the somewhat vague nature of the development projects that resulted from the funding, and the wide variety of potential outcomes, finding signficant results would have been relatively easy. In fact, the paper includes an example of how, if they had the latitude to define outcomes without a pre-analysis plan, the authors could have reported either large and significantly positive or negative outcomes, depending on their preferences. The paper also includes a discussion of the history and purpose of pre-analysis plans. The \href{http://emiguel.econ.berkeley.edu/assets/miguel_research/8/_Appendix__Reshaping_Institutions_-_Evidence__on__Aid__Impacts__Using__a__Pre___Analysis__Plan.pdf}{online appendix}
contains the PAP.
+
\item
Oregon expanded its Medicare enrollment through a random lottery in 2008, providing researchers with an ideal avenue to evaluate the benefits of enrollment. \cite{finkelstein_oregon_2012, doi:10.1056/NEJMsa1212321} and \cite{Taubman17012014} show that recipients did not improve in physical health measurements, but were more likely to have insurance, had better self-reported health outcomes, utilized emergency rooms more, and had better detection and management of diabetes. Pre-analsyis plans from the project are available at the \href{http://www.nber.org/oregon/documents.html}{National Bureau of Economics}' site devoted to the project. (See, for example, \cite{taubman_oregon_2013, baicker_katherine_oregon_2014}.)
+
+
\item
The shoe company Toms funded a rigorous evaluation of its in-kind shoe donation program. Researchers wrote a pre-analysis plan before conducting their research, and found no evidence that shoe donations displace local purchasing of shoes. See \cite{wydick_-kind_2014, katz_elizabeth_pre-analysis_2013}. The PAP is available in the \href{http://www.povertyactionlab.org/doc/pre-analysis-planwydick2-12-13pdf}{JPAL Hypothesis Registry}. This is one of many projects that has benefited from a pre-analysis plan because of the involvement of a group with a vested interest, such as a government or corporation. Even researchers skeptical of the need for PAPs in general admit the benefit to publicly pre-stating analysis plans when someone involved has such a clear incentive for results to go a certain direction.
+
\item
Researchers from UC San Diego and the World Bank evaluated job training programs run by the Turkish government and found only insignificant improvements and a strongly negative return on investment. See \cite{almeida_impact_2012, vocationalTurkey}. The PAP is available in the \href{http://www.povertyactionlab.org/Hypothesis-Registry}{J-PAL Hypothesis Registry} as well as the \href{http://blogs.worldbank.org/impactevaluations/files/impactevaluations/iskurie_analysisplan_v4a.pdf}{World Bank Development Impact Blog}.
+
+
+
\item
Teams led by Ben Olken have evaluated multiple randomized interventions in Indonesia. The \href{http://www.povertyactionlab.org/evaluation/project-generasi-conditional-community-block-grants-indonesia}{Generasi program} linked community block grants to performance. The PAP are \cite{olken_generasi_2009, olken_generasi_2010} and are available in the \href{http://www.povertyactionlab.org/Hypothesis-Registry}{J-PAL Hypothesis Registry}. The researchers found health improvement, but no education improvement \citep{olken_indonesias_2010, olken_should_2014}.
+
\item
Another project in Indonesia used a field experiment to evaluate different means of poverty targeting for cash transfer programs: proxy-means testing, community-based targeting, and a hybrid of the two. Results show that the proxy-means testing outperformed the other methods by 10\%, but that community members were far more satisfied with the community method. The PAP and final paper are available as \cite{olken_targeting_2009} and \cite{alatas_targeting_2012}
+
\item
An example from pyschology is a pre-registered replication of an implicit association test. Existing research showed evidence of stronger racial preferences among fertile women. \cite{NosekPreRegistered} failed to reproduce this effect in four tries, suggesting the association is weaker than originally found. The resulting manuscript, as well as the time-stamped registration of the analysis plan, can be found on the Open Science Framework at \url{https://osf.io/g3sca/}.
\end{itemize}
Additionally, Alejandro Ganimian developed \href{http://scholar.harvard.edu/files/alejandro_ganimian/files/pre-analysis_plan_template_0.pdf}{a template for pre-analysis plans} that instructors may find useful when teaching tranpsarency methods, or researcher themselves may find useful when developing their own pre-analysis plan.
+
\subsubsection{Observational Pre-Analysis Plans}
As with registration, part of the concern with pre-analysis plans is whether the ``pre'' aspect is verifiable. How can we be sure that a researcher didn't look at observational data, run a few hypothesis test, and then write the PAP? One way is to use reliably known availability dates of certain datasets such as government adminstrative data. Verifiable pre-specification of observational research was undertaken in \cite{neumark_employment_2001}, which prospectively detailed analysis of the unemployment effects of the minimum wage, a question that has been debated for at least a century \citep{Neumark_2014_bathwater}.\footnote{An editorial introduction to \cite{neumark_employment_2001} was accidentally left out of the issue, and published the following month, see \cite{Levine2001editorial}. Others followed Neumark's paper in spirit by using the same regression specifications on data from Canada \citep{CanadianMinWage2006}.}
+
The federal minimum wage changed in October 1996 and September 1997. Neumark submitted a pre-specified research design consisting of the exact estimating equations, variable definitions, and subgroups that would be used to analyze the effect of the minimum wage on unemployment of younger workers using October, November, and December CPS data from 1995 through 1998. This detailed plan was submitted to journal editors and reviewers prior to the end of May 1997; the October 1996 data started to become available at the end of May 1997, and Neumark assures readers he had not looked at any published data at the state level prior to submitting his analysis plan.
+
Neumark's hope was to eliminate the potential for bias due to ``author effects.'' The obvious time stamp of the federal government’s release of data indeed makes this possible, but the situation also benefits from the depth and intensity of the debate prior to this study. Neumark had an extensive extant literature to rely on when picking specific functional forms and subgroup analyses to be conducted. He tests two definitions of the minimum wage, the ratio of the minimum wage to the average wage (common in Neumark's previous work), as well as the fraction of workers who benefit from the newly raised minimum wage (used in David Card's earlier work \citep{Card1992minwage}) and tests both models with and without controls for the employment rate of higher-skilled prime age adults (as recommended by \cite{Deere1995minwage}).
+
These equations are also each estimated with only a lagged coefficient for minimum wage, and only for the outgoing rotation group (ORG) subset of the data. The equations are estimated for 16-19 year olds, 16-24 year olds, 16-24 year olds with no more than high school education, 20-24 year olds with no more than high school education, and 16-20 and 20-24 year olds (separately) with less than high school education. The results show mostly insignificant results, but that 18 of the 80 specifications result in statistically significant decreases in employment (at the .10 or .05 level), with elasticities ranging from -0.14 to -0.3 for significant estimates and smaller (closer to zero) for insignificant estimates.
+
How widely could Neumark's method of pre-specification prior to data-release become, and what do we gain from it? Thanks to pre-specification we know exactly how many hypothesis tests Neumark ran (at least 80 coefficients, and more with joint tests), so we know the context in which to appropriately interpret the p-values. This is a significant improvement over much other observational research. Of course, econometricians could still argue about the appropriateness of certain models (whether to include lags or look at contemporaneous effects, for instance) and the analysis suffers from noisy estimates given the relatively small number of changes to state minimum wages compared to other time periods that had been previously analyzed using similar methods.
+
It is also difficult to see how a researcher could reach this level of detail with a research question with which they were not already intimately familiar; it seems more likely that one would either pre-specify with either a completely inadequate level of detail, or a wildly inappropriate specification. But perhaps this familiarity is a good thing. Much exploratory observational research could continue as is, and for important and thoroughly debated and defined question, confirmatory research could be conducted in a pre-specified fashion. Although pre-specification is difficult, given the vast troves of data released on reliable schedules by governments of developed countries through the world, we are certain that the number of cases is which verifiable pre-specification of observational research could and should be conducted is far greater than the number of times it has been conducted.
+
\subsubsection{Project Protocols}\label{project-protocols}
+
A project protocol can be somewhat similar to a PAP, but is distinct. A protocol is a detailed recipe or instruction manual for others to use to reproduce an experiment. Protocols are important both in helping solve researcher degrees of freedom problems by making the exact details of analysis known and help avoid selective reporting, as well as in making one's work reproducible. Protocols are standard in the medical literature, as in areas of lab science, but may be less familiar to those used to working with administrative or observational data. Lab sciences are rife with examples of experiments failing to replicate because of supposedly minor changes such as the brand of bedding in mouse cages, the gender of the laboratory assistant, or the speed at which one stirs a reagent \citep{sorge2014olfactory, hines2014sorting}, and we assume the same situation exists in the social sciences. \textit{Nature} has decided to expand its methods section in order to encourage better reporting.\footnote{\url{http://www.nature.com/news/announcement-reducing-our-irreproducibility-1.12852}}
+
We believe the social sciences would benefit from more careful documentation of methods. When one uses administrative data this can be accomplished by sharing one's data and code so that analysis is transparent.\footnote{It should be noted that the need for documentation of survey method is not eliminated by using administrative data, the burden simply falls upon the administration.} This is discussed below in section~\ref{replication-and-reproducibility}. With original data collection, researchers should provide very detailed descriptions of what exactly they did. A 33-item checklist of suggested items is contained in the SPIRIT(\href{http://www.spirit-statement.org}{Standard Protocol Items: Recommendations for Interventional
Trials}) statement \citep{chan_spirit_2013}, including details on the participants, interventions, outcomes, assignment, blinding, data collection, data management, and statistical methods, among other things.
+
One area in which social sciences could improve involves details of randomization. \cite{bruhn_pursuit_2009} documents the lack of clear explanation pertaining to how randomization was conducted in RCTs published in economics journals. Variables used for stratification are not described, and the decision of whether to control for baseline characteristics was often done after the fact. While there seems to be internal disagreement in both
medicine and the social sciences over the appropriateness of including baseline
control variables in regression analysis of a randomized trial, having
researchers selectively report whatever method gives them the most
significant-seeming results is obviously not the optimal outcome.
+
The medical literature also exhibits much greater concern over the blinding and concealment of randomized assignment than some of the social science literature. In some situations, blinding is impossible or irrelevant in a social science field experiment: for example, the recipient of a cash transfer needs to know that they received cash in order for the program to have any effect. Also, a social scientist interested in the potential scaling up of a government program may rightfully be unperturbed by some respondents assigned to the control group somehow gaining access to treatment, since this behavior would undoubtedly occur if the program were scaled up and the researcher still has a valid intention to treat estimate. This is clearly not always the case, especially if one wants an accurate estimate of the efficacy of a program or a treatment on the treated estimate. Tales of trials ruined through carelessness with the original randomization assignment as well as tips on how to avoid the same problem are described in \cite{schulz_allocation_2002}. In addition to \cite{bruhn_pursuit_2009}, political science has produced guidelines for randomization and related disclosure, avaialable at \url{http://e-gap.org/resources/guides/randomization/}.
+
+
Some medicine and science journals have begun to publish protocols. While the
advantages of publishing a protocol related to the development of a new
procedure (e.g. ``we have developed a new method of isolating mRNA'')
should be obvious, the advantages of publishing protocols for randomized
trials under way are perhaps less obvious, but still exist. \emph{BioMed
Central} and \emph{BMJ Open}, among others, now publish protocols of
trials planned or ongoing, with the hopes that this will reduce
publication bias, allow patients to see trials in which they might like
to enroll, allow funders and researchers to learn of work underway to
avoid duplication, and to allow readers to compare what research was
originally proposed to what was actually completed.\footnote{See
\url{http://www.biomedcentral.com/authors/protocols} and
\url{http://bmjopen.bmj.com/site/about/guidelines.xhtml\#studyprotocols}.}
\emph{BMJ Open} suggests, but does not require, that its published
protocols include the items in the SPIRIT checklist.
+
Protocols are not a perfects solution, as even in published (or otherwise public) protocols, studies have found important differences between protocols and published results. 60-71\%
of outcomes described in protocols went unreported in the paper while
62\% had major discrepancies between primary outcomes in the protocols
and in the published papers, though there was a relatively even mix of
these discrepancies favoring significant or insignificant results \citep{chan_a_empirical_2004}. Another study found that appropriate level of statistical detail is often lacking in protocols, and there are often discrepancies between protocols and published results \citep{saquib_practices_2013}. 31\% of published papers had some sort of pre-specified plan for their regression adjustments (i.e.~specifying which baseline covariates would be controlled for), %while 70-74\% of those that published a design paper or provided a protocol to the authors,
but only 53\% of the plans matched what was published in the ultimate paper.
+
+
\subsection{Multiple Hypothesis Testing}
Several of the PAP and lists of suggestions above include corrections for multiple hypothesis testing. The idea of correcting for multiple tests is widespread in certain fields, but has yet to take hold in the social sciences. Simply put, the idea is that because we are aware of the fact that test statistics and p-values appear significant purely by chance a certain proportion of the time, we can report different, \textit{better} p-values that control for the fact that we are running multiple tests. There are several ways to do this, a few of which are used and explained in a simple and straightforward manner by \cite{anderson_fwer}:
\begin{itemize}
\item
Report index tests---instead of reporting the outcomes of numerous tests, standardize outcomes and combine them into a smaller number of indeces (e.g. instead of separately reporting whether a long-term health intervention reduced blood pressure, diabetes, obesity, cancer, heart disease, and Alzheimer's, report the results of a single health index.) \cite{kling2007experimental} implements an index test from the Moving to Opportunity field experiment, using methods developed in biomedicine by \cite{obrien1984procedures}.
+
\item
Control the Family-Wise Error Rate (FWER)---FWER is the probability that at least one true hypothesis in a group is rejected (a type I error), meaning it is advisable when the damage from incorrectly claiming \textit{any} hypotheses are false is important. There are several ways to do this, with the simplest (but very conservative) method being the Bonferroni correction of simply multiplying every original p-value by the number of tests done. Holm's sequential method involves ordering p-values by class and multiplying the lower p-values by higher dicsount factors \citep{holm_multipletesting}. An efficient recent method is the free step-down resampling method, developed by \cite{westfall_young_multiple}.
\item
Control the False Discovery Rate (FDR)---In situations where a single type I error is not catastrophic, researchers may be willing to use a less conservative method and trade off some incorrect rejections in exchange for greater power. This is possible by controlling the FDR, or the percentage of rejections that are type I errors. \cite{benjamini1995controlling} details a simple algorithm to control this rate at a chosen level, and \cite{benjamini2006adaptive} describes a two-step procedure with greater power.
+
+
\end{itemize}
+
\subsection{Subgroup Analysis}
One aspect of researcher degrees of freedom related to multiple hypothesis testing that seems to have taken hold widely in the medical literature is the aversion to sub-group analysis (``interactions'' to
most economists). Given the ability to test for a differential effect by many different groupings, crossed with each outcome variable, sub-groups analysis can almost always find some sort of supposedly significant effect. An oft-repeated story in the medical literature revolves around the publication
of a study on aspirin after heart attacks. When the editors suggested including 40 subgroup analyses, the authors relented on the condition
they include some of their own. Gemini and Libras had worse outcomes
when taking aspirin after heart attacks, despite the large beneficial
effects for everyone else. (Described in \cite{schulz_multiplicity_2005},
with the original finding in \cite{isis-2_second_international_study_of_infarct_survival_collaborative_group_randomised_1988}) Whether in a randomized trial or
not, we feel that social scientists could benefit from reporting the number of
interactions tested, possibly adjusting for multiple hypotheses, and
ideally specifying beforehand the interactions to be tested, with a justification from theory or previous evidence as to why the test is of interest.
+
+
\subsection{Results-Blind Reviewing}
A new development in research transparency which helps to address both publication bias and researcher degrees of freedom is results-blind reviewing. In results-blind reviewing, authors submit a detailed research plan \textit{before} conducting the research. They submit this plan to a journal, and the journal rejects or gives an in-principle acceptance of the not-yet-written article based on the scientific merit of the question being asked and the methods proposed to answer them, as opposed to whether the results pass an arbitrary threshold of statistical significance. Then the authors conduct the research, and their paper is published as long as they don't deviate too much from what they initially proposed. The editors and reviewers have tied their hands and have no ability to accept only significant results, and the authors have less incentive to game their statistical analysis in order to find something signficant.
+
This new mode of publication, called ``Registered Reports," is championed by Chris Chambers, psychologist at Cardiff University, and has been adopted by over a dozen journals. See a full list of journals adopting the procedure at \url{https://osf.io/8mpji/}. \textit{Social Pyschology} ran an issue dedicated to this type of article, with an editorial explaining the concept \citep{nosek2014registered}. \textit{AIMS Neuroscience}, which uses this format, published an editorial answering 25 frequently asked questions pertaining to registered reports \citep{chambers2014instead}.
+
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Replication and
Reproducibility}\label{replication-and-reproducibility}
+
\begin{quote}
``Economists treat replication the way teenagers treat chastity - as an
ideal to be professed but not to be practised.''---Daniel Hamermesh,
University of Texas at Austin Economics
\end{quote}
+
\begin{quote}
``Reproducibility is just collaboration with people you don't know,
including yourself next week''---Philip Stark, University of California Berkeley Statistics
\end{quote}
+
Replication, in both practice and principle, is a key part of social
science research. We first define what exactly we mean by replication
using the taxonomy developed in \cite{hamermesh_viewpoint:_2007} and \cite{hunter_desperate_2001}.
According to Hamermesh, replication comes in a few different shapes: pure, statistical, and
scientific.
+
\begin{itemize}
\item
Pure: Using the exact same data and the exact same model to see if the
published results are reproduced exactly.
\item
Scientific: Using a different sample from a different population, and
similar, but perhaps not identical model .
\item
Statistical: Using the same model and underlying population but a
different sample. In Hamermesh's view, this is less relevant to certain fields, such as economics,
where researchers are likely to already use as large a sample as is available.
\end{itemize}
+
One might imagine a fourth type that uses the same data but probes the data using additional robustness and sensitivity checks. Others have described replication in terms of a spectrum from full replication (independent collection of data and re-running analysis) to
reproducibility, where the same data and code are re-used by other
researchers \citep{peng_reproducible_2011}. Whatever the terminology used, replication or reproducibility, transparent research requires making data and code available to other researchers so they can try and get the same results.
+
+
\subsection{Code and Workflow}\label{code-and-workflow}
+
Reproducing research often involves using the exact code and statistical
programming done by the original researcher. To make this possible, code
needs to be both (1) easily available and (2) easily interpretable.
Thanks to several free and easy to use websites described below, code
can easily be made available by researchers without requiring funding or
website hosting. Making code easily interpretable is a somewhat more complicated
task, nevertheless, the extra effort spent to make a more manageable
code pays off with large dividends.
+
\subsubsection{Publicly Sharing Code}\label{publicly-sharing-code}
+
Once analysis is complete (or even before this stage) researchers should
share their data and code with the public. GitHub
(\url{http://www.github.com}), The Center for Open Science's Open
Science Framework (\url{http://osf.io}), and Harvard University's
Dataverse (\url{http://thedata.org}) are all free repositories for data
and code that include easy to use version control.\footnote{BitBucket
(\href{styles.xml}{http://www.bitbucket.org}) is another
web service that one can use for free version control and archiving
of public data and code.}
Version control is a powerful way to archive versions of files so that old versions are not lost and can be returned
to if needed. The most naive way to organize a project would be to just name a file ``MyAnalysis.do'' and then save over it any time you or your coauthor made changes. We sincerely hope no one is working this way, since you are losing tons of information. Most people are probably doing some version of the ``date and initials'' method. Instead of simply calling one's analysis code ``MyAnalysis.do'' and repeatedly saving over and losing old versions, when you save a new version, you add the date to the file name: ``MyAnalysis.2014.08.13.do'', and if a co-author makes changes on the same day, they add their initials:``MyAnalysis.2014.08.13\_GC.do'' or they change the date: ``MyAnalysis.2014.08.14.do''. This can work on small projects, but it really isn't the best way to work. Why does the operating system tell me the file from August 13 was modified in October? What if one analysis script file is called 14 times in 9 different files---did you remember to go through and update all the calls? \cite{GentzkowShapiro} provide strong evidence of the inadequacy of dating and initialing : ``\textit{Not one piece of commercial software you have on your PC, your phone, your tablet, your car, or any other modern computing device was written with the ``date and initial'' method.}'' [emphasis original]
+
The solution is version control (also referred to as revision control). Version control is free software (\href{https://git-scm.com/book/en/v2/Getting-Started-About-Version-Control}{Git} and \href{https://mercurial.selenic.com/}{Mercurial} are popular and free implementations) that stores all versions of files you create, and can easily compare versions of the files to highlight changes, revert to earlier versions, accept or reject changes proposed by a collaborator, and reconcile conflicting versions when different people make changes to the same file. Version control creates a repository, and stores all versions of your files in the repository\footnote{Repositories can take up a significant amount of space, but you can avoid this problem by not storing generated files or binaries (.pdf, .docx, .etc) since every user with access to the repository should be able to recompile and generate files on their own. Version control works best with simple text files.} Web services
such as \href{http:www.github.com}{GitHub} offer free hosting of these repositories with easy to use web-based interfaces and GUI software, so that you and your collaborators can all access the same files, but you can easily host the repository on a server of your own.
+
\subsubsection{Managing Workflow}\label{managing-workflow}
+
Code is just one aspect of a larger structure we refer to as
``workflow'' after \cite{long_workflow_2008}, by which we mean the
combination of data, code, organization, and documentation: everything
from file and variable names to folder organization as well as efficient
and readable programming, and data storage and documentation. We strongly recommend that Stata-users read \cite{long_workflow_2008} and R users read \cite{gandrud2013reproducible} for workflow recommendations both general and specific to their respective programming language\footnote{Also see and \cite{kirchkamp_workflow_????}}. Our suggestions here borrow heavily from their excellent work.
+
Another excellent, more brief work is Matthew Gentzkow and Jesse Shapiro's excellent manual on code and data \href{https://www.brown.edu/Research/Shapiro/pdfs/CodeAndData.pdf}{\citep{GentzkowShapiro}}. They come from the same background as us--they didn't take many programming classes, they're just clever and figured it out. But once their data got massive, and their programming problems got massive with it, they figured they should listen to the fulltime programmers and database managers who had spent years and billions of dollars solving these problems. Their manual adapts many of these solutions to data-based social science research.
+
We also refer undergraduate instructors and others who may be interested to Richard Ball and Norm Medeiros' \href{http://www.haverford.edu/TIER/}{Project TIER} (Teaching Integrity in Empirical Research), which is a ``protocol for comprehensively documenting all the steps of data management and analysis that go into an empirical research paper.'' A specific file organization using the Open Science Framework is taught so that teachers can exactly reproduce the work of every student (and so students can reliably get the same answer every time they conduct their analysis).
+
\paragraph{Software:}
+
Although we agree with the movement by many towards open source software such
as \href{http://www.r-project.org/}{R} and \href{https://www.python.org/}{Python}, we appreciate that many disciplines have long
traditions of using proprietary software such as SAS and STATA, and
learning a new programming language may be an undesirable additional
task in researchers' busy lives. That said, there are several general
coding rules that all researchers should use when organizing and
implementing their analysis, and researchers should strive to make their
work usable by as many others as possible.
+
\paragraph{Writing Code:}
Perhaps the most important rule is to write code instead of working by hand. By
that we mean:
+
\begin{itemize}
\item
Do not modify data by hand, such as with a spreadsheet. Which is to
say, don't use Excel.
\item
Use neither the command line nor drop-down menus nor point-and-click
options in statistical software.
\item
Instead, do everything with scripts.
\end{itemize}
+
The simple reason for this is reproducibility. Modifying data in Excel
or any similar spreadsheet program leaves no record of the changes made
to the data, nor any explanation of the reasoning or timing behind any
changes. Although it may seem easy or quick to do a one-time-only
cleaning of data in Excel, or make ``minor'' changes to get the data
into a format readable by a researcher's preferred statistical software,
unless these changes are written down in excruciating detail, this is
not reproducible by other researchers. It is better to write a programming
script that imports the raw data, does all necessary changes, with
comments in the code that explain changes, and saves any intermediate
data sets used in analysis. Then, researchers can share their initial
raw data and their code, and other researchers can reproduce their work
exactly.
+
Though we understand that a fair amount of research has been done using
pull down menus in SPSS or Stata, we advise against this. A bare minimum
if one insists on going this route is to use the built-in
command-logging features of the software. In Stata, this involves the
`cmdlog' command, in SPSS, this involves the paste button to add to a
syntax.
+
The ideal is to make everything, including changes like rounding and
formatting, done with scripts. Even downloading of data from websites
can be done through a script. For example, in R, the download.file()
function can be used to save data from a website. (Though of course this
opens the possibility to the data file changing. When reproducing
results from a given dataset is more important than the data from a
specific source, researchers should download their raw dataset once, and
never save over it, instead saving all modified intermediate datasets in
a separate location.) Another extremely important way to prevent
unintentional changes to data is to always set the seed for random
number generators whenever any random numbers are to be used (set.seed()
in R, set seed () in Stata). Additionally, information about the exact
software version used should be included (include the `version' command in Stata, or use the
session.info() command in R) as well as computer processor and operating
system information. The casual programmer may assume that sophisticated
software would always produce the exact same answer across multiple
versions of software and platforms, but this is not the case. This is also definitely not the case with user-written packages. R users can use the packageVersion() command, and can run old versions of packages since they are archived at \href{http://cran.r-project.org}{CRAN}. Stata users can use the viewsource command for any .ado they use, but since the Statistical Software Components (SSC) unfortunately does not archive old versions, reproducibility may be lost, so ideally researchers would include the actual code for the version of the user-written .ado along with their publicly archived data and code files
+
+
Finally, two simple organizing principles to consider are:
\begin{enumerate}
\item
Consider not saving statistical output, and just saving the code and data that generates it. Obviously this would be unrealistically time consuming for large projects, but the idea is that you should be able to reproduce all steps of your analysis that you could in theory take this approach.
+
\item
What would happen if you, or your laptop hard drive, were hit by a bus? How easily would anyone else be able to reproduce your work? Hopefully the probability is non-zero.
\end{enumerate}
+
\subsection{General Workflow
Suggestions:}\label{general-workflow-suggestions}
Here we offer some specific workflow organization suggestions that should be valid regardless of code or operating system.
\begin{itemize}
\item
Do not use spaces in directory or file names, as it complicates referring to them in certain software.
\item
Use ``naming directories'', .i.e.~a directory beginning with ``-'' (so
that it will appear first alphabetically) inside each directory to
explain the contents of the above directory.
\item
Add name, date, and describe contents, as well as updates, to all
scripting files.
\item
Keep a daily research log, i.e. a detailed written diary of what research is done on a given day. You'll be surprised how often this will be useful to answer questions about whether you ran a certain test or not, when you did it, and what you called the file.
\item
Make sure that your script files are self-contained. That is, don't write a program that only works if you run a group of other files previously in a specific order and then leave things a certain precarious way. You should be able to run intermediate steps of the workflow without disrupting things downstream.
\item
You can never comment too much.
\item
Indent your code
\item
Once you post/distribute code or data, any changes at all require a new file name.
\item
Separate your cleaning and analysis files; don't make any new variables
that need saving (or will be used by multiple analysis
files) in an analysis file---it is better to only create the variables once so you know they're the identical when used in different analysis files.
\item
Never name a file ``final'' because it won't be.
\item
Name binary variables ``male'' instead of ``gender'' so that the name is more informative.
\item
Use a prefix such as x\_ or temp\_ so you know which files can easily
be deleted.
\item
Never change the contents of a variable unless you give it a new name.
\item
Every variable should have a label.
\end{itemize}
+
\subsection{Stata-specific
Suggestions}\label{stata-specific-suggestions}
+
\begin{itemize}
\item
Use the full set of missing values available to you (``.a''-``.z'', not exclusively
``.'') in order to distinguish between ``don't know'' and ``didn't ask'' or other distinct reasons for missing data.
\item
Make sure code always produces same result---if you use anything randomly generated, set the seed. When sorting or merging, you need to be sure to uniquely specify observations, because if you don't, Stata does something arbitrary and not repeatable. So instead of just sorting or merging on `ID' when there are multiple observations per ID, sort by `ID' and `name.' You can use the `duplicates' command to test whether the \textit{varlist} you use uniquely indentifies observations. The `sort, stable' command can be used, though it is slower.
\item
Use the `version' command in your .do file to ensure that other researchers who run your code with a newer version of Stata get the same results.
\item
Don't use abbreviations for variables (which may become unstable after
adding variables) or commands (beyond reason)
\item
Avoid using global macros, and use locals instead. (This is a common piece of programming advice: since globals can be accessed across different functions or spaces, they can create contradictions or inconsistent dependencies. In Stata, it's possible they can be safely used to define directory paths so collaborators can work across different computers, but that should likely be the extent of their use.)
\item
Use locals for varlists to ensure that long lists of variables include the same variables whenever intended.
\item
Use computer-stored versions of numerical output instead of manually typing in numbers or copying and pasting. For example, instead of copying and pasting the mean after a `summ' command, refer to `r(mean)'. Use the `return list' command to see a full list of stored values after a regular command and the `ereturn list' after estimation commands.
\item
If you have a master .do file that calls other .do files, which each have their own .log file, you can run
multiple log files at the same time (so you have a master .log file)
\item
Use the `label data' and `notes' commands to label datasets and help yourself and other researchers easily identify the contents.
\item
Use the `notes' command for variables as well for identifying information that is too long for the variable label.
\item
Use the `datasignature' command to generate a hash or checksum and help ensure that data is
the same as before.
\item
In addition to labeling your variables, you should also use value labels for all categorical variables. Include the
numerical value in the label, however, since without it, it can be hard to tell what numerical value is actually meant by a given category.
\item
Even though Stata is case sensitive, don't use capital letters in variable names since not all software packages
are case sensitive.
\item
Make your files as non-proprietary as possible (use the `saveold'
command to enable those with earlier versions to use your data. This
is why trusted repositories are so useful--they'll do this for you.)
\end{itemize}
+
In addition to making code available to the public, the code itself
should be written in a reader-friendly format, referred to as ``Literate
Programming,'' introduced in \cite{knuth_literate_1984} and \cite{knuth_literate_1992}. The basic
idea is that ``the time is ripe for significantly better documentation
of programs, and that we can best achieve this by considering programs
to be \emph{works of literature}\ldots{}Instead of imagining that our
main task is to instruct a \emph{computer} what to do, let us
concentrate rather on explaining to \emph{human beings} what we want a
computer to do.'' {[}emphasis original{]} Simply put, code should be
written in as simple and easily understood a way as possible, and should
be very well commented, so that researchers other than the original
author can more easily understand the goal of the code.
+
One tool to make literate (statistical) programming significantly easier
is Knitr (see \cite{xie_dynamic_2013, xie_knitr:_2014}) which is built into R
Studio\footnote{R Studio is a popular free integrated implementation of
R, available at \href{stylesWithEffects.xml}{http://www.rstudio.com}.}.
Knitr uses R Markdown (a very simple plain text markup language,
described at \url{http://rmarkdown.rstudio.com/}) in which one writes
both code and comments that is automatically spun into an easily read
and shareable HTML, PDF, or MS Word document. These can be posted and
shared for free at \href{https://rpubs.com}{RPubs}, an easy to use
hosting service by Rstudio. For Stata users, dynamic documents are slightly less well developed, but \href{http://www.haghish.com}{E.F. Haghish} is actively developing packages (\href{http://www.haghish.com/statistics/stata-blog/reproducible-research/packages.php}{Markdoc, Weaver, Ketchup, and Synlight}) that allow users to write their .do files in such a way that the log files output by Stata are formatted and readable in Markdown, HMTL, or \LaTeX .
+
%ADD EXAMPLE
+
%R MarkdownMarkdownHTML (edit only the first)
+
\subsection{Sharing Data}\label{sharing-data}
+
In addition to code, researchers should share their data if at all
possible. Many journals do not require sharing of data, but the number
that do is increasing.
+
\subsubsection{The JMCB Project and
Economics}\label{the-jmcb-project-and-economics}
+
In the field of economics, few, if any journals required sharing of data
before ``The Journal of Money, Credit, and Banking Project,'' published
in \emph{The American Economic Review} in 1986 \citep{dewald_replication_1986}. \emph{The Journal of Money, Credit, and Banking} started
the \emph{JMCB Data Storage and Evaluation Project} with NSF funding in
1982, which requested data and code from authors who published in the
journal. With a great deal of research funded by the NSF, it should be
noted that they have long had an explicit policy of expecting
researchers to share their primary data\footnote{``Investigators are
expected to share with other researchers, at no more than incremental
cost and within a reasonable time, the primary data, samples, physical
collections and other supporting materials created or gathered in the
course of work under NSF grants. Grantees are expected to encourage
and facilitate such sharing.'' See
http://www.nsf.gov/bfa/dias/policy/dmp.jsp}. Despite this, and despite
the explicit policy of the \emph{Journal} during the project, at most
only 78\% of authors provided data to the authors within six months
after multiple requests. (This is admittedly an improvement over the
34\% from the control group---those who published before the
\emph{Journal} policy went into effect---who provided data.) Of the
papers that were still under review by the \emph{Journal} at the time of
the requests for data, one quarter did not even respond to the request,
despite the request coming from the same journal considering their
paper! The submitted data was often an unlabeled and undocumented mess.
Despite this, the authors attempted to replicate nine papers, and often
were completely unable to reproduce published results, despite detailed
assistance from the original authors.
+
Shockingly, nothing much changed with the publication of this important
article. A decade later, in a follow-up piece to the JMCB Project
published in the Federal Reserve Bank of St.~Louis \emph{Review}
\citep{anderson_replication_1994}, the authors note that only two economics
journals other than the \emph{Review} itself (\emph{Journal of Applied
Econometrics, Journal of Business and Economic Statistics}) requested
data from authors, and neither requested code. The \emph{JMCB} itself
discontinued the policy of requesting data in 1993, though it resumed
requesting data in 1996. The authors repeated their experiment with
papers presented at the St.~Louis Federal Reserve Bank conference in
1992, and obtained similar response rates as original JMCB Project. The
flagship economics journal, the \emph{American Economic Review} (AER),
did not start requesting data until 2003. Finally, after a 2003 article
showed that nonlinear maximization methods often produce wildly
different estimates across different software packages, that not a
single AER article tested their solution with different software, and
that fully half of queried authors from a chosen issue of the AER,
including a then editor of the journal, failed to comply with the policy
of providing data and code, editor Ben Bernanke made the data and code
policy mandatory in 2005 \citep{mccullough_verifying_2003, mccullough_got_2007}.
+
The current data policy from the \emph{American Economic Review} can be
seen here: \url{https://www.aeaweb.org/aer/data.php}. The AER conducted a self-review and
found good, but incomplete, compliance \citep{glandon_report_2010}; others believe much work remains. In addition to all
the journals published by the American Economic Association, several top journals, including \emph{Econometrica, The Journal of Applied
Econometrics, The Journal of Money Credit and Banking, the Journal of
Political Economy, The Review of Economics and Statistics, and the
Review of Economic Studies}, now explicitly require data and code to be
submitted at the time of publication. Sadly, one of the top journals, \textit{The Quarterly Journal of Economics} has no data-sharing requirement.
+
%EXPAND HERE ON JUST HOW MUCH DATA YOU SHOULD SHARE
+
+
\subsubsection{General Repositories}\label{general-repositories}
+
The previous section on the \emph{JMCB} describes only a few journals in
one field of the social sciences. Even if the journal to which you
submit your research does not require you to supply them with your code
and data, researchers should still share these materials. Though some
repositories, particularly Harvard's Dataverse, seem equipped to handle
data from practically any researcher (a free 1 TB of storage is
standard, with more possible upon request), many repositories specialize. \href{http://www.re3data.org}{The Registry of Research Data Repositories}
has described over 900 data repositories to help you find the right data
repository for your data. A key advantage to using a trusted repository
such as one listed here, in lieu of simply throwing the data up on your
own website or making your Dropbox folder public, is that many of these
repositories will take your data in its proprietary (Stata, SAS, SPSS,
etc.) form, and make it accessible in other formats. Storing your data in a repository with other similar datasets also makes it easier for others to find your data, instead of requiring that they already know of its existence, as would likely be the case with personal websites. Your own personal website is also more likely to be taken offline, should a researcher change schools or retire.
+
\subsubsection{Differential Privacy}\label{differential-privacy}
+
One important caveat to making data widely available, is that despite
anonymization, in the age of big data, sometimes individual subjects can
easily be identified. \cite{heffetz_privacy_2014} recount deliberate data
releases by Yahoo! Inc., the Massachusetts state government, and
Netflix, that could easily be used to identify individuals in the data,
despite the absence of direct identifiers such as names or social
security numbers. The problem is that ``de-identification does not
guarantee anonymization.'' This problem is well known in computer science, but solutions are still being developed and are not widely implemented.
+
+
\subsection{Reporting Standards}\label{reporting-standards}
In research, the devil truly is in the details. Whether it is for assessing the validity of a research design or for
attempting to replicate a study, details of what exactly was done must be recorded and made available to other researchers.
The exact details that are relevant will likely differ from field to field, but an increasing number of fields have
produced centralized checklists that describe (in excruciating detail) what disclosure is required of published studies. These checklists are not often published with the paper, but can be submitted with the original article so that reviewers can check that it has been completed. With infinite and easy web storage, researchers can easily post these materials on their website even if journal editors insist on cutting their methods sections for space reasons.
+
\subsubsection{Randomized Trials and CONSORT}\label{randomized-trials}
The most widely adopted reporting standard guideline is the Consolidated Standards of Reporting Trials (\href{http://www.consort-statement.org}{CONSORT}). Parallel to construction of \href{http://clinicaltrials.gov}{clinicaltrials.gov} and registration,
reporting standards evolved, and are now nearly universally adopted for randomized trials published in medical journals, required or requested by reviewers during the review process. This is still in its infancy in the social sciences.
+
The original CONSORT was developed in the mid 1990's \citep{begg_c_improving_1996}. After five years, research showed that reporting of essential details, as required by the checklist, had significantly increased in journals requiring the standard \citep{moher_d_use_2001}. The statement was revised in 2001, and simultaneously published in three of the top journals \citep{moher_consort_2001}). The statement was again revised in 2010 \citep{schulz_consort_2010}. The statement is a 25-item checklist pertaining to the title, abstract, introduction, methods, results, and discussion of the article in question, and seeks to delineate the minimum requirements of disclosure that may not be sufficiently addressed through other measures.
+
\subsection{Social Science Reporting Standards}\label{soc-sci-standards}
Though a standard akin to CONSORT has not been formally adopted by social science or behavioral science journals, at least as far as we are aware, there have been attempts to do this: In political science, the Experimental Research Section Standards Committee produced a detailed list of items required for disclosure of experiments in political science \citep{gerber_reporting_????}. This checklist is available \href{http://www.davidhendry.net/research-supplemental/gerberetal2014-reportingstandards/gerberetal2014-reportingstandards&appendix1.pdf}{here}.\footnote{\url{http://www.davidhendry.net/research-supplemental/gerberetal2014-reportingstandards/gerberetal2014-reportingstandards&appendix1.pdf}}
+
In economics, where no standards have been developed, one article has highlighted the fact that there is not much discussion of essential features of randomization (how was randomization stratified, if at all? How were control variables determined?), but no standards have been adopted. \citep{bruhn_pursuit_2009}
+
In psychological and behavioral research, an extention to CONSORT for Social and Psychological Interventions (CONSORT-SPI) has been developed in \citep{montgomery2013protocol}, but has so far not been widely adopted, or required by journals.
+
\subsection{Observational Reporting Standards}\label{observational-standards}
Social science has yet to make a serious push for reporting standards in RCTs, let alone observational work, but the medical/epidemiological literature has created standards in this type of work, though they are not as widely adopted as CONSORT. Perhaps the most well-known is the \href{http://www.strobe-statement.org}{STROBE Statement} (Strengthening the Reporting of Observational Studies in Epidemiology). STROBE provides checklists for reporting of cohort, case-control, and cross-sectional studies. These standards have been endorsed by approximately 100 journals in the field.\footnote{\url{http://www.strobe-statement.org/index.php?id=strobe-endorsement}}
+
Medicine has in fact come up with too many checklists to describe them all individually. Acknowledging that every field and type of research is different, the Equator Network (Enhancing the Quality of Transparency of Health Research) serves as an umbrella organization that seeks to keep tabs on all the best reporting standards and help researchers find which reporting standard is most relevant for their research. See \url{http://www.equator-network.org/} for more information.
+
+
+
+
+
\section{Conclusion}\label{conclusion}
+
As you may have noticed, many of the activities described in this manual
require extra work. Before you run an experiment, we're telling you to
write down the hypothesis, carefully explain how you are going to test
the hypothesis, write down the very regression analysis you're going to
run, write a detailed protocol of the exact experimental setting, and
then you have to post all of this publicly on the Internet with some
sort of Big Brother organization. Or at least that's one way to look at
it. But we strongly belive these steps are (1) not that difficult once
you get used to them and (2) well worth the reward. You'll get p-values
you can believe in. The next time someone asks you for your data, you
just point them to a website, where they'll download the data and
code, and the code will produce the exact results in the published
paper. The next time you open up a coding file you haven't looked at in
months to make a change suggested by a reviewer, your code will be so
thoroughly commented, you'll know exactly where to go to make the
changes. And the next time you want to extend the analysis of a
published paper, you click the link in the paper and have the data on
your own computer in seconds. Science moves forward.
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Glossary of Terms}
We feel that it is important to define the terms we use in this document. Although many of the concepts overlap, we suggest that researchers use the following terms:
\begin{itemize}
\item
Analysis Plan: See \hyperref[def-pap]{pre-analysis plan}
+
\item
Data citation: The practice of specifically citing datasets, and not just the paper in which a dataset was used. This helps other researchers to find data and rewards researchers who share data, leading to better science. Read more at \url{http://www.icpsr.umich.edu/icpsrweb/ICPSR/curation/citations.jsp}.
+
\item \label{def-mining}
Data mining: Searching blindly or repeatedly through data to find supposedly statistically significant relationships. While not inherently wrong, if done without a plan or without adjusting for multiple hypothesis testing, test statistics and \hyperref[def-pvalue]{p-values} that result no longer hold their traditional meaning, and can lead to research that cannot be replicated.
+
\item \label{def-datasharing}
Data sharing: Researchers making the data they use in an analysis available to other researchers, ideally through a trusted public archive.
+
\item\label{def-dbp}
Design-based publication: See \hyperref[def-rbr]{results-blind review}.
+
\item
Disclosure: In addition to the widely accepted norm of publicly declaring all potential conflicts of interest, researchers should detail all the ways in which they test a hypothesis (e.g. include the outcome from all regression specifications tested in the appendix.)
+
\item
Fishing: See \hyperref[def-mining]{data mining}.
+
\item
Literate Programming: The idea of writing programming code designed to be read and easily understood by a human. Use of this best practice can make a researcher’s code far more easily reproducible by others.
+
\item
Multiple hypothesis correction: Statistically taking into account the fact that multiple hypotheses have been tested. This tends to decrease the reported statistical significance of any individual test conducted. The oldest method, known as the Bonferroni correction, simply divides the significance threshold by the number of tests. This is quite conservative, and more modern methods are helpfully described in \cite{anderson_fwer}.
+
\item
Open Access: Journals, or articles that are freely available to the public, as opposed to available only to paid subscribers. See \href{https://www.plos.org/open-access/howopenisit/}{HowOpenIsIt?} for a detailed definition of the spectrum of openness.
+
\item
Open Data: See \hyperref[def-datasharing]{data sharing}.
+
\item
P-hacking: See \hyperref[def-mining]{data mining}.
+
\item \label{def-pvalue}
P-value: The statistic researchers use to make judgments regarding statistical significance, which is quite often misunderstood. It is the probability of obtaining a test statistic at least as extreme as the observed test statistic when the null hypothesis is true.
+
\item\label{def-pap}
Pre-analysis plan: A document that details, ahead of time, the statistical analysis that will be conducted for a given research project. Outcomes, control variables, and regression specifications are all written in as much detail as possible. This serves to make research more confirmatory in nature.
+
\item
Pre-specification: Detailing the method of analysis before actually beginning data work; the same as writing a \hyperref[def-pap]{pre-analysis plan}.
+
\item
Protocol: A general term meaning a document that provides a detailed description of a research project, ideally written before the project takes place, and in enough detail that other researchers may reproduce the project on their own. Often used in the context of human subjects IRB protocols, but increasingly used in connection with pre-analysis plans.
+
\item
Publication Bias: The unfortunate fact that research is often only published when it contains the rejection of a null hypothesis test, i.e. a statistically significant relationship. Reviewers or journal editors may consider a null finding to be of less interest, or a researcher may fail to write up a null result, even though the null result may be the truth.
+
\item
Registration: Publicly declaring that an investigation of a hypothesis is being or will be undertaken.
\begin{itemize}
\item Study Registration: Registering a research project generally.
\item Trial Registration: Registering a randomized trial.
\end{itemize}
\item
Registry: A database of registered studies or trials. For instance, \url{socialscienceregistry.org} or \url{clinicaltrials.gov}. Some of the largest registries only accept randomized trials, hence the frequent discussion of ‘trial registries.’
+
\item
Registered Reports: See \hyperref[def-dbp]{results-blind review}.
+
\item
Replicable: See \hyperref[def-repro]{reproducible}.
+
\item
Replication: The idea of conducting an existing research project over again, with the hope of obtaining the same result. A subtle taxonomy exists, and unfortunately there is disagreement, which is explained in \cite{hamermesh_viewpoint:_2007} and \cite{clemens_replication}.
\begin{itemize}
\item
Pure Replication: Re-running existing code, with error-checking, on the original dataset and seeing if the published results are obtained.
\item
Scientific Replication: Attempting to reproduce the published results with a new sample, with the same code or with slight variations from the original analysis.
\end{itemize}
+
\item \label{def-repro}
Reproducible: The test of whether research can redone by another researcher and produce the same results as the original.
\item
Researcher Degrees of Freedom: the flexibility that a researcher has in data analysis, whether consciously abused or not. See \hyperref[def-mining]{data mining}.
+
\item \label{def-rbr}
Results-blind Review: To help reduce publication bias, peer review can take place before results of a study are determined. Reviewers evaluate the design of a study to see whether the question is important and whether the study is well-designed. Good studies are given in-principle acceptance, and cannot be discriminated against for a null result, which, after all, may be the truth. See \url{https://osf.io/8mpji/wiki/home/} for journals practicing this form of publication.
+
\item Specification searching: Testing numerous regression specifications and reporting only the model that produces the desired results. See \hyperref[def-mining]{data mining}.
\item
Trusted Digital Repository: A location for storing data that others can believe will not be manipulated, and will be available into the future. Storing data here is superior to simply posting on an individual website, since it is more easily accessible and less easily changed.
\item
Version Control: A method of tracking every edit made to a computer file. This is often quite useful for empirical researchers who may edit their programming code hundreds or thousands of times.
\end{itemize}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newpage
\singlespacing
%\section{References}\label{references}
\bibliographystyle{plainnat}
\bibliography{Bibliography}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}
+ + + +
+ +
+ + + + +
+ +
+ +
+
+ +
+ + + + + + +
+ + + You can't perform that action at this time. +
+ + + + + + + + + + +
+ + You signed in with another tab or window. Reload to refresh your session. + You signed out in another tab or window. Reload to refresh your session. +
+ + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/README 1.md b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/README 1.md new file mode 100644 index 0000000..0ca58a6 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/README 1.md @@ -0,0 +1,33 @@ +# BITSS Annual Meeting 2017 + +## Tools for Research Transparency +Please see the full agenda for the Annual Meeting [http://www.bitss.org/wp-content/uploads/2015/12/2017-BITSS-Annual-Meeting-Agenda.pdf](here). + +### Agenda for Software Workshop +Time | Session | Presenter | +------------ | ------------- | ------------| +10:00AM-11:15AM | Hands-on with Dynamic Documents (using RMarkdown + brief references to Stata) | Fernando Hoces de la Guardia +11:15AM-12:00PM | Demonstration of Pandoc, and ShareLaTeX | Garret Christensen + + +### Software Installation +*Please install the following components before the beginning of the workshop!* We will have problems if everyone tries to download the ~4GB LaTeX installer at the same time. + +#### Dynamic Documents +You can write your code and your paper in one place. This means you won’t mess anything up +copying and pasting, and you’ll never have to wonder what code it was that produced which +figure, where on earth you saved it, or whether the paper has the updated version. + + * [Install R.](https://cloud.r-project.org) + * [Install RStudio.](https://www.rstudio.com/products/rstudio/download/#download) + * Download the contents of the repository into a folder in your desktop. ("Clone or download" button [here](https://github.com/BITSS/Annual2017)) + * *Optional:* if you want to replicate the section where we produce an output into PDF, you will need to have [LaTeX](https://www.latex-project.org/get/) installed in your computer (the default version, not the small one). + + * With RStudio, go to `File -> New file -> RMarkdown...` and hit `OK`. Then save the file. Then go to `File -> Knit file`. If it runs without problem you are ready to start. If it asks to install some packages, click OK. If you have any trouble send me [an email](fhoces@berkeley.edu). + +#### Pandoc + Pandoc is a tool to convert between document formats. It should be installed when you install RStudio. + You can test this by typing `which pandoc` into the command line. If you don't get the path to your installed version, you can find it [here](https://pandoc.org/installing.html). + +#### ShareLaTeX +ShareLaTeX allows you to collaboratively edit LaTeX documents. (No more choosing Word because of Tracked Changes). Create a [user account](https://www.sharelatex.com/). diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/README.md b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/README.md new file mode 100644 index 0000000..60b5123 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/README.md @@ -0,0 +1,823 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Annual2017/README.md at master · BITSS/Annual2017 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Skip to content +
+ + + + +
+
+
+
+ Help save net neutrality! + A free, open internet is once again at stake—and we need your help. +
+
+ Learn more + +
+
+
+ + + + + + + + + + +
+ +
+ +
+
+ + + +
+
+
+ + + + + + + + +
+
+ +
    +
  • +
    + +
    + + + +
    +
    +
    + + Notifications +
    + + + +
    +
    +
    +
    +
  • + +
  • + +
    +
    + + + +
    +
    + + + +
    + +
  • + +
  • + + + Fork + + + + + +
  • +
+ +

+ + /Annual2017 + +

+ +
+ + + + +
+ +
+
+ + + Permalink + + + +
+ +
+ + +
+ +
+
+ + Switch branches/tags +
+ +
+
+ +
+
+ +
+
+ + + +
+
+ + +
+ +
Nothing to show
+
+ +
+
+
+ +
+ + Find file + + +
+ +
+ + + +
+ + + aaf3dde + + Dec 5, 2017 + + + +
+ + +
+ + +
+ + +
+
+
+ +
+ Raw + Blame + History +
+ + + + + +
+ +
+ +
+ +
+ 93 lines (54 sloc) + + 4.52 KB +
+
+ + +
+

Intro to Pandoc

+

Garret Christensen

+

Pandoc is a command line tool to convert documents from one type to another. Kind of like Professor Farnsworth. +

+

Installing Pandoc

+

Pandoc is included when you install RStudio, and likely other things too (like the Anaconda installation of Python). +To make sure you have it, type which pandoc or pandoc --version in the command line. If it's not installed, download it here.

+

Using the Command Line

+

The command line is the way to type commands directly to your computer. On a Mac, look in ./Applications/Utilities/Terminal. Windows usually comes with 'Powershell' installed, but I like Git Bash. +If you're using Linux, I'm going to assume you know what you're doing.

+

We'll go over some basic commands, a more complete list is here. Software Carpentry has a nice tutorial here.

+

Just a few of the very basics:

+
    +
  • whoami to check the current user
  • +
  • cd to change directory. Follow with directoryname to go down into a directory, or .. to go up a level
  • +
  • pwd to see the present working directory
  • +
  • ls lists the contents of the current directory.
  • +
+

Reproducibility

+

What does this have to do with transparency or reproducibility?

+

It can help you automate stuff! If you routinely produce a report, and want to put it on the web, or you wrote in LaTeX and a journal doesn't accept that so you need to submit in Word, or if you hate the Microsoft Word Equation Editor (because who doesn't?) Pandoc is your friend.

+

Simple Conversion

+

pandoc --help to see all the options. The general setup is pandoc <existingfile.type> <newfile.type>.

+

Change Output Format

+

By default output is html.

+

So type pandoc militaryrecruiting.tex. But this just outputs the file to the command line.

+

To save the output to a new html file: pandoc militaryrecruiting.tex -o militaryrecruiting.html

+

(The easiest way to open any of your output files is just type open <filename>.)

+

Just change the extension to get whatever output file you like. +pandoc militaryrecruiting.tex -o militaryrecruiting.docx

+

pandoc militaryrecruiting.tex -o militaryrecruiting.odt

+

You'll notice some labels and references don't look so hot immediately, but footnotes are great, and a lot (but not all) of the math is nice.

+

To try Markdown, try converting the README file to html. To go from MS Word to LaTeX, send the Word paper back to TeX. How are the original and re-converted different?

+

References

+

To get the references, tell pandoc where the .bib file is. +pandoc militaryrecruiting.tex -o militaryrecruiting.docx --bibliography=military_bib.bib --csl=chicago-author-date.csl

+

Note that you also have to tell it about the citation style file.

+

Make a Script

+

If you wanted to really automate this, you could make the conversion part of a shell script. Put the command into a text file, which you can do directly, or with the echo command:

+

echo "pandoc militaryrecruiting.tex -o militaryrecruiting.docx --bibliography=military_bib.bib --csl=chicago-author-date.csl">>shellscript.sh

+

Then you can run that script by typing:

+

bash shellscript.sh

+

Or you could run that in Stata with:

+

!bash shellscript.sh

+

Issues

+

Do you have issues? Pandoc is open source and on GitHub.

+

I haven't been able to get complicated tables to work. That's potentially a big limitation.

+

Non-ASCII characters can screw things up. (Apparently, BibTeX doesn't support Unicode.) For example, try converting my "Manual of Best Practices" into HTML.

+

pandoc Manual.tex -o Manual.html --bibliography=test.bib

+

There definitely is a way to fix this!

+

One way would be to use iconv to clean up the .bib file. This should convert a UTF-8 file to ASCII. F is for from, t is for to, and c discards any unconvertable characters.

+

iconv -c -f utf-8 -t ascii test.bib>clean.bib

+

(You could also try using sed, the stream editor: sed -i 's/[\d128-\d255]//g' MYFILE.txt to remove all the non-ASCII characters.)

+

Harness the Full Power of Pandoc

+

Test your powers by seeing if you can convert Jake Bowers & Martin Voors paper "How to improve your relationship with your future self (V 2.0)" +

+
+
+ +
+ + + + +
+ +
+ +
+
+ +
+ + + + + + +
+ + + You can't perform that action at this time. +
+ + + + + + + + + + +
+ + You signed in with another tab or window. Reload to refresh your session. + You signed out in another tab or window. Reload to refresh your session. +
+ + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/chicago-author-date.csl b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/chicago-author-date.csl new file mode 100644 index 0000000..c0ca597 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/chicago-author-date.csl @@ -0,0 +1,3361 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Annual2017/chicago-author-date.csl at master · BITSS/Annual2017 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Skip to content +
+ + + + +
+
+
+
+ Help save net neutrality! + A free, open internet is once again at stake—and we need your help. +
+
+ Learn more + +
+
+
+ + + + + + + + + + +
+ +
+ +
+
+ + + +
+
+
+ + + + + + + + +
+
+ +
    +
  • +
    + +
    + + + +
    +
    +
    + + Notifications +
    + + + +
    +
    +
    +
    +
  • + +
  • + +
    +
    + + + +
    +
    + + + +
    + +
  • + +
  • + + + Fork + + + + + +
  • +
+ +

+ + /Annual2017 + +

+ +
+ + + + +
+ +
+
+ + + Permalink + + + +
+ +
+ + +
+ +
+
+ + Switch branches/tags +
+ +
+
+ +
+
+ +
+
+ + + +
+
+ + +
+ +
Nothing to show
+
+ +
+
+
+ +
+ + Find file + + +
+ +
+ + + +
+ Fetching contributors… +
+ +
+ + Cannot retrieve contributors at this time +
+
+ +
+
+
+ +
+ Raw + Blame + History +
+ + + + + +
+ +
+ +
+ +
+ 651 lines (650 sloc) + + 21.7 KB +
+
+ + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
<?xml version="1.0" encoding="utf-8"?>
<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" version="1.0" demote-non-dropping-particle="never" page-range-format="chicago">
<info>
<title>Chicago Manual of Style 16th edition (author-date)</title>
<id>http://www.zotero.org/styles/chicago-author-date</id>
<link href="http://www.zotero.org/styles/chicago-author-date" rel="self"/>
<link href="http://www.chicagomanualofstyle.org/tools_citationguide.html" rel="documentation"/>
<author>
<name>Julian Onions</name>
<email>julian.onions@gmail.com</email>
</author>
<contributor>
<name>Sebastian Karcher</name>
</contributor>
<contributor>
<name>Richard Karnesky</name>
<email>karnesky+zotero@gmail.com</email>
<uri>http://arc.nucapt.northwestern.edu/Richard_Karnesky</uri>
</contributor>
<contributor>
<name>Andrew Dunning</name>
<email>andrew.dunning@utoronto.ca</email>
</contributor>
<category citation-format="author-date"/>
<category field="generic-base"/>
<summary>The author-date variant of the Chicago style</summary>
<updated>2015-08-26T07:38:25+00:00</updated>
<rights license="http://creativecommons.org/licenses/by-sa/3.0/">This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License</rights>
</info>
<locale xml:lang="en">
<terms>
<term name="editor" form="verb-short">ed.</term>
<term name="container-author" form="verb">by</term>
<term name="translator" form="verb-short">trans.</term>
<term name="editortranslator" form="verb">
<single>edited and translated by</single>
<multiple>edited and translated by</multiple>
</term>
<term name="translator" form="short">trans.</term>
</terms>
</locale>
<macro name="secondary-contributors">
<choose>
<if type="chapter paper-conference" match="none">
<group delimiter=". ">
<names variable="editor translator" delimiter=". ">
<label form="verb" text-case="capitalize-first" suffix=" "/>
<name and="text" delimiter=", "/>
</names>
<names variable="director" delimiter=". ">
<label form="verb" text-case="capitalize-first" suffix=" "/>
<name and="text" delimiter=", "/>
</names>
</group>
</if>
</choose>
</macro>
<macro name="container-contributors">
<choose>
<if type="chapter paper-conference" match="any">
<group prefix=", " delimiter=", ">
<names variable="container-author" delimiter=", ">
<label form="verb" suffix=" "/>
<name and="text" delimiter=", "/>
</names>
<names variable="editor translator" delimiter=", ">
<label form="verb" suffix=" "/>
<name and="text" delimiter=", "/>
</names>
</group>
</if>
</choose>
</macro>
<macro name="editor">
<names variable="editor">
<name name-as-sort-order="first" and="text" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/>
<label form="short" prefix=", "/>
</names>
</macro>
<macro name="translator">
<names variable="translator">
<name name-as-sort-order="first" and="text" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/>
<label form="short" prefix=", "/>
</names>
</macro>
<macro name="recipient">
<choose>
<if type="personal_communication">
<choose>
<if variable="genre">
<text variable="genre" text-case="capitalize-first"/>
</if>
<else>
<text term="letter" text-case="capitalize-first"/>
</else>
</choose>
</if>
</choose>
<names variable="recipient" delimiter=", ">
<label form="verb" prefix=" " text-case="lowercase" suffix=" "/>
<name and="text" delimiter=", "/>
</names>
</macro>
<macro name="substitute-title">
<choose>
<if type="article-magazine article-newspaper review review-book" match="any">
<text macro="container-title"/>
</if>
</choose>
</macro>
<macro name="contributors">
<group delimiter=". ">
<names variable="author">
<name and="text" name-as-sort-order="first" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/>
<label form="short" prefix=", "/>
<substitute>
<names variable="editor"/>
<names variable="translator"/>
<names variable="director"/>
<text macro="substitute-title"/>
<text macro="title"/>
</substitute>
</names>
<text macro="recipient"/>
</group>
</macro>
<macro name="contributors-short">
<names variable="author">
<name form="short" and="text" delimiter=", " initialize-with=". "/>
<substitute>
<names variable="editor"/>
<names variable="translator"/>
<names variable="director"/>
<text macro="substitute-title"/>
<text macro="title"/>
</substitute>
</names>
</macro>
<macro name="interviewer">
<names variable="interviewer" delimiter=", ">
<label form="verb" prefix=" " text-case="capitalize-first" suffix=" "/>
<name and="text" delimiter=", "/>
</names>
</macro>
<macro name="archive">
<group delimiter=". ">
<text variable="archive_location" text-case="capitalize-first"/>
<text variable="archive"/>
<text variable="archive-place"/>
</group>
</macro>
<macro name="access">
<group delimiter=". ">
<choose>
<if type="graphic report" match="any">
<text macro="archive"/>
</if>
<else-if type="article-journal bill book chapter legal_case legislation motion_picture paper-conference" match="none">
<text macro="archive"/>
</else-if>
</choose>
<choose>
<if type="webpage post-weblog" match="any">
<date variable="issued" delimiter=" ">
<date-part name="month"/>
<date-part name="day"/>
</date>
</if>
</choose>
<choose>
<if variable="issued" match="none">
<group delimiter=" ">
<text term="accessed" text-case="capitalize-first"/>
<date variable="accessed" delimiter=" ">
<date-part name="month"/>
<date-part name="day"/>
</date>
</group>
</if>
</choose>
<choose>
<if type="legal_case" match="none">
<choose>
<if variable="DOI">
<text variable="DOI" prefix="doi:"/>
</if>
<else>
<text variable="URL"/>
</else>
</choose>
</if>
</choose>
</group>
</macro>
<macro name="title">
<choose>
<if variable="title" match="none">
<choose>
<if type="personal_communication" match="none">
<text variable="genre" text-case="capitalize-first"/>
</if>
</choose>
</if>
<else-if type="bill book graphic legislation motion_picture song" match="any">
<text variable="title" text-case="title" font-style="italic"/>
<group prefix=" (" suffix=")" delimiter=" ">
<text term="version"/>
<text variable="version"/>
</group>
</else-if>
<else-if variable="reviewed-author">
<choose>
<if variable="reviewed-title">
<group delimiter=". ">
<text variable="title" text-case="title" quotes="true"/>
<group delimiter=", ">
<text variable="reviewed-title" text-case="title" font-style="italic" prefix="Review of "/>
<names variable="reviewed-author">
<label form="verb-short" text-case="lowercase" suffix=" "/>
<name and="text" delimiter=", "/>
</names>
</group>
</group>
</if>
<else>
<group delimiter=", ">
<text variable="title" text-case="title" font-style="italic" prefix="Review of "/>
<names variable="reviewed-author">
<label form="verb-short" text-case="lowercase" suffix=" "/>
<name and="text" delimiter=", "/>
</names>
</group>
</else>
</choose>
</else-if>
<else-if type="legal_case interview patent" match="any">
<text variable="title"/>
</else-if>
<else>
<text variable="title" text-case="title" quotes="true"/>
</else>
</choose>
</macro>
<macro name="edition">
<choose>
<if type="bill book graphic legal_case legislation motion_picture report song" match="any">
<choose>
<if is-numeric="edition">
<group delimiter=" " prefix=". ">
<number variable="edition" form="ordinal"/>
<text term="edition" form="short" strip-periods="true"/>
</group>
</if>
<else>
<text variable="edition" text-case="capitalize-first" prefix=". "/>
</else>
</choose>
</if>
<else-if type="chapter paper-conference" match="any">
<choose>
<if is-numeric="edition">
<group delimiter=" " prefix=", ">
<number variable="edition" form="ordinal"/>
<text term="edition" form="short"/>
</group>
</if>
<else>
<text variable="edition" prefix=", "/>
</else>
</choose>
</else-if>
</choose>
</macro>
<macro name="locators">
<choose>
<if type="article-journal">
<choose>
<if variable="volume">
<text variable="volume" prefix=" "/>
<group prefix=" (" suffix=")">
<choose>
<if variable="issue">
<text variable="issue"/>
</if>
<else>
<date variable="issued">
<date-part name="month"/>
</date>
</else>
</choose>
</group>
</if>
<else-if variable="issue">
<group delimiter=" " prefix=", ">
<text term="issue" form="short"/>
<text variable="issue"/>
<date variable="issued" prefix="(" suffix=")">
<date-part name="month"/>
</date>
</group>
</else-if>
<else>
<date variable="issued" prefix=", ">
<date-part name="month"/>
</date>
</else>
</choose>
</if>
<else-if type="legal_case">
<text variable="volume" prefix=", "/>
<text variable="container-title" prefix=" "/>
<text variable="page" prefix=" "/>
</else-if>
<else-if type="bill book graphic legal_case legislation motion_picture report song" match="any">
<group prefix=". " delimiter=". ">
<group>
<text term="volume" form="short" text-case="capitalize-first" suffix=" "/>
<number variable="volume" form="numeric"/>
</group>
<group>
<number variable="number-of-volumes" form="numeric"/>
<text term="volume" form="short" prefix=" " plural="true"/>
</group>
</group>
</else-if>
<else-if type="chapter paper-conference" match="any">
<choose>
<if variable="page" match="none">
<group prefix=". ">
<text term="volume" form="short" text-case="capitalize-first" suffix=" "/>
<number variable="volume" form="numeric"/>
</group>
</if>
</choose>
</else-if>
</choose>
</macro>
<macro name="locators-chapter">
<choose>
<if type="chapter paper-conference" match="any">
<choose>
<if variable="page">
<group prefix=", ">
<text variable="volume" suffix=":"/>
<text variable="page"/>
</group>
</if>
</choose>
</if>
</choose>
</macro>
<macro name="locators-article">
<choose>
<if type="article-newspaper">
<group prefix=", " delimiter=", ">
<group delimiter=" ">
<text variable="edition"/>
<text term="edition"/>
</group>
<group>
<text term="section" form="short" suffix=" "/>
<text variable="section"/>
</group>
</group>
</if>
<else-if type="article-journal">
<choose>
<if variable="volume issue" match="any">
<text variable="page" prefix=": "/>
</if>
<else>
<text variable="page" prefix=", "/>
</else>
</choose>
</else-if>
</choose>
</macro>
<macro name="point-locators">
<choose>
<if variable="locator">
<choose>
<if locator="page" match="none">
<choose>
<if type="bill book graphic legal_case legislation motion_picture report song" match="any">
<choose>
<if variable="volume">
<group>
<text term="volume" form="short" suffix=" "/>
<number variable="volume" form="numeric"/>
<label variable="locator" form="short" prefix=", " suffix=" "/>
</group>
</if>
<else>
<label variable="locator" form="short" suffix=" "/>
</else>
</choose>
</if>
<else>
<label variable="locator" form="short" suffix=" "/>
</else>
</choose>
</if>
<else-if type="bill book graphic legal_case legislation motion_picture report song" match="any">
<number variable="volume" form="numeric" suffix=":"/>
</else-if>
</choose>
<text variable="locator"/>
</if>
</choose>
</macro>
<macro name="container-prefix">
<text term="in" text-case="capitalize-first"/>
</macro>
<macro name="container-title">
<choose>
<if type="chapter paper-conference" match="any">
<text macro="container-prefix" suffix=" "/>
</if>
</choose>
<choose>
<if type="legal_case" match="none">
<text variable="container-title" text-case="title" font-style="italic"/>
</if>
</choose>
</macro>
<macro name="publisher">
<group delimiter=": ">
<text variable="publisher-place"/>
<text variable="publisher"/>
</group>
</macro>
<macro name="date">
<choose>
<if variable="issued">
<group delimiter=" ">
<date variable="original-date" form="text" date-parts="year" prefix="(" suffix=")"/>
<date variable="issued">
<date-part name="year"/>
</date>
</group>
</if>
<else-if variable="accessed">
<date variable="accessed">
<date-part name="year"/>
</date>
</else-if>
<else-if variable="status">
<text variable="status" text-case="capitalize-first"/>
</else-if>
<else>
<text term="no date" form="short"/>
</else>
</choose>
</macro>
<macro name="date-in-text">
<choose>
<if variable="issued">
<group delimiter=" ">
<date variable="original-date" form="text" date-parts="year" prefix="[" suffix="]"/>
<date variable="issued">
<date-part name="year"/>
</date>
</group>
</if>
<else-if variable="accessed">
<date variable="accessed">
<date-part name="year"/>
</date>
</else-if>
<else-if variable="status">
<text variable="status"/>
</else-if>
<else>
<text term="no date" form="short"/>
</else>
</choose>
</macro>
<macro name="day-month">
<date variable="issued">
<date-part name="month"/>
<date-part name="day" prefix=" "/>
</date>
</macro>
<macro name="collection-title">
<choose>
<if match="none" type="article-journal">
<choose>
<if match="none" is-numeric="collection-number">
<group delimiter=", ">
<text variable="collection-title" text-case="title"/>
<text variable="collection-number"/>
</group>
</if>
<else>
<group delimiter=" ">
<text variable="collection-title" text-case="title"/>
<text variable="collection-number"/>
</group>
</else>
</choose>
</if>
</choose>
</macro>
<macro name="collection-title-journal">
<choose>
<if type="article-journal">
<group delimiter=" ">
<text variable="collection-title"/>
<text variable="collection-number"/>
</group>
</if>
</choose>
</macro>
<macro name="event">
<group>
<text term="presented at" suffix=" "/>
<text variable="event"/>
</group>
</macro>
<macro name="description">
<choose>
<if type="interview">
<group delimiter=". ">
<text macro="interviewer"/>
<text variable="medium" text-case="capitalize-first"/>
</group>
</if>
<else-if type="patent">
<group delimiter=" " prefix=". ">
<text variable="authority"/>
<text variable="number"/>
</group>
</else-if>
<else>
<text variable="medium" text-case="capitalize-first" prefix=". "/>
</else>
</choose>
<choose>
<if variable="title" match="none"/>
<else-if type="thesis personal_communication speech" match="any"/>
<else>
<group delimiter=" " prefix=". ">
<text variable="genre" text-case="capitalize-first"/>
<choose>
<if type="report">
<text variable="number"/>
</if>
</choose>
</group>
</else>
</choose>
</macro>
<macro name="issue">
<choose>
<if type="legal_case">
<text variable="authority" prefix=". "/>
</if>
<else-if type="speech">
<group prefix=". " delimiter=", ">
<group delimiter=" ">
<text variable="genre" text-case="capitalize-first"/>
<text macro="event"/>
</group>
<text variable="event-place"/>
<text macro="day-month"/>
</group>
</else-if>
<else-if type="article-newspaper article-magazine personal_communication" match="any">
<text macro="day-month" prefix=", "/>
</else-if>
<else-if type="patent">
<group delimiter=", " prefix=", ">
<group delimiter=" ">
<!--Needs Localization-->
<text value="filed"/>
<date variable="submitted" form="text"/>
</group>
<group delimiter=" ">
<choose>
<if variable="issued submitted" match="all">
<text term="and"/>
</if>
</choose>
<!--Needs Localization-->
<text value="issued"/>
<date variable="issued" form="text"/>
</group>
</group>
</else-if>
<else>
<group prefix=". " delimiter=", ">
<choose>
<if type="thesis">
<text variable="genre" text-case="capitalize-first"/>
</if>
</choose>
<text macro="publisher"/>
</group>
</else>
</choose>
</macro>
<citation et-al-min="4" et-al-use-first="1" disambiguate-add-year-suffix="true" disambiguate-add-names="true" disambiguate-add-givenname="true" givenname-disambiguation-rule="primary-name">
<layout prefix="(" suffix=")" delimiter="; ">
<group delimiter=", ">
<choose>
<if variable="issued accessed" match="any">
<group delimiter=" ">
<text macro="contributors-short"/>
<text macro="date-in-text"/>
</group>
</if>
<!---comma before forthcoming and n.d.-->
<else>
<group delimiter=", ">
<text macro="contributors-short"/>
<text macro="date-in-text"/>
</group>
</else>
</choose>
<text macro="point-locators"/>
</group>
</layout>
</citation>
<bibliography hanging-indent="true" et-al-min="11" et-al-use-first="7" subsequent-author-substitute="&#8212;&#8212;&#8212;" entry-spacing="0">
<sort>
<key macro="contributors"/>
<key variable="issued"/>
<key variable="title"/>
</sort>
<layout suffix=".">
<group delimiter=". ">
<text macro="contributors"/>
<text macro="date"/>
<text macro="title"/>
</group>
<text macro="description"/>
<text macro="secondary-contributors" prefix=". "/>
<text macro="container-title" prefix=". "/>
<text macro="container-contributors"/>
<text macro="edition"/>
<text macro="locators-chapter"/>
<text macro="collection-title-journal" prefix=", " suffix=", "/>
<text macro="locators"/>
<text macro="collection-title" prefix=". "/>
<text macro="issue"/>
<text macro="locators-article"/>
<text macro="access" prefix=". "/>
</layout>
</bibliography>
</style>
+ + + +
+ +
+ + + + +
+ +
+ +
+
+ +
+ + + + + + +
+ + + You can't perform that action at this time. +
+ + + + + + + + + + +
+ + You signed in with another tab or window. Reload to refresh your session. + You signed out in another tab or window. Reload to refresh your session. +
+ + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/military_bib.bib b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/military_bib.bib new file mode 100644 index 0000000..5985a10 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/military_bib.bib @@ -0,0 +1,3970 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Annual2017/military_bib.bib at master · BITSS/Annual2017 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Skip to content +
+ + + + +
+
+
+
+ Help save net neutrality! + A free, open internet is once again at stake—and we need your help. +
+
+ Learn more + +
+
+
+ + + + + + + + + + +
+ +
+ +
+
+ + + +
+
+
+ + + + + + + + +
+
+ +
    +
  • +
    + +
    + + + +
    +
    +
    + + Notifications +
    + + + +
    +
    +
    +
    +
  • + +
  • + +
    +
    + + + +
    +
    + + + +
    + +
  • + +
  • + + + Fork + + + + + +
  • +
+ +

+ + /Annual2017 + +

+ +
+ + + + +
+ +
+
+ + + Permalink + + + +
+ +
+ + +
+ +
+
+ + Switch branches/tags +
+ +
+
+ +
+
+ +
+
+ + + +
+
+ + +
+ +
Nothing to show
+
+ +
+
+
+ +
+ + Find file + + +
+ +
+ + + +
+ Fetching contributors… +
+ +
+ + Cannot retrieve contributors at this time +
+
+ +
+
+
+ +
+ Raw + Blame + History +
+ + + + + +
+ +
+ +
+ +
+ 768 lines (624 sloc) + + 33.3 KB +
+
+ + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
% This file was created with JabRef 2.10.
% Encoding: ISO8859_5
+
+
@Article{negbinom,
Title = {Fixed--effects negative binomial regression models},
Author = {Allison, Paul D and Waterman, Richard P},
Journal = {Sociological methodology},
Year = {2002},
Number = {1},
Pages = {247--265},
Volume = {32},
+
Publisher = {Wiley Online Library}
}
+
@Article{Altman-JHR-1969,
Title = {Earnings, Unemployment, and the Supply of Enlisted, Volunteers},
Author = {Stuart H. Altman},
Journal = {The Journal of Human Resources},
Year = {1969},
+
Month = {Winter},
Number = {1},
Pages = {38-59},
Volume = {4},
+
Date-added = {2010-09-17 16:51:16 -0700},
Date-modified = {2010-09-17 16:52:31 -0700}
}
+
@Article{Altman-Fechter-AER-1967,
Title = {The Supply of Military Personnel in the Absence of a Draft},
Author = {Stuart H. Altman and Alan E. Fechter},
Journal = {The American Economic Review},
Year = {1967},
+
Month = {May},
Number = {2},
Pages = {19-31},
Volume = {57},
+
Date-added = {2010-09-17 15:29:27 -0700},
Date-modified = {2010-09-17 15:31:22 -0700}
}
+
@Article{Anderson2008,
Title = {Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects},
Author = {Michael L. Anderson},
Journal = {Journal of the American Statistical Association},
Year = {2008},
Number = {484},
Pages = {1481-1495},
Volume = {103},
+
Doi = {10.1198/016214508000000841},
Eprint = {
http://dx.doi.org/10.1198/016214508000000841
},
Url = {
http://dx.doi.org/10.1198/016214508000000841
}
}
+
@Article{DMAsource2,
Title = {How campaigns respond to media prices: A study of campaign spending and broadcast advertising prices in us house elections, 1970-1972 and 1990-1992},
Author = {Ansolabehere, Stephen and Gerber, Alan and Snyder Jr, James M},
Journal = {Unpublished manuscript},
Year = {1999},
+
Date-added = {2014-04-21 19:55:56 +0000},
Date-modified = {2014-04-21 19:56:08 +0000}
}
+
@Article{Ansolabehere2006,
Title = {Television and the incumbency advantage in US elections},
Author = {Ansolabehere, Stephen and Snowberg, Erik C and Snyder, James M},
Journal = {Legislative Studies Quarterly},
Year = {2006},
Number = {4},
Pages = {469--490},
Volume = {31},
+
Date-added = {2014-04-21 19:54:17 +0000},
Date-modified = {2014-04-21 19:54:36 +0000},
Publisher = {Wiley Online Library}
}
+
@Unpublished{Heaton-Asch,
Title = {Monopsony and Labor Supply in the Army and Navy},
Author = {Beth Asch and Paul Heaton},
Note = {Working Paper Number 537, Princeton University Industrial Relations Section},
+
Month = {October},
Year = {2008},
+
Date-added = {2010-08-27 17:38:50 -0700},
Date-modified = {2010-09-25 23:29:53 -0700}
}
+
@Article{Asch-Incentives,
Title = {Do Incentives Matter? The Case of Navy Recruiters},
Author = {Beth J. Asch},
Journal = {Industrial and Labor Relations Review},
Year = {1990},
+
Month = {February},
Pages = {89S-106S},
Volume = {43},
+
Date-added = {2010-09-18 21:18:19 -0700},
Date-modified = {2010-09-18 21:20:05 -0700}
}
+
@Misc{BigRandDocument,
Title = {Cash Incentives and Military Enlistments, Attrition and Reenlistment},
+
Author = {Beth J. Asch and Paul Heaton and James Hosek and Francisco Martorell and Curtis Simon and John T. Warner},
HowPublished = {Monograph},
Month = {June},
Year = {2010},
+
Date-added = {2010-09-25 22:45:21 -0700},
Date-modified = {2010-09-25 22:48:12 -0700}
}
+
@Article{Ash-Udis-McNown-AER,
Title = {Enlistments in the All-Volunteer Force: A Military Personnel Supply Model and Its Forecasts},
Author = {Colin Ash and Bernard Udis and Robert F. McNown},
Journal = {The American Economic Review},
Year = {1983},
+
Month = {March},
Number = {1},
Pages = {145-155},
Volume = {73},
+
Date-added = {2010-09-17 22:16:32 -0700},
Date-modified = {2010-09-17 23:08:52 -0700}
}
+
@Article{BenHoch1995,
Title = {Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing},
Author = {Benjamini, Yoav AND Hochberg, Yosef},
Journal = {Journal of the Royal Statistical Society. Series B (Methodological)},
Year = {1995},
Number = {1},
Pages = {289-300},
Volume = {57},
+
Abstract = {The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach to problems of multiple significance testing is presented. It calls for controlling the expected proportion of falsely rejected hypotheses-the false discovery rate. This error rate is equivalent to the FWER when all hypotheses are true but is smaller otherwise. Therefore, in problems where the control of the false discovery rate rather than that of the FWER is desired, there is potential for a gain in power. A simple sequential Bonferroni-type procedure is proved to control the false discovery rate for independent test statistics, and a simulation study shows that the gain in power is substantial. The use of the new procedure and the appropriateness of the criterion are illustrated with examples.},
ISSN = {00359246},
Publisher = {[Royal Statistical Society, Wiley]},
Url = {http://www.jstor.org/stable/2346101}
}
+
@Article{BKY2006,
Title = {Adaptive linear step-up procedures that control the false discovery rate},
Author = {Benjamini, Yoav and Krieger, Abba M. and Yekutieli, Daniel},
Journal = {Biometrika},
Year = {2006},
Number = {3},
Pages = {491-507},
Volume = {93},
+
Abstract = {The linear step-up multiple testing procedure controls the false discovery rate at the desired level q for independent and positively dependent test statistics. When all null hypotheses are true, and the test statistics are independent and continuous, the bound is sharp. When some of the null hypotheses are not true, the procedure is conservative by a factor which is the proportion m0/m of the true null hypotheses among the hypotheses. We provide a new two-stage procedure in which the linear step-up procedure is used in stage one to estimate m0, providing a new level q' which is used in the linear step-up procedure in the second stage. We prove that a general form of the two-stage procedure controls the false discovery rate at the desired level q. This framework enables us to study analytically the properties of other procedures that exist in the literature. A simulation study is presented that shows that two-stage adaptive procedures improve in power over the original procedure, mainly because they provide tighter control of the false discovery rate. We further study the performance of the current suggestions, some variations of the procedures, and previous suggestions, in the case where the test statistics are positively dependent, a case for which the original procedure controls the false discovery rate. In the setting studied here the newly proposed two-stage procedure is the only one that controls the false discovery rate. The procedures are illustrated with two examples of biological importance.},
Doi = {10.1093/biomet/93.3.491},
Eprint = {http://biomet.oxfordjournals.org/content/93/3/491.full.pdf+html},
Url = {http://biomet.oxfordjournals.org/content/93/3/491.abstract}
}
+
@Article{BiddleZarkin,
Title = {Worker Preference and Market Compensation for Job Risk},
Author = {Biddle, Jeff E. and Zarkin, Gary A.},
Journal = {The Review of Economics and Statistics},
Year = {1988},
Number = {4},
Pages = {pp. 660-667},
Volume = {70},
+
Abstract = {Workers choose a job and receive in return a bundle consisting of income and a probability of job injury. We view this income-job risk bundle chosen by the worker as being exchanged in an implicit market. By jointly estimating the market income-job risk locus and the optimum conditions for utility maximization, we are able to identify the markets locus and parameters of the workers' utility function. In contrast to previous work, we are able to derive valuations of discrete changes in job risk for each individual in the sample. We present evidence that an increase in nonlabor income leads workers to select safer jobs.},
Copyright = {Copyright 1988 The MIT Press},
ISSN = {00346535},
Jstor_articletype = {research-article},
Jstor_formatteddate = {Nov., 1988},
Language = {English},
Publisher = {The MIT Press},
Url = {http://www.jstor.org/stable/1935830}
}
+
@Article{Charles-Brown-Unemployment,
Title = {Military Enlistments: What Can We Learn from Geographic Variation},
Author = {Charles Brown},
Journal = {The American Economic Review},
Year = {1985},
+
Month = {March},
Number = {1},
Pages = {228-234},
Volume = {75},
+
Date-added = {2010-09-17 22:10:40 -0700},
Date-modified = {2010-09-17 22:12:20 -0700}
}
+
@Article{eBayFinal,
Title = {Shrouded Attributes and Information Supression: Evidence from the Field},
Author = {Jennifer Brown and Tanjim Hossain and John Morgan},
Journal = {The Quarterly Journal of Economics},
Year = {2010},
+
Month = {May},
Pages = {859-876},
+
Date-added = {2010-11-16 22:08:08 -0800},
Date-modified = {2010-11-16 22:09:42 -0800}
}
+
@Book{cameron2013countdata,
Title = {Regression analysis of count data},
Author = {Cameron, A Colin and Trivedi, Pravin K},
Publisher = {Cambridge university press},
Year = {2013},
Volume = {53}
}
+
@Article{ChettySalience,
Title = {Salience and Taxation: Theory and Evidence},
Author = {Raj Chetty and Adam Looney and Kory Kroft},
Journal = {The American Economic Review},
Year = {2009},
Number = {4},
Pages = {1145-1177},
Volume = {99},
+
Date-added = {2010-11-16 22:09:50 -0800},
Date-modified = {2010-11-16 22:11:16 -0800}
}
+
@Unpublished{ChettyEarly,
Title = {Salience and Taxation: Theory and Evidence},
Author = {Raj Chetty and Adam Looney and Kory Kroft},
Year = {2007},
+
Date-added = {2011-01-16 13:54:31 -0800},
Date-modified = {2011-01-16 13:55:49 -0800},
Journal = {NBER WP 13330}
}
+
@Misc{AfghanIraqCasualties,
Title = {The Effect of Civilian Casualties in Afghanistan and Iraq},
+
Author = {Luke N. Condra and Joseph H. Felter and Radha K. Iyengar and Jacob N. Shapiro},
HowPublished = {NBER Working Paper 16152},
Month = {July},
Year = {2010},
+
Date-added = {2010-09-22 13:53:20 -0700},
Date-modified = {2010-09-22 13:55:13 -0700}
}
+
@Electronic{reghdfe,
Title = {reghdfe: Stata module for linear and instrumental-variable/GMM regression absorbing multiple levels of fixed effects.},
Author = {Sergio Correia},
Url = {https://ideas.repec.org/c/boc/bocode/s457874.html},
Year = {2015},
+
Owner = {garret},
Timestamp = {2016.06.24}
}
+
@Article{cunha2014expedited,
Title = {Expedited citizenship for sale: estimating the effect of Executive Order 13269 on noncitizen military enlistments},
Author = {Cunha, Jesse M and Sullivan, Ryan and Can, Melih and Yalcinkaya, Huseyin},
Journal = {Applied Economics},
Year = {2014},
Number = {11},
Pages = {1291--1300},
Volume = {46},
+
Publisher = {Taylor \& Francis}
}
+
@Article{Dale-Gilroy-AER-Note-1985,
Title = {Enlistments in the All-Volunteer Force: Note},
Author = {Charles Dale and Curtis Gilroy},
Journal = {The American Economic Review},
Year = {1985},
+
Month = {June},
Number = {3},
Pages = {547-551},
Volume = {75},
+
Date-added = {2010-09-17 22:28:10 -0700},
Date-modified = {2010-09-17 22:29:21 -0700}
}
+
@Article{DaleGilroy1984,
Title = {Determinants of Enlistments: A Macroeconomic Time-Series View},
Author = {Charles Dale and Curtis Gilroy},
Journal = {Armed Forces and Society},
Year = {1984},
+
Month = {Winter},
Number = {2},
Pages = {192-210},
Volume = {10},
+
Date-added = {2010-09-17 17:15:35 -0700},
Date-modified = {2010-09-19 19:26:29 -0700}
}
+
@Article{DoDBudgetPR,
Title = {Overview of the DOD Fiscal 2010 Budget Proposal},
Author = {Department of Defense},
Year = {2009},
+
Bdsk-url-1 = {http://www.defenselink.mil/news/2010%20Budget%20Proposal.pdf},
Date-added = {2010-08-27 17:02:38 -0700},
Date-modified = {2010-08-27 17:09:49 -0700},
LastChecked = {8/27/10},
Url = {http://www.defenselink.mil/news/2010%20Budget%20Proposal.pdf}
}
+
@Article{DeLeireLevy2004,
Title = {Worker Sorting and the Risk of Death on the Job},
Author = {Thomas DeLeire and Helen Levy},
Journal = {Journal of Labor Economics},
Year = {2004},
Number = {4},
Pages = {pp. 925-953},
Volume = {22},
+
Abstract = {This article examines worker sorting across occupations in response to the risk of death on the job. We use family structure as a proxy for willingness to trade safety for wages to test the proposition that workers with strong aversion to this risk sort into safer jobs. We estimate conditional logit models of occupation choice as a function of injury risk and other job attributes. Our results confirm the sorting hypothesis: within gender, single moms and dads are the most averse to risk. Overall, differences in the risk of death across occupations explain about one-quarter of occupational gender segregation.},
Copyright = {Copyright 2004 The University of Chicago},
ISSN = {0734306X},
Jstor_articletype = {research-article},
Jstor_formatteddate = {October 2004},
Language = {English},
Publisher = {The University of Chicago Press on behalf of the Society of Labor Economists and the NORC at the University of Chicago},
Url = {http://www.jstor.org/stable/10.1086/423159}
}
+
@Article{StefanoPsychReview,
Title = {Psychology and Economics: Evidence from the Field},
Author = {Stefano DellaVigna},
Journal = {Journal of Economic Literature},
Year = {2009},
Number = {2},
Pages = {315-372},
Volume = {47},
+
Date-added = {2010-11-16 23:00:43 -0800},
Date-modified = {2010-11-16 23:04:26 -0800}
}
+
@Misc{Dertouzos-RAND-1985,
Title = {Recruiter Incentives and Enlistment Supply},
+
Author = {James N. Dertouzos},
HowPublished = {Rand Corporation},
Month = {May},
Year = {1985},
+
Date-added = {2010-09-17 15:31:36 -0700},
Date-modified = {2010-09-17 15:34:09 -0700}
}
+
@Misc{Dertouzous-Polich-Army-Ads,
Title = {Recruiting Effects of Army Advertising},
+
Author = {James N. Dertouzos and J. Michael Polich},
HowPublished = {RAND National Defense Research Institute},
Month = {January},
Year = {1989},
+
Date-added = {2010-09-18 21:20:16 -0700},
Date-modified = {2010-09-18 21:22:44 -0700}
}
+
@Other{Budget2010NYT,
Title = {Victory for Obama over Military Lobby},
Author = {Christopher Drew},
Bdsk-url-1 = {http://www.nytimes.com/2009/10/29/business/29defense.html},
Date-added = {2010-08-27 16:46:59 -0700},
Date-modified = {2010-08-27 16:49:00 -0700},
Journal = {New York Times},
LastChecked = {8/27/10},
Url = {http://www.nytimes.com/2009/10/29/business/29defense.html},
Year = {2010}
}
+
@Article{FisherDraftModelAER1969,
Title = {The Cost of the Draft and the Cost of Ending the Draft},
Author = {Anthony C. Fisher},
Journal = {The American Economic Review},
Year = {1969},
+
Month = {June},
Number = {3},
Pages = {239-254},
Volume = {59},
+
Date-added = {2011-01-16 12:18:45 -0800},
Date-modified = {2011-01-16 12:20:06 -0800}
}
+
@Article{garen1988compensating,
Title = {Compensating wage differentials and the endogeneity of job riskiness},
Author = {Garen, John},
Journal = {The Review of Economics and Statistics},
Year = {1988},
Pages = {9--16},
+
Publisher = {JSTOR}
}
+
@Article{SSGartner-WarCasualtiesPublicOpinion,
Title = {War, Casualties, and Public Opinion},
Author = {Scott Sigmund Gartner and Gary M. Segura},
Journal = {The Journal of Conflict Resolution},
Year = {1998},
+
Month = {June},
Number = {3},
Pages = {278-300},
Volume = {42},
+
Date-added = {2010-09-19 18:58:26 -0700},
Date-modified = {2010-09-19 19:07:47 -0700}
}
+
@Article{SSGartnerAllPoliticsLocal,
Title = {All Politics are Local: Local Losses and Individual Attitudes toward the Vietnam War},
Author = {Scott Sigmund Gartner and Gary M. Segura and Michael Wilkening},
Journal = {The Journal of Conflict Resolution},
Year = {1997},
+
Month = {October},
Number = {5},
Pages = {669-694},
Volume = {41},
+
Date-added = {2010-09-19 19:07:53 -0700},
Date-modified = {2010-09-19 19:09:06 -0700}
}
+
@Unpublished{Gelber,
Title = {The Supply of Military Enlistments},
Author = {Alexander M. Gelber},
+
Month = {January},
Year = {2007},
+
Date-added = {2010-08-27 17:38:13 -0700},
Date-modified = {2010-08-27 17:38:47 -0700}
}
+
@Article{GlaeserSocialNetworks,
Title = {An Economic Approach to Social Capital},
Author = {Edward L. Glaeser and David Laibson and Bruce Sacerdote},
Journal = {The Economic Journal},
Year = {2002},
+
Month = {November},
Pages = {F437-F458},
Volume = {112},
+
Date-added = {2011-01-10 22:49:43 -0800},
Date-modified = {2011-01-10 22:51:36 -0800}
}
+
@Article{Hanssens-Levien-Advertising,
Title = {An Econometric Study of Recruitment Marketing in the U.S. Navy},
Author = {Dominique M. Hanssens and Henry A. Levien},
Journal = {Management Science},
Year = {1983},
+
Month = {October},
Number = {10},
Pages = {1167-1184},
Volume = {29},
+
Date-added = {2010-09-17 21:58:26 -0700},
Date-modified = {2010-09-17 22:00:01 -0700}
}
+
@Article{Hirshleifer-Lim-Teoh,
Title = {Driven to Distraction: Extraneous Events and Underreaction to Earnings News},
Author = {David Hirshleifer and Sonya Seongyeon Lim and Siew Hong Teoh},
Journal = {The Journal of Finance},
Year = {2009},
+
Month = {October},
Number = {5},
Pages = {2289-2325},
Volume = {64},
+
Date-added = {2010-11-16 22:15:58 -0800},
Date-modified = {2010-11-16 22:17:50 -0800}
}
+
@Article{eBayEarly,
Title = {...Plus Shipping and Handling: Revenue (Non) Equivalence in Field Experiments on eBay},
Author = {Tanjim Hossain and John Morgan},
Journal = {Advances in Economic Analysis and Policy},
Year = {2006},
Number = {2},
Volume = {6},
+
Date-added = {2010-11-16 22:13:36 -0800},
Date-modified = {2010-11-16 23:15:45 -0800}
}
+
@Article{WarnerRecProductivitySlowdown,
Title = {The Military Recruiting Productivity Slowdown: The Roles of Resources, Opportunity Cost and the Tastes of Youth},
Author = {John T. Warner, Curtis J. Simon and Deborah M. Payne},
Journal = {Defence and Peace Economics},
Year = {2003},
+
Month = {October},
Number = {5},
Pages = {329-342},
Volume = {14},
+
Date-added = {2010-09-13 16:22:55 -0700},
Date-modified = {2010-09-13 16:29:10 -0700}
}
+
@Article{kahn1987occupational,
Title = {Occupational safety and worker preferences: is there a marginal worker?},
Author = {Kahn, Shulamit},
Journal = {The Review of Economics and Statistics},
Year = {1987},
Pages = {262--268},
+
Publisher = {JSTOR}
}
+
@Article{Ted-Miguel-Bush-Deaths,
Title = {The Electoral Cost of War: Iraq Casualties and the 2004 U.S. Presidential Election},
Author = {David Karol and Edward Miguel},
Journal = {The Journal of Politics},
Year = {2007},
+
Month = {August},
Number = {3},
Pages = {633-648},
Volume = {69},
+
Date-added = {2010-09-19 18:01:16 -0700},
Date-modified = {2010-09-19 18:02:44 -0700}
}
+
@Article{Kocher-Vietnam,
Title = {Aerial Bombin and Counterinsurgency in the Vietnam War},
Author = {Matthew Adam Kocher and Thomas B. Pepinsky and Stathis N Kalyvas},
Journal = {American Journal of Political Science},
Year = {2011},
Pages = {201-218},
Volume = {55},
+
Owner = {Garret},
Timestamp = {2012.12.12}
}
+
@Article{Lyall,
Title = {Does Indiscriminate Violence Incite Insurgent Attacks? Evidence from Chechnya},
Author = {Jason Lyall},
Journal = {Journal of Conflict Resolution},
Year = {2009},
Pages = {331-362},
Volume = {53},
+
Owner = {Garret},
Timestamp = {2012.12.12}
}
+
@Article{Budget2010,
Title = {A New Era of Responsibility: Renewing America's Promise},
Author = {Office of Management and Budget},
Journal = {FY2010 Federal Budget Proposal},
Year = {2010},
+
Date-added = {2010-08-27 16:17:23 -0700},
Date-modified = {2010-08-27 16:41:05 -0700}
}
+
@Article{McNown-Udis-Ash-AF+S,
Title = {Economic Analysis of the All-Volunteer Force},
Author = {Robert F. McNown and Bernard Udis and Colin Ash},
Journal = {Armed Forces and Society},
Year = {1980},
+
Month = {Fall},
Number = {1},
Pages = {113-132},
Volume = {7},
+
Date-added = {2010-09-17 23:06:18 -0700},
Date-modified = {2010-09-19 19:26:44 -0700}
}
+
@Misc{Oken-Asch-RewardHistory,
Title = {Encouraging Recruiter Achievement: A Recent History of Military Recruiter Incentive Programs},
+
Author = {Carole Oken and Beth J. Asch},
HowPublished = {RAND National Defense Research Institute},
Year = {1997},
+
Date-added = {2010-09-19 19:27:49 -0700},
Date-modified = {2010-09-19 19:29:22 -0700}
}
+
@InCollection{RosenOccChoiceHandbook,
Title = {The Theory of Equalizing Differences},
Author = {Sherwin Rosen},
Booktitle = {Hanbook of Labor Economics},
Publisher = {Elsevier Science Publishers},
Year = {1986},
Chapter = {12},
Editor = {O. Ashenfelter and R. Layard},
Volume = {1},
+
Date-added = {2011-01-16 12:01:37 -0800},
Date-modified = {2011-01-16 12:04:08 -0800}
}
+
@Article{rosen1986theory,
Title = {The theory of equalizing differences},
Author = {Rosen, Sherwin},
Journal = {Handbook of labor economics},
Year = {1986},
Pages = {641--692},
Volume = {1},
+
Publisher = {Elsevier}
}
+
@Article{Roy,
Title = {Some Thoughts on the Distribution of Earnings},
Author = {A. D. Roy},
Journal = {Oxford Economic Papers},
Year = {1951},
+
Month = {June},
Number = {2},
Pages = {135-146},
Volume = {3},
+
Date-added = {2011-01-23 18:18:51 -0800},
Date-modified = {2011-01-23 18:20:00 -0800}
}
+
@Article{Simon-Warner-DEATHS!,
Title = {Managing the all-volunteer force in a time of war},
Author = {Curtis J. Simon and John T. Warner},
Journal = {The Economics of Peace and Security Journal},
Year = {2007},
Number = {1},
Pages = {20-29},
Volume = {2},
+
Date-added = {2010-09-25 23:20:33 -0700},
Date-modified = {2010-09-25 23:22:36 -0700}
}
+
@Article{SimonEarlyLimitedInfo,
Title = {A Behavioral Model of Rational Choice},
Author = {Herbert A. Simon},
Journal = {The Quarterly Journal of Economics},
Year = {1955},
+
Month = {February},
Number = {1},
Pages = {99-118},
Volume = {69},
+
Date-added = {2010-11-16 22:14:58 -0800},
Date-modified = {2010-11-16 22:15:49 -0800}
}
+
@Book{AdamSmith,
Title = {The Wealth of Nations},
Author = {Adam Smith},
Publisher = {Bantam Classics, New York},
Year = {1776/2003},
+
Owner = {garret},
Timestamp = {2014.12.06}
}
+
@Electronic{PersonnelNumber,
Title = {Military Personnel Statistics},
Author = {Statistical Information Analysis Division, Department of Defense},
Url = {http://siadapp.dmdc.osd.mil/personnel/MILITARY/miltop.htm},
Year = {2010},
+
Bdsk-url-1 = {http://siadapp.dmdc.osd.mil/personnel/MILITARY/miltop.htm},
Date-added = {2010-08-27 18:24:00 -0700},
Date-modified = {2010-08-27 18:26:27 -0700},
LastChecked = {8/27/10}
}
+
@InCollection{thaler1976value,
Title = {The value of saving a life: evidence from the labor market},
Author = {Thaler, Richard and Rosen, Sherwin},
Booktitle = {Household production and consumption},
Publisher = {NBER},
Year = {1976},
Pages = {265--302}
}
+
@Article{TverskyKahneman1974,
Title = {Judgment under Uncertainty: Heuristics and Biases},
Author = {Amos Tversky and Daniel Kahneman},
Journal = {Science},
Year = {1974},
+
Month = {September},
Number = {4157},
Pages = {1124-1131},
Volume = {185},
+
Date-added = {2010-11-16 22:11:33 -0800},
Date-modified = {2010-11-16 22:13:31 -0800}
}
+
@InBook{Handbook,
Title = {The Economics of Military Manpower},
Author = {John T. Warner and Beth J. Asch},
Chapter = {13},
Editor = {K. Hartley and T. Sandler},
Pages = {347-398},
Publisher = {Elsevier},
Year = {1995},
Series = {Handbook of Defense Economics},
Volume = {1},
+
Date-added = {2010-08-27 17:33:06 -0700},
Date-modified = {2010-08-27 17:36:18 -0700}
}
+
@Article{DMAsource1,
Date-added = {2014-04-21 19:54:02 +0000},
Date-modified = {2014-04-21 19:54:10 +0000}
}
+
@Article{StefanoFridayAnnounce,
Date-added = {2010-11-16 22:17:56 -0800},
Date-modified = {2010-11-16 22:18:07 -0800}
}
+
@Other{LAUSmeasurment,
Title = {Local Area Unemployment Statistics Estimation Methodology},
Bdsk-url-1 = {http://www.bls.gov/lau/laumthd.htm},
Date-added = {2010-09-10 20:47:34 -0700},
Date-modified = {2010-09-10 20:49:32 -0700},
LastChecked = {September 10, 2010},
Month = {September},
Url = {http://www.bls.gov/lau/laumthd.htm},
Year = {2009}
}
+
+ + + +
+ +
+ + + + +
+ +
+ +
+
+ +
+ + + + + + +
+ + + You can't perform that action at this time. +
+ + + + + + + + + + +
+ + You signed in with another tab or window. Reload to refresh your session. + You signed out in another tab or window. Reload to refresh your session. +
+ + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/militaryrecruiting.tex b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/militaryrecruiting.tex new file mode 100644 index 0000000..34c32cb --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/militaryrecruiting.tex @@ -0,0 +1,5805 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Annual2017/militaryrecruiting.tex at master · BITSS/Annual2017 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Skip to content +
+ + + + +
+
+
+
+ Help save net neutrality! + A free, open internet is once again at stake—and we need your help. +
+
+ Learn more + +
+
+
+ + + + + + + + + + +
+ +
+ +
+
+ + + +
+
+
+ + + + + + + + +
+
+ +
    +
  • +
    + +
    + + + +
    +
    +
    + + Notifications +
    + + + +
    +
    +
    +
    +
  • + +
  • + +
    +
    + + + +
    +
    + + + +
    + +
  • + +
  • + + + Fork + + + + + +
  • +
+ +

+ + /Annual2017 + +

+ +
+ + + + +
+ +
+
+ + + Permalink + + + +
+ +
+ + +
+ +
+
+ + Switch branches/tags +
+ +
+
+ +
+
+ +
+
+ + + +
+
+ + +
+ +
Nothing to show
+
+ +
+
+
+ +
+ + Find file + + +
+ +
+ + + +
+ Fetching contributors… +
+ +
+ + Cannot retrieve contributors at this time +
+
+ +
+
+
+ +
+ Raw + Blame + History +
+ + + + + +
+ +
+ +
+ +
+ 1207 lines (980 sloc) + + 111 KB +
+
+ + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
\documentclass[12pt] {article}
+
% The mathtime package uses a Times font instead of Computer Modern.
% Uncomment the line below if you wish to use the mathtime package:
%\usepackage[cmbold]{mathtime}
% Note that miktex, by default, configures the mathtime package to use commercial fonts
% which you may not have. If you would like to use mathtime but you are seeing error
% messages about missing fonts (mtex.pfb, mtsy.pfb, or rmtmi.pfb) then please see
% the technical support document at http://www.aeaweb.org/templates/technical_support.pdf
% for instructions on fixing this problem.
+
\usepackage{graphicx}
\usepackage{natbib}
\usepackage{fullpage}
\usepackage{amsmath}
\usepackage{hyperref}
\usepackage{pslatex}
\usepackage{standalone} %So you can outreg2 directly to a standalone doc and also input
%\renewcommand{\baselinestretch}{2}
+
+
\begin{document}
+
\title{Occupational Fatalities and the Labor Supply: Evidence from the Wars in Iraq and
Afghanistan}
\date{\today}
\author{Garret Christensen\thanks{
CEGA, UC Berkeley, 207 Giannini Hall, Berkeley CA, 94720-3310. E-mail: garret@berkeley.edu. This research did not receive any specific grant from funding agencies in the public, commercial, or
not-for-profit sectors. Many thanks to Dan Acland, John Warner, Alex Gelber and the personnel
of the Defense Manpower Data Center for data and assistance. Thanks
to Ted Miguel, Justin McCrary, David Card, Mark Borgschulte, Justin
Gallagher, Willa Friedman and UC Berkeley seminar participants for
suggestions. Data and replication files are available at \url{https://github.com/garretchristensen/military} and \url{https://dataverse.harvard.edu/dataverse/garretchristensen}.}}
\maketitle
+
+
+
\begin{abstract}
%\begin{center}
%\textbf{Job Market Paper}
%\end{center}
%PLEASE VISIT WEBSITE FOR LATEST VERSION: \url{http://www.ocf.berkeley.edu/~garret}
+
I find that full information utility maximization models are insufficient to explain the recruiting response to deaths of US soldiers in recent years. Using data of all applicants to the enlisted US military during the wars in Iraq and Afghanistan, deaths had a small but significant deterrent effect on recruiting in the soldiers' home counties. The deterrent is larger for deaths from the same county than for deaths from neighboring counties or for out of county but in state deaths. %This is not driven by media coverage; I find that recruits are over-emphasizing very local information in enlistment decisions.
The effect exhibits significant heterogeneity: deaths in Iraq decrease recruiting, while deaths in Afghanistan may increase recruiting, and the deterrent is more negative in less populous and more racially diverse counties; it is smaller or even positive in counties that voted for George W. Bush.
+
\end{abstract}
+
+
+
+
\section{Introduction\label{sec:Introduction}}
In a standard full information utility maximization model, one would predict that, all else equal, an increase in a profession's risk of death would decrease the desirability of employment in that profession. I present evidence from the United States military that this may not be the best model to explain behavior. Parts of the military can at times be similarly dangerous as the most dangerous civilian job, and the military employs many more people. I show that enlistees are responding differently to deaths from their county than to deaths from farther away, a finding that cannot be explained by easier access to news of local events. In contrast to standard predictions, I also show that job-related deaths can sometimes lead to increases in employment in an industry.
+
The purpose of this paper is to empirically test whether deaths of employees in a given occupation affect selection into that same occupation by other potential employees in the way that simple models predict. Since the time of Adam Smith, economists have used models with compensating differentials to explain wages in occupations entailing varying levels of risk or unpleasantness \citep{AdamSmith}. %Empirical studies have frequently estimated the value of a life using risk of death in different occupations.\footnote{See for example \cite{thaler1976value}. \cite{RosenOccChoiceHandbook} summarizes the extensive use of compensating differentials as a theory that explains supply of workers to jobs of different types in numerous occupations, including dangerous ones such as the military.}
Researchers have estimated how compensating differentials vary based on individual characteristics (e.g. age, gender, or being a single parent), but all these models either assume that safety is a normal good or estimate it as such.\footnote{See \cite{DeLeireLevy2004}, \cite{BiddleZarkin} and \cite{garen1988compensating}. \cite{thaler1976value} build a model involving insurance that indicates that job safety is not necessarily normal with respect to property income, since property income is not at risk in the labor market and reduces the need for insurance, acting as a substitution effect.} To examine this question empirically, I use new data from US soldiers in the wars in Iraq and Afghanistan and analyze how deaths affected the enlistment decisions of young Americans, but the results may also apply to a few other occupations.
+
One possibility is that when someone dies in an occupation, the siblings and friends of the deceased, and the population in general, would come to disapprove of the occupation (perhaps due to an increased assessment of their own risk) and become less likely to join. Since military pay is set at the same base rate across the country, I essentially hold compensation constant and look at the labor supply response to changes in perception of risk in my analysis. A standard model of compensating differentials would imply that if the wage were constant but assessment of risk increased, fewer would apply. However, if one was personally convinced of the virtue or necessity of the occupation in which the death occurred, then a sense of duty, patriotism, or pride might lead one to become more likely to join the occupation after an employee from the local area has been killed.
+
A few examples illustrate the point. The New York City Fire Department implements a recruitment policy giving a 10-point bonus on the employment exam to any applicant who is the child of a firefighter who was killed on 9/11.\footnote {See \url{http://www.nyc.gov/html/fdny/html/community/ff_faq_080106.shtm\#legacy}} The legacy points led to a minor uproar when they were initially refused to 13 applicants whose parent had died due to 9/11-related illnesses, but not at the World Trade Center on September 11, 2001.\footnote{See \url{http://www.nydailynews.com/new-york/gov-cuomo-13-fdny-sons-brave-future-article-1.1585065}} Dan Barta joined the Binghamton, NY police force after his father was killed in the line of duty, saying ``it has been my dream since that day [the day my father was killed] to be a Binghamton Police Officer.''
\footnote{See \url{http://www.wbng.com/home/Serving-the-community-A-family-legacy-230269421.html}} Similarly, the ``sandhogs'' who do the dangerous work of boring the tunnels under New York City often follow in (fallen) friends or family members' footsteps.\footnote{See \url{http://www.villagevoice.com/2012-04-11/news/sandhogs-tunneling-second-avenue-subway/full/} or \url{http://www.nydailynews.com/archives/news/brave-sandhogs-pause-reflect-article-1.569895}}
+
This possible incentive effect is not unique to a single instance, and perhaps it is not unique to the military, though at least one example from the military received media attention: 61-year-old orthopedic surgeon Bill Krissoff acquired an age waiver and enlisted in the Navy Medical Corps after his son Nathan was killed in the Marines in Iraq.\footnote{See \url{http://www.npr.org/templates/story/story.php?storyId=17013597}. There is also anecdotal evidence that recruiting stations were overwhelmed with potential recruits
after 9/11, but it is a goal of this paper to empirically determine whether increased risk actually led to more or fewer recruits. Compare \url{http://www.nytimes.com/2001/11/12/us/nation-challenged-recruit-self-described-slacker-decides-he-s-ready-be-soldier.html},
which describes an individual motivated to enlist to \url{http://www.nytimes.com/2001/09/16/us/after-attacks-military-despite-national-rush-emotion-recruiting-centers-aren-t.html}, in which recruiters claim not to have seen a significant increase
in qualified recruits.} Since some of the Marine Corps' support operations are provided by the Navy, serving in the Navy Medical Corps enabled Krissoff to give medical care to those with whom his son had fought and died.
+
Is there a model that can test whether these examples are evidence of a general phenomenon in military recruiting? This paper is an empirical test of the size of the deterrent or incentive effect of deaths in the military on recruiting, and an analysis of when the effect may be larger or smaller. My analysis draws on a valuable new dataset obtained through Freedom of Information Act requests, comprising the complete set of active duty enlisted applicants to the military from 2001 to 2006, matched with detailed data on every death of a US soldier that occurred in Operation Iraqi Freedom and Operation
Enduring Freedom (i.e., the war in Afghanistan) during the same period. I have made these datasets publicly available on the Internet using Harvard's Dataverse.\footnote{Please see \url{https://dataverse.harvard.edu/dataverse/garretchristensen} for data. All analysis files necessary to replicate this paper are also available online at \url{https://github.com/garretchristensen/military}.}
+
With detailed geographic and date information, I am able to analyze recruiting at the county-month level, a significant improvement upon much of the literature. I use data on the home locations of recently killed troops and correlate the deaths of soldiers from the local area with the local rates of recruiting. I flexibly control for the underlying characteristics of counties as well as nation-wide changes over time using county, monthly, and state-year fixed effects. After controlling for these underlying characteristics, the hometown of the fatality is arguably exogenous. I use this source of variation to analyze the causal effects of local deaths on local military recruiting.\footnote{I find that the recruiting response is always significantly different
for local deaths (deaths in the same county) than for deaths from
more distant locales (deaths from outside the county but in the same
state, and deaths from outside the state). Coefficients on deaths from outside the state (national deaths) cannot be estimated due to the monthly fixed effects in most models, however in (possibly misspecified) models without the fixed effects, local deaths
are multiple orders of magnitude larger for local deaths than
for national deaths. See \cite{ChettySalience} or the companion \cite{ChettyEarly},
which develops a structural model using bounded rationality to estimate salience parameters.%
}
+
I find that when a soldier died in Iraq or Afghanistan, that soldier's
home county saw a decrease in recruiting of over one percent. %This effect is similar for recruits in both of the stages of the recruiting process that I test. %I also obtain very similar semi-elasticity estimates using either Poisson or negative binomial regression, and the estimates are very stable across different sets of fixed effects. %I find that the effect is very localized---deaths in neighboring counties and deaths in counties in the same Nielsen media market produce no deterrent effect.
However, deaths of soldiers in Iraq lead to a larger reduction in recruiting, while deaths of soldiers in Afghanistan may actually lead to a small increase in recruiting. There are also differences across ethnic demographic, economic, and political lines. The deterrent effect is significantly larger in counties with higher than average African-American populations and is significantly smaller (and sometimes even positive) in counties with higher unemployment, and counties that voted for George W. Bush in 2000 or 2004.
+
%I also find that the effect is different for different types of recruits. %It is larger for recruits to the Marine Corps and smaller for recruits to the Air Force.
%It is significantly higher for recruits of the highest quality as measured by Armed Forces Qualification Test (AFQT) score and educational attainment. This evidence suggests that outside employment options and political opinions matter greatly in determining potential applicants' reactions to an on-the-job death, and is similar to what is found in \cite{kahn1987occupational} concerning education and higher compensating differentials for risk.
+
+
%I have also looked for these effects on recruiting with a larger dataset
%covering all deaths in the military, regardless of their cause, from
%the decade prior to the wars in Afghanistan and Iraq, and have found
%no significant effects. There were significantly fewer deaths during
%this earlier period, suggesting that some level of deaths, or general
%awareness of deaths or military operations, may be necessary before
%a significant effect is observed.
+
%Deaths at the national level are also of interest and likely have
%an effect on the difficulty the military services have in recruiting.
%National recruitment rates and deaths tend to fluctuate quite a bit,
%so I exploit this variation and briefly investigate this question in the appendix.
%I find a negative correlation between total national deaths and national
%recruiting. A one percent increase in deaths is correlated with a
%two percent decrease in national recruits. However, whether this relationship
%is a causal one is of course a difficult question. While a nation-wide
%causal relationship may in fact exist, controlling for all the unobserved
%factors in addition to total deaths that determine how many people
%in the entire nation are interested in joining the military is likely
%an impossible task. This paper thus does not address the question of the causal effect of %national deaths.
+
%Public support for the war, the state of the economy,
%and the state of the war effort itself are all extremely difficult
%to quantify, so omitted variable and endogeneity issues would likely
%be prohibitively difficult to overcome.
+
My findings on local deaths provide evidence that people
are not behaving in the way that a simple full-information utility-maximization
model would predict. I show that one's hometown when enlisting has
little to do with the likelihood of death given enlistment. Thus if
a potential recruit were to learn that a soldier from his or her
county had been killed in a war, that soldier's death has no more
bearing on his or her own risk from joining the military than the
death of a soldier from halfway across the country. If a potential
recruit were basing his enlistment decision on strictly rational factors such
as monetary compensation and the risk of death, a county death would
have the same effect as a state death. The data shows that this is
clearly not the case. Thus individuals must either be updating their
priors with incomplete information and misperceiving the actual risk
or basing the non-pecuniary benefits they receive from military service
on the proximity of deaths, and not just the number.
+
In addition to being of interest to labor economists who want to understand information processing in employment decisions, these findings may also be of interest to the military and policy makers who determine its funding. By analyzing the characteristics
of a county, the military could produce a detailed estimate of the
effect of deaths on recruiting in that county. If the military desires
a wide geographic recruiting base, or if they desire to minimize costs,
they could use the findings in this paper to help in their decision
to reallocate recruiting funding and manpower.
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Background\label{sec:Literature-Review}}
The military plays a very large role in the United States economy.
In 2010, President Obama signed a bill authorizing \$680 billion in
military spending, making up nearly 20\% of total federal expenditures.\footnote{\url{http://www.nytimes.com/2009/10/29/business/29defense.html}} \$130 billion of that was for the wars in Iraq and Afghanistan, which have so far claimed the lives of over 8,000 US and coalition
soldiers.\footnote{\url{http://www.gpoaccess.gov/usbudget/fy10/pdf/fy10-newera.pdf}}
\footnote{\url{http://www.cnn.com/SPECIALS/war.casualties/index.html}
} Another \$177.5 billion of this was to be spent on direct compensation
to military personnel and family.\footnote{\url{http://www.defenselink.mil/news/2010\%20Budget\%20Proposal.pdf}} Through 2010, this money was used to pay and support over
1.4 million active duty men and women in uniform, and another 1.4
million National Guard and Reserve troops.
+
Most of the soldiers in the military serve for only a few years, so the military needs
to recruit approximately 200,000 new troops every year. Recruiting this many troops costs a great deal. A 2003 GAO report lists the Defense Department annual recruiting budget as \$4
billion, roughly \$20,000 per recruit with over \$1900 per recruit
spent on advertising alone.%
\footnote{\url{http://www.gao.gov/new.items/d031005.pdf}
} Part of this recruiting budget is spent on the salaries of production
recruiters, active-duty men and women whose job it is to find new
recruits. As of 2010, the Army employed over 8,000 soldiers as recruiters,
and the Navy 4,897.\footnote{\url{http://www.2k.army.mil/faqs.htm}, \url{http://www.cnrc.navy.mil/PAO/facts\_stats.htm}
}
+
+
Military recruiters handle all recruiting of enlisted members of the military. Being a recruiter is similar to other military occupational
specialties---recruiters are mostly enlisted men and women who are
assigned to a specific location for a three year stint, with limited
control over where they are assigned. Recruiters work out of offices
spread all over the country, often in shopping malls or heavily trafficked
areas. Whenever anyone enlists in the military, it is through such
a recruiting office.
+
The military refers to potential recruits in the first stage of the enlistment process as {}``applicants.'' When a potential recruit first calls on
the telephone or walks in the door and expresses interest in joining
the service, the recruiter will make sure the candidate meets certain
medical and legal requirements, for instance, he or she can have no
felonies, cannot be on probation, and cannot be a single parent. The
recruiter will enter data on the potential recruit into the database
system as soon as possible after the initial expression of interest.
The interested party will typically take a short (30-minute) practice
version of the Armed Services Vocational Aptitude Battery (ASVAB).
If they don't perform very well, perhaps they'll be told to study
for a bit before taking the actual 3-hour ASVAB, but those who seem
prepared would soon travel to a regional processing center (at a location
other than the storefront recruiting center they've been visiting)
and take the ASVAB, in some cases as soon as the day after expressing
initial interest. Four of the 11 sections of the ASVAB are used as
the Armed Forces Qualification Test (AFQT). Assuming that a potential soldier passes the examination
(one needs a score of 31 or higher to enlist in the Army) the applicant will
then return to the recruiting center and be shown by their recruiter
what jobs are available and when. All recruits in the data have AFQT scores, so potential recruits are only entered into my data set as soon as they have taken the test.
+
Once a potential recruit has taken the exam, chosen a military career
(the availability of which depends on their test score) and is assigned
a departure and enrollment date, they can sign a contract and take
the oath of military service. This is the point at which a potential
recruit is recorded as a ``contract'' in the data. When a contracted
recruit finally ships off to training, they are recorded as an {}``accession.''
These accessions are the most commonly reported figures in the media
and in Defense Department press releases pertaining to the military
having reached its recruiting goals, but it is common in the literature
to use data on contracts, since the accession date is more under the
influence of the needs of the military, and is thus more demand-constrained
and exhibits very strong seasonal fluctuations.
+
Although a good deal of research has been conducted on the labor supply
elasticity of the all-volunteer US military, little, if any, has analyzed
the effect of war-time deaths on the labor supply. Most of the existing
research has focused on the supply elasticity with respect to salaries or unemployment, starting with 1960's estimates of labor supply in the absence of a draft, such as
\cite{Altman-Fechter-AER-1967} and \cite{Altman-JHR-1969}. Recent research has investigated the effect on supply of a faster route to citizenship for non-citizen legal residents \citep{cunha2014expedited}. \cite{Dale-Gilroy-AER-Note-1985} showed that higher unemployment typically led to higher recruiting levels and established the importance of using applicants and contracts in analysis instead of accessions, as the final stage in the recruiting process, defined as shipping off
to boot camp, is much more seasonal. I follow this convention and use applicants and contracts. %a slightly %earlier stage in the recruiting process
%referred to as {}``contracts'' is a much better dependent variable
%to use, because recruits can sign a binding contract to join the military
%up to a year in advance through the Delayed Entry Program. When they
%actually ship out is very seasonal and highest in the summer.)
Attempts have been made to account for both the supply and demand side of recruiting (\cite{Dertouzos-RAND-1985}, \cite{Hanssens-Levien-Advertising}, \cite{Dertouzous-Polich-Army-Ads}, \cite{Asch-Incentives}). Demand can affect the quality and quantity of recruits because waivers for drug use or criminal activity (which would normally preclude one from enlisting) are sometimes available in periods of high demand. In this paper I focus on the supply side only.
+
%More recent work has attempted to control for both the demand side
%of the process, acknowledging changes in the military's need for recruits,
%as well as the supply side. Some work that controls for the demand-side
%by using the quantity of production recruiters in a given location
%and time such as \cite{DaleGilroy1984} has not found significantly
%different estimates once the extra controls were added. Yet other
%work by \cite{Dertouzos-RAND-1985} controls for endogeneity by using
%recruiter quotas and explicitly models the tradeoff a recruiter faces
%between high and low quality recruits and finds that elasticities
%with respect to pay and unemployment are even higher once these extra
%controls are included.
+
%One of the more interesting data sets is that of \cite{Hanssens-Levien-Advertising},
%which has monthly advertising and recruiting spending for each of
%the 43 Navy Recruiting Districts from January 1976-December 1978.
%They find that television advertising does serve to increase recruiting,
%but less than the number of active recruiters and the money allocated
%to their efforts (appearing at schools, posting flyers, handing out
%brochures) and their effort (as proxied by their monthly recruiting
%quotas). Yet the number and effort of recruiters was found to matter
%less than economic or environmental conditions such as unemployment.
%Using similarly detailed advertising budgets for the Army from 1981
%to 1984 with detailed data on the number of recruiters, in addition
%to modeling recruiter effort using quotas for low and high-quality
%recruits, \cite{Dertouzous-Polich-Army-Ads} find very similar estimates---unemployment
%and wages have high supply elasticities. The number of recruiters
%is also associated with large increases in recruits, but advertising
%has very small elasticities. However, the marginal cost of an additional
%recruit through extra recruiters is found to be similar to the cost
%through advertising (\$6000), which is far cheaper than the marginal
%cost through enlistment bonuses (\$16,000). When adjusted to today's
%dollars, their estimated costs of a new recruit are roughly in line
%with the average cost before the war in Iraq according to the GAO estimate mentioned above.
+
%One attempt to estimate the effects of changes on the demand side
%of the recruiting process is \cite{Asch-Incentives}. The data set
%covers a five-month period for the Chicago Navy Recruiting District--monthly
%observations of 125 recruiters, with data on individual recruiter
%successes and their quotas and points based on a complex reward system
%that was in place at the time. She finds that recruiter effort does
%appear closely based on the potential of winning awards, as recruiters
%exhibit different amounts of effort over the different stages in the
%12-month award cycle. However, \cite{Oken-Asch-RewardHistory} show
%in a retrospective history of all four service branches that both
%the quota and reward systems have varied significantly across both
%time and service branches, and at times both the Army and the Navy
%did not even set individual recruiter goals, although they continued
%to set nationwide and recruiting district goals. More recent work
%has also focused on accounting for the demand side of the recruiting
%process, either using quotas and a control function approach as in
%\cite{Heaton-Asch} or by instrumenting for the military wage using
%the legislative formula for annual increases, as in \cite{Gelber}.
+
There has been some limited research analyzing the effect of US fatalities
in Iraq and Afghanistan at the national level (see\cite{BigRandDocument}, \cite{Simon-Warner-DEATHS!}).
%estimates that casualties in Iraq were responsible for anywhere from
%a 6 to 60 percent decline in Army recruiting depending on methodology,
%and \cite{Simon-Warner-DEATHS!} find that an additional 400 casualties
%per year in Iraq is associated with a decrease of high quality Army
%recruits of 6 percent.
However, no previous work has used the spatial variation in US military combat deaths in Iraq and Afghanistan (or those from any other war, for that matter) to examine the effect
on recruiting, though researchers have used spatial variation
in deaths for other purposes. For example, \cite{Ted-Miguel-Bush-Deaths}
used the plausibly exogenous variation in the geography of US deaths in Iraq at the state level to examine the effect on changes in voteshare for George W. Bush between
the presidential elections of 2000 and 2004. They find strong negative
localized effects of deaths--without the deaths, Bush might have won
an additional two percent of the national vote. Earlier work by \cite{SSGartner-WarCasualtiesPublicOpinion}
and \cite{SSGartnerAllPoliticsLocal} used the geographic variation
in casualties from the Vietnam War to show that local casualties have
a very strong relation to public approval of the President and his
handling of the war. %They find that this effect is strongest during
%the first half of the war, when casualties are accelerating, while
%in the second half of the war, as the rate of casualties is decreasing,
%socio-demographic characteristics play a more important role than
%local casualties.
+
A few papers in political science have found contradictory evidence regarding the effects of recruiting on the flip side of the insurgency/counter-insurgency coin. Research by \cite {Kocher-Vietnam} finds that civilian casualties in Vietnam helped the Viet Cong gain control in that area. \cite {Lyall} finds that Russian shelling in Chechnya led to a reduction in local insurgent attacks, and recent research by \cite{AfghanIraqCasualties} that uses civilian casualties resulting from the US military's presence in Afghanistan and Iraq shows that local civilian deaths lead to more incidence
of local violent attacks in Afghanistan, but not in Iraq. These results are consistent with my findings that the effects of war on recruiting vary in direction and magnitude by situation.
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
%Used to be a model here. Now it's gone.
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Data\label{sec:Data}}
+
The recruiting data used in previous literature has typically been analyzed at the quarterly or yearly level, often at the state level, while my data contains the exact dates of applications and the ZIP code for each applicant, which I have aggregated to the monthly-county level. I obtained this data through Freedom
of Information Act requests to the office of the Secretary of Defense. It consists of distinct sets of individuals: applicants, contracts,
and accessions (explained above) and contains the date on which these individuals were recorded as starting one of the three
specific parts of the recruitment process, ZIP code, AFQT score, educational
attainment, and branch and component of the military to which the
potential recruit was applying. The same data is available for applicants
and contracts, but the data are stored separately for each step in
the recruiting process and are unfortunately not linked by individuals
across datasets. Age of recruit is also unfortunately not included.
+
I have recruiting data for fiscal years 1990-2006. The applicants data set
contains 6.4 million active duty observations, the contracts data
set has 3.6 million active duty observations. I am able to match roughly 96\% of the applicant observations by ZIP code to a US county.
+
+
The main fatality data come from a publicly available list compiled by the Defense Manpower Data Center. Starting October 7, 2001, every fatality in Operation Iraqi Freedom
and Operation Enduring Freedom is listed, and includes the service branch, component (active/reserve/guard), name, rank,
pay grade, date of death, hostile status of death, age, gender, location of home of record, unit, incident geographic code, fatality geographic code, and race/ethnicity of the deceased.
+
The data includes {}``home of record'' which is where the soldier lived on the day they joined the service, and generally does not change over the course of a single military enlistment contract, but can change at the time of reenlistment. It is separate and distinct from a legal residence for tax purposes. Still, this is important with regards to my claim of plausible random assignment of death with respect to county after controlling for military population levels---service-men and women with especially dangerous military professions and long military careers could buy homes near their duty-base and changing their Home of Record to the county in which the base sits. This should still not be a problem for identification due to the inclusion of county fixed effects, but to test this, I have looked separately at deaths of low paygrade soldiers compared to deaths of high paygrade soldiers (who may have reenlisted and changed their home of record). These regressions are shown in the appendix; there are no significant differences in the deterrent effect by paygrade of the soldier who was killed.
+
Since the focus of this paper is recent combat deaths, the main 2001-2006 fatality data used in the majority of this paper does not include the fairly common deaths of military members unrelated to combat abroad, or unrelated to the military at all (e.g. heart attacks, car accidents). After earlier drafts of this paper were completed, a FOIA request for detailed death data for all deaths in the military (not just those in the wars) for the entire period for which I had recruiting data (1990-2006) was granted. I use these deaths as placebo tests of my analysis and find that, as expected, there is no significant recruiting effect of local non-combat deaths. This is discussed in the appendix and shown in Table~\ref{Flo:LNlinear90}.
%and \ref{Flo:alldeaths90}.
+
To summarize the data, Table \ref{Flo:Simple Deaths} shows total annual military deaths and the subsets of those recorded as hostile action from the FOIA data and those considered part of the Iraqi/Enduring Freedom operations as in the public data used for the majority of this paper. One can see that at most in any year, less than 50\% of the deaths of active duty US military members during the relevant time period (2001-2006) are classified as part of the Iraq/Afghanistan wars. Also, the Iraq/Afghanistan deaths outnumber the hostile deaths, so friendly fire or accidental deaths in the battle area can be considered part of the war data. During the entire period for which I have data, 1999---two years prior to the beginning of my analysis---experienced the lowest number of deaths, with 796, of which zero were classified as hostile.
+
%1--SUMMARY TABLE
\begin{table}
\caption{}
\label{Flo:Simple Deaths}\includegraphics[bb=45bp 0bp 612bp 730bp,clip,scale=0.95]{../Output/PDF/DeathsAnnualSimple}
%this is made by DataSummaryTable.do (though not reproduced automatically)
\end{table}
+
+
During the 2001-2006 period there were 2886 deaths in the combat death database, 2725 of which (94\%) I have been able to link to the home county of record of the deceased soldier. I thus have data for both deaths and recruits for 58 months, for all the roughly 3,125 U.S. counties or county-equivalents. The number of deaths in a county-month range from 0 (98.7\% of county-month observations) to a high of 8 in Los Angeles County in November, 2004. These and other summary statistics are presented in Table~\ref{Flo:Simple Deaths}. Figure \ref{Flo:Monthly Deaths vs. Monthly Recruits} shows monthly total national combat deaths and monthly total applicants to the military from October 2001 through July 2006.
+
%1--SUMMARY FIGURE
\begin{figure}
\includegraphics[scale=0.5]{../Output/graph_deathsvsrecruits_basic}
%This gets made in table1graph.do
\caption{Graph of Monthly Recruits and Monthly Iraq/Afghanistan Combat Deaths}
\label{Flo:Monthly Deaths vs. Monthly Recruits}
\end{figure}
%Publicly available yearly figures indicate that in the period for which I have recruiting data %(1990-2006),
%the year 2000 had the lowest number of military deaths--zero from
%hostile action, but 758 from accident, illness, homicide, or suicide.
%1991 had 1787 military deaths, only 147 of which were from hostile
%action (presumably in the Persian Gulf War).
+
+
Although my analysis primarily rests on the panel nature of the data
and the inclusion of area and time fixed effects to identify the effect
of local deaths, I have also included time varying characteristics
of counties to the extent that they are available. These include unemployment
at the state and county level as reported by the Bureau of Labor Statistics, and mortality for young males age 18-24 from the Multiple Cause of Death files at the National Center for Health Statistics National Vital Statistics System. Statewide numbers of recruiters by service branch have also been included in certain specifications.
A more complete description of datasets used is included in the appendix.
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
\section{Analysis\label{sec:Analysis}}
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{County-Level Analysis\label{sub:County OLS}}
+
The primary analysis is at the county level, the smallest region at which there is close to a one-to-one relationship from death data geographic unit to recruit data geographic unit. Main results are estimated using ordinary leasts squares and a log-linear model.\footnote{Log-linear estimates are calculated using Sergio Correia's reghdfe package in Stata 12 \citep{reghdfe} To interpret log-linear coefficients as percent changes, one should exponentiate coefficients, but at the small numbers in this papers values are nearly identical, and values reported in the body of the text refer strictly to values in tables. Since data on recruits is in count form, the Poisson model may also be appropriate for the data, as linear models of course ignore the restriction of the dependent variable of recruits to non-negative integers. See \cite{cameron2013countdata} for a complete discussion. For the sake of robustness and transparency, I have also tested Poisson regressions, which are shown in the Appendix. The results are qualitatively very similar, showing that a death leads to slightly larger decreases in percentage terms when estimated with log-linear OLS regressions than in the Poisson regressions. The appendix also includes a negative binomial specification. Again, the results are very similar.} The regression generally follows the model:
$$Recruits_{it}=\beta_{0}Deaths_{i,t}+\beta_{1}Deaths_{i,t-1}+\beta_{2}Unemployment_{it}\\
+x'_{it}\eta+\alpha_{i}+\gamma_{t}+u_{it}$$
where Deaths implies deaths from the given county, Unemployment county unemployment, and $x$ includes in-state (but out of county) deaths as well as
state unemployment. $\alpha_{i}$ is a set of fixed effects for each county, which flexibly control for any county characteristics fixed over time such as the presence of a military base or political support for the military. $\gamma_{t}$ is a set of fixed effects for every month, so national characteristics that are the same across counties in any given time period such as the total national number of deaths, national
unemployment rate, or the military wage rate are also flexibly controlled for and cannot be separately estimated.
+
% Linear with logged LHS--Main specification
% built by redefined.do
\begin{table}
\caption{}
\label{Flo:loglinear}
\scalebox{0.9}{
\input{../Output/LNLinearW.tex}
}
\end{table}
+
+
Table \ref{Flo:loglinear}
shows the results when linear regression is used to analyze the data
at the county-month level. The left half of these regressions show the analysis done for applicants, the right hand side for contracts. %The first specification, for applicants in column one and contracts in column five, is the `horse race' specification, comparing in-county deaths, out-of-county but in-state deaths, and out-of-state deaths. Of course this precludes the inclusion of monthly fixed effects, so this model is mis-specified to the extent that there is temporal heterogeneity in the recruiting response to deaths over the course of the wars, nevertheless, I find it illustrative that the coefficient for lagged in-county deaths is nearly five times that of the coefficient for an out-of-state death (1\% compared to 0.2\%). (A more detailed analysis of deaths at the national level is presented in the appendix.)
Fixed effects for each county and for each month are included in all specifications, as well as state-year fixed effects in specifications 3 and 6.
Observations are weighted by county population, and standard errors
are clustered by county. The results indicate that one additional
in-county death is followed in the next month by a 1.1\%
decrease in applicants and a similar reduction in contracts.\footnote{Note that all death figures have been divided by 100 to make more
useful digits of the coefficients visible, and thus all coefficients
for deaths should be interpreted as percents and not fractions (i.e.
0.4 is 0.4 percent, not 40 percent).}
+
Deaths from in-state but out-of-county
appear to have a small positive effect, from 0.1\% to 0.2\%, which leads to some concern about the possible behavioral interpretation of my results---perhaps recruiters avoid an area after a death and instead spend their time recruiting from neighboring areas. This alternative mechanism, however, is complicated by estimates below showing that deaths in contiguous counties do not lead to an increase in recruiting.
+
Unemployment at the state level has a negative effect: a one percentage point increase leads to a small but not always significant decrease in recruiting,
while a one percentage point increase in county unemployment leads
to a 1\% increase in recruiting.\footnote{It should not be surprising that specifications including both county and state unemployment show one positive and one negative coefficient: holding state unemployment constant and increasing county unemployment means the county in question is relatively unlucky within the state, so recruits in that county have fewer other options and are more likely to enlist. Conversely, hold county unemployment constant and increase state unemployment, and the county is relatively lucky employment-wise, leading to fewer recruits.} A simple comparison of the coefficients on lagged county deaths and county unemployment indicates that one fewer death of a soldier from the county would lead to the same increase in recruiting as a one percentage point increase in county unemployment.
+
%Unemployment levels are also closely correlated with recruiting levels.
%State unemployment has fairly volatile estimates depending on which
%fixed effects are included, but coefficients for county unemployment
%are stable; a one percentage point increase in county unemployment
%leads to two to four more recruits.
+
%One could conceivably be concerned that county unemployment was measured
%somewhat inaccurately and in a way that somehow biased the estimates,
%especially given the relatively small sample size used to calculate
%each county's unemployment rate and its mechanical relationship to
%state and national unemployment levels. I have tested this by adding
%normally distributed noise to the county unemployment estimates; none
%of the other estimates changed significantly.
+
\paragraph{Issues of Location}
Assuming that moving across county or state lines (or finding employment across
county or state lines) is costly, the idea that potential recruits are responding to county and state unemployment above and beyond the national unemployment level is in
accordance with rational utility maximizing individuals, as county and state unemployment
levels directly affect one's likelihood of employment, and thus income
and utility. As discussed at length in the appendix in section~\ref{sec:deathrate}, deaths of active duty soldiers from one's own county or state are largely unrelated to one's own likelihood of dying in the service, since the Army operates at a national level and recruits are put into
military careers irrespective of their state or county of origin. Clearly this is not quite the
case with Reserve and National Guard troops, as Reservists simply
report to the nearest base for one weekend a month and two weeks
a year of training, but their recruiting numbers are not included
in this analysis.
+
+
I performed this analysis using all of active duty, reserve, and guard duty deaths, because the main emphasis of my analysis is to determine the magnitude of the observed reaction to deaths. It may be true that the response to deaths of local soldiers from
reserve and guards units is a rational response based on an updated
assessment of the risk of death (since those who enlist would serve in the same location-based reserve or guard unit) but still, the magnitude of the observed
deterrence effect, rational or not, would be what is of interest to
policy makers. As a robustness check, however, I have run the analysis
using only the active-duty deaths, and under this specification, in-county
deaths are followed by a similar or slightly larger reduction in recruits in the
next month. The coefficients for out-of-county deaths and unemployment also
remain similar. These results are shown in Appendix Table~\ref{Flo:Rdeathslinear}.
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\paragraph{Additional Controls}
+
Despite the inclusion of fixed effects, the potential for omitted
variable bias still exists. One of the most obvious ways this might
occur is through the action of the military's production recruiters.
It seems likely that the number of production recruiters is positively
correlated with the number of recruits, and in the extreme this is true mechanically. If the number of recruiters (or their level
of effort) were also correlated with the number of deaths, my estimates
would be biased.
+
Given that recruiters serve for three years in one
place, it is highly unlikely that the military is relocating them
in a way that is correlated with monthly deaths. Without being relocated,
however, recruiters may change their level of effort. FOIA requests for data on recruiter quotas unfortunately have not been granted, so I am only able to use the number of recruiters by state and quarter until halfway through 2004\footnote{I am grateful to John Warner for sharing this data.}, which I have included as an extra control variable for that portion of the sample.
+
I also have detailed mortality data through 2004. It is conceivable that deaths unrelated to the military would play a role in determining recruiting (for example,
young men in a crime-ridden community may be anxious to join the military
as a means of escape) thus I include monthly male 18-24 year-old mortality figures
as well. Appendix table \ref{Flo:LNRecandMort} shows these results. The
analysis is done for both applicants and contracts, with the observations
limited to October 2001 to June 2004. County and monthly
fixed effects are included. The estimates are similar in this restricted time period to those from the full sample, and the effect of a death does not change significantly when I add the extra controls. %The estimates for contracts are only
%about half of what they are using the full data set, indicating that
%the response to deaths is not constant over the duration of the entire sample.
+
+
Table \ref{Flo:Media} probes the robustness further. Here I have included the number of deaths that occurred in contiguous counties and the number of deaths that occurred in counties that share
the same media market as the main county of interest. County contiguity
is defined using the 1991 ICPSR contiguous county file.\footnote{U.S. Dept. of Commerce, Bureau of the Census. CONTIGUOUS COUNTY FILE,
1991: {[}UNITED STATES{]} {[}Computer file{]}. Washington, DC: U.S.
Dept. of Commerce, Bureau of the Census {[}producer{]}, 1992. Ann
Arbor, MI: Inter- university Consortium for Political and Social Research
{[}distributor{]}, 1992. doi:10.3886/ICPSR09835
} Media markets are defined using the Nielsen Media Research's Designated
Market Area (DMA). In the year 2000, Nielsen divided the country into 208 DMAs based on a preponderence of residents having access to the same broadcast television and radio stations. (See \cite{DMAsource1} and \cite{DMAsource2} for more details.) The regressions show that deaths in nearby areas, whether defined using county borders or media market, have smaller effects on recruiting in a given county. The effect size is roughly half as large as that of a death from the county. While it seems that recruits do respond to information from outside the county, this is perhaps suggestive evidence that the county response to deaths is due to something more than information, since media markets are intended to share major news sources.
+
%MEDIA MARKET AND CONTIGUOUS
\begin{table}
%built by redefcontig.do
\caption{Deaths in Neighboring Counties and Same Media Market}
\label{Flo:Media}
\scalebox{0.65}{\input{../Output/redefcontigLN.tex}
}
\end{table}
+
\subsection{Lags and Leads of Deaths and Unemployment\label{sub:Lags}}
My main empirical method so far has been to compare county recruits
in a given calendar month to county-wide and state-wide deaths in
the previous month. It is possible that potential recruits initially
deterred from enlisting by a death eventually {}``forget'' about
local deaths and join the military. Table~\ref{Flo:Cumulative LagsLN}
shows regressions with cumulative death and unemployment lags
of two, four, six, and twelve months--that is, the sum of current
deaths plus all the deaths that occurred in the previous number of
months. The results indicate that deaths from previous months have a significantly smaller deterrent effect on recruiting than more recent deaths. Earlier regressions have shown a deterrent effect of over one percent for deaths in the previous month; these
regressions show a relative decline in the effect size the longer a time period included. The effect size decreases to one third a percent for applicants for twelve months of lagged deaths. The decline in effect size does not appear to be as steady for contracts. Poisson regressions produce very similar semi-elasticity
estimates.
+
%Cumulative Lags
\begin{table}
%builtby redefrunninglags.do
\caption{Cumulative Lags}
\label{Flo:Cumulative LagsLN}
\scalebox{0.75}{\input{../Output/runninglagsLN}}
\end{table}
+
As a robustness check on my main specification, I have also run regressions including deaths one month into the future. In the main log-linear specification, deaths from the same county one month into the future appear to have small but statistically significant relationships with recruits. However, the Poisson models, as well as the log-linear model using only active duty deaths, show clearly that deaths in the future do not correlate signficantly with current recruiting levels. These regressions are shown in Appendix Table~\ref{Flo:forwardbasicLN} and Table~\ref{Flo:forwardPbasic}.
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Heterogeneity of the Deterrent Effect\label{sub:interactions}}
+
The analysis in the previous two subsections makes it clear that in-county
deaths result in a significant decrease in county recruiting. An important
corollary question concerns the heterogeneity of this effect.
All counties are unlikely to observe the same deterrent effect of
death. Here I analyze the recruiting response to deaths based
on different characteristics of counties, the deaths, or the recruits. First, I test the effect of differences in a county's demographic and cultural makeup, specifically, its population, unemployment, racial makeup, rural/urban status, and political alignment. Second, I test the response for different types of recruits, and third, for different types of deaths.
+
In the Appendix section~\ref{sec:servicebranch} I have also tested the difference for different service branches in the armed forces, for both deaths and recruits. I find weak evidence that recruits to some service branches respond differently than recruits to other branches, but the patterns are not consistent across the stages of recruiting. I also find limited evidence that recruits respond based on the branch in which the service branch occurred.
+
\subsubsection{Heterogeneity by County Characteristics}
+
Table \ref{Flo:Linear Interactions} displays these regressions.
They all include monthly and county fixed effects, out of county but
in-state deaths, as well as unemployment, and I have added county
characteristics interacted with lagged in-county deaths. All variables
to be interacted have had the population weighted mean subtracted. %\footnote{ In a standard OLS regression this de-meaning would mean we could expect the main coefficient on lagged county deaths to remain essentially constant, but the weighting scheme used in Poisson regressions does not give the same sort of results. I have tested this and found that the main coefficient does stay very stable in least squares regressions regardless of what sort of weighted de-meaned interaction variables are included. I have also run these regressions using least squares and the results are qualitatively similar.}
Note that most of the county characteristics are fixed over time and thus perfectly collinear with fixed effects and cannot also be included.
+
%LINEAR INTERACTIONS
%built by interactionscontrols.do
\begin{table}
\caption{Linear Interactions}
\label{Flo:Linear Interactions}\input{../Output/LNinteractCOMBINE}
\end{table}
+
I have interacted lagged county deaths with inverse county population to estimate the effect in terms of deaths per population. Also included are interactions with the monthly county unemployment figure (the only county characteristic
that changes over time and thus is not collinear with the fixed effects
and can be included in the regression by itself), percent African-American
population as measured in 2005, %racial fractionalization using percent white, black, Native American, Asian, and Pacific Islander,
a binary measure of rural status using the USDA's Economic Research Service classification, and the percent of the county that voted for George Bush in 2004.
The regressions are run for applicants and contracts.%, the first column with percent black, and the second with racial fractionalization.
+
I have also run regressions including interactions of all these same
variables, but interacted with all four counts of deaths (in and out
of county, lagged and current) the coefficients on the original interaction
are very similar, and the coefficients for the interactions with out-of-county
deaths and current in-county deaths all either go in the same direction
as the ones shown in Table \ref{Flo:Linear Interactions} or are
statistically not different than zero.
+
+
%Deaths per Male 18-24 year-old County Population has a rather large and negative coefficient, which is actually quite reasonable. The coefficient implies that there is a level effect of deaths, and then an additional effect of deaths per capita, from -2086 to -2215 for applicants and a statistically insignificant -1510 to -1704 for contracts. This means that an increase of one death for every one young male (obviously improbably high) would result in a decrease of recruits by 1500 to 2000 percent (also improbably high). Simpler to imagine is that for every additional 1500 to 2000 young males in a county, the deterrent effect of deaths is one percent less negative. More-populated counties have a smaller percentage recruiting response to deaths than less-populated counties. With more people, perhaps other young men are less likely to hear the news of the death of a soldier, and if they hear it, perhaps they are less likely to have known the soldier who was killed and thus be relatively undeterred by his death. It is slightly puzzling, however, why this differential effect would be so strongly significant for applicants and not significant for contracts.
+
Death {*} County Unemployment yields positive but insignificant estimates
for applicants and small but significant estimates for contracts,
indicating that deaths in counties with higher unemployment are not
as large a deterrent effect, and could potentially even make the recruiting response
to deaths positive. %Under the first specification for contracts, a county with the weighted average level of county unemployment (5.5\% in the sample) would have a 0.5\% reduction in recruits for every death. A county with 6.5\% unemployment would actually see a 0.5+ (6.5-5.5)*0.8=1.3\% increase in recruiting with every death (not to mention the 1.6\% increase in recruiting thanks to the level effect of county unemployment) .
+
The percentage of county population that is African-American increases
the size of the effect of a death. A county with the weighted average
proportion of the population (13\%) African-American would see a 0.7\%
reduction in applicants for every death, a county with one standard deviation
(13\%) higher African-American population would see a -0.7+(13\%{*}-0.065)=1.55\%
reduction in recruits for each death. The estimates are slightly larger for contracts than for applicants. %I have also done the analysis using racial fractionalization, a more detailed description of the racial makeup of counties instead of simply percent African-American, but these estimates are not significant.
+
``Rural'' is a binary measure of whether the county is rural using USDA's Rural-Urban continuum, which is partly a measure of population and partly distance from a metropolitan area. The estimate is insignificant, but the sign does seem to go in the same direction as population, however, indicating that rural counties
(with lower populations not neighboring metropolitan areas) would have larger negative recruiting responses to deaths. Deaths per Male 18-24 year-old County Population has a positive but only marginally significant coefficient. This indicates that perhaps the size of the county does not strongly affect how its population responds to a death. %Running the regressions without the rural interaction does not change the coefficient on the population interaction significantly.
+
Finally, I have interacted the county percent of the vote that went
to George Bush in 2004 with deaths. The coefficient estimates are 0.09\% for applicants to 0.15\% for contracts. This indicates that a county with the weighted mean Bush voteshare would see a decrease in applicants of 0.7\% for every death, but a county with one percentage point higher vote for Bush would see a 0.09\% smaller (closer
to zero) decrease in recruiting. This indicates that a county with
roughly 6 to 8 percentage point higher than the weighted average Bush
vote would see \textit{increases} in recruiting after deaths. The average Bush
vote share is 50.6\%, and the standard deviation is nearly 14 percentage
points. Well over half the counties had a Bush vote share over 57\%, which indicates that the prospect of an increase in recruiting due to deaths is not at all unlikely.
+
\cite{Ted-Miguel-Bush-Deaths} showed that, at least at the state
level, war deaths led to poorer Bush election performance in 2004.
As a robustness check I have replaced the Bush '04 vote share with
Bush '00 county vote-share, which was obviously unaffected by Iraq
and Afghanistan combat deaths. The estimates are nearly identical.
+
These estimates all show that county characteristics are very important
in determining the response of a county's potential recruits to the
news of a death. High unemployment may dampen the deterrent effect of deaths slightly.
Counties with higher fractions of African-American population have
a larger response to deaths, as did counties that voted against George
W. Bush (in either 2000 or 2004).
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Recruiting Response for Different Types of Recruits \label{sub:Different Recruit Types}}
+
+
The military, like any other organization, has a strong interest in
recruiting high quality employees. The services have generally held {}``high
quality'' to mean a person in possession of a high school degree
and a score of 50 or higher on the AFQT. The services have often had
separate quotas for high and low quality enlistees, and they have
generally required that a high percentage of their recruits fall into
the high category, although these requirements have changed over time
with the needs of the services.
+
%LN QUALITY OF RECRUIT
\begin{table}
\caption{Recruits by Quality}
\label{Flo: Recs by QualityLN}
\scalebox{0.70}{\rotatebox{0}{
{\input{../Output/highqualitybytypeLN}}}}
\end{table}
+
Table \ref{Flo: Recs by QualityLN} shows results for recruits
of different quality levels. I have broken recruits into four groups, the first three of which attempt to use definitions explicitly used by the military.
Low Quality recruits either scored below 50 on the AFQT or do not
have a high school degree. High Quality have both a 50 or higher on
the AFQT as well as a high school degree. High Quality-Alt scored
50 or higher on the AFQT but may still be in their senior year of
high school (many recruits sign contracts while they are still in
school, but join through the Delayed Entry Program, so they do not
actually ship out until they graduate and are considered high quality
recruits by the military.) Very High Quality recruits is not a specific
distinction used by the military, but is meant to identify the most
sought after recruits--those who have a 75 or higher on the AFQT and
have taken at least some college courses. %There are fewer observations in this specification since 666 counties never have a very high quality recruit and cannot be included.
The results indicate that all but the Very High Quality recruits have roughly the same response to deaths: a 1\% reduction in recruits for every
death. Amongst Very High Quality recruits, the effect is almost 4\%.%
\footnote{It may be slightly surprising that higher-quality recruits are more
deterred by local deaths if one interprets the response to a county
death as an {}``over-response'' compared to deaths from out of county
or out of state, since higher quality recruits are better-educated
and might be expected to read national newspapers or acquire information
about distant deaths with lower cost. Indeed regressions not shown
indicate that the response to out-of-county and out-of-state deaths
is no larger for higher quality recruits than for lower quality recruits.
However, the results are consistent with a story of the local-death-deterrent
being due to personal knowledge of the soldier who was killed, since
evidence indicates that those with more education are likely to have
larger social networks. As written in \cite{GlaeserSocialNetworks},
{}``The connection between social capital and human capital is one
of the most robust empirical regularities in the social capital literature.''}
+
%Another interesting difference is the effect of unemployment on different types of recruits. \emph{A priori} it is unclear how unemployment would affect different types of recruits. Higher unemployment could raise low quality enlistment because low quality individuals have fewer outside options, or it could hurt low quality individuals, because high quality individuals have their outside options eliminated, then they join the military in greater numbers, and there isn't enough demand remaining for low quality individuals to enlist. (While High Quality can typically enlist in the military at any time with no cap on demand, low quality recruiting is frequently subject to both demand and supply constraints.) The table seems to indicate that county unemployment leads to a 2.5\% increase in high quality recruiting and a 1.5\% increase in low quality recruits.
Again, I have tested whether the results are the same when done using Poisson regression, and the results exhibit the same patterns for deaths---very high quality recruits are more deterred by deaths than other types of recruits.
+
%The appendix includes a similar analysis in Table \ref{Flo:Recruits by Service} with recruits broken out by military service branch.
+
\subsubsection{Recruiting Response for Different Types of Deaths\label{sub:Different Death Types}}
+
In addition to responses for different types of recruits, I have run analysis comparing the response to deaths of different types, specifically, the service branch in which the death occurred, the gender of the deceased, the classification of the death by the military as hostile or non-hostile, the race of
the deceased, and the war in which the deceased was killed (Iraq or
Afghanistan). Deaths are found to have no significantly different
deterrent effect based on gender, hostility-status, and race.%
\footnote{%Although statistical tests cannot reject that the race of the death does not affect the size of the deterrent effect, I have also run regressions interacting the race of the death with county racial characteristics. The coefficient of interaction between black deaths and black population is twice the magnitude of the coefficient on the interaction of white deaths and black population, indicating that perhaps counties with more blacks are even more deterred by black soldier deaths, although the difference between these two interactions is again not significant. The appendix includes results in Tables \ref{Flo:Death by Same-Service Other-Service} and \ref{Flo: Death by Service}
I have also compared deaths across military service branches, finding no significant differences by branch.}
+
However, the war in which the death occurred has a significant effect of the recruiting response. Table \ref{Flo:Deaths by WarLN} shows that county deaths from Iraq lead to a 2.1\% decrease in recruiting in the following month, while county deaths from Afghanistan lead to a statistically instignificant increase in recruiting of 1.4\% in the following month. The same pattern holds, though slightly less pronounced, when one restricts the analysis to after March 2003 when both wars were occurring simultaneously, and for Poisson regression specifications, which are shown in Appendix Table~\ref{Flo:Deaths by WarP}.
+
This is further evidence that recruits are responding not only to the risk of death, but also to a subjective valuation of the circumstances of the death, as well as how their politics affect that valuation. Perhaps the perception that the war in Afghanistan was just while the war in Iraq was not is enough to completely change the direction of the effect. (Only Representative Barbara Lee of California voted against the Authorization for Use of Military Force in September 2001, while 133 Representatives and 23 Senators voted against the Iraq War Resolution in October 2002.)
\begin{table}
%built by redefhighquality.do
\caption{Deaths in Different Wars}
\label{Flo:Deaths by WarLN}
\scalebox{0.8}{
\input{../Output/redefLNwar}
}
\end{table}
+
%New on Feb 18 2015:
%This is in redefLNwarINT and redefPwarINT if you want it.
+
I have also tested specifications that separately interact Iraq and Afghanistan deaths with county characteristics. The coefficients on the interactions are not statistically different from one another, and go in the same direction for county population, percent African-American, and county unemployment---that is, both wars have positive interactions with unemployment, both negative for percent African-American, and both negative for population. However, the coefficients for the interaction of deaths with county percent George Bush vote share have different directions but are not significantly different, with positive coefficients for Bush voteshare interacted with Iraq deaths, and negative coefficients for Bush voteshare interacted with Afghanistan deaths. This implies that the incentive effect of a death in Afghanistan may actually be smaller in Bush voting counties than in non-Bush counties. That is, deaths in Afghanistan drew out more new recruits overall, and relatively more new recruits in non-Bush counties than in Bush counties. Deaths in Iraq led to an overall decrease in recruits, with an incentive effect in some Bush counties and a deterent effect in most non-Bush counties. This again seems to fit with the not uncommon perception that the country was united in response to 9/11 and the war in Afghanistan (the effects go in the same direction), but deeply divided over Iraq (the effects go in opposite directions in some counties).
+
\section{Conclusion}\label{sec:Conclusion}
+
+
A perfectly rational, fully-informed individual conforming to a standard economic model would become less likely to start employment in a profession when they learned that the profession in question was more dangerous. This paper presents evidence that young men and women enlisting in the military are not behaving in this mannner. Individuals respond more to a local death than to a death from farther away, and the difference cannot be explained by media markets. In addition, the evidence suggests that opinions about the war affect these decisions as well, since counties with more Democratic voters have a more negative response to deaths, and the nation as a whole has a more negative response to deaths in Iraq compared to deaths in Afghanistan. Unfortunately, I am unable to distinguish between a purely information-based explanation and a more behavioral explanation based on saliency, though information on media markets provides suggestive evidence in favor of a behavioral explanation.
+
As far as policy is concerned, I have shown in this paper that military deaths make the difficult and expensive task of recruiting significantly
more complicated. At the national level (as shown in the appendix), a one percent increase in the death rate is associated with a 1.5 to 2.5 percent decrease in
national recruiting in the following month. This should not necessarily
be given a causal interpretation, due to the potential for omitted
variable bias.
+
However, I make the case that panel data regression
analysis at the county level warrants a causal interpretation, as
I can flexibly control for county characteristics that are fixed across
time, national trends that are constant across different counties,
and even state-level time trends. Using both weighted least squares
and Poisson regression shows remarkably similar and stable estimates
of the effect of deaths of local soldiers on local recruiting. Each
in-county death leads to a one percent decrease in that county's recruiting
in the next month, and this finding is robust across several specifications.
Therefore a large fraction of the overall deterrent effect of deaths appears
to be due to local deaths. I have also shown that the local effect
is in fact quite concentrated---deaths in contiguous counties and
deaths in counties in the same media market lead to smaller decreases in recruiting.
+
A one percent reduction may be small in terms of practical significance, but this effect is equal in magnitude to the effect of a one percentage point decrease in unemployment, and may be of use to the military, especially given that the localized deterrent effect also exhibits heterogeneity in interesting fashions. Counties that voted for George W. Bush in 2000 or 2004 see very different, and even positive recruiting responses
to local military deaths. Counties with higher than average African-American
populations see significantly more negative responses to local deaths. The military
also sees the largest reduction in recruits of the highest quality
(as measured by AFQT score and educational attainment) after a local
death. %However, it does not appear that recruits are responding significantly differently to deaths in their own service branch than to deaths from a different service branch.
+
To the economist who assumes actors have full information and are completely rational, it is nonetheless puzzling that there would be any difference in the response to a local death than to a death from farther away. I have documented that the likelihood of dying is not related to the location in which one enlists, so this paper provides evidence of
a larger response to local matters than is justified based on calculation
of risk alone. Models of non-standard decision making that include
a salience parameter such as \cite{ChettySalience} or \cite{eBayEarly}
may be able to better explain the observed recruiting phenomenon.
+
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%References here (manual or bibTeX). If you are using bibTeX, add your bib file
%name in place of BibFile in the bibliography command.
% Remove or comment out the next two lines if you are not using bibtex.
+
\newpage
\bibliographystyle{plainnat}
\bibliography{military_bib}
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% The appendix command is issued once, prior to all appendices, if any.
\newpage
\appendix
+
\section{Supplementary Appendix}
\setcounter{table}{0}
\renewcommand{\thetable}{A\arabic{table}}
+
\setcounter{figure}{0}
\renewcommand{\thefigure}{A\arabic{figure}}
+
+
\subsection{Detailed Data Description}
+
I obtained the recruiting data used in this paper through Freedom
of Information Act requests to the office of the Secretary of Defense, which were handled by the Defense Manpower Data Center. It consists of three
distinct sets of individuals: {}``applicants,'' {}``contracts,''
and {}``accessions'' (explained above) and contains the date on
which these individuals were recorded as starting one of the three
specific parts of the recruitment process, ZIP code, AFQT score, educational
attainment, and branch and component of the military to which the
potential recruit was applying. The same data is available for applicants
and contracts, but the data are stored separately for each step in
the recruiting process and are unfortunately not linked by individuals
across datasets. Age of recruit is also unfortunately not included.
+
I have recruiting data for fiscal years 1990-2006. (The military operates
on an October 1-September 30 fiscal year.) The applicants data set
contains 6.4 million active duty observations, the contracts data
set has 3.6 million active duty observations, and the accessions data
set has 3.0 million active duty observations.\footnote{There are nearly 50\%
more recruits when one includes reserve and guard recruits (I
observe roughly 9 million total applicants), however it appears that
much of the contracts data for Reservists and Army and Air National
Guard are missing. The data contain only 375 Army Reservist contracts,
an implausibly low number over a 17-year period. Note that all active duty (both applicants and contracts) and reserve/guard applicants data appears complete, at least as far as any civilian can confirm data obtained through FOIA. To account for the missing reserve and guard contracts, all of the analysis is run using only the applicants or contracts to the active duty components of the military. The main regression specification is repeated in the Appendix Table~\ref{Flo:allrecruits} using all recruits (active, reserve, and guard), which results in very similar, if not slightly larger, estimates.} I am able to match roughly 96\% of the applicant observations by ZIP code to a US county.
+
The military has an enlistment option referred to as the Delayed Entry Program (DEP) for enlistees who do not wish to or are not able to ship to basic training immediately upon signing their contract. DEP enlistees sign a contract to ship to training up on a specific date up to one year in advance. For example, many enlistees sign a contract during their senior year of high school and ship to basic training after graduating. Of course, some who sign a DEP contract do not access, and instead are discharged. My data unfortunately contains no information on whether an applicant/contract/accession enlists via the DEP or direct from contract to accession, and because the FOIA data does not enable me to track individuals across the enlistment process, I am unable to back out this information. A potentially fruitful avenue of research for those with direct access to administrative data would be to track individuals through the process, and to measure the rate of DEP discharges.
+
%%What does this do? Classical measurement error? Or since the X variable is likely to be under-reported if anything, is that not necessarily attenuation bias?
The military has typically not released or maintained publicly available
datasets of the deaths of its soldiers in the last two decades, which
were numerous even in times of peace. The onset of the wars in Iraq
and Afghanistan changed this, as the military made data on a large number of
deaths available to the public. The main fatality data come from a public list compiled by the Statistical
Information Analysis Division at the Defense Manpower Data Center
and freely obtained from their website.\footnote{This data was obtained from \url{http://siadapp.dmdc.osd.mil/personnel/CASUALTY/castop.htm} in 2010, but that site no longer operates. Similar data was available in 2017 from the Defense Casualty Analysis System\url{https://www.dmdc.osd.mil/dcas/pages/casualties.xhtml}. I archived this data at \url{https://dataverse.harvard.edu/dataverse/garretchristensen}.}
+
Starting October 7, 2001, every fatality in Operation Iraqi Freedom
and Operation Enduring Freedom is listed, and includes
the service branch, component (active/reserve/guard), name, rank,
pay grade, date of death, hostile status of death, age, gender, home
of record city, home of record county, home of record state, home
of record country, unit, incident geographic code, fatality geographic
code, fatality county, city of loss, and race/ethnicity of the deceased.
+
The data includes {}``home of record'' which is where the soldier lived on the day they joined the service, and generally does not change over the course of a single military enlistment contract, but can change at the time of reenlistment. It is separate and distinct from a legal residence for tax purposes. Still, this is important with regards to my claim of plausible random assignment of death with respect to county after controlling for military population levels---the data should not be biased by service-men and women with very dangerous military professions buying homes near their duty-base and changing their Home of Record to the county in which the base sits. To test this, however, I have looked separately at deaths of low paygrade soldiers compared to deaths of high paygrade soldiers (who may have reenlisted and changed their home of record). These regressions are shown in the appendix; there are no significant differences in the deterrent effect by paygrade of the soldier who was killed.
+
Since the focus of this paper is recent combat deaths, the main 2001-2006 fatality data used in the majority of this paper does not include the fairly common deaths of military members unrelated to combat abroad, or unrelated to the military at all (heart attacks, car accidents). After earlier drafts of this paper were completed, a FOIA request for detailed death data for all deaths in the military (not just those in the wars) for the entire period for which I had recruiting data (1990-2006) was granted. I use these deaths as placebo tests of my analysis and find that, as expected, there is no significant recruiting effect of local non-combat deaths. This is discussed in the appendix and shown in Table~\ref{Flo:LNlinear90}.
%and \ref{Flo:alldeaths90}.
+
\subsection{National Level Analysis}
Although this paper focuses on the effects of local deaths on local
recruiting, it is worthwhile to briefly discuss the effects of total national
deaths on national recruiting. Table \ref{Flo:TABLE: Simple Time Series}
shows a simple linear regression analysis of the the national time series of monthly combat deaths and log monthly total applicants from October 2001 through July 2006. Table \ref{Flo:TABLE: Simple Time Series P} shows the same specifications using Poisson regression. Graphically, spikes in deaths after the initial invasion of Iraq and the first
and second battles for Fallujah (the obvious high points in the figure)
are very clearly followed by decreases in recruits. With one observation
for every month nation-wide, there are only 58 observations, but there
is still a strong and consistent negative correlation between deaths
in the current and/or previous month and recruits. In terms of semi-elasticities,
as shown in the table, one deaths is associated with a 0.15 to 0.2 percent decrease in applicants and a similar reduction in contracts. (Standard non-logged OLS regressions show
that deaths are associated with 60-90 fewer applicants and 27 to 43
fewer contracted recruits.) So it would seem that deaths in the military
are followed by an overall decrease in the national number of recruits.
This should not necessarily be given a causal interpretation, as a
simple linear time trend is not nearly enough to control for all the
unobserved changes that occurred in the country over this nearly five-year
period, all of which could be biasing the estimate up or down. However, it is interesting to note that the correlation between national deaths is much smaller than the county effect (.15 compared to 1.1 percent).
+
%The same potential problem does not exist once I narrow the analysis to a finer geographic level and use repeated observations from multiple states or counties over time, where I can use fixed effects to flexibly control for unobserved characteristics. It seems prima facie obvious thatthere are significant differences between states and counties that might be correlated with both the number of deaths and the number of recruits. County population jumps to mind. A naive regression of counties' recruits on death without accounting for population would have an upward-biased estimate due to omitted variable bias, since with equal recruiting rates, the higher a county's population, the more recruits from that county, and mechanically, the more deaths from that county. With an obvious measure like population, it is possible to get rough estimates of the population that change over time, but with many more subtle county characteristics, such as abstract support for the military, it is not possible to get even one estimate for the county, let alone multiple measurements over time, thus the need for fixed effects to flexibly control for all immeasurable county characteristics fixed over time and reduce or eliminate omitted variable bias.
+
%both built by deathsvrecruits.do
%National Time Series
%National-LogLinear (SEMI)
\begin{table}[h]
\caption{}
\label{Flo:TABLE: Simple Time Series}\input{../Output/deathsvrecruitsSEMI.tex}
\end{table}
+
%National-Poisson
\begin{table}
\caption{}
\label{Flo:TABLE: Simple Time Series P}\input{../Output/deathsvrecruitsP.tex}
\end{table}
\clearpage{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\subsection{Distribution of Recruits Across States}
%Figure \ref{Flo: Figure: Recruits vs. Population}
%shows that, as one would expect, there are clearly differences in
%state populations. The horizontal axis is a state's percentage of
%the nation's male 18-24 year-old population, and the vertical axis
%shows a state's percentage of the applicants over the entire 16-year
%period for which I have data. With one observation for each state,
%a constant national propensity to enlist would yield a population
%weighted OLS regression coefficient of 1, which can be easily rejected
%statistically. Figure \ref{Flo:Figure: Pop vs. Deaths} shows a state's
%percentage of the young male population and its percentage of the
%deaths in Iraq and Afghanistan. This slope is also statistically different
%from 1, indicating that there is not one constant national likelihood
%of death in the military. This finding may be unsurprising, and is
%shown only since it may be interesting to know in and of itself which
%states have higher proportions of their young men enlist and die in
%the military.
+
%Graph of recruit pct by population percent by state
%\begin{figure}
%\includegraphics[scale=0.6]{../Output/graph_table1_rec_pop}
%\caption{State Percentage of Recruits by State Percentage of Young Male Population}
%\label{Flo: Figure: Recruits vs. Population}
%\end{figure}
+
%Graph of population percentage and death by state
%\begin{figure}
%\includegraphics[scale=0.6]{../Output/graph_table1_death_pop}
%\caption{State Percentage of Population vs. State Percentage of Deaths}
%\label{Flo:Figure: Pop vs. Deaths}
%\end{figure}
+
\subsection{Location of Origin and Rates of Death\label{sec:deathrate}}
The specific military occupational specialty (MOS) for which the residents of certain counties or states are likely to sign up, and the corresponding likelihood of death faced by those in certain MOS affects the interpretation of my results. One could imagine that those in the infantry are more likely to be killed than those in ancillary support operations, and one could imagine that recruits from certain states are more likely than others to sign up for more dangerous occupations. However, I find that although counties do tend to send their recruits to different service branches, the rate of death faced by a recruit is statistically the same for all but a very small number of counties. %(In the appendix, I establish the perhaps unsurprising fact that recruits and deaths are not uniformly distributed across the population. Here I restrict my attention to the likelihood of death \textit{given} enlistment.)
+
The first step in this analysis is conducted with a dataset separate from the deaths and recruits data, obtained after multiple FOIA requests, showing the total enlisted employment for each of the thousands of MOS by county and month. Using the midpoint month from my analysis (March 2004), I compare each county's distribution of employment across the four service branches to the national average distribution using Pearson's Chi-squared test. Using the observations in the dataset that I am able to match to a county, employment is split 35\% Army, 26\% Navy, 16\% Marines, and 23\% Air Force. The answer is clear: counties do not all send the same fraction of recruits to the four service branches, so the same applies to MOS. Without any correction for the multiplicity of hypothesis tests, as many as 1,225 of the 3,125 counties have ratios of employment that are different than that of the national average at the 95\% confidence level. Using the Bonferroni, Benjamini Hochberg, and Benjamini Krieger Yekutieli corrections indicate that 304, 901, and 977 of the 3,125 have statistically different employment distributions at the 95\% confidence level, respectively \citep{BenHoch1995, BKY2006}. Results are very similar using other months of the data.
+
The second step is more consequential. Just because counties send different fractions of their enlistees to different service branches and MOS does not mean that an enlistee from a certain county was more likely to die in Iraq or Afghanistan. To examine this issue, I compare the number of recruits from a state (or county) to the number of deaths from the same state (or county). Figure \ref{Flo:HISTOstatedeaths} shows histograms of the ratio of active duty deaths to total active duty applicants for each state over the whole period for which I have data, and Figure~\ref{Flo:HISTOcountydeaths} shows the same by county. The state ratios are centered around 0.3\%, but are clearly not all identical. I have repeated this exercise including both active duty and reserve and guard deaths (since service and death in the reserve and guard duty is clearly correlated with where one lives, including them might lead to complications) for both applicants and contracts, using both unweighted and population-weighted means. The coefficients of variation for each
of these eight methods of calculating the risk of death by state are
relatively small, ranging from 0.14 to 0.32. Looking at county figures shows that the majority of counties have zero deaths. While some counties clearly exhibit higher rates, testing is required to determine whether the variation is significant.
+
%State death hazard graphs
%built by Table1graph.do
\begin{figure}
\includegraphics[scale=0.5]{../Output/hist_state_combined.png}
\caption{Death Hazard Rates by State: Active Duty/Total Deaths (2001-2010) and Active Duty Applicants/Contracts (1990-2006)}
\label{Flo:HISTOstatedeaths}
\end{figure}
+
%County death hazard graphs
%built by table1bycounty.do
\begin{figure}
\includegraphics[scale=0.5]{../Output/hist_county_combined.png}
\caption{Death Hazard Rates by County: Active Duty/Total Deaths (2001-2010) and Active Duty Applicants/Contracts (1990-2006)}
\label{Flo:HISTOcountydeaths}
\end{figure}
+
To do this, I look at each individual county, and check the likelihood that it came from a binomial distribution with the hazard rate equal to that of the overall national hazard rate. (The number of active-duty deaths divided by the total number
of active duty applicants was .003.) I then tested the likelihood
that each county observation came from a binomial distribution with this
hazard rate of p=.003.
+
Figure \ref{Flo:Pat's Histo} displays
histograms of the p-value for each state and county, one set using active-duty deaths
and active duty applicants, the other using active-duty deaths and
active-duty contracts. If counties exhibit statistically indistinguishable rates of death, then p-values should be high. The histogram displays the p-values as calculated,
but to interpret, one should, as above, use an adjustment for the high degree of multiple testing present such as Bonferroni (i.e., for states divide the cutoff for significance by the number of tests, 51, thus replacing a cutoff of .05 with 0.05/51=0.0009). Only two
of the state observations (Florida and Massachusetts) reject the null
hypothesis that their true probability is in fact .003 using applicant
data, and only one, Massachusetts, rejects using contracts data.
+
Repeating this analysis with the 3,125 counties shows that rates of death given enlistment by county are also very rarely significantly different (six times for applicants and zero for contracts).\footnote{The Bonferroni correction is often considered to be quite conservative. Even using much more modern methods to adjust the False Discovery Rate (FDR), I obtain similar results. Both the Benjamini Hochberg q-values and Benjamini Krieger Yekutieli sharpened q-values indicate that 16 out of the more than 3,100 counties have significantly different death rates for applicants, and zero for contracts. See \cite{BenHoch1995, BKY2006, Anderson2008} for details on estimating FDR.} So while it is the case that recruits from certain counties are more likely to enter dangerous military occupations, according to the data, the idea that the risk of death is the same across all states can only be rejected very infrequently.
+
\begin{figure}
%built by Table1Graph.do AND table1bycounty.do
\includegraphics[scale=0.5]{../Output/hist_binomial_combined.png}
\caption{Histogram of P-Values Testing Whether Deaths Come from the
Same Binomial Distribution}
\label{Flo:Pat's Histo}
\end{figure}
\clearpage{}
+
This is not necessarily a cause for concern regarding omitted variable bias and my estimates of the deterrent effect of deaths,
since fixed effects for each county will still be able to control for this underlying characteristic of the state and county to the extent it is constant over time. However, it does give a slightly different meaning to the estimates I develop in the next few pages. If deaths in Iraq and Afghanistan were truly uniformly
distributed amongst all the troops, regardless of county of origin, then the fact that a soldier from a given county, say Fairfax
County, Virginia, had died would provide no more information regarding
the risk of death to a potential recruit from Fairfax County upon
enlisting than would the death of a soldier from Maricopa County,
Arizona. Any extra deterrent to enlisting because this death happened to a local soldier would thus be an emotional or behavioral response and not
an accurate updating of preferences based on risk. However, if a recruit
from Fairfax County were more likely to sign up for front-line occupations,
and those are the soldiers who were dying, then this death might actually
contain a useful signal as to the risk of death, and an extra deterrent
effect might be warranted for those reasons. Looking at the above
histograms of the rates of death by state, it seems that soldiers
from different locations have insignificantly different hazard rates, and
it is not the case that one state or another has a vastly different
rate of death of its soldiers, and the behavioral explanation is still reasonable.
+
\subsection {Additional Robustness Checks}
\paragraph{Other functional forms}
Economists often favor linear models in applied work, and these are shown in the main body of the paper, but some would consider Poisson regressions to be the most natural fit for count data with a large number of zeros that have no log, so I have exhaustively tested other models and find generally similar results across all specifications. I present these results here.
+
Poisson regression fits a generalized linear model of the form $log(\mu_{i})=x'_{i}\beta$, so $\mu_{i}=exp(x'_{i}\beta)$ and a one unit increase in $x_{j}$ multiplies $\mu_{j}$ by $exp(\beta_{j})$. However, as I am modeling an underlying rate of enlistment, $\mu_{i}=e^{x'_{i}\beta}$, the observed number of recruits is the rate times the exposure, which in my case is the population of young males. If $R_{i}$ is the expected number of recruits, then $R_{i}=Population_{i}\cdot e^{x'_{i}\beta}=e^{ln(Population_{i})+x'_{i}\beta}$.
Thus all the Poisson models have been fitted with a coefficient constrained
to 1 for the county's log young male population.\footnote{Regressions without the offset give qualitatively similar results.}
+
The detailed specification I estimate follows the equation:
$$Recruits_{it}=Population_{i}\cdot e^{\beta_{0}Deaths_{i,t}+\beta_{1}Deaths_{i,t-1}+\beta_{2}Unemployment_{it}\\
+x'_{it}\eta+\alpha_{i}+\gamma_{t}+u_{it}}$$
where Deaths implies deaths from the given county, Unemployment county unemployment, and $x$ includes in-state (but out of county) deaths as well as
state unemployment. $\alpha_{i}$ is a set of fixed effects for each county, which flexibly control for any county characteristics such as the presence of a military base or political support for the military. $\gamma_{t}$ is a set of fixed effects for every month, so national characteristics that are the same across counties in any
given time period such as the total national number of deaths, national
unemployment rate, or the military wage rate are also flexibly controlled for and cannot be separately estimated.
+
%Ordinary least squares results for recruit levels (as opposed to logs) are shown in Appendix Table \ref{Flo:County-basic}. All observations are weighted by county population.
%Completely Linear Regression
%could be done now in redefined.do but no one cares
%\pagebreak{}
%\begin{table}
%\caption{}
%\label{Flo:County-basic}\includegraphics[bb=45bp 0bp 612bp 730bp,clip, scale=0.95]{../Output/PDF/redefW}
%\end{table}
+
%(I have
%run un-weighted regressions, in which the results are equally significant,
%if only about 1/3 as large).
+
%One can see that with a full set of fixed effects, in-county deaths in the previous month are associated with 15 fewer applicants and nearly 10 fewer contracts. In-state deaths are associated with 0.3 to 1.1 fewer applicants, and 0.2 to 0.7 contracted recruits fewer, although the estimate is sometimes positive and not always statistically different from zero. The coefficient on lagged in-county deaths is the main estimate of interest, and is remarkably stable across different sets of fixed effects after county-level fixed effects are included.
+
%I have calculated, but do not show estimates of the elasticity of the recruiting response to death based on least squares regressions. They indicate that a one percent increase in the number of in-county deaths leads to a 2.7\% reduction in in-county recruits in the next month. However, I put less emphasis on this result for two reasons. First, the vast majority of county-months do not observe a death, and thus one cannot take the natural log of the data for simple elasticity calculations. Second, since most county-months have zero deaths, when any number of deaths is observed, there has actually been an infinite percent increase in the number of deaths.
+
%There are of course methods to deal with the first problem and other ways to calculate elasticities, however the second reason makes the interpretation of the 2.7 coefficient fairly odd. It's hard to know what sort of effect is implied by a $2.7*\infty\%$ decrease in recruits.
%Semi-elasticity estimates, where the right-hand side variable is in levels and the left hand side is logged eliminates some of the problems with logs of zero, but a non-trivial 30\% of county-months have zero active-duty recruits, so the log-linear OLS model is by no means ideal, even if it is more easily comparable to the Poisson estimates shown above. One advantage of these linear models, however, is that I can test inclusion of state-by-year fixed effects which improves my case for causal identification. Unfortunately I cannot include these state-by-year fixed effects in Poisson regression due to convergence issues, however, the extra fixed effects result in similar estimates as the other specification; in some cases they make the effect appear even larger.
+
Appendix Table \ref{Flo:Poisson Basic} shows the Poisson semi-elasticity estimates for both applicants and contracts. The dependent variable is active duty recruits. Observations have been weighted by county population. The semi-elasticity estimates show that an in-county death in the previous month leads to a 0.8 to 0.9\% reduction in both applicants and contracts, very similar to the log-linear regressions in the main body of the paper. Out-of-county deaths lead to small increases in recruiting.
+
%POISSON BASIC
%built by redefpoisson.do
+
\begin{table}
\caption{Poisson Regressions of County Recruits on Deaths and Unemployment}
\label{Flo:Poisson Basic}
\scalebox{0.9}{\input{../Output/redefPbasic}}
\end{table}
+
%Negative Binomial
%built by negbinom.do
\pagebreak{}
\clearpage{}
\begin{table}
\caption{}
\label{Flo:negbinom}
\scalebox{0.9}{\input{../Output/negbinom}}
\end{table}
+
Additional specifications include modeling %the left hand side variable as
with a negative binomial conditional fixed effects regression, with results shown in Table~\ref{Flo:negbinom}. Again, results are very similar: by far the largest deterrant effect comes from in-county deaths. Problems exist with negative binomial fixed effects regression, as explained in \cite{negbinom}, so these results are presented only to exhaustively test other functional forms and show that results are robust.
+
\paragraph{Tests of Additional Controls}
Table~\ref{Flo:LNRecandMort} shows additional specifications that control for county level young male mortality and the statewide number of recruiters. Comparing columns with and without the controls show no signficant differences.
+
\begin{table}
%built by interactionscontrols.do
\caption{Recruiter and Mortality Controls}
\label{Flo:LNRecandMort}\input{../Output/LNcontrolW.tex}
\end{table}
+
+
\paragraph{Tests of Additional Data}
In additionally to exhaustively testing functional forms and getting similarly robust results, I also obtain extremely similar results when I test only subsets on the data. For instance
in addition to testing every recruit, I also test the model by using only deaths from active duty soldiers, as opposed to deaths of all soldiers (active, reserve, and guard) as in all other specification. Results, shown in Appendix Table~\ref{Flo:Rdeathslinear} appear slightly \textit{stronger}, in that deterrent effects of in-county deaths are estimated to be slightly larger than one percent. Appendix Table~\ref{Flo:RdeathsP} shows Poisson regressions using only active duty deaths and again, the results are robust.
+
Tests of lead periods, i.e. testing the regression specification by testing for implausable effects of deaths in the future on current recruiting, are shown in tables~\ref{Flo:forwardbasicLN} and~\ref{Flo:forwardPbasic}. The main linear specificiation seems to have unexplained small effects in the lead period, but the Poisson specification, as well as specifications using only active duty deaths (not shown), both seem to pass this falsification test, in that deaths in the future have no relation to current recruiting.
+
+
+
%Using Only Active Duty DEATHS (all the R variables)
%Linear. Have Poisson as well.
\begin{table}
\caption{Active Duty Deaths Linear}
\label{Flo:Rdeathslinear}
\scalebox{0.9}{
\input{../Output/LNLinearWR.tex}
}
%built in redefined.do
\end{table}
+
+
\begin{table}
%built in redefinedpoisson.do
\caption{Active Duty Deaths Poisson}
\label{Flo:RdeathsP}
\scalebox{0.85}{\input{../Output/redefPbasicR}}
\end{table}
+
+
+
\clearpage{}
\begin{table}
%built by redefined.do
\caption{Testing Effect of Leads: Linear}
\label{Flo:forwardbasicLN}
\scalebox{0.8}{
\input{../Output/forwardbasicWLN}}
\end{table}
+
%Future Deaths Have No Effect
%built by redefinedpoisson.do
\pagebreak{}
\clearpage{}
\begin{table}
\caption{Testing Effect of Leads: Poisson}
\label{Flo:forwardPbasic}
\scalebox{0.9}{
\input{../Output/forwardPbasic}}
\end{table}
+
+
I also use additional data on deaths acquired through a FOIA request. This data consists of all deaths of anyone in the military from 1990-2006, regardless of circumstance or location. Thus it includes numerous deaths that are completely unrelated to the wars in Iraq or Afghanistan, and even unrelated to the military---heart attacks, car accidents, etc. As I expected given the comparatively banal circumstances of the majority of these deaths, all regressions indicate no significant recruiting response to in-county deaths, neither in the entire 1990-2006 period nor the 2001-2006 period used in the main regression tables. These results are shown in Appendix Table~\ref{Flo:LNlinear90}.% and ~\ref{Flo:alldeaths90}.
+
%Non-Combat Deaths, 1990-2006
%buily by PandLN90.do
\begin{table}
\caption{}
\label{Flo:LNlinear90}
\scalebox{0.8}{
\input{../Output/LNLinear90.tex}}
\end{table}
+
%Works, but why bother?
%\begin{table}
%\caption{}
%\label{Flo:alldeaths90}
%\scalebox{0.8}{
%\input{../Output/redefPbasic90.tex}}
%\end{table}
+
Tests of the entire set of recruits (active duty, guard, and reserve troops, not just active duty recruits as are used in most specifications) reveal similarly robust results--the deterrent effect appears even larger under this specification. Results are shown in Appendix Table~\ref{Flo:allrecruits}.
+
%made in redefined.do on 7/26/2016
\begin{table}
\caption{}
\label{Flo:allrecruits}
\scalebox{0.8}{
\input{../Output/allrecLNLinearW}}
\end{table}
+
\paragraph {Heterogeneity}
Tests of the heterogeneity of the effect by which the war in which the death occur are similar using a Poisson regression to those using linear regression in the main body of the paper. Results are shown in Appendix Table~\ref{Flo:Deaths by WarP}.
+
\begin{table}
\caption{Deaths by War, Poisson}
\label{Flo:Deaths by WarP}
\input{../Output/redefPwar}
\end{table}
+
I have also conducted tests by the paygrade of the soldier who was killed. This partially stems from a concern that Home of Record may change over the course of a military career. Note, however that legal residence and military Home of Record are separate and distinct. (For a thorough explanation see \url{https://www.army.mil/article/160640}.) Still, if Home of Record changes over the course of a military career when a soldier re-enlists, the death of a soldier may not be felt most accutely somewhere other than their Home of Record. To evaluate this, I separate the deaths by paygrade of the deceased, creating high paygrade and low paygrade groups for enlisted soldiers, and separately for officers (including warrant officers---officers and warrant officers make up 10\% of the total deaths in my sample) to approximate the ability of soldiers to change their Home of Record. Higher paygrade soldiers have, on average, been in the military longer and are more likely to reenlist. I test different definitions of the cutoff for high and low enlisted paygrades, paygrade E4 or lower (62\% of enlisted deaths), and higher than E4. I repeat this same division with E5 and below as the cutoff (81\% of the enlisted deaths). The results are shown in Appendix Table~\ref{Flo:paygrade}. Though it appears that lower paygrade soldiers have a larger coefficient of deterrent among applicants, coefficients are larger for contracts, and none of the differences are significant. I have tested other ways of dividing the sample (officers vs. enlisted, using E3 as the cutoff) and all these other specifications also showed no signficant differences in the size of the deterrent effect.
+
%made in paygrade.do on 4/6/2017
\begin{table}
\caption{}
\label{Flo:paygrade}
\scalebox{0.7}{
\input{../Output/paygrade}}
\end{table}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Recruits and Deaths by Service Branch}\label{sec:servicebranch}
In this section I examine how the individual services fare in their recruiting. There are clearly differences between the services in terms of their operations; perhaps stemming from this there are often assumed to be significant cultural differences between the branches, leading to a difference in the type of people who make up the potential applicant pool for each of the services, and the possibility for a difference in the potential applicants' response to a local death.
+
Table \ref{Flo:Recruits by Service} shows these estimates, again using weighted least squares analysis at the month-county level, with county, monthly, and state-year fixed effects. It appears that Marine recruiting decreases at a rate of over 2\% for every death, while the Air Force reaction to death is a statistically insignificant 0.1\% reduction in applicants. The pattern is the opposite for contracts, however, with the largest decrease among Air Force recruits. Data that could follow individual recruits from application to contract to accession could potentially further illuminate this question, but unfortunately my data cannot.
+
Deaths at the state level and state and county unemployment seem to have effects that are less clearly distinguishable across service branches. This makes sense given that very few of the deaths in Iraq and Afghanistan have come from the Air Force, while many have come from the Marines. But an even larger proportion have come from the Army, so if potential recruits were simply steeringaway from the branches of the military with the most deaths, the Army would be the branch with the largest recruiting response to deaths, which is not the case. I have repeated this exercise using Poisson regressions, and the deterrent effects by service branch maintain the same ratios relative to one another, exhibiting further evidence that the recruiting deterrent is greatest for the Marine Corps.
+
% Break recruits out by service branch
%made by servicebranch-simple.do
\begin{table}
\caption{}
\label{Flo:Recruits by Service}
\scalebox{0.8}{
\input{../Output/servicebranchrecLN}}
\end{table}
+
Amongst the four branches of the military, the deaths in Iraq and Afghanistan have been highly concentrated amongst soldiers in the Army and the Marines. In the data used in this paper, roughly 2000 deaths are from the Army, 800 from the Marines, 85 from the Navy, and 50 from the Air Force. Since there have been 40 times more deaths in the Army than the Air Force, it is entirely possible that potential Army recruits have instead gone on to join the Air Force instead.
+
One way to get at the question of deaths by service branch is to lump all the recruits together but show the specific response to deaths in a given service. These estimates are shown in Table \ref{Flo: Death by Service}. For applicants, the coefficients are large and positive for Air Force and the Navy in-county deaths, however they are not statistically significant for the Navy. I have also run specifications with the in-state but out-of-county deaths split by service branch, and I have run tests of equivalence of the coefficients for all the service branches. The null hypotheses of equality can be rejected for county deaths for applicants, but cannot be for state deaths. Neither tests reject for contracts.
+
% Break deaths out by service branch
%made by servicebranch-simple.do
\begin{table}
\caption{}
\label{Flo: Death by Service}
\scalebox{0.8}{
\input{../Output/servicebranchdeathLN}}
\end{table}
+
To test this another way, I have rearranged the data into county-month-service branch observations and test the recruiting response to a specific service branch after deaths from the same service branch and from other service branches. (There are four observations for each county month, and I control for same-service-branch deaths as well as other-service-branch deaths in a given county-month, so regressions shows the percent response to a death in the same service branch as the recruit, and in any of the other three service branches.) As before, I include a full set of fixed effects, and state and county unemployment. Results indicate that there are not statistically significant differences in the response to lagged same-service-branth deaths and lagged other-service-branch deaths. These results are not shown but are available upon request.
+
To some extent my results show a similar effect to that of \cite{cunha2014expedited}---their research shows that non-citizen enlistees switch away from the service branches with more fatalities (Army, Marines) towards the Navy and Air Force---but my results appear to find less clear evidence for a consistent or similar story. Contracts to the Air Force decrease the most due to a fatality, but Applicants decrease the least, and Navy and Air Force deaths do not appear to have consistent and significant effects on recruiting.
%Table \ref{Flo:Death by Same-Service Other-Service} shows the results of these tests.
+
%In these regressions I have included interacted county{*}service branch fixed effects as well. The bottom of the table shows pair-wise comparisons of the corresponding same-service and other-service death coefficients. For example, for applicants, a lagged same-service in-county death leads to a 0.7\% reduction in recruits in that service, while a lagged other-service in-county death leads to a 1.1\% reduction in recruits for that service. In all of the specifications, tests fail to reject the hypothesis that the effect of a lagged same-service in-county death is statistically identical to that of a lagged other-service in-county death. The same can be said of lagged same-service out-of-county deaths and lagged other-service out-of-county deaths. However, for several of the current-month deaths, statistical tests reject equivalence.
+
%They indicate that same-service deaths have a larger (and positive effect) than do other-service deaths. \emph{A priori} I would have assumed that if anything, same-service deaths would lead to a larger decrease in recruits, since recruits could either not sign up or substitute to a different service, but this does not seem to be the case. The results are further evidence that potential recruits are not using the information contained in a death (in this case, the service branch in which the death occurred) in a sophisticated or strictly risk-based manner.
+
%
+
+
+
+
%POISSON RECRUITER AND MORTALITY CONTROLS
%\clearpage{}
%\pagebreak{}
%\begin{table}
%\caption{DO IN LOG LINEAR!!!}
%\label{Flo:Poisson Rec&Mort}\includegraphics[bb=45bp 0bp 612bp 730bp,clip,scale=0.95]{../Output/PDF/redefPrec}
%\end{table}
+
%MEDIA MARKET AND CONTIGUOUS-IN POISSON
%built by redefcontig.do
%\pagebreak{}
%\begin{table}
%\caption{Poisson version of media and neighboring}
%\label{Flo:Media}\includegraphics[bb=45bp 0bp 612bp 730bp,clip, scale=0.91]{../Output/PDF/redefcontigP}
%\end{table}
+
%POISSON Cumulative Lags
%\clearpage{}
%\pagebreak{}
%\begin{table}
%\caption{}
%\label{Flo:Cumulative LagsP}\input{../Output/redefrunninglagsP}
%\end{table}
+
%POISSON INTERACTIONS
%\clearpage{}
%\pagebreak{}
%\begin{table}
%\caption{Interactions in Poisson}
%\label{Flo:Poisson Interactions}\includegraphics[bb=45bp 0bp 612bp 730bp,clip,scale=0.95]{../Output/PDF/redefPinteractions}
%\end{table}
+
%POISSON QUALITY OF RECRUIT
%\pagebreak{}
%\begin{table}
%\caption{}
%\label{Flo: Recs by QualityP}\input{../Output/highqualitybytypeP}
%\end{table}
+
+
%All Recruits, including the messed up reserve/guard
%\pagebreak{}
%\clearpage{}
%\begin{table}
%\caption{I NEED TO SWITCH THIS TO LOG LINEAR!}
%\label{Flo:allrecruits}
%\includegraphics[bb=45bp 0bp 612bp 730bp,clip,scale=0.95]{../Output/PDF/allrecPbasic}
%\end{table}
+
+
+
+
+
%11 Monthly county service branch observations
%\pagebreak{}
%\clearpage{}
%\begin{table}
%\caption{}
%\label{Flo:Death by Same-Service Other-Service}\includegraphics[bb=45bp 0bp 612bp 730bp,scale=0.91]{../Output/PDF/redefbyserviceP}
%\end{table}
+
+
+
+
\end{document}
+
+ + + +
+ +
+ + + + +
+ +
+ +
+
+ +
+ + + + + + +
+ + + You can't perform that action at this time. +
+ + + + + + + + + + +
+ + You signed in with another tab or window. Reload to refresh your session. + You signed out in another tab or window. Reload to refresh your session. +
+ + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/slides_w_beamer.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/slides_w_beamer.pdf new file mode 100644 index 0000000..88970cb Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/slides_w_beamer.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/test.bib b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/test.bib new file mode 100644 index 0000000..c87ff67 --- /dev/null +++ b/reading_material/3 - Reproducible Research and Data Analysis/Annual2017-master/test.bib @@ -0,0 +1,17774 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Annual2017/test.bib at master · BITSS/Annual2017 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ Skip to content +
+ + + + +
+
+
+
+ Help save net neutrality! + A free, open internet is once again at stake—and we need your help. +
+
+ Learn more + +
+
+
+ + + + + + + + + + +
+ +
+ +
+
+ + + +
+
+
+ + + + + + + + +
+
+ +
    +
  • +
    + +
    + + + +
    +
    +
    + + Notifications +
    + + + +
    +
    +
    +
    +
  • + +
  • + +
    +
    + + + +
    +
    + + + +
    + +
  • + +
  • + + + Fork + + + + + +
  • +
+ +

+ + /Annual2017 + +

+ +
+ + + + +
+ +
+
+ + + Permalink + + + +
+ +
+ + +
+ +
+
+ + Switch branches/tags +
+ +
+
+ +
+
+ +
+
+ + + +
+
+ + +
+ +
Nothing to show
+
+ +
+
+
+ +
+ + Find file + + +
+ +
+ + + +
+ Fetching contributors… +
+ +
+ + Cannot retrieve contributors at this time +
+
+ +
+
+
+ +
+ Raw + Blame + History +
+ + + + + +
+ +
+ +
+ +
+ 4094 lines (3450 sloc) + + 308 KB +
+
+ + + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
% This file was created with JabRef 2.10.
% Encoding: Cp1252
+
+
@Article{_registration_2010,
Title = {The Registration of Observational Studies When Metaphors Go Bad:},
Author = {{}},
Journal = {Epidemiology},
Year = {2010},
+
Month = jul,
Pages = {1},
+
Doi = {10.1097/EDE.0b013e3181eafbcf},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {The Registration of Observational Studies—When Metaphors Go Bad},
Url = {http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001648-900000000-99655},
Urldate = {2014-10-28}
}
+
@Article{acemoglu_colonial_2012,
Title = {The Colonial Origins of Comparative Development: An Empirical Investigation: Reply},
Author = {Acemoglu, Daron and Johnson, Simon and Robinson, James A.},
Journal = {The American Economic Review},
Year = {2012},
+
Month = oct,
Number = {6},
Pages = {3077--3110},
Volume = {102},
+
Abstract = {Acemoglu, Johnson, and Robinson (2001) established that economic institutions today are correlated with expected mortality of European colonialists. David Albouy argues this relationship is not robust. He drops all data from Latin America and much of the data from Africa, making up almost 60 percent of our sample, despite much information on the mortality of Europeans in those places during the colonial period. He also includes a "campaign" dummy that is coded inconsistently; even modest corrections undermine his claims. We also show that limiting the effect of outliers strengthens our results, making them robust to even extreme versions of Albouy's critiques.},
Doi = {10.1257/aer.102.6.3077},
Shorttitle = {The Colonial Origins of Comparative Development}
}
+
@Article{acemoglu_colonial_2001,
Title = {The Colonial Origins of Comparative Development: An Empirical Investigation},
Author = {Acemoglu, Daron and Johnson, Simon and Robinson, James A},
Journal = {American Economic Review},
Year = {2001},
+
Month = dec,
Number = {5},
Pages = {1369--1401},
Volume = {91},
+
Doi = {10.1257/aer.91.5.1369},
File = {AER (91,5) p. 1369 - The Colonial Origins of Comparative Development\: An Empirical Investigation:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\K96I5TIP\\articles.html:text/html},
ISSN = {0002-8282},
Language = {en},
Shorttitle = {The Colonial Origins of Comparative Development},
Url = {https://www-aeaweb-org.proxy.swarthmore.edu/articles.php?f=s&doi=10.1257/aer.91.5.1369},
Urldate = {2014-08-20}
}
+
@Article{alatas_targeting_2012,
Title = {Targeting the Poor: Evidence from a Field Experiment in Indonesia},
Author = {Alatas, Vivi and Banerjee, Abhijit and Hanna, Rema and Olken, Benjamin A and Tobias, Julia},
Journal = {American Economic Review},
Year = {2012},
+
Month = jun,
Number = {4},
Pages = {1206--1240},
Volume = {102},
+
Doi = {10.1257/aer.102.4.1206},
File = {AER (102,4) p. 1206 - Targeting the Poor\: Evidence from a Field Experiment in Indonesia:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\9DWN64BK\\articles.html:text/html},
ISSN = {0002-8282},
Language = {en},
Shorttitle = {Targeting the Poor},
Url = {https://www.aeaweb.org/articles.php?doi=10.1257/aer.102.4.1206&fnd=s},
Urldate = {2014-10-01}
}
+
@Article{albouy_colonial_2012,
Title = {The Colonial Origins of Comparative Development: An Empirical Investigation: Comment},
Author = {Albouy, David Y.},
Journal = {The American Economic Review},
Year = {2012},
+
Month = oct,
Number = {6},
Pages = {3059--3076},
Volume = {102},
+
Abstract = {Acemoglu, Johnson, and Robinson's (2001) seminal article argues property-rights institutions powerfully affect national income, using estimated mortality rates of early European settlers to instrument capital expropriation risk. However, 36 of the 64 countries in the sample are assigned mortality rates from other countries, often based on mistaken or conflicting evidence. Also, incomparable mortality rates from populations of laborers, bishops, and soldiers—often on campaign—are combined in a manner that favors the hypothesis. When these data issues are controlled for, the relationship between mortality and expropriation risk lacks robustness, and instrumental-variable estimates become unreliable, often with infinite confidence intervals.},
Doi = {10.1257/aer.102.6.3059},
Shorttitle = {The Colonial Origins of Comparative Development}
}
+
@Article{almeida_impact_2012,
Title = {The Impact of Vocational Training for the Unemployed in Turkey: Pre-Analysis Plan},
Author = {Almeida, Rita and Hirshleifer, Sarojini and McKenzie, David and Ridao-Cano, Cristobal and Yener, Ahmed Levent},
Journal = {Poverty Action Lab Hypothesis Registry},
Year = {2012},
+
Month = feb,
+
Url = {http://www.povertyactionlab.org/sites/default/files/documents/ISKURIE_AnalysisPlan_v4.pdf}
}
+
@Article{almond_is_2006,
Title = {Is the 1918 Influenza Pandemic Over? Long Term Effects of In Utero Influenza Exposure in the Post1940 U.S. Population},
Author = {Almond, Douglas},
Journal = {Journal of Political Economy},
Year = {2006},
+
Month = aug,
Number = {4},
Pages = {672--712},
Volume = {114},
+
Abstract = {This paper uses the 1918 influenza pandemic as a natural experiment for testing the fetal origins hypothesis. The pandemic arrived unexpectedly in the fall of 1918 and had largely subsided by January 1919, generating sharp predictions for long�term effects. Data from the 1960–80 decennial U.S. Census indicate that cohorts in utero during the pandemic displayed reduced educational attainment, increased rates of physical disability, lower income, lower socioeconomic status, and higher transfer payments compared with other birth cohorts. These results indicate that investments in fetal health can increase human capital.},
Copyright = {Copyright © 2006 The University of Chicago Press},
Doi = {10.1086/jpe.2006.114.issue-4},
ISSN = {0022-3808},
Shorttitle = {Is the 1918 Influenza Pandemic Over?},
Url = {http://www.jstor.org/stable/10.1086/507154},
Urldate = {2014-08-21}
}
+
@Article{anderson_fwer,
Title = {Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects},
Author = {Anderson, Michael L.},
Journal = {Journal of the American Statistical Association},
Year = {2008},
Number = {484},
Pages = {1481-1495},
Volume = {103},
+
Doi = {10.1198/016214508000000841},
Eprint = {
http://dx.doi.org/10.1198/016214508000000841
},
Url = {
http://dx.doi.org/10.1198/016214508000000841
}
}
+
@Article{anderson_registration_2013,
Title = {Registration and Replication: A Comment},
Author = {Anderson, Richard G.},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {38--39},
Volume = {21},
+
Doi = {10.1093/pan/mps034},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2BVUD8MF\\Anderson - 2013 - Registration and Replication A Comment.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\VCMHDK68\\38.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Shorttitle = {Registration and Replication},
Url = {http://pan.oxfordjournals.org/content/21/1/38},
Urldate = {2014-08-28}
}
+
@Article{anderson_replication_1994,
Title = {Replication and scientific standards in applied economics a decade after the Journal of Money, Credit and Banking project},
Author = {Anderson, Richard G. and Dewald, William G.},
Journal = {Federal Reserve Bank of St. Louis Review},
Year = {1994},
Number = {Nov},
Pages = {79--83},
+
File = {RePEc Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NKMSHAB8\\y_3a1994_3ai_3anov_3ap_3a79-83.html:text/html},
Keywords = {Research},
Url = {http://econpapers.repec.org/article/fipfedlrv/y_3a1994_3ai_3anov_3ap_3a79-83.htm},
Urldate = {2014-08-20}
}
+
@Article{anderson_role_2008,
Title = {The role of data/code archives in the future of economic research},
Author = {Anderson, Richard G. and Greene, William H. and McCullough, B. D. and Vinod, H. D.},
Journal = {Journal of Economic Methodology},
Year = {2008},
Number = {1},
Pages = {99--119},
Volume = {15},
+
Abstract = {This essay examines the role of data and program�code archives in making economic research ‘replicable.’ Replication of published results is recognized as an essential part of the scientific method. Yet, historically, both the ‘demand for’ and ‘supply of’ replicable results in economics has been minimal. ‘Respect for the scientific method’ is not sufficient to motivate either economists or editors of professional journals to ensure the replicability of published results. We enumerate the costs and benefits of mandatory data and code archives, and argue that the benefits far exceed the costs. Progress has been made since the gloomy assessment of Dewald, Thursby and Anderson some 20 years ago in the American Economic Review, but much remains to be done before empirical economics ceases to be a ‘dismal science’ when judged by the replicability of its published results.},
Doi = {10.1080/13501780801915574},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NNWSITCD\\Anderson et al. - 2008 - The role of datacode archives in the future of ec.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NT9DFTIR\\13501780801915574.html:text/html},
ISSN = {1350-178X},
Url = {http://dx.doi.org/10.1080/13501780801915574},
Urldate = {2014-08-20}
}
+
@Article{andreoli-versbach_open_????,
Title = {Open access to data: An ideal professed but not practised},
Author = {Andreoli-Versbach, Patrick and Mueller-Langer, Frank},
Journal = {Research Policy},
+
Abstract = {Data-sharing is an essential tool for replication, validation and extension of empirical results. Using a hand-collected data set describing the data-sharing behaviour of 488 randomly selected empirical researchers, we provide evidence that most researchers in economics and management do not share their data voluntarily. We derive testable hypotheses based on the theoretical literature on information-sharing and relate data-sharing to observable characteristics of researchers. We find empirical support for the hypotheses that voluntary data-sharing significantly increases with (a) academic tenure, (b) the quality of researchers, (c) the share of published articles subject to a mandatory data-disclosure policy of journals, and (d) personal attitudes towards “open science� principles. On the basis of our empirical evidence, we discuss a set of policy recommendations.},
Doi = {10.1016/j.respol.2014.04.008},
File = {ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\ETGVNNZQ\\S0048733314000699.html:text/html},
ISSN = {0048-7333},
Keywords = {Mandatory data-disclosure, Open science, Voluntary data-sharing},
Shorttitle = {Open access to data},
Url = {http://www.sciencedirect.com/science/article/pii/S0048733314000699},
Urldate = {2014-08-29}
}
+
@Article{angrist_credibility_2010,
Title = {The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics},
Author = {Angrist, Joshua D and Pischke, Jörn-Steffen},
Journal = {Journal of Economic Perspectives},
Year = {2010},
+
Month = may,
Number = {2},
Pages = {3--30},
Volume = {24},
+
Doi = {10.1257/jep.24.2.3},
File = {JEP (24,2) p. 3 - The Credibility Revolution in Empirical Economics\: How Better Research Design Is Taking the Con out of Econometrics:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NNXZ8AAF\\articles.html:text/html},
ISSN = {0895-3309},
Language = {en},
Shorttitle = {The Credibility Revolution in Empirical Economics},
Url = {https://www-aeaweb-org.proxy.swarthmore.edu/articles.php?doi=10.1257/jep.24.2.3},
Urldate = {2014-09-08}
}
+
@Book{angrist2008mostly,
Title = {Mostly harmless econometrics: An empiricist's companion},
Author = {Angrist, Joshua D and Pischke, J{\"o}rn-Steffen},
Publisher = {Princeton university press},
Year = {2008}
}
+
@Article{antonovics_does_2005,
Title = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation? Comment},
Author = {Antonovics, Kate L. and Goldberger, Arthur S.},
Journal = {The American Economic Review},
Year = {2005},
+
Month = dec,
Number = {5},
Pages = {1738--1744},
Volume = {95},
+
Copyright = {Copyright © 2005 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\95TJCXRX\\Antonovics and Goldberger - 2005 - Does Increasing Women's Schooling Raise the School.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation?},
Url = {http://www.jstor.org/stable/4132776},
Urldate = {2014-09-23}
}
+
@Article{ashenfelter1999review,
Title = {A review of estimates of the schooling/earnings relationship, with tests for publication bias},
Author = {Ashenfelter, Orley and Harmon, Colm and Oosterbeek, Hessel},
Journal = {Labour economics},
Year = {1999},
Number = {4},
Pages = {453--470},
Volume = {6},
+
Publisher = {Elsevier}
}
+
@Article{ashton_national_2010,
Title = {Do national soccer results really impact on the stock market?},
Author = {Ashton, J. K. and Gerrard, B. and Hudson, R.},
Journal = {Applied Economics},
Year = {2010},
+
Month = nov,
Number = {26},
Pages = {3709--3717},
Volume = {43},
+
Abstract = {This study is a response to Klein et al. (2008), which was highly critical of earlier work by Ashton et al. (2003). This work considering the link between international soccer results and stock market returns was challenged by Klein et al. (2008), who reject the presence and importance of this link. In response, this work provides a reassessment of the link between international soccer results and stock market returns within Ashton et al. (2003). This new analysis extends the original work by using a larger dataset, employing an extended range of tests and allowing for outliers. It is reported that, contrary to the findings of Klein et al. (2008), the link between international soccer results and stock market prices does indeed exist particularly within the sample period 1984–2002 used by Ashton et al. (2003). After extending the dataset to include observations from 2002 until 2009, it is reported that the effect on stock market returns has declined in importance over this period, particularly the impact of wins.},
Doi = {10.1080/00036841003689762},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\45JFF45V\\Ashton et al. - 2010 - Do national soccer results really impact on the st.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CD2H9AH5\\00036841003689762.html:text/html},
ISSN = {0003-6846},
Url = {http://dx.doi.org/10.1080/00036841003689762},
Urldate = {2014-08-28}
}
+
@Article{ashton_economic_2003,
Title = {Economic impact of national sporting success: evidence from the London stock exchange},
Author = {Ashton, J. K. and Gerrard, B. and Hudson, R.},
Journal = {Applied Economics Letters},
Year = {2003},
+
Month = oct,
Number = {12},
Pages = {783--785},
Volume = {10},
+
Abstract = {In this article strong association is reported between the performance of the England football team and subsequent daily changes in the {FTSE} 100 index, representing the price of shares in the 100 largest companies traded on the London stock exchange.},
Doi = {10.1080/1350485032000126712},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\N86J838Z\\Ashton et al. - 2003 - Economic impact of national sporting success evid.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\VEH27K34\\1350485032000126712.html:text/html},
ISSN = {1350-4851},
Shorttitle = {Economic impact of national sporting success},
Url = {http://dx.doi.org/10.1080/1350485032000126712},
Urldate = {2014-08-28}
}
+
@Article{auspurg_what_2011,
Title = {What Fuels Publication Bias? Theoretical and Empirical Analyses of Risk Factors Using the Caliper Test},
Author = {Auspurg, Katrin and Hinz, Thomas},
Journal = {Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik)},
Year = {2011},
Number = {5-6},
Pages = {636--660},
Volume = {231},
+
Abstract = {Significance tests were originally developed to enable more objective evaluations of research results. Yet the strong orientation towards statistical significance encourages biased results, a phenomenon termed “publication bias�. Publication bias occurs whenever the likelihood or time-lag of publication, or the prominence, language, impact factor of journal space or the citation rate of studies depend on the direction and significance of research findings. Although there is much evidence concerning the existence of publication bias in all scientific disciplines and although its detrimental consequences for the progress of the sciences have been known for a long time, all attempts to eliminate the bias have failed. The present article reviews the history and logic of significance testing, the state of research on publication bias, and existing practical recommendations. After demonstrating that more systematical research on the risk factors of publication bias is needed, the paper suggests two new directions for publication bias research. First, a more comprehensive theoretical model based on theories of rational choice and economics as well as on the sociology of science is sketched out. Publication bias is recognized as the outcome of a social dilemma that cannot be overcome by moral pleas alone. Second, detection methods for publication bias going beyond meta-analysis, ones that are more suitable for testing causal hypotheses, are discussed. In particular, the “caliper test� seems well-suited for conducting theoretically motivated comparisons across heterogeneous research fields like sociology. Its potential is demonstrated by testing hypotheses on (a) the relevance of explicitly vs. implicitly stated research propositions and on (b) the relevance of the number of authors on incidence rates of publication bias in 50 papers published in leading German sociology journals.},
File = {RePEc Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\XU8VSDT7\\v231y2011i5-6p636-660.html:text/html},
Keywords = {caliper test, Publication bias, rational choice, Significance testing, sociology of science.},
Shorttitle = {What Fuels Publication Bias?},
Url = {http://ideas.repec.org/a/jns/jbstat/v231y2011i5-6p636-660.html},
Urldate = {2014-08-19}
}
+
@Article{ayres_latest_2003,
Title = {The Latest Misfires in Support of the "More Guns, Less Crime" Hypothesis},
Author = {Ayres, Ian and Donohue III, John J.},
Journal = {Stanford Law Review},
Year = {2003},
+
Month = apr,
Number = {4},
Pages = {1371--1398},
Volume = {55},
+
Copyright = {Copyright © 2003 Stanford Law Review},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NJN8ETR2\\Ayres and III - 2003 - The Latest Misfires in Support of the More Guns, .pdf:application/pdf},
ISSN = {0038-9765},
Url = {http://www.jstor.org/stable/1229605},
Urldate = {2014-08-21}
}
+
@Article{ayres_shooting_2003,
Title = {Shooting down the "More Guns, Less Crime" Hypothesis},
Author = {Ayres, Ian and Donohue III, John J.},
Journal = {Stanford Law Review},
Year = {2003},
+
Month = apr,
Number = {4},
Pages = {1193--1312},
Volume = {55},
+
Copyright = {Copyright © 2003 Stanford Law Review},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\PXQNJN4Q\\Ayres and III - 2003 - Shooting down the More Guns, Less Crime Hypothes.pdf:application/pdf},
ISSN = {0038-9765},
Url = {http://www.jstor.org/stable/1229603},
Urldate = {2014-08-21}
}
+
@Article{baicker_katherine_oregon_2014,
Title = {The Oregon Health Insurance Experiment: Evidence from Criminal Charges Data, Analysis Plan},
Author = {Baicker, Katherine and Finkelstein, Amy and Taubman, Sarah},
Year = {2014},
+
Month = apr,
+
Series = {{NBER} Working Paper},
Url = {http://www.nber.org/oregon/files/oregon_hie_crime_analysis_plan.pdf}
}
+
@Article{doi:10.1056/NEJMsa1212321,
Title = {The Oregon Experiment — Effects of Medicaid on Clinical Outcomes},
Author = {Baicker, Katherine and Taubman, Sarah L. and Allen, Heidi L. and Bernstein, Mira and Gruber, Jonathan H. and Newhouse, Joseph P. and Schneider, Eric C. and Wright, Bill J. and Zaslavsky, Alan M. and Finkelstein, Amy N.},
Journal = {New England Journal of Medicine},
Year = {2013},
Note = {PMID: 23635051},
Number = {18},
Pages = {1713-1722},
Volume = {368},
+
Doi = {10.1056/NEJMsa1212321},
Eprint = {http://www.nejm.org/doi/pdf/10.1056/NEJMsa1212321},
Url = {http://www.nejm.org/doi/full/10.1056/NEJMsa1212321}
}
+
@Article{barrick1991big5,
Title = {The big five personality dimensions and job performance: a meta-analysis},
Author = {Barrick, Murray R and Mount, Michael K},
Journal = {Personnel psychology},
Year = {1991},
Number = {1},
Pages = {1--26},
Volume = {44},
+
Publisher = {Wiley Online Library}
}
+
@Article{begg_c_improving_1996,
Title = {Improving the quality of reporting of randomized controlled trials: The consort statement},
Author = {{Begg C} and {Cho M} and {Eastwood S} and {et al}},
Journal = {{JAMA}},
Year = {1996},
+
Month = aug,
Number = {8},
Pages = {637--639},
Volume = {276},
+
Abstract = {{THE} {RANDOMIZED} controlled trial ({RCT}), more than any other methodology, can have a powerful and immediate impact on patient care. Ideally, the report of such an evaluation needs to convey to the reader relevant information concerning the design, conduct, analysis, and generalizability of the trial. This information should provide the reader with the ability to make informed judgments regarding the internal and external validity of the trial. Accurate and complete reporting also benefits editors and reviewers in their deliberations regarding submitted manuscripts. For {RCTs} to ultimately benefit patients, the published report should be of the highest possible standard.For editorial comment see p 649. 
Evidence produced repeatedly over the last 30 years indicates a wide chasm between what a trial should report and what is actually published in the literature. In a review of 71 {RCTs} with negative results published between 1960 and 1975, the authors reported that the vast},
Doi = {10.1001/jama.1996.03540080059030},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HCK2H7WC\\Begg C et al. - 1996 - Improving the quality of reporting of randomized c.pdf:application/pdf},
ISSN = {0098-7484},
Shorttitle = {Improving the quality of reporting of randomized controlled trials},
Url = {http://dx.doi.org/10.1001/jama.1996.03540080059030},
Urldate = {2014-08-29}
}
+
@Article{behrman_does_2005,
Title = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation? Reply},
Author = {Behrman, Jere R. and Rosenzweig, Mark R.},
Journal = {The American Economic Review},
Year = {2005},
+
Month = dec,
Number = {5},
Pages = {1745--1751},
Volume = {95},
+
Copyright = {Copyright © 2005 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\7ZJXU3S3\\Behrman and Rosenzweig - 2005 - Does Increasing Women's Schooling Raise the School.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation?},
Url = {http://www.jstor.org/stable/4132777},
Urldate = {2014-09-23}
}
+
@Article{behrman_does_2002,
Title = {Does Increasing Women's Schooling Raise the Schooling of the Next Generation?},
Author = {Behrman, Jere R. and Rosenzweig, Mark R.},
Journal = {The American Economic Review},
Year = {2002},
+
Month = mar,
Number = {1},
Pages = {323--334},
Volume = {92},
+
Copyright = {Copyright © 2002 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\Z9FUB53C\\Behrman and Rosenzweig - 2002 - Does Increasing Women's Schooling Raise the School.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/3083336},
Urldate = {2014-09-23}
}
+
@Article{benjamini1995controlling,
Title = {Controlling the false discovery rate: a practical and powerful approach to multiple testing},
Author = {Benjamini, Yoav and Hochberg, Yosef},
Journal = {Journal of the Royal Statistical Society. Series B (Methodological)},
Year = {1995},
Pages = {289--300},
+
Publisher = {JSTOR}
}
+
@Article{benjamini2006adaptive,
Title = {Adaptive linear step-up procedures that control the false discovery rate},
Author = {Benjamini, Yoav and Krieger, Abba M and Yekutieli, Daniel},
Journal = {Biometrika},
Year = {2006},
Number = {3},
Pages = {491--507},
Volume = {93},
+
Publisher = {Biometrika Trust}
}
+
@Article{Bernanke2004,
Title = {Editorial Statement},
Author = {Ben S. Bernanke},
Journal = {The American Economic Review},
Year = {2004},
Number = {1},
Pages = {404-404},
Volume = {94},
+
ISSN = {00028282},
Publisher = {American Economic Association},
Url = {http://www.jstor.org/stable/3592790}
}
+
@Article{bhattacharjee_diederik_2013,
Title = {Diederik Stapel Audacious Academic Fraud},
Author = {Bhattacharjee, Yudhijit},
Journal = {The New York Times},
Year = {2013},
+
Month = apr,
+
Abstract = {Diederik Stapel, a Dutch social psychologist, perpetrated an audacious academic fraud by making up studies that told the world what it wanted to hear about human nature.},
Chapter = {Magazine},
File = {New York Times Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KVN625TA\\diederik-stapels-audacious-academic-fraud.html:text/html},
ISSN = {0362-4331},
Keywords = {Frauds and Swindling, Netherlands, Psychology and Psychologists, Research, Stapel, Diederik, Tilburg University},
Url = {http://www.nytimes.com/2013/04/28/magazine/diederik-stapels-audacious-academic-fraud.html},
Urldate = {2014-09-12}
}
+
@Article{bloom1995minimum,
Title = {Minimum detectable effects a simple way to report the statistical power of experimental designs},
Author = {Bloom, Howard S},
Journal = {Evaluation review},
Year = {1995},
Number = {5},
Pages = {547--556},
Volume = {19},
+
Publisher = {Sage Publications}
}
+
@Book{broad_betrayers_1983,
Title = {Betrayers of the truth},
Author = {Broad, William and Broad, William J. and Wade, Nicholas},
Publisher = {Simon and Schuster},
Year = {1983},
Month = jan,
+
Abstract = {"Fraud and deceit in the halls of science"--Cover subtitle.},
ISBN = {9780671447694},
Keywords = {Biography \& Autobiography / Science \& Technology, Deception, Fraud in science, Reference / Research, science, Science / Research \& Methodology},
Language = {en}
}
+
@TechReport{brodeur_star_2012,
Title = {Star Wars: The Empirics Strike Back},
Author = {Brodeur, Abel and Le, Mathias and Sangnier, Marc and Zylberberg, Yanos},
Institution = {Social Science Research Network},
Year = {2012},
+
Address = {Rochester, {NY}},
Month = jun,
Number = {{ID} 2089580},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {Journals favor rejections of the null hypothesis. This selection upon results may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the {AER}, {JPE} and {QJE}, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a camel shape with abundant p-values above .25, a valley between .25 and .10 and a bump slightly under .05. Missing tests are those which would have been accepted but close to being rejected (p-values between .25 and .10). We show that this pattern corresponds to a shift in the distribution of p-values: between 10\% and 20\% of marginally rejected tests are misallocated. Our interpretation is that researchers might be tempted to inflate the value of their tests by choosing the specification that provides the highest statistics. Note that Inflation is larger in articles where stars are used in order to highlight statistical significance and lower in articles with theoretical models.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\64W563JE\\papers.html:text/html},
Keywords = {distorting incentives, Hypothesis testing, research in economics, selection bias},
Shorttitle = {Star Wars},
Url = {http://papers.ssrn.com/abstract=2089580},
Urldate = {2014-08-21}
}
+
@Article{bronars_criminal_1998,
Title = {Criminal Deterrence, Geographic Spillovers, and the Right to Carry Concealed Handguns},
Author = {Bronars, Stephen G. and Lott, Jr., John R.},
Journal = {The American Economic Review},
Year = {1998},
+
Month = may,
Number = {2},
Pages = {475--479},
Volume = {88},
+
Copyright = {Copyright © 1998 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\AR4F3WXQ\\Bronars and Lott - 1998 - Criminal Deterrence, Geographic Spillovers, and th.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/116970},
Urldate = {2014-08-21}
}
+
@Article{broockman2015irreg,
Title = {Irregularities in LaCour (2014)},
Author = {Broockman, David and Kalla, Joshua and Aronow, Peter},
Year = {2015},
+
Month = {May},
+
Url = {http://stanford.edu/~dbroock/broockman_kalla_aronow_lg_irregularities.pdf}
}
+
@Article{brown_1918_2010,
Title = {The 1918 U.S. Influenza Pandemic as a Natural Experiment, Revisited},
Author = {Brown, Ryan},
Year = {2010},
+
Month = oct,
+
Abstract = {Douglas Almond's use of the 1918 U.S. in uenza pandemic as a
natural experiment led to the seminal works on the subject of in utero
health's impact on later life outcomes. The identi⬚cation strength and
clarity of his work, though, is driven by the inherent natural experi-
ment supposition of random assignment. By using data from the 1920
and 1930 U.S. census, this study investigates this keystone assump-
tion and shows that the families of the {\textbackslash}treatment" cohort used by
Douglas Almond were signi⬚cantly less literate and economically pros-
perous than the families of the {\textbackslash}control" group. The ⬚rst order e⬚ect
of parent's education and wealth on a child's long-run {SES} and health
conditions eliminates Almond's ability to make direct inference on fe
tal health's impact on one's long-term wellbeing using the 1918 U.S.
in uenza pandemic.},
Url = {http://mitsloan.mit.edu/neudc/papers/paper_276.pdf}
}
+
@Article{bruhn_pursuit_2009,
Title = {In Pursuit of Balance: Randomization in Practice in Development Field Experiments},
Author = {Bruhn, Miriam and McKenzie, David},
Journal = {American Economic Journal: Applied Economics},
Year = {2009},
+
Month = oct,
Number = {4},
Pages = {200--232},
Volume = {1},
+
Abstract = {We present new evidence on the randomization methods used in existing experiments, and new simulations comparing these methods. We find that many papers do not describe the randomization in detail, implying that better reporting is needed. Our simulations suggest that in samples of 300 or more, the different methods perform similarly. However, for very persistent outcome variables, and in smaller samples, pair-wise matching and stratification perform best and appear to dominate the rerandomization methods commonly used in practice. The simulations also point to specific recommendations for which variables to balance on, and for which controls to include in the ex post analysis.},
Doi = {10.1257/app.1.4.200},
Shorttitle = {In Pursuit of Balance}
}
+
@Article{burnside_aid_2004,
Title = {Aid, Policies, and Growth: Reply},
Author = {Burnside, Craig and Dollar, David},
Journal = {The American Economic Review},
Year = {2004},
+
Month = jun,
Number = {3},
Pages = {781--784},
Volume = {94},
+
Copyright = {Copyright © 2004 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JGFHQF96\\Burnside and Dollar - 2004 - Aid, Policies, and Growth Reply.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Aid, Policies, and Growth},
Url = {http://www.jstor.org/stable/3592955},
Urldate = {2014-08-20}
}
+
@Article{burnside_aid_2000,
Title = {Aid, Policies, and Growth},
Author = {Burnside, Craig and Dollar, David},
Journal = {The American Economic Review},
Year = {2000},
+
Month = sep,
Number = {4},
Pages = {847--868},
Volume = {90},
+
Abstract = {This paper uses a new database on foreign aid to examine the relationships among foreign aid, economic policies, and growth of per capita {GDP}. We find that aid has a positive impact on growth in developing countries with good fiscal, monetary, and trade policies but has little effect in the presence of poor policies. Good policies are ones that are themselves important for growth. The quality of policy has only a small impact on the allocation of aid. Our results suggest that aid would be more effective if it were more systematically conditioned on good policy.},
Copyright = {Copyright © 2000 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2Q463KRC\\Burnside and Dollar - 2000 - Aid, Policies, and Growth.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/117311},
Urldate = {2014-08-20}
}
+
@Article{button_power_2013,
Title = {Power failure: why small sample size undermines the reliability of neuroscience},
Author = {Button, Katherine S and Ioannidis, John PA and Mokrysz, Claire and Nosek, Brian A and Flint, Jonathan and Robinson, Emma SJ and Munaf{\`o}, Marcus R},
Journal = {Nature Reviews Neuroscience},
Year = {2013},
Number = {5},
Pages = {365--376},
Volume = {14},
+
Publisher = {Nature Publishing Group}
}
+
@Article{camfield_three_2013,
Title = {Three R's of Econometrics: Repetition, Reproduction and Replication},
Author = {Camfield, Laura and Palmer-Jones, Richard},
Journal = {Journal of Development Studies},
Year = {2013},
Number = {12},
Pages = {1607--1614},
Volume = {49},
+
Abstract = {Development economics has become increasingly quantified in recent years, reflecting the aspirations of economists to practise hard science. We argue that standard applied econometric methodology lacks one key feature of the claim of science to be scientific, namely replication as part of independent confirmation of findings. Replication plays a large role in understanding the confidence we can place in the quantitative studies on which much policy advice rests, which is particularly salient in a {UK} context given the emphasis placed on evidence-based policy-making by the {UK} Department for International Development.},
Doi = {10.1080/00220388.2013.807504},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\B3TF5BD3\\Camfield and Palmer-Jones - 2013 - Three ‘Rs’ of Econometrics Repetition, Reproducti.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\XHAG6H2I\\00220388.2013.html:text/html},
ISSN = {0022-0388},
Shorttitle = {Three ‘Rs’ of Econometrics},
Url = {http://dx.doi.org/10.1080/00220388.2013.807504},
Urldate = {2014-08-21}
}
+
@Article{CanadianMinWage2006,
Title = {Minimum Wage Impacts from a Prespecified Research Design: Canada 1981–1997},
Author = {CAMPOLIETI, MICHELE and GUNDERSON, MORLEY and RIDDELL, CHRIS},
Journal = {Industrial Relations: A Journal of Economy and Society},
Year = {2006},
Number = {2},
Pages = {195--216},
Volume = {45},
+
Doi = {10.1111/j.1468-232X.2006.00424.x},
ISSN = {1468-232X},
Publisher = {Blackwell Publishing Inc},
Url = {http://dx.doi.org/10.1111/j.1468-232X.2006.00424.x}
}
+
@Article{card_minimum_2000,
Title = {Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Reply},
Author = {Card, David and Krueger, Alan B.},
Journal = {The American Economic Review},
Year = {2000},
+
Month = dec,
Number = {5},
Pages = {1397--1420},
Volume = {90},
+
Copyright = {Copyright © 2000 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8UMCAMVW\\Card and Krueger - 2000 - Minimum Wages and Employment A Case Study of the .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Minimum Wages and Employment},
Url = {http://www.jstor.org/stable/2677856},
Urldate = {2014-08-20}
}
+
@Article{card1995time,
Title = {Time-series minimum-wage studies: a meta-analysis},
Author = {Card, David and Krueger, Alan B},
Journal = {The American Economic Review},
Year = {1995},
Pages = {238--243},
+
Publisher = {JSTOR}
}
+
@Article{card_minimum_1994,
Title = {Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania},
Author = {Card, David and Krueger, Alan B.},
Journal = {The American Economic Review},
Year = {1994},
+
Month = sep,
Number = {4},
Pages = {772--793},
Volume = {84},
+
Abstract = {On April 1, 1992, New Jersey's minimum wage rose from \$4.25 to \$5.05 per hour. To evaluate the impact of the law we surveyed 410 fast-food restaurants in New Jersey and eastern Pennsylvania before and after the rise. Comparisons of employment growth at stores in New Jersey and Pennsylvania (where the minimum wage was constant) provide simple estimates of the effect of the higher minimum wage. We also compare employment changes at stores in New Jersey that were initially paying high wages (above \$5) to the changes at lower-wage stores. We find no indication that the rise in the minimum wage reduced employment.},
Copyright = {Copyright © 1994 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\3GJU4UC9\\Card and Krueger - 1994 - Minimum Wages and Employment A Case Study of the .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Minimum Wages and Employment},
Url = {http://www.jstor.org/stable/2118030},
Urldate = {2014-08-20}
}
+
@Article{Card1992minwage,
Title = {Using Regional Variation in Wages to Measure the Effects of the Federal Minimum Wage},
Author = {Card, David AND Alan B Krueger},
Journal = {Industrial \& Labor Relations Review},
Year = {1992},
+
Month = {October},
Number = {1},
Pages = {22-37},
Volume = {46},
+
Owner = {garret},
Timestamp = {2015.07.03}
}
+
@Article{carey_noted_2011,
Title = {Noted Dutch Psychologist, Stapel, Accused of Research Fraud},
Author = {Carey, Benedict},
Journal = {The New York Times},
Year = {2011},
+
Month = nov,
+
Abstract = {A Dutch scholar was found to have falsified findings in dozens of papers, in a field that critics say is vulnerable to such abuses.},
Chapter = {Health / Research},
File = {New York Times Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HJIS8BZD\\noted-dutch-psychologist-stapel-accused-of-research-fraud.html:text/html},
ISSN = {0362-4331},
Keywords = {Falsification of Data, Frauds and Swindling, Psychology and Psychologists, Research, Stapel, Diederik},
Url = {http://www.nytimes.com/2011/11/03/health/research/noted-dutch-psychologist-stapel-accused-of-research-fraud.html},
Urldate = {2014-09-12}
}
+
@Article{carpenter_psychologys_2012,
Title = {Psychology's Bold Initiative},
Author = {Carpenter, Siri},
Journal = {Science},
Year = {2012},
+
Month = mar,
Number = {6076},
Pages = {1558--1561},
Volume = {335},
+
Abstract = {Some psychology researchers argue that a scientific culture that too heavily favors new and counterintuitive ideas over the confirmation of existing results has led to too many findings that are striking for their novelty and published in respected journals—but are nonetheless false. A new online site ({PsychFileDrawer}.org) lets psychologists post results of replications of experiments—whether they succeed or fail—that would ordinarily never leave their file drawer because most journals decline to publish straightforward replication studies. And a group of more than 50 academic psychologists, which calls itself the Open Science Collaboration, has begun an unprecedented, large-scale project to systematically replicate psychological experiments recently published in leading journals. Some researchers are optimistic that many published findings will be replicated. But others are concerned that if the project confirms few studies, it could unfairly indict psychology. Indeed, the prospect of exposing psychology's foibles has upset some scientists.},
Doi = {10.1126/science.335.6076.1558},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6E47UU9Q\\Carpenter - 2012 - Psychology's Bold Initiative.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WNTFGWVE\\1558.html:text/html},
ISSN = {0036-8075, 1095-9203},
Language = {en},
Pmid = {22461583},
Url = {http://www.sciencemag.org/content/335/6076/1558},
Urldate = {2014-09-08}
}
+
@Article{casey_reshaping_2012,
Title = {Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan*},
Author = {Casey, Katherine and Glennerster, Rachel and Miguel, Edward},
Journal = {The Quarterly Journal of Economics},
Year = {2012},
+
Month = nov,
Number = {4},
Pages = {1755--1812},
Volume = {127},
+
Abstract = {Despite their importance, there is limited evidence on how institutions can be strengthened. Evaluating the effects of specific reforms is complicated by the lack of exogenous variation in institutions, the difficulty of measuring institutional performance, and the temptation to “cherry pick� estimates from among the large number of indicators required to capture this multifaceted subject. We evaluate one attempt to make local institutions more democratic and egalitarian by imposing participation requirements for marginalized groups (including women) and test for learning-by-doing effects. We exploit the random assignment of a governance program in Sierra Leone, develop innovative real-world outcome measures, and use a preanalysis plan ({PAP}) to bind our hands against data mining. The intervention studied is a “community-driven development� program, which has become a popular strategy for foreign aid donors. We find positive short-run effects on local public goods and economic outcomes, but no evidence for sustained impacts on collective action, decision making, or the involvement of marginalized groups, suggesting that the intervention did not durably reshape local institutions. We discuss the practical trade-offs faced in implementing a {PAP} and show how in its absence we could have generated two divergent, equally erroneous interpretations of program impacts on institutions. {JEL} Codes: F35, H41, O4},
Doi = {10.1093/qje/qje027},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\MHIPH78X\\1755.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {Reshaping Institutions},
Url = {http://qje.oxfordjournals.org/content/127/4/1755},
Urldate = {2014-08-26}
}
+
@Article{chambers2014instead,
Title = {Instead of" playing the game" it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond},
Author = {Chambers, Christopher D and Feredoes, Eva and Muthukumaraswamy, Suresh Daniel and Etchells, Peter},
Journal = {AIMS Neuroscience},
Year = {2014},
Number = {1},
Pages = {4--17},
Volume = {1},
+
Publisher = {AIMS Press}
}
+
@Article{chan_a_empirical_2004,
Title = {Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles},
Author = {{Chan A} and {Hrbjartsson A} and {Haahr MT} and {Gtzsche PC} and {Altman DG}},
Journal = {{JAMA}},
Year = {2004},
+
Month = may,
Number = {20},
Pages = {2457--2465},
Volume = {291},
+
Abstract = {Context Selective reporting of outcomes within published studies based on the
nature or direction of their results has been widely suspected, but direct
evidence of such bias is currently limited to case reports.Objective To study empirically the extent and nature of outcome reporting bias
in a cohort of randomized trials.Design Cohort study using protocols and published reports of randomized trials
approved by the Scientific-Ethical Committees for Copenhagen and Frederiksberg,
Denmark, in 1994-1995. The number and characteristics of reported and unreported
trial outcomes were recorded from protocols, journal articles, and a survey
of trialists. An outcome was considered incompletely reported if insufficient
data were presented in the published articles for meta-analysis. Odds ratios
relating the completeness of outcome reporting to statistical significance
were calculated for each trial and then pooled to provide an overall estimate
of bias. Protocols and published articles were also compared to identify discrepancies
in primary outcomes.Main Outcome Measures Completeness of reporting of efficacy and harm outcomes and of statistically
significant vs nonsignificant outcomes; consistency between primary outcomes
defined in the most recent protocols and those defined in published articles.Results One hundred two trials with 122 published journal articles and 3736
outcomes were identified. Overall, 50\% of efficacy and 65\% of harm outcomes
per trial were incompletely reported. Statistically significant outcomes had
a higher odds of being fully reported compared with nonsignificant outcomes
for both efficacy (pooled odds ratio, 2.4; 95\% confidence interval [{CI}], 1.4-4.0)
and harm (pooled odds ratio, 4.7; 95\% {CI}, 1.8-12.0) data. In comparing published
articles with protocols, 62\% of trials had at least 1 primary outcome that
was changed, introduced, or omitted. Eighty-six percent of survey responders
(42/49) denied the existence of unreported outcomes despite clear evidence
to the contrary.Conclusions The reporting of trial outcomes is not only frequently incomplete but
also biased and inconsistent with protocols. Published articles, as well as
reviews that incorporate them, may therefore be unreliable and overestimate
the benefits of an intervention. To ensure transparency, planned trials should
be registered and protocols should be made publicly available prior to trial
completion.},
Doi = {10.1001/jama.291.20.2457},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\AFKZWBJ7\\Chan A et al. - 2004 - Empirical evidence for selective reporting of outc.pdf:application/pdf},
ISSN = {0098-7484},
Shorttitle = {Empirical evidence for selective reporting of outcomes in randomized trials},
Url = {http://dx.doi.org/10.1001/jama.291.20.2457},
Urldate = {2014-10-09}
}
+
@Article{chan_spirit_2013,
Title = {{SPIRIT} 2013 explanation and elaboration: guidance for protocols of clinical trials},
Author = {Chan, A.-W. and Tetzlaff, J. M. and Gotzsche, P. C. and Altman, D. G. and Mann, H. and Berlin, J. A. and Dickersin, K. and Hrobjartsson, A. and Schulz, K. F. and Parulekar, W. R. and Krleza-Jeric, K. and Laupacis, A. and Moher, D.},
Journal = {{BMJ}},
Year = {2013},
+
Month = jan,
Number = {jan08 15},
Pages = {e7586--e7586},
Volume = {346},
+
Doi = {10.1136/bmj.e7586},
ISSN = {1756-1833},
Language = {en},
Shorttitle = {{SPIRIT} 2013 explanation and elaboration},
Url = {http://www.bmj.com/cgi/doi/10.1136/bmj.e7586},
Urldate = {2014-10-09}
}
+
@Article{chemin_response_2012,
Title = {Response to High Noon for Microfinance Impact Evaluations},
Author = {Chemin, Matthieu},
Journal = {Journal of Development Studies},
Year = {2012},
Number = {12},
Pages = {1881--1885},
Volume = {48},
+
Abstract = {Abstract Duvendack and Palmer-Jones are critical of analysis and conclusions in Chemin (2008) because they are unable to replicate my results. This response identifies key differences between the two papers, especially regarding the sample and measurement of variables, which imply that Duvendack and Paler-Jones should not be considered as either a replication or a criticism of my work.},
Doi = {10.1080/00220388.2012.727561},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\48233AME\\Chemin - 2012 - Response to ‘High Noon for Microfinance Impact Eva.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WG76XX86\\00220388.2012.html:text/html},
ISSN = {0022-0388},
Url = {http://dx.doi.org/10.1080/00220388.2012.727561},
Urldate = {2014-08-21}
}
+
@Article{chemin_benefits_2008,
Title = {The Benefits and Costs of Microfinance: Evidence from Bangladesh},
Author = {Chemin, Matthieu},
Journal = {Journal of Development Studies},
Year = {2008},
Number = {4},
Pages = {463--484},
Volume = {44},
+
Abstract = {Abstract Using the latest developments from the evaluation literature, namely the technique of matching, this paper shows a positive, but lower than previously thought, effect of microfinance on expenditure per capita, supply of labour, and level of school enrolment for boys and girls. For instance, participants spend 3 per cent more on average than non-participants in control villages. This paper also takes into account repayment delays to calculate the cost of credit provision. It shows how a better investigation at the individual level of the benefits brought and the cost borne could help microfinance institutions to better select their customers. {JEL} Classification : C14, D10, G21, I38, O12, O16},
Doi = {10.1080/00220380701846735},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\A2VH4E68\\Chemin - 2008 - The Benefits and Costs of Microfinance Evidence f.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\B967ER6G\\00220380701846735.html:text/html},
ISSN = {0022-0388},
Shorttitle = {The Benefits and Costs of Microfinance},
Url = {http://dx.doi.org/10.1080/00220380701846735},
Urldate = {2014-08-21}
}
+
@Article{ciccone_economic_2011,
Title = {Economic Shocks and Civil Conflict: A Comment},
Author = {Ciccone, Antonio},
Journal = {American Economic Journal: Applied Economics},
Year = {2011},
+
Month = oct,
Number = {4},
Pages = {215--227},
Volume = {3},
+
Abstract = {Edward Miguel, Shanker Satyanath, and Ernest Sergenti (2004), henceforth {MSS}, argue that lower rainfall levels and negative rainfall shocks increase conflict risk in sub-Saharan Africa. This conclusion rests on their finding of a negative correlation between conflict in t and rainfall growth between t – 1 and t – 2. I show that this finding is driven by a (counterintuitive) positive correlation between conflict in t and rainfall levels in t – 2. If lower rainfall levels or negative rainfall shocks increased conflict, {MSS}'s finding should have been due to a negative correlation between conflict in t and rainfall levels in t – 1. In the latest data, conflict is unrelated to rainfall.},
Doi = {10.1257/app.3.4.215},
Shorttitle = {Economic Shocks and Civil Conflict}
}
+
@Article{ClemensRepTax,
Title = {The Meaning of Failed Replications: A Review and Proposal},
Author = {Michael Clemens},
Journal = {Journal of Economic Surveys},
Year = {2016},
+
Owner = {garret},
Timestamp = {2016.01.20}
}
+
@Article{clemens_replication,
Title = {The Meaning of Failed Replications: A Review and Proposal},
Author = {Michael Clemens},
Journal = {CGD Working Paper 399.},
Year = {2015},
+
Month = {April},
Number = {399},
+
Owner = {garret},
Timestamp = {2015.07.13},
Url = {http://www.cgdev.org/sites/default/files/CGD-Working-Paper-399-Clemens-Meaning-Failed-Replications.pdf}
}
+
@Article{collaboration_open_2012,
Title = {An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science},
Author = {Collaboration, Open Science},
Journal = {Perspectives on Psychological Science},
Year = {2012},
+
Month = nov,
Number = {6},
Pages = {657--660},
Volume = {7},
+
Abstract = {Reproducibility is a defining feature of science. However, because of strong incentives for innovation and weak incentives for confirmation, direct replication is rarely practiced or published. The Reproducibility Project is an open, large-scale, collaborative effort to systematically examine the rate and predictors of reproducibility in psychological science. So far, 72 volunteer researchers from 41 institutions have organized to openly and transparently replicate studies published in three prominent psychological journals in 2008. Multiple methods will be used to evaluate the findings, calculate an empirical rate of replication, and investigate factors that predict reproducibility. Whatever the result, a better understanding of reproducibility will ultimately improve confidence in scientific methodology and findings.},
Doi = {10.1177/1745691612462588},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JFFM4EBN\\Collaboration - 2012 - An Open, Large-Scale, Collaborative Effort to Esti.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KVKQRFJ8\\657.html:text/html},
ISSN = {1745-6916, 1745-6924},
Keywords = {methodology, open, psychological science, replication, reproducibility},
Language = {en},
Url = {http://pps.sagepub.com/content/7/6/657},
Urldate = {2014-09-08}
}
+
@Book{openmaximizing,
Title = {Maximizing the Reproducibility of Your Research},
Author = {Open Science Collaboration and others},
Editor = {Lilienfield, S.O. and Waldman, I.D.},
Publisher = {Wiley},
Year = {2014},
+
Address = {New York, NY},
+
Journal = {Psychological science under scrutiny: Recent challenges and proposed solutions.},
Url = {https://osf.io/nte3j/}
}
+
@Article{cyranoski_cloning_2014,
Title = {Cloning comeback},
Author = {Cyranoski, David},
Journal = {Nature},
Year = {2014},
+
Month = jan,
Number = {7484},
Pages = {468--471},
Volume = {505},
+
Doi = {10.1038/505468a},
File = {Cloning comeback \: Nature News & Comment:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\PCNSJG73\\cloning-comeback-1.html:text/html},
ISSN = {0028-0836, 1476-4687},
Url = {http://www.nature.com/news/cloning-comeback-1.14504},
Urldate = {2014-09-12}
}
+
@Article{dahl_rasmussen_walking_2011,
Title = {Walking the talk: the need for a trial registry for development interventions},
Author = {Dahl Rasmussen, Ole and Malchow-Møller, Nikolaj and Barnebeck Andersen, Thomas},
Journal = {Journal of Development Effectiveness},
Year = {2011},
+
Month = sep,
Number = {4},
Pages = {502--519},
Volume = {3},
+
Abstract = {Recent advances in the use of randomised control trials to evaluate the effect of development interventions promise to enhance our knowledge of what works and why. A core argument supporting randomised studies is the claim that they have high internal validity. The authors argue that this claim is weak as long as a trial registry of development interventions is not in place. Without a trial registry, the possibilities for data mining, created by analyses of multiple outcomes and subgroups, undermine internal validity. Drawing on experience from evidence-based medicine and recent examples from microfinance, they argue that a trial registry would also enhance external validity and foster innovative research.},
Doi = {10.1080/19439342.2011.605160},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\I3NG7FSN\\Dahl Rasmussen et al. - 2011 - Walking the talk the need for a trial registry fo.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2RWVI6V9\\19439342.2011.html:text/html},
ISSN = {1943-9342},
Shorttitle = {Walking the talk},
Url = {http://dx.doi.org/10.1080/19439342.2011.605160},
Urldate = {2014-08-27}
}
+
@Article{dal-re_making_2014,
Title = {Making Prospective Registration of Observational Research a Reality},
Author = {Dal-Re, Rafael and Ioannidis, John P. and Bracken, Michael B. and Buffler, Patricia A. and Chan, An-Wen and Franco, Eduardo L. and Vecchia, Carlo La and Weiderpass, Elisabete},
Journal = {Science Translational Medicine},
Year = {2014},
+
Month = feb,
Number = {224},
Pages = {224cm1--224cm1},
Volume = {6},
+
Abstract = {{AbstractBack} to Top
The vast majority of health-related observational studies are not prospectively registered and the advantages of registration have not been fully appreciated. Nonetheless, international standards require approval of study protocols by an independent ethics committee before the study can begin. We suggest that there is an ethical and scientific imperative to publicly preregister key information from newly approved protocols, which should be required by funders. Ultimately, more complete information may be publicly available by disclosing protocols, analysis plans, data sets, and raw data.},
Doi = {10.1126/scitranslmed.3007513},
ISSN = {1946-6234, 1946-6242},
Language = {en},
Pmid = {24553383},
Url = {http://stm.sciencemag.org/content/6/224/224cm1},
Urldate = {2014-10-28}
}
+
@Article{DeAngelis2004,
Title = {Clinical Trial Registration: A Statement from the International Committee of Medical Journal Editors},
Author = {De Angelis, Catherine AND Jeffrey M. Drazen AND Frank A. Frizelle AND Charlotte Haug AND John Hoey AND Richard Horton AND Sheldon Kotzin},
Journal = {New England Journal of Medicine},
Year = {2004},
Note = {doi:10.1056/NEJMe048225},
Number = {12},
Pages = {1250-1251},
Volume = {351},
+
Owner = {garret},
Timestamp = {2014.12.02}
}
+
@Misc{deaton_letter_2012,
Title = {Letter from America - Royal Economic Society},
+
Author = {Deaton, Angus},
Month = oct,
Year = {2012},
+
Abstract = {Angus Deaton reports on the creeping regulation affecting academic research in {US} universities.},
File = {Letter from America - Royal Economic Society:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\3EU96EGG\\article1Oct12Correspondence.html:text/html},
Journal = {Your wolf is interfering with my t-value!},
Url = {http://www.res.org.uk/view/article1Oct12Correspondence.html},
Urldate = {2014-10-23}
}
+
@Article{Deere1995minwage,
Title = {Employment and the 1990-1991 Minimum-Wage Hike},
Author = {Deere, Donald AND Kevin M. Murphy and Finis Welch. 1995},
Journal = {American Economic Review Papers and Proceedings},
Year = {1995},
+
Month = {May},
Number = {2},
Pages = {232-237},
Volume = {85},
+
Owner = {garret},
Timestamp = {2015.07.03}
}
+
@Article{delong_are_1992,
Title = {Are all Economic Hypotheses False?},
Author = {DeLong, J. Bradford and Lang, Kevin},
Journal = {Journal of Political Economy},
Year = {1992},
+
Month = dec,
Number = {6},
Pages = {1257--1272},
Volume = {100},
+
Abstract = {We develop an estimator that allows us to calculate an upper bound to the fraction of unrejected null hypotheses tested in economics journal articles that are in fact true. Our point estimate is that none of the unrejected nulls in our sample is true. We reject the hypothesis that more than one-third are true. We consider three explanations for this finding: that all null hypotheses are mere approximations, that data-mining biases reported standard errors downward, and that journals tend to publish papers that fail to reject their null hypotheses only when the null hypotheses are likely to be false. While all these explanations are important, the last seems best able to explain our findings.},
Copyright = {Copyright © 1992 The University of Chicago Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FUXR964Z\\Long and Lang - 1992 - Are all Economic Hypotheses False.pdf:application/pdf},
ISSN = {0022-3808},
Url = {http://www.jstor.org/stable/2138833},
Urldate = {2014-08-19}
}
+
@Article{denton_data_1985,
Title = {Data Mining as an Industry},
Author = {Denton, Frank T.},
Journal = {The Review of Economics and Statistics},
Year = {1985},
+
Month = feb,
Number = {1},
Pages = {124--127},
Volume = {67},
+
Abstract = {"Data mining" by an individual investigator can distort the probabilities in conventional significance tests. This paper argues that the same effect can occur when a given data set is used by more than one investigator, even if no individual investigator engages in data mining. A problem of publication selection bias is recalled and note is taken of its implications for the interpretation of published test results when there is collective data mining. Some illustrative calculations of probabilities associated with collective data mining are provided.},
Copyright = {Copyright © 1985 The {MIT} Press},
Doi = {10.2307/1928442},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BWD85GFD\\Denton - 1985 - Data Mining as an Industry.pdf:application/pdf},
ISSN = {0034-6535},
Url = {http://www.jstor.org/stable/1928442},
Urldate = {2014-08-19}
}
+
@Article{dewald_replication_1986,
Title = {Replication in Empirical Economics: The Journal of Money, Credit and Banking Project},
Author = {Dewald, William G. and Thursby, Jerry G. and Anderson, Richard G.},
Journal = {The American Economic Review},
Year = {1986},
+
Month = sep,
Number = {4},
Pages = {587--603},
Volume = {76},
+
Abstract = {This paper examines the role of replication in empirical economic research. It presents the findings of a two-year study that collected programs and data from authors and attempted to replicate their published results. Our research provides new and important information about the extent and causes of failures to replicate published results in economics. Our findings suggest that inadvertent errors in published empirical articles are a commonplace rather thana rare occurrence.},
Copyright = {Copyright © 1986 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\TT6PZMZX\\Dewald et al. - 1986 - Replication in Empirical Economics The Journal of.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Replication in Empirical Economics},
Url = {http://www.jstor.org/stable/1806061},
Urldate = {2014-08-20}
}
+
@Article{dezhbakhsh_lives_1998,
Title = {Lives Saved or Lives Lost? The Effects of Concealed-Handgun Laws on Crime},
Author = {Dezhbakhsh, Hashem and Rubin, Paul H.},
Journal = {The American Economic Review},
Year = {1998},
+
Month = may,
Number = {2},
Pages = {468--474},
Volume = {88},
+
Copyright = {Copyright © 1998 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FWNVERZE\\Dezhbakhsh and Rubin - 1998 - Lives Saved or Lives Lost The Effects of Conceale.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Lives Saved or Lives Lost?},
Url = {http://www.jstor.org/stable/116969},
Urldate = {2014-08-21}
}
+
@Article{donohue_iii_final_2003,
Title = {The Final Bullet in the Body of the More Guns, Less Crime Hypothesis},
Author = {Donohue III, John J.},
Journal = {Criminology \& Public Policy},
Year = {2003},
+
Month = jul,
Number = {3},
Pages = {397--410},
Volume = {2},
+
Doi = {10.1111/j.1745-9133.2003.tb00003.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\48D2B47B\\Iii - 2003 - The Final Bullet in the Body of the More Guns, Les.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WFSSXZI6\\abstract.html:text/html},
ISSN = {1745-9133},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1745-9133.2003.tb00003.x/abstract},
Urldate = {2014-08-21}
}
+
@Article{donohue_iii_further_2004,
Title = {Further Evidence That Legalized Abortion Lowered Crime: A Reply to Joyce},
Author = {Donohue III, John J. and Levitt, Steven D.},
Journal = {The Journal of Human Resources},
Year = {2004},
+
Month = jan,
Number = {1},
Pages = {29--49},
Volume = {39},
+
Abstract = {Donohue and Levitt (2001) suggest there is a causal link between legalized abortion and reductions in crime almost two decades later when the cohorts exposed to legalized abortion reach their peak crime years. Joyce (2003) examines crime committed in the period 1985-90 for the cohorts born immediately before and after abortion legalization. He finds little impact of legalized abortion. In this paper, we demonstrate that Joyce's failure to uncover a negative relationship between abortion and crime is a consequence of his decision to focus almost exclusively on one nonrepresentative six-year period during the peak of the crack epidemic. We provide empirical evidence that the crack-cocaine epidemic hit the high-abortion early-legalizing states earlier and more severely than other states. When we simply replicate his analyses, but extend the sample to cover the entire lives of these exact same cohorts, abortion is just as negatively related to crime as in our original analysis. Joyce's results appear to be purely an artifact of omitted variable bias due to focusing on the peak crack years without including adequate controls for crack.},
Copyright = {Copyright © 2004 The Board of Regents of the University of Wisconsin System},
Doi = {10.2307/3559004},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\UFCIIPCZ\\III and Levitt - 2004 - Further Evidence That Legalized Abortion Lowered C.pdf:application/pdf},
ISSN = {0022-166X},
Shorttitle = {Further Evidence That Legalized Abortion Lowered Crime},
Url = {http://www.jstor.org/stable/3559004},
Urldate = {2014-08-21}
}
+
@Article{donohue_iii_impact_2001,
Title = {The Impact of Legalized Abortion on Crime},
Author = {Donohue III, John J. and Levitt, Steven D.},
Journal = {The Quarterly Journal of Economics},
Year = {2001},
+
Month = may,
Number = {2},
Pages = {379--420},
Volume = {116},
+
Abstract = {We offer evidence that legalized abortion has contributed significantly to recent crime reductions. Crime began to fall roughly eighteen years after abortion legalization. The five states that allowed abortion in 1970 experienced declines earlier than the rest of the nation, which legalized in 1973 with Roe v. Wade. States with high abortion rates in the 1970s and 1980s experienced greater crime reductions in the 1990s. In high abortion states, only arrests of those born after abortion legalization fall relative to low abortion states. Legalized abortion appears to account for as much as 50 percent of the recent drop in crime.},
Copyright = {Copyright © 2001 Oxford University Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WSG9VI76\\III and Levitt - 2001 - The Impact of Legalized Abortion on Crime.pdf:application/pdf},
ISSN = {0033-5533},
Url = {http://www.jstor.org/stable/2696468},
Urldate = {2014-08-21}
}
+
@Article{donohue_measurement_2008,
Title = {Measurement Error, Legalized Abortion, and the Decline in Crime: A Response to Foote and Goetz},
Author = {Donohue, John J. and Levitt, Steven D.},
Journal = {The Quarterly Journal of Economics},
Year = {2008},
+
Month = feb,
Number = {1},
Pages = {425--440},
Volume = {123},
+
Abstract = {We are grateful to Foote and Goetz for noting that the final table of Donohue and Levitt (Quarterly Journal of Economics, 116 (2001), 379–420) inadvertently omitted state-year interactions. Correcting our mistake does not alter the sign or statistical significance of our estimates, although it does reduce their magnitude. Using a more carefully constructed measure of abortion that better links birth cohorts to abortion exposure (by using abortion data by state of residence rather than of occurrence, by adjusting for cross-state mobility, and by more precisely estimating birth years from age of arrest data), we present new evidence that abortion legalization reduces crime through both a cohort-size and a selection effect.},
Doi = {10.1162/qjec.2008.123.1.425},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\5ECJ5PZM\\Donohue and Levitt - 2008 - Measurement Error, Legalized Abortion, and the Dec.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BSQKM7CG\\425.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {Measurement Error, Legalized Abortion, and the Decline in Crime},
Url = {http://qje.oxfordjournals.org/content/123/1/425},
Urldate = {2014-08-21}
}
+
@Article{donohue_uses_2005,
Title = {Uses and Abuses of Empirical Evidence in the Death Penalty Debate},
Author = {Donohue, John J. and Wolfers, Justin},
Journal = {Stanford Law Review},
Year = {2005},
+
Month = dec,
Number = {3},
Pages = {791--845},
Volume = {58},
+
Copyright = {Copyright © 2005 Stanford Law Review},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\IT65BFI2\\Donohue and Wolfers - 2005 - Uses and Abuses of Empirical Evidence in the Death.pdf:application/pdf},
ISSN = {0038-9765},
Url = {http://www.jstor.org/stable/40040281},
Urldate = {2014-08-20}
}
+
@Article{doucouliagos_publication_2005,
Title = {Publication Bias in the Economic Freedom and Economic Growth Literature},
Author = {Doucouliagos, Chris},
Journal = {Journal of Economic Surveys},
Year = {2005},
+
Month = jul,
Number = {3},
Pages = {367--387},
Volume = {19},
+
Abstract = {Abstract.  The impact of institutions on economic performance has attracted significant attention from researchers, as well as from policy reformers. A rapidly growing area in this literature is the impact of economic freedom on economic growth. The aim of this paper was to explore publication bias in this literature by means of traditional funnel plots, meta-significance testing, as well as by bootstrapping these meta-significance tests. When all the available estimates are combined and averaged, there seems to be evidence of a genuine and positive economic freedom – economic growth effect. However, it is also shown that the economic freedom – economic growth literature is tainted strongly with publication bias. The existence of publication bias makes it difficult to identify the magnitude of the genuine effect of economic freedom on economic growth. The paper explores the differences between aggregate and disaggregate measures of economic freedom and shows that selection effects are stronger when aggregate measures are used.},
Doi = {10.1111/j.0950-0804.2005.00252.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\66ND5K27\\Doucouliagos - 2005 - Publication Bias in the Economic Freedom and Econo.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CHZTX6WD\\abstract.html:text/html},
ISSN = {1467-6419},
Keywords = {Economic freedom, Economic growth, Publication bias, Recursive meta-significance},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.0950-0804.2005.00252.x/abstract},
Urldate = {2014-08-19}
}
+
@Article{doucouliagos_are_2013,
Title = {Are All Economic Facts Greatly Exaggerated? Theory Competition and Selectivity},
Author = {Doucouliagos, Chris and Stanley, T.d.},
Journal = {Journal of Economic Surveys},
Year = {2013},
+
Month = apr,
Number = {2},
Pages = {316--339},
Volume = {27},
+
Abstract = {Abstract..There is growing concern and mounting evidence of selectivity in empirical economics. Most empirical economic literatures have a truncated distribution of results. The aim of this paper is to explore the link between publication selectivity and theory contests. This link is confirmed through the analysis of 87 distinct empirical economics literatures, involving more than three and a half thousand separate empirical studies, using objective measures of both selectivity and contests. Our meta–meta-analysis shows that publication selection is widespread, but not universal. It distorts scientific inference with potentially adverse effects on policy making, but competition and debate between rival theories reduces this selectivity and thereby improves economic inference.},
Copyright = {© 2011 Blackwell Publishing Ltd},
Doi = {10.1111/j.1467-6419.2011.00706.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NBSGU4V9\\Doucouliagos and Stanley - 2013 - Are All Economic Facts Greatly Exaggerated Theory.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\7T89ATRK\\full.html:text/html},
ISSN = {1467-6419},
Keywords = {Contested theory, Empirical economics, Meta-analysis, Publication selectivity},
Language = {en},
Shorttitle = {Are All Economic Facts Greatly Exaggerated?},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1467-6419.2011.00706.x/abstract},
Urldate = {2014-08-20}
}
+
@Article{doucouliagos2013all,
Title = {Are all economic facts greatly exaggerated? Theory competition and selectivity},
Author = {Doucouliagos, Chris and Stanley, TD},
Journal = {Journal of Economic Surveys},
Year = {2013},
Number = {2},
Pages = {316--339},
Volume = {27},
+
Publisher = {Wiley Online Library}
}
+
@Article{randomizationtoolkit,
Title = {Using randomization in development economics research: A toolkit},
Author = {Duflo, Esther and Glennerster, Rachel and Kremer, Michael},
Journal = {Handbook of development economics},
Year = {2007},
Pages = {3895--3962},
Volume = {4},
+
Publisher = {Elsevier}
}
+
@Article{duggan_more_2001,
Title = {More Guns, More Crime},
Author = {Duggan, Mark},
Journal = {Journal of Political Economy},
Year = {2001},
+
Month = oct,
Number = {5},
Pages = {1086--1114},
Volume = {109},
+
Abstract = {This paper examines the relationship between gun ownership and crime. Previous research has suffered from a lack of reliable data on gun ownership. I exploit a unique data set to reliably estimate annual rates of gun ownership at both the state and the county levels during the past two decades. My findings demonstrate that changes in gun ownership are significantly positively related to changes in the homicide rate, with this relationship driven almost entirely by an impact of gun ownership on murders in which a gun is used. The effect of gun ownership on all other crime categories is much less marked. Recent reductions in the fraction of households owning a gun can explain one�third of the differential decline in gun homicides relative to nongun homicides since 1993.},
Copyright = {Copyright © 2001 The University of Chicago Press},
Doi = {10.1086/322833},
ISSN = {0022-3808},
Url = {http://www.jstor.org/stable/10.1086/322833},
Urldate = {2014-08-21}
}
+
@Article{duvendack_replication_2013,
Title = {Replication of quantitative work in development studies: Experiences and suggestions},
Author = {Duvendack, Maren and Palmer-Jones, Richard},
Journal = {Progress in Development Studies},
Year = {2013},
+
Month = oct,
Number = {4},
Pages = {307--322},
Volume = {13},
+
Abstract = {There is a growing demand for replications of authoritative works in development studies, which reflects recent trends in other social sciences as well as challenges to important quantitative works in development studies where replications have made contested contributions to understanding. At the same time, there is a strong trend within development towards adoption of medical models of evidence-based policy to find out what policies and interventions work. Replication is a key practice of medical (and natural science) research and was advocated frequently over several decades without success. This article addresses the incentives for replication going beyond a narrow focus on extrinsic rewards, reviews some significant examples, discusses behaviour during replication and draws lessons for replicators and replicatees.},
Doi = {10.1177/1464993413490480},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BQ74ZZN5\\Duvendack and Palmer-Jones - 2013 - Replication of quantitative work in development st.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EXGEMR4N\\307.html:text/html},
ISSN = {1464-9934, 1477-027X},
Keywords = {economic methodology, incentives, professionalism, replication, research ethics, sociology of economics},
Language = {en},
Shorttitle = {Replication of quantitative work in development studies},
Url = {http://pdj.sagepub.com/content/13/4/307},
Urldate = {2014-08-21}
}
+
@Article{duvendack_high_2012,
Title = {High Noon for Microfinance Impact Evaluations: Re-investigating the Evidence from Bangladesh},
Author = {Duvendack, Maren and Palmer-Jones, Richard},
Journal = {Journal of Development Studies},
Year = {2012},
Number = {12},
Pages = {1864--1880},
Volume = {48},
+
Abstract = {Abstract Recently, microfinance has come under increasing criticism raising questions of the validity of iconic studies which have justified it, such as Pitt and Khandker. Chemin applied propensity score matching to the Pitt and Khandker data, finding different impacts, but does not disaggregate by gender of borrower. We first replicate Chemin and extend his analysis in two ways. We test the robustness of propensity score matching results to selection on unobservables using sensitivity analysis, and we investigate propensity score matching estimates of impacts by gender of borrowers. The mainly insignificant impacts of microfinance differ greatly by gender of borrower, but are all vulnerable to selection on unobservables. We are therefore not convinced that the relationships between microfinance and outcomes are causal with these data.},
Doi = {10.1080/00220388.2011.646989},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\MNR6R6A3\\Duvendack and Palmer-Jones - 2012 - High Noon for Microfinance Impact Evaluations Re-.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\E2ABQMHH\\00220388.2011.html:text/html},
ISSN = {0022-0388},
Shorttitle = {High Noon for Microfinance Impact Evaluations},
Url = {http://dx.doi.org/10.1080/00220388.2011.646989},
Urldate = {2014-08-21}
}
+
@Article{duvendack_response_2012,
Title = {Response to Chemin and to Pitt},
Author = {Duvendack, Maren and Palmer-Jones, Richard},
Journal = {Journal of Development Studies},
Year = {2012},
Number = {12},
Pages = {1892--1897},
Volume = {48},
+
Abstract = {Abstract We reply to the discussion and criticisms of Matthieu Chemin ({MC}) and Mark Pitt ({MP}) to our paper ((Duvendack and Palmer-Jones ({DPJ})) (all this issue). {MC} clarifies many issues which now make replication pure probable (but not yet achieved), and {MP} identifies a number of problems with {DPJ} (some of which are shared with Chemin, 2008). Chemin (2008) made at least one crucial undocumented and unrealistic assumption, and did not document many of his variable constructions. {MP} correctly identifies inappropriate members of control groups, and other problems, but his claim that his propensity score matching ({PSM}) results provide support for Pitt and Khandker's (1998 – {PnK}) most important claim is misleading as it is not robust. We see no reason to change our conclusion that {PnK} is limited as an evaluation of microfinance by a weak research design which cannot be convincingly mitigated by the sophisticated methods used in {PnK}, or by {PSM}.},
Doi = {10.1080/00220388.2012.747781},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DH9ZEJI2\\Duvendack and Palmer-Jones - 2012 - Response to Chemin and to Pitt.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\4IWADWAJ\\00220388.2012.html:text/html},
ISSN = {0022-0388},
Url = {http://dx.doi.org/10.1080/00220388.2012.747781},
Urldate = {2014-08-21}
}
+
@Article{easterly_aid_2004,
Title = {Aid, Policies, and Growth: Comment},
Author = {Easterly, William and Levine, Ross and Roodman, David},
Journal = {The American Economic Review},
Year = {2004},
+
Month = jun,
Number = {3},
Pages = {774--780},
Volume = {94},
+
Copyright = {Copyright © 2004 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JN48ADJB\\Easterly et al. - 2004 - Aid, Policies, and Growth Comment.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Aid, Policies, and Growth},
Url = {http://www.jstor.org/stable/3592954},
Urldate = {2014-08-20}
}
+
@TechReport{ebenstein_fertility_2007,
Title = {Fertility Choices and Sex Selection in Asia: Analysis and Policy},
Author = {Ebenstein, Avraham Y.},
Institution = {Social Science Research Network},
Year = {2007},
+
Address = {Rochester, {NY}},
Month = feb,
Number = {{ID} 965551},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {High sex ratios in China and India have historically concerned researchers (Sen 1990) and their recent increase has alarmed policymakers worldwide. This paper identifies sex selection via infanticide and abortion as the principal explanation for the sex ratio distortion, and rules out competing explanations such as biology (Oster 2005) or differential mortality rates. Consistent with recent work (Jha et al. 2006), I find that the sex ratio of first-order births is close to the natural rate and steeply rising following the birth of low-order daughters, indicating that mothers are practicing pre-natal sex selection or immediate infanticide. Sex ratios are found to be higher among those anticipating lower fertility, such as those under stricter government fertility limits. I present a model of a mother's fertility choice when she has access to a sex-selection technology and faces a mandated fertility limit. By exploiting variation in fines levied in China for unsanctioned births, I demonstrate that higher fine regimes discourage fertility but are associated with higher sex ratios among those who choose to have an additional child. I then estimate a structural model of parental preferences using China's 2000 census data that indicates that a son is worth 2.90 years of income more than a daughter, and the premium is highest among less educated mothers and rural families. I conclude with a set of simulations to model the effect on sex ratios and total fertility of a proposed subsidy to families who fail to have a son, and find that such a policy would reduce sex ratios and lower overall fertility.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DZ47VIV6\\papers.html:text/html},
Keywords = {China, missing girls, sex ratio, sex-selective abortion},
Shorttitle = {Fertility Choices and Sex Selection in Asia},
Url = {http://papers.ssrn.com/abstract=965551},
Urldate = {2014-08-20}
}
+
@Article{von_elm_strengthening_2007,
Title = {The Strengthening the Reporting of Observational Studies in Epidemiology ({STROBE}) Statement: Guidelines for reporting observational studies},
Author = {von Elm, Erik and Altman, Douglas G. and Egger, Matthias and Pocock, Stuart J. and Gtzsche, Peter C. and Vandenbroucke, Jan P.},
Journal = {Preventive Medicine},
Year = {2007},
+
Month = oct,
Number = {4},
Pages = {247--251},
Volume = {45},
+
Abstract = {Much biomedical research is observational. The reporting of such research is often inadequate, which hampers the assessment of its strengths and weaknesses and of a study's generalisability. The Strengthening the Reporting of Observational Studies in Epidemiology ({STROBE}) Initiative developed recommendations on what should be included in an accurate and complete report of an observational study. We defined the scope of the recommendations to cover three main study designs: cohort, case–control and cross-sectional studies. We convened a 2-day workshop in September 2004, with methodologists, researchers, and journal editors to draft a checklist of items. This list was subsequently revised during several meetings of the coordinating group and in e-mail discussions with the larger group of {STROBE} contributors, taking into account empirical evidence and methodological considerations. The workshop and the subsequent iterative process of consultation and revision resulted in a checklist of 22 items (the {STROBE} Statement) that relate to the title, abstract, introduction, methods, results, and discussion sections of articles. 18 items are common to all three study designs and four are specific for cohort, case–control, or cross-sectional studies. A detailed Explanation and Elaboration document is published separately and is freely available on the websites of {PLoS} Medicine, Annals of Internal Medicine and Epidemiology. We hope that the {STROBE} Statement will contribute to improving the quality of reporting of observational studies.},
Doi = {10.1016/j.ypmed.2007.08.012},
File = {ScienceDirect Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JCN73RKN\\von Elm et al. - 2007 - The Strengthening the Reporting of Observational S.pdf:application/pdf;ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\E7UD734I\\S009174350700374X.html:text/html},
ISSN = {0091-7435},
Keywords = {Epidemiology, Observational studies, Quality of reporting, Reporting guidelines},
Shorttitle = {The Strengthening the Reporting of Observational Studies in Epidemiology ({STROBE}) Statement},
Url = {http://www.sciencedirect.com/science/article/pii/S009174350700374X},
Urldate = {2014-08-29}
}
+
@Article{fanelli_negative_2012,
Title = {Negative results are disappearing from most disciplines and countries},
Author = {Fanelli, Daniele},
Journal = {Scientometrics},
Year = {2012},
+
Month = mar,
Note = {{WOS}:000300325800009},
Number = {3},
Pages = {891--904},
Volume = {90},
+
Abstract = {Concerns that the growing competition for funding and citations might distort science are frequently discussed, but have not been verified directly. Of the hypothesized problems, perhaps the most worrying is a worsening of positive-outcome bias. A system that disfavours negative results not only distorts the scientific literature directly, but might also discourage high-risk projects and pressure scientists to fabricate and falsify their data. This study analysed over 4,600 papers published in all disciplines between 1990 and 2007, measuring the frequency of papers that, having declared to have "tested" a hypothesis, reported a positive support for it. The overall frequency of positive supports has grown by over 22\% between 1990 and 2007, with significant differences between disciplines and countries. The increase was stronger in the social and some biomedical disciplines. The United States had published, over the years, significantly fewer positive results than Asian countries (and particularly Japan) but more than European countries (and in particular the United Kingdom). Methodological artefacts cannot explain away these patterns, which support the hypotheses that research is becoming less pioneering and/or that the objectivity with which results are produced and published is decreasing.},
Doi = {10.1007/s11192-011-0494-7},
ISSN = {0138-9130},
Keywords = {animal behavior, behavioral ecology, Bias, citation, Competition, journals, Misconduct, publication, Publication bias, publish, Publish or perish, replication research, Research evaluation, science, scientific publication, statistical power},
Language = {English}
}
+
@Article{feldstein_social_1974,
Title = {Social Security, Induced Retirement, and Aggregate Capital Accumulation},
Author = {Feldstein, Martin},
Journal = {Journal of Political Economy},
Year = {1974},
+
Month = sep,
Number = {5},
Pages = {905--926},
Volume = {82},
+
Copyright = {Copyright © 1974 The University of Chicago Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DA5K2B3P\\Feldstein - 1974 - Social Security, Induced Retirement, and Aggregate.pdf:application/pdf},
ISSN = {0022-3808},
Url = {http://www.jstor.org/stable/1829174},
Urldate = {2014-09-23}
}
+
@Article{feldstein_social_1982,
Title = {Social Security and Private Saving: Reply},
Author = {Feldstein, Martin S.},
Journal = {Journal of Political Economy},
Year = {1982},
+
Month = jun,
Number = {3},
Pages = {630--642},
Volume = {90},
+
Copyright = {Copyright © 1982 The University of Chicago Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\F2SBK6IT\\Feldstein - 1982 - Social Security and Private Saving Reply.pdf:application/pdf},
ISSN = {0022-3808},
Shorttitle = {Social Security and Private Saving},
Url = {http://www.jstor.org/stable/1831374},
Urldate = {2014-09-23}
}
+
@Article{FindleyCPS,
Title = {Introduction: Special Issue on Research Transparency in the Social Sciences},
Author = {Findley, Michael AND Nathan M. Jensen AND Edmund J. Malesky AND Thomas B. Pepinsky},
Journal = {Comparative Political Studies},
Year = {2016},
+
Owner = {garret},
Timestamp = {2016.01.20}
}
+
@Article{finkelstein_oregon_2012,
Title = {The Oregon Health Insurance Experiment: Evidence from the First Year*},
Author = {Finkelstein, Amy and Taubman, Sarah and Wright, Bill and Bernstein, Mira and Gruber, Jonathan and Newhouse, Joseph P. and Allen, Heidi and Baicker, Katherine},
Journal = {The Quarterly Journal of Economics},
Year = {2012},
+
Month = aug,
Number = {3},
Pages = {1057--1106},
Volume = {127},
+
Abstract = {In 2008, a group of uninsured low-income adults in Oregon was selected by lottery to be given the chance to apply for Medicaid. This lottery provides an opportunity to gauge the effects of expanding access to public health insurance on the health care use, financial strain, and health of low-income adults using a randomized controlled design. In the year after random assignment, the treatment group selected by the lottery was about 25 percentage points more likely to have insurance than the control group that was not selected. We find that in this first year, the treatment group had substantively and statistically significantly higher health care utilization (including primary and preventive care as well as hospitalizations), lower out-of-pocket medical expenditures and medical debt (including fewer bills sent to collection), and better self-reported physical and mental health than the control group. {JEL} Codes: H51, H75, I1.},
Doi = {10.1093/qje/qjs020},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\IM4CS48H\\1057.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {The Oregon Health Insurance Experiment},
Url = {http://qje.oxfordjournals.org/content/127/3/1057},
Urldate = {2014-08-26}
}
+
@Article{food1998guidance,
Title = {Guidance for industry: E9 statistical principles for clinical trials},
Author = {Food{\ }and{\ }Drug{\ }Administration},
Journal = {Food and Drug Administration: Rockville, Maryland, USA},
Year = {1998},
+
Url = {http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm073137.pdf}
}
+
@Article{foote_impact_2008,
Title = {The Impact of Legalized Abortion on Crime: Comment},
Author = {Foote, Christopher L. and Goetz, Christopher F.},
Journal = {The Quarterly Journal of Economics},
Year = {2008},
+
Month = feb,
Number = {1},
Pages = {407--423},
Volume = {123},
+
Abstract = {This comment makes three observations about Donohue and Levitt's paper on abortion and crime (Quarterly Journal of Economics 119(1) (2001), 249–275). First, there is a coding mistake in the concluding regressions, which identify abortion's effect on crime by comparing the experiences of different age cohorts within the same state and year. Second, correcting this error and using a more appropriate per capita specification for the crime variable generates much weaker results. Third, earlier tests in the paper, which exploit cross-state rather than within-state variation, are not robust to allowing for differential state trends based on statewide crime rates that predate the period when abortion could have had a causal effect on crime.},
Doi = {10.1162/qjec.2008.123.1.407},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KQR74GVH\\Foote and Goetz - 2008 - The Impact of Legalized Abortion on Crime Comment.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\55XP26KR\\407.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {The Impact of Legalized Abortion on Crime},
Url = {http://qje.oxfordjournals.org/content/123/1/407},
Urldate = {2014-08-21}
}
+
@Article{foote_impact_2008-1,
Title = {The Impact of Legalized Abortion on Crime: Comment},
Author = {Foote, Christopher L. and Goetz, Christopher F.},
Journal = {The Quarterly Journal of Economics},
Year = {2008},
+
Month = feb,
Number = {1},
Pages = {407--423},
Volume = {123},
+
Abstract = {This comment makes three observations about Donohue and Levitt's paper on abortion and crime (Quarterly Journal of Economics 119(1) (2001), 249–275). First, there is a coding mistake in the concluding regressions, which identify abortion's effect on crime by comparing the experiences of different age cohorts within the same state and year. Second, correcting this error and using a more appropriate per capita specification for the crime variable generates much weaker results. Third, earlier tests in the paper, which exploit cross-state rather than within-state variation, are not robust to allowing for differential state trends based on statewide crime rates that predate the period when abortion could have had a causal effect on crime.},
Doi = {10.1162/qjec.2008.123.1.407},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\92B6S3HB\\Foote and Goetz - 2008 - The Impact of Legalized Abortion on Crime Comment.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\UT77Z9WK\\407.html:text/html},
ISSN = {0033-5533, 1531-4650},
Language = {en},
Shorttitle = {The Impact of Legalized Abortion on Crime},
Url = {http://qje.oxfordjournals.org/content/123/1/407},
Urldate = {2014-08-21}
}
+
@Article{franco_publication_2014,
Title = {Publication bias in the social sciences: Unlocking the file drawer},
Author = {Franco, Annie and Malhotra, Neil and Simonovits, Gabor},
Journal = {Science},
Year = {2014},
+
Month = sep,
Number = {6203},
Pages = {1502--1505},
Volume = {345},
+
Abstract = {We studied publication bias in the social sciences by analyzing a known population of conducted studies—221 in total—in which there is a full accounting of what is published and unpublished. We leveraged Time-sharing Experiments in the Social Sciences ({TESS}), a National Science Foundation–sponsored program in which researchers propose survey-based experiments to be run on representative samples of American adults. Because {TESS} proposals undergo rigorous peer review, the studies in the sample all exceed a substantial quality threshold. Strong results are 40 percentage points more likely to be published than are null results and 60 percentage points more likely to be written up. We provide direct evidence of publication bias and identify the stage of research production at which publication bias occurs: Authors do not write up and submit null findings.
The file drawer is full. Should we worry?
Experiments that produce null results face a higher barrier to publication than those that yield statistically significant differences. Whether this is a problem depends on how many null but otherwise valid results might be trapped in the file drawer. Franco et al. use a Time-sharing Experiments in the Social Sciences archive of nearly 250 peer-reviewed proposals of social science experiments conducted on nationally representative samples. They find that only 10 out of 48 null results were published, whereas 56 out of 91 studies with strongly significant results made it into a journal.
Science, this issue p. 1502},
Doi = {10.1126/science.1255484},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JS8TRIMD\\1502.html:text/html},
ISSN = {0036-8075, 1095-9203},
Language = {en},
Pmid = {25170047},
Shorttitle = {Publication bias in the social sciences},
Url = {http://www.sciencemag.org/content/345/6203/1502},
Urldate = {2014-10-15}
}
+
@Article{fuess_replication_1996,
Title = {On Replication in Business and Economics Research: The {QJBE} Case},
Author = {Fuess, Jr., Scott M.},
Journal = {Quarterly Journal of Business and Economics},
Year = {1996},
+
Month = apr,
Number = {2},
Pages = {3--13},
Volume = {35},
+
Abstract = {In 1984 the Quarterly Journal of Business and Economics ({QJBE}) became the first journal to give priority to publishing econometric replications. This study is a retrospective on the research published in the {QJBE} under this policy. Over the 1984-1994 period replications occupied more than 21 percent of the journal's contents. Further, the acceptance rate for replications has been higher than for other studies. The journal also devoted more than 5 percent of its pages to reviews of empirical literature. Replication articles have focused on financial economics and industrial organization; they tend to use publicly available data sets and not to contradict entirely the findings of earlier studies. Econometric replications have been more heavily represented in the {QJBE} than in other outlets, but they have not dominated the contents of the journal. This finding suggests that editorial policy can bring forth a supply of replication research, but other active measures are necessary before such studies become more common.},
Copyright = {Copyright © 1996 University of Nebraska-Lincoln College of Business Administration},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CQV6HM3S\\Fuess - 1996 - On Replication in Business and Economics Research.pdf:application/pdf},
ISSN = {0747-5535},
Shorttitle = {On Replication in Business and Economics Research},
Url = {http://www.jstor.org/stable/40473179},
Urldate = {2014-10-01}
}
+
@Book{gandrud2013reproducible,
Title = {Reproducible Research with R and R Studio},
Author = {Gandrud, Christopher},
Publisher = {CRC Press},
Year = {2013}
}
+
@Article{gelman_preregistration_2013,
Title = {Preregistration of Studies and Mock Reports},
Author = {Gelman, Andrew},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {40--41},
Volume = {21},
+
Doi = {10.1093/pan/mps032},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CB399XI9\\Gelman - 2013 - Preregistration of Studies and Mock Reports.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\TZ2UKXAK\\40.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Url = {http://pan.oxfordjournals.org/content/21/1/40},
Urldate = {2014-08-28}
}
+
@Article{gelman_garden_2013,
Title = {The garden of forking paths: Why multiple comparisons can be a problem, even when there is no ‘fishing expedition’or ‘p-hacking’ and the research hypothesis was posited ahead of time.},
Author = {Gelman, Andrew and Loken, Eric},
Year = {2013},
+
Month = nov,
+
Url = {http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf}
}
+
@Article{GentzkowShapiro,
Title = {Code and Data for the Social Sciences: A Practioner's Guide},
Author = {Matthew Gentzkow and Jesse Shapiro},
Year = {2014},
+
Month = {March},
+
Owner = {garret},
Timestamp = {2015.07.13},
Url = {http://faculty.chicagobooth.edu/jesse.shapiro/research/CodeAndData.pdf}
}
+
@TechReport{gerber_reporting_????,
Title = {Reporting Guidelines for Experimental Research: A Report from the Experimental Research Section Standards Committee},
Author = {Gerber, Alan and Arceneaux, Kevin and Boudreau, Cheryl and Dowling, Conor and Hillygus, Sunshine and Palfrey, Thomas and Biggers, Daniel R. and Hendry, David J.},
Year = {2014},
+
Url = {http://www.davidhendry.net/research-supplemental/gerberetal2014-reportingstandards/gerberetal2014-reportingstandards&appendix1.pdf}
}
+
@Article{gerber_testing_2001,
Title = {Testing for Publication Bias in Political Science},
Author = {Gerber, Alan S. and Green, Donald P. and Nickerson, David},
Journal = {Political Analysis},
Year = {2001},
+
Month = jan,
Number = {4},
Pages = {385--392},
Volume = {9},
+
Abstract = {If the publication decisions of journals are a function of the statistical significance of research findings, the published literature may suffer from “publication bias.� This paper describes a method for detecting publication bias. We point out that to achieve statistical significance, the effect size must be larger in small samples. If publications tend to be biased against statistically insignificant results, we should observe that the effect size diminishes as sample sizes increase. This proposition is tested and confirmed using the experimental literature on voter mobilization.},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\XK39RAGF\\Gerber et al. - 2001 - Testing for Publication Bias in Political Science.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KJXWSKA4\\385.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Url = {http://pan.oxfordjournals.org/content/9/4/385},
Urldate = {2014-08-28}
}
+
@Book{gertler_impact_2011,
Title = {Impact Evaluation in Practice},
Author = {Gertler, Paul and Martinez, Sebastian and Premand, Patrick and Rawlings, Laura B. and Vermeersch, Christel M. J.},
Publisher = {World Bank Publications},
Year = {2011},
+
Abstract = {This book offers an accessible introduction to the topic of impact evaluation and its practice in development. While the book is geared principally towards development practitioners and policymakers designing prospective impact evaluations, we trust that it will be a valuable resource for students and others interested in using impact evaluation. Prospective impact evaluations should be used selectively to assess whether or not a program has achieved its intended results, or to test alternatives for achieving those results. We consider that more and better impact evaluation will help strengthen the evidence base for development policies and programs around the world. If governments and development practitioners can make policy decisions based on evidence - including evidence generated through impact evaluation - our hope is that development resources will be spent more effectively, and ultimately have a greater impact on reducing poverty and improving people's lives. The three chapters in this handbook provide a non-technical introduction to impact evaluations, including “Why Evaluate� in Chapter 1, “How to Evaluate� in Chapter 2 and “How to Implement Impact Evaluations� in Chapter 3. These elements are the basic 'tools' needed in order to successfully carry out an impact evaluation. From a methodological standpoint our approach to impact evaluation is largely pragmatic: we think that the most appropriate methods should be identified to fit the operational context, and not the other way around. This is best achieved at the outset of the program, through the design of prospective impact evaluation that can be built into the project's implementation. We argue that gaining consensus between key stakeholders and identifying an evaluation design that fits the political and operational context is as important as the method itself. We also believe strongly that impact evaluations should be upfront about their limitations and caveats. Finally, we strongly encourage policymakers and program managers to consider impact evaluations in a logical framework that clearly sets out the causal pathways by which the program works to produce outputs and influence final outcomes, and to combine impact evaluations with monitoring and selected complementary evaluation approach to gain a full picture of performance.This book builds on a core set of teaching materials developed for the “Turning Promises to Evidence� workshops organized by the office of the Chief Economist for Human Development ({HDNCE}) in partnership with regional units and the Development Economics Research Group ({DECRG}) at the World Bank.},
ISBN = {9780821385418},
Keywords = {Business \& Economics / Development / Economic Development, Education / Evaluation \& Assessment, Political Science / Public Policy / Economic Policy},
Language = {en}
}
+
@Article{gherghina_data_2013,
Title = {Data Availability in Political Science Journals},
Author = {Gherghina, Sergiu and Katsanidou, Alexia},
Journal = {European Political Science},
Year = {2013},
+
Month = sep,
Number = {3},
Pages = {333--349},
Volume = {12},
+
Abstract = {A characteristic of recent decades of scholarly work in the social sciences has been the increased amounts of empirical research. Access and availability of data are prerequisites for further research, replication work, and scientific development. As international peer-reviewed journals have gradually become the central forum for research debate, moves towards data sharing are dependent upon the policies of journals regarding data availability. This article examines contemporary data availability policies in political science and investigates the extent to which journals adopt such policies and their content. It also identifies a few factors associated with the existence of such policies.},
Doi = {10.1057/eps.2013.8},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NTP7MNDR\\Gherghina and Katsanidou - 2013 - Data Availability in Political Science Journals.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FQKE2CNP\\eps20138a.html:text/html},
ISSN = {1680-4333},
Keywords = {data availability, political science, replication policy},
Language = {en},
Url = {http://www.palgrave-journals.com/eps/journal/v12/n3/full/eps20138a.html},
Urldate = {2014-08-28}
}
+
@TechReport{glandon_report_2010,
Title = {Report on the American Economic Review Data Availability Compliance Project},
Author = {Glandon, Philip},
Institution = {Vanderbilt University},
Year = {2010},
Month = nov,
+
Url = {https://aeaweb.org/aer/2011_Data_Compliance_Report.pdf}
}
+
@Book{glennerster_running_2013,
Title = {Running Randomized Evaluations: A Practical Guide},
Author = {Glennerster, Rachel and Takavarasha, Kudzai},
Publisher = {Princeton University Press},
Year = {2013},
Month = nov,
+
Abstract = {This book provides a comprehensive yet accessible guide to running randomized impact evaluations of social programs. Drawing on the experience of researchers at the Abdul Latif Jameel Poverty Action Lab, which has run hundreds of such evaluations in dozens of countries throughout the world, it offers practical insights on how to use this powerful technique, especially in resource-poor environments. This step-by-step guide explains why and when randomized evaluations are useful, in what situations they should be used, and how to prioritize different evaluation opportunities. It shows how to design and analyze studies that answer important questions while respecting the constraints of those working on and benefiting from the program being evaluated. The book gives concrete tips on issues such as improving the quality of a study despite tight budget constraints, and demonstrates how the results of randomized impact evaluations can inform policy. With its self-contained modules, this one-of-a-kind guide is easy to navigate. It also includes invaluable references and a checklist of the common pitfalls to avoid. Provides the most up-to-date guide to running randomized evaluations of social programs, especially in developing countries Offers practical tips on how to complete high-quality studies in even the most challenging environments Self-contained modules allow for easy reference and flexible teaching and learning Comprehensive yet nontechnical},
ISBN = {9781400848447},
Keywords = {Business \& Economics / Development / Economic Development, Political Science / History \& Theory, Political Science / Public Policy / General},
Language = {en},
Shorttitle = {Running Randomized Evaluations}
}
+
@Article{greenwald2009understanding,
Title = {Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity.},
Author = {Greenwald, Anthony G and Poehlman, T Andrew and Uhlmann, Eric Luis and Banaji, Mahzarin R},
Journal = {Journal of personality and social psychology},
Year = {2009},
Number = {1},
Pages = {17},
Volume = {97},
+
Publisher = {American Psychological Association}
}
+
@Article{gupta_cultural_2006,
Title = {Cultural versus Biological Factors in Explaining Asia's Missing Women : Response to Oster},
Author = {Gupta, Monica Das},
Journal = {Population and Development Review},
Year = {2006},
+
Month = jun,
Number = {2},
Pages = {328--332},
Volume = {32},
+
Doi = {10.1111/j.1728-4457.2006.00121.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\QQ2J68BZ\\Gupta - 2006 - Cultural versus Biological Factors in Explaining A.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6STTEKME\\abstract.html:text/html},
ISSN = {1728-4457},
Language = {en},
Shorttitle = {Cultural versus Biological Factors in Explaining Asia's “Missing Women�},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2006.00121.x/abstract},
Urldate = {2014-08-20}
}
+
@Article{gupta_explaining_2005,
Title = {Explaining Asia's “Missing Women�: A New Look at the Data},
Author = {Gupta, Monica Das},
Journal = {Population and Development Review},
Year = {2005},
+
Month = sep,
Number = {3},
Pages = {529--535},
Volume = {31},
+
Abstract = {The fact that millions of females are “missing� in East Asia and South Asia has been attributed to cultural factors that support strong son preference in these countries. A widely disseminated paper by Emily Oster argues that a large part of this phenomenon can be attributed to excessively masculine sex ratios at birth resulting from maternal infection with hepatitis B. If her thesis is true, current policies to address this problem would need to be reframed to include biological factors in addition to cultural factors. The data show, however, that whether or not females “go missing� is determined by the existing sex composition of the family into which they are conceived. Girls with no older sisters have similar chances of survival as boys. However, girls conceived in families that already have a daughter experience steeply higher probabilities of being aborted or of dying in early childhood. This indicates that cultural factors still provide the overwhelming explanation for the “missing� females.},
Doi = {10.1111/j.1728-4457.2005.00082.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8CC7PZXJ\\Gupta - 2005 - Explaining Asia's “Missing Women� A New Look at t.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WBSC9R49\\abstract.html:text/html},
ISSN = {1728-4457},
Language = {en},
Shorttitle = {Explaining Asia's “Missing Women�},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2005.00082.x/abstract},
Urldate = {2014-08-20}
}
+
@Article{hamermesh_viewpoint:_2007,
Title = {Viewpoint: Replication in economics},
Author = {Hamermesh, Daniel S.},
Journal = {Canadian Journal of Economics/Revue canadienne d'economique},
Year = {2007},
+
Month = aug,
Number = {3},
Pages = {715--733},
Volume = {40},
+
Abstract = {Abstract.  This examination of the role and potential for replication in economics points out the paucity of both pure replication – checking on others' published papers using their data – and scientific replication – using data representing different populations in one's own work or in a comment. Several controversies in empirical economics are used to illustrate how and how not to behave when replicating others' work. The incentives for replication are examined, and proposals aimed at journal editors and authors are advanced that might stimulate an activity that most economists applaud but few perform.},
Doi = {10.1111/j.1365-2966.2007.00428.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\R8X563MJ\\Hamermesh - 2007 - Viewpoint Replication in economics.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\MZQCI34R\\full.html:text/html},
ISSN = {1540-5982},
Keywords = {A14, B41, C59},
Language = {en},
Shorttitle = {Viewpoint},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2966.2007.00428.x/abstract},
Urldate = {2014-08-21}
}
+
@Article{hartung_reporting_2014,
Title = {Reporting Discrepancies Between the {ClinicalTrials}.gov Results Database and Peer-Reviewed {PublicationsDiscrepancies} Between {ClinicalTrials}.gov and Peer-Reviewed Publications},
Author = {Hartung, Daniel M. and Zarin, Deborah A. and Guise, Jeanne-Marie and McDonagh, Marian and Paynter, Robin and Helfand, Mark},
Journal = {Annals of Internal Medicine},
Year = {2014},
+
Month = apr,
Number = {7},
Pages = {477--483},
Volume = {160},
+
Abstract = {Background: {ClinicalTrials}.gov requires reporting of result summaries for many drug and device trials.Purpose: To evaluate the consistency of reporting of trials that are registered in the {ClinicalTrials}.gov results database and published in the literature.Data Sources: {ClinicalTrials}.gov results database and matched publications identified through {ClinicalTrials}.gov and a manual search of 2 electronic databases.Study Selection: 10\% random sample of phase 3 or 4 trials with results in the {ClinicalTrials}.gov results database, completed before 1 January 2009, with 2 or more groups.Data Extraction: One reviewer extracted data about trial design and results from the results database and matching publications. A subsample was independently verified.Data Synthesis: Of 110 trials with results, most were industry-sponsored, parallel-design drug studies. The most common inconsistency was the number of secondary outcome measures reported (80\%). Sixteen trials (15\%) reported the primary outcome description inconsistently, and 22 (20\%) reported the primary outcome value inconsistently. Thirty-eight trials inconsistently reported the number of individuals with a serious adverse event ({SAE}); of these, 33 (87\%) reported more {SAEs} in {ClinicalTrials}.gov. Among the 84 trials that reported {SAEs} in {ClinicalTrials}.gov, 11 publications did not mention {SAEs}, 5 reported them as zero or not occurring, and 21 reported a different number of {SAEs}. Among 29 trials that reported deaths in {ClinicalTrials}.gov, 28\% differed from the matched publication.Limitation: Small sample that included earliest results posted to the database.Conclusion: Reporting discrepancies between the {ClinicalTrials}.gov results database and matching publications are common. Which source contains the more accurate account of results is unclear, although {ClinicalTrials}.gov may provide a more comprehensive description of adverse events than the publication.Primary Funding Source: Agency for Healthcare Research and Quality.},
Doi = {10.7326/M13-0480},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NET5TC3N\\Hartung et al. - 2014 - Reporting Discrepancies Between the ClinicalTrials.pdf:application/pdf},
ISSN = {0003-4819},
Url = {http://dx.doi.org/10.7326/M13-0480},
Urldate = {2014-09-18}
}
+
@TechReport{harvey_and_2014,
Title = {…and the Cross-Section of Expected Returns},
Author = {Harvey, Campbell R. and Liu, Yan and Zhu, Heqing},
Institution = {Social Science Research Network},
Year = {2014},
+
Address = {Rochester, {NY}},
Month = aug,
Number = {{ID} 2249314},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {Hundreds of papers and hundreds of factors attempt to explain the cross-section of expected returns. Given this extensive data mining, it does not make any economic or statistical sense to use the usual significance criteria for a newly discovered factor, e.g., a t-ratio greater than 2.0. However, what hurdle should be used for current research? Our paper introduces a multiple testing framework and provides a time series of historical significance cutoffs from the first empirical tests in 1967 to today. Our new method allows for correlation among the tests as well as publication bias. We also project forward 20 years assuming the rate of factor production remains similar to the experience of the last few years. The estimation of our model suggests that today a newly discovered factor needs to clear a much higher hurdle, with a t-ratio greater than 3.0. Echoing a recent disturbing conclusion in the medical literature, we argue that most claimed research findings in financial economics are likely false. Our key results are summarized:},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\5CRDDEIH\\papers.html:text/html},
Keywords = {3-factor model, Beta, Bonferroni, Factor zoo, {HML}, Idiosyncratic volatility, Liquidity, Momentum, Multiple tests, Risk factors, Skewness, {SMB}, Volatility},
Url = {http://papers.ssrn.com/abstract=2249314},
Urldate = {2014-08-21}
}
+
@Article{NosekPreRegistered,
Title = {In Search of an Association between Conception Risk and Prejudice},
Author = {Hawkins, Carlee Beth and Cailey E. Fitzgerald and Brian A. Nosek},
Journal = {Psychological Science},
Year = {2015},
+
Month = {February},
Number = {2},
Pages = {249-252},
Volume = {26},
+
Doi = {10.1177/0956797614553121},
Owner = {garret},
Timestamp = {2014.11.19},
Url = {http://pss.sagepub.com/content/26/2/249}
}
+
@Article{heffetz_privacy_2014,
Title = {Privacy and Data-Based Research},
Author = {Heffetz, Ori and Ligett, Katrina},
Journal = {Journal of Economic Perspectives},
Year = {2014},
+
Month = may,
Number = {2},
Pages = {75--98},
Volume = {28},
+
Doi = {10.1257/jep.28.2.75},
File = {JEP (28,2) p. 75 - Privacy and Data-Based Research:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\H3W29M6H\\articles.html:text/html},
ISSN = {0895-3309},
Language = {en},
Url = {https://www-aeaweb-org.proxy.swarthmore.edu/articles.php?doi=10.1257/jep.28.2.75},
Urldate = {2014-09-12}
}
+
@Article{helland_using_2004,
Title = {Using Placebo Laws to Test "More Guns, Less Crime"},
Author = {Helland, Eric and Tabarrok, Alexander},
Journal = {Advances in Economic Analysis \& Policy},
Year = {2004},
+
Month = jan,
Number = {1},
Volume = {4},
+
Abstract = {{AbstractWe} reexamine Mustard and Lott’s controversial study on the effect of “shall-issue� gun laws on crime using an empirical standard error function randomly generated from “placebo� laws. We find that the effect of shall-issue laws on crime is much less well-estimated than the Mustard and Lott (1997) and Lott (2000) results suggest. We also find, however, that the cross equation restrictions implied by the Lott-Mustard theory are supported. A boomlet has occurred in recent years in the use of quasi-natural experiments to answer important questions of public policy. The intuitive power of this approach, however, has sometimes diverted attention from the statistical assumptions that must be made, particularly regarding standard errors. Failing to take into account serial correlation and grouped data can dramatically reduce standard errors suggesting greater certainty in effects than is actually the case. We find that the placebo law technique (Bertrand, Duflo and Mullainathan 2002) is a useful addition to the econometrician’s toolkit.},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2FFS2ITD\\Helland and Tabarrok - 2004 - Using Placebo Laws to Test More Guns, Less Crime.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\R43HS2GK\\bejeap.2004.4.1.1182.html:text/html},
Url = {http://www.degruyter.com/view/j/bejeap.2004.4.issue-1/bejeap.2004.4.1.1182/bejeap.2004.4.1.1182.xml},
Urldate = {2014-08-21}
}
+
@Article{herndon_does_2014,
Title = {Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff},
Author = {Herndon, Thomas and Ash, Michael and Pollin, Robert},
Journal = {Cambridge Journal of Economics},
Year = {2014},
+
Month = mar,
Number = {2},
Pages = {257--279},
Volume = {38},
+
Abstract = {We replicate Reinhart and Rogoff (2010A and 2010B) and find that selective exclusion of available data, coding errors and inappropriate weighting of summary statistics lead to serious miscalculations that inaccurately represent the relationship between public debt and {GDP} growth among 20 advanced economies. Over 1946–2009, countries with public debt/{GDP} ratios above 90\% averaged 2.2\% real annual {GDP} growth, not −0.1\% as published. The published results for (i) median {GDP} growth rates for the 1946–2009 period and (ii) mean and median {GDP} growth figures over 1790–2009 are all distorted by similar methodological errors, although the magnitudes of the distortions are somewhat smaller than with the mean figures for 1946–2009. Contrary to Reinhart and Rogoff’s broader contentions, both mean and median {GDP} growth when public debt levels exceed 90\% of {GDP} are not dramatically different from when the public debt/{GDP} ratios are lower. The relationship between public debt and {GDP} growth varies significantly by period and country. Our overall evidence refutes {RR}’s claim that public debt/{GDP} ratios above 90\% consistently reduce a country’s {GDP} growth.},
Doi = {10.1093/cje/bet075},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\J9SBWB55\\Herndon et al. - 2014 - Does high public debt consistently stifle economic.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JC879UJJ\\257.html:text/html},
ISSN = {0309-166X, 1464-3545},
Keywords = {Austerity, E60, E62, E65, Public debt},
Language = {en},
Shorttitle = {Does high public debt consistently stifle economic growth?},
Url = {http://cje.oxfordjournals.org/content/38/2/257},
Urldate = {2014-08-20}
}
+
@Article{hines2014sorting,
Title = {Sorting Out the FACS: A Devil in the Details},
Author = {Hines, William C and Su, Ying and Kuhn, Irene and Polyak, Kornelia and Bissell, Mina J},
Journal = {Cell reports},
Year = {2014},
Number = {5},
Pages = {779--781},
Volume = {6},
+
Publisher = {Elsevier Science}
}
+
@Article{vocationalTurkey,
Title = {The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey},
Author = {Hirshleifer, Sarojini and McKenzie, David and Almeida, Rita and Ridao-Cano, Cristobal},
Journal = {The Economic Journal},
Year = {2014},
Pages = {n/a--n/a},
+
Doi = {10.1111/ecoj.12211},
ISSN = {1468-0297},
Keywords = {Vocational training, Active Labour Market Programs, Randomized Experiment, Private Provision},
Url = {http://dx.doi.org/10.1111/ecoj.12211}
}
+
@Article{holm_multipletesting,
Title = {A Simple Sequentially Rejective Multiple Test Procedure},
Author = {Holm, Sture},
Journal = {Scandinavian Journal of Statistics},
Year = {1979},
Number = {2},
Pages = {pp. 65-70},
Volume = {6},
+
Abstract = {This paper presents a simple and widely applicable multiple test procedure of the sequentially rejective type, i.e. hypotheses are rejected one at a time until no further rejections can be done. It is shown that the test has a prescribed level of significance protection against error of the first kind for any combination of true hypotheses. The power properties of the test and a number of possible applications are also discussed.},
Copyright = {Copyright © 1979 Board of the Foundation of the Scandinavian Journal of Statistics},
ISSN = {03036898},
Jstor_articletype = {research-article},
Jstor_formatteddate = {1979},
Language = {English},
Publisher = {Wiley on behalf of Board of the Foundation of the Scandinavian Journal of Statistics},
Url = {http://www.jstor.org/stable/4615733}
}
+
@Article{hoxby_does_2007,
Title = {Does Competition among Public Schools Benefit Students and Taxpayers? Reply},
Author = {Hoxby, Caroline M.},
Journal = {The American Economic Review},
Year = {2007},
+
Month = dec,
Number = {5},
Pages = {2038--2055},
Volume = {97},
+
Copyright = {Copyright © 2007 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2SKTX83D\\Hoxby - 2007 - Does Competition among Public Schools Benefit Stud.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Does Competition among Public Schools Benefit Students and Taxpayers?},
Url = {http://www.jstor.org/stable/30034600},
Urldate = {2014-08-20}
}
+
@Article{hoxby_does_2000,
Title = {Does Competition among Public Schools Benefit Students and Taxpayers?},
Author = {Hoxby, Caroline M.},
Journal = {The American Economic Review},
Year = {2000},
+
Month = dec,
Number = {5},
Pages = {1209--1238},
Volume = {90},
+
Abstract = {Tiebout choice among districts is the most powerful market force in American public education. Naive estimates of its effects are biased by endogenous district formation. I derive instruments from the natural boundaries in a metropolitan area. My results suggest that metropolitan areas with greater Tiebout choice have more productive public schools and less private schooling. Little of the effect of Tiebout choice works through its effect on household sorting. This finding may be explained by another finding: students are equally segregated by school in metropolitan areas with greater and lesser degrees of Tiebout choice among districts.},
Copyright = {Copyright © 2000 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SWX7PN98\\Hoxby - 2000 - Does Competition among Public Schools Benefit Stud.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/2677848},
Urldate = {2014-08-20}
}
+
@Article{hsiang2013climate,
Title = {Quantifying the influence of climate on human conflict},
Author = {Hsiang, Solomon M and Burke, Marshall and Miguel, Edward},
Journal = {Science},
Year = {2013},
Number = {6151},
Pages = {1235367},
Volume = {341},
+
Publisher = {American Association for the Advancement of Science}
}
+
@Article{humphreys_fishing_2013,
Title = {Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration},
Author = {Humphreys, Macartan and Sierra, Raul Sanchez de la and Windt, Peter van der},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {1--20},
Volume = {21},
+
Abstract = {Social scientists generally enjoy substantial latitude in selecting measures and models for hypothesis testing. Coupled with publication and related biases, this latitude raises the concern that researchers may intentionally or unintentionally select models that yield positive findings, leading to an unreliable body of published research. To combat this “fishing� problem in medical studies, leading journals now require preregistration of designs that emphasize the prior identification of dependent and independent variables. However, we demonstrate here that even with this level of advanced specification, the scope for fishing is considerable when there is latitude over selection of covariates, subgroups, and other elements of an analysis plan. These concerns could be addressed through the use of a form of comprehensive registration. We experiment with such an approach in the context of an ongoing field experiment for which we drafted a complete “mock report� of findings using fake data on treatment assignment. We describe the advantages and disadvantages of this form of registration and propose that a comprehensive but nonbinding approach be adopted as a first step to combat fishing by social scientists. Likely effects of comprehensive but nonbinding registration are discussed, the principal advantage being communication rather than commitment, in particular that it generates a clear distinction between exploratory analyses and genuine tests.},
Doi = {10.1093/pan/mps021},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\QVHCIGGE\\Humphreys et al. - 2013 - Fishing, Commitment, and Communication A Proposal.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BZWC5VG7\\1.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Shorttitle = {Fishing, Commitment, and Communication},
Url = {http://pan.oxfordjournals.org/content/21/1/1},
Urldate = {2014-08-27}
}
+
@Article{hunter_desperate_2001,
Title = {The Desperate Need for Replications},
Author = {Hunter, John},
Journal = {Journal of Consumer Research},
Year = {2001},
+
Month = jun,
Number = {1},
Pages = {149--158},
Volume = {28},
+
Abstract = {An overemphasis on creativity for evaluating research has lead to a serious devaluation of replication studies. However, we need a total sample size of \$N=153,669\$ to estimate a causal effect to two digits, which is quite rare for a single study. The only way to get accurate estimation is to average across replications. If the average sample size were as high as \$N=200\$ , we would need over 700 replication studies. Scientific replications are more problematic than pure statistical replications, and so we need even more replications to achieve reasonable accuracy.},
Copyright = {Copyright © 2001 Journal of Consumer Research, Inc.},
Doi = {10.1086/jcr.2001.28.issue-1},
ISSN = {0093-5301},
Url = {http://www.jstor.org/stable/10.1086/321953},
Urldate = {2014-09-23}
}
+
@Article{husereau_consolidated_2013,
Title = {Consolidated Health Economic Evaluation Reporting Standards ({CHEERS}) statement},
Author = {Husereau, Don and Drummond, Michael and Petrou, Stavros and Carswell, Chris and Moher, David and Greenberg, Dan and Augustovski, Federico and Briggs, Andrew H. and Mauskopf, Josephine and Loder, Elizabeth and {CHEERS Task Force}},
Journal = {Value in Health: The Journal of the International Society for Pharmacoeconomics and Outcomes Research},
Year = {2013},
+
Month = apr,
Number = {2},
Pages = {e1--5},
Volume = {16},
+
Abstract = {Economic evaluations of health interventions pose a particular challenge for reporting. There is also a need to consolidate and update existing guidelines and promote their use in a user friendly manner. The Consolidated Health Economic Evaluation Reporting Standards ({CHEERS}) statement is an attempt to consolidate and update previous health economic evaluation guidelines efforts into one current, useful reporting guidance. The primary audiences for the {CHEERS} statement are researchers reporting economic evaluations and the editors and peer reviewers assessing them for publication. The need for new reporting guidance was identified by a survey of medical editors. A list of possible items based on a systematic review was created. A two round, modified Delphi panel consisting of representatives from academia, clinical practice, industry, government, and the editorial community was conducted. Out of 44 candidate items, 24 items and accompanying recommendations were developed. The recommendations are contained in a user friendly, 24 item checklist. A copy of the statement, accompanying checklist, and this report can be found on the {ISPOR} Health Economic Evaluations Publication Guidelines Task Force website: (www.ispor.org/{TaskForces}/{EconomicPubGuidelines}.asp). We hope {CHEERS} will lead to better reporting, and ultimately, better health decisions. To facilitate dissemination and uptake, the {CHEERS} statement is being co-published across 10 health economics and medical journals. We encourage other journals and groups, to endorse {CHEERS}. The author team plans to review the checklist for an update in five years.},
Doi = {10.1016/j.jval.2013.02.010},
ISSN = {1524-4733},
Keywords = {Biomedical Research, Checklist, Cost-Benefit Analysis, Delphi Technique, Economics, Medical, Evaluation Studies as Topic, Guidelines as Topic, Health Policy, Humans, Journalism, Medical, Peer Review, Research, Resource Allocation},
Language = {eng},
Pmid = {23538200}
}
+
@Article{ioannidis_effectiveness_2008,
Title = {Effectiveness of antidepressants: an evidence myth constructed from a thousand randomized trials?},
Author = {Ioannidis, John PA},
Journal = {Philosophy, Ethics, and Humanities in Medicine},
Year = {2008},
+
Month = may,
Number = {1},
Pages = {14},
Volume = {3},
+
Abstract = {Antidepressants, in particular newer agents, are among the most widely prescribed medications worldwide with annual sales of billions of dollars. The introduction of these agents in the market has passed through seemingly strict regulatory control. Over a thousand randomized trials have been conducted with antidepressants. Statistically significant benefits have been repeatedly demonstrated and the medical literature is flooded with several hundreds of "positive" trials (both pre-approval and post-approval). However, two recent meta-analyses question this picture. The first meta-analysis used data that were submitted to {FDA} for the approval of 12 antidepressant drugs. While only half of these trials had formally significant effectiveness, published reports almost ubiquitously claimed significant results. "Negative" trials were either left unpublished or were distorted to present "positive" results. The average benefit of these drugs based on the {FDA} data was of small magnitude, while the published literature suggested larger benefits. A second meta-analysis using also {FDA}-submitted data examined the relationship between treatment effect and baseline severity of depression. Drug-placebo differences increased with increasing baseline severity and the difference became large enough to be clinically important only in the very small minority of patient populations with severe major depression. In severe major depression, antidepressants did not become more effective, simply placebo lost effectiveness. These data suggest that antidepressants may be less effective than their wide marketing suggests. Short-term benefits are small and long-term balance of benefits and harms is understudied. I discuss how the use of many small randomized trials with clinically non-relevant outcomes, improper interpretation of statistical significance, manipulated study design, biased selection of study populations, short follow-up, and selective and distorted reporting of results has built and nourished a seemingly evidence-based myth on antidepressant effectiveness and how higher evidence standards, with very large long-term trials and careful prospective meta-analyses of individual-level data may reach closer to the truth and clinically useful evidence.
{PMID}: 18505564},
Copyright = {2008 Ioannidis; licensee {BioMed} Central Ltd.},
Doi = {10.1186/1747-5341-3-14},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\S5SJ29NE\\Ioannidis - 2008 - Effectiveness of antidepressants an evidence myth.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HMHTZPTG\\abstract.html:text/html},
ISSN = {1747-5341},
Language = {en},
Pmid = {18505564},
Shorttitle = {Effectiveness of antidepressants},
Url = {http://www.peh-med.com/content/3/1/14/abstract},
Urldate = {2014-10-09}
}
+
@Article{ioannidis_why_2005,
Title = {Why Most Published Research Findings Are False},
Author = {Ioannidis, John P. A.},
Journal = {{PLoS} Med},
Year = {2005},
+
Month = aug,
Number = {8},
Pages = {e124},
Volume = {2},
+
Abstract = {Published research findings are sometimes refuted by subsequent evidence, says Ioannidis, with ensuing confusion and disappointment.},
Doi = {10.1371/journal.pmed.0020124},
File = {PLoS Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2V44N3ZE\\Ioannidis - 2005 - Why Most Published Research Findings Are False.pdf:application/pdf;PLoS Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NMZ3G7BJ\\journal.pmed.html:text/html},
Url = {http://dx.doi.org/10.1371/journal.pmed.0020124},
Urldate = {2014-08-19}
}
+
@Article{ioannidis_why_2005-1,
Title = {Why Most Published Research Findings Are False},
Author = {Ioannidis, John P. A.},
Journal = {{PLoS} Med},
Year = {2005},
+
Month = aug,
Number = {8},
Pages = {e124},
Volume = {2},
+
Abstract = {Published research findings are sometimes refuted by subsequent evidence, says Ioannidis, with ensuing confusion and disappointment.},
Doi = {10.1371/journal.pmed.0020124},
File = {PLoS Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SXACAGP3\\Ioannidis - 2005 - Why Most Published Research Findings Are False.pdf:application/pdf;PLoS Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CEMBCCE3\\journal.pmed.html:text/html},
Url = {http://dx.doi.org/10.1371/journal.pmed.0020124},
Urldate = {2014-08-28}
}
+
@Article{isis-2_second_international_study_of_infarct_survival_collaborative_group_randomised_1988,
Title = {{RANDOMISED} {TRIAL} {OF} {INTRAVENOUS} {STREPTOKINASE}, {ORAL} {ASPIRIN}, {BOTH}, {OR} {NEITHER} {AMONG} 17 187 {CASES} {OF} {SUSPECTED} {ACUTE} {MYOCARDIAL} {INFARCTION}: {ISIS}-2},
Author = {{ISIS-2 (SECOND INTERNATIONAL STUDY OF INFARCT SURVIVAL) COLLABORATIVE GROUP}},
Journal = {The Lancet},
Year = {1988},
+
Month = aug,
Number = {8607},
Pages = {349--360},
Volume = {332},
+
Abstract = {17 187 patients entering 417 hospitals up to 24 hours (median 5 hours) after the onset of suspected acute myocardial infarction were randomised, with placebo control, between: (i) a 1-hour intravenous infusion of 1·5 {MU} of streptokinase; (ii) one month of 160 mg/day enteric-coated aspirin; (iii) both active treatments; or (iv) neither. Streptokinase alone and aspirin alone each produced a highly significant reduction in 5-week vascular mortality: 791/8592 (9·2\%) among patients allocated streptokinase infusion vs 1029/8595 (12·0\%) among those allocated placebo infusion (odds reduction: 25\% {SD} 4; 2p\&lt;0·00001); 804/8587 (9·4\%) vascular deaths among patients allocated aspirin tablets vs 1016/8600 (11·8\%) among those allocated placebo tablets (odds reduction: 23\% {SD} 4; 2p \&lt;0·00001). The combination of streptokinase and aspirin was significantly (2p\&lt;0·0001) better than either agent alone. Their separate effects on vascular deaths appeared to be additive: 343/4292 (8·0\%) among patients allocated both active agents vs 568/4300 (13·2\%) among those allocated neither (odds reduction: 42\% {SD} 5; 95\% confidence limits 34-50\%). There was evidence of benefit from each agent even for patients treated late after pain onset (odds reductions at 0-4, 5-12, and 13-24 hours: 35\% {SD} 6, 16\% {SD} 7, and 21\% {SD} 12 for streptokinase alone; 25\% {SD} 7, 21\% {SD} 7, and 21\% {SD} 12 for aspirin alone; and 53\% {SD} 8, 32\% {SD} 9, and 38\% {SD} 15 for the combination of streptokinase and aspirin). Streptokinase was associated with an excess of bleeds requiring transfusion (0·5\% vs 0·2\%) and of confirmed cerebral haemorrhage (0·1\% vs 0·0\%), but with fewer other strokes (0·6\% vs 0·8\%). These "other" strokes may have included a few undiagnosed cerebral haemorrhages, but still there was no increase in total strokes (0·7\% streptokinase vs 0·8\% placebo infusion). Aspirin significantly reduced non-fatal reinfarction (1·0\% vs 2·0\%) and non-fatal stroke (0·3\% vs 0·6\%), and was not associated with any significant increase in cerebral haemorrhage or in bleeds requiring transfusion. An excess of non-fatal reinfarction was reported when streptokinase was used alone, but this appeared to be entirely avoided by the addition of aspirin. Those allocated the combination of streptokinase and aspirin had significantly fewer deaths (8·0\% vs 13·2\%) than those allocated neither. The differences in vascular and in all-cause mortality produced by streptokinase and by aspirin remain highly significant (2p\&lt;0·001 for each) after the median of 15 months of follow-up thus far available.},
Doi = {10.1016/S0140-6736(88)92833-4},
File = {ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NX7SNPHI\\S0140673688928334.html:text/html},
ISSN = {0140-6736},
Series = {Originally published as Volume 2, Issue 8607},
Shorttitle = {{RANDOMISED} {TRIAL} {OF} {INTRAVENOUS} {STREPTOKINASE}, {ORAL} {ASPIRIN}, {BOTH}, {OR} {NEITHER} {AMONG} 17 187 {CASES} {OF} {SUSPECTED} {ACUTE} {MYOCARDIAL} {INFARCTION}},
Url = {http://www.sciencedirect.com/science/article/pii/S0140673688928334},
Urldate = {2014-08-28}
}
+
@Article{johnr.lott_crime_1997,
Title = {Crime, Deterrence, and Right to Carry Concealed Handguns},
Author = {John R. Lott, Jr., By and Mustard, David B.},
Journal = {The Journal of Legal Studies},
Year = {1997},
+
Month = jan,
Number = {1},
Pages = {1--68},
Volume = {26},
+
Abstract = {Abstract Using cross�sectional time�series data for U.S. counties from 1977 to 1992, we find that allowing citizens to carry concealed weapons deters violent crimes, without increasing accidental deaths. If those states without right�to�carry concealed gun provisions had adopted them in 1992, county� and state�level data indicate that approximately 1,500 murders would have been avoided yearly. Similarly, we predict that rapes would have declined by over 4,000, robbery by over 11,000, and aggravated assaults by over 60,000. We also find criminals substituting into property crimes involving stealth, where the probability of contact between the criminal and the victim is minimal. Further, higher arrest and conviction rates consistently reduce crime. The estimated annual gain from all remaining states adopting these laws was at least \$5.74 billion in 1992. The annual social benefit from an additional concealed handgun permit is as high as \$5,000.},
Copyright = {Copyright © 1997 The University of Chicago},
Doi = {10.1086/467988},
ISSN = {0047-2530},
Url = {http://www.jstor.org/stable/10.1086/467988},
Urldate = {2014-08-21}
}
+
@Article{johnson_harvard_2012,
Title = {Harvard professor who resigned fabricated, manipulated data, {US} says - The Boston Globe},
Author = {Johnson, Carolyn Y.},
Journal = {{BostonGlobe}.com},
Year = {2012},
+
Month = sep,
+
Abstract = {Marc Hauser, a prolific scientist and popular psychology professor who resigned last summer from Harvard University, had fabricated data, manipulated results in multiple experiments, and incorrectly described how experiments were conducted, according to the findings of a federal research oversight agency posted online Wednesday.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\ZC9UU8TT\\story.html:text/html},
Keywords = {02138 (Magazine), Boston Globe, fraud, harvard, Harvard University, hauser, Health and Human Services Department, {LinkedIn}, {McMaster} University, monkeys, National Institutes of Health, New York Times, {NIH}, Office of Research Integrity, Public Health Service, research misconduct, Twitter},
Url = {https://www.bostonglobe.com/news/science/2012/09/05/harvard-professor-who-resigned-fabricated-manipulated-data-says/6gDVkzPNxv1ZDkh4wVnKhO/story.html},
Urldate = {2014-09-12}
}
+
@Article{joyce_did_2004,
Title = {Did Legalized Abortion Lower Crime?},
Author = {Joyce, Ted},
Journal = {The Journal of Human Resources},
Year = {2004},
+
Month = jan,
Number = {1},
Pages = {1--28},
Volume = {39},
+
Abstract = {In this paper I compare changes in homicide and arrest rates among cohorts born before and after the legalization of abortion to changes in crime in the same years among similar cohorts who were unexposed to legalized abortion. I find little consistent evidence that the legalization of abortion in selected states around 1970, and then in the remaining states following Roe v. Wade, had an effect on recent crime rates. I conclude that the dramatic association as reported in a recent study is most likely the result of unmeasured period effects such as changes in crack cocaine use.},
Copyright = {Copyright © 2004 The Board of Regents of the University of Wisconsin System},
Doi = {10.2307/3559003},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BHQI74T4\\Joyce - 2004 - Did Legalized Abortion Lower Crime.pdf:application/pdf},
ISSN = {0022-166X},
Url = {http://www.jstor.org/stable/3559003},
Urldate = {2014-08-21}
}
+
@Book{jr_more_2010,
Title = {More Guns, Less Crime: Understanding Crime and Gun Control Laws, Third Edition},
Author = {Jr, John R. Lott},
Publisher = {University Of Chicago Press},
Year = {2010},
+
Address = {Chicago},
Edition = {Third Edition edition},
Month = may,
+
Abstract = {John R. Lott, Jr., is the author five books, including Freedomnomics and Are Predatory Commitments Credible? Who Should the Courts Believe? , the latter also published by the University of Chicago Press.},
ISBN = {9780226493664},
Language = {English},
Shorttitle = {More Guns, Less Crime}
}
+
@Article{kane_why_1984,
Title = {{WHY} {JOURNAL} {EDITORS} {SHOULD} {ENCOURAGE} {THE} {REPLICATION} {OF} {APPLIED} {ECONOMETRIC} {RESEARCH}},
Author = {Kane, Edward J.},
Journal = {Quarterly Journal of Business and Economics},
Year = {1984},
+
Month = jan,
Number = {1},
Pages = {3--8},
Volume = {23},
+
Copyright = {Copyright © 1984 University of Nebraska-Lincoln College of Business Administration},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WWHAQMXS\\Kane - 1984 - WHY JOURNAL EDITORS SHOULD ENCOURAGE THE REPLICATI.pdf:application/pdf},
ISSN = {0747-5535},
Url = {http://www.jstor.org/stable/23526567},
Urldate = {2014-08-28}
}
+
@TechReport{katz_elizabeth_pre-analysis_2013,
Title = {Pre-Analysis Plan: {TOMS} Shoes Impact Study},
Author = {Katz, Elizabeth and Janet, Brendan and Wydick, Bruce and Gutierrez, Felipe},
Year = {2013},
Month = feb,
+
Url = {http://www.povertyactionlab.org/sites/default/files/documents/Pre-Analysis%20Plan_Wydick_2-12-13.pdf}
}
+
@Article{king_replication_1995,
Title = {Replication, Replication},
Author = {King, Gary},
Journal = {{PS}: Political Science \& Politics},
Year = {1995},
Number = {03},
Pages = {444--452},
Volume = {28},
+
Doi = {10.2307/420301},
File = {Cambridge Journals Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\PICSS99J\\displayAbstract.html:text/html}
}
+
@Book{kirchkamp_workflow_????,
Title = {Workflow of Statistical Data Analysis},
Author = {Kirchkamp, Oliver},
+
Url = {http://www.kirchkamp.de/oekonometrie/pdf/wf-screen2.pdf}
}
+
@Article{klein_reconsidering_2009,
Title = {Reconsidering the impact of national soccer results on the {FTSE} 100},
Author = {Klein, Christian and Zwergel, Bernhard and Henning Fock, J.},
Journal = {Applied Economics},
Year = {2009},
+
Month = nov,
Number = {25},
Pages = {3287--3294},
Volume = {41},
+
Abstract = {In past decades, many empirical studies revealed return anomalies in many different asset classes and markets. Very recent publications have, however, even found evidence that stock markets react to the results of soccer matches. In this article, we argue that such empirical studies should be analysed carefully; we thus endorse the use of replication studies to verify results. Consequently, by rebuilding the study of Ashton et al. (2003), we are able to detect mistakes in the empirical set-up. Based on these findings, we demonstrate how even minor flaws can have a crucial influence on the results of such studies and point out pitfalls that are frequently encountered. We furthermore emphasize the importance of robustness checks to validate the results of empirical studies.},
Doi = {10.1080/00036840802112471},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\NQ5RSRPD\\Klein et al. - 2009 - Reconsidering the impact of national soccer result.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SZW2TZ25\\00036840802112471.html:text/html},
ISSN = {0003-6846},
Url = {http://dx.doi.org/10.1080/00036840802112471},
Urldate = {2014-08-28}
}
+
@Article{many_labs,
Title = {Investigating Variation in Replicability},
Author = {Klein, Richard A and Ratliff, Kate A and Vianello, Michelangelo and Adams Jr, Reginald B and Bahn{\'\i}k, {\v{S}}t{\v{e}}p{\'a}n and Bernstein, Michael J and Bocian, Konrad and Brandt, Mark J and Brooks, Beach and Brumbaugh, Claudia Chloe and others},
Journal = {Social Psychology},
Year = {2014},
Number = {3},
Pages = {142--152},
Volume = {45},
+
Publisher = {Hogrefe \& Huber}
}
+
@Article{kling2007experimental,
Title = {Experimental analysis of neighborhood effects},
Author = {Kling, Jeffrey R and Liebman, Jeffrey B and Katz, Lawrence F},
Journal = {Econometrica},
Year = {2007},
Number = {1},
Pages = {83--119},
Volume = {75},
+
Publisher = {Wiley Online Library}
}
+
@Book{knuth_literate_1992,
Title = {Literate Programming},
Author = {Knuth, Donald Ervin},
Publisher = {Center for the Study of Language and Information},
Year = {1992},
Month = jan,
+
Abstract = {Literate programming is a programming methodology that combines a programming language with a documentation language, making programs more robust, more portable, and more easily maintained than programs written only in a high-level language. Computer programmers already know both kinds of languages; they need only learn a few conventions about alternating between languages to create programs that are works of literature. A literate programmer is an essayist who writes programs for humans to understand, instead of primarily writing instructions for machines to follow. When programs are written in the recommended style they can be transformed into documents by a document compiler and into efficient code by an algebraic compiler. This anthology of essays from the inventor of literate programming includes Knuth's early papers on related topics such as structured programming, as well as the Computer Journal article that launched literate programming itself.},
ISBN = {9780937073810},
Keywords = {Computers / Programming / General, Computers / Programming / Object Oriented},
Language = {en}
}
+
@Article{knuth_literate_1984,
Title = {Literate Programming},
Author = {Knuth, D. E.},
Journal = {The Computer Journal},
Year = {1984},
+
Month = jan,
Number = {2},
Pages = {97--111},
Volume = {27},
+
Abstract = {The author and his associates have been experimenting for the past several years with a programming language and documentation system called {WEB}. This paper presents {WEB} by example, and discusses why the new system appears to be an improvement over previous ones.},
Doi = {10.1093/comjnl/27.2.97},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\53H8RZZ3\\97.html:text/html},
ISSN = {0010-4620, 1460-2067},
Language = {en},
Url = {http://comjnl.oxfordjournals.org/content/27/2/97},
Urldate = {2014-09-25}
}
+
@Article{koenker_reproducible_2009,
Title = {On reproducible econometric research},
Author = {Koenker, Roger and Zeileis, Achim},
Journal = {Journal of Applied Econometrics},
Year = {2009},
+
Month = aug,
Number = {5},
Pages = {833--847},
Volume = {24},
+
Abstract = {Recent software developments are reviewed from the vantage point of reproducible econometric research. We argue that the emergence of new tools, particularly in the open-source community, have greatly eased the burden of documenting and archiving both empirical and simulation work in econometrics. Some of these tools are highlighted in the discussion of two small replication exercises. Copyright © 2009 John Wiley \& Sons, Ltd.},
Copyright = {Copyright © 2009 John Wiley \& Sons, Ltd.},
Doi = {10.1002/jae.1083},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\Q8JDP4K8\\Koenker and Zeileis - 2009 - On reproducible econometric research.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2TH4N2D2\\full.html:text/html},
ISSN = {1099-1255},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1002/jae.1083/abstract},
Urldate = {2014-08-20}
}
+
@Article{koole_rewarding_2012,
Title = {Rewarding Replications A Sure and Simple Way to Improve Psychological Science},
Author = {Koole, Sander L. and Lakens, Daniël},
Journal = {Perspectives on Psychological Science},
Year = {2012},
+
Month = nov,
Number = {6},
Pages = {608--614},
Volume = {7},
+
Abstract = {Although replications are vital to scientific progress, psychologists rarely engage in systematic replication efforts. In this article, we consider psychologists’ narrative approach to scientific publications as an underlying reason for this neglect and propose an incentive structure for replications within psychology. First, researchers need accessible outlets for publishing replications. To accomplish this, psychology journals could publish replication reports in files that are electronically linked to reports of the original research. Second, replications should get cited. This can be achieved by cociting replications along with original research reports. Third, replications should become a valued collaborative effort. This can be realized by incorporating replications in teaching programs and by stimulating adversarial collaborations. The proposed incentive structure for replications can be developed in a relatively simple and cost-effective manner. By promoting replications, this incentive structure may greatly enhance the dependability of psychology’s knowledge base.},
Doi = {10.1177/1745691612462586},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WXEQAJJP\\Koole and Lakens - 2012 - Rewarding Replications A Sure and Simple Way to Im.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\X672EUCA\\608.html:text/html},
ISSN = {1745-6916, 1745-6924},
Keywords = {philosophy of science, Publication bias, replication, scientific fraud, selective reporting},
Language = {en},
Url = {http://pps.sagepub.com/content/7/6/608},
Urldate = {2014-09-08}
}
+
@Article{kremer2009incentives,
Title = {Incentives to learn},
Author = {Kremer, Michael and Miguel, Edward and Thornton, Rebecca},
Journal = {The Review of Economics and Statistics},
Year = {2009},
Number = {3},
Pages = {437--456},
Volume = {91},
+
Publisher = {MIT Press}
}
+
@Article{laine_clinical_2007,
Title = {Clinical Trial Registration Looking Back and Moving Ahead},
Author = {Laine, Christine and Horton, Richard and DeAngelis, Catherine D. and Drazen, Jeffrey M. and Frizelle, Frank A. and Godlee, Fiona and Haug, Charlotte and Hébert, Paul C. and Kotzin, Sheldon and Marusic, Ana and Sahni, Peush and Schroeder, Torben V. and Sox, Harold C. and Weyden, Martin B. Van Der and Verheugt, Freek W.A.},
Journal = {New England Journal of Medicine},
Year = {2007},
+
Month = jun,
Number = {26},
Pages = {2734--2736},
Volume = {356},
+
Abstract = {In 2005, the International Committee of Medical Journal Editors ({ICMJE}) initiated a policy requiring investigators to deposit information about trial design into an accepted clinical trials registry before the onset of patient enrollment.1 This policy aimed to ensure that information about the existence and design of clinically directive trials was publicly available, an ideal that leaders in evidence-based medicine have advocated for decades.2 The policy precipitated much angst among research investigators and sponsors, who feared that registration would be burdensome and would stifle competition. Yet, the response to this policy has been overwhelming. The {ICMJE} promised to reevaluate the policy . . .},
Doi = {10.1056/NEJMe078110},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\D5BXWI59\\Laine et al. - 2007 - Clinical Trial Registration — Looking Back and Mov.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DUFG5DV3\\NEJMe078110.html:text/html},
ISSN = {0028-4793},
Pmid = {17548427},
Url = {http://www.nejm.org/doi/full/10.1056/NEJMe078110},
Urldate = {2014-08-28}
}
+
@Article{laitin_fisheries_2013,
Title = {Fisheries Management},
Author = {Laitin, David D.},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {42--47},
Volume = {21},
+
Doi = {10.1093/pan/mps033},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\7VZ62N93\\Laitin - 2013 - Fisheries Management.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\S8QFGJMS\\42.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Url = {http://pan.oxfordjournals.org/content/21/1/42},
Urldate = {2014-08-28}
}
+
@Article{laitin_fisheries_2013-1,
Title = {Fisheries Management},
Author = {Laitin, David D.},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {42--47},
Volume = {21},
+
Doi = {10.1093/pan/mps033},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\U7GPDX2C\\Laitin - 2013 - Fisheries Management.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\67M78Z9X\\42.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Url = {http://pan.oxfordjournals.org/content/21/1/42},
Urldate = {2014-08-28}
}
+
@Article{lalonde_evaluating_1986,
Title = {Evaluating the Econometric Evaluations of Training Programs with Experimental Data},
Author = {LaLonde, Robert J.},
Journal = {The American Economic Review},
Year = {1986},
+
Month = sep,
Number = {4},
Pages = {604--620},
Volume = {76},
+
Abstract = {This paper compares the effect on trainee earnings of an employment program that was run as a field experiment where participants were randomly assigned to treatment and control groups with the estimates that would have been produced by an econometrician. This comparison shows that many of the econometric procedures do not replicate the experimentally determined results, and it suggests that researchers should be aware of the potential for specification errors in other nonexperimental evaluations.},
Copyright = {Copyright © 1986 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6WQN3FIU\\LaLonde - 1986 - Evaluating the Econometric Evaluations of Training.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/1806062},
Urldate = {2014-09-08}
}
+
@Article{lash_preregistration_2010,
Title = {Preregistration of Study Protocols Is Unlikely to Improve the Yield From Our Science, But Other Strategies Might:},
Author = {Lash, Timothy L.},
Journal = {Epidemiology},
Year = {2010},
+
Month = sep,
Number = {5},
Pages = {612--613},
Volume = {21},
+
Doi = {10.1097/EDE.0b013e3181e9bba6},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {Preregistration of Study Protocols Is Unlikely to Improve the Yield From Our Science, But Other Strategies Might},
Url = {http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001648-201009000-00011},
Urldate = {2014-10-28}
}
+
@Article{leamer_lets_1983,
Title = {Let's Take the Con Out of Econometrics},
Author = {Leamer, Edward E.},
Journal = {The American Economic Review},
Year = {1983},
+
Month = mar,
Number = {1},
Pages = {31--43},
Volume = {73},
+
Copyright = {Copyright © 1983 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DK5VRK58\\Leamer - 1983 - Let's Take the Con Out of Econometrics.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/1803924},
Urldate = {2014-09-08}
}
+
@Article{leimer_social_1982,
Title = {Social Security and Private Saving: New Time-Series Evidence},
Author = {Leimer, Dean R. and Lesnoy, Selig D.},
Journal = {Journal of Political Economy},
Year = {1982},
+
Month = jun,
Number = {3},
Pages = {606--629},
Volume = {90},
+
Copyright = {Copyright © 1982 The University of Chicago Press},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KSMSTFMK\\Leimer and Lesnoy - 1982 - Social Security and Private Saving New Time-Serie.pdf:application/pdf},
ISSN = {0022-3808},
Shorttitle = {Social Security and Private Saving},
Url = {http://www.jstor.org/stable/1831373},
Urldate = {2014-09-23}
}
+
@Article{Levine2001editorial,
Title = {Editor's Introduction to “The Unemployment Effects of Minimum Wages: Evidence from a Prespecified Research Design”},
Author = {Levine, David I.},
Journal = {Industrial Relations: A Journal of Economy and Society},
Year = {2001},
Number = {2},
Pages = {161--162},
Volume = {40},
+
Doi = {10.1111/0019-8676.00204},
ISSN = {1468-232X},
Publisher = {Blackwell Publishers Inc.},
Url = {http://dx.doi.org/10.1111/0019-8676.00204}
}
+
@Article{levitt_using_2002,
Title = {Using Electoral Cycles in Police Hiring to Estimate the Effects of Police on Crime: Reply},
Author = {Levitt, Steven D.},
Journal = {The American Economic Review},
Year = {2002},
+
Month = sep,
Number = {4},
Pages = {1244--1250},
Volume = {92},
+
Copyright = {Copyright © 2002 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SJP8QQBP\\Levitt - 2002 - Using Electoral Cycles in Police Hiring to Estimat.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Using Electoral Cycles in Police Hiring to Estimate the Effects of Police on Crime},
Url = {http://www.jstor.org/stable/3083312},
Urldate = {2014-08-20}
}
+
@Article{levitt_using_1997,
Title = {Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime},
Author = {Levitt, Steven D.},
Journal = {The American Economic Review},
Year = {1997},
+
Month = jun,
Number = {3},
Pages = {270--290},
Volume = {87},
+
Abstract = {Previous empirical studies have uncovered little evidence that police reduce crime, possibly due to simultaneity problems. This paper uses the timing of mayoral and gubernatorial elections as an instrumental variable to identify a causal effect of police on crime. Increases in the size of police forces are shown to be disproportionately concentrated in mayoral and gubernatorial election years. Increases in police are shown to substantially reduce violent crime, but have a smaller impact on property crime. The null hypothesis that the marginal social benefit of reduced crime equals the costs of hiring additional police cannot be rejected.},
Copyright = {Copyright © 1997 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\J7GXNZH9\\Levitt - 1997 - Using Electoral Cycles in Police Hiring to Estimat.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/2951346},
Urldate = {2014-08-20}
}
+
@Article{lin_can_2008,
Title = {Can Hepatitis B Mothers Account for the Number of Missing Women? Evidence from Three Million Newborns in Taiwan},
Author = {Lin, Ming-Jen and Luoh, Ming-Ching},
Journal = {The American Economic Review},
Year = {2008},
+
Month = dec,
Number = {5},
Pages = {2259--2273},
Volume = {98},
+
Copyright = {Copyright © 2008 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EEWVARXU\\Lin and Luoh - 2008 - Can Hepatitis B Mothers Account for the Number of .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Can Hepatitis B Mothers Account for the Number of Missing Women?},
Url = {http://www.jstor.org/stable/29730171},
Urldate = {2014-08-20}
}
+
@Article{loder_registration_2010,
Title = {Registration of observational studies: The next step towards research transparency},
Author = {Loder, E. and Groves, T. and MacAuley, D.},
Journal = {{BMJ}},
Year = {2010},
+
Month = feb,
Number = {feb18 2},
Pages = {c950--c950},
Volume = {340},
+
Doi = {10.1136/bmj.c950},
ISSN = {0959-8138, 1468-5833},
Language = {en},
Url = {http://www.bmj.com/cgi/doi/10.1136/bmj.c950},
Urldate = {2014-10-28}
}
+
@Book{long_workflow_2008,
Title = {The Workflow of Data Analysis Using Stata},
Author = {Long, J. Scott},
Publisher = {Stata Press},
Year = {2008},
Month = dec,
+
Abstract = {The Workflow of Data Analysis Using Stata, by J. Scott Long, is an essential productivity tool for data analysts. Long presents lessons gained from his experience and demonstrates how to design and implement efficient workflows for both one-person projects and team projects. After introducing workflows and explaining how a better workflow can make it easier to work with data, Long describes planning, organizing, and documenting your work. He then introduces how to write and debug Stata do-files and how to use local and global macros. After a discussion of conventions that greatly simplify data analysis the author covers cleaning, analyzing, and protecting data.},
ISBN = {9781597180474},
Keywords = {Computers / Mathematical \& Statistical Software, Mathematics / Probability \& Statistics / General},
Language = {en}
}
+
@Article{loomis_journal_2011,
Title = {Journal Requirements to Register Observational Studies: {OEM}'s Policy},
Author = {Loomis, Dana},
Journal = {Occupational and Environmental Medicine},
Year = {2011},
+
Month = feb,
Number = {2},
Pages = {83--84},
Volume = {68},
+
Doi = {10.1136/oem.2010.064477},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DDGUADAD\\Loomis - 2011 - Journal Requirements to Register Observational Stu.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HCCBMA8X\\83.html:text/html},
ISSN = {, 1470-7926},
Keywords = {Epidemiology, health services research, hygiene / occupational hygiene, occupational health practice, toxicology},
Language = {en},
Pmid = {21242278},
Shorttitle = {Journal Requirements to Register Observational Studies},
Url = {http://oem.bmj.com/content/68/2/83},
Urldate = {2014-10-28}
}
+
@Article{lott_abortion_2007,
Title = {Abortion and Crime: Unwanted Children and Out-of-Wedlock Births},
Author = {Lott, John R. and Whitley, John},
Journal = {Economic Inquiry},
Year = {2007},
+
Month = apr,
Number = {2},
Pages = {304--324},
Volume = {45},
+
Abstract = {Legalizing abortion can either increase or decrease investments in children’s human capital. This article finds that abortion increases the number of out-of-wedlock births. Using data that more directly links the criminal with age when the crime was committed, not age when arrested, and fixing the assumption in previous research that no abortions took place prior to the Roe v. Wade decision in the 45 states affected by that decision, we find consistent significant evidence that legalizing abortions increased murders by over 7\%. Linear estimates indicate that legalization increased total annual victimization costs by at least \$3.2 billion. ({JEL} K42, K14, J24)},
Doi = {10.1111/j.1465-7295.2006.00040.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6QJ76BZ7\\Lott and Whitley - 2007 - Abortion and Crime Unwanted Children and Out-of-W.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\MJXAZWTD\\full.html:text/html},
ISSN = {1465-7295},
Language = {en},
Shorttitle = {Abortion and Crime},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1465-7295.2006.00040.x/abstract},
Urldate = {2014-08-21}
}
+
@Article{marwick_how,
Title = {How computers broke science – and what we can do to fix it},
Author = {Marwick, Ben},
Journal = {The Conversation},
Year = {2015},
+
Month = {November},
+
Abstract = {Virtually every researcher relies on computers to collect or analyze data. But when computers are opaque black boxes that manipulate data, it's impossible to replicate studies – a core value for science.},
Day = {9},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\RVZCJW3H\\how-computers-broke-science-and-what-we-can-do-to-fix-it-49938.html:text/html},
Url = {http://theconversation.com/how-computers-broke-science-and-what-we-can-do-to-fix-it-49938},
Urldate = {2016-02-02}
}
+
@Article{mathieu_s_comparison_2009,
Title = {{COmparison} of registered and published primary outcomes in randomized controlled trials},
Author = {{Mathieu S} and {Boutron I} and {Moher D} and {Altman DG} and {Ravaud P}},
Journal = {{JAMA}},
Year = {2009},
+
Month = sep,
Number = {9},
Pages = {977--984},
Volume = {302},
+
Abstract = {Context As of 2005, the International Committee of Medical Journal Editors required investigators to register their trials prior to participant enrollment as a precondition for publishing the trial's findings in member journals.Objective To assess the proportion of registered trials with results recently published in journals with high impact factors; to compare the primary outcomes specified in trial registries with those reported in the published articles; and to determine whether primary outcome reporting bias favored significant outcomes.Data Sources and Study Selection {MEDLINE} via {PubMed} was searched for reports of randomized controlled trials ({RCTs}) in 3 medical areas (cardiology, rheumatology, and gastroenterology) indexed in 2008 in the 10 general medical journals and specialty journals with the highest impact factors.Data Extraction For each included article, we obtained the trial registration information using a standardized data extraction form.Results Of the 323 included trials, 147 (45.5\%) were adequately registered (ie, registered before the end of the trial, with the primary outcome clearly specified). Trial registration was lacking for 89 published reports (27.6\%), 45 trials (13.9\%) were registered after the completion of the study, 39 (12\%) were registered with no or an unclear description of the primary outcome, and 3 (0.9\%) were registered after the completion of the study and had an unclear description of the primary outcome. Among articles with trials adequately registered, 31\% (46 of 147) showed some evidence of discrepancies between the outcomes registered and the outcomes published. The influence of these discrepancies could be assessed in only half of them and in these statistically significant results were favored in 82.6\% (19 of 23).Conclusion Comparison of the primary outcomes of {RCTs} registered with their subsequent publication indicated that selective outcome reporting is prevalent.},
Doi = {10.1001/jama.2009.1242},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HH9WTG2H\\Mathieu S et al. - 2009 - COmparison of registered and published primary out.pdf:application/pdf},
ISSN = {0098-7484},
Url = {http://dx.doi.org/10.1001/jama.2009.1242},
Urldate = {2014-10-09}
}
+
@Article{mccrary_using_2002,
Title = {Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime: Comment},
Author = {McCrary, Justin},
Journal = {The American Economic Review},
Year = {2002},
+
Month = sep,
Number = {4},
Pages = {1236--1243},
Volume = {92},
+
Copyright = {Copyright © 2002 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8PXGEEEX\\McCrary - 2002 - Using Electoral Cycles in Police Hiring to Estimat.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime},
Url = {http://www.jstor.org/stable/3083311},
Urldate = {2014-08-20}
}
+
@Article{mccullough_economics_2008,
Title = {Do economics journal archives promote replicable research?},
Author = {McCullough, B.d. and McGeary, Kerry Anne and Harrison, Teresa D.},
Journal = {Canadian Journal of Economics/Revue canadienn economique},
Year = {2008},
+
Month = nov,
Number = {4},
Pages = {1406--1420},
Volume = {41},
+
Abstract = {Abstract.  All the long-standing archives at economics journals do not facilitate the reproduction of published results. The data-only archives at Journal of Business and Economic Statistics and Economic Journal fail in part because most authors do not contribute data. Results published in the {FRB} St. Louis Review can rarely be reproduced using the data+code in the journal archive. Recently created archives at top journals should avoid the mistakes of their predecessors. We categorize reasons for archives' failures and identify successful policies.},
Copyright = {© Canadian Economics Association},
Doi = {10.1111/j.1540-5982.2008.00509.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JUWCUUJN\\McCullough et al. - 2008 - Do economics journal archives promote replicable r.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6V8H3493\\full.html:text/html},
ISSN = {1540-5982},
Keywords = {B40, C80},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1540-5982.2008.00509.x/abstract},
Urldate = {2014-08-20}
}
+
@Misc{mccullough_open_2009,
Title = {Open Access Economics Journals and the Market for Reproducible Economic Research},
+
Author = {McCullough, B. D.},
Month = mar,
Year = {2009},
+
Abstract = {Most economics journals take no substantive measures to ensure that the results they publish are replicable. To make the data and code available so that published results can be checked requires an archive. Top economics journals have been adopting mandatory data+code archives in the past few years. The movement toward mandatory data+code archives has yet to reach the open access journals. This is paradoxical; given their emphasis on making articles readily available, one would think that open access journals also would want to make data and code readily available. Open access economics journals should adopt mandatory data+code archives en masse. Doing so will give them a competitive advantage with respect to traditional economics journals.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FG55T8WN\\documentSummary\;dn=775889276720076\;res=IELBUS.html:text/html},
Keywords = {abstract and citation, Architecture Building and Design, Arts and Entertainment, Asian Resources, bibliographic databases, books, Business and Management, {CD} {ROMs}, Communications, Crime and Justice, Directory and Reference Resources, Early Childhood Resources, Education and Research, Engineering and Technology, Environment and Natural Resources, e-resources, e-titles, full text databases, genealogy, Genealogy (Family History), Health (Medicine), History and Heritage, Hospitality and Tourism, Indigenous Peoples, Information Management and Technology, journals, Law, Literature, monographs, Newspapers, New Zealand Resources, online resources, papers, print publishing, Public Affairs, Public Relations, reports, Science and Technology, Secondary School Resources, Social Sciences and Community Issues, Sports, Trade Skills},
Url = {http://search.informit.com.au/documentSummary;dn=775889276720076;res=IELBUS},
Urldate = {2014-08-28}
}
+
@Article{mccullough_got_2007,
Title = {Got Replicability? The Journal of Money, Credit and Banking Archive},
Author = {McCullough, B. D.},
Journal = {Econ Journal Watch},
Year = {2007},
+
Month = sep,
Number = {3},
Pages = {326--337},
Volume = {4},
+
Abstract = {In a paper published in the Journal of Money, Credit and Banking, {McGeary}, Harrison, and I showed that the {JMCB}’s data+code archive of generally did not support the replication of the journal’s published results. We recommended several procedures for ensuring that the archived data and code would reproduce the published results. The {JMCB} Editors recently adopted a few new procedures, ignoring most of the recommendations. This paper checks to see whether the new procedures are working. They are not.},
Url = {http://econjwatch.org/articles/got-replicability-the-journal-of-money-credit-and-banking-archive?ref=section-archive}
}
+
@Article{mccullough_lessons_2006,
Title = {Lessons from the {JMCB} Archive},
Author = {McCullough, B. D. and McGeary, Kerry Anne and Harrison, Teresa D.},
Journal = {Journal of Money, Credit, and Banking},
Year = {2006},
Note = {{\textless}p{\textgreater}Volume 38, Number 4, June 2006{\textless}/p{\textgreater}},
Number = {4},
Pages = {1093--1107},
Volume = {38},
+
Abstract = {Abstract We examine the online archive of the Journal of Money, Credit, and Banking, in which an author is required to deposit the data and code that replicate the results of his paper. We find that most authors do not fulfill this requirement. Of more than 150 empirical articles, fewer than 15 could be replicated. Despite all this, there is no doubt that a data/code archive is more conducive to replicable research than the alternatives. We make recommendations to improve the functioning of the archive.},
Doi = {10.1353/mcb.2006.0061},
File = {Project MUSE Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\P785GVJX\\McCullough et al. - 2006 - Lessons from the JMCB Archive.pdf:application/pdf;Project MUSE Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BEB9MB9B\\38.4mccullough.html:text/html},
ISSN = {1538-4616},
Url = {http://muse.jhu.edu/journals/journal_of_money_credit_and_banking/v038/38.4mccullough.pdf},
Urldate = {2014-08-20}
}
+
@Article{mccullough_verifying_2003,
Title = {Verifying the Solution from a Nonlinear Solver: A Case Study},
Author = {McCullough, B. D. and Vinod, H. D.},
Journal = {The American Economic Review},
Year = {2003},
+
Month = jun,
Number = {3},
Pages = {873--892},
Volume = {93},
+
Copyright = {Copyright © 2003 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8NB4E2JQ\\McCullough and Vinod - 2003 - Verifying the Solution from a Nonlinear Solver A .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Verifying the Solution from a Nonlinear Solver},
Url = {http://www.jstor.org/stable/3132121},
Urldate = {2014-08-20}
}
+
@Article{mckenzie2012caseformoreT,
Title = {Beyond baseline and follow-up: The case for more T in experiments},
Author = {McKenzie, David},
Journal = {Journal of Development Economics},
Year = {2012},
Number = {2},
Pages = {210--221},
Volume = {99},
+
Publisher = {Elsevier}
}
+
@Article{mcshane_2014,
Title = {You Cannot Step Into the Same River Twice When Power Analyses Are Optimistic},
Author = {McShane, Blakeley B and B{\"o}ckenholt, Ulf},
Journal = {Perspectives on Psychological Science},
Year = {2014},
Number = {6},
Pages = {612--625},
Volume = {9},
+
Publisher = {SAGE Publications}
}
+
@Book{merton1973sociology,
Title = {The sociology of science: Theoretical and empirical investigations},
Author = {Merton, Robert K},
Publisher = {University of Chicago press},
Year = {1973}
}
+
@Article{miguel_promoting_2014,
Title = {Promoting Transparency in Social Science Research},
Author = {Miguel, E. and Camerer, C. and Casey, K. and Cohen, J. and Esterling, K. M. and Gerber, A. and Glennerster, R. and Green, D. P. and Humphreys, M. and Imbens, G. and Laitin, D. and Madon, T. and Nelson, L. and Nosek, B. A. and Petersen, M. and Sedlmayr, R. and Simmons, J. P. and Simonsohn, U. and Laan, M. Van der},
Journal = {Science},
Year = {2014},
+
Month = jan,
Number = {6166},
Pages = {30--31},
Volume = {343},
+
Doi = {10.1126/science.1245317},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\QXKPAMAF\\Miguel et al. - 2014 - Promoting Transparency in Social Science Research.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WJ3W282B\\30.html:text/html},
ISSN = {0036-8075, 1095-9203},
Language = {en},
Pmid = {24385620},
Url = {http://www.sciencemag.org/content/343/6166/30},
Urldate = {2014-08-28}
}
+
@Article{miguel_re-examining_2011,
Title = {Re-examining Economic Shocks and Civil Conflict},
Author = {Miguel, Edward and Satyanath, Shanker},
Journal = {American Economic Journal: Applied Economics},
Year = {2011},
+
Month = oct,
Number = {4},
Pages = {228--232},
Volume = {3},
+
Abstract = {Miguel, Satyanath, and Ernest Sergenti (2004), henceforth {MSS}, show that economic growth is negatively related to civil conflict in Africa, using annual rainfall variation as an {IV} for growth. Antonio Ciccone (2011) argues that thanks to rainfall's mean-reverting nature, rainfall levels are preferable to annual changes. We make three points. First, {MSS}'s findings hold using rainfall levels as instruments. Second, Ciccone (2011) does not provide theoretical justification for preferring rainfall levels. Third, the first-stage relationship between rainfall and growth is weaker after 2000, suggesting that alternative instruments are needed when studying recent conflicts. We highlight the accumulating microeconomic evidence that adverse economic shocks lead to political violence.},
Doi = {10.1257/app.3.4.228}
}
+
@Article{miguel_economic_2004,
Title = {Economic Shocks and Civil Conflict: An Instrumental Variables Approach},
Author = {Miguel, Edward and Satyanath, Shanker and Sergenti, Ernest},
Journal = {Journal of Political Economy},
Year = {2004},
+
Month = aug,
Number = {4},
Pages = {725--753},
Volume = {112},
+
Abstract = {Estimating the impact of economic conditions on the likelihood of civil conflict is difficult because of endogeneity and omitted variable bias. We use rainfall variation as an instrumental variable for economic growth in 41 African countries during 1981–99. Growth is strongly negatively related to civil conflict: a negative growth shock of five percentage points increases the likelihood of conflict by one�half the following year. We attempt to rule out other channels through which rainfall may affect conflict. Surprisingly, the impact of growth shocks on conflict is not significantly different in richer, more democratic, or more ethnically diverse countries.},
Copyright = {Copyright © 2004 The University of Chicago Press},
Doi = {10.1086/jpe.2004.112.issue-4},
ISSN = {0022-3808},
Shorttitle = {Economic Shocks and Civil Conflict},
Url = {http://www.jstor.org/stable/10.1086/421174},
Urldate = {2014-08-21}
}
+
@Article{mittelstaedt_econometric_1984,
Title = {{ECONOMETRIC} {REPLICATION}: {LESSONS} {FROM} {THE} {EXPERIMENTAL} {SCIENCES}},
Author = {Mittelstaedt, Robert A. and Zorn, Thomas S.},
Journal = {Quarterly Journal of Business and Economics},
Year = {1984},
+
Month = jan,
Number = {1},
Pages = {9--15},
Volume = {23},
+
Copyright = {Copyright © 1984 University of Nebraska-Lincoln College of Business Administration},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\PFHSVF37\\Mittelstaedt and Zorn - 1984 - ECONOMETRIC REPLICATION LESSONS FROM THE EXPERIME.pdf:application/pdf},
ISSN = {0747-5535},
Shorttitle = {{ECONOMETRIC} {REPLICATION}},
Url = {http://www.jstor.org/stable/23526568},
Urldate = {2014-08-28}
}
+
@Article{moher_d_use_2001,
Title = {Use of the consort statement and quality of reports of randomized trials: A comparative before-and-after evaluation},
Author = {{Moher D} and {Jones A} and {Lepage L} and {for the CONSORT Group}},
Journal = {{JAMA}},
Year = {2001},
+
Month = apr,
Number = {15},
Pages = {1992--1995},
Volume = {285},
+
Abstract = {Context The Consolidated Standards for Reporting of Trials ({CONSORT}) statement
was developed to help improve the quality of reports of randomized controlled
trials ({RCTs}). To date, a paucity of data exists regarding whether it has
achieved this goal.Objective To determine whether use of the {CONSORT} statement is associated with
improvement in the quality of reports of {RCTs}.Design and Setting Comparative before-and-after evaluation in which reports of {RCTs} published
in 1994 (pre-{CONSORT}) were compared with {RCT} reports from the same journals
published in 1998 (post-{CONSORT}). We included 211 reports from {BMJ}, {JAMA}, and The Lancet (journals that
adopted {CONSORT}) as well as The New England Journal of Medicine (a journal that did not adopt {CONSORT} and was used as a comparator).Main Outcome Measures Number of {CONSORT} items included in a report, frequency of unclear reporting
of allocation concealment, and overall trial quality score based on the Jadad
scale, a 5-point quality assessment instrument.Results Compared with 1994, the number of {CONSORT} checklist items in reports
of {RCTs} increased in all 4 journals in 1998, and this increase was statistically
significant for the 3 adopter journals (pre-{CONSORT}, 23.4; mean change, 3.7;
95\% confidence interval [{CI}], 2.1-5.3). The frequency of unclear reporting
of allocation concealment decreased for each of the 4 journals, and this change
was statistically significant for adopters (pre-{CONSORT}, 61\%; mean change, −22\%;
95\% {CI}, −38\% to −6\%). Similarly, 3 of the 4 journals showed an
improvement in the quality score for reports of {RCTs}, and this increase was
statistically significant for adopter journals overall (pre-{CONSORT}, 2.7;
mean change, 0.4; 95\% {CI}, 0.1-0.8).Conclusion Use of the {CONSORT} statement is associated with improvements in the
quality of reports of {RCTs}.},
Doi = {10.1001/jama.285.15.1992},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\94CAFTPU\\Moher D et al. - 2001 - Use of the consort statement and quality of report.pdf:application/pdf},
ISSN = {0098-7484},
Shorttitle = {Use of the consort statement and quality of reports of randomized trials},
Url = {http://dx.doi.org/10.1001/jama.285.15.1992},
Urldate = {2014-08-29}
}
+
@Article{moher_consort_2001,
Title = {The {CONSORT} statement: revised recommendations for improving the quality of reports of parallel group randomized trials},
Author = {Moher, David and Schulz, Kenneth F. and Altman, Douglas G.},
Journal = {{BMC} Medical Research Methodology},
Year = {2001},
+
Month = apr,
Number = {1},
Pages = {2},
Volume = {1},
+
Abstract = {To comprehend the results of a randomized controlled trial ({RCT}), readers must understand its design, conduct, analysis and interpretation. That goal can only be achieved through complete transparency from authors. Despite several decades of educational efforts, the reporting of {RCTs} needs improvement. Investigators and editors developed the original {CONSORT} (Consolidated Standards of Reporting Trials) statement to help authors improve reporting by using a checklist and flow diagram. The revised {CONSORT} statement presented in this paper incorporates new evidence and addresses some criticisms of the original statement.
{PMID}: 11336663},
Copyright = {2001 Moher et al; licensee {BioMed} Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original {URL}.},
Doi = {10.1186/1471-2288-1-2},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\JF4V4KKT\\Moher et al. - 2001 - The CONSORT statement revised recommendations for.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\2HHPQ9N4\\2.html:text/html},
ISSN = {1471-2288},
Language = {en},
Pmid = {11336663},
Shorttitle = {The {CONSORT} statement},
Url = {http://www.biomedcentral.com/1471-2288/1/2/abstract},
Urldate = {2014-08-29}
}
+
@Article{monogan_case_2013,
Title = {A Case for Registering Studies of Political Outcomes: An Application in the 2010 House Elections},
Author = {Monogan, James E.},
Journal = {Political Analysis},
Year = {2013},
+
Month = jan,
Number = {1},
Pages = {21--37},
Volume = {21},
+
Abstract = {This article makes the case for the systematic registration of political studies. By proposing a research design before an outcome variable is observed, a researcher commits him- or herself to a theoretically motivated method for studying the object of interest. Further, study registration prompts peers of the discipline to evaluate a study’s quality on its own merits, reducing norms to accept significant results and reject null findings. To advance this idea, the Political Science Registered Studies Dataverse (http://dvn.iq.harvard.edu/dvn/dv/registration) has been created, in which scholars may create a permanent record of a research design before completing a study. This article also illustrates the method of registration through a study of the impact of the immigration issue in the 2010 election for the U.S. House of Representatives. Prior to the election, a design for this study was posted on the Society for Political Methodology website (http://polmeth.wustl.edu/{mediaDetail}.php?{docId}=1258). After the votes were counted, the study was completed in accord with the design. The treatment effect in this theoretically specified design was indiscernible, but a specification search could yield a significant result. Hence, this article illustrates the argument for study registration through a case in which the result could easily be manipulated.},
Doi = {10.1093/pan/mps022},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6X4Z7VZ3\\Monogan - 2013 - A Case for Registering Studies of Political Outcom.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WAR3Z6N2\\21.html:text/html},
ISSN = {1047-1987, 1476-4989},
Language = {en},
Shorttitle = {A Case for Registering Studies of Political Outcomes},
Url = {http://pan.oxfordjournals.org/content/21/1/21},
Urldate = {2014-08-27}
}
+
@Article{montgomery2013protocol,
Title = {Protocol for CONSORT-SPI: an extension for social and psychological interventions},
Author = {Montgomery, Paul and Grant, Sean and Hopewell, Sally and Macdonald, Geraldine and Moher, David and Michie, Susan and Mayo-Wilson, Evan},
Journal = {Implement Sci},
Year = {2013},
Number = {1},
Pages = {99},
Volume = {8}
}
+
@TechReport{mueller-langer_open_2014,
Title = {Open Access to Research Data: Strategic Delay and the Ambiguous Welfare Effects of Mandatory Data Disclosure},
Author = {Mueller-Langer, Frank and Versbach, Andreoli and Patrick},
Institution = {Social Science Research Network},
Year = {2014},
+
Address = {Rochester, {NY}},
Month = jun,
Number = {{ID} 2458362},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {Mandatory data disclosure is an essential feature for credible empirical work but comes at a cost: First, authors might invest less in data generation if they are not the full residual claimants of their data after their first publication. Second, authors might "strategically delay" the time of submission of papers in order to fully exploit their data in subsequent research. We analyze a three-stage model of publication and data disclosure. We derive exact conditions for positive welfare effects of mandatory data disclosure. However, we find that the transition to mandatory data disclosure has negative welfare properties if authors delay strategically.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\GK7J4693\\Papers.html:text/html},
Keywords = {data disclosure policy, strategic delay, welfare effects},
Shorttitle = {Open Access to Research Data},
Url = {http://papers.ssrn.com/abstract=2458362},
Urldate = {2014-08-29}
}
+
@Book{national_research_council_firearms_2004,
Title = {Firearms and Violence: A Critical Review},
Author = {{National Research Council}},
Publisher = {The National Academies Press},
Year = {2004},
+
Address = {Washington {DC}},
+
Abstract = {The Committee to Improve Research Information and Data on Firearms, which produced this book, was charged with providing an assessment of the strengths and limitations of the existing research and data on gun violence. One theme evident throughout the report is the relative absence of credible data that are central to answering the most basic questions about the link between firearms and violence, such as the effects of firearms on violence and the impacts of various violence-control policies. The inadequacy of data on gun ownership and use is among the most critical barriers to better understanding gun violence. The committee recommends a research effort to determine whether or not these kinds of data can be accurately collected with minimal risk to legitimate privacy concerns. Further, the committee recommends that for research purposes appropriate access be given to data maintained by regulatory and law enforcement agencies, including the trace data maintained by the Bureau of Alcohol, Tobacco, and Firearms; registration data maintained by the Federal Bureau of Investigation and State agencies; and firearms manufacturing and sales data. Further, there are difficult methodological issues regarding how various datasets might be used to address the complex causal issues of interest. The committee recommends that a methodological research program be established to address these problems. Regarding the focus of research on firearms and violence, the committee recommends further individual-level studies of the link between firearms and both lethal and nonlethal suicidal behavior. Other research recommendations pertain to deterrence and defense and interventions to reduce violence and suicide. Chapter references and tables and a subject index},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\5GX8AFGP\\abstract.html:text/html},
Url = {http://www.nap.edu/catalog.php?record_id=10881},
Urldate = {2014-08-21}
}
+
@Article{neumark_employment_2001,
Title = {The Employment Effects of Minimum Wages: Evidence from a Prespecified Research Design},
Author = {Neumark, David},
Journal = {Industrial Relations: A Journal of Economy and Society},
Year = {2001},
+
Month = jan,
Number = {1},
Pages = {121--144},
Volume = {40},
+
Abstract = {This article presentsevidence on the employment effects of recent minimum wage increases from a prespecified research design that entailed committing to a detailed set of statistical analyses prior to “going to� the data. The limited data to which the prespecified research design can be applied may preclude finding many significant effects. Nonetheless, the evidence is most consistent with disemployment effects of minimum wages for younger, less-skilled workers.},
Copyright = {2001 The Regents ofthe University of California},
Doi = {10.1111/0019-8676.00199},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\IQIM6D83\\abstract\;jsessionid=6B03069B6744E8116003AD9F7EE2C4E7.html:text/html},
ISSN = {1468-232X},
Language = {en},
Shorttitle = {The Employment Effects of Minimum Wages},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/0019-8676.00199/abstract},
Urldate = {2014-08-26}
}
+
@Article{Neumark_2014_bathwater,
Title = {Revisiting the Minimum Wage—Employment Debate: Throwing Out the Baby with the Bathwater?},
Author = {Neumark, David AND JM Ian Salas AND William Wascher},
Journal = {Industrial \& Labor Relations Review},
Year = {2014},
Number = {3},
Pages = {608-648},
Volume = {67},
+
Owner = {garret},
Timestamp = {2015.07.03}
}
+
@Article{neumark_minimum_2000,
Title = {Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Comment},
Author = {Neumark, David and Wascher, William},
Journal = {The American Economic Review},
Year = {2000},
+
Month = dec,
Number = {5},
Pages = {1362--1396},
Volume = {90},
+
Copyright = {Copyright © 2000 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\AIESKN8D\\Neumark and Wascher - 2000 - Minimum Wages and Employment A Case Study of the .pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Minimum Wages and Employment},
Url = {http://www.jstor.org/stable/2677855},
Urldate = {2014-08-20}
}
+
@Article{nosek2014registered,
Title = {Registered Reports},
Author = {Nosek, Brian A and Lakens, Dani{\"e}l},
Journal = {Social Psychology},
Year = {2014},
Number = {3},
Pages = {137--141},
Volume = {45},
+
Publisher = {Hogrefe \& Huber}
}
+
@Article{nosek_scientific_2012,
Title = {Scientific Utopia {II}. Restructuring Incentives and Practices to Promote Truth Over Publishability},
Author = {Nosek, Brian A. and Spies, Jeffrey R. and Motyl, Matt},
Journal = {Perspectives on Psychological Science},
Year = {2012},
+
Month = nov,
Number = {6},
Pages = {615--631},
Volume = {7},
+
Abstract = {An academic scientist’s professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, disciplinary incentives encourage design, analysis, and reporting decisions that elicit positive results and ignore negative results. Prior reports demonstrate how these incentives inflate the rate of false effects in published science. When incentives favor novelty over replication, false results persist in the literature unchallenged, reducing efficiency in knowledge accumulation. Previous suggestions to address this problem are unlikely to be effective. For example, a journal of negative results publishes otherwise unpublishable reports. This enshrines the low status of the journal and its content. The persistence of false findings can be meliorated with strategies that make the fundamental but abstract accuracy motive—getting it right—competitive with the more tangible and concrete incentive—getting it published. This article develops strategies for improving scientific practices and knowledge accumulation that account for ordinary human motivations and biases.},
Doi = {10.1177/1745691612459058},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\33HKNQZV\\Nosek et al. - 2012 - Scientific Utopia II. Restructuring Incentives and.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\GUZZU54I\\615.html:text/html},
ISSN = {1745-6916, 1745-6924},
Keywords = {false positives, incentives, methodology, motivated reasoning, replication},
Language = {en},
Url = {http://pps.sagepub.com/content/7/6/615},
Urldate = {2014-08-11}
}
+
@Article{obrien1984procedures,
Title = {Procedures for comparing samples with multiple endpoints},
Author = {O{'}Brien, Peter C},
Journal = {Biometrics},
Year = {1984},
Pages = {1079--1087},
+
Publisher = {JSTOR}
}
+
@Article{OlkenPAP,
Title = {Promises and Perils of Pre-Analysis Plans},
Author = {Olken, Benjamin A},
Journal = {The Journal of Economic Perspectives},
Year = {2015},
Number = {3},
Pages = {61--80},
Volume = {29},
+
Publisher = {American Economic Association}
}
+
@TechReport{olken_targeting_2009,
Title = {Targeting Analysis Protocol},
Author = {Olken, Benjamin A.},
Year = {2009},
Month = mar,
+
Url = {https://www.povertyactionlab.org/sites/default/files/documents/090318a%20Targeting%20Analysis%20Protocol.pdf}
}
+
@Article{olken_should_2014,
Title = {Should Aid Reward Performance? Evidence from a Field Experiment on Health and Education in Indonesia},
Author = {Olken, Benjamin A. and Onishi, Junko and Wong, Susan},
Journal = {American Economic Journal: Applied Economics},
Year = {2014},
Number = {4},
Pages = {1-34},
Volume = {6},
+
Doi = {10.1257/app.6.4.1},
Url = {http://www.aeaweb.org/articles.php?doi=10.1257/app.6.4.1}
}
+
@TechReport{olken_generasi_2010,
Title = {Generasi Analysis Plan: Wave {III}},
Author = {Olken, Benjamin A. and Onishi, Junko and Wong, Susan},
Year = {2010},
Month = jan,
+
Url = {http://www.povertyactionlab.org/sites/default/files/documents/100122_Generasi_AnalysisPlan_Wave_III_CLEAN.pdf}
}
+
@TechReport{olken_indonesias_2010,
Title = {Indonesia's {PNPM} Generasi program : interim impact evaluation report},
Author = {Olken, Benjamin A. and Onishi, Junko and Wong, Susan},
Institution = {The World Bank},
Year = {2010},
Month = jan,
Number = {59567},
+
Abstract = {This document describes the findings from the interim evaluation survey conducted between October and December 2007, after 15 to 18 months of Generasi implementation in 129 treatment sub districts. Since one full year's project cycle had been completed, authors refer to this survey as the one-year interim evaluation survey. A final evaluation survey is planned for October-December 2009, after the program will have been in operation for 27-30 months. The Generasi project is focused on 12 indicators of maternal and child health behavior and educational behavior. These indicators were chosen by the government of Indonesia to be as similar as possible to the conditions for the individual household conditional cash transfer program being piloted at the same time as Generasi. They are in the same spirit as the conditions used by conditional cash transfer programs in other countries, such as Progresa in Mexico. These 12 indicators respond to those seeking health and educational services that are within the direct control of villagers-such as the number of children who receive immunization, prenatal and postnatal care, and the number of children enrolled and attending school-rather than long-term outcomes, such as test scores or infant mortality. This study provides strong evidence that in this context, community incentives work and are more effective for focusing impacts on the poorest quintiles and increasing providers' efforts. The policy implications are that poverty programs may wish to experiment more with embedding incentives into their designs. The Generasi project is implemented by the government of Indonesia's Ministry of Home Affairs, and is funded in part with loans from the World Bank and grants from the Netherlands Embassy.},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\WRN822RU\\Olken et al. - 2010 - Indonesia's PNPM Generasi program interim impact.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DDJ8JRIR\\indonesias-pnpm-generasi-program-interim-impact-evaluation-report.html:text/html},
Keywords = {access to health services, achievement level, Acute respiratory infection, acute respiratory infections, aged, age groups},
Language = {en},
Pages = {1--154},
Shorttitle = {Indonesia's {PNPM} Generasi program},
Url = {http://documents.worldbank.org/curated/en/2010/01/13763479/indonesias-pnpm-generasi-program-interim-impact-evaluation-report},
Urldate = {2014-08-27}
}
+
@TechReport{olken_generasi_2009,
Title = {Generasi Analysis Plan},
Author = {Olken, Benjamin A. and Onishi, Junko and Wong, Susan},
Year = {2009},
Month = apr,
+
Url = {http://www.povertyactionlab.org/sites/default/files/documents/090408_Generasi_Analysis_Plan_CLEAN.pdf}
}
+
@Article{RPP,
Title = {Estimating the reproducibility of psychological science},
Author = {{Open Science Collaboration} and others},
Journal = {Science},
Year = {2015},
Number = {6251},
Pages = {aac4716},
Volume = {349},
+
Publisher = {American Association for the Advancement of Science}
}
+
@Article{oster_explaining_2006,
Title = {On Explaining Asia's Missing Women: Comment on Das Gupta},
Author = {Oster, Emily},
Journal = {Population and Development Review},
Year = {2006},
+
Month = jun,
Number = {2},
Pages = {323--327},
Volume = {32},
+
Doi = {10.1111/j.1728-4457.2006.00120.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EAZPQVE3\\Oster - 2006 - On Explaining Asia's “Missing Women� Comment on D.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\8TABKTFM\\abstract.html:text/html},
ISSN = {1728-4457},
Language = {en},
Shorttitle = {On Explaining Asia's “Missing Women�},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2006.00120.x/abstract},
Urldate = {2014-08-20}
}
+
@Article{oster_hepatitis_2005,
Title = {Hepatitis B and the Case of the Missing Women},
Author = {Oster, Emily},
Journal = {Journal of Political Economy},
Year = {2005},
+
Month = dec,
Number = {6},
Pages = {1163--1216},
Volume = {113},
+
Abstract = {In many Asian countries the ratio of male to female population is higher than in the West: as high as 1.07 in China and India, and even higher in Pakistan. A number of authors (most notably Amartya Sen) have suggested that this imbalance reflects excess female mortality and have argued that as many as 100 million women are “missing.� This paper proposes an explanation for some of the observed overrepresentation of men: the hepatitis B virus. I present new evidence, consistent with an existing scientific literature, that carriers of the hepatitis B virus have offspring sex ratios around 1.50 boys for each girl. This evidence includes both cross�country analyses and a natural experiment based on recent vaccination campaigns. Hepatitis B is common in many Asian countries, especially China, where some 10–15 percent of the population is infected. Using data on prevalence of the virus by country and estimates of the effect of hepatitis on the sex ratio, I argue that hepatitis B can account for about 45 percent of the “missing women�: around 75 percent in China, between 20 and 50 percent in Egypt and western Asia, and under 20 percent in India, Bangladesh, Pakistan, and Nepal.},
Copyright = {Copyright © 2005 The University of Chicago Press},
Doi = {10.1086/498588},
ISSN = {0022-3808},
Url = {http://www.jstor.org/stable/10.1086/498588},
Urldate = {2014-08-20}
}
+
@Article{oster_hepatitis_2010,
Title = {Hepatitis B does not explain male-biased sex ratios in China},
Author = {Oster, Emily and Chen, Gang and Yu, Xinsen and Lin, Wenyao},
Journal = {Economics Letters},
Year = {2010},
+
Month = may,
Number = {2},
Pages = {142--144},
Volume = {107},
+
Abstract = {Oster (2005) argued that parents with Hepatitis B ({HBV}) have more sons, which explained Asia's “missing women�. Lin and Luoh (2008) show no relationship between gender and mother's {HBV}. We test for a relationship between paternal {HBV} and son share and find none.},
Doi = {10.1016/j.econlet.2010.01.007},
File = {ScienceDirect Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\VUMXQRZT\\Oster et al. - 2010 - Hepatitis B does not explain male-biased sex ratio.pdf:application/pdf;ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DC7KUXQN\\S016517651000008X.html:text/html},
ISSN = {0165-1765},
Keywords = {China, Hepatitis B, Missing women},
Url = {http://www.sciencedirect.com/science/article/pii/S016517651000008X},
Urldate = {2014-08-20}
}
+
@Article{pearce_registration_2011,
Title = {Registration of protocols for observational research is unnecessary and would do more harm than good},
Author = {Pearce, Neil},
Journal = {Occupational and Environmental Medicine},
Year = {2011},
+
Month = feb,
Number = {2},
Pages = {86--88},
Volume = {68},
+
Doi = {10.1136/oem.2010.058917},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\A33K75TG\\Pearce - 2011 - Registration of protocols for observational resear.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\94WISPPT\\86.html:text/html},
ISSN = {, 1470-7926},
Keywords = {Epidemiology},
Language = {en},
Pmid = {21118848},
Url = {http://oem.bmj.com/content/68/2/86},
Urldate = {2014-10-28}
}
+
@Article{peng_reproducible_2011,
Title = {Reproducible Research in Computational},
Author = {Peng, Roger D.},
Journal = {Science},
Year = {2011},
+
Month = dec,
Number = {6060},
Pages = {1226--1227},
Volume = {334},
+
Abstract = {Computational science has led to exciting new developments, but the nature of the work has exposed limitations in our ability to evaluate published findings. Reproducibility has the potential to serve as a minimum standard for judging scientific claims when full independent replication of a study is not possible.},
Doi = {10.1126/science.1213847},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EZ7CXGIF\\Peng - 2011 - Reproducible Research in Computational.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\EUSFBQAF\\1226.html:text/html},
ISSN = {0036-8075, 1095-9203},
Language = {en},
Pmid = {22144613},
Url = {http://www.sciencemag.org/content/334/6060/1226},
Urldate = {2014-09-24}
}
+
@Article{perugini2014safeguard,
Title = {Safeguard power as a protection against imprecise power estimates},
Author = {Perugini, Marco and Gallucci, Marcello and Costantini, Giulio},
Journal = {Perspectives on Psychological Science},
Year = {2014},
Number = {3},
Pages = {319--332},
Volume = {9},
+
Publisher = {SAGE Publications}
}
+
@Article{pitt_gunfight_2012,
Title = {Gunfight at the Not {OK} Corral: Reply to High Noon for Microfinance},
Author = {Pitt, Mark M.},
Journal = {Journal of Development Studies},
Year = {2012},
Number = {12},
Pages = {1886--1891},
Volume = {48},
+
Abstract = {Abstract Duvendack and Palmer-Jones claim to replicate Chemin (2008) and Pitt and Khandker (1998) but obtain different results and hence challenge the two papers' estimates of the impact of microfinance in Bangladesh. This response details a number of reasons to demonstrate that Duvendack and Palmer-Jones is not a replication so their results provide no evidence about the validity of either of the earlier papers or on the effectiveness of microfinance.},
Doi = {10.1080/00220388.2012.727563},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\5W45VKWU\\Pitt - 2012 - Gunfight at the Not OK Corral Reply to ‘High Noon.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\TRD2QCD7\\00220388.2012.html:text/html},
ISSN = {0022-0388},
Shorttitle = {Gunfight at the Not {OK} Corral},
Url = {http://dx.doi.org/10.1080/00220388.2012.727563},
Urldate = {2014-08-21}
}
+
@Article{plassmann_confirming_2003,
Title = {Confirming "More Guns, Less Crime"},
Author = {Plassmann, Florenz and Whitley, John},
Journal = {Stanford Law Review},
Year = {2003},
+
Month = apr,
Number = {4},
Pages = {1313--1369},
Volume = {55},
+
Abstract = {Analyzing county-level data for the entire United States from 1977 to 2000, we find annual reductions in murder rates between 1.5\% and 2.3\% for each additional year that a right-to-carry law is in effect. For the first five years that such a law is in effect, the total benefit from reduced crimes usually ranges between approximately \$2 billion and \$3 billion per year. Ayres and Donohue have simply misread their own results. Their own most general specification that breaks down the impact of the law on a year-by-year basis shows large crime-reducing benefits. Virtually none of their claims that their county-level hybrid model implies initial significant increases in crime are correct. Overall, the vast majority of their estimates-based on data up to 1997-actually demonstrate that right-to-carry laws produce substantial crime-reducing benefits. We show that their models also do an extremely poor job of predicting the changes in crime rates after 1997.},
Copyright = {Copyright © 2003 Stanford Law Review},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\3C2EH4Z8\\Plassmann and Whitley - 2003 - Confirming More Guns, Less Crime.pdf:application/pdf},
ISSN = {0038-9765},
Url = {http://www.jstor.org/stable/1229604},
Urldate = {2014-08-21}
}
+
@Article{resultscompliance,
Title = {Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study},
Author = {Prayle, Andrew P and Hurley, Matthew N and Smyth, Alan R},
Journal = {BMJ},
Year = {2012},
Volume = {344},
+
Abstract = {Andrew P Prayle, NIHR doctoral research fellow, Matthew N Hurley, Wellcome Trust paediatric clinical research fellow, Alan R Smyth, professor of child health1University of Nottingham, Division of Child Health, School of Clinical Sciences, Queens Medical Centre, Nottingham NG7 2UH, UKCorrespondence to: A P Prayle andrew.prayle{at}nottingham.ac.ukAccepted 4 November 2011AbstractObjective To examine compliance with mandatory reporting of summary clinical trial results (within one year of completion of trial) on ClinicalTrials.gov for studies that fall under the recent Food and Drug Administration Amendments Act (FDAAA) legislation.Design Registry based study of clinical trial summaries. Data sources ClinicalTrials.gov, searched on 19 January 2011, with cross referencing with Drugs@FDA to determine for which trials mandatory reporting was required within one year.Selection criteria Studies registered on ClinicalTrials.gov with US sites which completed between 1 January and 31 December 2009.Main outcome measure Proportion of trials for which results had been reported.Results The ClinicalTrials.gov registry contained 83 579 entries for interventional trials, of which 5642 were completed within the timescale of interest. We identified trials as falling within the mandatory reporting rules if they were covered by the FDAAA (trials of a drug, device, or biological agent, which have at least one US site, and are of phase II or later) and if they investigated a drug that already had approval from the Food and Drug Administration. Of these, 163/738 (22\%) had reported results within one year of completion of the trial compared with 76/727 (10\%) trials that were not subject to mandatory reporting (95\% confidence interval for the difference in proportions 7.8\% to 15.5\%; ?2 test, P=2.6{\texttimes}10-9). Later phase trials were more likely to report results (P=4.4{\texttimes}10-11), as were industry funded trials (P=2.2{\texttimes}10-16).Conclusion Most trials subject to mandatory reporting did not report results within a year of completion.FootnotesContributors: APP, MNH, and ARS designed the study. APP wrote the script to webscrape additional data from ClinicalTrials.gov. APP categorised the trials, MNH cross checked a subset, and ARS arbitrated. All authors had full access to the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. All authors wrote the manuscript. APP and MNH contributed equally to this work. ARS is the guarantor.Funding: The study was not externally funded.Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that APP is supported by a National Institute for Health Research fellowship (DRF-2009-02-112) and MNH is supported by a Wellcome Trust fellowship (WT092295AIA). ARS declares relevant activities outside the submitted work of membership of a REMPEX steering committee, consultancies for Novartis and Biocontrol, and a lecture paid for by Chiesi Pharma. ARS has registered trials on ClinicalTrials.gov and other registries.Ethical approval: Not needed.Data sharing: Technical appendix, statistical code, and dataset available from the corresponding author (andrew.prayle{at}nottingham.ac.uk) or at http://dx.doi.org/10.5061/dryad.j512f21p. Participants{\textquoteright} consent for data sharing not required.This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.},
Doi = {10.1136/bmj.d7373},
ISBN = {1468-5833},
ISSN = {0959-8138},
Publisher = {BMJ Publishing Group Ltd}
}
+
@Article{reinhart_growth_2010,
Title = {Growth in a Time of Debt},
Author = {Reinhart, Carmen M. and Rogoff, Kenneth S.},
Journal = {The American Economic Review},
Year = {2010},
+
Month = may,
Number = {2},
Pages = {573--578},
Volume = {100},
+
Copyright = {Copyright © 2010 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HZU3ZQRI\\Reinhart and Rogoff - 2010 - Growth in a Time of Debt.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/27805061},
Urldate = {2014-08-20}
}
+
@Article{rosenthal1979file,
Title = {The file drawer problem and tolerance for null results.},
Author = {Rosenthal, Robert},
Journal = {Psychological bulletin},
Year = {1979},
Number = {3},
Pages = {638},
Volume = {86},
+
Publisher = {American Psychological Association}
}
+
@Article{rothstein_does_2007,
Title = {Does Competition among Public Schools Benefit Students and Taxpayers? Comment},
Author = {Rothstein, Jesse},
Journal = {The American Economic Review},
Year = {2007},
+
Month = dec,
Number = {5},
Pages = {2026--2037},
Volume = {97},
+
Copyright = {Copyright © 2007 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\BRTERQBH\\Rothstein - 2007 - Does Competition among Public Schools Benefit Stud.pdf:application/pdf},
ISSN = {0002-8282},
Shorttitle = {Does Competition among Public Schools Benefit Students and Taxpayers?},
Url = {http://www.jstor.org/stable/30034599},
Urldate = {2014-08-20}
}
+
@Article{rushton_should_2011,
Title = {Should protocols for observational research be registered?},
Author = {Rushton, L.},
Journal = {Occupational and Environmental Medicine},
Year = {2011},
+
Month = feb,
Number = {2},
Pages = {84--86},
Volume = {68},
+
Doi = {10.1136/oem.2010.056846},
ISSN = {, 1470-7926},
Keywords = {Epidemiology},
Language = {en},
Pmid = {21123807},
Url = {http://oem.bmj.com/content/68/2/84},
Urldate = {2014-10-28}
}
+
@Article{sala-i-martin_i_1997,
Title = {I Just Ran Two Million Regressions},
Author = {Sala-I-Martin, Xavier X.},
Journal = {The American Economic Review},
Year = {1997},
+
Month = may,
Number = {2},
Pages = {178--183},
Volume = {87},
+
Copyright = {Copyright © 1997 American Economic Association},
File = {JSTOR Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\7F85Z3X2\\Sala-I-Martin - 1997 - I Just Ran Two Million Regressions.pdf:application/pdf},
ISSN = {0002-8282},
Url = {http://www.jstor.org/stable/2950909},
Urldate = {2014-08-28}
}
+
@Article{samet_register_2010,
Title = {To Register or Not To Register:},
Author = {Samet, Jonathan M.},
Journal = {Epidemiology},
Year = {2010},
+
Month = sep,
Number = {5},
Pages = {610--611},
Volume = {21},
+
Doi = {10.1097/EDE.0b013e3181e9be54},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {To Register or Not To Register},
Url = {http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001648-201009000-00010},
Urldate = {2014-10-28}
}
+
@Article{saquib_practices_2013,
Title = {Practices and impact of primary outcome adjustment in randomized controlled trials: meta-epidemiologic study},
Author = {Saquib, N. and Saquib, J. and Ioannidis, J. P. A.},
Journal = {{BMJ}},
Year = {2013},
+
Month = jul,
Number = {jul12 2},
Pages = {f4313--f4313},
Volume = {347},
+
Doi = {10.1136/bmj.f4313},
File = {Practices and impact of primary outcome adjustment in randomized controlled trials\: meta-epidemiologic study | The BMJ:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\Q5M2NPIE\\bmj.html:text/html},
ISSN = {1756-1833},
Language = {en},
Shorttitle = {Practices and impact of primary outcome adjustment in randomized controlled trials},
Url = {http://www.bmj.com/content/347/bmj.f4313},
Urldate = {2014-10-09}
}
+
@Article{sarsons_rainfall_2011,
Title = {Rainfall and Conflict},
Author = {Sarsons, Heather},
Year = {2011},
+
Month = oct,
+
Abstract = {Starting with Miguel, Satyanath, and Sergenti (2004), a large literature has used rainfall variation as an instrument to study the impacts of income shocks on civil war and
conáict. These studies argue that in agriculturally-dependent regions, negative rain shocks lower income levels, which in turn incites violence. This identifcation strategy
relies on the assumption that rainfall shocks affect conflict only through their impacts on
income. I evaluate this exclusion restriction by identifying districts that are downstream from dams in India. In downstream districts, income is much less sensitive to rainfall fluctuations. However, rain shocks remain equally strong predictors of riot incidence in these districts. These results suggest that rainfall affects rioting through a channel other than income and cast doubt on the conclusion that income shocks incite riots.},
Url = {http://www.econ.yale.edu/conference/neudc11/papers/paper_199.pdf}
}
+
@Article{schulz_consort_2010,
Title = {{CONSORT} 2010 Statement: updated guidelines for reporting parallel group randomised trials},
Author = {Schulz, Kenneth F. and Altman, Douglas G. and Moher, David and \$author.lastName, \$author firstName},
Journal = {{BMC} Medicine},
Year = {2010},
+
Month = mar,
Number = {1},
Pages = {18},
Volume = {8},
+
Abstract = {The {CONSORT} statement is used worldwide to improve the reporting of randomised controlled trials. Kenneth Schulz and colleagues describe the latest version, {CONSORT} 2010, which updates the reporting guideline based on new methodological evidence and accumulating experience.
{PMID}: 20334633},
Copyright = {2010 Schulz et al; licensee {BioMed} Central Ltd.},
Doi = {10.1186/1741-7015-8-18},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\J65R5ZR7\\Schulz et al. - 2010 - CONSORT 2010 Statement updated guidelines for rep.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\UNG8BJ48\\18.html:text/html},
ISSN = {1741-7015},
Language = {en},
Pmid = {20334633},
Shorttitle = {{CONSORT} 2010 Statement},
Url = {http://www.biomedcentral.com/1741-7015/8/18/abstract},
Urldate = {2014-08-29}
}
+
@Article{schulz_multiplicity_2005,
Title = {Multiplicity in randomised trials {II}: subgroup and interim analyses},
Author = {Schulz, Kenneth F and Grimes, David A},
Journal = {The Lancet},
Year = {2005},
+
Month = may,
Number = {9471},
Pages = {1657--1661},
Volume = {365},
+
Abstract = {Summary
Subgroup analyses can pose serious multiplicity concerns. By testing enough subgroups, a false-positive result will probably emerge by chance alone. Investigators might undertake many analyses but only report the significant effects, distorting the medical literature. In general, we discourage subgroup analyses. However, if they are necessary, researchers should do statistical tests of interaction, rather than analyse every separate subgroup. Investigators cannot avoid interim analyses when data monitoring is indicated. However, repeatedly testing at every interim raises multiplicity concerns, and not accounting for multiplicity escalates the false-positive error. Statistical stopping methods must be used. The O'Brien-Fleming and Peto group sequential stopping methods are easily implemented and preserve the intended α level and power. Both adopt stringent criteria (low nominal p values) during the interim analyses. Implementing a trial under these stopping rules resembles a conventional trial, with the exception that it can be terminated early should a treatment prove greatly superior. Investigators and readers, however, need to grasp that the estimated treatment effects are prone to exaggeration, a random high, with early stopping.},
Doi = {10.1016/S0140-6736(05)66516-6},
File = {ScienceDirect Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\K5A5SQTH\\Schulz and Grimes - 2005 - Multiplicity in randomised trials II subgroup and.pdf:application/pdf;ScienceDirect Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\6QEUHN8R\\S0140673605665166.html:text/html},
ISSN = {0140-6736},
Shorttitle = {Multiplicity in randomised trials {II}},
Url = {http://www.sciencedirect.com/science/article/pii/S0140673605665166},
Urldate = {2014-08-28}
}
+
@Article{schulz_allocation_2002,
Title = {Allocation concealment in randomised trials: defending against deciphering},
Author = {Schulz, Kenneth F and Grimes, David A},
Journal = {The Lancet},
Year = {2002},
+
Month = feb,
Number = {9306},
Pages = {614--618},
Volume = {359},
+
Doi = {10.1016/S0140-6736(02)07750-4},
File = {Allocation concealment in randomised trials\: defending against deciphering \: The Lancet:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\CVFTHIJQ\\fulltext.html:text/html},
ISSN = {01406736},
Language = {en},
Shorttitle = {Allocation concealment in randomised trials},
Url = {http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(02)07750-4/fulltext},
Urldate = {2014-09-17}
}
+
@Article{simmons_false-positive_2011,
Title = {False-Positive Psychology Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant},
Author = {Simmons, Joseph P. and Nelson, Leif D. and Simonsohn, Uri},
Journal = {Psychological Science},
Year = {2011},
+
Month = nov,
Number = {11},
Pages = {1359--1366},
Volume = {22},
+
Abstract = {In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.},
Doi = {10.1177/0956797611417632},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\HT3VR6SV\\Simmons et al. - 2011 - False-Positive Psychology Undisclosed Flexibility .pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\DX68RTPX\\1359.html:text/html},
ISSN = {0956-7976, 1467-9280},
Keywords = {disclosure, methodology, motivated reasoning, publication},
Language = {en},
Pmid = {22006061},
Url = {http://pss.sagepub.com/content/22/11/1359},
Urldate = {2014-08-28}
}
+
@Article{simonsohn2013just,
Title = {Just post it: the lesson from two cases of fabricated data detected by statistics alone.},
Author = {Simonsohn, Uri},
Journal = {Psychological science},
Year = {2013},
Number = {10},
Pages = {1875},
Volume = {24}
}
+
@Article{simonsohn2014p,
Title = {P-curve: A key to the file-drawer.},
Author = {Simonsohn, Uri and Nelson, Leif D and Simmons, Joseph P},
Journal = {Journal of Experimental Psychology: General},
Year = {2014},
Number = {2},
Pages = {534},
Volume = {143},
+
Publisher = {American Psychological Association}
}
+
@TechReport{simonsohn_p-curve:_2013,
Title = {P-Curve: A Key to the File Drawer},
Author = {Simonsohn, Uri and Nelson, Leif D. and Simmons, Joseph P.},
Institution = {Social Science Research Network},
Year = {2013},
+
Address = {Rochester, {NY}},
Month = apr,
Number = {{ID} 2256237},
Type = {{SSRN} Scholarly Paper},
+
Abstract = {Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that “work�, readers must ask, “Are these effects true, or do they merely reflect selective reporting?� We introduce p-curve as a way to answer this question. P-curve is the distribution of statistically significant p-values for a set of studies (ps {\textless} .05). Because only true effects are expected to generate right-skewed p-curves – containing more low (.01s) than high (.04s) significant p-values – only right-skewed p-curves are diagnostic of evidential value. By telling us whether we can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\RI9CPIUI\\papers.html:text/html},
Keywords = {decision making, file drawer, judgment, p-hacking, Publication bias, science, statistics},
Shorttitle = {P-Curve},
Url = {http://papers.ssrn.com/abstract=2256237},
Urldate = {2014-09-04}
}
+
@Article{siskind_minimum_1977,
Title = {Minimum Wage Legislation in the United States: Comment},
Author = {Siskind, Frederic B.},
Journal = {Economic Inquiry},
Year = {1977},
+
Month = jan,
Number = {1},
Pages = {135--138},
Volume = {15},
+
Doi = {10.1111/j.1465-7295.1977.tb00457.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\RJK53NXB\\Siskind - 1977 - Minimum Wage Legislation in the United States Com.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\URRDFTHJ\\abstract.html:text/html},
ISSN = {1465-7295},
Language = {en},
Shorttitle = {Minimum Wage Legislation in the United States},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1465-7295.1977.tb00457.x/abstract},
Urldate = {2014-10-01}
}
+
@Article{sorge2014olfactory,
Title = {Olfactory exposure to males, including men, causes stress and related analgesia in rodents},
Author = {Sorge, Robert E and Martin, Loren J and Isbester, Kelsey A and Sotocinal, Susana G and Rosen, Sarah and Tuttle, Alexander H and Wieskopf, Jeffrey S and Acland, Erinn L and Dokova, Anastassia and Kadoura, Basil and others},
Journal = {Nature methods},
Year = {2014},
+
Publisher = {Nature Publishing Group}
}
+
@Article{stanley_beyond_2005,
Title = {Beyond Publication Bias},
Author = {Stanley, T. D.},
Journal = {Journal of Economic Surveys},
Year = {2005},
+
Month = jul,
Number = {3},
Pages = {309--345},
Volume = {19},
+
Abstract = {Abstract.  This review considers several meta-regression and graphical methods that can differentiate genuine empirical effect from publication bias. Publication selection exists when editors, reviewers, or researchers have a preference for statistically significant results. Because all areas of empirical research are susceptible to publication selection, any average or tally of significant/insignificant studies is likely to be biased and potentially misleading. Meta-regression analysis can see through the murk of random sampling error and selected misspecification bias to identify the underlying statistical structures that characterize genuine empirical effect. Meta-significance testing and precision-effect testing ({PET}) are offered as a means to identify empirical effect beyond publication bias and are applied to four areas of empirical economics research – minimum wage effects, union-productivity effects, price  elasticities, and tests of the natural rate hypothesis.},
Doi = {10.1111/j.0950-0804.2005.00250.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\GBSEMN2E\\Stanley - 2005 - Beyond Publication Bias.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\M72Q5RS2\\full.html:text/html},
ISSN = {1467-6419},
Keywords = {Elasticities, Funnel plot, Meta-Regression Analysis, Minimum wage, Natural rate hypothesis, Publication bias, Union-productivity},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.0950-0804.2005.00250.x/abstract},
Urldate = {2014-08-19}
}
+
@Article{stanley2005beyond,
Title = {Beyond publication bias},
Author = {Stanley, Tom D},
Journal = {Journal of Economic Surveys},
Year = {2005},
Number = {3},
Pages = {309--345},
Volume = {19},
+
Publisher = {Wiley Online Library}
}
+
@Misc{sven_vlaeminck_research_????,
Title = {Research Data Management in Economic Journals},
+
Author = {{Sven Vlaeminck}},
+
Abstract = {The Open Economics Working Group of the Open Knowledge Foundation – Identifying best practice as well as legal, regulatory and technical standards for open economic data},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\RPVJZQN8\\data-policies-of-economic-journals.html:text/html},
Journal = {Data Policies of Economics Journals},
Url = {http://openeconomics.net/resources/data-policies-of-economic-journals/},
Urldate = {2014-08-29}
}
+
@Article{ICMJEData,
Title = {Sharing Clinical Trial Data: A Proposal From the International Committee of Medical Journal EditorsSharing Clinical Trial Data},
Author = {Taichman, Darren B. and Backus, Joyce and Baethge, Christopher and Bauchner, Howard and de Leeuw, Peter W. and Drazen, Jeffrey M. and Fletcher, John and Frizelle, Frank A. and Groves, Trish and Haileamlak, Abraham and James, Astrid and Laine, Christine and Peiperl, Larry and Pinborg, Anja and Sahni, Peush and Wu, Sinan},
Journal = {Annals of Internal Medicine},
Year = {2016},
+
Doi = {10.7326/M15-2928},
Url = { + http://dx.doi.org/10.7326/M15-2928}
}
+
@Article{takkouche_meta-analysis_2010,
Title = {Meta-analysis Protocol Registration: Sed quis custodiet ipsos custodes? [But Who Will Guard the Guardians?]},
Author = {Takkouche, Bahi and Norman, Guy},
Journal = {Epidemiology},
Year = {2010},
+
Month = sep,
Number = {5},
Pages = {614--615},
Volume = {21},
+
Doi = {10.1097/EDE.0b013e3181e9bbbd},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {Meta-analysis Protocol Registration},
Url = {http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00001648-201009000-00012},
Urldate = {2014-10-28}
}
+
@Article{taubman_oregon_2013,
Title = {{THE} {OREGON} {HEALTH} {INSURANCE} {EXPERIMENT}: {EVIDENCE} {FROM} {EMERGENCY} {DEPARTMENT} {DATA} Analysis Plan},
Author = {Taubman, Sarah and Allen, Heidi and Baicker, Katherine and Wright, Bill and Finkelstein, Amy},
Year = {2013},
+
Month = mar,
+
Series = {{NBER} Working Paper},
Url = {http://www.nber.org/oregon/files/ED%20Analysis%20Plan.pdf}
}
+
@Article{Taubman17012014,
Title = {Medicaid Increases Emergency-Department Use: Evidence from Oregon's Health Insurance Experiment},
Author = {Taubman, Sarah L. and Allen, Heidi L. and Wright, Bill J. and Baicker, Katherine and Finkelstein, Amy N.},
Journal = {Science},
Year = {2014},
Number = {6168},
Pages = {263-268},
Volume = {343},
+
Abstract = {In 2008, Oregon initiated a limited expansion of a Medicaid program for uninsured, low-income adults, drawing names from a waiting list by lottery. This lottery created a rare opportunity to study the effects of Medicaid coverage by using a randomized controlled design. By using the randomization provided by the lottery and emergency-department records from Portland-area hospitals, we studied the emergency department use of about 25,000 lottery participants over about 18 months after the lottery. We found that Medicaid coverage significantly increases overall emergency use by 0.41 visits per person, or 40% relative to an average of 1.02 visits per person in the control group. We found increases in emergency-department visits across a broad range of types of visits, conditions, and subgroups, including increases in visits for conditions that may be most readily treatable in primary care settings.},
Doi = {10.1126/science.1246183},
Eprint = {http://www.sciencemag.org/content/343/6168/263.full.pdf},
Url = {http://www.sciencemag.org/content/343/6168/263.abstract}
}
+
@Article{the_lancet_should_2010,
Title = {Should protocols for observational research be registered?},
Author = {{The Lancet}},
Journal = {The Lancet},
Year = {2010},
+
Month = jan,
Number = {9712},
Pages = {348},
Volume = {375},
+
Doi = {10.1016/S0140-6736(10)60148-1},
ISSN = {01406736},
Language = {en},
Url = {http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(10)60148-1/fulltext},
Urldate = {2014-10-28}
}
+
@Article{turner_selective_2008,
Title = {Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy},
Author = {Turner, Erick H. and Matthews, Annette M. and Linardatos, Eftihia and Tell, Robert A. and Rosenthal, Robert},
Journal = {New England Journal of Medicine},
Year = {2008},
+
Month = jan,
Number = {3},
Pages = {252--260},
Volume = {358},
+
Abstract = {Medical decisions are based on an understanding of publicly reported clinical trials.1,2 If the evidence base is biased, then decisions based on this evidence may not be the optimal decisions. For example, selective publication of clinical trials, and the outcomes within those trials, can lead to unrealistic estimates of drug effectiveness and alter the apparent risk–benefit ratio.3,4 Attempts to study selective publication are complicated by the unavailability of data from unpublished trials. Researchers have found evidence for selective publication by comparing the results of published trials with information from surveys of authors,5 registries,6 institutional review boards,7,8 and . . .},
Doi = {10.1056/NEJMsa065779},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\K8IGA3UZ\\Turner et al. - 2008 - Selective Publication of Antidepressant Trials and.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\S6KX8JVP\\nejmsa065779.html:text/html},
ISSN = {0028-4793},
Pmid = {18199864},
Url = {http://www.nejm.org/doi/full/10.1056/NEJMsa065779},
Urldate = {2014-10-07}
}
+
@Article{van2011targeted,
Title = {Targeted learning},
Author = {Van der Laan, Mark J and Petersen, Maya L},
Journal = {Ensemble Machine Learning},
Year = {2011},
Pages = {117--156},
+
Publisher = {Springer}
}
+
@Article{strobestatement2007,
Title = {The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies},
Author = {Von Elm, Erik and Altman, Douglas G and Egger, Matthias and Pocock, Stuart J and G{\o}tzsche, Peter C and Vandenbroucke, Jan P and Strobe Initiative and others},
Journal = {Preventive medicine},
Year = {2007},
Number = {4},
Pages = {247--251},
Volume = {45},
+
Publisher = {Elsevier}
}
+
@Article{weiss_identification_2011,
Title = {The Identification and Prevention of Publication Bias in the Social Sciences and Economics},
Author = {Weiss, Bernd and Wagner, Michael},
Journal = {Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik)},
Year = {2011},
Number = {5-6},
Pages = {661--684},
Volume = {231},
+
Abstract = {Systematic research reviews have become essential in all empirical sciences. However, the validity of research syntheses is threatened by the fact that not all studies on a given topic can be summarized. Research reviews may suffer from missing data, and this is especially crucial in those cases where the selectivity of studies and their findings affects the summarized result. So-called publication bias is a type of missing data and a phenomenon that jeopardizes the validity of systematic or quantitative, as well as narrative, reviews. Publication bias exists if the preparation, submission or publication of research findings depend on characteristics of just these research results, e. g. their direction or statistical significance. This article describes methods to identify publication bias in the context of meta-analysis. It also reviews empirical studies on the prevalence of publication bias, especially in the social and economic sciences, where publication bias also seems to be prevalent. Several proposals to prevent publication bias are discussed.},
File = {RePEc Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\XDK5FC44\\v231y2011i5-6p661-684.html:text/html},
Keywords = {meta analysis, Publication bias, Systematic review},
Url = {http://ideas.repec.org/a/jns/jbstat/v231y2011i5-6p661-684.html},
Urldate = {2014-08-20}
}
+
@Article{welch_minimum_1977,
Title = {Minimum Wage Legislation in the United States: Reply},
Author = {Welch, Finis},
Journal = {Economic Inquiry},
Year = {1977},
+
Month = jan,
Number = {1},
Pages = {139--142},
Volume = {15},
+
Doi = {10.1111/j.1465-7295.1977.tb00458.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\QFPT7Z3T\\Welch - 1977 - Minimum Wage Legislation in the United States Rep.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\SIEIJ4D5\\abstract.html:text/html},
ISSN = {1465-7295},
Language = {en},
Shorttitle = {Minimum Wage Legislation in the United States},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1465-7295.1977.tb00458.x/abstract},
Urldate = {2014-10-01}
}
+
@Article{welch_minimum_1974,
Title = {Minimum Wage Legislation in the United States*},
Author = {Welch, Finis},
Journal = {Economic Inquiry},
Year = {1974},
+
Month = sep,
Number = {3},
Pages = {285--318},
Volume = {12},
+
Doi = {10.1111/j.1465-7295.1974.tb00401.x},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\ZVZUK3WG\\Welch - 1974 - Minimum Wage Legislation in the United States.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\46BQDM3X\\abstract.html:text/html},
ISSN = {1465-7295},
Language = {en},
Url = {http://onlinelibrary.wiley.com/doi/10.1111/j.1465-7295.1974.tb00401.x/abstract},
Urldate = {2014-10-01}
}
+
@Book{westfall_young_multiple,
Title = {Resampling-Based Multiple Testing},
Author = {Peter H. Westfall and S. Stanley Young},
Publisher = {Wiley},
Year = {1993},
+
Owner = {garret},
Timestamp = {2014.11.20}
}
+
@Article{williams_registration_2010,
Title = {Registration of observational studies: Is it time?},
Author = {Williams, Rebecca J. and Tse, Tony and Harlan, William R. and Zarin, Deborah A.},
Journal = {Canadian Medical Association Journal},
Year = {2010},
+
Month = oct,
Number = {15},
Pages = {1638--1642},
Volume = {182},
+
Doi = {10.1503/cmaj.092252},
ISSN = {0820-3946, 1488-2329},
Language = {en},
Pmid = {20643833},
Shorttitle = {Registration of observational studies},
Url = {http://www.cmaj.ca/content/182/15/1638},
Urldate = {2014-10-28}
}
+
@Misc{wohlrabe_open_2014,
Title = {Do open access articles in economics have a citation advantage?},
+
Author = {Wohlrabe, Klaus and Birkmeier, Daniel},
Month = jun,
Year = {2014},
+
Abstract = {We investigate whether articles in economics that are freely available on the web have a citation advantage over articles with a gated access. Our sample consists of articles from 2005 from 13 economic journals (including the top five journals). In addition to standard mean comparisons we also use a negative-binomial regression model with several covariates to control for potential selection effects and quality bias. Using citation data from three different databases (Web of Science, {RePEc} and Google Scholar) we show that articles that are freely available on the internet have indeed a significantly higher citation count.},
File = {Full Text PDF:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\D7HWA4F9\\Wohlrabe and Birkmeier - 2014 - Do open access articles in economics have a citati.pdf:application/pdf;Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\KUGBM567\\56842.html:text/html},
Keywords = {A12 - Relation of Economics to Other Disciplines, A14 - Sociology of Economics},
Language = {en},
Type = {{MPRA} Paper},
Url = {http://mpra.ub.uni-muenchen.de/56842/},
Urldate = {2014-08-20}
}
+
@Article{wydick_-kind_2014,
Title = {Do in-kind transfers damage local markets? The case of {TOMS} shoe donations in El Salvador},
Author = {Wydick, Bruce and Katz, Elizabeth and Janet, Brendan},
Journal = {Journal of Development Effectiveness},
Year = {2014},
+
Month = may,
Number = {3},
Pages = {249--267},
Volume = {6},
+
Abstract = {We carry out a cluster randomised trial among 979 households in rural El Salvador to test whether shoe donations exhibit negative impacts on local shoe markets. Households in half of the communities were given a pair of children’s shoes at baseline (treatment communities), while all households were given coupons that could be used for shoe purchases at a local shoe store. Although point estimates on coupon redemption and difference-in-difference estimations indicate shoe purchases to be slightly lower among households receiving the donated shoes, we find no statistically significant difference in market shoe purchases between treatment and control households.},
Doi = {10.1080/19439342.2014.919012},
File = {Snapshot:C\:\\Users\\garret\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\vbdhbrhn.default\\zotero\\storage\\FHPGCJRP\\19439342.2014.html:text/html},
ISSN = {1943-9342},
Shorttitle = {Do in-kind transfers damage local markets?},
Url = {http://dx.doi.org/10.1080/19439342.2014.919012},
Urldate = {2014-08-27}
}
+
@InCollection{xie_knitr:_2014,
Title = {knitr: A Comprehensive Tool for Reproducible Research in R},
Author = {Xie, Yihui},
Booktitle = {Implementing Reproducible Research},
Publisher = {{CRC} Press},
Year = {2014},
Month = apr,
Pages = {3--32},
+
Abstract = {In computational science, reproducibility requires that researchers make code and data available to others so that the data can be analyzed in a similar manner as in the original publication. Code must be available to be distributed, data must be accessible in a readable format, and a platform must be available for widely distributing the data and code. In addition, both data and code need to be licensed permissively enough so that others can reproduce the work without a substantial legal burden. Implementing Reproducible Research covers many of the elements necessary for conducting and distributing reproducible research. It explains how to accurately reproduce a scientific result. Divided into three parts, the book discusses the tools, practices, and dissemination platforms for ensuring reproducibility in computational science. It describes: Computational tools, such as Sweave, knitr, {VisTrails}, Sumatra, {CDE}, and the Declaratron system Open source practices, good programming practices, trends in open science, and the role of cloud computing in reproducible research Software and methodological platforms, including open source software packages, {RunMyCode} platform, and open access journals Each part presents contributions from leaders who have developed software and other products that have advanced the field. Supplementary material is available at www.{ImplementingRR}.org.},
ISBN = {9781466561595},
Keywords = {Computers / Mathematical \& Statistical Software, Mathematics / Probability \& Statistics / General, Science / Life Sciences / Biology},
Language = {en}
}
+
@Book{xie_dynamic_2013,
Title = {Dynamic Documents with R and knitr},
Author = {Xie, Yihui},
Publisher = {{CRC} Press},
Year = {2013},
Month = jul,
+
Abstract = {The cut-and-paste approach to writing statistical reports is not only tedious and laborious, but also can be harmful to scientific research, because it is inconvenient to reproduce the results. Dynamic Documents with R and knitr introduces a new approach via dynamic documents, i.e. integrating computing directly with reporting. A comprehensive guide to the R package knitr, the book covers examples, document editors, basic usage, detailed explanations of a wide range of options, tricks and solutions, extensions, and complete applications of this package. The book provides an overview of dynamic documents, introducing the idea of literate programming. It then explains the importance of dynamic documents to scientific research and its impact on reproducible research. Building on this, the author covers basic concepts, common text editors that support knitr, and the syntax for different document formats such as {LaTeX}, {HTML}, and Markdown before going on to discuss core functionality, how to control text and graphics output, caching mechanisms that can reduce computation time, and reuse of source code. He then explores advanced topics such as chunk hooks, integrating other languages such as Python and awk into one report in the knitr framework, and useful tricks that make it easier to write documents with knitr. Discussions of how to publish reports in a variety of formats, applications, and other tools complete the coverage. Suitable for both beginners and advanced users, this book shows you how to write reports in simple languages such as Markdown. The reports range from homework, projects, exams, books, blogs, and web pages to any documents related to statistical graphics, computing, and data analysis. While familiarity with {LaTeX} and {HTML} is helpful, the book requires no prior experience with advanced programs or languages. For beginners, the text provides enough features to get started on basic applications. For power users, the last several chapters enable an understanding of the extensibility of the knitr package.},
ISBN = {9781482203530},
Keywords = {Computers / Mathematical \& Statistical Software, Mathematics / Probability \& Statistics / General, Science / Life Sciences / Biology},
Language = {en}
}
+
@Article{_registration_2010,
Title = {The Registration of Observational Studies When Metaphors Go Bad:},
Journal = {Epidemiology},
Year = {2010},
+
Month = jul,
Pages = {1},
+
Doi = {10.1097/EDE.0b013e3181eafbcf},
ISSN = {1044-3983},
Language = {en},
Shorttitle = {The Registration of Observational Studies—When Metaphors Go Bad},
Url = {http://journals.lww.com/epidem/Fulltext/2010/09000/The_Registration_of_Observational_Studies_When.9.aspx},
Urldate = {2014-10-28}
}
+
+ + + +
+ +
+ + + + +
+ +
+ +
+
+ +
+ + + + + + +
+ + + You can't perform that action at this time. +
+ + + + + + + + + + +
+ + You signed in with another tab or window. Reload to refresh your session. + You signed out in another tab or window. Reload to refresh your session. +
+ + + + + + diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Barba, 2018.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Barba, 2018.pdf new file mode 100644 index 0000000..9fc3eec Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Barba, 2018.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Boettiger, 2015.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Boettiger, 2015.pdf new file mode 100644 index 0000000..651d59c Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Boettiger, 2015.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Brown and Heathers, 2016.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Brown and Heathers, 2016.pdf new file mode 100644 index 0000000..766279c Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Brown and Heathers, 2016.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Button et al., 2013.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Button et al., 2013.pdf new file mode 100644 index 0000000..24e17ab Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Button et al., 2013.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Chagas, 2018.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Chagas, 2018.pdf new file mode 100644 index 0000000..0c354f1 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Chagas, 2018.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Christensen and Migeul., 2017.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Christensen and Migeul., 2017.pdf new file mode 100644 index 0000000..c233aed Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Christensen and Migeul., 2017.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Edwards et al., 2017.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Edwards et al., 2017.pdf new file mode 100644 index 0000000..9a0a60d Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Edwards et al., 2017.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Goodman et al., 2016.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Goodman et al., 2016.pdf new file mode 100644 index 0000000..189f74d Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Goodman et al., 2016.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Green and Clyburne-Sherin, 2018.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Green and Clyburne-Sherin, 2018.pdf new file mode 100644 index 0000000..a92ff42 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Green and Clyburne-Sherin, 2018.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Howe, 2012.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Howe, 2012.pdf new file mode 100644 index 0000000..716099d Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Howe, 2012.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Klein et al., 2014.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Klein et al., 2014.pdf new file mode 100644 index 0000000..3212ece Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Klein et al., 2014.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Leek and Peng, 2015.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Leek and Peng, 2015.pdf new file mode 100644 index 0000000..37b525d Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Leek and Peng, 2015.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Lowndes et al., 2017.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Lowndes et al., 2017.pdf new file mode 100644 index 0000000..94c1742 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Lowndes et al., 2017.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Nosek et al., 2017.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Nosek et al., 2017.pdf new file mode 100644 index 0000000..ad09b5d Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Nosek et al., 2017.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Piccolo and Frampton, 2016.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Piccolo and Frampton, 2016.pdf new file mode 100644 index 0000000..88784aa Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Piccolo and Frampton, 2016.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Piento et al., 2010.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Piento et al., 2010.pdf new file mode 100644 index 0000000..979927d Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Piento et al., 2010.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Ram, 2013.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Ram, 2013.pdf new file mode 100644 index 0000000..8e7e198 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Ram, 2013.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Sandve et al., 2013.PDF b/reading_material/3 - Reproducible Research and Data Analysis/Sandve et al., 2013.PDF new file mode 100644 index 0000000..552b9f1 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Sandve et al., 2013.PDF differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Schimmack, 2012.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Schimmack, 2012.pdf new file mode 100644 index 0000000..609d580 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Schimmack, 2012.pdf differ diff --git a/reading_material/3 - Reproducible Research and Data Analysis/Toelch and Oswald, 2017.pdf b/reading_material/3 - Reproducible Research and Data Analysis/Toelch and Oswald, 2017.pdf new file mode 100644 index 0000000..7971a28 Binary files /dev/null and b/reading_material/3 - Reproducible Research and Data Analysis/Toelch and Oswald, 2017.pdf differ diff --git a/reading_material/4 - Open Research Data/Anderson et al., 2017.pdf b/reading_material/4 - Open Research Data/Anderson et al., 2017.pdf new file mode 100644 index 0000000..bbfc628 Binary files /dev/null and b/reading_material/4 - Open Research Data/Anderson et al., 2017.pdf differ diff --git a/reading_material/4 - Open Research Data/Bauch et al., 2011.pdf b/reading_material/4 - Open Research Data/Bauch et al., 2011.pdf new file mode 100644 index 0000000..82a0705 Binary files /dev/null and b/reading_material/4 - Open Research Data/Bauch et al., 2011.pdf differ diff --git a/reading_material/4 - Open Research Data/Bean et al., 2015.pdf b/reading_material/4 - Open Research Data/Bean et al., 2015.pdf new file mode 100644 index 0000000..38929f9 Binary files /dev/null and b/reading_material/4 - Open Research Data/Bean et al., 2015.pdf differ diff --git a/reading_material/4 - Open Research Data/Bechhofer et al., 2010.pdf b/reading_material/4 - Open Research Data/Bechhofer et al., 2010.pdf new file mode 100644 index 0000000..1f36fdd Binary files /dev/null and b/reading_material/4 - Open Research Data/Bechhofer et al., 2010.pdf differ diff --git a/reading_material/4 - Open Research Data/Benson et al., 2013.pdf b/reading_material/4 - Open Research Data/Benson et al., 2013.pdf new file mode 100644 index 0000000..8f98dc6 Binary files /dev/null and b/reading_material/4 - Open Research Data/Benson et al., 2013.pdf differ diff --git a/reading_material/4 - Open Research Data/Chavan and Penev, 2011.pdf b/reading_material/4 - Open Research Data/Chavan and Penev, 2011.pdf new file mode 100644 index 0000000..8773076 Binary files /dev/null and b/reading_material/4 - Open Research Data/Chavan and Penev, 2011.pdf differ diff --git a/reading_material/4 - Open Research Data/Dorch et al., 2015.pdf b/reading_material/4 - Open Research Data/Dorch et al., 2015.pdf new file mode 100644 index 0000000..9a939b9 Binary files /dev/null and b/reading_material/4 - Open Research Data/Dorch et al., 2015.pdf differ diff --git a/reading_material/4 - Open Research Data/Fecher et al., 2015.PDF b/reading_material/4 - Open Research Data/Fecher et al., 2015.PDF new file mode 100644 index 0000000..367ae6a Binary files /dev/null and b/reading_material/4 - Open Research Data/Fecher et al., 2015.PDF differ diff --git a/reading_material/4 - Open Research Data/Fecher et al., 2017.pdf b/reading_material/4 - Open Research Data/Fecher et al., 2017.pdf new file mode 100644 index 0000000..0f086f3 Binary files /dev/null and b/reading_material/4 - Open Research Data/Fecher et al., 2017.pdf differ diff --git a/reading_material/4 - Open Research Data/Fenner et al., 2018.pdf b/reading_material/4 - Open Research Data/Fenner et al., 2018.pdf new file mode 100644 index 0000000..884facc Binary files /dev/null and b/reading_material/4 - Open Research Data/Fenner et al., 2018.pdf differ diff --git "a/reading_material/4 - Open Research Data/Gonz\303\241lez-Beltr\303\241n et al., 2015.PDF" "b/reading_material/4 - Open Research Data/Gonz\303\241lez-Beltr\303\241n et al., 2015.PDF" new file mode 100644 index 0000000..a90b48e Binary files /dev/null and "b/reading_material/4 - Open Research Data/Gonz\303\241lez-Beltr\303\241n et al., 2015.PDF" differ diff --git a/reading_material/4 - Open Research Data/Gorgolewski et al., 2013.pdf b/reading_material/4 - Open Research Data/Gorgolewski et al., 2013.pdf new file mode 100644 index 0000000..1dedb39 Binary files /dev/null and b/reading_material/4 - Open Research Data/Gorgolewski et al., 2013.pdf differ diff --git a/reading_material/4 - Open Research Data/Kratz and Strasser, 2015.pdf b/reading_material/4 - Open Research Data/Kratz and Strasser, 2015.pdf new file mode 100644 index 0000000..bbeabd1 Binary files /dev/null and b/reading_material/4 - Open Research Data/Kratz and Strasser, 2015.pdf differ diff --git a/reading_material/4 - Open Research Data/Lecarpentier et al., 2013.pdf b/reading_material/4 - Open Research Data/Lecarpentier et al., 2013.pdf new file mode 100644 index 0000000..5f31330 Binary files /dev/null and b/reading_material/4 - Open Research Data/Lecarpentier et al., 2013.pdf differ diff --git a/reading_material/4 - Open Research Data/Mons et al., 2017.pdf b/reading_material/4 - Open Research Data/Mons et al., 2017.pdf new file mode 100644 index 0000000..46d3144 Binary files /dev/null and b/reading_material/4 - Open Research Data/Mons et al., 2017.pdf differ diff --git a/reading_material/4 - Open Research Data/Piwowar and Vision, 2013.pdf b/reading_material/4 - Open Research Data/Piwowar and Vision, 2013.pdf new file mode 100644 index 0000000..b34c852 Binary files /dev/null and b/reading_material/4 - Open Research Data/Piwowar and Vision, 2013.pdf differ diff --git a/reading_material/4 - Open Research Data/Poline et al., 2012.pdf b/reading_material/4 - Open Research Data/Poline et al., 2012.pdf new file mode 100644 index 0000000..a9ba4a6 Binary files /dev/null and b/reading_material/4 - Open Research Data/Poline et al., 2012.pdf differ diff --git a/reading_material/4 - Open Research Data/Roche et al., 2015.PDF b/reading_material/4 - Open Research Data/Roche et al., 2015.PDF new file mode 100644 index 0000000..7e03a68 Binary files /dev/null and b/reading_material/4 - Open Research Data/Roche et al., 2015.PDF differ diff --git a/reading_material/4 - Open Research Data/Sansone et al., 2012.pdf b/reading_material/4 - Open Research Data/Sansone et al., 2012.pdf new file mode 100644 index 0000000..3227e8e Binary files /dev/null and b/reading_material/4 - Open Research Data/Sansone et al., 2012.pdf differ diff --git a/reading_material/4 - Open Research Data/Starr et al., 2015.pdf b/reading_material/4 - Open Research Data/Starr et al., 2015.pdf new file mode 100644 index 0000000..edf8953 Binary files /dev/null and b/reading_material/4 - Open Research Data/Starr et al., 2015.pdf differ diff --git a/reading_material/4 - Open Research Data/Treadway et al., 2016.pdf b/reading_material/4 - Open Research Data/Treadway et al., 2016.pdf new file mode 100644 index 0000000..09ff120 Binary files /dev/null and b/reading_material/4 - Open Research Data/Treadway et al., 2016.pdf differ diff --git a/reading_material/4 - Open Research Data/Wenger et al., 2000.pdf b/reading_material/4 - Open Research Data/Wenger et al., 2000.pdf new file mode 100644 index 0000000..dd870c9 Binary files /dev/null and b/reading_material/4 - Open Research Data/Wenger et al., 2000.pdf differ diff --git a/reading_material/4 - Open Research Data/White et al., 2013.pdf b/reading_material/4 - Open Research Data/White et al., 2013.pdf new file mode 100644 index 0000000..abcee77 Binary files /dev/null and b/reading_material/4 - Open Research Data/White et al., 2013.pdf differ diff --git a/reading_material/4 - Open Research Data/Wilkinson et al., 2016.pdf b/reading_material/4 - Open Research Data/Wilkinson et al., 2016.pdf new file mode 100644 index 0000000..31b0a19 Binary files /dev/null and b/reading_material/4 - Open Research Data/Wilkinson et al., 2016.pdf differ diff --git a/reading_material/4 - Open Research Data/Wolstencroft et al., 2015.pdf b/reading_material/4 - Open Research Data/Wolstencroft et al., 2015.pdf new file mode 100644 index 0000000..47bdc98 Binary files /dev/null and b/reading_material/4 - Open Research Data/Wolstencroft et al., 2015.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Balasegaram et al., 2017.pdf b/reading_material/5 - Open Source and Open Research Software/Balasegaram et al., 2017.pdf new file mode 100644 index 0000000..f4546ba Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Balasegaram et al., 2017.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Boettiger and Eddelbuettel, 2017.pdf b/reading_material/5 - Open Source and Open Research Software/Boettiger and Eddelbuettel, 2017.pdf new file mode 100644 index 0000000..fffa09d Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Boettiger and Eddelbuettel, 2017.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Dryden et al., 2017.pdf b/reading_material/5 - Open Source and Open Research Software/Dryden et al., 2017.pdf new file mode 100644 index 0000000..fba05e5 Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Dryden et al., 2017.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Ince et al., 2012.pdf b/reading_material/5 - Open Source and Open Research Software/Ince et al., 2012.pdf new file mode 100644 index 0000000..bbdc84b Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Ince et al., 2012.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Iskoujina and Roberts, 2015.pdf b/reading_material/5 - Open Source and Open Research Software/Iskoujina and Roberts, 2015.pdf new file mode 100644 index 0000000..e17d123 Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Iskoujina and Roberts, 2015.pdf differ diff --git "a/reading_material/5 - Open Source and Open Research Software/Jim\303\251nez et al., 2018.pdf" "b/reading_material/5 - Open Source and Open Research Software/Jim\303\251nez et al., 2018.pdf" new file mode 100644 index 0000000..feea9cd Binary files /dev/null and "b/reading_material/5 - Open Source and Open Research Software/Jim\303\251nez et al., 2018.pdf" differ diff --git a/reading_material/5 - Open Source and Open Research Software/Martinez-Torres and Diaz-Fernandez, 2013.pdf b/reading_material/5 - Open Source and Open Research Software/Martinez-Torres and Diaz-Fernandez, 2013.pdf new file mode 100644 index 0000000..f60901f Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Martinez-Torres and Diaz-Fernandez, 2013.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Oishi et al., 2018.pdf b/reading_material/5 - Open Source and Open Research Software/Oishi et al., 2018.pdf new file mode 100644 index 0000000..b937c2a Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Oishi et al., 2018.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Sandve et al., 2013.PDF b/reading_material/5 - Open Source and Open Research Software/Sandve et al., 2013.PDF new file mode 100644 index 0000000..552b9f1 Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Sandve et al., 2013.PDF differ diff --git a/reading_material/5 - Open Source and Open Research Software/Shamir et al., 2013.pdf b/reading_material/5 - Open Source and Open Research Software/Shamir et al., 2013.pdf new file mode 100644 index 0000000..c043631 Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Shamir et al., 2013.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Steinmacher et al., 2014.pdf b/reading_material/5 - Open Source and Open Research Software/Steinmacher et al., 2014.pdf new file mode 100644 index 0000000..2427821 Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Steinmacher et al., 2014.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Stodden, 2010.pdf b/reading_material/5 - Open Source and Open Research Software/Stodden, 2010.pdf new file mode 100644 index 0000000..b4dfeb5 Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Stodden, 2010.pdf differ diff --git a/reading_material/5 - Open Source and Open Research Software/Vandewalle, 2012.pdf b/reading_material/5 - Open Source and Open Research Software/Vandewalle, 2012.pdf new file mode 100644 index 0000000..3daeb0d Binary files /dev/null and b/reading_material/5 - Open Source and Open Research Software/Vandewalle, 2012.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Bjork et al., 2013.pdf b/reading_material/6 - Open Access to Research Papers/Bjork et al., 2013.pdf new file mode 100644 index 0000000..4f285ec Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Bjork et al., 2013.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Crowfoot, 2017.pdf b/reading_material/6 - Open Access to Research Papers/Crowfoot, 2017.pdf new file mode 100644 index 0000000..f52e948 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Crowfoot, 2017.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Desjardins-Proulx et al., 2013.PDF b/reading_material/6 - Open Access to Research Papers/Desjardins-Proulx et al., 2013.PDF new file mode 100644 index 0000000..c8a25bf Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Desjardins-Proulx et al., 2013.PDF differ diff --git a/reading_material/6 - Open Access to Research Papers/Ellers et al., 2017.pdf b/reading_material/6 - Open Access to Research Papers/Ellers et al., 2017.pdf new file mode 100644 index 0000000..c25cc0e Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Ellers et al., 2017.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Greshake, 2017.pdf b/reading_material/6 - Open Access to Research Papers/Greshake, 2017.pdf new file mode 100644 index 0000000..417404c Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Greshake, 2017.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Himmelstein et al., 2018.pdf b/reading_material/6 - Open Access to Research Papers/Himmelstein et al., 2018.pdf new file mode 100644 index 0000000..a3cc017 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Himmelstein et al., 2018.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Johnson, 2018.pdf b/reading_material/6 - Open Access to Research Papers/Johnson, 2018.pdf new file mode 100644 index 0000000..713ebc1 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Johnson, 2018.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Kohls and Mele, 2018.pdf b/reading_material/6 - Open Access to Research Papers/Kohls and Mele, 2018.pdf new file mode 100644 index 0000000..4e0c12e Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Kohls and Mele, 2018.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Kuchma, 2018.pdf b/reading_material/6 - Open Access to Research Papers/Kuchma, 2018.pdf new file mode 100644 index 0000000..d8339e3 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Kuchma, 2018.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Laakso and Bjork, 2016.pdf b/reading_material/6 - Open Access to Research Papers/Laakso and Bjork, 2016.pdf new file mode 100644 index 0000000..eccff17 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Laakso and Bjork, 2016.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Laakso et al., 2011.PDF b/reading_material/6 - Open Access to Research Papers/Laakso et al., 2011.PDF new file mode 100644 index 0000000..e235159 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Laakso et al., 2011.PDF differ diff --git a/reading_material/6 - Open Access to Research Papers/Lariviere et al., 2013.pdf b/reading_material/6 - Open Access to Research Papers/Lariviere et al., 2013.pdf new file mode 100644 index 0000000..e3a1734 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Lariviere et al., 2013.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/McKiernan et al., 2016.pdf b/reading_material/6 - Open Access to Research Papers/McKiernan et al., 2016.pdf new file mode 100644 index 0000000..96e7c52 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/McKiernan et al., 2016.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Neylon et al., 2017.pdf b/reading_material/6 - Open Access to Research Papers/Neylon et al., 2017.pdf new file mode 100644 index 0000000..ca74e57 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Neylon et al., 2017.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Piwowar et al., 2018.pdf b/reading_material/6 - Open Access to Research Papers/Piwowar et al., 2018.pdf new file mode 100644 index 0000000..ec697b7 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Piwowar et al., 2018.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Schimmer et al., 2015.pdf b/reading_material/6 - Open Access to Research Papers/Schimmer et al., 2015.pdf new file mode 100644 index 0000000..861fed7 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Schimmer et al., 2015.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Siler et al., 2018.pdf b/reading_material/6 - Open Access to Research Papers/Siler et al., 2018.pdf new file mode 100644 index 0000000..d415635 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Siler et al., 2018.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Solomon and Bjork, 2012.pdf b/reading_material/6 - Open Access to Research Papers/Solomon and Bjork, 2012.pdf new file mode 100644 index 0000000..28a99e7 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Solomon and Bjork, 2012.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Solomon et al., 2016.pdf b/reading_material/6 - Open Access to Research Papers/Solomon et al., 2016.pdf new file mode 100644 index 0000000..aa1b0ee Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Solomon et al., 2016.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Tennant et al., 2016.pdf b/reading_material/6 - Open Access to Research Papers/Tennant et al., 2016.pdf new file mode 100644 index 0000000..ab48a81 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Tennant et al., 2016.pdf differ diff --git a/reading_material/6 - Open Access to Research Papers/Tennant et al., 2018.docx b/reading_material/6 - Open Access to Research Papers/Tennant et al., 2018.docx new file mode 100644 index 0000000..9a8f040 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Tennant et al., 2018.docx differ diff --git a/reading_material/6 - Open Access to Research Papers/Toledo, 2017.pdf b/reading_material/6 - Open Access to Research Papers/Toledo, 2017.pdf new file mode 100644 index 0000000..70431f5 Binary files /dev/null and b/reading_material/6 - Open Access to Research Papers/Toledo, 2017.pdf differ diff --git a/reading_material/7 - Open Evaluation/Adie, 2014.pdf b/reading_material/7 - Open Evaluation/Adie, 2014.pdf new file mode 100644 index 0000000..2800b03 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Adie, 2014.pdf differ diff --git a/reading_material/7 - Open Evaluation/Allen et al., 2014.pdf b/reading_material/7 - Open Evaluation/Allen et al., 2014.pdf new file mode 100644 index 0000000..36c8b37 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Allen et al., 2014.pdf differ diff --git a/reading_material/7 - Open Evaluation/Binswanger, 2014.pdf b/reading_material/7 - Open Evaluation/Binswanger, 2014.pdf new file mode 100644 index 0000000..f448c31 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Binswanger, 2014.pdf differ diff --git a/reading_material/7 - Open Evaluation/Bornmann et al., 2010.PDF b/reading_material/7 - Open Evaluation/Bornmann et al., 2010.PDF new file mode 100644 index 0000000..a37f325 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Bornmann et al., 2010.PDF differ diff --git a/reading_material/7 - Open Evaluation/Brembs et al., 2013.pdf b/reading_material/7 - Open Evaluation/Brembs et al., 2013.pdf new file mode 100644 index 0000000..1cd28fb Binary files /dev/null and b/reading_material/7 - Open Evaluation/Brembs et al., 2013.pdf differ diff --git a/reading_material/7 - Open Evaluation/Brembs, 2018.pdf b/reading_material/7 - Open Evaluation/Brembs, 2018.pdf new file mode 100644 index 0000000..f52541f Binary files /dev/null and b/reading_material/7 - Open Evaluation/Brembs, 2018.pdf differ diff --git a/reading_material/7 - Open Evaluation/European Commission, 2017 (next-generation metrics).pdf b/reading_material/7 - Open Evaluation/European Commission, 2017 (next-generation metrics).pdf new file mode 100644 index 0000000..e7a3345 Binary files /dev/null and b/reading_material/7 - Open Evaluation/European Commission, 2017 (next-generation metrics).pdf differ diff --git a/reading_material/7 - Open Evaluation/European Commission, 2017.pdf b/reading_material/7 - Open Evaluation/European Commission, 2017.pdf new file mode 100644 index 0000000..7cc32d2 Binary files /dev/null and b/reading_material/7 - Open Evaluation/European Commission, 2017.pdf differ diff --git a/reading_material/7 - Open Evaluation/Ferretti et al., 2018.pdf b/reading_material/7 - Open Evaluation/Ferretti et al., 2018.pdf new file mode 100644 index 0000000..db493cb Binary files /dev/null and b/reading_material/7 - Open Evaluation/Ferretti et al., 2018.pdf differ diff --git a/reading_material/7 - Open Evaluation/Haustein, 2016.pdf b/reading_material/7 - Open Evaluation/Haustein, 2016.pdf new file mode 100644 index 0000000..b0e9d65 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Haustein, 2016.pdf differ diff --git a/reading_material/7 - Open Evaluation/Helmer et al., 2017.pdf b/reading_material/7 - Open Evaluation/Helmer et al., 2017.pdf new file mode 100644 index 0000000..db5a0bd Binary files /dev/null and b/reading_material/7 - Open Evaluation/Helmer et al., 2017.pdf differ diff --git a/reading_material/7 - Open Evaluation/Kidwell et al., 2016.PDF b/reading_material/7 - Open Evaluation/Kidwell et al., 2016.PDF new file mode 100644 index 0000000..d9b07bc Binary files /dev/null and b/reading_material/7 - Open Evaluation/Kidwell et al., 2016.PDF differ diff --git a/reading_material/7 - Open Evaluation/Lariviere and Sugimoto, 2018.pdf b/reading_material/7 - Open Evaluation/Lariviere and Sugimoto, 2018.pdf new file mode 100644 index 0000000..796a887 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Lariviere and Sugimoto, 2018.pdf differ diff --git a/reading_material/7 - Open Evaluation/McKiernan et al., 2016.pdf b/reading_material/7 - Open Evaluation/McKiernan et al., 2016.pdf new file mode 100644 index 0000000..96e7c52 Binary files /dev/null and b/reading_material/7 - Open Evaluation/McKiernan et al., 2016.pdf differ diff --git a/reading_material/7 - Open Evaluation/Miles et al., 2018.pdf b/reading_material/7 - Open Evaluation/Miles et al., 2018.pdf new file mode 100644 index 0000000..d3c1092 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Miles et al., 2018.pdf differ diff --git a/reading_material/7 - Open Evaluation/Moher et al., 2018.pdf b/reading_material/7 - Open Evaluation/Moher et al., 2018.pdf new file mode 100644 index 0000000..05cad2a Binary files /dev/null and b/reading_material/7 - Open Evaluation/Moher et al., 2018.pdf differ diff --git a/reading_material/7 - Open Evaluation/Moore et al., 2017.pdf b/reading_material/7 - Open Evaluation/Moore et al., 2017.pdf new file mode 100644 index 0000000..a4ef4ba Binary files /dev/null and b/reading_material/7 - Open Evaluation/Moore et al., 2017.pdf differ diff --git a/reading_material/7 - Open Evaluation/Nichols and Twidale, 2016.pdf b/reading_material/7 - Open Evaluation/Nichols and Twidale, 2016.pdf new file mode 100644 index 0000000..ab19682 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Nichols and Twidale, 2016.pdf differ diff --git a/reading_material/7 - Open Evaluation/Ross-Hellauer et al., 2017.pdf b/reading_material/7 - Open Evaluation/Ross-Hellauer et al., 2017.pdf new file mode 100644 index 0000000..6fe3d7c Binary files /dev/null and b/reading_material/7 - Open Evaluation/Ross-Hellauer et al., 2017.pdf differ diff --git a/reading_material/7 - Open Evaluation/Ross-Hellauer, 2017.pdf b/reading_material/7 - Open Evaluation/Ross-Hellauer, 2017.pdf new file mode 100644 index 0000000..d73a574 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Ross-Hellauer, 2017.pdf differ diff --git a/reading_material/7 - Open Evaluation/Schmidt et al., 2018.pdf b/reading_material/7 - Open Evaluation/Schmidt et al., 2018.pdf new file mode 100644 index 0000000..2c230a3 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Schmidt et al., 2018.pdf differ diff --git a/reading_material/7 - Open Evaluation/Seglen, 1997.pdf b/reading_material/7 - Open Evaluation/Seglen, 1997.pdf new file mode 100644 index 0000000..d2c2c79 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Seglen, 1997.pdf differ diff --git a/reading_material/7 - Open Evaluation/Smith et al., 2016.pdf b/reading_material/7 - Open Evaluation/Smith et al., 2016.pdf new file mode 100644 index 0000000..bd96f1d Binary files /dev/null and b/reading_material/7 - Open Evaluation/Smith et al., 2016.pdf differ diff --git a/reading_material/7 - Open Evaluation/Tennant et al., 2017.pdf b/reading_material/7 - Open Evaluation/Tennant et al., 2017.pdf new file mode 100644 index 0000000..9ae088b Binary files /dev/null and b/reading_material/7 - Open Evaluation/Tennant et al., 2017.pdf differ diff --git a/reading_material/7 - Open Evaluation/Tomkins et al., 2017.pdf b/reading_material/7 - Open Evaluation/Tomkins et al., 2017.pdf new file mode 100644 index 0000000..2dc2237 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Tomkins et al., 2017.pdf differ diff --git a/reading_material/7 - Open Evaluation/Walsh et al., 2010.pdf b/reading_material/7 - Open Evaluation/Walsh et al., 2010.pdf new file mode 100644 index 0000000..8c0a4f3 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Walsh et al., 2010.pdf differ diff --git a/reading_material/7 - Open Evaluation/Ware, 2016.pdf b/reading_material/7 - Open Evaluation/Ware, 2016.pdf new file mode 100644 index 0000000..85a3709 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Ware, 2016.pdf differ diff --git a/reading_material/7 - Open Evaluation/Wicherts, 2016.PDF b/reading_material/7 - Open Evaluation/Wicherts, 2016.PDF new file mode 100644 index 0000000..65846b3 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Wicherts, 2016.PDF differ diff --git a/reading_material/7 - Open Evaluation/Wilsdon et al., 2015.pdf b/reading_material/7 - Open Evaluation/Wilsdon et al., 2015.pdf new file mode 100644 index 0000000..43b5c51 Binary files /dev/null and b/reading_material/7 - Open Evaluation/Wilsdon et al., 2015.pdf differ diff --git a/reading_material/7 - Open Evaluation/van Rooyen et al., 1999.pdf b/reading_material/7 - Open Evaluation/van Rooyen et al., 1999.pdf new file mode 100644 index 0000000..25842a2 Binary files /dev/null and b/reading_material/7 - Open Evaluation/van Rooyen et al., 1999.pdf differ diff --git a/reading_material/7 - Open Evaluation/van Rooyen et al., 2010.pdf b/reading_material/7 - Open Evaluation/van Rooyen et al., 2010.pdf new file mode 100644 index 0000000..692be54 Binary files /dev/null and b/reading_material/7 - Open Evaluation/van Rooyen et al., 2010.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Austrian Citizen Science Conference 2017.pdf b/reading_material/8 - Public Engagement with Science/Austrian Citizen Science Conference 2017.pdf new file mode 100644 index 0000000..fbe161a Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Austrian Citizen Science Conference 2017.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Ballard et al., 2017.pdf b/reading_material/8 - Public Engagement with Science/Ballard et al., 2017.pdf new file mode 100644 index 0000000..7a42300 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Ballard et al., 2017.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Baram-Tsabari and Osborne, 2015.pdf b/reading_material/8 - Public Engagement with Science/Baram-Tsabari and Osborne, 2015.pdf new file mode 100644 index 0000000..cbf90ae Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Baram-Tsabari and Osborne, 2015.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Bik and Goldstein, 2013.PDF b/reading_material/8 - Public Engagement with Science/Bik and Goldstein, 2013.PDF new file mode 100644 index 0000000..d1d7bfd Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Bik and Goldstein, 2013.PDF differ diff --git a/reading_material/8 - Public Engagement with Science/Chandler et al., 2017.pdf b/reading_material/8 - Public Engagement with Science/Chandler et al., 2017.pdf new file mode 100644 index 0000000..144f674 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Chandler et al., 2017.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Eitzel et al., 2017.pdf b/reading_material/8 - Public Engagement with Science/Eitzel et al., 2017.pdf new file mode 100644 index 0000000..b892e14 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Eitzel et al., 2017.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Ekins and Perlstein, 2014.PDF b/reading_material/8 - Public Engagement with Science/Ekins and Perlstein, 2014.PDF new file mode 100644 index 0000000..c75ad56 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Ekins and Perlstein, 2014.PDF differ diff --git a/reading_material/8 - Public Engagement with Science/Franzoni and Sauermann, 2014.pdf b/reading_material/8 - Public Engagement with Science/Franzoni and Sauermann, 2014.pdf new file mode 100644 index 0000000..dd8263a Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Franzoni and Sauermann, 2014.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Fulltext.doc b/reading_material/8 - Public Engagement with Science/Fulltext.doc new file mode 100644 index 0000000..77c54b0 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Fulltext.doc differ diff --git a/reading_material/8 - Public Engagement with Science/Kuehne and Olden, 2015.pdf b/reading_material/8 - Public Engagement with Science/Kuehne and Olden, 2015.pdf new file mode 100644 index 0000000..201fff2 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Kuehne and Olden, 2015.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Lewenstein, 2015.pdf b/reading_material/8 - Public Engagement with Science/Lewenstein, 2015.pdf new file mode 100644 index 0000000..91e23e4 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Lewenstein, 2015.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Lukyanenko et al., 2016.pdf b/reading_material/8 - Public Engagement with Science/Lukyanenko et al., 2016.pdf new file mode 100644 index 0000000..99adf6d Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Lukyanenko et al., 2016.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Newman et al., 2017.pdf b/reading_material/8 - Public Engagement with Science/Newman et al., 2017.pdf new file mode 100644 index 0000000..adc7ab7 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Newman et al., 2017.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Reisch, 2014.pdf b/reading_material/8 - Public Engagement with Science/Reisch, 2014.pdf new file mode 100644 index 0000000..2b5c172 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Reisch, 2014.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Scassa and Chung, 2015.pdf b/reading_material/8 - Public Engagement with Science/Scassa and Chung, 2015.pdf new file mode 100644 index 0000000..3e5a8a1 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Scassa and Chung, 2015.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Scheliga et al., 2016.pdf b/reading_material/8 - Public Engagement with Science/Scheliga et al., 2016.pdf new file mode 100644 index 0000000..08dad01 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Scheliga et al., 2016.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Scheufele, 2014.pdf b/reading_material/8 - Public Engagement with Science/Scheufele, 2014.pdf new file mode 100644 index 0000000..bde55e5 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Scheufele, 2014.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Stilgoe et al., 2014.pdf b/reading_material/8 - Public Engagement with Science/Stilgoe et al., 2014.pdf new file mode 100644 index 0000000..516149f Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Stilgoe et al., 2014.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Theobald et al., 2017.pdf b/reading_material/8 - Public Engagement with Science/Theobald et al., 2017.pdf new file mode 100644 index 0000000..7ba0156 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Theobald et al., 2017.pdf differ diff --git a/reading_material/8 - Public Engagement with Science/Trench, 2008.pdf b/reading_material/8 - Public Engagement with Science/Trench, 2008.pdf new file mode 100644 index 0000000..a403093 Binary files /dev/null and b/reading_material/8 - Public Engagement with Science/Trench, 2008.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Atenas and Havemann, 2015.pdf b/reading_material/9 - Open Educational Resources/Atenas and Havemann, 2015.pdf new file mode 100644 index 0000000..957fd69 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Atenas and Havemann, 2015.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Atenas et al., 2015.pdf b/reading_material/9 - Open Educational Resources/Atenas et al., 2015.pdf new file mode 100644 index 0000000..d33f450 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Atenas et al., 2015.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Camilleri et al., 2014.pdf b/reading_material/9 - Open Educational Resources/Camilleri et al., 2014.pdf new file mode 100644 index 0000000..f7c899d Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Camilleri et al., 2014.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Caswell et al., 2008.pdf b/reading_material/9 - Open Educational Resources/Caswell et al., 2008.pdf new file mode 100644 index 0000000..e7fdc54 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Caswell et al., 2008.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Commonwealth of Learning, 2015.pdf b/reading_material/9 - Open Educational Resources/Commonwealth of Learning, 2015.pdf new file mode 100644 index 0000000..b5e9a3d Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Commonwealth of Learning, 2015.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Conole, 2016.pdf b/reading_material/9 - Open Educational Resources/Conole, 2016.pdf new file mode 100644 index 0000000..8386bbf Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Conole, 2016.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Czerniewicz et al., 2017.pdf b/reading_material/9 - Open Educational Resources/Czerniewicz et al., 2017.pdf new file mode 100644 index 0000000..e638b26 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Czerniewicz et al., 2017.pdf differ diff --git a/reading_material/9 - Open Educational Resources/DeVries, 2013.pdf b/reading_material/9 - Open Educational Resources/DeVries, 2013.pdf new file mode 100644 index 0000000..b5feb9c Binary files /dev/null and b/reading_material/9 - Open Educational Resources/DeVries, 2013.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Downes, 2007.pdf b/reading_material/9 - Open Educational Resources/Downes, 2007.pdf new file mode 100644 index 0000000..9e93d96 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Downes, 2007.pdf differ diff --git a/reading_material/9 - Open Educational Resources/European Commission, 2017.pdf b/reading_material/9 - Open Educational Resources/European Commission, 2017.pdf new file mode 100644 index 0000000..bda6937 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/European Commission, 2017.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Goodova et al., 2015.pdf b/reading_material/9 - Open Educational Resources/Goodova et al., 2015.pdf new file mode 100644 index 0000000..3e18573 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Goodova et al., 2015.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Holotescu et al., 2015.pdf b/reading_material/9 - Open Educational Resources/Holotescu et al., 2015.pdf new file mode 100644 index 0000000..b56040e Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Holotescu et al., 2015.pdf differ diff --git "a/reading_material/9 - Open Educational Resources/Hyl\303\251n, 2005.pdf" "b/reading_material/9 - Open Educational Resources/Hyl\303\251n, 2005.pdf" new file mode 100644 index 0000000..544c890 Binary files /dev/null and "b/reading_material/9 - Open Educational Resources/Hyl\303\251n, 2005.pdf" differ diff --git a/reading_material/9 - Open Educational Resources/Islim et al., 2016.pdf b/reading_material/9 - Open Educational Resources/Islim et al., 2016.pdf new file mode 100644 index 0000000..f203bb5 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Islim et al., 2016.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Kaosaiyaporn et al., 2015.pdf b/reading_material/9 - Open Educational Resources/Kaosaiyaporn et al., 2015.pdf new file mode 100644 index 0000000..cb84977 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Kaosaiyaporn et al., 2015.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Marcus-Quinn and Diggins, 2013.pdf b/reading_material/9 - Open Educational Resources/Marcus-Quinn and Diggins, 2013.pdf new file mode 100644 index 0000000..e7d06af Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Marcus-Quinn and Diggins, 2013.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Meirani, 2015.pdf b/reading_material/9 - Open Educational Resources/Meirani, 2015.pdf new file mode 100644 index 0000000..f869e81 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Meirani, 2015.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Miao et al., 2016.pdf b/reading_material/9 - Open Educational Resources/Miao et al., 2016.pdf new file mode 100644 index 0000000..68d76b0 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Miao et al., 2016.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Nail et al., 2015.pdf b/reading_material/9 - Open Educational Resources/Nail et al., 2015.pdf new file mode 100644 index 0000000..8cc2654 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Nail et al., 2015.pdf differ diff --git a/reading_material/9 - Open Educational Resources/OECD, 2007.pdf b/reading_material/9 - Open Educational Resources/OECD, 2007.pdf new file mode 100644 index 0000000..488dca1 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/OECD, 2007.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Roeder et al., 2017.pdf b/reading_material/9 - Open Educational Resources/Roeder et al., 2017.pdf new file mode 100644 index 0000000..49b9731 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Roeder et al., 2017.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Salem Jr., 2017.pdf b/reading_material/9 - Open Educational Resources/Salem Jr., 2017.pdf new file mode 100644 index 0000000..6183358 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Salem Jr., 2017.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Shelton et al., 2010.pdf b/reading_material/9 - Open Educational Resources/Shelton et al., 2010.pdf new file mode 100644 index 0000000..9c6ab88 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Shelton et al., 2010.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Shu-Hsiang et al., 2015.pdf b/reading_material/9 - Open Educational Resources/Shu-Hsiang et al., 2015.pdf new file mode 100644 index 0000000..ab4da6c Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Shu-Hsiang et al., 2015.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Weiland, 2015.pdf b/reading_material/9 - Open Educational Resources/Weiland, 2015.pdf new file mode 100644 index 0000000..28007d2 Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Weiland, 2015.pdf differ diff --git a/reading_material/9 - Open Educational Resources/Yuan and Recker, 2015.pdf b/reading_material/9 - Open Educational Resources/Yuan and Recker, 2015.pdf new file mode 100644 index 0000000..489fa8d Binary files /dev/null and b/reading_material/9 - Open Educational Resources/Yuan and Recker, 2015.pdf differ diff --git a/Roadmap v.1.1.pdf b/roadmap_v.1.1.pdf similarity index 100% rename from Roadmap v.1.1.pdf rename to roadmap_v.1.1.pdf diff --git a/script_intro.md b/script_intro.md new file mode 100644 index 0000000..248ea61 --- /dev/null +++ b/script_intro.md @@ -0,0 +1,57 @@ +--- +title: "script_intro" +output: html_document +--- + +Script + +Course title: Open Science MOOC + +Video title: Welcome to the Open Science MOOC + +Presenters: Jon Tennant, Lisa Hehnke, Valentina Goglio, Bianca Kramer, Paola Masuzzo, Chris Hartgerink, Jo Havemann, Julien Colomb, Christopher Madan, Ivo Grigorov, Ricardo Hartley Belmar, Rutger Vos, Tobias Steiner, Nicolas Schmelling. + +# Background + +The intention of this video is to explain to learners what the Open Science MOOC is, how it came about, and why it might be useful for them. + +# Script + +Ricardo: Our world is facing huge challenges. From preserving life on land and combating the threat of climate change, to providing food and energy security for a rapidly growing global population. It has never been more important in all of human history that we make sure we are using scientific research to help to mitigate these major challenges of our time. + +Lisa: But are we using scientific research as effectively as we can? The simple answer is, no, we are not. Most scientific research remains inaccessible to most people on this planet. Researchers selectively communicate their results, in the hope of gaining prestige by publishing in the top scientific journals. Data, code, materials, and other critical elements of the research process remain locked away on harddrives, never to see the light of day. We are still largely using analog methods of communication within the digital age, where we have the enormous power of Web-based technologies to share and collaborate on knowledge like never before. But we are not using this power. + +Erzsebet: Enter Open Science. Open Science means many things to different people. Broadly, it can be thought of as transparent and collaborative ways to produce and share scholarly knowledge, while making the scientific process more rigorous and effective. + +Jo: While 'openness' has always been a fundamental part of research, ideally at least, Open Science has really taken off over the last few years, all around the world. We now have new expectations about how to work with each other to produce knowledge - more transparent, more collaborative, more continuous - and how to share that with the wider society. + +Jon: This is where the Open Science MOOC comes in. We are a mission-driven project to help make "Open" the default setting for all global research. We want to help create a welcoming and supporting community, with good tools, teachers, and role-models, and built upon a solid values-based foundation of freedom and equitable access to research. Therefore, we see Open Science as a goal: broad adoption of good scientific practices as a fundamental and essential part of the research process. + +Valentina: How are we working to achieve this? MOOC traditionally stands for 'Massively Open Online Course'. While the core of what we do is about producing learning materials in a course-like structure, we feel the C in MOOC better stands for community. This more accurately reflects the more peer-to-peer and engaging style of learning with each other that we try to promote. + +Ivo: What we are building then is a community of practice. We should be training ourselves to adapt to this new digital age of research. We are doing this through sustained community engagement across all research disciplines. We are constantly evolving and adapting, changing our practices and mindsets, and challenging the flaws of the older research system. + +Julien: We have a vision of the future, where research is defined by the values inherent to Open Science, such as freedom and equity. Science should be a freely available, public good. It should be rigorous and reproducible. It should be open to all. This is not 'Open' science, this is just good science. But we cannot achieve this alone. We need everyone to be working together if we are going to help to solve the major challenges that we face. We need science to work better for society. + +Bianca: Our mission is to help the global research community to do this. How? We will help to equip them with new knowledge and skills to save time and effort, making their research more efficient. By solving problems better together, we can advance research faster, and put it to better use. + +Rutger: The content and material we will provide will hopefully help shape an ecosystem of researchers more and more attentive to research equity, democratization, inclusiveness and open collaboration. By solving problems better together, we can advance research faster, and put it to better use. + +Tobias: Everything we produce is freely available, and openly licensed for re-use. It is all built in the open by a passionate community of developers. We place no restrictions on who can use what we produce, or who can join us in building it. + +Paola (and anyone else you can get locally to shout this?): Researchers can be world-changing heroes. [[No capes!](https://www.youtube.com/watch?v=RoFW2msxP6Q)] We will give them the power to achieve that. + + +# Licensing statement + +I/we grant the Open Science MOOC the right to re-use the content that I/we provide for the project. Specifically, I/we give permission to: + + * Use the material for educational purposes for the project + + * Publish the educational videos to the MOOC platform(s), and share any relevant ones on social media + + * Release the content under a Creative Commons CC BY 4.0 International license + +Signed: + +[please add names here] diff --git a/shuttleworth_funded.jpg b/shuttleworth_funded.jpg new file mode 100644 index 0000000..33a2b2b Binary files /dev/null and b/shuttleworth_funded.jpg differ