Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Nov 14, 2011

• fixed "No free spider slots" bug when calling fetch() from scrapy shell

  pablohoffman

  committed
  6d13de4

  Copy full SHA for 6d13de4

• improve handling of KeyError exception when creating spiders in spider manager. closes issue 49

  pablohoffman

  committed
  d37a788

  Copy full SHA for d37a788

• ignore meta-refresh redirects embedded in <script> tags. related to issue 18

  pablohoffman

  committed
  36df87b

  Copy full SHA for 36df87b

• ignore meta-refresh redirect when embedded inside <noscript> tag. closes issue 18

  pablohoffman

  committed
  ec1ef02

  Copy full SHA for ec1ef02

Commits on Nov 8, 2011

• fixed bug in MEMUSAGE_NOTIFY_MAIL setting

  pablohoffman

  committed
  6cc40dc

  Copy full SHA for 6cc40dc

Commits on Oct 28, 2011

• added support for ajax crawleable urls

  pablohoffman

  committed
  37ad4f8

  Copy full SHA for 37ad4f8

Commits on Oct 25, 2011

• ubuntu repos: added support for oneiric release

  pablohoffman

  committed
  992af8d

  Copy full SHA for 992af8d

• Do not raise PartialDownloadError if Content-Length doesn't match the body size. This fixes the error reported in: https://groups.google.com/d/topic/scrapy-users/FQ25O3KPQuU/discussion

  pablohoffman

  committed
  f4821a1

  Copy full SHA for f4821a1

• removed deprecation warning for spider.download_timeout attribute

  pablohoffman

  committed
  c085f81

  Copy full SHA for c085f81

• fixed PickeItemExporter bug, added unittest, and added pickle to suported feed exports formats

  pablohoffman

  committed
  c38c49d

  Copy full SHA for c38c49d

Commits on Oct 23, 2011

• made scrapyd doc more version agnostic

  pablohoffman

  committed
  8bdf288

  Copy full SHA for 8bdf288

• added support for using '-' in scrapy crawl -o, to dump items to standard output

  pablohoffman

  committed
  64b8e26

  Copy full SHA for 64b8e26

• feed exports: removed dependency on file.tell() method, so that stdout output works

  pablohoffman

  committed
  028bf33

  Copy full SHA for 028bf33

• changed feed exports storage api so that file/stdio outputs directly without using a temporary file

  pablohoffman

  committed
  10ced29

  Copy full SHA for 10ced29

Commits on Oct 22, 2011

• added -o option to scrapy crawl, a convenient shortcut for using feed exports

  pablohoffman

  committed
  ade5efd

  Copy full SHA for ade5efd

• remove deprecation warning for spider.user_agent attribute

  pablohoffman

  committed
  13cd9a1

  Copy full SHA for 13cd9a1

Commits on Sep 26, 2011

• remove usage of assertLess() which is only available on python 2.7+

  pablohoffman

  committed
  43b79af

  Copy full SHA for 43b79af

Commits on Sep 25, 2011

• updated documentation to remove references to old issue tracker and mercurial repos

  pablohoffman

  committed
  431441c

  Copy full SHA for 431441c

Commits on Sep 23, 2011

• updated documentation about DEPTH_PRIORITY and DFO/BFO crawls

  pablohoffman

  committed
  ce03ccd

  Copy full SHA for ce03ccd

• Some changes to persistent scheduler after some initial usage feedback:

  Show description for f850a44
  pablohoffman

  committed
  f850a44

  Copy full SHA for f850a44

• make SpiderState extension always available, regardless of whether there is a job dir, and make sure it always set spider.state attribute, for consistency between in-memory and on-disk runs

  pablohoffman

  committed
  cfddc31

  Copy full SHA for cfddc31

Commits on Sep 22, 2011

• Merge pull request #42 from noplay/deploy-git-version

  Show description for 2559f21
  pablohoffman

  committed
  2559f21

  Copy full SHA for 2559f21

Commits on Sep 21, 2011

• scrapy deploy support git version

  julien-duponchelle

  committed
  b7c4363

  Copy full SHA for b7c4363

• Merge pull request #39 from kalessin/master

  Show description for 0f0783e
  pablohoffman

  committed
  0f0783e

  Copy full SHA for 0f0783e

• fix autothrottle extension for working with new downloader

  kalessin

  committed
  dcc5020

  Copy full SHA for dcc5020

Commits on Sep 18, 2011

• moved rpm-install.sh to extras/

  pablohoffman

  committed
  b003687

  Copy full SHA for b003687

• Merge branch '0.12'

  dangra

  committed
  a5c1500

  Copy full SHA for a5c1500

• migrate hgignore to gitignore

  dangra

  committed
  fac5e5e

  Copy full SHA for fac5e5e

Commits on Sep 15, 2011

• new mechanism to override settings in scrapy commands before the Crawler object is available

  pablohoffman

  committed
  d788ba8

  Copy full SHA for d788ba8

Commits on Sep 14, 2011

• Merge pull request #38 from kalessin/master

  Show description for 77ffaa5
  pablohoffman

  committed
  77ffaa5

  Copy full SHA for 77ffaa5

• _extract_links requires extra parameter base_url in order to avoid

  Show description for 509b05d
  kalessin

  committed
  509b05d

  Copy full SHA for 509b05d

• removed documentation header notifying about other documentation versions, as that's provided by readthedocs already

  pablohoffman

  committed
  2fcb709

  Copy full SHA for 2fcb709

• removed documentation header notifying about other documentation versions, as that's provided by readthedocs already

  pablohoffman

  committed
  ab1c9cf

  Copy full SHA for ab1c9cf

Commits on Sep 11, 2011

• added support for generating version from git revision

  pablohoffman

  committed
  3b00b9c

  Copy full SHA for 3b00b9c

• added tests for SpiderState extension

  pablohoffman

  committed
  43ae7bd

  Copy full SHA for 43ae7bd

Pagination

Previous

Next