File tree Expand file tree Collapse file tree 1 file changed +7
-7
lines changed Expand file tree Collapse file tree 1 file changed +7
-7
lines changed Original file line number Diff line number Diff line change @@ -33,18 +33,18 @@ server) is, and this extension builds on that premise.
33
33
Throttling algorithm
34
34
====================
35
35
36
- This adjusts download delays and concurrency based on the following rules:
36
+ This adjusts download delays based on the following rules:
37
37
38
- 1. spiders always start with one concurrent request and a download delay of
39
- :setting: `AUTOTHROTTLE_START_DELAY `
38
+ 1. spiders always start with a download delay of
39
+ :setting: `AUTOTHROTTLE_START_DELAY `;
40
40
2. when a response is received, the download delay is adjusted to the
41
41
average of previous download delay and the latency of the response.
42
42
43
43
.. note :: The AutoThrottle extension honours the standard Scrapy settings for
44
- concurrency and delay. This means that it will never set a download delay
45
- lower than :setting: `DOWNLOAD_DELAY ` or a concurrency higher than
46
- :setting: `CONCURRENT_REQUESTS_PER_DOMAIN `
47
- (or :setting: ` CONCURRENT_REQUESTS_PER_IP `, depending on which one you use) .
44
+ concurrency and delay. This means that it will respect
45
+ :setting: `CONCURRENT_REQUESTS_PER_DOMAIN ` and
46
+ :setting: `CONCURRENT_REQUESTS_PER_IP ` options and
47
+ never set a download delay lower than :setting: ` DOWNLOAD_DELAY ` .
48
48
49
49
Settings
50
50
========
You can’t perform that action at this time.
0 commit comments