@@ -50,6 +50,11 @@ This 1.1 release brings a lot of interesting features and bug fixes:
50
50
``ImagesPipeline ``), the default ACL policy is now "private" instead
51
51
of "public" **Warning: backwards incompatible! **.
52
52
You can use :setting: `FILES_STORE_S3_ACL ` to change it.
53
+ - We've reimplemented ``canonicalize_url() `` for more correct output,
54
+ especially for URLs with non-ASCII characters (:issue: `1947 `).
55
+ This could change link extractors output compared to previous scrapy versions.
56
+ This may also invalidate some cache entries you could still have from pre-1.1 runs.
57
+ **Warning: backwards incompatible! **.
53
58
54
59
Keep reading for more details on other improvements and bug fixes.
55
60
@@ -72,7 +77,6 @@ some limitations in Python 3:
72
77
- FTP download handler is not supported (non-Python 3 ported Twisted
73
78
dependency)
74
79
- Telnet is not supported (non-Python 3 ported Twisted dependency)
75
- - Scrapy has problems handling non-ASCII URLs in Python 3
76
80
77
81
Additional New Features and Enhancements
78
82
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -112,11 +116,18 @@ Additional New Features and Enhancements
112
116
setting is set (:issue: `1723 `, :issue: `1725 `).
113
117
- Added method ``ExecutionEngine.close `` (:issue: `1423 `).
114
118
- Added method ``CrawlerRunner.create_crawler `` (:issue: `1528 `).
119
+ - Scheduler priority queue can now be customized via
120
+ :setting: `SCHEDULER_PRIORITY_QUEUE ` (:issue: `1822 `).
121
+ - ``.pps `` links are now ignored by default in link extractors (:issue: `1835 `).
122
+ - temporary data folder for FTP and S3 feed storages can be customized
123
+ using a new :setting: `FEED_TEMPDIR ` setting (:issue: `1847 `).
124
+ - ``FilesPipeline `` and ``ImagesPipeline `` settings are now instance attributes
125
+ instead of class attributes, enabling spider-specific behaviors (:issue: `1891 `).
115
126
- Tons of documentation updates and related fixes (:issue: `1291 `, :issue: `1302 `,
116
127
:issue: `1335 `, :issue: `1683 `, :issue: `1660 `, :issue: `1642 `, :issue: `1721 `,
117
- :issue: `1727 `).
128
+ :issue: `1727 `, :issue: ` 1879 ` ).
118
129
- Other refactoring, optimizations and cleanup (:issue: `1476 `, :issue: `1481 `,
119
- :issue: `1477 `, :issue: `1315 `, :issue: `1290 ` and :issue: `1750 `).
130
+ :issue: `1477 `, :issue: `1315 `, :issue: `1290 `, :issue: `1750 `, :issue: ` 1881 `).
120
131
121
132
.. _`Code of Conduct` : https://github.com/scrapy/scrapy/blob/master/CODE_OF_CONDUCT.md
122
133
@@ -126,6 +137,8 @@ Deprecations and Removals
126
137
127
138
- Added ``to_bytes `` and ``to_unicode ``, deprecated ``str_to_unicode `` and
128
139
``unicode_to_str `` functions (:issue: `778 `).
140
+ - ``binary_is_text `` is introduced, to replace use of ``isbinarytext ``
141
+ (but with inverse return value) (:issue: `1851 `)
129
142
- The ``optional_features `` set has been removed (:issue: `1359 `).
130
143
- The ``--lsprof `` command line option has been removed (:issue: `1689 `).
131
144
**Warning: backward incompatible **, but doesn't break user code.
@@ -171,6 +184,12 @@ Bugfixes
171
184
- Various logging related fixes (:issue: `1294 `, :issue: `1419 `, :issue: `1263 `,
172
185
:issue: `1624 `, :issue: `1654 `, :issue: `1722 `, :issue: `1726 ` and :issue: `1303 `).
173
186
- Fixed bug in ``utils.template.render_templatefile() `` (:issue: `1212 `).
187
+ - Fixed bug with filestorage HTTP cache checking wrong modified time (:issue: `1875 `).
188
+ - ``RetryMiddleware `` is now robust to non-standard HTTP status codes
189
+ (:issue: `1857 `).
190
+ - sitemaps extraction from ``robots.txt `` is now case-insensitive (:issue: `1902 `).
191
+ - HTTPS+CONNECT tunnels could get mixed up when using multiple proxies
192
+ to same remote host (:issue: `1912 `).
174
193
175
194
176
195
1.0.5 (2016-02-04)
0 commit comments