Skip to content

Commit 879fe88

Browse files
committed
Add comment + hide header spoofing.
1 parent cabbdd1 commit 879fe88

File tree

1 file changed

+12
-4
lines changed

1 file changed

+12
-4
lines changed

content/tutorial-deep-learning-on-mnist.md

+12-4
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,17 @@ data_sources = {
7272
**2.** Load the data. First check if the data is stored locally; if not, then
7373
download it.
7474

75+
```{code-cell} ipython3
76+
:tags: [remove-cell]
77+
78+
# Use responsibly! When running notebooks locally, be sure to keep local
79+
# copies of the datasets to prevent unnecessary server requests
80+
headers = {
81+
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0"
82+
}
83+
request_opts = {"headers": headers}
84+
```
85+
7586
```{code-cell} ipython3
7687
import requests
7788
import os
@@ -80,15 +91,12 @@ data_dir = "../_data"
8091
os.makedirs(data_dir, exist_ok=True)
8192
8293
base_url = "http://yann.lecun.com/exdb/mnist/"
83-
headers = {
84-
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20100101 Firefox/10.0"
85-
}
8694
8795
for fname in data_sources.values():
8896
fpath = os.path.join(data_dir, fname)
8997
if not os.path.exists(fpath):
9098
print("Downloading file: " + fname)
91-
resp = requests.get(base_url + fname, headers=headers, stream=True)
99+
resp = requests.get(base_url + fname, stream=True, **request_opts)
92100
with open(fpath, "wb") as fh:
93101
for chunk in resp.iter_content(chunk_size=128):
94102
fh.write(chunk)

0 commit comments

Comments
 (0)