From d63b38c92ceb224f61cf383207ce922096bde646 Mon Sep 17 00:00:00 2001 From: Daryl Van Dyke Date: Wed, 18 Apr 2018 17:25:50 -0700 Subject: [PATCH 1/3] added Cann --- 03B_Layers_CannGrows.ipynb | 2419 ++++++++++++++++++++++++++++++++++++ 1 file changed, 2419 insertions(+) create mode 100644 03B_Layers_CannGrows.ipynb diff --git a/03B_Layers_CannGrows.ipynb b/03B_Layers_CannGrows.ipynb new file mode 100644 index 0000000..4e2f4b1 --- /dev/null +++ b/03B_Layers_CannGrows.ipynb @@ -0,0 +1,2419 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# TensorFlow Tutorial #03-B\n", + "# Layers API\n", + "\n", + "by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n", + "/ [GitHub](https://github.com/Hvass-Labs/TensorFlow-Tutorials) / [Videos on YouTube](https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Introduction\n", + "\n", + "It is important to use a builder API when constructing Neural Networks in TensorFlow because it makes it easier to implement and modify the source-code. This also lowers the risk of bugs.\n", + "\n", + "Many of the other tutorials used the TensorFlow builder API called PrettyTensor for easy construction of Neural Networks. But there are several other builder APIs available for TensorFlow. PrettyTensor was used in these tutorials, because at the time in mid-2016, PrettyTensor was the most complete and polished builder API available for TensorFlow. But PrettyTensor is only developed by a single person working at Google and although it has some unique and elegant features, it is possible that it may become deprecated in the future.\n", + "\n", + "This tutorial is about a small builder API that has recently been added to TensorFlow version 1.1. It is simply called *Layers* or the *Layers API* or by its Python name `tf.layers`. This builder API is automatically installed as part of TensorFlow, so you no longer have to install a separate Python package as was needed with PrettyTensor.\n", + "\n", + "This tutorial is very similar to Tutorial #03 on PrettyTensor and shows how to implement the same Convolutional Neural Network using the Layers API. It is recommended that you are familiar with Tutorial #02 on Convolutional Neural Networks." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Flowchart" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following chart shows roughly how the data flows in the Convolutional Neural Network that is implemented below. See Tutorial #02 for a more detailed description of convolution." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![Flowchart](images/02_network_flowchart.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The input image is processed in the first convolutional layer using the filter-weights. This results in 16 new images, one for each filter in the convolutional layer. The images are also down-sampled using max-pooling so the image resolution is decreased from 28x28 to 14x14.\n", + "\n", + "These 16 smaller images are then processed in the second convolutional layer. We need filter-weights for each of these 16 channels, and we need filter-weights for each output channel of this layer. There are 36 output channels so there are a total of 16 x 36 = 576 filters in the second convolutional layer. The resulting images are also down-sampled using max-pooling to 7x7 pixels.\n", + "\n", + "The output of the second convolutional layer is 36 images of 7x7 pixels each. These are then flattened to a single vector of length 7 x 7 x 36 = 1764, which is used as the input to a fully-connected layer with 128 neurons (or elements). This feeds into another fully-connected layer with 10 neurons, one for each of the classes, which is used to determine the class of the image, that is, which number is depicted in the image.\n", + "\n", + "The convolutional filters are initially chosen at random, so the classification is done randomly. The error between the predicted and true class of the input image is measured as the so-called cross-entropy. The optimizer then automatically propagates this error back through the Convolutional Network using the chain-rule of differentiation and updates the filter-weights so as to improve the classification error. This is done iteratively thousands of times until the classification error is sufficiently low.\n", + "\n", + "These particular filter-weights and intermediate images are the results of one optimization run and may look different if you re-run this Notebook.\n", + "\n", + "Note that the computation in TensorFlow is actually done on a batch of images instead of a single image, which makes the computation more efficient. This means the flowchart actually has one more data-dimension when implemented in TensorFlow." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Imports" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "import matplotlib.pyplot as plt\n", + "import tensorflow as tf\n", + "import numpy as np\n", + "from sklearn.metrics import confusion_matrix\n", + "import math\n", + "import os\n", + "from itertools import islice" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This was developed using Python 3.6 and TensorFlow version:" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'1.7.0'" + ] + }, + "execution_count": 85, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "tf.__version__" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "metadata": {}, + "outputs": [], + "source": [ + "# Class for Image stuff\n", + "# `take` from :\n", + "#https://stackoverflow.com/questions/7971618/python-return-first-n-keyvalue-pairs-from-dict\n", + "from itertools import islice\n", + "\n", + "def take(n, iterable):\n", + " \"Return first n items of the iterable as a list\"\n", + " return list(islice(iterable, n))\n", + "\n", + "class TF_ImageDataset:\n", + " \"\"\" A class for TF Image Datasets: listImagePaths, listLabels , datasetName = 'generic' \"\"\"\n", + " def __init__(self,listImagePaths, listLabels, datasetName):\n", + " if len(listImagePaths) != len(listLabels):\n", + " print(\"BAD BAD\")\n", + " \n", + " self.dictDS = {}\n", + " self.datasetName = datasetName\n", + " self.listImagePaths = listImagePaths\n", + " self.listLabels = listLabels\n", + "\n", + " for i in range(len(listImagePaths)):\n", + " key = listImagePaths[i]\n", + " val = listLabels[i]\n", + " self.dictDS[key] = val\n", + " \n", + " def summarize(self,topN = 5):\n", + " print(\"summary :\",self.datasetName)\n", + " print(\"length of listImagePaths: \", len(self.listImagePaths))\n", + " print(\"length of listLabels: \", len(self.listLabels))\n", + " print(take(topN, self.dictDS.items() ))\n", + " \n", + " def __add__(self, otherClassImgDS, datasetName = \"merged\"):\n", + " newDict = {**self.dictDS, **otherClassImgDS.dictDS}\n", + " outKeys = []\n", + " outVals = []\n", + " for key in newDict:\n", + " outKeys.append(key)\n", + " outVals.append(newDict[key])\n", + " return TF_ImageDataset(outKeys, outVals, datasetName)\n", + " \n", + " " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load Data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Data are loaded in multiple steps. \n", + "- 1 \n", + "\n", + "Requires 2 lists - pathNames to the JPGs, and integer coding of Presence/Absence (1/0)" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "summary : RC\n", + "length of listImagePaths: 2006\n", + "length of listLabels: 2006\n", + "[('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aaajtvjlhod.jpg', 1), ('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aaaogunusqn.jpg', 1), ('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aabwgezsbkj.jpg', 1), ('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aacbnyzpydt.jpg', 1), ('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aackxswazps.jpg', 1)]\n", + "summary : Non\n", + "length of listImagePaths: 3077\n", + "length of listLabels: 3077\n", + "[('B:\\\\____MLearning\\\\03_\\\\non\\\\aaaesadvpab.jpg', 0), ('B:\\\\____MLearning\\\\03_\\\\non\\\\aaafsbrqzqn.jpg', 0), ('B:\\\\____MLearning\\\\03_\\\\non\\\\aaaqikcbxcj.jpg', 0), ('B:\\\\____MLearning\\\\03_\\\\non\\\\aaatomgskzv.jpg', 0), ('B:\\\\____MLearning\\\\03_\\\\non\\\\aaavnyeqjeq.jpg', 0)]\n", + "summary : merged\n", + "length of listImagePaths: 5083\n", + "length of listLabels: 5083\n", + "[('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aaajtvjlhod.jpg', 1), ('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aaaogunusqn.jpg', 1), ('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aabwgezsbkj.jpg', 1), ('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aacbnyzpydt.jpg', 1), ('B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\aackxswazps.jpg', 1)]\n" + ] + } + ], + "source": [ + "# https://stackoverflow.com/questions/37340129/tensorflow-training-on-my-own-image\n", + "# tf1.7\n", + "# step 1\n", + "pathInputs = \"B:\\\\____MLearning\\\\03_\\\\\"\n", + "pathRC = \"B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\\"\n", + "pathNon = \"B:\\\\____MLearning\\\\03_\\\\non\\\\\"\n", + "\n", + "dirListRC = os.listdir(\"B:\\\\____MLearning\\\\03_\\\\rowcrops\\\\\")\n", + "dirListNon = os.listdir(\"B:\\\\____MLearning\\\\03_\\\\non\\\\\")\n", + "#print(dirListRC)\n", + "\n", + "listRC = []\n", + "listRC_labels = []\n", + "for items in dirListRC:\n", + " listRC.append(pathRC+items)\n", + " listRC_labels.append(1)\n", + "\n", + "objRC = TF_ImageDataset(listRC, listRC_labels, \"RC\" )\n", + " \n", + "listNon = []\n", + "listNon_labels = []\n", + "for items in dirListNon:\n", + " listNon.append(pathNon+items)\n", + " listNon_labels.append(0)\n", + "#print(\"listRC & Labels\\n\", listRC[0:3])\n", + "#print(listRC_labels[0:3])\n", + "\n", + "\n", + "objNon = TF_ImageDataset(listNon, listNon_labels, \"Non\" )\n", + " \n", + "objRC.summarize()\n", + "objNon.summarize()\n", + "\n", + "objBoth = objRC + objNon\n", + "objBoth.summarize()" + ] + }, + { + "cell_type": "code", + "execution_count": 100, + "metadata": {}, + "outputs": [], + "source": [ + "# https://cs230-stanford.github.io/tensorflow-input-data.html#building-an-image-data-pipeline\n", + "\n", + "def parse_function(filename, label):\n", + " image_string = tf.read_file(filename)\n", + "\n", + " # Don't use tf.image.decode_image, or the output shape will be undefined\n", + " image = tf.image.decode_jpeg(image_string, channels=3)\n", + "\n", + " # This will convert to float values in [0, 1]\n", + " image = tf.image.convert_image_dtype(image, tf.float32)\n", + "\n", + " resized_image = tf.image.resize_images(image, [64, 64])\n", + " return resized_image, label\n", + "\n", + "def train_preprocess(image, label):\n", + " image = tf.image.random_flip_left_right(image)\n", + "\n", + " image = tf.image.random_brightness(image, max_delta=32.0 / 255.0)\n", + " image = tf.image.random_saturation(image, lower=0.5, upper=1.5)\n", + "\n", + " # Make sure the image is still in [0, 1]\n", + " image = tf.clip_by_value(image, 0.0, 1.0)\n", + "\n", + " return image, label\n" + ] + }, + { + "cell_type": "code", + "execution_count": 107, + "metadata": {}, + "outputs": [], + "source": [ + "# https://stackoverflow.com/questions/37340129/tensorflow-training-on-my-own-image \n", + "# but this is better:\n", + "# https://cs230-stanford.github.io/tensorflow-input-data.html#building-an-image-data-pipeline\n", + "\n", + "# using standard names\n", + "\n", + "batchsize = 100\n", + "nParCalls = 8\n", + "\n", + "filenames = objBoth.listImagePaths\n", + "labels = objBoth.listLabels\n", + "\n", + "dataset = tf.data.Dataset.from_tensor_slices(( filenames , labels ))\n", + "dataset = dataset.shuffle(len(filenames))\n", + "dataset = dataset.map(parse_function, num_parallel_calls = nParCalls)\n", + "dataset = dataset.map(train_preprocess, num_parallel_calls = nParCalls)\n", + "dataset = dataset.batch(batchsize)\n", + "dataset = dataset.prefetch(1)" + ] + }, + { + "cell_type": "code", + "execution_count": 108, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "# step 2: create a dataset returning slices of `filenames`\n", + "\n", + "print(dataset)" + ] + }, + { + "cell_type": "code", + "execution_count": 109, + "metadata": {}, + "outputs": [], + "source": [ + "# step 3: parse every image in the dataset using `map`\n", + "# currently NOT USED\n", + "def _parse_function(filename, label):\n", + " image_string = tf.read_file(filename)\n", + " image_decoded = tf.image.decode_jpeg(image_string, channels=3)\n", + " image = tf.cast(image_decoded, tf.float32)\n", + " return image, label" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "ename": "TypeError", + "evalue": "Tensor objects are not iterable when eager execution is not enabled. To iterate over this tensor use tf.map_fn.", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mTypeError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 14\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mdataset\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 15\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 16\u001b[1;33m \u001b[0mdataset\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtrain_input_fn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mimages\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlabels\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m100\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[1;32m\u001b[0m in \u001b[0;36mtrain_input_fn\u001b[1;34m(features, labels, batch_size)\u001b[0m\n\u001b[0;32m 6\u001b[0m \u001b[1;34m\"\"\"An input function for training\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 7\u001b[0m \u001b[1;31m# Convert the inputs to a Dataset.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 8\u001b[1;33m \u001b[0mdataset\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mDataset\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfrom_tensor_slices\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfeatures\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlabels\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 9\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 10\u001b[0m \u001b[1;31m# Shuffle, repeat, and batch the examples.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\u001b[0m in \u001b[0;36m__iter__\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m 398\u001b[0m \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mcontext\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mexecuting_eagerly\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 399\u001b[0m raise TypeError(\n\u001b[1;32m--> 400\u001b[1;33m \u001b[1;34m\"Tensor objects are not iterable when eager execution is not \"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 401\u001b[0m \"enabled. To iterate over this tensor use tf.map_fn.\")\n\u001b[0;32m 402\u001b[0m \u001b[0mshape\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_shape_tuple\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mTypeError\u001b[0m: Tensor objects are not iterable when eager execution is not enabled. To iterate over this tensor use tf.map_fn." + ] + } + ], + "source": [ + "# step 4: create iterator and final input tensor\n", + "iterator = dataset.make_one_shot_iterator()\n", + "images, labels = iterator.get_next()\n", + "\n", + "def train_input_fn(features, labels, batch_size):\n", + " \"\"\"An input function for training\"\"\"\n", + " # Convert the inputs to a Dataset.\n", + " dataset = tf.data.Dataset.from_tensor_slices((dict(features), labels))\n", + "\n", + " # Shuffle, repeat, and batch the examples.\n", + " dataset = dataset.shuffle(1000).repeat().batch(batch_size)\n", + "\n", + " # Return the dataset.\n", + " return dataset\n", + "\n", + "dataset = train_input_fn(images, labels, 100)" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "#from tensorflow.examples.tutorials.mnist import input_data\n", + "#data = input_data.read_data_sets('data/MNIST/', one_hot=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The MNIST data-set has now been loaded and consists of 70,000 images and associated labels (i.e. classifications of the images). The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Size of:\n" + ] + }, + { + "ename": "AttributeError", + "evalue": "'BatchDataset' object has no attribute 'train'", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mAttributeError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Size of:\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"- Training-set:\\t\\t{}\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtrain\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlabels\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 3\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"- Test-set:\\t\\t{}\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtest\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlabels\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"- Validation-set:\\t{}\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mvalidation\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlabels\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mAttributeError\u001b[0m: 'BatchDataset' object has no attribute 'train'" + ] + } + ], + "source": [ + "print(\"Size of:\")\n", + "print(\"- Training-set:\\t\\t{}\".format(len(data.train.labels)))\n", + "print(\"- Test-set:\\t\\t{}\".format(len(data.test.labels)))\n", + "print(\"- Validation-set:\\t{}\".format(len(data.validation.labels)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The class-labels are One-Hot encoded, which means that each label is a vector with 10 elements, all of which are zero except for one element. The index of this one element is the class-number, that is, the digit shown in the associated image. We also need the class-numbers as integers for the test-set, so we calculate it now." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "data.test.cls = np.argmax(data.test.labels, axis=1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Data Dimensions" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The data dimensions are used in several places in the source-code below. They are defined once so we can use these variables instead of numbers throughout the source-code below." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "# We know that MNIST images are 28 pixels in each dimension.\n", + "img_size = 28\n", + "\n", + "# Images are stored in one-dimensional arrays of this length.\n", + "img_size_flat = img_size * img_size\n", + "\n", + "# Tuple with height and width of images used to reshape arrays.\n", + "img_shape = (img_size, img_size)\n", + "\n", + "# Number of colour channels for the images: 1 channel for gray-scale.\n", + "num_channels = 1\n", + "\n", + "# Number of classes, one class for each of 10 digits.\n", + "num_classes = 10" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Helper-function for plotting images" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Function used to plot 9 images in a 3x3 grid, and writing the true and predicted classes below each image." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "def plot_images(images, cls_true, cls_pred=None):\n", + " assert len(images) == len(cls_true) == 9\n", + " \n", + " # Create figure with 3x3 sub-plots.\n", + " fig, axes = plt.subplots(3, 3)\n", + " fig.subplots_adjust(hspace=0.3, wspace=0.3)\n", + "\n", + " for i, ax in enumerate(axes.flat):\n", + " # Plot image.\n", + " ax.imshow(images[i].reshape(img_shape), cmap='binary')\n", + "\n", + " # Show true and predicted classes.\n", + " if cls_pred is None:\n", + " xlabel = \"True: {0}\".format(cls_true[i])\n", + " else:\n", + " xlabel = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n", + "\n", + " # Show the classes as the label on the x-axis.\n", + " ax.set_xlabel(xlabel)\n", + " \n", + " # Remove ticks from the plot.\n", + " ax.set_xticks([])\n", + " ax.set_yticks([])\n", + " \n", + " # Ensure the plot is shown correctly with multiple plots\n", + " # in a single Notebook cell.\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Plot a few images to see if data is correct" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAD5CAYAAAC9FVegAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHitJREFUeJzt3XmUFNXZx/HvA0IQEBVBQcWZE3CBEAXF4C5RIIoKSFwwLrzGaESDWwJG464xSlB4RU9YjMgJMSoKiEZFAUV82SMoiBuIKBKXEUIUERHu+8f07aqe6dl6uqp6xt/nHM9Ud1VXPeOl7zxVdzPnHCIi33cNkg5ARKQQqDIUEUGVoYgIoMpQRARQZSgiAqgyFBEBVBmKiACqDEVEAFWGIiIA7FSTg1u1auWKi4sjCqXwfPDBB5SUlFjSccRJZVz/qYyzq1FlWFxczJIlS3KPqo7p1q1b0iHETmVc/6mMs9NtsogIqgxFRABVhiIigCpDERFAlaGICFDD1mSRXI0YMQKALVu2APDGG28A8MQTT5Q7dvDgwQAceeSRAJx//vlxhCjfc8oMRURQZigRO/vsswGYPHly1v1m5fvCjhkzBoCZM2cCcPzxxwOw3377RRGiJOjdd98F4MADDwTgvvvuA2DIkCGxx6LMUEQEZYYSAZ8NQsUZ4UEHHQTASSedBMD777+f3jd9+nQAVq1aBcCkSZMAuP766/MfrCRq6dKlADRoUJqX7bPPPonFosxQRARlhpJHfrzr1KlTy+3r3LkzEGR9rVq1AqB58+YAfPvtt+lju3fvDsDrr78OwBdffBFRxJK0ZcuWAcG/gwEDBiQWizJDERFiyAx9P7Lx48cDsPfee6f3NWnSBIBzzz0XgDZt2gDQoUOHqMOSCPz73/8GwDmXfs9nhDNmzACgbdu2WT/r+yECvPXWWxn7Tj311LzGKclbvnw5AKNHjwbgggsuSDIcQJmhiAgQQ2Y4dOhQoHSCxYr4fmUtWrQAoFOnTnm5drt27QAYNmwY8P2cuy5Op512GhC0AgPssssuALRs2bLSzz722GPp7fDzQ6mf3nnnHQA2b94MZPZASIoyQxERVBmKiAAx3CY/+OCDQNBNInwLvHLlSiDoePnyyy8DsGDBAiAYfvXhhx9WeP5GjRoBQVcN/xA/fB5/u6zb5HgUFRVV+9g///nPQDAsK8x3sfE/pf4YPnw4ULoEARTGd1OZoYgIMWSGJ554YsbPMD8Uy9u4cSMQZIr+r8XixYsrPP8PfvADIBjo7Yd5AWzYsAGA9u3b5xS7ROeZZ54B4KabbgJg69at6X177bUXAHfddRcATZs2jTk6iUK4EdV/p/33tlmzZkmElEGZoYgIBTYcb/fddwfghBNOyHg/W1ZZ1pNPPgkE2SXAwQcfDMDAgQPzFaLkiR+6F84IPd/Nwk/dJfXDnDlzyr3XunXrBCLJTpmhiAgFlhnm4rPPPgPgsssuAzKHgvnnUVV1+JX49O/fHwiG53mDBg1Kb99xxx2xxiTx8Es9hPkBEYVAmaGICPUgM3zggQeAIEPcbbfd0vt8S5Ukz/f/nDdvHhA8K/TPjG644Yb0sX46J6kf5s+fD8CECRPS73Xt2hWAXr16JRJTNsoMRUSow5nhq6++CgR90bynnnoqve2nj5Lk+Uk7S0pKMt7307epL2j9NWvWLCCzp4fvY+yn8SsEygxFRFBlKCIC1OHb5GeffRYI5r7r2bMnAEceeWRiMUl5fs0TP8TS69GjBwC33XZb3CFJzPwkLWFnnnlmApFUTpmhiAh1MDPcsmULAM8//zwQTNRw6623AsGUXpKc8Gp2d955J1B+9uouXboA6kZTn33yyScAzJ07F8icROX0009PJKbKKDMUEaEOZoZ+MlD/DOrkk08G4KijjkosJsl0zz33pLcXLVqUsc8Px9Ozwvrv4YcfBuDTTz8Fgu9qoVJmKCJCHckM/USgALfffjsAu+66KwA33nhjIjFJxe69994K9/nhk3pWWP+tXbs247Wfoq9QKTMUEaHAM0PfKnnFFVek3/vuu+8A6NOnD6B+hXWNL9PqtPr77N8fu23bNgA2bdpU7lg/1GvkyJFZz9WwYcP09t133w1oOYGoPf300xmvTz311IQiqR5lhiIiqDIUEQEK9DZ5+/btQDCzxZo1a9L7OnToAAQNKVK3+HVpquOss84CoG3btkDQRePRRx+tVQx+9b3wHIqSP76TtS+vukKZoYgIBZoZrl69GghWUAvz3TY0/13h8o1bANOmTcv5PI8//niVx/jGlQYNMv+u9+3bFwjW3g475phjco5JqjZ16lQgaOz0s1oX+mqHygxFRCiwzNB30uzdu3fG+yNGjEhvF3rzvMCUKVPS28OHDwfKT9TgrVy5Eqj8OeBFF10EQFFRUbl9P//5zwHo2LFjbsFK3nz99dcAPPfccxnv++m6wt2bCpEyQxERCiwzHDt2LFB+GE/4WYOZxRqT1E5118V95JFHIo5Eouaf3/oVKvv16wfAlVdemVhMNaHMUESEAskMfb+k+++/P+FIRCRXPjP06yTXNcoMRUQokMzQr4H85ZdfZrzvR5touicRiZoyQxERVBmKiAAFcptcll85bdasWQC0bNkyyXBE5HtAmaGICAWSGV533XUZP0VE4qbMUEQEMOdc9Q82+xxYW+WB9UeRc6510kHESWVc/6mMs6tRZSgiUl/pNllEBFWGIiJAxK3JZrYHMCv1sg2wHfg89fonzrnsM37W7pqdgPB8UO2B65xzmgUiAgmVcREwEdgTcMBfVL7RSaKMU9edCPQBPnbOdYniGhnXi+uZoZndAnzlnBtR5n1LxbEjgmvuBKwHDnXOrcv3+SVTXGVsZnsDezrnlplZC2ApcLJz7t18nF8qFuf32MyOB7YA4+KoDBO5TTazDma2wszGAK8B7czsP6H9A83swdT2XmY2xcyWmNkiMzuiBpfqDbylijB+UZaxc269c25Zavu/wNvAPtH9NpJN1N9j59wcYENkv0AZST4z7AT81TnXFfi4kuPuA4Y757oBZwH+f273VCFUZiDwj3wEKzmJvIzN7IdAZ2BxfkKWGorjexyLJEegrHbOVecfcE/gwNB0/7ub2c7OuYXAwoo+ZGZNgFOAa2odqeQq6jJuATwJDHHOfVXraCUXkZZxnJKsDDeHtncA4cVNmoS2jdwe0p4CLHTOleQYn9ReZGVsZo2BKcDDzrnptYpSaiPq73FsCqJrTeqh60Yz29/MGgCnh3bPBC73L8ysug9Sz0G3yAUjn2Wcelj/MLDMOfe/EYQrOYjoexybgqgMU64Fnqe0CT/c4HE5cLSZvWFmK4GLofJnDWbWHPgpMC3akKWG8lXGx1P6x66XmS1L/feziGOX6snn93gyMBfoZGbrzOx/ogxcw/FERCiszFBEJDGqDEVEUGUoIgKoMhQRAVQZiogANex03apVK1dcXBxRKIXngw8+oKSkxKo+sv5QGdd/KuPsalQZFhcXs2TJktyjqmO6deuWdAixUxnXfyrj7HSbLCKCKkMREUCVoYgIoMpQRARQZSgiAqgyFBEBkp3ctUKbN5fOFzl06FAAxowJZvjxzeSTJ08GoKioKOboRKQ+UmYoIkKBZobr168HYPz48QA0bNgwvc93Fn366acB+M1vfhNzdJKL1157DYABAwYApaMCcvXCCy+ktzt27AhAu3btcg9OEuO/x3379gVg9OjRAAwePDh9TPj7HyVlhiIiFFhm+PnnnwMwaNCghCORfJsxYwYAW7durfW5pk8P1n966KGHAHj00UdrfV6JzxdffAFkZoAAQ4YMAeCiiy5Kv7fzzjvHEpMyQxERCiQzvO+++wCYNq10/abFi6tehnXu3LkA+DVcDjnkEACOO+64KEKUHH333XcAPPvss3k7Z3jg/b333gsEPRCaNWuWt+tIdF555RUAPv44c935c845B4AmTZqU+0zUlBmKiFAgmeFVV10F1KzVaMqUKRk/99tvPwAef/zx9DGHHXZYvkKUHL300ksAzJs3D4Brr7221ufcsGFDevvNN98E4OuvvwaUGRay8PPiO+64I+sx559/PgClS2PHS5mhiAiqDEVEgIRvk/v06QMEjSDbt2+v8jOtWrUCgtuhtWvXArBmzRoADj/88PSxO3bsyF+wUm3Lly9Pbw8cOBCADh06AHD99dfX+vzhrjVSd7zxxhvpbd8J39tpp9Kq6OSTT441pjBlhiIiJJAZzpkzJ7399ttvA8HD0ooaUC699NL0du/evQHYddddAZg9ezYAf/zjH8t97i9/+QtQvmOnRCtcFr5hY9KkSQA0b9485/P6hpPwv6EkHrRLbnxjZza9evWKMZLslBmKiBBjZugH5vtnSAAlJSVZj/XdZM444wwAbr755vS+pk2bZhzrp/AaO3ZsuXMOGzYMgG+++QYIJnVo1KhRbr+EVOqJJ54AMjtY+2eF4We5ufLdMcLZYI8ePQDYbbfdan1+iVY4o/caN24MwJ133hl3OOUoMxQRIcbMcNu2bUDF2SAEQ+kee+wxIGg5rozPDH0r5TXXXJPe54do+QzRTxPUvn37GsUu1eMn3PX/3yE/z2v9XcUjjzwCBC2PADfccAOgbL+Q+Q738+fPL7fP3+l16dIl1piyUWYoIkKBDMfzz5MmTJgAVC8jLMtnfX//+9/T7y1atCgP0UlVNm3aBMCCBQvK7bvssstqff5x48YBwRRvnTp1Su874YQTan1+iVZlE68UUk8PZYYiIiSQGWYbZbJw4cJan9ePYgmPOik7ssW3Svs+b5IffgD+unXrgGAapnxZvXp1xuvOnTvn9fwSrWyZoW/9z8edQ74oMxQRQZWhiAgQ422yX/s4qpWu/CpbS5cuTb9XdpjfrbfeGsm1v+922WUXIOgeEZ6owQ+ha9myZY3P+9lnnwFBlx3v6KOPzilOiderr74KBF2iwvxw2n333TfWmCqjzFBEhBgzw2eeeSav5/PdLFauXAlUPpzHd9VRx9xo+NXL/NA7PywP4JRTTgEyO8Nns2LFivS2bzDx07OVnYyhQQP9Da8L/Ap4viEzrBAmZihL/6pERCiQTte58NNEPfDAAxUeU1xcDMDEiROBYAIIicYtt9wCZGYC/o4gPEFHNq1bt05v+0ywoqGbF154YW3ClJiUfdYbnkzjkksuiTucKikzFBGhDmaGfqkAPzFsZfywrWOPPTbSmKRUx44dgcwVCn3rftmO02X56drCBg0aBJTvJO+fUUph8p3vy7Yih1uO8zGlW74pMxQRIcbMsLJFn5577rmM1xdffDEA69evr/A81ZnuPd8t2FJzXbt2zfhZEz/84Q+zvh/ux/jjH/84t8AkMn7KrrKtyP369UsinGpTZigigipDEREgxttkP2+Zn3U6zHfMLTtUL9vQPX+bXZ2V9KRu87dZZW+3dGtc2Hxna88PerjqqquSCKfalBmKiBBjZjhgwAAAhg8fnn6vsvVQquL/2vjuHOPHjwegbdu2OZ9TCotvJNPayHXLjBkzMl63a9cOCCZnKFTKDEVEiDEz9KvY+ZXvAKZNmwbAqFGjany+P/zhD0CwFrLUP369a0+drQubXwFz1apVGe83adIEKPyJUpQZioiQwHA8vzZyeLt3795AsAqan6j1tNNOA+DXv/51+jO+ZTG8QprUT361RD/A/6abbkoyHKmCn1rND7V78803Adh///0Ti6kmlBmKiFAgEzWcdNJJGT9FIMgwrr76akBrJBc63/fXT6/newEceuihicVUE8oMRUQokMxQJBv/7Fjqlr333huAhx56KOFIakaZoYgIqgxFRABVhiIigCpDERFAlaGICKDKUEQEAMu22n2FB5t9DqyNLpyCU+Sca131YfWHyrj+UxlnV6PKUESkvtJtsogIqgxFRABVhiIiQMRjk81sD2BW6mUbYDvweer1T5xz30Z03T7ASKAhMNY59+coriPJlXHq2jsBrwHvO+f6R3Wd77sEv8cTgT7Ax865LlFcI+N6cTWgmNktwFfOuRFl3rdUHDvydJ1GwDvAT4FPgCXAz51z7+bj/FKxuMo4dN5hQBegqSrDeMRZxmZ2PLAFGBdHZZjIbbKZdTCzFWY2htK/7O3M7D+h/QPN7MHU9l5mNsXMlpjZIjM7oorTHwG85Zxb65zbCjwO9Ivqd5HsIi5jzKwI6AVMiOp3kMpFXcbOuTnAhsh+gTKSfGbYCfirc64r8HElx90HDHfOdQPOAvz/3O6pQihrH+Cj0Ot1qfckflGVMcAoYCigvmHJirKMY5XkfIarnXOLq3FcT+DA0Nq5u5vZzs65hcDCLMdnW2RXX5hkRFLGZtYf+Mg5t8zMeuYvXMlBVN/j2CVZGW4Obe8gsxJrEto2avaQdh3QLvR6X2B9ThFKbUVVxkcBA8ysb+o8LcxsonNuUK2ilVxEVcaxK4iuNamHrhvNbH8zawCcHto9E7jcvzCzqh6kLgA6mVmRmf2A0pR8er5jlprJZxk754Y55/Z1zhUD5wEvqCJMXp6/x7EriMow5VrgeUqb8NeF3r8cONrM3jCzlcDFUPGzBufcNuAK4EVgJTDJOfdO1MFLteSljKWg5a2MzWwyMJfS5Gadmf1PlIFrbLKICIWVGYqIJEaVoYgIqgxFRABVhiIiQA37GbZq1coVFxdHFErh+eCDDygpKcnWibveUhnXfyrj7GpUGRYXF7NkyZLco6pjunXrlnQIsVMZ138q4+x0mywigipDERFAlaGICKDKUEQEUGUoIgKoMhQRAVQZiogAyU7uKiICwMaNGwH48MMPKzymqKgIgJEjRwLQuXNnAA444AAADjnkkFrFoMxQRISEM8PPPvsMgLPOOguAo446CoBLLrkEKO0pnw+bNm0C4JVXXgHgpJNOAqBRo0Z5Ob+I1MwzzzwDwNNPPw3Ayy+/DMB7771X4WcOPPBAoHR4HcDWrVsz9u/YUbtVSpUZioiQQGbonw0A/OhHPwKCzG2vvfYC8p8RHnrooQCUlJQApMdl7r///nm5jlTff//7XwB+//vfA/Dmm28CMHPmzPQxytjrh9WrVwPwwAMPADBu3Lj0vi1btgBQk5n233kn2tU7lBmKiBBjZuizMv98EOCLL74A4PLLSxfNGj16dF6veccddwCwZs0aIPjLpIwwfpMmTQLghhtuAMq3GvqMEWCPPfaILzCJzLp1petBjRo1qlbnOeigg4Cg9TgqygxFRIgxM3zttdeAoNUo7KabbsrbdVasWJHeHjFiBACnn166fOvZZ5+dt+tI9fjs4OqrrwaCOwSzzLk2hwwZkt6+//77AWjZsmUcIUoOfDlCkPkdc8wxQNBbo3HjxgDsuuuuADRv3jz9ma+++gqAn/3sZ0CQ9XXv3h2Arl27po/deeedAWjWrFmef4tMygxFRFBlKCICxHCb7DtWP/nkk+X2PfTQQwC0bt261tfxt8e9evUqt2/AgAEA7LLLLrW+jtSMf1ThG8sq8uijj6a3n3vuOSBobPG30P62S5KzefNmIPN79vrrrwMwbdq0jGOPPPJIAJYuXQpkdpnzDWj77rsvAA0aJJ+XJR+BiEgBiDwz/O1vfwsEXSt8B2iAM888M2/XefXVVwH45JNP0u9deOGFAJx33nl5u45Ube3atentCRMmZOzzg+l9B/sXX3yx3Od9Z3mfVZ577rkAtGnTJv/BSrV8++23APziF78AgmwQ4PrrrwegZ8+eWT+bbRDFfvvtl+cIa0+ZoYgIMWSGvguF/7nPPvuk99XmGZAfznPnnXcCwZCfcJcN/0xS4rVs2bL0tu9MfdxxxwEwZ84cAL755hsAHnnkEQD+9Kc/pT+zatUqIMjy+/XrBwTPEtXlJj6+C4z/nvmJFcLP+YcOHQpA06ZNY44uv5QZioiQwEQNfuoegN69ewOw2267ATB48OAqP+87bfufCxYsyNifz+eQkpvw1Eo+U/edrr0mTZoA8Mtf/hKAJ554Ir3PD/D3g/h9xqHW5Pj5FuK77roLCCZYnTt3bvoY36m6rlNmKCJCDJnhlVdeCcDs2bMBWL9+fXqff37kM4CnnnqqyvP5Y8sO52rfvj0QPNuQ5PzjH/8o994///lPAPr375/1M35atWyOOOIIIHM4l8Rj3rx5Ga/9MDnfP7A+UWYoIkIMmeFhhx0GwPLly4HMlsbnn38egOHDhwOw5557AjBo0KAKz3f++ecDcPDBB2e875cM8BmiJOecc85Jb/tsf/HixQC8/fbbQPDvYerUqUDmpL/+GbJ/z0+95su+U6dOkcUumcLPciFo0b/11lvT7/Xt2xfInFyhLlJmKCKCKkMREQCsJmsQdOvWzVX2oDsO77//PhDcDnfp0gWAF154AcjPpA9et27dWLJkiVV9ZP2RjzLesGFDetuXkx9iV1EDWHjgv+9Af+qppwLw7rvvAsGqiWPGjKlVfGEq48qVHTSRTcOGDQG49NJLgWBOwo8++giADh06AMGaR2F+DRw/qUMUDTPVLWNlhiIiJLxuci5uu+02IPhL5Rtf8pkRSu2Eh8tNnjwZgDPOOAMonyFeccUVANx9993pz/gO2X7qNT9Ub8aMGUDQKRvUYBa13/3udwDcc889FR6zfft2IMjo/c+a8I2nPXr0ADKndIuLMkMREepIZuizC4CJEycC0KJFC0ArqRU6P62T76LhJ2bw3Wd8pu+zwbAbb7wRgLfeegsIuun4z0Dw70Gi4Yfh+VUt/XRq27ZtSx/j17nxGWIu/CTQ/rseXgnPT/IbNWWGIiLUkczQd/QMO+WUU4DMyWKlcPkMsaIJQLPxq6L5VQ19ZvjSSy+lj/Et15rWKxq+pfjwww8Hgpb9sFmzZgFBtnjLLbcAsGjRohpfzz9L/te//lXjz9aWMkMREepgZujXTvWtXFL/+edV06dPBzJbGv0ay/lce1tq5sQTT8x47Yfc+sywUaNGQLAMB8DFF18MwMiRI4HgWXKSlBmKiKDKUEQEKPDbZD/sKrzinV9VTQ0n3x9+Td1hw4YBmevz+of1AwcOBOCAAw6INzgpx89g71fN8w0rfvYhgPfeew8IZqwvK7xWUlyUGYqIUEcyw/Ag8T59+mQc8+WXXwLB3HeFuB6r5IeflOP2229Pv+cb0q677jogWJ/bd8uR+HXs2BEIukQ99thj5Y4Jd48C2Gmn0qrId5kLD8+MizJDEREKPDPMxv8F8RmAb5r3w3c0PKv+u+CCC9LbY8eOBWDKlClA8Cyq7EzoEh+flY8aNQoI7t7CHak//fRTAIqLi4GgTP0z4CQoMxQRoQ5mhuPHjwfgwQcfBOBXv/oVEAzql/ovPF3bzJkzgWA9Xz+xQCF04v2+8z0//Frpf/vb39L75s+fDwSZoJ/CK0nKDEVEKPDMcPTo0QDcfPPN6feOO+44AAYPHgzA7rvvDkDjxo1jjk4Kge894JcN8EP2Vq5cCWglvULiVzcsu10olBmKiFDgmeGxxx4LwOzZsxOORAqdnzz2kEMOAWDVqlWAMkOpPmWGIiKoMhQRAQr8NlmkuvyaOGvWrEk4EqmrlBmKiKDKUEQEUGUoIgKA+dWoqnWw2efA2ujCKThFzrnWVR9Wf6iM6z+VcXY1qgxFROor3SaLiKDKUEQEiLifoZntAcxKvWwDbAc+T73+iXPu2wivvRPwGvC+c65/VNf5vkuqjM3sGuCi1MsxzrnRUVxHEi3jdcDG1PW2Oue6R3Gd9PXiemZoZrcAXznnRpR531Jx7Mjz9YYBXYCmqgzjEVcZm1kXYCJwBPAd8ALwS+ecelxHLM7vcaoy7Oyc+0++zlmZRG6TzayDma0wszGUZm/tzOw/of0DzezB1PZeZjbFzJaY2SIzO6Ia5y8CegETovodpHIRl3FHYL5zbotzbhvwCnB6VL+LZBf19zhuST4z7AT81TnXFfi4kuPuA4Y757oBZwH+f273VCFkMwoYCqipPFlRlfFyoIeZtTSzZsDJQLv8hi7VFOX32AGzzexfZnZRBcfkTZJjk1c75xZX47iewIGh5UJ3N7OdnXMLgYVlDzaz/sBHzrllZtYzf+FKDiIpY+fcCjO7F5gJfAUspfR2WeIXSRmndHfOrTezNsCLZvaWc25eHmLOKsnKcHNoewdgoddNQttGzR7SHgUMMLO+qfO0MLOJzrlBtYpWchFVGeOcGweMAzCz4cCqWsQpuYuyjNenfn5iZk8BPwEiqwwLomtN6qHrRjPb38wakPn8ZyZwuX+Renhe2bmGOef2dc4VA+cBL6giTF4+yzh1zJ6pn8VAP6D8SuUSq3yWsZk1N7PmfpvSNoAV+Y86UBCVYcq1wPOUNuGvC71/OXC0mb1hZiuBi6HKZw1SmPJZxtNSx04Dfu2c2xRh3FJ9+SrjtsD/mdnrlN5GT3XOzYwycA3HExGhsDJDEZHEqDIUEUGVoYgIoMpQRARQZSgiAqgyFBEBVBmKiACqDEVEAPh/EMZccjkjBQkAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Get the first images from the test-set.\n", + "images = data.test.images[0:9]\n", + "\n", + "# Get the true classes for those images.\n", + "cls_true = data.test.cls[0:9]\n", + "\n", + "# Plot the images and labels using our helper-function above.\n", + "plot_images(images=images, cls_true=cls_true)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## TensorFlow Graph\n", + "\n", + "The entire purpose of TensorFlow is to have a so-called computational graph that can be executed much more efficiently than if the same calculations were to be performed directly in Python. TensorFlow can be more efficient than NumPy because TensorFlow knows the entire computation graph that must be executed, while NumPy only knows the computation of a single mathematical operation at a time.\n", + "\n", + "TensorFlow can also automatically calculate the gradients that are needed to optimize the variables of the graph so as to make the model perform better. This is because the graph is a combination of simple mathematical expressions so the gradient of the entire graph can be calculated using the chain-rule for derivatives.\n", + "\n", + "TensorFlow can also take advantage of multi-core CPUs as well as GPUs - and Google has even built special chips just for TensorFlow which are called TPUs (Tensor Processing Units) and are even faster than GPUs.\n", + "\n", + "A TensorFlow graph consists of the following parts which will be detailed below:\n", + "\n", + "* Placeholder variables used for inputting data to the graph.\n", + "* Variables that are going to be optimized so as to make the convolutional network perform better.\n", + "* The mathematical formulas for the convolutional neural network.\n", + "* A so-called cost-measure or loss-function that can be used to guide the optimization of the variables.\n", + "* An optimization method which updates the variables.\n", + "\n", + "In addition, the TensorFlow graph may also contain various debugging statements e.g. for logging data to be displayed using TensorBoard, which is not covered in this tutorial." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Placeholder variables" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Placeholder variables serve as the input to the TensorFlow computational graph that we may change each time we execute the graph. We call this feeding the placeholder variables and it is demonstrated further below.\n", + "\n", + "First we define the placeholder variable for the input images. This allows us to change the images that are input to the TensorFlow graph. This is a so-called tensor, which just means that it is a multi-dimensional array. The data-type is set to `float32` and the shape is set to `[None, img_size_flat]`, where `None` means that the tensor may hold an arbitrary number of images with each image being a vector of length `img_size_flat`." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [], + "source": [ + "x = tf.placeholder(tf.float32, shape=[None, img_size_flat], name='x')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The convolutional layers expect `x` to be encoded as a 4-dim tensor so we have to reshape it so its shape is instead `[num_images, img_height, img_width, num_channels]`. Note that `img_height == img_width == img_size` and `num_images` can be inferred automatically by using -1 for the size of the first dimension. So the reshape operation is:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "x_image = tf.reshape(x, [-1, img_size, img_size, num_channels])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next we have the placeholder variable for the true labels associated with the images that were input in the placeholder variable `x`. The shape of this placeholder variable is `[None, num_classes]` which means it may hold an arbitrary number of labels and each label is a vector of length `num_classes` which is 10 in this case." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "y_true = tf.placeholder(tf.float32, shape=[None, num_classes], name='y_true')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We could also have a placeholder variable for the class-number, but we will instead calculate it using argmax. Note that this is a TensorFlow operator so nothing is calculated at this point." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "WARNING:tensorflow:From :1: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version.\n", + "Instructions for updating:\n", + "Use the `axis` argument instead\n" + ] + } + ], + "source": [ + "y_true_cls = tf.argmax(y_true, dimension=1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## PrettyTensor Implementation\n", + "\n", + "This section shows the implementation of a Convolutional Neural Network using PrettyTensor taken from Tutorial #03 so it can be compared to the implementation using the Layers API below. This code has been enclosed in an `if False:` block so it does not run here.\n", + "\n", + "The basic idea is to wrap the input tensor `x_image` in a PrettyTensor object which has helper-functions for adding new computational layers so as to create an entire Convolutional Neural Network. This is a fairly simple and elegant syntax." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "if False:\n", + " x_pretty = pt.wrap(x_image)\n", + "\n", + " with pt.defaults_scope(activation_fn=tf.nn.relu):\n", + " y_pred, loss = x_pretty.\\\n", + " conv2d(kernel=5, depth=16, name='layer_conv1').\\\n", + " max_pool(kernel=2, stride=2).\\\n", + " conv2d(kernel=5, depth=36, name='layer_conv2').\\\n", + " max_pool(kernel=2, stride=2).\\\n", + " flatten().\\\n", + " fully_connected(size=128, name='layer_fc1').\\\n", + " softmax_classifier(num_classes=num_classes, labels=y_true)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Layers Implementation\n", + "\n", + "We now implement the same Convolutional Neural Network using the Layers API that is included in TensorFlow version 1.1. This requires more code than PrettyTensor, although a lot of the following are just comments.\n", + "\n", + "We use the `net`-variable to refer to the last layer while building the Neural Network. This makes it easy to add or remove layers in the code if you want to experiment. First we set the `net`-variable to the reshaped input image." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "net = x_image" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The input image is then input to the first convolutional layer, which has 16 filters each of size 5x5 pixels. The activation-function is the Rectified Linear Unit (ReLU) described in more detail in Tutorial #02." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [], + "source": [ + "net = tf.layers.conv2d(inputs=net, name='layer_conv1', padding='same',\n", + " filters=16, kernel_size=5, activation=tf.nn.relu)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "One of the advantages of constructing neural networks in this fashion, is that we can now easily pull out a reference to a layer. This was more complicated in PrettyTensor.\n", + "\n", + "Further below we want to plot the output of the first convolutional layer, so we create another variable for holding a reference to that layer." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "layer_conv1 = net" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We now do the max-pooling on the output of the convolutional layer. This was also described in more detail in Tutorial #02." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "net = tf.layers.max_pooling2d(inputs=net, pool_size=2, strides=2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We now add the second convolutional layer which has 36 filters each with 5x5 pixels, and a ReLU activation function again." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "net = tf.layers.conv2d(inputs=net, name='layer_conv2', padding='same',\n", + " filters=36, kernel_size=5, activation=tf.nn.relu)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We also want to plot the output of this convolutional layer, so we keep a reference for later use." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [], + "source": [ + "layer_conv2 = net" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The output of the second convolutional layer is also max-pooled for down-sampling the images." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [], + "source": [ + "net = tf.layers.max_pooling2d(inputs=net, pool_size=2, strides=2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The tensors that are being output by this max-pooling are 4-rank, as can be seen from this:" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "net" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next we want to add fully-connected layers to the Neural Network, but these require 2-rank tensors as input, so we must first flatten the tensors.\n", + "\n", + "The `tf.layers` API was first located in `tf.contrib.layers` before it was moved into TensorFlow Core. But even though it has taken the TensorFlow developers a year to move these fairly simple functions, they have somehow forgotten to move the even simpler `flatten()` function. So we still need to use the one in `tf.contrib.layers`." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [], + "source": [ + "net = tf.contrib.layers.flatten(net)\n", + "\n", + "# This should eventually be replaced by:\n", + "# net = tf.layers.flatten(net)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This has now flattened the data to a 2-rank tensor, as can be seen from this:" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "net" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can now add fully-connected layers to the neural network. These are called *dense* layers in the Layers API." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [], + "source": [ + "net = tf.layers.dense(inputs=net, name='layer_fc1',\n", + " units=128, activation=tf.nn.relu)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We need the neural network to classify the input images into 10 different classes. So the final fully-connected layer has `num_classes=10` output neurons." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "net = tf.layers.dense(inputs=net, name='layer_fc_out',\n", + " units=num_classes, activation=None)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The output of the final fully-connected layer are sometimes called logits, so we have a convenience variable with that name." + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [], + "source": [ + "logits = net" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We use the softmax function to 'squash' the outputs so they are between zero and one, and so they sum to one." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [], + "source": [ + "y_pred = tf.nn.softmax(logits=logits)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This tells us how likely the neural network thinks the input image is of each possible class. The one that has the highest value is considered the most likely so its index is taken to be the class-number." + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "WARNING:tensorflow:From :1: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version.\n", + "Instructions for updating:\n", + "Use the `axis` argument instead\n" + ] + } + ], + "source": [ + "y_pred_cls = tf.argmax(y_pred, dimension=1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have now created the exact same Convolutional Neural Network in a few lines of code that required many complex lines of code in the direct TensorFlow implementation.\n", + "\n", + "The Layers API is perhaps not as elegant as PrettyTensor, but it has some other advantages, e.g. that we can more easily refer to intermediate layers, and it is also easier to construct neural networks with branches and multiple outputs using the Layers API." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Loss-Function to be Optimized" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To make the model better at classifying the input images, we must somehow change the variables of the Convolutional Neural Network.\n", + "\n", + "The cross-entropy is a performance measure used in classification. The cross-entropy is a continuous function that is always positive and if the predicted output of the model exactly matches the desired output then the cross-entropy equals zero. The goal of optimization is therefore to minimize the cross-entropy so it gets as close to zero as possible by changing the variables of the model.\n", + "\n", + "TensorFlow has a function for calculating the cross-entropy, which uses the values of the `logits`-layer because it also calculates the softmax internally, so as to to improve numerical stability." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_true, logits=logits)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have now calculated the cross-entropy for each of the image classifications so we have a measure of how well the model performs on each image individually. But in order to use the cross-entropy to guide the optimization of the model's variables we need a single scalar value, so we simply take the average of the cross-entropy for all the image classifications." + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [], + "source": [ + "loss = tf.reduce_mean(cross_entropy)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Optimization Method\n", + "\n", + "Now that we have a cost measure that must be minimized, we can then create an optimizer. In this case it is the Adam optimizer with a learning-rate of 1e-4.\n", + "\n", + "Note that optimization is not performed at this point. In fact, nothing is calculated at all, we just add the optimizer-object to the TensorFlow graph for later execution." + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [], + "source": [ + "optimizer = tf.train.AdamOptimizer(learning_rate=1e-4).minimize(loss)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Classification Accuracy\n", + "\n", + "We need to calculate the classification accuracy so we can report progress to the user.\n", + "\n", + "First we create a vector of booleans telling us whether the predicted class equals the true class of each image." + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [], + "source": [ + "correct_prediction = tf.equal(y_pred_cls, y_true_cls)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The classification accuracy is calculated by first type-casting the vector of booleans to floats, so that False becomes 0 and True becomes 1, and then taking the average of these numbers." + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [], + "source": [ + "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Getting the Weights\n", + "\n", + "Further below, we want to plot the weights of the convolutional layers. In the TensorFlow implementation we had created the variables ourselves so we could just refer to them directly. But when the network is constructed using a builder API such as `tf.layers`, all the variables of the layers are created indirectly by the builder API. We therefore have to retrieve the variables from TensorFlow.\n", + "\n", + "First we need a list of the variable names in the TensorFlow graph:" + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n" + ] + } + ], + "source": [ + "for var in tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES):\n", + " print(var)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Each of the convolutional layers has two variables. For the first convolutional layer they are named `layer_conv1/kernel:0` and `layer_conv1/bias:0`. The `kernel` variables are the ones we want to plot further below.\n", + "\n", + "It is somewhat awkward to get references to these variables, because we have to use the TensorFlow function `get_variable()` which was designed for another purpose; either creating a new variable or re-using an existing variable. The easiest thing is to make the following helper-function." + ] + }, + { + "cell_type": "code", + "execution_count": 96, + "metadata": {}, + "outputs": [], + "source": [ + "def get_weights_variable(layer_name):\n", + " # Retrieve an existing variable named 'kernel' in the scope\n", + " # with the given layer_name.\n", + " # This is awkward because the TensorFlow function was\n", + " # really intended for another purpose.\n", + "\n", + " with tf.variable_scope(layer_name, reuse=True):\n", + " variable = tf.get_variable('kernel')\n", + "\n", + " return variable" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Using this helper-function we can retrieve the variables. These are TensorFlow objects. In order to get the contents of the variables, you must do something like: `contents = session.run(weights_conv1)` as demonstrated further below." + ] + }, + { + "cell_type": "code", + "execution_count": 97, + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "weights_conv1 = get_weights_variable(layer_name='layer_conv1')\n", + "weights_conv2 = get_weights_variable(layer_name='layer_conv2')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## TensorFlow Run" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create TensorFlow session\n", + "\n", + "Once the TensorFlow graph has been created, we have to create a TensorFlow session which is used to execute the graph." + ] + }, + { + "cell_type": "code", + "execution_count": 98, + "metadata": {}, + "outputs": [], + "source": [ + "session = tf.Session()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Initialize variables\n", + "\n", + "The variables for the TensorFlow graph must be initialized before we start optimizing them." + ] + }, + { + "cell_type": "code", + "execution_count": 99, + "metadata": {}, + "outputs": [], + "source": [ + "session.run(tf.global_variables_initializer())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Helper-function to perform optimization iterations" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are 55,000 images in the training-set. It takes a long time to calculate the gradient of the model using all these images. We therefore only use a small batch of images in each iteration of the optimizer.\n", + "\n", + "If your computer crashes or becomes very slow because you run out of RAM, then you may try and lower this number, but you may then need to do more optimization iterations." + ] + }, + { + "cell_type": "code", + "execution_count": 100, + "metadata": {}, + "outputs": [], + "source": [ + "train_batch_size = 64" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This function performs a number of optimization iterations so as to gradually improve the variables of the neural network layers. In each iteration, a new batch of data is selected from the training-set and then TensorFlow executes the optimizer using those training samples. The progress is printed every 100 iterations." + ] + }, + { + "cell_type": "code", + "execution_count": 101, + "metadata": {}, + "outputs": [], + "source": [ + "# Counter for total number of iterations performed so far.\n", + "total_iterations = 0\n", + "\n", + "def optimize(num_iterations):\n", + " # Ensure we update the global variable rather than a local copy.\n", + " global total_iterations\n", + "\n", + " for i in range(total_iterations,\n", + " total_iterations + num_iterations):\n", + "\n", + " # Get a batch of training examples.\n", + " # x_batch now holds a batch of images and\n", + " # y_true_batch are the true labels for those images.\n", + " x_batch, y_true_batch = data.train.next_batch(train_batch_size)\n", + "\n", + " # Put the batch into a dict with the proper names\n", + " # for placeholder variables in the TensorFlow graph.\n", + " feed_dict_train = {x: x_batch,\n", + " y_true: y_true_batch}\n", + "\n", + " # Run the optimizer using this batch of training data.\n", + " # TensorFlow assigns the variables in feed_dict_train\n", + " # to the placeholder variables and then runs the optimizer.\n", + " session.run(optimizer, feed_dict=feed_dict_train)\n", + "\n", + " # Print status every 100 iterations.\n", + " if i % 100 == 0:\n", + " # Calculate the accuracy on the training-set.\n", + " acc = session.run(accuracy, feed_dict=feed_dict_train)\n", + "\n", + " # Message for printing.\n", + " msg = \"Optimization Iteration: {0:>6}, Training Accuracy: {1:>6.1%}\"\n", + "\n", + " # Print it.\n", + " print(msg.format(i + 1, acc))\n", + "\n", + " # Update the total number of iterations performed.\n", + " total_iterations += num_iterations" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Helper-function to plot example errors" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Function for plotting examples of images from the test-set that have been mis-classified." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "def plot_example_errors(cls_pred, correct):\n", + " # This function is called from print_test_accuracy() below.\n", + "\n", + " # cls_pred is an array of the predicted class-number for\n", + " # all images in the test-set.\n", + "\n", + " # correct is a boolean array whether the predicted class\n", + " # is equal to the true class for each image in the test-set.\n", + "\n", + " # Negate the boolean array.\n", + " incorrect = (correct == False)\n", + " \n", + " # Get the images from the test-set that have been\n", + " # incorrectly classified.\n", + " images = data.test.images[incorrect]\n", + " \n", + " # Get the predicted classes for those images.\n", + " cls_pred = cls_pred[incorrect]\n", + "\n", + " # Get the true classes for those images.\n", + " cls_true = data.test.cls[incorrect]\n", + " \n", + " # Plot the first 9 images.\n", + " plot_images(images=images[0:9],\n", + " cls_true=cls_true[0:9],\n", + " cls_pred=cls_pred[0:9])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Helper-function to plot confusion matrix" + ] + }, + { + "cell_type": "code", + "execution_count": 103, + "metadata": {}, + "outputs": [], + "source": [ + "def plot_confusion_matrix(cls_pred):\n", + " # This is called from print_test_accuracy() below.\n", + "\n", + " # cls_pred is an array of the predicted class-number for\n", + " # all images in the test-set.\n", + "\n", + " # Get the true classifications for the test-set.\n", + " cls_true = data.test.cls\n", + " \n", + " # Get the confusion matrix using sklearn.\n", + " cm = confusion_matrix(y_true=cls_true,\n", + " y_pred=cls_pred)\n", + "\n", + " # Print the confusion matrix as text.\n", + " print(cm)\n", + "\n", + " # Plot the confusion matrix as an image.\n", + " plt.matshow(cm)\n", + "\n", + " # Make various adjustments to the plot.\n", + " plt.colorbar()\n", + " tick_marks = np.arange(num_classes)\n", + " plt.xticks(tick_marks, range(num_classes))\n", + " plt.yticks(tick_marks, range(num_classes))\n", + " plt.xlabel('Predicted')\n", + " plt.ylabel('True')\n", + "\n", + " # Ensure the plot is shown correctly with multiple plots\n", + " # in a single Notebook cell.\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Helper-function for showing the performance" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Below is a function for printing the classification accuracy on the test-set.\n", + "\n", + "It takes a while to compute the classification for all the images in the test-set, that's why the results are re-used by calling the above functions directly from this function, so the classifications don't have to be recalculated by each function.\n", + "\n", + "Note that this function can use a lot of computer memory, which is why the test-set is split into smaller batches. If you have little RAM in your computer and it crashes, then you can try and lower the batch-size." + ] + }, + { + "cell_type": "code", + "execution_count": 104, + "metadata": {}, + "outputs": [], + "source": [ + "# Split the test-set into smaller batches of this size.\n", + "test_batch_size = 256\n", + "\n", + "def print_test_accuracy(show_example_errors=False,\n", + " show_confusion_matrix=False):\n", + "\n", + " # Number of images in the test-set.\n", + " num_test = len(data.test.images)\n", + "\n", + " # Allocate an array for the predicted classes which\n", + " # will be calculated in batches and filled into this array.\n", + " cls_pred = np.zeros(shape=num_test, dtype=np.int)\n", + "\n", + " # Now calculate the predicted classes for the batches.\n", + " # We will just iterate through all the batches.\n", + " # There might be a more clever and Pythonic way of doing this.\n", + "\n", + " # The starting index for the next batch is denoted i.\n", + " i = 0\n", + "\n", + " while i < num_test:\n", + " # The ending index for the next batch is denoted j.\n", + " j = min(i + test_batch_size, num_test)\n", + "\n", + " # Get the images from the test-set between index i and j.\n", + " images = data.test.images[i:j, :]\n", + "\n", + " # Get the associated labels.\n", + " labels = data.test.labels[i:j, :]\n", + "\n", + " # Create a feed-dict with these images and labels.\n", + " feed_dict = {x: images,\n", + " y_true: labels}\n", + "\n", + " # Calculate the predicted class using TensorFlow.\n", + " cls_pred[i:j] = session.run(y_pred_cls, feed_dict=feed_dict)\n", + "\n", + " # Set the start-index for the next batch to the\n", + " # end-index of the current batch.\n", + " i = j\n", + "\n", + " # Convenience variable for the true class-numbers of the test-set.\n", + " cls_true = data.test.cls\n", + "\n", + " # Create a boolean array whether each image is correctly classified.\n", + " correct = (cls_true == cls_pred)\n", + "\n", + " # Calculate the number of correctly classified images.\n", + " # When summing a boolean array, False means 0 and True means 1.\n", + " correct_sum = correct.sum()\n", + "\n", + " # Classification accuracy is the number of correctly classified\n", + " # images divided by the total number of images in the test-set.\n", + " acc = float(correct_sum) / num_test\n", + "\n", + " # Print the accuracy.\n", + " msg = \"Accuracy on Test-Set: {0:.1%} ({1} / {2})\"\n", + " print(msg.format(acc, correct_sum, num_test))\n", + "\n", + " # Plot some examples of mis-classifications, if desired.\n", + " if show_example_errors:\n", + " print(\"Example errors:\")\n", + " plot_example_errors(cls_pred=cls_pred, correct=correct)\n", + "\n", + " # Plot the confusion matrix, if desired.\n", + " if show_confusion_matrix:\n", + " print(\"Confusion Matrix:\")\n", + " plot_confusion_matrix(cls_pred=cls_pred)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Performance before any optimization\n", + "\n", + "The accuracy on the test-set is very low because the variables for the neural network have only been initialized and not optimized at all, so it just classifies the images randomly." + ] + }, + { + "cell_type": "code", + "execution_count": 105, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Accuracy on Test-Set: 0.0% (0 / 10000)\n" + ] + } + ], + "source": [ + "print_test_accuracy()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Performance after 1 optimization iteration\n", + "\n", + "The classification accuracy does not improve much from just 1 optimization iteration, because the learning-rate for the optimizer is set very low." + ] + }, + { + "cell_type": "code", + "execution_count": 106, + "metadata": {}, + "outputs": [ + { + "ename": "InvalidArgumentError", + "evalue": "logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]\n\nCaused by op 'SoftmaxCrossEntropyWithLogits_1', defined at:\n File \"c:\\pf\\python36\\Lib\\runpy.py\", line 193, in _run_module_as_main\n \"__main__\", mod_spec)\n File \"c:\\pf\\python36\\Lib\\runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel_launcher.py\", line 16, in \n app.launch_new_instance()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\traitlets\\config\\application.py\", line 658, in launch_instance\n app.start()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelapp.py\", line 486, in start\n self.io_loop.start()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 127, in start\n self.asyncio_loop.run_forever()\n File \"c:\\pf\\python36\\Lib\\asyncio\\base_events.py\", line 421, in run_forever\n self._run_once()\n File \"c:\\pf\\python36\\Lib\\asyncio\\base_events.py\", line 1426, in _run_once\n handle._run()\n File \"c:\\pf\\python36\\Lib\\asyncio\\events.py\", line 127, in _run\n self._callback(*self._args)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 117, in _handle_events\n handler_func(fileobj, events)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n return fn(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 450, in _handle_events\n self._handle_recv()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 480, in _handle_recv\n self._run_callback(callback, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 432, in _run_callback\n callback(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n return fn(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 283, in dispatcher\n return self.dispatch_shell(stream, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 233, in dispatch_shell\n handler(stream, idents, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 399, in execute_request\n user_expressions, allow_stdin)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\ipkernel.py\", line 208, in do_execute\n res = shell.run_cell(code, store_history=store_history, silent=silent)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\zmqshell.py\", line 537, in run_cell\n return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2662, in run_cell\n raw_cell, store_history, silent, shell_futures)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2785, in _run_cell\n interactivity=interactivity, compiler=compiler, result=result)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2903, in run_ast_nodes\n if self.run_code(code, result):\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2963, in run_code\n exec(code_obj, self.user_global_ns, self.user_ns)\n File \"\", line 1, in \n cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_true, logits=logits)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\ops\\nn_ops.py\", line 1597, in softmax_cross_entropy_with_logits\n precise_logits, labels, name=name)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\ops\\gen_nn_ops.py\", line 2385, in _softmax_cross_entropy_with_logits\n features=features, labels=labels, name=name)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py\", line 767, in apply_op\n op_def=op_def)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 2630, in create_op\n original_op=self._default_original_op, op_def=op_def)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 1204, in __init__\n self._traceback = self._graph._extract_stack() # pylint: disable=protected-access\n\nInvalidArgumentError (see above for traceback): logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]\n", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mInvalidArgumentError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_call\u001b[1;34m(self, fn, *args)\u001b[0m\n\u001b[0;32m 1326\u001b[0m \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1327\u001b[1;33m \u001b[1;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 1328\u001b[0m \u001b[1;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mOpError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run_fn\u001b[1;34m(session, feed_dict, fetch_list, target_list, options, run_metadata)\u001b[0m\n\u001b[0;32m 1305\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarget_list\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1306\u001b[1;33m status, run_metadata)\n\u001b[0m\u001b[0;32m 1307\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mc:\\pf\\python36\\Lib\\contextlib.py\u001b[0m in \u001b[0;36m__exit__\u001b[1;34m(self, type, value, traceback)\u001b[0m\n\u001b[0;32m 87\u001b[0m \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 88\u001b[1;33m \u001b[0mnext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mgen\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 89\u001b[0m \u001b[1;32mexcept\u001b[0m \u001b[0mStopIteration\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\errors_impl.py\u001b[0m in \u001b[0;36mraise_exception_on_not_ok_status\u001b[1;34m()\u001b[0m\n\u001b[0;32m 465\u001b[0m \u001b[0mcompat\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mas_text\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mpywrap_tensorflow\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTF_Message\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstatus\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 466\u001b[1;33m pywrap_tensorflow.TF_GetCode(status))\n\u001b[0m\u001b[0;32m 467\u001b[0m \u001b[1;32mfinally\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mInvalidArgumentError\u001b[0m: logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]", + "\nDuring handling of the above exception, another exception occurred:\n", + "\u001b[1;31mInvalidArgumentError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0moptimize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnum_iterations\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[1;32m\u001b[0m in \u001b[0;36moptimize\u001b[1;34m(num_iterations)\u001b[0m\n\u001b[0;32m 22\u001b[0m \u001b[1;31m# TensorFlow assigns the variables in feed_dict_train\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 23\u001b[0m \u001b[1;31m# to the placeholder variables and then runs the optimizer.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 24\u001b[1;33m \u001b[0msession\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0moptimizer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mfeed_dict_train\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 25\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 26\u001b[0m \u001b[1;31m# Print status every 100 iterations.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36mrun\u001b[1;34m(self, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m 893\u001b[0m \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 894\u001b[0m result = self._run(None, fetches, feed_dict, options_ptr,\n\u001b[1;32m--> 895\u001b[1;33m run_metadata_ptr)\n\u001b[0m\u001b[0;32m 896\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 897\u001b[0m \u001b[0mproto_data\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run\u001b[1;34m(self, handle, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m 1122\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mfinal_fetches\u001b[0m \u001b[1;32mor\u001b[0m \u001b[0mfinal_targets\u001b[0m \u001b[1;32mor\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mhandle\u001b[0m \u001b[1;32mand\u001b[0m \u001b[0mfeed_dict_tensor\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1123\u001b[0m results = self._do_run(handle, final_targets, final_fetches,\n\u001b[1;32m-> 1124\u001b[1;33m feed_dict_tensor, options, run_metadata)\n\u001b[0m\u001b[0;32m 1125\u001b[0m \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1126\u001b[0m \u001b[0mresults\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_run\u001b[1;34m(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m 1319\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mhandle\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1320\u001b[0m return self._do_call(_run_fn, self._session, feeds, fetches, targets,\n\u001b[1;32m-> 1321\u001b[1;33m options, run_metadata)\n\u001b[0m\u001b[0;32m 1322\u001b[0m \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1323\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_do_call\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0m_prun_fn\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeeds\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetches\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_call\u001b[1;34m(self, fn, *args)\u001b[0m\n\u001b[0;32m 1338\u001b[0m \u001b[1;32mexcept\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1339\u001b[0m \u001b[1;32mpass\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1340\u001b[1;33m \u001b[1;32mraise\u001b[0m \u001b[0mtype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0me\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnode_def\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mop\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 1341\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1342\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0m_extend_graph\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mInvalidArgumentError\u001b[0m: logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]\n\nCaused by op 'SoftmaxCrossEntropyWithLogits_1', defined at:\n File \"c:\\pf\\python36\\Lib\\runpy.py\", line 193, in _run_module_as_main\n \"__main__\", mod_spec)\n File \"c:\\pf\\python36\\Lib\\runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel_launcher.py\", line 16, in \n app.launch_new_instance()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\traitlets\\config\\application.py\", line 658, in launch_instance\n app.start()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelapp.py\", line 486, in start\n self.io_loop.start()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 127, in start\n self.asyncio_loop.run_forever()\n File \"c:\\pf\\python36\\Lib\\asyncio\\base_events.py\", line 421, in run_forever\n self._run_once()\n File \"c:\\pf\\python36\\Lib\\asyncio\\base_events.py\", line 1426, in _run_once\n handle._run()\n File \"c:\\pf\\python36\\Lib\\asyncio\\events.py\", line 127, in _run\n self._callback(*self._args)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 117, in _handle_events\n handler_func(fileobj, events)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n return fn(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 450, in _handle_events\n self._handle_recv()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 480, in _handle_recv\n self._run_callback(callback, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 432, in _run_callback\n callback(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n return fn(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 283, in dispatcher\n return self.dispatch_shell(stream, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 233, in dispatch_shell\n handler(stream, idents, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 399, in execute_request\n user_expressions, allow_stdin)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\ipkernel.py\", line 208, in do_execute\n res = shell.run_cell(code, store_history=store_history, silent=silent)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\zmqshell.py\", line 537, in run_cell\n return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2662, in run_cell\n raw_cell, store_history, silent, shell_futures)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2785, in _run_cell\n interactivity=interactivity, compiler=compiler, result=result)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2903, in run_ast_nodes\n if self.run_code(code, result):\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2963, in run_code\n exec(code_obj, self.user_global_ns, self.user_ns)\n File \"\", line 1, in \n cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_true, logits=logits)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\ops\\nn_ops.py\", line 1597, in softmax_cross_entropy_with_logits\n precise_logits, labels, name=name)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\ops\\gen_nn_ops.py\", line 2385, in _softmax_cross_entropy_with_logits\n features=features, labels=labels, name=name)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py\", line 767, in apply_op\n op_def=op_def)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 2630, in create_op\n original_op=self._default_original_op, op_def=op_def)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 1204, in __init__\n self._traceback = self._graph._extract_stack() # pylint: disable=protected-access\n\nInvalidArgumentError (see above for traceback): logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]\n" + ] + } + ], + "source": [ + "optimize(num_iterations=1)" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Accuracy on Test-Set: 14.4% (1443 / 10000)\n" + ] + } + ], + "source": [ + "print_test_accuracy()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Performance after 100 optimization iterations\n", + "\n", + "After 100 optimization iterations, the model has significantly improved its classification accuracy." + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Wall time: 2.07 s\n" + ] + } + ], + "source": [ + "%%time\n", + "optimize(num_iterations=99) # We already performed 1 iteration above." + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Accuracy on Test-Set: 76.8% (7681 / 10000)\n", + "Example errors:\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm8l3P+//HHK6GILFm6oY75SbLnm8LYBmXswtDYjWWsM/YxDJOx3AjJrvC1jGY0lUIUZZcsRSrZ9/JNhVCY0Pv3x+d6fa7rczrL5zrns56e99ut2/mc6/O+rut9ep/z/ryu92ohBEREJD+typ0BEZFqokpTRCQFVZoiIimo0hQRSUGVpohICqo0RURSUKUpIpKCKk0RkRRUaYqIpNC6OSd36NAh1NTUFCgr1WHKlCnzQwhrlTsfpaIybvlUxuk0q9Ksqalh8uTJzblE1TGzT8qdh1JSGbd8KuN09HguIpKCKk0RkRRUaYqIpKBKU0QkBVWaIiIpqNIUEUlBlaaISArNGqdZib7++msAPv3003rTdO7cGYDrr78egM033xyAjTfeGICtttqqmFkUWSZNmTIFgFGjRgEwcuTI7HvvvPMOAL79jpkB8D//8z8AdOvWLZv2r3/961LHSkmRpohIClUfaY4ZMwaARx55BIBnnnkGgPfee6/ec7p27QrAxx9/DMB///vfnPeXLFlS4FyKtGxDhgzJvn777bcBeP7553PSeKTpUWRyU0c/9sc//hGAvn37AtCnT58i5bjpFGmKiKRQFZHmBx98AMAtt9wC5H6q/fDDD0Dup1ZjvP1ERArDI0SIo8aVVloJiNsezzzzTAA22WQTADp06JA956CDDipJPgtBkaaISApVEWnOmjULgEGDBjXrOv4J573lUnnef/99AObPn5895r2t3l7dqlXms/7kk08GYIcddsim7dKlSymyKbUkI8XRo0cDcYT56quvliVPxaJIU0QkhbJHmsmIwiPJHXfcEYDf/va3AKywwgoAtG/fHoB27dplz1m4cCEAe+65JxBHkb169QKge/fu2bRt27YFYOWVVy7wTyFNNX36dCBur37wwQcBmDdvXqPnvvTSSwAsv/zy2WM+MsJ/h2644QYg/h2S4rj99tuzr1977TUAPvkks2Slj5nu1KlT6TNWBIo0RURSUKUpIpJC2R7PFy1aBEDv3r2zx9544w0gbkh222+/PQCvv/46kFme33nov/766wNxJ4FUpmnTpgHx4/iwYcMA+Oabb3LSeXkC7LTTTkBc7tdccw0QT7F7+eWXs2m//PJLAB577DEgnhLrnUZSHGutFW+3c+KJJwLwt7/9DYib4PR4LiKyDCp5pLl48WIADj/8cCCOLgEuvPBCAPbYY486z61rx7yW8unVkiUHPvvwododPV7mW2yxBQBXXnll9r02bdrkpJ00aRIAt912GwDHHXdc9r2pU6cCsO666wJw6qmnAnDwwQcDuRGRFIdPQ/YJJzNnzsz5vi4+PMkHxFcyRZoiIimULNL0oUEeQfgCG8lP/vPOOw+ojk8bqd+PP/4IwIABAwC44447su95tLH22msDcMoppwBx2eczHMzbLX/++WcALr300ux7PvTMF2OR0kg+Odx1111APJ3ymGOOAZZe9i0ZefoCHUcccQRQ2dMqFWmKiKRQskjTe8SvuuoqIF4IOLl8lA9el+rm0x29lzsZUay33npAPIi9Z8+ejV7vl19+AeCzzz4D4OijjwZgn332AeKFp+ty1FFHAbDaaqvlnX/Jn0eYO++8c/aYD2qvvYCwTzhwyScQHxDvvxcejfoUzOSCw+V+ElWkKSKSQskizRdffDHne5/emByPJy2DtzUut9xyS73nUx59bOWIESOAeOFa51NeAd56662cr76k2Jw5c+rNwzrrrAPEYwWTUy2lcLzcksst+kiF4cOHN3juSSedlH3tYznvv/9+IH4y3XbbbQHYdNNNs2n9utruQkSkCliaxXtr69GjR5g8eXJeab231D9RVlxxRQAuuOCCbJr9998fyF1ko9KY2ZQQQo9y56NU0pSx84WhfSzu+PHjs+99//33QP1j9lq3zjz8eLSaj+QsMO91vfHGGwHo2LFj3tdxKuPK4QuOJ9s/vc107NixQNx2mkZzyliRpohICqo0RURSKNnjuQ8h8K918Y4DX1zB18T0oSYbbbQRAJttttlS57755ptAvLhHsTqY9OiW3oIFC7KvfcjZxIkTAVhzzTWBeDqs7wyanF6bXJCjLj5AHuLJE80ZYqQyrjzJdXd9eNNXX30FwK233gqkGxCvx3MRkRIp2ZCjc889F4Drrruu3jQ+iNmXDfOvaXiH06677grAAw88kPoaUljJqM8jzcb4AHZYOtJcddVVARg4cCAAxx57bPa9uoY5SfVL7lzpq8Sfc845QPxk6stE+q6XxaJIU0QkhZK1aXoU6dOlfGL+Tz/9lE3ju0562ubwttPkYg4+0LmZ11V7VxH5Ih/Jskr+jgAMHToUiIc0FZrKuDp4O6e3cfoA+3zqD7VpioiUSMnaNL2tyadFvfvuu0ulefLJJ4E4sujfvz8Ar7zySur7eQQ9ZcqU1OdK6d15550AXH755cDS0SXEO436ND1Ztnk7p2+HUnsqbrEo0hQRSaHs+54n7b777jnf+9YFHmn6ogvJ7Q18E6frr78egH/9619Fz6cUjpet94R+9913S6VZZZVVgHh7C5+CK8s2X8DFF/dILupRTIo0RURSUKUpIpJCRT2e19anTx8g3qXSOwd85ROA9957D4hXC6/NVwqXyuR7RX377bc5x5N7BT388MPA0it/S+Xw5jGI9/068sgji3IvX+XooosuAmDRokUAPPvss0W5X22KNEVEUqjoSNNXZj7ssMMAGDZs2FJpnn766ZzvfT1G3z/m6quvLmYWpYm8w8cHs9eWjFJ8SqxUHt/TxzvyIN7nvimRpu85NGrUqJzjye99goxHtP/85z8B2GSTTVLfrykUaYqIpFDRkabvEzNo0CAgjk6SA9a/+OILAGpqaoB4oQcfGC+VZeHChUD8FLF48eKc97faaisgLnOpDsnp2IMHDwZg5MiRQLxkm6fxQei+LCDEw4bq2xs9uR+QT8H2vo7kYh6loEhTRCSFio40ne8sOGbMGCBuwwCYNGkSEEeWvjScVKannnoKgNmzZ9f5vi/31qZNm5LlSZrOo8hx48Zlj3nU6Lw9cu7cuUA8CD25ILm3g3rU2Ldv35xrJNsrte+5iEgVqYpIs7ajjjqqztdS+S6++OI6j59//vkA7LbbbqXMjhTInnvuWedriKe/thSKNEVEUqjKSFOql2+G5bwNuthbFIgUiiJNEZEUVGmKiKSgx3MpqbPPPjvnq3cMdezYsWx5EklDkaaISAqKNKWkzjrrrJyvItVGkaaISArN2vfczOYBnxQuO1WhcwhhrXJnolRUxi2fyjidZlWaIiLLGj2ei4ikoEpTRCSFBitNM1vTzKZG/+aY2ezE9ysUK1NmNsvMpkf3eTmP9CeY2bwo/Vtm9odm3v9+Mzswz7Tbm9kv+aavNGUs47PN7M3o3xl5pC95GZvZBYn/izfN7Gcza9+c+5aD/o4bTHOQmU2L7vmqme3Q6IVDCHn9A/oD59Zx3IBW+V4nz3vNAlZLkf4EYFD0el1gPtChVprWKa53P3BgHulaA08D4/JJX+n/SlXGwNbAG0BbYPno/3DDSizjRPq+wBPlLqNqKePomhX/dwy0I+7b2QaY0dh1m/R4bmYbmdkMM7sdeA3YwMwWJN7vZ2Z3Rq/XMbMHzWyymb1iZts15Z75CiHMAT4GOpnZ5WY22MzGA3ebWWszGxjlY5qZnRDlsZWZ3WpmM83sESDf9fPPBB4gU7gtSpHLuBswKYTwQwjhJ+A5MpVSXkpcxu73wL9TnlPR9HcMIYSFIaoxgZWBRnvGm9OmuSlwVwihO1D3MtwZNwIDQgg9gEMBL4ReUWHVJQBPmdkUMzs+TabMbCOgM/BhdKg7sF8I4SjgJGBuCKEnsC1wmpl1Ag4BNgQ2B04Bdkhc7woz27uO+3QC9gHuSJO/KlOsMp4O7Gpma5jZysBewAb5ZqpUZZx4vx2wB/BgvnmsIsv033H03iFm9g4wmky026DmzAj6IITwah7p9gC6Wry0/epm1jaE8DJQXztHrxDC52a2LjDezN4KIbzYyH2OMLNdgMXACSGEBdE9Hwoh/Bil6QN0M7N+0fftgS7AzsC/QwhLgFlm9oxfNIRwUT33GwScH0JYkvjZWpqilHEIYYaZDQQmAAuB14Gf87hPqcvYHQA8G0L4Jo88Vptl/e+YEMIIYISZ/Qa4LLp+vZpTaS5KvF5Cpk3EJTd4MaBnCCF328EGhBA+j77OMbOHgJ5AY//ZQ0MIdS3KmMynAaeGEJ5MJjCzvuQRltfSAxgeFWgHoI+Z/RJCeCTldSpZMct4CDAEwMwGAO/ncVqpy9j1A/7ZaKrqtKz/HSfz+7SZ3Wdmq4UQFtSXriBDjqKa/Wsz62Jmrchtn5oAnObfmNnWDV3LzNpFj0P+WNQbmBF9/2czO7kZWX0cONXMWkfX62pmbcm0qfWL2kTWA3Zp7EIhhE4hhJoQQg2ZsP6kFlZh5ihkGUdp1o6+1pCJ5IZF31dMGUfnr07mMa/Flq1bFv+Oo3Zdi173AGiowoTCjtP8C5le5CfJ9Jq504BfRw22M4ETowzW1xbSEZhoZm+QCftHhRAmRO91A75sRh4HA+8BU81sBnAbmWh7BPApmUK9mcx/PlE+G2zvWsYUqowBRkdpRwN/TDz6VloZHwyMDSH80Iw8VZNl7e/4UGCGmU0l0257WGM3r6pplGb2KHBACCGf9i+pQirjlq/ay7iqKk0RkXLTNEoRkRRUaYqIpKBKU0QkBVWaIiIpNGuPoA4dOoSampoCZaU6TJkyZX5Yhlb1Vhm3fCrjdJpVadbU1DB58uTmXKLqmNkytS2AyrjlUxmno8dzEZEUVGmKiKSgSlNEJAVVmiIiKajSlJIaOHAgAwcOxMwwMyZNmsSkSZPKnS2RvKnSFBFJoVlDjkTSGjRoULmzICWyYEFmWcpVV10VgFatWkaM1jJ+ChGRElGkKUX32WefLfX6uuuuA2D77bcvS56k+H73u98BsPLKKwNwwgmZPcv23Xffotxv7ty5AKyxxhoAtG5dnOpNkaaISApVH2l+8klmNtSNN94IkJ0OdssttwCw+eablydjkjV8+PCljq2//vplyImU0jbbbAPAgAEDANhll7y2ZWoyby//6aefALjmmmuKch9FmiIiKVRVpPnuu+8CcPPNN2eP3XfffQB8803ultS//e1vARgzZkz2mLende7cGYAtt9yyeJmVrLoiTbVltnwbbLBBSe4zfvx4IDMGGOC///0voEhTRKQiVHSkuWTJEgBmzpwJQO/evQGYM2dOo+fOnj0byG1H+fbbb4E4ynnhhReAljN+rNJ4ZP/SSy9lj3n0UaooRMrn1ltvLcl9nn76aSCOML0ttVhUW4iIpKBKU0QkhYp8PJ83bx4AN910EwCXXXZZvWlXW201IH709kd658eT3n777Zy0ejwvjuuvv36pY4ccckiTr+cLeyQHy0Pu478PqFZHU/nMmDEDgM8//7wk95swYULO93//+9+Lej/VFiIiKVRkpHnRRRcBcMcdd+QcX2GFFQC44YYbssc23HBDAPr37w/kRh21rbVWZh+lhx56CCjeNCvJmDVr1lLHtttuu9TX8QjzsMMOA5aONJM8un3xxRcBRZzl4OVVexigT6csFO/48cHsbdu2BWDnnXcu6H1qU6QpIpJC2UOtZBukt3d5JOhtjT4I/c477wTiwawAZ555JhC3UzbEhyIo+qguHj3WjjDrWvTDo9Gzzz4bQAscl8jChQuzr71cXN++fQE46aSTCnpPryemTp2ac33v5ygWRZoiIimUPdL0hTYARo0alfPeJptsAsAFF1wAwI477gjAjz/+mPf1N9544+zrwYMHNzmfUlrJCLH2NMxhw4YBcOihhy51nreZ1jV1U4rnrLPOyr5+5513ct4rVm/2//7v/xbluo1RpCkikkLZIk3v8br66qvrTePtlP369cs57ouMApxxxhlAPFZr4sSJOWn/8Ic/ZF/7Qh1S+eoa49lQhFmf//znP6nPkfw9/PDDQN2RfU1NDQBdu3Yt6D29V/6LL74o6HXzpUhTRCSFskWa3jPu4yxh6YU4fNzViiuuCMDpp58OxD2jEPeo1o5YvW3rlFNOKWS2JYW6Fhqua+xmkpdnMnLxslS0WDl8pp3P1qs9JhPiPoo2bdoU9N4fffQREPeau+OPP76g96mPIk0RkRRUaYqIpFC2x/PlllsOgMceeyx7zFdZ9+mNW2+9NRAPPXLJgbQ+fdKHIa2yyioA3HvvvUC857KUng9DSXbqnHPOOUBuE0uSv5/ki3Dkwx/vfb1OPdIXh+/86HtyJflg9i222KKkeVpzzTVLch9FmiIiKZR9cHtyytORRx6Z1zkjR47Mvq49IN6n0SUHtUt5eLSXXKTDF1Tx/VzqiziTGtu50ocVJa9feyqfFMbjjz8OwIUXXphzvEuXLtnXvhOsP02GEABYtGhRvdddfvnlgXgoovNFPsys3nM9sv3Vr37V+A9QAIo0RURSKHukmcZXX30F1B1FdOrUCYg/5aRyJKfY+ZOAt116ZOhp0uxc6RHmueeemz3m0W2adlDJny+S8dprr+Uc92XaAK644oqc93755RcAbr/99nqv2717dwBef/31nONDhw4FYL/99ssee+KJJ3LS+NNqQ9FoISnSFBFJoaoizX333ReA6dOnL/XeJZdcAsQLFUvlSPZg++D2QYMGAXFk2dACG754h38dMWJEvef44sPa7bI4kv0JSZ9++mn2dVOe9nxEjD8x+qiXI444AoAePXpk0/p2OK7UE1gUaYqIpFAVkeaHH34IxBs2JXn0eeyxx5YyS9JE3lvubY4+hrOuBTqct4PW5r3yyd5zRZjFdfHFFwPxQjkuuRjOOuusAyy9vcVuu+0GwLbbbrvUdXv27AnA999/D0CHDh0AeO6554B4k0WIx4b6OO5Sj5RRpCkikoIqTRGRFCr68Xz27NkA7L777gB89913QNxYDEsPpJXq4I/RPsjdv3rZ1rXjpA9L0qpH5XPyyScD0KtXr5zjHTt2zL729W5XWmml1NdfffXVc77v3bs3EA91SvKhaO3bt099n+ZQpCkikkJFR5o+gPbjjz/OOZ5cjT0ZdUr18x1J6+oY8uFKPuTIo1XtLlo6vphOXZ05xZTcZ8ifOJLRbSkp0hQRSaEiI81XXnkFgKOPPjrnuK/gvvfee5c8T1Ia3raZ5JGl8zbNxhbykJZjrbXWqvN1OSjSFBFJoaIiTV86ytsvFixYkPO+96y1a9eutBmTkktGnHVFnyLlokhTRCSFioo0hwwZAsC4ceNyjq+77roAjB07FoBu3bqVNmMiIhFFmiIiKVRUpOmzenxRUR+PdeKJJwLlG5clIuIUaYqIpKBKU0QkhYp6PP/Tn/6U81VEpNIo0hQRSUGVpohICqo0RURSsBBC0082mwd8UrjsVIXOIYTyrhhQQirjlk9lnE6zKk0RkWWNHs9FRFJQpSkikoIqTRGRFBqsNM1sTTObGv2bY2azE9+vUKxMmdnZZvZm9O+MPNKfYGbzony9ZWZ/aOycRq53v5kd2EiaCxL/F2+a2c9mVtpt8QqgjGW8hpk9aGZvR2XWs5H0JS/jRNrtzeyXfNNXmnKVcXTv1mY2zcxG55H28kTeppvZPs289wtmtnUe6X5vZjOjv+P7Gkvf4IygEMKXwNbRhfsDC0MI19a6oZHpUFrS2M3yEf2QxwA9gJ+BJ8xsTAjho0ZOHRpCONPM1gVmmNnDIYT5ieu2DiH8XIg8AoQQrgKuiq7dFzglhPBNoa5fKuUo48hNwMMhhIOiP9y2eZxT0jL2awJXAuMLed1SKmMZA5wNzADy3c/3mhDCIDPbHHjazNYOid7qQpexmW0CnAvsEEJYYGZrN3ZOkx7PzWwjM5thZrcDrwEbmNmCxPv9zOzO6PU6UUQx2cxeMbPtGrl8N2BSCOGHEMJPwHNA33zzFkKYA3wMdIo+uQab2Xjg7uhTb2CUj2lmdkKUx1Zmdmv0afMI0CHFfwfA74F/pzynohWzjM1sDaBXCOEegBDC4jQfOCUu4zOBB4D5jSWsNkX+O8bMOgO9gbvT5i2EMAMwYPXoqeA6M3sauNLM2pnZPVE+Xjez/aL7rWRmw6NyfwBok8etTgJuCiEsiO47t7ETmtOmuSlwVwihOzC7gXQ3AgNCCD2AQwEvhF5RYdU2HdjVMo9vKwN7ARvkmykz2wjoDHwYHeoO7BdCOIrMf9DcEEJPYFvgNDPrBBwCbAhsDpwC7JC43hVmVu9ObmbWDtgDeDDfPFaRYpXxr4B5ZnZf9Es/xMzyjURKVsbRefsAd+SbtypUrDIGGAScB6Qe12hmOwA/hhC+ig79P2D3EML5wCXAuKiMdwOuM7M2wOnA1yGELYGryfxe+PXurudRfWOgm5lNNLNJZtansbw1Z8GOD0IIr+aRbg+gayb6BzKfHG1DCC8DL9dOHEKYYWYDgQnAQuB1Mo/pjTnCzHYBFgMnRKE2wEMhhB+jNH3I/Af1i75vD3QBdgb+HT2azDKzZxL5uaiR+x4APFuNj+Z5KEoZk/m96wGcAUwh86h+HnBpI/cpdRkPAs4PISxJ/GwtTVHK2DLtv5+FEKaa2R4p8nOemR0LfAccljg+PNF00AfYy8wuiL5vA3QiU8YDAEIIr5vZm35yCOG4eu7XmsyH+C5kPoifNbNNQwjf1pfB5lSaixKvl5AJpV0yLDagZwhhcb4XDiEMAYYAmNkA4P08ThsaQjizkXwacGoI4clkAsu0STZ1lH8/4J9NPLfSFauMZwGfhBAmA5jZSDKPwY0pdRn3AIZHFUUHoI+Z/RJCeCTldSpZscp4B+AgM9s/us6qZnZvCOGYRs67JoQwqJF8GnBgCOGDZIKonNKW8Szgmaid9AMz+4BMVPt6fScUZMhR9AnwtZl1MbNW5LZBTgBO82/qCZFzWNQYa2Y1ZCK5YdH3fzazk5uR1ceBUy3TuI+ZdTWztmTaTftF7V7rkfnUaZSZrU7ml6Ml/RHVqZBlHEKYBcyNHrMBdgdmRudWTBmHEDqFEGpCCDXAaOCkFlZh5ihwGZ8fQlg/+r87EnjCK0wzG+DtkE30OJBdP9LM/DH8OeCI6NhWwGZ5XGs08JvonLXJVJgNdjoXcpzmX4BxwJNkam93GvDrqHF2JnBilMGG2kJGR2lHA39MPPp2A75sRh4HA+8BU81sBnAbmWh7BPApmV6+m8n85xPls6E2zYOBsSGEH5qRp2pSyDI+AxhmZtPI/HJfFR2vtDJe1hSyjOuzJTCnGXm8FFjJMsOS3gT6R8dvBtaMfqfOAib7CQ20aT4KLIx+pgnAWd4pVJ+qmntuZo8CBxR6WIlUDpVxy2aZZ+hxIYQ9y52XpqqqSlNEpNw0jVJEJAVVmiIiKajSFBFJoVm7UXbo0CHU1NQUKCvVYcqUKfOXpVW9VcYtn8o4nWZVmjU1NUyePLnxhC2ImS1T2wKojFs+lXE6ejwXEUlBlaaISAqqNEVEUlClKSKSgipNEZEUmtV7LpLWggWZtRB+85vfAPD9998D8M4775QtTyJpKNIUEUmhKiPNd999N/v62msz+0M9//zzAHz++ecA3H///QDst19zlu2TQvj666+zr3fffXcA3njjDQA23njjsuRJpKkUaYqIpFBVkeaoUaMAOPnkeGHv7bbLbIp3ww03ALDVVlsBsM4665Q4d1KbR5geXQJMnToVgOWWWw7Qk4BUH0WaIiIplD3STM55ve+++3Le+/nnzOLds2dndhb1drDzzz8/m+acc84pdhalia6++mogji6TTjzxRACuueaakuZJpLkUaYqIpFD2SPPQQw/Nvm7fvj0AW265JQBffPEFAPvvvz8AV1xxBQCbb755KbMoKc2fPx+AcePGLfWel/Gf//znkuZJKoc/OU6bNi17bPjw4QD89NNPAHz88cdAZgUmgKuuuiqbdr311itBLuunSFNEJIWyRZqLFmX2fvdxlQD33HMPADvvvHM5siQF4r3l06dPX+q93//+9wB07dq1pHmS8nnhhRcAuPzyywF48cUXAfjuu++yaTKbVC5t4sSJACy//PLZY0OGDAGgdevyVF+KNEVEUlClKSKSQtkez2fOnAnEDb/ScngDvz9yrbLKKtn3zj777LLkSUrntddeA+Dvf/87AOPHjwdg8eLFOemSU2jXXHNNAPbcc08gXsBlzJgxANx9993ZtMcccwwAu+yyS8Hzng9FmiIiKZQt0vzmm28ACCGUKwtSIiussEL29UYbbdTk6/hCLQsXLsz7nC222ALI7UiQwlmyZAkAAwcOzB4bMGAAEA898w6bI444AoCDDz4YgAMPPLDR6++9994AjB07NnvMOxgVaYqIVIGyRZoPP/wwkF+kWXs65RprrJF978svvwTigfDrrrsuAJ06dQLqH8oglW3ChAnZ14MGDQJg0qRJQO5Sc43x4U877bQTAMceeywQ/35I83iEmZza7H/Tm2yyCQCDBw8GCjeU0Kde+5ClZJt5KSjSFBFJoWyRZkOLz3777bdA3Ov2t7/9DYC3334bgA022CCb9rPPPqvzGnvttRcA5513XvaYb7Eglcujh4svvjh77OWXX85J0717dwBWXXVVADbbbLPsex06dADiRUIef/xxAJ588kkg7oU97rjjsudccsklhfsBlhH33nsvkPv35fxvz5dyXHHFFVNf39uvk1Mta997ww03BOJe+lJRpCkikkJFRZq+RcVRRx0FxFMsDznkEAAuu+wyII4wIF502Hl0OmzYMCAe9wXxQsWnnHJK838AydvRRx+dd1qPMGtHlxAv7uLT6JK/B/W5+eabAbjllluAePxf//79s2kUaab30UcfAXGfwbbbbpt9rykRpi/Q4X0T3gOfnGbt/J6/+93vUua6MBRpioikULZIc+uttwa72IXZAAAJjklEQVRgm222yR7zdiePDv/6178CcdtFPo488sicrzfeeGP2vVNPPRWAHj16ALmfjlI8Xq75+PTTT+t977TTTgPyizDd6aefDsQjMDQjqTiSs32ee+45IF7WzcfIrrbaagD85z//AWDEiBHZc1599VUgHr/dEH9y6dKlSzNz3TSKNEVEUlClKSKSQtkez9dee20ApkyZUtT7+F40AHfeeScQrwT/f//3f0W9t2T4pIRKovU8m2fTTTcF4kdv378LoE+fPkDcYdOmTRsg3iH2k08+AXIntqSZhLLDDjvk3LvUFGmKiKRQ9j2Ciq1t27bZ175zpUefPljep3tJcSQ7Cd5//32gaQt3+HQ8nxLZHMkhR5KeD//6/vvvgdzJCLNmzcpJ++OPPwJxhOkdu8khQ7vuuisQD2XyDlwfIpbcF+jwww8vzA/RRIo0RURSaPGRZpJ/wvnCxz4FU5FmYd1+++1APIkguRfMjjvuCMBNN90ELD1A2Qeh+1RJgNtuuw2A0aNHA/FyYb5IhEcpDak9HU/t2YXhC6D4V4j3BPIB6767rH9tyLPPPgvET4EuuRtlu3btmprdglCkKSKSwjIVafritT69ywe5S2Edf/zxQBxxJntW586dC8R72NeONDt27AjktpGtvPLKAFx77bVAvJ+6D6K+6667smkPO+ywnOv5tiq+r7br3Llzuh9K8uZPE/41DS/j2tMzyzVlsi6KNEVEUlimIk1fvME/xVZfffVyZqfF8u0NHnvsMSC359N5BOhTW316Y1296v6eL2J7zz33APDhhx8Cucu8+dJvPhb3yiuvBGDRokU57+ez1YKUjve4v/TSSznHzz33XKBpy8sViyJNEZEUVGmKiKRQ9sfz5FQqX+GmEI30ft3kfjK+mnevXr2afX1pnO/X9K9//St77OqrrwbiziHvLPK1VL3svTOpLv4I7/v8JCcw+FqOTzzxBBCvt/rKK68AcUeT9o6qLN6Z53t+tW/fHoDevXuXLU/1UaQpIpJC2SPN5MBnX1vTG+v33XdfAFq1yr9u/+qrrwA466yzgHh4CsRrePogaSkuj+b69euXPeaLN/jK7I8++igAM2bMAODNN98E0q17mZyc8Je//AWIy9o7fCqpI0GW5n+3ztfK9DU4K4kiTRGRFMoeaSZX4b7ooosAOOCAAwAYOnQoEC8F5TsNejQCcTvoyJEjAXjqqadyrp/cD+jCCy8EFHWUk0d+/tX3ffK9YXz/nzROPvnk7Gtvs5TKl5z04MPIvC/C/+YrkSJNEZEUyh5pJv3pT38CYMGCBQBccMEFQNze4RFisv3D28h8XyGffue9blqMo7L5QHgfAH/ppZeWMztSQr4HGMR9Gz4SYpdddilLnvKhSFNEJIWKijQ96vjHP/6R81VEWg5fpm/s2LHZYz7SwncP7du3b+kzlidFmiIiKVRUpCkiLZ8vQJ3kM8F22223UmcnNUWaIiIpqNIUEUlBj+ciUlLHHHMMkLsPkA8V9D3TK5kiTRGRFBRpikhJ+d5Bzz//fJlz0jSKNEVEUrDkIsCpTzabB3xSuOxUhc4hhLXKnYlSURm3fCrjdJpVaYqILGv0eC4ikoIqTRGRFBqsNM1sTTObGv2bY2azE9+vUKxMmdksM5se3eflPNKfYGbzovRvmdkfmnn/+82swY2xzewgM5sW3fNVM6vcVVMbUMYyXsPMHjSzt6My69lI+nKU8dHR7+E0M5toZls0557lUsYyPtvM3oz+nZFH+pKXcSLt9mb2Sz7pGxxyFEL4Etg6umh/YGEI4dpaNzMybaNL8slcCjuFEBakSD80hHCmma0LzDCzh0MI8xP5bB1C+LmA+XsCGBVCCGa2DXAfsHkBr18SZSzjm4CHQwgHRX+4bRs7gdKX8QdEv4dmth9wO/DrAl6/JMpRxma2NXAM0AP4GXjCzMaEED5q5NRSlzFm1hq4EhifT/omPZ6b2UZmNsPMbgdeAzYwswWJ9/uZ2Z3R63WiiGKymb1iZts15Z75CiHMAT4GOpnZ5WY22MzGA3ebWWszGxjlY5qZnRDlsZWZ3WpmM83sEaBDHvdZGOJetJWBFtWjVswyNrM1gF4hhHsAQgiLQwjf5Ju3EpbxxMQH90vA+vnmsRoU+e+4GzAphPBDCOEn4Dkg7/XeSlXGkTOBB4D5jSWE5rVpbgrcFULoDsxuIN2NwIAQQg/gUMALoVdUWHUJwFNmNsXM6t8Auw5mthHQGfgwOtQd2C+EcBRwEjA3hNAT2BY4zcw6AYcAG5KJFE8Bdkhc7woz27ueex1iZu8Ao4ET0uSzShSrjH8FzDOz+8zsdTMbYmYr5ZupUpZxwvHA2EbSVKNilfF0YFfLNMOsDOwFbJBvpkpVxtF5+wB35Ju35swI+iCE8Goe6fYAulq0yCiwupm1DSG8DNTXXtkrhPB5FKKPN7O3QggvNnKfI8xsF2AxcEL0SAXwUAjhxyhNH6Cbmfmesu2BLsDOwL+jR5NZZvaMXzSEcFF9NwwhjABGmNlvgMui67ckxSrj1mQe284AppB5VD8PaGyvi5KXMYCZ7QEcBezYSP6qUVHKOIQww8wGAhOAhcDrZB7TG1PqMh4EnB9CWJL42RrUnEpzUeL1EiB5xzaJ1wb0DCEszvfCIYTPo69zzOwhoCfQWKU5NIRwZiP5NODUEMKTyQRm1pdmPF6HEJ6OoqbVUrbDVrpilfEs4JMQwmQAMxtJ5hGpMSUvY8u0zQ0G9gwhfJ32/CpQzL/jIcAQADMbALyfx2mlLuMewPCowuwA9DGzX0IIj9R3QkGGHEU1+9dm1sXMWpHbdjEBOM2/iX4J62Vm7cysnb8GegMzou//bGYnN3R+Ix4HTrVMwy9m1tXM2pJpb+kXtYmsBzS6q1PUHmTR6x4ALazCzFHIMg4hzALmRo9gALsDM6NzK6mMa4ARwOEhhHz+4KtaIcs4SrN29LUGOAAYFn1fMWUcQugUQqgJIdSQaWY7qaEKEwo7TvMvwDjgSTKRhDsN+HXUYDsTOBEabAvpCEw0szfIhP2jQggTove6AV82I4+DgfeAqWY2A7iNTLQ9AviUTOV8M5n/fKJ81tfedSiZ3r2pZNp7DmtGvqpFocoYMo/mw8xsGrAZcFV0vJLKuD+wBjDY8hz+1gIUsoxHR2lHA39MdPZVUhmnVlXTKM3sUeCAQg85kMqhMm75qr2Mq6rSFBEpN02jFBFJQZWmiEgKqjRFRFJQpSkikoIqTRGRFFRpioikoEpTRCSF/w+H2cYDWb+BuAAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "print_test_accuracy(show_example_errors=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Performance after 1000 optimization iterations\n", + "\n", + "After 1000 optimization iterations, the model has greatly increased its accuracy on the test-set to more than 90%." + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimization Iteration: 1001, Training Accuracy: 95.3%\n", + "Optimization Iteration: 1101, Training Accuracy: 96.9%\n", + "Optimization Iteration: 1201, Training Accuracy: 95.3%\n", + "Optimization Iteration: 1301, Training Accuracy: 93.8%\n", + "Optimization Iteration: 1401, Training Accuracy: 93.8%\n", + "Optimization Iteration: 1501, Training Accuracy: 92.2%\n", + "Optimization Iteration: 1601, Training Accuracy: 96.9%\n", + "Optimization Iteration: 1701, Training Accuracy: 92.2%\n", + "Optimization Iteration: 1801, Training Accuracy: 90.6%\n", + "Wall time: 18.1 s\n" + ] + } + ], + "source": [ + "%%time\n", + "optimize(num_iterations=900) # We performed 100 iterations above." + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Accuracy on Test-Set: 94.9% (9494 / 10000)\n", + "Example errors:\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm8leP+//HXJ0WFSJlKtfmllKEiRRyhhJwkY45jLGQeT8fwPYZz5EvnIGM4KEOmzHQUJUPnS5SKZMxYpKIMHQ7p+v2x7mvd99rD2uvea977/Xw89mPf6173cNW117U+13VfgznnEBGRzDQqdgJERMqJCk0RkRhUaIqIxKBCU0QkBhWaIiIxqNAUEYlBhaaISAwqNEVEYlChKSISQ+NsTm7durWrqKjIUVLKw+zZs5c75zYudjoKRXlc/ymP48mq0KyoqGDWrFnZXKLsmNlnxU5DISmP6z/lcTyqnouIxKBCU0QkBhWaIiIxqNAUEYlBhaaISAwqNEVEYlChKSISQ1b9NOvixx9/BOCLL74AYOzYsVWOOeGEEwDo3r174RImBbVy5UoAWrRoAUCjRvr+lvKgv1QRkRgKFmn6CPPvf/87AH/7299qPPbWW28F4IgjjgDg+uuvB2CjjTbKZxKlgA477DAA1l13XQCGDx8OwO9///u83G/p0qVA+DfUuHHBK1lSTyjSFBGJoWBft1deeSUAV111Va3Hrl69GoAJEyYAMG3aNADGjx8PwIABA/KQQimkHXfcEYDRo0cD0Ldv37zeb8yYMQD8+uuvQFjjEYlLkaaISAwFizS33HLLlNdmBsDpp5+e3LftttsC8MsvvwBwySWXALBkyRIABg8eDMCf//zn5DkjR44EoHnz5vlItuRJu3btCnKf559/HoBrr70WgP/+97+AIs18uuKKKwB48803Abj44osB2HrrrYGwx8TPP/+cPOe5554D4Pjjjwdg6tSpAPTo0aMAKY5HkaaISAwFizQff/zxlNeHH344ED4Zr063bt0AOPjggwH45ptvAPjrX/+aPGbhwoUA3HXXXQA0adIkRymWfLrlllsKcp/p06cDYYTp21Ilf3wN0dcmn3zySQC6du0KwMYbJ+b+XbVqVfKcyvN5+prBvffem9/E1oEiTRGRGFRoiojEULDq+bPPPguEIbtvHE7nd7/7HRCG9xdeeCEAr7zySvIY3y3JOQeE3ZLUebk0zZ8/H4Avv/yyIPfzDxS8Sy+9tCD3laoWLFiQ8tp/ZiEsF7xHHnkEgL/85S8AdOrUKc+py5wiTRGRGAoWjvXv3x8IO6qvt956GZ/bp08fIOwIPXDgwOR7K1asAOD+++8H4MADDwTCB01SWl599VUAvvvuu5T9fjhlrvgHP74ze7NmzQDYY489cnofSZgyZUqN7/3zn/8E4LXXXgPCmuL7779f4zm+26Ef6FJKFGmKiMRQsEizS5cuQBhpVueOO+4Awqjx5JNPrva4P/zhD8ntm2++OeW9Dz74IKt0Su75yVoArrnmmpT3hgwZAsBJJ52U03v6dvC5c+emXH/DDTfM6X0kwXf9q86gQYOAcMrHb7/9FggHrUT55xh+6sBSpEhTRCSGgkWaPXv2THn91ltvAalDqfyQSt+e8eKLL8a+z5133gnANttsA8A+++yTfG+DDTaIfT3J3jnnnJPcrtyOla+n2X6wgxTGmjVrktvRp+LV8dPzVTfVo+/14q/ho9JSokhTRCSGgkWaBx10EAD33HMPAHvvvTcAX3/9dfKYpk2bAmGkWRefffYZED49j07k4Z/i+Yk/NMlHfj311FMATJw4scp7FRUVAHTu3Dmn9/RP5aN/V5J/0eVKfJ/Lyn0vM1H53IceegiA3XffPdsk5owiTRGRGAoWafrpoP74xz+m7I/217zvvvuAMDLx7RmTJk2q833/85//JLePOuooALbffnsgHE203Xbb1fn6UtX3338PhEuaVO6TCeEELr52kSuffPIJED4194YNG5bT+0jDpUhTRCQGFZoiIjGU1KwWfiVC//u3334D4Icffkg5LtrI7xuMN9lkk5RjfFeWaNcTP3/f22+/DcB5550HwNVXXw1onfVc8Ss/Vp4jEcLO7L6JpFBatWpV0Ps1NG3atElut23bFijcpCyFpkhTRCSGkoo0ly9fDoRDIf1EHZWHvmUyFM7PCO/XTgc45ZRTgDDS9OvH+O4Sfvo6qRs/acNFF12Ust+vDQPhsNe11loLCDsxR2fxrszPxu8n3/D8JB/purb4yHarrbaq/R8gdea7FEI4jdvYsWOBcLKU+kKRpohIDEWPNJ9++unk9llnnQXAV199BcCDDz4IhJ3R68JHqwAzZswAwnVi/CQDfrqyyZMnA7DffvvV+X4NmZ8kw69C6Plp2gBGjRqV8p5vt7711ltrvK5fkXDOnDkp+32XMT8hBISrGnq+VlKXjtZSN34toBtvvDH2ub7mUdtQzGJSpCkiEkPRI83ok3EfYfrIxK9C6SPEXXfdNat7rb/++kA49ZyPQn1nbP8UXZFm3Tz66KPV7v/888+T25Wn8suEn1quffv2QDhQwg9WiE4Gs2zZspRzfTu2lIdshmAWiiJNEZEYih5pRicU9v26Ro4cCYTtGr7dK1f8tHTR6awAdthhh5zep6Hxi2CdccYZKfs7dOiQ3N50002Bqstb+Alcdt555yrX7dWrFxAOiW3dujUAL7/8MpDadub7hvo+t6W0IJfUD4o0RURiUKEpIhJD0avnUX4dF9/JfPr06QAcc8wxAOy5554AXHDBBclzaqt++U7uEK5B9NFHHwGl3a2hHI0YMQKA3r17p+zffPPNk9t+tu66zGXasmXLlNd+Vn7f1SnKPzTUbP2Sa4o0RURiKKlI03cl8ZGDfzDjuyKNHz8egHvvvTd5TnTG6OpUHnpXHf+g4ZJLLomXYEnh13ep7mFOPkXXGfLrEUWjW5FcUqQpIhJDSUWanp/N/eOPPwbg7rvvBsJhlX7CDajb9FO77bYbAPvuuy8AJ554IqDpw8rVxhtvXO22lJ/Kwyh9t7JSokhTRCSGkow0Kzv22GNTfi9ZsiT5nh+G6Vea9E/YfSfn6NP1nXbaCQiH462zzjp5TLWIxFV5GOX8+fOLmZxqKdIUEYmhLCLNyjbbbLMq26NHj045ZuDAgQVNk4g0DIo0RURiUKEpIhKDCk0RkRhUaIqIxFCWD4JEpH7yK5oefvjhQOGH5GZCkaaISAyKNEWkZHTr1g2A999/v8gpqZkiTRGRGCybiXjNbBnwWe6SUxY6OOcazKwQyuP6T3kcT1aFpohIQ6PquYhIDCo0RURiSFtomlkrM5sb/Cwxs8WR12vnI0Fmtq6ZvR7cY4GZ1boGhZldEUnb22Z2QJZpmGFm3Ws5psLMXjKzOWY2z8z2y+aexVKMPI7cu7GZvWVmT2RwbMHzODjuyODv8B0zuyebexZLkT7HXSP3mGtmP5jZ6bWcM9zMlgXHv2tmJ2SZhvvM7KBajtnIzJ4K/g5nmlnXWi/snMvoB7gMOL+a/QY0yvQ6GdynEbBusN0EmAX0rOWcK4Czg+3tgGUE7bWRYxrHSMMMoHstx9wFnBhs7wB8lKv/g2L9FCqPI9cdCdwPPJHBscXI422A2cCGwetNip1H5ZbHPl+ApcAWtRw3HBgTbG8GLAdaZ5HH9wEH1XLMdcDFwfa2wPO1XbdO1XMz62hm883sVuBNoJ2ZrYy8P9TM7gi2NzWzx8xsVhBB7pLu2s65Nc65VcHLtUkUnBk/rXLOzSfxB9Ay+Ka5xsymA1ea2XpmNj5IxxwzGxSksbmZTQy+bR4EmmZyK6BFsL0BEH/djRKWzzwOzukA7AOMi5u2AubxScCNzrmVwX2Xxk1rKct3HkcMAN51zi3K9ATn3BLgU6B9UMu4zcyeB8YFNZRrg3S8ZWbDgzQ2MrNbgprB00DrDG7VFZgW3PMdoJOZpV33Jps2za7Anc65HsDiNMfdAIx2zvUEDgd8JvQOMqsKM1vbzOYCXwPPOOdmZ5ooM+sD/Oyc+zbY9f+Afs65kcAlwGTnXC9gb+AaM2sKnA6scM7tAFwN9Ihcb1wN1bhLgBPMbBHwJHBWpmksI3nLY2AM8CdifCF6BczjTkAXM/u3mb1qZgPiprUM5DOPvaHAA3ESZWYdgQ7Ax8GuHsAg59zRJL7MlgZ5vDNwmpm1Bw4FtiRREzkF6BO53igzq26S3XnAIcExuwJbBD81ymZE0ELn3BsZHNcf6GzB9PUkooNmzrmZwMzqTnDO/QJ0N7OWwONm1sU5924t9/mTmR0H/AAcEdk/0Tm3JtgeAOxvZhcEr5sC7YE9gNHBveeY2TuRtBxfw/2OAm53zl1vZrsD95rZ9i6I8+uJvORx0M70hXNurpn1j5GeQudxY2AroC+JD/BLZtbVOfd9jDSXurx9jgGCL6wDgHMzTM9RZtYX+AUY7pxbGdzzSefcz8ExA0h8mQ0NXm8AbE0ijx8I/hYWmdmL/qLOuYtruN8o4IYgSJsX/KxOl8BsCs1Vke01JKpLXrTqY0CvoCCMxTm3wsxeAfYFais0/+6cG1NLOo1EG8fC6AFBpsQt7IYBewbpnGFmLYCWwLfpTioz+crjPsDBZnZgcJ0WZna3c+7YWs4rdB4vAl50zq0GFprZQhJR7ZyY1yll+f4cHwDMdM4tz/D4Cc65s6vZXzmPT3XOTYseYGZDiJnHzrnvgGOD8xuRaBL4NN05OelyFJTsK8xs6+DGQyJvTwVO8y9qqAYReX8TM9sg2G4O9APeC16P9m1UdTQFODNyL19Fe5lE5IiZdSPRIFybz4O0YWbbkmhEr08FZopc5rFzbqRzbgvnXAXwR+A5X2CWWB4/AewVnLMJiQLzkyzSVtJymccRR1Kpam5mZ5nZiCySOgU41cwaB9frbGbNSOTx0KBtsy2JGkJaZrahmTUJXp4MTI08U6lWLvtp/hmYTKJRNdrgexqwW9BguwA4MUhsTW0hbUhUg+YBrwOTnHOTg/d2AJZUc06mLgeaW6LLyjskniQC3AS0MrO3gHNIPLEnSGdN7V3nkMi4eSSe0h2XRbrKRa7yOJ1SyuNJwI/Bv2kqcI5/KFSP5SyPzWw9El86lbuUdQG+ySKNtwEfAnPNbD4wlkSt+RESwcx8EvmdXDQ9TZvm9sACM3uPRBBUazNC2QyjtET9arJzbt9ip0XyQ3ncMJjZJGBw0OxRdsqm0BQRKQUaRikiEoMKTRGRGFRoiojEoEJTRCSGrNYIat26tauoqMhRUsrD7Nmzl7sGNKu38rj+Ux7Hk1WhWVFRwaxZs2o/sB4xswa1LIDyuP5THsej6rmISAwqNEVEYlChKSISgwpNEZEYVGiKiMSgQlNEJAYVmiIiMajQFBGJIavO7cXy448/JrdHjEhMAD1hwgQAdt11VwBefPFFANZeO69Ld0uOrF6dmFrRr0Gz1lprFTM5kgNffplYoHXs2LEA3H///QB8/PHHVY49/vjEMk177703AEcckVgCqkmTJlWOLTZFmiIiMZR0pNm1a1cA9ttvPwA+/PBDAF566aXkMXvssQeQGAoG8OqrrwLhN9YBBxyQPPbCCy/Mb4IltmeeeQaAo48+GoDWrRNLVV900UXJY449NrHeWqNG+o4vVWvWrElu+4hy1KhRALz33nu1nj9u3LiU3/7cadMSa6e1adMmd4nNkv4KRURiKKlI8+uvvwZg8ODBALz77rspv6vzf//3f0DYFrbhhhsC8OabbwLwxhvhks7+G+/uu+/OZbIlC9tttx0AJ5xwAgCPPPIIAMOGDUse46OPO+64A4BOnToVMomSAZ83ACeffHLKey1atADgmGOOAaBjx45Vzve1yNtuuw0IP6vnnptY58w/s4Dit3cr0hQRiaHokaZ/yg1w5JFHArBkSfUruHbvnlhl9YILLkju+93vfgeEkaZvW1m6dCkQtm0CvP766zlKteSKb4u+5pprUn4/99xzyWP8k9SddtoJCKOQtm3bFiqZUoMHH3wQgOuuu67Ke507dwbg2WefBWDLLbes9Xp9+yaWKj/rrLMAeOihhwAYPXp08pj27dtnkeLsKdIUEYlBhaaISAxFq577hl/fnQSqVsvXWWcdIKyO+y5Dfn86/uHRypUrk/t23nlnAH799VegNDvOSsKAAQOS2/4hw+GHHw7A22+/Dah6XgpefvllILVb0WabbQbAv/71LyCzarl32GGHAXD55ZcD8NVXX+UknbmkSFNEJIaCR5r+Qc3w4cMB+Pzzz2s81neSPfjgg2Pf57fffquyz3c/8t9exW5QlswccsghQNjVaM6cOUA46EFKix+osNVWWxU5JfmhSFNEJIaCR5pz584FwraQKN9W6bsxHHjggXW+j++qEOXbWhRhFs/ixYuBMH/8kNiWLVsCcOihhwKw++67J89ZtGgRAKtWrQJgyJAhhUms1In/nGXD1y7feecdAB599NHke+ecc07W18+GIk0RkRiK3rk9yj85O+igg+p8DT8U0w+vjBo6dGidryvx/fDDDwBMnDgxuc9HCc45AFq1agWEbdB+iGs0WvE1g969ewOwzTbb5DPZkiU/FNIPgayL+fPnp7xesGBBVmnKJUWaIiIxFDzS3GGHHQCYNGkSEE4NBnD99ddnff0bb7wRgPfffx8Ih+kBnHrqqVlfXzI3ffp0AKZMmZLcd/bZZwPhhBw+ivzll19Sjo22Z/v+u37ih88++wyADh065C3tUnd+8mH/3MJP3xjHBx98kNM05ZIiTRGRGAoeaTZunLjlwIEDU35ny7eRRaeCg9SRJRtvvHFO7iXpzZgxAwj760Wn9fr9739f7Tl+WZLoUiaeH1HiIxfftvnkk0+mvJbC81P6Pf7448l9vmZw+umnp/z2E3h4n3zySXI7+jcCYW2i8jUhrJUUaykbRZoiIjGo0BQRiaGkuhzVZsWKFQB89913yX2+a5Ff4S46DyNAz549C5Q68c4//3wgHEbn50hMx8+5eN555wHh3KkAkydPBmDmzJlAONfinnvuCYTDKkHdkQrNf77OOOOM5L6LL74YCCdWqTyTe11EHxh/++23QG460deFIk0RkRjKItL0EabvpuIfNAAsW7as2nPatWsHqEN7Mfgp2/baay8A1l9//RqP9RO2nHjiiQA0bdoUCCNPgE033RQIuyH5VUr32WcfIHUwhK9paKhsYfkaAoS1hFtuuQWAjz76CAjXBqpuBYVevXoB4d+Bj1L/8pe/5CnFdadIU0QkhpKONH1ndD8JrZ88OBO+e0O6KEfyw08a7YfFRjuhDxo0CAjbqHz7p48m/UQe6dqrfMTy/PPPA2HECWEXs9mzZwOw7rrrZvNPkQxFJwav3J3Qdxfyeeojz6jKK1T+9NNPNd7LT3isNk0RkTJQkpHmww8/DMCtt94KhB3X4/BtItGB/36Nbckvv6yIX+c6OiTST/nlh9FuvvnmALzwwgsAtGnTJuP7VI44AXbccUcgXMHSTxbSrFmzmP8KyZXKEWF1657H4Sfv8L0nCk2RpohIDCUVafp2DL/+uY8w/aQbn376aY3n+gjjzDPPBMLhXf5pHMCrr76a0/RKeiNHjgRSJ5z2C+r5WoSPPP1kHHURjVx8ZOmXyNhll12A8IltJovySeaWLl0KpE4aPXjwYABOO+00IHXSnFyoy/I3uaRIU0QkBhWaIiIxlFT13PMrVnrpquW+E/Njjz0GhN1bRowYUeXc5cuXA9C6detcJVXSWG+99YBwGCRAo0aJ7+l8zVCz7777AuFsV77Zxs+u5Ofr9OmQ7PiZ9/3AEwi7nPmZ+/3rXFfTi0V/OSIiMZRkpFmb6NDIG264Aah5rszoPHyvvPIKoNUMC80PjSykLl26AHDnnXcC4dyeN910ExA+MJTsrLXWWgAMHz48uW/cuHFAuFaQX3HUT+rhu5WlWwvMD07wfH4CNG/ePNtkZ0WRpohIDCUVafpZ3bt16wbAvHnzgLCbiF/dLjqIv6ZOy/vvvz+QOqO0H7qnSLPh+MMf/gCEE7v41TD9hC6gv4dc8G2bEE6a4oe0+mGPvgtSkyZNgLDNuzq+PdSLrmyZTfe0XFCkKSISQ0lFmv4byA+Lu/TSSwE47rjjgHD6qEz4dpPoU9r+/fvnIplShvw6Nb7zu18VExRp5prv0eI/x5dccgkA48ePB8KJd/yUj+n4oc+HHnporpNZZ4o0RURisLpMhuH17NnTzZo1K4fJyb1om6ZvC+nXr1+dr2dms51zDWYNjXLI4zh8H+BoX2Dflu4pj3PLlzGrV68G4L777gNSV6P0T9z9Eil+oum//vWvQO5Xks0mjxVpiojEUFJtmvmg9iqJ8iOBNCKocMwMCJ9ZHH/88VWO8RFlOdBfjohIDCo0RURiUKEpIhKDCk0RkRhUaIqIxKBCU0Qkhqw6t5vZMuCz3CWnLHRwzuW2p20JUx7Xf8rjeLIqNEVEGhpVz0VEYlChKSISgwpNEZEY0haaZtbKzOYGP0vMbHHkdV6WEzSzDmb2opktMLN3zOz0DM4ZbmbLgnS9a2YnZJmG+8ys5gVMEsdsZGZPmdlbZjbTzLpmc89iUR5ndOyuZvZbpseXmmLkcXDfc4P8fcfMzsjg+GJ8jlua2SQzmxek85haL+ycy+gHuAw4v5r9BjTK9DoZ3KcN0D3YbgEsBDrVcs5wYEywvRmwHGhd6ZjGMdJwH3BQLcdcB1wcbG8LPJ+r/4Ni/SiPqz2uMTAdmJzJ8aX+U8A87g7MA5oBTYL/wy1LLY+BS4BRwfamwIra7lGn6rmZdTSz+WZ2K/Am0M7MVkbeH2pmdwTbm5rZY2Y2y8xeN7Nd0l3bOfelc25usP098B7QNtO0OeeWAJ8C7c3sCjO7zcyeB8aZWWMzuzZIx1tmNjxIYyMzuyWIfJ4GMlkYvSswLbjnO0AnM2uV/pTyoTxOOht4kMQHuF7JZx4DXYBXnXM/Oed+BV4GMp5yrIB57ID1g+31SOTzb+lOyKZNsytwp3OuB7A4zXE3AKNdYsLPwwGfCb2DzKqRmW0FbAe8kWmizKwj0AH4ONjVAxjknDsaOAlY6pzrBewMnGZm7YFDgS2De50C9Ilcb5SZDazmVvOAQ4JjdgW2CH7qkwadx8F5BwD/zDRtZShfefw2sKclmrHWBfYH2lVzXLUK+Dm+HuhuZl+S+Eyf4YKwsybZzKe50DmXyR96f6CzBXPqAS3NrJlzbiYws6aTzKwF8CiJf8SPGdznKDPrC/wCDHfOrQzu+aRz7ufgmAFAFzPzC6dvAGwN7AE84JxbAywysxf9RZ1zF9dwv1HADWY2l8R/9jxgdQbpLCcNPY/HACOdc2si/7b6Ji957Jybb2bXAlOBH4E5ZPb5KHQeDwReB/oCnYDJZrZ9ur/HbArNVZHtNSTaRLymkW0Dejnnfsn0wpZonH4MGO+ceyrD0yY4586uZn80nQac6pybVul+Q0iE6Rlzzn0HHBuc34hEVeLTONcoAw06j4GewMTgQ9saGGBmvznnno55nVKWtzx2zt0O3A5gZqOBjzI4rdB5fDxwWRBdvm9mX5AoPN+s6YScdDkKSvYVZrZ1UIBE2y6mAqf5F2bWPd21LPEXOh6Y65y7vtJ7Z5nZiCySOgU41cwaB9frbGbNSLS3DA3aRNqS+NZJy8w2NLMmwcuTganOuVXpzilnDTGPnXPtnXMVzrkK4AngpHpWYKbIZR4Hx2wS/K4ABgMPBa9LJo+Bz4F+wXU2BzoCn6Q7IZf9NP9M4gnjNGBRZP9pwG5Bg+0C4MQggTW1hfQFjgT2sbBbxL7Be12Ab7JI423Ah8BcM5sPjCURbT9C4j9vPnATif98gnTW1BayPbDAzN4j8Z9+bjXH1DcNLY8bolzlMcATwbFPACcHtTMorTy+DOhrZm8Bz5PoWZB2beGyGntuZpOAwc65+tZ2KAHlcf1X7nlcVoWmiEixaRiliEgMKjRFRGJQoSkiEkM2/TRp3bq1q6ioyFFSysPs2bOXuwY0q7fyuP5THseTVaFZUVHBrFmzsrlE2TGzBrUsgPK4/lMex6PquYhIDCo0RURiyKp6Xgr+85//ADB0aGLs/lZbbQXAmDFjipYmEam/FGmKiMSgQlNEJIayr54vWpSYU+DppxOTzzRr1gyASy+9FICWLVsWJ2EiAsDbb78NwN577w3A8uWJSfDfeCOcxrNnz56FT1gdKdIUEYmh7CPNyjbddFMA1l47b4vsiUgGhg0bBsC9994LwOrViUmNOnXqBMBmm21WnIRlSZGmiEgM9S7S3H///QFYd911i5wSSefnnxPLvVx77bUAHH300QC0a1d17a0vvvgCgLPPTqyCsGDBAgDmzZsHqFZRqqZMmQJUjTAnT54MwBZblOc6hIo0RURiKPtIc+zYsQCss846QBiNSGnyUcepp54KwLhx4wB4+OGHAZg7d26Vc/75z8QKuo899ljK/n/84x8AXHTRRflJrNSJz9uvv/4agM6dOwPw7LPPAomx7uVMkaaISAxlGWl+/vnnye3x48cD0Lx5cyBsN5HSdMoppwBhhOn5vnyPP/44AIMHD06+N2fOnGqvdcsttwBwwQUXJPc1aqQ4oNgmTpwIwG+//QaEtYhyjzA9/YWJiMRQlpHm1KlTk9srV64E4KqrripWciQG365V2Zo1awBYtSqxdPxHH32UfG/atGnVnrN06VIApk+fntzXr1+/nKRT4rnrrruS2/4zecQRRwDQpUuXas/58ssvk9svvPBCynt+9FCbNm1yms5cUKQpIhKDCk0RkRjKqnruq2OjR49O7vNDsY477rhiJElyZP311wegf//+QOoQO9+d7Keffko5xw+ZVZW8+L7//vvktn8A1Lt3bwAaN04UM75p5uqrrwbg448/Tp7jJ97xfMd3P0ildevWyffOPPNMIJzkY8stt8zRvyIzijRFRGIoq0jTf1O9//77yX2HHXYYEEYdPhrxnah9BCPF8+STTya3v/rqq2qPWWuttYAwwnzvvfeS7/khl1K6br755ir7/IOgZ555Bgg/q//9739rvV7lyDP6mf/3v/8NQNeuXVOuX6guTYo0RURiKItI03c9E9lnAAAKz0lEQVRDueeee6q8N3LkSCCMLP1aQX4I17/+9a/ksRtttFFe0ympfNR/xRVXJPf5rkWVHXnkkSmvt9lmm+R206ZNgaoR56+//gqEXVwANtxwwyxSLHHdfffdAHz66adV3vPDXP2ABR9h7r777gCcf/75yWPbtm2b9j4PPfRQcvv+++8HwolbbrzxRgCuueaa2OmvC0WaIiIxlEWked111wFhB9i99tor+Z5/gvbcc88B8NRTT6Wc66cVA0WahbZw4UIAZs2aVeuxvXr1SnkdjVx8RFmZr01EO7cPGTIkbjIlCz4P/BPzKP+59fxzBz/02a8cm4mddtopuX3AAQcAYTlw0003AbDLLrsAYdtpvijSFBGJoaQjzfnz5wNw++23p+w/4YQTktt+kaYzzjgj5ZjNN98cKN8p9euDTJ6Sej5PfRt19Fzfpl2Tyy+/PLm9xx57ANCqVauM7y35tckmmwDw4IMPAvEizOpsvfXWQBi5+mj3tddeAxRpioiUlJKKNH3blZ8O308jtnjx4pTjDj744OS2n1L/gw8+SDnGj0LwT9UhjF78CBPJLz95cCaccwAsW7Ys9n18jwlQhFlKNt54YwBGjBgBQN++fXNyXT+Jh49cfdvmnXfeCeT/KboiTRGRGFRoiojEUPTq+XfffZfc9t1Fol1IqpPJSpO+q1F0xbv27dsDcMcddwCwzz77xEusxOIb5G+77ba83sevQSOlpXL1Odc6duyYl+vWRpGmiEgMRYs0fYQZHUpVOcJcb731Uo5p0aIFAA888EDymDfeeCPje/qHQ2+++SagSDPf9txzTyC1c/uhhx4KwJIlS4DsJuPw6wHpwV5pig6FzQff3bDQFGmKiMRQ8EjTdyvy0aNvX6yO77R87rnnAmFU8re//a3KsWYGQLdu3YBwjZFBgwYlj9lxxx2BMGKV/PLTvUWHwH3yyScAvP766ymvK/uf//mf5HZ0vaAo3/Vk4MCB2SdWyoYvQ6688sqU/YccckhB7q9IU0QkhoJHmh9++CGQPsI8+uijATjrrLNS9vvpoVasWFHlnP322w9InQpOSpefoKPyRB3etddeW8jkSB3tv//+AFx66aXJfX4QyYQJEwAYNmwYAC1btszJPf2UcH59dT/R+HnnnZeT69dGkaaISAwFjzSji6JV5hdI8m2Wvk3Mq26I3THHHAPAuHHjcpVEKQF9+vRJbvv2z8r88gZ+YheA7bbbLq/pklTbb789kNqe6CcJ9pOv+Frl6aefDoTDKn1vlnT8MOhou7ZfRsPzPTL88hf5pkhTRCQGFZoiIjEUrHr+zTffAFU7sEc7JvtO6x06dKj2Gn62I79mDIShuu/oLPWD7x6WzowZM4Dw4SKoel4s0Ycwvvvfo48+CoQzkPn1ymfOnAmkrmVeme8y+PDDDwPhQyUIZ7IaM2YMkL9hmjVRSSMiEkPBIk3fIbXysLlJkyYlt3v37p32Gr5h2T/8AejRo0eukihlKvogSGsEFUf0c3jvvfcC4efVP/ytLmqsyfXXX5/yOroCw2WXXQbASSedVPcEZ0GRpohIDAWLNP03hV/Poy78uj/+twgUrquJxOO7I/nI88ILLwTgf//3f4H0Eaef0tFHk9EuTV26dMl9YmNQpCkiEkPRJyEWqY6fjAOgSZMmQNX1z/3T1912261wCZM68zUCH3n63+VGkaaISAyKNKUk9evXL7m96667AvDyyy8DcOSRRwLhqqRa214KSZGmiEgMijSl5L300kvFToJIkiJNEZEYVGiKiMSgQlNEJAYVmiIiMajQFBGJQYWmiEgM5pyr+8lmy4DPcpecstDBObdxsRNRKMrj+k95HE9WhaaISEOj6rmISAwqNEVEYkhbaJpZKzObG/wsMbPFkddr5zNhZtbYzN4ysycyOPaKSNreNrMDsrz3DDPrnsFxR5rZAjN7x8zuyeaexaI8TntMhZm9EKRxupm1SXd8qSpWHpvZRmb2mJm9Z2bvmlmvWo4fbmbLgnS9a2YnZHn/+8zsoFqOMTO7xcw+MrN5mXzu0449d859A3QPLn4Z8KNz7h+Vb0qibXRNbTeL6VxgPtA8w+P/7pwbY2bbAdPNbBMXabA1s8bOudW5SpyZbQOcD/Rxzq00s01yde1CUh6ndR1wp3NugpkNAEYBx+fw+gVRxDy+EXjKOXdwUDg3y+CcCc65s81sM2C+mT3lnFseSWeu83gQ0M4519HMdgduBtJO0Fqn6rmZdTSz+WZ2K/Am0M7MVkbeH2pmdwTbmwbfNrPM7HUz2yWD63cA9gHGxU2bc24+YEDL4JvmGjObDlxpZuuZ2fggHXPMbFBwv+ZmNjGIKB4Emqa7R+Ak4Ebn3MrgvkvjprWUKY8B6ApMC7anAQfHTWspy2cem9lGQG/n3HgA59wvzrnvMk2bc24J8CnQPqhl3GZmzwPjLFFDuTZIx1tmNjy4Z6MgalxgZk8DNa8RHBoM3BPccwawmZmlfaqeTZtmVxLfwj2AxWmOuwEY7ZzrCRwO+EzoHWRWdcYAfwJiP9o3sz7Az865b4Nd/w/o55wbCVwCTHbO9QL2Bq4xs6bA6cAK59wOwNVAj8j1xtUQsncCupjZv83s1SASqW8aeh7PA/ziNIcALcxsg7jpLXH5yuOtgGVmdk/w5XW7mWVao8DMOgIdgI+DXT2AQc65o0kELEuDPN4ZOM3M2gOHAlsC2wGnAH0i1xtlZgOruVVb4IvI60XBvhplMzXcQufcGxkc1x/obMEC8iSig2bOuZnAzMoHW6IN4gvn3Fwz6x8jPX8ys+OAH4AjIvsnRqocA4D9zeyC4HVToD2wBzAawDk3x8ze8Sc752qqjjUm8YfRl0TmvmRmXZ1z38dIc6lr6Hl8DnCTmQ0DXgKWALmsGpaCvOQxic9HT+AMYDaJqvqfgMtruc9RZtYX+AUYHjR9ATzpnPPrfw8gEbAMDV5vAGxNIo8fCP4WFpnZi/6izrmLa7ifVbMv7Rd5NoXmqsj2mko3j1Z9DOjlnPslw+v2AQ42swOD67Qws7udc8fWct7fnXNjakmnAQc55xZGDwgyJW7Eswh4MWhfWWhmC0lEPHNiXqeUNeg8ds4tBoYE57cADnHOrUp/VtnJVx4vAj5zzs0CMLNHgbMzOG+Cc6664yrn8anOuWnRA8xsCHX7HLcDXgtebwF8me6EnHQ5Ckr2FWa2tZk1IvhDC0wFTvMvaqgGRa810jm3hXOuAvgj8Jz/MJnZaN9GVUdTgDMjafFVtJeBo4J93YBtM7jWE8BewTmbkCgwP8kibSWtIeaxmbW2MLS6iKBKWl/lOI8XAUuDajZAP2BBcO5ZZjYii6ROAU41s8bB9TqbWTMSeTw0aNtsS6IWWJungGOC6+wOfO2cW5buhFz20/wzMJlEg/miyP7TgN2CBtsFwIlBAtO1d9VkBxJVpLq6HGhuiS4r7wCXBftvAlqZ2VskqmSz/Alp2rsmAT8G/6apwDn+oVA91tDyuB/wgZl9AGwEXJVFuspFLvP4DOCh4P98W8L/vy7AN1mk8TbgQ2Cumc0HxpKoNT8CfE6iR8ZNJApRgnTW1Kb5NLA4qCneQuSLoSZlM4wy+Maf7Jzbt9hpkfxQHjcMZjYJGJzjrkMFUzaFpohIKdAwShGRGFRoiojEoEJTRCQGFZoiIjGo0BQRiUGFpohIDCo0RURi+P/1ZRbaO2r82AAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "print_test_accuracy(show_example_errors=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Performance after 10,000 optimization iterations\n", + "\n", + "After 10,000 optimization iterations, the model has a classification accuracy on the test-set of about 99%." + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Optimization Iteration: 1901, Training Accuracy: 93.8%\n", + "Optimization Iteration: 2001, Training Accuracy: 93.8%\n", + "Optimization Iteration: 2101, Training Accuracy: 98.4%\n", + "Optimization Iteration: 2201, Training Accuracy: 98.4%\n", + "Optimization Iteration: 2301, Training Accuracy: 96.9%\n", + "Optimization Iteration: 2401, Training Accuracy: 98.4%\n", + "Optimization Iteration: 2501, Training Accuracy: 92.2%\n", + "Optimization Iteration: 2601, Training Accuracy: 98.4%\n", + "Optimization Iteration: 2701, Training Accuracy: 96.9%\n", + "Optimization Iteration: 2801, Training Accuracy: 93.8%\n", + "Optimization Iteration: 2901, Training Accuracy: 96.9%\n", + "Optimization Iteration: 3001, Training Accuracy: 95.3%\n", + "Optimization Iteration: 3101, Training Accuracy: 96.9%\n", + "Optimization Iteration: 3201, Training Accuracy: 95.3%\n", + "Optimization Iteration: 3301, Training Accuracy: 92.2%\n", + "Optimization Iteration: 3401, Training Accuracy: 98.4%\n", + "Optimization Iteration: 3501, Training Accuracy: 95.3%\n", + "Optimization Iteration: 3601, Training Accuracy: 95.3%\n", + "Optimization Iteration: 3701, Training Accuracy: 95.3%\n", + "Optimization Iteration: 3801, Training Accuracy: 96.9%\n", + "Optimization Iteration: 3901, Training Accuracy: 98.4%\n", + "Optimization Iteration: 4001, Training Accuracy: 98.4%\n", + "Optimization Iteration: 4101, Training Accuracy: 98.4%\n", + "Optimization Iteration: 4201, Training Accuracy: 93.8%\n", + "Optimization Iteration: 4301, Training Accuracy: 98.4%\n", + "Optimization Iteration: 4401, Training Accuracy: 100.0%\n", + "Optimization Iteration: 4501, Training Accuracy: 98.4%\n", + "Optimization Iteration: 4601, Training Accuracy: 92.2%\n", + "Optimization Iteration: 4701, Training Accuracy: 96.9%\n", + "Optimization Iteration: 4801, Training Accuracy: 95.3%\n", + "Optimization Iteration: 4901, Training Accuracy: 93.8%\n", + "Optimization Iteration: 5001, Training Accuracy: 98.4%\n", + "Optimization Iteration: 5101, Training Accuracy: 92.2%\n", + "Optimization Iteration: 5201, Training Accuracy: 96.9%\n", + "Optimization Iteration: 5301, Training Accuracy: 100.0%\n", + "Optimization Iteration: 5401, Training Accuracy: 100.0%\n", + "Optimization Iteration: 5501, Training Accuracy: 90.6%\n", + "Optimization Iteration: 5601, Training Accuracy: 96.9%\n", + "Optimization Iteration: 5701, Training Accuracy: 95.3%\n", + "Optimization Iteration: 5801, Training Accuracy: 98.4%\n", + "Optimization Iteration: 5901, Training Accuracy: 95.3%\n", + "Optimization Iteration: 6001, Training Accuracy: 100.0%\n", + "Optimization Iteration: 6101, Training Accuracy: 95.3%\n", + "Optimization Iteration: 6201, Training Accuracy: 96.9%\n", + "Optimization Iteration: 6301, Training Accuracy: 90.6%\n", + "Optimization Iteration: 6401, Training Accuracy: 98.4%\n", + "Optimization Iteration: 6501, Training Accuracy: 96.9%\n", + "Optimization Iteration: 6601, Training Accuracy: 100.0%\n", + "Optimization Iteration: 6701, Training Accuracy: 100.0%\n", + "Optimization Iteration: 6801, Training Accuracy: 98.4%\n", + "Optimization Iteration: 6901, Training Accuracy: 100.0%\n", + "Optimization Iteration: 7001, Training Accuracy: 96.9%\n", + "Optimization Iteration: 7101, Training Accuracy: 100.0%\n", + "Optimization Iteration: 7201, Training Accuracy: 96.9%\n", + "Optimization Iteration: 7301, Training Accuracy: 92.2%\n", + "Optimization Iteration: 7401, Training Accuracy: 96.9%\n", + "Optimization Iteration: 7501, Training Accuracy: 95.3%\n", + "Optimization Iteration: 7601, Training Accuracy: 95.3%\n", + "Optimization Iteration: 7701, Training Accuracy: 98.4%\n", + "Optimization Iteration: 7801, Training Accuracy: 96.9%\n", + "Optimization Iteration: 7901, Training Accuracy: 95.3%\n", + "Optimization Iteration: 8001, Training Accuracy: 96.9%\n", + "Optimization Iteration: 8101, Training Accuracy: 95.3%\n", + "Optimization Iteration: 8201, Training Accuracy: 95.3%\n", + "Optimization Iteration: 8301, Training Accuracy: 95.3%\n", + "Optimization Iteration: 8401, Training Accuracy: 100.0%\n", + "Optimization Iteration: 8501, Training Accuracy: 100.0%\n", + "Optimization Iteration: 8601, Training Accuracy: 98.4%\n", + "Optimization Iteration: 8701, Training Accuracy: 96.9%\n", + "Optimization Iteration: 8801, Training Accuracy: 100.0%\n", + "Optimization Iteration: 8901, Training Accuracy: 96.9%\n", + "Optimization Iteration: 9001, Training Accuracy: 98.4%\n", + "Optimization Iteration: 9101, Training Accuracy: 96.9%\n", + "Optimization Iteration: 9201, Training Accuracy: 98.4%\n", + "Optimization Iteration: 9301, Training Accuracy: 100.0%\n", + "Optimization Iteration: 9401, Training Accuracy: 98.4%\n", + "Optimization Iteration: 9501, Training Accuracy: 96.9%\n", + "Optimization Iteration: 9601, Training Accuracy: 96.9%\n", + "Optimization Iteration: 9701, Training Accuracy: 100.0%\n", + "Optimization Iteration: 9801, Training Accuracy: 93.8%\n", + "Optimization Iteration: 9901, Training Accuracy: 96.9%\n", + "Optimization Iteration: 10001, Training Accuracy: 95.3%\n", + "Optimization Iteration: 10101, Training Accuracy: 96.9%\n", + "Optimization Iteration: 10201, Training Accuracy: 98.4%\n", + "Optimization Iteration: 10301, Training Accuracy: 96.9%\n", + "Optimization Iteration: 10401, Training Accuracy: 100.0%\n", + "Optimization Iteration: 10501, Training Accuracy: 98.4%\n", + "Optimization Iteration: 10601, Training Accuracy: 100.0%\n", + "Optimization Iteration: 10701, Training Accuracy: 100.0%\n", + "Optimization Iteration: 10801, Training Accuracy: 100.0%\n", + "Wall time: 3min 3s\n" + ] + } + ], + "source": [ + "%%time\n", + "optimize(num_iterations=9000) # We performed 1000 iterations above." + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Accuracy on Test-Set: 98.8% (9884 / 10000)\n", + "Example errors:\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe4FdW9//H3F1EBsYOCSok/pSixi4qKBWNXsAVCDIJB\nTLBAvLZYErkCtthIxBoRRb15EBSUq8YS8EIUBUG6YiWgKBgBEUvU9ftjZu2Zfdrec3Y/fF7Pw3Nm\nz57y5ayz1/7OmjVrmXMOERHJTqNSByAiUklUaYqIJKBKU0QkAVWaIiIJqNIUEUlAlaaISAKqNEVE\nElClKSKSgCpNEZEEGueyc4sWLVz79u3zFEplmD179mrnXMtSx1EsKuOGT2WcTE6VZvv27Zk1a1Yu\nh6g4ZvZRqWMoJpVxw6cyTkaX5yIiCajSFBFJQJWmiEgCqjRFRBJQpSkikkBOd89FRPLhnXfeAeD8\n888HoG/fvgCcd955JYupNso0RUQSKOtMc/369QA88cQTtW4zY8YMAP76178C0KtXLwAGDRoEwPHH\nH1/IEEWknnx2CXDSSScB8P777wPw4YcfAso0RUQqXllmmj7DvOSSSwB46aWXAOjcuTMA22+/fbV9\nfEY5bdq0tJ933nlnapuzzz67QBGLSLb8Z/KOO+5IrVu2bFnaNu3atStqTEko0xQRSaAsM813330X\ngC5dugBw3333Zb3vsGHDABg+fDgAAwYMSL2nTLN8rFmzBoClS5cC8Nhjj6W9H89CzKzGY7Rq1QqA\nV199NbWunDOUjd33338PwKJFiwD46KPo8W9fxh06dABg3LhxRY4ue8o0RUQSKMtMc5999kn7mcSQ\nIUMAePDBBwH4+OOPU+8tWbIEgE6dOuUaotSTzyBGjhwJwNtvv13jdvHscu+99wbgP//5DwCLFy8G\n4NNPPwVg5cqVqW2VaZave+65B4AHHnig1m1atGgBwC677FKUmOpDmaaISAJlmWnmYurUqQCsXbu2\n2nurV68ucjQC6e2Vv/3tbwHYsGEDANtttx0Ap59+OhBlld27d0/t47NH3ybWpk0bAL7++utqxz/o\noIPy/x+QnPirPd+X2jmX9jPulltuKV5g9aRMU0QkAVWaIiIJNLjL87lz5wKwbt06AA4//PDUe4cd\ndlhJYtpY+UvweMP//vvvD8A111wDwKGHHgpA06ZNMx7PX45X7YJ01lln5R6sFIzvWjRv3jyg5i5k\np556KgD77bdf8QKrJ2WaIiIJlDzT/Oabb1LLI0aMAGD69OlA9W8k30jcsWPHasfx3YkefvhhADbZ\nZBMAfv3rX+c5YslWs2bNAHj55Zfzcrxbb70ViDLY3XffHYger5XytOWWWwJRd6Kabsj6BxT8IB7+\nwZZypExTRCSBkmea8Y7Jm266KQBHHXUUAGPHjgWih/kPPvhgAE455ZRqx3n66afTXvt5nH0XFqlc\nb7zxBgA33XRT2nrffammAVykfPis0Q/bWFPndp99jh49Ou1nOVKmKSKSQMkzTZ8RAvzhD39Ie893\nePaPRvpBHuJZ5Q8//ABEbZje3/72N0CZZqX68ccfU8vPP/88ELVlbr311kB0RSKVwfeYqOsxSv/Z\n9tNelOPnV5mmiEgCJc806+LbQvwgxP369QOi/l51mTRpElAZ/b6kOv/IHcAf//jHtPduvPFGAPba\na6+ixiS58Y+/+ivH22+/vdo2K1asAKJ+m/Hh48qFMk0RkQTKOtP0kyvtuuuutW5zzjnnAPDVV18B\n0SRs119/fbV9/bZS/p555plq69q2bQuoHCudv3I44IADUut8G6Zvt/a9ai6++GIAzj333NS29Rky\nMp+UaYqIJKBKU0QkgbK8PPeje/sOrr47UY8ePQC46qqrUtv6cRf945jNmzcH4JFHHgFqn19GytOc\nOXOA9G5lvgwvu+wyADbffPPiByZ547uM9e3bN7Vu/PjxQDQerh9w56677gKiZjeAt956C4CWLVsW\nPNaaKNMUEUmgLDNN37XIZ5g+m/Qd1v03VVyTJk2A6nPEPPXUU9WOK+XH38i77rrrgPRRvf0VxuDB\ng4selxTHk08+CcC9994LVC/r+OPW3333XfECq4EyTRGRBMoq0/RtVp6fkW7ixIlAzRlmJvFBiKV8\njRkzBoi6GsUHJY7PXS8NWyU8sKBMU0QkgbLKND///PO014MGDQKyyzD90FL+8UlPw4aVt6VLlwJw\n9dVXp62PX3XE77JKeZo2bVq1dUcccUTW+99///0AjBw5Eqg+U2VNM1eWijJNEZEESp5pxu+Kvfvu\nu0D0rdKtW7eM+/t9fIbpJ1Zr1aoVUPcjmFI6voxvuOEGANavX5/2/sknn1z0mCQ5P6d5z549U+t8\nb5fPPvusxn0mT54MpGenn376KRDNbe/75vpHJv0+EH22S0WZpohIAqo0RUQSKPnlue+UDtEjkD41\nr3pjqKZZ7Px8Qf4y3d/48Y9iaq7z8jRhwgQgmgfK69+/PwAHHnhgsUOSevAzJ3z55Zepdb7b2JQp\nU+rcN35zx3/mt9pqKyCaD8p/vlu3bp2niHOnTFNEJIGSZ5rbbLNNarlqA+95550HwPz584Hq3YkA\n3n77bSB65PKiiy4C4Oijj85/sJI3fn7rqvw8MnXxj9P27t07rzFJcv5z5zNEgLVr12a1rx/JHWDf\nffcFolHdy3n+J2WaIiIJlDzTjOvYsWPaaz881PDhw2vdx2envm2sa9euBYpO8mnWrFlpr6+99log\nGp3922+/Tb3nH6P1o/H/+c9/LkaIkoWddtoJSB8Yxw/v540aNQqAI488EogelRw6dGgRIsw/ZZoi\nIgmUVaZ5xRVXAFH26O+o+W8x/8hd/BE733ldd8kry6uvvpr2+t///jcAixYtAuCXv/xl6j0/I6F/\n1DLJ43lSHPEyqVo+lZpR1kaZpohIAmWVaXpVZxvU4MENz2mnnQZEg876aQ38z3gfPj9wy+WXX17M\nEEVqpExTRCSBssw0peEbNmwYADNmzABgwYIFQDRAQ7y/5nHHHVfk6ERqp0xTRCQBVZoiIgno8lxK\nws9Z7eewFqkUyjRFRBJQpSkikoAqTRGRBCyXWd7MbBXwUf7CqQjtnHMtSx1EsaiMGz6VcTI5VZoi\nIhsbXZ6LiCSgSlNEJIE6K00z297M5ob/VprZitjrzQoVlJldamYLzWyBmT1qZptn2H54LLb5ZnZS\njuefbmb7ZNjmQjObF57z/8ysUy7nLJVSlLGZtTOzqWa2KCznC7PYZ6CZrQrjWmxm5+YYwzgz65Vh\nmytjv4uFZva9mW2dy3lLoYSf4+Xh53Gumc3MYvtSlPHpsc/xG2bWLeOBnXNZ/QOuAy6tYb0BjbI9\nThbnaQe8CzQJjz0BODvDPsOBoeFyF2AVYXttbJvGCWKYDuyTYZutYsunA8/k63dQqn9FLOOd/O8X\n2Ap4D+iQYZ+BwB3hcitgNdAihzIeB/RKsP1pwN9LXUaVUsbhMZcD2yTYvuhlDDQnurezH7Ag03Hr\ndXluZruFWcKjwEKgjZmtib3fx8weCJd3NLOJZjbLzF43s4OzOMWmBJVmY6AZ8HG2sTnnFhD8AWwb\nftPcbWavAyPNrLmZPRTGMcfMTgljbGZm48NvtwnhuTOdZ13s5RZAg7qjVsgyds597JybGy6vA5YA\nO2cbm3NuJfAh0Da8ynjYzGYAD5lZYzO7LYxjnpkNDGNsZGajzWyJmb0AtEj0C4FfAI8n3KesFeFz\nXG/FKmPn3HoX1phk+TnO5THKTkA/59wsM6vrOKOAm51zr5lZe+AZoIuZHQQMcM79Jr6xc+4jM7sT\n+BfwLTDFOfdytkGF6fU3zrl/WzDye2vgYOfcj2Z2M/Ccc66/mW0LzAx/uRcCXzjnOpvZvsCs2PHG\nAHf6D3mVc10MDCGo5Mt3+rz6K0gZx5nZrgRXB29kG5SZ7UZwRfJ+LM7uzrlvzGww8JlzrqsFzTqv\nmdnfgYOBnwB7EGS6i4B7wuONAGY45/63lvM1B44Bzss2xgpSyDJ2wMtm5oDRzrm/ZhtUMcvYzM4E\nRhBUsidmii2XSvM959yszJtxDNAxrMAgyACbOudmAtXaOcxse+Bkgv/8OmCCmfVxzv1PhvNcZmb9\ngS+B+Nyu451zP4bLxwInmNmV4esmQFugO3AzgHNujpkt9Ds75wbUdkLn3ChglJn1A64Cfp0hxkpT\nkDL2zGwrguaXi5xz67M4zy/N7EiCL9OBzrk14TknOee+Cbc5FuhsZn3C11sDuxOU8ePh38JyM5vq\nD+qcuzrDeXsC05xz2c1NW1kKWcYHO+dWmFkr4AUzW+yc+2eG8xS9jJ1zTwBPmNlRwPXh8WuVS6X5\nVWz5R4JLYi9+eWtAV+fcd1ke91hgqXNuNYCZPQl0AzJVmrc45+7IEKcRtHG8F98g9odQX48Bd9Lw\nKs1ClTEW3ICYCIxxzk3OcrdHnXM1TThTtYwHO+deqnK+07KNrQZ9gEdy2L+cFayMnXMrwp8rzWwS\n0BXIVGmWqoxxzv3DzMaa2TbOuTW1bZeXLkdhzf6Fme1uZo0IGs29F4EL/AvLcFcaWAYcYmZNLajN\negCLw31v9u2Q9fQ8cFEsln3DxVeAvuG6vYE9Mx3IzHaPvTwFeDuHuMpePss4LNeHgLlhth5/b4iZ\n1Xo5n4XngcH+UtPMOppZU4Iy7h22e+0MZDU7W9iM0w14OoeYKkKey7h52KyBmW0B/AxYEL4umzIO\n23UtXD6A4KZQrRUm5Lef5hUE/5l/Etw18y4ADg0bbBcRtguZ2UFmdk/VgzjnZgCTgTnAfOB7wLeF\n7AWszCHGYcAWFnSDWEhwJxHgL8D2ZrYYuDY8N2GcY2r5AxlqQTeUuQRtorVexjcgeSljgj/mXwA/\ns6jrix+evTPweQ4x3gssBeaa2QLgboIrqicIvpAXAWOA1HSYZjbCzGpryzoDeNY593UOMVWSfJVx\na2CGmb0FvA486Zx7MXyvnMr458CC8HM8ivSmvRpVzGOU4bfBs86540sdixSOmU0Bejrnvi91LFIY\nlV7GFVNpioiUAz1GKSKSgCpNEZEEVGmKiCSgSlNEJIGcZqNs0aKFa9++fZ5CqQyzZ89e7TaiUb1V\nxg2fyjiZnCrN9u3bM2tWNk9gNRxmtlFNC6AybvhUxsno8lxEJAFVmiIiCajSFBFJQJWmiEgCOd0I\nEimG2bNnA3DMMccAsM022wDw/PPPA9ChQ4fSBCYbJWWaIiIJKNOUsrJhwwYAzj///NS6Z555BoB1\n69al/TzrrLMAeOutt4oZouRJfPDv008/HcBPdsaeewZD2l5//fXFDywDZZoiIgmUdaY5alQwqPfF\nF19c4kik0ObPnw/AuecGU12/+eabqfd89lF1WpIjjzyyOMFJQcTL86mnngKisp40aRIA++4bTK7g\nM9FyoExTRCSBsso0v/oqmDvpyiuDySI/+OADQJlmQ/bxx8GU9nfcEcyJF88wMxkzZgwABx54YGrd\n2WefncfopJDuuaf6LBnXXHMNAKtXrwbghhtuAJRpiohUrLLKNH1meddddwHw+uuvlzIcKYIbb7wR\ngIceeijxvv7K5Jxzzkmt85nqPvsEc+H169cvxwilUAYNGlRtnS+/+++/v9jhZE2ZpohIAmWVaQ4Z\nMgSAn/70pwA0adKkrs2lgvmnfB555BEgumtakyST/91+++0A9O4dzMSqTLMy+TI//PDDSxxJdco0\nRUQSUKUpIpJAyS/PX3jhhdTyDz/8ANTvsbj33nsPgDVr1gCw//77A/CPf/wjtc2MGTNq3HfvvfcG\n4JRTTkl8Xqmfv/zlL0D0SGTVjuv+Rg7A5MmTgejvwndPeumll2o9/pQpU4CoW9KAAQPyEbYU2JNP\nPglEfw+nnXZaKcOpkTJNEZEESp5pPvfcc6nlRo3qrsN9R+hevXpVe89nLN9++y0AO++8MxB1kgV4\n5513ajxuy5bB/Ert2rUD1NWpGMaOHQtUzzC7du0KwMSJE1PrWrduDURluu222wJ1Z5rNmjUDYMcd\nd8xTxFIMPrO87777AN0IEhGpeCXLNH3WOG/evNS6Bx54ACA1M17btm0B2GGHHYBoMAefVULUNWHp\n0qVpx/cdnn07KcCIESNqjGXVqlVAlOVI6QwcOBCApk2bptZ9+eWXQNRe7f9O6nL00UcDcOKJJ+Y7\nRMkT/7mD6HFJ36a5xx57lCSmbCjTFBFJoGSZph9YYerUqal1fuDZZcuWAfDoo48CUaa5xRZbADB+\n/PjUPj7T/PTTT9OO3717dwD+9a9/pdY98cQTQPS45nfffQfA8ccfD8CDDz6Y0/9JcnfppZcC6YM5\nNG/eHIBXXnkl6+Oceuqp+Q1McvbRR8FU4/4ewrhx41Lv+R4Rvi162rRpRY4ue8o0RUQSKHqmOXPm\nTCBqt/SDjEI0eMOtt94KwHbbbZe274QJExKfb7fddkst+7viF1xwARB90/m7s/4bUArnkksuAWp/\nNHLt2rVA9JhlfNuqd9pr4tvEevbsmVOckn/+noH/fPvPO0Rle9VVVwHQqVOnIkeXPWWaIiIJFD3T\nvPfeewFYv349kD5orH+K57HHHivIuX27Z7wtRQrvt7/9bWr58ccfB6LMIpvs0ctmW2WY5cf3uf3s\ns88AGDlyZNprgM6dOwNRplnOlGmKiCSgSlNEJIGiXZ7/93//NxBdGvvHoy666KKCnve6665LLd90\n000ADB06FIguEzbZZJOCxrCx8jNMxm/g+Y7qVW211VZAVEYffvhh6r34DYNMfv/73wMwbNgwADbb\nbLPsA5a8WLx4MRCVuy9T37xy5plnArBw4cLUPn42yuHDhwPRXEHlSJmmiEgCRcs0//jHPwLRt40f\nnKNx48KE4BuU40PPXXHFFQAcd9xxgEaGL7S7774bgM8//7zWbbp16wZEVyJHHXVUtW38TUN/E/H7\n77+v9Xg+q9lyyy2Byrix0BD4jusAV199NRB1/zriiCOA6Oqhb9++QDTHE0SPTV577bUAtG/fHijP\n2UWVaYqIJFCyxyj9oBsrV65MrWvVqlW9j+c7y/vsxs894zuuQzRfzK677lrv80hmc+fOBeDpp5/O\nuO3gwYOBmjNMb9SoUQBMmjQJgOXLl2c8rm9PleKIz8U0ffp0IHr8+bbbbgOiAXhatGgBwIYNG1L7\n+C5Hfmg4P7iOf6xS856LiFSokmWac+bMAdLnrPYdn6s+PllVfDg5P3jHzTffDERDgfm75n7gDlCG\nWSx+sOcVK1bUuo1/fLbq0G3+7np8+Df/6GU2ndvLeRbDhsgP7xYfTMW3YcYH46mJzyLj9ttvPyC6\n8+6zVd/GGd+mVJRpiogkULRM0w+c4b+ZfJtm/O52nz59ABg9ejQAl112GQDvvvtu2rHigxBffPHF\nQDTAw0477QRkzlalcLJ5RNKXqb9b6tsr/aDRn3zySaLjeX4owFzaxyV7VSdCg/xOhubvTSxatCi1\nTpmmiEgFUaUpIpJA0S7P/Rw+fp4fPxr3a6+9ltrmxRdfBKBDhw41HmPTTTcFYMiQIal1PXr0AKBL\nly55jlgKyd/wueuuu+p9jM033xxIf1TWz1gpxeG7D/mfED2E0KZNG6B+3YX8yEhnnHEGkH75X+oO\n78o0RUQSKHqXoyuvvBKIuv/4+YAgmtel6nw/nh+EwXeIlvJ0yCGHANEVQ23zzWfLZyz+0Vtv0KBB\nAFx++eU5HV/qz2eR8c+x7y7muxMuWbIEyO6RVj9gR9VBPsppAA9lmiIiCRQ906zaXhnvcL5gwYJi\nhyMFsMsuuwDQv39/INmgGT472WuvvVLr/FB+Ur7iZeQHxDnhhBOA6IqgLr/61a+AKCv1baRjx44F\n9BiliEjFKtljlNLw+aH4/E/ZOPjBN3zH9Nr4wYohGoTYDyLts9P4XflyoUxTRCQBZZoiUhCZBk3x\nGSnUPg1KOVKmKSKSgCpNEZEEVGmKiCSgSlNEJAFVmiIiCajSFBFJQJWmiEgCqjRFRBIwP3tfvXY2\nWwV8lL9wKkI751zLUgdRLCrjhk9lnExOlaaIyMZGl+ciIgmo0hQRSUCVpohIAnVWmma2vZnNDf+t\nNLMVsdebFSooM9vOzCaa2RIzW2xmXTNsP9DMVoVxLTazc3M8/zgz65VhGzOz0Wb2rpnNM7N9cjln\nqZSwjC81s4VmtsDMHjWzzTNsPzwW23wzOynH80/PpszM7BdmtiiM9eFczlkqKuM6t7k8rDPeMrMX\nzKxNxgM757L6B1wHXFrDegMaZXucLM/1KNA/XN4M2DrD9gOBO8LlVsBqoEWVbRonOP84oFeGbU4F\nng6XDwNm5PN3UIp/xSpjoB3wLtAkPPYE4OwM+wwHhobLXYBVhDcy61nG04F9MmzTCZgNbBO+3qHU\nZaQyznsZHw00DZcvAh7NdNx6XZ6b2W7ht++jwEKgjZmtib3fx8weCJd3DLPGWWb2upkdnOHY2wEH\nOeceAnDOfeecW5ttbM65lcCHQNvwm+thM5sBPGRmjc3stjCOeWY2MDxnozBrXGJmLwDZDBfdE3g4\nPOd0oJWZNZhuKoUs49CmBB+oxkAz4ONsY3POLSD4IG4bXhXcbWavAyPNrLmZPRTGMcfMTgljbGZm\n48OsYkJ47kwGAX92zq0Jz/tZtjFWApUxOOdeds59Hb58Ddgl0z65tGl2Am53zu0BrKhju1HAzc65\nA4CfA74QDjKze2rYfldgVVjZzTGz+8ysWbZBmdluBN9y78fi7OGcO5vgQ/CZc64rcCBwgZm1Bc4E\nfgLsAQwAusWON8LMTqzhVDsD/4q9Xh6ua0gKUsbOuY+AOwl+f58QlMnL2QZlZt2Ab5xz/w5XtQYO\nds5dDvwBeC4s46OBW82sCXAh8IVzrjNBRrNv7HhjarmM6wB0NrMZZvaqmR2bbYwVZGMv47hfA89m\nii2Xkdvfc87NymK7Y4COFs5fTPDN0dQ5NxOYWUtMBxCkyrOBPwOXAcMynOeXZnYk8C0w0Dm3Jjzn\nJOfcN+E2xxJ8CPqEr7cGdge6A487534ElpvZVH9Q59zVWfwfG6qClLGZbQ+cTPBFtQ6YYGZ9nHP/\nk+E8l5lZf+BLoHds/fiw7CAo4xPM7MrwdROgLUEZ3wzgnJtjZgv9zs65AbWcrzHBl/gRBF/E08xs\nD+fcugxxVpKNvYx9vP2BnwIXZ4gvp0rzq9jyjwSptBdPiw3o6pz7LsvjLgeW+YIM0+xs5nB91DlX\n03bxOA0Y7Jx7Kb6BmZ2WZWxxK4A2BCk9BGl9Xd/UlahQZXwssNQ5txrAzJ4kyO4zfaBucc7dkSFO\nI2iPfi++QezDnsRyYJpz7nvgPTN7D/h/wJz6HKxMbexljJkdT5CYHZHN/y8vXY7Cb4AvzGx3M2sE\nxCuhF4ELYgHWmSI755YDn4aX2QA9gEXhvkPM7Dc5hPo8MNjMGofH62hmTYFXgN5h2+bOBJlFJpOB\nfuFxDgM+dc6tyiG2spbPMgaWAYeYWVML/tJ7AIvDfW/2bVT19DzBVYqPxV+ivQL0DdftDeyZxbGe\nAo4M99mBoML8IIfYytrGWMZmdgBwF3Cqr+AzyWc/zSsI/jP/JPiG9i4ADrXgxssi4Lww2NraNCH4\nhfzNzOYR/MdvDNd3Bj7PIcZ7gaXAXDNbANxNkG0/QVDIi4AxwKt+hzraNJ8GVoTZx93E/qAasLyU\nsXNuBsGXzhxgPvA98Nfw7b2AlTnEOAzYwoIuKwsJ7hYD/AXY3swWA9cSyxbraO+aAqwP/08vAr/z\nN4UasI2tjP8EbEHQfDA3zIjrVFHPnpvZFKBneLkkDUyYkTzrnDu+1LFIYTSEMq6oSlNEpNT0GKWI\nSAKqNEVEElClKSKSQC79NGnRooVr3759nkKpDLNnz17tNqJRvVXGDZ/KOJmcKs327dsza1Y2DxM0\nHGa2UU0LoDJu+FTGyejyXEQkAVWaIiIJqNIUEUlAlaaISAKqNEVEElClKSKSgCpNEZEEcuqnKZKN\ncePGpZbPOeectPceeeQRAPr27VvUmETqS5mmiEgCZZlpfvVVMLL9kiVLALj//vvT3v/ss2hSwKee\negqAQYMGpW3jM5fu3bsXLE7JTjy73GSTTdLe69+/PwBffvklAHvssQcAhx9+eHGCkzr95z//AeDz\nz4OxvxctWgTA6tXBIOdvvPEGAM8+G81H5j+/Z511VtqxLrnkEgC23nprAJo2bVqosAtKmaaISAI5\nDUJ8wAEHuFyfWR0+fHhqedKkSUD0TfX2228D+EndUxMnxWOuus6/3m+//YD0b8AWLbKZzrxuZjY7\nnMZ0o5CPMo5nl1UzTe+HH34AYM89g2ldRo8enXrvsMMOy+n8SW3sZfzxx9H05KNGjQLglltuqXHf\nqp+7uvhtfXkOGxZNMHvUUUcljDo3uZSxMk0RkQSK3qbp76T+13/9F5DePlk1a+zcuTMA7dq1A+C0\n06rPtHv66aen7dO1a1eA1Kgty5YtS22bj0xTkvN3yCFqw6yNb8f2P6H4mebG7o47ohl0b731ViD6\n7PgrOM9/7tavX59a9+qrr1KXGTNmAHDFFVek1r30UjCr9pZbblnfsItGmaaISAJFzzT9XTf/8ze/\nqT6N+XnnnQdAp06dAGjWrFnG4y5evDjtuPWdOF7yr0OHDqll33ZZ1Y8//pj2+vzzz08t+/JXX87i\n+N3vfpda/tWvfgVA8+bNAfjJT35S4z5ff/11avnFF18EonZQn1lWNXv27NTylClTAOjTp099wy4a\nZZoiIgmo0hQRSaDkXY7yxaf6Bx54IBA1XMfja9u2bc7n2di7o9THRx9FMwsMGDAAgOnTp6dt4y/b\na+uSBPDdd9/lFEe2VMb54S/Zf/7znwPRJXhNTWe+o/v48eMBOOGEE/IeT5y6HImIFElZPkZZH08+\n+SQQfYvpRlD58F3GIOq0PnjwYKB6xikNh88en376aQBOPPFEAJ577rlq227YsAGAk046CYgeq91i\niy0KHmdSyjRFRBJoMJmmH0jAt9H69su62jF9W5vvpuS3bdlyo5nyuuh8N7KOHTsCyjQ3Jg8++CAA\nO++8c8YAVfjMAAAJPklEQVRtTznlFABefvnlgsZUH8o0RUQSaDCZph8izrdl+qHF7rvvvmrb+vbP\nN998E6iead52222pbWt6dFNyd/fddwNR74Zs7t76x2p9m1i8rVTK37bbbgtAjx49gOjRyZr43jD+\nCtIPGVgOlGmKiCRQ8ZmmH1quan/T22+/HUi/i151IJBddtkFgOOOOw6A3//+92nvS+H5AVd81l9X\nP8133nkHgJtuuglIHz5Oyt/mm28OwNFHHw3UnWk2ahTkc02aNCl8YAkp0xQRSaAiM00/iABUb8v0\nP/00F/GssT4DgUhh+ez+mmuuKXEkUix+WMgvvvgite5Pf/pT2jZr164FoF+/fkB59bJQpikikoAq\nTRGRBMr68nzixIkAXH311UD1OYMguhz3l9p+lHB1FWq4fJexY489FoBevXqVMhzJ0sqVK4Fo3q4P\nPvgg9V7VG7n+tb9M//bbb1Pv+RtKpaJMU0QkgbLKNH1m6TMJf5PHz055xhlnADBhwoTUPj7T9F1X\nlGFWpqojt9e1jZ9Xyj+UIOXl/fffB2DmzJlA9DCCvwqsaTCd2gbY8Z3b/WOVADfeeCNQfb6iYlGm\nKSKSQMkyzVWrVgFw7bXXptb5DNNnlj57rNrpvKZvJc1YWJn81YXvzFxX53bPb+O7ocTbNDXjaGn4\nodwAevfuDUQPLORDvCN8z549AZg3bx4QPZ5ZLMo0RUQSKHqm6WeN9AOSxqdC8A/l33PPPUD19km/\nbzzTrNqmKZXFt0/Vx2OPPQbAkCFDUuuUaZZGvKP6dtttV+M2+++/PwB77rlntff8QMVr1qypcd+d\ndtopteznS8/mqqQQlGmKiCRQ9EzTZ4Q+w/RZZfy92rKFkSNHAul9unwfTmUYlenhhx8Gas4+pHLE\nB/uePHkyEPWx9PzUFTVNYXHvvfcC0TQoVXXp0iW1fOGFF+YWbI6UaYqIJKBKU0QkgaJfnvtHIf0N\nnPhldW2X2L5bStURjUCd2SudH3GqQ4cOQDRmZk2y6QAvpecfc9xhhx2y3ufQQw8FoHnz5gCsW7cu\n7f0lS5aklj/55BMAWrdunVOc9aVMU0QkgaJnmlOnTgWirNHfyIFoTEV/Q8iPfzlu3Dgg6vQ+dOjQ\n1D6lepRK8suX8SGHHJJxW9/VxD/QoJuAlc/f6Gnfvj0A8+fPB6KryniXpvXr1xc3uCqUaYqIJFD0\nTNOPqO5/+jl+IGq3OPvsswF44IEHgGiABv+tc9VVVxUnWCkaP7Nk3759gajjel38tm3atClcYFJU\nflaGyy+/PG29b/OG6PHZ3XffvXiBxSjTFBFJoORDw8Xn6fHtk344qEsuuQSIZpb02anasBoeX6bd\nunUDsss0peE555xzABg7diwQ1QV+HnSASy+9FIBTTz0VgO23376YISrTFBFJouSZZl18Hz7flqk+\nmQ3foEGD0n7KxqVly5YA3HDDDUDUR3vDhg2pbfw9jlJNe6FMU0QkgbLONJV1iGycTj755LSf5USZ\npohIAqo0RUQSUKUpIpKAKk0RkQRUaYqIJKBKU0QkAYvPt5N4Z7NVwEcZN2xY2jnnWpY6iGJRGTd8\nKuNkcqo0RUQ2Nro8FxFJQJWmiEgCdVaaZra9mc0N/600sxWx15sVIiAza2dmU81skZktNLOMkxyb\n2UAzWxXGtdjMzs0xhnFm1ivDNlfGfhcLzex7M9s6l/OWQinKODzvWF9mWW5fijLe1symmNlbYRn3\ny+WcpVKiz/EesXPMNbMvM32WS1TGp5vZvPCcb5hZt4wHds5l9Q+4Dri0hvUGNMr2OFmcZydgn3B5\nK+A9oEOGfQYCd4TLrYDVQIsq2zROEMM4oFeC7U8D/p6v30Gp/hWrjMNjHgF0BeZmuX3Ryxj4AzAi\nXN4R+CLJOcrxXzHLOHbsTYHPgF3KsIybE93b2Q9YkOm49bo8N7PdwkzwUWAh0MbM1sTe72NmD4TL\nO5rZRDObZWavm9nBdR3bOfexc25uuLwOWALsnG1szrmVwIdAWzMbbmYPm9kM4CEza2xmt4VxzDOz\ngWGMjcxstJktMbMXgKSjHP8CeDzhPmWtkGUM4JybBvy7PrEVsYwdsGW43JzgQ/xDfWIuR4Uu45if\nAYudc8uz3aFYZeycW+/CGhPYgqDM65TLKEedgH7OuVlmVtdxRgE3O+deM7P2wDNAFzM7CBjgnPtN\nbTua2a5AF+CNbIMys92AdsD7sTi7O+e+MbPBwGfOua5mtjnwmpn9HTgY+AmwB0Gmuwi4JzzeCGCG\nc+5/azlfc+AY4LxsY6wgBS/j+ihiGd8JPGNmHxNc9ZwZ+4A1FMUo4z4kTCqK+Tk2szOBEQSV7ImZ\nYsul0nzPOTcri+2OATpaOJAwsK2ZNXXOzQRm1raTmW0FTAAucs5lM2fnL83sSOBbYKBzbk14zknO\nuW/CbY4FOptZn/D11sDuQHfgcefcj8ByM5vqD+qci+YYrllPYJpzbm0WMVaagpZxPRS7jE8EXido\nSugAPGdmP83y77FSFPpz3AQ4Cbgky3iK/jl2zj0BPGFmRwHXh8evVS6V5lex5R8J2kS8JrFlA7o6\n577L9sAWNE5PBMY45yZnudujzrmhNayPx2nAYOfcS1XOl8uQ8H2AR3LYv5wVrIzrqdhlPAC4Lswu\n3zazfxFUnm/W41jlqtBlfBIw0zm3OsvtS/U5xjn3DwtuUG7jnFtT23Z56XIU1uxfmNnuZtaI4MaI\n9yJwgX9hZvvUdSwLvlYeIrhBMKrKe0PMLJdLveeBwf4yxMw6mllT4BWgd9gmsjNBZpGRmW0LdAOe\nziGmipDPMq5LmZXxMqBHeJzWwG7ABznEVtYKVMbV2vvLqYzDdl0Llw8guClUa4UJ+e2neQXBf+af\nQLzB9wLg0LDBdhFh25+ZHWRm99RwnCMIftE/s6i7wnHhe52Bz3OI8V5gKTDXzBYAdxNk208QfEAW\nAWOAV/0OZjbCzGpr5zgDeNY593UOMVWSfJUxZjYe+D9gDzNbbmb9w7fKqYyvA44ws3nACwR3nb/I\nIbZKkM8y3hI4CniqylvlVMY/BxZY0PVtFNA708kr6jFKM5sC9HTOfV/qWKQwVMYNX6WXcUVVmiIi\npabHKEVEElClKSKSgCpNEZEEVGmKiCSgSlNEJAFVmiIiCajSFBFJ4P8DvJniUZggGdAAAAAASUVO\nRK5CYII=\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Confusion Matrix:\n", + "[[ 975 0 0 0 0 1 2 1 1 0]\n", + " [ 0 1127 2 0 0 0 2 2 2 0]\n", + " [ 2 0 1024 1 1 0 0 4 0 0]\n", + " [ 0 0 0 1006 0 1 0 1 2 0]\n", + " [ 0 0 1 0 979 0 1 0 0 1]\n", + " [ 2 0 0 7 0 879 3 0 1 0]\n", + " [ 5 2 0 1 1 3 946 0 0 0]\n", + " [ 0 1 4 2 0 0 0 1018 1 2]\n", + " [ 1 0 3 10 1 1 1 2 951 4]\n", + " [ 1 3 0 6 8 4 0 6 2 979]]\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAARIAAAD3CAYAAADRydumAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGrdJREFUeJzt3XuwXWWd5vHvQwKEm9xlIMEGGxqlmRYwhVFaSonQigxg\nj23BjIoO1cz00Apqj2L31DDt9MzotONtuodqGlBsES8BypTaXMQL7ZSkgRC5BSGAQGIgIBERFEjO\nM3+s98gmJifr7LXW2ZfzfKpWnb3XXvt9330uv/O+73rXb8k2ERFNbDPoBkTE6EsgiYjGEkgiorEE\nkohoLIEkIhpLIImIxhJIIqKxBJKIaCyBJCIaSyCJiMbmDroBEbPZH7x+J//08Y21jr351meutv3G\njpvUlwSSiAF67PGNLLt6Qa1jt9333r06bk7fEkgiBsps9MSgG9FYAknEABmYYPSvwE8giRggY55z\nvTmSYTYyZ20kvVHSjyStknRun2VcLGmdpNsbtmV/Sd+RdKekOySd3Wc58yT9s6QflnL+skGb5ki6\nRdLXG5TxY0m3SVoh6aYG5ewmaYmkuyStlPTqPso4pLRjcvu5pHP6bM/7yvf3dkmXSZrXZzlnlzLu\n6LctmzOBa23DbCQCiaQ5wN8CbwIOBU6TdGgfRX0OaGPWewPwAduHAouAs/pszzPAsbZfARwOvFHS\noj7bdDawss/39nq97cNtL2xQxqeBq2y/DHhFP+2y/aPSjsOBVwJPA1dOtxxJ84H3AgttHwbMAU7t\no5zDgD8GjqL6TCdKOmi65WzKwEZcaxtmIxFIqH54q2zfZ/tZ4EvAydMtxPb1wONNG2N7re3l5fGT\nVH8o8/sox7Z/UZ5uW7Zp/8ZIWgC8Gbhwuu9tm6RdgWOAiwBsP2v7Zw2LXQzca/uBPt8/F9hB0lxg\nR+AnfZTxcmCZ7adtbwC+B/xhn+15gfRIZs584KGe56vp4w+3C5IOAI4AlvX5/jmSVgDrgGtt91PO\np4APAk2n/w1cI+lmSWf2WcaBwKPAZ8tQ60JJOzVs16nAZf280fYa4OPAg8Ba4Anb1/RR1O3AayXt\nKWlH4ARg/37a9IL2ARvtWtswG5VAMpQk7QxcDpxj++f9lGF7Y+m+LwCOKl3o6bThRGCd7Zv7qX8T\nv2/7SKoh5FmSjumjjLnAkcD5to8AngL6mtMCkLQdcBLw1T7fvztV7/VAYD9gJ0lvn245tlcCHwOu\nAa4CVgCtzJJO1NyG2agEkjW8MPovKPsGRtK2VEHkUttXNC2vdP+/w/TncI4GTpL0Y6oh37GSvtBn\nG9aUr+uo5iOO6qOY1cDqnp7VEqrA0q83ActtP9Ln+98A3G/7UdvPAVcAr+mnINsX2X6l7WOA9cDd\nfbbp+TJrzo9kjqQdNwIHSzqw/Ic6FVg6qMZIEtUcwErbn2hQzt6SdiuPdwCOA+6aThm2P2x7ge0D\nqL4v37Y97f+4knaStMvkY+B4qu78tNh+GHhI0iFl12LgzumW0+M0+hzWFA8CiyTtWH5ui+lzUlrS\ni8vXl1DNj3yxQbsAsOG5mtswG4l1JLY3SPpT4GqqWfeLbd8x3XIkXQa8DthL0mrgPNsX9dGko4F3\nALeV+Q2AP7f9zWmWsy9wSTkrtQ3wFdt9n75taB/gyupvjbnAF21f1WdZ7wEuLUH/PuDd/RRSAtpx\nwL/vsx3YXiZpCbCc6mzbLcAFfRZ3uaQ9geeAs1qYRAbERtS8mAFT7msTMTiH/d52vvwb9S6hedlL\n1t7c8LR8Z0aiRxIxzsahR5JAEjFA1YK0BJKIaGjCCSQR0UB6JBHRmBHPec6gm9HYqKwj+bUGS7db\nLSPlzEw5w9SWNsuZNNkjqbMNs5ELJEAbP8i2fhlSTvflDFNb2iynEBu9Ta1tmA136yLGXJUhbZta\nWx2by7kjaQ9J10q6p3zdveyXpM+UHD+3Sjqy5z2nl+PvkXT61uodqjmSXfaY673nbz/lMXvttx0v\n/Zc7TbmK7rHbpy5jHjvyIu3ReCVeyum+nGFqS91yfsVTPOtnao9FWh62fA74G+DzPfvOBa6z/VFV\nScHOBT5EdR3TwWV7FXA+8CpJewDnAQupYt3NkpbaXr+lSocqkOw9f3v+6orfbVzOZw/5rRZaE9Gf\nZb6u9rG2Wh222L6+pLbodTLVpSEAlwDfpQokJwOfd7W8/YaS2W7fcuy1th8HkHQt1cWkW7zmaagC\nScRsNNH9ROo+tteWxw9TXVcFW87zM+38PwkkEQNkxLOu/We41ya5dC+wPa0LEG1bUusX2CWQRAzQ\n5GRrTY/1edHeI5L2tb22DF3Wlf1byvOzhueHQpP7vztVBZ2etVELmd8jxt1Gq9bWwFJg8szL6cDX\neva/s5y9WUSVhnItVbqO4yXtXs7wHF/2bVFnPZKezO/HUY2xbiwzv02S3ESMFSM2tvj/fHM5d4CP\nAl+RdAbwAPC2cvg3qXLPrqLK0v9uANuPS/pvVAnFAD4yOfG6JV0ObX6d+R1A0mTm9wSSiB4T7Z61\nOW0LLy3ezLEGztpCORcDF9ett8tAsrmZ31d1WF/EyKmWyI/+utCBT7aWaxfOhGqxWcRsMi4X7XUZ\nSGplfi+nry4AtrpiNWLc2Az9dTR1dPkJhirze8RwEhM1t2HWWY+krczvEeOsutPe6PdIOp0jKbdn\nmO4tGiJmlUy2RkQjRsnZGhHNpUcSEY3k9G8HHrt9+1ZyiVz9kxVbP6iGP9jv8FbKidgS0+7K1kEZ\nqkASMRsNe2LnOhJIIgbIVnokEdFc1pFERCNVYqMMbSKikXaTPw9Kl4mNLgZOBNbZPqyreiJGmWEs\nTv92GQo/R5XCPiK2YHJla51tmHV50d7m7q8REZuYRvLnoZU5kogBqvKRDHdvo46BB5LeDGnz2HHA\nrYmYecM+bKlj4IGkN0NaG/dmjRgl1RxJhjYR0dA4LJHvLBSW+2v8ADhE0upyT42I6GHEhok5tbZh\n1uVZmy3dXyMiemRla0Q0krM2EdGKTLZGRCPJ2TrE2sps9l/uW95KOR956ZGtlBPjKXMkEdFIlWox\ngSQimrCG/tRuHaM/yxMxwiYTG7V1y05J75N0h6TbJV0maV65be4ySaskfbncQhdJ25fnq8rrB/T7\nORJIIgasrTQCkuYD7wUWlhxAc6juuf0x4JO2DwLWA5OLQ88A1pf9nyzH9SWBJGKAJudIWsxHMhfY\nQdJcYEdgLXAssKS8fglwSnl8cnlOeX2xpL4mbLpcIr+/pO9IurN0tc7uqq6IUdZWILG9Bvg48CBV\nAHkCuBn4me0N5bDVwPzyeD7wUHnvhnL8nv18hi57JBuAD9g+FFgEnCXp0A7rixg508yQtpekm3q2\nM3vLkrQ7VS/jQGA/YCdmKEthl9farKWKith+UtJKqgh4Z1d1Rowcw4b6K1sfs71witffANxv+1EA\nSVcARwO7SZpbeh0LgDXl+DXA/sDqMhTaFfhpH59iZuZIymzwEcCymagvYlS0PEfyILBI0o5lrmMx\n1T/u7wBvLcecDnytPF5anlNe/7btvnICdb6ORNLOwOXAObZ/vpnXkyEtZrW2FqTZXiZpCbCcamrh\nFqqkYd8AviTpr8q+i8pbLgL+QdIq4HGqMzx96TSQSNqWKohcavuKzR2TDGkxm7V9rY3t84DzNtl9\nH3DUZo79FfBHbdTb5X1tRBXxVtr+RFf1RIw6j8ES+S7nSI4G3gEcK2lF2U7osL6IkdTmytZB6fKs\nzfdhyD99xIDZuWgvIhoTGydGf4F5AknEgI3DHEkCScQAJR/JLNBWZrMPrLqjlXL+90G/20o5MURc\nzZOMugSSiAEb9jMydSSQRAyQyRxJRDSWLPIR0YKJiQSSiGjAztBmSpLmAdcD25d6lpQLiiKiR4Y2\nU3sGONb2L8pVwN+X9I+2b+iwzoiRk9O/UygJUn5Rnm5btjH4lkW0axyGNp0u8pc0R9IKYB1wre3f\nyJAm6czJHJTP8UyXzYkYOkbY9bZh1mkgsb3R9uFUeSKPknTYZo65wPZC2wu3ZfsumxMxlFxzG2Yz\nctmh7Z9R5Y2ckYzWESPD4AnV2oZZl/e12VvSbuXxDsBxwF1d1RcxqsZhaNPlWZt9gUskzaEKWF+x\n/fUO64sYSTlrMwXbt1LdgiIitiDX2kREcwYSSCKiqQxtIqK5BJKoo63MZmfcfX/jMi76nQNbaEm0\nZ/hP7daRQBIxSLn6NyJakaFNRDSXHklENDUGPZLOr7UpVwDfIimrWiM2Zwyu2puJHsnZwErgRTNQ\nV8RoKRftjbqu85EsAN4MXNhlPREjrcUeiaTdJC2RdJeklZJeLWkPSddKuqd83b0cK0mfkbRK0q2S\n+r4jXO1AIqmfZCGfAj4ITPTx3ojZwaq31fNp4CrbLwNeQTUaOBe4zvbBwHXlOcCbgIPLdiZwfr8f\nYauBRNJRkm4D7inPXyHp/9R434nAOts3b+W4ZEiLWU2ut221HGlX4BjgIgDbz5ZcQCcDl5TDLgFO\nKY9PBj7vyg3AbpL27ecz1OmRfAY4EfhpadwPgdfXeN/RwEmSfgx8CThW0hc2PSgZ0mJWqzusqTe0\nORB4FPhsOcFxoaSdgH1sry3HPAzsUx7PBx7qef/qsm/a6gSSbWw/sMm+jVt7k+0P215g+wDgVODb\ntt/eRxsjxljNYU01tNlrsvdetjM3KWwucCRwvu0jgKd4fhgD/Dope+vngOqctXlI0lGAS5Ki9wB3\nt92QiFmr/p/1Y7YXTvH6amB1T5L1JVSB5BFJ+9peW4Yu68rra4D9e96/oOybtjo9kj8B3g+8BHgE\nWFT21Wb7u7ZPnH7zImaBiZrbVth+mOof/yFl12LgTmApcHrZdzrwtfJ4KfDOcvZmEfBEzxBoWrba\nI7G9jmpoEhFtaz+x0XuASyVtB9wHvJuS6lTSGcADwNvKsd8ETgBWAU+XY/uy1UAi6e/ZTOfL9qbj\ns4joQ50zMnXZXgFsbvizeDPHGjirjXrrzJF8q+fxPOAtvHCmNyKaGPLl73XUGdp8ufe5pH8Avt9Z\ni8aR2um6tpGU6OU3t3NVxMpXbmilnBgP/fxWHcjz56EjoqE2hzaDUmeOZD3Pd762AR5nk3PTEdHA\nuGdIkySq9fqT55YnygRNRLTBjMWVaFOuIylB45vlZuAbE0Qi2tfWtTaDVGdB2gpJuWNeRFfGObGR\npLm2N1DddvNGSfdSrd0XVWdlq7kLygV7T1Jdm7NhK8t7I2anIQ8SdUw1R/LPVBcAndSwjtfbfqxh\nGRFjaRSGLXVMFUgEYPveGWpLxOw05mdt9pb0/i29aPsTNco3cI0kA39n+4LpNjBi7I15j2QOsDPN\nbrrx+7bXSHoxcK2ku2xf33tAyalwJsA8dmxQVcRo0hic/p0qkKy1/ZEmhdteU76uk3QlcBRw/SbH\nXABcAPAi7TEGsTliGsZkjmSq07+NBm6SdpK0y+Rj4Hjg9iZlRoylcT79y2YuO56mfYArq8WxzAW+\naPuqhmVGjJ8hDxJ1bDGQ2H68ScG276NaXh8RUxj3oU1ERC25iXjEoI1BjySBJGKQPP6nfwejjWxi\nw3aR8hC1p63MZifd+dNWyll66J6tlDPShufXo2/DF0giZhExHpOtCSQRg5ZAEhGNjMnK1gSSiEFL\nIImIpsbhrE2nC9Ik7SZpiaS7JK2U9Oou64sYSWN+rU0bPg1cZfut5V6kyRMQ0WsEgkQdnQUSSbsC\nxwDvArD9LPBsV/VFjKpxmGztcmhzIPAo8FlJt0i6sKQTiIheYzC06TKQzKVKHn2+7SOoMtD/xh36\nJJ0p6SZJNz3HMx02J2I4zZb72vRrNbDa9rLyfAlVYHkB2xfYXmh74bZs32FzIoZUeiRbZvth4CFJ\nh5Rdi4E7u6ovYhTV7Y1Mp0ciaU6ZTvh6eX6gpGWSVkn6cjnxgaTty/NV5fUD+v0cXecjeQ9wqaRb\ngcOB/9FxfRGjp/0eydnAyp7nHwM+afsgYD1wRtl/BrC+7P9kOa4vnQYS2yvKsOX3bJ9ie32X9UWM\nojZ7JJIWAG8GLizPBRxLNbUAcAlwSnl8cnlOeX1xOX7akiEtYtDa7ZF8CvggMLledk/gZ+X2u1DN\nXc4vj+cDDwGU158ox09bAknEoNUPJHtNnuEs25m9xUg6EVhn++YZbD2Qa20iBmt6E6mP2V44xetH\nAydJOgGYB7yIanX5bpLmll7HAmBNOX4NsD+wWtJcYFegr4xVwxdIhiibWGxZW5nN/uK+Fa2U899f\nengr5QxES7/ytj8MfBhA0uuAP7P9byV9FXgr8CXgdOBr5S1Ly/MflNe/bff3B5ihTcSAaaLe1sCH\ngPdLWkU1B3JR2X8RsGfZ/342s2C0ruHrkUTMMl2sWrX9XeC75fF9VLfL3fSYXwF/1EZ9CSQRgzQC\nq1brSCCJGLQEkohoYlyyyHc22SrpEEkrerafSzqnq/oiRtYYXLTXWY/E9o+orq9B0hyqc9ZXdlVf\nxKjSGCx5mKmhzWLgXtsPzFB9EaMht+ycllOBy2aorojRMvodku4XpJXcBycBX93C68mQFrNaMqTV\n8yZgue1HNvdiMqTFrJfJ1lpOI8OaiM0bgd5GHV3fIGsn4Djgii7riRhp6ZFMzfZT9JkoJWI2GJcF\naVnZGjFgmhj9SJJAEjFIIzBsqSOBJGLAsiAt6tlmTjvlTGxsp5wh0lZms3+9cl0r5Vz+8he3Us60\npEcSEU1lsjUimjFjkac4gSRiwDJHEhGNZB1JRDRnj8XQpusl8u+TdIek2yVdJmlel/VFjKJc/TsF\nSfOB9wILbR8GzKHKSxIRvXKtTa3yd5D0HLAj8JOO64sYOcPe26ijsx6J7TXAx4EHgbXAE7av6aq+\niJFkYML1tiHW5dBmd+Bk4EBgP2AnSW/fzHHJkBaz2gzcsrNzXU62vgG43/ajtp+jyknymk0PSoa0\nmPUmz9xsbRtiXc6RPAgskrQj8EuqTPI3dVhfxEjKHMkUbC8DlgDLgdtKXRd0VV/ESKp7xmbIg03X\nGdLOA87rso6IUVatbB3yKFHDTGSRj4ipTNTctkLS/pK+I+nOshD07LJ/D0nXSrqnfN297Jekz0ha\nJelWSUf2+xESSCIGTHatrYYNwAdsHwosAs6SdChwLnCd7YOB68pzqG4Vc3DZzgTO7/czJJBEDJJr\nriGpsY7E9lrby8vjJ4GVwHyqZRiXlMMuAU4pj08GPu/KDcBukvbt52Pkor2Z4JYWAUjNyxi28Xgb\nn4n2Mpudcff9jcu49y3TWw/VxVkbSQcARwDLgH1sry0vPQzsUx7PBx7qedvqsm8t05RAEjFo9YP7\nXpJ6l1BcYPs3zoRK2hm4HDjH9s/VE6xtW2o/dCWQRAySp7Vq9THbC6c6QNK2VEHkUtuTN6Z7RNK+\ntteWoctkgts1wP49b19Q9k1b5kgiBq2lla2quh4XASttf6LnpaXA6eXx6cDXeva/s5y9WUR1Pdy0\nhzWQHknE4LU30DgaeAdwm6QVZd+fAx8FviLpDOAB4G3ltW8CJwCrgKeBd/dbcQJJxIC1tSDN9vep\n1rhtzuLNHG/grDbq7jpD2tklO9odks7psq6IkWRgo+ttQ6zLNAKHAX8MHAW8AjhR0kFd1RcxikS9\nxWjDvoy+yx7Jy4Fltp+2vQH4HvCHHdYXMZrGII1Al4HkduC1kvYsqQRO4IWnmiICxiKQdDbZanul\npI8B1wBPASuA37h5raQzqdb5M48du2pOxHAytS7IG3adTrbavsj2K20fA6wH7t7MMcmQFrPaOMyR\ndHr6V9KLba+T9BKq+ZFFXdYXMZKGPEjU0fU6kssl7Qk8B5xl+2cd1xcxWmyYGP2xTdcZ0l7bZfkR\nY2H040hWtkYM2rDPf9SRQBIxaAkkEdHI5J32RtxQBZInWf/Yt7zkga0cthfwWMOq2iijfjlb/z2Z\n2fYMUzlD9r351sGtlPNb9ZoEMPyLzeoYqkBie++tHSPppq0ld5mJMlLOzJQzTG1ps5wXSCCJiEYM\nbBz90zYJJBED5faSgw/QKAaSNm772datQ1NO9+UMU1vaLOd5YzC0kcfgQ4wbSRup7pc8l+reJKfb\nfrrPsl4H/JntEyWdBBxq+6NbOHY34N/Y/r/TrOO/Ar+w/fF+2jib7brdPn7Nvzit1rFXPfTpm1uf\nn2lJkj8Pp1/aPtz2YcCzwH/ofbEk6532z8720i0FkWI34D9Ot9xoaAzSCCSQDL9/Ag6SdICkH0n6\nPFWul/0lHS/pB5KWS/pquZ8Jkt4o6S5Jy+lJJiXpXZL+pjzeR9KVkn5YttdQJQn+bUkrJP11Oe4/\nSbqx3Bv2L3vK+gtJd0v6PnDIjH03xtEYBJJRnCOZNSTNpbo/61Vl18FUw5wbJO0F/GfgDbafkvQh\n4P2S/hfw98CxVNnBv7yF4j8DfM/2WyTNAXamuifsYbYPL/UfX+o8iiqp8FJJx1DllzkVOJzqd2g5\ncHO7n36WsGHjb6TpGTkJJMNph57bCfwT1b1K9gMeKPdohSolw6HA/yt3UtsO+AHwMuB+2/cASPoC\nJXHUJo4F3glgeyPwxORd6nscX7ZbyvOdqQLLLsCVk/M2kpY2+rSz3ZD3NupIIBlOv5zsFUwqweKp\n3l3AtbZP2+S4F7yvIQH/0/bfbVJH7gjQpjEIJJkjGV03AEdPZuaXtJOk3wHuAg6Q9NvluC2dErgO\n+JPy3jmSdgWepOptTLoa+Hc9cy/zJb0YuB44RdIOknYB/lXLn20WcXWtTZ1tiCWQjCjbjwLvAi6T\ndCtlWGP7V1RDmW+UydZ1WyjibOD1km6jmt841PZPqYZKt0v6a9vXAF8EflCOWwLsYns51dzLD4F/\nBG7s7IOOO4M9UWsbZllHEjFAu87d269+0Sm1jr16/YVDu44kcyQRgzYG/8wTSCIGKad/I6INTvLn\niGhm+Fet1pFAEjFIY5JqMad/IwbNE/W2Gsp1Vj+StErSuR23/NfSI4kYIANuqUdSrpn6W+A4YDVw\no6Sltu9spYIppEcSMUh2mz2So4BVtu+z/SzwJeDkTttfpEcSMWBu7/TvfOChnuergVe1VfhUEkgi\nBuhJ1l/9LS/Zq+bh8yTd1PP8Atvtp37sQwJJxADZfmOLxa0B9u95vqDs61zmSCLGx43AwZIOlLQd\nVfKpGckVkx5JxJiwvUHSn1Klf5gDXGz7jpmoO1f/RkRjGdpERGMJJBHRWAJJRDSWQBIRjSWQRERj\nCSQR0VgCSUQ0lkASEY39f3WS57bNTBYlAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "print_test_accuracy(show_example_errors=True,\n", + " show_confusion_matrix=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Visualization of Weights and Layers" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Helper-function for plotting convolutional weights" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": {}, + "outputs": [], + "source": [ + "def plot_conv_weights(weights, input_channel=0):\n", + " # Assume weights are TensorFlow ops for 4-dim variables\n", + " # e.g. weights_conv1 or weights_conv2.\n", + " \n", + " # Retrieve the values of the weight-variables from TensorFlow.\n", + " # A feed-dict is not necessary because nothing is calculated.\n", + " w = session.run(weights)\n", + "\n", + " # Get the lowest and highest values for the weights.\n", + " # This is used to correct the colour intensity across\n", + " # the images so they can be compared with each other.\n", + " w_min = np.min(w)\n", + " w_max = np.max(w)\n", + "\n", + " # Number of filters used in the conv. layer.\n", + " num_filters = w.shape[3]\n", + "\n", + " # Number of grids to plot.\n", + " # Rounded-up, square-root of the number of filters.\n", + " num_grids = math.ceil(math.sqrt(num_filters))\n", + " \n", + " # Create figure with a grid of sub-plots.\n", + " fig, axes = plt.subplots(num_grids, num_grids)\n", + "\n", + " # Plot all the filter-weights.\n", + " for i, ax in enumerate(axes.flat):\n", + " # Only plot the valid filter-weights.\n", + " if i" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "image1 = data.test.images[0]\n", + "plot_image(image1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Plot another example image from the test-set." + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADjVJREFUeJzt3XGInPWdx/HPV00RbJBoNtvFRrcWKbcEm8qwHOTQPYrFntUYJKERQsSSrZiAxYgnEdLEeLKaS3uCZ3F7Ls1CYxtsPKMYr6JHbPWImZiQpJfTiuyluSybXU2oxT+q5nt/7LNlm+z8ZjPzzDyz+b5fEGbm+c4zz9fHfPLMzO955mfuLgDxXFB0AwCKQfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwR1UTM3NnfuXO/s7GzmJoFQhoaGNDY2ZtN5bl3hN7ObJD0h6UJJ/+bufannd3Z2qlwu17NJAAmlUmnaz635bb+ZXSjpXyV9W1KXpOVm1lXr6wForno+83dLet/dP3D3P0v6haTF+bQFoNHqCf8Vkv4w6fGxbNlfMbNeMyubWXl0dLSOzQHIUz3hn+pLhbOuD3b3fncvuXupra2tjs0ByFM94T8maf6kx1+WdLy+dgA0Sz3h3yvpGjP7ipl9QdJ3Je3Mpy0AjVbzUJ+7f2ZmayT9h8aH+gbc/Xe5dQagoeoa53f3lyW9nFMvAJqI03uBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8Iqq5Zes1sSNLHkj6X9Jm7l/JoCvk5efJksr5///5k/ZVXXknWN2/enKybWcXa0qVLk+teddVVyfratWuT9fb29mQ9urrCn/l7dx/L4XUANBFv+4Gg6g2/S/q1me0zs948GgLQHPW+7V/k7sfNbJ6kV83sf9z9jclPyP5R6JWkK6+8ss7NAchLXUd+dz+e3Z6Q9Lyk7ime0+/uJXcvtbW11bM5ADmqOfxmdomZzZ64L+lbkg7n1RiAxqrnbX+7pOezoZyLJG1z9/S4EICWUXP43f0DSV/PsRdU8OmnnybrW7ZsqVh78sknk+sODw/X1NOE1Dh+tfpzzz1X17bHxtIjzAMDA3W9/vmOoT4gKMIPBEX4gaAIPxAU4QeCIvxAUHlc1YcGe/rpp5P1hx56qEmdnK2npydZ3717d8O2vXXr1mSdob40jvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/C3g8OH0b6Bs2rSpSZ2c7bHHHkvW77333mR9/fr1FWuPP/54TT0hHxz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvmboNo4/rp165L10dHRZD3189jVprneuXNnst7V1ZWsX3BB+vjx8MMPV6wtWbIkue6tt96arFfbL9dee23F2sGDB5PrRsCRHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCqjrOb2YDkr4j6YS7L8iWXSbpl5I6JQ1JWubuJxvX5sy2f//+ZP2ll15K1t09WZ81a1bF2urVq5PrLliwIFmvV6q37u7u5Lp33nlnsp6amlySDh06VLHW29ubXLe/vz9ZPx9M58j/M0k3nbHsQUmvufs1kl7LHgOYQaqG393fkPTRGYsXS5qYLmWrpNty7gtAg9X6mb/d3YclKbudl19LAJqh4V/4mVmvmZXNrFztXGwAzVNr+EfMrEOSstsTlZ7o7v3uXnL3UltbW42bA5C3WsO/U9LK7P5KSS/k0w6AZqkafjN7VtJ/SfqamR0zs+9J6pN0o5n9XtKN2WMAM0jVcX53X16h9M2cezlv7dq1K1lPXY8/HT09PRVra9eureu1i9TXlz6mVNuvqXH+vXv31tTT+YQz/ICgCD8QFOEHgiL8QFCEHwiK8ANB8dPdOfjwww+T9T179jR0+ytWrGjo67eqav/dDzzwQJM6mZk48gNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIzz52Dfvn3J+tDQUF2vf/311yfrN998c12vH9GpU6eS9eHh4WS9o6Mjz3YKwZEfCIrwA0ERfiAowg8ERfiBoAg/EBThB4JinD8H5XK5oa+/cePGZH3OnDkN3f756OjRo8n64cOHk3XG+QHMWIQfCIrwA0ERfiAowg8ERfiBoAg/EFTVcX4zG5D0HUkn3H1BtmyDpFWSRrOnrXP3lxvVZKv75JNPknV3r+v1b7jhhrrWj6re/X6+m86R/2eSbppi+Y/dfWH2J2zwgZmqavjd/Q1JHzWhFwBNVM9n/jVmdtDMBsyM80uBGabW8P9E0lclLZQ0LGlLpSeaWa+Zlc2sPDo6WulpAJqspvC7+4i7f+7upyX9VFJ34rn97l5y91JbW1utfQLIWU3hN7PJlzQtkZS+BApAy5nOUN+zknokzTWzY5J+KKnHzBZKcklDkr7fwB4BNEDV8Lv78ikWP9OAXmasatfzm1mTOsFkqf3O/xPO8APCIvxAUIQfCIrwA0ERfiAowg8ExU93I6TZs2cn65dffnmTOikOR34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIpxfsxYg4ODNa+7YcOGZP26666r+bVnCo78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4/w56OvrS9YPHDiQrFebxuyuu+5K1gcGBpL181W1/TZv3ryKtbvvvjvvdmYcjvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFTVcX4zmy9pUNKXJJ2W1O/uT5jZZZJ+KalT0pCkZe5+snGttq6FCxcm65s3b07WV65cmaxv3749WV+zZk3F2ky+Ln3VqlXJ+sjISLK+bNmyirWLL764pp7OJ9M58n8maa27/42kv5W02sy6JD0o6TV3v0bSa9ljADNE1fC7+7C7v5Pd/1jSEUlXSFosaWv2tK2SbmtUkwDyd06f+c2sU9I3JO2R1O7uw9L4PxCSKp9LCaDlTDv8ZvZFSb+S9AN3/+M5rNdrZmUzK1c7FxtA80wr/GY2S+PB/7m778gWj5hZR1bvkHRiqnXdvd/dS+5eamtry6NnADmoGn4zM0nPSDri7j+aVNopaeJr6pWSXsi/PQCNMp1LehdJWiHpkJlNXJu6TlKfpO1m9j1JRyUtbUyLM9+iRYuS9TvuuCNZ37ZtW7K+e/fuirVWHup7/fXXk/UdO3Yk6+3t7cn6+vXrz7mnSKqG391/K8kqlL+ZbzsAmoUz/ICgCD8QFOEHgiL8QFCEHwiK8ANB8dPdTXD11Vcn64888kiy/uabbybrGzdurFirdkr1o48+mqxX89577yXrb7/9dsXafffdl1z31KlTyfr999+frHd1dSXr0XHkB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgGOdvAZ2dncn6W2+9laynppt+6qmnkuvu2rWr5teWql8zPzY2lqyn3HLLLcl6b29vza8NjvxAWIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/DNAR0dHsj44OFix9u677ybX3bRpU7J+zz33JOvVrqlPuf3225P1anMOXHQRf33rwZEfCIrwA0ERfiAowg8ERfiBoAg/EBThB4KqOlBqZvMlDUr6kqTTkvrd/Qkz2yBplaSJH4Zf5+4vN6pRVHbppZdWrHV3dyfXffHFF/NuBzPEdM6S+EzSWnd/x8xmS9pnZq9mtR+7+z83rj0AjVI1/O4+LGk4u/+xmR2RdEWjGwPQWOf0md/MOiV9Q9KebNEaMztoZgNmNqfCOr1mVjazcrWpowA0z7TDb2ZflPQrST9w9z9K+omkr0paqPF3BlumWs/d+9295O6ltra2HFoGkIdphd/MZmk8+D939x2S5O4j7v65u5+W9FNJ6W+WALSUquE3M5P0jKQj7v6jScsnX2q2RNLh/NsD0CjT+bZ/kaQVkg6Z2YFs2TpJy81soSSXNCTp+w3pEEBDTOfb/t9KsilKjOkDMxhn+AFBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Iyd2/exsxGJf3vpEVzJY01rYFz06q9tWpfEr3VKs/ernL3af1eXlPDf9bGzcruXiqsgYRW7a1V+5LorVZF9cbbfiAowg8EVXT4+wvefkqr9taqfUn0VqtCeiv0Mz+A4hR95AdQkELCb2Y3mdm7Zva+mT1YRA+VmNmQmR0yswNmVi64lwEzO2Fmhyctu8zMXjWz32e3U06TVlBvG8zs/7J9d8DM/qGg3uab2X+a2REz+52Z3ZstL3TfJfoqZL81/W2/mV0o6T1JN0o6JmmvpOXu/t9NbaQCMxuSVHL3wseEzex6SX+SNOjuC7Jlj0v6yN37sn8457j7P7ZIbxsk/anomZuzCWU6Js8sLek2SXeqwH2X6GuZCthvRRz5uyW97+4fuPufJf1C0uIC+mh57v6GpI/OWLxY0tbs/laN/+Vpugq9tQR3H3b3d7L7H0uamFm60H2X6KsQRYT/Ckl/mPT4mFprym+X9Gsz22dmvUU3M4X2bNr0ienT5xXcz5mqztzcTGfMLN0y+66WGa/zVkT4p5r9p5WGHBa5+3WSvi1pdfb2FtMzrZmbm2WKmaVbQq0zXuetiPAfkzR/0uMvSzpeQB9Tcvfj2e0JSc+r9WYfHpmYJDW7PVFwP3/RSjM3TzWztFpg37XSjNdFhH+vpGvM7Ctm9gVJ35W0s4A+zmJml2RfxMjMLpH0LbXe7MM7Ja3M7q+U9EKBvfyVVpm5udLM0ip437XajNeFnOSTDWX8i6QLJQ24+z81vYkpmNnVGj/aS+OTmG4rsjcze1ZSj8av+hqR9ENJ/y5pu6QrJR2VtNTdm/7FW4XeejT+1vUvMzdPfMZucm9/J+k3kg5JOp0tXqfxz9eF7btEX8tVwH7jDD8gKM7wA4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8Q1P8DdB0CkDYcK7EAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "image2 = data.test.images[13]\n", + "plot_image(image2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Convolution Layer 1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now plot the filter-weights for the first convolutional layer.\n", + "\n", + "Note that positive weights are red and negative weights are blue." + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'weights_conv1' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mplot_conv_weights\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mweights\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mweights_conv1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[1;31mNameError\u001b[0m: name 'weights_conv1' is not defined" + ] + } + ], + "source": [ + "plot_conv_weights(weights=weights_conv1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Applying each of these convolutional filters to the first input image gives the following output images, which are then used as input to the second convolutional layer." + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUoAAADuCAYAAABf005JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvelv1Wd6//86hmBjY8cbEGww2CzBTgIhhCRkI8kok0wy\n0ajTzHSUdlRpuo7UVmqfVFVVqX9B1UpV1fbBdDTpjObbJZnJNJpptkIIgQk7hEAAE4MNweANL8f7\nOb8Hn9/7+hyvn2M4x8vx9Xpi4hybw3Xuz3W/r+W+7lgymcRxHMeZmry5fgOO4zjzHXeUjuM4Ebij\ndBzHicAdpeM4TgTuKB3HcSJwR+k4jhOBO0rHcZwI3FE6juNE4I7ScRwngqUzeXFlZWVyw4YNWXor\n2aWpqYm2trbYXL+PqXDbZo+FbFuAo0ePtiWTyZVz/T6mYiHbN921OyNHuWHDBg4fPnz772oO2bVr\n11y/hWnZsGEDR44cmeu3cVs8/PDDc/0WpmUh2xYgFotdnuv3MB0L2b7prl0PvR3HcSJwR+k4jhOB\nO0rHcZwI3FE6juNE4I7ScRwnAneUjuM4EcyoPSgdEokES5Ysua2f9WnrkzM4OAjA8PAwK1asmON3\nk1uMjo4CEIuFrXSdnZ32Z32/oKAAgPz8fADy8vLG/IwzObJvb28vt27dAkL79vf3m13vvvtuAEpL\nSwEoKirirrvuApgXds6Yo5QRGhsbzVHKSDt37kzrd2TCILnobLWwBgYGzKZaWM6d0dvbC8CyZcvo\n6OgA4IsvvgCCjWloaAiAwsJCIHSYFRUV9ufphIHW9NKlS+2zW758OcCi2PS0yTc3N3P27FkALl26\nBMDVq1fJywuC2uLiYgDKy8sBqKysNJtrc5qOgoICeyYqKyvta1FREYD9PbeLh96O4zgRZExRNjU1\nAfB///d/7Nu3D4ArV64AsGXLFnbs2AHAqlWrAHjwwQcBWL16NSUlJUCoQBOJhMluoZ1Zr11MSLHc\nuHGDa9euAXD//fcDi9MemUTranBw0BTlmTNnAPjkk0/sdVI8ZWVl9t9SP1I+o6Oj9vsSiQQQKFUY\nqx6lKDdt2kRVVdWY95FrKMJrbW2ltbUVgC+//BKAixcvmuKTvaQeBwYGxqjx1NdAaC99LSoqQsco\n6+rqAKivr+fee+8F7ly9u6J0HMeJIGOKUsqmrKyM++67DwjyEgDnzp3j4MGDQJjLlIevqKiw3Vrf\n6+3ttZ1bKNfx1FNP8Zu/+ZsAPPDAA5l6+/MaKcp4PE57ezsAJ0+eBGBkZMRsunJlMDdh8+bNANTU\n1JgdlaPJz883BSQFpV2/oqIi6/+W+YbW3MWLF229nj9/HoCzZ8/aupPSkY2GhoZMDSlXOTIyYgqn\np6cHCIsTEOQ8ATZu3AjAc889Z+pS6jTXUJS4ZMkSU9eyyeOPP26RY2rRB6Crq2tM0QcCRSlVqdd3\ndXXZ75d99ZmuX7/ePr87VZQZc5SSu3V1dVy+HJzhl8N89913LTRXIlcL76677jJpre/19fWZARW+\nX79+HQgM093dDcBf/dVfAaGDyFXk0OLxOJ9//jkAhw4dAoIwJjX8gHBRVFZW2p+1sPLz822x3bx5\nc8zfs2fPHv7oj/4om/+UeYeKNcuWLbMHubq6GgjTGxA+rFp73d3d3LhxAxhbQOzr6wNCR6lNKZFI\n2O/QWl67di1btmwBctdRam1WVFTQ0NAAhIMoampqzP4Ky7UmR0ZGTBTI2SWTSXOG8XgcgKNHjwJB\nqL527Vog/Pyqq6snCK7bxUNvx3GcCDLeRwmB5AV44okngEBqazdta2sDsKJEPB63XUXhe09Pj31P\nSlS7cH9/vyki7fgvvfSS7cjji0C5gBTLqlWrWLduHRCGKGvXrjWVo+9J6bS0tNj3xPDwsH0GUj9K\nfaxevZoTJ04AoUofHBy0AlwutrMobK6srLSIRkWBZ555hoGBASBUkgoHu7u7Tf1oPQ4NDdn3pO5V\nuDh58qSlP6RcIb3Wl4WM1OC6detM8U2m8lTUSg3V9WfZrb+/n5GREQBrNdLnMzAwYEpSkayi3Ezg\nitJxHCeCrChKUVNTAwTqRElr5SO18zY3N1u+QbtFf38/+/fvB8Lk+WOPPQbAmjVrLK+j/MeFCxds\n99DOlEtoJ165cqWpHdljZGTE1I5smpoL066snf3WrVuWD1Jj9datWwH4rd/6LVOSFy5cAAIVm6ut\nK6msWLHC1N2aNWsm/H8pF0U6sVjMFLnsE4/HJ+QoP/74YyDIrbe0tABhxNXQ0JCxHNp8RU3gyWRy\njJIWipZkw9Tm/fGN/IWFhba2VQeRTSGMBDZt2pShdx/iitJxHCeCjCnKyY4OakdYsWLFBFUi5VJZ\nWWlKUr+jpaWFxsZGIMxD1tfXA0E+Ta9TPjKZTJpizUX0b0v9N+qYFoQ5MqlGqfbu7m77DLQTf/nl\nl9ZScfHiRQAeffRRIIgA1Bqj3GZdXZ1VbnOd6fLbymXKtkuXLrXcrlR7d3e35dllPymfzs5Oa4uR\n4tm8ebP93lwl3ZqBqtl6fSKRmHDssLu72w4BvPHGG0Bo3/r6erZv3w6QFZtmNfROZboz2Fp8Clsa\nGxvtYa6trQXC3qvKykrrG5QTWLNmzaI+oaIFNX6zSO2LTB3uoLYLFYbuueceIFiIKp7pYV4soXe6\naK2monA8Ly/PbKsNR+1cra2ttiGpPSbX29rSQWtrvENNdZKyb0tLCwcOHADCIq9+bvv27eYos4GH\n3o7jOBHMmqJMh9TpLQpntOsqKdzZ2WmJdSmhysrK2x7ttliQWr927ZrZWfZT6H7mzBnb4WXvyRSU\nMxaFjaOjo2ZbzTvQAYt169bxyCOPAFjj9WTFDWciet4bGxs5deoUECpJTSZ79tlns2pPV5SO4zgR\nzAu5IEWjtpVkMmnnYZV3S50komStGqFzuZBzp0hJqoXo+vXrY2b+QdhE3dHRYefE1bzrinJqlDvT\n+u3q6rJGaDX1a61u2rSJJ598EgjbvZzpkVKXLY8fP26KXWt39+7dAGzbti2r72VePAVykPqan59v\nYeH4Ez2FhYW20PRQ5/rphttlaGhowlTp/Px865tUv6Aur1+1apX1ovnmMz2jo6MWEoobN27YsBIV\ndVL7XrVenWhGR0dtc1e4feLECRNMOn2jcY3ZxkNvx3GcCOZcUXZ3d/Ppp58C4aSgkpISO1esIo16\nLVPVpqueyVErVldXl4XearfasmWL9abqvL16/jZt2pTzJ0UyRTweHzNQGYLQUP2/Cht1Yuzhhx8e\nM3LNmZ7+/n4rhOmU3rVr1yxtpNN5GsybbVxROo7jRDBnilIK8fz581y9ehUI24O2bt1qDafjz9OW\nl5ebovRG6MlJHXgq+0nN1NfX2wmezz77DAhnJq5atcpbViJQznzJkiWWU1de8vDhw9ZormKkigx1\ndXW+XtNAed+rV6/y0UcfAeF5+Xg8bsWbxx9/fFbflytKx3GcCGZdUSp/plzO8ePHraVCVcGlS5fa\nRCGpTOUm7rnnHs9NToGa9FXpbm1ttYlCujajt7fXbK+rD9QI7Tm0qdG6VT63oKDA7KdjdUeOHLG8\npdaybOvXC0+P7Kvn/vTp0xw+fBgIfUBVVRUvvfQSkJ0JQdMx645SYZ8Os7e3t1u7is5r33XXXVbg\nUQiuEzrqnXRC1MenkDv1/hHZTQ/wlStXLJRRInz16tWAnxSZDD3ACrk1kPf8+fMcP34cwIYdd3R0\nmGN8+umngbA9yMPuyZF9lSLSoJYPP/zQUkMSSd/61rd49tln5+BdeujtOI4TyawpSp1i0C4hRRmL\nxWzArxqg29raTBVpAo5OinjYPZHxp2/UZjU8PGxDYhUynjt3zkJsta7k6sVWmUDrVl+V1mhpaeHX\nv/41EKqgtWvX8txzzwHh6DqNYnMmR0VdHSjRZWGHDx+2SEmtQF//+tfn7KSYK0rHcZwIZs09qylX\n7RNqCdq8efOEC7OSyeSEO6p9dt/kDA8Pm5KUstQVD08++aSpR+XISktLLSepo6A+eWlyEomE5dRl\nI0VC+/fvtzy6VNFjjz3Gnj17gHAykzM1iURiwjHFX/7yl0Cg2PXMS6Ur8pwLZsVR9vX12QDTc+fO\nAeGpkIKCAgurVYzYunWrheEKHb3QMDn9/f1mN3UP6MGNxWITBiY/+OCDVjRbLJPLb5fBwUEL/7Re\n5RxPnz7N6dOngbAo1tDQYA/z+OnczkRGRkZMQKkX9cyZM0DgKJ966ikgvN1gLvFP03EcJ4JZUZRd\nXV0WsqgnSrfUnTlzxu4+VuvP1q1bra3CVc/0DA8PWz+fdmOF1C0tLWZnDTotKytzdZ4myWTSQm+d\nlZdqf//9982mKooVFxd7GmMGjIyMWApOa1cKvra21tIY82HqkitKx3GcCGZFUaq1AsJmaLWwtLe3\n27lYtQL19PTYzuyNutOTeqGVzh7rRMP69evNpmqx0PBYJ5q8vDxTlIqEUu+fVrFBefT169f7oOMZ\nEIvFzB+ooKtBx7t37+b5558HmBcTrVxROo7jRDAr2191dTXf+MY3gPCqTlUMjx49avlL5Xx6e3tt\np3GmZ2hoyHZhNY6rw+DGjRtmR531doWePgUFBdYwriq2puk/88wzVu3+yle+AgS5NM//ps/IyIgd\nCdVVs/IBr7zyyryqT8yKo8zPz7cFprsutKAuXbpkD696z9asWZP2xemLnaGhIRu4oJYqDeaNxWKW\n1lCLxbp16zw8nAHaaFQge/XVV4Ggt08bk2y7mO+Wvx3y8vKora0FwpN3sul8cpLgobfjOE4ksfEN\nydO+OBa7CVzO3tvJKuuTyeS8Pd7jts0eC9y24PbNJmnZdkaO0nEcZzHiobfjOE4E7igdx3EicEfp\nOI4TgTtKx3GcCNxROo7jROCO0nEcJwJ3lI7jOBG4o3Qcx4lgRod+Kysrkxqou9Boamqira1t3k6E\ncNtmj4VsW4CjR4+2zeeTOQvZvumu3Rk5yg0bNnDkyJHbf1dziKYWzVfcttljIdsWIBaLzevjgRs2\nbLCrexcaulY4Cg+9HcdxInBH6TiOE4E7SsdxnAjcUTqO40TgjtJxHCcCd5SO4zgRZOXylN7eXgD6\n+vqA4CJ5XZvqd+HcPj09PTQ3NwMwPDwMBPfi6AIs3Z2jy8b0FcLLsZxobt26BUBbW5tdtbx8+XIA\nVqxYAUBxcbGtZbft1OjOoXg8TldXFxDaN5lM2lW0umRM9w7l5eWZXefDcPGMO8rLly/bLYDXrl0D\noKGhgUQiAWAPtb4mk0n7fwMDA1P+Xr1maGjILh7S4s11ZJe9e/dy/PhxILycbfXq1XYnsh5iLbqB\ngQG7l1r2m468vDyqqqrs9wIsXbrULiPTXeu5ijZ43RB67Ngxu/hOl+OtWrUKCC7J00M93c2LqetW\nD7w+p9Tfkau27enpAeCzzz7j3LlzAJw5cwYInKg2c9lVF+StXr3a1vF0G5E+n2XLlpktdUFZaWmp\nvS6d9T8dvhU6juNEkDFFqZDw5MmT7Nu3D4Du7m4gUI2S29qZRX5+vu0KUj8jIyO2i2in1df8/Hzb\nMRbqsamZ8sUXXwCB0tGuvHnzZgA2btxoIffIyAgQ3OcNQeqjo6MDCMOdvLw8UzYKHWX/0tJSVq4M\nTsppN19MSOkpXIzFYmbT1HAcAlsrotHaTCQSY342lZ6eHrO7rmXetWsXDQ0NQBhh5RoKt69du8b5\n8+cBLCo6f/68rUHZUOtu+fLl9nlIdab6hfH+obKy0q6+1R32O3bsoKamJiP/DleUjuM4EWRMUWoH\nLSoqMqXX398PBLkfqaLOzs7gL/7/815Lly41lak8Qk9Pj+UhVQRSYWh0dJSNGzcCi0dRKke5fPly\nHn/8cQCefvppALZt22avkyJvb28HgryYVL2UZSwWM3tr97548SIQfBb33ntvVv8t8xmtTSmY9evX\nT8iP6TWpOS+pmqGhIeLxOBAW1mTrixcv8umnnwKhKq2urrbIIFeRHWpqaiyXu379eiCwyfXr14Ew\nCtLX69evW85YvmJgYMD+LPtrfS9fvtzWrp6DiooKy1Mqf3m7ZMxRKgk7NDRkhQA5RwgXh6S2wpbl\ny5dbKK3QZHBw0JK7MsSlS5eAwLjjw/dcp7KyEghCNW0c9fX1E14nu6ggky7ahBTqLCYUGl66dMk2\nmC+//BKAs2fP2sOtz0DdBkVFRVYR18Oo/wfhBq8Nqr29nZaWFiC0c0dHR85XzGU3fZ2K1tZWIEzh\nNTc3m9OUXZPJpKU2rly5AgSpPgjEWHV1NRD6mOHhYfuM7pTc/pQcx3EyQMYUpcLs5cuXm0eXKmxu\nbjbVqP+nnWF4eNhUo3bvkZERk90qDL399ttAoFh37NiRqbe9IJA6GR4eNjtmggsXLgDw1ltvAfDd\n7343Y797oaD1uGzZMvuzIqJr166ZImxqagJCdTM4OGgFHqWJRkdHLcSTOpXiOXjwoCmkRx55xH6X\nQvXUVpbFiGyur+mO7jt48CAQ+BiF5VKva9assc/mTnFF6TiOE0HGFKUS4Kk5FxULKisrLcEqJalG\n1P7+ftt9U3fm8c2pamLfs2fPhILD4ODgmMbTXEP53Mlyj6Ojo6ZKZFu9fqpc7okTJwD4m7/5GyA4\nJADwO7/zOxl81wsDrbmNGzeydu1aIMyVNzQ0WA5TNlb0Mzg4aHn3VFWoXL1ybR999BEQFCekNqWa\nysvLM5ZDW2yowKk1v27dOlOSsm9qy5U3nDuO42SZjCnK1HYfod26vLzcdmk18Ep5piqi1DYA5X+k\nRB966CEAvv/97/Piiy/az+r1qbmmXENqOfXstliyZIntnLJH6u6pP+tra2sr//qv/wrAO++8A8C3\nv/1tAGt+Ho9+NhcrtLLt8uXLJxyJLSsrs+Ohsq2+JpNJW8tao6n54/379wNjG6J37twJwIMPPmjf\ny/XcpOw1FbJ/OmtrcHDQahaffPIJEHYQ3HffffZZyY+Mb/q/E7IyFGMy9KbHD8VYsmTJBOfW3Nxs\n58QlsZ955hkg7B+E8GxuMplcNOe+p2P8KSYIF6Ae6gMHDvDBBx8AYSj/l3/5l9P+3kwuuIXKZLYd\n348KodP8+OOPgbCoU1dXZydHtCGtXbv2jvv7FjoSUNM5VLWvHTt2jGPHjgFhCkQF47KyMhNmqQWc\nOw25Re5JBMdxnAwza4oyHbQbNzU1mbTW2eOvfvWrQFDyl5JUq0ZJSYmrngikbPbu3WuNz3/yJ38C\nwP333z/tz7pt0+fs2bMAdiuhGqm3b99up3AUoq9atSon0xmZQmpT6rG1tdUiTJ3K27JlCwC1tbUT\nWueSyWTGRrT5p+Q4jhPBvFKUasg9d+6cFXO2b98OjD3TrOLP+MKQMzWa2HLw4EFLev/Zn/1Z5M8l\nEgm3b5oMDQ3x/vvvA+ExyNTmZ+UoZf/FOKFpJuj5lirv6OiwnK5sKZWuAcDZYl44ShlEpxgOHTpk\nMvr5558HwrPk8XjcqlqqGKZW2p2xqJ/vF7/4BRCclf/jP/5jYPoz4blc6c4W+/fvt2q3Nnr1/N5/\n//3W36ehED7tf2oSiYQNzNCch87OTttkZEsVcSdbp5kq5ICH3o7jOJHMCymmsPDAgQNAELb8xm/8\nBjC2HQhy/xROJonH46Ykf/WrXwFBEvzP//zPI3/WlWT6qFD27rvv0tjYCISKp66uzr5KwU/WD+sE\nqPhy69YtG0unAlk8HjcVrv7Jyc5yR/Vu3g7+NDiO40Qw54qysbGRQ4cOAWGTbn19PS+88AIw9qIs\n4Unw9HjnnXd4/fXXgXDu4p/+6Z96bjLDqIDzwQcfWMuaig2aG1pRUWF5dmciqbNoIVCRmkmgwdIP\nPPCAFW00EHi2okp/GhzHcSKYM0WpFp9Tp06xd+9eINxVvva1r004d6xpQ3fddZfnJiNQlfCtt97i\nyJEjAHzrW98C4C/+4i+m/VlXkumjLo2f/vSnQDC/UutW1VlN3a6qqlp0k/lngp79q1evAkGLoP6s\nlqA1a9aYQk+9/1tkIzcp5sxR6szm/v37rYVFBZwnn3zSXqdTJJOd43TGovTEm2++CcDPf/5zO3Xz\nt3/7t5E/7z2T6dPV1cXPf/5zIBxbV1lZyaZNmwDsq/oodfOiMxY5SJ2203jFixcvmiDatWsXEAzG\nUZFstjcdfyocx3EimHVFqalAStCeOHGCrVu3AvDEE08AwS6s2+6EprZ42D0RFWB0vlgtQUNDQ3a9\ng87ETvfzribT56233rIxdVJDDQ0NZmeF3PrqYffkqHija0l0IWF3d7elMdRitWXLlikPl2Qz7AZX\nlI7jOJHMmqLUyHsVF1TA6erq4pvf/CYwdnCs2iykIBfjVarpohzvf//3fwPhhUt1dXWRxRvw6UAz\nQdekvv3226bgdYVEaWnphCKO5iU6E0kkEty8eROAzz77DAiVZSwWs4HUulxvMjWZqelAUcyao5Sk\nVkVWxZzS0lJLdKfe/avTC3KUqQNTnZChoSEOHz4MYAN5tSm98sor0/6swhW3bTR6IGXrkydP2owC\n/b/q6mpzkOOnbTsT6e/vt41HRVtt+hUVFbaONRSjqqpqzD04kNnz3NPhobfjOE4Es7Ld9fX12SQQ\nnevWDrJjxw4bwpnKYh+Rny6dnZ2cPn0aCMNCKfO//uu/nvZnXUmmj4qQGih9+fJli4T0tba21iYD\neRtbNPF43KYsKdUmW5aUlNh0MKXd2trarCg222vXFaXjOE4Es6Ioe3p6bMKKco66SXH79u123YNa\ngkpKSlztpElXV5flGtWo/+qrrwLheVjnzlHuTHmzxx57zPLoGi5dXV3tRccZsGTJEmsg3717NxAW\nbkZGRiwfKZVZXFw8Zy1srigdx3EimBVFuWLFCqsG7tmzBwirVWVlZfT39wPhVOg1a9bYzuJN0NNT\nWFjIAw88AIQTa773ve/N5VvKGUZGRiy3rgp3auO+ZiPqnu65VDwLkZUrV1o0mS7yG7PVFiRmxVEW\nFhbaSHw5TBVzuru77a4cSeyhoSE/LZImZWVl7Ny5EyCtu81HR0dndOn8YmZgYMAcpe7AUahYWlpq\nG7x6JWtra/0ETpbR2p1tR+lPiuM4TgSxmXjmWCx2E7icvbeTVdYnk8mZ6fxZxG2bPRa4bcHtm03S\nsu2MHKXjOM5ixENvx3GcCNxROo7jROCO0nEcJwJ3lI7jOBG4o3Qcx4nAHaXjOE4E7igdx3EimNER\nxsrKyuRksyMXAk1NTbS1tc3bOw/cttljIdsW4OjRo23zueF8Ids33bU7I0e5YcMGG4W/0NDdwPOV\nDRs22H1CC42HH354rt/CtCxk2wLEYrF5feplIds33bXrobfjOE4E7igdx3EicEfpOI4TgTtKx3Gc\nCNxROo7jROCO0nEcJwJ3lI7jOBFk7M6c1Lssent7gfBqz5s3b7J0afBX6WtJSQkQ3POiS5r8itpo\ndFF8d3c3APn5+X5F6h2gu5l6enrs7qabN28CwT05ul65rKwMCC6+A6isrKSwsBAILxlzpqa7u9su\nDxwYGACCO5t035DsPF/vHMr45WKxWMzu4718OeiTPXz4sF3SJGeo29dWrlxp909PdzmWJrGPjo7a\nAtXrS0pKzPHKEecqeih1K6BsAaGNtGnNlGQyeds/u1DRhvPhhx9a0/Qnn3wCwPXr1+3u7nXr1gFQ\nXl4OBPdPa4NK51K3u+66y5yCLiirqamhqqoKmL8O4k6Rczx8+DBtbW1AcBc9wKZNm+yCNtlQDnN4\neNjWswTXZGi9FhQU2Gcj/5PJteyht+M4TgRZlV/r168Hgp1D3r2jowOA5uZmADo7O20XKSoqsp8d\nHR0FQgWqXaWtrY2+vj4gCH8A6uvreeihh4Bwx891ZINly5aZjbRTSwVJZadLf3+/7fo1NTWZeqvz\nGinKVPVRV1cHQDweNzt/+umnQGj3np4e+1ld+zs6OjpB1SvCqaqqsjX62GOP2e+QktJVzbmGnvf2\n9naampqAUD0mk0mLjHp6eoDwOR8YGDBbyxcMDw/bWpfN9XX16tWUlpYCmVWSwhWl4zhOBLOS0Nu9\neze7d+8e8z3lLAcGBqz4o51mZGTEvqddQon2xsZG9u/fD8DJkyeBYJevra0Fcl9RXrx4EYDTp08D\nwb9dBQbZb3BwEAgKEFLdytu0t7dbnlN5NxWI9u7dy9mzZwH45je/CWB2zVWknGtqanjllVeAUNU0\nNjaaklS+XYWeWCw2oTgRi8VMIUn5tLS0AIGNFTlJPS1btoxcvwU1tUCr9blx40YAdu7cadGPbKK1\nODw8bGpUBbf+/n6LOvU92be/v9/smw3mrPIx01BDxt2yZYst3kOHDgFw33335fSC0wO5b98+jh49\nCoQLqqSkxBajigutra1AsPi04WiBdXZ2mqPU5BT9/rfffttC723btgGBA1ls3Qh64Orr66mvr7+t\n33HlyhUA3nnnHQA++OADK9jImS5dutQKmbmK1l9eXp6tQRUgU0VNalFSaJPWxg9h0evSpUsAXL16\n1b4/2e/IFB56O47jRDArivL69etWWJDX1y4xPDxsyddUWa3vjU/M9vb2WvvGF198AQShT662V0C4\na7a3t1uoIoVdXl5uoaKS3lKMhYWFtqNLKULYC6jXKdzu6urihRdeAGDHjh1AEDrF43H7fU56VFdX\nA7B161YgCA2lMh944AF7Ta63s+nfV15ebmtXir25udnWrFS2FHZxcbG9PtUXaC0q7aa1m+3n3xWl\n4zhOBFndzpQ3LC4unjLPNTg4SGdnJxDuCkuXLrX8xfid4vTp05w/fx4IFejKlStzulFaBYfly5fb\nDqzdFsKcpFSjCjclJSVmF+3Eq1atMpWj3/Xhhx/afz/66KNA2HoFriRvB61prdGtW7daMUO2ra6u\nzmoBYj6gFsHKykorwMgvLFmyxFraZC9FR52dnbaOxeDgIBcuXADgjTfeAMJijiKsbOGK0nEcJ4KM\nKcrpqs6FhYUTFJ+UYn5+/oTK33R5mytXrpja1DHI++67b0yzeq6havZUZ7qVc1Q+MhU15ErRp6rD\nEydOAPAgL2e6AAASdUlEQVTuu+8CgT1zXeFkk/b2do4fPw6MnX0AcO7cOdauXQsEBzBg6s8zl0h9\nzidDz7KOcmqdDg8Pmx9Q/rKpqcna4nTctKKiAsj+nVizlkmezpHKONOFz+rS379/vznWPXv2AMFJ\nipmeQsklZL+Ztpr8wz/8A8CY3kmdHnHSR2vz7Nmz9lDLCSrlUVxcbJv5Yjn1NBNSU0kwdtCIUkRX\nrlyxHmrZ+etf/zoQnqbKFh56O47jRDAvehPSKcT86Ec/AoKWIMn0Rx55BAhOmCy2pug74a233gLg\nzTffBDAV+fzzz1uo7qSPFGXqhCCpRx0QqKiosOZ1X6szQ4We48ePc+7cOSAM1X/3d393Vt6DPxWO\n4zgRzAtFOR2a0PLee+8BQaHn8ccfB8j5WX7Z4sc//jEQtmK8+OKLQNhk7qSH8mQ6AlpUVGTf03Qs\ntcSUlJRYE3out7JlEtnu888/B+DgwYOWr9S5fBUys82cO8qoRfNv//ZvQHi2c+vWrRZyq3fKHWX6\n/OQnP+HgwYMAfPvb3wZmL3zJJRKJhDlDnZxKDak1vETOcfPmzZ7WmCE6yaQCzvnz561z4Hvf+96s\nvhf/5BzHcSKYM0UZpSR1+mbfvn1AOIT2iSeesPFg6YzgdwI0CuzNN9+0QsPLL78MhCkMJxq1uXV1\nddl5Y4XbBQUFNi5M/ao6hTNbIWKu0NnZOeFqjkQiwWuvvQbM/jhFV5SO4zgRzHmOcir+/d//HQhH\nyd97770APPjgg3Y5kyfF0+c//uM/ADhz5owNUd65c+dcvqUFiQpgJ06csLP1KjqsXr3azi5rluKG\nDRsAX6vpoqlix44d48CBA0CwZgGeffZZfv/3f39O3pcrSsdxnAhmXVGms7O++eablp9QblLTuNet\nW+e5yRmg89x/93d/BwTHHJ977jkgqMQ66aHcpC7Iunr1qh2z04ScpqYmy/cqJ+m5yZmResW1ujM0\n0+EP/uAP5ux9zZqjTMdBKhH+s5/9zL6nkWDbt28HQqM50YyMjPAv//IvQHhH0WuvvWZn5J1o5CC1\nNvUgFxYWWlua2oJWr15tRYYtW7bM9ltd0Khf+tixY0Bwf5PGB37/+98HmHDv1mziobfjOE4E86qY\no2k2N27csMvHdApHjbsedqfP3//93/PTn/4UCE/dvPrqq9Ze5USj4s2pU6eA8Fx3LBazc906DFFQ\nUGDFG1+n6SH76sJADZFubGy0NfuHf/iHc/PmUnBF6TiOE8GsKMqo/OT//M//APCf//mfQNBu8dJL\nLwFhrke7txONjta9/vrrdq2t7umWQneiSSaTltvVMUUdQ+zt7TU7K0dZV1eX0wOks8G1a9eAcMrS\nL3/5SyBo6NcR29LS0rl5cynMi9D79ddfB8IBso888oj1TSpMTB3k6UyPFtupU6d4+umngbBrwEmf\neDzOl19+CYSh9M2bNwG4cOHChPtadu7cuSimlmeK/v5+G3jx0UcfAWFXwbPPPst3vvOduXprE/DQ\n23EcJ4I5V5SfffaZneVUmPjQQw9Zj5/v0OmjoaY//OEPASgrK+O73/0u4KdwboehoSHKysqAMLxW\n6H3x4kU7x62bKxUFOenR19dnivLXv/71mP/327/923PxlqbEFaXjOE4Ec64om5ub7fSCTjjs3LnT\ndnInfdTCojxaWVmZKSDleBOJhBXX/Pzx9JSWlk65Dnfv3m3numXb2Z5os9AZGBiwoceaM6mm8t/7\nvd+bs/c1Ga4oHcdxIphzRVlcXGwTy4uLi4GgzcLvl06PoaEhqxiqWVe5s127dtnRT+FTttNnOsVd\nUVFhcwhk02Qy6Sp9BiSTSZsEpuZyRZXzjTl3lIlEgoaGBiCU3ytXrvSb6tKkvb2dGzduAGEI+Oyz\nzwJB76QXcbKHBvb6KZzbI5lMWn+0xNJ8vbfJ5YXjOE4EMU1HSevFsdhN4HL23k5WWZ9MJuft6CG3\nbfZY4LYFt282Scu2M3KUjuM4ixEPvR3HcSJwR+k4jhOBO0rHcZwI3FE6juNE4I7ScRwnAneUjuM4\nEbijdBzHiWBGRxgrKyuTujxpodHU1ERbW9u8PYjrts0eC9m2AEePHm2bzw3nC9m+6a7dGTnKDRs2\ncOTIkdt/V3PIfL8KwW2bPRaybQFisdi8PvWykO2b7tr10NtxHCcCd5SO4zgRuKN0HMeJwB2l4zhO\nBO4oHcdxInBH6TiOE0FWr4Lo6+sDoL+/n0QiAUBhYWHwFy8N/ur8/Hx7vd83MnO6u7vp7u4Gwqsg\ndN9QYWGhfc/vypmcgYEBANra2ujo6ACw2xVHR0ftmoeKigoguJkRgvudZFu/tmRqhoeHAbh165bZ\ntb29HQj8g+wru+q+pxUrVphvmA9rNyuO8uzZswC0tLQAwcKTE5RBdKnQqlWrzCBynpOh+0m6u7vN\ncLoedNmyZbZYF4uz1ZW0Fy5csMUoB1lQUAAE19Xqz6kb0ng0vHlwcJB4PA6ETrekpMQufctFh9DW\n1gbA559/zoULF4Bw/V64cMEc6cqVQb+37niprq62y8WmuzNHa/Xuu+82J3DPPfcAgfOVbadb+wsZ\nOccLFy5w6dIlALsM7+rVq7bOZFddXV1WVmb2nc42sm9hYaH5g1Q733333cCdr925d9WO4zjznIxt\nY1Il8XickydPAsEuDdDZ2WlKT7uvdury8nJTPSIWi5mC1I6jXWVwcNB+1wMPPADAtm3b7KL68b8r\nl4jH41y5cgWA06dPA4Fql62kemSDgoIC+3Pq9b96vZA94/E4g4ODY35HTU0NtbW1ALY75xKyRSKR\nMNWhtdzc3Exra+uE10Fgn9QUx1RI8axcuZKNGzcC2K2jO3fuZNu2bUAY2ucaUoXbt2+3NXXt2jUA\nzpw5Y/a9desWECq/FStWmF0VDaWuW9lVr6+srKSmpgYI7bt7927q6+uB6T+jdHBF6TiOE0HGFKVU\nSW9vr6lG3dXb2dlpO7FyYFevXgWgo6ODoaEhAFMziUTCvqcL0fXzx48ft3yG8jvl5eX2d+aiorx+\n/bp93bt3L4Cp9mvXrpmalqKU+h4ZGbEdV7txLBazIptsVVVVBQSfjRLtUjiJRMLsnYuKUmuotraW\nuro6AB588EEANm3aRFNTExDm2qSGWltb7Xtaq7FYzNap/p++Xr16lZ6eHiDMi8ZiMaqrq4HcVZRS\ng/n5+Tz00EMA9nX37t22jj/77DMgtNfAwICt4xUrVtj3RH9/P4Ap0kuXLnH+/HkAzp07BwTRq4Z1\n3KmizHgGubm52R5OLYL6+npbQPoqent76e3tBcIqeTKZtIdYDvDy5WAuQEtLizkGJX7z8/Mn/N5c\nQqFgW1ub2UWL57777rPX6YFV9ba7u9s2H1XGU9MaClXkHGVjgHvvvdf+7lwtNEDooFIdlULkXbt2\n2ff0kKqIdu3aNXOao6OjQJAmUgh55swZAHO0AEVFRUAoFvr6+uwzW4w89dRTPPXUU2O+J/v19PTY\n2tUGPTAwYJuNUlD6+sknn/Bf//VfQCDMICggad3f6UbkobfjOE4EGZMKCqWLi4ttx9TXvLw82zml\nTiSxq6qqJoTLJSUlppj0/95//30gUEubNm0CsK8bNmyYF71W2ULhy6ZNmywVofBwxYoVtvNK4UiZ\nt7W1mV0UqoyOjlobhf6fQpYvv/zS/k7t4jU1NRZ6L2a0DtetWzfm61QoJFSr0ejoqH0+elZqampy\nMp1xJ8geUXZRkUasWbPGRr0p9L5x44Y9C3dK7noXx3GcDJExRSmVsmLFCstNKveYSCQsLyYPr+Rq\nao5GKrK8vNzynMo3qEn11KlTfOc73wHCJtWCgoKcVpRqsSgqKrJ2qdQil/7tyq2J4eFhy2/q54aH\nh81ujY2NQKh+8vPz7TPT7xr/O53pUa5cBTgVOZPJpH2O+l5DQ4Pl2507Y926daYylecsLS3NWH49\nd72L4zhOhsiYolR1OvU4lxQihDutFKTybr29vaaIVBVM5Z/+6Z+AsMH6kUcesQqWduZcVpMQ5nVT\nd8fUBnKpRtlbr0skEmMqsuNRS8Z7770HBDlOVbvVwqEjp040LS0tFvmoBUiKvK+vz54BHd9duXLl\ntMcfnbEMDAxMqISLvLw88x9qON++fbu1zN0ps9b3IWc2vnCjPrbxKCH74YcfAmELxssvv2wSO9UR\nL2a0YYwPM/Ly8ibdRC5evAjAz372MwC++OILILDnE088AYSnnpxo1I515swZ+7NSS2pnycvLs81N\nbW3FxcU5eX4+08iGPT091vM7nmPHjnHjxg0gbHurr6/PWGojt6WY4zhOBpi3ncQ//OEPgbCh9Ktf\n/SoQhDLaVVLDTyc9EokEn3zyCRC2BSl037RpE7t37wY85E4H2U0N+x0dHZZS2rJlCxCqof7+flM6\nUpQedqeH7DyZmtREoiNHjpg/0InAjRs3Ziwt54rScRwngnmpKD/99FOOHj0KhEWfl19+GQh2ifHz\nF530aWxstLyvlJDmI77yyis8+uijc/beFhpqpVIj+dDQkBVqlHvU+k0mk9ZCl86cRWfs2e6p2Ldv\nHxC0Y23duhWA+++/H8hsDWNeflL/7//9Pys4fO1rXwOwgQWVlZUestwBhw4dsgG1eoh1ymfnzp2T\nVsediQwODtrZ+ObmZiDoUR0/wEGh9z333DOhK2GxDJm+HUZGRqwXVamKVJQ20pAYgB07dgBh1Xu6\nYdUzxUNvx3GcCOaVoty/fz8QyGm1Az333HNAOInI2yluj1OnTgGBjTUBRwUbhdupk4icyVEvZGtr\nq53jlvIpLy+3U05qD1J6qLKycsLJHGciKtx0dHRYz+RkylC+QtOBtm/fbuv3TkeqTYYrSsdxnAjm\nhaJUo+iPfvQjIGjcfe211wDsGgLtxl7AmRk6IfKrX/0KCNS6dm3lch5++GHA1fp0pM4EhWCNKo+u\n4uLq1atNBSkPqUjo7rvv9vxvGkghtrW1TTpB6NChQwBW7NVrduzYYe1X2cAVpeM4TgTzQlH+4Ac/\nAELVU1dXx/PPPw+EU3IyWcFaTHz88ccAvPHGG0Cg3rdv3w6ETfxqp3CmRjlzzey8evWqfU8tQcPD\nw3Zkbvw91cXFxZ6bnAblftWyVl5ebm1roq2tzRSljopq7TY0NGS1G2bOHeV7773H22+/DYRDMV55\n5RUzwPgBGE76nD592k446QRDaWkpL7zwAoCd63amZryD1HCW9vZ2a1vR4IXh4WFzkJ4ySo/U4g2E\nhRhtMKmcOHHC2oIUcitttHbt2qy+Tw+9HcdxIpgzRamd5Ac/+AGffvopAK+++ioQNJkrCe5Kcuao\n4PCLX/yC48ePA2HB4cknn+Sll14CwpDRmRo1jGvCkq4wSR3Eq/PEhYWFpjIVCWlalq/jydF1MSro\n6tbE1FNLaujfu3evNfJrJoFO42T7EIorSsdxnAjmTFH++Mc/BuDAgQP2PeXMqqqqcn4YbzbRXclH\njhyx5LiUzle+8hWfNZkmQ0NDtLS0AOElYVI3VVVVdkmYznAXFxdb7kzFRz/PPTWjo6Om0JXbTW0W\n18GIf/zHfwSCHKWut1VuUpFntpn1T1EDBH7yk58AwRi1b3zjGwBWMezv7/ees9tAI+k0KOD48eMW\nquzcuRMIOgp8E0qPvr4+K+LorifdZtnZ2WnT+nWvU21trYXYWr8eck9NPB43O2mzEe3t7fzzP/8z\nEPZX19fX21yCzZs3A5NP7s8G/sQ4juNEMOuKUv18Z86cAYKpKrraQZ31U10P4UyOVKNsql6zpqYm\na11RuK2+VCeaeDxutlV7j4oHfX19FlarkLBs2TJXkjMgPz9/ymf98uXL/O///i8QnqV/6aWX5uwa\nGFeUjuM4EcyaolTDsxLgKtwkk0l27doFhPPknPQZGBiw2xR1Ckd54Pr6estNaid2RZk+RUVFbNu2\nDQivF1Ae/YsvvrC5nirgFBUV+azUGTBdE35tba017auFaO3atdaSNdu4onQcx4lg1hSlcja6dEkN\nuU8//bTt1s7M6erqMgWpI6BS60uWLOHFF18EMNXupE9paemUl6zde++9NnVf55QHBwf9qGKGKCsr\ns7PeymNu27ZtwnXXs8WsOMp4PG7tFTrDqRvV3EneGYWFhVYEU9uPQpb6+noLHZ3MM741xZ1kZtHw\nFtm1oaFhzobjeOjtOI4TQUxnrtN6cSx2E7icvbeTVdYnk8mVc/0mpsJtmz0WuG3B7ZtN0rLtjByl\n4zjOYsRDb8dxnAjcUTqO40TgjtJxHCcCd5SO4zgRuKN0HMeJwB2l4zhOBO4oHcdxInBH6TiOE4E7\nSsdxnAj+P9VNNSIcvuqDAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot_conv_layer(layer=layer_conv1, image=image1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following images are the results of applying the convolutional filters to the second image." + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": {}, + "outputs": [ + { + "ename": "AttributeError", + "evalue": "'AxesSubplot' object has no attribute 'flat'", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mAttributeError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mplot_conv_layer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlayer\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mlayer_conv1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mimage\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mimage2\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[1;32m\u001b[0m in \u001b[0;36mplot_conv_layer\u001b[1;34m(layer, image)\u001b[0m\n\u001b[0;32m 24\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 25\u001b[0m \u001b[1;31m# Plot the output images of all the filters.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 26\u001b[1;33m \u001b[1;32mfor\u001b[0m \u001b[0mi\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0max\u001b[0m \u001b[1;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0maxes\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mflat\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 27\u001b[0m \u001b[1;31m# Only plot the images for valid filters.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 28\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mi\u001b[0m\u001b[1;33m<\u001b[0m\u001b[0mnum_filters\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mAttributeError\u001b[0m: 'AxesSubplot' object has no attribute 'flat'" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADYBJREFUeJzt3HGI33d9x/Hny8ROprWO5QRJou1YuhrKoO7oOoRZ0Y20fyT/FEmguEppwK0OZhE6HCr1rylDELJptolT0Fr9Qw+J5A9X6RAjudJZmpTALTpzROhZu/5TtGZ774/fT++4XHLf3v3uLt77+YDA7/v7fX6/e+fD3TO/fH/3+6WqkCRtf6/a6gEkSZvD4EtSEwZfkpow+JLUhMGXpCYMviQ1sWrwk3wuyXNJnrnC7Uny6SRzSZ5O8rbJjylJWq8hz/A/Dxy4yu13AfvGf44C/7T+sSRJk7Zq8KvqCeBnV1lyCPhCjZwC3pDkTZMaUJI0GTsn8Bi7gQtLjufH1/1k+cIkRxn9L4DXvva1f3TLLbdM4MtLUh9PPvnkT6tqai33nUTws8J1K35eQ1UdB44DTE9P1+zs7AS+vCT1keS/13rfSfyWzjywd8nxHuDiBB5XkjRBkwj+DPDe8W/r3AG8WFWXnc6RJG2tVU/pJPkycCewK8k88FHg1QBV9RngBHA3MAe8BLxvo4aVJK3dqsGvqiOr3F7AX01sIknShvCdtpLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDUxKPhJDiQ5l2QuycMr3P7mJI8neSrJ00nunvyokqT1WDX4SXYAx4C7gP3AkST7ly37O+CxqroNOAz846QHlSStz5Bn+LcDc1V1vqpeBh4FDi1bU8Drx5dvAC5ObkRJ0iQMCf5u4MKS4/nxdUt9DLg3yTxwAvjASg+U5GiS2SSzCwsLaxhXkrRWQ4KfFa6rZcdHgM9X1R7gbuCLSS577Ko6XlXTVTU9NTX1yqeVJK3ZkODPA3uXHO/h8lM29wOPAVTV94DXALsmMaAkaTKGBP80sC/JTUmuY/Si7MyyNT8G3gWQ5K2Mgu85G0m6hqwa/Kq6BDwInASeZfTbOGeSPJLk4HjZQ8ADSX4AfBm4r6qWn/aRJG2hnUMWVdUJRi/GLr3uI0sunwXePtnRJEmT5DttJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNDAp+kgNJziWZS/LwFda8J8nZJGeSfGmyY0qS1mvnaguS7ACOAX8GzAOnk8xU1dkla/YBfwu8vapeSPLGjRpYkrQ2Q57h3w7MVdX5qnoZeBQ4tGzNA8CxqnoBoKqem+yYkqT1GhL83cCFJcfz4+uWuhm4Ocl3k5xKcmClB0pyNMlsktmFhYW1TSxJWpMhwc8K19Wy453APuBO4AjwL0necNmdqo5X1XRVTU9NTb3SWSVJ6zAk+PPA3iXHe4CLK6z5RlX9sqp+CJxj9A+AJOkaMST4p4F9SW5Kch1wGJhZtubrwDsBkuxidIrn/CQHlSStz6rBr6pLwIPASeBZ4LGqOpPkkSQHx8tOAs8nOQs8Dnyoqp7fqKElSa9cqpafjt8c09PTNTs7uyVfW5J+UyV5sqqm13Jf32krSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSE4OCn+RAknNJ5pI8fJV19ySpJNOTG1GSNAmrBj/JDuAYcBewHziSZP8K664H/hr4/qSHlCSt35Bn+LcDc1V1vqpeBh4FDq2w7uPAJ4CfT3A+SdKEDAn+buDCkuP58XW/luQ2YG9VffNqD5TkaJLZJLMLCwuveFhJ0toNCX5WuK5+fWPyKuBTwEOrPVBVHa+q6aqanpqaGj6lJGndhgR/Hti75HgPcHHJ8fXArcB3kvwIuAOY8YVbSbq2DAn+aWBfkpuSXAccBmZ+dWNVvVhVu6rqxqq6ETgFHKyq2Q2ZWJK0JqsGv6ouAQ8CJ4Fngceq6kySR5Ic3OgBJUmTsXPIoqo6AZxYdt1HrrD2zvWPJUmaNN9pK0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqYlDwkxxIci7JXJKHV7j9g0nOJnk6ybeTvGXyo0qS1mPV4CfZARwD7gL2A0eS7F+27Clguqr+EPga8IlJDypJWp8hz/BvB+aq6nxVvQw8ChxauqCqHq+ql8aHp4A9kx1TkrReQ4K/G7iw5Hh+fN2V3A98a6UbkhxNMptkdmFhYfiUkqR1GxL8rHBdrbgwuReYBj650u1VdbyqpqtqempqaviUkqR12zlgzTywd8nxHuDi8kVJ3g18GHhHVf1iMuNJkiZlyDP808C+JDcluQ44DMwsXZDkNuCzwMGqem7yY0qS1mvV4FfVJeBB4CTwLPBYVZ1J8kiSg+NlnwReB3w1yX8mmbnCw0mStsiQUzpU1QngxLLrPrLk8rsnPJckacJ8p60kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNDAp+kgNJziWZS/LwCrf/VpKvjG//fpIbJz2oJGl9Vg1+kh3AMeAuYD9wJMn+ZcvuB16oqt8HPgX8/aQHlSStz5Bn+LcDc1V1vqpeBh4FDi1bcwj4t/HlrwHvSpLJjSlJWq+dA9bsBi4sOZ4H/vhKa6rqUpIXgd8Ffrp0UZKjwNHx4S+SPLOWobehXSzbq8bci0XuxSL3YtEfrPWOQ4K/0jP1WsMaquo4cBwgyWxVTQ/4+tuee7HIvVjkXixyLxYlmV3rfYec0pkH9i453gNcvNKaJDuBG4CfrXUoSdLkDQn+aWBfkpuSXAccBmaWrZkB/mJ8+R7g36vqsmf4kqSts+opnfE5+QeBk8AO4HNVdSbJI8BsVc0A/wp8Mckco2f2hwd87ePrmHu7cS8WuReL3ItF7sWiNe9FfCIuST34TltJasLgS1ITGx58P5Zh0YC9+GCSs0meTvLtJG/Zijk3w2p7sWTdPUkqybb9lbwhe5HkPePvjTNJvrTZM26WAT8jb07yeJKnxj8nd2/FnBstyeeSPHel9ypl5NPjfXo6ydsGPXBVbdgfRi/y/hfwe8B1wA+A/cvW/CXwmfHlw8BXNnKmrfozcC/eCfz2+PL7O+/FeN31wBPAKWB6q+fewu+LfcBTwO+Mj9+41XNv4V4cB94/vrwf+NFWz71Be/GnwNuAZ65w+93Atxi9B+oO4PtDHnejn+H7sQyLVt2Lqnq8ql4aH55i9J6H7WjI9wXAx4FPAD/fzOE22ZC9eAA4VlUvAFTVc5s842YZshcFvH58+QYuf0/QtlBVT3D19zIdAr5QI6eANyR502qPu9HBX+ljGXZfaU1VXQJ+9bEM282QvVjqfkb/gm9Hq+5FktuAvVX1zc0cbAsM+b64Gbg5yXeTnEpyYNOm21xD9uJjwL1J5oETwAc2Z7RrzivtCTDsoxXWY2Ify7ANDP57JrkXmAbesaETbZ2r7kWSVzH61NX7NmugLTTk+2Ino9M6dzL6X99/JLm1qv5ng2fbbEP24gjw+ar6hyR/wuj9P7dW1f9t/HjXlDV1c6Of4fuxDIuG7AVJ3g18GDhYVb/YpNk222p7cT1wK/CdJD9idI5yZpu+cDv0Z+QbVfXLqvohcI7RPwDbzZC9uB94DKCqvge8htEHq3UzqCfLbXTw/ViGRavuxfg0xmcZxX67nqeFVfaiql6sql1VdWNV3cjo9YyDVbXmD426hg35Gfk6oxf0SbKL0Sme85s65eYYshc/Bt4FkOStjIK/sKlTXhtmgPeOf1vnDuDFqvrJanfa0FM6tXEfy/AbZ+BefBJ4HfDV8evWP66qg1s29AYZuBctDNyLk8CfJzkL/C/woap6fuum3hgD9+Ih4J+T/A2jUxj3bccniEm+zOgU3q7x6xUfBV4NUFWfYfT6xd3AHPAS8L5Bj7sN90qStALfaStJTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ18f+GmWq6NWLIwgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot_conv_layer(layer=layer_conv1, image=image2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Convolution Layer 2" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now plot the filter-weights for the second convolutional layer.\n", + "\n", + "There are 16 output channels from the first conv-layer, which means there are 16 input channels to the second conv-layer. The second conv-layer has a set of filter-weights for each of its input channels. We start by plotting the filter-weigths for the first channel.\n", + "\n", + "Note again that positive weights are red and negative weights are blue." + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4VOW1P/A1uUOGkMsAISSwoRGppWA1Smop5fAg9rQc\nUIsQMVIOpYiCivxyBC0H0IdSLxStYlGk1B8FvFGliJfDoUApR9EGRQSNEHEMCIEMSQi5k2SfPyQ8\nnrPW2pN5XJnf48/v58/vznK/m5ksJ7Pf/b4+13UJAAC+mpj/1wMAAPj/AZopAIABNFMAAANopgAA\nBtBMAQAMoJkCABhAMwUAMIBmCgBgAM0UAMBAXCQ/HEhPd53sbJbXNieoNX5fnXygpEStafzO5Sz7\n/PMgVVWFfOFH+dUEUlNdJyuLHzh2TC/q0kXOc3L0mk8+YVGwvp5CTU2df42xsa4TH88PaNdBRHur\nB4h5//76cJOS5Pyjj/aGXNft4TlIA4G0NPG1bCD9Ors0VcsHzp5Va07E8de5ujpIdXVReL9mZLiO\n8D6rrNF/tZua5Lx3z1a15sSpWJZF6xqJiPz+gJuR4bC8VR8yxfIhExFRj/rP1JrajH4sKy8PUnV1\n+OuMqJk62dlUvGULy3cF+6o1I5LekQ8MG6bWHHqpmGXXX58XfoAGnKwsKl63jh+YP18vGjxYzpct\n02smTGBR3o4dYUZnw4mPp+K+wmumXQcR+V7eIOZLligdk4gGDZLzyy/36e9mQ05WFhU/+yzL99MQ\ntWZIcLN8YOdOteb+1OUse+qpKL1fc3KoePt2lm94I12tKS2V84VzatSa+x9NYVm0rpGIKCPDoV/9\niveFqiq9Ji1NzmcUz1BrdhWu4j8/o2PXiT/zAQAMoJkCABhAMwUAMBDRd6YNrQm0v5p/1zZiz0Nq\nTem8eWKee9FFas3ASd9jWdKRjzswQgOff060YAGLj/3nf6ol2YGAfGDfPv08c+bw7MMPw43ORvfu\nRGPHsnj/VP7dX7vee+R8ctwL+nkGXxvpyGxVVxO98gqLQ9/XvzOl0aPFeEPtOLUkJPzbtLSEHZ2J\nlvffp1BGBsvHeNRMvu8+MX+3dKFas3DkLpZt3lAbdnxWehx9l2bcyW8cPri4Qa2Z8cfvi/mRPcqb\nmYhGCHdN/VVHOzBCfDIFADCBZgoAYADNFADAAJopAIABNFMAAANopgAABiKaGtUlpomG+I/wA9fq\nU2DWVN8t5kvzlcf2iKh+/HiWtYUfno2MDKKbbmJxzOuv6zWZmXLu8QjiiE1zWfbx5/5wo7ORnU30\nwAMsHvLwr9WSkSN/JeZl+RPVmkJ5llH09OxJdPvtLB4VV6+WvP1BVzGfXKC/Ayev49PM3mw83IEB\nfnVx/fpRYKEwpcnjd3KfMJWKiMi/aJFa8+FBvotxY1yU3q9ERKmp4rS1ef/m0RluWC/GA7weDRem\nRdIuPi1Mgk+mAAAG0EwBAAygmQIAGEAzBQAwgGYKAGAgorv51NJCFArx3HHUkqW1d8gHilPVmq5/\n+QvLYubyu9+d4f2j6dTzLn43f/o9PGu39DfKItwnT6o1uwqOsyzvn8+FH6AFn4/a4vjuCDFnzqgl\nG/IfE/M125TXl8T1r4mI6O9/9x6emVOniB5/nOenT6sl/R95RMyzevO72e0KC19j2bF3o7NwcoUb\noN83TmP5zzzeSgPOyNeSMv82vehRvqBy0smorPH9haoqanvxRRbHVCs7IxDRmoKtYr6lRV+cp0hY\nOLu2sWNtEp9MAQAMoJkCABhAMwUAMIBmCgBgAM0UAMAAmikAgIHIpkbV1BBJeyG9/bZaMjdXXtBk\neeEhtWZrcCA/dcz94cdnoF8/ouXCVkjl5R5FdXVi3JycrJYk/Pa3PKysDDM6G8o2VzR7jr6XV1aS\nPLZpb72qnygovy/u9BydoZYWcXraTmm61Hl8B6AvHK/i+w9dMHUvi7a/1hhudCbSyvbSxFl8al7q\nLL1myo3y1KgNUz327JL2M0tMDDc8O4mJFNOX7z/X4rE327Tn5Pdsfn66WiNsAUXx8eGHR4RPpgAA\nJtBMAQAMoJkCABhAMwUAMIBmCgBgwOe6+gIO7Id9vgoiiuLqBv9DP9d1e3T2SXCNUfFNuE5co6Gv\nw3VG1EwBAECGP/MBAAygmQIAGEAzBQAwgGYKAGAgomfzA0lJruP38wNez5QPGSLGrT791LHNDSwL\nHj9OoaoqZX8QO4Fu3VwnEGB5+bkMtSZDOdTcrJ8nubmKZcGKCgrV1HT6Nfr9ATc93WF5T3+9WlMW\n6irmOTn6eaQdboiIysr2hqJxFzgjI+BmZzssP3VKr+nT/KmYN3m8x6V3chkRnXbdTn8tk5Pl19Lr\nvnJmY1A+0EN/SY5W8nUmamqC1NAQ6vRrJCJK9/ncbCFPiI1Va9zWVjH3ZWVFdO5gdTWF6urCXmdE\nzdTx+6l43Dh+YN06vWirvA9LTVJPtSQluJ9leTfeGHZ8FpxAgIrvu4/lD56YotbcfLOcHz2qn2fY\nZ3wfmrx77gk7Pgvp6Q4VFRWz/I7h76o1tz59mZj/7nf6eVavlvNZs3xRmeKSne3Q1q38Or3GvDQ4\nWcxLn31WrZHeyT8KNzgj6ekO3XUXv8ZzHntAzfuY7xlFREQzZ6o1c5+7kmXr10dnnysiomwi4jtt\nEWV366bWNCv7QyXccktE58576qkO/Rz+zAcAMIBmCgBgAM0UAMBAZItDZ2URLV7McylrN3++GKf8\n7GdqSc0Pf8qy1gSPxXktue4Xiwr/Lx5brZNwv4qIiA4c0GuedyeyrIr0xZkt9aw+RHdsGsUPhIar\nNSv/+Zh8YPpGtea2ZcvEfJbHwsWW4j8rpV6/5N/xx10qL1hORLS5YIOYj8uWbn+cN3gwi2IXLQo/\nQANduoinp2PKy0VERL+UXxd66y215Fph3ejXpC8xO0lCSgpl/+AHLH97kT6IYe+vkg+kpuonkn6Z\n168PNzwiwidTAAATaKYAAAbQTAEADKCZAgAYQDMFADCAZgoAYCCyqVFnzxLt3MnzkhK95uWXxXhi\n7Rq15IXpvVkWqz3obS0jg6iwkMUPDeeP07XbvvsdMR9D8qO0RET0W75x/cPlH4Ufn4XkZKL8fJ6/\n8opeM326GBePH6+W5F1zTaQjM9V25gzVC9c0/xX9MWth2/QveMyNmziT78N+5Mxj4YZnIqXmGI3Z\ndjc/UFSkFxXvE+MNZ/iUxHaT/zqXZf4qj+elrWVmitfk9QT2unUzxLy0VK+Rlh6pj9UfWf0yfDIF\nADCAZgoAYADNFADAAJopAIABNFMAAAOR3c1PSyOaMIHFE6fKq7ATEb3Q+Iicz3lTP8/gW3nWwQVa\nv7KqKqIXX2Tx8gL5jj0RUZIymWFUoXDH/Ly2PcJ/78roLLZ7IqYP3Z+0lOUL79PHS7W1YnxJnb6k\n+6nkqCzCrorJyaGu8+ax/PmAvtLKDQXymO9dxu/Yt5PWQImPDz8+C20nT1L9ww+zvKvX4kPSjBwi\nmjzaox0Iv/dRXemktlZciCU7W1iw57ysku1yPnKkfh5hgeyuFR1byxyfTAEADKCZAgAYQDMFADCA\nZgoAYADNFADAAJopAICBiKZGHS+PoYUP8GlQL5QM0Yt++Usxrhx0lVoyeAI/Fgrp+/aYSkoi+va3\nWbz7z3rJS4v3i/m9D+j/LmPH8kyZfWTO7ycaLm73JKzy0E7ZN6dr+RG1pPA6ZdrUy9GZMnUmsSe9\n1p9Pg/perl6T3Vse8/FBa9Wax6qnsCwxMfz4LJwZcDm9srSY5ZO8pqX9WXkzb9qk1zzzDM/q6rwH\nZ+nECaIlS1i8dpm+n9OhbHkK3MDR+nSqhcP5dKrj8e92YID4ZAoAYALNFADAAJopAIABNFMAAANo\npgAABnyuqy9UwX7Y56sgoo499W+vn+u6PTr7JLjGqPgmXCeu0dDX4TojaqYAACDDn/kAAAbQTAEA\nDKCZAgAYQDMFADAQ0bP5yckBNzXVYXnvLtV6UVJSZDkRHREe966rC1JTU6jTH+oOZGS4Tt++/EB9\nvV4UJ/8z1rV1UUuS606xLFhZSaHa2s6/xvR015H22vjoI71o6FA5P3xYLWmpqRHz94lC0bgLnJwc\ncNPTHZb3SmlQaz4tl18zr3UTevbkWWVlkOrqovB+TUpyHT9fU6Hp9Gm1JvHii8W89eOPIzp3GRGd\ndt2oLLQQiI93HWHBg2aP9QESsrLkAydO6CcaMIBFwVOnKFRTE/Y6I2qmqakOzZrFF1W4d7DHIiSD\nBsl5rr7axMQC/oF527bo7I/k9O1LxX/7Gz9QzK/7gkBAjN9u0Bc6GVb8BMvyHnww7PgsONnZVLxl\nCz+Qp/8bt70jX3/M2J+oNZWvvy7mGVGa4pKe7tBdd/Fxzx0tL0xDRDRZWZxmzx79PLNn8+yRR6L0\nfvX7qXjcOJaX/vGPak3umjViXvODH6g1bUL2T2FHZ8dJTKTi736X5WUeL0zfW26RDwgLplywfDmL\n8ubODTs+IvyZDwBgAs0UAMAAmikAgIGIvjPt7R6nexsXsvzD3Pv1ohY5nn+tXrJ5MV+MNe+Axw0g\nQ5+WxdKU2SksX+vs1IuU71O/u1HfV/yqu/jCtSWV+vdcppqbiY4d47nH92wtsfL37wm3366f57Ty\ndF1GdBaH7tKFaPBg4YD2PT4RbShXFg7OVt7IRESXLub/na5nvQdnRXktc2+8Ua/JzxfjFI9/l/0l\nJfzU4UdnZ+BAor/+lcUHdvLF6tstUda6HruR97ALpxnP35uN4UdHRPhkCgBgAs0UAMAAmikAgAE0\nUwAAA2imAAAG0EwBAAxENDWK4uLERydTvqNPdcn+138V8+de8ZgGdIvwqOO5c2GHZ6F/bBmt9d/G\nDwz3mMvVKE+e2LZNLxk/nmdlZWEGZ6Qp3k9HMq9i+YqNes3yK64Q8w35j6k1k+fPiHhsllJSiMaM\nFh6EnDpdrSndsUPMc9etU2tq8vh0qtau3cIP0EKfPuLjkVNWXKmWrN30knzghhvUmnWNfPrj6bXR\neWSWiKj8VAw9+DifBvX883pNYaGcP/CAXhPszafzhUIdu058MgUAMIBmCgBgAM0UAMAAmikAgAE0\nUwAAAxHdzW9M6UmHfnwHywcmzVNrWpTFM7reeqt+oj/8gWehUNjxmdAWARFXzPjC81VjxHzvbv00\nwSDPvBbzt5RYU0EDtq1i+fL8VLVmQ947Yj5ypH6eVbX8HERE9PTTXsOzc/QoUVERi/cXrVVLhgwb\nJuZv9r9JrblqAf+diD1+tAMD/OrOtCTTayF+537CBL3GN16+079+/fVqzUOl/G7+9liPFeuNxccT\n9e7Nc69FuxPy5IW+586Zoxel8t+BvLs9dhL5EnwyBQAwgGYKAGAAzRQAwACaKQCAATRTAAADaKYA\nAAYimhp18KBLF1/MFxxx607rJ6itFfNp83uqNUXCttaNE6OzqEJTTi4deXQzy5Nj9ZpJ798r5q+W\nL1VrpClFb78dbnRG2tqIpNeloEAtmXzN9+UDgfvUmhl3Cqu5EJGym7m9Hj2IpvNFTcqFmW/thtx8\ns5jn+/WaMYv5Yi+HGt4MOzwL3c+F6Cfla/iBhx9Wa9wbvycf2Jetn2joUJ4lJoYZnZ2Mo/toyp1p\nLC//uT5tSe48RKm/+IVaE/jd73jY3LHdrvDJFADAAJopAIABNFMAAANopgAABtBMAQAM+FyXL9Ov\n/rDPV0FEn3XecDz1c123R2efBNcYFd+E68Q1Gvo6XGdEzRQAAGT4Mx8AwACaKQCAATRTAAADaKYA\nAAbQTAEADES00EkgLs514uP5AWHflAuqlYUI0viiBRe0tbEoeOYMhRoafGGG+JWlpQXcrCyH5V1O\ne6yOcY4v/kJEdDSuv1oi/TNWVQWpri7U6deYmhpwMzMdlpeX6zVxyjvFaw2Ii87uFfN3iULRmFIT\nFxdwExIcll90kV7zySdyPqhXlV4kvMeDdXUUamzs9NeyW7eAm5HhsDxw6kO1pqH/JWKekKCfJ/bT\nUpYF6+sp1Nzc6ddIRBRITnYdqc/4PE4fK69OdDZJf+t1a+aLNgVDIQqdPRv2OiNqpk58PBU7Dj9w\n3XV60csvy/kNN+g1jY0sylurb4JmKSvLoWefLWb5kHV360XSBnxEdEdgg1rSpw/PHn88OitjZWY6\ntGoVv8Zly/SaQEDOpY0B272xQ37/JUZpvmBCgkODBvHr3LJFr9E2ontzzgt6kfAfzHv11XDDM5GR\n4dDChfwapz2urAxFRPv/L/95IiLpV7tdSuE4luXt2hV2fFac1FQqvkVYbywpSS9SPuRtz52hlow6\nxvtM3qJFYcdHhD/zAQBMoJkCABhAMwUAMBDRd6bU1CR/Qz96tF6zYIGca19OEdHx1a+x7Nx/bA83\nOhNdfI00JOkQP+B1jXv2iPG1w/WS3bt5Fq0ne/3JLo3I53eOnvTrdyDWDBdWcyeiQ/OnqTUJ1crW\nAcOGeQ/QyCX96qj4yXdYPrnoSrXmzaKXxHx76kS1prGAHzvzVnS+/w4cfY+mzUnhBwYNUmucofJ3\n2Sked+aqDx9mWWv44Zk52tKb5lYvZPny6fqNNlqxQoxHNfKdES4oLOTZ8uXhhkdE+GQKAGACzRQA\nwACaKQCAATRTAAADaKYAAAbQTAEADEQ0Nepkn8to+V38UbS4A3rNVGWGSMqmTWpNVm5flsV7PThu\nqLYliXaVD2T5gVKetZu5YIyYj5qpP7ZWmreKZdrz7+YqKoiefprF+fmz1JIpO+UpUGt7eTw2+fzz\nEQ/NUmNJCR0SpmHxByO/JPVWMR5V+qRe89xzLFqY3BJmdEYGDCB69FEWN44dq5akPPWUfKCiQq1J\nEKY4RuWh/PNaW5VlPm66Sa159w/vifllyybrJ9q4kWfagg3/Cz6ZAgAYQDMFADCAZgoAYADNFADA\nAJopAICBiO4f9wq00tzpNSwfmCcstHDe6tVyPnKkvqjGY9u28fD668OOz0JDA9EBYXbCbSV36EV7\nCsT4yHx+x77djJl8BsCqKmGBlU5w0u1Jy5v4nfudO/WazVPlBUCOf09/XbLq6uQDf/qT1/DMJGVl\n0UBhQeFsr8V+d+wQ40MlJXpNKJ1FjS3RmZpx6GR3GvXbn7J8+7p1as3mbvId8NlL9PM4P/wVyw6/\nF53FXIiIep7eS7P/yOcPLP21vjrQvSeUmSbKYu5ERHTNNTzD3XwAgOhBMwUAMIBmCgBgAM0UAMAA\nmikAgAE0UwAAAxHN3zhbH0vbi/k0qEOp+p46xzfxPXiI9ClTRETjiviiIqXHPPbHNtSjoYxuO3Ab\ny+cm/V6tWf7pejEf8FM+ZeWC22/nmTQnqxP08p2iuYlPsHzukh+qNc9/JE+BmlSkLxrx4YINypFJ\nnuOzcuB0bxq4ju8bNOEenrXT9o6fMS9NrWnL5ZnXdu6WBvZtpO1PClPqSrqpNUVFcv6Pf+jnueIK\nnjU1hRmcoQQiyhZyacumC/zfF+NdS3apJSN+3p+HoZDn2NrhkykAgAE0UwAAA2imAAAG0EwBAAyg\nmQIAGPC5rr5QAPthn6+CiD7rvOF46ue6bo/OPgmuMSq+CdeJazT0dbjOiJopAADI8Gc+AIABNFMA\nAANopgAABtBMAQAMRPRsfqB7d9fJzOQH/H61praObzVAROT//GP9RD34jbNgRQWFzp6V/2OGAgkJ\nriM8WF3Vg68X0C6NquQDiYlqTUlZV5Y1NQXp3LlQ519jly6u0707P9DQoBfFKP/fld4P551LSBbz\n/fv3hqJxFzg5OeCmpjos792rTS/SHjgvLdVrUvh6FcGzZynU2Nj5r2VcnOsk8C2AKjMvUWvSGz6X\nD1Qp72MiahP+XcqIKOS6nX6NRESB2FjXiY1ledvgIWqN9kh9z/QWteZEBW+J1dVBqqsL/3sZUTN1\nMjOpeOVKfmD4cLVm1x55r6cRC0boJxL27cn7938POz4LTlISFefns/z5X2xVayb5XpAP5AorYJx3\n1ezLWLZ/f3T21HG6d6fiKVP4Aa+FVrSVO+bPV0uOZ8sL4PTp44vKFJfUVIdmzSpm+b1z6vUirWmO\nH6/XXH01i/Jefjnc8Ew4CQlULLzPNszn191u8oF75QMbN6o1tYcPs8zjN9icExtLxcL/uOt369ep\nLaZ0R2GlWrP0Sb6f1xNPdOz3En/mAwAYQDMFADCAZgoAYCCi70ybE7tRWe4olvedqi8Q7C+SFwie\nmKkv0Dpa2G69ovWR8AM0UJM5kLYW8e9HJ8Vt14ve4t8nERHRG2+oJQUFa1jmtZ23qa5diYYOZXHz\nkofUkoTZM+QDzz2n1mx09EXDo6FnT6LZs3l+6//hN//aXXSRfEPDf8+nas2M/P08/K//Cjs+E337\nEq1YweLJ8+SFkYmIRnV5S8ynL16q1owdyzP3R9H5jp+IqHbAEHpzDf9+9KpL9RvDl64WFs0mog/L\n+fei7aTbBo2N4cdHhE+mAAAm0EwBAAygmQIAGEAzBQAwgGYKAGAAzRQAwEBEU6NOniRatoznY6dq\n+6MTjUk9IuYvDHpGrSn78f0sE2Z/dIqUri005tJTLK/38ylh7XY2ysdGLNAf572jlT/rtnZtBwZo\noDomnV7qchPLry+4Xi967z0xnnudPmXIiXRgxqqriTZt4vnKHgvVmprp/L1HRDR6tH6eGdcK6xPE\nx4cbno2KCvm5ybvuUktGlsj5oEH6aVIOvMmy2IbacKMz468so6vW3cbyQ1vk6U9ERCMe5T9PREQt\n+rP5G47xf5y8Zo91RL4En0wBAAygmQIAGEAzBQAwgGYKAGAAzRQAwEBEd/NzcogefZTnMatXqTXb\n4+QFMkoy5bumRES3HeN3DhOao3Tn8OBBcRGQ1fecUEu2bZPznwiLTLfbLawaHq17oymf7KUf/0yY\naXDypF40c6YYT5qklwzzvSPmd3oNzlBKCtE11/B8yjb9vTdcWbfl8ss9TrRuHc9On/YenBWfjyhO\n+DX2uDW/cIm8mMuqTGHBlvMuSxVW4Tl3LuzwzAQCRNOns3jxYr1kw+I58oESZToDEVF2Ns8++cR7\nbOfhkykAgAE0UwAAA2imAAAG0EwBAAygmQIAGEAzBQAwENHUKDpzhmJef5XnK1eqJc6f5alRo3bq\ni01MW82nrgRD/vDjszB0KNE7fEpP7QN6SWqqcqAlTa2J3+OyzDc1OnvqxHznO9T1hRf4gd279SJl\nmtewX4/Ta44ejXBktuLdZurVVMby1NS+as2Mil/LBz75m36iq4VpY4mJ4YZn48wZotdf5/m3vqWW\nbJgvT4EqvElfmOdkOX+/nvPre4aZi4khSkpisbTGywXKnmr379Pfswtrhd+B1tYwg/sCPpkCABhA\nMwUAMIBmCgBgAM0UAMAAmikAgAGf6/K7dOoP+3wVRPRZ5w3HUz/XdXt09klwjVHxTbhOXKOhr8N1\nRtRMAQBAhj/zAQAMoJkCABhAMwUAMIBmCgBgIKJn87t3D7i9ejksP35cr8nIkPM0/bF1ijtyiGXB\nxkYKNTfrDw8b6dYt4AYCDsvPntVrUlLkvNZjHxKp5tSpINXUhDr9GgOxsa4jbJvi+aLEyP/fdcvL\n1RJfD/kG6N6KilA07gIHMjJcJyeH5TX1+ts+5fOPxPxc7rfVmvhKvt1LsLKSQnV1nf5apqUF3D59\nHJa3tOg12j3nbj6PN2xVFYuCNTUUamjo9GskIsrICLg5OQ7LpR1bLqipkfO6OrXko+reLGtqClJL\nS/jfy4iaaa9eDq1YUcxyr31YCgvlvKBAr0mfMIplecX8vJ0hEHDovvv4uXbu1GtGj5bzv/9dr7n6\nap7dfXd0FjpxYmOpODOTH5gwQS8SFpkgImr8zW/0EuW/51u5MipTXJycHCreupXlW/f1VGvGLLhS\nzE++Iu9nRUTUa/1yluU98kgHRvjV9enj0Esv8ferx//j1EY7Km6XXrRpE4vy1q8PNzwzOTkObd/O\nr1NdZIiIYrbx156IiPbsUWuu3MIXYDp4sGO/l/gzHwDAAJopAIABNFMAAAMRfWeacu40jSlfy/Jt\nw6eoNdp3jbcVVOonCgR45vlNs5304F4q+Dn/rvmtmfqTYpOTXhLzgieuV2tiSj5k2dKkxg6M0EBa\nmvz96L59asn9I7eL+SK6Tq1x8+WbOV6LiZsKhYieeYbFYzy+UDy+Sf5u1N9FP82uvLksq03eEHZ4\nFpKSiAbmtrF84AH+HecF0vflREQbPWqku6lt/LydJa7yFKU/+wQ/cPCgWrM2//diXtI4Rq0JhXjm\ndTPvy/DJFADAAJopAIABNFMAAANopgAABtBMAQAMoJkCABiIaL7RB8czaMBiPg3qwAG9Zvp05cCy\nZWpNy4svsixaS1jXEpG0e/zKvfJjhkRElU/+Q8zTu3XTT7RiBc88nhk2FR8vTo9p2bFDLVmYO0PO\nm4TrOG/y1CuUIz/3HJ4ZbQpYdbVakhVoFvOaxES15iohSw43NisffED0rW/x/PRpvaa0VM5nz1ZL\nKlMHsKzlnXfDjc5OczPRZ8JTyB5TJqc89xMxXzPhNbXmyLV8mlve+qPhx0f4ZAoAYALNFADAAJop\nAIABNFMAAANopgAABiK6m3/uHNGxYzwfPFivOVKsLGiyZZA+qPx8lvk++CDc8Ez4Lr6cktYIC1HX\nKgvNElG6sJgGERF5LWgtLarstaKvJZ9PvAsa9y//otdIi8945US0Stme4FnPwdnZ/3Ei9R3J70KX\nbdmvF82ZI8YTrtbnk0glNXdGZ6Fv6t+faNUqnv/oR3qNsjL7wGPyYjZERIdW8Pd/XL2ykn0nqE3N\npl1jH2K5x9uPLsmXt8CYNvhuvWjkSJ5t3hxmdF/AJ1MAAANopgAABtBMAQAMoJkCABhAMwUAMIBm\nCgBgIKKpUdnZREVFPFcXMyEiWrBEzjMy1JJdD77FstoZ0ZlqEhenTLfYo6/mMi1J3u9nzezH9BNJ\n+y1VVIQZnY3qxF60OZcv6DDOYzEPuvlmOfeYm+L/05/kAx57TVm6+GKiv/xFOHAgqNacXCTvG7T1\nP/jeZxe3aZYMAAAAwElEQVR8yqeALWw6FWZ0Rg4fJho7lucei+yUKAvaHNrrsXBJUjbPEhLCjc6M\n/8h+GlHYlx9QpnkREdEbb8j5e+/pNdIiOK2t3oM7D59MAQAMoJkCABhAMwUAMIBmCgBgAM0UAMCA\nz3U7viGIz+erICJh74Co6Oe6bo/OPgmuMSq+CdeJazT0dbjOiJopAADI8Gc+AIABNFMAAANopgAA\nBtBMAQAMoJkCABhAMwUAMIBmCgBgAM0UAMAAmikAgIH/BkJomYa+y8y2AAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot_conv_weights(weights=weights_conv2, input_channel=0)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There are 16 input channels to the second convolutional layer, so we can make another 15 plots of filter-weights like this. We just make one more with the filter-weights for the second channel. " + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X18lOWZL/Df5J2QhAADJLzlMaAiokVNka0eZDlI68si\nh7KVKmWR4ssq63rUCouUKloKilQpsh6Wg9RSl7aRIgcrRcoqpQqcwaK8FBBheAsJmYQQkpCEwLN/\nSPz09LquZ5hyZ/bj8ff98/fk4rmfzMzFZOZ+7jvk+z6IiOjipPxXD4CI6P8HbKZERA6wmRIROcBm\nSkTkAJspEZEDbKZERA6wmRIROcBmSkTkAJspEZEDaYn8cG5u2O/c2RN5uLnMrPGPHVPz5gHXmTWZ\n506LLFpWhtiJE6H4o7w44Q4dfK9bN3mgqsouyspS408bCs2SPu3LRRY9cQKx+vq2v8Zw2PeKikR+\nIGqfOj9fzzs2yev4XKdOarx1+/aY7/tdAgfpQLhzZ9/r3VvkpxpSzZq6Oj0vPL3frGk5cUJkhwFU\n+X5yHkvPE/nZs3aNMlwAQEuLXVMQlgejhw8jVlXV5tcIAOFOnXyvRw954MwZs6Y+lKPmQS9l7Slb\nXh5FTU0s7nUm1Ew7d/YwY0ZE5BOjM8yaxmeeUfOyN+W/06q47mORlXz72xcwwovndeuGyIIF8sDS\npXbRlVeq8eitT5olKwY/J7KS+fPjDc8Jr6gIkQ8+EPn4SRlmzW236fmdB+V1fG7sWDUOFRUdDByg\nI17v3oi8957I10fyzJqNG/V8xo5vmTXVv/qVyIbFH54Tnudhyxb5WrL+UwCA0lI9r6y0a6bcWy2y\nkmHJukrA69EDkTfflAeOHDFrNmcOUfOgl/J3viOziRNL4ozuM/wzn4jIATZTIiIH2EyJiBxI6DPT\ncMVOTHz+CpE37t5t1mQZn/YWF9ifz5VFm0V2Jq3dBYzQgQMHgHHjRHwk4AMl/WsW4JH37M9Mb/nR\nEyL75NQv4w7PiYYGYNs2Ebe0DDJLfvc7Pb/zxgL7PMo5kmrPHuCmm0Q87O/+zix5FzPV/Np99mOz\nbKfMWr51YZ+zXayaGmD1apmPHHzcrJkIpQAADtrfY2DbGJmdll8Ut5mWFiAWE/HXpuqfiwLA+y9u\nUfPrXxponyccFlFOfX388YHvTImInGAzJSJygM2UiMgBNlMiIgfYTImIHGAzJSJyIKGpUWjXDrjq\nKhFnvfGGWbKrXJ841P+BB8ya7pFVIkuvr7mAATqQmqreiN5z9myzZP93v6vmQ26yb+cdni53hQ26\nN9qp9HSgQE5pev0RfSoJAFT31adNNeeMN2v++Z8TH5pTZ89+NnfoLwVM2Zr5on4P/vTpxWZNNJro\nwNzJbyzHyN3KLb2lO+yiCRPUuNZ4HgNAaclCkcWacuMNz51QCEiT7SroOTZzjf6cnXFkhV20cqXM\n/vEf440OAN+ZEhE5wWZKROQAmykRkQNspkREDrCZEhE5kNi3+SkpQI5cvXri3P5myZKhr+kHygNW\naNdWbw1aHtuh+ksGYPNSueDD9QfthS6K771XzXf927+ZNaNGyWzduvjjc6KsDHjqKRGPrllilqw4\nYiyCoqzy3uqfntJ/Z6+8EjQ4h7p00b+JNRbzBoCP6/Rv7b+Suc+seeutviJraoo/PCcqKoC5c0V8\nrtxe6CQlqs9YyLv5ZrNm4urRIlt48tMLGKAjlZXqE6d+8CKz5PHHjQOlAStna7OMrK0J/gLfmRIR\nOcBmSkTkAJspEZEDbKZERA6wmRIROcBmSkTkQEJTo1p6eqieK6fPLNm23i5arS8q8fooe6rRgAEy\nO70nOXvq7N0LaDNE3nnH3jf9+q/o+0P1/+gjs+ZFudUMbrkl7vDcyM4GBsp9cFb0DFgAolRO/wGA\nFWNeN0s2Lk54ZE7V5xZg801yr63cwfYCNFe/9JKa+7esMWtCt/1YSRvjjs+J4mLgX/9VxCkFXe2a\n0lI9D1o1pLBQZgF7vzlnLM4zRtmaqlX2aqPHKM/9VgufklPKKudcWO/hO1MiIgfYTImIHGAzJSJy\ngM2UiMgBNlMiIgdCvi+3zzB/OBSqBHCw7YYTqMj3/S5tfRJeY1J8Ga6T1+jQF+E6E2qmRESk45/5\nREQOsJkSETnAZkpE5ACbKRGRAwndmx/OzfW9sLypvDa9s1mTl6nv31BzOtOs0XYJqKuLoqkpZt9U\n7Ug4P9/3lHuAj56U27W0Sk3V80z7EtEx54zIoocPI1Zd/V92jYHbM3QxvsysC9gCQnmuAMDWrVtj\nyfgWOBwO+717eyJvDLht3nrMUmsCts3JyBBRtLwcsZMn2/6x7NzZ93r1kgd27jRraoq+ouZpAd2g\nrExmjY1RNDe3/WsSADp0CPtdu3oy//RDu+jSS/W8ocGuUV7M0aoqxE6dinudCTVTLxxG5OmnRb62\nYLxZM6Kvvt/Mqh36XjsAsHy5zH772+QsdOIVFCCySO4rM23NELOmQwfj3/Ls89z53+SzsyRJK51Y\n14iVK+2iSZP0fNMmu2bCBDUOpaYmZYpL794eNm6U+3kFrc/RV1/PBXkrjb3MAKBnTxGVaHtPtQGv\nVy9E1q6VB7TVgs5bNU/+TgDz/z4AwPTpMotEkvOaBICuXT289JIc963fbGcXKQvAAAC26YsvAQDy\n80VUMnNmvOEB4J/5REROsJkSETnAZkpE5EBCn5mitlbd3H1E+TK7RtmfHQBGzp1glgxYukFk27fH\nG5wjxp7yswI2ex//7GVqPuWKVfZ51sRkdvJkvNG5UVEBvPiiiD9+yl4cep2xNvLSpf3NmtvtreaT\nIqW5EdlH9oq833WXmzXZ772nH9A+NGylfQbXLuCzPJeqqoBl8vU3+ka5yHGrBcZHnX362Kc5veeQ\nyEpub447PFfS9m1F+Db5HVDAJ/YY/MEH+oGgz7OPHJFZ+/bBgzuP70yJiBxgMyUicoDNlIjIATZT\nIiIH2EyJiBxgMyUiciCxqVGFhcDUqSKesdyeHnObcd96x8Vy+lOry2JbRJbZUh9/fC5ccok61eSK\n/97dLPne9/R8c7eRZs1Vw2V2bsHCuMNzwpjiVv6AXfLo8I/VfMRj+n3eADBgij5l7kfBo3OmOSUL\nh7LktLXeWVl2UUyZsgbgF8/LqUGt7hwzTIb79duoXTvQ0A13RR4V+YoJ9rS8/Y368/L01l1mTWPR\nlSJL5rLyGVdfh6K18nZSa8kIAJg1W8+nTX/QLtKmwAUtWvBn+M6UiMgBNlMiIgfYTImIHGAzJSJy\ngM2UiMiBxL7NT09XF8Kd+Iy9CHX2M3r+H8vt7wLDNw8SWUvmhS02cLFiJ9OxZI385v6HP7RrNm7U\n86FD7ZrsZXJx5pSqyjijc6SwEHj4YRGPqLMXOqnuOVrNe58M+E538byEh+ZSRupZ9M6vlQcGDrSL\n3n1Xje+cbC+2jMWLZXbHHcGDc+SSk9vw+tsd5YFvvGTWZPQxXq8BC4BUH5WPc8styVscOr3sILp9\n/z55IGABmkikt37gmzeYNe36yNd+U1N63PEBfGdKROQEmykRkQNspkREDrCZEhE5wGZKROQAmykR\nkQOJTY06dUqdOuIFlKx6U586E/2TXVNXJ7Nz5wJH5kxeHjBcWYRE2U77cyXGDJHed/6NXVRTI7Pj\n9r49LtXlFGDD4CdEPuTwz80aa/qX59nn6fuAXIADAPDYYwGjc+fMuVRUnM4T+c31xt5AANbIdXwA\nAN2j75s1oRu+qqQZ8YbnhucB85QpaMoeX63OHTSmsy2Qz4lW3X/9ssjSTyTn+QoAaG7W92cKmBq1\nYmBf/cBRe6GbF16Q2Zw58Qb3Gb4zJSJygM2UiMgBNlMiIgfYTImIHGAzJSJyIOT7F775QCgUqgRw\nsO2GE6jI9/2ATQrc4DUmxZfhOnmNDn0RrjOhZkpERDr+mU9E5ACbKRGRA2ymREQOsJkSETmQ0L35\nKSlhPy3NE/mZM6fNmusG6vco/2lvqllzRd8zIosePoxYdbW9P4oj4XDY95Qbzlu2bjVr0nr1UvPa\nrK5mTV7NIZFFT51C7PTptr/G9u19T1tsoKzMLuout3MAgKPnCs2SHllVar41Go0l41vgcH6+7xXK\n8R2vt7fAOW08lYsyy+0TheRDFq2uRqy+vs0fy/z8sF9Q4Ik8p/KAWVPb+RI1r6+3z1PYKP+9aH09\nYo2NbX6NABBOT/e9zEx5IMV+P3i2z2Vq3rLNfi1nFhWJLFpVhdipU3GvM6FmmpbmIRyOiPzYsY/M\nmsh7+gM3aLhcgKLVlpXyRV1yyy0XMMKL53ketmyR11idav8uw1OmqPnaSx8ya0asfFBkJaWlFzDC\ni+fl5yPyoDz/uYBFI1KUnweAafVPmjWz+r2m5qF/+IekTHHxCgsR+elPRT5/k9xjrNWOHXq+qO9z\n9onS5Muo5Mc/jjs+FwoKPCxaJJ+vQ165y6xZO+F1Nd+0yT7PjH3jRVby1lvxB+iIl5mJyFVXyQO5\nuWZNbelaNY91sF/LxTNmiKxk5sz4AwT/zCcicoLNlIjIATZTIiIHEvrM9OrUnYh0uELkj461V3rO\n66nnN9hbV2PWUvllx7HYhe1dfdFqa5GyTn7WEj4Y8DHf9u1q/Otf2yUj9uyWYWNjvNG50dQEfPqp\niFua7LvhMsaOVvNZA+RnTJ9b/G6iI3PrwAFg3DgRP2ytdA3oK5MDWLXDXjh55A+ukWGV/uWbazn+\nKQxpWS8PfOMbZs2IZ4eoef/f/94+0XvvyWzLlnjDcyczE7j0Upk/+6xZEo3q+YCg84waJbP584Mq\nPsd3pkREDrCZEhE5wGZKROQAmykRkQNspkREDrCZEhE5kNDUKIRCQLqcojRvsX1rqPdsrZoPHWqf\nRrutTbljr00cPpmHh1ePEPn8bfbthBN369Nm5s61z7O/Rk5nabqjJP4AXejZUx1cxkZlis15r41a\noebjV3/LPk9BQcJDc6n+kgHYvFTeann9MmWf+fO2PfaYmo/8+783a1Zu2yaymgsYnwsNqbn4MH+Y\nyI/oM7wAACMXDFTznosX20XLlsksSdO/AKC+6yXY/JC8PbnKuP0XALKy9HzM/7CnAM5VHrimsxfW\nfPjOlIjIATZTIiIH2EyJiBxgMyUicoDNlIjIgcS+Iy8oALSFkLUFCM57eLr8ZhwAdg3XF24FgPs8\neWxRpj4rwLVehS2YP/24PLDM/lUteaVZzc+l6bsMAECnjatElnk6Sd8B79wJ9Osn83XrzJLx66bp\nedYvzZrHHzcO/Copi7OjfcV+XP+CMttA2Umh1cCTJ/UD1lfDAEaNHSuyZ5+wF0ZxqapK/6I9YJ1v\nrNp4tZqXD7AX9HjlFZntrv8w3vCcOXMGqKiQed++do21btCYMXaNts6JsiaQiu9MiYgcYDMlInKA\nzZSIyAE2UyIiB9hMiYgcYDMlInIgsalRJ08C2l7ZQauWLF2qxtsDtpvpv0dZ6aS+PnBozhw4AEyY\nIHNjbyAA6PTso2q+fLl9mt3RkSI73nxh+3NftE6dgLvvTqzG2LTrlb+1S4KuPxnO9CpGxU/k1K2A\nmXyYYEynefFFu2b2brk/1rHTs+INz4m0NCAclvluZYuxVitX6vmSqXvNmvuWPyCykjT7513L+nQr\n+t4hp9T137PHrGm57nI1v/aNN8yau848KbISPxp/gOA7UyIiJ9hMiYgcYDMlInKAzZSIyAE2UyIi\nB0K+by/hL344FKoEcLDthhOoyPf9Lm19El5jUnwZrpPX6NAX4ToTaqZERKTjn/lERA6wmRIROcBm\nSkTkAJspEZEDbKZERA4ktNBJfijkFyp5+3btzJrG06fV/NwV15k12U0nRBatrESstrbNNw8Kt2vn\ne3l5Ij8d7mXWHDqk56GA0RYVyezo0Siqq2Ntf40dO/pejx4i37oz3azp0ydVzfMrP7FPlJurxluP\nHo0lY0pNWlrYz8jwRJ6TY9dY69n0zzEeZADbT/YWWUtLFGfPtv1j2aFD2O/a1ZN5yim7qKVFz5ua\nzJK9tQUia2yMorm57a8RAMK5ub6nregSsADSqY7ycQGAVP2pDAA4d05m5eVR1NTEv86EmmkhgFeV\nfHDArla7tm9X88ZlEbPm2n1ypZ+Sf/mXeMNzwsvLQ+TOO0X+8SR7s7HJk/U8YA82dYOyO+4oiTc8\nJ7wePRBZsULkocvlC6bVvHnyPxgAGPnKrfaJjNXEQlOmJGW+YEaGh7595fPMWAALALB5s55HBj9o\n1hSvWSiyo0eT81h27erhpZfkNd6atd4uisX0PBo1S4atkRsERiLJuUYA8MJhRJ5+Wh7YpKwwd976\nMfJxAYD8fPs82iZ8Eyde2HXyz3wiIgfYTImIHGAzJSJyIKHPTNunpGBw+/bywJgxZk3/G2/UD2Tt\nMmtWZX1LZDWh5+KOz4Xq9r3w+mD5+ejq2XbNhoEPq/n6UfbnrMWl8noyT5THH6AD1Q1ZeD1ymciv\nucauGZm/Qc0b3n7brMk2dlnAlClBw3Omf3EjIqXKavDWh9wAQn9cpuYNG/XP3wBg/6QPRVYyriH+\nAB3o0HQctx54WR746lftoqee0vMFC8ySdY/LbNCg4LE5deYMUC5fH++Psx+X/2vs5nH0qH2a+btH\niCyn7MJ2FOA7UyIiB9hMiYgcYDMlInKAzZSIyAE2UyIiB9hMiYgcSGhq1Mep16B7jrx1rWzcfrvI\nmG5xKKe/WTJygZyeMPP4hU1PuFidju3EXc9cIfK7ugTcSj50nBoPK7VvQcT06TL7+c/jDc+JpiZg\n3z6Zf5hmz3VpKNmi5nOftndqmDEh4FbTJGjZuROxyy8XuXEzJQBgz56uar5tm13ztRblhn7tJu+2\ncvaszILuZTZu88XvjblEAFLuuEOGZWXB43Lok1MFuOU/5C2thbvtmiWzj6t5Q47+GANAxam1Ijsz\ngreTEhElDZspEZEDbKZERA6wmRIROcBmSkTkQELf5l/dsxqRZ+Q3zpsr7zZrrjcWleg9V18cBACW\njJXfqMX2JmchWr//lWj+QM5YyIgGzCawFohQFmZoNXNxd5Edi9kr3bvUqRMwTpmAUD1Z/8YeADrN\nnanmY8bMsE9kr9ubFE0AtHkmg5bpi5kAACbLmSQAkPbOO2bJrp1yRkNjWsBy/g6dC3dFwyT5Wpo6\n1a558smr1bzbQfvx/8XsAyI7MS15i0NfGj6Bt++Ri8bjBz+wi9boC8pn33+/XfJzuTNIwGL+/w++\nMyUicoDNlIjIATZTIiIH2EyJiBxgMyUicoDNlIjIgYSmRuHkSeCtt0R8fc57ZknFM4vUvFvAtKE1\na2RWWxt/eC6EKsqR8aKy31RpqVlTtlKfUtL9gZFmzYypzSJb9aa9aIhLmbs/QvENhSLfvPKYWXP9\nkSNq3r+g2qyZUfIb/cDboeABOlLV5Tq8+k05zW1QxTy76Pbb1bjY2s8KQK0yCyo9ObPccPgw8Mgj\nMl/0gNyXqtWKP1yr5uXl9kI3MWV1GG2P+TZz7Bjwox/JXFsw6Lw5R/QpmxOi482a0S9NE9ms+oBN\no/4M35kSETnAZkpE5ACbKRGRA2ymREQOsJkSETkQ8v0L/wY5FApVAjjYdsMJVOT7fsDeIW7wGpPi\ny3CdvEaHvgjXmVAzJSIiHf/MJyJygM2UiMgBNlMiIgfYTImIHEjo3vxwKOT3VvKUoiKzJloXVnPv\nzCf2iVpa5L/T1IRYS0ub39Qd7tjR97rLLUWwN2Dbkt7abwVAXZ1ZEj3bS/nxKBobY21/jR06+F63\nbvKAdgN2q+JiNQ64ROTU6Pfzb62oiCXjW+BwaqrvKTfJV3QZYNZ0K/9IP3D2rH0i5XcZPXkSsYaG\nNn8s09LCfkaGJ/L+Xe3HsjpFf03mHNhq1mRkZ4ssWa9JAMjICPtZWZ7Ic3PtmsJ0/Xdw7Ix+/QBQ\nKJesQDQaRSwW/3WZUDPtDWCjkmfPsPcBmrhxopovKb/VPpHyoi7ZuTPO6NzwundH5N//XR74+tft\nIm0BBgDYZG+CNLFGLraxalVy9tTxunVDRNu3avFiu2j5cjXesNH+42bI6ifUPPT880mZ4uKlpyOi\n/Ec37wG5+EmrR2d31Q/U1NgnuuceEZW8+mrc8bmQkeGhXz95PZHJS8ya17P01+SNd9v9oveVV4os\nWa9JAMjK8jB4sLzOoUPtmmkF+u9g5hH9+gFgxvRzIisZZC8A8+f4Zz4RkQNspkREDrCZEhE5kNBn\npint2yP7qqtEHvQZxJLbV+gHNvazT/TUUzK76aY4o3Okvh6IyM9mGj61F07Onmxc/x//aNbc+E8y\nW78+7ujaVsCC3dixQ42HTJ9slqx4ZINx5PkEBvXXq+k5AKvmysfygeF2zazG42o+bdQus+YX2/uL\n7ET22vgDdKB/z1pEZstzzYrYr0lrUecbDwbcDfl/XpbZnDnxhufMZcUtWLtcLkT+m02d7KIB+gM9\no9H+MvnhRy4T2eHD8ccH8J0pEZETbKZERA6wmRIROcBmSkTkAJspEZEDbKZERA4kNDUKXbsCk+VU\nmBk1yrSJ86ZFHlLzWaPs+2PV2zDr6+MOzwnfV9cG2LbNLsl5RL9t7eqvKzf6njdx5UiRLazZF398\nLnzyiXp77P5P7akxxQse1Q9MmWLWjP7bhoSH5lJ2NjBwoMxLS+2aceP0fORkOf2p1aq00SJ7vvrT\neMNzo7ER2L1bxNOW2lPW0Lmzno/5X2bJonT5Oq4MJeeWWQBAbS3w9tsivrWiwq4ZbvwO0uy2N2GC\nzNatizO28/jOlIjIATZTIiIH2EyJiBxgMyUicoDNlIjIgYS+za882wkLT94t8gcr7MWhZ0XvUvP1\nLa+bNbuVL7Urm/LiD9CFrCygb18RFxTYJcWLp6l5S8DCIWnvvCPDpqa4w3MiLw+44QYRe15AzQB9\ndfr17W4zSwYaC2okS0bNcfReOV/kG3c8bNaMz1+l5qvuTzVrZmyWi/mUbUrOQt9ISQFycmT+7LN2\njbXSSYD7JslFkxctSvif+atV+Z3w2lnZe4aPtWus+UJjRtk1qw5fI7Ls6J44o/sM35kSETnAZkpE\n5ACbKRGRA2ymREQOsJkSETnAZkpE5EBCU6O6dD6HByfIxSvOZc00a5Yt0/PxT/6NWfM1ZaGT/x1/\neE7UhXLxftYwke/baNcUK4u/AEDa1Kl2kbaYi/HvOHfppcDq1SJO6dzRLFnywgk1nxh5zj7Pk79O\neGhOpaerc9qWPm6XLAorjwuA1/rNMmvGjJHZG2/EHZ0b+fnAKGWuj7LPfatmY8peRtD8v3vvlVlZ\nWbzROdMZVRiP1+SBgEVIFtaNV/NVU9+3i/r+VmYjRsQZ3Wf4zpSIyAE2UyIiB9hMiYgcYDMlInKA\nzZSIyIGQ79tbVYgfDoUqARxsu+EEKvJ9v0tbn4TXmBRfhuvkNTr0RbjOhJopERHp+Gc+EZEDbKZE\nRA6wmRIROcBmSkTkQEL35qenh/3MTE/k/TIP2EUdjfu9G+Q9/q0q07uLrKoqirq6WCjeGC9W+/Zh\nPz/fE3mHDnZNerqehz7eatak5cltWKKnTyPW3Nzm19iuXdjPy/NE3quwxS6y7sPu1MmuqaxU463V\n1bFkfAsczs/3PeV+8yM1yjYf51mPZbcU/VoAoDFXXsrRo1GcONH2z9dOncJ+r16eyKuq7BrzFvya\nGrtI2eokeuIEYvX1bX6NAJCfH/a7d/dEnh3dZRf16aPGDWczzZLsGvk8j9bUINbQEPc6E2qmmZke\nrr46IvL3PX2fJwD6KhAAsG2bWbKwQC6cMmdOcvbUyc/3cP/98hpvv92u6dFDz1ML7N9/WNmDqeQP\nf4g7Phfy8jzcfbe8xnnTq+2i6dP1fGzAJjyLF6tx6Gc/S8oUF6+gAJElS0T+xMqvmTVWo3k0x97w\naO/Q+0Q2enRynq+9enl4+235WP7sZ3bNlO/J/ZwAqIvffG73bhGVzJf7a7WV7t09LFsmr/Pa78o9\nmz5nrDbzYU2xWXLtSrmfXYnxPP5L/DOfiMgBNlMiIgfYTImIHEjoM9N+/p/wfssg5YjcZ77V/oGj\n1bzY2IcdAAYoa9e2axdvdG4U4hhmpP5Q5BU9njRrur0lP5cDANTX2yd69VWZ7dgRb3hO5OYCQ4fK\nfG3E/jJp9u6Fat4z4OOk1wboCy0nzZEjwGOPifi5oG9nPvlEzwNWe74Me0WWhcT3pv9rnDgBlJbK\nfMp3AhZufnyuGi8ZMM+uCY8UUSztl/GG50xmJtBXaTPrX/ijWTMsul7Nr007Yp9IO0mm/YXVn+M7\nUyIiB9hMiYgcYDMlInKAzZSIyAE2UyIiB9hMiYgcSGhqVGX4CiycsEXkDw61749dvlzPpzUuM2sG\nPCJvJ83Kij8+F+rKyrBBuXVySH6+WXPE2O++Z8Ats+o0qKB7ox3Kz2nByMHHRf6bSFezZv2Ry9R8\nUJ2cFtQqZdkTxpEpgeNzJi8PuPlmmXfrZtdcY9yeqNz++7l77pGZsTe9a6EQkKa8ip94Ua5v0Sot\nS58CNThsn2fku4+KbOHpw3HH50rq2Wbk1RwS+bDv3mQXffvbel5XZ9dMmCCzC5yXyXemREQOsJkS\nETnAZkpE5ACbKRGRA2ymREQOJPRt/qFDZ/HQQ7Ui3/Sd/maN5xkHBg40azqNHSGytP32t8YupQPo\nqR2oqDBr9hv5tp/8xKy5/fvfl6GyAG+baGkBYjER33qjPWWiNqL//jd1sBfATtG+SQcQeifO+Fxp\nbFR/p7vGytkirfpDn5ny6P+0t0S/8UaZ1WxI0mLmh7Zi9EPyMSj43vfsonL52APA/gHGgj0A5kfl\nDIDjmRviD9CVaBSYNEnmc+bYNcbsmIo75GLerbrlKjuAXOBUIr4zJSJygM2UiMgBNlMiIgfYTImI\nHGAzJSJygM2UiMiBhKZGdeyYiuHD80T+0EN2zfUVq/QD9wcUvfyyzPZbE5Dcarr0OuxbIPfnjgb8\nptY06lNtnnvE3odnyRq5EEUs4zfxB+jCuXOfTRv6C8UD5WPbat06Pc/73e/s87z7rp6/k5y5UZW5\nxVg0XO48Vc1GAAAA2ElEQVRTdF+pPTWqbJLcNx0A5k342D7R1KkimlVp7CXlmA+gRTvQU53gBwAY\nsuk5Nd+QJhcSafXw7tkie63R/nnXarpehlWT14p85J0Bi5D89KdqnFtgT+fDwYMya1F/wwLfmRIR\nOcBmSkTkAJspEZEDbKZERA6wmRIRORDyfXsBB/HDoVAlAOXrrqQo8n2/S1ufhNeYFF+G6+Q1OvRF\nuM6EmikREen4Zz4RkQNspkREDrCZEhE5wGZKROQAmykRkQNspkREDrCZEhE5wGZKROQAmykRkQP/\nCbwTyk8Td2ZOAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot_conv_weights(weights=weights_conv2, input_channel=1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It can be difficult to understand and keep track of how these filters are applied because of the high dimensionality.\n", + "\n", + "Applying these convolutional filters to the images that were ouput from the first conv-layer gives the following images.\n", + "\n", + "Note that these are down-sampled to 14 x 14 pixels which is half the resolution of the original input images, because the first convolutional layer was followed by a max-pooling layer with stride 2. Max-pooling is also done after the second convolutional layer, but we retrieve these images before that has been applied." + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXucVOV5x78zy+4isCBxRRBhF1FRVECokaj1lsR7tN4v\nDSYxabw0GquxBq3Jx1xsY2tMW01T0jZ+1IZ6i0mMwRtRUFCiRCPiHV0w2ACLNwwU2N3pH/v5nfc9\nhzOzMztnzpzV5/vPzp45M3Pe857znuf+5AqFAoZhGEZ15Ot9AIZhGB8GbDE1DMNIAFtMDcMwEsAW\nU8MwjASwxdQwDCMBbDE1DMNIAFtMDcMwEsAWU8MwjASwxdQwDCMBBlWyc2tra6G9vb2sfeMyq3K5\nXCU/F6Kjo4POzs7+f0GZVDLGpPkojBFg6dKlnYVCYcda/47G6V+L3d3dAPT09ADummxoaAj2yeer\nlzGyNJelshwHwj0JbpyaN3BzGJ2vuPFqn/6Mt9xxVrSYtre3s2TJktCFV4wtW7a4HxnU+zOlLtIP\nPvgAgK1btwIwZMiQ4L3m5mb+7M/+rJJD7Tft7e08/fTToW2dnZ0AtLa21ux3u7u7OeCAA2r2/T5x\nY/y///s/wF1szc3NRT+vi1ULE7i5jc5xV1dX8FrXQS6XW9nfY68EjXPz5s3BtieeeAKABx98EIDX\nXnsNCM/tuHHjANh+++0BaGxsDN4bOXIkAJMnTwZg0qRJwLbjTut6HT9+PAsWLOCdd94JtrW0tABu\nsVm3bh0Ab7/9drCP7k/N8/Dhw4P3tG3jxo2h/z/2sY8F+wwfPpyZM2cmO5gStLW1sWjRomAsPrp2\nn3zySQCWLVsWvKdj11xq/gB23nnn0HvbbbcdEL4WWltb+cQnPlHWMZqabxiGkQC2mBqGYSRARWp+\nV1cX69evD1RxgLFjx8bu29TUVNGBSGVcvXo1ALvsskvwXimVMw3eeOMNADZs2ADAhAkTEv+NamxX\nSTB48OCy99WxrlmzJtgmNVNqr8w27777brCPr2KliX/9HHbYYYBT7f75n/8ZgOeffz7Yp6OjA4Ch\nQ4cC4XGuXbsWgD322AOACy+8EICjjz66BkfeN1u3bmXNmjW8/PLLwTaZyEaPHh3631fTn3nmGSD+\nftPY3n//fYDATCJ1Gnrv7zQrzuVyOZqbm0PHGUXj800emvsddtgBCNvG33vvPQCeffbZ0P8HHnhg\nsM/BBx8cstOWwiRTwzCMBKhYMn377bcDoz3AH//4RwDGjBkDwIgRIwD3VC8XfU5PP0kOWUCG++XL\nlwPOIbX//vsX/YwvvfsOjGIk4UGuBh1vOccqZPAH55TaccdeJ70cPb4jce+99676OJNi2rRpAPzk\nJz/Z5j2dC0kxDz30UPDe1VdfDcC9994LOM1s1113DfaRZJcGDQ0NjBgxInS/SJKUJjBq1KhtPlft\nMW7evLnu2lQUSaYXXXRRWfsvWLAAgGuvvRaA3//+9wBceumlwT577713yIlaCpNMDcMwEqAiybSx\nsZHRo0cHkhnA4sWLAReGMWPGDKDX1tAfdtppp359rpbstttuAKxfvx6AJUuWAITsVJK6JKnfdddd\nwXsHHXQQAOeee27tD7YCXnnlleC1pJeoRiA7EjjtQcjWBHDGGWcAzvY9f/58AD71qU8F+/jhN1km\nKp0fddRRwWvZTG+++WbASa++PXHz5s2p2RNzuRyDBg0KzZvuzzlz5gBO4tL1C85Gqs99/OMfD947\n9thjgbCNNUpzc3OqkmmhUGDLli3cfffdwbZbb70VgMMPPxyAyy67DAjbvzVPksTPP//84D1dz3vu\nuSfg7Me+5v3222+HQgBLYZKpYRhGAlQkmTY0NLD99tsH0ic46eSll14C4KmnngLcUxHcSi+7km9f\n0lNBNlc9ZbKEnt56mvv2UKHx33777QD84he/CN576623ADe2WkQDVII8wJKyoLgNLSqNAtx///2A\nC+oG2GeffQCnqUh69yVTBZOnSaFQSFSCUjC+pPq2tjbAec6h19OdltQmm2ncPCnY/Nvf/jYQviYf\nffRRwNm6/Xty4cKFAJx55pmAsy8r6QLS1zI2bdrEsmXLgigEgIkTJwKw1157AS4Kw09OuOqqqwDn\nzffRtq997WuAi8h4+OGHg33effddk0wNwzDSxBZTwzCMBKhIzRfKYQU48sgjgd48aIDHH38cgPvu\nuy/YR3nACpvxw1H+8Ic/AHDTTTcB2VHz/XAIORnGjx8PuHAY3+gvZ5RMASeeeGLw3sknnwyUNuh3\ndXWl5rTI5/M0NzcHjrVK+dGPfgQ4FcpHZh+FRBVL6kiDQqFAd3d3SD2tFoUbSUWM5ndD/RMwhMZ9\nzTXXhP6CC9pXHYJSdSdkEth9992DbWmr+U1NTbS3twcqOcSHfEHYZBHlT3/6U/BaoX2nnXYa4BzN\nSjiB3uu43LBFk0wNwzASoOpHthwXkkynTJkChKvmKGxIEurcuXOD92TcVVpeFigUCqGn0bBhwwAn\nkcelt8oBocpPfnqh0tOiToJoZa00nRb9SYpQWIrCpeKSFuQEOOGEEwDnJKgX3d3doapRCnuStlFO\nBTQfOVgl9UlbqTR9ut7ouOMcM0LznAUtY9CgQSWPtS+0ztxyyy3BtgsuuCC0jyRSXwIfP3582XNr\nkqlhGEYCJGZMUtD+ihUrgHD4j+xKCs3wg4dnz56d1CEkRi6XC0mJklKjtjdfslQxDCUd+E+3YrbS\nuN/IGn7RiL/9278F4Ac/+ME2+82bNw9wEsAxxxwD1K+4CbjiGHGSRSVagILewUnekpIU0pclZHtX\nkRnZR/1QNj+8sRiqHaqwt3rT09MTmjeNUwkTfupyFKUB+wkYUTS3Wq+gd57Ltbln8w42DMMYYNhi\nahiGkQBVq/kStZXlJKO1HFHgVEWpS374U6kwhqzjq38at9R7hZzEobCrJEN2asUXv/jF4LXUvc98\n5jNAb1aKUNaIajLUO8urGJWo9wrbW7VqVbBNzkedizinSJp1PuOQw+31118HnGNFjlQozzwh55wf\nCpkldB+VqsX7ne98B3Bzeeihh26zj1+vFsL1QSqpQWCSqWEYRgJULRqp+ryC1pUn6z+xVTdQOe0K\nkh2oqDKUEhQA9t13X6CyUKCsBHfHodz6e+65J9imqjrCT75Q0LdC4iqp3J81NK9+VSyhUECFFmUR\n5aavXLky9NevmFSKBx54AIAjjjgCyO51WixkSeFrAL/5zW8AQsH+UeR4UrigH8JYydhNMjUMw0iA\nqiVTSWlawVUZyK/1KZvErFmzgMqquWeRN998EwhXYJeE7ldJiiIb40CQ2r73ve8BcPrppwfbFDKi\nepF+3cjjjz8ecCnDAxlJKKqA5afdKk04mvRQbzupj45F16QSakrZPv1KaLIHZ2lMlaDwTIDrr78e\ngP3222+b/aR9yXchW2l/1yeTTA3DMBKgaslUgbJ6+r344otAuEK7iqHUO7UwafyiCb/73e8AV+cy\nrj6oJNKs2qDA2QnlAX7wwQe32Uc2KT+dVF78LKZVVnq+JZlpDlVzF4onIWRpTuW1V0TF1KlT+/yM\nL43VM220FMXO8bJlywC48sorAZcwAttKpH7CkLoOqxOGtI24xIByMMnUMAwjAWwxNQzDSICq1XzV\n79RfVRZSGw+AmTNnVvszmUJhQH4TMuUHS+XwW13LKVePth3lojxuJSJ86UtfKrqvWnX4udADpVme\nageUUy1K161fAc0PfM8qCu055JBDiu6jJBu1KFEYFDiTXdaRSfGKK64AXC2BUhXofLVdc6nzFXdN\nWGiUYRhGyuQqMbDmcrl1wMraHU5J2gqFQs3jbmyMqfBRGKeNMUEGwjgrWkwNwzCMeEzNNwzDSABb\nTA3DMBLAFlPDMIwEsMXUMAwjASqKM21tbS3UKwato6ODzs7Omufs9XeMcuRVk1aYxTHGOSirTZ1c\nunRpZxpe4GrnUvRnvFmcy6RJa4zgxunPTfSeq1VKb7njrGgxbW9v5+mnn+7/UVWBHzhdS9ra2li8\neHGotoCCeqN5536lHXUTUGM8P6Bdr9WAT9/tV48aNmxYbOvkWtDfeVRlc1Vy9xsK6j1d4KpI749R\n23K5XCohLmlcr6odqoBxgFGjRoWC4GtJe3s7S5YsqbhldRKkdU9C73355JNPBhXzwZ171RLQvec3\ngdS1qtoD/vWoXHwlnOh+9s/loEGDyh6nqfmGYRgJYIupYRhGAmSuo5typ5XrDr3qdVrJBblcjkGD\nBgXqAbiizlE1//333w9eKxdfakSc2qXPS1XxTQF+AeJaUygU6Orqqrihn/bXX7/+wEcV2elUehF6\nVUqZPdKgmIqvEnOvvvoq4GpKgFNx4+pGqKi06jSoFVE9S/P19PSwceNG/vd//zfYpnH7DfAgXP5T\n6r1UeH9dUWslmQc0bv8+r6SehkmmhmEYCZC4ZKrqQ9G2DsVQ6wBJf2pBO3r06GCfwYMHp1Z8d+vW\nraxZsyYo9gyuOpQcUb/61a+AcHFoHffatWuBsEf4lFNOAZxE+tvf/haA6dOnB/uk6UDo7u7m3Xff\nDVX2kkagtiOqAjZQqkHVC0lvfguX0aNHpyqZgps/IHC6zZs3D3BOmI0bNwb7qA207ldf6nzhhRcA\nJ3X/zd/8DeBafEN8i+taIslUUiRQ1GFbSsvzz1P0nlNbF18ytapRhmEYKdMvydSXupYuXQq42oKS\naGRngbCUGeVjH/sY4BrzaV+/3WqaFAoFNm/eHLR88I9JjBs3Dgi3PlaYhp7qfrsEPU1vueUWwLWf\nnTFjRtKHXxb5fJ7m5uaQ9HT//fcDTmpWMznfdixtQ22ODz300OA91ZBUrdNSzdvSoqenhw0bNnDH\nHXcE29RyRRK4bGh+Ows1zdPfzs7O4D1pLJLydL4k4UHvOUvLxt/V1cW6detYtGhRsO3hhx8GnFYx\nbdo0AN56661gHx2vWpT7SIP6i7/4C8BJpP61sHXr1lQb7uXzeYYMGcIBBxxQ1feU0gCr9QGYZGoY\nhpEAFUmm3d3dvPPOO0GVbnAtnSWZ6Qkv2yHASSedBMQ3W1uwYAHggr7//M//vJJDSpympqY+K42r\nQVlcozI1lvM9p0I2Snn849rPpkE+n6elpYXDDjss2KbXsl3Pnz8fgOeeey7YR8HQeoJLqwD49a9/\nHfoN2a1OOOGEZA++AjZt2sQLL7wQaE8ATz75JOCuU2kdvj1QEqnw7aGS5CTlSRPxtZfW1taKIyX6\nSy6Xo6mpKZA+wVXNr8Te7SegyA5cKljdT9hIg4aGhn51qpDmoMiGM844o8/P+HbVSjDJ1DAMIwFs\nMTUMw0iAinSRXC5Hc3NzKPRAzhgZ6RUu5O8TVe+//e1vB6/nzp0LwNlnn13JoWSWOPVeKMj/s5/9\nLJBuOFS5yHF0/PHHh/72hcJxpBLHmUp8B0YaNDc3M2HCBD7/+c8H2yZNmgS4kDyN9xvf+MY2n//O\nd74DwC9/+ctg28UXXwy4XHyp0n5o0ZgxY0J96GtJQ0MDI0aMqNphe/PNNwevZ82aFbuP73DyQ5TS\nIhqmJOdhR0cH4MxoDz74YLDPv/3bvwEE10Ccmi9nq+5d/77caaed6OnpKev4TDI1DMNIgIokU4Un\n+GmQolT4k1i5srdY0BNPPLHN54477rhKDiUTyLgNzhEX91RX2JCM/HFOma6urlRDTZJGoUZyysWF\n3KQlrYlBgwYxatQoRo0aFWyTU+WrX/1qn5+XU85PrpBDTnOpUMDJkycH+4wYMSKTWkcccrwp6QTg\nr/7qr2L39R0zuVwutUSaYkjj1bH76d1C9+NPfvKTot8jLUqf90MGm5ubyx6nSaaGYRgJkGqhk//4\nj/8AwuE255xzDlC/MKFKkDSi0LDHHnsseO+yyy4L7avwEoAf//jHANx0001Ab+3SKFl40vtUWuxa\nNvOdd9459Dk/5CZtyRR6x+GPoRJbn8L9/DCnYuFh0WIbWaDUHF533XWAs3X7BUSiKIzMTxHPguSt\nudR1FZfKrUSZOCSByv6tQie+v6exsdEkU8MwjDSxxdQwDCMBUlHzFaqgEBPfoH/eeeelcQhVodAI\n1UlUdpDCwuJQ+BPAqaeeCsCXv/zl0D5+qJBaemQFjbmUOudXndLxS92VqlUP1b5aFi5cCDhV/vbb\nbw/e05zJObXnnnsC5VdJSwPNXdSk8cwzzwSvFQp21VVX9fl9cjhnQbX3kUNsxYoVgMvCO/zww8v6\nvOZS49Ic9rcilkmmhmEYCVAzydQPL1Dws/K+/fAhVRkaCMippDCor3zlK9vsI2eTn8/9T//0T7Hf\nF1eroN5o3srJLf/ggw+C17vuuivgpKFKvicL+BXAVBVLNRT8PHRJq9JKFBrlU+8Qt6jDRBLYt771\nrW22XX311UW/Rw64elVw6wuNQfUR5Ow8/fTTy/q8HFZyPO2xxx5A/7Upk0wNwzASoGZiwzXXXBO8\nVgiRbIeqIjVQ0JNr2bJlAKFqS1FuuOEGAP76r/862HbUUUeF9tETNUt2Uo0xWlc2rlLPqlWrABcG\nBS5ESGPLotQdhypKzZ49O9h21llnAU7K9sc5cuRIwNXrzWIfLEmmSrecM2cOAA888ECwzz333FP0\n85pDSaRZs5UKpa5rnBdddFGfn/E1RoVEKeGkWgncJFPDMIwESFwylefT94DKe68UvoFiRxPRPjnq\n6eSjDpDyCMbZU0WWJFIhO5+e1nqC+2mGkl7ffPNNIFxpX2RtbFH7paQ2BXN/7nOfA8JB9/LQy57v\ne+r33ntvgFCKatzv1BPN00MPPQS4Gq5KGgFXRT8OXQP+3GcRaQ6qITxx4sQ+P+OvPSpyIxt5teuS\nSaaGYRgJYIupYRhGAlStbyvcSRWUFKZw7LHHBvvIYSNxfKChmpWnnXZa0X1kDFdbXKmKPgrdyGIg\nu8Z44IEHAi7wW5W+wIWGySHhJx0oTCqu7kC98VVwNcS76667AFcxyHeKKsxLDQ/9UCOZejSHWVLv\nheZFc3rFFVcA8W124pB6X4+apZWg8ZUTZK/reM2aNcE2Nc3sTzuUOLJ9tgzDMAYIuUqerLlcbh2w\nss8da0NboVDYsdY/YmNMhY/COG2MCTIQxlnRYmoYhmHEY2q+YRhGAthiahiGkQC2mBqGYSSALaaG\nYRgJUFGcaWtrayGuH3oadHR00NnZWfMmSRqj75iL9tKpVa+mLIwx+r8/1qTGv3Tp0s40vMBx44zy\nYZlLv7e7iiZH+737caOKFa5mTtMaI5R3zYq4sVQzz+WOs6LFtL29PWjAlTZq0Vtrxo8fzyOPPBLk\nJ4OrZ6nqMqqQ5Ne5VOUaXbD9CQROa4xjx47lvvvuC5rDgWuopqpR+uu391UlKP1VTjO42p8KpFY+\nu84ZuKo8uVwulRCXtrY2Fi1aFBqDFpFSNQQU9L5x40YgfCNq7Pp8XGB7oVDg4x//eJVHXx7jxo3j\noYceCiqagWuSp/oKqi3hn4focftVvpSYoCB//fXne/jw4RxwwAGJjaMvxo8fz4IFC3jrrbeCbbpG\nde2qVbO/yKrKl65HP6lE16OuWb2nnH3ozdcvdy5NzTcMw0gAW0wNwzASYGDVwkuBQqFAT09PqCRb\nsQLAvqqYtdJzpWhoaGDYsGGhMnLKb5adbffddwdczQWAn//854BTH/0xS52aMmUK4Era+SpSuY3O\nkiKXy/VrXvSZ/s5preywxcjn86H2KTJDnXjiiYAryffaa68F+6gZosop+qqt1HmVINR3+6UI8/l8\n6uPUvSl07CoxqL+vvPJKsI/Grs/597LalOy///4AgdnCv2YnTpy4je25GCaZGoZhJEBFkmlPTw8b\nNmwIqgeBax+raix6+vlPLVWhUSWlffbZJ3hv2rRpgGsnW28aGhoYPnx4TZ+6nZ2dQPgcSZpIg3w+\nv420LUfRJz/5ydB23yl24YUXFv3OYg30nnvuueC177D7MLNly5bUqkk1NDQwYsSIslpu6F4biOTz\neYYNG8akSZOCbXp99tlnh/ZVdTaAJ554AoDly5cDTpoFV/FOEndcNbSNGzeaZGoYhpEmFdtM8/l8\nKNxFNQH1NFDogprogVv5JZHKHgew3377Ae5poBCbz3/+88E+adfI7EsqlYQV1zRO7XHjJE2Fr7z+\n+uuAs9WU85tZp1jLB9lQ60F3dzfvv/9+qCar5kB2X4XX+GFivmQCcMghhwSv1RxRdsT33ntvm8+P\nHz8+1OrcSBe/XrDmzp/DSujp6Sm7nYlJpoZhGAlQkWQqW1uc3UJceumlfX7P2rVrg9eytcpTrABj\n37YR/Y16oeNWgL6q64NrQlbqCXjbbbcBvYHW4LytabNlyxZWrVoV8u4uWLAAcMco6dl/Kmts2qbm\ncgCf/vSnATjmmGNqeOSVUygUQlJ/tEK+pFbZ/sFJmbLxq+I+uABxdRbQ9eprT+Xa2JKgq6uL9evX\n8/LLLwfbZB+Ud/vFF18E3JyCm0N5rs8777zgvWhr8izQ09PDxo0bg8B8cO3Ia0kl3QZMMjUMw0gA\nW0wNwzASoC5B+36wuF5LFZPa76uXPT09dW1cppxmHaOcC3/4wx+CfXynXJR/+Id/AOBnP/sZAJdf\nfnlNjrNcZK5R0DK4JnIK1L711lsBWL16dbCPnGtS75XDDbB+/fraHnQ/aGhooKWlhb322ivYJuen\nVFkFdcuR5DN58uSi3/3ss88CzuGq8wa9dRkUZpMG3d3dIXVUJgfVh1CNAb+ZXNSp6yep6F7T2HRu\n/MD+sWPHpnpPFgoFtmzZEoQVggvD1P0p06BvIpQjWNeqb4ZTTn5SmGRqGIaRAJlJJ422p1V6IvRK\nqfUMHZJDQaFQcjr4joZSAdE//OEPAdh5552B9KpDFWPQoEGMHDkytkXu9OnTAWd490suSlvQX3/8\n++67L+CkdT+9sd74Uls0VVTXmy9ll4O0E0l0vmYyZMiQ1NokNzQ0sP3224ecgXLYzpw5E4AjjjgC\ncBIquFRKzXccml+NxdcW09YU8/k8gwcPDhKAAFpbWwE3LoUs+mnA0nx1rZcjjfqhcd3d3Ra0bxiG\nkSZ1l0xll9HqrydHVtJLwYXTSBrRk7pUCMlll10WvJat8eKLLwbiA9nr8aT30fmXhHXmmWcC4WB0\nJVT4WkOULHa79Y9J41R4m6SYcgOzZUOWdqLwHP+cpCWVQq8dv7GxMRSorjEqXVKhePoLTqorRbRG\nry/Vpa0tqmiNL3VqnLJ7a579fSRl+hJtX2jc0HvNlztOk0wNwzASwBZTwzCMBKiLmu8bwhV2IVVC\nBuMsqfkS+5X94oeIRNF4lEkEzgFw9NFHA/EqZb1z86UqyZShWgt++5VyaiQkHW6SNNGc+WLVrnz8\nsC/l9kutl5ofp37WC6n3Mi9pTspR7cGZMHROdC/KzAPpmjKKoblTJmK0dQ5U1j5IWW3RjDlT8w3D\nMFKkLpKpHyAtA3FcM6usoCe9ArFL1Qo455xzgLAEdNZZZwHhOq5ZQ4HPfvA5uLH3RbFKWr6UlgVp\nRlJGse4JcbzxxhvBa0nskn760zix1kiTUsC6QvJK4Tsa5WSTpCfJ1Jfe661JgZuLaIdVv3FgJehz\n/hpUyTjrf3UbhmF8CKhaMpXkobqQCnXyQzUUbqOqNf7KH20J7NtlsoKOyU9LjHLPPfcA8Mtf/hKA\nWbNmBe8deuihNTy6ZChW6b9cTSGutitkQxr1KXaccahCkW8zVfC3khL0fVkKCSvXNurjayC6d6Ut\naoxZm0sdX3/rHUdDq+KSEyohW2fHMAxjgFK1ZCpPmjyHHR0dQDhAWNKqikP4XQ7l6c6irVSUc2xX\nX3014NL0Tj311OC9cmxWHxai9qo0C36Ug2y7Kpgh6VORGuC82bIj+lK7bMpZ0qCSsF/6UqfGK4lP\nkmoW7KRJojHLvyE7eH/HaZKpYRhGAthiahiGkQBVq/kKDVHYT6nwH6lSftiQVAjfYTVQmDNnTvBa\nyQaf+cxngP438KoXUm2j7W79Vs2qEylHnFpe+GRNrYewc0hqvcJ/Xn311dD/PgrI92t9ap41zrgA\n/XqrwzomVfCSKW7VqlXBPi+99BLgkg98s5zu4aw5npI+rzL5RM0Z/SUbZ8kwDGOAk6skpCOXy60D\nVva5Y21oKxQKxcvZJ4SNMRU+CuO0MSbIQBhnRYupYRiGEY+p+YZhGAlgi6lhGEYC2GJqGIaRALaY\nGoZhJIAtpoZhGAlQUdB+a2trwW/96xONCkg6wLajo4POzs6aR0OXGmOtycIYNY/6W4uA7aVLl3am\nEVLzUZ3LciJ0krg/0xojVDaX/viTuI7LHWdFi2l7eztPP/10aJuyCJRtoWwCZYokRVq95tva2li0\naFGoQ+GIESNS+e20xhg3j2rZoCy1aF95cFk/ymCLtgCB8op853K5VOIF48aZFvWcy2L4RWjUOija\ncx7cPKsotObSb80C6Y0RajuXKmijUot+y6RRo0ZxwAEHlPU9puYbhmEkgC2mhmEYCVB1oZNosYCk\n1fu0KRQK9PT0BDVYARYtWgTAypW92qk6VPoVvvfff3/AFcXw613OmDEDyGYRELFu3TrAqTyq2+mr\nPCoSot47qlAOToVUcYws1af11VsVNFGBHhX6+CjgX38afxZ7WMVRKBTYvHnzNqaGJNB9/OijjwLh\ngiczZ84su6eUSaaGYRgJ0C/JVOXZwDks/BJeA5l8Ps92220X6vek0mWSTB955BEgXLbt1ltvBVzH\nAd+gr7J8Bx10EOC6lY4fP74mY+iLQqFAV1dXqOvm888/DzhJTWXc1qxZE+wjSVSeYJVxA+eMGjt2\nbNHfLbfTadL4Y9A4f/e73wGwfPlyIOxMU2cE/ZVzDmDy5MmAm8MsUCgU6l72r9Z0d3fzwQcfsGLF\nimCbuhxHHaJ+D6e1a9cCTpuaOXNm8J6iA+69914Abr/9dsD1o4Pe69m/l0thkqlhGEYC9Esy9SUM\nhUSpp45sh/3t8JdFjjrqKAAmTZoEuOLIvl318MMPB5yt0O9o+fLLLwPOnlxvO113dzfvvfdeqO+R\nnurqvimQUykyAAAZX0lEQVSbr/9UHjNmDOD6yPuSqc6NH0oV/XxcIeVaUigU2Lp1a8h+O3XqVMDZ\nDyXdPPXUU8E+6qKrfmbSNsCFA0lLOe644wCYOHFisM+gQYNS61aqudQxg7snFd6nufRt/LoW1clU\nhbCzTNSXsXTp0tD7CmGSvR+cdqHr2u8/J04++WTA+Tb867qxsbFsqd8kU8MwjATol/joe6r1hFd7\ni8WLFwPhVg+yM8leNXfu3OC9yy+/HHBdSrNMNAPDD+zfc889Q+/JOw5w5513Ak4KkO20Xp7UXC5H\nPp8PPaUlqey2225Aac1C8+jbImfPnl30t0S9WtP4moBey7YrrSMOSZ++lKnrWpENQ4cOBcKRDS0t\nLalJprlcjsbGxpD0rTYlDz/8MAALFy4EnPYIrk2NOpEedthhwXtf+MIXQtvkI1GAvz6XpqbR09PD\nxo0bA6kb3Npz4oknArD77rtX9RuSaH3JFsqPTDHJ1DAMIwFsMTUMw0iAfqn5ftEAGXilWsip4asE\nCiX68Y9/DDiHDMA111zTn0OoK+UUXPDDK3SOFI4jx0a9jP65XI6mpqZQgoVUmXIch3fffTfgxlWK\nenadlQrcX6Lqnk9U9fOdO5BeR898Ps/QoUNDoXya1z322AOA0047DQibZZScIdOTTHHgxq39FRoo\nJw701qtIs+WR5tIPJ9x3330BF2Ko4PpSyTG+OUbmC52DajHJ1DAMIwEqlkwLhULoaa9wGTlTFGLj\np2D95je/AZyx/rvf/W7wXlzv9Q8bCtmQhO4/4etBPp9n8ODBoSd4OeEfN9xwAwD3338/APPmzavN\nARpVoetLziXNrcLAYNtQuDj8AHkIO0ybmppSk76hV9psaWkJOXqV6qyxSBuOCz3UvXfjjTcG2/7z\nP/8z0WM0ydQwDCMBKpZMoxKMpE39FX6wtoKejz32WACuvPLKin4zK+2olZJWyg6nffQkBLj55psB\nOProo4HqQziSoKGhoazCK7/+9a+D19dffz0A06dPB2DChAm1Obg64NvxlZQybdq0eh1ORURDlHSP\n6q/e9xMqSmkiSp+VXVUSrl/XN+2knHw+T0tLS8lwwuga5PP9738fcNcubBvAL8m9v/WLTTI1DMNI\nAFtMDcMwEqBmsrqcTgCvvfYaANdee23Zn/cdWPWuAyo1qZzj0HH7xnmFQKl61EBAYTFyOvnbrrvu\nurocUy1QJtObb74ZbFOG2kAlqsLL9FRuLVBluCn8S9evX9u2oaFhQFSqmjNnDgCvvvoqADfddFPR\nfaPhbZVikqlhGEYCJC6ZynitfGBwtU7jKrYUI82wi76QM82vSRBFSQsKzFd9U3COJz+wOuvcd999\ngMvvBrjggguAbesQDGRUU8J3qGSpS0ASlCOR+mFQqiuh+1YSWz0TMPqLHMGXXHIJULqGcLUacHZW\nLMMwjAFM4pLpCy+8AISfYpVUJS8nJSxtohKp7C9+PUVte/bZZwHYddddg/eyVJW9LyShqPq4L4X+\n8Ic/rMsx1QJVYFdqr58aK+2qWhtavankXvJtxgp6V0q0JPWs2UhV+UqVpHTv3XbbbcE+qkZ3/vnn\nb/N5pb4r8ahaTDI1DMNIgMQkU6V0qRK2An2hvMIgsksOBLuMgu79Y9VTTk/4c845J/0DqwIFaC9Z\nsgRwRSCK1SkdqCgBRDZuVc73A9qfeOIJwJ2DKVOmBO/Vq29XX/hSo8bmFzaBcOdg1TZ98cUXgXAP\nLF3fCl7XdZ41yVTHrGv2q1/9KuDGDy71OY7+BucXwyRTwzCMBLDF1DAMIwESU/Oj+cC+inHHHXcA\nrr1AXKhGVlWJUvjmC4WTKPxLYx0oKDFBThclGPzlX/5l3Y6pFsicpLmLu97kjMqiM7QYfv0KzaUc\nab///e8Bp9KDc9rIyXTwwQcH7yk4v6mpCXDnKCs1MoTaBslkoXm65557yvq8n4SQBCaZGoZhJECu\nkqdNLpdbB6ys3eGUpK1QKOzY927VYWNMhY/COG2MCTIQxlnRYmoYhmHEY2q+YRhGAthiahiGkQC2\nmBqGYSSALaaGYRgJUFGcaWtra6Gc1NBa0NHRQWdnZ82DUD+qY1RsohySKoFYi7jfpUuXdqbhBdY4\nfSerXkfjopMm7bmMcyTXOmY7rTFC/DWrMUev2aQpd5wVLaZtbW08/vjjoeZjO+20E+CqcdcK5VDX\nmvb2dp5++ulUfstvOtjQ0BC0hK4148ePZ+HChUEHBHAV51VbQQHbqmkJsM8++wDlV2wvRi6XSyXE\npa2tjcWLFwcB6uDGOWnSJMBVRPLbIOu1FiO/vmlra2vsbymPH3pro6Y5l4899ljo93Vd6QGpRcZv\ngZxEDYy07kmIvy81PuXiK8nCf7Bo7Aro9+vW6j1d68Uod5ym5huGYSSALaaGYRgJUJGav3nzZlau\nXMm8efOCbWPHjgVg1qxZRT+nhl5RkTuOjRs3AoRUs3Hjxm3TG7xWdHd38+677wZFrsHlLyfd7375\n8uXB69bW1pDaX2sKhUKoTKLUX82NGiKuXr062EfqvcrQSe2HcDFsgAMPPLAGR10ZPT09/OlPfwpy\nuMEVAo62JvHLsUkFlrpY6tqTiqkSjNBb6s5vCJkG/j2lMpCLFy8OHdumTZuCfaTyKz/dL8+ncoS7\n7bYbALvssgsQPkdp19AoFAohUwa467GlpSW03TfZiKFDhwLhc7BhwwbAmUV0P/hmgqamprJrEphk\nahiGkQAVSaZdXV2sX78+ZLz+9Kc/HdpHFVzUFgJc6ws5rko1llu/fj1ASJpIs61sd3c37733Hg8+\n+GCw7bHHHgPgi1/8IgAnnHAC4JqxgSsorIK1hx12WPBe1BkhScdvVnfwwQcHEnytyefzDB06NCSd\nScKaMGEC4I7ZPw+33HILAHPnzgV6HTxCc6qqU3IWXHzxxTUZQzkUCgW6urpCDgY5TEshaU1tMUp5\nid944w0gLJkOGTIkNU0qn88zePDgUHFnjVdzqWNbsGBBsI80L0l70jDBXZ9yvMycORMIX8d77bVX\natcr9EqPq1evDtpQg9MYVeVL96DWIIDJkycDTjL1HVDSHnTNas2p9HoRJpkahmEkQEWS6bBhw5g5\nc2ZJe9gOO+wAxIeQ6CleSjJ96qmnADj55JND29OSTBsaGhg+fHioLbWegAcddBDgnmR+3Nvbb78N\nwKpVq4BwSFEUSTovvfRSsG3q1Kmp29l8iUuhQlHOPffc2NflsnKli4JK287W0NDAyJEjg/krF9UB\nld3etw1HmT9/fvBbYtKkSanW/szlciFtcd999w39Pf744wG49tprg300L3HXotp86zqXPd33Y4wZ\nMybV67WpqYmxY8duY5sHF86n1jt+nVJJ7K+88goQtvvKJiwJW9L6/vvvH/rdcq9bk0wNwzASoOJK\n+31lGWgV95905513HuDsqMcee+w2n4u2aY1Kpmkhaea0004Ltqk5nu/9hnB7YD3F5f3234vy+OOP\nA+HK51nh9ddfB5w04kuW0joktc+ZMyd4T5X5ozZ0eUwh+crmfZHL5UI2snKRrbScwHZJptOmTQu2\nVSLNJEGhUKj493x7N4Q1Kc2h5k6V+n1bZNpaRi6XK5osonvtggsuKPp53Zf+ccuW/K//+q+As49+\n8pOf7NcxmmRqGIaRALaYGoZhJEC/Gur5QbEKJ5CIrABYXwVcuHAh4JrOxaH9/fzheuKHipSDAp19\ntbYY//Vf/wWEwy7GjBmTSL50f1mxYgXgHIcdHR0AoeQFqUiPPPIIEA7UV7KFUN6/HzIUNQFkATkO\npcqCOxelHG6333474Mwgp5xySvDeyJEj+2Ve6A8KV9Q4wN2DcrxVEt7jo2B4ObL8a2HIkCE1KyxS\nC+LuZ82d6op87WtfA8JrUCWOxIFzNgzDMDJMxY/PQqEQciT4oQbgnvB+gPDll18OFK+4A07CvfDC\nCys9pEygp1w5EokkOyUBQK9DoK/qNbVk4sSJof+POOKI0F+fr3/9631+3x//+EfApW/WEz+ER+mE\nSg5ZtmwZAPfee2+wj0JmSkle119/PQCDBw8GwgHtzc3NqYbytbS0hK47SaYaqwLd/cpuuhflbPND\nAaPoHvfv35EjR6beArtQKITSQRX2JK2wUn76058CcNZZZwHx61Ml82iSqWEYRgJULJlGA4SF7KE/\n+tGPAJeeCHDdddcV/b5///d/B1y4VJo1EvuLUvCUYOBz8MEHF/3c7NmzAWeT8aW+lpaWVJ/0hUIh\nZA+S9CapRqFRlUqWug4UQF3qfKSFf14VwiaJQ3M5derUYJ9S2pHs3dLArrjiCiCcwJEmuVyOpqam\nkFajeZXUqbRhX/JSAo1srUr5BhcSFtWUfFvidtttl7rNNJfLlQyvU7KFv/ZE7cUXXXRR8HrRokWA\nC42qFpNMDcMwEsAWU8MwjASoOn5DRuAHHngAgDvvvBOAX/ziF0U/47c9UTWXU089tdpDSQ2FzviV\ngQ455JA+P6dqS8oIU8gJpJ9REkXzKFUpLp9dudmqbxmHMqimTJmS9CEmgswZytBTFtoZZ5xR1ue/\n+93vAjB9+nQATj/9dMA5orKATDVy6ipzz6+Xq/DGGTNmAE7tB1dtSqxZswYIO6nSrORWjGg/L43B\nv6+ExnDjjTcG20qtUf3BJFPDMIwEqFoyVVC2DPMyEB9++OFFP/M///M/wWs/Bz7ryOnw6KOPAsUr\nLUU56aSTAFd96Oijjwa2rRBeTyRl65gUauMnIZSSSIWqZsU1lPNrbtYLhdZIypaDolQXhfPPPz94\nLcn7kksuAcKOm6yg8ywHkebW1wilechJpboLcciBl3YoVLlI6oyTSIUqZ33iE58ItqkucVKYZGoY\nhpEAVUumSjsUf/d3f1d0X1WW91O7VAk7y0iK+e///m8A/vEf/xGAn//850U/47dRlm1GYWNZlGZk\nu47iS8/SQhQu5SdmSCL10yqhdP+keqC0VwWiS2KJQ3UuFb4HLpztxBNPBMpL0kgbHZOC9GVbVKIC\nOOlVknpcjWJVidI1kLX0UY3n1VdfBeLTZtWvTp0f/M4RSZOts2MYhjFAqfqxqmDl733ve0C4SnUU\neRNLdTLNIrIVyWMr76Gkkzj8MR555JGAK/SRJc9vJUQD+H17qgLCo0HVvmRab+8vuKB9JROUsgN+\n5StfAcLS+Ze+9CWgdCeFehO9vnTe41KD4zriqrupogCyKH2DC84vlRii6IurrroKqG2xHZNMDcMw\nEsAWU8MwjASoWn6XulOO2iMVq57VkfqDDPlyVuyxxx59fsZXDRXYrfYKWVB3obzjWLx4cfBaTgpV\nhPLVyU996lOhz0UDqsv9vVpTSesUORH98D21tMjCWJIg7l6U403qfdYcTyLaekXccccdwWs17/zG\nN77R5/dV2wQxm2fJMAxjgJGrZDXO5XLrgJV97lgb2gqFQmU9e/uBjTEVPgrjtDEmyEAYZ0WLqWEY\nhhGPqfmGYRgJYIupYRhGAthiahiGkQC2mBqGYSRARXGmra2thWK9buTIKuXQqiZeraOjg87OzpoH\n95UaY63JwhiV/qm//nwqtlLz2N8Y0qVLl3am4QUuZy7j4mGTIAtzWauxibTGCANjLitaTNvb24Pq\nK0I3nQK5VWHJb2oVbdDl52srQFj7qGmX3/xrxIgRsfUxa0FbWxuLFy8OtQdWQ7Jak1YzwXHjxjF/\n/nzeeuutYJsqQqn6k6or+RX3NTfKyVczNnAV7JWsoMpg48ePD/ZR8sN2222XSohL3PUq1DBQgenl\nBvOXe8OmPZfRluvg2o+rVqmSZpIizeaXcXOpZoiqHtXc3AzE369RIcF/rXVJ10BUICxVb8TH1HzD\nMIwEsMXUMAwjAfqVm++LylKX1DpA6qHffEv7SxxXMy9wDdyiapbfozuXy6WWC60+5B9murq6WLt2\nLXfffXewTa+fffZZAPbee28gnJcuNV0l2vzrQO0gjjnmGCBbZdv8dikyR0lFnDhxYkXflcWc/GJ+\nCtXLUI0B33QlM8zw4cOLfu8777wDhO/FLKG1Q6YpHeeoUaOCfVRiUOZEzTu486FzoDXIP0/d3d1l\n5+ybZGoYhpEAFYsPPT09IeeSnBha+VVZqJR050s0+rwcUHGtB+qBGo2Bawssg76ebr7RX5K4jPx+\n87msjEk0NjYyevToUHFrPY31V0/0F154IdhHT2g5pXzn1H777QdkSyLt6elhw4YNoRYyuk5LNdAb\nSDQ0NIS0QB9FXUhCVUNIgPnz5wOw2267AeEGiNJK5JTUX2kmWUFrhjQPtbb2i5brHMg59corrwTv\n6b6Wg0njk3YNvZqISaaGYRgpUrEYkcvlAlsFOBuUaguWY2/0401VE9MPhaonPT09bNq0KQjxAtfe\nVxLOsmXLAFiyZEmwj+yICg1SHUVwLaHVXmHq1KlAOLQkzTa6+XyelpaWUGtcvf7mN7/Z5+d1bm67\n7bZg27nnngvAFVdcAbi6n/Wkp6eHLVu2hGxguj41X5pLX1tS3VnZDBctWhS8p1q2pRrx1YN169YF\nr6VNSKLU/ebft08++STgJFN/jApn07WQNYlUSHvS+NQU0rfx6n7UNSD7KsCLL74IbNtM0Lcjb926\ntWw7uUmmhmEYCVCxZFooFIL2sODsFr79rC/U+hicbfJzn/tc0f03bNiQWstgRQ7oiQ2wzz77hPaR\nDeXll18Otvk2VnBRDuC8hLLb6Enqt8neYYcdQhJUlpE9+Otf/3qw7bjjjgMISbtRZNNKi1wuF9iH\nhaQZ2b/9ORSS2ubOnQvAc889F7yndt9ZYevWraxZsyaQtMFpOYpUUBC7pHBw7crVJPHOO+8M3jvz\nzDMBOO+880K/5dsiof5tvKUpSiKVdqw1yUfnxE9U0XmRphhHY2OjSaaGYRhpYoupYRhGAlSs5ufz\n+aCfNjhRuZxe8I899hgAf//3fx9s+/KXvxy7rx9cu2nTplTV/L7GIrF/zz33rOq35CCAXpUpra4H\nhUKBzZs3B2aHJCim3j/yyCPB6wkTJiT2e+WQz+dpbm4OBXFr7nTdKnjdV+XmzZsHOJPAJZdcErx3\n9tln9/m7aXavaGxsZMcddywZfifzStRc5ePfbyeddFLsPp2dnaH/FWaWJn49CDmKZGKMU++FEob8\nMDk5hpNKxDDJ1DAMIwH6FWHtV2WJOlX0tPKfzjL2X3fddUDY6K+Qmihr164NXo8cOTKz7WarQYZz\n6D2PaYVHFQoFenp6QsHJChmShKKwkdWrVwf7yJGmuSgV/qSUYX8e0w6Uz+VyRaVvOV7013cYKtxv\nxowZANxwww0V/26a9HVv+Om0Ue6//34g3Jq8WAtl/54eMmRIXe5Jv/KVNJ1SEqm48cYbAfjVr34V\nbIu2J6+WD98KZRiGUQf6JZmWsrXp6eUHCOvpp6fC7Nmz+z4wLy2xXk/BKLKhxSUmlHovikKgfEm0\nubk5NYkmn88XPU5Jq7Ix+WFw2lZMm/D57W9/C7gAeCC25mZWeOCBB4LXCmC/8sor63U4iaDiQ3H2\nVGkOsoN+9rOfLfo9upd9u+ouu+xSl9ThSosQLV26FICf/vSngNM2AE4//fTkDgyTTA3DMBLBFlPD\nMIwESFxOl6rqV6i56aabgN7WAwDXXntt0c8rk8MPv6q3ii+1PKqG+3VZpVIpd7sUMgmk1Q6lHOQ4\nlMNJqp2vVpXjQFImmMJxFH4C2RqveP755wG44447gm1TpkwB4KijjqrLMVWLzn2pEK1vfetbAJxy\nyil9fp9qMfj1M5qamjJZ2zXKv/zLvwDOnOGvPUmbKUwyNQzDSIDEJVNVr/nZz34WbFMolL8tiozb\n0eZWWUCSsRxGcsr4ucrlSKRy7khiqLfE7aMnt+oFSOrwQ2Z8430xFixYALhKQ1nSMHw0F48++igQ\nbgB5wQUX1OOQEkPaQbTa08KFC4PXd911FwCXXnpp0e+RhKvrPqsV96P41czkeDrnnHOA8iTx/pKd\nq9swDGMAU7VkqjRPPcVUmeapp54K9lF1oWJpauAkI0kyWbLH6FgkkS5fvhwIV0FS8Hc5gfdZtB0K\nVcuSXbdURR0fSXYav6q7Z22ssg2r15Uqr5911lnBPpMnT07/wBKkWP1RP0lDgfmS3C6//PJt9peW\nonTcLHVRKIWfZKGEhYsvvrjmv2uSqWEYRgJU/aiR1Cbvr570fhEQv7BJFBUuUGpl1iQZcNKMbL/q\nHeP33pFErqe3HyitwikaWxZsh1HJXzVK/d5VlRCt3K5z4/9OFrSNaJqr+v/MmjWr6GckpUPlQeNZ\nQB0DfIn1C1/4AuB6PykhA1xkiu7JLCdb+LzxxhuAGy+4BJNyNaxqqP9dbRiG8SHAFlPDMIwESEzN\nV2iTjNxHHnlkWZ+XSF6sUk0WkKNphx12AFx4RZxJQuNX5SFwRnBVvEmz3mUtkEroG/oV6D59+nQg\nvn5DFsYtNV3mDIXg+aqhHC8y2fh1WFWhKAsmi2LIGfjMM88A7rqbNm1asI//OsqKFSsAN+40mz1W\ng8Lc/EaV5TSITAqTTA3DMBIgV4m0kMvl1gEra3c4JWkrFArld+3rJzbGVPgojNPGmCADYZwVLaaG\nYRhGPKbmG4ZhJIAtpoZhGAlgi6lhGEYC2GJqGIaRALaYGoZhJIAtpoZhGAlgi6lhGEYC2GJqGIaR\nALaYGoZhJMD/A/8aY2yG4odpAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot_conv_layer(layer=layer_conv2, image=image1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "And these are the results of applying the filter-weights to the second image." + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXeYJHWZxz/VM5tgl7RLUMKOAktmYZWc4wICkrPIwSMI\nd6eIgiei8oiCeCTlgEORU1Y8JUnOyURaliw5LDnsICC7wIbpuj/m+Va9VVNdUz1TXd0c7+ef6anQ\nXb8Kv3rzG4RhiOM4jjM8au0+AMdxnP8P+GTqOI5TAj6ZOo7jlIBPpo7jOCXgk6njOE4J+GTqOI5T\nAj6ZOo7jlIBPpo7jOCXgk6njOE4JdDez8YQJE8Kenp5SD2D+/PkA1Ot1AEaMGAFArZac52fOnElv\nb29Q6o9n0IoxNsJmnwVB8P9qjH19fUDyOgZB/9BmzJjRG4bhki09APLHqXO/YMECAObNmxetmzt3\nLhDfm/Y6jRkzBoCFF14YgO7u7EeoE66lnikdv66JXSZ0bdLbQTxGO9Yq71fIH6eOV9fLjkWfdZ2z\nzoHu0ZEjRwLxHKT9i46zqcm0p6eH+++/P7Hso48+AmD27NkAjB07FoDRo0dH22gAXV1dDb/71Vdf\nBWCRRRYBYNy4cYn1n//855s51CGTNcY0H3zwARCffGj8UDVDJ40xD002jz76aLTsU5/6FBDfvO+9\n9x6QvI7LLbectnlxyD/eBBpn+qVl0bV85ZVXomXPP/88AHfffTcAN91004D9Dz74YAAOPfRQAEaN\nGpX43qqvpR2jzr3Goev19ttvR9vouZ0yZQoAn/3sZwd899NPPw3EL5plllkmWjd+/HjWW2+90sYx\nGD09PUyfPj2xbM6cOUA8lgkTJjTc/5///CcA119/fbTsnnvuAeLzs/nmmwOw5ZZbRtssvfTSrL/+\n+oWO0dV8x3GcEvDJ1HEcpwSGpJtau8Ozzz4LwGuvvQbE6r1Vpx588EEADjjgACBbHF922WUb/l6n\nVbaaOXMmkBzHUkstldjm9ddfjz5LBc5j/vz5HTVOqU7WXCNeeuklAG655ZZo2eTJk4HYpii7k71X\n0ueoKtKqvWWhhRYCYNKkSdEyfd5hhx2ApHr77W9/G4Czzz4b6Fc/AXbcccfyDngI6HpBbLJ4//33\ngdgcteqqq0bbyPyy9NJLN/xOnYd333038X3Qrxq3436195Ps1jIt5iEzon0WZb64+eabgXgOs3PR\n+PHjC4/TJVPHcZwSGJJk2tvbG32WE0Kz92c+8xkA7rzzzmibn/70p0D8Ftx3332b+r08yaJKJJHN\nmjULiB0qlnfeeQeAk08+OVq29tprA/CVr3yl4XePGDGiI8b5xhtvAPDb3/4WgAMPPDBap+snbcRK\nNZISdI50H1jsffNx4sgjj4w+636fNm0aALfeeisAG220UbTNYostVuHR9SOnE8Cbb74JxI4VXZvl\nl19+wH6/+MUvgNiZA/CNb3wjsY3GY+/PDz/8MIoWqJIijt4TTjgh+nzdddcBcPjhhwPJa7nFFlsA\ncNRRRwFw4YUXArDWWmtF26y++uoDohsa4ZKp4zhOCQxJMv373/8eff7www+BOMRC9pq//OUv0TYK\nSzj++OOBpN1C4QgfB2R3kfSWZUtRzNqMGTOiZRdccAFAFGIh+2Inoje4QmWsnVNSmGxnVsNQHN/L\nL78MwHPPPQckbZHp8KFOQhoFxMcpe6rllFNOAeCxxx4DYmn7rbfeirapWjKt1+sJyVQ2zsUXXxzI\nDtOS1CoNxD6vOhc//OEPE/ssuuii0Wc901UShmGu9vbNb34TgDPOOCNaJs1q2223bbjfueeeC8D/\n/u//AnD55ZdH6/bbb78ofnUwXDJ1HMcpgaYk0zAMWbBgQeItuOSS/Yks8vrK9iJJBeD73/8+EHv1\np06dGq07//zzAfjSl74EdI59NAsdv2xFWW8svb2txK3gb/3tRMn0oYceAuCaa64BYruoTbT485//\nDMAmm2wCxLY4i9bputr7YOutty77sEvDJjHoXt5ss80GbKfru/rqqwOxliJpsGr0TFqbp2x8eedb\n0vePf/xjIGnP1ndJ+l5zzTUz909nKbaaRr+npArZhB9//PFo3WqrrVb4+2VPPe+886JlzzzzTCTF\nD3p8hX/JcRzHaYhPpo7jOCXQlJpfr9eZM2dOwjCvgGY5JWSYtvmtcjyJPfbYI/p8ww03AHFA8fjx\n4wFYccUVmzm0lpJWe3RsebnAUnchDhNSHngncvXVVwNxWEjW+Zfqv/vuuwNx8gLEwesaqwKgbT64\nNe90Crqmd9xxR7SsSM65wuLkeJIjtmrCMKSvry+h5ucF4qfJMmUIW68gzahRo9puktPzpDnj6KOP\nbrit7kNtm4UcWNdee2207OWXX04UwcnDJVPHcZwSaNoBNW/evChlEOLwi3QqVp5kKckGYongiSee\nAGDllVcedP+q0VtNkliRN7+V3uWsSFfC6iQkhRxyyCGJ5Vb6lNQpJ8zPf/7zaJ0NR4E4vdE6K/Oq\nhrULOQVtNaEi5Qnl2FCSQrsIgoAgCBLnNk/6Ei++2F+4S84VG8ImspJSOgmlLOdV6FKqqO7Dvffe\nu+G2Om8bbLBBtGzu3LmFkxNcMnUcxymBpiTTWq3GmDFjopqjEAfwq9jATjvtBAwsMGtReiXEwcJP\nPvkkAOuss04zh1QJCiORXeqpp54adJ8lllgi+qw3XVaKZaegY7PpdgB33XVX9FnhXgrqvuqqq6J1\naclUEo89D50omUvKfvjhh6Nl9957LwCHHXYYkC1RSztROE47UkhFd3d3ojiHLW7ciHTxkizJNI21\nHdpavlWS5XfIG6/uS9nB8yRTseGGG0afu7q6Cp1PcMnUcRynFHwydRzHKYGm1fyxY8eyyiqrRMtu\nu+02IFaBFWqR52yw6q4qMMlhY00AnYIM03IkybShUCGAlVZaKbGPzbxQdklWJomougJPvV6P8ugB\nPve5zwEknIsAK6ywwoB9Nf68kBGZRqz6WKTuZNnU6/XcTJ199tkHiOuTQhzmpEwaW0VISN389Kc/\nDeSHybWSIAjo7u5m4sSJDbdRDQkbyqR78U9/+tOgv6EqVFbFrtpkFYYhc+fOTTj8FJaZVUNBSGX/\nxz/+AcStSiDOztt0000T+9ixzZ8/39V8x3GcKhlS1SgrvcjwftFFFwGxk0FvfBhYRd86sBTcrzzi\ndgcCZ6Eaiqq8LuO9At0BjjnmmMQ+VhpSmFdeSFWVVcvr9Tpz585NBHqrxkIa+9bWNb7xxhuBbGO+\npFVJSnlB4VVQ9LzaqknK088KkVKIjc6dwoc60bkmVA3JVu3ac889gWL1EiSZNZMMUDYKy7T1MPIk\nUnHiiScCcZcPG853xBFHZO5jq2ONGjXKJVPHcZwqGXZ/4v333x+Ayy67DIglNDu7qx1uFl//+teH\newiVofAKhZPI3gtxcLDqf9r0QkktkmokzVs7aZUVeIIgYMSIEQkbZpHfl41Y0ud3v/vdAdvInqrU\n0U6wgTcb0pMXBC7pTiFfut5ZFbQ6BWkHZ511VrRMoWuS2LKQVC+bqe0hVTVBEDB69OhEPy6Nocg1\nVXKQbdusSvtprAS/zDLLuGTqOI5TJcOWTGW3UN8Y2RNVlR3ilFN5EK3txdpPOx159fWmVwotwOzZ\nswF44YUXgGQnx7TXUW9S+8Yr+vYrA3mAbdqhoi8kPVvNQmy33XZAbEfNSluUd1vXuB0efEutVkvU\nVB0uklrkB9A4OzFVViixQNcP4iQFFXqx3S/UWTedatpOf4a0KZsEouOThqciS7YzsJ5RpQuvu+66\nDX9Dz6y998eNG1f42rpk6jiOUwI+mTqO45TAsNV8sf322wOxKqvwGYDp06cDcbM1GwQtw70auLUr\n+LkIUs+l3lrVXEHB+mtDjeSkkPogtaFI29pWkuU0UbWnLIqEOencFG310GrSFZWk8g/13EvVVTJD\nu3LUh4JC+9Kf0+gaFm1xXBVhGCauZTo0So5ee22baVsiE46tszBy5MjC5g2XTB3HcUogaCZYPAiC\nWcCLrTucXCaGYZgdWV4iPsZK+CSM08dYIh+HcTY1mTqO4zjZuJrvOI5TAj6ZOo7jlIBPpo7jOCXg\nk6njOE4JNBVsN2HChLBI58ZWMHPmTHp7e1uez5Y3RqWt6a+NecuLRZOTTzGc2i8do9gJYxwq6eLW\nOh9Z52XGjBm9VXiBW3m/Kl5VRVTSqcEf52tZlKrGCPnj1LVQbLONj9X9p9hTW8RE67S/nst00Z+i\n42xqMu3p6YlqPZaFKi8pR18Dsg/nyJEjcyv5lEnWGDUJqpFcunKQRYkJNvlAtS8fffRRIH7wVN0e\n+mvEtnOMVRIEQSUhLq0cp2oY3H777UCyLfKUKVMS7YJbSdYY9ezkVQJT1fy33noLIFEnVPUlVGeh\nUYeIqu5XyB6nroG6BTzwwAMATJ48OdpGCUJKmMmq2/vGG28k/reVqaD4OF3NdxzHKQGfTB3HcUqg\n0uTwF1/s1+7uvPPOaJnya7faaiugc3Lzrd1F5fWefvppIM61t2r+NddcA8CPf/xjAHbbbbdonfqv\nq8SX7Dc2f93acpzO4plnnok+r7zyykCsNuqaqpQd9Je8s2pzFdjkm0bq/S233BJ9vvjii4HY5GTN\nFGpLJBOAajhU3UQvC3teH3zwQQAuvPBCIK7vsfvuuzf1nVLrZTawNJPU5JKp4zhOCbRMMrVvQb05\nJH1uvvnm0Tp5tPMk0nnz5lXacA6SDjA1W7v33nsB+M53vjNg+2nTpgHw1FNPASRa76YlBUkzKqQN\n/W//dqb2yiEhZ9nbb78NwF133RVt84Mf/ACAV155BYCTTjopWnf00UcD7S8GPVSsc0MFhaVBSDOB\nuNC57le1+FbLc+h3aFQtmeZFk5x22mkAHHvssdEyFXw+6qijAPjCF74QrdO9qwLSqoSmItPQvopn\n9plRqyBdryuvvHJY3y1tw/6GrSA1GC6ZOo7jlEDpr5df/vKXABx++OHRMtmZtMw25lKrgTyaqSlY\nFrYliWy8G220EZDd0kNv6vPOOw+A/fbbb8A2Og9/+MMfgORbL90Ouwps7VK1elBNSF0je1yKx1Pr\nYDVRhPg6qi2EQlIahdV0CqeccgoQ28MhlsjUovtb3/pWtC5d71XjvPzyy6Nlzz77bOX1XK1Wo8+6\nby+44AIgrjkMcN111wH5EuaUKVMAuO+++wDo7e2N1qXDh6ri1VdfjT5Lm/ja175WeP88P4UaB1pN\nxCVTx3GciilNMpWnXm9BazM84ogjgNhmmrWf/u68885lHdKwsJKKbJxpadO+qRWQf+655zb8TlX9\nluRjpYLtt9++cunb2vUUVaFIBSFJFWLNYqeddgLg/PPPj9bJzqQ3vySITpVML730UgBGjx4NwMYb\nbxyt+93vfgfE7bplI4bYYyx0fuQBh34veJlN/IpgJS4Foesa7LXXXgD86Ec/auo7ZetX5pEdYzvo\n6+vjpZdeiv6XZHnQQQc13OeJJ54A4rbkf/zjH6N1emZ1j2a1aW8Gl0wdx3FKwCdTx3GcEihNzVf4\nkNSbfffdN1r3zW9+s+F+cvQoAFeqsIz/VROGIfPnz4/y8CF2HKXZdttto89SqR555BEgGf6V5qGH\nHgKS4WBVqvhhGLJgwYKEmcH2Tbfcc8890WeFBa2yyipAnIwAcSiZzAVyWnRqYoJqQMh0YQuV/PWv\nfwViE4AN91LIWDrHO8spWRVhGCZUUznJdE2t42koyNmk8Dnov65VhvLpubSB9Wusscag+8nRJtOH\naoDAQBOUnkFr1ll22WULj9MlU8dxnBIoTTKVBKJKM0XeGnY/SXZKy2unZDp37tyEpGKdExal2wHs\nuuuuQPz2VkAxDJQM5Kyxb8mq6evrK+QkefbZZ6PPX/ziFxPrbEk0XcfnnnsOyG8Z3QmonXFWu+s9\n9tgDiCVpK3VKk5LUp3NonTPjxo3LrdhUNrpnhX5bGsTSSy/d1PcpREjnRpK5HdP8+fMrl0wXLFiQ\ncIgqVDGNva91jNJE5DzNQink9np3d3d7q2fHcZwqGbZkqtlc9j+lnK2wwgqF9lewt1LW2t0tNQxD\n6vV64g2oQPQ0m2yySfRZ49Y4bGKCkNQmyXT11Vcv56CbJAgCRo0alXiDS5KUnVDYUCB7TtJIinv8\n8ceB+DyoeLLdpp0onE1FO3R8tlB3WgK/++67o8+S2lT8RJqIlUyXXXbZAYW/W0kYhgnpSbZwm/6Z\n5vnnnwcGhnpBbDO09XYheW8MNXxouNiCLI2KRVtfgEIupUWqEFEWei6tjbwZ27BLpo7jOCXgk6nj\nOE4JNK3mh2GYyKdXCJGM3Ouvvz4QO6IafYdQ6I2yjKwY3w5qtRojR45MqD9pR5HUAZvRpdAgmT2U\n12xRTrucc7vsskuJR9481vkitU+ZX7p+zdaXVRaJVMI800CrqdfrzJkzh7///e/RMqn1MkOpJoHN\n2Etj6+9qPFIlVdPWnqflllsu4cCsAuscymrNkUb365NPPgmQCAW0pplGjBgxotJwviAIGD16dCJM\nsUg4mswteh7zQhZ1DmwI2CKLLJKobZyHS6aO4zgl0JRkqvAE1byEOCRGoQd64+dJpuecc070WQHS\n22yzDVBtk64s9AbMMswLVZKxDqRLLrkEyK4/INTwS7U/03nw7UTjVe6yatDaxAQ5WfIcG5LY1l57\n7QHrir7hyyQMw0jrgVjyVo63qj5loTqgF110UbTMJqNAdqO2T33qU5U6oGq1WpRQAcVqjWqb3/zm\nN0CyGtP3vve9zH2swzIIgsol0+7u7qaTIw4++GAA1llnnUG3lRPWat6NklmycMnUcRynBJqSTIMg\noKurKxEGku5ZrXQ7W91F9lSFmChECODQQw8F4F/+5V+aPvh2Y0OmJGFnBQUrEUHB+2npBqpPz2vE\n1KlTAbjxxhuBuLo+wOmnnw7EoUP77LNPtE5Sp9L9mnmjt4parcaYMWMS9lBV7NJfHa+tX/vrX/8a\ngN///vcAbLbZZgO+W/e00mdtfc+qg/YltTWDnlNVDtt///2jdY16PVnJtErJezA09yhcy9YgLSKR\nCl1Lq3mPGzcu0roHwyVTx3GcEmjam1+r1RJBrZJI9NbS7D5jxoxoGwVKS1qxVfhV2OTjiLXfnHDC\nCQ230znabrvtgGwPd9G3X6tR8YczzzwTiG2nFtnDVSsS4uuvcyJvtn3LZ6VutpogCBIJJApEV2C6\nepVZO7CiNxTgbW3j6YB4SUG2U22nYvuaaUw/+clPBt1P9286oaNT0Lik2TVbXEdapSI7rN9m9OjR\nnk7qOI5TJT6ZOo7jlMCQcvOtsVtqjtQ6idy2apDCRrLCpiSaV92yo0omT56cudyeo2bUiSqQun/G\nGWc03EaONYCZM2cCce1X3RftUO0taeeMVDiZJa6++mogGaiuseuvNcvIHCMHlEKSOqHuwGBYp1gz\nLUg66b7MQtdXwfY2AUGfZ82aBcSt2CGusyATje6NoTYLdMnUcRynBIJmwnGCIJgFvNi6w8llYhiG\ng+fJDRMfYyV8EsbpYyyRj8M4m5pMHcdxnGxczXccxykBn0wdx3FKwCdTx3GcEvDJ1HEcpwSaijOd\nMGFC2KjvSquZOXMmvb29LQ94yxqjnHSKTVRxCFtSToUfhhNv2Alj1NgUL2zjL/XZpiUKxRkrJlFx\njFnnY8aMGb1VeIHz7leNIWssiiUdTnxlJ1xLXS+VIFRRc4DZs2cD2amiihFWarCuoY3XDYKgsjFC\n/jjTKe26hyE+B1pnY231zOqv7mGb2t3MOJuaTHt6erj//vub2aU0qqpzqjHaKId0EzU9gFtssUWp\nv131GO1EorxkBTmrOr2qKkFcCUw3pH249MDp5tVkapvTKaA/CIJKQlx6enq47777EtdSlYU09la0\n267X61HHiVaja2krOuk6XX/99QBcccUVQHyNIU4k2WGHHYBkdSXV0NCkmvVCrHKM0D/O6dOnJ+5Z\nvSSUMKJmjnaOUu0FJZjYseh5U1Uw1SK241p00UVZb731Ch2jq/mO4zgl4JOp4zhOCQwpN78IVrW6\n+OKLgbhpmy1pJhE6r6FZO7Cl46Tmy4aU1ZIjD5V7k4oiNarqpmtprAqvY1POufLSh8qVV14JJHOh\nGxUdbjVWBZY9zJaRHA46b9ZOV3WefrrJpQpd65m64YYbSj+uer3elmLm1oav+1emG9VJ2HnnnaNt\nDjroICC2F7/xxhvROtmN06UFVTgbmmsI6ZKp4zhOCZQumcr4qxbOEDdp0xtzo402itZNnz4diCu1\nHHnkkUD7C9HqrQWxV7eIRKqxWseGHDUrrbQS0H6JFOLmiCKvSd5Q2G233YC48C4kpYIqsRJUkWpJ\nL77Y7x9T25K33norWqcGg+lWJu1s6RGGYSSdQVydbdNNN23Zb3Z3d1deTSp9z+o8S7tttnVLGs1P\nVsqfPXu2t3p2HMepkmFLpnrr682ocAMb5iMpRaEM1iaht8Bll10GxG11bWuTqqnX64k3/SqrrDLo\nPnvuuScQh5jsvvvu0Tq9QTXW8ePHA0kbVpVv+TAM6evrK2Q3tI0RdU5kf7KNERUutd9++yX2t9KR\n/a6qCMMwETeolhZp+6HVpE499VQgtvva1joK71L4jNr0yK4O/XbZrPjVViBpzWpCzdj5fvaznwHx\nPQlx48dOakUehiH1ej0hfZbdtFF1lm0sajONLl0ydRzHKYFhS6aSqPSGzmK55ZYb9HtkM502bRoA\nL7zwQrSuSi+w3oD27dTIC2rb495xxx0AHHjggUDSG66xSJJ79dVXgWSg9IQJE8o4/KbIskvrLSwt\nwjajE7KD2oiH733ve0Ac7H/SSScN2C/ru6ogKyNGEo6k1s9+9rPRNmp8qIwbq5lI0raSnP0+6E98\nsLa9Ksh7/oS1/e61115A7LNQwDrEUSuSvhXh0QlU4Uux13bevHmF23a7ZOo4jlMCPpk6juOUQNNq\nfhiGTTtLpDLmGcanTJkCxA6oGTNmROuqDvau1+tRAHAWDzzwABCHzgBsvfXWAJl5vDr+P/3pT0Dc\n3Ms27qpSzQ+CgK6urkTIh9TUIjnrUnWlDkLsgJHzJkvNbxdWzc8qZgFJ1U4hcDJLWJNNo8Ip9plo\nxmkxXIIgKByKZR2AW265JQCbbLIJEN+/ABtuuCEw8Hm1TrWiqm9ZBEFArVZLnGeZbIqM/9FHHwWS\nz1kjB5a9N/r6+twB5TiOUyVNS6ZFpVIFwNp9ioRsSFr6xz/+0eyhlUYQBLmSmdJj1cIa4gpSeeEa\nckooDMe2gB5u+mazpFsgC72VpRl87nOfi9Y9++yzQDwO67Q5++yzgbhCUScgZ6KVqBo5MKw0Ioej\nEhnyHGeSWmygd61Wa0uq5WDYcD+NTedDCSXQ+DmtWhod7PeLSKQK45PzrUiVMPs7I0aMKDznuWTq\nOI5TAsMOjVJ4zMILL5xYft9990WflYJX6IBStjuAjz76qFIb1GBpaZK6bcEW1X7M2/eJJ54A4oIv\nabtdJ6GUSiuZSnq59tprgWQK5aqrrgq0NoVxqNh7SZpP3rmXxqFCGnkSmWrc2lqhnVa0R9hnVMer\nZbZwdCOs9J1+3juVBx98EIjtxc2GeY0ZM8ZDoxzHcarEJ1PHcZwSGLaaLwN2OkxBaiLENS2tytgI\nZZtYR878+fMrNegPZnBWqJMNLbLOmEbcdtttQBye066MIGBAllcaVc2yVZ+kwus6XnDBBdG6qVOn\nArDGGmsAcahUu7Nn+vr6EuYIOSJ032aFpDXT50znyZ7LsWPHdqQJx45Vpgxl4a277roN9/voo4+A\npFN56aWXLlxNqZ3oPtR8ZGv4lu30dcnUcRynBIYtmeoNrL9689vc+m9/+9sAHH300QAcdthh0TpV\n4Xn55ZeBWFqzAe0jR46svHZiHqplaeub2ureaY477jggzs1XRax2VZ4Pw5D58+cPCAGBOCRNUsg2\n22wzYH9pDcrHt0yaNAmIm5y1WzLt6upKSIkffPABAL29vUAsWcqBmP7cCN3nakBoKywttthilUmm\nYRjy0UcfRYkg+n0YGOJkNSHVDS6CnFNWwm9HmFRfX190viFOtMibG1TFTPOLrdTfCJvksWDBgsIV\nwFwydRzHKYHSK+2rRqa1z6iG5FVXXQUka50qxEKSgiQkGxDfTOBsFcjOlFc5Xq1lAW699VYAdtll\nFyCu2DPcyuDDoVarJSQN2aQlcWlsOnaAL3/5y4N+r66nrYjVLpQ2a+u2pls9K0To4YcfjraR9Kp7\nztoK5RNYYoklgNi+atNRR48eXbnkJrsmxJW7JC0XsednoXMl6d2mD48aNarSMaoGr+YSiH0xCsvL\no5lOEnauacYu7JKp4zhOCZQuGqmu4gknnBAtO+aYY4A4WN166rW9bHWysVm7VbvT2NJIKrHoLa6i\nLjfddFO0ThKpkhea7W5aNpLY7HmVZqBaq1qnawdw1llnAXDmmWcCcZEMiO1MepN3Qp8rYdMOpfFI\n+lAihaQ5iKVUnQN7v6o277LLLgvE1dntPVGlFhUEASNGjEg8L/JYK3FGERlWqtNnnRtbm1bB+Suu\nuCIQ28Ftymk7tMVarZZIFtCcIS2wLO+8PU+eTuo4jlMxPpk6juOUQCUeEKnyeWK4VAmpiY1ahXQC\nOjbr2FDoxSuvvAIkW11IJZRDIKvaTdVVhmq1WsK4LvVXhno5kJ5++uloG4V23XvvvUBSnZWaKcej\nQlg0duiMWgQK1pfqLqeKdWLsuuuuQKzuWceLwo30V/dCO01RXV1dCTVfx60wIFUAUx1eiFV/qcr2\nflVtU4Un6lzZNtntqGfa3d2dUPN1XHrm/vznPwPJugFyNGZVx9J4dA5kqrKVxcaMGeNqvuM4TpUE\nzUhEQRDMAl4cdMPWMDEMwyUH32x4+Bgr4ZMwTh9jiXwcxtnUZOo4juNk42q+4zhOCfhk6jiOUwI+\nmTqO45SAT6aO4zgl4JOp4zhOCTQVtD9hwoSwaBVyGyWgYFgFvw4lb3vmzJn09va2PBm4mTGWTSeO\n0dZyTOfcCBi1AAAdc0lEQVTf2yB8XdMiwdwzZszorSKkxq9lP3oW7TOpz3omhxKEX9UYoTXXUve2\n6mrofrZJNbVarfA4m5pMe3p6uP/++3O30YP2+OOPR8uUUaBC0EPBlu1rJVlj1CSik51146l03R13\n3AHA4osvHq1bbbXVgMHLgLVzjGlUds1mzahIsK6nLZOoMm82WwiSD7Ae3CAIKokXLDLOVlHltbzv\nvvvakoFV1RihNddSWXr//d//DcTP9e677x5tM2nSpMLjdDXfcRynBHwydRzHKYHSC51IFFcFc4Dd\ndtut7J9pObYIiMT/tCplO7CeeOKJAPz6178GkoUj1Pvqq1/9aisOtVRUsV2dR61qtdVWWwFxd1Jb\nAKJRMQjbc6eTiteoy4Dqmaq6PsSFMlQP1Jo61P9r3333reQ4neGhojzqDKyeUBCbplQw5frrrwdg\nkUUWibZZbrnlvAeU4zhOlZQmmapPzu233w7A5MmTy/rqtmDfRunoA61TSTqIy7bJAWMrt9sq5p3O\nxRdfDMCpp54KwHrrrRet23zzzYH4jf7SSy9F6+zb3GLPXdE3fBXcfffdADz55JNAsmeZyus9++yz\nAFx00UXRunPOOQeAZ555Bkh2lHA6A/Wag1grVgePo446asD2hxxyCBCX57v22mujdW+88UaiW2ke\nLpk6juOUQGmSqexKKpJ86KGHNtz2V7/6VfRZkk+7+yKlyYuFlT3VFodWP6Sdd94ZgKuvvjpaJ/ux\nCvJuuumm5R5sCbzwwgtAfG3Uy+q4446LtkmHtqXDoLKwdmbF87UT9bhS0WsVr95pp50GbKueXVOn\nTo2W6fwUGXtVpG356uGkELasYuTnnnsuAP/6r/8KJO3ZZ5xxBpAtxbUb68tIFxvX8/WHP/whWqZ5\nSOFOtoh2Gj2XDz30ULTs+eefT/SEysMlU8dxnBIYtmQqr6g8n2+//TYQ2ygs8n5PmzYtWnb66acD\ncNlllwGw+uqrD/eQWo6kVnWqhIGe6j333DP6fMQRRwCxx7gTJdO//e1vQGxLlLc671izJJ48dK+0\nkzfffBOIr52VOhuxxRZbRJ/lzdf3dArWrietIu/6SJqVfdjas9UGRBJZJ0Vh2OgQSamKyJA3fp99\n9om2kcah1iRF2HLLLaPP8gUVwSVTx3GcEvDJ1HEcpwSGreZL3VEo0Pjx4xtuO3HiRACmTJkSLTvz\nzDOBuLPgx0HNF4899lj0WepHVjGGww47DIDTTjsNgBtuuAGAHXfcscVHmI91CD344INArBoeeeSR\ng+7/1FNPRZ+V5yynTRadEBol9Xbdddcd1v4K9O4UrGNGqrvtsgkkHCk6foV2yTQAcQhcJ6n30H//\nWPOKQtdklpCzyXYgFQqXUsIJNK4tYB3LY8eOLVz3wCVTx3GcEhi2ZKpqSXoz2v7jjbCplnr72V7X\nnY7Ca+SsgXxJZf3110/8L0N5uyVT+5afNWsWABtssAGQdLqkkYRz5ZVXRssk0eZJpu3qLW8lYoU0\nNes861TCMGTBggWJ6lyNEijseVhhhRWA2Jna3R1PBapylsf7779fqaahcUoKhfg52njjjYFsiVRJ\nGXJSKeEkDxsWueSSSxYuGeqSqeM4TgkMWzJ97733gLh+pw0XaoQN45CUut122w33UFqObKQKyJed\nEGCttdYadH/ZU20RmHYQhiHz5s1LSKbSELbffvtB99db/uabb46WqQhKHu2SBq3UJru9QmWkXWy0\n0Ual/V5aUmw19Xo9UahF5zkd1D5mzJjoc7r+rH0mdS8olE1jef7556NtRowYUTjNsizq9Xri2bnl\nlluAWJvKQs+c7KBZ2pitywtJLbNWqxW2Hbtk6jiOUwI+mTqO45TAsNV8GaHlXFA9zDysurf//vsD\nsMwyywz3UFqO2o4ozMLWMz3rrLMa7qewKRn5bRhLOwjDkPnz5ydCozS2Nddcc9D9le1mq0Zl5ban\naUeoTb1eH6DuQlwl6rrrrgOSDhhbKWsodHd3N6zv2gpqtVoi7Ek1aJdaaikgVnXtMUm9L1Jj4JFH\nHgGSZq0i5rwyCcNwgOlE1djyqrIpPOwLX/gCELcogXjsu+yyS2KfoTpKXTJ1HMcpgWFLpksssUTi\nfwXS5rHDDjtEn23juU5HbzIZqB9++OFo3e9//3sgzmW30reC2z/zmc8AnSGF12q1RHCyJA1b17MR\nStDYZpttomUK9O40BpMQJYXedddd0bJrrrkGiO/tL3/5y9G6TrtfgyCgq6srkSyj+1IOQiUoqKJZ\nUSQJqm6v/Y2FF1648lC3MAwT2o2uXZbmkUbHbmvTKhwzLZmqcSTkV5lK45Kp4zhOCQxbMpU9Rra2\nGTNmAHDFFVdE2+yxxx6JfRQw/HHlvPPOA+AnP/lJtExvvixbkiQc2VjXWGONVh9iLkEQ0N3dnQgJ\nkc3Uhs+kkX1coShFwsHazWCSqdKXrc30f/7nf4DYVmgrr3/6058GYgle+xcN7G4FQRAkJDZdS0nb\nqtKmZxNg6623BmKp1WopCgG88847gXisthLc2LFjC0mEZRIEQUKr032o41B42EILLRRt88YbbwBw\nySWXALDOOutE60455ZTM32lGGrW4ZOo4jlMCw5ZMNYvLfqZAX1sERNKBJFL7htMbXVJSlh2mygDo\nIshm+u///u/RsjyPolLaNG6lv7ULSabW3i17rgKxdR2txKZrc8ABB1R1qJUxadKk6HMjiQUaB3p3\nEgrIVyKMAtWtvVBddNVpwNqC9bwqrVSJNdbzP2rUqEojFoIgYMSIEZFmAHF3DhVpUS1lK6UrKUPS\nugr6tAKXTB3HcUrAJ1PHcZwSKK2hntQkqYJWzVdlKakb1kAsB5YCyKVK2CpSnVrhJ6tKTRY6/nRD\nunYSBEHiOsjJoOugIHCr5judqd739fUlnEG6rnK26H9bv1MB+KqtYXP7ZY5S7U+p+dZJVavVKlfz\nu7u7E+FZMmfI9KJ6Cza0Sep9kWdVyTVDnW9cMnUcxymBoBnnThAEs4AXB92wNUwMw7DlYoGPsRI+\nCeP0MZbIx2GcTU2mjuM4Tjau5juO45SAT6aO4zgl4JOp4zhOCfhk6jiOUwJNBRFOmDAhzOoLD3ER\nDKUh2sIPZcSjzZw5k97e3pYHtuWNsSxUHNqel1qt1tYxyhGpvyryrdg7iK+x4vBs2l4zhT5mzJjR\nW4UXuMj9qvRZ64hVbG1ejK22TxdHh/7r2gnXUsem/9PHaBnKM1rVGKG55zKva6pdp9jcwcZedJxN\nTaY9PT1RFW+hQNkbb7wRiPPQ99xzz2ibyZMnF/6NZ555Bkg2tRo7dmwi4LiVZI2xbJQvPG7cuGjZ\nmmuuWekYp0+fnrixFKSverQKhLbdBO69914gDuq210iTqaoQKeA7iyAIKglxybqWCk5//PHHAejt\n7QWSDQGLdAT45z//CcTdBmwd2GWWWabya2lfBgrET7+0bTO62267DYCHHnoISAaqqxqYKmLp/3TC\nQlVjhOaeS927MPCYbVNA1eVVXQr9VXV+6J9wi47T1XzHcZwS8MnUcRynBIaUeG2bd/3xj38E4IQT\nTgDiFgBZqr1y820JvjTKt60y77coKjQr9diWAxOvvvoqEJc2y/se1SyA/nHn2Xpage17LnONrq3K\nmUllBNhxxx2BWKVdccUVo3UqQLz33nsDcYHlLbbYoiXH3gy2gaHMSI8++igQ36fNNvvTeZLqbHPj\nF1988crLRtpGlioHqWdIJg2p9BCfB5lxZHqy37XbbrsBcXlNXX9IXvt2I9X99ttvB5K1PxZZZBEg\ntn+rhRDE85DUfRWetnVBmsElU8dxnBIYkmQ6c+bM6PPZZ58NxF78X/ziFw33+6//+i8ATjrppIbb\nZLVCmDNnTuVSm0Vvs+nTpwNxa5IsyVQSXp5kKqO4lQZWXnnlhOe81YRhmPDu6s2tsUma22effQp9\n33HHHQfETim11N1ss82ibapuwCZmz54dfbYONYApU6YMur+VbHV/6q8kdxvNMH78+ErbeYdhmGjb\nrWORZKoiylZbVGFwVY8688wzo3WS2vWc33PPPUBc4S39uV288MILQNxAUE5EKzVrXL/61a8A+M//\n/M8B3yOn62uvvQYkm4TW6/XCWoZLpo7jOCUwJMnUNubSW+E3v/lNw+333XdfIG5QlieZZtGOtrL2\nTf/AAw8AsWR60EEHNdxPdmGFngAcffTRiW3UDvrSSy+Nlj3//PMJW3QrCcOQvr6+RDhMuv20DXtK\n8/LLLwPwta99LVomCfZLX/oSAK+88grQPmnUovAniCUx26a6Eb/97W+BZANItbSW9KJzqHMC/RqL\nNLUqCMMw8XtLLbUUUMzvIC3p5JNPHrBOWuYTTzwBJG2ua621VqWalLDhXbLr77777g23V+iamlja\ncEQ9q2rZIm3C/sayyy5b2H/T/jvdcRzn/wFNSaZhGDJ37lyee+65aJnezAcffHDD/eRN3GmnnQb9\njZtvvhlIeoGb9bSWgQ38le3IVvluxC9/+UsAbr311mhZWjIVtq3yU089lfDIdjKyC1955ZXRMgV2\nS0L7zne+U/2BZVCv1xM2U3lq89qNq6Ge7L7/8R//Ea2TZCokwT/99NPRsnfeeacym6m0DBtonpak\nJLVa21+RjLXDDz8cgB/96EdAMmLg9ddfT0SDVIWVhhsF09vrJWQztd0C0tx3331ArFVBvu8jjUum\njuM4JeCTqeM4Tgk0peb39fUxZ86chNpUpEmc+sQrlMjmxypIX1x77bVAsg99noG5FfT19SXGqCD7\nr371q4Pue8kllwDFGq9ZJ9drr71WmdqkHuRZyFkjlU6hUhaF2MgBYLG91duNVGCr3hZR244//ngg\nDhfLUw313Qqrgf6A/k7oYKH6AQpcz0LhRcpLz2KHHXYA4Oqrr46Wvfzyy5WH8s2bNy/XMSqz06mn\nnhotu+uuu4D8ayi0jU1UaQaXTB3HcUqgaQfU/PnzEw6hIhVVJk6cCMRv7zvvvDNal5ZMZci3Ek6V\nkmm9XmfevHkJB5SOe7XVVmu43+mnnw7EITN5VZOEJAfod2S0M4VWTiVpDeuvv/6g+9jgZhnt7bJO\noF6vJ6SSLEkbYueDRRqV7t8sdL7ef//9aNno0aMru5ZBENDV1ZUI+RG6JostthiQTDLR8cqZbMsN\nSvrTMt3LCg2E/nbKVSYmQP+1tI62NHIcbr311tGyjTbaaNDvlTamNNQ8ST4Pl0wdx3FKoCnJtFar\nMXLkSJZffvlo2ZprrjnofpI+FZS+xx57NNxWgf2y01RNEAR0d3dn1vrMQ4HS2267LVBMMn333Xej\nz5MnT84tRtxq9FZWuqUkUzv2Y489FoCf//znA/ZvJPG1m1qtlpA0bN1Ri1IoIZZEFeidJ6UrgcWG\nDS2yyCKZadFVI7u9gtOtH0Lhfrq+9plO2+4lZSv8DfpDo6ocYxAEg4ZIyh6uYjtF0TWUVrXeeusN\n4QhdMnUcxykFn0wdx3FKoCm9squri0UXXTSh9jRSgZUDDXErE6nAMohnoaygItV8WoHChqzaI9VQ\nNSCzwsGUsaVsryKhUdaU0NPT05ZMLyHDfjp0yKqvqtUq55x1sjRSnx977LHocxGTUJnoWhZxitk6\nBRtssAEQ11DIc3pIXbYhO+PHj6/UZNNI3dY9KOeUqiNB3IJGDigb2vXFL34RGFh32F7jBQsWVG6W\nauTUu/zyywE44ogjAJg6dWpT33vxxRcDccbXUHHJ1HEcpwSafrXUarWE1Gar5Vhs+NODDz4IwIkn\nntjwexU8rApT6RzoqllppZWizzJoyzmTJZmqepYkU7t/GiUmWGlmpZVWaqtkao/DYkPUJAGofsJN\nN90UrVM9U0kzqtZj6zhULZmKIhKUpFCIa0HkOTJU41aJC1YamjBhQmVSm8IVbdUoaTyqQyDJ2p5/\n1R1QULvNR5czVbVoVZPCSqbd3d0Jab6dyLGWpc3qeVRzQMsBBxyQ2MbeA0PBJVPHcZwSGNLr0/ZY\nSdfBFDbQvkgQrCSarIov7Ua9bxSgrTQ6+2ZWQLTegKrrmYXqQso2B/2hKZ3wppdUohRaa6fSNdp+\n++0Tfy2yoauvTjNVd9qJTafUPZ0XeK/URVWc33DDDaN1VSdfhGGYSBpQDdm8XkbqeqHqZlnPcbpK\nmk0fXWqppdoaymfZddddgVi6tmnOaYnUatK6hqpbO1xcMnUcxymBIb1arPewSGELpZzKdrruuusO\n2MZKu51KOvXVogDvvGB9dSNVeqPdtlMCvUUjjcNiC0LIRir7lVINi6Qbdxq2zmwadV2QJCjpPM9G\n3mpqtVri3tE1kI0/Lx1WETZFsJrTqFGjOqKLAsTRQXlRQuKYY46JPqtjRl4SUTN0xtlwHMf5mOOT\nqeM4TgkMSc23hmfVbVRwd1aAs0I1iqixUg87SeUtQl5gt/jb3/4GwCabbAIkQ6zaWTGqKDLUX3HF\nFQBst9120To5YhQgb9d1Irpvi5x3VdSCuMKQHE5yIha5/q0gq2qUnICqSqbc8yL1IvKwJr2urq6P\nxT2bZs8994w+77fffoNu30xdWpdMHcdxSiBoZuYNgmAW8GLrDieXiWEYDp6jOUx8jJXwSRinj7FE\nPg7jbGoydRzHcbJxNd9xHKcEfDJ1HMcpAZ9MHcdxSsAnU8dxnBJoKs50woQJoeIJB8M6tlQgQfFv\ntmCCYtVU9kwpa7avexAEzJw5k97e3pYHtuWN0RZzBkpPp+uEMeq6aay2+PeHH34IEJV7s7HASsHU\n9dM6G4uozzNmzOitwgvczP1q0fh0Dux1Llrco5OuZaviQasaIwz9WipuXefCzku6Rwd7jouOs6nJ\ntKenh/vvvz93GwXv2wrrClZXVXJbt1N5+sptV41PG4Q8cuTIynK8e3p6Ei1tIR5TeqIomyrHmG5t\n/MEHHwBx/dE77rgDgGuuuSba5vbbbwfiupg/+MEPonV77bXXoL9rkjcqCXFp5n697LLLomXKu1fl\nL9squihVXsv0GFUzoUjdjCyuuuoqIK6434gq6y4UuZZCHSEgvpa6v+2LRVWxVNlME68V9saOHVt4\nnK7mO47jlIBPpo7jOCVQWnVXichqOnfppZdG69QWQEVcbeHkRjnNtsd3EARN5cgOlyAIEr3Di6j3\njzzyCBAXRbYtSaQWN2o61w6CIEioMyoGLROH2lk88cQT0TZrr702ALvssguQ3Qoij05KEFGR4B/+\n8IdA0iyle3co6n27sI0Prb+hKHb8u+22GwAHHnggUF7x5Fajc/D6669Hy/Tsap2eT4jLTKp4vcwi\ndu7RvFYEl0wdx3FKoDTJVK0CZLxW0zWIK7V85StfGdJ3t7tCTVoivfvuuwH43e9+Fy2TRCfJ3Baq\nVRtoNfw65JBDgPZKPmEYMnv27Oh/VUZS+w41G7NjTHuyrTRz6qmnAvBv//ZvQHbLjE6oBPbuu+8C\ncM455wBwwQUXAHFDQIDlllsusY+VTjphDFmoQhTErWeEnDDf+ta3omVqs/ONb3wDSDbb++lPfwrE\nz7Ke7XQbk05DGrBtaa2C7jo/b775ZrRO11KOUc0x1nE3Z86cAVE8jXDJ1HEcpwSGJJlmxZAq1Gba\ntGkATJo0KdrmhBNOaPo3bOuIqtsjhGGY+E1JbZLEZB+dPHlytM3+++8PxNLb008/Ha3TOdJfhR3J\n9tgOwjBMSFxLLtkf9ik7aBGp2Uozp512GgDHH388AD/72c9KO9bhYu/Xv/71rwBcf/31AKy66qpA\nLFlbFBYmKRZiSUdty9NSYDsIwzC3nqpaddjQIjWutHZzceyxxwKx3VzthpppcdIOLrzwQgBee+21\naJmkcdUQtpqFwqVsLDUk55t58+a5ZOo4jlMlw7aZzpo1C4Abb7wRiL3CeR5Aa09VsH66JbB9O3z4\n4YeF3w5loSwYiFvISsKRtzNtW7Osv/76A5bJRimPom07u/zyyw/ziIsjqdRK/8OVsGRj1XW/5557\ngGQL5HahexTgL3/5CxBLZLLxZqFtbGSH7gXZIdstmYZhyIIFCzJt1Hpm9Ex+//vfj9bZVuyNWGGF\nFQB45513yjjUlnH55ZcDsQZx+OGHR+s23njjxLa2mafmLD2HNgJHjBgxorC/xiVTx3GcEvDJ1HEc\npwSaVvPDMEyoPTL23nvvvUDslFlvvfUG7CtDrxqyATz88MNAbPTOopnwhLKQGmdRAzyFlTSLnDoK\nkVLISTsIw3BIwd2NUP94FUMp0sO8CsIw5K233or+l8qv+1OJJFlIRZSzCmKVUM6LTiEvIUImpGab\nHMopudpqqw39wCrg/PPPT/wvk1MW9p5XOKOcxlmmue7ublfzHcdxqmRIDigbSvDii/0FgJSCtffe\nezfcT+FT1rmkSkT77rsvEBu9LVUH7Muon/59pZ8NF70dbThLldJ3EARRmt1QUKiY2jpb5HBaeuml\nh/z9ZRGGIfPmzUukF0qCU4vmrPtNyCFhg7hfeuklIHvs7SAIAmq1WuJ+laSl50xagkL6YKBjJgs5\nh20InHj11VcTGmo7UIroo48+CsBaa60FxEkyWdjkhgceeABIOqXSjB49unBopkumjuM4JdC0ZBoE\nQWQXg7hGqcKElCqZhUKDJNlAHJ6gNL8sSWHevHmVFslIB+0LvQkXWmihUn7HSty9vb0J6aLVDEfa\nzzvOPInUSgVVIA3Dps2qwLCC9Ytgba4rrbQSMDCUr510dXXlSolTp04FkrVpJa3mFavZaaedGq6r\n1+ttL1yjuqUaS5afJo3SvSH2i+Rpac0kDLlk6jiOUwI+mTqO45TAsDOgpPLLOZOXEaJ2JVbUFqqV\nmcX777/fVF3B4RCGIfV6PSHeKxRKY5V6YNuvKJvILhsMm4HUaVkmMurbFi5ymG211VaFv0fZN1C9\nmg/95gxbZ0BOJV1L1VvIcrJce+21QNL09N3vfrdlxzoc8kwvcsjYVjUKSVxllVWAZM66rpPqfArr\nIG1nFbc0CsfMcyQJjRviMSvkcbi4ZOo4jlMCQ5JMrcFWUps18jdClebtm/6II44YdL9mwhPKwkoz\n6Yo8ckhYR5rCLGQUtwHSjYz8tj7ouHHjKh1jGIa50oWqI33961+Plsl5s8022xT+HSuZZuWPt5Ig\nCOjq6krkXOs+1XEppM/WRpAzRwH+J598ciXHO1T6+voS95K0uHTtVZsYo+QD1Sy19QuEpNbNNtsM\nSFbzr9VqbZdOpQXuvPPOAEycOHHQfawzUePT3+HikqnjOE4JDEkytZKaZnW9zVXlWiEkFr3ZDjvs\nsGiZ3ippbKD1oosuWlmF8yAIGDFiRG6qpezCtqK3UtpuvfVWAA466KBonaqZSyKXhJZuKVvVGFU1\nykrCaalYx6iKPAB33nknUCwgXxXNbchO1SmmSk7IqgYk6U2hfbbVszSvTTfdtOF39/b2AvGY0l0I\nqqSrqyvhU5DNU+3Ss45NErrasNsUaSU0pMdvtc/Ro0e3XTJViKJspnn+Cknidgyqz5q2DQ8Vl0wd\nx3FKYNg20xVXXBGI0/QUfJ+FbG6f//znB/0NKymNGzeuI3vv2JQ8VZqXp99KQ0pHzJJIxciRIyt7\n0wdBMKADa6O3+uKLLx59LlIDUzbIF154AUimYraj51UQBAnJQ/eRIil031r7tyQ6Sdc2fVrj0nlR\nBXcbwN4Oic0+kzoWSajywttkG92Dqs1ro2nsNbPYMbbrmbQRBXqedL3ytElFEEnqhqT2WAYumTqO\n45SAT6aO4zglMCQ136rgCneS+C0Drw3QlppVpMXDe++9l9gn/Xudim2uNxhSMdsZBF2r1QY0DoP8\nPOUiqIqYVHrrrBrudw8Vq/7puOS8kHPGhszoOKXKW9OV2h3LwdpuJ0wWMtlIDdczqWcVmmuTo7q7\n1kzTTDuPMrH3rK6dnIAyx9gEGC1T2NTWW2/dumNr2Tc7juN8ggiaqfwSBMEs4MXWHU4uE8MwXLLV\nP+JjrIRPwjh9jCXycRhnU5Op4ziOk42r+Y7jOCXgk6njOE4J+GTqOI5TAj6ZOo7jlIBPpo7jOCXg\nk6njOE4J+GTqOI5TAj6ZOo7jlIBPpo7jOCXwf+1R9NvS3jKEAAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot_conv_layer(layer=layer_conv2, image=image2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Close TensorFlow Session" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We are now done using TensorFlow, so we close the session to release its resources." + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": {}, + "outputs": [], + "source": [ + "# This has been commented out in case you want to modify and experiment\n", + "# with the Notebook without having to restart it.\n", + "# session.close()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "This tutorial showed how to use the so-called *Layers API* for easily building Convolutional Neural Networks in TensorFlow. The syntax is different and more verbose than that of PrettyTensor. Both builder API's have advantages and disadvantages, but since PrettyTensor is only developed by one person and the Layers API is now an official part of TensorFlow Core, it is possible that PrettyTensor will become deprecated in the future. If this happens, we might hope that some of its unique and elegant features will become integrated into TensorFlow Core as well.\n", + "\n", + "I have been trying to get a clear answer from the TensorFlow developers for almost a year, on which of their APIs will be the main builder API for TensorFlow. They still seem to be undecided and very slow to implement it." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercises\n", + "\n", + "These are a few suggestions for exercises that may help improve your skills with TensorFlow. It is important to get hands-on experience with TensorFlow in order to learn how to use it properly.\n", + "\n", + "You may want to backup this Notebook before making any changes.\n", + "\n", + "* Change the activation function to sigmoid for some of the layers.\n", + "* Can you find a simple way of changing the activation function for all the layers?\n", + "* Add a dropout-layer after the fully-connected layer. If you want a different probability during training and testing then you will need a placeholder variable and set it in the feed-dict.\n", + "* Plot the output of the max-pooling layers instead of the conv-layers.\n", + "* Replace the 2x2 max-pooling layers with stride=2 in the convolutional layers. Is there a difference in classification accuracy? What if you optimize it again and again? The difference is random, so how would you measure if there really is a difference? What are the pros and cons of using max-pooling vs. stride in the conv-layer?\n", + "* Change the parameters for the layers, e.g. the kernel, depth, size, etc. What is the difference in time usage and classification accuracy?\n", + "* Add and remove some convolutional and fully-connected layers.\n", + "* What is the simplest network you can design that still performs well?\n", + "* Retrieve the bias-values for the convolutional layers and print them. See `get_weights_variable()` for inspiration.\n", + "* Remake the program yourself without looking too much at this source-code.\n", + "* Explain to a friend how the program works." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## License (MIT)\n", + "\n", + "Copyright (c) 2016-2017 by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n", + "\n", + "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n", + "\n", + "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n", + "\n", + "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE." + ] + } + ], + "metadata": { + "anaconda-cloud": {}, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.2" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} From 6dd31008c3c960da94c29305471f74bd42a18e96 Mon Sep 17 00:00:00 2001 From: Daryl Van Dyke Date: Wed, 18 Apr 2018 17:59:59 -0700 Subject: [PATCH 2/3] now just helper functions ... --- 03B_Layers_CannGrows.ipynb | 773 ++++++++----------------------------- 1 file changed, 168 insertions(+), 605 deletions(-) diff --git a/03B_Layers_CannGrows.ipynb b/03B_Layers_CannGrows.ipynb index 4e2f4b1..fd35e54 100644 --- a/03B_Layers_CannGrows.ipynb +++ b/03B_Layers_CannGrows.ipynb @@ -73,9 +73,18 @@ }, { "cell_type": "code", - "execution_count": 51, + "execution_count": 1, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\h5py\\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n", + " from ._conv import register_converters as _register_converters\n" + ] + } + ], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", @@ -96,7 +105,7 @@ }, { "cell_type": "code", - "execution_count": 85, + "execution_count": 2, "metadata": {}, "outputs": [ { @@ -105,7 +114,7 @@ "'1.7.0'" ] }, - "execution_count": 85, + "execution_count": 2, "metadata": {}, "output_type": "execute_result" } @@ -116,12 +125,14 @@ }, { "cell_type": "code", - "execution_count": 86, + "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "# Class for Image stuff\n", - "# `take` from :\n", + "# Daryl Van Dyke GeospatialDaryl@github.com\n", + "## `take` from :\n", + "# \n", "#https://stackoverflow.com/questions/7971618/python-return-first-n-keyvalue-pairs-from-dict\n", "from itertools import islice\n", "\n", @@ -159,6 +170,11 @@ " outKeys.append(key)\n", " outVals.append(newDict[key])\n", " return TF_ImageDataset(outKeys, outVals, datasetName)\n", + " \n", + " def n(self):\n", + " lenSamp = len(self.dictDS)\n", + " self.n = lenSamp\n", + " return lenSamp\n", " \n", " " ] @@ -182,7 +198,7 @@ }, { "cell_type": "code", - "execution_count": 87, + "execution_count": 21, "metadata": {}, "outputs": [ { @@ -244,7 +260,7 @@ }, { "cell_type": "code", - "execution_count": 100, + "execution_count": 22, "metadata": {}, "outputs": [], "source": [ @@ -276,7 +292,7 @@ }, { "cell_type": "code", - "execution_count": 107, + "execution_count": 23, "metadata": {}, "outputs": [], "source": [ @@ -302,7 +318,7 @@ }, { "cell_type": "code", - "execution_count": 108, + "execution_count": 24, "metadata": {}, "outputs": [ { @@ -321,7 +337,7 @@ }, { "cell_type": "code", - "execution_count": 109, + "execution_count": 25, "metadata": {}, "outputs": [], "source": [ @@ -331,33 +347,7 @@ " image_string = tf.read_file(filename)\n", " image_decoded = tf.image.decode_jpeg(image_string, channels=3)\n", " image = tf.cast(image_decoded, tf.float32)\n", - " return image, label" - ] - }, - { - "cell_type": "code", - "execution_count": 45, - "metadata": {}, - "outputs": [ - { - "ename": "TypeError", - "evalue": "Tensor objects are not iterable when eager execution is not enabled. To iterate over this tensor use tf.map_fn.", - "output_type": "error", - "traceback": [ - "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[1;31mTypeError\u001b[0m Traceback (most recent call last)", - "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 14\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mdataset\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 15\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 16\u001b[1;33m \u001b[0mdataset\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtrain_input_fn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mimages\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlabels\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m100\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", - "\u001b[1;32m\u001b[0m in \u001b[0;36mtrain_input_fn\u001b[1;34m(features, labels, batch_size)\u001b[0m\n\u001b[0;32m 6\u001b[0m \u001b[1;34m\"\"\"An input function for training\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 7\u001b[0m \u001b[1;31m# Convert the inputs to a Dataset.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 8\u001b[1;33m \u001b[0mdataset\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mDataset\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfrom_tensor_slices\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdict\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfeatures\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlabels\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 9\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 10\u001b[0m \u001b[1;31m# Shuffle, repeat, and batch the examples.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\u001b[0m in \u001b[0;36m__iter__\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m 398\u001b[0m \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mcontext\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mexecuting_eagerly\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 399\u001b[0m raise TypeError(\n\u001b[1;32m--> 400\u001b[1;33m \u001b[1;34m\"Tensor objects are not iterable when eager execution is not \"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 401\u001b[0m \"enabled. To iterate over this tensor use tf.map_fn.\")\n\u001b[0;32m 402\u001b[0m \u001b[0mshape\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_shape_tuple\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;31mTypeError\u001b[0m: Tensor objects are not iterable when eager execution is not enabled. To iterate over this tensor use tf.map_fn." - ] - } - ], - "source": [ - "# step 4: create iterator and final input tensor\n", - "iterator = dataset.make_one_shot_iterator()\n", - "images, labels = iterator.get_next()\n", - "\n", + " return image, label\n", "def train_input_fn(features, labels, batch_size):\n", " \"\"\"An input function for training\"\"\"\n", " # Convert the inputs to a Dataset.\n", @@ -367,52 +357,63 @@ " dataset = dataset.shuffle(1000).repeat().batch(batch_size)\n", "\n", " # Return the dataset.\n", - " return dataset\n", - "\n", - "dataset = train_input_fn(images, labels, 100)" + " return dataset" ] }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 26, "metadata": {}, "outputs": [], "source": [ - "#from tensorflow.examples.tutorials.mnist import input_data\n", - "#data = input_data.read_data_sets('data/MNIST/', one_hot=True)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The MNIST data-set has now been loaded and consists of 70,000 images and associated labels (i.e. classifications of the images). The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial." + "# step 4: create iterator and final input tensor\n", + "iterator = dataset.make_one_shot_iterator()\n", + "images, labels = iterator.get_next()\n", + "#Now we can run directly sess.run([images, labels]) without feeding any data through placeholders." ] }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "Size of:\n" - ] - }, - { - "ename": "AttributeError", - "evalue": "'BatchDataset' object has no attribute 'train'", - "output_type": "error", - "traceback": [ - "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[1;31mAttributeError\u001b[0m Traceback (most recent call last)", - "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Size of:\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"- Training-set:\\t\\t{}\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtrain\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlabels\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 3\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"- Test-set:\\t\\t{}\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtest\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlabels\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"- Validation-set:\\t{}\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mvalidation\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlabels\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;31mAttributeError\u001b[0m: 'BatchDataset' object has no attribute 'train'" + "Total Samples: 5083\n", + "N: \n", + " train = 3049 \n", + " test = 1016 \n", + " val = 1018 \n", + "\n" ] } ], + "source": [ + "Train = 0.6\n", + "Test = 0.2\n", + "Val = 0.2\n", + "\n", + "n = len(objBoth.listLabels)\n", + "print(\"Total Samples:\",n)\n", + "\n", + "nTrain = int(0.6 * n)\n", + "nTest = int(0.2 * n)\n", + "nVal = n - nTrain - nTest\n", + "print(\"N:\",\"\\n\", \"train = \", nTrain,\"\\n\", \"test = \", nTest, \"\\n\", \"val = \", nVal, \"\\n\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The MNIST data-set has now been loaded and consists of 70,000 images and associated labels (i.e. classifications of the images). The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, "source": [ "print(\"Size of:\")\n", "print(\"- Training-set:\\t\\t{}\".format(len(data.train.labels)))\n", @@ -428,10 +429,8 @@ ] }, { - "cell_type": "code", - "execution_count": 5, + "cell_type": "markdown", "metadata": {}, - "outputs": [], "source": [ "data.test.cls = np.argmax(data.test.labels, axis=1)" ] @@ -452,24 +451,35 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 46, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(?, 64, 64, 3)\n" + ] + } + ], "source": [ - "# We know that MNIST images are 28 pixels in each dimension.\n", - "img_size = 28\n", + "print(dataset.output_shapes[0])\n", + "theShapes = dataset.output_shapes[0]\n", + "w = theShapes[1]\n", + "h = theShapes[2]\n", + "c = theShapes[3]\n", "\n", "# Images are stored in one-dimensional arrays of this length.\n", - "img_size_flat = img_size * img_size\n", + "img_size_flat = h * w\n", "\n", "# Tuple with height and width of images used to reshape arrays.\n", - "img_shape = (img_size, img_size)\n", + "img_shape = (w, i)\n", "\n", "# Number of colour channels for the images: 1 channel for gray-scale.\n", - "num_channels = 1\n", + "num_channels = 3\n", "\n", "# Number of classes, one class for each of 10 digits.\n", - "num_classes = 10" + "num_classes = 2" ] }, { @@ -488,7 +498,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -530,20 +540,9 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAD5CAYAAAC9FVegAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHitJREFUeJzt3XmUFNXZx/HvA0IQEBVBQcWZE3CBEAXF4C5RIIoKSFwwLrzGaESDWwJG464xSlB4RU9YjMgJMSoKiEZFAUV82SMoiBuIKBKXEUIUERHu+8f07aqe6dl6uqp6xt/nHM9Ud1VXPeOl7zxVdzPnHCIi33cNkg5ARKQQqDIUEUGVoYgIoMpQRARQZSgiAqgyFBEBVBmKiACqDEVEAFWGIiIA7FSTg1u1auWKi4sjCqXwfPDBB5SUlFjSccRJZVz/qYyzq1FlWFxczJIlS3KPqo7p1q1b0iHETmVc/6mMs9NtsogIqgxFRABVhiIigCpDERFAlaGICFDD1mSRXI0YMQKALVu2APDGG28A8MQTT5Q7dvDgwQAceeSRAJx//vlxhCjfc8oMRURQZigRO/vsswGYPHly1v1m5fvCjhkzBoCZM2cCcPzxxwOw3377RRGiJOjdd98F4MADDwTgvvvuA2DIkCGxx6LMUEQEZYYSAZ8NQsUZ4UEHHQTASSedBMD777+f3jd9+nQAVq1aBcCkSZMAuP766/MfrCRq6dKlADRoUJqX7bPPPonFosxQRARlhpJHfrzr1KlTy+3r3LkzEGR9rVq1AqB58+YAfPvtt+lju3fvDsDrr78OwBdffBFRxJK0ZcuWAcG/gwEDBiQWizJDERFiyAx9P7Lx48cDsPfee6f3NWnSBIBzzz0XgDZt2gDQoUOHqMOSCPz73/8GwDmXfs9nhDNmzACgbdu2WT/r+yECvPXWWxn7Tj311LzGKclbvnw5AKNHjwbgggsuSDIcQJmhiAgQQ2Y4dOhQoHSCxYr4fmUtWrQAoFOnTnm5drt27QAYNmwY8P2cuy5Op512GhC0AgPssssuALRs2bLSzz722GPp7fDzQ6mf3nnnHQA2b94MZPZASIoyQxERVBmKiAAx3CY/+OCDQNBNInwLvHLlSiDoePnyyy8DsGDBAiAYfvXhhx9WeP5GjRoBQVcN/xA/fB5/u6zb5HgUFRVV+9g///nPQDAsK8x3sfE/pf4YPnw4ULoEARTGd1OZoYgIMWSGJ554YsbPMD8Uy9u4cSMQZIr+r8XixYsrPP8PfvADIBjo7Yd5AWzYsAGA9u3b5xS7ROeZZ54B4KabbgJg69at6X177bUXAHfddRcATZs2jTk6iUK4EdV/p/33tlmzZkmElEGZoYgIBTYcb/fddwfghBNOyHg/W1ZZ1pNPPgkE2SXAwQcfDMDAgQPzFaLkiR+6F84IPd/Nwk/dJfXDnDlzyr3XunXrBCLJTpmhiAgFlhnm4rPPPgPgsssuAzKHgvnnUVV1+JX49O/fHwiG53mDBg1Kb99xxx2xxiTx8Es9hPkBEYVAmaGICPUgM3zggQeAIEPcbbfd0vt8S5Ukz/f/nDdvHhA8K/TPjG644Yb0sX46J6kf5s+fD8CECRPS73Xt2hWAXr16JRJTNsoMRUSow5nhq6++CgR90bynnnoqve2nj5Lk+Uk7S0pKMt7307epL2j9NWvWLCCzp4fvY+yn8SsEygxFRFBlKCIC1OHb5GeffRYI5r7r2bMnAEceeWRiMUl5fs0TP8TS69GjBwC33XZb3CFJzPwkLWFnnnlmApFUTpmhiAh1MDPcsmULAM8//zwQTNRw6623AsGUXpKc8Gp2d955J1B+9uouXboA6kZTn33yyScAzJ07F8icROX0009PJKbKKDMUEaEOZoZ+MlD/DOrkk08G4KijjkosJsl0zz33pLcXLVqUsc8Px9Ozwvrv4YcfBuDTTz8Fgu9qoVJmKCJCHckM/USgALfffjsAu+66KwA33nhjIjFJxe69994K9/nhk3pWWP+tXbs247Wfoq9QKTMUEaHAM0PfKnnFFVek3/vuu+8A6NOnD6B+hXWNL9PqtPr77N8fu23bNgA2bdpU7lg/1GvkyJFZz9WwYcP09t133w1oOYGoPf300xmvTz311IQiqR5lhiIiqDIUEQEK9DZ5+/btQDCzxZo1a9L7OnToAAQNKVK3+HVpquOss84CoG3btkDQRePRRx+tVQx+9b3wHIqSP76TtS+vukKZoYgIBZoZrl69GghWUAvz3TY0/13h8o1bANOmTcv5PI8//niVx/jGlQYNMv+u9+3bFwjW3g475phjco5JqjZ16lQgaOz0s1oX+mqHygxFRCiwzNB30uzdu3fG+yNGjEhvF3rzvMCUKVPS28OHDwfKT9TgrVy5Eqj8OeBFF10EQFFRUbl9P//5zwHo2LFjbsFK3nz99dcAPPfccxnv++m6wt2bCpEyQxERCiwzHDt2LFB+GE/4WYOZxRqT1E5118V95JFHIo5Eouaf3/oVKvv16wfAlVdemVhMNaHMUESEAskMfb+k+++/P+FIRCRXPjP06yTXNcoMRUQokMzQr4H85ZdfZrzvR5touicRiZoyQxERVBmKiAAFcptcll85bdasWQC0bNkyyXBE5HtAmaGICAWSGV533XUZP0VE4qbMUEQEMOdc9Q82+xxYW+WB9UeRc6510kHESWVc/6mMs6tRZSgiUl/pNllEBFWGIiJAxK3JZrYHMCv1sg2wHfg89fonzrnsM37W7pqdgPB8UO2B65xzmgUiAgmVcREwEdgTcMBfVL7RSaKMU9edCPQBPnbOdYniGhnXi+uZoZndAnzlnBtR5n1LxbEjgmvuBKwHDnXOrcv3+SVTXGVsZnsDezrnlplZC2ApcLJz7t18nF8qFuf32MyOB7YA4+KoDBO5TTazDma2wszGAK8B7czsP6H9A83swdT2XmY2xcyWmNkiMzuiBpfqDbylijB+UZaxc269c25Zavu/wNvAPtH9NpJN1N9j59wcYENkv0AZST4z7AT81TnXFfi4kuPuA4Y757oBZwH+f273VCFUZiDwj3wEKzmJvIzN7IdAZ2BxfkKWGorjexyLJEegrHbOVecfcE/gwNB0/7ub2c7OuYXAwoo+ZGZNgFOAa2odqeQq6jJuATwJDHHOfVXraCUXkZZxnJKsDDeHtncA4cVNmoS2jdwe0p4CLHTOleQYn9ReZGVsZo2BKcDDzrnptYpSaiPq73FsCqJrTeqh60Yz29/MGgCnh3bPBC73L8ysug9Sz0G3yAUjn2Wcelj/MLDMOfe/EYQrOYjoexybgqgMU64Fnqe0CT/c4HE5cLSZvWFmK4GLofJnDWbWHPgpMC3akKWG8lXGx1P6x66XmS1L/feziGOX6snn93gyMBfoZGbrzOx/ogxcw/FERCiszFBEJDGqDEVEUGUoIgKoMhQRAVQZiogANex03apVK1dcXBxRKIXngw8+oKSkxKo+sv5QGdd/KuPsalQZFhcXs2TJktyjqmO6deuWdAixUxnXfyrj7HSbLCKCKkMREUCVoYgIoMpQRARQZSgiAqgyFBEBkp3ctUKbN5fOFzl06FAAxowJZvjxzeSTJ08GoKioKOboRKQ+UmYoIkKBZobr168HYPz48QA0bNgwvc93Fn366acB+M1vfhNzdJKL1157DYABAwYApaMCcvXCCy+ktzt27AhAu3btcg9OEuO/x3379gVg9OjRAAwePDh9TPj7HyVlhiIiFFhm+PnnnwMwaNCghCORfJsxYwYAW7durfW5pk8P1n966KGHAHj00UdrfV6JzxdffAFkZoAAQ4YMAeCiiy5Kv7fzzjvHEpMyQxERCiQzvO+++wCYNq10/abFi6tehnXu3LkA+DVcDjnkEACOO+64KEKUHH333XcAPPvss3k7Z3jg/b333gsEPRCaNWuWt+tIdF555RUAPv44c935c845B4AmTZqU+0zUlBmKiFAgmeFVV10F1KzVaMqUKRk/99tvPwAef/zx9DGHHXZYvkKUHL300ksAzJs3D4Brr7221ufcsGFDevvNN98E4OuvvwaUGRay8PPiO+64I+sx559/PgClS2PHS5mhiAiqDEVEgIRvk/v06QMEjSDbt2+v8jOtWrUCgtuhtWvXArBmzRoADj/88PSxO3bsyF+wUm3Lly9Pbw8cOBCADh06AHD99dfX+vzhrjVSd7zxxhvpbd8J39tpp9Kq6OSTT441pjBlhiIiJJAZzpkzJ7399ttvA8HD0ooaUC699NL0du/evQHYddddAZg9ezYAf/zjH8t97i9/+QtQvmOnRCtcFr5hY9KkSQA0b9485/P6hpPwv6EkHrRLbnxjZza9evWKMZLslBmKiBBjZugH5vtnSAAlJSVZj/XdZM444wwAbr755vS+pk2bZhzrp/AaO3ZsuXMOGzYMgG+++QYIJnVo1KhRbr+EVOqJJ54AMjtY+2eF4We5ufLdMcLZYI8ePQDYbbfdan1+iVY4o/caN24MwJ133hl3OOUoMxQRIcbMcNu2bUDF2SAEQ+kee+wxIGg5rozPDH0r5TXXXJPe54do+QzRTxPUvn37GsUu1eMn3PX/3yE/z2v9XcUjjzwCBC2PADfccAOgbL+Q+Q738+fPL7fP3+l16dIl1piyUWYoIkKBDMfzz5MmTJgAVC8jLMtnfX//+9/T7y1atCgP0UlVNm3aBMCCBQvK7bvssstqff5x48YBwRRvnTp1Su874YQTan1+iVZlE68UUk8PZYYiIiSQGWYbZbJw4cJan9ePYgmPOik7ssW3Svs+b5IffgD+unXrgGAapnxZvXp1xuvOnTvn9fwSrWyZoW/9z8edQ74oMxQRQZWhiAgQ422yX/s4qpWu/CpbS5cuTb9XdpjfrbfeGsm1v+922WUXIOgeEZ6owQ+ha9myZY3P+9lnnwFBlx3v6KOPzilOiderr74KBF2iwvxw2n333TfWmCqjzFBEhBgzw2eeeSav5/PdLFauXAlUPpzHd9VRx9xo+NXL/NA7PywP4JRTTgEyO8Nns2LFivS2bzDx07OVnYyhQQP9Da8L/Ap4viEzrBAmZihL/6pERCiQTte58NNEPfDAAxUeU1xcDMDEiROBYAIIicYtt9wCZGYC/o4gPEFHNq1bt05v+0ywoqGbF154YW3ClJiUfdYbnkzjkksuiTucKikzFBGhDmaGfqkAPzFsZfywrWOPPTbSmKRUx44dgcwVCn3rftmO02X56drCBg0aBJTvJO+fUUph8p3vy7Yih1uO8zGlW74pMxQRIcbMsLJFn5577rmM1xdffDEA69evr/A81ZnuPd8t2FJzXbt2zfhZEz/84Q+zvh/ux/jjH/84t8AkMn7KrrKtyP369UsinGpTZigigipDEREgxttkP2+Zn3U6zHfMLTtUL9vQPX+bXZ2V9KRu87dZZW+3dGtc2Hxna88PerjqqquSCKfalBmKiBBjZjhgwAAAhg8fnn6vsvVQquL/2vjuHOPHjwegbdu2OZ9TCotvJNPayHXLjBkzMl63a9cOCCZnKFTKDEVEiDEz9KvY+ZXvAKZNmwbAqFGjany+P/zhD0CwFrLUP369a0+drQubXwFz1apVGe83adIEKPyJUpQZioiQwHA8vzZyeLt3795AsAqan6j1tNNOA+DXv/51+jO+ZTG8QprUT361RD/A/6abbkoyHKmCn1rND7V78803Adh///0Ti6kmlBmKiFAgEzWcdNJJGT9FIMgwrr76akBrJBc63/fXT6/newEceuihicVUE8oMRUQokMxQJBv/7Fjqlr333huAhx56KOFIakaZoYgIqgxFRABVhiIigCpDERFAlaGICKDKUEQEAMu22n2FB5t9DqyNLpyCU+Sca131YfWHyrj+UxlnV6PKUESkvtJtsogIqgxFRABVhiIiQMRjk81sD2BW6mUbYDvweer1T5xz30Z03T7ASKAhMNY59+coriPJlXHq2jsBrwHvO+f6R3Wd77sEv8cTgT7Ax865LlFcI+N6cTWgmNktwFfOuRFl3rdUHDvydJ1GwDvAT4FPgCXAz51z7+bj/FKxuMo4dN5hQBegqSrDeMRZxmZ2PLAFGBdHZZjIbbKZdTCzFWY2htK/7O3M7D+h/QPN7MHU9l5mNsXMlpjZIjM7oorTHwG85Zxb65zbCjwO9Ivqd5HsIi5jzKwI6AVMiOp3kMpFXcbOuTnAhsh+gTKSfGbYCfirc64r8HElx90HDHfOdQPOAvz/3O6pQihrH+Cj0Ot1qfckflGVMcAoYCigvmHJirKMY5XkfIarnXOLq3FcT+DA0Nq5u5vZzs65hcDCLMdnW2RXX5hkRFLGZtYf+Mg5t8zMeuYvXMlBVN/j2CVZGW4Obe8gsxJrEto2avaQdh3QLvR6X2B9ThFKbUVVxkcBA8ysb+o8LcxsonNuUK2ilVxEVcaxK4iuNamHrhvNbH8zawCcHto9E7jcvzCzqh6kLgA6mVmRmf2A0pR8er5jlprJZxk754Y55/Z1zhUD5wEvqCJMXp6/x7EriMow5VrgeUqb8NeF3r8cONrM3jCzlcDFUPGzBufcNuAK4EVgJTDJOfdO1MFLteSljKWg5a2MzWwyMJfS5Gadmf1PlIFrbLKICIWVGYqIJEaVoYgIqgxFRABVhiIiQA37GbZq1coVFxdHFErh+eCDDygpKcnWibveUhnXfyrj7GpUGRYXF7NkyZLco6pjunXrlnQIsVMZ138q4+x0mywigipDERFAlaGICKDKUEQEUGUoIgKoMhQRAVQZiogAyU7uKiICwMaNGwH48MMPKzymqKgIgJEjRwLQuXNnAA444AAADjnkkFrFoMxQRISEM8PPPvsMgLPOOguAo446CoBLLrkEKO0pnw+bNm0C4JVXXgHgpJNOAqBRo0Z5Ob+I1MwzzzwDwNNPPw3Ayy+/DMB7771X4WcOPPBAoHR4HcDWrVsz9u/YUbtVSpUZioiQQGbonw0A/OhHPwKCzG2vvfYC8p8RHnrooQCUlJQApMdl7r///nm5jlTff//7XwB+//vfA/Dmm28CMHPmzPQxytjrh9WrVwPwwAMPADBu3Lj0vi1btgBQk5n233kn2tU7lBmKiBBjZuizMv98EOCLL74A4PLLSxfNGj16dF6veccddwCwZs0aIPjLpIwwfpMmTQLghhtuAMq3GvqMEWCPPfaILzCJzLp1petBjRo1qlbnOeigg4Cg9TgqygxFRIgxM3zttdeAoNUo7KabbsrbdVasWJHeHjFiBACnn166fOvZZ5+dt+tI9fjs4OqrrwaCOwSzzLk2hwwZkt6+//77AWjZsmUcIUoOfDlCkPkdc8wxQNBbo3HjxgDsuuuuADRv3jz9ma+++gqAn/3sZ0CQ9XXv3h2Arl27po/deeedAWjWrFmef4tMygxFRFBlKCICxHCb7DtWP/nkk+X2PfTQQwC0bt261tfxt8e9evUqt2/AgAEA7LLLLrW+jtSMf1ThG8sq8uijj6a3n3vuOSBobPG30P62S5KzefNmIPN79vrrrwMwbdq0jGOPPPJIAJYuXQpkdpnzDWj77rsvAA0aJJ+XJR+BiEgBiDwz/O1vfwsEXSt8B2iAM888M2/XefXVVwH45JNP0u9deOGFAJx33nl5u45Ube3atentCRMmZOzzg+l9B/sXX3yx3Od9Z3mfVZ577rkAtGnTJv/BSrV8++23APziF78AgmwQ4PrrrwegZ8+eWT+bbRDFfvvtl+cIa0+ZoYgIMWSGvguF/7nPPvuk99XmGZAfznPnnXcCwZCfcJcN/0xS4rVs2bL0tu9MfdxxxwEwZ84cAL755hsAHnnkEQD+9Kc/pT+zatUqIMjy+/XrBwTPEtXlJj6+C4z/nvmJFcLP+YcOHQpA06ZNY44uv5QZioiQwEQNfuoegN69ewOw2267ATB48OAqP+87bfufCxYsyNifz+eQkpvw1Eo+U/edrr0mTZoA8Mtf/hKAJ554Ir3PD/D3g/h9xqHW5Pj5FuK77roLCCZYnTt3bvoY36m6rlNmKCJCDJnhlVdeCcDs2bMBWL9+fXqff37kM4CnnnqqyvP5Y8sO52rfvj0QPNuQ5PzjH/8o994///lPAPr375/1M35atWyOOOIIIHM4l8Rj3rx5Ga/9MDnfP7A+UWYoIkIMmeFhhx0GwPLly4HMlsbnn38egOHDhwOw5557AjBo0KAKz3f++ecDcPDBB2e875cM8BmiJOecc85Jb/tsf/HixQC8/fbbQPDvYerUqUDmpL/+GbJ/z0+95su+U6dOkcUumcLPciFo0b/11lvT7/Xt2xfInFyhLlJmKCKCKkMREQCsJmsQdOvWzVX2oDsO77//PhDcDnfp0gWAF154AcjPpA9et27dWLJkiVV9ZP2RjzLesGFDetuXkx9iV1EDWHjgv+9Af+qppwLw7rvvAsGqiWPGjKlVfGEq48qVHTSRTcOGDQG49NJLgWBOwo8++giADh06AMGaR2F+DRw/qUMUDTPVLWNlhiIiJLxuci5uu+02IPhL5Rtf8pkRSu2Eh8tNnjwZgDPOOAMonyFeccUVANx9993pz/gO2X7qNT9Ub8aMGUDQKRvUYBa13/3udwDcc889FR6zfft2IMjo/c+a8I2nPXr0ADKndIuLMkMREepIZuizC4CJEycC0KJFC0ArqRU6P62T76LhJ2bw3Wd8pu+zwbAbb7wRgLfeegsIuun4z0Dw70Gi4Yfh+VUt/XRq27ZtSx/j17nxGWIu/CTQ/rseXgnPT/IbNWWGIiLUkczQd/QMO+WUU4DMyWKlcPkMsaIJQLPxq6L5VQ19ZvjSSy+lj/Et15rWKxq+pfjwww8Hgpb9sFmzZgFBtnjLLbcAsGjRohpfzz9L/te//lXjz9aWMkMREepgZujXTvWtXFL/+edV06dPBzJbGv0ay/lce1tq5sQTT8x47Yfc+sywUaNGQLAMB8DFF18MwMiRI4HgWXKSlBmKiKDKUEQEKPDbZD/sKrzinV9VTQ0n3x9+Td1hw4YBmevz+of1AwcOBOCAAw6INzgpx89g71fN8w0rfvYhgPfeew8IZqwvK7xWUlyUGYqIUEcyw/Ag8T59+mQc8+WXXwLB3HeFuB6r5IeflOP2229Pv+cb0q677jogWJ/bd8uR+HXs2BEIukQ99thj5Y4Jd48C2Gmn0qrId5kLD8+MizJDEREKPDPMxv8F8RmAb5r3w3c0PKv+u+CCC9LbY8eOBWDKlClA8Cyq7EzoEh+flY8aNQoI7t7CHak//fRTAIqLi4GgTP0z4CQoMxQRoQ5mhuPHjwfgwQcfBOBXv/oVEAzql/ovPF3bzJkzgWA9Xz+xQCF04v2+8z0//Frpf/vb39L75s+fDwSZoJ/CK0nKDEVEKPDMcPTo0QDcfPPN6feOO+44AAYPHgzA7rvvDkDjxo1jjk4Kge894JcN8EP2Vq5cCWglvULiVzcsu10olBmKiFDgmeGxxx4LwOzZsxOORAqdnzz2kEMOAWDVqlWAMkOpPmWGIiKoMhQRAQr8NlmkuvyaOGvWrEk4EqmrlBmKiKDKUEQEUGUoIgKA+dWoqnWw2efA2ujCKThFzrnWVR9Wf6iM6z+VcXY1qgxFROor3SaLiKDKUEQEiLifoZntAcxKvWwDbAc+T73+iXPu2wivvRPwGvC+c65/VNf5vkuqjM3sGuCi1MsxzrnRUVxHEi3jdcDG1PW2Oue6R3Gd9PXiemZoZrcAXznnRpR531Jx7Mjz9YYBXYCmqgzjEVcZm1kXYCJwBPAd8ALwS+ecelxHLM7vcaoy7Oyc+0++zlmZRG6TzayDma0wszGUZm/tzOw/of0DzezB1PZeZjbFzJaY2SIzO6Ia5y8CegETovodpHIRl3FHYL5zbotzbhvwCnB6VL+LZBf19zhuST4z7AT81TnXFfi4kuPuA4Y757oBZwH+f273VCFkMwoYCqipPFlRlfFyoIeZtTSzZsDJQLv8hi7VFOX32AGzzexfZnZRBcfkTZJjk1c75xZX47iewIGh5UJ3N7OdnXMLgYVlDzaz/sBHzrllZtYzf+FKDiIpY+fcCjO7F5gJfAUspfR2WeIXSRmndHfOrTezNsCLZvaWc25eHmLOKsnKcHNoewdgoddNQttGzR7SHgUMMLO+qfO0MLOJzrlBtYpWchFVGeOcGweMAzCz4cCqWsQpuYuyjNenfn5iZk8BPwEiqwwLomtN6qHrRjPb38wakPn8ZyZwuX+Renhe2bmGOef2dc4VA+cBL6giTF4+yzh1zJ6pn8VAP6D8SuUSq3yWsZk1N7PmfpvSNoAV+Y86UBCVYcq1wPOUNuGvC71/OXC0mb1hZiuBi6HKZw1SmPJZxtNSx04Dfu2c2xRh3FJ9+SrjtsD/mdnrlN5GT3XOzYwycA3HExGhsDJDEZHEqDIUEUGVoYgIoMpQRARQZSgiAqgyFBEBVBmKiACqDEVEAPh/EMZccjkjBQkAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "# Get the first images from the test-set.\n", "images = data.test.images[0:9]\n", @@ -596,7 +595,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -612,7 +611,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -628,7 +627,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -644,19 +643,9 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "WARNING:tensorflow:From :1: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version.\n", - "Instructions for updating:\n", - "Use the `axis` argument instead\n" - ] - } - ], + "outputs": [], "source": [ "y_true_cls = tf.argmax(y_true, dimension=1)" ] @@ -674,7 +663,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": null, "metadata": { "scrolled": true }, @@ -707,7 +696,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -723,7 +712,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -742,7 +731,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -758,7 +747,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -774,7 +763,7 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -791,7 +780,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -807,7 +796,7 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -823,20 +812,9 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 21, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "net" ] @@ -852,7 +830,7 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -871,20 +849,9 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 23, - "metadata": {}, - "output_type": "execute_result" - } - ], + "outputs": [], "source": [ "net" ] @@ -898,7 +865,7 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -915,7 +882,7 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -932,7 +899,7 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -948,7 +915,7 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -964,19 +931,9 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "WARNING:tensorflow:From :1: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version.\n", - "Instructions for updating:\n", - "Use the `axis` argument instead\n" - ] - } - ], + "outputs": [], "source": [ "y_pred_cls = tf.argmax(y_pred, dimension=1)" ] @@ -1010,7 +967,7 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1026,7 +983,7 @@ }, { "cell_type": "code", - "execution_count": 30, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1046,7 +1003,7 @@ }, { "cell_type": "code", - "execution_count": 31, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1066,7 +1023,7 @@ }, { "cell_type": "code", - "execution_count": 32, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1082,7 +1039,7 @@ }, { "cell_type": "code", - "execution_count": 33, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1102,44 +1059,9 @@ }, { "cell_type": "code", - "execution_count": 95, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n" - ] - } - ], + "outputs": [], "source": [ "for var in tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES):\n", " print(var)" @@ -1156,7 +1078,7 @@ }, { "cell_type": "code", - "execution_count": 96, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1181,7 +1103,7 @@ }, { "cell_type": "code", - "execution_count": 97, + "execution_count": null, "metadata": { "scrolled": true }, @@ -1209,7 +1131,7 @@ }, { "cell_type": "code", - "execution_count": 98, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1227,7 +1149,7 @@ }, { "cell_type": "code", - "execution_count": 99, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1252,7 +1174,7 @@ }, { "cell_type": "code", - "execution_count": 100, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1268,7 +1190,7 @@ }, { "cell_type": "code", - "execution_count": 101, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1328,7 +1250,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1369,7 +1291,7 @@ }, { "cell_type": "code", - "execution_count": 103, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1425,7 +1347,7 @@ }, { "cell_type": "code", - "execution_count": 104, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1510,17 +1432,9 @@ }, { "cell_type": "code", - "execution_count": 105, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Accuracy on Test-Set: 0.0% (0 / 10000)\n" - ] - } - ], + "outputs": [], "source": [ "print_test_accuracy()" ] @@ -1536,52 +1450,20 @@ }, { "cell_type": "code", - "execution_count": 106, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "ename": "InvalidArgumentError", - "evalue": "logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]\n\nCaused by op 'SoftmaxCrossEntropyWithLogits_1', defined at:\n File \"c:\\pf\\python36\\Lib\\runpy.py\", line 193, in _run_module_as_main\n \"__main__\", mod_spec)\n File \"c:\\pf\\python36\\Lib\\runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel_launcher.py\", line 16, in \n app.launch_new_instance()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\traitlets\\config\\application.py\", line 658, in launch_instance\n app.start()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelapp.py\", line 486, in start\n self.io_loop.start()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 127, in start\n self.asyncio_loop.run_forever()\n File \"c:\\pf\\python36\\Lib\\asyncio\\base_events.py\", line 421, in run_forever\n self._run_once()\n File \"c:\\pf\\python36\\Lib\\asyncio\\base_events.py\", line 1426, in _run_once\n handle._run()\n File \"c:\\pf\\python36\\Lib\\asyncio\\events.py\", line 127, in _run\n self._callback(*self._args)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 117, in _handle_events\n handler_func(fileobj, events)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n return fn(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 450, in _handle_events\n self._handle_recv()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 480, in _handle_recv\n self._run_callback(callback, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 432, in _run_callback\n callback(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n return fn(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 283, in dispatcher\n return self.dispatch_shell(stream, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 233, in dispatch_shell\n handler(stream, idents, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 399, in execute_request\n user_expressions, allow_stdin)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\ipkernel.py\", line 208, in do_execute\n res = shell.run_cell(code, store_history=store_history, silent=silent)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\zmqshell.py\", line 537, in run_cell\n return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2662, in run_cell\n raw_cell, store_history, silent, shell_futures)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2785, in _run_cell\n interactivity=interactivity, compiler=compiler, result=result)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2903, in run_ast_nodes\n if self.run_code(code, result):\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2963, in run_code\n exec(code_obj, self.user_global_ns, self.user_ns)\n File \"\", line 1, in \n cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_true, logits=logits)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\ops\\nn_ops.py\", line 1597, in softmax_cross_entropy_with_logits\n precise_logits, labels, name=name)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\ops\\gen_nn_ops.py\", line 2385, in _softmax_cross_entropy_with_logits\n features=features, labels=labels, name=name)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py\", line 767, in apply_op\n op_def=op_def)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 2630, in create_op\n original_op=self._default_original_op, op_def=op_def)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 1204, in __init__\n self._traceback = self._graph._extract_stack() # pylint: disable=protected-access\n\nInvalidArgumentError (see above for traceback): logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]\n", - "output_type": "error", - "traceback": [ - "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[1;31mInvalidArgumentError\u001b[0m Traceback (most recent call last)", - "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_call\u001b[1;34m(self, fn, *args)\u001b[0m\n\u001b[0;32m 1326\u001b[0m \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1327\u001b[1;33m \u001b[1;32mreturn\u001b[0m \u001b[0mfn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 1328\u001b[0m \u001b[1;32mexcept\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mOpError\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run_fn\u001b[1;34m(session, feed_dict, fetch_list, target_list, options, run_metadata)\u001b[0m\n\u001b[0;32m 1305\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetch_list\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarget_list\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1306\u001b[1;33m status, run_metadata)\n\u001b[0m\u001b[0;32m 1307\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;32mc:\\pf\\python36\\Lib\\contextlib.py\u001b[0m in \u001b[0;36m__exit__\u001b[1;34m(self, type, value, traceback)\u001b[0m\n\u001b[0;32m 87\u001b[0m \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 88\u001b[1;33m \u001b[0mnext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mgen\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 89\u001b[0m \u001b[1;32mexcept\u001b[0m \u001b[0mStopIteration\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\errors_impl.py\u001b[0m in \u001b[0;36mraise_exception_on_not_ok_status\u001b[1;34m()\u001b[0m\n\u001b[0;32m 465\u001b[0m \u001b[0mcompat\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mas_text\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mpywrap_tensorflow\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTF_Message\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstatus\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 466\u001b[1;33m pywrap_tensorflow.TF_GetCode(status))\n\u001b[0m\u001b[0;32m 467\u001b[0m \u001b[1;32mfinally\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;31mInvalidArgumentError\u001b[0m: logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]", - "\nDuring handling of the above exception, another exception occurred:\n", - "\u001b[1;31mInvalidArgumentError\u001b[0m Traceback (most recent call last)", - "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0moptimize\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnum_iterations\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", - "\u001b[1;32m\u001b[0m in \u001b[0;36moptimize\u001b[1;34m(num_iterations)\u001b[0m\n\u001b[0;32m 22\u001b[0m \u001b[1;31m# TensorFlow assigns the variables in feed_dict_train\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 23\u001b[0m \u001b[1;31m# to the placeholder variables and then runs the optimizer.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 24\u001b[1;33m \u001b[0msession\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0moptimizer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeed_dict\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mfeed_dict_train\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 25\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 26\u001b[0m \u001b[1;31m# Print status every 100 iterations.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36mrun\u001b[1;34m(self, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m 893\u001b[0m \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 894\u001b[0m result = self._run(None, fetches, feed_dict, options_ptr,\n\u001b[1;32m--> 895\u001b[1;33m run_metadata_ptr)\n\u001b[0m\u001b[0;32m 896\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mrun_metadata\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 897\u001b[0m \u001b[0mproto_data\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTF_GetBuffer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mrun_metadata_ptr\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_run\u001b[1;34m(self, handle, fetches, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m 1122\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mfinal_fetches\u001b[0m \u001b[1;32mor\u001b[0m \u001b[0mfinal_targets\u001b[0m \u001b[1;32mor\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mhandle\u001b[0m \u001b[1;32mand\u001b[0m \u001b[0mfeed_dict_tensor\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1123\u001b[0m results = self._do_run(handle, final_targets, final_fetches,\n\u001b[1;32m-> 1124\u001b[1;33m feed_dict_tensor, options, run_metadata)\n\u001b[0m\u001b[0;32m 1125\u001b[0m \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1126\u001b[0m \u001b[0mresults\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_run\u001b[1;34m(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)\u001b[0m\n\u001b[0;32m 1319\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mhandle\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1320\u001b[0m return self._do_call(_run_fn, self._session, feeds, fetches, targets,\n\u001b[1;32m-> 1321\u001b[1;33m options, run_metadata)\n\u001b[0m\u001b[0;32m 1322\u001b[0m \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1323\u001b[0m \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_do_call\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0m_prun_fn\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_session\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhandle\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfeeds\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfetches\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;32mc:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\client\\session.py\u001b[0m in \u001b[0;36m_do_call\u001b[1;34m(self, fn, *args)\u001b[0m\n\u001b[0;32m 1338\u001b[0m \u001b[1;32mexcept\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1339\u001b[0m \u001b[1;32mpass\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1340\u001b[1;33m \u001b[1;32mraise\u001b[0m \u001b[0mtype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0me\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnode_def\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mop\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 1341\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 1342\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0m_extend_graph\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;31mInvalidArgumentError\u001b[0m: logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]\n\nCaused by op 'SoftmaxCrossEntropyWithLogits_1', defined at:\n File \"c:\\pf\\python36\\Lib\\runpy.py\", line 193, in _run_module_as_main\n \"__main__\", mod_spec)\n File \"c:\\pf\\python36\\Lib\\runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel_launcher.py\", line 16, in \n app.launch_new_instance()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\traitlets\\config\\application.py\", line 658, in launch_instance\n app.start()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelapp.py\", line 486, in start\n self.io_loop.start()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 127, in start\n self.asyncio_loop.run_forever()\n File \"c:\\pf\\python36\\Lib\\asyncio\\base_events.py\", line 421, in run_forever\n self._run_once()\n File \"c:\\pf\\python36\\Lib\\asyncio\\base_events.py\", line 1426, in _run_once\n handle._run()\n File \"c:\\pf\\python36\\Lib\\asyncio\\events.py\", line 127, in _run\n self._callback(*self._args)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\platform\\asyncio.py\", line 117, in _handle_events\n handler_func(fileobj, events)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n return fn(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 450, in _handle_events\n self._handle_recv()\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 480, in _handle_recv\n self._run_callback(callback, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\zmq\\eventloop\\zmqstream.py\", line 432, in _run_callback\n callback(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tornado\\stack_context.py\", line 276, in null_wrapper\n return fn(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 283, in dispatcher\n return self.dispatch_shell(stream, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 233, in dispatch_shell\n handler(stream, idents, msg)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\kernelbase.py\", line 399, in execute_request\n user_expressions, allow_stdin)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\ipkernel.py\", line 208, in do_execute\n res = shell.run_cell(code, store_history=store_history, silent=silent)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\ipykernel\\zmqshell.py\", line 537, in run_cell\n return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2662, in run_cell\n raw_cell, store_history, silent, shell_futures)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2785, in _run_cell\n interactivity=interactivity, compiler=compiler, result=result)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2903, in run_ast_nodes\n if self.run_code(code, result):\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\IPython\\core\\interactiveshell.py\", line 2963, in run_code\n exec(code_obj, self.user_global_ns, self.user_ns)\n File \"\", line 1, in \n cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_true, logits=logits)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\ops\\nn_ops.py\", line 1597, in softmax_cross_entropy_with_logits\n precise_logits, labels, name=name)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\ops\\gen_nn_ops.py\", line 2385, in _softmax_cross_entropy_with_logits\n features=features, labels=labels, name=name)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py\", line 767, in apply_op\n op_def=op_def)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 2630, in create_op\n original_op=self._default_original_op, op_def=op_def)\n File \"c:\\users\\dkvandyke\\envs\\jupyter-tf-cpu\\lib\\site-packages\\tensorflow\\python\\framework\\ops.py\", line 1204, in __init__\n self._traceback = self._graph._extract_stack() # pylint: disable=protected-access\n\nInvalidArgumentError (see above for traceback): logits and labels must be same size: logits_size=[64,784] labels_size=[64,10]\n\t [[Node: SoftmaxCrossEntropyWithLogits_1 = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Reshape_5, Reshape_6)]]\n" - ] - } - ], + "outputs": [], "source": [ "optimize(num_iterations=1)" ] }, { "cell_type": "code", - "execution_count": 47, + "execution_count": null, "metadata": { "scrolled": true }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Accuracy on Test-Set: 14.4% (1443 / 10000)\n" - ] - } - ], + "outputs": [], "source": [ "print_test_accuracy()" ] @@ -1597,19 +1479,11 @@ }, { "cell_type": "code", - "execution_count": 48, + "execution_count": null, "metadata": { "scrolled": true }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Wall time: 2.07 s\n" - ] - } - ], + "outputs": [], "source": [ "%%time\n", "optimize(num_iterations=99) # We already performed 1 iteration above." @@ -1617,28 +1491,9 @@ }, { "cell_type": "code", - "execution_count": 49, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Accuracy on Test-Set: 76.8% (7681 / 10000)\n", - "Example errors:\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm8l3P+//HHK6GILFm6oY75SbLnm8LYBmXswtDYjWWsM/YxDJOx3AjJrvC1jGY0lUIUZZcsRSrZ9/JNhVCY0Pv3x+d6fa7rczrL5zrns56e99ut2/mc6/O+rut9ep/z/ryu92ohBEREJD+typ0BEZFqokpTRCQFVZoiIimo0hQRSUGVpohICqo0RURSUKUpIpKCKk0RkRRUaYqIpNC6OSd36NAh1NTUFCgr1WHKlCnzQwhrlTsfpaIybvlUxuk0q9Ksqalh8uTJzblE1TGzT8qdh1JSGbd8KuN09HguIpKCKk0RkRRUaYqIpKBKU0QkBVWaIiIpqNIUEUlBlaaISArNGqdZib7++msAPv3003rTdO7cGYDrr78egM033xyAjTfeGICtttqqmFkUWSZNmTIFgFGjRgEwcuTI7HvvvPMOAL79jpkB8D//8z8AdOvWLZv2r3/961LHSkmRpohIClUfaY4ZMwaARx55BIBnnnkGgPfee6/ec7p27QrAxx9/DMB///vfnPeXLFlS4FyKtGxDhgzJvn777bcBeP7553PSeKTpUWRyU0c/9sc//hGAvn37AtCnT58i5bjpFGmKiKRQFZHmBx98AMAtt9wC5H6q/fDDD0Dup1ZjvP1ERArDI0SIo8aVVloJiNsezzzzTAA22WQTADp06JA956CDDipJPgtBkaaISApVEWnOmjULgEGDBjXrOv4J573lUnnef/99AObPn5895r2t3l7dqlXms/7kk08GYIcddsim7dKlSymyKbUkI8XRo0cDcYT56quvliVPxaJIU0QkhbJHmsmIwiPJHXfcEYDf/va3AKywwgoAtG/fHoB27dplz1m4cCEAe+65JxBHkb169QKge/fu2bRt27YFYOWVVy7wTyFNNX36dCBur37wwQcBmDdvXqPnvvTSSwAsv/zy2WM+MsJ/h2644QYg/h2S4rj99tuzr1977TUAPvkks2Slj5nu1KlT6TNWBIo0RURSUKUpIpJC2R7PFy1aBEDv3r2zx9544w0gbkh222+/PQCvv/46kFme33nov/766wNxJ4FUpmnTpgHx4/iwYcMA+Oabb3LSeXkC7LTTTkBc7tdccw0QT7F7+eWXs2m//PJLAB577DEgnhLrnUZSHGutFW+3c+KJJwLwt7/9DYib4PR4LiKyDCp5pLl48WIADj/8cCCOLgEuvPBCAPbYY486z61rx7yW8unVkiUHPvvwododPV7mW2yxBQBXXnll9r02bdrkpJ00aRIAt912GwDHHXdc9r2pU6cCsO666wJw6qmnAnDwwQcDuRGRFIdPQ/YJJzNnzsz5vi4+PMkHxFcyRZoiIimULNL0oUEeQfgCG8lP/vPOOw+ojk8bqd+PP/4IwIABAwC44447su95tLH22msDcMoppwBx2eczHMzbLX/++WcALr300ux7PvTMF2OR0kg+Odx1111APJ3ymGOOAZZe9i0ZefoCHUcccQRQ2dMqFWmKiKRQskjTe8SvuuoqIF4IOLl8lA9el+rm0x29lzsZUay33npAPIi9Z8+ejV7vl19+AeCzzz4D4OijjwZgn332AeKFp+ty1FFHAbDaaqvlnX/Jn0eYO++8c/aYD2qvvYCwTzhwyScQHxDvvxcejfoUzOSCw+V+ElWkKSKSQskizRdffDHne5/emByPJy2DtzUut9xyS73nUx59bOWIESOAeOFa51NeAd56662cr76k2Jw5c+rNwzrrrAPEYwWTUy2lcLzcksst+kiF4cOHN3juSSedlH3tYznvv/9+IH4y3XbbbQHYdNNNs2n9utruQkSkCliaxXtr69GjR5g8eXJeab231D9RVlxxRQAuuOCCbJr9998fyF1ko9KY2ZQQQo9y56NU0pSx84WhfSzu+PHjs+99//33QP1j9lq3zjz8eLSaj+QsMO91vfHGGwHo2LFj3tdxKuPK4QuOJ9s/vc107NixQNx2mkZzyliRpohICqo0RURSKNnjuQ8h8K918Y4DX1zB18T0oSYbbbQRAJttttlS57755ptAvLhHsTqY9OiW3oIFC7KvfcjZxIkTAVhzzTWBeDqs7wyanF6bXJCjLj5AHuLJE80ZYqQyrjzJdXd9eNNXX30FwK233gqkGxCvx3MRkRIp2ZCjc889F4Drrruu3jQ+iNmXDfOvaXiH06677grAAw88kPoaUljJqM8jzcb4AHZYOtJcddVVARg4cCAAxx57bPa9uoY5SfVL7lzpq8Sfc845QPxk6stE+q6XxaJIU0QkhZK1aXoU6dOlfGL+Tz/9lE3ju0562ubwttPkYg4+0LmZ11V7VxH5Ih/Jskr+jgAMHToUiIc0FZrKuDp4O6e3cfoA+3zqD7VpioiUSMnaNL2tyadFvfvuu0ulefLJJ4E4sujfvz8Ar7zySur7eQQ9ZcqU1OdK6d15550AXH755cDS0SXEO436ND1Ztnk7p2+HUnsqbrEo0hQRSaHs+54n7b777jnf+9YFHmn6ogvJ7Q18E6frr78egH/9619Fz6cUjpet94R+9913S6VZZZVVgHh7C5+CK8s2X8DFF/dILupRTIo0RURSUKUpIpJCRT2e19anTx8g3qXSOwd85ROA9957D4hXC6/NVwqXyuR7RX377bc5x5N7BT388MPA0it/S+Xw5jGI9/068sgji3IvX+XooosuAmDRokUAPPvss0W5X22KNEVEUqjoSNNXZj7ssMMAGDZs2FJpnn766ZzvfT1G3z/m6quvLmYWpYm8w8cHs9eWjFJ8SqxUHt/TxzvyIN7nvimRpu85NGrUqJzjye99goxHtP/85z8B2GSTTVLfrykUaYqIpFDRkabvEzNo0CAgjk6SA9a/+OILAGpqaoB4oQcfGC+VZeHChUD8FLF48eKc97faaisgLnOpDsnp2IMHDwZg5MiRQLxkm6fxQei+LCDEw4bq2xs9uR+QT8H2vo7kYh6loEhTRCSFio40ne8sOGbMGCBuwwCYNGkSEEeWvjScVKannnoKgNmzZ9f5vi/31qZNm5LlSZrOo8hx48Zlj3nU6Lw9cu7cuUA8CD25ILm3g3rU2Ldv35xrJNsrte+5iEgVqYpIs7ajjjqqztdS+S6++OI6j59//vkA7LbbbqXMjhTInnvuWedriKe/thSKNEVEUqjKSFOql2+G5bwNuthbFIgUiiJNEZEUVGmKiKSgx3MpqbPPPjvnq3cMdezYsWx5EklDkaaISAqKNKWkzjrrrJyvItVGkaaISArN2vfczOYBnxQuO1WhcwhhrXJnolRUxi2fyjidZlWaIiLLGj2ei4ikoEpTRCSFBitNM1vTzKZG/+aY2ezE9ysUK1NmNsvMpkf3eTmP9CeY2bwo/Vtm9odm3v9+Mzswz7Tbm9kv+aavNGUs47PN7M3o3xl5pC95GZvZBYn/izfN7Gcza9+c+5aD/o4bTHOQmU2L7vmqme3Q6IVDCHn9A/oD59Zx3IBW+V4nz3vNAlZLkf4EYFD0el1gPtChVprWKa53P3BgHulaA08D4/JJX+n/SlXGwNbAG0BbYPno/3DDSizjRPq+wBPlLqNqKePomhX/dwy0I+7b2QaY0dh1m/R4bmYbmdkMM7sdeA3YwMwWJN7vZ2Z3Rq/XMbMHzWyymb1iZts15Z75CiHMAT4GOpnZ5WY22MzGA3ebWWszGxjlY5qZnRDlsZWZ3WpmM83sESDf9fPPBB4gU7gtSpHLuBswKYTwQwjhJ+A5MpVSXkpcxu73wL9TnlPR9HcMIYSFIaoxgZWBRnvGm9OmuSlwVwihO1D3MtwZNwIDQgg9gEMBL4ReUWHVJQBPmdkUMzs+TabMbCOgM/BhdKg7sF8I4SjgJGBuCKEnsC1wmpl1Ag4BNgQ2B04Bdkhc7woz27uO+3QC9gHuSJO/KlOsMp4O7Gpma5jZysBewAb5ZqpUZZx4vx2wB/BgvnmsIsv033H03iFm9g4wmky026DmzAj6IITwah7p9gC6Wry0/epm1jaE8DJQXztHrxDC52a2LjDezN4KIbzYyH2OMLNdgMXACSGEBdE9Hwoh/Bil6QN0M7N+0fftgS7AzsC/QwhLgFlm9oxfNIRwUT33GwScH0JYkvjZWpqilHEIYYaZDQQmAAuB14Gf87hPqcvYHQA8G0L4Jo88Vptl/e+YEMIIYISZ/Qa4LLp+vZpTaS5KvF5Cpk3EJTd4MaBnCCF328EGhBA+j77OMbOHgJ5AY//ZQ0MIdS3KmMynAaeGEJ5MJjCzvuQRltfSAxgeFWgHoI+Z/RJCeCTldSpZMct4CDAEwMwGAO/ncVqpy9j1A/7ZaKrqtKz/HSfz+7SZ3Wdmq4UQFtSXriBDjqKa/Wsz62Jmrchtn5oAnObfmNnWDV3LzNpFj0P+WNQbmBF9/2czO7kZWX0cONXMWkfX62pmbcm0qfWL2kTWA3Zp7EIhhE4hhJoQQg2ZsP6kFlZh5ihkGUdp1o6+1pCJ5IZF31dMGUfnr07mMa/Flq1bFv+Oo3Zdi173AGiowoTCjtP8C5le5CfJ9Jq504BfRw22M4ETowzW1xbSEZhoZm+QCftHhRAmRO91A75sRh4HA+8BU81sBnAbmWh7BPApmUK9mcx/PlE+G2zvWsYUqowBRkdpRwN/TDz6VloZHwyMDSH80Iw8VZNl7e/4UGCGmU0l0257WGM3r6pplGb2KHBACCGf9i+pQirjlq/ay7iqKk0RkXLTNEoRkRRUaYqIpKBKU0QkBVWaIiIpNGuPoA4dOoSampoCZaU6TJkyZX5Yhlb1Vhm3fCrjdJpVadbU1DB58uTmXKLqmNkytS2AyrjlUxmno8dzEZEUVGmKiKSgSlNEJAVVmiIiKajSlJIaOHAgAwcOxMwwMyZNmsSkSZPKnS2RvKnSFBFJoVlDjkTSGjRoULmzICWyYEFmWcpVV10VgFatWkaM1jJ+ChGRElGkKUX32WefLfX6uuuuA2D77bcvS56k+H73u98BsPLKKwNwwgmZPcv23Xffotxv7ty5AKyxxhoAtG5dnOpNkaaISApVH2l+8klmNtSNN94IkJ0OdssttwCw+eablydjkjV8+PCljq2//vplyImU0jbbbAPAgAEDANhll7y2ZWoyby//6aefALjmmmuKch9FmiIiKVRVpPnuu+8CcPPNN2eP3XfffQB8803ultS//e1vARgzZkz2mLende7cGYAtt9yyeJmVrLoiTbVltnwbbLBBSe4zfvx4IDMGGOC///0voEhTRKQiVHSkuWTJEgBmzpwJQO/evQGYM2dOo+fOnj0byG1H+fbbb4E4ynnhhReAljN+rNJ4ZP/SSy9lj3n0UaooRMrn1ltvLcl9nn76aSCOML0ttVhUW4iIpKBKU0QkhYp8PJ83bx4AN910EwCXXXZZvWlXW201IH709kd658eT3n777Zy0ejwvjuuvv36pY4ccckiTr+cLeyQHy0Pu478PqFZHU/nMmDEDgM8//7wk95swYULO93//+9+Lej/VFiIiKVRkpHnRRRcBcMcdd+QcX2GFFQC44YYbssc23HBDAPr37w/kRh21rbVWZh+lhx56CCjeNCvJmDVr1lLHtttuu9TX8QjzsMMOA5aONJM8un3xxRcBRZzl4OVVexigT6csFO/48cHsbdu2BWDnnXcu6H1qU6QpIpJC2UOtZBukt3d5JOhtjT4I/c477wTiwawAZ555JhC3UzbEhyIo+qguHj3WjjDrWvTDo9Gzzz4bQAscl8jChQuzr71cXN++fQE46aSTCnpPryemTp2ac33v5ygWRZoiIimUPdL0hTYARo0alfPeJptsAsAFF1wAwI477gjAjz/+mPf1N9544+zrwYMHNzmfUlrJCLH2NMxhw4YBcOihhy51nreZ1jV1U4rnrLPOyr5+5513ct4rVm/2//7v/xbluo1RpCkikkLZIk3v8br66qvrTePtlP369cs57ouMApxxxhlAPFZr4sSJOWn/8Ic/ZF/7Qh1S+eoa49lQhFmf//znP6nPkfw9/PDDQN2RfU1NDQBdu3Yt6D29V/6LL74o6HXzpUhTRCSFskWa3jPu4yxh6YU4fNzViiuuCMDpp58OxD2jEPeo1o5YvW3rlFNOKWS2JYW6Fhqua+xmkpdnMnLxslS0WDl8pp3P1qs9JhPiPoo2bdoU9N4fffQREPeau+OPP76g96mPIk0RkRRUaYqIpFC2x/PlllsOgMceeyx7zFdZ9+mNW2+9NRAPPXLJgbQ+fdKHIa2yyioA3HvvvUC857KUng9DSXbqnHPOOUBuE0uSv5/ki3Dkwx/vfb1OPdIXh+/86HtyJflg9i222KKkeVpzzTVLch9FmiIiKZR9cHtyytORRx6Z1zkjR47Mvq49IN6n0SUHtUt5eLSXXKTDF1Tx/VzqiziTGtu50ocVJa9feyqfFMbjjz8OwIUXXphzvEuXLtnXvhOsP02GEABYtGhRvdddfvnlgXgoovNFPsys3nM9sv3Vr37V+A9QAIo0RURSKHukmcZXX30F1B1FdOrUCYg/5aRyJKfY+ZOAt116ZOhp0uxc6RHmueeemz3m0W2adlDJny+S8dprr+Uc92XaAK644oqc93755RcAbr/99nqv2717dwBef/31nONDhw4FYL/99ssee+KJJ3LS+NNqQ9FoISnSFBFJoaoizX333ReA6dOnL/XeJZdcAsQLFUvlSPZg++D2QYMGAXFk2dACG754h38dMWJEvef44sPa7bI4kv0JSZ9++mn2dVOe9nxEjD8x+qiXI444AoAePXpk0/p2OK7UE1gUaYqIpFAVkeaHH34IxBs2JXn0eeyxx5YyS9JE3lvubY4+hrOuBTqct4PW5r3yyd5zRZjFdfHFFwPxQjkuuRjOOuusAyy9vcVuu+0GwLbbbrvUdXv27AnA999/D0CHDh0AeO6554B4k0WIx4b6OO5Sj5RRpCkikoIqTRGRFCr68Xz27NkA7L777gB89913QNxYDEsPpJXq4I/RPsjdv3rZ1rXjpA9L0qpH5XPyyScD0KtXr5zjHTt2zL729W5XWmml1NdfffXVc77v3bs3EA91SvKhaO3bt099n+ZQpCkikkJFR5o+gPbjjz/OOZ5cjT0ZdUr18x1J6+oY8uFKPuTIo1XtLlo6vphOXZ05xZTcZ8ifOJLRbSkp0hQRSaEiI81XXnkFgKOPPjrnuK/gvvfee5c8T1Ia3raZ5JGl8zbNxhbykJZjrbXWqvN1OSjSFBFJoaIiTV86ytsvFixYkPO+96y1a9eutBmTkktGnHVFnyLlokhTRCSFioo0hwwZAsC4ceNyjq+77roAjB07FoBu3bqVNmMiIhFFmiIiKVRUpOmzenxRUR+PdeKJJwLlG5clIuIUaYqIpKBKU0QkhYp6PP/Tn/6U81VEpNIo0hQRSUGVpohICqo0RURSsBBC0082mwd8UrjsVIXOIYTyrhhQQirjlk9lnE6zKk0RkWWNHs9FRFJQpSkikoIqTRGRFBqsNM1sTTObGv2bY2azE9+vUKxMmdnZZvZm9O+MPNKfYGbzony9ZWZ/aOycRq53v5kd2EiaCxL/F2+a2c9mVtpt8QqgjGW8hpk9aGZvR2XWs5H0JS/jRNrtzeyXfNNXmnKVcXTv1mY2zcxG55H28kTeppvZPs289wtmtnUe6X5vZjOjv+P7Gkvf4IygEMKXwNbRhfsDC0MI19a6oZHpUFrS2M3yEf2QxwA9gJ+BJ8xsTAjho0ZOHRpCONPM1gVmmNnDIYT5ieu2DiH8XIg8AoQQrgKuiq7dFzglhPBNoa5fKuUo48hNwMMhhIOiP9y2eZxT0jL2awJXAuMLed1SKmMZA5wNzADy3c/3mhDCIDPbHHjazNYOid7qQpexmW0CnAvsEEJYYGZrN3ZOkx7PzWwjM5thZrcDrwEbmNmCxPv9zOzO6PU6UUQx2cxeMbPtGrl8N2BSCOGHEMJPwHNA33zzFkKYA3wMdIo+uQab2Xjg7uhTb2CUj2lmdkKUx1Zmdmv0afMI0CHFfwfA74F/pzynohWzjM1sDaBXCOEegBDC4jQfOCUu4zOBB4D5jSWsNkX+O8bMOgO9gbvT5i2EMAMwYPXoqeA6M3sauNLM2pnZPVE+Xjez/aL7rWRmw6NyfwBok8etTgJuCiEsiO47t7ETmtOmuSlwVwihOzC7gXQ3AgNCCD2AQwEvhF5RYdU2HdjVMo9vKwN7ARvkmykz2wjoDHwYHeoO7BdCOIrMf9DcEEJPYFvgNDPrBBwCbAhsDpwC7JC43hVmVu9ObmbWDtgDeDDfPFaRYpXxr4B5ZnZf9Es/xMzyjURKVsbRefsAd+SbtypUrDIGGAScB6Qe12hmOwA/hhC+ig79P2D3EML5wCXAuKiMdwOuM7M2wOnA1yGELYGryfxe+PXurudRfWOgm5lNNLNJZtansbw1Z8GOD0IIr+aRbg+gayb6BzKfHG1DCC8DL9dOHEKYYWYDgQnAQuB1Mo/pjTnCzHYBFgMnRKE2wEMhhB+jNH3I/Af1i75vD3QBdgb+HT2azDKzZxL5uaiR+x4APFuNj+Z5KEoZk/m96wGcAUwh86h+HnBpI/cpdRkPAs4PISxJ/GwtTVHK2DLtv5+FEKaa2R4p8nOemR0LfAccljg+PNF00AfYy8wuiL5vA3QiU8YDAEIIr5vZm35yCOG4eu7XmsyH+C5kPoifNbNNQwjf1pfB5lSaixKvl5AJpV0yLDagZwhhcb4XDiEMAYYAmNkA4P08ThsaQjizkXwacGoI4clkAsu0STZ1lH8/4J9NPLfSFauMZwGfhBAmA5jZSDKPwY0pdRn3AIZHFUUHoI+Z/RJCeCTldSpZscp4B+AgM9s/us6qZnZvCOGYRs67JoQwqJF8GnBgCOGDZIKonNKW8Szgmaid9AMz+4BMVPt6fScUZMhR9AnwtZl1MbNW5LZBTgBO82/qCZFzWNQYa2Y1ZCK5YdH3fzazk5uR1ceBUy3TuI+ZdTWztmTaTftF7V7rkfnUaZSZrU7ml6Ml/RHVqZBlHEKYBcyNHrMBdgdmRudWTBmHEDqFEGpCCDXAaOCkFlZh5ihwGZ8fQlg/+r87EnjCK0wzG+DtkE30OJBdP9LM/DH8OeCI6NhWwGZ5XGs08JvonLXJVJgNdjoXcpzmX4BxwJNkam93GvDrqHF2JnBilMGG2kJGR2lHA39MPPp2A75sRh4HA+8BU81sBnAbmWh7BPApmV6+m8n85xPls6E2zYOBsSGEH5qRp2pSyDI+AxhmZtPI/HJfFR2vtDJe1hSyjOuzJTCnGXm8FFjJMsOS3gT6R8dvBtaMfqfOAib7CQ20aT4KLIx+pgnAWd4pVJ+qmntuZo8CBxR6WIlUDpVxy2aZZ+hxIYQ9y52XpqqqSlNEpNw0jVJEJAVVmiIiKajSFBFJoVm7UXbo0CHU1NQUKCvVYcqUKfOXpVW9VcYtn8o4nWZVmjU1NUyePLnxhC2ImS1T2wKojFs+lXE6ejwXEUlBlaaISAqqNEVEUlClKSKSgipNEZEUmtV7LpLWggWZtRB+85vfAPD9998D8M4775QtTyJpKNIUEUmhKiPNd999N/v62msz+0M9//zzAHz++ecA3H///QDst19zlu2TQvj666+zr3fffXcA3njjDQA23njjsuRJpKkUaYqIpFBVkeaoUaMAOPnkeGHv7bbLbIp3ww03ALDVVlsBsM4665Q4d1KbR5geXQJMnToVgOWWWw7Qk4BUH0WaIiIplD3STM55ve+++3Le+/nnzOLds2dndhb1drDzzz8/m+acc84pdhalia6++mogji6TTjzxRACuueaakuZJpLkUaYqIpFD2SPPQQw/Nvm7fvj0AW265JQBffPEFAPvvvz8AV1xxBQCbb755KbMoKc2fPx+AcePGLfWel/Gf//znkuZJKoc/OU6bNi17bPjw4QD89NNPAHz88cdAZgUmgKuuuiqbdr311itBLuunSFNEJIWyRZqLFmX2fvdxlQD33HMPADvvvHM5siQF4r3l06dPX+q93//+9wB07dq1pHmS8nnhhRcAuPzyywF48cUXAfjuu++yaTKbVC5t4sSJACy//PLZY0OGDAGgdevyVF+KNEVEUlClKSKSQtkez2fOnAnEDb/ScngDvz9yrbLKKtn3zj777LLkSUrntddeA+Dvf/87AOPHjwdg8eLFOemSU2jXXHNNAPbcc08gXsBlzJgxANx9993ZtMcccwwAu+yyS8Hzng9FmiIiKZQt0vzmm28ACCGUKwtSIiussEL29UYbbdTk6/hCLQsXLsz7nC222ALI7UiQwlmyZAkAAwcOzB4bMGAAEA898w6bI444AoCDDz4YgAMPPLDR6++9994AjB07NnvMOxgVaYqIVIGyRZoPP/wwkF+kWXs65RprrJF978svvwTigfDrrrsuAJ06dQLqH8oglW3ChAnZ14MGDQJg0qRJQO5Sc43x4U877bQTAMceeywQ/35I83iEmZza7H/Tm2yyCQCDBw8GCjeU0Kde+5ClZJt5KSjSFBFJoWyRZkOLz3777bdA3Ov2t7/9DYC3334bgA022CCb9rPPPqvzGnvttRcA5513XvaYb7Eglcujh4svvjh77OWXX85J0717dwBWXXVVADbbbLPsex06dADiRUIef/xxAJ588kkg7oU97rjjsudccsklhfsBlhH33nsvkPv35fxvz5dyXHHFFVNf39uvk1Mta997ww03BOJe+lJRpCkikkJFRZq+RcVRRx0FxFMsDznkEAAuu+wyII4wIF502Hl0OmzYMCAe9wXxQsWnnHJK838AydvRRx+dd1qPMGtHlxAv7uLT6JK/B/W5+eabAbjllluAePxf//79s2kUaab30UcfAXGfwbbbbpt9rykRpi/Q4X0T3gOfnGbt/J6/+93vUua6MBRpioikULZIc+uttwa72IXZAAAJjklEQVRgm222yR7zdiePDv/6178CcdtFPo488sicrzfeeGP2vVNPPRWAHj16ALmfjlI8Xq75+PTTT+t977TTTgPyizDd6aefDsQjMDQjqTiSs32ee+45IF7WzcfIrrbaagD85z//AWDEiBHZc1599VUgHr/dEH9y6dKlSzNz3TSKNEVEUlClKSKSQtkez9dee20ApkyZUtT7+F40AHfeeScQrwT/f//3f0W9t2T4pIRKovU8m2fTTTcF4kdv378LoE+fPkDcYdOmTRsg3iH2k08+AXIntqSZhLLDDjvk3LvUFGmKiKRQ9j2Ciq1t27bZ175zpUefPljep3tJcSQ7Cd5//32gaQt3+HQ8nxLZHMkhR5KeD//6/vvvgdzJCLNmzcpJ++OPPwJxhOkdu8khQ7vuuisQD2XyDlwfIpbcF+jwww8vzA/RRIo0RURSaPGRZpJ/wvnCxz4FU5FmYd1+++1APIkguRfMjjvuCMBNN90ELD1A2Qeh+1RJgNtuuw2A0aNHA/FyYb5IhEcpDak9HU/t2YXhC6D4V4j3BPIB6767rH9tyLPPPgvET4EuuRtlu3btmprdglCkKSKSwjIVafritT69ywe5S2Edf/zxQBxxJntW586dC8R72NeONDt27AjktpGtvPLKAFx77bVAvJ+6D6K+6667smkPO+ywnOv5tiq+r7br3Llzuh9K8uZPE/41DS/j2tMzyzVlsi6KNEVEUlimIk1fvME/xVZfffVyZqfF8u0NHnvsMSC359N5BOhTW316Y1296v6eL2J7zz33APDhhx8Cucu8+dJvPhb3yiuvBGDRokU57+ez1YKUjve4v/TSSznHzz33XKBpy8sViyJNEZEUVGmKiKRQ9sfz5FQqX+GmEI30ft3kfjK+mnevXr2afX1pnO/X9K9//St77OqrrwbiziHvLPK1VL3svTOpLv4I7/v8JCcw+FqOTzzxBBCvt/rKK68AcUeT9o6qLN6Z53t+tW/fHoDevXuXLU/1UaQpIpJC2SPN5MBnX1vTG+v33XdfAFq1yr9u/+qrrwA466yzgHh4CsRrePogaSkuj+b69euXPeaLN/jK7I8++igAM2bMAODNN98E0q17mZyc8Je//AWIy9o7fCqpI0GW5n+3ztfK9DU4K4kiTRGRFMoeaSZX4b7ooosAOOCAAwAYOnQoEC8F5TsNejQCcTvoyJEjAXjqqadyrp/cD+jCCy8EFHWUk0d+/tX3ffK9YXz/nzROPvnk7Gtvs5TKl5z04MPIvC/C/+YrkSJNEZEUyh5pJv3pT38CYMGCBQBccMEFQNze4RFisv3D28h8XyGffue9blqMo7L5QHgfAH/ppZeWMztSQr4HGMR9Gz4SYpdddilLnvKhSFNEJIWKijQ96vjHP/6R81VEWg5fpm/s2LHZYz7SwncP7du3b+kzlidFmiIiKVRUpCkiLZ8vQJ3kM8F22223UmcnNUWaIiIpqNIUEUlBj+ciUlLHHHMMkLsPkA8V9D3TK5kiTRGRFBRpikhJ+d5Bzz//fJlz0jSKNEVEUrDkIsCpTzabB3xSuOxUhc4hhLXKnYlSURm3fCrjdJpVaYqILGv0eC4ikoIqTRGRFBqsNM1sTTObGv2bY2azE9+vUKxMmdksM5se3eflPNKfYGbzovRvmdkfmnn/+82swY2xzewgM5sW3fNVM6vcVVMbUMYyXsPMHjSzt6My69lI+nKU8dHR7+E0M5toZls0557lUsYyPtvM3oz+nZFH+pKXcSLt9mb2Sz7pGxxyFEL4Etg6umh/YGEI4dpaNzMybaNL8slcCjuFEBakSD80hHCmma0LzDCzh0MI8xP5bB1C+LmA+XsCGBVCCGa2DXAfsHkBr18SZSzjm4CHQwgHRX+4bRs7gdKX8QdEv4dmth9wO/DrAl6/JMpRxma2NXAM0AP4GXjCzMaEED5q5NRSlzFm1hq4EhifT/omPZ6b2UZmNsPMbgdeAzYwswWJ9/uZ2Z3R63WiiGKymb1iZts15Z75CiHMAT4GOpnZ5WY22MzGA3ebWWszGxjlY5qZnRDlsZWZ3WpmM83sEaBDHvdZGOJetJWBFtWjVswyNrM1gF4hhHsAQgiLQwjf5Ju3EpbxxMQH90vA+vnmsRoU+e+4GzAphPBDCOEn4Dkg7/XeSlXGkTOBB4D5jSWE5rVpbgrcFULoDsxuIN2NwIAQQg/gUMALoVdUWHUJwFNmNsXM6t8Auw5mthHQGfgwOtQd2C+EcBRwEjA3hNAT2BY4zcw6AYcAG5KJFE8Bdkhc7woz27ueex1iZu8Ao4ET0uSzShSrjH8FzDOz+8zsdTMbYmYr5ZupUpZxwvHA2EbSVKNilfF0YFfLNMOsDOwFbJBvpkpVxtF5+wB35Ju35swI+iCE8Goe6fYAulq0yCiwupm1DSG8DNTXXtkrhPB5FKKPN7O3QggvNnKfI8xsF2AxcEL0SAXwUAjhxyhNH6Cbmfmesu2BLsDOwL+jR5NZZvaMXzSEcFF9NwwhjABGmNlvgMui67ckxSrj1mQe284AppB5VD8PaGyvi5KXMYCZ7QEcBezYSP6qUVHKOIQww8wGAhOAhcDrZB7TG1PqMh4EnB9CWJL42RrUnEpzUeL1EiB5xzaJ1wb0DCEszvfCIYTPo69zzOwhoCfQWKU5NIRwZiP5NODUEMKTyQRm1pdmPF6HEJ6OoqbVUrbDVrpilfEs4JMQwmQAMxtJ5hGpMSUvY8u0zQ0G9gwhfJ32/CpQzL/jIcAQADMbALyfx2mlLuMewPCowuwA9DGzX0IIj9R3QkGGHEU1+9dm1sXMWpHbdjEBOM2/iX4J62Vm7cysnb8GegMzou//bGYnN3R+Ix4HTrVMwy9m1tXM2pJpb+kXtYmsBzS6q1PUHmTR6x4ALazCzFHIMg4hzALmRo9gALsDM6NzK6mMa4ARwOEhhHz+4KtaIcs4SrN29LUGOAAYFn1fMWUcQugUQqgJIdSQaWY7qaEKEwo7TvMvwDjgSTKRhDsN+HXUYDsTOBEabAvpCEw0szfIhP2jQggTove6AV82I4+DgfeAqWY2A7iNTLQ9AviUTOV8M5n/fKJ81tfedSiZ3r2pZNp7DmtGvqpFocoYMo/mw8xsGrAZcFV0vJLKuD+wBjDY8hz+1gIUsoxHR2lHA39MdPZVUhmnVlXTKM3sUeCAQg85kMqhMm75qr2Mq6rSFBEpN02jFBFJQZWmiEgKqjRFRFJQpSkikoIqTRGRFFRpioikoEpTRCSF/w+H2cYDWb+BuAAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "print_test_accuracy(show_example_errors=True)" ] @@ -1654,28 +1509,11 @@ }, { "cell_type": "code", - "execution_count": 51, + "execution_count": null, "metadata": { "scrolled": false }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Optimization Iteration: 1001, Training Accuracy: 95.3%\n", - "Optimization Iteration: 1101, Training Accuracy: 96.9%\n", - "Optimization Iteration: 1201, Training Accuracy: 95.3%\n", - "Optimization Iteration: 1301, Training Accuracy: 93.8%\n", - "Optimization Iteration: 1401, Training Accuracy: 93.8%\n", - "Optimization Iteration: 1501, Training Accuracy: 92.2%\n", - "Optimization Iteration: 1601, Training Accuracy: 96.9%\n", - "Optimization Iteration: 1701, Training Accuracy: 92.2%\n", - "Optimization Iteration: 1801, Training Accuracy: 90.6%\n", - "Wall time: 18.1 s\n" - ] - } - ], + "outputs": [], "source": [ "%%time\n", "optimize(num_iterations=900) # We performed 100 iterations above." @@ -1683,30 +1521,11 @@ }, { "cell_type": "code", - "execution_count": 52, + "execution_count": null, "metadata": { "scrolled": true }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Accuracy on Test-Set: 94.9% (9494 / 10000)\n", - "Example errors:\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm8leP+//HXJ0WFSJlKtfmllKEiRRyhhJwkY45jLGQeT8fwPYZz5EvnIGM4KEOmzHQUJUPnS5SKZMxYpKIMHQ7p+v2x7mvd99rD2uvea977/Xw89mPf6173cNW117U+13VfgznnEBGRzDQqdgJERMqJCk0RkRhUaIqIxKBCU0QkBhWaIiIxqNAUEYlBhaaISAwqNEVEYlChKSISQ+NsTm7durWrqKjIUVLKw+zZs5c75zYudjoKRXlc/ymP48mq0KyoqGDWrFnZXKLsmNlnxU5DISmP6z/lcTyqnouIxKBCU0QkBhWaIiIxqNAUEYlBhaaISAwqNEVEYlChKSISQ1b9NOvixx9/BOCLL74AYOzYsVWOOeGEEwDo3r174RImBbVy5UoAWrRoAUCjRvr+lvKgv1QRkRgKFmn6CPPvf/87AH/7299qPPbWW28F4IgjjgDg+uuvB2CjjTbKZxKlgA477DAA1l13XQCGDx8OwO9///u83G/p0qVA+DfUuHHBK1lSTyjSFBGJoWBft1deeSUAV111Va3Hrl69GoAJEyYAMG3aNADGjx8PwIABA/KQQimkHXfcEYDRo0cD0Ldv37zeb8yYMQD8+uuvQFjjEYlLkaaISAwFizS33HLLlNdmBsDpp5+e3LftttsC8MsvvwBwySWXALBkyRIABg8eDMCf//zn5DkjR44EoHnz5vlItuRJu3btCnKf559/HoBrr70WgP/+97+AIs18uuKKKwB48803Abj44osB2HrrrYGwx8TPP/+cPOe5554D4Pjjjwdg6tSpAPTo0aMAKY5HkaaISAwFizQff/zxlNeHH344ED4Zr063bt0AOPjggwH45ptvAPjrX/+aPGbhwoUA3HXXXQA0adIkRymWfLrlllsKcp/p06cDYYTp21Ilf3wN0dcmn3zySQC6du0KwMYbJ+b+XbVqVfKcyvN5+prBvffem9/E1oEiTRGRGFRoiojEULDq+bPPPguEIbtvHE7nd7/7HRCG9xdeeCEAr7zySvIY3y3JOQeE3ZLUebk0zZ8/H4Avv/yyIPfzDxS8Sy+9tCD3laoWLFiQ8tp/ZiEsF7xHHnkEgL/85S8AdOrUKc+py5wiTRGRGAoWjvXv3x8IO6qvt956GZ/bp08fIOwIPXDgwOR7K1asAOD+++8H4MADDwTCB01SWl599VUAvvvuu5T9fjhlrvgHP74ze7NmzQDYY489cnofSZgyZUqN7/3zn/8E4LXXXgPCmuL7779f4zm+26Ef6FJKFGmKiMRQsEizS5cuQBhpVueOO+4Awqjx5JNPrva4P/zhD8ntm2++OeW9Dz74IKt0Su75yVoArrnmmpT3hgwZAsBJJ52U03v6dvC5c+emXH/DDTfM6X0kwXf9q86gQYOAcMrHb7/9FggHrUT55xh+6sBSpEhTRCSGgkWaPXv2THn91ltvAalDqfyQSt+e8eKLL8a+z5133gnANttsA8A+++yTfG+DDTaIfT3J3jnnnJPcrtyOla+n2X6wgxTGmjVrktvRp+LV8dPzVTfVo+/14q/ho9JSokhTRCSGgkWaBx10EAD33HMPAHvvvTcAX3/9dfKYpk2bAmGkWRefffYZED49j07k4Z/i+Yk/NMlHfj311FMATJw4scp7FRUVAHTu3Dmn9/RP5aN/V5J/0eVKfJ/Lyn0vM1H53IceegiA3XffPdsk5owiTRGRGAoWafrpoP74xz+m7I/217zvvvuAMDLx7RmTJk2q833/85//JLePOuooALbffnsgHE203Xbb1fn6UtX3338PhEuaVO6TCeEELr52kSuffPIJED4194YNG5bT+0jDpUhTRCQGFZoiIjGU1KwWfiVC//u3334D4Icffkg5LtrI7xuMN9lkk5RjfFeWaNcTP3/f22+/DcB5550HwNVXXw1onfVc8Ss/Vp4jEcLO7L6JpFBatWpV0Ps1NG3atElut23bFijcpCyFpkhTRCSGkoo0ly9fDoRDIf1EHZWHvmUyFM7PCO/XTgc45ZRTgDDS9OvH+O4Sfvo6qRs/acNFF12Ust+vDQPhsNe11loLCDsxR2fxrszPxu8n3/D8JB/purb4yHarrbaq/R8gdea7FEI4jdvYsWOBcLKU+kKRpohIDEWPNJ9++unk9llnnQXAV199BcCDDz4IhJ3R68JHqwAzZswAwnVi/CQDfrqyyZMnA7DffvvV+X4NmZ8kw69C6Plp2gBGjRqV8p5vt7711ltrvK5fkXDOnDkp+32XMT8hBISrGnq+VlKXjtZSN34toBtvvDH2ub7mUdtQzGJSpCkiEkPRI83ok3EfYfrIxK9C6SPEXXfdNat7rb/++kA49ZyPQn1nbP8UXZFm3Tz66KPV7v/888+T25Wn8suEn1quffv2QDhQwg9WiE4Gs2zZspRzfTu2lIdshmAWiiJNEZEYih5pRicU9v26Ro4cCYTtGr7dK1f8tHTR6awAdthhh5zep6Hxi2CdccYZKfs7dOiQ3N50002Bqstb+Alcdt555yrX7dWrFxAOiW3dujUAL7/8MpDadub7hvo+t6W0IJfUD4o0RURiUKEpIhJD0avnUX4dF9/JfPr06QAcc8wxAOy5554AXHDBBclzaqt++U7uEK5B9NFHHwGl3a2hHI0YMQKA3r17p+zffPPNk9t+tu66zGXasmXLlNd+Vn7f1SnKPzTUbP2Sa4o0RURiKKlI03cl8ZGDfzDjuyKNHz8egHvvvTd5TnTG6OpUHnpXHf+g4ZJLLomXYEnh13ep7mFOPkXXGfLrEUWjW5FcUqQpIhJDSUWanp/N/eOPPwbg7rvvBsJhlX7CDajb9FO77bYbAPvuuy8AJ554IqDpw8rVxhtvXO22lJ/Kwyh9t7JSokhTRCSGkow0Kzv22GNTfi9ZsiT5nh+G6Vea9E/YfSfn6NP1nXbaCQiH462zzjp5TLWIxFV5GOX8+fOLmZxqKdIUEYmhLCLNyjbbbLMq26NHj045ZuDAgQVNk4g0DIo0RURiUKEpIhKDCk0RkRhUaIqIxFCWD4JEpH7yK5oefvjhQOGH5GZCkaaISAyKNEWkZHTr1g2A999/v8gpqZkiTRGRGCybiXjNbBnwWe6SUxY6OOcazKwQyuP6T3kcT1aFpohIQ6PquYhIDCo0RURiSFtomlkrM5sb/Cwxs8WR12vnI0Fmtq6ZvR7cY4GZ1boGhZldEUnb22Z2QJZpmGFm3Ws5psLMXjKzOWY2z8z2y+aexVKMPI7cu7GZvWVmT2RwbMHzODjuyODv8B0zuyebexZLkT7HXSP3mGtmP5jZ6bWcM9zMlgXHv2tmJ2SZhvvM7KBajtnIzJ4K/g5nmlnXWi/snMvoB7gMOL+a/QY0yvQ6GdynEbBusN0EmAX0rOWcK4Czg+3tgGUE7bWRYxrHSMMMoHstx9wFnBhs7wB8lKv/g2L9FCqPI9cdCdwPPJHBscXI422A2cCGwetNip1H5ZbHPl+ApcAWtRw3HBgTbG8GLAdaZ5HH9wEH1XLMdcDFwfa2wPO1XbdO1XMz62hm883sVuBNoJ2ZrYy8P9TM7gi2NzWzx8xsVhBB7pLu2s65Nc65VcHLtUkUnBk/rXLOzSfxB9Ay+Ka5xsymA1ea2XpmNj5IxxwzGxSksbmZTQy+bR4EmmZyK6BFsL0BEH/djRKWzzwOzukA7AOMi5u2AubxScCNzrmVwX2Xxk1rKct3HkcMAN51zi3K9ATn3BLgU6B9UMu4zcyeB8YFNZRrg3S8ZWbDgzQ2MrNbgprB00DrDG7VFZgW3PMdoJOZpV33Jps2za7Anc65HsDiNMfdAIx2zvUEDgd8JvQOMqsKM1vbzOYCXwPPOOdmZ5ooM+sD/Oyc+zbY9f+Afs65kcAlwGTnXC9gb+AaM2sKnA6scM7tAFwN9Ihcb1wN1bhLgBPMbBHwJHBWpmksI3nLY2AM8CdifCF6BczjTkAXM/u3mb1qZgPiprUM5DOPvaHAA3ESZWYdgQ7Ax8GuHsAg59zRJL7MlgZ5vDNwmpm1Bw4FtiRREzkF6BO53igzq26S3XnAIcExuwJbBD81ymZE0ELn3BsZHNcf6GzB9PUkooNmzrmZwMzqTnDO/QJ0N7OWwONm1sU5924t9/mTmR0H/AAcEdk/0Tm3JtgeAOxvZhcEr5sC7YE9gNHBveeY2TuRtBxfw/2OAm53zl1vZrsD95rZ9i6I8+uJvORx0M70hXNurpn1j5GeQudxY2AroC+JD/BLZtbVOfd9jDSXurx9jgGCL6wDgHMzTM9RZtYX+AUY7pxbGdzzSefcz8ExA0h8mQ0NXm8AbE0ijx8I/hYWmdmL/qLOuYtruN8o4IYgSJsX/KxOl8BsCs1Vke01JKpLXrTqY0CvoCCMxTm3wsxeAfYFais0/+6cG1NLOo1EG8fC6AFBpsQt7IYBewbpnGFmLYCWwLfpTioz+crjPsDBZnZgcJ0WZna3c+7YWs4rdB4vAl50zq0GFprZQhJR7ZyY1yll+f4cHwDMdM4tz/D4Cc65s6vZXzmPT3XOTYseYGZDiJnHzrnvgGOD8xuRaBL4NN05OelyFJTsK8xs6+DGQyJvTwVO8y9qqAYReX8TM9sg2G4O9APeC16P9m1UdTQFODNyL19Fe5lE5IiZdSPRIFybz4O0YWbbkmhEr08FZopc5rFzbqRzbgvnXAXwR+A5X2CWWB4/AewVnLMJiQLzkyzSVtJymccRR1Kpam5mZ5nZiCySOgU41cwaB9frbGbNSOTx0KBtsy2JGkJaZrahmTUJXp4MTI08U6lWLvtp/hmYTKJRNdrgexqwW9BguwA4MUhsTW0hbUhUg+YBrwOTnHOTg/d2AJZUc06mLgeaW6LLyjskniQC3AS0MrO3gHNIPLEnSGdN7V3nkMi4eSSe0h2XRbrKRa7yOJ1SyuNJwI/Bv2kqcI5/KFSP5SyPzWw9El86lbuUdQG+ySKNtwEfAnPNbD4wlkSt+RESwcx8EvmdXDQ9TZvm9sACM3uPRBBUazNC2QyjtET9arJzbt9ip0XyQ3ncMJjZJGBw0OxRdsqm0BQRKQUaRikiEoMKTRGRGFRoiojEoEJTRCSGrNYIat26tauoqMhRUsrD7Nmzl7sGNKu38rj+Ux7Hk1WhWVFRwaxZs2o/sB4xswa1LIDyuP5THsej6rmISAwqNEVEYlChKSISgwpNEZEYVGiKiMSgQlNEJAYVmiIiMajQFBGJIavO7cXy448/JrdHjEhMAD1hwgQAdt11VwBefPFFANZeO69Ld0uOrF6dmFrRr0Gz1lprFTM5kgNffplYoHXs2LEA3H///QB8/PHHVY49/vjEMk177703AEcckVgCqkmTJlWOLTZFmiIiMZR0pNm1a1cA9ttvPwA+/PBDAF566aXkMXvssQeQGAoG8OqrrwLhN9YBBxyQPPbCCy/Mb4IltmeeeQaAo48+GoDWrRNLVV900UXJY449NrHeWqNG+o4vVWvWrElu+4hy1KhRALz33nu1nj9u3LiU3/7cadMSa6e1adMmd4nNkv4KRURiKKlI8+uvvwZg8ODBALz77rspv6vzf//3f0DYFrbhhhsC8OabbwLwxhvhks7+G+/uu+/OZbIlC9tttx0AJ5xwAgCPPPIIAMOGDUse46OPO+64A4BOnToVMomSAZ83ACeffHLKey1atADgmGOOAaBjx45Vzve1yNtuuw0IP6vnnptY58w/s4Dit3cr0hQRiaHokaZ/yg1w5JFHArBkSfUruHbvnlhl9YILLkju+93vfgeEkaZvW1m6dCkQtm0CvP766zlKteSKb4u+5pprUn4/99xzyWP8k9SddtoJCKOQtm3bFiqZUoMHH3wQgOuuu67Ke507dwbg2WefBWDLLbes9Xp9+yaWKj/rrLMAeOihhwAYPXp08pj27dtnkeLsKdIUEYlBhaaISAxFq577hl/fnQSqVsvXWWcdIKyO+y5Dfn86/uHRypUrk/t23nlnAH799VegNDvOSsKAAQOS2/4hw+GHHw7A22+/Dah6XgpefvllILVb0WabbQbAv/71LyCzarl32GGHAXD55ZcD8NVXX+UknbmkSFNEJIaCR5r+Qc3w4cMB+Pzzz2s81neSPfjgg2Pf57fffquyz3c/8t9exW5QlswccsghQNjVaM6cOUA46EFKix+osNVWWxU5JfmhSFNEJIaCR5pz584FwraQKN9W6bsxHHjggXW+j++qEOXbWhRhFs/ixYuBMH/8kNiWLVsCcOihhwKw++67J89ZtGgRAKtWrQJgyJAhhUms1In/nGXD1y7feecdAB599NHke+ecc07W18+GIk0RkRiK3rk9yj85O+igg+p8DT8U0w+vjBo6dGidryvx/fDDDwBMnDgxuc9HCc45AFq1agWEbdB+iGs0WvE1g969ewOwzTbb5DPZkiU/FNIPgayL+fPnp7xesGBBVmnKJUWaIiIxFDzS3GGHHQCYNGkSEE4NBnD99ddnff0bb7wRgPfffx8Ih+kBnHrqqVlfXzI3ffp0AKZMmZLcd/bZZwPhhBw+ivzll19Sjo22Z/v+u37ih88++wyADh065C3tUnd+8mH/3MJP3xjHBx98kNM05ZIiTRGRGAoeaTZunLjlwIEDU35ny7eRRaeCg9SRJRtvvHFO7iXpzZgxAwj760Wn9fr9739f7Tl+WZLoUiaeH1HiIxfftvnkk0+mvJbC81P6Pf7448l9vmZw+umnp/z2E3h4n3zySXI7+jcCYW2i8jUhrJUUaykbRZoiIjGo0BQRiaGkuhzVZsWKFQB89913yX2+a5Ff4S46DyNAz549C5Q68c4//3wgHEbn50hMx8+5eN555wHh3KkAkydPBmDmzJlAONfinnvuCYTDKkHdkQrNf77OOOOM5L6LL74YCCdWqTyTe11EHxh/++23QG460deFIk0RkRjKItL0EabvpuIfNAAsW7as2nPatWsHqEN7Mfgp2/baay8A1l9//RqP9RO2nHjiiQA0bdoUCCNPgE033RQIuyH5VUr32WcfIHUwhK9paKhsYfkaAoS1hFtuuQWAjz76CAjXBqpuBYVevXoB4d+Bj1L/8pe/5CnFdadIU0QkhpKONH1ndD8JrZ88OBO+e0O6KEfyw08a7YfFRjuhDxo0CAjbqHz7p48m/UQe6dqrfMTy/PPPA2HECWEXs9mzZwOw7rrrZvNPkQxFJwav3J3Qdxfyeeojz6jKK1T+9NNPNd7LT3isNk0RkTJQkpHmww8/DMCtt94KhB3X4/BtItGB/36Nbckvv6yIX+c6OiTST/nlh9FuvvnmALzwwgsAtGnTJuP7VI44AXbccUcgXMHSTxbSrFmzmP8KyZXKEWF1657H4Sfv8L0nCk2RpohIDCUVafp2DL/+uY8w/aQbn376aY3n+gjjzDPPBMLhXf5pHMCrr76a0/RKeiNHjgRSJ5z2C+r5WoSPPP1kHHURjVx8ZOmXyNhll12A8IltJovySeaWLl0KpE4aPXjwYABOO+00IHXSnFyoy/I3uaRIU0QkBhWaIiIxlFT13PMrVnrpquW+E/Njjz0GhN1bRowYUeXc5cuXA9C6detcJVXSWG+99YBwGCRAo0aJ7+l8zVCz7777AuFsV77Zxs+u5Ofr9OmQ7PiZ9/3AEwi7nPmZ+/3rXFfTi0V/OSIiMZRkpFmb6NDIG264Aah5rszoPHyvvPIKoNUMC80PjSykLl26AHDnnXcC4dyeN910ExA+MJTsrLXWWgAMHz48uW/cuHFAuFaQX3HUT+rhu5WlWwvMD07wfH4CNG/ePNtkZ0WRpohIDCUVafpZ3bt16wbAvHnzgLCbiF/dLjqIv6ZOy/vvvz+QOqO0H7qnSLPh+MMf/gCEE7v41TD9hC6gv4dc8G2bEE6a4oe0+mGPvgtSkyZNgLDNuzq+PdSLrmyZTfe0XFCkKSISQ0lFmv4byA+Lu/TSSwE47rjjgHD6qEz4dpPoU9r+/fvnIplShvw6Nb7zu18VExRp5prv0eI/x5dccgkA48ePB8KJd/yUj+n4oc+HHnporpNZZ4o0RURisLpMhuH17NnTzZo1K4fJyb1om6ZvC+nXr1+dr2dms51zDWYNjXLI4zh8H+BoX2Dflu4pj3PLlzGrV68G4L777gNSV6P0T9z9Eil+oum//vWvQO5Xks0mjxVpiojEUFJtmvmg9iqJ8iOBNCKocMwMCJ9ZHH/88VWO8RFlOdBfjohIDCo0RURiUKEpIhKDCk0RkRhUaIqIxKBCU0Qkhqw6t5vZMuCz3CWnLHRwzuW2p20JUx7Xf8rjeLIqNEVEGhpVz0VEYlChKSISgwpNEZEY0haaZtbKzOYGP0vMbHHkdV6WEzSzDmb2opktMLN3zOz0DM4ZbmbLgnS9a2YnZJmG+8ys5gVMEsdsZGZPmdlbZjbTzLpmc89iUR5ndOyuZvZbpseXmmLkcXDfc4P8fcfMzsjg+GJ8jlua2SQzmxek85haL+ycy+gHuAw4v5r9BjTK9DoZ3KcN0D3YbgEsBDrVcs5wYEywvRmwHGhd6ZjGMdJwH3BQLcdcB1wcbG8LPJ+r/4Ni/SiPqz2uMTAdmJzJ8aX+U8A87g7MA5oBTYL/wy1LLY+BS4BRwfamwIra7lGn6rmZdTSz+WZ2K/Am0M7MVkbeH2pmdwTbm5rZY2Y2y8xeN7Nd0l3bOfelc25usP098B7QNtO0OeeWAJ8C7c3sCjO7zcyeB8aZWWMzuzZIx1tmNjxIYyMzuyWIfJ4GMlkYvSswLbjnO0AnM2uV/pTyoTxOOht4kMQHuF7JZx4DXYBXnXM/Oed+BV4GMp5yrIB57ID1g+31SOTzb+lOyKZNsytwp3OuB7A4zXE3AKNdYsLPwwGfCb2DzKqRmW0FbAe8kWmizKwj0AH4ONjVAxjknDsaOAlY6pzrBewMnGZm7YFDgS2De50C9Ilcb5SZDazmVvOAQ4JjdgW2CH7qkwadx8F5BwD/zDRtZShfefw2sKclmrHWBfYH2lVzXLUK+Dm+HuhuZl+S+Eyf4YKwsybZzKe50DmXyR96f6CzBXPqAS3NrJlzbiYws6aTzKwF8CiJf8SPGdznKDPrC/wCDHfOrQzu+aRz7ufgmAFAFzPzC6dvAGwN7AE84JxbAywysxf9RZ1zF9dwv1HADWY2l8R/9jxgdQbpLCcNPY/HACOdc2si/7b6Ji957Jybb2bXAlOBH4E5ZPb5KHQeDwReB/oCnYDJZrZ9ur/HbArNVZHtNSTaRLymkW0Dejnnfsn0wpZonH4MGO+ceyrD0yY4586uZn80nQac6pybVul+Q0iE6Rlzzn0HHBuc34hEVeLTONcoAw06j4GewMTgQ9saGGBmvznnno55nVKWtzx2zt0O3A5gZqOBjzI4rdB5fDxwWRBdvm9mX5AoPN+s6YScdDkKSvYVZrZ1UIBE2y6mAqf5F2bWPd21LPEXOh6Y65y7vtJ7Z5nZiCySOgU41cwaB9frbGbNSLS3DA3aRNqS+NZJy8w2NLMmwcuTganOuVXpzilnDTGPnXPtnXMVzrkK4AngpHpWYKbIZR4Hx2wS/K4ABgMPBa9LJo+Bz4F+wXU2BzoCn6Q7IZf9NP9M4gnjNGBRZP9pwG5Bg+0C4MQggTW1hfQFjgT2sbBbxL7Be12Ab7JI423Ah8BcM5sPjCURbT9C4j9vPnATif98gnTW1BayPbDAzN4j8Z9+bjXH1DcNLY8bolzlMcATwbFPACcHtTMorTy+DOhrZm8Bz5PoWZB2beGyGntuZpOAwc65+tZ2KAHlcf1X7nlcVoWmiEixaRiliEgMKjRFRGJQoSkiEkM2/TRp3bq1q6ioyFFSysPs2bOXuwY0q7fyuP5THseTVaFZUVHBrFmzsrlE2TGzBrUsgPK4/lMex6PquYhIDCo0RURiyKp6Xgr+85//ADB0aGLs/lZbbQXAmDFjipYmEam/FGmKiMSgQlNEJIayr54vWpSYU+DppxOTzzRr1gyASy+9FICWLVsWJ2EiAsDbb78NwN577w3A8uWJSfDfeCOcxrNnz56FT1gdKdIUEYmh7CPNyjbddFMA1l47b4vsiUgGhg0bBsC9994LwOrViUmNOnXqBMBmm21WnIRlSZGmiEgM9S7S3H///QFYd911i5wSSefnnxPLvVx77bUAHH300QC0a1d17a0vvvgCgLPPTqyCsGDBAgDmzZsHqFZRqqZMmQJUjTAnT54MwBZblOc6hIo0RURiKPtIc+zYsQCss846QBiNSGnyUcepp54KwLhx4wB4+OGHAZg7d26Vc/75z8QKuo899ljK/n/84x8AXHTRRflJrNSJz9uvv/4agM6dOwPw7LPPAomx7uVMkaaISAxlGWl+/vnnye3x48cD0Lx5cyBsN5HSdMoppwBhhOn5vnyPP/44AIMHD06+N2fOnGqvdcsttwBwwQUXJPc1aqQ4oNgmTpwIwG+//QaEtYhyjzA9/YWJiMRQlpHm1KlTk9srV64E4KqrripWciQG365V2Zo1awBYtSqxdPxHH32UfG/atGnVnrN06VIApk+fntzXr1+/nKRT4rnrrruS2/4zecQRRwDQpUuXas/58ssvk9svvPBCynt+9FCbNm1yms5cUKQpIhKDCk0RkRjKqnruq2OjR49O7vNDsY477rhiJElyZP311wegf//+QOoQO9+d7Keffko5xw+ZVZW8+L7//vvktn8A1Lt3bwAaN04UM75p5uqrrwbg448/Tp7jJ97xfMd3P0ildevWyffOPPNMIJzkY8stt8zRvyIzijRFRGIoq0jTf1O9//77yX2HHXYYEEYdPhrxnah9BCPF8+STTya3v/rqq2qPWWuttYAwwnzvvfeS7/khl1K6br755ir7/IOgZ555Bgg/q//9739rvV7lyDP6mf/3v/8NQNeuXVOuX6guTYo0RURiKItI03c9E9lnAAAKz0lEQVRDueeee6q8N3LkSCCMLP1aQX4I17/+9a/ksRtttFFe0ympfNR/xRVXJPf5rkWVHXnkkSmvt9lmm+R206ZNgaoR56+//gqEXVwANtxwwyxSLHHdfffdAHz66adV3vPDXP2ABR9h7r777gCcf/75yWPbtm2b9j4PPfRQcvv+++8HwolbbrzxRgCuueaa2OmvC0WaIiIxlEWked111wFhB9i99tor+Z5/gvbcc88B8NRTT6Wc66cVA0WahbZw4UIAZs2aVeuxvXr1SnkdjVx8RFmZr01EO7cPGTIkbjIlCz4P/BPzKP+59fxzBz/02a8cm4mddtopuX3AAQcAYTlw0003AbDLLrsAYdtpvijSFBGJoaQjzfnz5wNw++23p+w/4YQTktt+kaYzzjgj5ZjNN98cKN8p9euDTJ6Sej5PfRt19Fzfpl2Tyy+/PLm9xx57ANCqVauM7y35tckmmwDw4IMPAvEizOpsvfXWQBi5+mj3tddeAxRpioiUlJKKNH3blZ8O308jtnjx4pTjDj744OS2n1L/gw8+SDnGj0LwT9UhjF78CBPJLz95cCaccwAsW7Ys9n18jwlQhFlKNt54YwBGjBgBQN++fXNyXT+Jh49cfdvmnXfeCeT/KboiTRGRGFRoiojEUPTq+XfffZfc9t1Fol1IqpPJSpO+q1F0xbv27dsDcMcddwCwzz77xEusxOIb5G+77ba83sevQSOlpXL1Odc6duyYl+vWRpGmiEgMRYs0fYQZHUpVOcJcb731Uo5p0aIFAA888EDymDfeeCPje/qHQ2+++SagSDPf9txzTyC1c/uhhx4KwJIlS4DsJuPw6wHpwV5pig6FzQff3bDQFGmKiMRQ8EjTdyvy0aNvX6yO77R87rnnAmFU8re//a3KsWYGQLdu3YBwjZFBgwYlj9lxxx2BMGKV/PLTvUWHwH3yyScAvP766ymvK/uf//mf5HZ0vaAo3/Vk4MCB2SdWyoYvQ6688sqU/YccckhB7q9IU0QkhoJHmh9++CGQPsI8+uijATjrrLNS9vvpoVasWFHlnP322w9InQpOSpefoKPyRB3etddeW8jkSB3tv//+AFx66aXJfX4QyYQJEwAYNmwYAC1btszJPf2UcH59dT/R+HnnnZeT69dGkaaISAwFjzSji6JV5hdI8m2Wvk3Mq26I3THHHAPAuHHjcpVEKQF9+vRJbvv2z8r88gZ+YheA7bbbLq/pklTbb789kNqe6CcJ9pOv+Frl6aefDoTDKn1vlnT8MOhou7ZfRsPzPTL88hf5pkhTRCQGFZoiIjEUrHr+zTffAFU7sEc7JvtO6x06dKj2Gn62I79mDIShuu/oLPWD7x6WzowZM4Dw4SKoel4s0Ycwvvvfo48+CoQzkPn1ymfOnAmkrmVeme8y+PDDDwPhQyUIZ7IaM2YMkL9hmjVRSSMiEkPBIk3fIbXysLlJkyYlt3v37p32Gr5h2T/8AejRo0eukihlKvogSGsEFUf0c3jvvfcC4efVP/ytLmqsyfXXX5/yOroCw2WXXQbASSedVPcEZ0GRpohIDAWLNP03hV/Poy78uj/+twgUrquJxOO7I/nI88ILLwTgf//3f4H0Eaef0tFHk9EuTV26dMl9YmNQpCkiEkPRJyEWqY6fjAOgSZMmQNX1z/3T1912261wCZM68zUCH3n63+VGkaaISAyKNKUk9evXL7m96667AvDyyy8DcOSRRwLhqqRa214KSZGmiEgMijSl5L300kvFToJIkiJNEZEYVGiKiMSgQlNEJAYVmiIiMajQFBGJQYWmiEgM5pyr+8lmy4DPcpecstDBObdxsRNRKMrj+k95HE9WhaaISEOj6rmISAwqNEVEYkhbaJpZKzObG/wsMbPFkddr5zNhZtbYzN4ysycyOPaKSNreNrMDsrz3DDPrnsFxR5rZAjN7x8zuyeaexaI8TntMhZm9EKRxupm1SXd8qSpWHpvZRmb2mJm9Z2bvmlmvWo4fbmbLgnS9a2YnZHn/+8zsoFqOMTO7xcw+MrN5mXzu0449d859A3QPLn4Z8KNz7h+Vb0qibXRNbTeL6VxgPtA8w+P/7pwbY2bbAdPNbBMXabA1s8bOudW5SpyZbQOcD/Rxzq00s01yde1CUh6ndR1wp3NugpkNAEYBx+fw+gVRxDy+EXjKOXdwUDg3y+CcCc65s81sM2C+mT3lnFseSWeu83gQ0M4519HMdgduBtJO0Fqn6rmZdTSz+WZ2K/Am0M7MVkbeH2pmdwTbmwbfNrPM7HUz2yWD63cA9gHGxU2bc24+YEDL4JvmGjObDlxpZuuZ2fggHXPMbFBwv+ZmNjGIKB4Emqa7R+Ak4Ebn3MrgvkvjprWUKY8B6ApMC7anAQfHTWspy2cem9lGQG/n3HgA59wvzrnvMk2bc24J8CnQPqhl3GZmzwPjLFFDuTZIx1tmNjy4Z6MgalxgZk8DNa8RHBoM3BPccwawmZmlfaqeTZtmVxLfwj2AxWmOuwEY7ZzrCRwO+EzoHWRWdcYAfwJiP9o3sz7Az865b4Nd/w/o55wbCVwCTHbO9QL2Bq4xs6bA6cAK59wOwNVAj8j1xtUQsncCupjZv83s1SASqW8aeh7PA/ziNIcALcxsg7jpLXH5yuOtgGVmdk/w5XW7mWVao8DMOgIdgI+DXT2AQc65o0kELEuDPN4ZOM3M2gOHAlsC2wGnAH0i1xtlZgOruVVb4IvI60XBvhplMzXcQufcGxkc1x/obMEC8iSig2bOuZnAzMoHW6IN4gvn3Fwz6x8jPX8ys+OAH4AjIvsnRqocA4D9zeyC4HVToD2wBzAawDk3x8ze8Sc752qqjjUm8YfRl0TmvmRmXZ1z38dIc6lr6Hl8DnCTmQ0DXgKWALmsGpaCvOQxic9HT+AMYDaJqvqfgMtruc9RZtYX+AUYHjR9ATzpnPPrfw8gEbAMDV5vAGxNIo8fCP4WFpnZi/6izrmLa7ifVbMv7Rd5NoXmqsj2mko3j1Z9DOjlnPslw+v2AQ42swOD67Qws7udc8fWct7fnXNjakmnAQc55xZGDwgyJW7Eswh4MWhfWWhmC0lEPHNiXqeUNeg8ds4tBoYE57cADnHOrUp/VtnJVx4vAj5zzs0CMLNHgbMzOG+Cc6664yrn8anOuWnRA8xsCHX7HLcDXgtebwF8me6EnHQ5Ckr2FWa2tZk1IvhDC0wFTvMvaqgGRa810jm3hXOuAvgj8Jz/MJnZaN9GVUdTgDMjafFVtJeBo4J93YBtM7jWE8BewTmbkCgwP8kibSWtIeaxmbW2MLS6iKBKWl/lOI8XAUuDajZAP2BBcO5ZZjYii6ROAU41s8bB9TqbWTMSeTw0aNtsS6IWWJungGOC6+wOfO2cW5buhFz20/wzMJlEg/miyP7TgN2CBtsFwIlBAtO1d9VkBxJVpLq6HGhuiS4r7wCXBftvAlqZ2VskqmSz/Alp2rsmAT8G/6apwDn+oVA91tDyuB/wgZl9AGwEXJVFuspFLvP4DOCh4P98W8L/vy7AN1mk8TbgQ2Cumc0HxpKoNT8CfE6iR8ZNJApRgnTW1Kb5NLA4qCneQuSLoSZlM4wy+Maf7Jzbt9hpkfxQHjcMZjYJGJzjrkMFUzaFpohIKdAwShGRGFRoiojEoEJTRCQGFZoiIjGo0BQRiUGFpohIDCo0RURi+P/1ZRbaO2r82AAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "print_test_accuracy(show_example_errors=True)" ] @@ -1722,109 +1541,11 @@ }, { "cell_type": "code", - "execution_count": 53, + "execution_count": null, "metadata": { "scrolled": true }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Optimization Iteration: 1901, Training Accuracy: 93.8%\n", - "Optimization Iteration: 2001, Training Accuracy: 93.8%\n", - "Optimization Iteration: 2101, Training Accuracy: 98.4%\n", - "Optimization Iteration: 2201, Training Accuracy: 98.4%\n", - "Optimization Iteration: 2301, Training Accuracy: 96.9%\n", - "Optimization Iteration: 2401, Training Accuracy: 98.4%\n", - "Optimization Iteration: 2501, Training Accuracy: 92.2%\n", - "Optimization Iteration: 2601, Training Accuracy: 98.4%\n", - "Optimization Iteration: 2701, Training Accuracy: 96.9%\n", - "Optimization Iteration: 2801, Training Accuracy: 93.8%\n", - "Optimization Iteration: 2901, Training Accuracy: 96.9%\n", - "Optimization Iteration: 3001, Training Accuracy: 95.3%\n", - "Optimization Iteration: 3101, Training Accuracy: 96.9%\n", - "Optimization Iteration: 3201, Training Accuracy: 95.3%\n", - "Optimization Iteration: 3301, Training Accuracy: 92.2%\n", - "Optimization Iteration: 3401, Training Accuracy: 98.4%\n", - "Optimization Iteration: 3501, Training Accuracy: 95.3%\n", - "Optimization Iteration: 3601, Training Accuracy: 95.3%\n", - "Optimization Iteration: 3701, Training Accuracy: 95.3%\n", - "Optimization Iteration: 3801, Training Accuracy: 96.9%\n", - "Optimization Iteration: 3901, Training Accuracy: 98.4%\n", - "Optimization Iteration: 4001, Training Accuracy: 98.4%\n", - "Optimization Iteration: 4101, Training Accuracy: 98.4%\n", - "Optimization Iteration: 4201, Training Accuracy: 93.8%\n", - "Optimization Iteration: 4301, Training Accuracy: 98.4%\n", - "Optimization Iteration: 4401, Training Accuracy: 100.0%\n", - "Optimization Iteration: 4501, Training Accuracy: 98.4%\n", - "Optimization Iteration: 4601, Training Accuracy: 92.2%\n", - "Optimization Iteration: 4701, Training Accuracy: 96.9%\n", - "Optimization Iteration: 4801, Training Accuracy: 95.3%\n", - "Optimization Iteration: 4901, Training Accuracy: 93.8%\n", - "Optimization Iteration: 5001, Training Accuracy: 98.4%\n", - "Optimization Iteration: 5101, Training Accuracy: 92.2%\n", - "Optimization Iteration: 5201, Training Accuracy: 96.9%\n", - "Optimization Iteration: 5301, Training Accuracy: 100.0%\n", - "Optimization Iteration: 5401, Training Accuracy: 100.0%\n", - "Optimization Iteration: 5501, Training Accuracy: 90.6%\n", - "Optimization Iteration: 5601, Training Accuracy: 96.9%\n", - "Optimization Iteration: 5701, Training Accuracy: 95.3%\n", - "Optimization Iteration: 5801, Training Accuracy: 98.4%\n", - "Optimization Iteration: 5901, Training Accuracy: 95.3%\n", - "Optimization Iteration: 6001, Training Accuracy: 100.0%\n", - "Optimization Iteration: 6101, Training Accuracy: 95.3%\n", - "Optimization Iteration: 6201, Training Accuracy: 96.9%\n", - "Optimization Iteration: 6301, Training Accuracy: 90.6%\n", - "Optimization Iteration: 6401, Training Accuracy: 98.4%\n", - "Optimization Iteration: 6501, Training Accuracy: 96.9%\n", - "Optimization Iteration: 6601, Training Accuracy: 100.0%\n", - "Optimization Iteration: 6701, Training Accuracy: 100.0%\n", - "Optimization Iteration: 6801, Training Accuracy: 98.4%\n", - "Optimization Iteration: 6901, Training Accuracy: 100.0%\n", - "Optimization Iteration: 7001, Training Accuracy: 96.9%\n", - "Optimization Iteration: 7101, Training Accuracy: 100.0%\n", - "Optimization Iteration: 7201, Training Accuracy: 96.9%\n", - "Optimization Iteration: 7301, Training Accuracy: 92.2%\n", - "Optimization Iteration: 7401, Training Accuracy: 96.9%\n", - "Optimization Iteration: 7501, Training Accuracy: 95.3%\n", - "Optimization Iteration: 7601, Training Accuracy: 95.3%\n", - "Optimization Iteration: 7701, Training Accuracy: 98.4%\n", - "Optimization Iteration: 7801, Training Accuracy: 96.9%\n", - "Optimization Iteration: 7901, Training Accuracy: 95.3%\n", - "Optimization Iteration: 8001, Training Accuracy: 96.9%\n", - "Optimization Iteration: 8101, Training Accuracy: 95.3%\n", - "Optimization Iteration: 8201, Training Accuracy: 95.3%\n", - "Optimization Iteration: 8301, Training Accuracy: 95.3%\n", - "Optimization Iteration: 8401, Training Accuracy: 100.0%\n", - "Optimization Iteration: 8501, Training Accuracy: 100.0%\n", - "Optimization Iteration: 8601, Training Accuracy: 98.4%\n", - "Optimization Iteration: 8701, Training Accuracy: 96.9%\n", - "Optimization Iteration: 8801, Training Accuracy: 100.0%\n", - "Optimization Iteration: 8901, Training Accuracy: 96.9%\n", - "Optimization Iteration: 9001, Training Accuracy: 98.4%\n", - "Optimization Iteration: 9101, Training Accuracy: 96.9%\n", - "Optimization Iteration: 9201, Training Accuracy: 98.4%\n", - "Optimization Iteration: 9301, Training Accuracy: 100.0%\n", - "Optimization Iteration: 9401, Training Accuracy: 98.4%\n", - "Optimization Iteration: 9501, Training Accuracy: 96.9%\n", - "Optimization Iteration: 9601, Training Accuracy: 96.9%\n", - "Optimization Iteration: 9701, Training Accuracy: 100.0%\n", - "Optimization Iteration: 9801, Training Accuracy: 93.8%\n", - "Optimization Iteration: 9901, Training Accuracy: 96.9%\n", - "Optimization Iteration: 10001, Training Accuracy: 95.3%\n", - "Optimization Iteration: 10101, Training Accuracy: 96.9%\n", - "Optimization Iteration: 10201, Training Accuracy: 98.4%\n", - "Optimization Iteration: 10301, Training Accuracy: 96.9%\n", - "Optimization Iteration: 10401, Training Accuracy: 100.0%\n", - "Optimization Iteration: 10501, Training Accuracy: 98.4%\n", - "Optimization Iteration: 10601, Training Accuracy: 100.0%\n", - "Optimization Iteration: 10701, Training Accuracy: 100.0%\n", - "Optimization Iteration: 10801, Training Accuracy: 100.0%\n", - "Wall time: 3min 3s\n" - ] - } - ], + "outputs": [], "source": [ "%%time\n", "optimize(num_iterations=9000) # We performed 1000 iterations above." @@ -1832,57 +1553,11 @@ }, { "cell_type": "code", - "execution_count": 52, + "execution_count": null, "metadata": { "scrolled": true }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Accuracy on Test-Set: 98.8% (9884 / 10000)\n", - "Example errors:\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe4FdW9//H3F1EBsYOCSok/pSixi4qKBWNXsAVCDIJB\nTLBAvLZYErkCtthIxBoRRb15EBSUq8YS8EIUBUG6YiWgKBgBEUvU9ftjZu2Zfdrec3Y/fF7Pw3Nm\nz57y5ayz1/7OmjVrmXMOERHJTqNSByAiUklUaYqIJKBKU0QkAVWaIiIJqNIUEUlAlaaISAKqNEVE\nElClKSKSgCpNEZEEGueyc4sWLVz79u3zFEplmD179mrnXMtSx1EsKuOGT2WcTE6VZvv27Zk1a1Yu\nh6g4ZvZRqWMoJpVxw6cyTkaX5yIiCajSFBFJQJWmiEgCqjRFRBJQpSkikkBOd89FRPLhnXfeAeD8\n888HoG/fvgCcd955JYupNso0RUQSKOtMc/369QA88cQTtW4zY8YMAP76178C0KtXLwAGDRoEwPHH\nH1/IEEWknnx2CXDSSScB8P777wPw4YcfAso0RUQqXllmmj7DvOSSSwB46aWXAOjcuTMA22+/fbV9\nfEY5bdq0tJ933nlnapuzzz67QBGLSLb8Z/KOO+5IrVu2bFnaNu3atStqTEko0xQRSaAsM813330X\ngC5dugBw3333Zb3vsGHDABg+fDgAAwYMSL2nTLN8rFmzBoClS5cC8Nhjj6W9H89CzKzGY7Rq1QqA\nV199NbWunDOUjd33338PwKJFiwD46KPo8W9fxh06dABg3LhxRY4ue8o0RUQSKMtMc5999kn7mcSQ\nIUMAePDBBwH4+OOPU+8tWbIEgE6dOuUaotSTzyBGjhwJwNtvv13jdvHscu+99wbgP//5DwCLFy8G\n4NNPPwVg5cqVqW2VaZave+65B4AHHnig1m1atGgBwC677FKUmOpDmaaISAJlmWnmYurUqQCsXbu2\n2nurV68ucjQC6e2Vv/3tbwHYsGEDANtttx0Ap59+OhBlld27d0/t47NH3ybWpk0bAL7++utqxz/o\noIPy/x+QnPirPd+X2jmX9jPulltuKV5g9aRMU0QkAVWaIiIJNLjL87lz5wKwbt06AA4//PDUe4cd\ndlhJYtpY+UvweMP//vvvD8A111wDwKGHHgpA06ZNMx7PX45X7YJ01lln5R6sFIzvWjRv3jyg5i5k\np556KgD77bdf8QKrJ2WaIiIJlDzT/Oabb1LLI0aMAGD69OlA9W8k30jcsWPHasfx3YkefvhhADbZ\nZBMAfv3rX+c5YslWs2bNAHj55Zfzcrxbb70ViDLY3XffHYger5XytOWWWwJRd6Kabsj6BxT8IB7+\nwZZypExTRCSBkmea8Y7Jm266KQBHHXUUAGPHjgWih/kPPvhgAE455ZRqx3n66afTXvt5nH0XFqlc\nb7zxBgA33XRT2nrffammAVykfPis0Q/bWFPndp99jh49Ou1nOVKmKSKSQMkzTZ8RAvzhD39Ie893\nePaPRvpBHuJZ5Q8//ABEbZje3/72N0CZZqX68ccfU8vPP/88ELVlbr311kB0RSKVwfeYqOsxSv/Z\n9tNelOPnV5mmiEgCJc806+LbQvwgxP369QOi/l51mTRpElAZ/b6kOv/IHcAf//jHtPduvPFGAPba\na6+ixiS58Y+/+ivH22+/vdo2K1asAKJ+m/Hh48qFMk0RkQTKOtP0kyvtuuuutW5zzjnnAPDVV18B\n0SRs119/fbV9/bZS/p555plq69q2bQuoHCudv3I44IADUut8G6Zvt/a9ai6++GIAzj333NS29Rky\nMp+UaYqIJKBKU0QkgbK8PPeje/sOrr47UY8ePQC46qqrUtv6cRf945jNmzcH4JFHHgFqn19GytOc\nOXOA9G5lvgwvu+wyADbffPPiByZ547uM9e3bN7Vu/PjxQDQerh9w56677gKiZjeAt956C4CWLVsW\nPNaaKNMUEUmgLDNN37XIZ5g+m/Qd1v03VVyTJk2A6nPEPPXUU9WOK+XH38i77rrrgPRRvf0VxuDB\ng4selxTHk08+CcC9994LVC/r+OPW3333XfECq4EyTRGRBMoq0/RtVp6fkW7ixIlAzRlmJvFBiKV8\njRkzBoi6GsUHJY7PXS8NWyU8sKBMU0QkgbLKND///PO014MGDQKyyzD90FL+8UlPw4aVt6VLlwJw\n9dVXp62PX3XE77JKeZo2bVq1dUcccUTW+99///0AjBw5Eqg+U2VNM1eWijJNEZEESp5pxu+Kvfvu\nu0D0rdKtW7eM+/t9fIbpJ1Zr1aoVUPcjmFI6voxvuOEGANavX5/2/sknn1z0mCQ5P6d5z549U+t8\nb5fPPvusxn0mT54MpGenn376KRDNbe/75vpHJv0+EH22S0WZpohIAqo0RUQSKPnlue+UDtEjkD41\nr3pjqKZZ7Px8Qf4y3d/48Y9iaq7z8jRhwgQgmgfK69+/PwAHHnhgsUOSevAzJ3z55Zepdb7b2JQp\nU+rcN35zx3/mt9pqKyCaD8p/vlu3bp2niHOnTFNEJIGSZ5rbbLNNarlqA+95550HwPz584Hq3YkA\n3n77bSB65PKiiy4C4Oijj85/sJI3fn7rqvw8MnXxj9P27t07rzFJcv5z5zNEgLVr12a1rx/JHWDf\nffcFolHdy3n+J2WaIiIJlDzTjOvYsWPaaz881PDhw2vdx2envm2sa9euBYpO8mnWrFlpr6+99log\nGp3922+/Tb3nH6P1o/H/+c9/LkaIkoWddtoJSB8Yxw/v540aNQqAI488EogelRw6dGgRIsw/ZZoi\nIgmUVaZ5xRVXAFH26O+o+W8x/8hd/BE733ldd8kry6uvvpr2+t///jcAixYtAuCXv/xl6j0/I6F/\n1DLJ43lSHPEyqVo+lZpR1kaZpohIAmWVaXpVZxvU4MENz2mnnQZEg876aQ38z3gfPj9wy+WXX17M\nEEVqpExTRCSBssw0peEbNmwYADNmzABgwYIFQDRAQ7y/5nHHHVfk6ERqp0xTRCQBVZoiIgno8lxK\nws9Z7eewFqkUyjRFRBJQpSkikoAqTRGRBCyXWd7MbBXwUf7CqQjtnHMtSx1EsaiMGz6VcTI5VZoi\nIhsbXZ6LiCSgSlNEJIE6K00z297M5ob/VprZitjrzQoVlJldamYLzWyBmT1qZptn2H54LLb5ZnZS\njuefbmb7ZNjmQjObF57z/8ysUy7nLJVSlLGZtTOzqWa2KCznC7PYZ6CZrQrjWmxm5+YYwzgz65Vh\nmytjv4uFZva9mW2dy3lLoYSf4+Xh53Gumc3MYvtSlPHpsc/xG2bWLeOBnXNZ/QOuAy6tYb0BjbI9\nThbnaQe8CzQJjz0BODvDPsOBoeFyF2AVYXttbJvGCWKYDuyTYZutYsunA8/k63dQqn9FLOOd/O8X\n2Ap4D+iQYZ+BwB3hcitgNdAihzIeB/RKsP1pwN9LXUaVUsbhMZcD2yTYvuhlDDQnurezH7Ag03Hr\ndXluZruFWcKjwEKgjZmtib3fx8weCJd3NLOJZjbLzF43s4OzOMWmBJVmY6AZ8HG2sTnnFhD8AWwb\nftPcbWavAyPNrLmZPRTGMcfMTgljbGZm48NvtwnhuTOdZ13s5RZAg7qjVsgyds597JybGy6vA5YA\nO2cbm3NuJfAh0Da8ynjYzGYAD5lZYzO7LYxjnpkNDGNsZGajzWyJmb0AtEj0C4FfAI8n3KesFeFz\nXG/FKmPn3HoX1phk+TnO5THKTkA/59wsM6vrOKOAm51zr5lZe+AZoIuZHQQMcM79Jr6xc+4jM7sT\n+BfwLTDFOfdytkGF6fU3zrl/WzDye2vgYOfcj2Z2M/Ccc66/mW0LzAx/uRcCXzjnOpvZvsCs2PHG\nAHf6D3mVc10MDCGo5Mt3+rz6K0gZx5nZrgRXB29kG5SZ7UZwRfJ+LM7uzrlvzGww8JlzrqsFzTqv\nmdnfgYOBnwB7EGS6i4B7wuONAGY45/63lvM1B44Bzss2xgpSyDJ2wMtm5oDRzrm/ZhtUMcvYzM4E\nRhBUsidmii2XSvM959yszJtxDNAxrMAgyACbOudmAtXaOcxse+Bkgv/8OmCCmfVxzv1PhvNcZmb9\ngS+B+Nyu451zP4bLxwInmNmV4esmQFugO3AzgHNujpkt9Ds75wbUdkLn3ChglJn1A64Cfp0hxkpT\nkDL2zGwrguaXi5xz67M4zy/N7EiCL9OBzrk14TknOee+Cbc5FuhsZn3C11sDuxOU8ePh38JyM5vq\nD+qcuzrDeXsC05xz2c1NW1kKWcYHO+dWmFkr4AUzW+yc+2eG8xS9jJ1zTwBPmNlRwPXh8WuVS6X5\nVWz5R4JLYi9+eWtAV+fcd1ke91hgqXNuNYCZPQl0AzJVmrc45+7IEKcRtHG8F98g9odQX48Bd9Lw\nKs1ClTEW3ICYCIxxzk3OcrdHnXM1TThTtYwHO+deqnK+07KNrQZ9gEdy2L+cFayMnXMrwp8rzWwS\n0BXIVGmWqoxxzv3DzMaa2TbOuTW1bZeXLkdhzf6Fme1uZo0IGs29F4EL/AvLcFcaWAYcYmZNLajN\negCLw31v9u2Q9fQ8cFEsln3DxVeAvuG6vYE9Mx3IzHaPvTwFeDuHuMpePss4LNeHgLlhth5/b4iZ\n1Xo5n4XngcH+UtPMOppZU4Iy7h22e+0MZDU7W9iM0w14OoeYKkKey7h52KyBmW0B/AxYEL4umzIO\n23UtXD6A4KZQrRUm5Lef5hUE/5l/Etw18y4ADg0bbBcRtguZ2UFmdk/VgzjnZgCTgTnAfOB7wLeF\n7AWszCHGYcAWFnSDWEhwJxHgL8D2ZrYYuDY8N2GcY2r5AxlqQTeUuQRtorVexjcgeSljgj/mXwA/\ns6jrix+evTPweQ4x3gssBeaa2QLgboIrqicIvpAXAWOA1HSYZjbCzGpryzoDeNY593UOMVWSfJVx\na2CGmb0FvA486Zx7MXyvnMr458CC8HM8ivSmvRpVzGOU4bfBs86540sdixSOmU0Bejrnvi91LFIY\nlV7GFVNpioiUAz1GKSKSgCpNEZEEVGmKiCSgSlNEJIGcZqNs0aKFa9++fZ5CqQyzZ89e7TaiUb1V\nxg2fyjiZnCrN9u3bM2tWNk9gNRxmtlFNC6AybvhUxsno8lxEJAFVmiIiCajSFBFJQJWmiEgCOd0I\nEimG2bNnA3DMMccAsM022wDw/PPPA9ChQ4fSBCYbJWWaIiIJKNOUsrJhwwYAzj///NS6Z555BoB1\n69al/TzrrLMAeOutt4oZouRJfPDv008/HcBPdsaeewZD2l5//fXFDywDZZoiIgmUdaY5alQwqPfF\nF19c4kik0ObPnw/AuecGU12/+eabqfd89lF1WpIjjzyyOMFJQcTL86mnngKisp40aRIA++4bTK7g\nM9FyoExTRCSBsso0v/oqmDvpyiuDySI/+OADQJlmQ/bxx8GU9nfcEcyJF88wMxkzZgwABx54YGrd\n2WefncfopJDuuaf6LBnXXHMNAKtXrwbghhtuAJRpiohUrLLKNH1meddddwHw+uuvlzIcKYIbb7wR\ngIceeijxvv7K5Jxzzkmt85nqPvsEc+H169cvxwilUAYNGlRtnS+/+++/v9jhZE2ZpohIAmWVaQ4Z\nMgSAn/70pwA0adKkrs2lgvmnfB555BEgumtakyST/91+++0A9O4dzMSqTLMy+TI//PDDSxxJdco0\nRUQSUKUpIpJAyS/PX3jhhdTyDz/8ANTvsbj33nsPgDVr1gCw//77A/CPf/wjtc2MGTNq3HfvvfcG\n4JRTTkl8Xqmfv/zlL0D0SGTVjuv+Rg7A5MmTgejvwndPeumll2o9/pQpU4CoW9KAAQPyEbYU2JNP\nPglEfw+nnXZaKcOpkTJNEZEESp5pPvfcc6nlRo3qrsN9R+hevXpVe89nLN9++y0AO++8MxB1kgV4\n5513ajxuy5bB/Ert2rUD1NWpGMaOHQtUzzC7du0KwMSJE1PrWrduDURluu222wJ1Z5rNmjUDYMcd\nd8xTxFIMPrO87777AN0IEhGpeCXLNH3WOG/evNS6Bx54ACA1M17btm0B2GGHHYBoMAefVULUNWHp\n0qVpx/cdnn07KcCIESNqjGXVqlVAlOVI6QwcOBCApk2bptZ9+eWXQNRe7f9O6nL00UcDcOKJJ+Y7\nRMkT/7mD6HFJ36a5xx57lCSmbCjTFBFJoGSZph9YYerUqal1fuDZZcuWAfDoo48CUaa5xRZbADB+\n/PjUPj7T/PTTT9OO3717dwD+9a9/pdY98cQTQPS45nfffQfA8ccfD8CDDz6Y0/9JcnfppZcC6YM5\nNG/eHIBXXnkl6+Oceuqp+Q1McvbRR8FU4/4ewrhx41Lv+R4Rvi162rRpRY4ue8o0RUQSKHqmOXPm\nTCBqt/SDjEI0eMOtt94KwHbbbZe274QJExKfb7fddkst+7viF1xwARB90/m7s/4bUArnkksuAWp/\nNHLt2rVA9JhlfNuqd9pr4tvEevbsmVOckn/+noH/fPvPO0Rle9VVVwHQqVOnIkeXPWWaIiIJFD3T\nvPfeewFYv349kD5orH+K57HHHivIuX27Z7wtRQrvt7/9bWr58ccfB6LMIpvs0ctmW2WY5cf3uf3s\ns88AGDlyZNprgM6dOwNRplnOlGmKiCSgSlNEJIGiXZ7/93//NxBdGvvHoy666KKCnve6665LLd90\n000ADB06FIguEzbZZJOCxrCx8jNMxm/g+Y7qVW211VZAVEYffvhh6r34DYNMfv/73wMwbNgwADbb\nbLPsA5a8WLx4MRCVuy9T37xy5plnArBw4cLUPn42yuHDhwPRXEHlSJmmiEgCRcs0//jHPwLRt40f\nnKNx48KE4BuU40PPXXHFFQAcd9xxgEaGL7S7774bgM8//7zWbbp16wZEVyJHHXVUtW38TUN/E/H7\n77+v9Xg+q9lyyy2Byrix0BD4jusAV199NRB1/zriiCOA6Oqhb9++QDTHE0SPTV577bUAtG/fHijP\n2UWVaYqIJFCyxyj9oBsrV65MrWvVqlW9j+c7y/vsxs894zuuQzRfzK677lrv80hmc+fOBeDpp5/O\nuO3gwYOBmjNMb9SoUQBMmjQJgOXLl2c8rm9PleKIz8U0ffp0IHr8+bbbbgOiAXhatGgBwIYNG1L7\n+C5Hfmg4P7iOf6xS856LiFSokmWac+bMAdLnrPYdn6s+PllVfDg5P3jHzTffDERDgfm75n7gDlCG\nWSx+sOcVK1bUuo1/fLbq0G3+7np8+Df/6GU2ndvLeRbDhsgP7xYfTMW3YcYH46mJzyLj9ttvPyC6\n8+6zVd/GGd+mVJRpiogkULRM0w+c4b+ZfJtm/O52nz59ABg9ejQAl112GQDvvvtu2rHigxBffPHF\nQDTAw0477QRkzlalcLJ5RNKXqb9b6tsr/aDRn3zySaLjeX4owFzaxyV7VSdCg/xOhubvTSxatCi1\nTpmmiEgFUaUpIpJA0S7P/Rw+fp4fPxr3a6+9ltrmxRdfBKBDhw41HmPTTTcFYMiQIal1PXr0AKBL\nly55jlgKyd/wueuuu+p9jM033xxIf1TWz1gpxeG7D/mfED2E0KZNG6B+3YX8yEhnnHEGkH75X+oO\n78o0RUQSKHqXoyuvvBKIuv/4+YAgmtel6nw/nh+EwXeIlvJ0yCGHANEVQ23zzWfLZyz+0Vtv0KBB\nAFx++eU5HV/qz2eR8c+x7y7muxMuWbIEyO6RVj9gR9VBPsppAA9lmiIiCRQ906zaXhnvcL5gwYJi\nhyMFsMsuuwDQv39/INmgGT472WuvvVLr/FB+Ur7iZeQHxDnhhBOA6IqgLr/61a+AKCv1baRjx44F\n9BiliEjFKtljlNLw+aH4/E/ZOPjBN3zH9Nr4wYohGoTYDyLts9P4XflyoUxTRCQBZZoiUhCZBk3x\nGSnUPg1KOVKmKSKSgCpNEZEEVGmKiCSgSlNEJAFVmiIiCajSFBFJQJWmiEgCqjRFRBIwP3tfvXY2\nWwV8lL9wKkI751zLUgdRLCrjhk9lnExOlaaIyMZGl+ciIgmo0hQRSUCVpohIAnVWmma2vZnNDf+t\nNLMVsdebFSooM9vOzCaa2RIzW2xmXTNsP9DMVoVxLTazc3M8/zgz65VhGzOz0Wb2rpnNM7N9cjln\nqZSwjC81s4VmtsDMHjWzzTNsPzwW23wzOynH80/PpszM7BdmtiiM9eFczlkqKuM6t7k8rDPeMrMX\nzKxNxgM757L6B1wHXFrDegMaZXucLM/1KNA/XN4M2DrD9gOBO8LlVsBqoEWVbRonOP84oFeGbU4F\nng6XDwNm5PN3UIp/xSpjoB3wLtAkPPYE4OwM+wwHhobLXYBVhDcy61nG04F9MmzTCZgNbBO+3qHU\nZaQyznsZHw00DZcvAh7NdNx6XZ6b2W7ht++jwEKgjZmtib3fx8weCJd3DLPGWWb2upkdnOHY2wEH\nOeceAnDOfeecW5ttbM65lcCHQNvwm+thM5sBPGRmjc3stjCOeWY2MDxnozBrXGJmLwDZDBfdE3g4\nPOd0oJWZNZhuKoUs49CmBB+oxkAz4ONsY3POLSD4IG4bXhXcbWavAyPNrLmZPRTGMcfMTgljbGZm\n48OsYkJ47kwGAX92zq0Jz/tZtjFWApUxOOdeds59Hb58Ddgl0z65tGl2Am53zu0BrKhju1HAzc65\nA4CfA74QDjKze2rYfldgVVjZzTGz+8ysWbZBmdluBN9y78fi7OGcO5vgQ/CZc64rcCBwgZm1Bc4E\nfgLsAQwAusWON8LMTqzhVDsD/4q9Xh6ua0gKUsbOuY+AOwl+f58QlMnL2QZlZt2Ab5xz/w5XtQYO\nds5dDvwBeC4s46OBW82sCXAh8IVzrjNBRrNv7HhjarmM6wB0NrMZZvaqmR2bbYwVZGMv47hfA89m\nii2Xkdvfc87NymK7Y4COFs5fTPDN0dQ5NxOYWUtMBxCkyrOBPwOXAcMynOeXZnYk8C0w0Dm3Jjzn\nJOfcN+E2xxJ8CPqEr7cGdge6A487534ElpvZVH9Q59zVWfwfG6qClLGZbQ+cTPBFtQ6YYGZ9nHP/\nk+E8l5lZf+BLoHds/fiw7CAo4xPM7MrwdROgLUEZ3wzgnJtjZgv9zs65AbWcrzHBl/gRBF/E08xs\nD+fcugxxVpKNvYx9vP2BnwIXZ4gvp0rzq9jyjwSptBdPiw3o6pz7LsvjLgeW+YIM0+xs5nB91DlX\n03bxOA0Y7Jx7Kb6BmZ2WZWxxK4A2BCk9BGl9Xd/UlahQZXwssNQ5txrAzJ4kyO4zfaBucc7dkSFO\nI2iPfi++QezDnsRyYJpz7nvgPTN7D/h/wJz6HKxMbexljJkdT5CYHZHN/y8vXY7Cb4AvzGx3M2sE\nxCuhF4ELYgHWmSI755YDn4aX2QA9gEXhvkPM7Dc5hPo8MNjMGofH62hmTYFXgN5h2+bOBJlFJpOB\nfuFxDgM+dc6tyiG2spbPMgaWAYeYWVML/tJ7AIvDfW/2bVT19DzBVYqPxV+ivQL0DdftDeyZxbGe\nAo4M99mBoML8IIfYytrGWMZmdgBwF3Cqr+AzyWc/zSsI/jP/JPiG9i4ADrXgxssi4Lww2NraNCH4\nhfzNzOYR/MdvDNd3Bj7PIcZ7gaXAXDNbANxNkG0/QVDIi4AxwKt+hzraNJ8GVoTZx93E/qAasLyU\nsXNuBsGXzhxgPvA98Nfw7b2AlTnEOAzYwoIuKwsJ7hYD/AXY3swWA9cSyxbraO+aAqwP/08vAr/z\nN4UasI2tjP8EbEHQfDA3zIjrVFHPnpvZFKBneLkkDUyYkTzrnDu+1LFIYTSEMq6oSlNEpNT0GKWI\nSAKqNEVEElClKSKSQC79NGnRooVr3759nkKpDLNnz17tNqJRvVXGDZ/KOJmcKs327dsza1Y2DxM0\nHGa2UU0LoDJu+FTGyejyXEQkAVWaIiIJqNIUEUlAlaaISAKqNEVEElClKSKSgCpNEZEEcuqnKZKN\ncePGpZbPOeectPceeeQRAPr27VvUmETqS5mmiEgCZZlpfvVVMLL9kiVLALj//vvT3v/ss2hSwKee\negqAQYMGpW3jM5fu3bsXLE7JTjy73GSTTdLe69+/PwBffvklAHvssQcAhx9+eHGCkzr95z//AeDz\nz4OxvxctWgTA6tXBIOdvvPEGAM8+G81H5j+/Z511VtqxLrnkEgC23nprAJo2bVqosAtKmaaISAI5\nDUJ8wAEHuFyfWR0+fHhqedKkSUD0TfX2228D+EndUxMnxWOuus6/3m+//YD0b8AWLbKZzrxuZjY7\nnMZ0o5CPMo5nl1UzTe+HH34AYM89g2ldRo8enXrvsMMOy+n8SW3sZfzxx9H05KNGjQLglltuqXHf\nqp+7uvhtfXkOGxZNMHvUUUcljDo3uZSxMk0RkQSK3qbp76T+13/9F5DePlk1a+zcuTMA7dq1A+C0\n06rPtHv66aen7dO1a1eA1Kgty5YtS22bj0xTkvN3yCFqw6yNb8f2P6H4mebG7o47ohl0b731ViD6\n7PgrOM9/7tavX59a9+qrr1KXGTNmAHDFFVek1r30UjCr9pZbblnfsItGmaaISAJFzzT9XTf/8ze/\nqT6N+XnnnQdAp06dAGjWrFnG4y5evDjtuPWdOF7yr0OHDqll33ZZ1Y8//pj2+vzzz08t+/JXX87i\n+N3vfpda/tWvfgVA8+bNAfjJT35S4z5ff/11avnFF18EonZQn1lWNXv27NTylClTAOjTp099wy4a\nZZoiIgmo0hQRSaDkXY7yxaf6Bx54IBA1XMfja9u2bc7n2di7o9THRx9FMwsMGDAAgOnTp6dt4y/b\na+uSBPDdd9/lFEe2VMb54S/Zf/7znwPRJXhNTWe+o/v48eMBOOGEE/IeT5y6HImIFElZPkZZH08+\n+SQQfYvpRlD58F3GIOq0PnjwYKB6xikNh88en376aQBOPPFEAJ577rlq227YsAGAk046CYgeq91i\niy0KHmdSyjRFRBJoMJmmH0jAt9H69su62jF9W5vvpuS3bdlyo5nyuuh8N7KOHTsCyjQ3Jg8++CAA\nO++8c8YAVfjMAAAJPklEQVRtTznlFABefvnlgsZUH8o0RUQSaDCZph8izrdl+qHF7rvvvmrb+vbP\nN998E6iead52222pbWt6dFNyd/fddwNR74Zs7t76x2p9m1i8rVTK37bbbgtAjx49gOjRyZr43jD+\nCtIPGVgOlGmKiCRQ8ZmmH1quan/T22+/HUi/i151IJBddtkFgOOOOw6A3//+92nvS+H5AVd81l9X\nP8133nkHgJtuuglIHz5Oyt/mm28OwNFHHw3UnWk2ahTkc02aNCl8YAkp0xQRSaAiM00/iABUb8v0\nP/00F/GssT4DgUhh+ez+mmuuKXEkUix+WMgvvvgite5Pf/pT2jZr164FoF+/fkB59bJQpikikoAq\nTRGRBMr68nzixIkAXH311UD1OYMguhz3l9p+lHB1FWq4fJexY489FoBevXqVMhzJ0sqVK4Fo3q4P\nPvgg9V7VG7n+tb9M//bbb1Pv+RtKpaJMU0QkgbLKNH1m6TMJf5PHz055xhlnADBhwoTUPj7T9F1X\nlGFWpqojt9e1jZ9Xyj+UIOXl/fffB2DmzJlA9DCCvwqsaTCd2gbY8Z3b/WOVADfeeCNQfb6iYlGm\nKSKSQMkyzVWrVgFw7bXXptb5DNNnlj57rNrpvKZvJc1YWJn81YXvzFxX53bPb+O7ocTbNDXjaGn4\nodwAevfuDUQPLORDvCN8z549AZg3bx4QPZ5ZLMo0RUQSKHqm6WeN9AOSxqdC8A/l33PPPUD19km/\nbzzTrNqmKZXFt0/Vx2OPPQbAkCFDUuuUaZZGvKP6dtttV+M2+++/PwB77rlntff8QMVr1qypcd+d\ndtopteznS8/mqqQQlGmKiCRQ9EzTZ4Q+w/RZZfy92rKFkSNHAul9unwfTmUYlenhhx8Gas4+pHLE\nB/uePHkyEPWx9PzUFTVNYXHvvfcC0TQoVXXp0iW1fOGFF+YWbI6UaYqIJKBKU0QkgaJfnvtHIf0N\nnPhldW2X2L5bStURjUCd2SudH3GqQ4cOQDRmZk2y6QAvpecfc9xhhx2y3ufQQw8FoHnz5gCsW7cu\n7f0lS5aklj/55BMAWrdunVOc9aVMU0QkgaJnmlOnTgWirNHfyIFoTEV/Q8iPfzlu3Dgg6vQ+dOjQ\n1D6lepRK8suX8SGHHJJxW9/VxD/QoJuAlc/f6Gnfvj0A8+fPB6KryniXpvXr1xc3uCqUaYqIJFD0\nTNOPqO5/+jl+IGq3OPvsswF44IEHgGiABv+tc9VVVxUnWCkaP7Nk3759gajjel38tm3atClcYFJU\nflaGyy+/PG29b/OG6PHZ3XffvXiBxSjTFBFJoORDw8Xn6fHtk344qEsuuQSIZpb02anasBoeX6bd\nunUDsss0peE555xzABg7diwQ1QV+HnSASy+9FIBTTz0VgO23376YISrTFBFJouSZZl18Hz7flqk+\nmQ3foEGD0n7KxqVly5YA3HDDDUDUR3vDhg2pbfw9jlJNe6FMU0QkgbLONJV1iGycTj755LSf5USZ\npohIAqo0RUQSUKUpIpKAKk0RkQRUaYqIJKBKU0QkAYvPt5N4Z7NVwEcZN2xY2jnnWpY6iGJRGTd8\nKuNkcqo0RUQ2Nro8FxFJQJWmiEgCdVaaZra9mc0N/600sxWx15sVIiAza2dmU81skZktNLOMkxyb\n2UAzWxXGtdjMzs0xhnFm1ivDNlfGfhcLzex7M9s6l/OWQinKODzvWF9mWW5fijLe1symmNlbYRn3\ny+WcpVKiz/EesXPMNbMvM32WS1TGp5vZvPCcb5hZt4wHds5l9Q+4Dri0hvUGNMr2OFmcZydgn3B5\nK+A9oEOGfQYCd4TLrYDVQIsq2zROEMM4oFeC7U8D/p6v30Gp/hWrjMNjHgF0BeZmuX3Ryxj4AzAi\nXN4R+CLJOcrxXzHLOHbsTYHPgF3KsIybE93b2Q9YkOm49bo8N7PdwkzwUWAh0MbM1sTe72NmD4TL\nO5rZRDObZWavm9nBdR3bOfexc25uuLwOWALsnG1szrmVwIdAWzMbbmYPm9kM4CEza2xmt4VxzDOz\ngWGMjcxstJktMbMXgKSjHP8CeDzhPmWtkGUM4JybBvy7PrEVsYwdsGW43JzgQ/xDfWIuR4Uu45if\nAYudc8uz3aFYZeycW+/CGhPYgqDM65TLKEedgH7OuVlmVtdxRgE3O+deM7P2wDNAFzM7CBjgnPtN\nbTua2a5AF+CNbIMys92AdsD7sTi7O+e+MbPBwGfOua5mtjnwmpn9HTgY+AmwB0Gmuwi4JzzeCGCG\nc+5/azlfc+AY4LxsY6wgBS/j+ihiGd8JPGNmHxNc9ZwZ+4A1FMUo4z4kTCqK+Tk2szOBEQSV7ImZ\nYsul0nzPOTcri+2OATpaOJAwsK2ZNXXOzQRm1raTmW0FTAAucs5lM2fnL83sSOBbYKBzbk14zknO\nuW/CbY4FOptZn/D11sDuQHfgcefcj8ByM5vqD+qci+YYrllPYJpzbm0WMVaagpZxPRS7jE8EXido\nSugAPGdmP83y77FSFPpz3AQ4Cbgky3iK/jl2zj0BPGFmRwHXh8evVS6V5lex5R8J2kS8JrFlA7o6\n577L9sAWNE5PBMY45yZnudujzrmhNayPx2nAYOfcS1XOl8uQ8H2AR3LYv5wVrIzrqdhlPAC4Lswu\n3zazfxFUnm/W41jlqtBlfBIw0zm3OsvtS/U5xjn3DwtuUG7jnFtT23Z56XIU1uxfmNnuZtaI4MaI\n9yJwgX9hZvvUdSwLvlYeIrhBMKrKe0PMLJdLveeBwf4yxMw6mllT4BWgd9gmsjNBZpGRmW0LdAOe\nziGmipDPMq5LmZXxMqBHeJzWwG7ABznEVtYKVMbV2vvLqYzDdl0Llw8guClUa4UJ+e2neQXBf+af\nQLzB9wLg0LDBdhFh25+ZHWRm99RwnCMIftE/s6i7wnHhe52Bz3OI8V5gKTDXzBYAdxNk208QfEAW\nAWOAV/0OZjbCzGpr5zgDeNY593UOMVWSfJUxZjYe+D9gDzNbbmb9w7fKqYyvA44ws3nACwR3nb/I\nIbZKkM8y3hI4CniqylvlVMY/BxZY0PVtFNA708kr6jFKM5sC9HTOfV/qWKQwVMYNX6WXcUVVmiIi\npabHKEVEElClKSKSgCpNEZEEVGmKiCSgSlNEJAFVmiIiCajSFBFJ4P8DvJniUZggGdAAAAAASUVO\nRK5CYII=\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Confusion Matrix:\n", - "[[ 975 0 0 0 0 1 2 1 1 0]\n", - " [ 0 1127 2 0 0 0 2 2 2 0]\n", - " [ 2 0 1024 1 1 0 0 4 0 0]\n", - " [ 0 0 0 1006 0 1 0 1 2 0]\n", - " [ 0 0 1 0 979 0 1 0 0 1]\n", - " [ 2 0 0 7 0 879 3 0 1 0]\n", - " [ 5 2 0 1 1 3 946 0 0 0]\n", - " [ 0 1 4 2 0 0 0 1018 1 2]\n", - " [ 1 0 3 10 1 1 1 2 951 4]\n", - " [ 1 3 0 6 8 4 0 6 2 979]]\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAARIAAAD3CAYAAADRydumAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGrdJREFUeJzt3XuwXWWd5vHvQwKEm9xlIMEGGxqlmRYwhVFaSonQigxg\nj23BjIoO1cz00Apqj2L31DDt9MzotONtuodqGlBsES8BypTaXMQL7ZSkgRC5BSGAQGIgIBERFEjO\nM3+s98gmJifr7LXW2ZfzfKpWnb3XXvt9330uv/O+73rXb8k2ERFNbDPoBkTE6EsgiYjGEkgiorEE\nkohoLIEkIhpLIImIxhJIIqKxBJKIaCyBJCIaSyCJiMbmDroBEbPZH7x+J//08Y21jr351meutv3G\njpvUlwSSiAF67PGNLLt6Qa1jt9333r06bk7fEkgiBsps9MSgG9FYAknEABmYYPSvwE8giRggY55z\nvTmSYTYyZ20kvVHSjyStknRun2VcLGmdpNsbtmV/Sd+RdKekOySd3Wc58yT9s6QflnL+skGb5ki6\nRdLXG5TxY0m3SVoh6aYG5ewmaYmkuyStlPTqPso4pLRjcvu5pHP6bM/7yvf3dkmXSZrXZzlnlzLu\n6LctmzOBa23DbCQCiaQ5wN8CbwIOBU6TdGgfRX0OaGPWewPwAduHAouAs/pszzPAsbZfARwOvFHS\noj7bdDawss/39nq97cNtL2xQxqeBq2y/DHhFP+2y/aPSjsOBVwJPA1dOtxxJ84H3AgttHwbMAU7t\no5zDgD8GjqL6TCdKOmi65WzKwEZcaxtmIxFIqH54q2zfZ/tZ4EvAydMtxPb1wONNG2N7re3l5fGT\nVH8o8/sox7Z/UZ5uW7Zp/8ZIWgC8Gbhwuu9tm6RdgWOAiwBsP2v7Zw2LXQzca/uBPt8/F9hB0lxg\nR+AnfZTxcmCZ7adtbwC+B/xhn+15gfRIZs584KGe56vp4w+3C5IOAI4AlvX5/jmSVgDrgGtt91PO\np4APAk2n/w1cI+lmSWf2WcaBwKPAZ8tQ60JJOzVs16nAZf280fYa4OPAg8Ba4Anb1/RR1O3AayXt\nKWlH4ARg/37a9IL2ARvtWtswG5VAMpQk7QxcDpxj++f9lGF7Y+m+LwCOKl3o6bThRGCd7Zv7qX8T\nv2/7SKoh5FmSjumjjLnAkcD5to8AngL6mtMCkLQdcBLw1T7fvztV7/VAYD9gJ0lvn245tlcCHwOu\nAa4CVgCtzJJO1NyG2agEkjW8MPovKPsGRtK2VEHkUttXNC2vdP+/w/TncI4GTpL0Y6oh37GSvtBn\nG9aUr+uo5iOO6qOY1cDqnp7VEqrA0q83ActtP9Ln+98A3G/7UdvPAVcAr+mnINsX2X6l7WOA9cDd\nfbbp+TJrzo9kjqQdNwIHSzqw/Ic6FVg6qMZIEtUcwErbn2hQzt6SdiuPdwCOA+6aThm2P2x7ge0D\nqL4v37Y97f+4knaStMvkY+B4qu78tNh+GHhI0iFl12LgzumW0+M0+hzWFA8CiyTtWH5ui+lzUlrS\ni8vXl1DNj3yxQbsAsOG5mtswG4l1JLY3SPpT4GqqWfeLbd8x3XIkXQa8DthL0mrgPNsX9dGko4F3\nALeV+Q2AP7f9zWmWsy9wSTkrtQ3wFdt9n75taB/gyupvjbnAF21f1WdZ7wEuLUH/PuDd/RRSAtpx\nwL/vsx3YXiZpCbCc6mzbLcAFfRZ3uaQ9geeAs1qYRAbERtS8mAFT7msTMTiH/d52vvwb9S6hedlL\n1t7c8LR8Z0aiRxIxzsahR5JAEjFA1YK0BJKIaGjCCSQR0UB6JBHRmBHPec6gm9HYqKwj+bUGS7db\nLSPlzEw5w9SWNsuZNNkjqbMNs5ELJEAbP8i2fhlSTvflDFNb2iynEBu9Ta1tmA136yLGXJUhbZta\nWx2by7kjaQ9J10q6p3zdveyXpM+UHD+3Sjqy5z2nl+PvkXT61uodqjmSXfaY673nbz/lMXvttx0v\n/Zc7TbmK7rHbpy5jHjvyIu3ReCVeyum+nGFqS91yfsVTPOtnao9FWh62fA74G+DzPfvOBa6z/VFV\nScHOBT5EdR3TwWV7FXA+8CpJewDnAQupYt3NkpbaXr+lSocqkOw9f3v+6orfbVzOZw/5rRZaE9Gf\nZb6u9rG2Wh222L6+pLbodTLVpSEAlwDfpQokJwOfd7W8/YaS2W7fcuy1th8HkHQt1cWkW7zmaagC\nScRsNNH9ROo+tteWxw9TXVcFW87zM+38PwkkEQNkxLOu/We41ya5dC+wPa0LEG1bUusX2CWQRAzQ\n5GRrTY/1edHeI5L2tb22DF3Wlf1byvOzhueHQpP7vztVBZ2etVELmd8jxt1Gq9bWwFJg8szL6cDX\neva/s5y9WUSVhnItVbqO4yXtXs7wHF/2bVFnPZKezO/HUY2xbiwzv02S3ESMFSM2tvj/fHM5d4CP\nAl+RdAbwAPC2cvg3qXLPrqLK0v9uANuPS/pvVAnFAD4yOfG6JV0ObX6d+R1A0mTm9wSSiB4T7Z61\nOW0LLy3ezLEGztpCORcDF9ett8tAsrmZ31d1WF/EyKmWyI/+utCBT7aWaxfOhGqxWcRsMi4X7XUZ\nSGplfi+nry4AtrpiNWLc2Az9dTR1dPkJhirze8RwEhM1t2HWWY+krczvEeOsutPe6PdIOp0jKbdn\nmO4tGiJmlUy2RkQjRsnZGhHNpUcSEY3k9G8HHrt9+1ZyiVz9kxVbP6iGP9jv8FbKidgS0+7K1kEZ\nqkASMRsNe2LnOhJIIgbIVnokEdFc1pFERCNVYqMMbSKikXaTPw9Kl4mNLgZOBNbZPqyreiJGmWEs\nTv92GQo/R5XCPiK2YHJla51tmHV50d7m7q8REZuYRvLnoZU5kogBqvKRDHdvo46BB5LeDGnz2HHA\nrYmYecM+bKlj4IGkN0NaG/dmjRgl1RxJhjYR0dA4LJHvLBSW+2v8ADhE0upyT42I6GHEhok5tbZh\n1uVZmy3dXyMiemRla0Q0krM2EdGKTLZGRCPJ2TrE2sps9l/uW95KOR956ZGtlBPjKXMkEdFIlWox\ngSQimrCG/tRuHaM/yxMxwiYTG7V1y05J75N0h6TbJV0maV65be4ySaskfbncQhdJ25fnq8rrB/T7\nORJIIgasrTQCkuYD7wUWlhxAc6juuf0x4JO2DwLWA5OLQ88A1pf9nyzH9SWBJGKAJudIWsxHMhfY\nQdJcYEdgLXAssKS8fglwSnl8cnlOeX2xpL4mbLpcIr+/pO9IurN0tc7uqq6IUdZWILG9Bvg48CBV\nAHkCuBn4me0N5bDVwPzyeD7wUHnvhnL8nv18hi57JBuAD9g+FFgEnCXp0A7rixg508yQtpekm3q2\nM3vLkrQ7VS/jQGA/YCdmKEthl9farKWKith+UtJKqgh4Z1d1Rowcw4b6K1sfs71witffANxv+1EA\nSVcARwO7SZpbeh0LgDXl+DXA/sDqMhTaFfhpH59iZuZIymzwEcCymagvYlS0PEfyILBI0o5lrmMx\n1T/u7wBvLcecDnytPF5anlNe/7btvnICdb6ORNLOwOXAObZ/vpnXkyEtZrW2FqTZXiZpCbCcamrh\nFqqkYd8AviTpr8q+i8pbLgL+QdIq4HGqMzx96TSQSNqWKohcavuKzR2TDGkxm7V9rY3t84DzNtl9\nH3DUZo79FfBHbdTb5X1tRBXxVtr+RFf1RIw6j8ES+S7nSI4G3gEcK2lF2U7osL6IkdTmytZB6fKs\nzfdhyD99xIDZuWgvIhoTGydGf4F5AknEgI3DHEkCScQAJR/JLNBWZrMPrLqjlXL+90G/20o5MURc\nzZOMugSSiAEb9jMydSSQRAyQyRxJRDSWLPIR0YKJiQSSiGjAztBmSpLmAdcD25d6lpQLiiKiR4Y2\nU3sGONb2L8pVwN+X9I+2b+iwzoiRk9O/UygJUn5Rnm5btjH4lkW0axyGNp0u8pc0R9IKYB1wre3f\nyJAm6czJHJTP8UyXzYkYOkbY9bZh1mkgsb3R9uFUeSKPknTYZo65wPZC2wu3ZfsumxMxlFxzG2Yz\nctmh7Z9R5Y2ckYzWESPD4AnV2oZZl/e12VvSbuXxDsBxwF1d1RcxqsZhaNPlWZt9gUskzaEKWF+x\n/fUO64sYSTlrMwXbt1LdgiIitiDX2kREcwYSSCKiqQxtIqK5BJKoo63MZmfcfX/jMi76nQNbaEm0\nZ/hP7daRQBIxSLn6NyJakaFNRDSXHklENDUGPZLOr7UpVwDfIimrWiM2Zwyu2puJHsnZwErgRTNQ\nV8RoKRftjbqu85EsAN4MXNhlPREjrcUeiaTdJC2RdJeklZJeLWkPSddKuqd83b0cK0mfkbRK0q2S\n+r4jXO1AIqmfZCGfAj4ITPTx3ojZwaq31fNp4CrbLwNeQTUaOBe4zvbBwHXlOcCbgIPLdiZwfr8f\nYauBRNJRkm4D7inPXyHp/9R434nAOts3b+W4ZEiLWU2ut221HGlX4BjgIgDbz5ZcQCcDl5TDLgFO\nKY9PBj7vyg3AbpL27ecz1OmRfAY4EfhpadwPgdfXeN/RwEmSfgx8CThW0hc2PSgZ0mJWqzusqTe0\nORB4FPhsOcFxoaSdgH1sry3HPAzsUx7PBx7qef/qsm/a6gSSbWw/sMm+jVt7k+0P215g+wDgVODb\ntt/eRxsjxljNYU01tNlrsvdetjM3KWwucCRwvu0jgKd4fhgD/Dope+vngOqctXlI0lGAS5Ki9wB3\nt92QiFmr/p/1Y7YXTvH6amB1T5L1JVSB5BFJ+9peW4Yu68rra4D9e96/oOybtjo9kj8B3g+8BHgE\nWFT21Wb7u7ZPnH7zImaBiZrbVth+mOof/yFl12LgTmApcHrZdzrwtfJ4KfDOcvZmEfBEzxBoWrba\nI7G9jmpoEhFtaz+x0XuASyVtB9wHvJuS6lTSGcADwNvKsd8ETgBWAU+XY/uy1UAi6e/ZTOfL9qbj\ns4joQ50zMnXZXgFsbvizeDPHGjirjXrrzJF8q+fxPOAtvHCmNyKaGPLl73XUGdp8ufe5pH8Avt9Z\ni8aR2um6tpGU6OU3t3NVxMpXbmilnBgP/fxWHcjz56EjoqE2hzaDUmeOZD3Pd762AR5nk3PTEdHA\nuGdIkySq9fqT55YnygRNRLTBjMWVaFOuIylB45vlZuAbE0Qi2tfWtTaDVGdB2gpJuWNeRFfGObGR\npLm2N1DddvNGSfdSrd0XVWdlq7kLygV7T1Jdm7NhK8t7I2anIQ8SdUw1R/LPVBcAndSwjtfbfqxh\nGRFjaRSGLXVMFUgEYPveGWpLxOw05mdt9pb0/i29aPsTNco3cI0kA39n+4LpNjBi7I15j2QOsDPN\nbrrx+7bXSHoxcK2ku2xf33tAyalwJsA8dmxQVcRo0hic/p0qkKy1/ZEmhdteU76uk3QlcBRw/SbH\nXABcAPAi7TEGsTliGsZkjmSq07+NBm6SdpK0y+Rj4Hjg9iZlRoylcT79y2YuO56mfYArq8WxzAW+\naPuqhmVGjJ8hDxJ1bDGQ2H68ScG276NaXh8RUxj3oU1ERC25iXjEoI1BjySBJGKQPP6nfwejjWxi\nw3aR8hC1p63MZifd+dNWyll66J6tlDPShufXo2/DF0giZhExHpOtCSQRg5ZAEhGNjMnK1gSSiEFL\nIImIpsbhrE2nC9Ik7SZpiaS7JK2U9Oou64sYSWN+rU0bPg1cZfut5V6kyRMQ0WsEgkQdnQUSSbsC\nxwDvArD9LPBsV/VFjKpxmGztcmhzIPAo8FlJt0i6sKQTiIheYzC06TKQzKVKHn2+7SOoMtD/xh36\nJJ0p6SZJNz3HMx02J2I4zZb72vRrNbDa9rLyfAlVYHkB2xfYXmh74bZs32FzIoZUeiRbZvth4CFJ\nh5Rdi4E7u6ovYhTV7Y1Mp0ciaU6ZTvh6eX6gpGWSVkn6cjnxgaTty/NV5fUD+v0cXecjeQ9wqaRb\ngcOB/9FxfRGjp/0eydnAyp7nHwM+afsgYD1wRtl/BrC+7P9kOa4vnQYS2yvKsOX3bJ9ie32X9UWM\nojZ7JJIWAG8GLizPBRxLNbUAcAlwSnl8cnlOeX1xOX7akiEtYtDa7ZF8CvggMLledk/gZ+X2u1DN\nXc4vj+cDDwGU158ox09bAknEoNUPJHtNnuEs25m9xUg6EVhn++YZbD2Qa20iBmt6E6mP2V44xetH\nAydJOgGYB7yIanX5bpLmll7HAmBNOX4NsD+wWtJcYFegr4xVwxdIhiibWGxZW5nN/uK+Fa2U899f\nengr5QxES7/ytj8MfBhA0uuAP7P9byV9FXgr8CXgdOBr5S1Ly/MflNe/bff3B5ihTcSAaaLe1sCH\ngPdLWkU1B3JR2X8RsGfZ/342s2C0ruHrkUTMMl2sWrX9XeC75fF9VLfL3fSYXwF/1EZ9CSQRgzQC\nq1brSCCJGLQEkohoYlyyyHc22SrpEEkrerafSzqnq/oiRtYYXLTXWY/E9o+orq9B0hyqc9ZXdlVf\nxKjSGCx5mKmhzWLgXtsPzFB9EaMht+ycllOBy2aorojRMvodku4XpJXcBycBX93C68mQFrNaMqTV\n8yZgue1HNvdiMqTFrJfJ1lpOI8OaiM0bgd5GHV3fIGsn4Djgii7riRhp6ZFMzfZT9JkoJWI2GJcF\naVnZGjFgmhj9SJJAEjFIIzBsqSOBJGLAsiAt6tlmTjvlTGxsp5wh0lZms3+9cl0r5Vz+8he3Us60\npEcSEU1lsjUimjFjkac4gSRiwDJHEhGNZB1JRDRnj8XQpusl8u+TdIek2yVdJmlel/VFjKJc/TsF\nSfOB9wILbR8GzKHKSxIRvXKtTa3yd5D0HLAj8JOO64sYOcPe26ijsx6J7TXAx4EHgbXAE7av6aq+\niJFkYML1tiHW5dBmd+Bk4EBgP2AnSW/fzHHJkBaz2gzcsrNzXU62vgG43/ajtp+jyknymk0PSoa0\nmPUmz9xsbRtiXc6RPAgskrQj8EuqTPI3dVhfxEjKHMkUbC8DlgDLgdtKXRd0VV/ESKp7xmbIg03X\nGdLOA87rso6IUVatbB3yKFHDTGSRj4ipTNTctkLS/pK+I+nOshD07LJ/D0nXSrqnfN297Jekz0ha\nJelWSUf2+xESSCIGTHatrYYNwAdsHwosAs6SdChwLnCd7YOB68pzqG4Vc3DZzgTO7/czJJBEDJJr\nriGpsY7E9lrby8vjJ4GVwHyqZRiXlMMuAU4pj08GPu/KDcBukvbt52Pkor2Z4JYWAUjNyxi28Xgb\nn4n2Mpudcff9jcu49y3TWw/VxVkbSQcARwDLgH1sry0vPQzsUx7PBx7qedvqsm8t05RAEjFo9YP7\nXpJ6l1BcYPs3zoRK2hm4HDjH9s/VE6xtW2o/dCWQRAySp7Vq9THbC6c6QNK2VEHkUtuTN6Z7RNK+\ntteWoctkgts1wP49b19Q9k1b5kgiBq2lla2quh4XASttf6LnpaXA6eXx6cDXeva/s5y9WUR1Pdy0\nhzWQHknE4LU30DgaeAdwm6QVZd+fAx8FviLpDOAB4G3ltW8CJwCrgKeBd/dbcQJJxIC1tSDN9vep\n1rhtzuLNHG/grDbq7jpD2tklO9odks7psq6IkWRgo+ttQ6zLNAKHAX8MHAW8AjhR0kFd1RcxikS9\nxWjDvoy+yx7Jy4Fltp+2vQH4HvCHHdYXMZrGII1Al4HkduC1kvYsqQRO4IWnmiICxiKQdDbZanul\npI8B1wBPASuA37h5raQzqdb5M48du2pOxHAytS7IG3adTrbavsj2K20fA6wH7t7MMcmQFrPaOMyR\ndHr6V9KLba+T9BKq+ZFFXdYXMZKGPEjU0fU6kssl7Qk8B5xl+2cd1xcxWmyYGP2xTdcZ0l7bZfkR\nY2H040hWtkYM2rDPf9SRQBIxaAkkEdHI5J32RtxQBZInWf/Yt7zkga0cthfwWMOq2iijfjlb/z2Z\n2fYMUzlD9r351sGtlPNb9ZoEMPyLzeoYqkBie++tHSPppq0ld5mJMlLOzJQzTG1ps5wXSCCJiEYM\nbBz90zYJJBED5faSgw/QKAaSNm772datQ1NO9+UMU1vaLOd5YzC0kcfgQ4wbSRup7pc8l+reJKfb\nfrrPsl4H/JntEyWdBBxq+6NbOHY34N/Y/r/TrOO/Ar+w/fF+2jib7brdPn7Nvzit1rFXPfTpm1uf\nn2lJkj8Pp1/aPtz2YcCzwH/ofbEk6532z8720i0FkWI34D9Ot9xoaAzSCCSQDL9/Ag6SdICkH0n6\nPFWul/0lHS/pB5KWS/pquZ8Jkt4o6S5Jy+lJJiXpXZL+pjzeR9KVkn5YttdQJQn+bUkrJP11Oe4/\nSbqx3Bv2L3vK+gtJd0v6PnDIjH03xtEYBJJRnCOZNSTNpbo/61Vl18FUw5wbJO0F/GfgDbafkvQh\n4P2S/hfw98CxVNnBv7yF4j8DfM/2WyTNAXamuifsYbYPL/UfX+o8iiqp8FJJx1DllzkVOJzqd2g5\ncHO7n36WsGHjb6TpGTkJJMNph57bCfwT1b1K9gMeKPdohSolw6HA/yt3UtsO+AHwMuB+2/cASPoC\nJXHUJo4F3glgeyPwxORd6nscX7ZbyvOdqQLLLsCVk/M2kpY2+rSz3ZD3NupIIBlOv5zsFUwqweKp\n3l3AtbZP2+S4F7yvIQH/0/bfbVJH7gjQpjEIJJkjGV03AEdPZuaXtJOk3wHuAg6Q9NvluC2dErgO\n+JPy3jmSdgWepOptTLoa+Hc9cy/zJb0YuB44RdIOknYB/lXLn20WcXWtTZ1tiCWQjCjbjwLvAi6T\ndCtlWGP7V1RDmW+UydZ1WyjibOD1km6jmt841PZPqYZKt0v6a9vXAF8EflCOWwLsYns51dzLD4F/\nBG7s7IOOO4M9UWsbZllHEjFAu87d269+0Sm1jr16/YVDu44kcyQRgzYG/8wTSCIGKad/I6INTvLn\niGhm+Fet1pFAEjFIY5JqMad/IwbNE/W2Gsp1Vj+StErSuR23/NfSI4kYIANuqUdSrpn6W+A4YDVw\no6Sltu9spYIppEcSMUh2mz2So4BVtu+z/SzwJeDkTttfpEcSMWBu7/TvfOChnuergVe1VfhUEkgi\nBuhJ1l/9LS/Zq+bh8yTd1PP8Atvtp37sQwJJxADZfmOLxa0B9u95vqDs61zmSCLGx43AwZIOlLQd\nVfKpGckVkx5JxJiwvUHSn1Klf5gDXGz7jpmoO1f/RkRjGdpERGMJJBHRWAJJRDSWQBIRjSWQRERj\nCSQR0VgCSUQ0lkASEY39f3WS57bNTBYlAAAAAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "print_test_accuracy(show_example_errors=True,\n", " show_confusion_matrix=True)" @@ -1904,7 +1579,7 @@ }, { "cell_type": "code", - "execution_count": 54, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -1963,7 +1638,7 @@ }, { "cell_type": "code", - "execution_count": 60, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -2021,7 +1696,7 @@ }, { "cell_type": "code", - "execution_count": 61, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -2042,20 +1717,9 @@ }, { "cell_type": "code", - "execution_count": 62, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADWtJREFUeJzt3X+oXPWZx/HPZ900gqmakKuJNu7tJqIbgpsuQ1h1WV1/hEQCsX9UEqRkoTQFK26h6EpAq8hCWG26glJNNDRCa1tM3QQJbiWsaGAtGY1Wa3a3/rim2Vxyb4zQFISQ5Nk/7km5jXfOjPPrzM3zfoHMzHnOmfN4yOeemfmema8jQgDy+bOqGwBQDcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiCpP+/nzubOnRvDw8P93CWQysjIiI4cOeJW1u0o/LZXSHpU0jmSnoqIjWXrDw8Pq16vd7JLACVqtVrL67b9st/2OZIel7RS0mJJa20vbvf5APRXJ+/5l0l6LyI+iIjjkn4qaXV32gLQa52E/1JJv5v0+GCx7E/YXm+7brs+Pj7ewe4AdFMn4Z/qQ4XPfD84IjZHRC0iakNDQx3sDkA3dRL+g5IWTHr8JUmHOmsHQL90Ev69ki63/WXbX5C0RtLO7rQFoNfaHuqLiBO275T0H5oY6tsaEb/pWmcAeqqjcf6I2CVpV5d6AdBHXN4LJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUh3N0mt7RNIxSSclnYiIWjeaAtB7HYW/8A8RcaQLzwOgj3jZDyTVafhD0i9tv257fTcaAtAfnb7svzYiDtm+SNJLtv87Il6ZvELxR2G9JF122WUd7g5At3R05o+IQ8XtmKTnJS2bYp3NEVGLiNrQ0FAnuwPQRW2H3/Z5tr94+r6k5ZLe6VZjAHqrk5f9F0t63vbp5/lJRLzYla4A9Fzb4Y+IDyT9dRd7AdBHDPUBSRF+ICnCDyRF+IGkCD+QFOEHkurGt/pSeO655xrWtmzZUrrtJZdcUlo/99xzS+u33357aX3evHkNa4sWLSrdFnlx5geSIvxAUoQfSIrwA0kRfiApwg8kRfiBpBjnb9Hdd9/dsDYyMtLTfT/xxBOl9fPPP79hbfHixd1uZ9pYsGBBw9o999xTum2tdvb/Cj1nfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IinH+Fj311FMNa2+99Vbpts3G2t99993S+r59+0rrL7/8csPaa6+9VrptsynUDhw4UFrvxIwZM0rrc+fOLa2Pjo6W1sv+38uuAZAY5wdwFiP8QFKEH0iK8ANJEX4gKcIPJEX4gaSajvPb3ipplaSxiFhSLJsj6WeShiWNSLotIj7pXZvVu/HGG9uqtWLFihUdbf/JJ40PfbNrBJqNZ+/du7etnloxc+bM0voVV1xRWr/yyitL60ePHm1YW7hwYem2GbRy5v+RpDP/dd4raXdEXC5pd/EYwDTSNPwR8YqkM/+Erpa0rbi/TdKtXe4LQI+1+57/4ogYlaTi9qLutQSgH3r+gZ/t9bbrtuvj4+O93h2AFrUb/sO250tScTvWaMWI2BwRtYioDQ0Ntbk7AN3Wbvh3SlpX3F8naUd32gHQL03Db/tZSf8l6QrbB21/Q9JGSTfb/q2km4vHAKaRpuP8EbG2QamzwW10zezZsxvWbrjhho6eu9NrGDqxffv20nrZ9Q2SdNVVVzWsrVmzpq2eziZc4QckRfiBpAg/kBThB5Ii/EBShB9Iip/uRmXGxhpeGCpJuuOOO0rrEVFav//++xvW5syZU7ptBpz5gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApxvlRmccff7y03uw6gAsvvLC03uynv7PjzA8kRfiBpAg/kBThB5Ii/EBShB9IivADSTHOj57as2dPw9rGjZ1N97BjR/lcMUuWLOno+c92nPmBpAg/kBThB5Ii/EBShB9IivADSRF+IKmm4/y2t0paJWksIpYUyx6Q9E1J48VqGyJiV6+axPS1a1fjfxbHjx8v3famm24qrV999dVt9YQJrZz5fyRpxRTLfxARS4v/CD4wzTQNf0S8IuloH3oB0EedvOe/0/avbW+1PbtrHQHoi3bD/0NJCyUtlTQq6fuNVrS93nbddn18fLzRagD6rK3wR8ThiDgZEackbZG0rGTdzRFRi4ja0NBQu30C6LK2wm97/qSHX5X0TnfaAdAvrQz1PSvpeklzbR+U9D1J19teKikkjUj6Vg97BNADTcMfEWunWPx0D3rBNPTpp5+W1l988cWGtZkzZ5Zu++CDD5bWZ8yYUVpHOa7wA5Ii/EBShB9IivADSRF+ICnCDyTFT3ejIw8//HBpfd++fQ1rK1euLN32mmuuaasntIYzP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kxTg/Sr3wwgul9Yceeqi0fsEFFzSs3XfffW31hO7gzA8kRfiBpAg/kBThB5Ii/EBShB9IivADSTHOn9zHH39cWr/rrrtK6ydOnCit33LLLQ1rTLFdLc78QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5BU03F+2wskPSNpnqRTkjZHxKO250j6maRhSSOSbouIT3rXKtpx8uTJ0vqKFStK6x9++GFpfdGiRaX1Zt/3R3VaOfOfkPTdiPgrSX8r6du2F0u6V9LuiLhc0u7iMYBpomn4I2I0It4o7h+TtF/SpZJWS9pWrLZN0q29ahJA932u9/y2hyV9RdKvJF0cEaPSxB8ISRd1uzkAvdNy+G3PkrRd0nci4vefY7v1tuu26+Pj4+30CKAHWgq/7RmaCP6PI+IXxeLDtucX9fmSxqbaNiI2R0QtImpDQ0Pd6BlAFzQNv21LelrS/ojYNKm0U9K64v46STu63x6AXmnlK73XSvq6pLdtv1ks2yBpo6Sf2/6GpAOSvtabFtGJ999/v7Rer9c7ev5NmzaV1hcuXNjR86N3moY/IvZIcoPyjd1tB0C/cIUfkBThB5Ii/EBShB9IivADSRF+ICl+uvss8NFHHzWsLV++vKPnfuSRR0rrq1at6uj5UR3O/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOP8Z4Enn3yyYa3sGoBWXHfddaX1id96wXTEmR9IivADSRF+ICnCDyRF+IGkCD+QFOEHkmKcfxp49dVXS+uPPfZYnzrB2YQzP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8k1XSc3/YCSc9ImifplKTNEfGo7QckfVPSeLHqhojY1atGM9uzZ09p/dixY20/96JFi0rrs2bNavu5MdhaucjnhKTvRsQbtr8o6XXbLxW1H0RE+awOAAZS0/BHxKik0eL+Mdv7JV3a68YA9Nbnes9ve1jSVyT9qlh0p+1f295qe3aDbdbbrtuuj4+PT7UKgAq0HH7bsyRtl/SdiPi9pB9KWihpqSZeGXx/qu0iYnNE1CKiNjQ01IWWAXRDS+G3PUMTwf9xRPxCkiLicEScjIhTkrZIWta7NgF0W9Pwe+LnWZ+WtD8iNk1aPn/Sal+V9E732wPQK6182n+tpK9Letv2m8WyDZLW2l4qKSSNSPpWTzpER5YuXVpa3717d2l9zpw53WwHA6SVT/v3SJrqx9kZ0wemMa7wA5Ii/EBShB9IivADSRF+ICnCDyTliOjbzmq1WtTr9b7tD8imVqupXq+3NG86Z34gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSKqv4/y2xyV9NGnRXElH+tbA5zOovQ1qXxK9taubvf1FRLT0e3l9Df9ndm7XI6JWWQMlBrW3Qe1Lord2VdUbL/uBpAg/kFTV4d9c8f7LDGpvg9qXRG/tqqS3St/zA6hO1Wd+ABWpJPy2V9j+H9vv2b63ih4asT1i+23bb9qu9PvHxTRoY7bfmbRsju2XbP+2uJ1ymrSKenvA9v8Vx+5N27dU1NsC2/9pe7/t39j+p2J5pceupK9KjlvfX/bbPkfS/0q6WdJBSXslrY2Id/vaSAO2RyTVIqLyMWHbfy/pD5KeiYglxbJ/lXQ0IjYWfzhnR8Q/D0hvD0j6Q9UzNxcTysyfPLO0pFsl/aMqPHYlfd2mCo5bFWf+ZZLei4gPIuK4pJ9KWl1BHwMvIl6RdPSMxaslbSvub9PEP56+a9DbQIiI0Yh4o7h/TNLpmaUrPXYlfVWiivBfKul3kx4f1GBN+R2Sfmn7ddvrq25mChcX06afnj79oor7OVPTmZv76YyZpQfm2LUz43W3VRH+qX5iaJCGHK6NiL+RtFLSt4uXt2hNSzM398sUM0sPhHZnvO62KsJ/UNKCSY+/JOlQBX1MKSIOFbdjkp7X4M0+fPj0JKnF7VjF/fzRIM3cPNXM0hqAYzdIM15XEf69ki63/WXbX5C0RtLOCvr4DNvnFR/EyPZ5kpZr8GYf3ilpXXF/naQdFfbyJwZl5uZGM0ur4mM3aDNeV3KRTzGU8W+SzpG0NSL+pe9NTMH2X2ribC9NTGL6kyp7s/2spOs18a2vw5K+J+nfJf1c0mWSDkj6WkT0/YO3Br1dr4mXrn+cufn0e+w+9/Z3kl6V9LakU8XiDZp4f13ZsSvpa60qOG5c4QckxRV+QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeS+n89yrzr7tkdMgAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "image1 = data.test.images[0]\n", "plot_image(image1)" @@ -2070,20 +1734,9 @@ }, { "cell_type": "code", - "execution_count": 63, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADjVJREFUeJzt3XGInPWdx/HPV00RbJBoNtvFRrcWKbcEm8qwHOTQPYrFntUYJKERQsSSrZiAxYgnEdLEeLKaS3uCZ3F7Ls1CYxtsPKMYr6JHbPWImZiQpJfTiuyluSybXU2oxT+q5nt/7LNlm+z8ZjPzzDyz+b5fEGbm+c4zz9fHfPLMzO955mfuLgDxXFB0AwCKQfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwR1UTM3NnfuXO/s7GzmJoFQhoaGNDY2ZtN5bl3hN7ObJD0h6UJJ/+bufannd3Z2qlwu17NJAAmlUmnaz635bb+ZXSjpXyV9W1KXpOVm1lXr6wForno+83dLet/dP3D3P0v6haTF+bQFoNHqCf8Vkv4w6fGxbNlfMbNeMyubWXl0dLSOzQHIUz3hn+pLhbOuD3b3fncvuXupra2tjs0ByFM94T8maf6kx1+WdLy+dgA0Sz3h3yvpGjP7ipl9QdJ3Je3Mpy0AjVbzUJ+7f2ZmayT9h8aH+gbc/Xe5dQagoeoa53f3lyW9nFMvAJqI03uBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8Iqq5Zes1sSNLHkj6X9Jm7l/JoCvk5efJksr5///5k/ZVXXknWN2/enKybWcXa0qVLk+teddVVyfratWuT9fb29mQ9urrCn/l7dx/L4XUANBFv+4Gg6g2/S/q1me0zs948GgLQHPW+7V/k7sfNbJ6kV83sf9z9jclPyP5R6JWkK6+8ss7NAchLXUd+dz+e3Z6Q9Lyk7ime0+/uJXcvtbW11bM5ADmqOfxmdomZzZ64L+lbkg7n1RiAxqrnbX+7pOezoZyLJG1z9/S4EICWUXP43f0DSV/PsRdU8OmnnybrW7ZsqVh78sknk+sODw/X1NOE1Dh+tfpzzz1X17bHxtIjzAMDA3W9/vmOoT4gKMIPBEX4gaAIPxAU4QeCIvxAUHlc1YcGe/rpp5P1hx56qEmdnK2npydZ3717d8O2vXXr1mSdob40jvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/C3g8OH0b6Bs2rSpSZ2c7bHHHkvW77333mR9/fr1FWuPP/54TT0hHxz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvmboNo4/rp165L10dHRZD3189jVprneuXNnst7V1ZWsX3BB+vjx8MMPV6wtWbIkue6tt96arFfbL9dee23F2sGDB5PrRsCRHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCqjrOb2YDkr4j6YS7L8iWXSbpl5I6JQ1JWubuJxvX5sy2f//+ZP2ll15K1t09WZ81a1bF2urVq5PrLliwIFmvV6q37u7u5Lp33nlnsp6amlySDh06VLHW29ubXLe/vz9ZPx9M58j/M0k3nbHsQUmvufs1kl7LHgOYQaqG393fkPTRGYsXS5qYLmWrpNty7gtAg9X6mb/d3YclKbudl19LAJqh4V/4mVmvmZXNrFztXGwAzVNr+EfMrEOSstsTlZ7o7v3uXnL3UltbW42bA5C3WsO/U9LK7P5KSS/k0w6AZqkafjN7VtJ/SfqamR0zs+9J6pN0o5n9XtKN2WMAM0jVcX53X16h9M2cezlv7dq1K1lPXY8/HT09PRVra9eureu1i9TXlz6mVNuvqXH+vXv31tTT+YQz/ICgCD8QFOEHgiL8QFCEHwiK8ANB8dPdOfjwww+T9T179jR0+ytWrGjo67eqav/dDzzwQJM6mZk48gNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIzz52Dfvn3J+tDQUF2vf/311yfrN998c12vH9GpU6eS9eHh4WS9o6Mjz3YKwZEfCIrwA0ERfiAowg8ERfiBoAg/EBThB4JinD8H5XK5oa+/cePGZH3OnDkN3f756OjRo8n64cOHk3XG+QHMWIQfCIrwA0ERfiAowg8ERfiBoAg/EFTVcX4zG5D0HUkn3H1BtmyDpFWSRrOnrXP3lxvVZKv75JNPknV3r+v1b7jhhrrWj6re/X6+m86R/2eSbppi+Y/dfWH2J2zwgZmqavjd/Q1JHzWhFwBNVM9n/jVmdtDMBsyM80uBGabW8P9E0lclLZQ0LGlLpSeaWa+Zlc2sPDo6WulpAJqspvC7+4i7f+7upyX9VFJ34rn97l5y91JbW1utfQLIWU3hN7PJlzQtkZS+BApAy5nOUN+zknokzTWzY5J+KKnHzBZKcklDkr7fwB4BNEDV8Lv78ikWP9OAXmasatfzm1mTOsFkqf3O/xPO8APCIvxAUIQfCIrwA0ERfiAowg8ExU93I6TZs2cn65dffnmTOikOR34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIpxfsxYg4ODNa+7YcOGZP26666r+bVnCo78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4/w56OvrS9YPHDiQrFebxuyuu+5K1gcGBpL181W1/TZv3ryKtbvvvjvvdmYcjvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFTVcX4zmy9pUNKXJJ2W1O/uT5jZZZJ+KalT0pCkZe5+snGttq6FCxcm65s3b07WV65cmaxv3749WV+zZk3F2ky+Ln3VqlXJ+sjISLK+bNmyirWLL764pp7OJ9M58n8maa27/42kv5W02sy6JD0o6TV3v0bSa9ljADNE1fC7+7C7v5Pd/1jSEUlXSFosaWv2tK2SbmtUkwDyd06f+c2sU9I3JO2R1O7uw9L4PxCSKp9LCaDlTDv8ZvZFSb+S9AN3/+M5rNdrZmUzK1c7FxtA80wr/GY2S+PB/7m778gWj5hZR1bvkHRiqnXdvd/dS+5eamtry6NnADmoGn4zM0nPSDri7j+aVNopaeJr6pWSXsi/PQCNMp1LehdJWiHpkJlNXJu6TlKfpO1m9j1JRyUtbUyLM9+iRYuS9TvuuCNZ37ZtW7K+e/fuirVWHup7/fXXk/UdO3Yk6+3t7cn6+vXrz7mnSKqG391/K8kqlL+ZbzsAmoUz/ICgCD8QFOEHgiL8QFCEHwiK8ANB8dPdTXD11Vcn64888kiy/uabbybrGzdurFirdkr1o48+mqxX89577yXrb7/9dsXafffdl1z31KlTyfr999+frHd1dSXr0XHkB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgGOdvAZ2dncn6W2+9laynppt+6qmnkuvu2rWr5teWql8zPzY2lqyn3HLLLcl6b29vza8NjvxAWIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/DNAR0dHsj44OFix9u677ybX3bRpU7J+zz33JOvVrqlPuf3225P1anMOXHQRf33rwZEfCIrwA0ERfiAowg8ERfiBoAg/EBThB4KqOlBqZvMlDUr6kqTTkvrd/Qkz2yBplaSJH4Zf5+4vN6pRVHbppZdWrHV3dyfXffHFF/NuBzPEdM6S+EzSWnd/x8xmS9pnZq9mtR+7+z83rj0AjVI1/O4+LGk4u/+xmR2RdEWjGwPQWOf0md/MOiV9Q9KebNEaMztoZgNmNqfCOr1mVjazcrWpowA0z7TDb2ZflPQrST9w9z9K+omkr0paqPF3BlumWs/d+9295O6ltra2HFoGkIdphd/MZmk8+D939x2S5O4j7v65u5+W9FNJ6W+WALSUquE3M5P0jKQj7v6jScsnX2q2RNLh/NsD0CjT+bZ/kaQVkg6Z2YFs2TpJy81soSSXNCTp+w3pEEBDTOfb/t9KsilKjOkDMxhn+AFBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Iyd2/exsxGJf3vpEVzJY01rYFz06q9tWpfEr3VKs/ernL3af1eXlPDf9bGzcruXiqsgYRW7a1V+5LorVZF9cbbfiAowg8EVXT4+wvefkqr9taqfUn0VqtCeiv0Mz+A4hR95AdQkELCb2Y3mdm7Zva+mT1YRA+VmNmQmR0yswNmVi64lwEzO2Fmhyctu8zMXjWz32e3U06TVlBvG8zs/7J9d8DM/qGg3uab2X+a2REz+52Z3ZstL3TfJfoqZL81/W2/mV0o6T1JN0o6JmmvpOXu/t9NbaQCMxuSVHL3wseEzex6SX+SNOjuC7Jlj0v6yN37sn8457j7P7ZIbxsk/anomZuzCWU6Js8sLek2SXeqwH2X6GuZCthvRRz5uyW97+4fuPufJf1C0uIC+mh57v6GpI/OWLxY0tbs/laN/+Vpugq9tQR3H3b3d7L7H0uamFm60H2X6KsQRYT/Ckl/mPT4mFprym+X9Gsz22dmvUU3M4X2bNr0ienT5xXcz5mqztzcTGfMLN0y+66WGa/zVkT4p5r9p5WGHBa5+3WSvi1pdfb2FtMzrZmbm2WKmaVbQq0zXuetiPAfkzR/0uMvSzpeQB9Tcvfj2e0JSc+r9WYfHpmYJDW7PVFwP3/RSjM3TzWztFpg37XSjNdFhH+vpGvM7Ctm9gVJ35W0s4A+zmJml2RfxMjMLpH0LbXe7MM7Ja3M7q+U9EKBvfyVVpm5udLM0ip437XajNeFnOSTDWX8i6QLJQ24+z81vYkpmNnVGj/aS+OTmG4rsjcze1ZSj8av+hqR9ENJ/y5pu6QrJR2VtNTdm/7FW4XeejT+1vUvMzdPfMZucm9/J+k3kg5JOp0tXqfxz9eF7btEX8tVwH7jDD8gKM7wA4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8Q1P8DdB0CkDYcK7EAAAAASUVORK5CYII=\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "image2 = data.test.images[13]\n", "plot_image(image2)" @@ -2107,23 +1760,11 @@ }, { "cell_type": "code", - "execution_count": 64, + "execution_count": null, "metadata": { "scrolled": false }, - "outputs": [ - { - "ename": "NameError", - "evalue": "name 'weights_conv1' is not defined", - "output_type": "error", - "traceback": [ - "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", - "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mplot_conv_weights\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mweights\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mweights_conv1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", - "\u001b[1;31mNameError\u001b[0m: name 'weights_conv1' is not defined" - ] - } - ], + "outputs": [], "source": [ "plot_conv_weights(weights=weights_conv1)" ] @@ -2137,20 +1778,9 @@ }, { "cell_type": "code", - "execution_count": 59, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUoAAADuCAYAAABf005JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztvelv1Wd6//86hmBjY8cbEGww2CzBTgIhhCRkI8kok0wy\n0ajTzHSUdlRpuo7UVmqfVFVVqX9B1UpV1fbBdDTpjObbJZnJNJpptkIIgQk7hEAAE4MNweANL8f7\nOb8Hn9/7+hyvn2M4x8vx9Xpi4hybw3Xuz3W/r+W+7lgymcRxHMeZmry5fgOO4zjzHXeUjuM4Ebij\ndBzHicAdpeM4TgTuKB3HcSJwR+k4jhOBO0rHcZwI3FE6juNE4I7ScRwngqUzeXFlZWVyw4YNWXor\n2aWpqYm2trbYXL+PqXDbZo+FbFuAo0ePtiWTyZVz/T6mYiHbN921OyNHuWHDBg4fPnz772oO2bVr\n11y/hWnZsGEDR44cmeu3cVs8/PDDc/0WpmUh2xYgFotdnuv3MB0L2b7prl0PvR3HcSJwR+k4jhOB\nO0rHcZwI3FE6juNE4I7ScRwnAneUjuM4EcyoPSgdEokES5Ysua2f9WnrkzM4OAjA8PAwK1asmON3\nk1uMjo4CEIuFrXSdnZ32Z32/oKAAgPz8fADy8vLG/IwzObJvb28vt27dAkL79vf3m13vvvtuAEpL\nSwEoKirirrvuApgXds6Yo5QRGhsbzVHKSDt37kzrd2TCILnobLWwBgYGzKZaWM6d0dvbC8CyZcvo\n6OgA4IsvvgCCjWloaAiAwsJCIHSYFRUV9ufphIHW9NKlS+2zW758OcCi2PS0yTc3N3P27FkALl26\nBMDVq1fJywuC2uLiYgDKy8sBqKysNJtrc5qOgoICeyYqKyvta1FREYD9PbeLh96O4zgRZExRNjU1\nAfB///d/7Nu3D4ArV64AsGXLFnbs2AHAqlWrAHjwwQcBWL16NSUlJUCoQBOJhMluoZ1Zr11MSLHc\nuHGDa9euAXD//fcDi9MemUTranBw0BTlmTNnAPjkk0/sdVI8ZWVl9t9SP1I+o6Oj9vsSiQQQKFUY\nqx6lKDdt2kRVVdWY95FrKMJrbW2ltbUVgC+//BKAixcvmuKTvaQeBwYGxqjx1NdAaC99LSoqQsco\n6+rqAKivr+fee+8F7ly9u6J0HMeJIGOKUsqmrKyM++67DwjyEgDnzp3j4MGDQJjLlIevqKiw3Vrf\n6+3ttZ1bKNfx1FNP8Zu/+ZsAPPDAA5l6+/MaKcp4PE57ezsAJ0+eBGBkZMRsunJlMDdh8+bNANTU\n1JgdlaPJz883BSQFpV2/oqIi6/+W+YbW3MWLF229nj9/HoCzZ8/aupPSkY2GhoZMDSlXOTIyYgqn\np6cHCIsTEOQ8ATZu3AjAc889Z+pS6jTXUJS4ZMkSU9eyyeOPP26RY2rRB6Crq2tM0QcCRSlVqdd3\ndXXZ75d99ZmuX7/ePr87VZQZc5SSu3V1dVy+HJzhl8N89913LTRXIlcL76677jJpre/19fWZARW+\nX79+HQgM093dDcBf/dVfAaGDyFXk0OLxOJ9//jkAhw4dAoIwJjX8gHBRVFZW2p+1sPLz822x3bx5\nc8zfs2fPHv7oj/4om/+UeYeKNcuWLbMHubq6GgjTGxA+rFp73d3d3LhxAxhbQOzr6wNCR6lNKZFI\n2O/QWl67di1btmwBctdRam1WVFTQ0NAAhIMoampqzP4Ky7UmR0ZGTBTI2SWTSXOG8XgcgKNHjwJB\nqL527Vog/Pyqq6snCK7bxUNvx3GcCDLeRwmB5AV44okngEBqazdta2sDsKJEPB63XUXhe09Pj31P\nSlS7cH9/vyki7fgvvfSS7cjji0C5gBTLqlWrWLduHRCGKGvXrjWVo+9J6bS0tNj3xPDwsH0GUj9K\nfaxevZoTJ04AoUofHBy0AlwutrMobK6srLSIRkWBZ555hoGBASBUkgoHu7u7Tf1oPQ4NDdn3pO5V\nuDh58qSlP6RcIb3Wl4WM1OC6detM8U2m8lTUSg3V9WfZrb+/n5GREQBrNdLnMzAwYEpSkayi3Ezg\nitJxHCeCrChKUVNTAwTqRElr5SO18zY3N1u+QbtFf38/+/fvB8Lk+WOPPQbAmjVrLK+j/MeFCxds\n99DOlEtoJ165cqWpHdljZGTE1I5smpoL066snf3WrVuWD1Jj9datWwH4rd/6LVOSFy5cAAIVm6ut\nK6msWLHC1N2aNWsm/H8pF0U6sVjMFLnsE4/HJ+QoP/74YyDIrbe0tABhxNXQ0JCxHNp8RU3gyWRy\njJIWipZkw9Tm/fGN/IWFhba2VQeRTSGMBDZt2pShdx/iitJxHCeCjCnKyY4OakdYsWLFBFUi5VJZ\nWWlKUr+jpaWFxsZGIMxD1tfXA0E+Ta9TPjKZTJpizUX0b0v9N+qYFoQ5MqlGqfbu7m77DLQTf/nl\nl9ZScfHiRQAeffRRIIgA1Bqj3GZdXZ1VbnOd6fLbymXKtkuXLrXcrlR7d3e35dllPymfzs5Oa4uR\n4tm8ebP93lwl3ZqBqtl6fSKRmHDssLu72w4BvPHGG0Bo3/r6erZv3w6QFZtmNfROZboz2Fp8Clsa\nGxvtYa6trQXC3qvKykrrG5QTWLNmzaI+oaIFNX6zSO2LTB3uoLYLFYbuueceIFiIKp7pYV4soXe6\naK2monA8Ly/PbKsNR+1cra2ttiGpPSbX29rSQWtrvENNdZKyb0tLCwcOHADCIq9+bvv27eYos4GH\n3o7jOBHMmqJMh9TpLQpntOsqKdzZ2WmJdSmhysrK2x7ttliQWr927ZrZWfZT6H7mzBnb4WXvyRSU\nMxaFjaOjo2ZbzTvQAYt169bxyCOPAFjj9WTFDWciet4bGxs5deoUECpJTSZ79tlns2pPV5SO4zgR\nzAu5IEWjtpVkMmnnYZV3S50komStGqFzuZBzp0hJqoXo+vXrY2b+QdhE3dHRYefE1bzrinJqlDvT\n+u3q6rJGaDX1a61u2rSJJ598EgjbvZzpkVKXLY8fP26KXWt39+7dAGzbti2r72VePAVykPqan59v\nYeH4Ez2FhYW20PRQ5/rphttlaGhowlTp/Px865tUv6Aur1+1apX1ovnmMz2jo6MWEoobN27YsBIV\ndVL7XrVenWhGR0dtc1e4feLECRNMOn2jcY3ZxkNvx3GcCOZcUXZ3d/Ppp58C4aSgkpISO1esIo16\nLVPVpqueyVErVldXl4XearfasmWL9abqvL16/jZt2pTzJ0UyRTweHzNQGYLQUP2/Cht1Yuzhhx8e\nM3LNmZ7+/n4rhOmU3rVr1yxtpNN5GsybbVxROo7jRDBnilIK8fz581y9ehUI24O2bt1qDafjz9OW\nl5ebovRG6MlJHXgq+0nN1NfX2wmezz77DAhnJq5atcpbViJQznzJkiWWU1de8vDhw9ZormKkigx1\ndXW+XtNAed+rV6/y0UcfAeF5+Xg8bsWbxx9/fFbflytKx3GcCGZdUSp/plzO8ePHraVCVcGlS5fa\nRCGpTOUm7rnnHs9NToGa9FXpbm1ttYlCujajt7fXbK+rD9QI7Tm0qdG6VT63oKDA7KdjdUeOHLG8\npdaybOvXC0+P7Kvn/vTp0xw+fBgIfUBVVRUvvfQSkJ0JQdMx645SYZ8Os7e3t1u7is5r33XXXVbg\nUQiuEzrqnXRC1MenkDv1/hHZTQ/wlStXLJRRInz16tWAnxSZDD3ACrk1kPf8+fMcP34cwIYdd3R0\nmGN8+umngbA9yMPuyZF9lSLSoJYPP/zQUkMSSd/61rd49tln5+BdeujtOI4TyawpSp1i0C4hRRmL\nxWzArxqg29raTBVpAo5OinjYPZHxp2/UZjU8PGxDYhUynjt3zkJsta7k6sVWmUDrVl+V1mhpaeHX\nv/41EKqgtWvX8txzzwHh6DqNYnMmR0VdHSjRZWGHDx+2SEmtQF//+tfn7KSYK0rHcZwIZs09qylX\n7RNqCdq8efOEC7OSyeSEO6p9dt/kDA8Pm5KUstQVD08++aSpR+XISktLLSepo6A+eWlyEomE5dRl\nI0VC+/fvtzy6VNFjjz3Gnj17gHAykzM1iURiwjHFX/7yl0Cg2PXMS6Ur8pwLZsVR9vX12QDTc+fO\nAeGpkIKCAgurVYzYunWrheEKHb3QMDn9/f1mN3UP6MGNxWITBiY/+OCDVjRbLJPLb5fBwUEL/7Re\n5RxPnz7N6dOngbAo1tDQYA/z+OnczkRGRkZMQKkX9cyZM0DgKJ966ikgvN1gLvFP03EcJ4JZUZRd\nXV0WsqgnSrfUnTlzxu4+VuvP1q1bra3CVc/0DA8PWz+fdmOF1C0tLWZnDTotKytzdZ4myWTSQm+d\nlZdqf//9982mKooVFxd7GmMGjIyMWApOa1cKvra21tIY82HqkitKx3GcCGZFUaq1AsJmaLWwtLe3\n27lYtQL19PTYzuyNutOTeqGVzh7rRMP69evNpmqx0PBYJ5q8vDxTlIqEUu+fVrFBefT169f7oOMZ\nEIvFzB+ooKtBx7t37+b5558HmBcTrVxROo7jRDAr2191dTXf+MY3gPCqTlUMjx49avlL5Xx6e3tt\np3GmZ2hoyHZhNY6rw+DGjRtmR531doWePgUFBdYwriq2puk/88wzVu3+yle+AgS5NM//ps/IyIgd\nCdVVs/IBr7zyyryqT8yKo8zPz7cFprsutKAuXbpkD696z9asWZP2xemLnaGhIRu4oJYqDeaNxWKW\n1lCLxbp16zw8nAHaaFQge/XVV4Ggt08bk2y7mO+Wvx3y8vKora0FwpN3sul8cpLgobfjOE4ksfEN\nydO+OBa7CVzO3tvJKuuTyeS8Pd7jts0eC9y24PbNJmnZdkaO0nEcZzHiobfjOE4E7igdx3EicEfp\nOI4TgTtKx3GcCNxROo7jROCO0nEcJwJ3lI7jOBG4o3Qcx4lgRod+Kysrkxqou9Boamqira1t3k6E\ncNtmj4VsW4CjR4+2zeeTOQvZvumu3Rk5yg0bNnDkyJHbf1dziKYWzVfcttljIdsWIBaLzevjgRs2\nbLCrexcaulY4Cg+9HcdxInBH6TiOE4E7SsdxnAjcUTqO40TgjtJxHCcCd5SO4zgRZOXylN7eXgD6\n+vqA4CJ5XZvqd+HcPj09PTQ3NwMwPDwMBPfi6AIs3Z2jy8b0FcLLsZxobt26BUBbW5tdtbx8+XIA\nVqxYAUBxcbGtZbft1OjOoXg8TldXFxDaN5lM2lW0umRM9w7l5eWZXefDcPGMO8rLly/bLYDXrl0D\noKGhgUQiAWAPtb4mk0n7fwMDA1P+Xr1maGjILh7S4s11ZJe9e/dy/PhxILycbfXq1XYnsh5iLbqB\ngQG7l1r2m468vDyqqqrs9wIsXbrULiPTXeu5ijZ43RB67Ngxu/hOl+OtWrUKCC7J00M93c2LqetW\nD7w+p9Tfkau27enpAeCzzz7j3LlzAJw5cwYInKg2c9lVF+StXr3a1vF0G5E+n2XLlpktdUFZaWmp\nvS6d9T8dvhU6juNEkDFFqZDw5MmT7Nu3D4Du7m4gUI2S29qZRX5+vu0KUj8jIyO2i2in1df8/Hzb\nMRbqsamZ8sUXXwCB0tGuvHnzZgA2btxoIffIyAgQ3OcNQeqjo6MDCMOdvLw8UzYKHWX/0tJSVq4M\nTsppN19MSOkpXIzFYmbT1HAcAlsrotHaTCQSY342lZ6eHrO7rmXetWsXDQ0NQBhh5RoKt69du8b5\n8+cBLCo6f/68rUHZUOtu+fLl9nlIdab6hfH+obKy0q6+1R32O3bsoKamJiP/DleUjuM4EWRMUWoH\nLSoqMqXX398PBLkfqaLOzs7gL/7/815Lly41lak8Qk9Pj+UhVQRSYWh0dJSNGzcCi0dRKke5fPly\nHn/8cQCefvppALZt22avkyJvb28HgryYVL2UZSwWM3tr97548SIQfBb33ntvVv8t8xmtTSmY9evX\nT8iP6TWpOS+pmqGhIeLxOBAW1mTrixcv8umnnwKhKq2urrbIIFeRHWpqaiyXu379eiCwyfXr14Ew\nCtLX69evW85YvmJgYMD+LPtrfS9fvtzWrp6DiooKy1Mqf3m7ZMxRKgk7NDRkhQA5RwgXh6S2wpbl\ny5dbKK3QZHBw0JK7MsSlS5eAwLjjw/dcp7KyEghCNW0c9fX1E14nu6ggky7ahBTqLCYUGl66dMk2\nmC+//BKAs2fP2sOtz0DdBkVFRVYR18Oo/wfhBq8Nqr29nZaWFiC0c0dHR85XzGU3fZ2K1tZWIEzh\nNTc3m9OUXZPJpKU2rly5AgSpPgjEWHV1NRD6mOHhYfuM7pTc/pQcx3EyQMYUpcLs5cuXm0eXKmxu\nbjbVqP+nnWF4eNhUo3bvkZERk90qDL399ttAoFh37NiRqbe9IJA6GR4eNjtmggsXLgDw1ltvAfDd\n7343Y797oaD1uGzZMvuzIqJr166ZImxqagJCdTM4OGgFHqWJRkdHLcSTOpXiOXjwoCmkRx55xH6X\nQvXUVpbFiGyur+mO7jt48CAQ+BiF5VKva9assc/mTnFF6TiOE0HGFKUS4Kk5FxULKisrLcEqJalG\n1P7+ftt9U3fm8c2pamLfs2fPhILD4ODgmMbTXEP53Mlyj6Ojo6ZKZFu9fqpc7okTJwD4m7/5GyA4\nJADwO7/zOxl81wsDrbmNGzeydu1aIMyVNzQ0WA5TNlb0Mzg4aHn3VFWoXL1ybR999BEQFCekNqWa\nysvLM5ZDW2yowKk1v27dOlOSsm9qy5U3nDuO42SZjCnK1HYfod26vLzcdmk18Ep5piqi1DYA5X+k\nRB966CEAvv/97/Piiy/az+r1qbmmXENqOfXstliyZIntnLJH6u6pP+tra2sr//qv/wrAO++8A8C3\nv/1tAGt+Ho9+NhcrtLLt8uXLJxyJLSsrs+Ohsq2+JpNJW8tao6n54/379wNjG6J37twJwIMPPmjf\ny/XcpOw1FbJ/OmtrcHDQahaffPIJEHYQ3HffffZZyY+Mb/q/E7IyFGMy9KbHD8VYsmTJBOfW3Nxs\n58QlsZ955hkg7B+E8GxuMplcNOe+p2P8KSYIF6Ae6gMHDvDBBx8AYSj/l3/5l9P+3kwuuIXKZLYd\n348KodP8+OOPgbCoU1dXZydHtCGtXbv2jvv7FjoSUNM5VLWvHTt2jGPHjgFhCkQF47KyMhNmqQWc\nOw25Re5JBMdxnAwza4oyHbQbNzU1mbTW2eOvfvWrQFDyl5JUq0ZJSYmrngikbPbu3WuNz3/yJ38C\nwP333z/tz7pt0+fs2bMAdiuhGqm3b99up3AUoq9atSon0xmZQmpT6rG1tdUiTJ3K27JlCwC1tbUT\nWueSyWTGRrT5p+Q4jhPBvFKUasg9d+6cFXO2b98OjD3TrOLP+MKQMzWa2HLw4EFLev/Zn/1Z5M8l\nEgm3b5oMDQ3x/vvvA+ExyNTmZ+UoZf/FOKFpJuj5lirv6OiwnK5sKZWuAcDZYl44ShlEpxgOHTpk\nMvr5558HwrPk8XjcqlqqGKZW2p2xqJ/vF7/4BRCclf/jP/5jYPoz4blc6c4W+/fvt2q3Nnr1/N5/\n//3W36ehED7tf2oSiYQNzNCch87OTttkZEsVcSdbp5kq5ICH3o7jOJHMCymmsPDAgQNAELb8xm/8\nBjC2HQhy/xROJonH46Ykf/WrXwFBEvzP//zPI3/WlWT6qFD27rvv0tjYCISKp66uzr5KwU/WD+sE\nqPhy69YtG0unAlk8HjcVrv7Jyc5yR/Vu3g7+NDiO40Qw54qysbGRQ4cOAWGTbn19PS+88AIw9qIs\n4Unw9HjnnXd4/fXXgXDu4p/+6Z96bjLDqIDzwQcfWMuaig2aG1pRUWF5dmciqbNoIVCRmkmgwdIP\nPPCAFW00EHi2okp/GhzHcSKYM0WpFp9Tp06xd+9eINxVvva1r004d6xpQ3fddZfnJiNQlfCtt97i\nyJEjAHzrW98C4C/+4i+m/VlXkumjLo2f/vSnQDC/UutW1VlN3a6qqlp0k/lngp79q1evAkGLoP6s\nlqA1a9aYQk+9/1tkIzcp5sxR6szm/v37rYVFBZwnn3zSXqdTJJOd43TGovTEm2++CcDPf/5zO3Xz\nt3/7t5E/7z2T6dPV1cXPf/5zIBxbV1lZyaZNmwDsq/oodfOiMxY5SJ2203jFixcvmiDatWsXEAzG\nUZFstjcdfyocx3EimHVFqalAStCeOHGCrVu3AvDEE08AwS6s2+6EprZ42D0RFWB0vlgtQUNDQ3a9\ng87ETvfzribT56233rIxdVJDDQ0NZmeF3PrqYffkqHija0l0IWF3d7elMdRitWXLlikPl2Qz7AZX\nlI7jOJHMmqLUyHsVF1TA6erq4pvf/CYwdnCs2iykIBfjVarpohzvf//3fwPhhUt1dXWRxRvw6UAz\nQdekvv3226bgdYVEaWnphCKO5iU6E0kkEty8eROAzz77DAiVZSwWs4HUulxvMjWZqelAUcyao5Sk\nVkVWxZzS0lJLdKfe/avTC3KUqQNTnZChoSEOHz4MYAN5tSm98sor0/6swhW3bTR6IGXrkydP2owC\n/b/q6mpzkOOnbTsT6e/vt41HRVtt+hUVFbaONRSjqqpqzD04kNnz3NPhobfjOE4Es7Ld9fX12SQQ\nnevWDrJjxw4bwpnKYh+Rny6dnZ2cPn0aCMNCKfO//uu/nvZnXUmmj4qQGih9+fJli4T0tba21iYD\neRtbNPF43KYsKdUmW5aUlNh0MKXd2trarCg222vXFaXjOE4Es6Ioe3p6bMKKco66SXH79u123YNa\ngkpKSlztpElXV5flGtWo/+qrrwLheVjnzlHuTHmzxx57zPLoGi5dXV3tRccZsGTJEmsg3717NxAW\nbkZGRiwfKZVZXFw8Zy1srigdx3EimBVFuWLFCqsG7tmzBwirVWVlZfT39wPhVOg1a9bYzuJN0NNT\nWFjIAw88AIQTa773ve/N5VvKGUZGRiy3rgp3auO+ZiPqnu65VDwLkZUrV1o0mS7yG7PVFiRmxVEW\nFhbaSHw5TBVzuru77a4cSeyhoSE/LZImZWVl7Ny5EyCtu81HR0dndOn8YmZgYMAcpe7AUahYWlpq\nG7x6JWtra/0ETpbR2p1tR+lPiuM4TgSxmXjmWCx2E7icvbeTVdYnk8mZ6fxZxG2bPRa4bcHtm03S\nsu2MHKXjOM5ixENvx3GcCNxROo7jROCO0nEcJwJ3lI7jOBG4o3Qcx4nAHaXjOE4E7igdx3EimNER\nxsrKyuRksyMXAk1NTbS1tc3bOw/cttljIdsW4OjRo23zueF8Ids33bU7I0e5YcMGG4W/0NDdwPOV\nDRs22H1CC42HH354rt/CtCxk2wLEYrF5feplIds33bXrobfjOE4E7igdx3EicEfpOI4TgTtKx3Gc\nCNxROo7jROCO0nEcJwJ3lI7jOBFk7M6c1Lssent7gfBqz5s3b7J0afBX6WtJSQkQ3POiS5r8itpo\ndFF8d3c3APn5+X5F6h2gu5l6enrs7qabN28CwT05ul65rKwMCC6+A6isrKSwsBAILxlzpqa7u9su\nDxwYGACCO5t035DsPF/vHMr45WKxWMzu4718OeiTPXz4sF3SJGeo29dWrlxp909PdzmWJrGPjo7a\nAtXrS0pKzPHKEecqeih1K6BsAaGNtGnNlGQyeds/u1DRhvPhhx9a0/Qnn3wCwPXr1+3u7nXr1gFQ\nXl4OBPdPa4NK51K3u+66y5yCLiirqamhqqoKmL8O4k6Rczx8+DBtbW1AcBc9wKZNm+yCNtlQDnN4\neNjWswTXZGi9FhQU2Gcj/5PJteyht+M4TgRZlV/r168Hgp1D3r2jowOA5uZmADo7O20XKSoqsp8d\nHR0FQgWqXaWtrY2+vj4gCH8A6uvreeihh4Bwx891ZINly5aZjbRTSwVJZadLf3+/7fo1NTWZeqvz\nGinKVPVRV1cHQDweNzt/+umnQGj3np4e+1ld+zs6OjpB1SvCqaqqsjX62GOP2e+QktJVzbmGnvf2\n9naampqAUD0mk0mLjHp6eoDwOR8YGDBbyxcMDw/bWpfN9XX16tWUlpYCmVWSwhWl4zhOBLOS0Nu9\neze7d+8e8z3lLAcGBqz4o51mZGTEvqddQon2xsZG9u/fD8DJkyeBYJevra0Fcl9RXrx4EYDTp08D\nwb9dBQbZb3BwEAgKEFLdytu0t7dbnlN5NxWI9u7dy9mzZwH45je/CWB2zVWknGtqanjllVeAUNU0\nNjaaklS+XYWeWCw2oTgRi8VMIUn5tLS0AIGNFTlJPS1btoxcvwU1tUCr9blx40YAdu7cadGPbKK1\nODw8bGpUBbf+/n6LOvU92be/v9/smw3mrPIx01BDxt2yZYst3kOHDgFw33335fSC0wO5b98+jh49\nCoQLqqSkxBajigutra1AsPi04WiBdXZ2mqPU5BT9/rfffttC723btgGBA1ls3Qh64Orr66mvr7+t\n33HlyhUA3nnnHQA++OADK9jImS5dutQKmbmK1l9eXp6tQRUgU0VNalFSaJPWxg9h0evSpUsAXL16\n1b4/2e/IFB56O47jRDArivL69etWWJDX1y4xPDxsyddUWa3vjU/M9vb2WvvGF198AQShT662V0C4\na7a3t1uoIoVdXl5uoaKS3lKMhYWFtqNLKULYC6jXKdzu6urihRdeAGDHjh1AEDrF43H7fU56VFdX\nA7B161YgCA2lMh944AF7Ta63s+nfV15ebmtXir25udnWrFS2FHZxcbG9PtUXaC0q7aa1m+3n3xWl\n4zhOBFndzpQ3LC4unjLPNTg4SGdnJxDuCkuXLrX8xfid4vTp05w/fx4IFejKlStzulFaBYfly5fb\nDqzdFsKcpFSjCjclJSVmF+3Eq1atMpWj3/Xhhx/afz/66KNA2HoFriRvB61prdGtW7daMUO2ra6u\nzmoBYj6gFsHKykorwMgvLFmyxFraZC9FR52dnbaOxeDgIBcuXADgjTfeAMJijiKsbOGK0nEcJ4KM\nKcrpqs6FhYUTFJ+UYn5+/oTK33R5mytXrpja1DHI++67b0yzeq6havZUZ7qVc1Q+MhU15ErRp6rD\nEydOAPAgL2e6AAASdUlEQVTuu+8CgT1zXeFkk/b2do4fPw6MnX0AcO7cOdauXQsEBzBg6s8zl0h9\nzidDz7KOcmqdDg8Pmx9Q/rKpqcna4nTctKKiAsj+nVizlkmezpHKONOFz+rS379/vznWPXv2AMFJ\nipmeQsklZL+Ztpr8wz/8A8CY3kmdHnHSR2vz7Nmz9lDLCSrlUVxcbJv5Yjn1NBNSU0kwdtCIUkRX\nrlyxHmrZ+etf/zoQnqbKFh56O47jRDAvehPSKcT86Ec/AoKWIMn0Rx55BAhOmCy2pug74a233gLg\nzTffBDAV+fzzz1uo7qSPFGXqhCCpRx0QqKiosOZ1X6szQ4We48ePc+7cOSAM1X/3d393Vt6DPxWO\n4zgRzAtFOR2a0PLee+8BQaHn8ccfB8j5WX7Z4sc//jEQtmK8+OKLQNhk7qSH8mQ6AlpUVGTf03Qs\ntcSUlJRYE3out7JlEtnu888/B+DgwYOWr9S5fBUys82cO8qoRfNv//ZvQHi2c+vWrRZyq3fKHWX6\n/OQnP+HgwYMAfPvb3wZmL3zJJRKJhDlDnZxKDak1vETOcfPmzZ7WmCE6yaQCzvnz561z4Hvf+96s\nvhf/5BzHcSKYM0UZpSR1+mbfvn1AOIT2iSeesPFg6YzgdwI0CuzNN9+0QsPLL78MhCkMJxq1uXV1\nddl5Y4XbBQUFNi5M/ao6hTNbIWKu0NnZOeFqjkQiwWuvvQbM/jhFV5SO4zgRzHmOcir+/d//HQhH\nyd97770APPjgg3Y5kyfF0+c//uM/ADhz5owNUd65c+dcvqUFiQpgJ06csLP1KjqsXr3azi5rluKG\nDRsAX6vpoqlix44d48CBA0CwZgGeffZZfv/3f39O3pcrSsdxnAhmXVGms7O++eablp9QblLTuNet\nW+e5yRmg89x/93d/BwTHHJ977jkgqMQ66aHcpC7Iunr1qh2z04ScpqYmy/cqJ+m5yZmResW1ujM0\n0+EP/uAP5ux9zZqjTMdBKhH+s5/9zL6nkWDbt28HQqM50YyMjPAv//IvQHhH0WuvvWZn5J1o5CC1\nNvUgFxYWWlua2oJWr15tRYYtW7bM9ltd0Khf+tixY0Bwf5PGB37/+98HmHDv1mziobfjOE4E86qY\no2k2N27csMvHdApHjbsedqfP3//93/PTn/4UCE/dvPrqq9Ze5USj4s2pU6eA8Fx3LBazc906DFFQ\nUGDFG1+n6SH76sJADZFubGy0NfuHf/iHc/PmUnBF6TiOE8GsKMqo/OT//M//APCf//mfQNBu8dJL\nLwFhrke7txONjta9/vrrdq2t7umWQneiSSaTltvVMUUdQ+zt7TU7K0dZV1eX0wOks8G1a9eAcMrS\nL3/5SyBo6NcR29LS0rl5cynMi9D79ddfB8IBso888oj1TSpMTB3k6UyPFtupU6d4+umngbBrwEmf\neDzOl19+CYSh9M2bNwG4cOHChPtadu7cuSimlmeK/v5+G3jx0UcfAWFXwbPPPst3vvOduXprE/DQ\n23EcJ4I5V5SfffaZneVUmPjQQw9Zj5/v0OmjoaY//OEPASgrK+O73/0u4KdwboehoSHKysqAMLxW\n6H3x4kU7x62bKxUFOenR19dnivLXv/71mP/327/923PxlqbEFaXjOE4Ec64om5ub7fSCTjjs3LnT\ndnInfdTCojxaWVmZKSDleBOJhBXX/Pzx9JSWlk65Dnfv3m3numXb2Z5os9AZGBiwoceaM6mm8t/7\nvd+bs/c1Ga4oHcdxIphzRVlcXGwTy4uLi4GgzcLvl06PoaEhqxiqWVe5s127dtnRT+FTttNnOsVd\nUVFhcwhk02Qy6Sp9BiSTSZsEpuZyRZXzjTl3lIlEgoaGBiCU3ytXrvSb6tKkvb2dGzduAGEI+Oyz\nzwJB76QXcbKHBvb6KZzbI5lMWn+0xNJ8vbfJ5YXjOE4EMU1HSevFsdhN4HL23k5WWZ9MJuft6CG3\nbfZY4LYFt282Scu2M3KUjuM4ixEPvR3HcSJwR+k4jhOBO0rHcZwI3FE6juNE4I7ScRwnAneUjuM4\nEbijdBzHiWBGRxgrKyuTujxpodHU1ERbW9u8PYjrts0eC9m2AEePHm2bzw3nC9m+6a7dGTnKDRs2\ncOTIkdt/V3PIfL8KwW2bPRaybQFisdi8PvWykO2b7tr10NtxHCcCd5SO4zgRuKN0HMeJwB2l4zhO\nBO4oHcdxInBH6TiOE0FWr4Lo6+sDoL+/n0QiAUBhYWHwFy8N/ur8/Hx7vd83MnO6u7vp7u4Gwqsg\ndN9QYWGhfc/vypmcgYEBANra2ujo6ACw2xVHR0ftmoeKigoguJkRgvudZFu/tmRqhoeHAbh165bZ\ntb29HQj8g+wru+q+pxUrVphvmA9rNyuO8uzZswC0tLQAwcKTE5RBdKnQqlWrzCBynpOh+0m6u7vN\ncLoedNmyZbZYF4uz1ZW0Fy5csMUoB1lQUAAE19Xqz6kb0ng0vHlwcJB4PA6ETrekpMQufctFh9DW\n1gbA559/zoULF4Bw/V64cMEc6cqVQb+37niprq62y8WmuzNHa/Xuu+82J3DPPfcAgfOVbadb+wsZ\nOccLFy5w6dIlALsM7+rVq7bOZFddXV1WVmb2nc42sm9hYaH5g1Q733333cCdr925d9WO4zjznIxt\nY1Il8XickydPAsEuDdDZ2WlKT7uvdury8nJTPSIWi5mC1I6jXWVwcNB+1wMPPADAtm3b7KL68b8r\nl4jH41y5cgWA06dPA4Fql62kemSDgoIC+3Pq9b96vZA94/E4g4ODY35HTU0NtbW1ALY75xKyRSKR\nMNWhtdzc3Exra+uE10Fgn9QUx1RI8axcuZKNGzcC2K2jO3fuZNu2bUAY2ucaUoXbt2+3NXXt2jUA\nzpw5Y/a9desWECq/FStWmF0VDaWuW9lVr6+srKSmpgYI7bt7927q6+uB6T+jdHBF6TiOE0HGFKVU\nSW9vr6lG3dXb2dlpO7FyYFevXgWgo6ODoaEhAFMziUTCvqcL0fXzx48ft3yG8jvl5eX2d+aiorx+\n/bp93bt3L4Cp9mvXrpmalqKU+h4ZGbEdV7txLBazIptsVVVVBQSfjRLtUjiJRMLsnYuKUmuotraW\nuro6AB588EEANm3aRFNTExDm2qSGWltb7Xtaq7FYzNap/p++Xr16lZ6eHiDMi8ZiMaqrq4HcVZRS\ng/n5+Tz00EMA9nX37t22jj/77DMgtNfAwICt4xUrVtj3RH9/P4Ap0kuXLnH+/HkAzp07BwTRq4Z1\n3KmizHgGubm52R5OLYL6+npbQPoqent76e3tBcIqeTKZtIdYDvDy5WAuQEtLizkGJX7z8/Mn/N5c\nQqFgW1ub2UWL57777rPX6YFV9ba7u9s2H1XGU9MaClXkHGVjgHvvvdf+7lwtNEDooFIdlULkXbt2\n2ff0kKqIdu3aNXOao6OjQJAmUgh55swZAHO0AEVFRUAoFvr6+uwzW4w89dRTPPXUU2O+J/v19PTY\n2tUGPTAwYJuNUlD6+sknn/Bf//VfQCDMICggad3f6UbkobfjOE4EGZMKCqWLi4ttx9TXvLw82zml\nTiSxq6qqJoTLJSUlppj0/95//30gUEubNm0CsK8bNmyYF71W2ULhy6ZNmywVofBwxYoVtvNK4UiZ\nt7W1mV0UqoyOjlobhf6fQpYvv/zS/k7t4jU1NRZ6L2a0DtetWzfm61QoJFSr0ejoqH0+elZqampy\nMp1xJ8geUXZRkUasWbPGRr0p9L5x44Y9C3dK7noXx3GcDJExRSmVsmLFCstNKveYSCQsLyYPr+Rq\nao5GKrK8vNzynMo3qEn11KlTfOc73wHCJtWCgoKcVpRqsSgqKrJ2qdQil/7tyq2J4eFhy2/q54aH\nh81ujY2NQKh+8vPz7TPT7xr/O53pUa5cBTgVOZPJpH2O+l5DQ4Pl2507Y926daYylecsLS3NWH49\nd72L4zhOhsiYolR1OvU4lxQihDutFKTybr29vaaIVBVM5Z/+6Z+AsMH6kUcesQqWduZcVpMQ5nVT\nd8fUBnKpRtlbr0skEmMqsuNRS8Z7770HBDlOVbvVwqEjp040LS0tFvmoBUiKvK+vz54BHd9duXLl\ntMcfnbEMDAxMqISLvLw88x9qON++fbu1zN0ps9b3IWc2vnCjPrbxKCH74YcfAmELxssvv2wSO9UR\nL2a0YYwPM/Ly8ibdRC5evAjAz372MwC++OILILDnE088AYSnnpxo1I515swZ+7NSS2pnycvLs81N\nbW3FxcU5eX4+08iGPT091vM7nmPHjnHjxg0gbHurr6/PWGojt6WY4zhOBpi3ncQ//OEPgbCh9Ktf\n/SoQhDLaVVLDTyc9EokEn3zyCRC2BSl037RpE7t37wY85E4H2U0N+x0dHZZS2rJlCxCqof7+flM6\nUpQedqeH7DyZmtREoiNHjpg/0InAjRs3Ziwt54rScRwngnmpKD/99FOOHj0KhEWfl19+GQh2ifHz\nF530aWxstLyvlJDmI77yyis8+uijc/beFhpqpVIj+dDQkBVqlHvU+k0mk9ZCl86cRWfs2e6p2Ldv\nHxC0Y23duhWA+++/H8hsDWNeflL/7//9Pys4fO1rXwOwgQWVlZUestwBhw4dsgG1eoh1ymfnzp2T\nVsediQwODtrZ+ObmZiDoUR0/wEGh9z333DOhK2GxDJm+HUZGRqwXVamKVJQ20pAYgB07dgBh1Xu6\nYdUzxUNvx3GcCOaVoty/fz8QyGm1Az333HNAOInI2yluj1OnTgGBjTUBRwUbhdupk4icyVEvZGtr\nq53jlvIpLy+3U05qD1J6qLKycsLJHGciKtx0dHRYz+RkylC+QtOBtm/fbuv3TkeqTYYrSsdxnAjm\nhaJUo+iPfvQjIGjcfe211wDsGgLtxl7AmRk6IfKrX/0KCNS6dm3lch5++GHA1fp0pM4EhWCNKo+u\n4uLq1atNBSkPqUjo7rvv9vxvGkghtrW1TTpB6NChQwBW7NVrduzYYe1X2cAVpeM4TgTzQlH+4Ac/\nAELVU1dXx/PPPw+EU3IyWcFaTHz88ccAvPHGG0Cg3rdv3w6ETfxqp3CmRjlzzey8evWqfU8tQcPD\nw3Zkbvw91cXFxZ6bnAblftWyVl5ebm1roq2tzRSljopq7TY0NGS1G2bOHeV7773H22+/DYRDMV55\n5RUzwPgBGE76nD592k446QRDaWkpL7zwAoCd63amZryD1HCW9vZ2a1vR4IXh4WFzkJ4ySo/U4g2E\nhRhtMKmcOHHC2oIUcitttHbt2qy+Tw+9HcdxIpgzRamd5Ac/+AGffvopAK+++ioQNJkrCe5Kcuao\n4PCLX/yC48ePA2HB4cknn+Sll14CwpDRmRo1jGvCkq4wSR3Eq/PEhYWFpjIVCWlalq/jydF1MSro\n6tbE1FNLaujfu3evNfJrJoFO42T7EIorSsdxnAjmTFH++Mc/BuDAgQP2PeXMqqqqcn4YbzbRXclH\njhyx5LiUzle+8hWfNZkmQ0NDtLS0AOElYVI3VVVVdkmYznAXFxdb7kzFRz/PPTWjo6Om0JXbTW0W\n18GIf/zHfwSCHKWut1VuUpFntpn1T1EDBH7yk58AwRi1b3zjGwBWMezv7/ees9tAI+k0KOD48eMW\nquzcuRMIOgp8E0qPvr4+K+LorifdZtnZ2WnT+nWvU21trYXYWr8eck9NPB43O2mzEe3t7fzzP/8z\nEPZX19fX21yCzZs3A5NP7s8G/sQ4juNEMOuKUv18Z86cAYKpKrraQZ31U10P4UyOVKNsql6zpqYm\na11RuK2+VCeaeDxutlV7j4oHfX19FlarkLBs2TJXkjMgPz9/ymf98uXL/O///i8QnqV/6aWX5uwa\nGFeUjuM4EcyaolTDsxLgKtwkk0l27doFhPPknPQZGBiw2xR1Ckd54Pr6estNaid2RZk+RUVFbNu2\nDQivF1Ae/YsvvrC5nirgFBUV+azUGTBdE35tba017auFaO3atdaSNdu4onQcx4lg1hSlcja6dEkN\nuU8//bTt1s7M6erqMgWpI6BS60uWLOHFF18EMNXupE9paemUl6zde++9NnVf55QHBwf9qGKGKCsr\ns7PeymNu27ZtwnXXs8WsOMp4PG7tFTrDqRvV3EneGYWFhVYEU9uPQpb6+noLHZ3MM741xZ1kZtHw\nFtm1oaFhzobjeOjtOI4TQUxnrtN6cSx2E7icvbeTVdYnk8mVc/0mpsJtmz0WuG3B7ZtN0rLtjByl\n4zjOYsRDb8dxnAjcUTqO40TgjtJxHCcCd5SO4zgRuKN0HMeJwB2l4zhOBO4oHcdxInBH6TiOE4E7\nSsdxnAj+P9VNNSIcvuqDAAAAAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "plot_conv_layer(layer=layer_conv1, image=image1)" ] @@ -2164,32 +1794,9 @@ }, { "cell_type": "code", - "execution_count": 65, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "ename": "AttributeError", - "evalue": "'AxesSubplot' object has no attribute 'flat'", - "output_type": "error", - "traceback": [ - "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", - "\u001b[1;31mAttributeError\u001b[0m Traceback (most recent call last)", - "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mplot_conv_layer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlayer\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mlayer_conv1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mimage\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mimage2\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", - "\u001b[1;32m\u001b[0m in \u001b[0;36mplot_conv_layer\u001b[1;34m(layer, image)\u001b[0m\n\u001b[0;32m 24\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 25\u001b[0m \u001b[1;31m# Plot the output images of all the filters.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 26\u001b[1;33m \u001b[1;32mfor\u001b[0m \u001b[0mi\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0max\u001b[0m \u001b[1;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0maxes\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mflat\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 27\u001b[0m \u001b[1;31m# Only plot the images for valid filters.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 28\u001b[0m \u001b[1;32mif\u001b[0m \u001b[0mi\u001b[0m\u001b[1;33m<\u001b[0m\u001b[0mnum_filters\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", - "\u001b[1;31mAttributeError\u001b[0m: 'AxesSubplot' object has no attribute 'flat'" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADYBJREFUeJzt3HGI33d9x/Hny8ROprWO5QRJou1YuhrKoO7oOoRZ0Y20fyT/FEmguEppwK0OZhE6HCr1rylDELJptolT0Fr9Qw+J5A9X6RAjudJZmpTALTpzROhZu/5TtGZ774/fT++4XHLf3v3uLt77+YDA7/v7fX6/e+fD3TO/fH/3+6WqkCRtf6/a6gEkSZvD4EtSEwZfkpow+JLUhMGXpCYMviQ1sWrwk3wuyXNJnrnC7Uny6SRzSZ5O8rbJjylJWq8hz/A/Dxy4yu13AfvGf44C/7T+sSRJk7Zq8KvqCeBnV1lyCPhCjZwC3pDkTZMaUJI0GTsn8Bi7gQtLjufH1/1k+cIkRxn9L4DXvva1f3TLLbdM4MtLUh9PPvnkT6tqai33nUTws8J1K35eQ1UdB44DTE9P1+zs7AS+vCT1keS/13rfSfyWzjywd8nxHuDiBB5XkjRBkwj+DPDe8W/r3AG8WFWXnc6RJG2tVU/pJPkycCewK8k88FHg1QBV9RngBHA3MAe8BLxvo4aVJK3dqsGvqiOr3F7AX01sIknShvCdtpLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDUxKPhJDiQ5l2QuycMr3P7mJI8neSrJ00nunvyokqT1WDX4SXYAx4C7gP3AkST7ly37O+CxqroNOAz846QHlSStz5Bn+LcDc1V1vqpeBh4FDi1bU8Drx5dvAC5ObkRJ0iQMCf5u4MKS4/nxdUt9DLg3yTxwAvjASg+U5GiS2SSzCwsLaxhXkrRWQ4KfFa6rZcdHgM9X1R7gbuCLSS577Ko6XlXTVTU9NTX1yqeVJK3ZkODPA3uXHO/h8lM29wOPAVTV94DXALsmMaAkaTKGBP80sC/JTUmuY/Si7MyyNT8G3gWQ5K2Mgu85G0m6hqwa/Kq6BDwInASeZfTbOGeSPJLk4HjZQ8ADSX4AfBm4r6qWn/aRJG2hnUMWVdUJRi/GLr3uI0sunwXePtnRJEmT5DttJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNDAp+kgNJziWZS/LwFda8J8nZJGeSfGmyY0qS1mvnaguS7ACOAX8GzAOnk8xU1dkla/YBfwu8vapeSPLGjRpYkrQ2Q57h3w7MVdX5qnoZeBQ4tGzNA8CxqnoBoKqem+yYkqT1GhL83cCFJcfz4+uWuhm4Ocl3k5xKcmClB0pyNMlsktmFhYW1TSxJWpMhwc8K19Wy453APuBO4AjwL0necNmdqo5X1XRVTU9NTb3SWSVJ6zAk+PPA3iXHe4CLK6z5RlX9sqp+CJxj9A+AJOkaMST4p4F9SW5Kch1wGJhZtubrwDsBkuxidIrn/CQHlSStz6rBr6pLwIPASeBZ4LGqOpPkkSQHx8tOAs8nOQs8Dnyoqp7fqKElSa9cqpafjt8c09PTNTs7uyVfW5J+UyV5sqqm13Jf32krSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSE4OCn+RAknNJ5pI8fJV19ySpJNOTG1GSNAmrBj/JDuAYcBewHziSZP8K664H/hr4/qSHlCSt35Bn+LcDc1V1vqpeBh4FDq2w7uPAJ4CfT3A+SdKEDAn+buDCkuP58XW/luQ2YG9VffNqD5TkaJLZJLMLCwuveFhJ0toNCX5WuK5+fWPyKuBTwEOrPVBVHa+q6aqanpqaGj6lJGndhgR/Hti75HgPcHHJ8fXArcB3kvwIuAOY8YVbSbq2DAn+aWBfkpuSXAccBmZ+dWNVvVhVu6rqxqq6ETgFHKyq2Q2ZWJK0JqsGv6ouAQ8CJ4Fngceq6kySR5Ic3OgBJUmTsXPIoqo6AZxYdt1HrrD2zvWPJUmaNN9pK0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqYlDwkxxIci7JXJKHV7j9g0nOJnk6ybeTvGXyo0qS1mPV4CfZARwD7gL2A0eS7F+27Clguqr+EPga8IlJDypJWp8hz/BvB+aq6nxVvQw8ChxauqCqHq+ql8aHp4A9kx1TkrReQ4K/G7iw5Hh+fN2V3A98a6UbkhxNMptkdmFhYfiUkqR1GxL8rHBdrbgwuReYBj650u1VdbyqpqtqempqaviUkqR12zlgzTywd8nxHuDi8kVJ3g18GHhHVf1iMuNJkiZlyDP808C+JDcluQ44DMwsXZDkNuCzwMGqem7yY0qS1mvV4FfVJeBB4CTwLPBYVZ1J8kiSg+NlnwReB3w1yX8mmbnCw0mStsiQUzpU1QngxLLrPrLk8rsnPJckacJ8p60kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNGHxJasLgS1ITBl+SmjD4ktSEwZekJgy+JDVh8CWpCYMvSU0YfElqwuBLUhMGX5KaMPiS1ITBl6QmDL4kNWHwJakJgy9JTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ1YfAlqQmDL0lNDAp+kgNJziWZS/LwCrf/VpKvjG//fpIbJz2oJGl9Vg1+kh3AMeAuYD9wJMn+ZcvuB16oqt8HPgX8/aQHlSStz5Bn+LcDc1V1vqpeBh4FDi1bcwj4t/HlrwHvSpLJjSlJWq+dA9bsBi4sOZ4H/vhKa6rqUpIXgd8Ffrp0UZKjwNHx4S+SPLOWobehXSzbq8bci0XuxSL3YtEfrPWOQ4K/0jP1WsMaquo4cBwgyWxVTQ/4+tuee7HIvVjkXixyLxYlmV3rfYec0pkH9i453gNcvNKaJDuBG4CfrXUoSdLkDQn+aWBfkpuSXAccBmaWrZkB/mJ8+R7g36vqsmf4kqSts+opnfE5+QeBk8AO4HNVdSbJI8BsVc0A/wp8Mckco2f2hwd87ePrmHu7cS8WuReL3ItF7sWiNe9FfCIuST34TltJasLgS1ITGx58P5Zh0YC9+GCSs0meTvLtJG/Zijk3w2p7sWTdPUkqybb9lbwhe5HkPePvjTNJvrTZM26WAT8jb07yeJKnxj8nd2/FnBstyeeSPHel9ypl5NPjfXo6ydsGPXBVbdgfRi/y/hfwe8B1wA+A/cvW/CXwmfHlw8BXNnKmrfozcC/eCfz2+PL7O+/FeN31wBPAKWB6q+fewu+LfcBTwO+Mj9+41XNv4V4cB94/vrwf+NFWz71Be/GnwNuAZ65w+93Atxi9B+oO4PtDHnejn+H7sQyLVt2Lqnq8ql4aH55i9J6H7WjI9wXAx4FPAD/fzOE22ZC9eAA4VlUvAFTVc5s842YZshcFvH58+QYuf0/QtlBVT3D19zIdAr5QI6eANyR502qPu9HBX+ljGXZfaU1VXQJ+9bEM282QvVjqfkb/gm9Hq+5FktuAvVX1zc0cbAsM+b64Gbg5yXeTnEpyYNOm21xD9uJjwL1J5oETwAc2Z7RrzivtCTDsoxXWY2Ify7ANDP57JrkXmAbesaETbZ2r7kWSVzH61NX7NmugLTTk+2Ino9M6dzL6X99/JLm1qv5ng2fbbEP24gjw+ar6hyR/wuj9P7dW1f9t/HjXlDV1c6Of4fuxDIuG7AVJ3g18GDhYVb/YpNk222p7cT1wK/CdJD9idI5yZpu+cDv0Z+QbVfXLqvohcI7RPwDbzZC9uB94DKCqvge8htEHq3UzqCfLbXTw/ViGRavuxfg0xmcZxX67nqeFVfaiql6sql1VdWNV3cjo9YyDVbXmD426hg35Gfk6oxf0SbKL0Sme85s65eYYshc/Bt4FkOStjIK/sKlTXhtmgPeOf1vnDuDFqvrJanfa0FM6tXEfy/AbZ+BefBJ4HfDV8evWP66qg1s29AYZuBctDNyLk8CfJzkL/C/woap6fuum3hgD9+Ih4J+T/A2jUxj3bccniEm+zOgU3q7x6xUfBV4NUFWfYfT6xd3AHPAS8L5Bj7sN90qStALfaStJTRh8SWrC4EtSEwZfkpow+JLUhMGXpCYMviQ18f+GmWq6NWLIwgAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "plot_conv_layer(layer=layer_conv1, image=image2)" ] @@ -2214,22 +1821,11 @@ }, { "cell_type": "code", - "execution_count": 61, + "execution_count": null, "metadata": { "scrolled": false }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4VOW1P/A1uUOGkMsAISSwoRGppWA1Smop5fAg9rQc\nUIsQMVIOpYiCivxyBC0H0IdSLxStYlGk1B8FvFGliJfDoUApR9EGRQSNEHEMCIEMSQi5k2SfPyQ8\nnrPW2pN5XJnf48/v58/vznK/m5ksJ7Pf/b4+13UJAAC+mpj/1wMAAPj/AZopAIABNFMAAANopgAA\nBtBMAQAMoJkCABhAMwUAMIBmCgBgAM0UAMBAXCQ/HEhPd53sbJbXNieoNX5fnXygpEStafzO5Sz7\n/PMgVVWFfOFH+dUEUlNdJyuLHzh2TC/q0kXOc3L0mk8+YVGwvp5CTU2df42xsa4TH88PaNdBRHur\nB4h5//76cJOS5Pyjj/aGXNft4TlIA4G0NPG1bCD9Ors0VcsHzp5Va07E8de5ujpIdXVReL9mZLiO\n8D6rrNF/tZua5Lx3z1a15sSpWJZF6xqJiPz+gJuR4bC8VR8yxfIhExFRj/rP1JrajH4sKy8PUnV1\n+OuMqJk62dlUvGULy3cF+6o1I5LekQ8MG6bWHHqpmGXXX58XfoAGnKwsKl63jh+YP18vGjxYzpct\n02smTGBR3o4dYUZnw4mPp+K+wmumXQcR+V7eIOZLligdk4gGDZLzyy/36e9mQ05WFhU/+yzL99MQ\ntWZIcLN8YOdOteb+1OUse+qpKL1fc3KoePt2lm94I12tKS2V84VzatSa+x9NYVm0rpGIKCPDoV/9\niveFqiq9Ji1NzmcUz1BrdhWu4j8/o2PXiT/zAQAMoJkCABhAMwUAMBDRd6YNrQm0v5p/1zZiz0Nq\nTem8eWKee9FFas3ASd9jWdKRjzswQgOff060YAGLj/3nf6ol2YGAfGDfPv08c+bw7MMPw43ORvfu\nRGPHsnj/VP7dX7vee+R8ctwL+nkGXxvpyGxVVxO98gqLQ9/XvzOl0aPFeEPtOLUkJPzbtLSEHZ2J\nlvffp1BGBsvHeNRMvu8+MX+3dKFas3DkLpZt3lAbdnxWehx9l2bcyW8cPri4Qa2Z8cfvi/mRPcqb\nmYhGCHdN/VVHOzBCfDIFADCBZgoAYADNFADAAJopAIABNFMAAANopgAABiKaGtUlpomG+I/wA9fq\nU2DWVN8t5kvzlcf2iKh+/HiWtYUfno2MDKKbbmJxzOuv6zWZmXLu8QjiiE1zWfbx5/5wo7ORnU30\nwAMsHvLwr9WSkSN/JeZl+RPVmkJ5llH09OxJdPvtLB4VV6+WvP1BVzGfXKC/Ayev49PM3mw83IEB\nfnVx/fpRYKEwpcnjd3KfMJWKiMi/aJFa8+FBvotxY1yU3q9ERKmp4rS1ef/m0RluWC/GA7weDRem\nRdIuPi1Mgk+mAAAG0EwBAAygmQIAGEAzBQAwgGYKAGAgorv51NJCFArx3HHUkqW1d8gHilPVmq5/\n+QvLYubyu9+d4f2j6dTzLn43f/o9PGu39DfKItwnT6o1uwqOsyzvn8+FH6AFn4/a4vjuCDFnzqgl\nG/IfE/M125TXl8T1r4mI6O9/9x6emVOniB5/nOenT6sl/R95RMyzevO72e0KC19j2bF3o7NwcoUb\noN83TmP5zzzeSgPOyNeSMv82vehRvqBy0smorPH9haoqanvxRRbHVCs7IxDRmoKtYr6lRV+cp0hY\nOLu2sWNtEp9MAQAMoJkCABhAMwUAMIBmCgBgAM0UAMAAmikAgIHIpkbV1BBJeyG9/bZaMjdXXtBk\neeEhtWZrcCA/dcz94cdnoF8/ouXCVkjl5R5FdXVi3JycrJYk/Pa3PKysDDM6G8o2VzR7jr6XV1aS\nPLZpb72qnygovy/u9BydoZYWcXraTmm61Hl8B6AvHK/i+w9dMHUvi7a/1hhudCbSyvbSxFl8al7q\nLL1myo3y1KgNUz327JL2M0tMDDc8O4mJFNOX7z/X4rE327Tn5Pdsfn66WiNsAUXx8eGHR4RPpgAA\nJtBMAQAMoJkCABhAMwUAMIBmCgBgwOe6+gIO7Id9vgoiiuLqBv9DP9d1e3T2SXCNUfFNuE5co6Gv\nw3VG1EwBAECGP/MBAAygmQIAGEAzBQAwgGYKAGAgomfzA0lJruP38wNez5QPGSLGrT791LHNDSwL\nHj9OoaoqZX8QO4Fu3VwnEGB5+bkMtSZDOdTcrJ8nubmKZcGKCgrV1HT6Nfr9ATc93WF5T3+9WlMW\n6irmOTn6eaQdboiIysr2hqJxFzgjI+BmZzssP3VKr+nT/KmYN3m8x6V3chkRnXbdTn8tk5Pl19Lr\nvnJmY1A+0EN/SY5W8nUmamqC1NAQ6vRrJCJK9/ncbCFPiI1Va9zWVjH3ZWVFdO5gdTWF6urCXmdE\nzdTx+6l43Dh+YN06vWirvA9LTVJPtSQluJ9leTfeGHZ8FpxAgIrvu4/lD56YotbcfLOcHz2qn2fY\nZ3wfmrx77gk7Pgvp6Q4VFRWz/I7h76o1tz59mZj/7nf6eVavlvNZs3xRmeKSne3Q1q38Or3GvDQ4\nWcxLn31WrZHeyT8KNzgj6ekO3XUXv8ZzHntAzfuY7xlFREQzZ6o1c5+7kmXr10dnnysiomwi4jtt\nEWV366bWNCv7QyXccktE58576qkO/Rz+zAcAMIBmCgBgAM0UAMBAZItDZ2URLV7McylrN3++GKf8\n7GdqSc0Pf8qy1gSPxXktue4Xiwr/Lx5brZNwv4qIiA4c0GuedyeyrIr0xZkt9aw+RHdsGsUPhIar\nNSv/+Zh8YPpGtea2ZcvEfJbHwsWW4j8rpV6/5N/xx10qL1hORLS5YIOYj8uWbn+cN3gwi2IXLQo/\nQANduoinp2PKy0VERL+UXxd66y215Fph3ejXpC8xO0lCSgpl/+AHLH97kT6IYe+vkg+kpuonkn6Z\n168PNzwiwidTAAATaKYAAAbQTAEADKCZAgAYQDMFADCAZgoAYCCyqVFnzxLt3MnzkhK95uWXxXhi\n7Rq15IXpvVkWqz3obS0jg6iwkMUPDeeP07XbvvsdMR9D8qO0RET0W75x/cPlH4Ufn4XkZKL8fJ6/\n8opeM326GBePH6+W5F1zTaQjM9V25gzVC9c0/xX9MWth2/QveMyNmziT78N+5Mxj4YZnIqXmGI3Z\ndjc/UFSkFxXvE+MNZ/iUxHaT/zqXZf4qj+elrWVmitfk9QT2unUzxLy0VK+Rlh6pj9UfWf0yfDIF\nADCAZgoAYADNFADAAJopAIABNFMAAAOR3c1PSyOaMIHFE6fKq7ATEb3Q+Iicz3lTP8/gW3nWwQVa\nv7KqKqIXX2Tx8gL5jj0RUZIymWFUoXDH/Ly2PcJ/78roLLZ7IqYP3Z+0lOUL79PHS7W1YnxJnb6k\n+6nkqCzCrorJyaGu8+ax/PmAvtLKDQXymO9dxu/Yt5PWQImPDz8+C20nT1L9ww+zvKvX4kPSjBwi\nmjzaox0Iv/dRXemktlZciCU7W1iw57ysku1yPnKkfh5hgeyuFR1byxyfTAEADKCZAgAYQDMFADCA\nZgoAYADNFADAAJopAICBiKZGHS+PoYUP8GlQL5QM0Yt++Usxrhx0lVoyeAI/Fgrp+/aYSkoi+va3\nWbz7z3rJS4v3i/m9D+j/LmPH8kyZfWTO7ycaLm73JKzy0E7ZN6dr+RG1pPA6ZdrUy9GZMnUmsSe9\n1p9Pg/perl6T3Vse8/FBa9Wax6qnsCwxMfz4LJwZcDm9srSY5ZO8pqX9WXkzb9qk1zzzDM/q6rwH\nZ+nECaIlS1i8dpm+n9OhbHkK3MDR+nSqhcP5dKrj8e92YID4ZAoAYALNFADAAJopAIABNFMAAANo\npgAABnyuqy9UwX7Y56sgoo499W+vn+u6PTr7JLjGqPgmXCeu0dDX4TojaqYAACDDn/kAAAbQTAEA\nDKCZAgAYQDMFADAQ0bP5yckBNzXVYXnvLtV6UVJSZDkRHREe966rC1JTU6jTH+oOZGS4Tt++/EB9\nvV4UJ/8z1rV1UUuS606xLFhZSaHa2s6/xvR015H22vjoI71o6FA5P3xYLWmpqRHz94lC0bgLnJwc\ncNPTHZb3SmlQaz4tl18zr3UTevbkWWVlkOrqovB+TUpyHT9fU6Hp9Gm1JvHii8W89eOPIzp3GRGd\ndt2oLLQQiI93HWHBg2aP9QESsrLkAydO6CcaMIBFwVOnKFRTE/Y6I2qmqakOzZrFF1W4d7DHIiSD\nBsl5rr7axMQC/oF527bo7I/k9O1LxX/7Gz9QzK/7gkBAjN9u0Bc6GVb8BMvyHnww7PgsONnZVLxl\nCz+Qp/8bt70jX3/M2J+oNZWvvy7mGVGa4pKe7tBdd/Fxzx0tL0xDRDRZWZxmzx79PLNn8+yRR6L0\nfvX7qXjcOJaX/vGPak3umjViXvODH6g1bUL2T2FHZ8dJTKTi736X5WUeL0zfW26RDwgLplywfDmL\n8ubODTs+IvyZDwBgAs0UAMAAmikAgIGIvjPt7R6nexsXsvzD3Pv1ohY5nn+tXrJ5MV+MNe+Axw0g\nQ5+WxdKU2SksX+vs1IuU71O/u1HfV/yqu/jCtSWV+vdcppqbiY4d47nH92wtsfL37wm3366f57Ty\ndF1GdBaH7tKFaPBg4YD2PT4RbShXFg7OVt7IRESXLub/na5nvQdnRXktc2+8Ua/JzxfjFI9/l/0l\nJfzU4UdnZ+BAor/+lcUHdvLF6tstUda6HruR97ALpxnP35uN4UdHRPhkCgBgAs0UAMAAmikAgAE0\nUwAAA2imAAAG0EwBAAxENDWK4uLERydTvqNPdcn+138V8+de8ZgGdIvwqOO5c2GHZ6F/bBmt9d/G\nDwz3mMvVKE+e2LZNLxk/nmdlZWEGZ6Qp3k9HMq9i+YqNes3yK64Q8w35j6k1k+fPiHhsllJSiMaM\nFh6EnDpdrSndsUPMc9etU2tq8vh0qtau3cIP0EKfPuLjkVNWXKmWrN30knzghhvUmnWNfPrj6bXR\neWSWiKj8VAw9+DifBvX883pNYaGcP/CAXhPszafzhUIdu058MgUAMIBmCgBgAM0UAMAAmikAgAE0\nUwAAAxHdzW9M6UmHfnwHywcmzVNrWpTFM7reeqt+oj/8gWehUNjxmdAWARFXzPjC81VjxHzvbv00\nwSDPvBbzt5RYU0EDtq1i+fL8VLVmQ947Yj5ypH6eVbX8HERE9PTTXsOzc/QoUVERi/cXrVVLhgwb\nJuZv9r9JrblqAf+diD1+tAMD/OrOtCTTayF+537CBL3GN16+079+/fVqzUOl/G7+9liPFeuNxccT\n9e7Nc69FuxPy5IW+586Zoxel8t+BvLs9dhL5EnwyBQAwgGYKAGAAzRQAwACaKQCAATRTAAADaKYA\nAAYimhp18KBLF1/MFxxx607rJ6itFfNp83uqNUXCttaNE6OzqEJTTi4deXQzy5Nj9ZpJ798r5q+W\nL1VrpClFb78dbnRG2tqIpNeloEAtmXzN9+UDgfvUmhl3Cqu5EJGym7m9Hj2IpvNFTcqFmW/thtx8\ns5jn+/WaMYv5Yi+HGt4MOzwL3c+F6Cfla/iBhx9Wa9wbvycf2Jetn2joUJ4lJoYZnZ2Mo/toyp1p\nLC//uT5tSe48RKm/+IVaE/jd73jY3LHdrvDJFADAAJopAIABNFMAAANopgAABtBMAQAM+FyXL9Ov\n/rDPV0FEn3XecDz1c123R2efBNcYFd+E68Q1Gvo6XGdEzRQAAGT4Mx8AwACaKQCAATRTAAADaKYA\nAAbQTAEADES00EkgLs514uP5AWHflAuqlYUI0viiBRe0tbEoeOYMhRoafGGG+JWlpQXcrCyH5V1O\ne6yOcY4v/kJEdDSuv1oi/TNWVQWpri7U6deYmhpwMzMdlpeX6zVxyjvFaw2Ii87uFfN3iULRmFIT\nFxdwExIcll90kV7zySdyPqhXlV4kvMeDdXUUamzs9NeyW7eAm5HhsDxw6kO1pqH/JWKekKCfJ/bT\nUpYF6+sp1Nzc6ddIRBRITnYdqc/4PE4fK69OdDZJf+t1a+aLNgVDIQqdPRv2OiNqpk58PBU7Dj9w\n3XV60csvy/kNN+g1jY0sylurb4JmKSvLoWefLWb5kHV360XSBnxEdEdgg1rSpw/PHn88OitjZWY6\ntGoVv8Zly/SaQEDOpY0B272xQ37/JUZpvmBCgkODBvHr3LJFr9E2ontzzgt6kfAfzHv11XDDM5GR\n4dDChfwapz2urAxFRPv/L/95IiLpV7tdSuE4luXt2hV2fFac1FQqvkVYbywpSS9SPuRtz52hlow6\nxvtM3qJFYcdHhD/zAQBMoJkCABhAMwUAMBDRd6bU1CR/Qz96tF6zYIGca19OEdHx1a+x7Nx/bA83\nOhNdfI00JOkQP+B1jXv2iPG1w/WS3bt5Fq0ne/3JLo3I53eOnvTrdyDWDBdWcyeiQ/OnqTUJ1crW\nAcOGeQ/QyCX96qj4yXdYPrnoSrXmzaKXxHx76kS1prGAHzvzVnS+/w4cfY+mzUnhBwYNUmucofJ3\n2Sked+aqDx9mWWv44Zk52tKb5lYvZPny6fqNNlqxQoxHNfKdES4oLOTZ8uXhhkdE+GQKAGACzRQA\nwACaKQCAATRTAAADaKYAAAbQTAEADEQ0Nepkn8to+V38UbS4A3rNVGWGSMqmTWpNVm5flsV7PThu\nqLYliXaVD2T5gVKetZu5YIyYj5qpP7ZWmreKZdrz7+YqKoiefprF+fmz1JIpO+UpUGt7eTw2+fzz\nEQ/NUmNJCR0SpmHxByO/JPVWMR5V+qRe89xzLFqY3BJmdEYGDCB69FEWN44dq5akPPWUfKCiQq1J\nEKY4RuWh/PNaW5VlPm66Sa159w/vifllyybrJ9q4kWfagg3/Cz6ZAgAYQDMFADCAZgoAYADNFADA\nAJopAICBiO4f9wq00tzpNSwfmCcstHDe6tVyPnKkvqjGY9u28fD668OOz0JDA9EBYXbCbSV36EV7\nCsT4yHx+x77djJl8BsCqKmGBlU5w0u1Jy5v4nfudO/WazVPlBUCOf09/XbLq6uQDf/qT1/DMJGVl\n0UBhQeFsr8V+d+wQ40MlJXpNKJ1FjS3RmZpx6GR3GvXbn7J8+7p1as3mbvId8NlL9PM4P/wVyw6/\nF53FXIiIep7eS7P/yOcPLP21vjrQvSeUmSbKYu5ERHTNNTzD3XwAgOhBMwUAMIBmCgBgAM0UAMAA\nmikAgAE0UwAAAxHN3zhbH0vbi/k0qEOp+p46xzfxPXiI9ClTRETjiviiIqXHPPbHNtSjoYxuO3Ab\ny+cm/V6tWf7pejEf8FM+ZeWC22/nmTQnqxP08p2iuYlPsHzukh+qNc9/JE+BmlSkLxrx4YINypFJ\nnuOzcuB0bxq4ju8bNOEenrXT9o6fMS9NrWnL5ZnXdu6WBvZtpO1PClPqSrqpNUVFcv6Pf+jnueIK\nnjU1hRmcoQQiyhZyacumC/zfF+NdS3apJSN+3p+HoZDn2NrhkykAgAE0UwAAA2imAAAG0EwBAAyg\nmQIAGPC5rr5QAPthn6+CiD7rvOF46ue6bo/OPgmuMSq+CdeJazT0dbjOiJopAADI8Gc+AIABNFMA\nAANopgAABtBMAQAMRPRsfqB7d9fJzOQH/H61praObzVAROT//GP9RD34jbNgRQWFzp6V/2OGAgkJ\nriM8WF3Vg68X0C6NquQDiYlqTUlZV5Y1NQXp3LlQ519jly6u0707P9DQoBfFKP/fld4P551LSBbz\n/fv3hqJxFzg5OeCmpjos792rTS/SHjgvLdVrUvh6FcGzZynU2Nj5r2VcnOsk8C2AKjMvUWvSGz6X\nD1Qp72MiahP+XcqIKOS6nX6NRESB2FjXiY1ledvgIWqN9kh9z/QWteZEBW+J1dVBqqsL/3sZUTN1\nMjOpeOVKfmD4cLVm1x55r6cRC0boJxL27cn7938POz4LTlISFefns/z5X2xVayb5XpAP5AorYJx3\n1ezLWLZ/f3T21HG6d6fiKVP4Aa+FVrSVO+bPV0uOZ8sL4PTp44vKFJfUVIdmzSpm+b1z6vUirWmO\nH6/XXH01i/Jefjnc8Ew4CQlULLzPNszn191u8oF75QMbN6o1tYcPs8zjN9icExtLxcL/uOt369ep\nLaZ0R2GlWrP0Sb6f1xNPdOz3En/mAwAYQDMFADCAZgoAYCCi70ybE7tRWe4olvedqi8Q7C+SFwie\nmKkv0Dpa2G69ovWR8AM0UJM5kLYW8e9HJ8Vt14ve4t8nERHRG2+oJQUFa1jmtZ23qa5diYYOZXHz\nkofUkoTZM+QDzz2n1mx09EXDo6FnT6LZs3l+6//hN//aXXSRfEPDf8+nas2M/P08/K//Cjs+E337\nEq1YweLJ8+SFkYmIRnV5S8ynL16q1owdyzP3R9H5jp+IqHbAEHpzDf9+9KpL9RvDl64WFs0mog/L\n+fei7aTbBo2N4cdHhE+mAAAm0EwBAAygmQIAGEAzBQAwgGYKAGAAzRQAwEBEU6NOniRatoznY6dq\n+6MTjUk9IuYvDHpGrSn78f0sE2Z/dIqUri005tJTLK/38ylh7XY2ysdGLNAf572jlT/rtnZtBwZo\noDomnV7qchPLry+4Xi967z0xnnudPmXIiXRgxqqriTZt4vnKHgvVmprp/L1HRDR6tH6eGdcK6xPE\nx4cbno2KCvm5ybvuUktGlsj5oEH6aVIOvMmy2IbacKMz468so6vW3cbyQ1vk6U9ERCMe5T9PREQt\n+rP5G47xf5y8Zo91RL4En0wBAAygmQIAGEAzBQAwgGYKAGAAzRQAwEBEd/NzcogefZTnMatXqTXb\n4+QFMkoy5bumRES3HeN3DhOao3Tn8OBBcRGQ1fecUEu2bZPznwiLTLfbLawaHq17oymf7KUf/0yY\naXDypF40c6YYT5qklwzzvSPmd3oNzlBKCtE11/B8yjb9vTdcWbfl8ss9TrRuHc9On/YenBWfjyhO\n+DX2uDW/cIm8mMuqTGHBlvMuSxVW4Tl3LuzwzAQCRNOns3jxYr1kw+I58oESZToDEVF2Ns8++cR7\nbOfhkykAgAE0UwAAA2imAAAG0EwBAAygmQIAGEAzBQAwENHUKDpzhmJef5XnK1eqJc6f5alRo3bq\ni01MW82nrgRD/vDjszB0KNE7fEpP7QN6SWqqcqAlTa2J3+OyzDc1OnvqxHznO9T1hRf4gd279SJl\nmtewX4/Ta44ejXBktuLdZurVVMby1NS+as2Mil/LBz75m36iq4VpY4mJ4YZn48wZotdf5/m3vqWW\nbJgvT4EqvElfmOdkOX+/nvPre4aZi4khSkpisbTGywXKnmr379Pfswtrhd+B1tYwg/sCPpkCABhA\nMwUAMIBmCgBgAM0UAMAAmikAgAGf6/K7dOoP+3wVRPRZ5w3HUz/XdXt09klwjVHxTbhOXKOhr8N1\nRtRMAQBAhj/zAQAMoJkCABhAMwUAMIBmCgBgIKJn87t3D7i9ejksP35cr8nIkPM0/bF1ijtyiGXB\nxkYKNTfrDw8b6dYt4AYCDsvPntVrUlLkvNZjHxKp5tSpINXUhDr9GgOxsa4jbJvi+aLEyP/fdcvL\n1RJfD/kG6N6KilA07gIHMjJcJyeH5TX1+ts+5fOPxPxc7rfVmvhKvt1LsLKSQnV1nf5apqUF3D59\nHJa3tOg12j3nbj6PN2xVFYuCNTUUamjo9GskIsrICLg5OQ7LpR1bLqipkfO6OrXko+reLGtqClJL\nS/jfy4iaaa9eDq1YUcxyr31YCgvlvKBAr0mfMIplecX8vJ0hEHDovvv4uXbu1GtGj5bzv/9dr7n6\nap7dfXd0FjpxYmOpODOTH5gwQS8SFpkgImr8zW/0EuW/51u5MipTXJycHCreupXlW/f1VGvGLLhS\nzE++Iu9nRUTUa/1yluU98kgHRvjV9enj0Esv8ferx//j1EY7Km6XXrRpE4vy1q8PNzwzOTkObd/O\nr1NdZIiIYrbx156IiPbsUWuu3MIXYDp4sGO/l/gzHwDAAJopAIABNFMAAAMRfWeacu40jSlfy/Jt\nw6eoNdp3jbcVVOonCgR45vlNs5304F4q+Dn/rvmtmfqTYpOTXhLzgieuV2tiSj5k2dKkxg6M0EBa\nmvz96L59asn9I7eL+SK6Tq1x8+WbOV6LiZsKhYieeYbFYzy+UDy+Sf5u1N9FP82uvLksq03eEHZ4\nFpKSiAbmtrF84AH+HecF0vflREQbPWqku6lt/LydJa7yFKU/+wQ/cPCgWrM2//diXtI4Rq0JhXjm\ndTPvy/DJFADAAJopAIABNFMAAANopgAABtBMAQAMoJkCABiIaL7RB8czaMBiPg3qwAG9Zvp05cCy\nZWpNy4svsixaS1jXEpG0e/zKvfJjhkRElU/+Q8zTu3XTT7RiBc88nhk2FR8vTo9p2bFDLVmYO0PO\nm4TrOG/y1CuUIz/3HJ4ZbQpYdbVakhVoFvOaxES15iohSw43NisffED0rW/x/PRpvaa0VM5nz1ZL\nKlMHsKzlnXfDjc5OczPRZ8JTyB5TJqc89xMxXzPhNbXmyLV8mlve+qPhx0f4ZAoAYALNFADAAJop\nAIABNFMAAANopgAABiK6m3/uHNGxYzwfPFivOVKsLGiyZZA+qPx8lvk++CDc8Ez4Lr6cktYIC1HX\nKgvNElG6sJgGERF5LWgtLarstaKvJZ9PvAsa9y//otdIi8945US0Stme4FnPwdnZ/3Ei9R3J70KX\nbdmvF82ZI8YTrtbnk0glNXdGZ6Fv6t+faNUqnv/oR3qNsjL7wGPyYjZERIdW8Pd/XL2ykn0nqE3N\npl1jH2K5x9uPLsmXt8CYNvhuvWjkSJ5t3hxmdF/AJ1MAAANopgAABtBMAQAMoJkCABhAMwUAMIBm\nCgBgIKKpUdnZREVFPFcXMyEiWrBEzjMy1JJdD77FstoZ0ZlqEhenTLfYo6/mMi1J3u9nzezH9BNJ\n+y1VVIQZnY3qxF60OZcv6DDOYzEPuvlmOfeYm+L/05/kAx57TVm6+GKiv/xFOHAgqNacXCTvG7T1\nP/jeZxe3aZYMAAAAwElEQVR8yqeALWw6FWZ0Rg4fJho7lucei+yUKAvaHNrrsXBJUjbPEhLCjc6M\n/8h+GlHYlx9QpnkREdEbb8j5e+/pNdIiOK2t3oM7D59MAQAMoJkCABhAMwUAMIBmCgBgAM0UAMCA\nz3U7viGIz+erICJh74Co6Oe6bo/OPgmuMSq+CdeJazT0dbjOiJopAADI8Gc+AIABNFMAAANopgAA\nBtBMAQAMoJkCABhAMwUAMIBmCgBgAM0UAMAAmikAgIH/BkJomYa+y8y2AAAAAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "plot_conv_weights(weights=weights_conv2, input_channel=0)" ] @@ -2243,20 +1839,9 @@ }, { "cell_type": "code", - "execution_count": 62, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X18lOWZL/Df5J2QhAADJLzlMaAiokVNka0eZDlI68si\nh7KVKmWR4ssq63rUCouUKloKilQpsh6Wg9RSl7aRIgcrRcoqpQqcwaK8FBBheAsJmYQQkpCEwLN/\nSPz09LquZ5hyZ/bj8ff98/fk4rmfzMzFZOZ+7jvk+z6IiOjipPxXD4CI6P8HbKZERA6wmRIROcBm\nSkTkAJspEZEDbKZERA6wmRIROcBmSkTkAJspEZEDaYn8cG5u2O/c2RN5uLnMrPGPHVPz5gHXmTWZ\n506LLFpWhtiJE6H4o7w44Q4dfK9bN3mgqsouyspS408bCs2SPu3LRRY9cQKx+vq2v8Zw2PeKikR+\nIGqfOj9fzzs2yev4XKdOarx1+/aY7/tdAgfpQLhzZ9/r3VvkpxpSzZq6Oj0vPL3frGk5cUJkhwFU\n+X5yHkvPE/nZs3aNMlwAQEuLXVMQlgejhw8jVlXV5tcIAOFOnXyvRw954MwZs6Y+lKPmQS9l7Slb\nXh5FTU0s7nUm1Ew7d/YwY0ZE5BOjM8yaxmeeUfOyN+W/06q47mORlXz72xcwwovndeuGyIIF8sDS\npXbRlVeq8eitT5olKwY/J7KS+fPjDc8Jr6gIkQ8+EPn4SRlmzW236fmdB+V1fG7sWDUOFRUdDByg\nI17v3oi8957I10fyzJqNG/V8xo5vmTXVv/qVyIbFH54Tnudhyxb5WrL+UwCA0lI9r6y0a6bcWy2y\nkmHJukrA69EDkTfflAeOHDFrNmcOUfOgl/J3viOziRNL4ozuM/wzn4jIATZTIiIH2EyJiBxI6DPT\ncMVOTHz+CpE37t5t1mQZn/YWF9ifz5VFm0V2Jq3dBYzQgQMHgHHjRHwk4AMl/WsW4JH37M9Mb/nR\nEyL75NQv4w7PiYYGYNs2Ebe0DDJLfvc7Pb/zxgL7PMo5kmrPHuCmm0Q87O/+zix5FzPV/Np99mOz\nbKfMWr51YZ+zXayaGmD1apmPHHzcrJkIpQAADtrfY2DbGJmdll8Ut5mWFiAWE/HXpuqfiwLA+y9u\nUfPrXxponyccFlFOfX388YHvTImInGAzJSJygM2UiMgBNlMiIgfYTImIHGAzJSJyIKGpUWjXDrjq\nKhFnvfGGWbKrXJ841P+BB8ya7pFVIkuvr7mAATqQmqreiN5z9myzZP93v6vmQ26yb+cdni53hQ26\nN9qp9HSgQE5pev0RfSoJAFT31adNNeeMN2v++Z8TH5pTZ89+NnfoLwVM2Zr5on4P/vTpxWZNNJro\nwNzJbyzHyN3KLb2lO+yiCRPUuNZ4HgNAaclCkcWacuMNz51QCEiT7SroOTZzjf6cnXFkhV20cqXM\n/vEf440OAN+ZEhE5wWZKROQAmykRkQNspkREDrCZEhE5kNi3+SkpQI5cvXri3P5myZKhr+kHygNW\naNdWbw1aHtuh+ksGYPNSueDD9QfthS6K771XzXf927+ZNaNGyWzduvjjc6KsDHjqKRGPrllilqw4\nYiyCoqzy3uqfntJ/Z6+8EjQ4h7p00b+JNRbzBoCP6/Rv7b+Suc+seeutviJraoo/PCcqKoC5c0V8\nrtxe6CQlqs9YyLv5ZrNm4urRIlt48tMLGKAjlZXqE6d+8CKz5PHHjQOlAStna7OMrK0J/gLfmRIR\nOcBmSkTkAJspEZEDbKZERA6wmRIROcBmSkTkQEJTo1p6eqieK6fPLNm23i5arS8q8fooe6rRgAEy\nO70nOXvq7N0LaDNE3nnH3jf9+q/o+0P1/+gjs+ZFudUMbrkl7vDcyM4GBsp9cFb0DFgAolRO/wGA\nFWNeN0s2Lk54ZE7V5xZg801yr63cwfYCNFe/9JKa+7esMWtCt/1YSRvjjs+J4mLgX/9VxCkFXe2a\n0lI9D1o1pLBQZgF7vzlnLM4zRtmaqlX2aqPHKM/9VgufklPKKudcWO/hO1MiIgfYTImIHGAzJSJy\ngM2UiMgBNlMiIgdCvi+3zzB/OBSqBHCw7YYTqMj3/S5tfRJeY1J8Ga6T1+jQF+E6E2qmRESk45/5\nREQOsJkSETnAZkpE5ACbKRGRAwndmx/OzfW9sLypvDa9s1mTl6nv31BzOtOs0XYJqKuLoqkpZt9U\n7Ug4P9/3lHuAj56U27W0Sk3V80z7EtEx54zIoocPI1Zd/V92jYHbM3QxvsysC9gCQnmuAMDWrVtj\nyfgWOBwO+717eyJvDLht3nrMUmsCts3JyBBRtLwcsZMn2/6x7NzZ93r1kgd27jRraoq+ouZpAd2g\nrExmjY1RNDe3/WsSADp0CPtdu3oy//RDu+jSS/W8ocGuUV7M0aoqxE6dinudCTVTLxxG5OmnRb62\nYLxZM6Kvvt/Mqh36XjsAsHy5zH772+QsdOIVFCCySO4rM23NELOmQwfj3/Ls89z53+SzsyRJK51Y\n14iVK+2iSZP0fNMmu2bCBDUOpaYmZYpL794eNm6U+3kFrc/RV1/PBXkrjb3MAKBnTxGVaHtPtQGv\nVy9E1q6VB7TVgs5bNU/+TgDz/z4AwPTpMotEkvOaBICuXT289JIc963fbGcXKQvAAAC26YsvAQDy\n80VUMnNmvOEB4J/5REROsJkSETnAZkpE5EBCn5mitlbd3H1E+TK7RtmfHQBGzp1glgxYukFk27fH\nG5wjxp7yswI2ex//7GVqPuWKVfZ51sRkdvJkvNG5UVEBvPiiiD9+yl4cep2xNvLSpf3NmtvtreaT\nIqW5EdlH9oq833WXmzXZ772nH9A+NGylfQbXLuCzPJeqqoBl8vU3+ka5yHGrBcZHnX362Kc5veeQ\nyEpub447PFfS9m1F+Db5HVDAJ/YY/MEH+oGgz7OPHJFZ+/bBgzuP70yJiBxgMyUicoDNlIjIATZT\nIiIH2EyJiBxgMyUiciCxqVGFhcDUqSKesdyeHnObcd96x8Vy+lOry2JbRJbZUh9/fC5ccok61eSK\n/97dLPne9/R8c7eRZs1Vw2V2bsHCuMNzwpjiVv6AXfLo8I/VfMRj+n3eADBgij5l7kfBo3OmOSUL\nh7LktLXeWVl2UUyZsgbgF8/LqUGt7hwzTIb79duoXTvQ0A13RR4V+YoJ9rS8/Y368/L01l1mTWPR\nlSJL5rLyGVdfh6K18nZSa8kIAJg1W8+nTX/QLtKmwAUtWvBn+M6UiMgBNlMiIgfYTImIHGAzJSJy\ngM2UiMiBxL7NT09XF8Kd+Iy9CHX2M3r+H8vt7wLDNw8SWUvmhS02cLFiJ9OxZI385v6HP7RrNm7U\n86FD7ZrsZXJx5pSqyjijc6SwEHj4YRGPqLMXOqnuOVrNe58M+E538byEh+ZSRupZ9M6vlQcGDrSL\n3n1Xje+cbC+2jMWLZXbHHcGDc+SSk9vw+tsd5YFvvGTWZPQxXq8BC4BUH5WPc8styVscOr3sILp9\n/z55IGABmkikt37gmzeYNe36yNd+U1N63PEBfGdKROQEmykRkQNspkREDrCZEhE5wGZKROQAmykR\nkQOJTY06dUqdOuIFlKx6U586E/2TXVNXJ7Nz5wJH5kxeHjBcWYRE2U77cyXGDJHed/6NXVRTI7Pj\n9r49LtXlFGDD4CdEPuTwz80aa/qX59nn6fuAXIADAPDYYwGjc+fMuVRUnM4T+c31xt5AANbIdXwA\nAN2j75s1oRu+qqQZ8YbnhucB85QpaMoeX63OHTSmsy2Qz4lW3X/9ssjSTyTn+QoAaG7W92cKmBq1\nYmBf/cBRe6GbF16Q2Zw58Qb3Gb4zJSJygM2UiMgBNlMiIgfYTImIHGAzJSJyIOT7F775QCgUqgRw\nsO2GE6jI9/2ATQrc4DUmxZfhOnmNDn0RrjOhZkpERDr+mU9E5ACbKRGRA2ymREQOsJkSETmQ0L35\nKSlhPy3NE/mZM6fNmusG6vco/2lvqllzRd8zIosePoxYdbW9P4oj4XDY95Qbzlu2bjVr0nr1UvPa\nrK5mTV7NIZFFT51C7PTptr/G9u19T1tsoKzMLuout3MAgKPnCs2SHllVar41Go0l41vgcH6+7xXK\n8R2vt7fAOW08lYsyy+0TheRDFq2uRqy+vs0fy/z8sF9Q4Ik8p/KAWVPb+RI1r6+3z1PYKP+9aH09\nYo2NbX6NABBOT/e9zEx5IMV+P3i2z2Vq3rLNfi1nFhWJLFpVhdipU3GvM6FmmpbmIRyOiPzYsY/M\nmsh7+gM3aLhcgKLVlpXyRV1yyy0XMMKL53ketmyR11idav8uw1OmqPnaSx8ya0asfFBkJaWlFzDC\ni+fl5yPyoDz/uYBFI1KUnweAafVPmjWz+r2m5qF/+IekTHHxCgsR+elPRT5/k9xjrNWOHXq+qO9z\n9onS5Muo5Mc/jjs+FwoKPCxaJJ+vQ165y6xZO+F1Nd+0yT7PjH3jRVby1lvxB+iIl5mJyFVXyQO5\nuWZNbelaNY91sF/LxTNmiKxk5sz4AwT/zCcicoLNlIjIATZTIiIHEvrM9OrUnYh0uELkj461V3rO\n66nnN9hbV2PWUvllx7HYhe1dfdFqa5GyTn7WEj4Y8DHf9u1q/Otf2yUj9uyWYWNjvNG50dQEfPqp\niFua7LvhMsaOVvNZA+RnTJ9b/G6iI3PrwAFg3DgRP2ytdA3oK5MDWLXDXjh55A+ukWGV/uWbazn+\nKQxpWS8PfOMbZs2IZ4eoef/f/94+0XvvyWzLlnjDcyczE7j0Upk/+6xZEo3q+YCg84waJbP584Mq\nPsd3pkREDrCZEhE5wGZKROQAmykRkQNspkREDrCZEhE5kNDUKIRCQLqcojRvsX1rqPdsrZoPHWqf\nRrutTbljr00cPpmHh1ePEPn8bfbthBN369Nm5s61z7O/Rk5nabqjJP4AXejZUx1cxkZlis15r41a\noebjV3/LPk9BQcJDc6n+kgHYvFTeann9MmWf+fO2PfaYmo/8+783a1Zu2yaymgsYnwsNqbn4MH+Y\nyI/oM7wAACMXDFTznosX20XLlsksSdO/AKC+6yXY/JC8PbnKuP0XALKy9HzM/7CnAM5VHrimsxfW\nfPjOlIjIATZTIiIH2EyJiBxgMyUicoDNlIjIgcS+Iy8oALSFkLUFCM57eLr8ZhwAdg3XF24FgPs8\neWxRpj4rwLVehS2YP/24PLDM/lUteaVZzc+l6bsMAECnjatElnk6Sd8B79wJ9Osn83XrzJLx66bp\nedYvzZrHHzcO/Copi7OjfcV+XP+CMttA2Umh1cCTJ/UD1lfDAEaNHSuyZ5+wF0ZxqapK/6I9YJ1v\nrNp4tZqXD7AX9HjlFZntrv8w3vCcOXMGqKiQed++do21btCYMXaNts6JsiaQiu9MiYgcYDMlInKA\nzZSIyAE2UyIiB9hMiYgcYDMlInIgsalRJ08C2l7ZQauWLF2qxtsDtpvpv0dZ6aS+PnBozhw4AEyY\nIHNjbyAA6PTso2q+fLl9mt3RkSI73nxh+3NftE6dgLvvTqzG2LTrlb+1S4KuPxnO9CpGxU/k1K2A\nmXyYYEynefFFu2b2brk/1rHTs+INz4m0NCAclvluZYuxVitX6vmSqXvNmvuWPyCykjT7513L+nQr\n+t4hp9T137PHrGm57nI1v/aNN8yau848KbISPxp/gOA7UyIiJ9hMiYgcYDMlInKAzZSIyAE2UyIi\nB0K+by/hL344FKoEcLDthhOoyPf9Lm19El5jUnwZrpPX6NAX4ToTaqZERKTjn/lERA6wmRIROcBm\nSkTkAJspEZEDbKZERA4ktNBJfijkFyp5+3btzJrG06fV/NwV15k12U0nRBatrESstrbNNw8Kt2vn\ne3l5Ij8d7mXWHDqk56GA0RYVyezo0Siqq2Ntf40dO/pejx4i37oz3azp0ydVzfMrP7FPlJurxluP\nHo0lY0pNWlrYz8jwRJ6TY9dY69n0zzEeZADbT/YWWUtLFGfPtv1j2aFD2O/a1ZN5yim7qKVFz5ua\nzJK9tQUia2yMorm57a8RAMK5ub6nregSsADSqY7ycQGAVP2pDAA4d05m5eVR1NTEv86EmmkhgFeV\nfHDArla7tm9X88ZlEbPm2n1ypZ+Sf/mXeMNzwsvLQ+TOO0X+8SR7s7HJk/U8YA82dYOyO+4oiTc8\nJ7wePRBZsULkocvlC6bVvHnyPxgAGPnKrfaJjNXEQlOmJGW+YEaGh7595fPMWAALALB5s55HBj9o\n1hSvWSiyo0eT81h27erhpZfkNd6atd4uisX0PBo1S4atkRsERiLJuUYA8MJhRJ5+Wh7YpKwwd976\nMfJxAYD8fPs82iZ8Eyde2HXyz3wiIgfYTImIHGAzJSJyIKHPTNunpGBw+/bywJgxZk3/G2/UD2Tt\nMmtWZX1LZDWh5+KOz4Xq9r3w+mD5+ejq2XbNhoEPq/n6UfbnrMWl8noyT5THH6AD1Q1ZeD1ymciv\nucauGZm/Qc0b3n7brMk2dlnAlClBw3Omf3EjIqXKavDWh9wAQn9cpuYNG/XP3wBg/6QPRVYyriH+\nAB3o0HQctx54WR746lftoqee0vMFC8ySdY/LbNCg4LE5deYMUC5fH++Psx+X/2vs5nH0qH2a+btH\niCyn7MJ2FOA7UyIiB9hMiYgcYDMlInKAzZSIyAE2UyIiB9hMiYgcSGhq1Mep16B7jrx1rWzcfrvI\nmG5xKKe/WTJygZyeMPP4hU1PuFidju3EXc9cIfK7ugTcSj50nBoPK7VvQcT06TL7+c/jDc+JpiZg\n3z6Zf5hmz3VpKNmi5nOftndqmDEh4FbTJGjZuROxyy8XuXEzJQBgz56uar5tm13ztRblhn7tJu+2\ncvaszILuZTZu88XvjblEAFLuuEOGZWXB43Lok1MFuOU/5C2thbvtmiWzj6t5Q47+GANAxam1Ijsz\ngreTEhElDZspEZEDbKZERA6wmRIROcBmSkTkQELf5l/dsxqRZ+Q3zpsr7zZrrjcWleg9V18cBACW\njJXfqMX2JmchWr//lWj+QM5YyIgGzCawFohQFmZoNXNxd5Edi9kr3bvUqRMwTpmAUD1Z/8YeADrN\nnanmY8bMsE9kr9ubFE0AtHkmg5bpi5kAACbLmSQAkPbOO2bJrp1yRkNjWsBy/g6dC3dFwyT5Wpo6\n1a558smr1bzbQfvx/8XsAyI7MS15i0NfGj6Bt++Ri8bjBz+wi9boC8pn33+/XfJzuTNIwGL+/w++\nMyUicoDNlIjIATZTIiIH2EyJiBxgMyUicoDNlIjIgYSmRuHkSeCtt0R8fc57ZknFM4vUvFvAtKE1\na2RWWxt/eC6EKsqR8aKy31RpqVlTtlKfUtL9gZFmzYypzSJb9aa9aIhLmbs/QvENhSLfvPKYWXP9\nkSNq3r+g2qyZUfIb/cDboeABOlLV5Tq8+k05zW1QxTy76Pbb1bjY2s8KQK0yCyo9ObPccPgw8Mgj\nMl/0gNyXqtWKP1yr5uXl9kI3MWV1GG2P+TZz7Bjwox/JXFsw6Lw5R/QpmxOi482a0S9NE9ms+oBN\no/4M35kSETnAZkpE5ACbKRGRA2ymREQOsJkSETkQ8v0L/wY5FApVAjjYdsMJVOT7fsDeIW7wGpPi\ny3CdvEaHvgjXmVAzJSIiHf/MJyJygM2UiMgBNlMiIgfYTImIHEjo3vxwKOT3VvKUoiKzJloXVnPv\nzCf2iVpa5L/T1IRYS0ub39Qd7tjR97rLLUWwN2Dbkt7abwVAXZ1ZEj3bS/nxKBobY21/jR06+F63\nbvKAdgN2q+JiNQ64ROTU6Pfzb62oiCXjW+BwaqrvKTfJV3QZYNZ0K/9IP3D2rH0i5XcZPXkSsYaG\nNn8s09LCfkaGJ/L+Xe3HsjpFf03mHNhq1mRkZ4ssWa9JAMjICPtZWZ7Ic3PtmsJ0/Xdw7Ix+/QBQ\nKJesQDQaRSwW/3WZUDPtDWCjkmfPsPcBmrhxopovKb/VPpHyoi7ZuTPO6NzwundH5N//XR74+tft\nIm0BBgDYZG+CNLFGLraxalVy9tTxunVDRNu3avFiu2j5cjXesNH+42bI6ifUPPT880mZ4uKlpyOi\n/Ec37wG5+EmrR2d31Q/U1NgnuuceEZW8+mrc8bmQkeGhXz95PZHJS8ya17P01+SNd9v9oveVV4os\nWa9JAMjK8jB4sLzOoUPtmmkF+u9g5hH9+gFgxvRzIisZZC8A8+f4Zz4RkQNspkREDrCZEhE5kNBn\npint2yP7qqtEHvQZxJLbV+gHNvazT/TUUzK76aY4o3Okvh6IyM9mGj61F07Onmxc/x//aNbc+E8y\nW78+7ujaVsCC3dixQ42HTJ9slqx4ZINx5PkEBvXXq+k5AKvmysfygeF2zazG42o+bdQus+YX2/uL\n7ET22vgDdKB/z1pEZstzzYrYr0lrUecbDwbcDfl/XpbZnDnxhufMZcUtWLtcLkT+m02d7KIB+gM9\no9H+MvnhRy4T2eHD8ccH8J0pEZETbKZERA6wmRIROcBmSkTkAJspEZEDbKZERA4kNDUKXbsCk+VU\nmBk1yrSJ86ZFHlLzWaPs+2PV2zDr6+MOzwnfV9cG2LbNLsl5RL9t7eqvKzf6njdx5UiRLazZF398\nLnzyiXp77P5P7akxxQse1Q9MmWLWjP7bhoSH5lJ2NjBwoMxLS+2aceP0fORkOf2p1aq00SJ7vvrT\neMNzo7ER2L1bxNOW2lPW0Lmzno/5X2bJonT5Oq4MJeeWWQBAbS3w9tsivrWiwq4ZbvwO0uy2N2GC\nzNatizO28/jOlIjIATZTIiIH2EyJiBxgMyUicoDNlIjIgYS+za882wkLT94t8gcr7MWhZ0XvUvP1\nLa+bNbuVL7Urm/LiD9CFrCygb18RFxTYJcWLp6l5S8DCIWnvvCPDpqa4w3MiLw+44QYRe15AzQB9\ndfr17W4zSwYaC2okS0bNcfReOV/kG3c8bNaMz1+l5qvuTzVrZmyWi/mUbUrOQt9ISQFycmT+7LN2\njbXSSYD7JslFkxctSvif+atV+Z3w2lnZe4aPtWus+UJjRtk1qw5fI7Ls6J44o/sM35kSETnAZkpE\n5ACbKRGRA2ymREQOsJkSETnAZkpE5EBCU6O6dD6HByfIxSvOZc00a5Yt0/PxT/6NWfM1ZaGT/x1/\neE7UhXLxftYwke/baNcUK4u/AEDa1Kl2kbaYi/HvOHfppcDq1SJO6dzRLFnywgk1nxh5zj7Pk79O\neGhOpaerc9qWPm6XLAorjwuA1/rNMmvGjJHZG2/EHZ0b+fnAKGWuj7LPfatmY8peRtD8v3vvlVlZ\nWbzROdMZVRiP1+SBgEVIFtaNV/NVU9+3i/r+VmYjRsQZ3Wf4zpSIyAE2UyIiB9hMiYgcYDMlInKA\nzZSIyIGQ79tbVYgfDoUqARxsu+EEKvJ9v0tbn4TXmBRfhuvkNTr0RbjOhJopERHp+Gc+EZEDbKZE\nRA6wmRIROcBmSkTkQEL35qenh/3MTE/k/TIP2EUdjfu9G+Q9/q0q07uLrKoqirq6WCjeGC9W+/Zh\nPz/fE3mHDnZNerqehz7eatak5cltWKKnTyPW3Nzm19iuXdjPy/NE3quwxS6y7sPu1MmuqaxU463V\n1bFkfAsczs/3PeV+8yM1yjYf51mPZbcU/VoAoDFXXsrRo1GcONH2z9dOncJ+r16eyKuq7BrzFvya\nGrtI2eokeuIEYvX1bX6NAJCfH/a7d/dEnh3dZRf16aPGDWczzZLsGvk8j9bUINbQEPc6E2qmmZke\nrr46IvL3PX2fJwD6KhAAsG2bWbKwQC6cMmdOcvbUyc/3cP/98hpvv92u6dFDz1ML7N9/WNmDqeQP\nf4g7Phfy8jzcfbe8xnnTq+2i6dP1fGzAJjyLF6tx6Gc/S8oUF6+gAJElS0T+xMqvmTVWo3k0x97w\naO/Q+0Q2enRynq+9enl4+235WP7sZ3bNlO/J/ZwAqIvffG73bhGVzJf7a7WV7t09LFsmr/Pa78o9\nmz5nrDbzYU2xWXLtSrmfXYnxPP5L/DOfiMgBNlMiIgfYTImIHEjoM9N+/p/wfssg5YjcZ77V/oGj\n1bzY2IcdAAYoa9e2axdvdG4U4hhmpP5Q5BU9njRrur0lP5cDANTX2yd69VWZ7dgRb3hO5OYCQ4fK\nfG3E/jJp9u6Fat4z4OOk1wboCy0nzZEjwGOPifi5oG9nPvlEzwNWe74Me0WWhcT3pv9rnDgBlJbK\nfMp3AhZufnyuGi8ZMM+uCY8UUSztl/GG50xmJtBXaTPrX/ijWTMsul7Nr007Yp9IO0mm/YXVn+M7\nUyIiB9hMiYgcYDMlInKAzZSIyAE2UyIiB9hMiYgcSGhqVGX4CiycsEXkDw61749dvlzPpzUuM2sG\nPCJvJ83Kij8+F+rKyrBBuXVySH6+WXPE2O++Z8Ats+o0qKB7ox3Kz2nByMHHRf6bSFezZv2Ry9R8\nUJ2cFtQqZdkTxpEpgeNzJi8PuPlmmXfrZtdcY9yeqNz++7l77pGZsTe9a6EQkKa8ip94Ua5v0Sot\nS58CNThsn2fku4+KbOHpw3HH50rq2Wbk1RwS+bDv3mQXffvbel5XZ9dMmCCzC5yXyXemREQOsJkS\nETnAZkpE5ACbKRGRA2ymREQOJPRt/qFDZ/HQQ7Ui3/Sd/maN5xkHBg40azqNHSGytP32t8YupQPo\nqR2oqDBr9hv5tp/8xKy5/fvfl6GyAG+baGkBYjER33qjPWWiNqL//jd1sBfATtG+SQcQeifO+Fxp\nbFR/p7vGytkirfpDn5ny6P+0t0S/8UaZ1WxI0mLmh7Zi9EPyMSj43vfsonL52APA/gHGgj0A5kfl\nDIDjmRviD9CVaBSYNEnmc+bYNcbsmIo75GLerbrlKjuAXOBUIr4zJSJygM2UiMgBNlMiIgfYTImI\nHGAzJSJygM2UiMiBhKZGdeyYiuHD80T+0EN2zfUVq/QD9wcUvfyyzPZbE5Dcarr0OuxbIPfnjgb8\nptY06lNtnnvE3odnyRq5EEUs4zfxB+jCuXOfTRv6C8UD5WPbat06Pc/73e/s87z7rp6/k5y5UZW5\nxVg0XO48Vc1GAAAA2ElEQVRTdF+pPTWqbJLcNx0A5k342D7R1KkimlVp7CXlmA+gRTvQU53gBwAY\nsuk5Nd+QJhcSafXw7tkie63R/nnXarpehlWT14p85J0Bi5D89KdqnFtgT+fDwYMya1F/wwLfmRIR\nOcBmSkTkAJspEZEDbKZERA6wmRIRORDyfXsBB/HDoVAlAOXrrqQo8n2/S1ufhNeYFF+G6+Q1OvRF\nuM6EmikREen4Zz4RkQNspkREDrCZEhE5wGZKROQAmykRkQNspkREDrCZEhE5wGZKROQAmykRkQP/\nCbwTyk8Td2ZOAAAAAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "plot_conv_weights(weights=weights_conv2, input_channel=1)" ] @@ -2274,20 +1859,9 @@ }, { "cell_type": "code", - "execution_count": 63, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXucVOV5x78zy+4isCBxRRBhF1FRVECokaj1lsR7tN4v\nDSYxabw0GquxBq3Jx1xsY2tMW01T0jZ+1IZ6i0mMwRtRUFCiRCPiHV0w2ACLNwwU2N3pH/v5nfc9\nhzOzMztnzpzV5/vPzp45M3Pe857znuf+5AqFAoZhGEZ15Ot9AIZhGB8GbDE1DMNIAFtMDcMwEsAW\nU8MwjASwxdQwDCMBbDE1DMNIAFtMDcMwEsAWU8MwjASwxdQwDCMBBlWyc2tra6G9vb2sfeMyq3K5\nXCU/F6Kjo4POzs7+f0GZVDLGpPkojBFg6dKlnYVCYcda/47G6V+L3d3dAPT09ADummxoaAj2yeer\nlzGyNJelshwHwj0JbpyaN3BzGJ2vuPFqn/6Mt9xxVrSYtre3s2TJktCFV4wtW7a4HxnU+zOlLtIP\nPvgAgK1btwIwZMiQ4L3m5mb+7M/+rJJD7Tft7e08/fTToW2dnZ0AtLa21ux3u7u7OeCAA2r2/T5x\nY/y///s/wF1szc3NRT+vi1ULE7i5jc5xV1dX8FrXQS6XW9nfY68EjXPz5s3BtieeeAKABx98EIDX\nXnsNCM/tuHHjANh+++0BaGxsDN4bOXIkAJMnTwZg0qRJwLbjTut6HT9+PAsWLOCdd94JtrW0tABu\nsVm3bh0Ab7/9drCP7k/N8/Dhw4P3tG3jxo2h/z/2sY8F+wwfPpyZM2cmO5gStLW1sWjRomAsPrp2\nn3zySQCWLVsWvKdj11xq/gB23nnn0HvbbbcdEL4WWltb+cQnPlHWMZqabxiGkQC2mBqGYSRARWp+\nV1cX69evD1RxgLFjx8bu29TUVNGBSGVcvXo1ALvsskvwXimVMw3eeOMNADZs2ADAhAkTEv+NamxX\nSTB48OCy99WxrlmzJtgmNVNqr8w27777brCPr2KliX/9HHbYYYBT7f75n/8ZgOeffz7Yp6OjA4Ch\nQ4cC4XGuXbsWgD322AOACy+8EICjjz66BkfeN1u3bmXNmjW8/PLLwTaZyEaPHh3631fTn3nmGSD+\nftPY3n//fYDATCJ1Gnrv7zQrzuVyOZqbm0PHGUXj800emvsddtgBCNvG33vvPQCeffbZ0P8HHnhg\nsM/BBx8cstOWwiRTwzCMBKhYMn377bcDoz3AH//4RwDGjBkDwIgRIwD3VC8XfU5PP0kOWUCG++XL\nlwPOIbX//vsX/YwvvfsOjGIk4UGuBh1vOccqZPAH55TaccdeJ70cPb4jce+99676OJNi2rRpAPzk\nJz/Z5j2dC0kxDz30UPDe1VdfDcC9994LOM1s1113DfaRZJcGDQ0NjBgxInS/SJKUJjBq1KhtPlft\nMW7evLnu2lQUSaYXXXRRWfsvWLAAgGuvvRaA3//+9wBceumlwT577713yIlaCpNMDcMwEqAiybSx\nsZHRo0cHkhnA4sWLAReGMWPGDKDX1tAfdtppp359rpbstttuAKxfvx6AJUuWAITsVJK6JKnfdddd\nwXsHHXQQAOeee27tD7YCXnnlleC1pJeoRiA7EjjtQcjWBHDGGWcAzvY9f/58AD71qU8F+/jhN1km\nKp0fddRRwWvZTG+++WbASa++PXHz5s2p2RNzuRyDBg0KzZvuzzlz5gBO4tL1C85Gqs99/OMfD947\n9thjgbCNNUpzc3OqkmmhUGDLli3cfffdwbZbb70VgMMPPxyAyy67DAjbvzVPksTPP//84D1dz3vu\nuSfg7Me+5v3222+HQgBLYZKpYRhGAlQkmTY0NLD99tsH0ic46eSll14C4KmnngLcUxHcSi+7km9f\n0lNBNlc9ZbKEnt56mvv2UKHx33777QD84he/CN576623ADe2WkQDVII8wJKyoLgNLSqNAtx///2A\nC+oG2GeffQCnqUh69yVTBZOnSaFQSFSCUjC+pPq2tjbAec6h19OdltQmm2ncPCnY/Nvf/jYQviYf\nffRRwNm6/Xty4cKFAJx55pmAsy8r6QLS1zI2bdrEsmXLgigEgIkTJwKw1157AS4Kw09OuOqqqwDn\nzffRtq997WuAi8h4+OGHg33effddk0wNwzDSxBZTwzCMBKhIzRfKYQU48sgjgd48aIDHH38cgPvu\nuy/YR3nACpvxw1H+8Ic/AHDTTTcB2VHz/XAIORnGjx8PuHAY3+gvZ5RMASeeeGLw3sknnwyUNuh3\ndXWl5rTI5/M0NzcHjrVK+dGPfgQ4FcpHZh+FRBVL6kiDQqFAd3d3SD2tFoUbSUWM5ndD/RMwhMZ9\nzTXXhP6CC9pXHYJSdSdkEth9992DbWmr+U1NTbS3twcqOcSHfEHYZBHlT3/6U/BaoX2nnXYa4BzN\nSjiB3uu43LBFk0wNwzASoOpHthwXkkynTJkChKvmKGxIEurcuXOD92TcVVpeFigUCqGn0bBhwwAn\nkcelt8oBocpPfnqh0tOiToJoZa00nRb9SYpQWIrCpeKSFuQEOOGEEwDnJKgX3d3doapRCnuStlFO\nBTQfOVgl9UlbqTR9ut7ouOMcM0LznAUtY9CgQSWPtS+0ztxyyy3BtgsuuCC0jyRSXwIfP3582XNr\nkqlhGEYCJGZMUtD+ihUrgHD4j+xKCs3wg4dnz56d1CEkRi6XC0mJklKjtjdfslQxDCUd+E+3YrbS\nuN/IGn7RiL/9278F4Ac/+ME2+82bNw9wEsAxxxwD1K+4CbjiGHGSRSVagILewUnekpIU0pclZHtX\nkRnZR/1QNj+8sRiqHaqwt3rT09MTmjeNUwkTfupyFKUB+wkYUTS3Wq+gd57Ltbln8w42DMMYYNhi\nahiGkQBVq/kStZXlJKO1HFHgVEWpS374U6kwhqzjq38at9R7hZzEobCrJEN2asUXv/jF4LXUvc98\n5jNAb1aKUNaIajLUO8urGJWo9wrbW7VqVbBNzkedizinSJp1PuOQw+31118HnGNFjlQozzwh55wf\nCpkldB+VqsX7ne98B3Bzeeihh26zj1+vFsL1QSqpQWCSqWEYRgJULRqp+ryC1pUn6z+xVTdQOe0K\nkh2oqDKUEhQA9t13X6CyUKCsBHfHodz6e+65J9imqjrCT75Q0LdC4iqp3J81NK9+VSyhUECFFmUR\n5aavXLky9NevmFSKBx54AIAjjjgCyO51WixkSeFrAL/5zW8AQsH+UeR4UrigH8JYydhNMjUMw0iA\nqiVTSWlawVUZyK/1KZvErFmzgMqquWeRN998EwhXYJeE7ldJiiIb40CQ2r73ve8BcPrppwfbFDKi\nepF+3cjjjz8ecCnDAxlJKKqA5afdKk04mvRQbzupj45F16QSakrZPv1KaLIHZ2lMlaDwTIDrr78e\ngP3222+b/aR9yXchW2l/1yeTTA3DMBKgaslUgbJ6+r344otAuEK7iqHUO7UwafyiCb/73e8AV+cy\nrj6oJNKs2qDA2QnlAX7wwQe32Uc2KT+dVF78LKZVVnq+JZlpDlVzF4onIWRpTuW1V0TF1KlT+/yM\nL43VM220FMXO8bJlywC48sorAZcwAttKpH7CkLoOqxOGtI24xIByMMnUMAwjAWwxNQzDSICq1XzV\n79RfVRZSGw+AmTNnVvszmUJhQH4TMuUHS+XwW13LKVePth3lojxuJSJ86UtfKrqvWnX4udADpVme\nageUUy1K161fAc0PfM8qCu055JBDiu6jJBu1KFEYFDiTXdaRSfGKK64AXC2BUhXofLVdc6nzFXdN\nWGiUYRhGyuQqMbDmcrl1wMraHU5J2gqFQs3jbmyMqfBRGKeNMUEGwjgrWkwNwzCMeEzNNwzDSABb\nTA3DMBLAFlPDMIwEsMXUMAwjASqKM21tbS3UKwato6ODzs7Omufs9XeMcuRVk1aYxTHGOSirTZ1c\nunRpZxpe4GrnUvRnvFmcy6RJa4zgxunPTfSeq1VKb7njrGgxbW9v5+mnn+7/UVWBHzhdS9ra2li8\neHGotoCCeqN5536lHXUTUGM8P6Bdr9WAT9/tV48aNmxYbOvkWtDfeVRlc1Vy9xsK6j1d4KpI749R\n23K5XCohLmlcr6odqoBxgFGjRoWC4GtJe3s7S5YsqbhldRKkdU9C73355JNPBhXzwZ171RLQvec3\ngdS1qtoD/vWoXHwlnOh+9s/loEGDyh6nqfmGYRgJYIupYRhGAmSuo5typ5XrDr3qdVrJBblcjkGD\nBgXqAbiizlE1//333w9eKxdfakSc2qXPS1XxTQF+AeJaUygU6Orqqrihn/bXX7/+wEcV2elUehF6\nVUqZPdKgmIqvEnOvvvoq4GpKgFNx4+pGqKi06jSoFVE9S/P19PSwceNG/vd//zfYpnH7DfAgXP5T\n6r1UeH9dUWslmQc0bv8+r6SehkmmhmEYCZC4ZKrqQ9G2DsVQ6wBJf2pBO3r06GCfwYMHp1Z8d+vW\nraxZsyYo9gyuOpQcUb/61a+AcHFoHffatWuBsEf4lFNOAZxE+tvf/haA6dOnB/uk6UDo7u7m3Xff\nDVX2kkagtiOqAjZQqkHVC0lvfguX0aNHpyqZgps/IHC6zZs3D3BOmI0bNwb7qA207ldf6nzhhRcA\nJ3X/zd/8DeBafEN8i+taIslUUiRQ1GFbSsvzz1P0nlNbF18ytapRhmEYKdMvydSXupYuXQq42oKS\naGRngbCUGeVjH/sY4BrzaV+/3WqaFAoFNm/eHLR88I9JjBs3Dgi3PlaYhp7qfrsEPU1vueUWwLWf\nnTFjRtKHXxb5fJ7m5uaQ9HT//fcDTmpWMznfdixtQ22ODz300OA91ZBUrdNSzdvSoqenhw0bNnDH\nHXcE29RyRRK4bGh+Ows1zdPfzs7O4D1pLJLydL4k4UHvOUvLxt/V1cW6detYtGhRsO3hhx8GnFYx\nbdo0AN56661gHx2vWpT7SIP6i7/4C8BJpP61sHXr1lQb7uXzeYYMGcIBBxxQ1feU0gCr9QGYZGoY\nhpEAFUmm3d3dvPPOO0GVbnAtnSWZ6Qkv2yHASSedBMQ3W1uwYAHggr7//M//vJJDSpympqY+K42r\nQVlcozI1lvM9p0I2Snn849rPpkE+n6elpYXDDjss2KbXsl3Pnz8fgOeeey7YR8HQeoJLqwD49a9/\nHfoN2a1OOOGEZA++AjZt2sQLL7wQaE8ATz75JOCuU2kdvj1QEqnw7aGS5CTlSRPxtZfW1taKIyX6\nSy6Xo6mpKZA+wVXNr8Te7SegyA5cKljdT9hIg4aGhn51qpDmoMiGM844o8/P+HbVSjDJ1DAMIwFs\nMTUMw0iAinSRXC5Hc3NzKPRAzhgZ6RUu5O8TVe+//e1vB6/nzp0LwNlnn13JoWSWOPVeKMj/s5/9\nLJBuOFS5yHF0/PHHh/72hcJxpBLHmUp8B0YaNDc3M2HCBD7/+c8H2yZNmgS4kDyN9xvf+MY2n//O\nd74DwC9/+ctg28UXXwy4XHyp0n5o0ZgxY0J96GtJQ0MDI0aMqNphe/PNNwevZ82aFbuP73DyQ5TS\nIhqmJOdhR0cH4MxoDz74YLDPv/3bvwEE10Ccmi9nq+5d/77caaed6OnpKev4TDI1DMNIgIokU4Un\n+GmQolT4k1i5srdY0BNPPLHN54477rhKDiUTyLgNzhEX91RX2JCM/HFOma6urlRDTZJGoUZyysWF\n3KQlrYlBgwYxatQoRo0aFWyTU+WrX/1qn5+XU85PrpBDTnOpUMDJkycH+4wYMSKTWkcccrwp6QTg\nr/7qr2L39R0zuVwutUSaYkjj1bH76d1C9+NPfvKTot8jLUqf90MGm5ubyx6nSaaGYRgJkGqhk//4\nj/8AwuE255xzDlC/MKFKkDSi0LDHHnsseO+yyy4L7avwEoAf//jHANx0001Ab+3SKFl40vtUWuxa\nNvOdd9459Dk/5CZtyRR6x+GPoRJbn8L9/DCnYuFh0WIbWaDUHF533XWAs3X7BUSiKIzMTxHPguSt\nudR1FZfKrUSZOCSByv6tQie+v6exsdEkU8MwjDSxxdQwDCMBUlHzFaqgEBPfoH/eeeelcQhVodAI\n1UlUdpDCwuJQ+BPAqaeeCsCXv/zl0D5+qJBaemQFjbmUOudXndLxS92VqlUP1b5aFi5cCDhV/vbb\nbw/e05zJObXnnnsC5VdJSwPNXdSk8cwzzwSvFQp21VVX9fl9cjhnQbX3kUNsxYoVgMvCO/zww8v6\nvOZS49Ic9rcilkmmhmEYCVAzydQPL1Dws/K+/fAhVRkaCMippDCor3zlK9vsI2eTn8/9T//0T7Hf\nF1eroN5o3srJLf/ggw+C17vuuivgpKFKvicL+BXAVBVLNRT8PHRJq9JKFBrlU+8Qt6jDRBLYt771\nrW22XX311UW/Rw64elVw6wuNQfUR5Ow8/fTTy/q8HFZyPO2xxx5A/7Upk0wNwzASoGZiwzXXXBO8\nVgiRbIeqIjVQ0JNr2bJlAKFqS1FuuOEGAP76r/862HbUUUeF9tETNUt2Uo0xWlc2rlLPqlWrABcG\nBS5ESGPLotQdhypKzZ49O9h21llnAU7K9sc5cuRIwNXrzWIfLEmmSrecM2cOAA888ECwzz333FP0\n85pDSaRZs5UKpa5rnBdddFGfn/E1RoVEKeGkWgncJFPDMIwESFwylefT94DKe68UvoFiRxPRPjnq\n6eSjDpDyCMbZU0WWJFIhO5+e1nqC+2mGkl7ffPNNIFxpX2RtbFH7paQ2BXN/7nOfA8JB9/LQy57v\ne+r33ntvgFCKatzv1BPN00MPPQS4Gq5KGgFXRT8OXQP+3GcRaQ6qITxx4sQ+P+OvPSpyIxt5teuS\nSaaGYRgJYIupYRhGAlStbyvcSRWUFKZw7LHHBvvIYSNxfKChmpWnnXZa0X1kDFdbXKmKPgrdyGIg\nu8Z44IEHAi7wW5W+wIWGySHhJx0oTCqu7kC98VVwNcS76667AFcxyHeKKsxLDQ/9UCOZejSHWVLv\nheZFc3rFFVcA8W124pB6X4+apZWg8ZUTZK/reM2aNcE2Nc3sTzuUOLJ9tgzDMAYIuUqerLlcbh2w\nss8da0NboVDYsdY/YmNMhY/COG2MCTIQxlnRYmoYhmHEY2q+YRhGAthiahiGkQC2mBqGYSSALaaG\nYRgJUFGcaWtrayGuH3oadHR00NnZWfMmSRqj75iL9tKpVa+mLIwx+r8/1qTGv3Tp0s40vMBx44zy\nYZlLv7e7iiZH+737caOKFa5mTtMaI5R3zYq4sVQzz+WOs6LFtL29PWjAlTZq0Vtrxo8fzyOPPBLk\nJ4OrZ6nqMqqQ5Ne5VOUaXbD9CQROa4xjx47lvvvuC5rDgWuopqpR+uu391UlKP1VTjO42p8KpFY+\nu84ZuKo8uVwulRCXtrY2Fi1aFBqDFpFSNQQU9L5x40YgfCNq7Pp8XGB7oVDg4x//eJVHXx7jxo3j\noYceCiqagWuSp/oKqi3hn4focftVvpSYoCB//fXne/jw4RxwwAGJjaMvxo8fz4IFC3jrrbeCbbpG\nde2qVbO/yKrKl65HP6lE16OuWb2nnH3ozdcvdy5NzTcMw0gAW0wNwzASYGDVwkuBQqFAT09PqCRb\nsQLAvqqYtdJzpWhoaGDYsGGhMnLKb5adbffddwdczQWAn//854BTH/0xS52aMmUK4Era+SpSuY3O\nkiKXy/VrXvSZ/s5preywxcjn86H2KTJDnXjiiYAryffaa68F+6gZosop+qqt1HmVINR3+6UI8/l8\n6uPUvSl07CoxqL+vvPJKsI/Grs/597LalOy///4AgdnCv2YnTpy4je25GCaZGoZhJEBFkmlPTw8b\nNmwIqgeBax+raix6+vlPLVWhUSWlffbZJ3hv2rRpgGsnW28aGhoYPnx4TZ+6nZ2dQPgcSZpIg3w+\nv420LUfRJz/5ydB23yl24YUXFv3OYg30nnvuueC177D7MLNly5bUqkk1NDQwYsSIslpu6F4biOTz\neYYNG8akSZOCbXp99tlnh/ZVdTaAJ554AoDly5cDTpoFV/FOEndcNbSNGzeaZGoYhpEmFdtM8/l8\nKNxFNQH1NFDogprogVv5JZHKHgew3377Ae5poBCbz3/+88E+adfI7EsqlYQV1zRO7XHjJE2Fr7z+\n+uuAs9WU85tZp1jLB9lQ60F3dzfvv/9+qCar5kB2X4XX+GFivmQCcMghhwSv1RxRdsT33ntvm8+P\nHz8+1OrcSBe/XrDmzp/DSujp6Sm7nYlJpoZhGAlQkWQqW1uc3UJceumlfX7P2rVrg9eytcpTrABj\n37YR/Y16oeNWgL6q64NrQlbqCXjbbbcBvYHW4LytabNlyxZWrVoV8u4uWLAAcMco6dl/Kmts2qbm\ncgCf/vSnATjmmGNqeOSVUygUQlJ/tEK+pFbZ/sFJmbLxq+I+uABxdRbQ9eprT+Xa2JKgq6uL9evX\n8/LLLwfbZB+Ud/vFF18E3JyCm0N5rs8777zgvWhr8izQ09PDxo0bg8B8cO3Ia0kl3QZMMjUMw0gA\nW0wNwzASoC5B+36wuF5LFZPa76uXPT09dW1cppxmHaOcC3/4wx+CfXynXJR/+Id/AOBnP/sZAJdf\nfnlNjrNcZK5R0DK4JnIK1L711lsBWL16dbCPnGtS75XDDbB+/fraHnQ/aGhooKWlhb322ivYJuen\nVFkFdcuR5DN58uSi3/3ss88CzuGq8wa9dRkUZpMG3d3dIXVUJgfVh1CNAb+ZXNSp6yep6F7T2HRu\n/MD+sWPHpnpPFgoFtmzZEoQVggvD1P0p06BvIpQjWNeqb4ZTTn5SmGRqGIaRAJlJJ422p1V6IvRK\nqfUMHZJDQaFQcjr4joZSAdE//OEPAdh5552B9KpDFWPQoEGMHDkytkXu9OnTAWd490suSlvQX3/8\n++67L+CkdT+9sd74Uls0VVTXmy9ll4O0E0l0vmYyZMiQ1NokNzQ0sP3224ecgXLYzpw5E4AjjjgC\ncBIquFRKzXccml+NxdcW09YU8/k8gwcPDhKAAFpbWwE3LoUs+mnA0nx1rZcjjfqhcd3d3Ra0bxiG\nkSZ1l0xll9HqrydHVtJLwYXTSBrRk7pUCMlll10WvJat8eKLLwbiA9nr8aT30fmXhHXmmWcC4WB0\nJVT4WkOULHa79Y9J41R4m6SYcgOzZUOWdqLwHP+cpCWVQq8dv7GxMRSorjEqXVKhePoLTqorRbRG\nry/Vpa0tqmiNL3VqnLJ7a579fSRl+hJtX2jc0HvNlztOk0wNwzASwBZTwzCMBKiLmu8bwhV2IVVC\nBuMsqfkS+5X94oeIRNF4lEkEzgFw9NFHA/EqZb1z86UqyZShWgt++5VyaiQkHW6SNNGc+WLVrnz8\nsC/l9kutl5ofp37WC6n3Mi9pTspR7cGZMHROdC/KzAPpmjKKoblTJmK0dQ5U1j5IWW3RjDlT8w3D\nMFKkLpKpHyAtA3FcM6usoCe9ArFL1Qo455xzgLAEdNZZZwHhOq5ZQ4HPfvA5uLH3RbFKWr6UlgVp\nRlJGse4JcbzxxhvBa0nskn760zix1kiTUsC6QvJK4Tsa5WSTpCfJ1Jfe661JgZuLaIdVv3FgJehz\n/hpUyTjrf3UbhmF8CKhaMpXkobqQCnXyQzUUbqOqNf7KH20J7NtlsoKOyU9LjHLPPfcA8Mtf/hKA\nWbNmBe8deuihNTy6ZChW6b9cTSGutitkQxr1KXaccahCkW8zVfC3khL0fVkKCSvXNurjayC6d6Ut\naoxZm0sdX3/rHUdDq+KSEyohW2fHMAxjgFK1ZCpPmjyHHR0dQDhAWNKqikP4XQ7l6c6irVSUc2xX\nX3014NL0Tj311OC9cmxWHxai9qo0C36Ug2y7Kpgh6VORGuC82bIj+lK7bMpZ0qCSsF/6UqfGK4lP\nkmoW7KRJojHLvyE7eH/HaZKpYRhGAthiahiGkQBVq/kKDVHYT6nwH6lSftiQVAjfYTVQmDNnTvBa\nyQaf+cxngP438KoXUm2j7W79Vs2qEylHnFpe+GRNrYewc0hqvcJ/Xn311dD/PgrI92t9ap41zrgA\n/XqrwzomVfCSKW7VqlXBPi+99BLgkg98s5zu4aw5npI+rzL5RM0Z/SUbZ8kwDGOAk6skpCOXy60D\nVva5Y21oKxQKxcvZJ4SNMRU+CuO0MSbIQBhnRYupYRiGEY+p+YZhGAlgi6lhGEYC2GJqGIaRALaY\nGoZhJIAtpoZhGAlQUdB+a2trwW/96xONCkg6wLajo4POzs6aR0OXGmOtycIYNY/6W4uA7aVLl3am\nEVLzUZ3LciJ0krg/0xojVDaX/viTuI7LHWdFi2l7eztPP/10aJuyCJRtoWwCZYokRVq95tva2li0\naFGoQ+GIESNS+e20xhg3j2rZoCy1aF95cFk/ymCLtgCB8op853K5VOIF48aZFvWcy2L4RWjUOija\ncx7cPKsotObSb80C6Y0RajuXKmijUot+y6RRo0ZxwAEHlPU9puYbhmEkgC2mhmEYCVB1oZNosYCk\n1fu0KRQK9PT0BDVYARYtWgTAypW92qk6VPoVvvfff3/AFcXw613OmDEDyGYRELFu3TrAqTyq2+mr\nPCoSot47qlAOToVUcYws1af11VsVNFGBHhX6+CjgX38afxZ7WMVRKBTYvHnzNqaGJNB9/OijjwLh\ngiczZ84su6eUSaaGYRgJ0C/JVOXZwDks/BJeA5l8Ps92220X6vek0mWSTB955BEgXLbt1ltvBVzH\nAd+gr7J8Bx10EOC6lY4fP74mY+iLQqFAV1dXqOvm888/DzhJTWXc1qxZE+wjSVSeYJVxA+eMGjt2\nbNHfLbfTadL4Y9A4f/e73wGwfPlyIOxMU2cE/ZVzDmDy5MmAm8MsUCgU6l72r9Z0d3fzwQcfsGLF\nimCbuhxHHaJ+D6e1a9cCTpuaOXNm8J6iA+69914Abr/9dsD1o4Pe69m/l0thkqlhGEYC9Esy9SUM\nhUSpp45sh/3t8JdFjjrqKAAmTZoEuOLIvl318MMPB5yt0O9o+fLLLwPOnlxvO113dzfvvfdeqO+R\nnurqvimQUykyAAAZX0lEQVSbr/9UHjNmDOD6yPuSqc6NH0oV/XxcIeVaUigU2Lp1a8h+O3XqVMDZ\nDyXdPPXUU8E+6qKrfmbSNsCFA0lLOe644wCYOHFisM+gQYNS61aqudQxg7snFd6nufRt/LoW1clU\nhbCzTNSXsXTp0tD7CmGSvR+cdqHr2u8/J04++WTA+Tb867qxsbFsqd8kU8MwjATol/joe6r1hFd7\ni8WLFwPhVg+yM8leNXfu3OC9yy+/HHBdSrNMNAPDD+zfc889Q+/JOw5w5513Ak4KkO20Xp7UXC5H\nPp8PPaUlqey2225Aac1C8+jbImfPnl30t0S9WtP4moBey7YrrSMOSZ++lKnrWpENQ4cOBcKRDS0t\nLalJprlcjsbGxpD0rTYlDz/8MAALFy4EnPYIrk2NOpEedthhwXtf+MIXQtvkI1GAvz6XpqbR09PD\nxo0bA6kb3Npz4oknArD77rtX9RuSaH3JFsqPTDHJ1DAMIwFsMTUMw0iAfqn5ftEAGXilWsip4asE\nCiX68Y9/DDiHDMA111zTn0OoK+UUXPDDK3SOFI4jx0a9jP65XI6mpqZQgoVUmXIch3fffTfgxlWK\nenadlQrcX6Lqnk9U9fOdO5BeR898Ps/QoUNDoXya1z322AOA0047DQibZZScIdOTTHHgxq39FRoo\nJw701qtIs+WR5tIPJ9x3330BF2Ko4PpSyTG+OUbmC52DajHJ1DAMIwEqlkwLhULoaa9wGTlTFGLj\np2D95je/AZyx/rvf/W7wXlzv9Q8bCtmQhO4/4etBPp9n8ODBoSd4OeEfN9xwAwD3338/APPmzavN\nARpVoetLziXNrcLAYNtQuDj8AHkIO0ybmppSk76hV9psaWkJOXqV6qyxSBuOCz3UvXfjjTcG2/7z\nP/8z0WM0ydQwDCMBKpZMoxKMpE39FX6wtoKejz32WACuvPLKin4zK+2olZJWyg6nffQkBLj55psB\nOProo4HqQziSoKGhoazCK7/+9a+D19dffz0A06dPB2DChAm1Obg64NvxlZQybdq0eh1ORURDlHSP\n6q/e9xMqSmkiSp+VXVUSrl/XN+2knHw+T0tLS8lwwuga5PP9738fcNcubBvAL8m9v/WLTTI1DMNI\nAFtMDcMwEqBmsrqcTgCvvfYaANdee23Zn/cdWPWuAyo1qZzj0HH7xnmFQKl61EBAYTFyOvnbrrvu\nurocUy1QJtObb74ZbFOG2kAlqsLL9FRuLVBluCn8S9evX9u2oaFhQFSqmjNnDgCvvvoqADfddFPR\nfaPhbZVikqlhGEYCJC6ZynitfGBwtU7jKrYUI82wi76QM82vSRBFSQsKzFd9U3COJz+wOuvcd999\ngMvvBrjggguAbesQDGRUU8J3qGSpS0ASlCOR+mFQqiuh+1YSWz0TMPqLHMGXXHIJULqGcLUacHZW\nLMMwjAFM4pLpCy+8AISfYpVUJS8nJSxtohKp7C9+PUVte/bZZwHYddddg/eyVJW9LyShqPq4L4X+\n8Ic/rMsx1QJVYFdqr58aK+2qWhtavankXvJtxgp6V0q0JPWs2UhV+UqVpHTv3XbbbcE+qkZ3/vnn\nb/N5pb4r8ahaTDI1DMNIgMQkU6V0qRK2An2hvMIgsksOBLuMgu79Y9VTTk/4c845J/0DqwIFaC9Z\nsgRwRSCK1SkdqCgBRDZuVc73A9qfeOIJwJ2DKVOmBO/Vq29XX/hSo8bmFzaBcOdg1TZ98cUXgXAP\nLF3fCl7XdZ41yVTHrGv2q1/9KuDGDy71OY7+BucXwyRTwzCMBLDF1DAMIwESU/Oj+cC+inHHHXcA\nrr1AXKhGVlWJUvjmC4WTKPxLYx0oKDFBThclGPzlX/5l3Y6pFsicpLmLu97kjMqiM7QYfv0KzaUc\nab///e8Bp9KDc9rIyXTwwQcH7yk4v6mpCXDnKCs1MoTaBslkoXm65557yvq8n4SQBCaZGoZhJECu\nkqdNLpdbB6ys3eGUpK1QKOzY927VYWNMhY/COG2MCTIQxlnRYmoYhmHEY2q+YRhGAthiahiGkQC2\nmBqGYSSALaaGYRgJUFGcaWtra6Gc1NBa0NHRQWdnZ82DUD+qY1RsohySKoFYi7jfpUuXdqbhBdY4\nfSerXkfjopMm7bmMcyTXOmY7rTFC/DWrMUev2aQpd5wVLaZtbW08/vjjoeZjO+20E+CqcdcK5VDX\nmvb2dp5++ulUfstvOtjQ0BC0hK4148ePZ+HChUEHBHAV51VbQQHbqmkJsM8++wDlV2wvRi6XSyXE\npa2tjcWLFwcB6uDGOWnSJMBVRPLbIOu1FiO/vmlra2vsbymPH3pro6Y5l4899ljo93Vd6QGpRcZv\ngZxEDYy07kmIvy81PuXiK8nCf7Bo7Aro9+vW6j1d68Uod5ym5huGYSSALaaGYRgJUJGav3nzZlau\nXMm8efOCbWPHjgVg1qxZRT+nhl5RkTuOjRs3AoRUs3Hjxm3TG7xWdHd38+677wZFrsHlLyfd7375\n8uXB69bW1pDaX2sKhUKoTKLUX82NGiKuXr062EfqvcrQSe2HcDFsgAMPPLAGR10ZPT09/OlPfwpy\nuMEVAo62JvHLsUkFlrpY6tqTiqkSjNBb6s5vCJkG/j2lMpCLFy8OHdumTZuCfaTyKz/dL8+ncoS7\n7bYbALvssgsQPkdp19AoFAohUwa467GlpSW03TfZiKFDhwLhc7BhwwbAmUV0P/hmgqamprJrEphk\nahiGkQAVSaZdXV2sX78+ZLz+9Kc/HdpHFVzUFgJc6ws5rko1llu/fj1ASJpIs61sd3c37733Hg8+\n+GCw7bHHHgPgi1/8IgAnnHAC4JqxgSsorIK1hx12WPBe1BkhScdvVnfwwQcHEnytyefzDB06NCSd\nScKaMGEC4I7ZPw+33HILAHPnzgV6HTxCc6qqU3IWXHzxxTUZQzkUCgW6urpCDgY5TEshaU1tMUp5\nid944w0gLJkOGTIkNU0qn88zePDgUHFnjVdzqWNbsGBBsI80L0l70jDBXZ9yvMycORMIX8d77bVX\natcr9EqPq1evDtpQg9MYVeVL96DWIIDJkycDTjL1HVDSHnTNas2p9HoRJpkahmEkQEWS6bBhw5g5\nc2ZJe9gOO+wAxIeQ6CleSjJ96qmnADj55JND29OSTBsaGhg+fHioLbWegAcddBDgnmR+3Nvbb78N\nwKpVq4BwSFEUSTovvfRSsG3q1Kmp29l8iUuhQlHOPffc2NflsnKli4JK287W0NDAyJEjg/krF9UB\nld3etw1HmT9/fvBbYtKkSanW/szlciFtcd999w39Pf744wG49tprg300L3HXotp86zqXPd33Y4wZ\nMybV67WpqYmxY8duY5sHF86n1jt+nVJJ7K+88goQtvvKJiwJW9L6/vvvH/rdcq9bk0wNwzASoOJK\n+31lGWgV95905513HuDsqMcee+w2n4u2aY1Kpmkhaea0004Ltqk5nu/9hnB7YD3F5f3234vy+OOP\nA+HK51nh9ddfB5w04kuW0joktc+ZMyd4T5X5ozZ0eUwh+crmfZHL5UI2snKRrbScwHZJptOmTQu2\nVSLNJEGhUKj493x7N4Q1Kc2h5k6V+n1bZNpaRi6XK5osonvtggsuKPp53Zf+ccuW/K//+q+As49+\n8pOf7NcxmmRqGIaRALaYGoZhJEC/Gur5QbEKJ5CIrABYXwVcuHAh4JrOxaH9/fzheuKHipSDAp19\ntbYY//Vf/wWEwy7GjBmTSL50f1mxYgXgHIcdHR0AoeQFqUiPPPIIEA7UV7KFUN6/HzIUNQFkATkO\npcqCOxelHG6333474Mwgp5xySvDeyJEj+2Ve6A8KV9Q4wN2DcrxVEt7jo2B4ObL8a2HIkCE1KyxS\nC+LuZ82d6op87WtfA8JrUCWOxIFzNgzDMDJMxY/PQqEQciT4oQbgnvB+gPDll18OFK+4A07CvfDC\nCys9pEygp1w5EokkOyUBQK9DoK/qNbVk4sSJof+POOKI0F+fr3/9631+3x//+EfApW/WEz+ER+mE\nSg5ZtmwZAPfee2+wj0JmSkle119/PQCDBw8GwgHtzc3NqYbytbS0hK47SaYaqwLd/cpuuhflbPND\nAaPoHvfv35EjR6beArtQKITSQRX2JK2wUn76058CcNZZZwHx61Ml82iSqWEYRgJULJlGA4SF7KE/\n+tGPAJeeCHDdddcV/b5///d/B1y4VJo1EvuLUvCUYOBz8MEHF/3c7NmzAWeT8aW+lpaWVJ/0hUIh\nZA+S9CapRqFRlUqWug4UQF3qfKSFf14VwiaJQ3M5derUYJ9S2pHs3dLArrjiCiCcwJEmuVyOpqam\nkFajeZXUqbRhX/JSAo1srUr5BhcSFtWUfFvidtttl7rNNJfLlQyvU7KFv/ZE7cUXXXRR8HrRokWA\nC42qFpNMDcMwEsAWU8MwjASoOn5DRuAHHngAgDvvvBOAX/ziF0U/47c9UTWXU089tdpDSQ2FzviV\ngQ455JA+P6dqS8oIU8gJpJ9REkXzKFUpLp9dudmqbxmHMqimTJmS9CEmgswZytBTFtoZZ5xR1ue/\n+93vAjB9+nQATj/9dMA5orKATDVy6ipzz6+Xq/DGGTNmAE7tB1dtSqxZswYIO6nSrORWjGg/L43B\nv6+ExnDjjTcG20qtUf3BJFPDMIwEqFoyVVC2DPMyEB9++OFFP/M///M/wWs/Bz7ryOnw6KOPAsUr\nLUU56aSTAFd96Oijjwa2rRBeTyRl65gUauMnIZSSSIWqZsU1lPNrbtYLhdZIypaDolQXhfPPPz94\nLcn7kksuAcKOm6yg8ywHkebW1wilechJpboLcciBl3YoVLlI6oyTSIUqZ33iE58ItqkucVKYZGoY\nhpEAVUumSjsUf/d3f1d0X1WW91O7VAk7y0iK+e///m8A/vEf/xGAn//850U/47dRlm1GYWNZlGZk\nu47iS8/SQhQu5SdmSCL10yqhdP+keqC0VwWiS2KJQ3UuFb4HLpztxBNPBMpL0kgbHZOC9GVbVKIC\nOOlVknpcjWJVidI1kLX0UY3n1VdfBeLTZtWvTp0f/M4RSZOts2MYhjFAqfqxqmDl733ve0C4SnUU\neRNLdTLNIrIVyWMr76Gkkzj8MR555JGAK/SRJc9vJUQD+H17qgLCo0HVvmRab+8vuKB9JROUsgN+\n5StfAcLS+Ze+9CWgdCeFehO9vnTe41KD4zriqrupogCyKH2DC84vlRii6IurrroKqG2xHZNMDcMw\nEsAWU8MwjASoWn6XulOO2iMVq57VkfqDDPlyVuyxxx59fsZXDRXYrfYKWVB3obzjWLx4cfBaTgpV\nhPLVyU996lOhz0UDqsv9vVpTSesUORH98D21tMjCWJIg7l6U403qfdYcTyLaekXccccdwWs17/zG\nN77R5/dV2wQxm2fJMAxjgJGrZDXO5XLrgJV97lgb2gqFQmU9e/uBjTEVPgrjtDEmyEAYZ0WLqWEY\nhhGPqfmGYRgJYIupYRhGAthiahiGkQC2mBqGYSRARXGmra2thWK9buTIKuXQqiZeraOjg87OzpoH\n95UaY63JwhiV/qm//nwqtlLz2N8Y0qVLl3am4QUuZy7j4mGTIAtzWauxibTGCANjLitaTNvb24Pq\nK0I3nQK5VWHJb2oVbdDl52srQFj7qGmX3/xrxIgRsfUxa0FbWxuLFy8OtQdWQ7Jak1YzwXHjxjF/\n/nzeeuutYJsqQqn6k6or+RX3NTfKyVczNnAV7JWsoMpg48ePD/ZR8sN2222XSohL3PUq1DBQgenl\nBvOXe8OmPZfRluvg2o+rVqmSZpIizeaXcXOpZoiqHtXc3AzE369RIcF/rXVJ10BUICxVb8TH1HzD\nMIwEsMXUMAwjAfqVm++LylKX1DpA6qHffEv7SxxXMy9wDdyiapbfozuXy6WWC60+5B9murq6WLt2\nLXfffXewTa+fffZZAPbee28gnJcuNV0l2vzrQO0gjjnmGCBbZdv8dikyR0lFnDhxYkXflcWc/GJ+\nCtXLUI0B33QlM8zw4cOLfu8777wDhO/FLKG1Q6YpHeeoUaOCfVRiUOZEzTu486FzoDXIP0/d3d1l\n5+ybZGoYhpEAFYsPPT09IeeSnBha+VVZqJR050s0+rwcUHGtB+qBGo2Bawssg76ebr7RX5K4jPx+\n87msjEk0NjYyevToUHFrPY31V0/0F154IdhHT2g5pXzn1H777QdkSyLt6elhw4YNoRYyuk5LNdAb\nSDQ0NIS0QB9FXUhCVUNIgPnz5wOw2267AeEGiNJK5JTUX2kmWUFrhjQPtbb2i5brHMg59corrwTv\n6b6Wg0njk3YNvZqISaaGYRgpUrEYkcvlAlsFOBuUaguWY2/0401VE9MPhaonPT09bNq0KQjxAtfe\nVxLOsmXLAFiyZEmwj+yICg1SHUVwLaHVXmHq1KlAOLQkzTa6+XyelpaWUGtcvf7mN7/Z5+d1bm67\n7bZg27nnngvAFVdcAbi6n/Wkp6eHLVu2hGxguj41X5pLX1tS3VnZDBctWhS8p1q2pRrx1YN169YF\nr6VNSKLU/ebft08++STgJFN/jApn07WQNYlUSHvS+NQU0rfx6n7UNSD7KsCLL74IbNtM0Lcjb926\ntWw7uUmmhmEYCVCxZFooFIL2sODsFr79rC/U+hicbfJzn/tc0f03bNiQWstgRQ7oiQ2wzz77hPaR\nDeXll18Otvk2VnBRDuC8hLLb6Enqt8neYYcdQhJUlpE9+Otf/3qw7bjjjgMISbtRZNNKi1wuF9iH\nhaQZ2b/9ORSS2ubOnQvAc889F7yndt9ZYevWraxZsyaQtMFpOYpUUBC7pHBw7crVJPHOO+8M3jvz\nzDMBOO+880K/5dsiof5tvKUpSiKVdqw1yUfnxE9U0XmRphhHY2OjSaaGYRhpYoupYRhGAlSs5ufz\n+aCfNjhRuZxe8I899hgAf//3fx9s+/KXvxy7rx9cu2nTplTV/L7GIrF/zz33rOq35CCAXpUpra4H\nhUKBzZs3B2aHJCim3j/yyCPB6wkTJiT2e+WQz+dpbm4OBXFr7nTdKnjdV+XmzZsHOJPAJZdcErx3\n9tln9/m7aXavaGxsZMcddywZfifzStRc5ePfbyeddFLsPp2dnaH/FWaWJn49CDmKZGKMU++FEob8\nMDk5hpNKxDDJ1DAMIwH6FWHtV2WJOlX0tPKfzjL2X3fddUDY6K+Qmihr164NXo8cOTKz7WarQYZz\n6D2PaYVHFQoFenp6QsHJChmShKKwkdWrVwf7yJGmuSgV/qSUYX8e0w6Uz+VyRaVvOV7013cYKtxv\nxowZANxwww0V/26a9HVv+Om0Ue6//34g3Jq8WAtl/54eMmRIXe5Jv/KVNJ1SEqm48cYbAfjVr34V\nbIu2J6+WD98KZRiGUQf6JZmWsrXp6eUHCOvpp6fC7Nmz+z4wLy2xXk/BKLKhxSUmlHovikKgfEm0\nubk5NYkmn88XPU5Jq7Ix+WFw2lZMm/D57W9/C7gAeCC25mZWeOCBB4LXCmC/8sor63U4iaDiQ3H2\nVGkOsoN+9rOfLfo9upd9u+ouu+xSl9ThSosQLV26FICf/vSngNM2AE4//fTkDgyTTA3DMBLBFlPD\nMIwESFxOl6rqV6i56aabgN7WAwDXXntt0c8rk8MPv6q3ii+1PKqG+3VZpVIpd7sUMgmk1Q6lHOQ4\nlMNJqp2vVpXjQFImmMJxFH4C2RqveP755wG44447gm1TpkwB4KijjqrLMVWLzn2pEK1vfetbAJxy\nyil9fp9qMfj1M5qamjJZ2zXKv/zLvwDOnOGvPUmbKUwyNQzDSIDEJVNVr/nZz34WbFMolL8tiozb\n0eZWWUCSsRxGcsr4ucrlSKRy7khiqLfE7aMnt+oFSOrwQ2Z8430xFixYALhKQ1nSMHw0F48++igQ\nbgB5wQUX1OOQEkPaQbTa08KFC4PXd911FwCXXnpp0e+RhKvrPqsV96P41czkeDrnnHOA8iTx/pKd\nq9swDGMAU7VkqjRPPcVUmeapp54K9lF1oWJpauAkI0kyWbLH6FgkkS5fvhwIV0FS8Hc5gfdZtB0K\nVcuSXbdURR0fSXYav6q7Z22ssg2r15Uqr5911lnBPpMnT07/wBKkWP1RP0lDgfmS3C6//PJt9peW\nonTcLHVRKIWfZKGEhYsvvrjmv2uSqWEYRgJU/aiR1Cbvr570fhEQv7BJFBUuUGpl1iQZcNKMbL/q\nHeP33pFErqe3HyitwikaWxZsh1HJXzVK/d5VlRCt3K5z4/9OFrSNaJqr+v/MmjWr6GckpUPlQeNZ\nQB0DfIn1C1/4AuB6PykhA1xkiu7JLCdb+LzxxhuAGy+4BJNyNaxqqP9dbRiG8SHAFlPDMIwESEzN\nV2iTjNxHHnlkWZ+XSF6sUk0WkKNphx12AFx4RZxJQuNX5SFwRnBVvEmz3mUtkEroG/oV6D59+nQg\nvn5DFsYtNV3mDIXg+aqhHC8y2fh1WFWhKAsmi2LIGfjMM88A7rqbNm1asI//OsqKFSsAN+40mz1W\ng8Lc/EaV5TSITAqTTA3DMBIgV4m0kMvl1gEra3c4JWkrFArld+3rJzbGVPgojNPGmCADYZwVLaaG\nYRhGPKbmG4ZhJIAtpoZhGAlgi6lhGEYC2GJqGIaRALaYGoZhJIAtpoZhGAlgi6lhGEYC2GJqGIaR\nALaYGoZhJMD/A/8aY2yG4odpAAAAAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "plot_conv_layer(layer=layer_conv2, image=image1)" ] @@ -2301,20 +1875,9 @@ }, { "cell_type": "code", - "execution_count": 64, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAADuCAYAAACEaORrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXeYJHWZxz/VM5tgl7RLUMKOAktmYZWc4wICkrPIwSMI\nd6eIgiei8oiCeCTlgEORU1Y8JUnOyURaliw5LDnsICC7wIbpuj/m+Va9VVNdUz1TXd0c7+ef6anQ\nXb8Kv3rzG4RhiOM4jjM8au0+AMdxnP8P+GTqOI5TAj6ZOo7jlIBPpo7jOCXgk6njOE4J+GTqOI5T\nAj6ZOo7jlIBPpo7jOCXgk6njOE4JdDez8YQJE8Kenp5SD2D+/PkA1Ot1AEaMGAFArZac52fOnElv\nb29Q6o9n0IoxNsJmnwVB8P9qjH19fUDyOgZB/9BmzJjRG4bhki09APLHqXO/YMECAObNmxetmzt3\nLhDfm/Y6jRkzBoCFF14YgO7u7EeoE66lnikdv66JXSZ0bdLbQTxGO9Yq71fIH6eOV9fLjkWfdZ2z\nzoHu0ZEjRwLxHKT9i46zqcm0p6eH+++/P7Hso48+AmD27NkAjB07FoDRo0dH22gAXV1dDb/71Vdf\nBWCRRRYBYNy4cYn1n//855s51CGTNcY0H3zwARCffGj8UDVDJ40xD002jz76aLTsU5/6FBDfvO+9\n9x6QvI7LLbectnlxyD/eBBpn+qVl0bV85ZVXomXPP/88AHfffTcAN91004D9Dz74YAAOPfRQAEaN\nGpX43qqvpR2jzr3Goev19ttvR9vouZ0yZQoAn/3sZwd899NPPw3EL5plllkmWjd+/HjWW2+90sYx\nGD09PUyfPj2xbM6cOUA8lgkTJjTc/5///CcA119/fbTsnnvuAeLzs/nmmwOw5ZZbRtssvfTSrL/+\n+oWO0dV8x3GcEvDJ1HEcpwSGpJtau8Ozzz4LwGuvvQbE6r1Vpx588EEADjjgACBbHF922WUb/l6n\nVbaaOXMmkBzHUkstldjm9ddfjz5LBc5j/vz5HTVOqU7WXCNeeuklAG655ZZo2eTJk4HYpii7k71X\n0ueoKtKqvWWhhRYCYNKkSdEyfd5hhx2ApHr77W9/G4Czzz4b6Fc/AXbcccfyDngI6HpBbLJ4//33\ngdgcteqqq0bbyPyy9NJLN/xOnYd333038X3Qrxq3436195Ps1jIt5iEzon0WZb64+eabgXgOs3PR\n+PHjC4/TJVPHcZwSGJJk2tvbG32WE0Kz92c+8xkA7rzzzmibn/70p0D8Ftx3332b+r08yaJKJJHN\nmjULiB0qlnfeeQeAk08+OVq29tprA/CVr3yl4XePGDGiI8b5xhtvAPDb3/4WgAMPPDBap+snbcRK\nNZISdI50H1jsffNx4sgjj4w+636fNm0aALfeeisAG220UbTNYostVuHR9SOnE8Cbb74JxI4VXZvl\nl19+wH6/+MUvgNiZA/CNb3wjsY3GY+/PDz/8MIoWqJIijt4TTjgh+nzdddcBcPjhhwPJa7nFFlsA\ncNRRRwFw4YUXArDWWmtF26y++uoDohsa4ZKp4zhOCQxJMv373/8eff7www+BOMRC9pq//OUv0TYK\nSzj++OOBpN1C4QgfB2R3kfSWZUtRzNqMGTOiZRdccAFAFGIh+2Inoje4QmWsnVNSmGxnVsNQHN/L\nL78MwHPPPQckbZHp8KFOQhoFxMcpe6rllFNOAeCxxx4DYmn7rbfeirapWjKt1+sJyVQ2zsUXXxzI\nDtOS1CoNxD6vOhc//OEPE/ssuuii0Wc901UShmGu9vbNb34TgDPOOCNaJs1q2223bbjfueeeC8D/\n/u//AnD55ZdH6/bbb78ofnUwXDJ1HMcpgaYk0zAMWbBgQeItuOSS/Yks8vrK9iJJBeD73/8+EHv1\np06dGq07//zzAfjSl74EdI59NAsdv2xFWW8svb2txK3gb/3tRMn0oYceAuCaa64BYruoTbT485//\nDMAmm2wCxLY4i9bputr7YOutty77sEvDJjHoXt5ss80GbKfru/rqqwOxliJpsGr0TFqbp2x8eedb\n0vePf/xjIGnP1ndJ+l5zzTUz909nKbaaRr+npArZhB9//PFo3WqrrVb4+2VPPe+886JlzzzzTCTF\nD3p8hX/JcRzHaYhPpo7jOCXQlJpfr9eZM2dOwjCvgGY5JWSYtvmtcjyJPfbYI/p8ww03AHFA8fjx\n4wFYccUVmzm0lpJWe3RsebnAUnchDhNSHngncvXVVwNxWEjW+Zfqv/vuuwNx8gLEwesaqwKgbT64\nNe90Crqmd9xxR7SsSM65wuLkeJIjtmrCMKSvry+h5ucF4qfJMmUIW68gzahRo9puktPzpDnj6KOP\nbrit7kNtm4UcWNdee2207OWXX04UwcnDJVPHcZwSaNoBNW/evChlEOLwi3QqVp5kKckGYongiSee\nAGDllVcedP+q0VtNkliRN7+V3uWsSFfC6iQkhRxyyCGJ5Vb6lNQpJ8zPf/7zaJ0NR4E4vdE6K/Oq\nhrULOQVtNaEi5Qnl2FCSQrsIgoAgCBLnNk/6Ei++2F+4S84VG8ImspJSOgmlLOdV6FKqqO7Dvffe\nu+G2Om8bbLBBtGzu3LmFkxNcMnUcxymBpiTTWq3GmDFjopqjEAfwq9jATjvtBAwsMGtReiXEwcJP\nPvkkAOuss04zh1QJCiORXeqpp54adJ8lllgi+qw3XVaKZaegY7PpdgB33XVX9FnhXgrqvuqqq6J1\naclUEo89D50omUvKfvjhh6Nl9957LwCHHXYYkC1RSztROE47UkhFd3d3ojiHLW7ciHTxkizJNI21\nHdpavlWS5XfIG6/uS9nB8yRTseGGG0afu7q6Cp1PcMnUcRynFHwydRzHKYGm1fyxY8eyyiqrRMtu\nu+02IFaBFWqR52yw6q4qMMlhY00AnYIM03IkybShUCGAlVZaKbGPzbxQdklWJomougJPvV6P8ugB\nPve5zwEknIsAK6ywwoB9Nf68kBGZRqz6WKTuZNnU6/XcTJ199tkHiOuTQhzmpEwaW0VISN389Kc/\nDeSHybWSIAjo7u5m4sSJDbdRDQkbyqR78U9/+tOgv6EqVFbFrtpkFYYhc+fOTTj8FJaZVUNBSGX/\nxz/+AcStSiDOztt0000T+9ixzZ8/39V8x3GcKhlS1SgrvcjwftFFFwGxk0FvfBhYRd86sBTcrzzi\ndgcCZ6Eaiqq8LuO9At0BjjnmmMQ+VhpSmFdeSFWVVcvr9Tpz585NBHqrxkIa+9bWNb7xxhuBbGO+\npFVJSnlB4VVQ9LzaqknK088KkVKIjc6dwoc60bkmVA3JVu3ac889gWL1EiSZNZMMUDYKy7T1MPIk\nUnHiiScCcZcPG853xBFHZO5jq2ONGjXKJVPHcZwqGXZ/4v333x+Ayy67DIglNDu7qx1uFl//+teH\newiVofAKhZPI3gtxcLDqf9r0QkktkmokzVs7aZUVeIIgYMSIEQkbZpHfl41Y0ud3v/vdAdvInqrU\n0U6wgTcb0pMXBC7pTiFfut5ZFbQ6BWkHZ511VrRMoWuS2LKQVC+bqe0hVTVBEDB69OhEPy6Nocg1\nVXKQbdusSvtprAS/zDLLuGTqOI5TJcOWTGW3UN8Y2RNVlR3ilFN5EK3txdpPOx159fWmVwotwOzZ\nswF44YUXgGQnx7TXUW9S+8Yr+vYrA3mAbdqhoi8kPVvNQmy33XZAbEfNSluUd1vXuB0efEutVkvU\nVB0uklrkB9A4OzFVViixQNcP4iQFFXqx3S/UWTedatpOf4a0KZsEouOThqciS7YzsJ5RpQuvu+66\nDX9Dz6y998eNG1f42rpk6jiOUwI+mTqO45TAsNV8sf322wOxKqvwGYDp06cDcbM1GwQtw70auLUr\n+LkIUs+l3lrVXEHB+mtDjeSkkPogtaFI29pWkuU0UbWnLIqEOencFG310GrSFZWk8g/13EvVVTJD\nu3LUh4JC+9Kf0+gaFm1xXBVhGCauZTo0So5ee22baVsiE46tszBy5MjC5g2XTB3HcUogaCZYPAiC\nWcCLrTucXCaGYZgdWV4iPsZK+CSM08dYIh+HcTY1mTqO4zjZuJrvOI5TAj6ZOo7jlIBPpo7jOCXg\nk6njOE4JNBVsN2HChLBI58ZWMHPmTHp7e1uez5Y3RqWt6a+NecuLRZOTTzGc2i8do9gJYxwq6eLW\nOh9Z52XGjBm9VXiBW3m/Kl5VRVTSqcEf52tZlKrGCPnj1LVQbLONj9X9p9hTW8RE67S/nst00Z+i\n42xqMu3p6YlqPZaFKi8pR18Dsg/nyJEjcyv5lEnWGDUJqpFcunKQRYkJNvlAtS8fffRRIH7wVN0e\n+mvEtnOMVRIEQSUhLq0cp2oY3H777UCyLfKUKVMS7YJbSdYY9ezkVQJT1fy33noLIFEnVPUlVGeh\nUYeIqu5XyB6nroG6BTzwwAMATJ48OdpGCUJKmMmq2/vGG28k/reVqaD4OF3NdxzHKQGfTB3HcUqg\n0uTwF1/s1+7uvPPOaJnya7faaiugc3Lzrd1F5fWefvppIM61t2r+NddcA8CPf/xjAHbbbbdonfqv\nq8SX7Dc2f93acpzO4plnnok+r7zyykCsNuqaqpQd9Je8s2pzFdjkm0bq/S233BJ9vvjii4HY5GTN\nFGpLJBOAajhU3UQvC3teH3zwQQAuvPBCIK7vsfvuuzf1nVLrZTawNJPU5JKp4zhOCbRMMrVvQb05\nJH1uvvnm0Tp5tPMk0nnz5lXacA6SDjA1W7v33nsB+M53vjNg+2nTpgHw1FNPASRa76YlBUkzKqQN\n/W//dqb2yiEhZ9nbb78NwF133RVt84Mf/ACAV155BYCTTjopWnf00UcD7S8GPVSsc0MFhaVBSDOB\nuNC57le1+FbLc+h3aFQtmeZFk5x22mkAHHvssdEyFXw+6qijAPjCF74QrdO9qwLSqoSmItPQvopn\n9plRqyBdryuvvHJY3y1tw/6GrSA1GC6ZOo7jlEDpr5df/vKXABx++OHRMtmZtMw25lKrgTyaqSlY\nFrYliWy8G220EZDd0kNv6vPOOw+A/fbbb8A2Og9/+MMfgORbL90Ouwps7VK1elBNSF0je1yKx1Pr\nYDVRhPg6qi2EQlIahdV0CqeccgoQ28MhlsjUovtb3/pWtC5d71XjvPzyy6Nlzz77bOX1XK1Wo8+6\nby+44AIgrjkMcN111wH5EuaUKVMAuO+++wDo7e2N1qXDh6ri1VdfjT5Lm/ja175WeP88P4UaB1pN\nxCVTx3GciilNMpWnXm9BazM84ogjgNhmmrWf/u68885lHdKwsJKKbJxpadO+qRWQf+655zb8TlX9\nluRjpYLtt9++cunb2vUUVaFIBSFJFWLNYqeddgLg/PPPj9bJzqQ3vySITpVML730UgBGjx4NwMYb\nbxyt+93vfgfE7bplI4bYYyx0fuQBh34veJlN/IpgJS4Foesa7LXXXgD86Ec/auo7ZetX5pEdYzvo\n6+vjpZdeiv6XZHnQQQc13OeJJ54A4rbkf/zjH6N1emZ1j2a1aW8Gl0wdx3FKwCdTx3GcEihNzVf4\nkNSbfffdN1r3zW9+s+F+cvQoAFeqsIz/VROGIfPnz4/y8CF2HKXZdttto89SqR555BEgGf6V5qGH\nHgKS4WBVqvhhGLJgwYKEmcH2Tbfcc8890WeFBa2yyipAnIwAcSiZzAVyWnRqYoJqQMh0YQuV/PWv\nfwViE4AN91LIWDrHO8spWRVhGCZUUznJdE2t42koyNmk8Dnov65VhvLpubSB9Wusscag+8nRJtOH\naoDAQBOUnkFr1ll22WULj9MlU8dxnBIoTTKVBKJKM0XeGnY/SXZKy2unZDp37tyEpGKdExal2wHs\nuuuuQPz2VkAxDJQM5Kyxb8mq6evrK+QkefbZZ6PPX/ziFxPrbEk0XcfnnnsOyG8Z3QmonXFWu+s9\n9tgDiCVpK3VKk5LUp3NonTPjxo3LrdhUNrpnhX5bGsTSSy/d1PcpREjnRpK5HdP8+fMrl0wXLFiQ\ncIgqVDGNva91jNJE5DzNQink9np3d3d7q2fHcZwqGbZkqtlc9j+lnK2wwgqF9lewt1LW2t0tNQxD\n6vV64g2oQPQ0m2yySfRZ49Y4bGKCkNQmyXT11Vcv56CbJAgCRo0alXiDS5KUnVDYUCB7TtJIinv8\n8ceB+DyoeLLdpp0onE1FO3R8tlB3WgK/++67o8+S2lT8RJqIlUyXXXbZAYW/W0kYhgnpSbZwm/6Z\n5vnnnwcGhnpBbDO09XYheW8MNXxouNiCLI2KRVtfgEIupUWqEFEWei6tjbwZ27BLpo7jOCXgk6nj\nOE4JNK3mh2GYyKdXCJGM3Ouvvz4QO6IafYdQ6I2yjKwY3w5qtRojR45MqD9pR5HUAZvRpdAgmT2U\n12xRTrucc7vsskuJR9481vkitU+ZX7p+zdaXVRaJVMI800CrqdfrzJkzh7///e/RMqn1MkOpJoHN\n2Etj6+9qPFIlVdPWnqflllsu4cCsAuscymrNkUb365NPPgmQCAW0pplGjBgxotJwviAIGD16dCJM\nsUg4mswteh7zQhZ1DmwI2CKLLJKobZyHS6aO4zgl0JRkqvAE1byEOCRGoQd64+dJpuecc070WQHS\n22yzDVBtk64s9AbMMswLVZKxDqRLLrkEyK4/INTwS7U/03nw7UTjVe6yatDaxAQ5WfIcG5LY1l57\n7QHrir7hyyQMw0jrgVjyVo63qj5loTqgF110UbTMJqNAdqO2T33qU5U6oGq1WpRQAcVqjWqb3/zm\nN0CyGtP3vve9zH2swzIIgsol0+7u7qaTIw4++GAA1llnnUG3lRPWat6NklmycMnUcRynBJqSTIMg\noKurKxEGku5ZrXQ7W91F9lSFmChECODQQw8F4F/+5V+aPvh2Y0OmJGFnBQUrEUHB+2npBqpPz2vE\n1KlTAbjxxhuBuLo+wOmnnw7EoUP77LNPtE5Sp9L9mnmjt4parcaYMWMS9lBV7NJfHa+tX/vrX/8a\ngN///vcAbLbZZgO+W/e00mdtfc+qg/YltTWDnlNVDtt///2jdY16PVnJtErJezA09yhcy9YgLSKR\nCl1Lq3mPGzcu0roHwyVTx3GcEmjam1+r1RJBrZJI9NbS7D5jxoxoGwVKS1qxVfhV2OTjiLXfnHDC\nCQ230znabrvtgGwPd9G3X6tR8YczzzwTiG2nFtnDVSsS4uuvcyJvtn3LZ6VutpogCBIJJApEV2C6\nepVZO7CiNxTgbW3j6YB4SUG2U22nYvuaaUw/+clPBt1P9286oaNT0Lik2TVbXEdapSI7rN9m9OjR\nnk7qOI5TJT6ZOo7jlMCQcvOtsVtqjtQ6idy2apDCRrLCpiSaV92yo0omT56cudyeo2bUiSqQun/G\nGWc03EaONYCZM2cCce1X3RftUO0taeeMVDiZJa6++mogGaiuseuvNcvIHCMHlEKSOqHuwGBYp1gz\nLUg66b7MQtdXwfY2AUGfZ82aBcSt2CGusyATje6NoTYLdMnUcRynBIJmwnGCIJgFvNi6w8llYhiG\ng+fJDRMfYyV8EsbpYyyRj8M4m5pMHcdxnGxczXccxykBn0wdx3FKwCdTx3GcEvDJ1HEcpwSaijOd\nMGFC2KjvSquZOXMmvb29LQ94yxqjnHSKTVRxCFtSToUfhhNv2Alj1NgUL2zjL/XZpiUKxRkrJlFx\njFnnY8aMGb1VeIHz7leNIWssiiUdTnxlJ1xLXS+VIFRRc4DZs2cD2amiihFWarCuoY3XDYKgsjFC\n/jjTKe26hyE+B1pnY231zOqv7mGb2t3MOJuaTHt6erj//vub2aU0qqpzqjHaKId0EzU9gFtssUWp\nv131GO1EorxkBTmrOr2qKkFcCUw3pH249MDp5tVkapvTKaA/CIJKQlx6enq47777EtdSlYU09la0\n267X61HHiVaja2krOuk6XX/99QBcccUVQHyNIU4k2WGHHYBkdSXV0NCkmvVCrHKM0D/O6dOnJ+5Z\nvSSUMKJmjnaOUu0FJZjYseh5U1Uw1SK241p00UVZb731Ch2jq/mO4zgl4JOp4zhOCQwpN78IVrW6\n+OKLgbhpmy1pJhE6r6FZO7Cl46Tmy4aU1ZIjD5V7k4oiNarqpmtprAqvY1POufLSh8qVV14JJHOh\nGxUdbjVWBZY9zJaRHA46b9ZOV3WefrrJpQpd65m64YYbSj+uer3elmLm1oav+1emG9VJ2HnnnaNt\nDjroICC2F7/xxhvROtmN06UFVTgbmmsI6ZKp4zhOCZQumcr4qxbOEDdp0xtzo402itZNnz4diCu1\nHHnkkUD7C9HqrQWxV7eIRKqxWseGHDUrrbQS0H6JFOLmiCKvSd5Q2G233YC48C4kpYIqsRJUkWpJ\nL77Y7x9T25K33norWqcGg+lWJu1s6RGGYSSdQVydbdNNN23Zb3Z3d1deTSp9z+o8S7tttnVLGs1P\nVsqfPXu2t3p2HMepkmFLpnrr682ocAMb5iMpRaEM1iaht8Bll10GxG11bWuTqqnX64k3/SqrrDLo\nPnvuuScQh5jsvvvu0Tq9QTXW8ePHA0kbVpVv+TAM6evrK2Q3tI0RdU5kf7KNERUutd9++yX2t9KR\n/a6qCMMwETeolhZp+6HVpE499VQgtvva1joK71L4jNr0yK4O/XbZrPjVViBpzWpCzdj5fvaznwHx\nPQlx48dOakUehiH1ej0hfZbdtFF1lm0sajONLl0ydRzHKYFhS6aSqPSGzmK55ZYb9HtkM502bRoA\nL7zwQrSuSi+w3oD27dTIC2rb495xxx0AHHjggUDSG66xSJJ79dVXgWSg9IQJE8o4/KbIskvrLSwt\nwjajE7KD2oiH733ve0Ac7H/SSScN2C/ru6ogKyNGEo6k1s9+9rPRNmp8qIwbq5lI0raSnP0+6E98\nsLa9Ksh7/oS1/e61115A7LNQwDrEUSuSvhXh0QlU4Uux13bevHmF23a7ZOo4jlMCPpk6juOUQNNq\nfhiGTTtLpDLmGcanTJkCxA6oGTNmROuqDvau1+tRAHAWDzzwABCHzgBsvfXWAJl5vDr+P/3pT0Dc\n3Ms27qpSzQ+CgK6urkTIh9TUIjnrUnWlDkLsgJHzJkvNbxdWzc8qZgFJ1U4hcDJLWJNNo8Ip9plo\nxmkxXIIgKByKZR2AW265JQCbbLIJEN+/ABtuuCEw8Hm1TrWiqm9ZBEFArVZLnGeZbIqM/9FHHwWS\nz1kjB5a9N/r6+twB5TiOUyVNS6ZFpVIFwNp9ioRsSFr6xz/+0eyhlUYQBLmSmdJj1cIa4gpSeeEa\nckooDMe2gB5u+mazpFsgC72VpRl87nOfi9Y9++yzQDwO67Q5++yzgbhCUScgZ6KVqBo5MKw0Ioej\nEhnyHGeSWmygd61Wa0uq5WDYcD+NTedDCSXQ+DmtWhod7PeLSKQK45PzrUiVMPs7I0aMKDznuWTq\nOI5TAsMOjVJ4zMILL5xYft9990WflYJX6IBStjuAjz76qFIb1GBpaZK6bcEW1X7M2/eJJ54A4oIv\nabtdJ6GUSiuZSnq59tprgWQK5aqrrgq0NoVxqNh7SZpP3rmXxqFCGnkSmWrc2lqhnVa0R9hnVMer\nZbZwdCOs9J1+3juVBx98EIjtxc2GeY0ZM8ZDoxzHcarEJ1PHcZwSGLaaLwN2OkxBaiLENS2tytgI\nZZtYR878+fMrNegPZnBWqJMNLbLOmEbcdtttQBye066MIGBAllcaVc2yVZ+kwus6XnDBBdG6qVOn\nArDGGmsAcahUu7Nn+vr6EuYIOSJ032aFpDXT50znyZ7LsWPHdqQJx45Vpgxl4a277roN9/voo4+A\npFN56aWXLlxNqZ3oPtR8ZGv4lu30dcnUcRynBIYtmeoNrL9689vc+m9/+9sAHH300QAcdthh0TpV\n4Xn55ZeBWFqzAe0jR46svHZiHqplaeub2ureaY477jggzs1XRax2VZ4Pw5D58+cPCAGBOCRNUsg2\n22wzYH9pDcrHt0yaNAmIm5y1WzLt6upKSIkffPABAL29vUAsWcqBmP7cCN3nakBoKywttthilUmm\nYRjy0UcfRYkg+n0YGOJkNSHVDS6CnFNWwm9HmFRfX190viFOtMibG1TFTPOLrdTfCJvksWDBgsIV\nwFwydRzHKYHSK+2rRqa1z6iG5FVXXQUka50qxEKSgiQkGxDfTOBsFcjOlFc5Xq1lAW699VYAdtll\nFyCu2DPcyuDDoVarJSQN2aQlcWlsOnaAL3/5y4N+r66nrYjVLpQ2a+u2pls9K0To4YcfjraR9Kp7\nztoK5RNYYoklgNi+atNRR48eXbnkJrsmxJW7JC0XsednoXMl6d2mD48aNarSMaoGr+YSiH0xCsvL\no5lOEnauacYu7JKp4zhOCZQuGqmu4gknnBAtO+aYY4A4WN166rW9bHWysVm7VbvT2NJIKrHoLa6i\nLjfddFO0ThKpkhea7W5aNpLY7HmVZqBaq1qnawdw1llnAXDmmWcCcZEMiO1MepN3Qp8rYdMOpfFI\n+lAihaQ5iKVUnQN7v6o277LLLgvE1dntPVGlFhUEASNGjEg8L/JYK3FGERlWqtNnnRtbm1bB+Suu\nuCIQ28Ftymk7tMVarZZIFtCcIS2wLO+8PU+eTuo4jlMxPpk6juOUQCUeEKnyeWK4VAmpiY1ahXQC\nOjbr2FDoxSuvvAIkW11IJZRDIKvaTdVVhmq1WsK4LvVXhno5kJ5++uloG4V23XvvvUBSnZWaKcej\nQlg0duiMWgQK1pfqLqeKdWLsuuuuQKzuWceLwo30V/dCO01RXV1dCTVfx60wIFUAUx1eiFV/qcr2\nflVtU4Un6lzZNtntqGfa3d2dUPN1XHrm/vznPwPJugFyNGZVx9J4dA5kqrKVxcaMGeNqvuM4TpUE\nzUhEQRDMAl4cdMPWMDEMwyUH32x4+Bgr4ZMwTh9jiXwcxtnUZOo4juNk42q+4zhOCfhk6jiOUwI+\nmTqO45SAT6aO4zgl4JOp4zhOCTQVtD9hwoSwaBVyGyWgYFgFvw4lb3vmzJn09va2PBm4mTGWTSeO\n0dZyTOfcCBi1AAAdc0lEQVTf2yB8XdMiwdwzZszorSKkxq9lP3oW7TOpz3omhxKEX9UYoTXXUve2\n6mrofrZJNbVarfA4m5pMe3p6uP/++3O30YP2+OOPR8uUUaBC0EPBlu1rJVlj1CSik51146l03R13\n3AHA4osvHq1bbbXVgMHLgLVzjGlUds1mzahIsK6nLZOoMm82WwiSD7Ae3CAIKokXLDLOVlHltbzv\nvvvakoFV1RihNddSWXr//d//DcTP9e677x5tM2nSpMLjdDXfcRynBHwydRzHKYHSC51IFFcFc4Dd\ndtut7J9pObYIiMT/tCplO7CeeOKJAPz6178GkoUj1Pvqq1/9aisOtVRUsV2dR61qtdVWWwFxd1Jb\nAKJRMQjbc6eTiteoy4Dqmaq6PsSFMlQP1Jo61P9r3333reQ4neGhojzqDKyeUBCbplQw5frrrwdg\nkUUWibZZbrnlvAeU4zhOlZQmmapPzu233w7A5MmTy/rqtmDfRunoA61TSTqIy7bJAWMrt9sq5p3O\nxRdfDMCpp54KwHrrrRet23zzzYH4jf7SSy9F6+zb3GLPXdE3fBXcfffdADz55JNAsmeZyus9++yz\nAFx00UXRunPOOQeAZ555Bkh2lHA6A/Wag1grVgePo446asD2hxxyCBCX57v22mujdW+88UaiW2ke\nLpk6juOUQGmSqexKKpJ86KGHNtz2V7/6VfRZkk+7+yKlyYuFlT3VFodWP6Sdd94ZgKuvvjpaJ/ux\nCvJuuumm5R5sCbzwwgtAfG3Uy+q4446LtkmHtqXDoLKwdmbF87UT9bhS0WsVr95pp50GbKueXVOn\nTo2W6fwUGXtVpG356uGkELasYuTnnnsuAP/6r/8KJO3ZZ5xxBpAtxbUb68tIFxvX8/WHP/whWqZ5\nSOFOtoh2Gj2XDz30ULTs+eefT/SEysMlU8dxnBIYtmQqr6g8n2+//TYQ2ygs8n5PmzYtWnb66acD\ncNlllwGw+uqrD/eQWo6kVnWqhIGe6j333DP6fMQRRwCxx7gTJdO//e1vQGxLlLc671izJJ48dK+0\nkzfffBOIr52VOhuxxRZbRJ/lzdf3dArWrietIu/6SJqVfdjas9UGRBJZJ0Vh2OgQSamKyJA3fp99\n9om2kcah1iRF2HLLLaPP8gUVwSVTx3GcEvDJ1HEcpwSGreZL3VEo0Pjx4xtuO3HiRACmTJkSLTvz\nzDOBuLPgx0HNF4899lj0WepHVjGGww47DIDTTjsNgBtuuAGAHXfcscVHmI91CD344INArBoeeeSR\ng+7/1FNPRZ+V5yynTRadEBol9Xbdddcd1v4K9O4UrGNGqrvtsgkkHCk6foV2yTQAcQhcJ6n30H//\nWPOKQtdklpCzyXYgFQqXUsIJNK4tYB3LY8eOLVz3wCVTx3GcEhi2ZKpqSXoz2v7jjbCplnr72V7X\nnY7Ca+SsgXxJZf3110/8L0N5uyVT+5afNWsWABtssAGQdLqkkYRz5ZVXRssk0eZJpu3qLW8lYoU0\nNes861TCMGTBggWJ6lyNEijseVhhhRWA2Jna3R1PBapylsf7779fqaahcUoKhfg52njjjYFsiVRJ\nGXJSKeEkDxsWueSSSxYuGeqSqeM4TgkMWzJ97733gLh+pw0XaoQN45CUut122w33UFqObKQKyJed\nEGCttdYadH/ZU20RmHYQhiHz5s1LSKbSELbffvtB99db/uabb46WqQhKHu2SBq3UJru9QmWkXWy0\n0Ual/V5aUmw19Xo9UahF5zkd1D5mzJjoc7r+rH0mdS8olE1jef7556NtRowYUTjNsizq9Xri2bnl\nlluAWJvKQs+c7KBZ2pitywtJLbNWqxW2Hbtk6jiOUwI+mTqO45TAsNV8GaHlXFA9zDysurf//vsD\nsMwyywz3UFqO2o4ozMLWMz3rrLMa7qewKRn5bRhLOwjDkPnz5ydCozS2Nddcc9D9le1mq0Zl5ban\naUeoTb1eH6DuQlwl6rrrrgOSDhhbKWsodHd3N6zv2gpqtVoi7Ek1aJdaaikgVnXtMUm9L1Jj4JFH\nHgGSZq0i5rwyCcNwgOlE1djyqrIpPOwLX/gCELcogXjsu+yyS2KfoTpKXTJ1HMcpgWFLpksssUTi\nfwXS5rHDDjtEn23juU5HbzIZqB9++OFo3e9//3sgzmW30reC2z/zmc8AnSGF12q1RHCyJA1b17MR\nStDYZpttomUK9O40BpMQJYXedddd0bJrrrkGiO/tL3/5y9G6TrtfgyCgq6srkSyj+1IOQiUoqKJZ\nUSQJqm6v/Y2FF1648lC3MAwT2o2uXZbmkUbHbmvTKhwzLZmqcSTkV5lK45Kp4zhOCQxbMpU9Rra2\nGTNmAHDFFVdE2+yxxx6JfRQw/HHlvPPOA+AnP/lJtExvvixbkiQc2VjXWGONVh9iLkEQ0N3dnQgJ\nkc3Uhs+kkX1coShFwsHazWCSqdKXrc30f/7nf4DYVmgrr3/6058GYgle+xcN7G4FQRAkJDZdS0nb\nqtKmZxNg6623BmKp1WopCgG88847gXisthLc2LFjC0mEZRIEQUKr032o41B42EILLRRt88YbbwBw\nySWXALDOOutE60455ZTM32lGGrW4ZOo4jlMCw5ZMNYvLfqZAX1sERNKBJFL7htMbXVJSlh2mygDo\nIshm+u///u/RsjyPolLaNG6lv7ULSabW3i17rgKxdR2txKZrc8ABB1R1qJUxadKk6HMjiQUaB3p3\nEgrIVyKMAtWtvVBddNVpwNqC9bwqrVSJNdbzP2rUqEojFoIgYMSIEZFmAHF3DhVpUS1lK6UrKUPS\nugr6tAKXTB3HcUrAJ1PHcZwSKK2hntQkqYJWzVdlKakb1kAsB5YCyKVK2CpSnVrhJ6tKTRY6/nRD\nunYSBEHiOsjJoOugIHCr5judqd739fUlnEG6rnK26H9bv1MB+KqtYXP7ZY5S7U+p+dZJVavVKlfz\nu7u7E+FZMmfI9KJ6Cza0Sep9kWdVyTVDnW9cMnUcxymBoBnnThAEs4AXB92wNUwMw7DlYoGPsRI+\nCeP0MZbIx2GcTU2mjuM4Tjau5juO45SAT6aO4zgl4JOp4zhOCfhk6jiOUwJNBRFOmDAhzOoLD3ER\nDKUh2sIPZcSjzZw5k97e3pYHtuWNsSxUHNqel1qt1tYxyhGpvyryrdg7iK+x4vBs2l4zhT5mzJjR\nW4UXuMj9qvRZ64hVbG1ejK22TxdHh/7r2gnXUsem/9PHaBnKM1rVGKG55zKva6pdp9jcwcZedJxN\nTaY9PT1RFW+hQNkbb7wRiPPQ99xzz2ibyZMnF/6NZ555Bkg2tRo7dmwi4LiVZI2xbJQvPG7cuGjZ\nmmuuWekYp0+fnrixFKSverQKhLbdBO69914gDuq210iTqaoQKeA7iyAIKglxybqWCk5//PHHAejt\n7QWSDQGLdAT45z//CcTdBmwd2GWWWabya2lfBgrET7+0bTO62267DYCHHnoISAaqqxqYKmLp/3TC\nQlVjhOaeS927MPCYbVNA1eVVXQr9VXV+6J9wi47T1XzHcZwS8MnUcRynBIaUeG2bd/3xj38E4IQT\nTgDiFgBZqr1y820JvjTKt60y77coKjQr9diWAxOvvvoqEJc2y/se1SyA/nHn2Xpage17LnONrq3K\nmUllBNhxxx2BWKVdccUVo3UqQLz33nsDcYHlLbbYoiXH3gy2gaHMSI8++igQ36fNNvvTeZLqbHPj\nF1988crLRtpGlioHqWdIJg2p9BCfB5lxZHqy37XbbrsBcXlNXX9IXvt2I9X99ttvB5K1PxZZZBEg\ntn+rhRDE85DUfRWetnVBmsElU8dxnBIYkmQ6c+bM6PPZZ58NxF78X/ziFw33+6//+i8ATjrppIbb\nZLVCmDNnTuVSm0Vvs+nTpwNxa5IsyVQSXp5kKqO4lQZWXnnlhOe81YRhmPDu6s2tsUma22effQp9\n33HHHQfETim11N1ss82ibapuwCZmz54dfbYONYApU6YMur+VbHV/6q8kdxvNMH78+ErbeYdhmGjb\nrWORZKoiylZbVGFwVY8688wzo3WS2vWc33PPPUBc4S39uV288MILQNxAUE5EKzVrXL/61a8A+M//\n/M8B3yOn62uvvQYkm4TW6/XCWoZLpo7jOCUwJMnUNubSW+E3v/lNw+333XdfIG5QlieZZtGOtrL2\nTf/AAw8AsWR60EEHNdxPdmGFngAcffTRiW3UDvrSSy+Nlj3//PMJW3QrCcOQvr6+RDhMuv20DXtK\n8/LLLwPwta99LVomCfZLX/oSAK+88grQPmnUovAniCUx26a6Eb/97W+BZANItbSW9KJzqHMC/RqL\nNLUqCMMw8XtLLbUUUMzvIC3p5JNPHrBOWuYTTzwBJG2ua621VqWalLDhXbLr77777g23V+iamlja\ncEQ9q2rZIm3C/sayyy5b2H/T/jvdcRzn/wFNSaZhGDJ37lyee+65aJnezAcffHDD/eRN3GmnnQb9\njZtvvhlIeoGb9bSWgQ38le3IVvluxC9/+UsAbr311mhZWjIVtq3yU089lfDIdjKyC1955ZXRMgV2\nS0L7zne+U/2BZVCv1xM2U3lq89qNq6Ge7L7/8R//Ea2TZCokwT/99NPRsnfeeacym6m0DBtonpak\nJLVa21+RjLXDDz8cgB/96EdAMmLg9ddfT0SDVIWVhhsF09vrJWQztd0C0tx3331ArFVBvu8jjUum\njuM4JeCTqeM4Tgk0peb39fUxZ86chNpUpEmc+sQrlMjmxypIX1x77bVAsg99noG5FfT19SXGqCD7\nr371q4Pue8kllwDFGq9ZJ9drr71WmdqkHuRZyFkjlU6hUhaF2MgBYLG91duNVGCr3hZR244//ngg\nDhfLUw313Qqrgf6A/k7oYKH6AQpcz0LhRcpLz2KHHXYA4Oqrr46Wvfzyy5WH8s2bNy/XMSqz06mn\nnhotu+uuu4D8ayi0jU1UaQaXTB3HcUqgaQfU/PnzEw6hIhVVJk6cCMRv7zvvvDNal5ZMZci3Ek6V\nkmm9XmfevHkJB5SOe7XVVmu43+mnnw7EITN5VZOEJAfod2S0M4VWTiVpDeuvv/6g+9jgZhnt7bJO\noF6vJ6SSLEkbYueDRRqV7t8sdL7ef//9aNno0aMru5ZBENDV1ZUI+RG6JostthiQTDLR8cqZbMsN\nSvrTMt3LCg2E/nbKVSYmQP+1tI62NHIcbr311tGyjTbaaNDvlTamNNQ8ST4Pl0wdx3FKoCnJtFar\nMXLkSJZffvlo2ZprrjnofpI+FZS+xx57NNxWgf2y01RNEAR0d3dn1vrMQ4HS2267LVBMMn333Xej\nz5MnT84tRtxq9FZWuqUkUzv2Y489FoCf//znA/ZvJPG1m1qtlpA0bN1Ri1IoIZZEFeidJ6UrgcWG\nDS2yyCKZadFVI7u9gtOtH0Lhfrq+9plO2+4lZSv8DfpDo6ocYxAEg4ZIyh6uYjtF0TWUVrXeeusN\n4QhdMnUcxykFn0wdx3FKoCm9squri0UXXTSh9jRSgZUDDXErE6nAMohnoaygItV8WoHChqzaI9VQ\nNSCzwsGUsaVsryKhUdaU0NPT05ZMLyHDfjp0yKqvqtUq55x1sjRSnx977LHocxGTUJnoWhZxitk6\nBRtssAEQ11DIc3pIXbYhO+PHj6/UZNNI3dY9KOeUqiNB3IJGDigb2vXFL34RGFh32F7jBQsWVG6W\nauTUu/zyywE44ogjAJg6dWpT33vxxRcDccbXUHHJ1HEcpwSafrXUarWE1Gar5Vhs+NODDz4IwIkn\nntjwexU8rApT6RzoqllppZWizzJoyzmTJZmqepYkU7t/GiUmWGlmpZVWaqtkao/DYkPUJAGofsJN\nN90UrVM9U0kzqtZj6zhULZmKIhKUpFCIa0HkOTJU41aJC1YamjBhQmVSm8IVbdUoaTyqQyDJ2p5/\n1R1QULvNR5czVbVoVZPCSqbd3d0Jab6dyLGWpc3qeVRzQMsBBxyQ2MbeA0PBJVPHcZwSGNLr0/ZY\nSdfBFDbQvkgQrCSarIov7Ua9bxSgrTQ6+2ZWQLTegKrrmYXqQso2B/2hKZ3wppdUohRaa6fSNdp+\n++0Tfy2yoauvTjNVd9qJTafUPZ0XeK/URVWc33DDDaN1VSdfhGGYSBpQDdm8XkbqeqHqZlnPcbpK\nmk0fXWqppdoaymfZddddgVi6tmnOaYnUatK6hqpbO1xcMnUcxymBIb1arPewSGELpZzKdrruuusO\n2MZKu51KOvXVogDvvGB9dSNVeqPdtlMCvUUjjcNiC0LIRir7lVINi6Qbdxq2zmwadV2QJCjpPM9G\n3mpqtVri3tE1kI0/Lx1WETZFsJrTqFGjOqKLAsTRQXlRQuKYY46JPqtjRl4SUTN0xtlwHMf5mOOT\nqeM4TgkMSc23hmfVbVRwd1aAs0I1iqixUg87SeUtQl5gt/jb3/4GwCabbAIkQ6zaWTGqKDLUX3HF\nFQBst9120To5YhQgb9d1Irpvi5x3VdSCuMKQHE5yIha5/q0gq2qUnICqSqbc8yL1IvKwJr2urq6P\nxT2bZs8994w+77fffoNu30xdWpdMHcdxSiBoZuYNgmAW8GLrDieXiWEYDp6jOUx8jJXwSRinj7FE\nPg7jbGoydRzHcbJxNd9xHKcEfDJ1HMcpAZ9MHcdxSsAnU8dxnBJoKs50woQJoeIJB8M6tlQgQfFv\ntmCCYtVU9kwpa7avexAEzJw5k97e3pYHtuWN0RZzBkpPp+uEMeq6aay2+PeHH34IEJV7s7HASsHU\n9dM6G4uozzNmzOitwgvczP1q0fh0Dux1Llrco5OuZaviQasaIwz9WipuXefCzku6Rwd7jouOs6nJ\ntKenh/vvvz93GwXv2wrrClZXVXJbt1N5+sptV41PG4Q8cuTIynK8e3p6Ei1tIR5TeqIomyrHmG5t\n/MEHHwBx/dE77rgDgGuuuSba5vbbbwfiupg/+MEPonV77bXXoL9rkjcqCXFp5n697LLLomXKu1fl\nL9squihVXsv0GFUzoUjdjCyuuuoqIK6434gq6y4UuZZCHSEgvpa6v+2LRVWxVNlME68V9saOHVt4\nnK7mO47jlIBPpo7jOCVQWnVXichqOnfppZdG69QWQEVcbeHkRjnNtsd3EARN5cgOlyAIEr3Di6j3\njzzyCBAXRbYtSaQWN2o61w6CIEioMyoGLROH2lk88cQT0TZrr702ALvssguQ3Qoij05KEFGR4B/+\n8IdA0iyle3co6n27sI0Prb+hKHb8u+22GwAHHnggUF7x5Fajc/D6669Hy/Tsap2eT4jLTKp4vcwi\ndu7RvFYEl0wdx3FKoDTJVK0CZLxW0zWIK7V85StfGdJ3t7tCTVoivfvuuwH43e9+Fy2TRCfJ3Baq\nVRtoNfw65JBDgPZKPmEYMnv27Oh/VUZS+w41G7NjTHuyrTRz6qmnAvBv//ZvQHbLjE6oBPbuu+8C\ncM455wBwwQUXAHFDQIDlllsusY+VTjphDFmoQhTErWeEnDDf+ta3omVqs/ONb3wDSDbb++lPfwrE\nz7Ke7XQbk05DGrBtaa2C7jo/b775ZrRO11KOUc0x1nE3Z86cAVE8jXDJ1HEcpwSGJJlmxZAq1Gba\ntGkATJo0KdrmhBNOaPo3bOuIqtsjhGGY+E1JbZLEZB+dPHlytM3+++8PxNLb008/Ha3TOdJfhR3J\n9tgOwjBMSFxLLtkf9ik7aBGp2Uozp512GgDHH388AD/72c9KO9bhYu/Xv/71rwBcf/31AKy66qpA\nLFlbFBYmKRZiSUdty9NSYDsIwzC3nqpaddjQIjWutHZzceyxxwKx3VzthpppcdIOLrzwQgBee+21\naJmkcdUQtpqFwqVsLDUk55t58+a5ZOo4jlMlw7aZzpo1C4Abb7wRiL3CeR5Aa09VsH66JbB9O3z4\n4YeF3w5loSwYiFvISsKRtzNtW7Osv/76A5bJRimPom07u/zyyw/ziIsjqdRK/8OVsGRj1XW/5557\ngGQL5HahexTgL3/5CxBLZLLxZqFtbGSH7gXZIdstmYZhyIIFCzJt1Hpm9Ex+//vfj9bZVuyNWGGF\nFQB45513yjjUlnH55ZcDsQZx+OGHR+s23njjxLa2mafmLD2HNgJHjBgxorC/xiVTx3GcEvDJ1HEc\npwSaVvPDMEyoPTL23nvvvUDslFlvvfUG7CtDrxqyATz88MNAbPTOopnwhLKQGmdRAzyFlTSLnDoK\nkVLISTsIw3BIwd2NUP94FUMp0sO8CsIw5K233or+l8qv+1OJJFlIRZSzCmKVUM6LTiEvIUImpGab\nHMopudpqqw39wCrg/PPPT/wvk1MW9p5XOKOcxlmmue7ublfzHcdxqmRIDigbSvDii/0FgJSCtffe\nezfcT+FT1rmkSkT77rsvEBu9LVUH7Muon/59pZ8NF70dbThLldJ3EARRmt1QUKiY2jpb5HBaeuml\nh/z9ZRGGIfPmzUukF0qCU4vmrPtNyCFhg7hfeuklIHvs7SAIAmq1WuJ+laSl50xagkL6YKBjJgs5\nh20InHj11VcTGmo7UIroo48+CsBaa60FxEkyWdjkhgceeABIOqXSjB49unBopkumjuM4JdC0ZBoE\nQWQXg7hGqcKElCqZhUKDJNlAHJ6gNL8sSWHevHmVFslIB+0LvQkXWmihUn7HSty9vb0J6aLVDEfa\nzzvOPInUSgVVIA3Dps2qwLCC9Ytgba4rrbQSMDCUr510dXXlSolTp04FkrVpJa3mFavZaaedGq6r\n1+ttL1yjuqUaS5afJo3SvSH2i+Rpac0kDLlk6jiOUwI+mTqO45TAsDOgpPLLOZOXEaJ2JVbUFqqV\nmcX777/fVF3B4RCGIfV6PSHeKxRKY5V6YNuvKJvILhsMm4HUaVkmMurbFi5ymG211VaFv0fZN1C9\nmg/95gxbZ0BOJV1L1VvIcrJce+21QNL09N3vfrdlxzoc8kwvcsjYVjUKSVxllVWAZM66rpPqfArr\nIG1nFbc0CsfMcyQJjRviMSvkcbi4ZOo4jlMCQ5JMrcFWUps18jdClebtm/6II44YdL9mwhPKwkoz\n6Yo8ckhYR5rCLGQUtwHSjYz8tj7ouHHjKh1jGIa50oWqI33961+Plsl5s8022xT+HSuZZuWPt5Ig\nCOjq6krkXOs+1XEppM/WRpAzRwH+J598ciXHO1T6+voS95K0uHTtVZsYo+QD1Sy19QuEpNbNNtsM\nSFbzr9VqbZdOpQXuvPPOAEycOHHQfawzUePT3+HikqnjOE4JDEkytZKaZnW9zVXlWiEkFr3ZDjvs\nsGiZ3ippbKD1oosuWlmF8yAIGDFiRG6qpezCtqK3UtpuvfVWAA466KBonaqZSyKXhJZuKVvVGFU1\nykrCaalYx6iKPAB33nknUCwgXxXNbchO1SmmSk7IqgYk6U2hfbbVszSvTTfdtOF39/b2AvGY0l0I\nqqSrqyvhU5DNU+3Ss45NErrasNsUaSU0pMdvtc/Ro0e3XTJViKJspnn+Cknidgyqz5q2DQ8Vl0wd\nx3FKYNg20xVXXBGI0/QUfJ+FbG6f//znB/0NKymNGzeuI3vv2JQ8VZqXp99KQ0pHzJJIxciRIyt7\n0wdBMKADa6O3+uKLLx59LlIDUzbIF154AUimYraj51UQBAnJQ/eRIil031r7tyQ6Sdc2fVrj0nlR\nBXcbwN4Oic0+kzoWSajywttkG92Dqs1ro2nsNbPYMbbrmbQRBXqedL3ytElFEEnqhqT2WAYumTqO\n45SAT6aO4zglMCQ136rgCneS+C0Drw3QlppVpMXDe++9l9gn/Xudim2uNxhSMdsZBF2r1QY0DoP8\nPOUiqIqYVHrrrBrudw8Vq/7puOS8kHPGhszoOKXKW9OV2h3LwdpuJ0wWMtlIDdczqWcVmmuTo7q7\n1kzTTDuPMrH3rK6dnIAyx9gEGC1T2NTWW2/dumNr2Tc7juN8ggiaqfwSBMEs4MXWHU4uE8MwXLLV\nP+JjrIRPwjh9jCXycRhnU5Op4ziOk42r+Y7jOCXgk6njOE4J+GTqOI5TAj6ZOo7jlIBPpo7jOCXg\nk6njOE4J+GTqOI5TAj6ZOo7jlIBPpo7jOCXwf+1R9NvS3jKEAAAAAElFTkSuQmCC\n", - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], + "outputs": [], "source": [ "plot_conv_layer(layer=layer_conv2, image=image2)" ] @@ -2335,7 +1898,7 @@ }, { "cell_type": "code", - "execution_count": 65, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ From c530e37c739877f1aee37338f463b0af2180e51f Mon Sep 17 00:00:00 2001 From: Daryl Van Dyke Date: Thu, 19 Apr 2018 11:02:35 -0700 Subject: [PATCH 3/3] now just helper functions and more --- 03B_Layers_CannGrows.ipynb | 72 ++++++++++++++++++++++++++++++++------ 1 file changed, 62 insertions(+), 10 deletions(-) diff --git a/03B_Layers_CannGrows.ipynb b/03B_Layers_CannGrows.ipynb index fd35e54..3575726 100644 --- a/03B_Layers_CannGrows.ipynb +++ b/03B_Layers_CannGrows.ipynb @@ -451,7 +451,7 @@ }, { "cell_type": "code", - "execution_count": 46, + "execution_count": 48, "metadata": {}, "outputs": [ { @@ -473,10 +473,10 @@ "img_size_flat = h * w\n", "\n", "# Tuple with height and width of images used to reshape arrays.\n", - "img_shape = (w, i)\n", + "img_shape = (w, h)\n", "\n", "# Number of colour channels for the images: 1 channel for gray-scale.\n", - "num_channels = 3\n", + "num_channels = c\n", "\n", "# Number of classes, one class for each of 10 digits.\n", "num_classes = 2" @@ -498,7 +498,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 52, "metadata": {}, "outputs": [], "source": [ @@ -511,6 +511,35 @@ "\n", " for i, ax in enumerate(axes.flat):\n", " # Plot image.\n", + " image = plt.imread(images[i])\n", + " ax.imshow(image.reshape(img_shape), cmap='binary')\n", + "\n", + " # Show true and predicted classes.\n", + " if cls_pred is None:\n", + " xlabel = \"True: {0}\".format(cls_true[i])\n", + " else:\n", + " xlabel = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n", + "\n", + " # Show the classes as the label on the x-axis.\n", + " ax.set_xlabel(xlabel)\n", + " \n", + " # Remove ticks from the plot.\n", + " ax.set_xticks([])\n", + " ax.set_yticks([])\n", + " \n", + " # Ensure the plot is shown correctly with multiple plots\n", + " # in a single Notebook cell.\n", + " plt.show()\n", + " \n", + "def plot_images_ori(images, cls_true, cls_pred=None):\n", + " assert len(images) == len(cls_true) == 9\n", + " \n", + " # Create figure with 3x3 sub-plots.\n", + " fig, axes = plt.subplots(3, 3)\n", + " fig.subplots_adjust(hspace=0.3, wspace=0.3)\n", + "\n", + " for i, ax in enumerate(axes.flat):\n", + " # Plot image.\n", " ax.imshow(images[i].reshape(img_shape), cmap='binary')\n", "\n", " # Show true and predicted classes.\n", @@ -540,16 +569,39 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 54, "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "ValueError", + "evalue": "cannot reshape array of size 4800 into shape (64,64)", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 6\u001b[0m \u001b[0mcls_true\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mobjBoth\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlistLabels\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m0\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;36m9\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 7\u001b[0m \u001b[1;31m# Plot the images and labels using our helper-function above.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 8\u001b[1;33m \u001b[0mplot_images\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mimages\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mimages\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcls_true\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcls_true\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[1;32m\u001b[0m in \u001b[0;36mplot_images\u001b[1;34m(images, cls_true, cls_pred)\u001b[0m\n\u001b[0;32m 9\u001b[0m \u001b[1;31m# Plot image.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 10\u001b[0m \u001b[0mimage\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mplt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mimread\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mimages\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 11\u001b[1;33m \u001b[0max\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mimshow\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mimage\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mimg_shape\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcmap\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'binary'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 12\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 13\u001b[0m \u001b[1;31m# Show true and predicted classes.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mValueError\u001b[0m: cannot reshape array of size 4800 into shape (64,64)" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD8CAYAAAB0IB+mAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAEtFJREFUeJzt3U+I3Hf9x/Hn65cYhRwUbA4lDaTFYgjSQ7K0PRVBhLaH5FAP6UUjLaFofp4LHoRcxJNYLL+SYrD10AZz2oJSKAo9WTOBGhtLZS1IlxaatlIQITXw/h1mxHUy2/m6OzO78/08H7Aw3/l+d+a9ecFrv/n+2UlVIUnqv//Z6QEkSYth4UtSIyx8SWqEhS9JjbDwJakRFr4kNWJq4Se5kOT9JG9ssj5JnkqyluRqkmOzH1PzZMb9Z8aCbnv4Pwce/JT1DwF3j77OAP+3/bG0YD/HjPvu55hx86YWflW9Cnz0KZucBJ6vod8BX0hy+6wG1PyZcf+ZsQD2zuA1DgLvbFheHz333viGSc4w3Htg//79x48cOTKDt1dXV65c+aCqDmzhW814SZhx/20j45kUfiY8N/HvNVTVeeA8wMrKSg0Ggxm8vbpK8tetfuuE58x4FzLj/ttGxjO5SmcdOLRh+Q7g3Rm8rnYPM+4/M27ALAp/Ffjm6Cz//cDHVXXLfwO11My4/8y4AVMP6SR5AfgqcFuSdeAHwGcAquoZ4FfAw8Aa8A/g2/MaVvNhxv1nxoIOhV9Vj05ZX8B3ZzaRFs6M+8+MBd5pK0nNsPAlqREWviQ1wsKXpEZY+JLUCAtfkhph4UtSIyx8SWqEhS9JjbDwJakRFr4kNcLCl6RGWPiS1AgLX5IaYeFLUiMsfElqhIUvSY2w8CWpERa+JDXCwpekRlj4ktSIToWf5MEkbyVZS/LkhPWnk1xP8vro6/HZj6p5MuP+M2PtnbZBkj3A08DXgXXgcpLVqvrT2KYXq+rsHGbUnJlx/5mxoNse/r3AWlW9XVWfAC8CJ+c7lhbMjPvPjNWp8A8C72xYXh89N+6RJFeTXEpyaCbTaVHMuP/MWJ0KPxOeq7Hll4DDVXUP8Arw3MQXSs4kGSQZXL9+/b+bVPNkxv1nxupU+OvAxt/0dwDvbtygqj6sqhujxWeB45NeqKrOV9VKVa0cOHBgK/NqPsy4/8xYnQr/MnB3kjuT7ANOAasbN0hy+4bFE8CbsxtRC2DG/WfGmn6VTlXdTHIWeBnYA1yoqmtJzgGDqloFvpfkBHAT+Ag4PceZNWNm3H9mLIBUjR/GW4yVlZUaDAY78t6tSnKlqlYW9X5mvHhm3H/bydg7bSWpERa+JDXCwpekRlj4ktQIC1+SGmHhS1IjLHxJaoSFL0mNsPAlqREWviQ1wsKXpEZY+JLUCAtfkhph4UtSIyx8SWqEhS9JjbDwJakRFr4kNcLCl6RGWPiS1AgLX5IaYeFLUiM6FX6SB5O8lWQtyZMT1n82ycXR+teSHJ71oJovM+4/M9bUwk+yB3gaeAg4Cjya5OjYZo8Bf6uqLwE/Bn4060E1P2bcf2Ys6LaHfy+wVlVvV9UnwIvAybFtTgLPjR5fAr6WJLMbU3Nmxv1nxmJvh20OAu9sWF4H7ttsm6q6meRj4IvABxs3SnIGODNavJHkja0MvQvcxtjPtiS+vMnzZvyfljVfMOOu+pjxVF0Kf9Jv+NrCNlTVeeA8QJJBVa10eP9dZ1lnTzLYbNWE55rNeFnnBjPualnnhk/NeKouh3TWgUMblu8A3t1smyR7gc8DH211KC2cGfefGatT4V8G7k5yZ5J9wClgdWybVeBbo8ffAH5TVbfsGWjXMuP+M2NNP6QzOpZ3FngZ2ANcqKprSc4Bg6paBX4G/CLJGsM9glMd3vv8Nubeacs6+8S5zfgWyzo3mHFXyzo3bGP2+AtcktrgnbaS1AgLX5IaMffCX9bbuTvMfTrJ9SSvj74e34k5xyW5kOT9za6NztBTo5/rapJjM3hPM14gM+7OjMdU1dy+GJ4c+gtwF7AP+ANwdGyb7wDPjB6fAi7Oc6YZzn0a+OlOzzph9geAY8Abm6x/GPg1w2uu7wdeM2MzNuOdz3URGc97D39Zb+fuMveuVFWv8unXTp8Enq+h3wFfSHL7Nt7SjBfMjDsz4zHzLvxJt3Mf3GybqroJ/Ot27p3UZW6AR0b/nbqU5NCE9btR159tlq9nxotlxkNmPGbehT+z27kXrMtMLwGHq+oe4BX+vXez283639uMdx8zHjLjMfMu/GW9nXvq3FX1YVXdGC0+Cxxf0Gzb1SWTWb+eGS+WGQ+Z8Zh5F/6y3s49de6x42UngDcXON92rALfHJ3lvx/4uKre28brmfHuY8ZDZjyuw9niC8D7bH62OMBTwBpwFTg24WzynxmeLf/+6LlzwInR488Bvxx9/++Bu3b6DHnHuX8IXGN45v+3wJGdnnk01wvAe8A/Ge4FPAY8ATyxIa+nRz/XH4EVMzZjM17+jLu87tQ/rZDkAeDvDM8If2XC+oeB/x39w94H/KSqxv/OtnYxM+4/MxZ0OKRTi78ETAtmxv1nxoJuH4AyzWaXB91yPCkbPiln//79x48cOTKDt1dXV65c+aCqDmzhW814SZhx/20j45kUfufLg2rDJ+WsrKzUYLDlD27RFiT561a/dcJzZrwLmXH/bSPjmVylM+tLwLT7mHH/mXEDZlH4s74ETLuPGfefGTdg6iGdJC8AXwVuS7IO/AD4DEBVPQP8iuGZ/TXgH8C35zWs5sOM+8+MBd0+4vDRKesL+O7MJtLCmXH/mbHAD0CRpGZY+JLUCAtfkhph4UtSIyx8SWqEhS9JjbDwJakRFr4kNcLCl6RGWPiS1AgLX5IaYeFLUiMsfElqhIUvSY2w8CWpERa+JDXCwpekRlj4ktQIC1+SGmHhS1IjLHxJakSnwk/yYJK3kqwleXLC+tNJrid5ffT1+OxH1TyZcf+ZsfZO2yDJHuBp4OvAOnA5yWpV/Wls04tVdXYOM2rOzLj/zFjQbQ//XmCtqt6uqk+AF4GT8x1LC2bG/WfG6lT4B4F3Niyvj54b90iSq0kuJTk06YWSnEkySDK4fv36FsbVnJhx/5mxOhV+JjxXY8svAYer6h7gFeC5SS9UVeeraqWqVg4cOPDfTap5MuP+M2N1Kvx1YONv+juAdzduUFUfVtWN0eKzwPHZjKcFMeP+M2N1KvzLwN1J7kyyDzgFrG7cIMntGxZPAG/ObkQtgBn3nxlr+lU6VXUzyVngZWAPcKGqriU5BwyqahX4XpITwE3gI+D0HGfWjJlx/5mxAFI1fhhvMVZWVmowGOzIe7cqyZWqWlnU+5nx4plx/20nY++0laRGWPiS1AgLX5IaYeFLUiMsfElqhIUvSY2w8CWpERa+JDXCwpekRlj4ktQIC1+SGmHhS1IjLHxJaoSFL0mNsPAlqREWviQ1wsKXpEZY+JLUCAtfkhph4UtSIyx8SWpEp8JP8mCSt5KsJXlywvrPJrk4Wv9aksOzHlTzZcb9Z8aaWvhJ9gBPAw8BR4FHkxwd2+wx4G9V9SXgx8CPZj2o5seM+8+MBd328O8F1qrq7ar6BHgRODm2zUngudHjS8DXkmR2Y2rOzLj/zFjs7bDNQeCdDcvrwH2bbVNVN5N8DHwR+GDjRknOAGdGizeSvLGVoXeB2xj72ZbElzd53oz/07LmC2bcVR8znqpL4U/6DV9b2IaqOg+cB0gyqKqVDu+/6yzr7EkGm62a8FyzGS/r3GDGXS3r3PCpGU/V5ZDOOnBow/IdwLubbZNkL/B54KOtDqWFM+P+M2N1KvzLwN1J7kyyDzgFrI5tswp8a/T4G8BvquqWPQPtWmbcf2as6Yd0RsfyzgIvA3uAC1V1Lck5YFBVq8DPgF8kWWO4R3Cqw3uf38bcO21ZZ584txnfYlnnBjPualnnhm3MHn+BS1IbvNNWkhph4UtSI+Ze+Mt6O3eHuU8nuZ7k9dHX4zsx57gkF5K8v9m10Rl6avRzXU1ybAbvacYLZMbdmfGYqprbF8OTQ38B7gL2AX8Ajo5t8x3gmdHjU8DFec40w7lPAz/d6VknzP4AcAx4Y5P1DwO/ZnjN9f3Aa2Zsxma887kuIuN57+Ev6+3cXebelarqVT792umTwPM19DvgC0lu38ZbmvGCmXFnZjxm3oU/6Xbug5ttU1U3gX/dzr2TuswN8Mjov1OXkhyasH436vqzzfL1zHixzHjIjMfMu/Bndjv3gnWZ6SXgcFXdA7zCv/dudrtZ/3ub8e5jxkNmPGbehb+st3NPnbuqPqyqG6PFZ4HjC5ptu7pkMuvXM+PFMuMhMx4z78Jf1tu5p849drzsBPDmAufbjlXgm6Oz/PcDH1fVe9t4PTPefcx4yIzHdThbfAF4n83PFgd4ClgDrgLHJpxN/jPDs+XfHz13Djgxevw54Jej7/89cNdOnyHvOPcPgWsMz/z/Fjiy0zOP5noBeA/4J8O9gMeAJ4AnNuT19Ojn+iOwYsZmbMbLn3GX1536pxWSPAD8neEZ4a9MWP8w8L+jf9j7gJ9U1fjf2dYuZsb9Z8aCDod0avGXgGnBzLj/zFjQ7QNQptns8qBbjidlwyfl7N+///iRI0dm8Pbq6sqVKx9U1YEtfKsZLwkz7r9tZDyTwu98eVBt+KSclZWVGgy2/MEt2oIkf93qt054zox3ITPuv21kPJOrdGZ9CZh2HzPuPzNuwCwKf9aXgGn3MeP+M+MGTD2kk+QF4KvAbUnWgR8AnwGoqmeAXzE8s78G/AP49ryG1XyYcf+ZsaDbRxw+OmV9Ad+d2URaODPuPzMW+AEoktQMC1+SGmHhS1IjLHxJaoSFL0mNsPAlqREWviQ1wsKXpEZY+JLUCAtfkhph4UtSIyx8SWqEhS9JjbDwJakRFr4kNcLCl6RGWPiS1AgLX5IaYeFLUiMsfElqhIUvSY3oVPhJHkzyVpK1JE9OWH86yfUkr4++Hp/9qJonM+4/M9beaRsk2QM8DXwdWAcuJ1mtqj+NbXqxqs7OYUbNmRn3nxkLuu3h3wusVdXbVfUJ8CJwcr5jacHMuP/MWJ0K/yDwzobl9dFz4x5JcjXJpSSHZjKdFsWM+8+M1anwM+G5Glt+CThcVfcArwDPTXyh5EySQZLB9evX/7tJNU9m3H9mrE6Fvw5s/E1/B/Duxg2q6sOqujFafBY4PumFqup8Va1U1cqBAwe2Mq/mw4z7z4zVqfAvA3cnuTPJPuAUsLpxgyS3b1g8Abw5uxG1AGbcf2as6VfpVNXNJGeBl4E9wIWqupbkHDCoqlXge0lOADeBj4DTc5xZM2bG/WfGAkjV+GG8xVhZWanBYLAj792qJFeqamVR72fGi2fG/bedjL3TVpIaYeFLUiMsfElqhIUvSY2w8CWpERa+JDXCwpekRlj4ktQIC1+SGmHhS1IjLHxJaoSFL0mNsPAlqREWviQ1wsKXpEZY+JLUCAtfkhph4UtSIyx8SWqEhS9JjbDwJakRFr4kNaJT4Sd5MMlbSdaSPDlh/WeTXBytfy3J4VkPqvky4/4zY00t/CR7gKeBh4CjwKNJjo5t9hjwt6r6EvBj4EezHlTzY8b9Z8aCbnv49wJrVfV2VX0CvAicHNvmJPDc6PEl4GtJMrsxNWdm3H9mLPZ22OYg8M6G5XXgvs22qaqbST4Gvgh8sHGjJGeAM6PFG0ne2MrQu8BtjP1sS+LLmzxvxv9pWfMFM+6qjxlP1aXwJ/2Gry1sQ1WdB84DJBlU1UqH9991lnX2JIPNVk14rtmMl3VuMOOulnVu+NSMp+pySGcdOLRh+Q7g3c22SbIX+Dzw0VaH0sKZcf+ZsToV/mXg7iR3JtkHnAJWx7ZZBb41evwN4DdVdcuegXYtM+4/M9b0QzqjY3lngZeBPcCFqrqW5BwwqKpV4GfAL5KsMdwjONXhvc9vY+6dtqyzT5zbjG+xrHODGXe1rHPDNmaPv8AlqQ3eaStJjbDwJakRcy/8Zb2du8Pcp5NcT/L66OvxnZhzXJILSd7f7NroDD01+rmuJjk2g/c04wUy4+7MeExVze2L4cmhvwB3AfuAPwBHx7b5DvDM6PEp4OI8Z5rh3KeBn+70rBNmfwA4BryxyfqHgV8zvOb6fuA1MzZjM975XBeR8bz38Jf1du4uc+9KVfUqn37t9Eng+Rr6HfCFJLdv4y3NeMHMuDMzHjPvwp90O/fBzbapqpvAv27n3kld5gZ4ZPTfqUtJDk1Yvxt1/dlm+XpmvFhmPGTGY+Zd+DO7nXvBusz0EnC4qu4BXuHfeze73az/vc149zHjITMeM+/CX9bbuafOXVUfVtWN0eKzwPEFzbZdXTKZ9euZ8WKZ8ZAZj5l34S/r7dxT5x47XnYCeHOB823HKvDN0Vn++4GPq+q9bbyeGe8+ZjxkxuMWcLb5YeDPDM+Wf3/03DngxOjx54BfAmvA74G7dvoMece5fwhcY3jm/7fAkZ2eeTTXC8B7wD8Z7gU8BjwBPDFaH4YfhPEX4I/AihmbsRm3kbF/WkGSGuGdtpLUCAtfkhph4UtSIyx8SWqEhS9JjbDwJakRFr4kNeL/AaPkcsPJ1CCYAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], "source": [ "# Get the first images from the test-set.\n", - "images = data.test.images[0:9]\n", - "\n", + "#images = data.test.images[0:9]\n", + "images = objBoth.listImagePaths[0:9]\n", "# Get the true classes for those images.\n", - "cls_true = data.test.cls[0:9]\n", - "\n", + "#cls_true = data.test.cls[0:9]\n", + "cls_true = objBoth.listLabels[0:9]\n", "# Plot the images and labels using our helper-function above.\n", "plot_images(images=images, cls_true=cls_true)" ]