Newer
Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Introduction to Neural Networks\n",
"\n",
"## TO DO: Almost all the figues and schematics will be replaced or improved slowly\n",
"\n",
"<img src=\"./images/neuralnets/Colored_neural_network.svg\"/>\n",
"source: https://en.wikipedia.org/wiki/Artificial_neural_network\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## History of Neural networks\n",
"\n",
"**TODO: Make it more complete and format properly**\n",
"\n",
"1943 - Threshold Logic\n",
"\n",
"1940s - Hebbian Learning\n",
"\n",
"1958 - Perceptron\n",
"\n",
"1975 - Backpropagation\n",
"\n",
"1980s - Neocognitron\n",
"\n",
"1982: Hopfield Network\n",
"\n",
"1986: Convolutional Neural Networks\n",
"\n",
"1997: Long-short term memory (LSTM) model\n",
"\n",
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Building blocks\n",
"### Perceptron\n",
"\n",
"Smallest unit of a neural network is a **perceptron** like node.\n",
"\n",
"**What is a Perceptron?**\n",
"\n",
"It is a simple function which has multiple inputs and a single output.\n",
"\n",
"Step 1: Weighted sum of the inputs is calculated\n",
"\n",
"\\begin{equation*}\n",
"weighted\\_sum = \\sum_{k=1}^{num\\_inputs} w_{i} x_{i}\n",
"\\end{equation*}\n",
"\n",
"Step 2: The following activation function is applied\n",
"\n",
"$$\n",
"f(weighted\\_sum) = \\left\\{\n",
" \\begin{array}{ll}\n",
" 0 & \\quad weighted\\_sum < threshold \\\\\n",
" 1 & \\quad weighted\\_sum \\geq threshold\n",
"\n",
"You can see that this is also a linear classifier as we introduced in script 02."
"metadata": {},
"outputs": [],
"source": [
"%matplotlib inline\n",
"%config IPCompleter.greedy=True"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1"
]
},
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"def perceptron(X, w, threshold=1):\n",
" # This function computes sum(w_i*x_i) and \n",
" # applies a perceptron activation\n",
" linear_sum = np.dot(X,w)\n",
" output=0\n",
" if linear_sum >= threshold:\n",
" output = 1\n",
" # print(\"The perceptron has peaked\")\n",
" return output\n",
"X = [1,0]\n",
"w = [1,1]\n",
"perceptron(X,w)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Boolean AND\n",
"\n",
"| x$_1$ | x$_2$ | output |\n",
"| --- | --- | --- |\n",
"| 0 | 0 | 0 |\n",
"| 1 | 0 | 0 |\n",
"| 0 | 1 | 0 |\n",
"| 1 | 1 | 1 |"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Perceptron output for x1, x2 = 0, 0 is 0\n",
"Perceptron output for x1, x2 = 1, 0 is 0\n",
"Perceptron output for x1, x2 = 0, 1 is 0\n",
"Perceptron output for x1, x2 = 1, 1 is 1\n"
]
}
],
"source": [
"# Calculating Boolean AND using a perceptron\n",
"import matplotlib.pyplot as plt\n",
"w=[1,1]\n",
"X=[[0,0],[1,0],[0,1],[1,1]]\n",
"for i in X:\n",
" print(\"Perceptron output for x1, x2 = \" + str(i)[1:-1] + \" is \" + str(perceptron(i,w,threshold)))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this simple case we can rewrite our equation to $x_2 = ...... $ which describes a line in 2D:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X2cjXX+x/HXZ2ZyL7nLXUVSmzPunXW7RGxibVllsd3Q0oSEam3KdidL7Eq00W+kH7qRpEKkJDepqKEYxvbr5rGFJKFBYQzf3x9zzE4azMw5c65z834+Hucx13Wd7znX53KN857rXNf5HHPOISIikuB1ASIiEhkUCCIiAigQREQkQIEgIiKAAkFERAIUCCIiAoQgEMzsQjNbaWYZZrbVzIbnM8bMbKqZfW5mm82sWbDrFRGR0EoKwXNkA3c75zaaWXlgg5ktd85l5BnTFbg0cGsJTA/8FBGRCBH0EYJzbpdzbmNg+iCwDah1yrBrgTkuxzrgPDOrEey6RUQkdEJxhJDLzOoATYH1p9xVC9ieZ35HYNmuUx6fAqQAlC1btvnll18eyvIiwpEjR/jqq6/Izs7G5/NhZl6XJCIxZMOGDd8756oW5bEhCwQzKwcsAEY45w4U5Tmcc6lAKoDf73dpaWmhKi+inDhxgm+//ZaaNWty6NAh5s6dy4ABA0hI0Dl+EQmOmX1V1MeG5BXIzM4hJwyed869ks+QncCFeeYvCCyLSwkJCdSsWROAOXPmkJKSQvv27fn3v//tcWUiEs9CcZWRATOBbc65x04zbBFwc+Bqo1ZApnNu12nGxpXBgwcze/Zstm3bRuPGjRk3bhzHjh3zuiwRiUOhOEJoC9wEXGlmnwRu3cxskJkNCoxZCnwJfA7MAIaEYL0xwcy4+eabycjIoEePHowePZphw4Z5XZaIxKGgzyE459YCZzwz6nJ6bN8e7LpiWbVq1Zg3bx59+/alQYMGAOzdu5cyZcpQunRpj6sTkXigs5gRpkePHtSrVw+AAQMG0KRJE9auXetxVSISDxQIEWzo0KFkZWXRrl07hg4dysGDB70uSURimAIhgnXu3Jn09HSGDx/OtGnTSE5OZuPGjV6XJSIxSoEQ4cqVK8fjjz/Oe++9xyWXXELt2rW9LklEYpQCIUq0bt2alStXUrlyZbKzs/nd737H/Pnz0Xdii0ioKBCi0J49e9i9ezd//OMf6dmzJ7t26SMdIhI8BUIUqlGjBuvWrWPixIksW7aM+vXr88wzz+hoQUSCokCIUklJSYwcOZLNmzfTuHFjJk2aRFZWltdliUgUUyBEuUsvvZSVK1eyYsUKSpYsycGDB3nqqac4fvy416WJSJRRIMSAhIQEqlevDsBzzz3H4MGD+c1vfkNGRsZZHiki8l8KhBgzaNAgnnvuOT777DOaNm3KI488oreSRKRAFAgxxsy44YYbyMjIoGfPnjzwwAMMH/6Lr7kWEfmFkH5jmkSO888/n7lz59K3b1+Sk5MB+P777ylbtqya5YlIvnSEEOOuueYaLrnkEiCnWV6jRo1YvXq1x1WJSCRSIMSR4cOHc+LECTp06MDgwYM5cKBI33QqIjFKgRBHrrzySjZv3sxdd91FamoqycnJbNiwweuyRCRCKBDiTNmyZZk0aRLvv/8+v/rVr7j44ou9LklEIoQCIU61bNmSt99+m0qVKpGdnc3VV1/Niy++qPYXInFMgSB8//337Nu3j759+9KjRw927tzpdUki4gEFglC9enU++OADJk2axPLly/H5fMyYMUNHCyJxRoEgACQmJnLXXXeRnp5O8+bNefzxx/UJZ5E4o0CQn7nkkktYsWJFbrO8AwcOMG3aNDXLE4kDCgT5BTPLbZb3wgsvcPvtt9OmTRu2bNnicWUiUpwUCHJGt912G3PnzuXLL7+kWbNmPPTQQ3orSSRGKRDkjMyMPn36sG3bNnr16sXDDz/MHXfc4XVZIlIM1NxOCqRKlSo8//zz9O3bl/r16wM53+1ctmxZypQp43F1IhIKOkKQQunevfvPmuU1bNiQlStXelyViIRCSALBzJ4xs+/MLN+zjmbWwcwyzeyTwO2BUKxXvHX33XeTkJDAlVdeSUpKCpmZmV6XJCJBCNURwizg6rOMedc51yRwGxOi9YqHrrjiCjZt2sTIkSOZOXMmPp+PtLQ0r8sSkSIKSSA459YA+0LxXBJdypQpw8SJE1m/fj3JycnUrVvX65JEpIjCeQ6htZltMrM3zCw5jOuVMPD7/bz11lu5zfK6dOnC888/r/YXIlEkXIGwEajtnGsMPAG8lt8gM0sxszQzS9uzZ0+YSpNQ27t3L5mZmdx44438/ve/Z/v27V6XJCIFEJZAcM4dcM4dCkwvBc4xsyr5jEt1zvmdc/6qVauGozQpBtWqVeO9995j8uTJrFy5kuTkZP7nf/6HEydOeF2aiJxBWALBzKqbmQWmWwTWuzcc6xZvJCYmMmLECNLT02nRogVTp07l2LFjXpclImcQkg+mmdlcoANQxcx2AA8C5wA4554CrgcGm1k2cBjo4/TmclyoW7cuy5cvZ8+ePbnN8mbPns3gwYNJStLnIkUiiUXq67Lf73e6hDH2PPXUUwwePJjmzZszc+ZMGjdu7HVJIjHFzDY45/xFeaw+qSxhddttt/HSSy+xfft2/H4/999/P0ePHvW6LBFBgSBhZmb06tWLjIwM+vbty9ixYxk2bJjXZYkIam4nHqlcuTJz5syhb9++XH755QB89913lC1blrJly3pcnUh80hGCeKpr165cfPHFwH+b5a1YscLjqkTikwJBIsbIkSNJSkqic+fODBw4kB9++MHrkkTiigJBIkb79u3ZtGkTo0aNYtasWfh8Pj766COvyxKJGwoEiSilS5dm/PjxrF+/nkaNGlGvXj0A9UQSCQMFgkSk5s2bs2zZMipWrMixY8f47W9/y5w5cxQMIsVIgSARb9++fRw+fJh+/frRrVs3vv76a69LEolJCgSJeNWqVePdd99l6tSpvPvuuyQnJ/Pkk0+qWZ5IiCkQJCokJCRwxx13sGXLFtq0acP06dPJzs72uiyRmKJAkKhSp04dli1bxsqVKylRogQHDhxgypQpCgeREFAgSNQxM05+X8aLL77IiBEjaNmyJZ988onHlYlENwWCRLWUlBRefvlldu7cid/vZ/To0Rw5csTrskSikgJBot51111HRkYGN954I+PGjVOzPJEiUnM7iQmVKlVi1qxZ9O3bl8suuwzIaZZXpkwZypUr53F1ItFBRwgSU7p06fKzZnkNGjTgrbfe8rgqkeigQJCYNWrUKEqXLk2XLl245ZZb2Ldvn9cliUQ0BYLErLZt2/Lxxx8zevRonn32WXw+Hx9++KHXZYlELAWCxLRSpUoxduxY0tLS8Pv9uecX1BNJ5JcUCBIXmjRpwuuvv855553HsWPH6Ny5M7NmzVIwiOShQJC4s3//frKysrjlllvo0qUL//nPf7wuSSQiKBAk7px//vmsXr2aJ598kg8++IAGDRrwxBNPqFmexD0FgsSlhIQEhgwZwpYtW2jXrh2pqanqhyRxT4Egca127dosXbo0t1leZmYmjz32GMeOHfO6NJGwUyBI3DMzqlSpAsBLL73E3Xffza9//Ws2btzocWUi4aVAEMnj1ltv5dVXX2X37t20aNGCUaNGcfjwYa/LEgkLBYLIKXr06EFGRgb9+/dnwoQJDB8+3OuSRMIiJIFgZs+Y2XdmtuU095uZTTWzz81ss5k1C8V6JfIMGQJJSWCW83PIEK8rKpqKFSvy9NNPs3z5cu677z4Adu/ezcGDBz2urHjFyv6TognVEcIs4Ooz3N8VuDRwSwGmh2i9EkGGDIHp0+H48Zz548dz5qP5RaVz587UqVMHyGmWl5yczBtvvOFtUcUkFvefFE5IAsE5twY4U+ewa4E5Lsc64DwzqxGKdUvkSE0t3PJoM3r0aMqVK0e3bt24+eab2bt3r9clhVSs7z85u3CdQ6gFbM8zvyOw7GfMLMXM0swsbc+ePWEqTULl5F+WBV0ebVq3bs3HH3/M/fffz9y5c2OuWV6s7z85u4g6qeycS3XO+Z1z/pPfmSvRIzGxcMujUcmSJRkzZgwbNmygRYsWMdUsLx72n5xZuAJhJ3BhnvkLAsskhqSkFG55NGvUqBGLFy/ObZbXsWNHZs6cGdXBEE/7T/IXrkBYBNwcuNqoFZDpnNsVpnVLmEybBoMH//cvysTEnPlp07ytq7jt378fgIEDB9K5c2e+/PJLjysqmnjdf/JfFoq/aMxsLtABqALsBh4EzgFwzj1lZgb8i5wrkX4CbnHOpZ3pOf1+v0tLO+MQkYhx4sQJZsyYwciRIzl+/Dhjx45l2LBhJOr9FgkzM9vgnPMX6bGReoirQJBotGPHDgYNGsTXX39NWloaJUqU8LokiTPBBEJEnVQWiXYXXHABixcvZtWqVbnN8v75z3+SlZXldWkiZ6VAEAkxM6NSpUoAzJ8/n5EjR+L3+/noo488rkzkzBQIIsVo4MCBLFy4kL1799KqVSv++te/8tNPP3ldlki+FAgixeyaa64hIyODAQMG8I9//EPN8iRiJXldgEg8qFChAqmpqfTp04dLLrkEgG+//ZbSpUtToUIFj6sTyaEjBJEwuvLKK6lduzbw32Z5S5Ys8bgqkRwKBBGPPPjgg1SsWJHu3btzww03oP5d4jUFgohHWrRowYYNG3jooYeYP38+Pp+PdevWeV2WxDEFgoiHSpQowYMPPsjGjRtp27Yt9evXB2KjWZ5EHwWCSARo0KABr732GhUqVCArK4sOHTowY8YMBYOElQJBJMJkZmaSmJhISkoKnTp14osvvvC6JIkTCgSRCFO1alVWrFhBamoqGzZsoGHDhjz22GMc1zfVSDFTIIhEIDPj1ltvJSMjg86dOzN79mxOnDjhdVkS4xQIIhGsVq1aLFy4kFWrVnHOOefwww8/MGHCBDXLk2KhQBCJcGZGxYoVAViwYAGjRo2iWbNmrF+/3uPKJNYoEESiyIABA3j99dfJzMykdevW3HXXXfz4449elyUxQoEgEmV+97vfsXXrVgYNGsTkyZPVLE9CRs3tRKLQueeey7Rp0+jduzd16tQBYNeuXZQpU0bN8qTIdIQgEsWuuOKK3GZ5AwcOxOfzsWjRIo+rkmilQBCJEQ8//DCVK1fm2muvpU+fPnz33XdelyRRRoEgEiP8fj9paWk88sgjvPrqq/h8Pj744AOvy5IookAQiSElSpTgb3/7Gx9//DHt27fH5/MBapYnBaNAEIlBPp+PV155JbdZXrt27Zg+fbo+7SxnpEAQiXGZmZmULl2aIUOG0LFjRz777DOvS5IIpUAQiXFVq1blrbfeYubMmWzatIlGjRoxceJEsrOzvS5NIowCQSQOmBl//vOfycjI4Oqrr+aFF17QeQX5BQWCSBypWbMmr7zyys+a5Y0fP56jR496XZpEgJAEgpldbWafmtnnZjYqn/v7m9keM/skcBsYivWKSOGZGeeddx4Ar776Kvfddx9NmzbVJaoSfCCYWSLwJNAV8AF9zcyXz9B5zrkmgdvTwa5XRIJ3yy23sHTpUg4dOkTbtm0ZMWKEmuXFsVAcIbQAPnfOfemcywJeBK4NwfOKSBh07dqVrVu3MmTIEKZMmaJmeXEsFM3tagHb88zvAFrmM+46M2sP/B9wp3Nu+6kDzCwFSAG46KKLQlCaiBRE+fLl+de//kXv3r1zeyN98803lC5dOve7GCT2heuk8mKgjnOuEbAcmJ3fIOdcqnPO75zzV61aNUylichJ7dq1y/1j7GSzvFdffdXjqiRcQhEIO4EL88xfEFiWyzm31zl38jKGp4HmIViviBSjsWPHUr16dXr27EmvXr349ttvvS5JilkoAuEj4FIzu9jMSgB9gJ/13zWzGnlmrwG2hWC9IlKMmjVrxocffsi4ceNYvHixmuXFgaADwTmXDQwF3iTnhf4l59xWMxtjZtcEhg0zs61mtgkYBvQPdr0iUvzOOecc7r33Xj755BM6depEcnIyoGZ5scoidcf6/X6XlpbmdRkicoqsrCw6dOjADTfcwODBg0lI0OdbI4mZbXDO+YvyWO1JESmUAwcOUL58eYYOHcoVV1zBp59+6nVJEiIKBBEplCpVqrBs2TJmzZrF1q1bady4MY8++qia5cUABYKIFJqZ0a9fPzIyMujevTvz5s3TeYUYoEAQkSKrXr06L7/8cm6zvP379/PII49w5MgRr0uTIlAgiEjQKlSoAMDChQt54IEHaNKkCe+9957HVUlhKRBEJGT69+/Pm2++ydGjR2nXrh133HEHBw8e9LosKSAFgoiE1FVXXUV6ejp33HEHTz75JCNGjPC6JCmgUDS3ExH5mXLlyjFlyhR69+7NhRfmdLbZuXMnpUuXplKlSh5XJ6ejIwQRKTZt2rTJDYRbb70Vn8/HggULPK5KTkeBICJhMW7cOGrVqsX111/Pddddx65du7wuSU6hQBCRsGjSpAnr16/n0UcfZcmSJfh8Pl2JFGEUCCISNklJSdxzzz1s3ryZq666ioYNGwJw4sQJjysTUCCIiAcuu+wy5s2bx7nnnktWVhZt2rRh6tSpHD9+3OvS4poCQUQ8dfDgQSpWrMjw4cNp164d27bp61K8okAQEU9VrlyZpUuXMmfOHD799FOaNGnC2LFjOXbsmNelxR0Fgoh4zsy46aabyMjIoEePHrzyyitelxSXFAgiEjGqVavGvHnzWL16dW6zvIcffpjDhw97XVpcUCCISMQpX748AIsWLeKhhx6icePGrFmzxuOqYp8CQUQiVr9+/Xj77bfJzs7miiuu4Pbbb+fAgQNelxWzFAgiEtE6depEeno6I0aMYPr06dx5551elxSz1NxORCJe2bJlmTx5Mr179+aCCy4AYMeOHZQqVYoqVap4XF3s0BGCiESNVq1a5QbCyWZ5+vrO0FEgiEhUmjBhAhdddBF9+vThD3/4A998843XJUU9BYKIRKVGjRqxbt06/vGPf/Dmm2/i8/lYu3at12VFNQWCiEStpKQk/vKXv5Cenk63bt1o1KgRoGZ5RaVAEJGoV69ePV544QXOPfdcjh49SqtWrXj88cfVLK+QFAgiElMOHTrE+eefz5133knbtm3ZunWr1yVFjZAEgpldbWafmtnnZjYqn/tLmtm8wP3rzaxOKNYrkWfIEEhKArOcn0OGeF2RFEYs7L/KlSuzePFinn/+eT7//HOaNm3KmDFj1CyvAIIOBDNLBJ4EugI+oK+Z+U4ZNgDY75yrB0wGJgS7Xok8Q4bA9Olw8ij9+PGc+Wh8UYlHsbT/zIw//elPbNu2jeuvv56FCxdiZl6XFfEs2Ot3zaw18JBzrktg/l4A59z4PGPeDIz5wMySgG+Bqu4MK/f7/S4tLS2o2iS8kpL++2KSV2IiZGeHvx4pnFjefwcPHqR8+fLs27ePKVOmcM8991CmTBmvyyoWZrbBOecvymND8ZZRLWB7nvkdgWX5jnHOZQOZQOVTn8jMUswszczS9uzZE4LSJJxOd/5O5/WiQyzvv5PN8l5//XXGjBlD48aNWbVqlbdFRaCIOqnsnEt1zvmdc/6qVat6XY4UUmJi4ZZLZImH/XfzzTfzzjvv4JyjY8eODBo0iMzMTK/LihihCISdwIV55i8ILMt3TOAtowrA3hCsWyJISkrhlktkiZf917FjRzZv3szdd9/NjBkz1Cwvj1A0t/sIuNTMLibnhb8P8KdTxiwC+gEfANcD75zp/IFEp2nTcn6mpua8zZCYmPNicnK5RLZ42n9lypThn//8J71796ZGjRpATrO8kiVLEs/vTgR9UhnAzLoBjwOJwDPOub+b2RggzTm3yMxKAc8CTYF9QB/n3Jdnek6dVBaRcOratStpaWlMnTqVPn36RO1VScGcVA5JIBQHBYKIhNOWLVsYMGAAH374Id27d2f69Om5nVWjiddXGYmIRL0GDRrw/vvv89hjj7FixQqSk5N59913vS4rrBQIIiIBiYmJ3HnnnWzZsoXf//73NG7cGIifZnkKBBGRU9StW5fnnnsut1ley5YtmTRpUsw3y1MgiIicwY8//kjNmjX5y1/+QuvWrUlPT/e6pGKjQBAROYNKlSrx2muv8eKLL/Kf//yHZs2a8eCDD5KVleV1aSGnQBAROQszo3fv3mRkZNCnTx+WLFlCQkLsvXzG3haJiBSTKlWq8Oyzz7J69WqSkpLYt28ff/vb3/jxxx+9Li0kFAgiIoVUtmxZAJYsWcLf//53GjVqxDvvvONxVcFTIIiIFNFNN93E6tWrSUxMpFOnTtx666388MMPXpdVZAoEEZEgtG/fnk2bNvHXv/6VZ555hrvuusvrkoosFM3tRETiWunSpZkwYQK9evXKbZa3fft2SpYsyfnnn+9xdQWnIwQRkRDx+/3UqpXz/WApKSnUr1+f5557jkjtGXcqBYKISDGYNGkSl112GTfddBPdu3dn+/btZ3+QxxQIIiLFwOfzsXbtWqZMmcKqVavw+XwR3yxPgSAiUkwSExMZNmwYW7ZsoWfPnjRp0gQgYnsiKRBERIrZxRdfzOzZsylfvjxHjhzh17/+NRMnTiQ7O9vr0n5GgSAiEkY//fQTderU4Z577qFly5Zs2rTJ65JyKRBERMKoUqVKLFiwgPnz57Njxw78fj/3339/RDTLUyCIiISZmXH99deTkZHBDTfcwLJlyyKiWZ73FYiIxKnKlSsza9as3GZ5e/fu5b777uPQoUOe1KNAEBHxWJkyZQB44403GD9+PA0bNmT58uVhr0OBICISIW688UbWrFlDiRIluOqqq/jzn//M/v37w7Z+BYKISARp164dmzZt4t5772XOnDlhbZan5nYiIhGmVKlSjBs3jl69euU2x/v6668pUaIE1atXL7b16ghBRCRCNW3aNLdZ3m233YbP52P27NnF1ixPgSAiEgUmT56Mz+ejf//+dO3ala+++irk61AgiIhEgcsvv5w1a9bwxBNPsHbtWpKTk1m9enVI1xFUIJhZJTNbbmafBX5WPM2442b2SeC2KJh1iojEq4SEBIYOHcrWrVvp1asXzZs3B0LXLC/YI4RRwArn3KXAisB8fg4755oEbtcEuU4RkbhWu3Zt/vd//5dy5cpx5MgR/H4/48eP59ixY0E9b7CBcC0wOzA9G+gR5POJiEghHD58mHr16nHffffRsmXLoJ4r2ECo5pzbFZj+Fqh2mnGlzCzNzNaZmUJDRCREKlasyPz581mwYAHffPNNUM911s8hmNnbQH4Xvo7OO+Occ2Z2umuhajvndppZXeAdM0t3zn2Rz7pSgBSAiy666KzFi4hIjp49e9KxY0cqVapU5Oc4ayA45zqf7j4z221mNZxzu8ysBvDdaZ5jZ+Dnl2a2CmgK/CIQnHOpQCqA3++Pjm+lFhGJEBUr5ntdT4EF+5bRIqBfYLofsPDUAWZW0cxKBqarAG2BjCDXKyIiIRZsIDwK/NbMPgM6B+YxM7+ZPR0YUx9IM7NNwErgUeecAkFEJMIE1cvIObcX6JTP8jRgYGD6faBhMOsREZHip08qi4gIoEAQEZEABYKIiAAKBBERCVAgiIgIoEAQEZEABYKIiAAKBBERCVAgiIgIoEAQEZEABYKIiAAKBBERCVAgiIgIoEAQEZEABYKIiAAKBBERCVAgiIgIoEAQEZEABYKIiAAKBBERCVAgiIgIoEAQEZEABYKIiAAKBBERCVAgiIgIoEAQEZEABYKIiAAKBBERCQgqEMysl5ltNbMTZuY/w7irzexTM/vczEYFs04RESkewR4hbAF6AmtON8DMEoEnga6AD+hrZr4g1ysiIiGWFMyDnXPbAMzsTMNaAJ87574MjH0RuBbICGbdIiISWkEFQgHVArbnmd8BtMxvoJmlACmB2aNmtqWYa/NSFeB7r4soRtq+6BbL2xfL2wbwq6I+8KyBYGZvA9XzuWu0c25hUVecH+dcKpAaWG+ac+605yWinbYvumn7olcsbxvkbF9RH3vWQHDOdS7qkwfsBC7MM39BYJmIiESQcFx2+hFwqZldbGYlgD7AojCsV0RECiHYy07/YGY7gNbAEjN7M7C8ppktBXDOZQNDgTeBbcBLzrmtBXj61GBqiwLavuim7YtesbxtEMT2mXMulIWIiEiU0ieVRUQEUCCIiEhAxARCrLfBMLNKZrbczD4L/Kx4mnHHzeyTwC3iT76fbX+YWUkzmxe4f72Z1Ql/lUVXgO3rb2Z78uyzgV7UWRRm9oyZfXe6z/tYjqmBbd9sZs3CXWMwCrB9HcwsM8++eyDcNRaVmV1oZivNLCPwujk8nzGF33/OuYi4AfXJ+UDFKsB/mjGJwBdAXaAEsAnweV17AbdvIjAqMD0KmHCacYe8rrUQ23TW/QEMAZ4KTPcB5nldd4i3rz/wL69rLeL2tQeaAVtOc3834A3AgFbAeq9rDvH2dQBe97rOIm5bDaBZYLo88H/5/G4Wev9FzBGCc26bc+7TswzLbYPhnMsCTrbBiAbXArMD07OBHh7WEioF2R95t/tloJOdpddJBInm37ezcs6tAfadYci1wByXYx1wnpnVCE91wSvA9kUt59wu59zGwPRBcq7grHXKsELvv4gJhALKrw3Gqf8Ikaqac25XYPpboNppxpUyszQzW2dmkR4aBdkfuWNcziXImUDlsFQXvIL+vl0XOCR/2cwuzOf+aBXN/98KqrWZbTKzN8ws2etiiiLwNmxTYP0pdxV6/4Wjl1GucLbB8MKZti/vjHPOmdnprvet7ZzbaWZ1gXfMLN0590Woa5WQWQzMdc4dNbPbyDkautLjmqRgNpLz/+2QmXUDXgMu9bimQjGzcsACYIRz7kCwzxfWQHAx3gbjTNtnZrvNrIZzblfgsO270zzHzsDPL81sFTnJH6mBUJD9cXLMDjNLAioAe8NTXtDOun3Oubzb8jQ554piRUT/fwtW3hdQ59xSM5tmZlWcc1HR+M7MziEnDJ53zr2Sz5BC779oe8somttgLAL6Bab7Ab84IjKzimZWMjBdBWhLZLcJL8j+yLvd1wNkdXu0AAABEklEQVTvuMAZryhw1u075T3Za8h5LzdWLAJuDlyt0grIzPO2Z9Qzs+onz2eZWQtyXg+j4o+VQN0zgW3OucdOM6zw+8/rs+V5zoj/gZz3uI4Cu4E3A8trAktPOXP+f+T81Tza67oLsX2VgRXAZ8DbQKXAcj/wdGC6DZBOztUs6cAAr+suwHb9Yn8AY4BrAtOlgPnA58CHQF2vaw7x9o0Htgb22Urgcq9rLsS2zQV2AccC//cGAIOAQYH7jZwvt/oi8PuY79V/kXorwPYNzbPv1gFtvK65ENv2G8ABm4FPArduwe4/ta4QEREg+t4yEhGRYqJAEBERQIEgIiIBCgQREQEUCCIiEqBAEBERQIEgIiIB/w/o5ubntGCXYwAAAABJRU5ErkJggg==\n",
"<Figure size 432x288 with 1 Axes>"
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# Plotting the decision boundary\n",
"plt.xlim(-1,2)\n",
"plt.ylim(-1,2)\n",
"for i in X:\n",
" plt.plot(i,\"o\",color=\"b\");\n",
"# Plotting the decision boundary\n",
"# that is a line given by w_1*x_1+w_2*x_2-threshold=0\n",
"plt.plot(np.arange(-3,4), 1.5-np.arange(-3,4), \"--\", color=\"black\");"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Exercise :Can you compute a Boolean \"OR\" using a perceptron?**\n",
"\n",
"Hint: copy the code from the \"AND\" example and edit the weights and/or threshold"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Boolean OR\n",
"\n",
"| x$_1$ | x$_2$ | output |\n",
"| --- | --- | --- |\n",
"| 0 | 0 | 0 |\n",
"| 1 | 0 | 1 |\n",
"| 0 | 1 | 1 |\n",
"| 1 | 1 | 1 |"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [],
"source": [
"# Calculating Boolean OR using a perceptron\n",
"# Edit the code below"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Perceptron output for x1, x2 = 0, 0 is 0\n",
"Perceptron output for x1, x2 = 1, 0 is 1\n",
"Perceptron output for x1, x2 = 0, 1 is 1\n",
"Perceptron output for x1, x2 = 1, 1 is 1\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XucTfX+x/HXZ2bcc4SR3HJJxUyT23ZLSqkcp5ojQ1G/cFIS1XGObsotJR11nJSQ8AvnKOXnVpRDkUrUkMuY6UInud8SSRG+vz9m5UyaYS57Zu3L+/l47Mfstfd37/VeFvO21trzHXPOISIiEuN3ABERCQ0qBBERAVQIIiLiUSGIiAigQhAREY8KQUREgCAUgpnVMLMlZpZuZhvM7M/ZjDEze87MNprZOjNrXND1iohIcMUF4T2OAf2dc6vNrCywyswWOefSs4xpD1zg3ZoD47yvIiISIgp8hOCc2+GcW+3d/x7IAKqdMuyPwFSXaQVwtplVKei6RUQkeIJxhHCSmdUCGgErT3mqGrAly/JW77Edp7y+F9ALoEyZMk3q1asXzHgiIhFv1apVe51zlfLz2qAVgpmdBfwf0M85dzA/7+GcmwBMAAgEAi41NTVY8UREooKZbc7va4PyKSMzK0ZmGfzLOTcrmyHbgBpZlqt7j4mISIgIxqeMDJgEZDjnRuUwbB7Qzfu0UQvggHNuRw5jRUTEB8E4ZdQKuA1Yb2ZrvMceAc4DcM6NBxYAfwA2AoeBPwVhvSIiEkQFLgTn3AeAnWGMA/oWdF0iIlJ49JPKIiICqBBERMSjQhAREUCFICIiHhWCiIgAKgQREfGoEEREBFAhiIiIR4UgIiKACkFERDwqBBERAVQIIiLiUSGIiAigQhAREY8KQUREABWCiIh4VAgiIgKoEERExKNCEBERQIUgIiIeFYKIiAAqBBER8agQREQECOFCcM7RsWNHPvroI7+jiIhEhZAthCNHjrBq1SpatWpFv379OHTokN+RREQiWsgWQsmSJUlLS6NPnz6MHj2apKQkFi1a5HcsEZGIFbKFAFC2bFnGjBnDsmXLKF68OI888ggnTpzwO5aISEQKSiGY2WQz221maTk838bMDpjZGu82OC/v37p1a9auXcusWbOIiYlh3759zJ07NxjRRUTEE6wjhJeB359hzPvOuYbebVheV1CyZElq1KgBwLPPPkuHDh246aab2LVrV97TiojIbwSlEJxzy4Bvg/FeuTF48GCGDx/O3LlzqV+/PlOnTsU5V1SrFxGJSEV5DaGlma01s7fMLLEgb1SsWDEeeeQR1q5dS/369enevTvDhw8PVk4RkagUV0TrWQ3UdM4dMrM/AHOAC04dZGa9gF4A55133hnftF69erz//vuMGzeODh06ALB//37KlStHTExIXy8XEQk5FqxTLWZWC3jTOXdxLsZ+DQScc3tzGhMIBFxqamqeMjjnuOqqqzh27BgTJ07koosuytPrRUTCnZmtcs4F8vPaIvlvtJmda2bm3W/mrXdfYayrR48ebNiwgQYNGvDUU0/x888/F8ZqREQiTrA+dvoK8BFwkZltNbOeZtbbzHp7QzoBaWa2FngO6OIK4SqwmdG9e3fS09O54YYbGDBgAM2bN+err74K9qpERCJO0E4ZBVt+ThmdatasWYwcOZJFixZRtmzZICUTEQldIX/KyC+/TI5XtmxZjhw5wo033siHH37odywRkZAU0YUAmaeRAL7++ms+/fRTWrduzX333afJ8kREThHxhfCLiy66iLS0NO69917GjBlDYmIiCxcu9DuWiEjIiJpCADjrrLMYPXo077//PqVKlWLgwIGaLE9ExBNVhfCLVq1asWbNGmbPnn1ysrxZs2b5HUtExFdRWQiQOVle9erVARg9ejQpKSmkpKSwY8cOn5OJiPgjagshq8GDB/PUU08xf/58EhIS+N///V9NliciUUeFAMTFxfHQQw+xbt06kpKSuP3223niiSf8jiUiUqSKanK7sHDhhReydOlSJkyYQHJyMgDffvst5cqVIzY21ud0IiKFS0cIp4iJiaF3795UrVqVEydO0LFjRy6//HIyMjL8jiYiUqhUCKdhZvTs2ZPPPvuMhg0bMnz4cE2WJyIRS4VwGmbGbbfdRnp6Oh06dGDgwIE0bdpUk+WJSERSIeRC5cqVmTFjBrNnz6Z06dKcc845fkcSEQk6FUIedOjQgQ8//JCzzjqLn376ieTkZJYtW+Z3LBGRoFAh5NEvk+V98803pKWlccUVV9C3b18OHjzoczIRkYJRIeTThRdeyPr16/nLX/7CuHHjuPjii1mwYIHfsURE8k2FUABlypRh1KhRLF++nLJlyzJ06FBNliciYUuFEAQtWrRg9erVJyfL27t3LzNnztT0FyISVlQIQVKiRAmqVasGwPPPP0/nzp3p2LEj27dv9zmZiEjuqBAKwaBBg3j66ad5++23SUhIYNKkSTpaEJGQp0IoBHFxcdx///2sX7+ehg0bcscddzBs2DC/Y4mInJYmtytEdevW5d1332XixIlcf/31AOzdu5fy5ctrsjwRCTk6QihkMTEx9OrV6+RkeSkpKbRq1YoNGzb4HU1E5FdUCEXIzOjduzebNm2iUaNGDBs2jKNHj/odS0QEUCEUKTOja9eupKen06lTJ4YMGUKTJk3YtGmT39FERFQIfqhUqRLTp09n3rx5nH322VSuXNnvSCIiKgQ/3XDDDSxbtuzkZHnXX389S5cu9TuWiEQpFYLPsk6W99lnn3HllVfSu3dvDhw44HMyEYk2QSkEM5tsZrvNLC2H583MnjOzjWa2zswaB2O9keTCCy9k3bp19O/fn5deeonExETmz5/vd6w869MH4uLALPNrnz5+J5K80P6LbsE6QngZ+P1pnm8PXODdegHjgrTeiFK6dGmeeeYZPvroI8qXL8+wYcPCarK8Pn1g3Dg4fjxz+fjxzGV9UwkP2n9iwZpSwcxqAW865y7O5rkXgaXOuVe85c+BNs65HTm9XyAQcKmpqUHJFo6OHj3K3r17qVq1Knv37mXx4sXcfPPNJ08xhaK4uP9+M8kqNhaOHSv6PJI32n+RwcxWOecC+XltUV1DqAZsybK81XvsV8ysl5mlmlnqnj17iihaaCpevDhVq1YFYMyYMXTt2pXk5GS2bt3qc7KcZffN5HSPS2jR/pOQuqjsnJvgnAs45wKVKlXyO07IGDRoEKNGjeKdd94hMTGRCRMmhOSppJxm49AsHeFB+0+KqhC2ATWyLFf3HpNciI2N5S9/+QtpaWkEAgHuuusuHn/8cb9j/UavXnl7XEKL9p8UVSHMA7p5nzZqARw43fUDyV6dOnVYvHgxkyZNopf3r3Tv3r0cC5ETvGPHwt13//d/lLGxmctjx/qbS3JH+0+CclHZzF4B2gDxwC5gCFAMwDk33jKvhI4h85NIh4E/OedOe8U42i8q58aJEydo06YNP/30E5MmTSIpKcnvSCLiM98vKjvnujrnqjjnijnnqjvnJjnnxjvnxnvPO+dcX+fc+c65pDOVgeSOmdG3b1++/vprGjduzJAhQzhy5IjfsUQkTIXURWXJGzPj5ptvJj09nS5dujBs2DCaNGnCxo0b/Y4mImFIhRAB4uPjmTZtGm+++SYVK1akSpUqfkcSkTCkQogg1113He+99x5lypThxx9/pH379rzzzjt+xxKRMKFCiFBbt25l06ZNXH311dx555189913fkcSkRCnQohQF1xwAWvXruXBBx9k8uTJJCQkMHfuXL9jiUgIUyFEsFKlSvG3v/2NlStXUqlSJUaMGBGSP+EsIqFBhRAFAoEAqampzJ49m5iYGPbs2cP06dMJ1sSGIhIZVAhRolixYic/ffTCCy9w6623cv3117Nly5YzvFJEooUKIQoNGjSI0aNHs3TpUhITExk3bpxOJYmICiEaxcbGct9995GWlkbz5s3p06cPjz32mN+xRMRncX4HEP/Url2bf//730yZMoV27doBsHv3bipUqEBcnP5qiEQbHSFEOTOjR48eVKlShRMnTpCSkkLz5s1Zu3at39FEpIipEOSkmJgY+vXrx7Zt2wgEAgwcOJCffvrJ71giUkRUCPIrKSkppKenc+uttzJ8+HAaNWrEl19+6XcsESkCKgT5jQoVKvDyyy/z9ttvU6VKlZO/21lEIpsKQXLUrl073n333ZOT5bVr145Fixb5HUtECokKQXJl27ZtbN68mWuvvZbbb7+d/fv3+x1JRIJMhSC5UrduXdasWcOAAQOYOnUqCQkJzJ492+9YIhJEKgTJtZIlS/Lkk0/y8ccfc+655zJy5Ej9hLNIBFEhSJ41btyYjz/++FeT5U2bNk2T5YmEORWC5EuxYsU499xzARg7dizdunWjffv2bN682edkIpJfKgQpsEGDBvH888/zwQcfkJiYyJgxY3QqSSQMqRCkwGJiYrjnnnvYsGEDl112Gffeey9Dhw71O5aI5JFmMJOgqVmzJm+99Rb//Oc/ueaaawDYtWsXFSpUoFixYj6nE5Ez0RGCBJWZcdttt3Huuef+arK8Tz/91O9oInIGKgQpNDExMfz1r39l+/btNG3alEceeUST5YmEMBWCFKqOHTuSkZFBt27dGDFiBA0bNuSLL77wO5aIZCMohWBmvzezz81so5k9nM3zPcxsj5mt8W53BGO9Eh7Kly/P5MmTWbhwIdWrV6datWp+RxKRbBS4EMwsFngBaA8kAF3NLCGboTOccw2928SCrlfCz7XXXsvixYspU6YMhw8f5uqrr+btt9/2O5aIeIJxhNAM2Oic+8o5dxR4FfhjEN5XItiOHTvYvn077du3p3v37uzbt8/vSCJRLxiFUA3YkmV5q/fYqVLMbJ2ZzTSzGtm9kZn1MrNUM0vds2dPEKJJqDr//PP59NNPGThwINOnTychIYGZM2dq+gsRHxXVReU3gFrOuUuARcCU7AY55yY45wLOuUClSpWKKJr4pUSJEjz++OOkpqZSo0YNRo0apUIQ8VEwCmEbkPV//NW9x05yzu1zzh3xFicCTYKwXokQDRo0YMWKFScny9u9ezdTpkxROYgUsWAUwifABWZW28yKA12AeVkHmFmVLIvJQEYQ1isRJC4ujsqVKwMwfvx4evTowbXXXst//vMfn5OJRI8CF4Jz7hhwD7CQzG/0rznnNpjZMDNL9obdZ2YbzGwtcB/Qo6Drlcg1cOBAxo4dy4oVK7j44ot57rnnOH78uN+xRCKehepheSAQcKmpqX7HEB9988039O7dm7feeouBAwfy+OOP+x1JJOSZ2SrnXCA/r9XkdhKyzjvvPObPn8/06dO5+uqrAdi5cycVK1bUZHkihUBTV0hIMzNuvfVWKleujHOOlJQUAoEAq1at8juaSMRRIUjYMDMeeugh9u7dS7NmzXjooYf48ccf/Y4lEjFUCBJWkpOT2bBhA7fffjsjR46kQYMGmixPJEhUCBJ2zj77bF566SUWL15M7dq1qV69ut+RRCKCCkHCVtu2bVm4cCGlS5fm8OHDXHXVVSxYsMDvWCJhS4UgEWHnzp3s2rWL6667jttuu429e/f6HUkk7KgQJCLUqVOH1atXM2TIEF599VUSEhJ47bXXNP2FSB6oECRilChRgqFDh7J69Wpq1qzJ6NGjVQgieaBCkIiTlJTERx99dHKyvF27djF58mSVg8gZqBAkIsXFxXHOOecA8OKLL9KzZ0/atm3Lpk2bfE4mErpUCBLxBg4cyPjx40lNTSUpKYlRo0ZpsjyRbKgQJOLFxMRw1113kZ6eTtu2benfvz9Dhw71O5ZIyNHkdhI1qlevzrx583jttde48sorgczf7VyxYkWKFy/uczoR/+kIQaKKmXHzzTdzzjnnnJwsr0mTJnzyySd+RxPxnQpBopaZMWDAAPbv30+LFi24//77OXz4sN+xRHyjQpCodsMNN7BhwwbuvPNO/v73v3PJJZfw+eef+x1LxBcqBIl65cqVY/z48SxZsoS6detSo0YNvyOJ+EKFIOJp06YNb7/99snJ8tq0acMbb7zhdyyRIqNCEMnGzp072bdvH8nJydxyyy3s2bPH70gihU6FIJKNOnXqsGrVKh577DFmzpxJ/fr1mT59uqa/kIimQhDJQfHixRk8eDCffvopdevW5YUXXlAhSERTIYicQWJiIh9++CFz5sw5OVnexIkTOXHihN/RRIJKhSCSC7GxsVSqVAmAl156iTvvvJO2bduyceNGn5OJBI8KQSSPHn30UV566SVWr15NUlISzzzzDMeOHfM7lkiBqRBE8sjMuOOOO0hPT+faa6/lgQceYMiQIX7HEikwTW4nkk/VqlVjzpw5zJw5kyuuuALInCyvQoUKlChRwud0InkXlCMEM/u9mX1uZhvN7OFsni9hZjO851eaWa1grFdCT58+EBcHZplf+/TxO1HhMjM6d+78m8nyVq5c6Xe0fIm2/Se/VuBCMLNY4AWgPZAAdDWzhFOG9QT2O+fqAv8A/lbQ9Uro6dMHxo2DX373zPHjmcvR8k3FzBg4cCAHDhygZcuW/PWvf+WHH37wO1auRfv+E7CCfq7azFoCQ51z7bzlAQDOuRFZxiz0xnxkZnHATqCSO83KA4GAS01NLVA2KVpxcf/9ZpJVbCxE0zXXgwcPMmDAAMaOHUvt2rVZsGAB9erV8zvWGWn/RQYzW+WcC+TntcE4ZVQN2JJleav3WLZjnHPHgANAxVPfyMx6mVmqmaVqqoDwk9NvpYy231b5u9/9jhdeeIH33nuP+vXrU7NmTYCQ/6E27T8JqU8ZOecmOOcCzrnAL5/5lvARG5u3xyPd5Zdfzvz58ylVqhQ//PADV1xxBXPnzvU7Vo60/yQYhbANyDpfcHXvsWzHeKeMygH7grBuCSG9euXt8Wiye/duDh48SIcOHbj55pvZtWuX35F+Q/tPglEInwAXmFltMysOdAHmnTJmHtDdu98JePd01w8kPI0dC3ff/d//UcbGZi6PHetvrlBQu3ZtPvnkE5544gnmzJlDQkIC//znP0PqNJL2nxT4ojKAmf0BeBaIBSY754ab2TAg1Tk3z8xKAtOARsC3QBfn3Fene09dVJZIlZGRQc+ePYmJiWHZsmXExITUmVsJcwW5qByUQigMKgSJZMePH2f//v3Ex8ezc+dO5s6dy5133qlykALz+1NGIpJHsbGxxMfHAzBp0iR69+5NmzZt+OKLL3xOJtFMhSDis0ceeYTJkyezfv16GjRowMiRIzVZnvhChSDiMzPjT3/6E+np6bRv356HHnpIk+WJLzS5nUiIqFKlCrNmzWLWrFm0bt0agG3btlGxYkVKlizpczqJBjpCEAkxHTt2pFKlSjjn6NSpE40aNWL58uV+x5IooEIQCVFmxtChQzl8+DCXXXYZf/7znzl06JDfsSSCqRBEQli7du1IS0ujb9++PP/881x88cVkZGT4HUsilApBJMSVLVuW559/nmXLlpGUlEStWrWA0J8sT8KPCkEkTFx22WW88cYbJyfLa926NbNmzfI7lkQQFYJIGNqzZw8//PADKSkpdOrUiZ07d/odSSKACkEkDNWqVYuPP/6YJ598kjfffJOEhARefvllnUaSAlEhiISpYsWKMWDAANasWUNCQgKTJ09WIUiBqBBEwly9evVYtmwZc+bMISYmhp07dzJ27FhOnDjhdzQJMyoEkQgQExNDhQoVAJg8eTJ9+/bl8ssv57PPPvM5mYQTFYJIhBkwYABTpkwhPT2dBg0a8OSTT/Lzzz/7HUvCgApBJMKYGd26dSMjI4Pk5GQeffRRhg4d6ncsCQOa3E4kQlWuXJnXX3+dOXPm0KpVK0CT5cnp6QhBJMJ16NDhV5PlNWjQgA8++MDvWBKCVAgiUcLMGDZsGEePHqV169bcc889fP/9937HkhCiQhCJItdccw3r16/nvvvuY+zYsSQmJpKenu53LAkRKgSRKHPWWWcxevRoPvjgAxo1akTt2rUBTZYnKgSRqHXppZcyd+7ck5PlXXrppbz++usqhiimQhAR9uzZw88//8xNN91ESkoKO3bs8DuS+ECFICLUqlWLFStWMHLkSN566y3NjRSlVAgiAkBcXBwPPPAAa9eu5ZJLLmHKlCkqhCijQhCRX7nwwgtZsmQJs2fPJiYmhh07djBmzBiOHz/udzQpZCoEEfmNrJPlvfzyy9x77720bt1av885whWoEMysgpktMrMvva/lcxh33MzWeLd5BVmniBSthx9+mGnTpvHFF1/QsGFDnnjiCU2WF6EKeoTwMPCOc+4C4B1vOTs/OucaerfkAq5TRIqQmfE///M/pKenc+ONNzJo0CAGDx7sdywpBFaQi0Zm9jnQxjm3w8yqAEudcxdlM+6Qc+6svLx3IBBwqamp+c4mIoVj3rx5tGzZkkqVKrFlyxbi4+MpVaqU37HEY2arnHOB/Ly2oEcIlZ1zv3xgeSdQOYdxJc0s1cxWmFmHAq5TRHyUnJz8q8nyLrnkEt577z2/Y0kQnLEQzGyxmaVlc/tj1nEu81Ajp8ONml5j3QI8a2bn57CuXl5xpO7Zsyev2yIiRcjMGDFiBCdOnKBNmzbcfffdHDx40O9YUgBFcsrolNe8DLzpnJt5unE6ZSQSHn744QcGDx7Ms88+S9WqVVm4cCEJCQl+x4pafp4ymgd09+53B+aeOsDMyptZCe9+PNAK0PSKIhGiTJky/P3vf2f58uU0bdqUOnXqAJosLxwVtBCeAq4xsy+Bq71lzCxgZhO9MfWBVDNbCywBnnLOqRBEIkzz5s2ZNWsWJUuW5NChQ7Ro0YIZM2aoGMJIgQrBObfPOdfWOXeBc+5q59y33uOpzrk7vPvLnXNJzrkG3tdJwQguIqFr3759OOfo0qULHTp0YPv27X5HklzQTyqLSNDVrFmT5cuX88wzz/Dvf/+bhIQEJk6cqKOFEKdCEJFCERcXR//+/Vm/fj0NGzZk2rRpKoQQp0IQkUJVt25d3n33XebMmXNysrzRo0drsrwQpEIQkUIXExND+fKZU51NnTqVfv36cemll5KWluZzMslKhSAiRerBBx/klVde4auvvqJx48Y89thjHD161O9YggpBRIqYmdGlSxcyMjLo3LkzQ4cOZciQIX7HEiDO7wAiEp3i4+P517/+xS233ELz5s0B+Oabb4iPj6d06dI+p4tOOkIQEV9dd911xMfH45yjc+fOJCUlsWTJEr9jRSUVgoiEBDPjqaeewsy46qqruOuuuzhw4IDfsaKKCkFEQsaVV17JunXruP/++5k4cSIJCQls2LDB71hRQ4UgIiGldOnSPP3006xYsYKWLVty/vmZs+Xrh9oKnwpBREJS06ZNmTlzJiVLluT777+nadOmTJ8+XcVQiFQIIhLy9u/fT1xcHLfeeis33HADW7Zs8TtSRFIhiEjIO++88/jwww/5xz/+wZIlS0hMTOTFF1/kxIkTfkeLKCoEEQkLsbGx9OvXj/Xr19OsWTNeffVVvyNFHBWCiISVOnXqsGjRImbPnk1MTAzbt29n1KhRHDt2zO9oYU+FICJhx8w4++yzAZg2bRr9+/enZcuWrFu3zudk4U2FICJh7cEHH2TGjBls3ryZJk2aMGTIEI4cOeJ3rLCkQhCRsGZm3HTTTWRkZNC1a1eGDRumyfLySZPbiUhEqFixIlOnTqVr1640a9YMgM2bNxMfH0+ZMmV8ThcedIQgIhGlffv2VKxYEeccN910E0lJSSxevNjvWGFBhSAiEcnMeOaZZ4iLi+Oaa66hZ8+efPfdd37HCmkqBBGJWK1bt2bt2rU8/PDDTJkyhYSEBP3aztNQIYhIRCtVqhQjRoxg5cqVtG7dmrp16wKaLC87KgQRiQpNmjRhxowZJyfLCwQCTJs2TcWQhQpBRKLOd999R4kSJejWrRvXXXcd33zzjd+RQoIKQUSiTo0aNXj//fd57rnnWLZsGYmJiYwbNy7qJ8tTIYhIVIqNjeXee+8lLS2Nli1b8vrrr2NmfsfyVYEKwcw6m9kGMzthZoHTjPu9mX1uZhvN7OGCrFNEJJhq1arFwoULmT17NmbGtm3bePrpp6NysryCHiGkAR2BZTkNMLNY4AWgPZAAdDWzhAKuV0QkaMyMcuXKATB9+nQefPBBmjdvzpo1a3xOVrQKVAjOuQzn3OdnGNYM2Oic+8o5dxR4FfhjQdYrIlJYHnjgAWbOnMm2bdsIBAI8+uij/PTTT37HKhIWjI9cmdlS4H7nXGo2z3UCfu+cu8Nbvg1o7py7J5uxvYBe3uLFZB6BRKp4YK/fIQqRti+8RfL2RfK2AVzknCubnxeecXI7M1sMnJvNU4865+bmZ6U5cc5NACZ46011zuV4XSLcafvCm7YvfEXytkHm9uX3tWcsBOfc1fl9c882oEaW5ereYyIiEkKK4mOnnwAXmFltMysOdAHmFcF6RUQkDwr6sdMbzWwr0BKYb2YLvcermtkCAOfcMeAeYCGQAbzmnNuQi7efUJBsYUDbF960feErkrcNCrB9QbmoLCIi4U8/qSwiIoAKQUREPCFTCJE+DYaZVTCzRWb2pfe1fA7jjpvZGu8W8hffz7Q/zKyEmc3wnl9pZrWKPmX+5WL7epjZniz77A4/cuaHmU02s91mlu3P+1im57xtX2dmjYs6Y0HkYvvamNmBLPtucFFnzC8zq2FmS8ws3fu++edsxuR9/znnQuIG1AcuApYCgRzGxAKbgDpAcWAtkOB39lxu30jgYe/+w8Dfchh3yO+sedimM+4PoA8w3rvfBZjhd+4gb18PYIzfWfO5fZcDjYG0HJ7/A/AWYEALYKXfmYO8fW2AN/3Omc9tqwI09u6XBb7I5u9mnvdfyBwhuMifBuOPwBTv/hSgg49ZgiU3+yPrds8E2lr4TCkZzn/fzsg5twz49jRD/ghMdZlWAGebWZWiSVdwudi+sOWc2+GcW+3d/57MT3BWO2VYnvdfyBRCLlUDtmRZ3spv/xBCVWXn3A7v/k6gcg7jSppZqpmtMLNQL43c7I+TY1zmR5APABWLJF3B5fbvW4p3SD7TzGpk83y4Cud/b7nV0szWmtml6qCuAAACD0lEQVRbZpbod5j88E7DNgJWnvJUnvffGX9SOZiKchoMP5xu+7IuOOecmeX0ed+azrltZlYHeNfM1jvnNgU7qwTNG8ArzrkjZnYXmUdDV/mcSXJnNZn/3g6Z2R+AOcAFPmfKEzM7C/g/oJ9z7mBB369IC8FF+DQYp9s+M9tlZlWcczu8w7bdObzHNu/rV96kgY3IPI8dinKzP34Zs9XM4oBywL6iiVdgZ9w+51zWbZlI5rWiSBHS/94KKus3UOfcAjMba2bxzrmwmPjOzIqRWQb/cs7NymZInvdfuJ0yCudpMOYB3b373YHfHBGZWXkzK+HdjwdaAelFljDvcrM/sm53J+Bd513xCgNn3L5Tzskmk3kuN1LMA7p5n1ZpARzIctoz7JnZub9czzKzZmR+PwyL/6x4uScBGc65UTkMy/v+8/tqeZYr4jeSeY7rCLALWOg9XhVYcMqV8y/I/F/zo37nzsP2VQTeAb4EFgMVvMcDwETv/qXAejI/zbIe6Ol37lxs12/2BzAMSPbulwReBzYCHwN1/M4c5O0bAWzw9tkSoJ7fmfOwba8AO4CfvX97PYHeQG/veSPzl1tt8v4+Zvvpv1C95WL77smy71YAl/qdOQ/bdhnggHXAGu/2h4LuP01dISIiQPidMhIRkUKiQhAREUCFICIiHhWCiIgAKgQREfGoEEREBFAhiIiI5/8BJfnPDvcczfcAAAAASUVORK5CYII=\n",
"<Figure size 432x288 with 1 Axes>"
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# Solution\n",
"# Calculating Boolean OR using a perceptron\n",
"import matplotlib.pyplot as plt\n",
"threshold=0.6\n",
"w=[1,1]\n",
"X=[[0,0],[1,0],[0,1],[1,1]]\n",
"for i in X:\n",
" print(\"Perceptron output for x1, x2 = \" + str(i)[1:-1] + \" is \" + str(perceptron(i,w,threshold)))\n",
"# Plotting the decision boundary\n",
"plt.xlim(-1,2)\n",
"plt.ylim(-1,2)\n",
"for i in X:\n",
" plt.plot(i,\"o\",color=\"b\");\n",
"# Plotting the decision boundary\n",
"# that is a line given by w_1*x_1+w_2*x_2-threshold=0\n",
"plt.plot(np.arange(-3,4), threshold-np.arange(-3,4), \"--\", color=\"black\");"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Optional exercise: Create a NAND gate with perceptrons**"
]
},
{
"cell_type": "code",
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
"metadata": {},
"outputs": [],
"source": [
"# Calculating Boolean NAND using a perceptron\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In fact a single perceptron can compute \"AND\", \"OR\" and \"NOT\" boolean functions.\n",
"However, it cannot compute some other boolean functions such as \"XOR\"\n",
"\n",
"WHAT CAN WE DO?\n",
"Hint: What is the significance of the NAND gate we created above\n",
"\n",
"We said a single perceptron can't compute these functions. We didn't say that about **multiple Perceptrons**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**XOR function**\n",
"\n",
"**TO DO: INSERT IMAGE HERE!!!!!!!!!!!!!!**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Activation Functions\n",
"\n",
"Some of the commonly used activation functions"
]
},
{
"cell_type": "code",
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlYAAAD8CAYAAAC1veq+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt8XPV57/vPo6vvtmTLF3zBBszFGDBGgdxLiEOAUEwCpNAbtKS8ck7SNk3bXbKzT9pNd/ch7es0fbVJm9KGDe3OhsQCgts4IUAg5EqQjMEXMDbm4pFlS7ZkSZasy8w8549ZY8byjD22Z2atmfm+Xy+hNWv9ZuYRnt9az/zWbz3L3B0REREROX01YQcgIiIiUimUWImIiIgUiBIrERERkQJRYiUiIiJSIEqsRERERApEiZWIiIhIgSixEhERESkQJVYiIiIiBaLESkRERKRA6sJ64zlz5vjSpUvDenuRo3R0dOx395YwY1CfkChRnxA5Wr59IrTEaunSpbS3t4f19iJHMbO3wo5BfUKiRH1C5Gj59gmdChQREREpECVWIiIiIgWixEpERESkQJRYiYiIiBSIEisREcmLmS02s2fMbJuZbTWzPwzWN5vZk2a2I/jdlOP5twdtdpjZ7aWNXqQ0TphYmdn9ZtZtZltybDcz+3sz22lmL5vZ6sKHKVJ6p/PZ1wFEKlQc+GN3XwG8G/iMma0A7gaedvflwNPB46OYWTPw58AVwOXAn+dKwETKWT4jVg8A1xxn+7XA8uDnLuCfTj8skUh4gFP47OsAIpXK3bvcfWOwPAi8AiwE1gIPBs0eBG7M8vSPAk+6e6+79wFPcvz+JVKWTljHyt2fM7Olx2myFvg3d3fgF2Y2y8wWuHtXgWKUELg7PYdG6R4YZXAkzuDIOIdG44yMJ0m4k0gkiSedRNKP/Haf8Bp4ltfN8l65gzjtvyPTe8+Zw7vPmp13+1P97ANXEhxAAMwsfQB56BRDl1M0PBbnh692s+fgYYZGE0fWH/PJKvBnrVxcef5cVi85tZw/6BuXAs8D8zL2+XuBeVmeshDYnfE4Fqyb+Lp3kfqiwpIlS04pNpGTFU8k+Z8bXuXX3rWY8+ZPP63XKkSB0Fyd5ZjESh0mugZGxnlq2z6e39XLpt0HefPAEKPxZNhhYVa416qvrTmpxCoPuT77eR1AQH2imPqGxlj7tZ/ydu9wXu0L+VkrF81TG04psTKzacAjwOfcfcAy/ue5u5vZKWeq7n4fcB9Aa2trdWa8UnI/3rGf+3/6Bpcva4pEYpU3dZjo2Tcwwt8/vYN1HTHG4klmTaln9ZImPnjuHBY1TWHejEnMmFzHjEn1TGuso7G+hrqaGupqjNpaS/2uMWrNqMlyZMp2sLJqPILloD5RPP/y4110HjzM/Xe0cvmy2Uypr6WmRp+902Vm9aSSqm+6+6PB6n3pMxXBqG13lqd2khrNTVsEPFvMWEXy1dYRo2lKPVedn22w9eQUIrHqBBZnPF4UrJOI+86LnXzxsc2MJZLcfNkibmldzKpFs3TwyV+uz74OIBHw/a17ee/Zswuyo5QUS30r+gbwirv/bcam9cDtwL3B78ezPP0J4H9mzDe8GvhCEcMVycvB4TGe3LaPX79iCQ11p18soRDlFtYDvx1cIfVuoF/zq6Lv6z96nc99axMrzpjB05+/kv/3ExezekmTkqqTk+uz/wRwtZk1BQeRq4N1UiKHRuPs6hniXUubww6l0rwP+C3gKjPbFPxcRyqh+oiZ7QDWBI8xs1Yz+1eAYM7hXwIvBD/3pOchioRp/Ut7jgwwFMIJR6zM7CFS377nmFmM1NVO9QDu/nVgA3AdsBMYBn6nIJFJ0Ty6Mca933uV6y9ewFd+bRX1tSpnls2pfvbdvdfM0gcQ0AGk5F7bNwjA+ac5V0KO5u4/AXJ9+/pwlvbtwKcyHt8P3F+c6EROTVtHjAsWzGDlwpkFeb18rgq87QTbHfhMQaKRotvZfYj/+thmrljWzN/92irqlFTldDqffR1AwtV1cASAxc1TQo5ERKJs+95BXo718/9cv6Jgr6mjahVxd/77f2ylobaGf7jtUiVVUrH2DaQSq3kzJoUciYhEWVvHbupqjBtXnVGw19SRtYr88NVufrxjP3/0kXOZqwOOVLDuwVHqa42mKfVhhyIiETWeSPLYi3u46vy5zJ7WWLDXVWJVRb7+o9dZ1DSZ33z3mWGHIlJUPYOjtExrVGkPEcnpudd62H9otGCT1tOUWFWJzbF+Xnizjzveu1ST1aXiDY6MM2OyRqtEJLd17TFmT23gQ+fPLejr6ghbJb75/FtMaajlk+9afOLGImXu0GicaY0lrX8sImWkd2iMp1/dx42XLiz4YIMSqyownkjy/a17uXrFPGZM0rd4qXyDI3GmT1JiJSLZPb6pk/GEF/w0ICixqgo/e/0AB4fH+djFhbvqQSTKDo3GmaYvESKSQ1tHjJULZ3DBghkFf20lVlVgw8tdTG+s4wPL54QdikhJDI6M61SgiGS1bc8AW/cMcPPqwo9WgRKriufu/HhHDx84dw6T6mvDDkekJAZH4szQqUARyaKtI0Z9rXHDqoVFeX0lVhXurQPD7Okf4b1na7RKqkMi6YzGk0xu0BcJETnaWDzJdzZ1suaCeTRPbSjKeyixqnA/fX0/AO89e3bIkYiUxmg8AaARWhE5xrPbu+kdGivKpPU0JVYV7mevH2D+jEksmzM17FBESmJkPAnApDrt3kTkaOs6YsyZ1sivnNtStPfQnqfCtb/ZyxVnNasCtVSN9IhVo0asRCTD/kOjPPNqN59YvbCo98pVYlXBugdG2DcwyiWLZoUdikjJHBmxqtfuTUTe8fimPcSTxaldlUmXzVSwzZ39AFy8aGbIkYiUzpERqzqNWBWamd0PXA90u/vKYN23gPOCJrOAg+6+Kstz3wQGgQQQd/fWkgQtQuoK+XXtu7lk0UzOnTe9qO+lr3QV7OVYPzUGK84ofAG0amBm15jZdjPbaWZ3Z9n+FTPbFPy8ZmYHM7YlMratL23k1U0jVkX1AHBN5gp3/zV3XxUkU48Ajx7n+R8K2iqpkpLaumeAV/cOFn20CjRiVdE2d/ZzztxpTGnQP/PJMrNa4GvAR4AY8IKZrXf3bek27v5HGe1/H7g04yUOZ/vWLsU3Mh5cFagRq4Jz9+fMbGm2bZaayPlJ4KpSxiSSj7aOGA21NfzqJcW/A4m+0lWwzZ39rFyo04Cn6HJgp7vvcvcx4GFg7XHa3wY8VJLI5LhG46kRq0aNWJXaB4B97r4jx3YHfmBmHWZ2Vwnjkio3Gk/wnU2dfOTCecyaUpzaVZm056lQfUNj9AyOcsF8nQY8RQuB3RmPY8G6Y5jZmcAy4IcZqyeZWbuZ/cLMbixemDJResRKc6xK7kRfLt7v7quBa4HPmNkHszUys7uCvtPe09NTjDilyjzzajcHh8dLchoQlFhVrJ09hwA4Z960kCOpCrcCbe6eyFh3ZjCP5NeBvzOzs7M9UQeRwkuPWGmOVemYWR3wCeBbudq4e2fwuxt4jNSocLZ297l7q7u3trQUr9aQVI917THmzWjkg8tL83nSnqdC7dgXJFYtSqxOUSewOOPxomBdNrcy4Zt6xkFkF/AsR8+/ymyng0iBjWrEKgxrgFfdPZZto5lNNbPp6WXgamBLCeOTKtU9OMKzr/Xw8UsXUVtTmnqOSqwq1M7uQ0yur2XhrMlhh1KuXgCWm9kyM2sglTwdc3WfmZ0PNAE/z1jXZGaNwfIc4H3AtonPleKIJx2A+iIWAKxWZvYQqc/6eWYWM7M7g03HfLkwszPMbEPwcB7wEzN7Cfgl8F13/36p4pbq9fiLe0iUoHZVJl0uVqF29hzi7LlTqSlRhl5p3D1uZp8FngBqgfvdfauZ3QO0u3s6yboVeNjdPePpFwD/bGZJUl9e7s28mlCKazyROhVYV6vPfqG5+2051t+RZd0e4LpgeRdwSVGDE5nA3VnXsZtLl8zinLmlO3ujxKpC7dw3yOXLmsMOo6y5+wZgw4R1X5rw+C+yPO9nwEVFDU5yGk8EI1Y1GrESqWabO/t5bd8h/urjK0v6vtrzVKCh0Th7+kdKmqGLREU8GLGqr9OIlUg1a+uI0VhXw/UXF792VSYlVhXo7d5hAJbOmRpyJCKld+RUoEasRKrWyHiCxzft4aMXzmfm5PqSvrf2PBVod5BYLW6aEnIkIqV35FSg5liJVK2nX+mm/3DpaldlUmJVgXb3HQZgSbMSK6k+8WSSuhojdYcVEalG6zp2s2DmJN53zpySv7cSqwq0u3eYaY11zJpS2uFPkSgYT7iuCBSpYvsGRnjutR4+sXphyWpXZVJiVYF29w6zuHmKvrFLVRpPJFXDSqSKPfZiJ0mHm1aX/jQg5JlYmdk1ZrbdzHaa2d1Zti8xs2fM7EUze9nMrit8qJKvt3uHWdykwqBSnZRYiVQvd2dd+25az2zirJDuPHLCvY+Z1QJfI3XjzBXAbWa2YkKz/wZ8290vJVUw8R8LHajkx92J9R1mseZXSZWKJ1wT10Wq1KbdB3m9ZyiUSetp+XytuxzY6e673H0MeBhYO6GNAzOC5ZnAnsKFKCdj/6ExDo8nNHFdqtZ4wlVqQaRKtXXEmFRfw8cuXhBaDPlUXl8I7M54HAOumNDmL4AfmNnvA1NJ3ZBTQhDrS5VaWKRTgVKlxhNJGuqUWIlUm5HxBOtf2sO1KxcwfVJ4F28Vau9zG/CAuy8idW+ofzezY17bzO4ys3Yza+/p6SnQW0umvf0jACyYqcRKqlO63IKIVJcfbNvH4Eg81NOAkF9i1Qkszni8KFiX6U7g2wDu/nNgEnBM8Qh3v8/dW929taWl5dQiluPqOpJYTQo5EpFwpMotaMRKpNqsa9/NwlmTec9Zs0ONI5+9zwvAcjNbZmYNpCanr5/Q5m3gwwBmdgGpxEpDUiHYOzBCY12NalhJ1RpPJGnQ5HWRqtLVf5if7NzPTasXUhPyiPUJEyt3jwOfBZ4AXiF19d9WM7vHzG4Imv0x8Htm9hLwEHCHu3uxgpbcuvpHWDBzkmpYSdWKa8RKpOo8urETd7gp5NOAkN/kddx9A7BhwrovZSxvA95X2NDkVOztP8x8nQaUKhZPJkOptiwi4XB32jpiXL6smTNnTw07HFVerzSpEStNXJfqlUi6Jq8XiZndb2bdZrYlY91fmFmnmW0KfrIWiD5RoWmRU7Xx7T7e2B9u7apMSqwqSDLp7BsY0YhVAeVx14E7zKwn46DyqYxtt5vZjuDn9tJGXr0SSdeIVfE8AFyTZf1X3H1V8LNh4sY8C02LnJK2jhiT62u57qLwaldlyutUoJSHA0NjjCdcVwQWSMbB4COk6re9YGbrg1Pfmb7l7p+d8Nxm4M+BVlIFdDuC5/aVIPSqpsSqeNz9OTNbegpPPVJoGsDM0oWmJ/YlkZNyeCzBf7zUxXUXLWBaYzRSGo1YVZCu/sMAzJ+hxKpA8rnrQC4fBZ50994gmXqS7N/0pcASrlOBIfhscJ/Y+82sKcv2bIWmF5YmNKlkT2zdy6HR8GtXZVJiVUHSxUF1KrBg8j0Y3BQcVNrMLF3zTQeSkMQTTo2uii2lfwLOBlYBXcD/dzovpkLScjLaOmIsaprMFcuaww7lCCVWFaTn0CgAc6crsSqh/wCWuvvFpEalHjyZJ+sgUnhJd+pUx6pk3H2fuyfcPQn8C6mR3onyKTSdfj0Vkpa8dB48zE9f38/Nly0KvXZVJiVWFaRnMJVYzZ7WEHIkFeOEBwN3P+Duo8HDfwUuy/e5wfN1ECmweFIjVqVkZpkzhj8ObMnSLJ9C0yIn5dGOWKp21eronAYEJVYVpWdwlOapDdSrOGKhnPBgMOGgcgOpIrqQKqh7tZk1BXNOrg7WSZElVW6haMzsIeDnwHlmFjOzO4G/NrPNZvYy8CHgj4K2Z5jZBshdaDqUP0IqgrvTtjHGu89qZnHzlLDDOUo0ptBLQew/NMocjVYVjLvHzSx9MKgF7k/fdQBod/f1wB8EdyCIA73AHcFze83sL0klZwD3uHtvyf+IKhRPeqROC1QSd78ty+pv5Gi7B7gu4/ExhaZFTtULb/bx1oFh/uCq5WGHcgwlVhWkZ3CUlumNYYdRUfK468AXgC/keO79wP1FDVCOkUw6tToVKFLR2jp2M7Whlmsvmh92KMfQOaMKsv/QGHOmKbGS6hZPavK6SCUbHovz3Ze7+NjFC5jSEL3xISVWFcLdUyNWSqykyiVdk9dFKtn3Nu9laCzBzZctPnHjECixqhBDYwkOjyeYo1OBUuXimrwuUtHaOmKcOXsK71qarRZt+JRYVYj9QakFjVhJtUto8rpIxdrdO8zPdx3g5tWLsIiOTCuxqhDp4qCavC7VLqERK5GK9cjGGGbwiQjdwmYiJVYVIj1ipcnrUu00YiVSmZJJ55GNMd579mwWzpocdjg5KbGqEBqxEknRiJVIZXr+jV529x7mlohOWk9TYlUhegZHqTFonqoCoVLdEq46ViKVqK0jxvTGOj56YfRqV2VSYlUh9h8apXlqI7X6pi5VLJl03KG2Rrs2kUpyaDTOhs1dXH/JAiY31IYdznFp71MhegZ1OxuRhDsAul2mSGXZsLmLw+MJbo7wpPU07X4qRM+hMc2vkqqXSKYTK+3aRCpJW0eMs+ZMZfWSaNauyqS9T4XoHRrVFYFS9d5JrEIOREQK5q0DQ/zyjV5uuiy6tasyafdTIfqGxmmaolOBUt3iGrESqTiPdAS1q1YvDDuUvGjvUwFG4wkOjcZpnlofdigioUqmE6vof6kVkTykald18v5z5rBgZnRrV2VSYlUBDg6PA9CkUgtS5Y6MWOlcoEhF+PmuA3QePMwtrdGuXZVJe58K0Ds0BkCzTgVKlUumrwosg3kY5cjM7jezbjPbkrHub8zsVTN72cweM7NZOZ77ppltNrNNZtZeuqilnLV1xJg+qY6rV8wLO5S8KbGqAH1BYqURq8Iys2vMbLuZ7TSzu7Ns/7yZbQsOKE+b2ZkZ2xLBAWSTma0vbeTVK67J68X2AHDNhHVPAivd/WLgNeALx3n+h9x9lbu3Fik+qSADI+N8b0sXv3rJGUyqj3btqkza/VSA3uFgxEqJVcGYWS3wNeBaYAVwm5mtmNDsRaA1OKC0AX+dse1wcABZ5e43lCRoeWeOlSavF4W7Pwf0Tlj3A3ePBw9/AUS/0JCUhQ0vdzEynuSWMqhdlUl7nwpwZMRKpwIL6XJgp7vvcvcx4GFgbWYDd3/G3YeDhzqgRIBGrEL3u8D3cmxz4Adm1mFmd5UwJilTbR0xzm6ZyqrFWc8uR5Z2PxWgdyg1eX3WFF0VWEALgd0Zj2PBulzu5OgDyiQzazezX5jZjbmeZGZ3Be3ae3p6Ti9iUYHQEJnZF4E48M0cTd7v7qtJjQJ/xsw+mON11CeEXT2HaH+rj1taF5dF7apM2vtUgL7hMWZMqqNeX9NDYWa/CbQCf5Ox+sxgHsmvA39nZmdne6673+fure7e2tLSUoJoK9uRxKrMdsTlzszuAK4HfsM9uIJgAnfvDH53A4+RGhXO1k59QnhkY4wag49fWh61qzLldSQ+0STeoM0ng4m8W83s/xQ2TDme3qExza8qvE4g8/reRcG6o5jZGuCLwA3uPppen3EQ2QU8C1xazGAl5Z0RKyVWpWJm1wD/hVQfGM7RZqqZTU8vA1cDW7K1FUkknUc3dvLBc1uYN2NS2OGctBMmVvlM4jWz5aSuBHmfu18IfK4IsUoOfcNjuiKw8F4AlpvZMjNrAG4Fjrq6z8wuBf6Z1AGlO2N9k5k1BstzgPcB20oWeRVTYlVcZvYQ8HPgPDOLmdmdwFeB6cCTwVWwXw/anmFmG4KnzgN+YmYvAb8Evuvu3w/hT5Ay8NOd++nqH+GWy8qndlWmujzaHJnEC2Bm6Um8mQeK3wO+5u59cGSoV0qkd2iM+WWY1UeZu8fN7LPAE0AtcL+7bzWze4B2d19P6tTfNGBdMAfg7eAKwAuAfzazJKkvL/e6uxKrEkgEZ6HqlFgVhbvflmX1N3K03QNcFyzvAi4pYmhSQdo6YsycXM+HL5gbdiinJJ/EKtsk3ismtDkXwMx+Suog9BfZvo0EV4LcBbBkyZJTiVey6Bsa44IFM8IOo+K4+wZgw4R1X8pYXpPjeT8DLipudJJNIpkEoEaJlUhZ6j88zhNb9/LJ1sVlVbsqU6FmO9cBy4ErgduAf8lWfVeTEoujd1hzrEQAEqm8SiNWImXqP1/ew2g8yS2t5Vu9Jp/EKp9JvDFgvbuPu/sbpKrvLi9MiHI8h8cSjIwnVcNKBIinR6x0VaBIWWrriHHuvGlctHBm2KGcsnwSqxNO4gW+Q2q0Kj1Z91xgVwHjlBzeqbquGlYiyfSIVa0SK5Fys7N7kBffPsjNly0qu9pVmU6YWAW3KkhP4n0F+HZ6Eq+ZpW/V8QRwwMy2Ac8Af+ruB4oVtLxDVddF3pGevK4RK5Hy09bRSW2NcWMZ1q7KlM/k9Xwm8Trw+eBHSqh3SPcJFElLT17XHCuR8pJIOo+9GOPKc1uYO728r3JXqe4y1xecClQdK5F3Jq+rjpVIeXluRw/7BkbLetJ6mhKrMndkxEqnAkWOjFgpsRIpL20dMZqm1HPV+fPCDuW0KbEqc31DY9QYzJisyesiGrESKT/9w+M8uXUfa1ctpKGu/NOS8v8Lqlzv8BizpjToQCKCyi2IlKP1L3Uylkhy82XlfxoQlFiVvb6hcZqmaLRKBCCpW9qIlJ22jhjnz5/OhWdUxh1ElFiVud4hVV0XSYsndBNmkXLy2r5BXor1l33tqkxKrMpc3/CYaliJBNIjVkqsRMpDW0eMugqoXZVJiVWZ04iVyDviSSVWIuUinkjy6MZOPnT+XOZMaww7nIJRYlXG3D01YqXESgSApBIrkbLxo9d62H9olFsqZNJ6mhKrMnZoNM54wlXDSiRwZMSqQuZqiFSyto4Ys6c28KHz54YdSkEpsSpjfUPjgKquF5OZXWNm281sp5ndnWV7o5l9K9j+vJktzdj2hWD9djP7aCnjrlaJdGKlmzAXhZndb2bdZrYlY12zmT1pZjuC3005nnt70GaHmd1euqglivqGxnjqlVTtqvraykpFKuuvqTK9w+n7BKrcQjGYWS3wNeBaYAVwm5mtmNDsTqDP3c8BvgJ8OXjuCuBW4ELgGuAfg9eTIkpoxKrYHiD1ec50N/C0uy8Hng4eH8XMmoE/B64ALgf+PFcCJtXh8U2djCe8Im5hM5ESqzLWF9zORlcFFs3lwE533+XuY8DDwNoJbdYCDwbLbcCHLXXN8FrgYXcfdfc3gJ3B60kRJXRVYFG5+3NA74TVmX3gQeDGLE/9KPCku/e6ex/wJMcmaFJF2jbGuPCMGVywoDJqV2VSYlXGjtwnUKcCi2UhsDvjcSxYl7WNu8eBfmB2ns+VAkuojlUY5rl7V7C8F8h2szf1Bznila4BtnQOVEyl9YmUWJWxvuBUoOZYlS8zu8vM2s2svaenJ+xwyt6RESudCgyFuzvgp/Ma6hOVr60jRn2tsXZVZebWSqzKWO/QGHU1xvTGurBDqVSdwOKMx4uCdVnbmFkdMBM4kOdzcff73L3V3VtbWloKGHp1SiQdM6jRiFUp7TOzBQDB7+4sbfLqD6A+UenGE0m+82InHz5/XsWebVFiVcbSNawq5TYAEfQCsNzMlplZA6nJ6OsntFkPpK9wuhn4YfCtfT1wa3DV4DJgOfDLEsVdtRJJ130CSy+zD9wOPJ6lzRPA1WbWFExavzpYJ1XmmVe7OTA0VrGnAQE01FHGeofGVMOqiNw9bmafJXUAqAXud/etZnYP0O7u64FvAP9uZjtJTeq9NXjuVjP7NrANiAOfcfdEKH9IFUkknRp90SgaM3sIuBKYY2YxUlf63Qt828zuBN4CPhm0bQU+7e6fcvdeM/tLUl9WAO5x94mT4KUKtHXEmDOtkV85r3JHI5VYlbG+oXGaVGqhqNx9A7BhwrovZSyPALfkeO5fAX9V1ADlKBqxKi53vy3Hpg9nadsOfCrj8f3A/UUKTcrAgUOj/PDVbn7nfUsrrnZVpsr9y6pA77DuEyiSKZ50za8SiajvbNpDPOncfNniEzcuY0qsyljf0JhqWIlkSLqr1IJIRLV1xLh40UzOmz897FCKSolVmUomUzdg1oiVyDviOhUoEklb9/TzSlfl1q7KpMSqTA2MjJN0VV0XyZTU5HWRSFrXHqOhtoYbLjkj7FCKTolVmUpXXdfkdZF3aMRKJHrG4kke39TJR1bMY1YVDAYosSpT79zOpjHkSESiI6nJ6yKR88NXu+kbHq+K04CgxKpsHUmsqiD7F8lXwjViJRI1bR27mTu9kQ8snxN2KCWhxKpMpe8T2DxNiZVImsotiERLz+Aoz2zv4eOrF1JXwbWrMlXHX1mBDmjESuQYSc2xEomU77zYSSLp3FIlpwFBiVXZ6hsaY3J9LZMbasMORSQy4roqUCQy3J22jhirFs/inLmVXbsqkxKrMnVgSDWsRCZKJp26WiVWIlGwpXOA7fsGq2bSelpeiZWZXWNm281sp5ndfZx2N5mZBzfflCLqU2Ilcox40qnViJVIJKzr2E1DXQ2/WgW1qzKdMLEys1rga8C1wArgNjNbkaXddOAPgecLHaQcq3dojCYlViJH0S1tRKJhNJ7g8U17+OiF85k5ubrqLeYzYnU5sNPdd7n7GPAwsDZLu78EvgyMFDA+yaF3eIzmKdX1YRU5kXhCiZVIFDz9Sjf9h6undlWmfBKrhcDujMexYN0RZrYaWOzu3y1gbHIcfUPjKg4qMkFCI1YikbCufTfzZ0zi/edUR+2qTKc9ed3MaoC/Bf44j7Z3mVm7mbX39PSc7ltXrdF4gkOjcZp1OxuRoySSSqxEwtY9MMKPXuvhE6sXVmV/zCex6gQWZzxeFKxLmw6sBJ41szeBdwPrs01gd/f73L3V3VtbWlpOPeoq1zc0Duh2NsVkZs1m9qSZ7Qh+N2Vps8rMfm5mW83sZTP7tYxtD5jZG2a2KfhZVdq/oDqlEitd7FyneX2vAAAaVUlEQVRqZnZexmd9k5kNmNnnJrS50sz6M9p8Kax4pbgee7GTpFOVpwEB6vJo8wKw3MyWkUqobgV+Pb3R3fuBI2N9ZvYs8Cfu3l7YUCXtwNAogEasiutu4Gl3vze4EvZu4M8mtBkGftvdd5jZGUCHmT3h7geD7X/q7m0ljLnqJZKOqi2UnrtvB1bBkQueOoHHsjT9sbtfX8rYpLTcnXUdMS47s4mzWqaFHU4oTvjVzt3jwGeBJ4BXgG+7+1Yzu8fMbih2gHIsjViVxFrgwWD5QeDGiQ3c/TV33xEs7wG6AQ3FhkinAiPhw8Dr7v5W2IFI6b0U62dn96GqHa2C/EascPcNwIYJ67IO47r7lacflhyPRqxKYp67dwXLe4F5x2tsZpcDDcDrGav/Kjjd8TRwt7uPFiVSOUKJVSTcCjyUY9t7zOwlYA+pMxtbJzYws7uAuwCWLFlStCClONa172ZSfQ0fu3hB2KGERpMRylBf+j6BGrE6LWvWrGHlypWsXLkS4EIz2xL8HFVOxN0d8FyvY2YLgH8Hfsfdk8HqLwDnA+8Cmjn2NGL6ubqgo4B0VWC4zKwBuAFYl2XzRuBMd78E+AfgO9leQ3Nxy9fIeIL1L+3hmgvnM2NS9X7xz2vESqKld2gMM6qu6FqhPfXUU0eWzWyru7dmPN5nZgvcvStInLqzvYaZzQC+C3zR3X+RXp8x2jVqZv8L+JNsz3f3+4D7AFpbW3Mmb5IfTV4P3bXARnffN3GDuw9kLG8ws380sznuvr+kEUrRPLltH4MjcW6+bPGJG1cw7YHKUO/wGE1TGvTNvLjWA7cHy7cDj09sEHw7fwz4t4mT1INkDDMzUvOzthQ1WgE0eT0CbiPHaUAzmx/0h/Sp8xrgQAljkyJb1xHjjJmTeO/Zs8MOJVRKrMpQ39A4Taq6Xmz3Ah8xsx3AmuAxZtZqZv8atPkk8EHgjixlFb5pZpuBzaSumv0fpQ2/OmnEKjxmNhX4CPBoxrpPm9mng4c3A1uCOVZ/D9wanGaXCrC3f4Sf7OjhpssWUVPlX/p1KrAMHRgaZbbmVxWVux8gdXXTxPXtwKeC5f8N/O8cz7+qqAFKVqnEKuwoqpO7DwGzJ6z7esbyV4GvljouKY1HX4yRdLhpdfVeDZimXVAZ6hsap0lXBIocIzV5Xbs1kVJyd9raY1y+tJmlc6aGHU7otAcqQweGxnRFoEgWGrESKb2Nbx9k1/6hqq5dlUm7oDLj7vQNj6mGlUgWiaRTpxErkZJq64gxub6W66q4dlUm7YHKzMDhOImka8RKJItE0qmx6p44K1JKh8cS/OdLe7j2ovlMa9S0bVBiVXZ6Do0AMGdaQ8iRiERPIunUqd6CSMn8YNteBkfjOg2YQYlVmekeTN0VpWW6RqxEJtKIlUhprWuPsahpMu9eVt21qzIpsSoz+w+lbmczV4mVyDES7tRVeQ0dkVLpPHiYn76+n5tWq3ZVJiVWZaYnGLGaM02JlUgmd0+NWGkHL1ISj22M4Y5OA06gxKrM7D80Sn2t6T6BIhMkgxreGrESKT53p60jxrvPamZx85Sww4kUJVZlpmdwlDnTGjHNIxE5SjyZBNA9NEVKoP2tPt48MFz1N1zORolVmekZHNXEdZEsgrxKk9dFSqCtPcaUhlquXTk/7FAiR4lVmdl/aJQWza8SOUZ6xEqnAkWKa3gszn++vIePXbSAqapddQwlVmUmfSpQRI52ZMRKiZVIUX1/y16GxhKatJ6DEqsykkw6B4bGdCpQJIuEp2ava8RKpLjWtcdY0jyFy5c1hx1KJCmxKiN9w2Mkkq6q6yJZpE8FasRKpHh29w7z810HuPmyRbqIKgclVmWk51C66vqkkCMRiZ70qUCNWIkUz6MbOzGDT6xeGHYokaXEqozsH0xVXdepwOIzs2Yze9LMdgS/m3K0S5jZpuBnfcb6ZWb2vJntNLNvmZmGGYvsSLkFfYsOhZm9aWabg77QnmW7mdnfB33iZTNbHUaccuqSSadt427ee/ZsFjWpdlUuSqzKiG7AXFJ3A0+7+3Lg6eBxNofdfVXwc0PG+i8DX3H3c4A+4M7ihivpESvVsQrVh4K+0Jpl27XA8uDnLuCfShqZnLZfvtnL7t7DmrR+AkqsykiPbsBcSmuBB4PlB4Eb832ipSYeXAW0ncrz5dSoQGjkrQX+zVN+AcwyswVhByX5a+uIMa2xjmsu1D/b8SixKiM9g6M01tUwTXVDSmGeu3cFy3uBeTnaTTKzdjP7hZmlk6fZwEF3jwePY0DWCQlmdlfw/Paenp6CBV+NksFVgUqsQuPAD8ysw8zuyrJ9IbA743HWfqE+EU1Do3E2bO7i+osXMLmhNuxwIk1H6DKyd2CUBTMn6UqMAlmzZg179+5NP7zQzLYEy1/MbOfubmae42XOdPdOMzsL+KGZbQb6843B3e8D7gNobW3N9R6Sh3hSiVXI3h/0hbnAk2b2qrs/d7Ivoj4RTRs2dzGs2lV5UWJVRvb2H2b+TF0RWChPPfXUkWUz25o5L8TM9pnZAnfvCk5XdGd7DXfvDH7vMrNngUuBR0id5qgLRq0WAZ3F+0sEIKHEKlQZfaHbzB4DLgcyE6tOIPPGcuoXZaStI8ayOVO57Mys1/FIBp0KLCNd/SMsmDk57DCqxXrg9mD5duDxiQ3MrMnMGoPlOcD7gG3u7sAzwM3He74U1pHESiO6JWdmU81senoZuBrYMqHZeuC3g6sD3w30Z5xulwh7+8Awz7/Rq9pVeVJiVSaSSWffwIhGrErnXuAjZrYDWBM8xsxazexfgzYXAO1m9hKpROped98WbPsz4PNmtpPUnKtvlDT6KnQksarVjj8E84CfBH3hl8B33f37ZvZpM/t00GYDsAvYCfwL8H+HE6qcrLaNMczg45eqdlU+dCqwTBwYGmM84SxQYlUS7n4A+HCW9e3Ap4LlnwEX5Xj+LlKnQqRENGIVnuDzfkmW9V/PWHbgM6WMS05fMuk80hHj/efM4YxZOmOSD41YlYm9/akaVvNnKLESySY9eV2V10UK5xe7DtB5ULWrTkZeiZWZXWNm24OKuccUSjSzz5vZtqCa7tNmdmbhQ61uXf2HAXQqUCSHeCKVWNXX6fuiSKG0dcSYPqmOj144P+xQysYJ90BmVgt8jVTV3BXAbWa2YkKzF4FWd7+YVFHEvy50oNVu70AwYqXESiSr8USqQKhGrEQKY3BknA1buvjVS85gUr1qV+Urn692lwM73X2Xu48BD5OqoHuEuz/j7sPBw1+QuoxWCqirf4S6GmPOVFVdF8kmnVjV12rESqQQNmzuYmQ8qdOAJymfPVBe1XIz3Al8L9sGVdQ9dV0HDzNvxiRq9G1cJKvx9KlAJVYiBdHWEeOslqlcunhW2KGUlYLugczsN4FW4G+ybXf3+9y91d1bW1paCvnWFS/Wd5iFTboiQySX9L0C61RuQeS0vbF/iBfe7OOWyxardtVJyiexyqtarpmtIXUrkBvcfbQw4Una7r5hljRPCTsMkchKj1g1aMRK5LQ90hGjxuATq1W76mTlswd6AVhuZsvMrAG4lVQF3SPM7FLgn0klVVlv/SGnbmQ8wb6BURY3KbESyeXI5HWNWImclkTSeWRjjA+e28I8lfg5aSdMrIJ7nX0WeAJ4Bfi2u281s3vM7Iag2d8A04B1ZrbJzNbneDk5BbG+VKmFJbN1KlAkl/iRqwI1YiVyOn72+n66+kc0af0U5VV53d03kLodQea6L2UsrylwXJJhd2/qgkuNWInkNqZTgSIF0dYRY8akOtZcMC/sUMqS9kBlYHdfKrHSHCuR3OI6FShy2voPj/P9LXtZu2qhaledIiVWZWB37zCNdTW0TFcNK5Fc0re0UbkFkVP33Ze7GI2rdtXp0B6oDLzdO8zi5im65FXkOMbi6QKh6icip6qtYzfL507j4kUzww6lbCmxKgNvHVCpBZETiSeT1NWYvoCInKKd3YfY+PZBbmldpH50GpRYRVwi6ezaP8Q5c6eFHYpIpI0nXPOrRE7DIxtj1NYYN16q2lWnQ4lVxO3uHWYsnuScFiVWIscznkhSr1ILoTCzxWb2jJltM7OtZvaHWdpcaWb9QUmeTWb2pWyvJeFIJJ1HN8a48twW5k5X7arTob1QxO3sPgTA2RqxKikzazazJ81sR/C7KUubD2UcJDaZ2YiZ3Rhse8DM3sjYtqr0f0V1GU8kqa/TLi0kceCP3X0F8G7gM2a2Iku7H7v7quDnntKGKMfz4x097BsY1aT1AtBeKOJ29qQSK50KLLm7gafdfTnwdPD4KO7+TPogAVwFDAM/yGjypxkHkU0libqKxRNOnW5SHgp373L3jcHyIKli0jqfVEbaOmLMmlLPVRfMDTuUsqfEKuJ2dh+iZXojMyfXhx1KtVkLPBgsPwjceIL2NwPfc/fhokYlOY3FkzTWa5cWNjNbClwKPJ9l83vM7CUz+56ZXVjSwCSn/uFxfrBtHzeuWkhjnWpXnS7thSJuZ/chza8Kxzx37wqW9wInKkF8K/DQhHV/ZWYvm9lXzCxrETIzu8vM2s2svaen5zRDrm4j8YQOCiEzs2nAI8Dn3H1gwuaNwJnufgnwD8B3cryG+kSJrX95D2OqXVUwSqwiLJl0duwb5Nx5SqyKYc2aNaxcuZKVK1cCXGhmW4KftZnt3N0Bz/U6ZrYAuIjU/TTTvgCcD7wLaAb+LNtz3f0+d29199aWlpbT+4Oq3Mh4kkkasQqNmdWTSqq+6e6PTtzu7gPufihY3gDUm9mcLO3UJ0qsrSPG+fOnc+EZM8IOpSLkda9ACceu/UMMjSVYuVCF2orhqaeeOrJsZlvdvTXj8T4zW+DuXUHi1H2cl/ok8Ji7j6dXZIx2jZrZ/wL+pLDRy0SjGrEKjaWKHn0DeMXd/zZHm/nAPnd3M7uc1Bf7AyUMU7LYsW+Ql3Yf5L997ALVrioQJVYRtrnzIAAXL5oVciRVaT1wO3Bv8Pvx47S9jdQI1REZSZmRmp+1pViBSopGrEL1PuC3gM1mlr5Q478CSwDc/euk5iH+X2YWBw4DtwajwRKito4YdapdVVBKrCLs5Vg/k+prOLtlatihVKN7gW+b2Z3AW6RGpTCzVuDT7v6p4PFSYDHwownP/6aZtQAGbAI+XZqwq9doPKGLPELi7j8h9Vk/XpuvAl8tTUSSj3giyaMvdnLleXOZM033oi0UJVYRtjnWz4VnzKRON5UtOXc/AHw4y/p24FMZj98ky2Xl7n5VMeOTY2nESuTkPLejh57BUW5p1aT1QtJeKKLiiSRb9wxwkeZXieRFc6xETk5bR4zmqQ186DzVriokJVYRtWXPAIfHE6w+85iC3yKShUasRPLXNzTGU9u6uXHVQhp0x4KC0v/NiPrZ6/sBeM9Zs0OORKQ8jI5rxEokX+tf2sNYQrWrikGJVUT9/PUDnDdvOi3TNaFQJB8jqrwukre2jhgrFsxghWpXFZz2QhE0Gk/wwpu9vPccjVaJ5COeSDIWTzK5XiNWIify6t4BNnf2a9J6kSixiqCfv36AkfEk7z/nmKLEIpLF0GgCgOmTVG5B5ETa2mPU1xprV6l2VTEosYqgDZu7mN5Yx/uXK7ESycfASKro/fRGVZAROZ7xRJLvbOrkqvPn0jy1IexwKpISq4gZTyR5Yus+1qyYp4m4Ink6NBoHYPokJVYix/Ps9h72HxrjlssWhx1KxVJiFTHPbu+h//A41120IOxQRMpGOrGapsRK5LjaOnYzZ1oDv3KebnBdLEqsIuaBn73BgpmTuFIfepG8DaZPBWqOlUhOBw6N8vQr3Xz80oXU644eRaP/sxHy6t4BfrrzAL/1njP1oRc5CYMjwYiV5liJ5PT4pj3Ek85Nql1VVDp6R8hff3870xrruO1dS8IORaSs9AyOAtCiG8mK5NTWEeOihTM5f75qVxWTEquIeGZ7Nz98tZvfv+ocmnSlhshJ6R4cpaGuhhmTNWIlks3WPf1s6xpQ7aoSUGIVAd0DI/zpupc5Z+407njf0rDDESk7+wZGmDejETMLOxSRSGrriNFQW8OvXnxG2KFUPH29C9nB4TF+98EXGBqN839+7wqVWBA5BXsOHmb+jElhhyESSWPxJI9v2sOaFXN1RqQE8hqxMrNrzGy7me00s7uzbG80s28F2583s6WFDrQSbdszwE3/9DNe23eIf/yN1Zw7b3rYIUnAzG4xs61mljSz1uO0y9o3zGxZ0Bd2Bn1De7MicXe27x1U/wmZjhPR9cNXu+kdUu2qUjlhYmVmtcDXgGuBFcBtZrZiQrM7gT53Pwf4CvDlQgdaKdydrXv6+S9tL3H9P/yYgZE4//a7l/Oh8+eGHZocbQvwCeC5XA1O0De+DHwl6BN9pPqIFMGu/UMMjMR1M9kQ6TgRbW0dMVqmN/IB3c2jJPI5FXg5sNPddwGY2cPAWmBbRpu1wF8Ey23AV83M3N0LGGvZiCeSDI0mODQW5+DwGJ19h9ndd5htewZof6uXtw4M01Bbwx3vXcZnrzpHtxWIIHd/BTjRnJ2sfcPMXgGuAn49aPcgqf7xT8WKt1olk84DP30TgKv05SRMOk5EVM/gKM9s7+ZT719Gncr4lEQ+idVCYHfG4xhwRa427h43s35gNrD/ZAPatmeAP3j4RdJ9zY/858gv3D1jOb3Njzye2E1ztfdjXvedR8duy/IaGevSC+PJJCPjyax/2+ypDVx2ZhO/94GzuO6iBUqoyl+uvjEbOOju8Yz1p3y30/Y3e7n70c0AR/eLND92MfNYldl2Yn85al2Ow1u29/Sj3tOzrMvWNst7ZnmfzPVHvWaW7fGEc3g8wW9csYQFMydn/wOkFEp6nPjRaz38j//cduKGwtBonETSuVm1q0qmpJPXzewu4C6AJUuy12qa3FDLeem5EvbOr/TIQXr8wCxz+ehtGFjwyI56jfTyO9ss80n5tJ/wnkxoV1sD0xrrmTapjmmNtcyYVM/CpsksbprCrCn1umopQtasWcPevXvTDy80sy3B8hfd/fFSxJBPn5jSUPdOn4Cj+kXG60zcTOZHLVvboz6JR7U9ui/kfs0sfSDHi1rWmHO/ztHvmb3PWNDPL1k8k+t1pVPFyKdPTGusY/m8aaUMq6x9ct50lmsOYsnkk1h1Apkz3hYF67K1iZlZHTATODDxhdz9PuA+gNbW1qzfj5fNmcrXfmN1HmGJnJ6nnnrqyLKZbXX3nJPUc8jVNw4As8ysLhi1ytZngPz6xIozZqhPSNSV9Dhx2ZlNXHbmZQUIW6Tw8jnh+gKwPLjKqQG4FVg/oc164PZg+WbghzpvLlUga98IPvvPkOoLkOobJRkBEwmJjhMigRMmVsE37s8CTwCvAN92961mdo+Z3RA0+wYw28x2Ap8HjrnUVqScmNnHzSwGvAf4rpk9Eaw/w8w2QO6+EbzEnwGfD/rEbFJ9RKQi6Tgh8o685li5+wZgw4R1X8pYHgFuKWxoIuFx98eAx7Ks3wNcl/H4mL4RrN9F6kopkaqg44RIiq69FBERESkQJVYiIiIiBaLESkRERKRAlFiJiIiIFIgSKxEREZECsbDKiJhZD/DWcZrM4RRudVBkiil/UYzreDGd6e4tpQxmohP0iSj+/4RoxhXFmCCacalPFF4U44piTBDNuE67T4SWWJ2ImbWfQiXsolJM+YtiXFGMKV9RjT2KcUUxJohmXFGMKV9RjT2KcUUxJohmXIWISacCRURERApEiZWIiIhIgUQ5sbov7ACyUEz5i2JcUYwpX1GNPYpxRTEmiGZcUYwpX1GNPYpxRTEmiGZcpx1TZOdYiYiIiJSbKI9YiYiIiJSVSCVWZvY3Zvaqmb1sZo+Z2ayMbV8ws51mtt3MPlriuG4xs61mljSz1gnbwozrmuB9d5pZaHeKN7P7zazbzLZkrGs2syfNbEfwu6nEMS02s2fMbFvwb/eHUYjrZEWxT0S1PwTvH3qfUH8oLvWJk45NfSJ7TMXrE+4emR/gaqAuWP4y8OVgeQXwEtAILANeB2pLGNcFwHnAs0BrxvrQ4gJqg/c7C2gI4lgR0r/bB4HVwJaMdX8N3B0s353+tyxhTAuA1cHydOC14N8r1LhO4e+IXJ+IYn8I3j8SfUL9oeh/i/pE/nGpT+SOqWh9IlIjVu7+A3ePBw9/ASwKltcCD7v7qLu/AewELi9hXK+4+/Ysm8KM63Jgp7vvcvcx4OEgnpJz9+eA3gmr1wIPBssPAjeWOKYud98YLA8CrwALw47rZEWxT0S0P0BE+oT6Q3GpT5wU9YncMRWtT0QqsZrgd4HvBcsLgd0Z22LBurCFGVdU/5+kzXP3rmB5LzAvrEDMbClwKfA8EYrrFES9T4QdU9jvfzyR+dxVUH8A9Ymov//xROazV+g+UVewyPJkZk8B87Ns+qK7Px60+SIQB74Zpbjk1Li7m1kol5+a2TTgEeBz7j5gZpGIK1MU+4T6Q/GoP5yY+kR1qbQ+UfLEyt3XHG+7md0BXA982IOTnEAnsDij2aJgXcniyqHocUX0vfOxz8wWuHuXmS0AuksdgJnVk+ow33T3R6MS10RR7BNl2B+i8P7HE/rnrlz6A6hPFFDY7388oX/2itUnInUq0MyuAf4LcIO7D2dsWg/camaNZrYMWA78MowYJwgzrheA5Wa2zMwagFuDeKJiPXB7sHw7UNJvdJb62vEN4BV3/9uoxHWyyqxPhB1TlPuE+kOBqE+cFPWJHIraJwo1w74QP6Qm9u0GNgU/X8/Y9kVSVzdsB64tcVwfJ3VuehTYBzwRkbiuI3Ulw+ukhqPD+nd7COgCxoP/T3cCs4GngR3AU0BziWN6P+DAyxmfp+vCjusU/o7I9Ymo9ofg/UPvE+oPRf9b1CdOLjb1iewxFa1PqPK6iIiISIFE6lSgiIiISDlTYiUiIiJSIEqsRERERApEiZWIiIhIgSixEhERESkQJVYiIiIiBaLESkRERKRAlFiJiIiIFMj/DzVCVit1tBkyAAAAAElFTkSuQmCC\n",
"<Figure size 720x288 with 3 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"\n",
"plt.figure(figsize=(10, 4), constrained_layout=False)\n",
"plt.subplot(1, 3, 1)\n",
"\n",
"#fig, ax = plt.subplots(1,3)\n",
"#ax=ax.flatten()\n",
"\n",
"pts=np.arange(-20,20, 0.1)\n",
"\n",
"# Sigmoid\n",
"plt.plot(pts, 1/(1+np.exp(-pts))) ;\n",
"\n",
"plt.plot(pts, np.tanh(pts*np.pi)) ;\n",
"plt.subplot(1, 3, 3)\n",
"\n",
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Suggestion Uwe:\n",
"\n",
"1. more layers might improve power of single perctptron.\n",
"\n",
"2. regrettably math show that just \"stacking\" perceptrons only adds little improvements\n",
"\n",
"3. way around: look at nature how neuron works and introduce non linear activation functions.\n",
"\n",
"4. theoretical background: universal approximation theorem.\n",
"\n",
"\n",
"\n",
"### Multi-layer preceptron neural network\n",
"Universal function theorem\n",
"\n",
"#### Gradient based learning\n",
"\n",
"#### back propogation\n",
"\n",
"#### loss function\n",
"\n",
"#### optimization\n",
"\n",
"epochs\n",
"\n",
"### Google Playground\n",
"\n",
"UWE: move up before discussing gradient stuff etc\n",
"\n",
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
"https://playground.tensorflow.org/\n",
"\n",
"<img src=\"./images/neuralnets/google_playground.png\"/>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Introduction to Keras"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"What is **Keras**?\n",
"\n",
"* It is a high level API to create and work with neural networks\n",
"* Supports multiple backends such as TensorFlow and Keras\n",
"* Very good for creating neural nets very quickly and hides awy a lot of tedious work\n",
"* Has been incorporated into official TensorFlow (which obviously only works with tensforflow)\n"
]
},
{
"cell_type": "code",
"ename": "ModuleNotFoundError",
"evalue": "No module named 'keras'",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mModuleNotFoundError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-17-0e94e6863fea>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0;31m# Say hello to keras\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mkeras\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mSequential\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 4\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mkeras\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlayers\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mDense\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mActivation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'keras'"
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
]
}
],
"source": [
"# Say hello to keras\n",
"\n",
"from keras.models import Sequential\n",
"from keras.layers import Dense, Activation\n",
"\n",
"# Creating a model\n",
"model = Sequential()\n",
"\n",
"# Adding layers to this model\n",
"# 1st Hidden layer\n",
"model.add(Dense(units=4, input_dim=4))\n",
"model.add(Activation(\"relu\"))\n",
"\n",
"# The output layer\n",
"model.add(Dense(units=1))\n",
"model.add(Activation(\"sigmoid\"))\n",
"\n",
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {},
"outputs": [],
"source": [
"# Fittind the model "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Network results on dataset used in previous notebooks\n",
"\n",
"## Network Architecture\n",
"\n",
"## CNN examples"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"TODO: \n",
"\n",
"- does keras support scikit-learn api ? (.fit and .predict methods) ?\n",
"- if yes: we could use cross validation and hyper parameter optimzation for scikit-learn to evaluae / improve keras network. \n",
" \n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.2"
},
"latex_envs": {
"LaTeX_envs_menu_present": true,
"autoclose": false,
"autocomplete": true,
"bibliofile": "biblio.bib",
"cite_by": "apalike",
"current_citInitial": 1,
"eqLabelWithNumbers": true,
"eqNumInitial": 1,
"hotkeys": {
"equation": "Ctrl-E",
"itemize": "Ctrl-I"
},
"labels_anchors": false,
"latex_user_defs": false,
"report_style_numbering": false,
"user_envs_cfg": false