01_introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 1: General Introduction to machine learning (ML)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ML = \"learning models from data\"\n",
    "\n",
    "\n",
    "### About models\n",
    "\n",
    "A \"model\" allows us to explain observations and to answer questions. For example:\n",
    "\n",
    "   1. Where will my car at given velocity stop when I break now ?\n",
    "   2. Where on the night sky will I see the moon tonight ?\n",
    "   2. Is the email I received spam ? \n",
    "   4. What article X should I recommend to my customers Y ?\n",
    "   \n",
    "- The first two questions can be answered based on existing physical models (formulas). \n",
    "\n",
    "- For the  questions 3 and 4 it is difficult to develop explicitly formulated models. \n",
    "\n",
    "### What is needed to apply ML ?\n",
    "\n",
    "Problems 3 and 4 have the following in common:\n",
    "\n",
    "- No exact model known or implementable because we have a vague understanding of the problem domain.\n",
    "- But enough data with sufficient and implicit information is available.\n",
    "\n",
    "\n",
    "\n",
    "E.g. for the spamming example:\n",
    "\n",
    "- We have no explicit formula for such a task\n",
    "- We have a vague understanding of the problem domeani, because we know that some words are specific for spam emails, other words are specific for my personal and job emails.\n",
    "- My mailbox is full with examples for spam vs non-spam.\n",
    "\n",
    "\n",
    "**In such cases machine learning offers approaches to build models based on example data.**\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## ML: what is \"learning\" ?\n",
    "\n",
    "To create a predictive model, we first must \"learn\" such a model on given data. \n",
    "\n",
    "All ML algorithms have in common that they rely on internal data structures and/or parameters. Learning then builds up such data structures or adjusts parameters based on the given data. After that such models can be used to explain observations or to answer questions.\n",
    "\n",
    "The important difference between explicit models and models learned from data:\n",
    "\n",
    "- Explicit models usually offer exact answers to questions\n",
    "- Models we learn from data usually come with inherent uncertainty."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Some history\n",
    "\n",
    "Some parts of ML are older than you might think. This is a rough time line with a few selected achievements from this field:\n",
    "\n",
    " \n",
    "    1812: Bayes Theorem\n",
    "    1913: Markov Chains\n",
    "    1951: First neural network\n",
    "    1959: first use or term \"machine learning\" AI pioneer Arthur Samuel\n",
    "    1969: Book \"Perceptrons\": Limitations of Neural Networks\n",
    "    1986: Backpropagation to learn neural networks\n",
    "    1995: Randomized Forests and Support Vector Machines\n",
    "    1998: Public appearance of ML: naive Bayes Classifier for Spam detection\n",
    "    2000+: Deep learning\n",
    "    \n",
    "So the field is not as new as one might think, but due to \n",
    "\n",
    "- more available data\n",
    "- more processing power \n",
    "- development of better algorithms \n",
    "\n",
    "more applications of machine learning appeared during the last 15 years."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Machine learning with Python\n",
    "\n",
    "Currently (2018) `Python` is the  dominant programming language for ML. Especially the advent of deep-learning pushed this forward. First releases of frameworks such as `TensorFlow` or `PyTorch` were released with`Python` support early.\n",
    "\n",
    "The prevalent packages in the Python eco-system used for ML include:\n",
    "\n",
    "- `pandas` for handling tabualar data\n",
    "- `matplotlib` and `seaborn` for plotting\n",
    "- `scikit-learn` for classical (non-deep-learning) ML\n",
    "- `tensorflow`, `PyTorch` and `Keras` for deep-learning.\n",
    "\n",
    "`scikit-learn` is very comprehensive and the online-documentation itself provides a good introducion into ML."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ML terms: What are \"features\" ?\n",
    "\n",
    "A typical and very common situation is that our data is presented as a table, as in the following example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>alcohol_content</th>\n",
       "      <th>bitterness</th>\n",
       "      <th>darkness</th>\n",
       "      <th>fruitiness</th>\n",
       "      <th>is_yummy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3.739295</td>\n",
       "      <td>0.422503</td>\n",
       "      <td>0.989463</td>\n",
       "      <td>0.215791</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.207849</td>\n",
       "      <td>0.841668</td>\n",
       "      <td>0.928626</td>\n",
       "      <td>0.380420</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.709494</td>\n",
       "      <td>0.322037</td>\n",
       "      <td>5.374682</td>\n",
       "      <td>0.145231</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.684743</td>\n",
       "      <td>0.434315</td>\n",
       "      <td>4.072805</td>\n",
       "      <td>0.191321</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4.148710</td>\n",
       "      <td>0.570586</td>\n",
       "      <td>1.461568</td>\n",
       "      <td>0.260218</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   alcohol_content  bitterness  darkness  fruitiness  is_yummy\n",
       "0         3.739295    0.422503  0.989463    0.215791         0\n",
       "1         4.207849    0.841668  0.928626    0.380420         0\n",