course_layout.md

Targeted audience

• Researchers from DBIOL and DGESS having no machine learning experience yet.
• Basic Python knowledge.
• Almost no math knowledge.

Concepts

• 3 days workshop: 2 days lectures with exercises + 0.5 day real life example walk through + 0.5 day working on own data / prepared data.
• smooth learning curve
• explain fundamental concepts first, discuss exceptions, corner cases, pitfalls late.
• plotting / pandas / numpy first. Else participants might be fight with these basics during coding sessions and will be disctracted from the actual learning goal of an exercise.
• jupyter notebooks / conda, extra notebooks with solutions.
• use prepared computers in computer room, setting up personal computer during last day if required.
• exercises: empty holes to fill

Course structure

Home prep

Introductions to NumPy, Pandas and Matplotlib (plus Python, if needed).

Prep materials to send out:

• Python, ca. 6h: https://siscourses.ethz.ch/python_one_day/script.html
• NumPy, ca. 3h: https://siscourses.ethz.ch/python-scientific/01_numpy.html
  • WARN: a bit too advanced
  • alt, ext: http://scipy-lectures.org/intro/numpy/index.html
• Pandas, ca. 1.5h: https://siscourses.ethz.ch/python-scientific/02_pandas.html
  • alt, ext: http://www.scipy-lectures.org/packages/statistics/index.html#data-representation-and-interaction
  • cheat sheet: https://pandas.pydata.org/Pandas_Cheat_Sheet.pdf
• Matplotlib + Seaborn
  • ext:
    • http://scipy-lectures.org/intro/matplotlib/index.html
    • http://scipy-lectures.org/packages/statistics/index.html#more-visualization-seaborn-for-statistical-exploration
  • cheat sheets:
    • https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Python_Matplotlib_Cheat_Sheet.pdf
    • https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Python_Seaborn_Cheat_Sheet.pdf

Day 1

Intro and superficial overview of classifiers including quality assessment, pipelines and hyperparams optim.

Total time: 6h (8 x uni hour (uh))

Part 0: Preparation

Time: 15 min (1/3 uh)

• organizational announcements
• installation/machines preparation

Part 1: General introduction

Time: 75 min (5/3 uh)

• What is machine learning?

  • learning from examples
  • working with hard to understand data.
  • automation

• What are features / samples / feature matrix?

  • always numerical / categorical vectors
  • examples: beer, movies, images, text to numerical examples

• Learning problems:

  • unsupervised:

    • find structure in set of features
    • beers: find groups of beer types

  • supervised:

    • classification: do I like this beer? example: draw decision tree or surface

Part 2: Supervised learning: concepts of classification

Time: 60 min (4/3 uh)

Intention: demonstrate one / two simple examples of classifiers, also introduce the concept of decision boundary

• idea of simple linear classifier: take features, produce real value ("beer score"), use threshold to decide

  • simple linear classifier (linear SVM e.g.)
  • beer example with some weights

• show code example with logistic regression for beer data, show weights, plot decision surface

Coding session:

• change given code to use a linear SVM classifier
• use different data set which can not be classified well with a linear classifier
• tell to transform data and run again

Part 3: Overfitting and cross-validation

Time: 60 min (4/3 uh)

Needs: simple accuracy measure.

Classifiers (regressors) have parameters / degrees of freedom.

• underfitting:

  • linear classifier for points on a quadratic function

• overfitting:

  • features have actual noise, or not enough information not enough information: orchid example in 2d. elevate to 3d using another feature.
  • polnome of degree 5 to fit points on a line + noise
  • points in a circle: draw very exact boundary line

• how to check underfitting / overfitting?

  • measure accuracy or other metric on test dataset
  • cross validation

Coding session:

• How to do cross validation with scikit-learn
• use different beer feature set with redundant feature (+)
• run crossvalidation on classifier

Part 4: accuracy, F1, ROC, ...

Time: 60 min (4/3 uh)

Intention: pitfalls of simple accuracy

• how to measure accuracy?

  • classifier accuracy:
    • confusion matrix metrics
    • pitfalls for unbalanced data sets e.g. diagnose HIV
    • precision / recall
    • mention ROC?

• exercise (pen and paper): determine precision / recall

Coding session

• do cross val with multiple metrics: evaluate linear beer classifier from latest section
• fool them: give them other dataset where classifier fails.

Part 5: Pipelines and hyperparameters tuning w/ extended exercise

Time: 1.5h (2 uh)

• Scikit-Learn API: recall what we have seen up to now.
• preprocessing (scaler, PCA, function/column transformers)
• cross validation
• parameter tuning: grid search / random search.

Coding session

• build SVM and LinearRegression crossval pipelines for previous examples
• use PCA in pipeline for (+) to improve performance
• find optimal SVM parameters
• find optimal pca components number
• extended: full process for best pipeline/model selection incl. preprocessing steps selection, hyperparams tunning w/ cross-validation

Day 2

Total time: 6h (8 x uni hour (uh))

Part 6 a+b: classifiers overview (NNs & regression-based + tree-based & ensembles)

Intention: quick walk through reliable classifiers, give some background idea if suitable, let them play with some, incl. modification of parameters.

Summary: decision graph (mind-map) from ScikitLearn, and come up with easy to understand summary table.

Part 6a

Time: 1h (4/3 uh)

• Nearest neighbours
• Logistic regression
• Linear + kernel SVM classifier (SVC)
  • demo for Radial Basis Function (RBF) kernel trick: different parameters influence on decision line

Part 6b

Time: 1h (4/3 uh)

• Decision trees
• Averaging: Random forests
• Boosting AdaBoost and mention Gradient Tree Boosting (hist; xgboost)
• mentions
  • text classification: Naive Bayes for text classification
  • big data:
    • Stochastic Gradient Descent classifier,
    • kernel approximation transformation (explicitly approx. kernel trick)
      • opt, compare SVC incl. RBF vs. Random Kitchen Sinks (RBFSampler) + linear SVC (https://scikit-learn.org/stable/auto_examples/plot_kernel_approximation.html#sphx-glr-auto-examples-plot-kernel-approximation-py)
• summary/overview

Topics to include

• interoperability of results (in terms features importance, e.g. SVN w/ high deg. poly. kernel)
• some rules of thumbs: don't use kNN classifiers for 10 or more dimensions (why? paper link)
• show decision surfaces for diff classifiers (extend exercise in sec 3 using hyperparams)

Coding session

• apply SVM, Random Forests, boosting to specific examples
• MNIST example

Part 7: Supervised learning: regression

Time: 1h (4/3 uh)

Intention: demonstrate one / two simple examples of regression

• regression: how would I rate this movie? example: use weighted sum, also example for linear regressor example: fit a quadratic function

• learn regressor for movie scores / salmon weight.

Part 8: Supervised learning: neuronal networks

Time: 3h (4 uh)

Intention: Introduction to neural networks and deep learning with keras

• include real-life tumor example (maybe in day 3 walk-through)

• overview, history

• perceptron

• multi layer

• multi layer demo with google online tool

• where neural networks work well

• keras demo

Coding Session

• keras reuse network and play with it.

Day 3

Total time: 6h (8 uh)

1. Hands-on walk-through real life example.
2. Assisted programming session where participants can start to work on their own machine learning application. Assist to setup own machines. Offer some example data sets from https://www.kaggle.com/datasets

Misc

Best practices

Rather include/repeat in relevant workshop parts/examples

• visualize features: pairwise scatter, UMAP/tSNE
• PCA to simplify/understand data
• check balance of data set, what if not?
• start with baseline classifier/regressor
• augment data to introduce variance