Fundamentals of Deep Learning

Deep learning is a machine learning technique that has dramatically improved results in many areas such as computer vision, speech recognition, machine translation, etc. Deep learning techniques make it possible, using data, to solve many problems in many areas of the economy such as health, transport, trade, finance and energy. It is a technology which imposes itself as a powerful AI approach that uses multi-layered artificial neural networks to deliver state-of-the-art accuracy in tasks such as object detection, speech recognition, and language translation. Using deep learning, computers can learn and recognize patterns from data that are considered too complex or subtle for expert-written software.

In this tutorial, you’ll learn how deep learning works through hands-on exercises in computer vision and natural language processing. You’ll train deep learning models from scratch, learning tools and tricks to achieve highly accurate results. You’ll also learn to leverage freely available, state-of-the-art pre-trained models to save time and get your deep learning application up and running quickly.

Learning Objectives

By participating in this tutorial, you’ll:

  • Learn the fundamental techniques and tools required to train a deep learning model
  • Gain experience with common deep learning data types and model architectures
  • Enhance datasets through data augmentation to improve model accuracy
  • Build confidence to take on your own project with a modern deep learning framework

Tutorial Details

  • Duration: 3 half days.
  • Prerequisites: An understanding of fundamental programming concepts in Python 3, such as functions, loops, dictionaries, and arrays; familiarity with Pandas data structures; and an understanding of how to compute a regression line.
  • Suggested materials to satisfy prerequisites: Python Beginner’s Guide.
  • Technologies: Tensorflow 2 with Keras, Pandas
  • Assessment Type: Skills-based coding assessments evaluate students’ ability to train a deep learning model to high accuracy.
  • Hardware Requirements: 
    • Desktop or laptop computer capable of running the latest version of Chrome or Firefox and anaconda distribution.
    • Internet connection.
  • Operating System: Windows with 3 GB space or more to install anaconda distribution and databases.
  • Installation required:
    • Anaconda Distribution 64-bit with Python 3: latest version
    • TensorFlow: latest version
    • Karas: latest version
  • Number of trainees: 30 trainees max.
  • Languages: French.

Tutorial Outline

Class 0 — Jupyter notebooks manipulating and learning

  • Episode 1 — Linear regression with direct resolution

    • Low-level implementation, using numpy, of a direct resolution for a linear regression

  • Episode 2 — Linear regression with gradient descent

    • Low level implementation of a solution by gradient descent. Basic and stochastic approach.

  • Episode 3 — Logistic regression

    • Simple example of logistic regression with a sklearn solution

Class 1 — Basic regression using DNN

  • Episode 1 — Regression with a Dense Network (DNN)

    • Simple example of a regression with the dataset Boston Housing Prices Dataset (BHPD)

  • Episode 2 — Regression with a Dense Network (DNN) - Advanced code

    • A more advanced implementation of the precedent example

Class 2 — Basic classification using a DNN

  • Episode 1 — Simple classification with DNN

    • An example of classification using a dense neural network for the famous MNIST dataset

  • Episode 2 — Simple classification with CNN

    • An example of classification using a convolutional neural network for the famous MNIST dataset

Class 3 — Working with Convolutional Neural Networks (CNN) : Case of Images classification

  • Episode 1 — Dataset analysis and preparation

    • Analysis of the German Traffic Sign Recognition Benchmark (GTSRB) dataset and creation of an enhanced dataset

  • Episode 2 — First convolutions

    • First convolutions and first classification of our traffic signs

  • Episode 3 — Training monitoring

    • Monitoring, analysis and check points during a training session

  • Episode 4 — Data augmentation

    • Adding data by data augmentation when we lack it, to improve our results

  • Episode 5 — Full convolutions

    • A lot of models, a lot of datasets and a lot of results.

Class 4 — Working with Recurrent Neural Networks (RNN) : Case of Sentiment Analysis

  • Episode 1 — Sentiment analysis with hot-one encoding

    • A basic example of sentiment analysis with sparse encoding, using a dataset from Internet Movie Database (IMDB)

  • Episode 2 — Sentiment analysis with text embedding

    • A very classical example of word embedding with a dataset from Internet Movie Database (IMDB)

  • Episode 3 — Reload and reuse a saved model

    • Retrieving a saved model to perform a sentiment analysis (movie review)

  • Episode 4 — Reload embedded vectors

    • Retrieving embedded vectors from our trained model

  • Episode 5 — Sentiment analysis with a RNN network

    • Still the same problem, but with a network combining embedding and RNN

Class 5 — Unsupervised learning with an autoencoder neural network (AE)

  • Episode 1 — Prepare a noisy MNIST dataset

    • Preparation of a noisy MNIST dataset

  • Episode 2 — Building and training an AE denoiser model

    • Construction of a denoising autoencoder and training of it with a noisy MNIST dataset.

  • Episode 3 — Playing with our denoiser model

    • Using the previously trained autoencoder to denoise data

  • Episode 4 — Denoiser and classifier model

    • Construction of a denoiser and classifier model

  • Episode 5 — Advanced denoiser and classifier model

    • Construction of an advanced denoiser and classifier model

Class 6 — Generative network with Variational Autoencoder (VAE)

  • Episode 1 — First VAE, using functional API (MNIST dataset)

    • Construction and training of a VAE, using functional APPI, with a latent space of small dimension.

  • Episode 2 — VAE, using a custom model class (MNIST dataset)

    • Construction and training of a VAE, using model subclass, with a latent space of small dimension.

  • Episode 3 — Analysis of the VAE's latent space of MNIST dataset

    • Visualization and analysis of the VAE's latent space of the dataset MNIST

Class 7 — Generative Adversarial Networks (GANs)

  • Episode 1 — A first DCGAN to Draw a Sheep

    • Draw me a sheep, revisited with a DCGAN

  • Episode 2 — A WGAN-GP to Draw a Sheep

    • Draw me a sheep, revisited with a WGAN-GP