Predict EPL results (Part 2: Neural Network example)

Neural Network explained

  • Artificial Neural Network is a model that receives input in vector/matrix form and uses them to compute the output 
  • It is made up of 3 different layers namely input layer, hidden layer and output layer. 
  • Each layer is made up of 1 or more artificial neurons (aka nodes) 
    • The nodes are interconnected with one another
    • Each node is assigned to a weight value denoted by w that determines how much emphasis is given to the learned information extracted from the raw training data 
  • Note: A Neural Network with 1 hidden layer is known as Shallow Neural Network

Types of Neural Networks

  1. Feedforward Neural Network
  2. Convolutional Neural Network (CNN)
  3. Recurrent Neural Network - Long Short Term Memory (LSTM) Networks
  4. Recursive Neural Network

Neural Network Architecture

  • The figure below shows a Neural Network Architecture that is made up of n features, h hidden layers and y output values
    • The number of hidden layers is a hyperparameter that is set by us and we can change it to fine tune the performance of the model
    • If we are solving binary classification problem (i.e. y = 0 or 1), there will be only 2 nodes in the output layer
    • If we are dealing with multiclass classification problem (i.e. EPL prediction), there will be 3 nodes in the output layer
  • Let's take the EPL data that we used for our Logistic Regression model as an example
    • Recall that there are 
      • 280 training set 
      • 100 test set
      • 16 features (n = 16)
      • 3 output values (i.e. win, draw and lose) (y = 3)
    • Therefore, for every example, the input layer will have 16 nodes to represent the features (x1, x2, x3, ... , x16) and the output layer will have 3 nodes to represent the possible outcomes. We can choose the number of hidden layers that we want

Neural Network Architecture

How does Neural Network learn? 

Before we start, let's go through some notations as shown in the picture below.

Notation
Notation

Shallow Neural Network Model

Shallow Neural Network Model
Shallow Neural Network Model

Forward Propagation

Forward Propagation
Forward Propagation


Backward Propagation

  • Helps to reduce error thus optimizing the cost of the function

Analysis of our NN model

  • We included the rest of the seasons data and still using 16 features
  • We built a NN model with 1 hidden layer with 4 nodes and here's our results
    • Train accuracy = 67% 
    • Test accuracy = 64%
  • The gap between train and test accuracy has closed up a lot so there's no need for us to do regularization

What's next

  • We are going to find ways to further improve the accuracy (e.g. change the model, add more features etc)
  • And we are going to build an interface to use the trained parameters so that we can use it to predict the new game
  • And also we are going to learn Tensorflow 2.0 (heard it's easier now) and Keras 

Comments

Post a Comment

Popular posts from this blog

How to connect Python to MySQL Workbench

Image
Step-by-Step Guide 1) Download MySQL: For Windows user , you can download  MySQL Installer that allows you to install all the products in one package. Link as follows https://dev.mysql.com/downloads/windows/installer/8.0.html Download MySQL Installer (For Windows only!) Begin download Otherwise, you can download the product 1 by 1  (Remember to select your Operating System for all!) MySQL Workbench  MySQL Connector / Python   (Note: You have to download Python first. And remember to select the OS and Python version!) MySQL Community Server For Windows users... 2) How to setup MySQL Installer We are going to show how to download and install MySQL Community Server only... Double click the file and install it Once it has finished installing, you will see this window below and click on the 'Add' option Click 'Add' option Agree on their License terms and click 'next' Agree on License Terms Selec...
Read more

Bias and Variance in Machine Learning

Sorry for the lack of updates...Have been rushing to complete the Machine Learning course as fast as possible... Good news!!! We managed to survive through the whole course!!!! Hahaha Summary of Week 6 lesson: Some definitions first before delving into the details... Bias:  The hypothesis is known to be biased if it tends to overestimate or underestimate a parameter Variance:  The hypothesis is known to have high variance if the data are far away from its average value Given the complexity of some data, a linear regression may not be able to formulate an accurate prediction. As the linear regression only factors in 1 specific feature of the data, the 'best-fit' line does not fit the data well enough. Such under-fitting problem leads to high training costs and also causes the hypothesis to be biased. Ways to reduce the bias: Add more features / Increase the degree of the polynomial Decrease λ  Use larger neural network As mentioned a...
Read more