Tensorflow application on the mnist data set
A perceprtron is a single layer neural network.
This tutorial deals with creating a multilayer neural network using the mnist dataset
1. Dataset¶
We use mnist dataset that consist of images of hand-written digits.
Each image size is 24 * 24 pixels = 784 pixels
The data for each image is therefore a 24*24 array
Each data-point in the image array represet a white space (0) or a value less than or equal to one.
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("tmp/data", one_hot=True)
# tmp/data is a temporary project data storage for our dataset
# It gets created automatically
mnist
mnist.train.images
# Pixel representation for first image
# 784 pixels represents one image as stated before
len(mnist.train.images[0])
# Total images / observations / rows = 55000
mnist.train.images.shape
We can plot any sample image in the dataset using matplotlib imshow()
import matplotlib.pyplot as plt
%matplotlib inline
sample = mnist.train.images[1].reshape(28,28)
plt.imshow(sample, cmap='Greys')
2. CREATE TENSORFLOW GRAPH INPUTS¶
2.1. Define learning parameters
learning_rate = 0.01 # how quickly to adjust cost function
training_epochs = 15 # Total number of training cycles
batch_size = 500 # number of samples per iteration
Example: if you have 1000 training examples, and your batch size is 500, then it will take 2 iterations to complete 1 epoch.
2.2. Define neural network parameters
# Hidden layer 1 & 2 tensor should be able to hold 8-bit color data
n_hidden_1 = 256
n_hidden_2 = 256 # Also represents the number of features for each image
n_input = 784 # MNIST data input (img shape: 28*28)
n_classes = 10 # MNIST total classes (0-9 digits)
# Total number of images or rows
samples = mnist.train.num_examples
2.3. Graph inputs
x = tf.placeholder("float", [None, n_input])
y = tf.placeholder("float", [None, n_classes])
3.TRAIN DEEP LEARNING MODEL¶
len(mnist.test.labels[0])
3.1. Define multi-layer model (graph)
Each layer performs the following computations:
relu_actiction ( weights * inputs + biases )
We'll use relu activation to make the network non-linnear to prevent the network from being a simple linear regression model!
def multi_layer_network( x, weights, biases):
# Hidden Layer 1 Computations
layer_1 = tf.add( tf.matmul(x ,weights['h1']), biases['b1'] )
layer_1 = tf.nn.relu(layer_1)
# Hidden Layer 2 Computations
layer_2 = tf.add( tf.matmul( layer_1 , weights['h2']), biases['b2'] )
layer_2 = tf.nn.relu(layer_2)
# Finally, compute the output layer
# Use a linear activation function here
out_layer = tf.matmul ( layer_2 , weights['out'] ) + biases['out']
return out_layer
# Define RANDOMLY assigned weights
weights = {
'h1': tf.Variable( tf.random_normal([n_input ,n_hidden_1]) ),
'h2': tf.Variable( tf.random_normal([n_hidden_1, n_hidden_2]) ),
'out': tf.Variable( tf.random_normal([n_hidden_2, n_classes]) )
}
# Define RANDOMLY assigned biases
biases = {
'b1': tf.Variable(tf.random_normal([n_hidden_1])),
'b2': tf.Variable(tf.random_normal([n_hidden_2])),
'out': tf.Variable(tf.random_normal([n_classes]))
}
# Specify prediction function
predictions = multi_layer_network( x, weights, biases)
3.2. Define Cost Optimization Function
# Define loss and optimizer
cost = tf.reduce_mean( tf.nn.softmax_cross_entropy_with_logits_v2( logits=predictions, labels=y) )
optimizer = tf.train.AdamOptimizer ( learning_rate=learning_rate).minimize(cost)
# Initialise all global variables
init = tf.global_variables_initializer()
3.3. Train model
# We'll use the next_batch function
# Example usage:
Xsamp, ysamp = mnist.train.next_batch(batch_size=1)
plt.imshow(Xsamp.reshape(28,28))
ysamp
As it can be seen above, this function returns a tuple conatining lists of input and labels
Training session¶
Training epochs - Essentially the max amount of loops possible before we stop, may stop earlier if cost/loss limit was set
# The only difference with a regular `Session` is that an `InteractiveSession` installs itself as the default
# session on construction.
# The methods @{tf.Tensor.eval} nd @{tf.Operation.run} will use that session to run ops
sess = tf.InteractiveSession()
sess.run(init)
# Training nested loop ( Outer_loop = no. of epochs, Inner_loop = no_of_iterations )
# No_of_iterations = sample_size / batch_size
for epoch in range(training_epochs):
avg_cost = 0
no_of_iterations = int( samples / batch_size )
for i in range(no_of_iterations):
batch_x, batch_y = mnist.train.next_batch(batch_size)
# Feed dictionary for optimization and loss value
# Returns a tuple, but we only need 'c' the cost
# So we set an underscore as a "throwaway"
_, current_cost = sess.run( [optimizer, cost], feed_dict={ x: batch_x, y: batch_y } )
avg_cost += current_cost / no_of_iterations
print( "Epoch: {} cost={:.4f}".format (epoch+1, avg_cost) )
print("Model has completed {} Epochs of Training".format(training_epochs))
The model is now ready for testing and further tuning if necessary.
Model Evaluation¶
correct_predictions = tf.equal(tf.argmax(predictions, 1), tf.argmax(y, 1))
correct_predictions = tf.cast(correct_predictions, "float")
accuracy = tf.reduce_mean(correct_predictions)
print("Accuracy:", accuracy.eval({x: mnist.test.images, y: mnist.test.labels}))
We can improve the accuracy through increasing no. of layers or using many other methods that have been described here: http://neuralnetworksanddeeplearning.com/chap6.html
Comments
Post a Comment