Sichkar V. N. Real time detection and classification of traffic signs based on YOLO version 3 algorithm. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2020, vol. 20, no. 3, pp. 418–424. DOI: 10.17586/2226-1494-2020-20-3-418-424 (Full-text available on ResearchGate here: Real time detection and classification of traffic signs based on YOLO version 3 algorithm
Sichkar V. N. Effect of various dimension convolutional layer filters on traffic sign classification accuracy. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2019, vol. 19, no. 3, pp. 546–552. DOI: 10.17586/2226-1494-2019-19-3-546-552 (Full-text available on ResearchGate here: Effect of various dimension convolutional layer filters on traffic sign classification accuracy
[ ⛳Goal ]
Design, Train 7 Test deep CNN for Image Classification!!! TRY TO IMPROVE YOUR DEEP LEARNING SKILLS!!
[ 🔭 IMPORT LIBRARIES ]
numpyfor linear algebra
pandasfor data processing & deal with CSV data
mataplotlibfor visualization
picklefor large capacity data
timeit
for provides to time small bits of Python code
keras.utils.np-utils __ to_categorical
for one-hot-encoding
keras.models __ Sequentialfor deep learning model
keras.layer __ Dense, Dropout, Flatten, Conv2D, MaxPool2D, AvgPool2D, BatchNormalization, Reshapefor deep learning model
keras.preprocessing.image import ImageDataGeneratorfor training image data
keras.callbacks import LearningRateSchedulerfor moderate lerning rate by accuracy
In [1]:
import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import pickle
import matplotlib.pyplot as plt
from timeit import default_timer as timer
from keras.utils.np_utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, AvgPool2D, BatchNormalization, Reshape
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import LearningRateScheduler
# Input data files are available in the read-only "../input/" directory
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory
import os
for dirname, _, filenames in os.walk('/kaggle/input'):
for filename in filenames:
print(os.path.join(dirname, filename))
print(os.listdir('../input'))
# Any results we write to the current directory are saved as output
# Opening file for reading in binary mode
with open('../input/traffic-signs-preprocessed/data2.pickle', 'rb') as f:
data = pickle.load(f, encoding='latin1') # dictionary type, latin1 for data loss
print('<Before tranpose>')
# Showing loaded data from file
for i, j in data.items(): # i=key, j=value
if i == 'labels':
print(i + ':', len(j))
else:
print(i + ':', j.shape)
print(" ")
print('<Before tranpose key: x_train>')
print(data['x_train'][0])
# Preparing y_train and y_validation for using in Keras
data['y_train'] = to_categorical(data['y_train'], num_classes=43)
data['y_validation'] = to_categorical(data['y_validation'], num_classes=43)
# Making channels come at the end
data['x_train'] = data['x_train'].transpose(0, 2, 3, 1)
data['x_validation'] = data['x_validation'].transpose(0, 2, 3, 1)
data['x_test'] = data['x_test'].transpose(0, 2, 3, 1)
print('<After tranpose>')
# Showing loaded data from file
for i, j in data.items(): # i=key, j=value
if i == 'labels':
print(i + ':', len(j))
else:
print(i + ':', j.shape)
print(" ")
print('<After tranpose key: x_train>')
print(data['x_train'][0])
As input it will take 4D tensor and convert it to gridBy Nnmpy library
Values will be scaled to the range [0,255]
In [5]:
%matplotlib inline
#return Rich output(image, sound, animation... ect)
def convert_to_grid(x_input):
N, H, W, C = x_input.shape #Number of data
grid_size = int(np.ceil(np.sqrt(N))) # as formula
grid_height = H * grid_size + 1 * (grid_size - 1)
grid_width = W * grid_size + 1 * (grid_size - 1)
grid = np.zeros((grid_height, grid_width, C)) + 255
next_idx = 0
y0, y1 = 0, H
# fill the grid data
for y in range(grid_size):
x0, x1 = 0, W
for x in range(grid_size):
if next_idx < N:
img = x_input[next_idx]
low, high = np.min(img), np.max(img)
grid[y0:y1, x0:x1] = 255.0 * (img - low) / (high - low) # normalize 0~225
next_idx += 1
x0 += W + 1
x1 += W + 1
y0 += H + 1
y1 += H + 1
return grid
# Visualizing some examples of training data
examples = data['x_train'][:81, :, :, :]
print(examples.shape)
# Plotting some examples
fig = plt.figure()
grid = convert_to_grid(examples)
plt.imshow(grid.astype('uint8'), cmap='gray')
plt.axis('off')
plt.gcf().set_size_inches(15, 15)
plt.title('Some examples of training data', fontsize=18)
# Showing the plot
plt.show()
# Saving the plot
fig.savefig('training_examples.png')
plt.close()
(81, 32, 32, 3)
[ 🏗️ Building model of CNN with Keras ]
Trying one model with filters of size 3x3
use relu(Can avoid Vanishing Gradient Problem)
Conv2D (Make a Convolution calculate process)
MaxPool2D (Conist kernel with representative value)
Flatten for training data
Dense
Dense (with relu) for Hiden layer
Dense (with softmax) for Output layer of Multi-class classification
Compile with adam loss func = categorical_crossentropy!
adam
categorical_crossentropy
In [6]:
model = Sequential() # Generate in the order given
model.add(Conv2D(32, kernel_size=3, padding='same', activation='relu', input_shape=(32, 32, 3)))
model.add(MaxPool2D(pool_size=2))
model.add(Flatten())
model.add(Dense(500, activation='relu'))
model.add(Dense(43, activation='softmax'))
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
2022-05-16 07:42:24.371724: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-05-16 07:42:24.486417: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-05-16 07:42:24.487171: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-05-16 07:42:24.488532: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-16 07:42:24.488831: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-05-16 07:42:24.489557: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-05-16 07:42:24.490320: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-05-16 07:42:26.660706: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-05-16 07:42:26.661518: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-05-16 07:42:26.662201: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-05-16 07:42:26.663543: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 15403 MB memory: -> device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:04.0, compute capability: 6.0
[ 🤏 Overfitting the 3x3 model with small amount of data ]
2022-05-16 07:42:27.090346: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
Epoch 1/15
2022-05-16 07:42:28.277460: I tensorflow/stream_executor/cuda/cuda_dnn.cc:369] Loaded cuDNN version 8005
Trying different models with different dimensions of filters
Trying one model with filters of size 3x3
use relu(Can avoid Vanishing Gradient Problem)
Conv2D (Make a Convolution calculate process)
MaxPool2D (Conist kernel with representative value)
Flatten for training data
Dense
Dense (with relu) for Hiden layer
Dense (with softmax) for Output layer of Multi-class classification
Compile with adam loss func = categorical_crossentropy!
adam
categorical_crossentropy
In [10]:
filters = [3, 5, 9, 13, 15, 19, 23, 25, 31]
model = [0] * len(filters)
for i in range(len(model)):
model[i] = Sequential()
model[i].add(Conv2D(32, kernel_size=filters[i], padding='same', activation='relu', input_shape=(32, 32, 3)))
model[i].add(MaxPool2D(pool_size=2))
model[i].add(Flatten())
model[i].add(Dense(500, activation='relu'))
model[i].add(Dense(43, activation='softmax'))
model[i].compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
[ 💡 Training set of models of CNN with Keras ]
And with different dimensions of filters
Trying one model with filters of size 3x3
use relu(Can avoid Vanishing Gradient Problem)
Conv2D (Make a Convolution calculate process)
MaxPool2D (Conist kernel with representative value)
Flatten for training data
Dense
Dense (with relu) for Hiden layer
Dense (with softmax) for Output layer of Multi-class classification
Compile with adam loss func = categorical_crossentropy!
adam
categorical_crossentropy
In [11]:
annealer = LearningRateScheduler(lambda x: 1e-3 * 0.95 ** (x + epochs))
epochs = 5
h = [0] * len(model)
for i in range(len(h)):
h[i] = model[i].fit(data['x_train'], data['y_train'],
batch_size=1, epochs = epochs,
validation_data = (data['x_validation'], data['y_validation']),
callbacks=[annealer], verbose=0)
print('Model with filters {0:d}x{0:d}, epochs={1:d}, training accuracy={2:.5f}, validation accuracy={3:.5f}'.\
format(filters[i], epochs, max(h[i].history['accuracy']), max(h[i].history['val_accuracy'])))
2022-05-16 07:42:56.616622: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 1068920832 exceeds 10% of free system memory.
2022-05-16 07:42:57.691048: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 1068920832 exceeds 10% of free system memory.
Model with filters 3x3, epochs=5, training accuracy=0.98096, validation accuracy=0.84399
2022-05-16 07:55:52.103681: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 1068920832 exceeds 10% of free system memory.
2022-05-16 07:55:53.347220: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 1068920832 exceeds 10% of free system memory.
Model with filters 5x5, epochs=5, training accuracy=0.97805, validation accuracy=0.86939
2022-05-16 08:09:17.129617: W tensorflow/core/framework/cpu_allocator_impl.cc:80] Allocation of 1068920832 exceeds 10% of free system memory.
Model with filters 9x9, epochs=5, training accuracy=0.96858, validation accuracy=0.81406
Model with filters 13x13, epochs=5, training accuracy=0.94320, validation accuracy=0.81429
Model with filters 15x15, epochs=5, training accuracy=0.91650, validation accuracy=0.77914
Model with filters 19x19, epochs=5, training accuracy=0.84381, validation accuracy=0.74875
Model with filters 23x23, epochs=5, training accuracy=0.73501, validation accuracy=0.60431
Model with filters 25x25, epochs=5, training accuracy=0.64470, validation accuracy=0.52857
Model with filters 31x31, epochs=5, training accuracy=0.02368, validation accuracy=0.04762
[ 📈 Plotting comparison results for accuracy ]
You might see the accuracy rate foes 100% which means overfitting occur
! Caution !
You have to put accuracy & val_accuract instead acc & val_acc -> it's occur error
In [12]:
%matplotlib inline
plt.rcParams['figure.figsize'] = (15.0, 15.0) # Setting default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['font.family'] = 'Times New Roman'
# Plotting history of training accuracy
fig = plt.figure()
plt.subplot(2, 1, 1)
plt.plot(h[8].history['accuracy'], '-o', linewidth=3.0)
plt.plot(h[7].history['accuracy'], '-s', linewidth=3.0)
plt.plot(h[6].history['accuracy'], '-D', linewidth=3.0)
plt.plot(h[5].history['accuracy'], '-D', linewidth=3.0)
plt.plot(h[4].history['accuracy'], '-o', linewidth=3.0)
plt.plot(h[3].history['accuracy'], '-o', linewidth=3.0)
plt.plot(h[2].history['accuracy'], '-o', linewidth=3.0)
plt.plot(h[1].history['accuracy'], '-o', linewidth=3.0)
plt.plot(h[0].history['accuracy'], '-o', linewidth=3.0)
plt.legend(['filter 31', 'filter 25', 'filter 23', 'filter 19', 'filter 15', 'filter 13', 'filter 9', 'filter 5', 'filter 3'], loc='lower right', fontsize='xx-large', borderpad=2)
plt.xlabel('Epoch', fontsize=20, fontname='Times New Roman')
plt.ylabel('Training Accuracy', fontsize=20, fontname='Times New Roman')
plt.yscale('linear') # {"linear", "log", "symlog", "logit", ...}
plt.ylim(0.85, 1.0)
plt.xlim(0.5, 5.3)
plt.title('Accuracy for different sizes of filters', fontsize=22)
plt.tick_params(labelsize=18)
plt.subplot(2, 1, 2)
# plt.gca().set_title('Validation accuracy')
plt.plot(h[8].history['val_accuracy'], '-o', linewidth=3.0)
plt.plot(h[7].history['val_accuracy'], '-s', linewidth=3.0)
plt.plot(h[6].history['val_accuracy'], '-D', linewidth=3.0)
plt.plot(h[5].history['val_accuracy'], '-D', linewidth=3.0)
plt.plot(h[4].history['val_accuracy'], '-o', linewidth=3.0)
plt.plot(h[3].history['val_accuracy'], '-o', linewidth=3.0)
plt.plot(h[2].history['val_accuracy'], '-o', linewidth=3.0)
plt.plot(h[1].history['val_accuracy'], '-o', linewidth=3.0)
plt.plot(h[0].history['val_accuracy'], '-o', linewidth=3.0)
plt.legend(['filter 31', 'filter 25', 'filter 23', 'filter 19', 'filter 15', 'filter 13', 'filter 9', 'filter 5', 'filter 3'], loc='lower right', fontsize='xx-large', borderpad=2)
plt.xlabel('Epoch', fontsize=20, fontname='Times New Roman')
plt.ylabel('Validation Accuracy', fontsize=20, fontname='Times New Roman')
plt.yscale('linear') # {"linear", "log", "symlog", "logit", ...}
plt.ylim(0.75, 0.9)
plt.xlim(0.5, 5.3)
plt.tick_params(labelsize=18)
# Showing the plot
plt.show()
# Saving the plot
fig.savefig('models_accuracy.png')
plt.close()
# Showing values of accuracy for different filters
for i in range(len(h)):
print('data2 filter {0:d} training accuracy = {1:.5f}'.\
format(filters[i], np.max(h[i].history['accuracy'])))
print()
for i in range(len(h)):
print('data2 filter {0:d} validation accuracy = {1:.5f}'.\
format(filters[i], np.max(h[i].history['val_accuracy'])))
Through test with lots of filters You can find the most efficienct model
In [13]:
for i in range(len(model)):
temp = model[i].predict(data['x_test'])
temp = np.argmax(temp, axis=1)
# We compare predicted class with correct class for all input images
# And calculating mean value among all values of following numpy array
# By saying 'testing_accuracy == data['y_test']' we create numpy array with True and False values
# 'np.mean' function will return average of the array elements
# The average is taken over the flattened array by default
temp = np.mean(temp == data['y_test'])
print('data2 filter {0:d} testing accuracy = {1:.5f}'.format(filters[i], temp))
# Getting scores from forward pass of one input image
# Scores are given for each image with 43 numbers of predictions for each class
# Measuring at the same time execution time
for i in range(len(model)):
start = timer()
temp = model[i].predict(data['x_test'][:1, :, :, :])
end = timer()
print('data2 filter {0:d} classification time = {1:.5f}'.format(filters[i], end - start))
data2 filter 3 classification time = 0.03712
data2 filter 5 classification time = 0.03142
data2 filter 9 classification time = 0.03025
data2 filter 13 classification time = 0.03037
data2 filter 15 classification time = 0.03015
data2 filter 19 classification time = 0.03133
data2 filter 23 classification time = 0.03899
data2 filter 25 classification time = 0.03316
data2 filter 31 classification time = 0.04968
[ 👁️ Visualizing filters of convolutional layer ]
In [15]:
for i in range(len(model)):
w = model[i].get_weights()
print(w[0].shape)
# print(model[i].get_config())
# l = model[i].layers
# print(l[0].get_weights()[0].shape)
# Visualizing filters
temp = w[0].transpose(3, 0, 1, 2)
print(temp.shape) # (81, 32, 32, 3)
# Plotting
fig = plt.figure()
grid = convert_to_grid(temp)
plt.imshow(grid.astype('uint8'), cmap='gray')
plt.axis('off')
plt.gcf().set_size_inches(10, 10)
name = 'Trained filters ' + str(filters[i]) + 'x' + str(filters[i])
plt.title(name, fontsize=18)
# Showing the plot
plt.show()
# Saving the plot
name = 'filters-' + str(filters[i]) + 'x' + str(filters[i]) + '.png'
fig.savefig(name)
plt.close()
(3, 3, 3, 32)
(32, 3, 3, 3)
(5, 5, 3, 32)
(32, 5, 5, 3)
(9, 9, 3, 32)
(32, 9, 9, 3)
(13, 13, 3, 32)
(32, 13, 13, 3)
(15, 15, 3, 32)
(32, 15, 15, 3)
(19, 19, 3, 32)
(32, 19, 19, 3)
(23, 23, 3, 32)
(32, 23, 23, 3)
(25, 25, 3, 32)
(32, 25, 25, 3)
(31, 31, 3, 32)
(32, 31, 31, 3)
[ 🖼️ Predicting with one image from test dataset ]
It Works nice!! (even I couldn't get well..)
In [16]:
%matplotlib inline
# Preparing image for predicting from test dataset
x_input = data['x_test'][100:101]
print(x_input.shape)
y_input = data['y_test'][100:101]
print(y_input)
plt.rcParams['figure.figsize'] = (2.5, 2.5) # Setting default size of plots
plt.imshow(x_input[0, :, :, :])
plt.axis('off')
# Showing the plot
plt.show()
# Getting scores from forward pass of input image
scores = model[0].predict(x_input)
print(scores[0].shape) # (43,)
# Scores is given for image with 43 numbers of predictions for each class
# Getting only one class with maximum value
prediction = np.argmax(scores)
print('ClassId:', prediction)
# Defining function for getting texts for every class - labels
def label_text(file):
# Defining list for saving label in order from 0 to 42
label_list = []
# Reading 'csv' file and getting image's labels
r = pd.read_csv(file)
# Going through all names
for name in r['SignName']:
# Adding from every row second column with name of the label
label_list.append(name)
# Returning resulted list with labels
return label_list
# Getting labels
labels = label_text('../input/traffic-signs-preprocessed/label_names.csv')
# Printing label for classified Traffic Sign
print('Label:', labels[prediction])
(1, 32, 32, 3)
[3]
(43,)
ClassId: 5
Label: Speed limit (80km/h)
[ 💾 Saving models ]
Saving model locally without committing
from IPython.display import FileLink
FileLink('model-3x3.h5')
In [17]:
for i in range(len(model)):
name = 'model-' + str(filters[i]) + 'x' + str(filters[i]) + '.h5'
model[i].save(name)
