Continue with Machine Learning - Noise Detection (Classification)

Noise has pattern that we can identify. If our model is good enough to classify a specific group of noises such as gun shot, mirror broken, car horn, ..., then we can use the model in a very useful ways such as to identify crime or abnormality in a running machine by just detecting the noise.

Today we'll build a noise classifier based on the data from https://drive.google.com/drive/folders/0By0bAi7hOBAFUHVXd1JCN3MwTEU The data is the collection of noises from 10 classes. Our goal is to learn from the training data in order to identify one among those noises.

Note that we'll be using librosa libray to audio analysing. We need to convert wave form to interval of frequency. Here we change each noise into MFCCs (Mel Frequency Cepstral Coefficient), which is an array of some numbers.

All we'll be using Deep Learning algorithm to train our data with [keras](https://keras.io/) and [tensorflow](https://github.com/tensorflow/tensorflow) libraries.

We'll skip the installation process by going directly into the implementation.

Understand Data

The attributes of data are as follows: ID – Unique ID of sound excerpt Class – type of sound

Each noise is a wave file with .wav extension. Try to listen to some noises.

import IPython.display as ipd
ipd.Audio('data/Train/2022.wav')

Output

Now let's plot some waves to see their patterns.

% pylab inline
import os
import pandas as pd
import librosa
import glob
import librosa.display

data, sampling_rate = librosa.load('data/Train/1045.wav')
plt.figure(figsize=(12, 4))
librosa.display.waveplot(data, sr=sampling_rate)

Output

Let's check some wave plots and observe its patterns. Pay attention to how the patterns of wave of the same class looks similar to each other.

import time
train = pd.read_csv('data/train.csv')
data_dir = os.getcwd() + '/data'
def load_wave():
    i = random.choice(train.index)

    audio_name = train.ID[i]
    path = os.path.join(data_dir, 'Train', str(audio_name) + '.wav')

    print('Class: ', train.Class[i])
    x, sr = librosa.load(path)

    plt.figure(figsize=(12, 4))
    librosa.display.waveplot(x, sr=sr)
    plt.show()

for i in range(10):
    load_wave()

Output

Class: jackhammer

Class: street_music

Class: gun_shot

Class: street_music

Class: siren

Class: engine idling

Class: children_playing

Class: gun_shot

Class: children_playing

Class: children_playing

Let's check number of values for each class:

train.Class.value_counts()

Output:

jackhammer          668
engine_idling       624
siren               607
children_playing    600
street_music        600
drilling            600
air_conditioner     600
dog_bark            600
car_horn            306
gun_shot            230
Name: Class, dtype: int64

Let's create function to convert wave to mfcc.

def parser(row):
   # function to load files and extract features
   file_name = os.path.join(os.path.abspath(data_dir), 'Train', str(row.ID) + '.wav')

   # handle exception to check if there isn't a file which is corrupted
   try:
      # here kaiser_fast is a technique used for faster extraction
      X, sample_rate = librosa.load(file_name, res_type='kaiser_fast') 
      # we extract mfcc feature from data
      mfccs = np.mean(librosa.feature.mfcc(y=X, sr=sample_rate, n_mfcc=40).T,axis=0) 
   except Exception as e:
      print("Error encountered while parsing file: ", file)
      return None, None
 
   feature = mfccs
   label = row.Class
 
   return [feature, label]

temp = train.apply(parser, axis=1)
temp.columns = ['feature', 'label']
temp.head(2)

Output

	feature	label
0	[-82.12358939071989, 139.5059159813099, -42.43...	siren
1	[-15.744005405358056, 124.1199599305049, -29.4...	street_music
2	[-123.39365145003913, 15.181946313102896, -50....	drilling
3	[-213.27878814908152, 89.32358896182456, -55.2...	siren
4	[-237.92647882472895, 135.90246127730546, 39.2...	dog_bark

[array([-82.12358939, 139.50591598, -42.43086489,  24.82786139,
       -11.62076447,  23.49708426, -12.19458986,  25.89713885,
        -9.40527728,  21.21042898,  -7.36882138,  14.25433903,
        -8.67870015,   7.75023765, -10.1241154 ,   3.2581183 ,
       -11.35261914,   2.80096779,  -7.04601346,   3.91331351,
        -2.3349743 ,   2.01242254,  -2.79394367,   4.12927394,
        -1.62076864,   4.32620082,  -1.03440959,  -1.23297714,
        -3.11085341,   0.32044827,  -1.787786  ,   0.44295495,
        -1.79164752,  -0.76361758,  -1.24246428,  -0.27664012,
         0.65718559,  -0.50237115,  -2.60428533,  -1.05346291]), array([-15.74400541, 124.11995993, -29.42888126,  39.44719325,
       -23.50191209,  16.55081468, -21.73682007,  16.533573  ,
       -16.97172924,   4.48358393, -17.38768904,   0.73712233,
       -16.28922845,   5.11214906, -10.55923116,   2.91787297,
       -10.39084829,   0.6512996 , -10.04633806,  -1.78348022,
        -6.09971424,   5.62978658,  -4.65111382,  -1.3691931 ,
        -8.24916556,  -2.36192798,  -4.79620618,  -0.50256975,
        -5.41067503,   2.07804459,   7.18600337,   8.1857473 ,
         0.76736086,   0.32726166,  -2.21366512,  -3.1068377 ,
        -5.72384895,   0.82370563,   1.7193221 ,  -0.33146235]), array([-123.39365145,   15.18194631,  -50.09332904,    7.14187248,
        -26.81703338,   -0.69250356,   -8.22307572,   13.51293887,
        -11.38205589,   19.94935211,  -11.19345959,    9.59290493,
         -8.26916969,    4.59170834,   -4.1160931 ,   -0.12661012,
         -9.26636096,   12.86464874,   -6.76813103,    0.17970622,
         -5.58614496,    6.82406367,   -7.44342262,    6.7138549 ,
          0.88696144,    7.95247415,   -7.80404736,    4.75135726,
         -5.91704383,   -0.51082848,   -2.89312164,    3.75250478,
         -4.3756492 ,    5.6246255 ,   -4.87082627,    1.88768287,
         -3.88603327,    1.57439023,   -3.9967419 ,    3.24574944]), array([-2.13278788e+02,  8.93235890e+01, -5.52561899e+01,  1.26320999e+01,
       -4.77753793e+01,  1.47029095e+01,  1.90393420e+01,  1.59744018e+01,
       -3.44622589e-01, -3.85278488e+00, -5.71352012e+00,  1.45023797e+01,
        7.35625117e+00,  2.77150926e+00, -1.21664120e+01, -7.54264413e+00,
       -1.07718201e+00, -8.32686797e+00,  1.13106934e+01,  1.30546411e+01,
       -1.24408594e+01, -1.77511089e+01,  5.21535203e-02,  2.10836481e+00,
        5.23872284e+00,  9.59365485e+00, -3.76473392e+00, -2.26955796e+00,
       -6.74119185e+00, -8.63759137e+00,  1.20566440e+01, -4.87823860e+00,
       -5.16628578e+00,  1.11927687e+00, -2.28136843e+00,  5.82950793e+00,
        1.19403556e+00,  7.46546796e+00, -1.78587829e+00, -1.50114553e+01]), array([-2.37926479e+02,  1.35902461e+02,  3.92684403e+01,  2.12402387e+01,
        9.53132848e+00,  1.38851206e+01, -3.99444661e+00,  1.24814870e+01,
       -2.60462664e+00,  6.07091558e+00,  2.23836723e+00,  4.17497228e+00,
       -1.90301314e+00,  2.30779460e+00, -2.66080009e+00, -6.64915491e-01,
        4.49824368e+00,  3.77204298e+00,  3.37126391e+00,  1.59958680e+00,
       -5.34918903e-01, -1.95140379e-01,  6.27361166e-01,  3.20973300e+00,
        1.33894133e+00,  1.04329816e-01,  9.07274421e-01, -2.27093950e+00,
       -5.29897600e-01,  2.00067343e-01,  5.41832293e-01, -3.01083238e-01,
        5.31196471e-01, -5.16148851e-01,  1.73844662e+00,  1.10963680e+00,
        2.75794074e+00,  2.15940254e+00,  6.26566792e-01,  6.92017477e-01])]

Let's convert label to dummy variable

from sklearn.preprocessing import LabelEncoder
from keras.utils import np_utils

X = np.array(temp.feature.tolist())
y = np.array(temp.label.tolist())

lb = LabelEncoder()
print(lb.fit_transform(y)[:5])
y = np_utils.to_categorical(lb.fit_transform(y))
print(y.shape)
print(y[:10])

Output

[8 9 4 8 3]
(5435, 10)
[[0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]
 [0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]
 [0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]
 [0. 0. 1. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]
 [0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]
 [0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]]

Now let's build the model:

model = Sequential()

model.add(Dense(256, input_shape=(40,)))
model.add(Activation('relu'))
model.add(Dropout(0.5))

model.add(Dense(256))
model.add(Activation('relu'))
model.add(Dropout(0.5))

model.add(Dense(num_labels))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer='adam')

model.fit(X, y, batch_size=32, epochs=200, validation_split=0.1, shuffle=True)

Here is some last few output:

Epoch 196/200
4891/4891 [==============================] - 0s 84us/step - loss: 0.3572 - acc: 0.8779 - val_loss: 0.3988 - val_acc: 0.8989
Epoch 197/200
4891/4891 [==============================] - 0s 94us/step - loss: 0.3450 - acc: 0.8867 - val_loss: 0.3363 - val_acc: 0.9044
Epoch 198/200
4891/4891 [==============================] - 0s 84us/step - loss: 0.3603 - acc: 0.8796 - val_loss: 0.3784 - val_acc: 0.8934
Epoch 199/200
4891/4891 [==============================] - 0s 84us/step - loss: 0.3349 - acc: 0.8902 - val_loss: 0.3603 - val_acc: 0.8952
Epoch 200/200
4891/4891 [==============================] - 0s 84us/step - loss: 0.3700 - acc: 0.8814 - val_loss: 0.3686 - val_acc: 0.8952
<keras.callbacks.History at 0x7f6fb6eb6b00>

We'll see that accuracy rate is 0.8814, and validation accuracy 0.8952 the loss is around 0.37.

Pretty good result. But I think we can tune to make better result!

Conclusion

We cannot work on wave sound directly so we need to convert it to digital format first. The we use deep learning to learn from those pattern to predict the other noises.