This blog is used to record the progress in my Master Project:”Efficient Human Re-Identification in Video Sequences”.

Current rank-1 accuracy: 94.23%.

Introduction

Kaggle is a brave new world for me. Sometimes I really miss the programming/project competitions I played in my first Uni. I should know this website earlier. Anyway, it’s time to start a new journey.

As a “hello world”-level competition, Data Science London + Scikit-learn is binary classification task, which helps me to practice the PCA & SVM, as well as the GMM and QQ-plot. Here are my notes from this simple competition.

Final result:

• GMM+Random Forest: 0.99218
• Simple SVC: 0.93443 (n_components=12)
• DNN:0.85544

Source Code: here

Read more »

In these days ,I am trying to integrate my re-id network. Then I found the official example of Siamese CNNs of the Keras based on the MNIST dataset.

Actually it is a very good example about how to integrate multiple networks. But there are 2 annoying bugs..

Wrong contrastive_loss function

LinHungShi pointed this bug in This issue. Therefore, the correct contrastive loss function should be:

1
2
3
4
5
6
7

def contrastive_loss(y_true, y_pred):
    '''Contrastive loss from Hadsell-et-al.'06
    http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf
    '''
    margin = 1
    return K.mean((1 - y_true) * K.square(y_pred) +
                  y_true * K.square(K.maximum(margin - y_pred, 0)))

Wrong accuracy function

The original accuracy function in this example is:

1
2
3
4

def compute_accuracy(predictions, labels):
    '''Compute classification accuracy with a fixed threshold on distances.
    '''
    return labels[predictions.ravel() < 0.5].mean()

Actually this code cannot calculate the accuracy. The output is the mean value of labels that with the false index. For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

In [1]: import numpy as np

In [2]: def compute_accuracy(predictions, labels):
   ...:     return labels[predictions.ravel() < 0.5].mean()
   ...: a=np.array([1,0,0,1])
   ...: b=np.array([0,1,1,0])
   ...: print(compute_accuracy(a,b))
   ...:
1.0

In [3]: print(a.ravel()<0.5)
[False  True  True False]

In [5]: print (b[a.ravel()<0.5])
[1 1]

The accuracy should be 0 because 2 arrays (a and b) are totally different. It calculate the mean of b[a.ravel()<0.5].

I saw there is a fix which gives a correct method of computing the accuracy, but they reverted this fix and I don’t know why.

Anyway, this sample code is a good example about how to set my own loss function, and how to integrate multiple networks into one model.

Show one image in 5 line

1
2
3
4
5

import matplotlib.pyplot as plt
import matplotlib.image as mpimg
path='/Users/typewind/Desktop/xpy/test.jpg'
img = mpimg.imread(path)
plt.imshow(img)

The output look like this:

Turn off the axis

The axis looks not good. So just turn it off by one line:

1
2
3
4
5
6
7

import matplotlib.pyplot as plt
import matplotlib.image as mpimg
path='/Users/typewind/Desktop/xpy/test.jpg'
img = mpimg.imread(path)
# Turn off the axis
plt.axis("off")
plt.imshow(img)

Use the open() function to read the binary file. Each hex bit will be saved as one element in the list. Here is an example about how to read the first 8 bit hex code and transform it to decimal int number.

1
2
3
4
5
6

def read_8_bit_data(file):
    with open(file) as f:
        # 32 bit signed interger = 8 bit hex bit
        data_8_bit_hex = f.read(8)
        data_8_bit = int(data_size_hex, 16)
    return data_8_bit

Installation

Fink (for Mac OS)

Basic installation tutorial of TF-Object Detection used apt-get to install packages. I tried use brew to replace it but some packages are missed. Fortunately, we can use apt-get after installing the Fink.

Fink provided a helper script to install all things automatically. The Installation cost nearly half an hour on my computer, here is the whole process:

1. Save the script on the Desktop (or whatever you like) as “Fink.sh”
2. Run ~/Desktop/Fink.sh
3. If you didn’t install the full Xcode or Xquartz, the script will ask you whether you’d like to install them. After installing the Xcode and Xquartz, we should run ~/Desktop/Fink.sh again to continue the process of Fink.
4. Waiting for the installation until the Fink ask you several questions like

   • I’ll ask you some questions and update the configuration file in
     ‘/sw/etc/fink.conf’.

• In what additional directory should Fink look for downloaded tarballs?
• Which directory should Fink use to build packages? (If you don’t know what this means, it is safe to leave it at its default.)
• ….

Here just need to press “Enter” again and again to leave them at its default. Then have a coffee and wait for a few minutes :)

After the installation is done, remember to restart the terminal and run fink update-all to active the fink. Now apt-get is available on Mac OS :D

Install Object Detection model

The basic version of Tensorflow didn’t contain the models. So we should run

1
2

# From /tensorflow
git clone https://github.com/tensorflow/models

Then compile the photoruf libraries by running:

1
2

# From tensorflow/models
protoc object_detection/protos/*.proto --python_out=.

If photoruf was not installed, then run

1

brew photoruf

to install it first.

Then set the pre-trained model path by:

1
2

# From tensorflow/models/
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

From this line, we can find it is obvious that object detection use the TF-slim to extract the features.

To test whether the model has been installed successfully or not, run:

1

python object_detection/builders/model_builder_test.py

under your environment. If you use the VirtualEnv, remember run this first:

1

source ~/tensorflow/bin/active

Reference:

1. TF-Object Detection Model
2. Fink
3. Document of Fink

I’m following the CS20SI course and found something maybe useless but interesting.

The definition of tf.ones_like is:

1
2
3
4
5
6

ones_like(
    tensor,
    dtype=None,
    name=None,
    optimize=True
)

For example, given a tensor a that:

1

a=tf.constant([1,2,3],[4,5,6])

The tf.ones_like can return a tensor with the same shape of a but all values are set to 1.

1

b=tf.ones_like(a, tf.int16)

The ourpur of b should be:

1
2

[[1 1 1]
 [1 1 1]]

As an accident, I found we can use the numbers instead of the data type and get the same output, for example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

>>> b=tf.ones_like(a,1)
>>> sess.run(b)
array([[ 1., 1., 1.],
  [ 1., 1., 1.]], dtype=float32)
>>> b=tf.ones_like(a,2)
>>> sess.run(b)
array([[ 1., 1., 1.],
  [ 1., 1., 1.]])
>>> print(b)
Tensor("ones_like_16:0", shape=(2, 3), dtype=float64)
>>> b=tf.ones_like(a,3)
>>> sess.run(b)
array([[1, 1, 1],
  [1, 1, 1]], dtype=int32)
>>> b=tf.ones_like(a,5)
>>> sess.run(b)
array([[1, 1, 1],
  [1, 1, 1]], dtype=int16)
>>> b=tf.ones_like(a,6)
>>> sess.run(b)
array([[1, 1, 1],
  [1, 1, 1]], dtype=int8)
>>> b=tf.ones_like(a,9)
>>> sess.run(b)
array([[1, 1, 1],
>>> print(b)
Tensor("ones_like_17:0", shape=(2, 3), dtype=int64)

By check the core code of tensorflow, the proto of enum data type as shown below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

DT_FLOAT = 1;
DT_DOUBLE = 2;
DT_INT32 = 3;
DT_UINT8 = 4;
DT_INT16 = 5;
DT_INT8 = 6;
DT_STRING = 7;
DT_COMPLEX64 = 8;  // Single-precision complex
DT_INT64 = 9;
DT_BOOL = 10;
DT_QINT8 = 11;     // Quantized int8
DT_QUINT8 = 12;    // Quantized uint8
DT_QINT32 = 13;    // Quantized int32
DT_BFLOAT16 = 14;  // Float32 truncated to 16 bits.  Only for cast ops.
DT_QINT16 = 15;    // Quantized int16
DT_QUINT16 = 16;   // Quantized uint16
DT_UINT16 = 17;
DT_COMPLEX128 = 18;  // Double-precision complex
DT_HALF = 19;
DT_RESOURCE = 20;

That’s why we can pass several constant numbers to the dtype. Notice that the definition of tf.ones_like’s dtype is

dtype: A type for the returned Tensor. Must be float32, float64, int8, int16, int32, int64, uint8, complex64, complex128 or bool.

Thus, the available numbers are 1,2,3,4,5,6,8,9,10,18. (7 is string)

The string of dtype can be replaced by the key value diectly for convenience. But the code becomes harder to understand.

TensorFlow-Slim

Reference

1.https://github.com/tensorflow/models/pull/1346

Type of knowledge transfer[2]

Instance transfer

Transfer the instance is same as reuse the same instance. But the weight of it is different in the target dataset.
For example, an instance is appeared both in the source set and target set. Then we can calculate the similarity between the source label and target label of the same instance and give a new weight value of it in the target set.[1]

Feature representation transfer

The general idea of feature represenation transfer is to reduce the difference between the source and target featurespace. It can be done by mapping(copying :P) the feature space from the source data to target data.

Feature representation in DNN/CNN

Because the DNN/CNN has a modular structure, we get the representation of features for each layers by pre-train, then use the high-level features to classify the target. So we can transfer/reuse the low level features from the source data set (eg. ImageNet), then tune the hyper-parameters of our own network.

Parameter transfer

The key of the parameter transfer is to find the common priori knowledge between the source and target data. It can be applied on SVM, K-means, etc.

Relational knowledge transfer

It used to transfer the relational knowledges between graphs or networks.

Basic strategy of transfer learning in CNN [3]

1. Copy the first layers to the corresponding layers
   Because the low-level features in ther first layers (such as layer/color) are very similar among the different computer vision(CV) tasks. So we can reuse it directly. Sounds tricky but useful.
2. Randomly initialise the remaining layers
   Different CV tasks have different high level features. So just update the remaining network with new/random values.
3. Tune the random genelized layers or all layers
   The hell of tuning parameter is coming…

Reference

1. Cross-Domain Activity Recognition, Vincent Wenchen Zheng, Derek Hao Hu, Qiang Yang
2. Transfer learning for activity recognition: a survey, Diane Cook, Kyle D. Feuz, and Narayanan C. Krishnan
3. Deep Transfer Learning for Person Re-identification, M Geng, Y Wang, T Xiang, Y Tian

Rank-r matching rate means the percentage of the p images with correct matches found in the top rank r ranks against the p gallery images.
Rank 1 matching rate is equal to the correct matching/recognition rate.

Reference:
Computer Vision, CCF Chinese Conference, CCCV 2015, Xi’an, China, September 18-20, 2015, Proceedings, Part I, page 225. Editors: Zha, H., Chen, X., Wang, L., Miao, Q. (Eds.)