Poppy-Kine : S4 project 2015-2016

This project is organised under the “Projet d'ingénieur” of Telecom BRETAGNE for the scholar year 2015-2016. Its main objective is to evaluate the performance of the patients in their reeducation exercises and to give the feedbacks by a humanoid robot based on a text-to-speech system. The robot will be used is Poppy (see Poppy project and Poppy robot wiki page), the movements are captured by the camera Kinect (v2).
For the recorded and reproduction parts, we have used the work of projet S5 (see poppy-kine-2015-s5).
We implemented the filtering in order to have the proper movements without noise of the patients. This will help us to build the GMM model to compare the movements of the patients with the reference ones of the kinesitherapist. After having the resultat of evaluation, we used the MaryTTS (see marytts) to give the oral feedbacks.

1. First step: getting the data from the camera Kinect. In front of the camera Kinect, we have performed a simple movement, and we would use this movement as the reference in the reeducation exercise.
2. Second step: filter the data, after the first step, we will have the movements' data in form of <name_of_the_exercise_x>.txt file (x starts from zero (first movement)). The Butterworth filter is used in order to obtain the proper movements without noise.
3. Third step: detection of the beginning and the end of the movements. After that building the GMM model in MATLAB.

See the clear instruction at poppy-kine-2015-s5

We used a low pass filter to decrease the effects of the noise. In this case, we decided to choose the filter Butterworth because of its advantage which is its formula of the transfer function is the same for all the orders:

After considering the performance of the filter with several orders, we chose n=5 for our filter. The figure below shows the filtered data for the joint “Head” correspond to the skeleton of the camera Kinect.

To detect the beginning of the movement, we supposed that the noise always has the amplitude which is smaller than the movements' amplitudes. We have calculated an interval outside of which the signal can be considered as the movements.
This figure below presents the detection of the beginning for the movements of the joint “Hip-Left”.

From the detection of the beginning, we can fin the end of the movement as we know its period which is the inverse of the frequency obtained by the FFT transform of the signal.
In reality, we consider the case where there are many repetition in one exercise, so that we to cut the movements into many cycles in order to compare with the reference movements. We have calculated the period of the cycles thanks to the FFT transform, and then obtained the number of cycles in one movement.

All the source code is located in the folder C:/Utilisateurs/ihsev/Documents/2016projetS4PoppyKine/redmine, you can also get the code source via https://redmine.telecom-bretagne.eu/projects/poppy-kine-2016-s4

You can run our program by executing main.py(you can use PythonWin or PyCharm), and there are two inputs, for exemple, “16mai_slow_0.json bruit_0.json”. The first is data of movement, and the second is a sample of noise.

In the main.py (line 38), you can select the joints that you want, or comment the joints no needed.

To see the results of the data filtered, you can change the parameter “debug” in the function “filter” as “filter()”(line69).

You can also refer to Curve.py and calcules.py to see the method used for detecting the start and period.

The number of cycles detected is showed by the output“number of cycles cutted”, and the files cutted are saved in the foleder C:/Utilisateurs/ihsev/Documents/2016projetS4PoppyKine/redmine/filtered

Here is a synthesis of the files:
main.py : execute the program.
Curve.py : contains the functions used for one curve(a joint, a dimension), functions : filter, detect the start and period.
Data.py : contains the functions used to determine the global start and period for the movement, and cut the cycles into different files.
calcules.py : contains the calculation of the transform FFT and the detection of start.
filtrage.py : constitute a filter low-pass, and determine the cutoff frequency.