Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
poppy-kine:poppy-kine-2015-s5 [2016/03/09 11:36]
vngo
poppy-kine:poppy-kine-2015-s5 [2019/04/25 14:08] (current)
Line 1: Line 1:
-{{tag>​Poppy Project}} +{{tag>​Poppy Project ​poppy-kine Kinect}} 
-====== Poppy-Kine project ======+====== Poppy-Kine ​: S5 project ​2015-2016 ​======
  
 ===== Overview ===== ===== Overview =====
  
 +{{http://​breddydotorg.files.wordpress.com/​2013/​07/​shoulder-flexion-and-extension.png?​100}}
 The main objective of this project is to help patients with **functional reeducation** exercices. The main objective of this project is to help patients with **functional reeducation** exercices.
  
 Patient A's kinesitherapist will show him/her some movements that A will need to repeat more or less regularly, //at home and without the doctor//​. ​ Patient A's kinesitherapist will show him/her some movements that A will need to repeat more or less regularly, //at home and without the doctor//​. ​
-\\ Therefore, for better memorization,​ the movement will be done by a **Poppy Robot** (see [[http://​poppy-project.org/​|Poppy project]] and [[poppy|Poppy robot wiki page]]).+\\ Therefore, for better memorization,​ the movement will be done by a **Poppy Robot** (see [[http://​poppy-project.org/​|Poppy project]] and [[:poppy|Poppy robot wiki page]]).
 \\ First, the moves are done in front of a **Kinect camera** (v2).  \\ First, the moves are done in front of a **Kinect camera** (v2). 
 \\ The movements (captured by position of the Skeleton) are saved, and replayed by the robot. \\ The movements (captured by position of the Skeleton) are saved, and replayed by the robot.
Line 19: Line 19:
   - First step : capturing the Skeleton (Kinect v2) as **cartesian** coordinates (x,y,z)   - First step : capturing the Skeleton (Kinect v2) as **cartesian** coordinates (x,y,z)
   - Second step : converting the Skeleton data into Poppy robot'​s referential (**angular**)   - Second step : converting the Skeleton data into Poppy robot'​s referential (**angular**)
 +\\ __**NB : **__  ​
 +  * If you want to **make Poppy rigid**, we wrote a program that puts it to the //zero position// (ie all the motors'​ angles are set to zero): **Init_poppy.py**
 +  * Don't forget to **plug Poppy** !
  
 ==== First step: getting the Skeleton ==== ==== First step: getting the Skeleton ====
 +{{:​kinect_skeleton.png?​200 |}}
   * Plug the Kinect camera to the computer **before** turning it on (or you will probably need to restart the computer)   * Plug the Kinect camera to the computer **before** turning it on (or you will probably need to restart the computer)
   * Connect the Kinect camera   * Connect the Kinect camera
 +\\  
 +\\  
 +\\  
 +The Kinect recovers the position of the following joints: 
 +  * Unordered List Item 
 +{{https://​naokinect.files.wordpress.com/​2012/​02/​image_thumb_6f4828ec1.png}}
 ==== Second step: the movement ==== ==== Second step: the movement ====
-All the programs we wrote are in **projets5/​poppy_humanoid/​software/poppy_humanoid/​** folder+All the programs we wrote are in **C:​\Users\ihsev\Documents\2015projetS5PoppyKine-VirginieNgoGuillaumeVillette\poppy-humanoid\software\poppy_humanoid/​** folder
 === First option: manually save, convert, then play the movement=== === First option: manually save, convert, then play the movement===
   * Save the movement   * Save the movement
 <​hidden>​ Click **save.bat** <​hidden>​ Click **save.bat**
-\\ Enter the name of the exercise as asked by the shell.+\\ Enter the //name of the exercise// as asked by the shell.
 \\ A new window should appear, with the video captured by the Kinect camera, and the skeleton of the person in front of it (The first time, it can take a while... be patient). \\ A new window should appear, with the video captured by the Kinect camera, and the skeleton of the person in front of it (The first time, it can take a while... be patient).
 \\ Movements are separated by //space key// pressures. \\ Movements are separated by //space key// pressures.
Line 42: Line 51:
 \\ x starts from zero (first movement). \\ x starts from zero (first movement).
 \\ To finish capturing the movements, just close the window (click on the red cross..). \\ To finish capturing the movements, just close the window (click on the red cross..).
 +
 +You can also use save_movement_and_video.bat to save the video too (info to be checked).
 +
 </​hidden>​ </​hidden>​
  
   * Convert the angles   * Convert the angles
 <​hidden>​ Click **convert.bat** <​hidden>​ Click **convert.bat**
-\\ Enter the name of the exercise as asked by the shell+\\ Enter the name of the exercise ​(name of the folder) ​as asked by the shell
 \\ The program will convert all the movements saved in the corresponding folder \\ The program will convert all the movements saved in the corresponding folder
 \\ The cartesian coordinates and the quaternions captured by the Kinect are saved in .txt files \\ The cartesian coordinates and the quaternions captured by the Kinect are saved in .txt files
Line 79: Line 91:
 </​hidden>​ </​hidden>​
 \\  \\ 
 +----
 ===== How we proceeded ===== ===== How we proceeded =====
-All the programs we wrote are stored in **projets5/​poppy_humanoid/​software/​poppy_humanoid/​** folder+All the programs we wrote are stored in **2015projetS5PoppyKine-VirginieNgoGuillaumeVillette/​poppy_humanoid/​software/​poppy_humanoid/​** folder
 ==== First step: getting the Skeleton ==== ==== First step: getting the Skeleton ====
 We used the //​pykinect2//​ library in Python. We used the //​pykinect2//​ library in Python.
 \\ The code is written in **poppy_kinect_save_mouvement.py** file \\ The code is written in **poppy_kinect_save_mouvement.py** file
 \\ For each movement, a .txt file is saved in **⁄exercices⁄<​name_of_the_exercise>/​** folder, in <​name_of_the_exercise_x>​.txt file (x starts from zero (first movement)). \\ For each movement, a .txt file is saved in **⁄exercices⁄<​name_of_the_exercise>/​** folder, in <​name_of_the_exercise_x>​.txt file (x starts from zero (first movement)).
-\\ A .txt file is a //Python dictionnary//​ stored with //json// (in order to save it as a dictionnary and not just some text, and treat the data for the conversion)+\\ A .txt file is a //Python dictionnary//​ stored with //​json// ​format ​(in order to save it as a dictionnary and not just some text, and treat the data for the conversion)
 \\  \\ 
 \\ The **cartesian coordinates** are written in the subdictionnary **"​positions"​**,​ and each time unit (depends on the framerate) has its own dictionnary \\ The **cartesian coordinates** are written in the subdictionnary **"​positions"​**,​ and each time unit (depends on the framerate) has its own dictionnary
Line 105: Line 118:
 \\  \\ 
 \\ Well, you need to **convert the cartesian coordinates into angles**. How ?! \\ Well, you need to **convert the cartesian coordinates into angles**. How ?!
-\\ Take 4 points: we can define 2 vectors. You can calculate the angles between 2 vectors thanks to Euclidean geometry (the functions needed are defined in **calculangles.py** file). +<​hidden>​ 
-\\ Here, the main difficulty is not to calculate the angles; but to find __which__ vectors to choose. Indeed, we need to consider all the possible configurations in 3D space and find the good formula.+{{:​anglesbras.png?​200 |}} 
 +\\ Take 4 points: we can define 2 vectors. ​ 
 +\\ You can then calculate the angles between 2 vectors thanks to Euclidean geometry (the python ​functions needed are defined in **calculangles.py** file). 
 +\\ Here, the main difficulty is not to calculate the angles; but to find __which__ vectors to choose. Indeed, we need to consider all the possible configurations in 3D space and find the good formula.</​hidden>​
 \\ You can find our approximations in **Kinect_to_Poppy_angles.py** file.  \\ You can find our approximations in **Kinect_to_Poppy_angles.py** file. 
 \\  \\ 
 \\ For each movement, a .txt file is saved in //​⁄exercices⁄<​name_of_the_exercise>///​ folder, in **<​name_of_the_exercise_x>​_poppy.txt** file (x starts from zero (first movement)). \\ For each movement, a .txt file is saved in //​⁄exercices⁄<​name_of_the_exercise>///​ folder, in **<​name_of_the_exercise_x>​_poppy.txt** file (x starts from zero (first movement)).
-\\ A .txt file is a //Python dictionnary//​ stored with //json// (in order to save it as a dictionnary and not just some text, and treat the data for Poppy to play the movement)+\\ A .txt file is a //Python dictionnary//​ stored with //​json// ​format ​(in order to save it as a dictionnary and not just some text, and treat the data for Poppy to play the movement)
 \\ Basically, the structure of a .txt file (ie the dictionnary that stores the movement) is:  \\ Basically, the structure of a .txt file (ie the dictionnary that stores the movement) is: 
 <​code>​ <​code>​
Line 123: Line 139:
 \\ It needs some strong geometrical skills, and unfortunately we did not manage to get to it successfully. \\ It needs some strong geometrical skills, and unfortunately we did not manage to get to it successfully.
 </​hidden>​ </​hidden>​
 +---- 
 +\\ 
 ===== For further improvements ===== ===== For further improvements =====
 The angles we calculated are unfortunately not accurate enough for complex movements. That is why using quaternions can be another way to improve the conversion. The angles we calculated are unfortunately not accurate enough for complex movements. That is why using quaternions can be another way to improve the conversion.
-Moreover, Gaussian mixture models can be used to get one movement from several samples of the same movement; Gaussian mixture regressions can also be considered for the recognition phase (when the human repeats Poppy'​s movement +\\ Moreover, GMM (Gaussian mixture models) and GMR (Gaussian mixture regressions) can be used to synthetize samples of the same movement, and make Poppy then the patient reproduce it.
-Moreover, GMM (Gaussian mixture models) and GMR (Gaussian mixture regressions) can be used to synthetize samples of the same movement, and make Poppy then the patient reproduce it.+
  • poppy-kine/poppy-kine-2015-s5.1457523407.txt.gz
  • Last modified: 2019/04/25 14:08
  • (external edit)