Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
pepper-imitation:pepper-imitation [2018/06/28 19:37] s4pepper2018 |
pepper-imitation:pepper-imitation [2018/06/28 20:12] s4pepper2018 |
||
|---|---|---|---|
| Line 11: | Line 11: | ||
| {{:pepper-imitation:gestures.jpg?200|}} | {{:pepper-imitation:gestures.jpg?200|}} | ||
| - | As such, the project is made up of two main pillars, namely: | + | As such, the project is built on top of two main pillars, namely: |
| * Body pose detection: Windows Kinect V1 was the sensor chosen for this task. | * Body pose detection: Windows Kinect V1 was the sensor chosen for this task. | ||
| * Robot control and synchronization. | * Robot control and synchronization. | ||
| Line 19: | Line 19: | ||
| The chosen framework was ROS (the only tested version was kinetic), written mainly in C++, with some parts in python. | The chosen framework was ROS (the only tested version was kinetic), written mainly in C++, with some parts in python. | ||
| - | ===== Installation ===== | + | ===== Installation and Set Up ===== |
| The code is fully available [[https://github.com/Thordreck/pepper-imitation-pkg|here]]. The repo should be clone directly into the src folder of a catkin workspace. | The code is fully available [[https://github.com/Thordreck/pepper-imitation-pkg|here]]. The repo should be clone directly into the src folder of a catkin workspace. | ||
| Line 63: | Line 63: | ||
| * When all the gestures are done and the music ends, Pepper thanks the user and the game goes back to its first state. The user can then click on the tablet again to re-start the game without having to input his name again. | * When all the gestures are done and the music ends, Pepper thanks the user and the game goes back to its first state. The user can then click on the tablet again to re-start the game without having to input his name again. | ||
| - | ===== Known issues and limitations ===== | + | ===== Known Issues and Limitations ===== |
| + | |||
| + | * The synchronization process is hardcoded. As such, correct synchronization will only be achieved if the same file is used (it is available in the robot's internal memory in /home/pepper/resources/audio), The state machine waits until the file is played to a given time (usually the time when a new gesture is started). | ||
| + | * The way openni_tracker publish the skeleton information (using TF frames following the convention <body_part>_<id>, for example torso_1) poses some problems when people are lost and new ids are assigned. This is due to old TF frames still being listed in by ROS even after a while. An easy solution to this would be to query the TF frames, but keep only those with the highest id value. | ||
| + | * As of now, only one person is tracked. However, adding multiple person's body poses verification should be rather trivial. | ||
| + | |||
| + | ===== Future Work ===== | ||
| + | |||
| + | * The detection provided by the kinect and the openni libraries is not enough. The calibration phase is a limiting factor when interacting with the subjects. A possible solution to this would be to adopt body pose detection based on RGB alone. An example of this would be [[https://github.com/CMU-Perceptual-Computing-Lab/openpose|OpenPose]]. This particular game could be reproduced entirely using only joints angles data in 2D detections, even allowing us to get rid of the external kinect completely, and using the built-in cameras. More complex scenarios and gestures may need visual and depth data fusion. For a real time application, a GPU should be used. | ||
| + | * The concept of this game can be generalized a bit, using some simple config files. These files could define the audio file to be used and the time when each posture should be adopted. Postures may be defined here as well. | ||
| + | * Furthermore, these files could be generated using a user-friendly GUI, where the user could set an audio file and set up the sync times and the robot poses. | ||
| + | |||
| + | ===== Contact ===== | ||
| + | |||
| + | Álvaro Páez Guerra | ||
| + | |||
| + | paezguerraalvaro@gmail.com | ||