This is an old revision of the document!


S4 project 2018 : A reinforcement learning approach based on imitation using kinect and Pepper.

The project aims to be the first steps for a development of a more general learn-by-imitation approach for autistic children. It was developed in collaboration with the CHU Brest, more specifically with Dr. Nathalie Collot as the main contact.

The main goal was to simulate a well-known nursery rhyme used by Ms Collot's team to interact with the patients. The song is divided up in several sections, where the entertainer would assume a pre-defined posture that the kids should try to imitate.

The gestures currently used are show in the image below:

As such, the project is made up of two main pillars, namely: * Body pose detection: Windows Kinect V1 was the sensor chosen for this task. * Robot control and synchronization.

Body pose detection was entirely done using publicly available software. Thus, the main focus of our work was to familiarize ourselves with the robot itself and synchronize all the tasks, as well as fine-tune everything, taking into account the nature of the project and the target subjects.

The chosen framework was ROS (Robotics Operative System), written mainly in C++, with some parts in python.

The code is fully available here.

  • pepper-imitation/pepper-imitation.1530212118.txt.gz
  • Last modified: 2019/04/25 14:08
  • (external edit)