Combining Pepper and the smarthome: Link ROS and xAAL

20180118: Current status naoqi can be imported, and web_video_server can boradbast kinect video and we can see the skeleton on rviz The source address is :

- To start the fall detection with kinect, please plug the USB cable of Kinect into the left USB port of the computer, please make sure that the LED light on the cable of Kinect is green.

Then run in different terminals the following commands or use “command &”:

rosrun openni_tracker openni_tracker
roslaunch openni_launch openni.launch

In order to follow the movement of the person, the openni tracker first needs to calibrate the person shown before the camera and extracts the skeleton of the person. To realize this, the user can use a “surrender” posture before the camera to help the process find exactly the position of the skeleton.

To open the web streaming

rosrun web_video_server web_video_server

and open a browser and go directly to

or see all available images on
  1. To start the fall detection with sensfloor:
    • With the Sensfloor Receiver, please plug the sensfloor recever into the USB port of the computer.

then run in another terminal, choose one of the two methods:

cd /home/user/P204/xaal/code/C/trunk/xaafloor
sudo chmod 777 /dev/ttyUSB0
./xaafloor -D /dev/ttyUSB0 -a -p 1235 -s xaal

Because using the USB device in ubuntu system needs a root permission, so a “chmod” is used to change the execution mode. If you see the result: “Device: 920de19d-63ac-451e-9c42-7eb7e48c0f90” It means that it works normally.

  • Or With the Sensfloor Server:

Please use another computer to work as a gateway, because this one need to be connected with LABO_SID

cd /home/user/P204/xaal/code/C/trunk/xaafloor
./xaafloor -S -P 5000 -a -p 1235 -s xaal

If Sensfloor server is used, please make sure the computer is connected with the network: “sensfloor-03555”, code:wlan1234. Only one of the above connection is needed.

  1. To connect the Pepper, please first launch the pepper and wait for 2~5 minutes, until it says “on est neckmouque”. Then please run in another terminal:
roslaunch pepper_bringup pepper_full_py.launch nao_ip:= roscore_ip:=

Please make sure the IP address of pepper is correct. It can be verified by pressing the chest button of the Pepper one time.

  1. To start the controller, firstly, goto the corresponding repository.
cd /home/user/P204/xaalros/scripts/

Then make sure that your are in the branch “pepper”

git branch (see the available branch) 
git checkout pepper (goto branch pepper)
  1. Then, run in another terminal:
  1. If a falling is detected, the lights will be turned on and an email will be sent to the contact list. In the email there is an IP address of the video stream, please tap on the address to see the actual scene captured by the robot.
  1. Install ros kinetic with command sudo apt-get install ros-kinetic-desktop-full. The details can be read here : You will have created a catkin workspace CATKIN_WS (for instance ~/catkin_ws)
  2. For the kinect. Run:
    sudo apt install ros-kinetic-openni-camera
    sudo apt install ros-kinetic-openni-launch

    Install NITE v1.5.2.23 by downloading from: then run ./ Clone the git repo of ros-openni-tracker from:

Place it inside a catkin workspace and use catkin_make to build it. Then source devel/setup.bash to load the built packages.

To test, initiate roscore. Then roslaunch openni_launch openni.launch and rosrun openni_tracker openni_tracker as decribed in Use rviz to visualize the skeleton (launch rosrun rviz rviz AFTER the openni_tracker has started its tracking). In the display window, under Global Options, set Fixed Frame to openni_depth_frame. Then add (button Add) a TF. Set the scale to 0,5 and you should see the frames for each joint. More details on

  1. Install web_video_server ROS package with the command sudo apt install ros-kinetic-web-video-server.
    1. download from aldebaran the Python SDK the compressed file pynaoqi-python2.7- and install according to the instructions In details :
    2. Set the environnement variable PYTHONPATH to /path/to/python-sdk/lib/python2.7/site-packages by doing for example:
      export PYTHONPATH=${PYTHONPATH}:/home/user/NaoPepper/pynaoqi-python2.7- 
    3. check the installation with python
      import naoqi

      If no error message is written, the library is imported. Else read

    4. ROS packages for the Nao robot
      sudo apt-get install ros-kinetic-move-base-msgs ros-kinetic-octomap ros-kinectic-octomap-msgs ros-kinetic-humanoid-msgs ros-kinetic-humanoid-nav-msgs ros-kinetic-camera-info-manager ros-kinetic-camera-info-manager-py
      sudo apt-get install ros-kinetic-nao-robot 
    5. For Pepper, refer to

NOTE sudo apt-get install ros-driver-base did not work. Package driver-base is deprecated and is not compatible with ros kinetic

A former install was done as described in Since, we have changed the version of Ubuntu, of ROS and naoqi.

All ros packages are installed in folder ~/catkin_ws. Do not forget to execute source ~/catkin_ws/devel/setup.bash before using these packages. I installed choregraphe for Pepper in folder /home/user/NaoPepper/choregraphe-suite-

to connect Pepper to ROS:

roslaunch pepper_bringup pepper_full_py.launch nao_ip:= roscore_ip:=

1.Today I did not manage to have the kinect run.

  • First, when executing openni_launch, I had the error :
    kinect ros no devices connected

so I followed ans installed freenect packages. It did not work at first but after a few attempts, roslaunch freenect_launch freenect.launch eventually worked. I don't know what heppened.

  • Then opennitracker had an error : InitFromXml failed: Can't create any node of the requested type! So I followed the instructions in!topic/ros-by-example/cPVGgHHWtEU

eventually the command rosrun openni_tracker openni_tracker worked. I don't know what happened.

2.These kinds of error appears because the pepper is not connected successfully. Please verify the IP address of the Pepper.

[ERROR] [1520344155.442555]: Could not create Proxy to "ALxxxxxx".
connect: No route to host
[FATAL] [1520344155.707677]: Could not connect to required "joint_trajectory" action server, is the nao_controller node running?
REQUIRED process [naoqi_moveto-3] has died!

3.This happens because the Kinect is not well plugged in or the machine can use this device. Please try to replug the Kinect USB cable again and plug it into the left port of the labo computer. By running this demo, please try to avoid using virtual machines.

[ INFO] [1520344443.904870037]: No devices connected.... waiting for devices to be connected
  • How to control the intelligent equipment of the connected habitat from the web page?


  • How to monitor the Sensfloor status?

Connect to the network of the sensfloor and then visit:

  • How to make xaafloor?

Please download the code of xaal from and follow the instructions from this page. Make sure that the is generated in the code. Then go to the following repository:

cd ~/P204/xaal/code/C/trunk/xaafloor(or your path)

Please make sure the library dependency is satisfied. As described in first lines of the Readme.txt of the repository libxaal.

  • How to change xaal hardware address/email address configuration?

Please refer to the file nao_xaal.conf in the repository of nao_xaal. The correct hardware address can be found from the web

  • Useful xaal tools: Firstly, clean all:
sudo pip uninstall xaal.lib
sudo pip uninstall

Secondly, install xaal_test. The tool virtualenv is recommanded, it may need to be installed first.

$ virtualenv xaal_test
Using base prefix '/usr'
New python executable in /home/jkx/xaal_test/bin/python
Installing setuptools, pip, wheel...done.
$ source xaal_test/bin/activate
(xaal_test) $ cd xaal_svn/libs/lib/
(xaal_test) $ python develop
(xaal_test) $ cd ../../apps/tools/
(xaal_test) $ python develop

Here the xaal_svn is the repository of the python trunk, ex.“/home/user/P204/xaal/code/Python/trunk/libs/lib/”. Then a list of function is available for you!

source xaal_test/bin/activate
xaal-isalive (show the list of the active device)
xaal-walker (show the list of the active device with )
xaal-dumper (show all the message exchanging on the multi-cast channel xaal)
xaal-tail 1|2|3 (show the message exchanging by the message level)

And one more time, please pay attention that before these command, the command “source xaal_test/bin/activate” is always necessary. Make sure that the terminal is then started with “(xaal_test) $”.

  • How to check the connection with pepper?
ping pepper.local
  • Cannot read sensfloor message from xaal?

Firstly, please make sure the computer is connected with the LABO_SID. Secondly, make sure that the gateway xaafloor is working. Use the xaal-dumper or xaal-tail to check. Thirdly, verify the encoding key:

./xaafloor -D /dev/ttyUSB0 -a -p 1235 -s xaal

In this command the option “-s” is the encode key for the sensfloor message from the gateway. If you want to change it, please make sure that it is the same key in the message receiving process.

  • smarthome/xaal_ros_pepper.txt
  • Last modified: 2020/07/03 17:33
  • by mai