Title image Spring 2017

Interactive Robot

The purpose of this project is to add more perception to your robot. You will use both lasers and vision to identify attributes of the environment and respond to them appropriately.

The vision module, SVM (Social Vision Module), is a general purpose vision system designed for interactive applications. It runs as a separate program and broadcasts messages via a system called IPC (Inter-Process Communication), developed at Carnegie Mellon University. Programs can connect to an IPC central server and subscribe to messages sent by other programs or send their own messages. A program can tell SVM to turn on various algorithms, and SVM will broadcast information about what it detects.


On both robots there is now an executable of the SVM module in /usr/local/bin. To run SVM, you need to start two programs. Then you can use a third program to control SVM and also view what it detects while debugging.

To run SVM and view what it is detecting, create three terminals. In one terminal, start the central server, if one is not already running.


There is, in fact, very little point in quitting the central server, so leaving one running at all times is fine. You can check if a central server is running by looking at all active processes and searching for central.

ps aux | grep central

In the second terminal, first copy the files colorCalibration, camera0.conf and camera1.conf from /usr/local/etc to the directory where you will run svm. Then use the following command.


That will run svm using the built-in camera on the laptop. If you want to use a different camera (good idea) then you'll need to give it the command line argument of the video device to use, such as /dev/video1 and you'll need to modify the camera1.conf file so it is active and camera0 is not.

In the third terminal, type the following to run dispipc, which creates a window in which you can see what SVM is doing.

dispipc localhost

The program dispipc lets you see what the robot sees and control what operators are active in SVM. An operator is a vision algorithm that detects or recognizes something and is able to broadcast that via IPC. You can active several operators at once. By typing keys or clicking in the video window, you can do a number of things.

clickSave current video frame to a file
1Toggle greyscale video
2Toggle color video
3Set video size to small
4Set video size to medium
5Set video size to large
6Set video quality to low
7Set video quality to medum
8Set video quality to high
rtoggle face detection
mtoggle motion detection
ptoggle pink blob detection
itoggle italian flag detection
Ltoggle soccer marker detection
qquit dispipc, but leave SVM running
xquit dispipc and terminate SVM

Note that disipc does not have to run on the same computer as SVM. I hope to have local disipic versions on the dwarves, so that you can run it locally and connect to either hasuful or arod by giving telling it the hostname, as below.

dispipc hasufel.cs


  1. Download the two files makefile.listener and listener.c. Take a look at listener.c, which is an example of how to turn on an SVM operator, listen for responses, and turn off an operator. You can use the following command to compile it.

    make -f makefile.listener listener

    If SVM is running, you can use the listener program to turn on an operator and subscribe to messages. To run listener and turn on face detection, make sure SVM is running and then use the following command.

    ./listener -o 24

    That will start the face detector and print out any detections. Typing 'q' will terminate listener. The pink badge operator is operator 0. You can figure out what the other operator ids are by looking at the file SVM_VisionModule.h which is located in /usr/local/include.

  2. Write a stand-alone program that connects to central, tells the vision module to turn on either the pink badge or face detector operator, and then prints out any detections. Your program will look similar to the listener operator, but you don't need to do any of the special stuff with the terminal settings. You will need to link the ipc and gcm libraries with your code during compilation, as in the makefile.listener file.
  3. In your navigation main program, have your program try to connect to IPC when it starts up. Then put a call to IPC_listenClear() inside your main loop. note that if you set the argument to IPC_listenClear to 100, then you do not need a usleep in your main loop any more, as the function will listen for 100ms before returning. Then terminate the IPC connection when your program quits.
  4. Add a new top-level state to your system PINK_TRACK. When entering that state, the robot should turn on the pink badge detector. It should then try to approach anything pink it sees in the environment and stop a certain distance away from it, using sonar/lasers to determine the distance. If it sees nothing, it should wander. The robot should not run into anything. Note that you can substitute any of the other vision operators for the pink badge detector, if you wish.

    Note that the operators in SVM should be returning the bounding box and location of the detection. They should also be returning pan and tilt angles, relative to the forward direction of the robot. These may not be quite accurate, but they do tell you if you are left or right of the target.



The report for each project should be a brief summary of what you did along with some code examples, terminal output, or screen shots, depending upon the assignment. Please organize the writeup as follows.

  1. Title of the project and your name(s)
  2. An abstract describing what you did in 200 words or less.
  3. A brief description of code you wrote and experiments you ran.
  4. Results.
  5. A brief description of what you learned.


Make your writeup for the project a wiki page in your personal space.

Once you have written up your assignment, give the page the label:


Do not put code on your writeup page or anywhere it can be publicly accessed. To hand in code, put it into a project4 directory in the Private folder of your handin folder on Courses.