CS 363: Assignment #5

Interactive Robot

The purpose of this project is to add more perception to your robot. You will use both lasers and vision to identify attributes of the environment and respond to them appropriately.

The vision module, SVM (Social Vision Module), is a general purpose vision system designed for interactive applications. It runs as a separate program and broadcasts messages via a system called IPC (Inter-Process Communication), developed at Carnegie Mellon University. Programs can connect to an IPC central server and subscribe to messages sent by other programs or send their own messages. A program can tell SVM to turn on various algorithms, and SVM will broadcast information about what it detects.


On both robots, in the cs363-S13 directory, there is now an instance of the SVM module. To run SVM, you need to start two programs. Then you can use a third program to control SVM and also view what it detects while debugging.

To run SVM and view what it is detecting, create three terminals. In one terminal, start the central server, if one is not already running.


There is, in fact, very little point in quitting the central server, so leaving one running at all times is fine. You can check if a central server is running by looking at all active processes and searching for central.

ps aux | grep central

In the second terminal, first copy the files colorCalibration, camera0.conf and camera1.conf from /usr/local/etc to the directory where you will run svm. Then use the following command.

svm-v4l2 /dev/video0

In the third terminal, type the following to run dispipc, which creates a window in which you can see what SVM is doing.

dispipc localhost

The program dispipc lets you see what the robot sees and control what operators are active in SVM. An operator is a vision algorithm that detects or recognizes something and is able to broadcast that via IPC. You can active several operators at once. By typing keys or clicking in the video window, you can do a number of things.

clickSave current video frame to a file
1Toggle greyscale video
2Toggle color video
3Set video size to small
4Set video size to medium
5Set video size to large
6Set video quality to low
7Set video quality to medum
8Set video quality to high
rtoggle face detection
mtoggle motion detection
ptoggle pink blob detection
itoggle italian flag detection
Ltoggle soccer marker detection
qquit dispipc, but leave SVM running
xquit dispipc and terminate SVM

Note that disipc does not have to run on the same computer as SVM. I hope to have local disipic versions on the dwarves, so that you can run it locally and connect to either hasuful or arod by giving telling it the hostname, as below.

dispipc hasufel.cs


  1. Download the two files makefile.listener and listener.c. Take a look at listener.c, which is an example of how to turn on an SVM operator, listen for responses, and turn off an operator. You can use the following command to compile it.

    make -f makefile.listener listener

    If SVM is running, you can use the listener program to turn on an operator and subscribe to messages. To run listener and turn on face detection, make sure SVM is running and then use the following command.

    ./listener -o 24

    That will start the face detector and print out any detections. Typing 'q' will terminate listener. The pink badge operator is operator 0. You can figure out what the other operator ids are by looking at the file SVM_VisionModule.h which is located in /usr/local/include.

  2. Write a stand-alone program that connects to central, tells the vision module to turn on either the pink badge or face detector operator, and then prints out any detections. Your program will look similar to the listener operator, but you don't need to do any of the special stuff with the terminal settings. You will need to link the ipc and gcm libraries with your code during compilation, as in the makefile.listener file.
  3. In your navigation main program, have your program try to connect to IPC when it starts up. Then put a call to IPC_listenClear() inside your main loop. note that if you set the argument to IPC_listenClear to 100, then you do not need a usleep in your main loop any more, as the function will listen for 100ms before returning. Then terminate the IPC connection when your program quits.
  4. Add a new top-level state to your system PINK_TRACK. When entering that state, the robot should turn on the pink badge detector. It should then try to approach anything pink it sees in the environment and stop a certain distance away from it, using sonar/lasers to determine the distance. If it sees nothing, it should wander. The robot should not run into anything. Note that you can substitute any of the other vision operators for the pink badge detector, if you wish.

    Note that the operators in SVM should be returning the bounding box and location of the detection. They should also be returning pan and tilt angles, relative to the forward direction of the robot. These may not be quite accurate, but they do tell you if you are left or right of the target.



The writeup for each project should be a brief summary of what you did along with some code examples, terminal output, or screen shots, depending upon the assignment. Please organize the writeup as follows.

  1. Title of the project and your name
  2. An abstract describing what you did in 200 words or less.
  3. A brief description of code you wrote and experiments you ran.
  4. Results.
  5. A brief description of what you learned.

In addition, you will demo your programs during the following lab.


Make your writeup for the project a wiki page in your personal space. If you have questions about making a wiki page, stop by my office or ask in class.

Once you have written up your assignment, give the page the label:


To hand in the code, email it to the prof or put it in your private handin directory.