INSTALL
-------

  Download the June 21, 2004 snapshot of Tekkotsu:
    $ cvs -d :pserver:anonymous@cvs.tekkotsu.org:/cvs checkout -D 2004/06/21 Tekkotsu_cvs
    $ cd Tekkotsu_cvs

  To apply patch file:

    $ patch -p1 -i [path/to/]tekkotsu-telepresence.patch

  Now you are ready to build. See also the install guide on the
  Tekkotsu Homepage to build and use it.


ORIGINAL SUBMISSION EMAIL FOLLOWS
---------------------------------

From: Alexander Klyubin <A.Kljubin@herts.ac.uk>
Date: June 21, 2004 2:46:29 AM EDT
To: Ethan Tira-Thompson <ejt@andrew.cmu.edu>
Subject: PATCH: Telepresence for Tekkotsu

Hello!

Here's the patch with telepresence interface for Tekkotsu. Briefly, the patch adds full-duplex audio support and unifies various sensory and actuatory modalities into one single-window interface, where one can customize how keyboard and mouse events map to commands/actions.

I'm sorry I did not provide detailed instructions. I'm leaving for a conference and then for a vacation in a couple of hours. Even worse, I couldn't test on a robot whether changes to the C code work fine with the latest Tekkotsu CVS checkout (they do with Tekkotsu-2.0.1 though). This is because my flash card reader broke down unexpectedly.

If you don't want to integrate the patch because of its lacking documentation or for some other reason, then I can try resubmitting it around mid August.


The patch is against the latest CVS checkout as of the date I send and time this e-mail. To apply the patch, cd into tekkotsu directory and run:
  $ bzcat tekkotsu-telepresencee.patch.bz2 | patch -p1
Then download bsh-2.0b.jar from http://www.beanshell.org and save it as tools/mon/bsh.jar.


CHANGES TO EXISTING CODE

I've tried to touch existing code as little as possible, but to reuse it as much as possible.

* MMCombo/MMCombo.c
  MMCombo/MMCombo.h
  MMCombo/stub.cfg
  project/ms/open-r/mw/conf/connect.cfg
  project/ms/config/tekkotsu.cfg

  Audio IO and simple scripting sockets. I gave a try at integrating the audio layer into event routing similar to how video layer is. However, the code changes required were quite huge and it didn't work at the end. I think I just don't fully understand the event routing. As a quick hack, I added audio and scripting right into MMCombo.
  The scripting socket allows reconfiguring the AIBO (through Config object) and also executing postures and motions. As postures and motions sometimes crash the AIBO, I suspect the way I execute them is flawed.

* Shared/Config.cc
  Shared/Config.h

  Added streaming_config structure into sound_config. The structure specifies how the stream from the AIBO's microphone is resampled before it is streamed over the network.

* tools/mon/Makefile

  Added 'fps.jar' and 'fps.tar.gz' targets. The latter creates a self-sufficient distribution of telepresence UI.

* tools/mon/org/tekkotsu/mon/ControllerListener.java

  Added connect/disconnect notification interface.
  Fixed a bug in exception handling.

* tools/mon/org/tekkotsu/mon/EStopListener.java

  Made listener add/remove methods public.
  Moved update notification to proper (I hope) place.

* tools/mon/org/tekkotsu/mon/HeadPointListener.java
  tools/mon/org/tekkotsu/mon/MechaController.java

  sendCommand does nothing in case connection is not available.
  Previously it threw a NullPointerException.

* tools/mon/org/tekkotsu/mon/Listener.java

  Added read/written byte counter.
  Simplified read/write code.
  Added readShort/writeShort.

* tools/mon/org/tekkotsu/mon/VisionListener.java

  Made listener add/remove methods public.

* tools/mon/org/tekkotsu/mon/Joints.java
  tools/mon/org/tekkotsu/mon/WorldStateJointsListener.java

  Added named getters for various joints and sensors.
  Added update listener to WorldStateJointsListener.
  Improved code to avoid frequent memory allocation.



NEW CODE

* project/ms/data/motion
  Added postures and motions to control the mouth, tail, sitting, lying and standing.

* tools/mon
  Added telepresence interface (a.k.a First-Person-Shooter UI).
  The Java code is in org.tekkotsu.fps package and relies on org.tekkotsu.mon for lower-level connection services.

There are roughly two layers: the GUI itself, and the Controller, which provides a facade to various 'Listeners' and their notifications.

The GUI integrates various helpful sensors, such as accelerometers, head joints, camera, microphone, IR, battery readings, pressure sensors (see FpsGUI2.png). Also, it provides a single-point interface for actuation. With the default setup the user can use the mouse to move the head, and keyboard for the rest, such as moving, changing posture, executing motions, changing sensoric settings (e.g., shutter speed). The idea is taken from First Person Shooter games, where the user doesn't have to switch between many different windows as is the case with the current Tekkotsu Monitoring interface.

The GUI works with the AIBO solely through the Controller. As Controller does not depend on the GUI, it should be possible to use the Controller in standalone mode. For example, one could run some AI by connecting it to the robot through the Controller interface without having to write C code and update the controller. Obviously, there are also some disadvantages to such an approach: less control, higher latency, lack of full autonomy.


The GUI uses InputMapper class to map inputs to commands. This approach enables one to bind commands and even scripts on the fly to various input events from keyboard and mouse.


The SoundIO, used by the GUI, provides audio recording and playback via Java Sound API. The class may be of interest if someone wants to incorporate audio support into other client side code.


BeanShell

The patch relies on BeanShell (http://www.beanshell.org, SPL/LGPL license) for scripting, which can be used to customize application behavior. For example, all the keyboard/mouse binding are scripted and can thus be reprogrammed on the fly. The console invoked by default through ~ (tilde) also provides a scripting interface.

I do not attach the BeanShell JAR file to save mail server some work. The JAR file can be downloaded from http://www.beanshell.org. I am using BeanShell version 2.0b1. The JAR file is to be placed into tools/mon as bsh.jar.




VERY BRIEF USAGE INSTRUCTIONS

* Configuring audio

  Go to tools/mon. Run:
  $ SoundIO -o help
  This will scan available output mixers. Careful, the test sound may be loud and annoying. Select a mixer (index) which works best. In config.bsh set sound.outputMixerIndex to the index of the mixer.

  $ SoundIO -o <your output mixer index> -i help
  This will scan available input mixers. Again, the test sound may be loud and annoying -- the applications first wants to make sure that your output mixers works and you can hear the audio. The application will scan input mixers. Some of them might work. When it starts recording 5 seconds of audio, say something into the microphone. Then see if this recording is properly played back. In config.bsh set sound.inputMixerIndex to the index of the input mixer which works for you.

  Now you can start the FpsGUI and enable the sound.

  To enable audio input from the robot's camera either press DEL/dot key of the keypad (default config) or in the BeanShell console execute:
  $ ui.audioOutputEnabled = true;
To enable audio output through the AIBO's microphone either press 0/INS key of the keypad or in the BeanShell console execute:
  $ ui.audioInputEnabled = true;

* Compile the controller. Flash it into an AIBO.

* Telepresence UI.
  To start, go to tools/mon. Compile. Run using FpsGUI script/BAT file.
  When the GUI starts, open console window using ~(tidle). I think keyboard layout doesn't influence scan codes. Thus, this is usually the button which is left from button 1 and above TAB.
  In the console run:
  $ connect("<AIBO IP/hostname>");
  If everything works fine, the GUI will get connected in a couple of seconds and will start showing telemetry from the AIBO.

  To close the console press the tilde key again.

  If all of the sensors seem to show fine, but not the video from the camera, try running
  $ controller.reconnect();

  In default setup mouse movements move AIBO's head. To capture mouse movements make sure the FpsGUI main windows is active and then press ESC. Now the mouse pointer will disappear and mouse movements will start  to move the robot's head. To stop capturing mouse movements and thus free up the mouse pointer press ESC again.

  Default keyboard/mouse event map from klyubin.bsh is as follows:

E    - lower/raise ears (press/depress)
R    - open/close mouth (press/depress)

A    - walk forward
Z    - walk backward
Q    - turn left
W    - turn right
CTRL - strafe left
ALT  - strafe right
T    - wiggle tail (press once)

F    - stand on the hind legs
S    - stand high on four legs
D    - sit
X    - lie down

The four postures above should only be traversed through adjacent ones. For example, to lie down from a standing posture one should first sit down and only then lie down. Otherwise, the AIBO may fall or hit the floor heavily.


F1   - layout keypad for audio control
F2   - layout keypad for video control
In either of the modes NumLock has to be enabled for the keypad to work.

Audio Control Layout
0    - Toggle robot's speakers
DEL  - Toggle AIBO's microphone
One can also bind some keys to resample audio streams, so that less bandwidth is used. Changing controller.microphoneSampleRate, controller.microphoneBitsPerSample, controller.microphoneStereo properties should do the trick.

Video Control Layout
DEL   - Toggle camera
0-3   - resolution (high-low). It is best to use 2 as a setting.
        Setting 1 results in a noticeable delay and low frame rate.
        Setting 0 halts the AIBO almost to a halt.
4-6   - brightness
7-9   - shutter speed

One can change input maps and even create own commands/actions using either .bsh files or direct programming through the BeanShell console.



Kind Regards,
Alexander Klyubin