DMCPP

An experimental dialog manager for galatea, using libjulius and opencv, written in C++.

Source file: http://github.com/nishimotz/dmcpp

An experiment which replaces the dialog manager (DM) creaded by galatea-generate by an application written in C++.

The DM application also handles speech recognition engine julius, with the same mannar of the Java based galatea_dialog_studio. opencv face detection is also integrated.

The project is tested with Ubuntu 9.04, 10.04, 10.10 (i386). In this project, the character encoding is UTF-8. Japanese speech engines are used, currently.

Ubuntu official packages:

$ sudo aptitude install ruby
$ sudu aptitude install rake
$ sudo aptitude install freeglut3
$ sudo aptitude install openjdk-6-jre
$ sudo aptitude install chasen
$ sudo aptitude install rhino
$ sudo aptitude install libreadline5
$ sudo aptitude install ttf-sazanami-gothic

$ sudo aptitude install libasound2-dev
$ sudo aptitude install libavcodec52
$ sudo aptitude install libavformat52

sourceforge.jp packages:

• http://sourceforge.jp/projects/galatea/releases/
  • galatea-dialog-20100530
  • galatea-engine-20090604
  • galatea-ja-chaone-1.3.2
  • galatea-ja-unidic-1.3.9

packages by nishimotz:

* http://files.nishimotz.com/ruby-1.9.2_preview3-2_i386.deb * or http://files.nishimotz.com/ruby-1.9.1_p376-1_i386.deb * rake is required for ruby1.9. “aptitude install rake” is maybe enough for this.

• http://ja.nishimotz.com/_media/julius_4.1.5-2_i386.deb
• http://files.nishimotz.com/opencv-local_2.1.0-1_i386.deb
  • opencv 1.x is not tested.

An USB camera, working with Linux, is also required.

Clone source files using git

$ git clone git://github.com/nishimotz/dmcpp.git
$ cd dmcpp

or download dmcpp tar ball. Visit github and click “download source” and select tar.

nishimotz-dmcpp-ec191ea.tar.gz (the revision may be different.)

$ tar xvfz nishimotz-dmcpp-ec191ea.tar.gz
$ cd nishimotz-dmcpp-ec191ea

compile and run:

$ make
$ ./app/script/runner

If you get error while doing make, something is wrong.

Speak “Yasuda Kodo” to microphone, then the agent will speak “Yasuda Kodo desu ne.”

gram.txt contains the lines of Japanese words, such as “やすだこうどう 安田講堂”, which consists of the pronunciation and desctiption.

Within the make, gram is generated. The procedure is written in rakefile.rb. A class within the Java verion of Dialog Studio is invoked.

http://github.com/nishimotz/dmcpp/blob/master/dm.cpp

In main() thread, stdin is read and processed.

julius_worker(), cv_worker() and main() run simultaneously as the multiple threads.

struct thread_info_t can be used for sharing information among the thread. It can be used for dialog management.

The task is “parrot.”

In the callback function output_result(), command such as “to @AM-MCL set Speak = %sですね” is send to the AgentManager via stdout.

INTERIM julius_progout is just send to stderr, currently.

julius_worker() is created and detached using pthread.

The use of workorder_t is a future work.

opencv based face detection is also integrated experimentally.

The worker thread, cv_worker(), is running face detecion loop.

It replaces galatea_dialog_studio dialog manager in the following way:

app/skel/am.conf.txt

# module:DM     commands: <%= am_commands_dm %>
module:DM     commands: cd <%= File.join($runner_dir, '..') %>; ./dm

Although the extention of file is txt, this is ERB.

$runner_dir is a global value defined at app/script/runner.