Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
project:dmcpp [2010/06/16 20:31] – created nishimotzproject:dmcpp [2010/11/09 08:58] (current) nishimotz
Line 1: Line 1:
 in Japanese: http://ja.nishimotz.com/project:dmcpp in Japanese: http://ja.nishimotz.com/project:dmcpp
 +====== DMCPP ======
 +
 +An experimental dialog manager for [[http://sourceforge.jp/projects/galatea/|galatea]], using lib[[http://sourceforge.jp/projects/julius/|julius]] and [[http://sourceforge.net/projects/opencvlibrary/|opencv]], written in C++.
 +
 +Source file: http://github.com/nishimotz/dmcpp
 +
 +An experiment which replaces the dialog manager (DM) creaded by galatea-generate by an application written in C++.
 +
 +The DM application also handles speech recognition engine [[:julius]], with the same mannar of the Java based [[:galatea_dialog_studio]].
 +[[:opencv]] face detection is also integrated.
 +
 +The project is tested with Ubuntu 9.04, 10.04, 10.10 (i386).
 +In this project, the character encoding is UTF-8.
 +Japanese speech engines are used, currently.
 +
 +===== setup =====
 +
 +Ubuntu official packages:
 +
 +<code>
 +$ sudo aptitude install ruby
 +$ sudu aptitude install rake
 +$ sudo aptitude install freeglut3
 +$ sudo aptitude install openjdk-6-jre
 +$ sudo aptitude install chasen
 +$ sudo aptitude install rhino
 +$ sudo aptitude install libreadline5
 +$ sudo aptitude install ttf-sazanami-gothic
 +</code>
 +
 +<code>
 +$ sudo aptitude install libasound2-dev
 +$ sudo aptitude install libavcodec52
 +$ sudo aptitude install libavformat52
 +</code>
 +
 +
 +sourceforge.jp packages:
 +
 +  * http://sourceforge.jp/projects/galatea/releases/
 +    * galatea-dialog-20100530 
 +    * galatea-engine-20090604
 +    * galatea-ja-chaone-1.3.2
 +    * galatea-ja-unidic-1.3.9
 +
 +packages by nishimotz:
 +
 +<del>
 +  * http://files.nishimotz.com/ruby-1.9.2_preview3-2_i386.deb
 +    * or http://files.nishimotz.com/ruby-1.9.1_p376-1_i386.deb
 +    *  [[:rake]] is required for [[:ruby1.9]]. "aptitude install rake" is maybe enough for this.
 +</del>
 +
 +  * http://ja.nishimotz.com/_media/julius_4.1.5-2_i386.deb
 +  * http://files.nishimotz.com/opencv-local_2.1.0-1_i386.deb
 +    * opencv 1.x is not tested.
 +
 +An USB camera, working with Linux, is also required. 
 +
 +===== compilation and execution =====
 +
 +Clone source files using git
 +
 +<code>
 +$ git clone git://github.com/nishimotz/dmcpp.git
 +$ cd dmcpp
 +</code>
 +
 +or download dmcpp tar ball.
 +Visit [[http://github.com/nishimotz/dmcpp|github]] and click "download source" and select tar.
 +
 +nishimotz-dmcpp-ec191ea.tar.gz (the revision may be different.)
 +
 +<code>
 +$ tar xvfz nishimotz-dmcpp-ec191ea.tar.gz
 +$ cd nishimotz-dmcpp-ec191ea
 +</code>
 +
 +compile and run:
 +
 +<code>
 +$ make
 +$ ./app/script/runner
 +</code>
 +
 +If you get error while doing make, something is wrong.
 +
 +Speak "Yasuda Kodo" to microphone, then the agent will speak "Yasuda Kodo desu ne."
 +
 +===== speech recognition grammar =====
 +
 +gram.txt contains the lines of Japanese words,
 +such as "やすだこうどう 安田講堂", 
 +which consists of the pronunciation and desctiption.
 +
 +Within the make, gram is generated.
 +The procedure is written in rakefile.rb.
 +A class within the Java verion of Dialog Studio is invoked.
 +
 +===== dm.cpp =====
 +
 +http://github.com/nishimotz/dmcpp/blob/master/dm.cpp
 +
 +In main() thread, stdin is read and processed.
 +
 +julius_worker(), cv_worker() and main() run simultaneously as the multiple threads.
 +
 +struct thread_info_t can be used for sharing information among the thread.
 +It can be used for dialog management.
 +
 +==== speech recognition and synthesis ====
 +
 +The task is "parrot."
 +
 +In the callback function output_result(), command such as "to @AM-MCL set Speak = %sですね" is send to the AgentManager via stdout.
 +
 +INTERIM [[:julius_progout]] is just send to stderr, currently.
 +
 +julius_worker() is created and detached using pthread.
 +
 +The use of workorder_t is a future work.
 +
 +==== face detectiuon ====
 +
 +[[:opencv]] based face detection is also integrated experimentally.
 +
 +The worker thread, cv_worker(), is running face detecion loop.
 +
 +===== app =====
 +
 +It replaces [[:galatea_dialog_studio]] dialog manager in the following way:
 +
 +app/skel/am.conf.txt
 +
 +<code>
 +# module:DM     commands: <%= am_commands_dm %>
 +module:DM     commands: cd <%= File.join($runner_dir, '..') %>; ./dm
 +</code>
 +
 +Although the extention of file is txt, this is ERB.
 +
 +$runner_dir is a global value defined at app/script/runner.
project/dmcpp.txt · Last modified: 2010/11/09 08:58 by nishimotz
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0