Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
project:dmcpp [2010/06/16 20:31]
nishimotz created
project:dmcpp [2010/11/09 08:58] (current)
nishimotz
Line 1: Line 1:
 in Japanese: http://​ja.nishimotz.com/​project:​dmcpp in Japanese: http://​ja.nishimotz.com/​project:​dmcpp
 +====== DMCPP ======
 +
 +An experimental dialog manager for [[http://​sourceforge.jp/​projects/​galatea/​|galatea]],​ using lib[[http://​sourceforge.jp/​projects/​julius/​|julius]] and [[http://​sourceforge.net/​projects/​opencvlibrary/​|opencv]],​ written in C++.
 +
 +Source file: http://​github.com/​nishimotz/​dmcpp
 +
 +An experiment which replaces the dialog manager (DM) creaded by galatea-generate by an application written in C++.
 +
 +The DM application also handles speech recognition engine [[:​julius]],​ with the same mannar of the Java based [[:​galatea_dialog_studio]].
 +[[:opencv]] face detection is also integrated.
 +
 +The project is tested with Ubuntu 9.04, 10.04, 10.10 (i386).
 +In this project, the character encoding is UTF-8.
 +Japanese speech engines are used, currently.
 +
 +===== setup =====
 +
 +Ubuntu official packages:
 +
 +<​code>​
 +$ sudo aptitude install ruby
 +$ sudu aptitude install rake
 +$ sudo aptitude install freeglut3
 +$ sudo aptitude install openjdk-6-jre
 +$ sudo aptitude install chasen
 +$ sudo aptitude install rhino
 +$ sudo aptitude install libreadline5
 +$ sudo aptitude install ttf-sazanami-gothic
 +</​code>​
 +
 +<​code>​
 +$ sudo aptitude install libasound2-dev
 +$ sudo aptitude install libavcodec52
 +$ sudo aptitude install libavformat52
 +</​code>​
 +
 +
 +sourceforge.jp packages:
 +
 +  * http://​sourceforge.jp/​projects/​galatea/​releases/​
 +    * galatea-dialog-20100530 ​
 +    * galatea-engine-20090604
 +    * galatea-ja-chaone-1.3.2
 +    * galatea-ja-unidic-1.3.9
 +
 +packages by nishimotz:
 +
 +<del>
 +  * http://​files.nishimotz.com/​ruby-1.9.2_preview3-2_i386.deb
 +    * or http://​files.nishimotz.com/​ruby-1.9.1_p376-1_i386.deb
 +    *  [[:rake]] is required for [[:​ruby1.9]]. "​aptitude install rake" is maybe enough for this.
 +</​del>​
 +
 +  * http://​ja.nishimotz.com/​_media/​julius_4.1.5-2_i386.deb
 +  * http://​files.nishimotz.com/​opencv-local_2.1.0-1_i386.deb
 +    * opencv 1.x is not tested.
 +
 +An USB camera, working with Linux, is also required. ​
 +
 +===== compilation and execution =====
 +
 +Clone source files using git
 +
 +<​code>​
 +$ git clone git://​github.com/​nishimotz/​dmcpp.git
 +$ cd dmcpp
 +</​code>​
 +
 +or download dmcpp tar ball.
 +Visit [[http://​github.com/​nishimotz/​dmcpp|github]] and click "​download source"​ and select tar.
 +
 +nishimotz-dmcpp-ec191ea.tar.gz (the revision may be different.)
 +
 +<​code>​
 +$ tar xvfz nishimotz-dmcpp-ec191ea.tar.gz
 +$ cd nishimotz-dmcpp-ec191ea
 +</​code>​
 +
 +compile and run:
 +
 +<​code>​
 +$ make
 +$ ./​app/​script/​runner
 +</​code>​
 +
 +If you get error while doing make, something is wrong.
 +
 +Speak "​Yasuda Kodo" to microphone, then the agent will speak "​Yasuda Kodo desu ne."
 +
 +===== speech recognition grammar =====
 +
 +gram.txt contains the lines of Japanese words,
 +such as "​やすだこうどう 安田講堂", ​
 +which consists of the pronunciation and desctiption.
 +
 +Within the make, gram is generated.
 +The procedure is written in rakefile.rb.
 +A class within the Java verion of Dialog Studio is invoked.
 +
 +===== dm.cpp =====
 +
 +http://​github.com/​nishimotz/​dmcpp/​blob/​master/​dm.cpp
 +
 +In main() thread, stdin is read and processed.
 +
 +julius_worker(),​ cv_worker() and main() run simultaneously as the multiple threads.
 +
 +struct thread_info_t can be used for sharing information among the thread.
 +It can be used for dialog management.
 +
 +==== speech recognition and synthesis ====
 +
 +The task is "​parrot."​
 +
 +In the callback function output_result(),​ command such as "to @AM-MCL set Speak = %sですね"​ is send to the AgentManager via stdout.
 +
 +INTERIM [[:​julius_progout]] is just send to stderr, currently.
 +
 +julius_worker() is created and detached using pthread.
 +
 +The use of workorder_t is a future work.
 +
 +==== face detectiuon ====
 +
 +[[:opencv]] based face detection is also integrated experimentally.
 +
 +The worker thread, cv_worker(),​ is running face detecion loop.
 +
 +===== app =====
 +
 +It replaces [[:​galatea_dialog_studio]] dialog manager in the following way:
 +
 +app/​skel/​am.conf.txt
 +
 +<​code>​
 +# module:​DM ​    ​commands:​ <%= am_commands_dm %>
 +module:​DM ​    ​commands:​ cd <%= File.join($runner_dir,​ '​..'​) %>; ./dm
 +</​code>​
 +
 +Although the extention of file is txt, this is ERB.
 +
 +$runner_dir is a global value defined at app/​script/​runner.
project/dmcpp.1276687878.txt.gz · Last modified: 2010/06/16 11:31 (external edit)
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0