This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
project:dmcpp [2010/06/16 20:37] – nishimotz | project:dmcpp [2010/11/09 08:58] (current) – nishimotz | ||
---|---|---|---|
Line 1: | Line 1: | ||
in Japanese: http:// | in Japanese: http:// | ||
- | |||
====== DMCPP ====== | ====== DMCPP ====== | ||
- | dialog manager for [[: | + | An experimental |
Source file: http:// | Source file: http:// | ||
Line 9: | Line 8: | ||
An experiment which replaces the dialog manager (DM) creaded by galatea-generate by an application written in C++. | An experiment which replaces the dialog manager (DM) creaded by galatea-generate by an application written in C++. | ||
- | The DM application also handles speech recognition engine [[: | + | The DM application also handles speech recognition engine [[: |
[[:opencv]] face detection is also integrated. | [[:opencv]] face detection is also integrated. | ||
- | The project is tested with Ubuntu 9.04 and Ubuntu | + | The project is tested with Ubuntu 9.04, 10.04, 10.10 (i386). |
In this project, the character encoding is UTF-8. | In this project, the character encoding is UTF-8. | ||
Japanese speech engines are used, currently. | Japanese speech engines are used, currently. | ||
Line 22: | Line 21: | ||
< | < | ||
$ sudo aptitude install ruby | $ sudo aptitude install ruby | ||
+ | $ sudu aptitude install rake | ||
$ sudo aptitude install freeglut3 | $ sudo aptitude install freeglut3 | ||
$ sudo aptitude install openjdk-6-jre | $ sudo aptitude install openjdk-6-jre | ||
Line 29: | Line 29: | ||
$ sudo aptitude install ttf-sazanami-gothic | $ sudo aptitude install ttf-sazanami-gothic | ||
</ | </ | ||
+ | |||
+ | < | ||
+ | $ sudo aptitude install libasound2-dev | ||
+ | $ sudo aptitude install libavcodec52 | ||
+ | $ sudo aptitude install libavformat52 | ||
+ | </ | ||
+ | |||
sourceforge.jp packages: | sourceforge.jp packages: | ||
Line 40: | Line 47: | ||
packages by nishimotz: | packages by nishimotz: | ||
+ | <del> | ||
* http:// | * http:// | ||
* or http:// | * or http:// | ||
- | * [[: | + | * [[: |
- | * {{: | + | </ |
- | * [[:julius]] を見てください | + | |
+ | * http:// | ||
* http:// | * http:// | ||
- | * opencv 1.x 系は試していない | + | * opencv 1.x is not tested. |
+ | |||
+ | An USB camera, working with Linux, is also required. | ||
- | Linux で認識できる USB カメラをつないでおく必要がある。 | + | ===== compilation and execution |
- | ===== コンパイルと実行 | + | |
- | git からレポジトリを複製する。 | + | Clone source files using git |
< | < | ||
Line 58: | Line 68: | ||
</ | </ | ||
- | または | + | or download |
- | [[http:// | + | Visit [[http:// |
- | nishimotz-dmcpp-ec191ea.tar.gz | + | nishimotz-dmcpp-ec191ea.tar.gz |
< | < | ||
Line 68: | Line 78: | ||
</ | </ | ||
- | コンパイルして実行。 | + | compile and run: |
< | < | ||
Line 75: | Line 85: | ||
</ | </ | ||
- | make でエラーが出たらどこか失敗している。 | + | If you get error while doing make, something is wrong. |
- | マイクに「安田講堂」と喋ると「安田講堂ですね」という応答が得られるはず。 | + | Speak " |
- | ===== 音声認識文法 | + | ===== speech recognition grammar |
- | gram.txt | + | gram.txt |
+ | such as "やすだこうどう 安田講堂", | ||
+ | which consists of the pronunciation and desctiption. | ||
+ | |||
+ | Within the make, gram is generated. | ||
+ | The procedure is written in rakefile.rb. | ||
+ | A class within the Java verion of Dialog Studio is invoked. | ||
- | make の中で gram の生成が行われるが、この処理は rakefile.rb に書かれており、 | ||
- | Java 版 Dialog Studio のクラスが呼び出されている。 | ||
===== dm.cpp ===== | ===== dm.cpp ===== | ||
http:// | http:// | ||
- | main() | + | In main() |
- | julius_worker | + | julius_worker(), cv_worker() and main() run simultaneously as the multiple threads. |
- | マルチスレッドで同時に動く。 | + | |
- | 構造体 | + | struct |
- | これを使って顔検出と音声入力の状態の組み合わせから | + | It can be used for dialog management. |
- | 対話状態を切り替えたり音声応答をしたりすることができるはず。 | + | |
- | ==== 音声認識と応答 | + | ==== speech recognition and synthesis |
- | オウム返しタスク。 | + | The task is "parrot." |
- | output_result() コールバックの中で標準出力に | + | |
- | INTERIM [[: | + | In the callback function output_result(), |
- | julius_worker() は pthread で create & detach されている。 | + | INTERIM [[: |
- | workorder_t, | + | julius_worker() is created and detached using pthread. |
- | ==== 顔検出 ==== | + | |
- | [[:opencv]] の顔検出を組み合わせてみた。 | + | The use of workorder_t is a future work. |
- | 新たなworkerスレッド | + | ==== face detectiuon ==== |
+ | |||
+ | [[:opencv]] based face detection is also integrated experimentally. | ||
+ | |||
+ | The worker | ||
- | 対話制御に反映させるのはこれから。 | ||
===== app ===== | ===== app ===== | ||
- | どうやって | + | It replaces |
app/ | app/ | ||
Line 126: | Line 139: | ||
</ | </ | ||
- | 拡張子が | + | Although the extention of file is txt, this is ERB. |
- | + | ||
- | $runner_dir は app/ | + | |
+ | $runner_dir is a global value defined at app/ |