I spent several days now (non-continuus work :)) to install CMU Sphinx on my Raspberry Pi.
First there were problems installing it, cause of a too lower version of cmake on debian. After upgrading debian(Raspbian) to Jessie cmake was fine and CMU Sphinx installation was straight forward. Though after a restart, the Pi didn't boot anymore.
So I tried it again with Arch Linux which was rather painful:
CMU Sphinx wants(!) to use pulseaudio or jack for its audio-input. Though these are not so easy to setup (especially if you don't use a GUI). To force the CMU Sphinx configure script to use simple ALSA run:
sudo mv /usr/include/jack/jack.h /usr/include/jack/jack.h.old
sudo mv /usr/include/pulse/pulseaudio.h /usr/include/pulse/pulseaudio.h.old
Then follow the common installation guides
Have fun.