Check your paths in the config.php as said above, these seem to be your issue.

If you need the paths ask your hosting provider, they will be more than happy to help you.

You are looking for the following:

Path to mplayer - TRY - /usr/bin/mplayer
Path to mencoder - TRY - /usr/bin/mencoder
Path to ffmpeg - TRY - /usr/local/bin/ffmpeg
Path to flvtool2 - TRY- /usr/bin/flvtool2

If these don't work as said above ask your hosting company. Which hosting company are you using?