Transcoding MKV/OGM to MP4

After endlessly mucking around with transcoding MKV files to MP4 and going through many false leads from various Google searches, I've finally figured out a decent way to transcode from MKV to MP4 without too much of a problem. Here are the prerequisites in order to do the following task:

• Some Unix variant. In this case, I used Mac OS X Leopard. This should definitely work in Linux, though.
• x264/XviD codec libraries. For this example, I will be using x264, so be sure that the required libraries and binaries are available.
• faac/faad audio codec libraries. We want to be able to create a video file that is playable in both QuickTime and on the PSP/PS3, so using the AAC audio codec is a must.
  
• mplayer/mencoder binaries. The newer, the better. SSA/ASS support needs to be compiled for embedded font support for subtitles.
• (Optional) Multiple processors. This will definitely speed things up, so it is a must if you have a bunch of stuff to transcode.

After the prerequisites are met, you can do the following in order to transcode between formats:

1. Open two terminal windows.
   
2. In one terminal, create a work directory. I created a folder in my home directory called encoding.
3. In that directory, create a fifo pipe: mkfifo stream.yuv.
4. Dump the audio from the video file to a .wav file:
   # mplayer FILENAME -vo null -vc null -ao pcm:file=audiodump.wav
   
5. Start playback of the video file that needs to be transcoded within your work directory:
   # mplayer FILENAME -vo yuv4mpeg -ao null -nosound -noframedrop
6. In the other terminal, cd into your work directory and start the transcoding with the following:
   # cat stream.yuv | mencoder - -o OUTFILENAME.mp4 -mc 0 \
   -ovc x264 -x264encopts global_header:bitrate=400:\
   vbv_maxrate=768:vbv_bufsize=2000:partitions=all:\
   trellis=1:level_idc=30:threads=auto \
   -vf harddup,scale=480:-10,expand=480:272 \
   -sws 9 -audiofile audiodump.wav -oac faac \
   -faacopts br=128:mpeg=4:object=2:raw \
   -channels 2 -srate 48000 -ofps 24000/1001 \
   -of lavf -lavfopts format=psp
7. Wait for completion.

In this case, the mencoder command specifically creates a file that is playable on the PSP/PS3. As a consequence of the settings though, it is also playable in QuickTime. These settings are completely tunable, depending on what is needed. An entirely different codec, e.g. XviD/DivX, can be used instead in order to fit whatever requirements you may have.

From my limited experience, two-pass encoding with x264 is almost always better than only doing one-pass, even if it is more time consuming. I am not sure whether or not the resultant video stream that is piped into the fifo will be the same for both passes, but it should be a good estimation regardless. You probably do a two-pass encoding as follows, given that you have performed steps 1-4 beforehand:

# mplayer FILENAME -ass -embedded-fonts -sid 0 -correct-pts -vo yuv4mpeg -ao null -noframedrop -benchmark -nosound -really-quiet &; cat stream.yuv | mencoder - -o /dev/null -mc 0 -ovc x264 \
-x264encopts pass=1:turbo:global_header:\
bitrate=400:vbv_maxrate=768:vbv_bufsize=2000:\
partitions=all:trellis=1:level_idc=30:threads=auto \
-vf harddup,scale=480:-10,expand=480:272 \
-sws 9 -audiofile audiodump.wav -oac faac \
-faacopts br=128:mpeg=4:object=2:raw \
-channels 2 -srate 48000 -ofps 24000/1001 \
-of lavf -lavfopts format=psp
# mplayer FILENAME -ass -embedded-fonts -sid 0 -correct-pts -vo yuv4mpeg -ao null -noframedrop -benchmark -nosound -really-quiet &; cat stream.yuv | mencoder - -o OUTFILENAME.mp4 -mc 0 -ovc x264 \
-x264encopts pass=2:subq=7:me=umh:global_header:\
bitrate=400:vbv_maxrate=768:vbv_bufsize=2000:\
partitions=all:trellis=1:level_idc=30:threads=auto \
-vf harddup,scale=480:-10,expand=480:272 \
-sws 9 -audiofile audiodump.wav -oac faac \
-faacopts br=128:mpeg=4:object=2:raw \
-channels 2 -srate 48000 -ofps 24000/1001 \
-of lavf -lavfopts format=psp

Two new options for the x264 encoder: subq and me. These options will make the encoding significantly longer, so be forewarned. I prefer that the video to be of good quality, so I usually use them for the second pass. Which is why the recommendation for a multicore/multiprocessor system. Also, the quiet flag will completely squelch the output from mplayer, so take this out if you don't want this. For more information about any of the mplayer/mencoder options, please refer to the Mplayer documentation or man page.

Hopefully, this has been somewhat informative. So, get on with some transcoding now, eh?