How many processor cores do you have in your computer?
There is a severe bug in Apple's H.264 QuickTime component that causes failures when a large number of threads are in use. The failure seems to occur at 24 threads, which you'll encounter with 12 processor cores with hyperthreading enabled.
well I'm using dual Xeons with 12 cores and 24 threads due to hyperthreading....
So what should I do to resolve the issue?
That's really a question for Apple, since it's about how to work past their bug.
I have seen people work around this bug by disabling hyperthreading.
Are you certain that H.264 in a .mov file is what you need? That is very seldom the best choice for output.