Every time without fail After Effects crashes when I try to change my render settings. When I'm in Render Queue > Output Module > Click Lossless > Under Video Output > Format Options button. When I try to click the Format Options Button to change from animation quality to like h.256, After Effects Crashes. Below is the Alert message it gives me when it crashes. I have a late 2013 13" macbook pro with retina. I've purged all memory and disk cache as well and still it crashes every time. I can't have 10 second videos being 300 MB so any suggestions?
First, make sure your AE is up-to-date.
Second, you should be rendering your compressed files from the Adobe Media Encoder. H.264 from the render cue and QuickTime is a mess. Don't even try. Apple really found that up.
This is a Late Reply but the reason maybe the scale and layout of your display settings on your system
try to move the render settings window if it becomes smaller and starts to work correctly then that is your problem