I'm looking to compress a video (footage+animation) down to just beyond YT's upload limit of 128GB. I've calculated that a spatial resolution of 60 with QT's animation codec will help me achieve this, resulting in an 80GB file. An alternative is to render at 100% spatial res, generating a 170GB file and compressing via H.264 (with high bitrate)
Which method would result in a higher quality video?
It's usually recommended to create your video in h.264 using the Adobe Media Encoder to match the specs YouTube requests. (Check their help pages for exact info.) Render a lossless file out of AE and use that in the AME to make your final file. This way you can test a few different settings without having to re-render your whole comp again.