Does anyone know what setting(s) to use for Open GL that would not degrade the final HD render?
OpenGL is more about smooth interactions than it is about final rendering. Many operations work quickly, but with draft-like results.
Quality will not be as high as with the software renderrer, plus many effects and operations in AE are simply not accelerated by OpenGL.
I personally use the OpenGL - Interactive mode in Fast Previews to get faster interactions when dragging elements or adjusting parameters. As soon as you release the mouse button, the software renderer kicks in and processes the frame at the highest quality. Same for Render Queue renders. You normally don't want to use the OGL renderer for this.
Though, as mentioned, OpenGL is not exactly aimed at rendering final output quality, it still can be used for such a workflow. in your case this seems to be more of an issue with the graphics card configuration. The way i figure it, it limits texture size to 1024x1024 or something like that and then simply scales up, which of course will look soft. Therefore a small excursion into the graphics card settings is advisable. Also note, that by default AE will only use 200 MB texture buffer, so if you increase that value in the OpenGL prefs (Advanced), the problem may also resolve itself. Depending on the number of effects and OpenGL accelerated features you use, you may still run out of resources eventually, so check, whether it only occurs with a given number of such features or if e.g. turning off an accelerated effect improves the result. As a last tip, you may consider setting layers to draft quality. With sub-pixel calculations turned off, you may also be able to circumvent some of the limitations. And naturally, even if it is scaled, it will look less soft. It's not perfect, but for interlaced footage it would still provide an acceptable compromise.