Best/Most efficient way of chromakeying long 4K source for 720p output?

[A.I.1]

I'm doing keying work for a video with a man in front of a green screen. The source is 4K (3840x2160@25 fps), just over 46 minutes in duration but only needs to be output at 720p.

It seems to key reasonably well (using the colour range keyer) except at various points the man keeps moving his hand/arm outside of the greenscreen background. This will need rotoscoping. I created a couple of mattes and scaled the video a bit (to get rid of things on the sides too), and have keyframed the matte around the hand/arm in some places. It's far from perfect though around the hand (though it's so quick when he moves his hands that I don't think it will matter all that much), but I'm wondering what's the best/most efficient way. He doesn't seem to move his hands outside the greenscreen all that much. Should I be using seperate masks for the hand? Is it simpler to use rotobrush or something else?

What about the resolution and file format? Since the output for the client will be only 720p and the source is 3840x2160@ 25 fps and is 46 mins long approx, should I convert to a 720p I-frame only format for more efficient working with it? Or to 1080p (I-frame only?)? What will be the best, most efficient way of doing this that will give reasonable results at 720p?

edit: I've created a separate mask for the hand, to hopefully make thing easier (so I can just keyframe that mask so it separate from the other 'garbage' masks). I've also added a spill suppressor and that's improved things so it looks quite good at 720p. I'm still wondering if there's a better/more efficient way though. Also, the keyed hand even though it's not exact (the mask is more basic than the hand outline, for speed), because of the composited background and a bit because of the hand is moving, the discrepancies don't stand out very much.

edit2: I'm currently pre-composing everything (actually doing 2 pre-composes) to a 720p resolution comp since it only needs to be output at 720p. But I'm wondering if there are better ways that will render everything in approx the same quality (looks about the same) but rendered faster (it uses over 80% of RAM which is good).

I'm currently using the old CS5.5 version of AE with Windows 10.

[Dave_LaRonde]

You're doing a forty-six-minute-long chroma key?  Ugh.

What's the media container & codec of this 4K source footage?

You may find renders will go a lot faster if you transcode to a different codec.  If you would benefit from the transcode, keep it in the same dimensions -- you wouldn't have to do the masking work over again.

[A.I.1]

Yes, about 46 mins . But it doesn't seem as though it will be too bad, since it keys reasonably well (apart from the issue with the hand occasionally moving past the greenscreen). And hopefully I can output it at a low bitrate 720p (He said 720p is okay, and I need it low bitrate on output too due to broadband not being very fast).

This is the video/codec info:

MPEG-4

Format profile                          : Base Media / Version 2

Codec ID                                : mp42 (mp42/mp41)

File size                                : 5.28 GiB

Duration                                : 46 min 19 s

Overall bit rate mode                    : Variable

Overall bit rate                        : 16.3 Mb/s

Video

ID                                      : 1

Format                                  : AVC

Format/Info                              : Advanced Video Codec

Format profile                          : High@L5.2

Format settings, CABAC                  : Yes

Format settings, ReFrames                : 4 frames

Codec ID                                : avc1

Codec ID/Info                            : Advanced Video Coding

Duration                                : 46 min 19 s

Bit rate mode                            : Variable

Bit rate                                : 16.0 Mb/s

Width                                    : 3 840 pixels

Height                                  : 2 160 pixels

Display aspect ratio                    : 16:9

Frame rate mode                          : Constant

Frame rate                              : 25.000 FPS

Standard                                : PAL

Color space                              : YUV

Chroma subsampling                      : 4:2:0

Bit depth                                : 8 bits

Scan type                                : Progressive

Stream size                              : 5.18 GiB (98%)

Color range                              : Limited

Color primaries                          : BT.709

So you think render the source video to I-frame only, still 4K, then replace the source footage in the comp, and that will give quite big increase in render time (and probably faster if I need to do more mask keyframes too) when I render to the 720p output (with all the chroma keys and chroma suppression effect and masks (with keyframes))?

[Dave_LaRonde · Accepted solution]

A.I.1  wrote

So you think render the source video to I-frame only, still 4K, then replace the source footage in the comp, and that will give quite big increase in render time (and probably faster if I need to do more mask keyframes too) when I render to the 720p output (with all the chroma keys and chroma suppression effect and masks (with keyframes))?

Just as I suspected -- 4K video with the bejeezus compressed out of it.

Mp4 and avc-intra are pretty much death on AE CS 5.  Frankly, I'm surprised you've been able to do any work at all!  You'll have a LOT easier time of this chore by transcoding.

So what to transcode TO?  Personally, I like Quicktime Movies in JPEG 2000, PNG, Photo JPEG or Animation codecs.  If you have access to ProRes, use it.  Be prepared for astoundingly - huge file sizes compared to the source.

[A.I.1]

Thanks. I'm currently (after reading your suggestion), rendering it out as Quicktime, Jpeg2000 codec with 100% quality (maybe I could reduce quality if it helps). It's currently saying it will take about 18 hours to encode . And rending it the other way was a lot faster. But I assume once it's rendered, any changes needed for the actual composition (eg. for keyframing where the hand goes past the greenscreen occasionally) will be a lot more efficient and faster to render and edit (though the whole thing needs to be done and uploaded this Sunday).

Note: I don't have the ProRes codec installed.

edit: because It was taking about 18 hours to encode the QuickTime Jpeg2000 100% quality version, I'm trying encoding it as H264 (same 3840x2160@25 fps) target bitrate 100 Mbps target (or I think a bit higher, and a max bitrate of about 150 Mbps I think) and a keyframe distance of 1 (I assume that will be I-Frame only). The estimated render time for this is around 2.5 hours. I assume I can replace the original source footage in the other comp with the output of this and it will still be a lot faster to edit/render than using the originally sent H264 file (and I assume/hope picture quality won't be affected too much). edit2: Now trying quicktime with anim codec 100% quality with keyframe every 3 frames instead in case that will preserve more quality.

[A.I.1]

I did try a high bitrate version (>=100 Mbps target) of H264 with keyframes every 1 frame (though I stopped it). I'm not sure whether or not that would be good (not sure if it would affect quality too much etc.).

I'm currently outputting using the Quicktime PhotoJpeg codec instead. It's faster rendering to that than quicktime Jpeg2000 (about 3 hours 50 min to encode rather than 18 hours), and hopefully the quality will be good and it should in theory be fast at decoding.

EDIT:

What's confusing is when I replaced the 3840x2160@25 fps AVC, about 5 GB footage in the comp with the the QuickTime PhotoJpeg footage - same res but 192 GB, the render of approx the whole clip duration took longer with the PhotoJpeg clip.

With the PhotoJpeg clip (192 GB) the render of the comp (with the masks and keying etc.) to 720p said it had 8 hours remaining.

With the AVC 3840x2160 clip (around 5 GB), the render of the comp (with masks and keying etc.) to 720p said and says around 5 hours remaining (or just under).

Perhaps it (using PhotoJpeg as the source) would be more benefit for editing speed rather than render speed. Perhaps the faster render using the AVC source rather than PhotoJpeg source is because the AVC (5 GB) clip could be cached in RAM (the compressed version) whereas the PhotoJpeg compressed version (192 GB) couldn't.