Thank you for your response. I am not really interested in identifing objects or surfaces in frame.
I am refereing to http://en.wikipedia.org/wiki/Motion_vector that would descibe a pixels or block of pixels movement from one frame to another. These might be used for creating motion compensated frames (http://en.wikipedia.org/wiki/Motion_compensation).
So I am hearing this is a no?
I am not an expert in video compression, but it seems to be the field specific term for the same concept.
After Effects does not provide any image processing algorithms beyond standard pixel blending and colour conversion, so yes, you would need to obtain pixels of the layer, call a custom algorithm and then write it back to the output layer/world. Beware that after effects uses ARGB channel sequence in a pixel, which is different to what most other libraries use (e.g. BGRA or RGBA).