I use AE CC on Mac OS X 10.8.5 I feel Rotobrush is too slow. More than one minute per frame. The edge is a single sitted person
CC is already 64-bit
then what is wrong with my rotobrush ? My Mac is 8GB RAM, should increasing RAM will rotobrush faster ?
Rotobrush depends on the type of footage and resolution. It is a resource hog and works much better in CC 2014 than it does in 2015. Slowness could be depend on the type of footage (codec), the size of the frame, the frame rate, the complexity of the background or a bunch of other issues.
All things being equal I would suggest that you convert your video to a 1920 x 1080 JPEG or PNG Image sequence and use that for Roto brush if the original video is any flavor of MP4 or nonstandard camera original.
Without complete system and workflow details that's the best suggestion I can give.