What is the scale level of the solid/how close to the camera is it? Is the continuously rasterize switch turned on for the layer?
I'm not sure about the continuously rasterize switch (I've never used that, where is it?) But it was the scale possibly combined with the layers proximity to the camera, it was scaled about 400-500%, I scaled it back down to 100% and up the layer resolution instead and re-drew my masks and it's working fine now. I'm still confused as to why that would cause so much blur on the edges of the mask? Either way thanks for the help, as soon as I read the scale level I was like "thaaaaat's probably what it is." kind of feel stupid for not thinking to try that first. Thanks a ton!
Anytime you try to scale something up beyond 100%, the program has got to try to make up information about what it would look like at that larger size. This means interpolating. Thus blurriness can ensue. The continuous rasterization button tells After Effects to keep rasterizing vector information. This means that Illustrator files (and sometimes masks) can be scaled up thousands of times and still remain crisp and clear.
More info on the continuous rasterization button here.