I'm trying to recreate these lines in After Effects, which will keep the color and shape out of the square, but once they come across they mask the square. I usually keep searching on google until I find the solution, but this time I don't even know how to query this. Please, can anybody give me a hint? thank you!
You simply need to use duplicates and pre-comps and invert and mask those pre-comps or use them as mattes, respectively. Just keeping it dumb and trying to do it in one single comp is not going to get you anywhere. Nothing at all complicated here, just a bit of extra work.
I don't even know how to query this.
you are having mask issues. you should learn more about masks to figure out how to create this. in addition to what M suggested, there is valuable information here: Use alpha channels, masks, and mattes in After Effects. if you are still having specific problems, let us know.