I essentially have two compositions, one has a box with a frame around it and the other is the background. The box with the frame around it, has the fill set as a blurry piece of glass, which works fine within that comp; it blurs anything I set behind it. But when I nest that composition into another comp the effect is lost. How do I set up my box composition to be treated as a graphic that I can place into any comp and it will blur anything that is behind it?
You can't. That's just not how it works. You need to use a duplicate comp with just the insert and use it as an adjustment layer or matte on the other blurry stuff. You may even need to conenct multiple items across comps using expressions to keep everything aligned.