do you have a layer style applied to the bottom layer? then 3D+Layer style+Track matte won't work in this case and never did. if this is another issue please be more specific and show as screenshots and specify exactly where is the problem.
No layer styles applied. Just simple basic shapes.
I have tested the same shapes with 2015 and then recreated in 2017. It doesn't work in 2017 but works fine in 2015.
See Images below.
On the first I have the track mattes enabled on 2d layer in 2017.
The next image I have made them 3d layers and now they break. The bottom image is the track mattes with 3d layers recreated in 2015.
I also just noticed Blending modes do not work in 3d layers either. Once I change it to a 3d layer blending mode disappears.
wow thank you sir!