Tables of contents created using Layout > Table of Contents pull paragraph styles already included in your document. Have you formatted the numbers with a paragraph style?
If so, maybe you can duplicate the paragraph style. Use one version of the paragraph style on odd-numbered pages and one version on even-numbered pages, but only call out the even-numbered style in the TOC.
It would help if you could explain what the pages are like.