I have set up bunch of color groups in the swatches panel, and int he book sync options, have clicked "Smart Match Style Groups." But when I sync the book, all the swatches are dented, but the swatches that get imported are just in a regular list, but in groups.
The only way I have found to get the swatches to be in groups is by importing them manually as an ASE file. Am I missing something?
No, Smart Match Style Groups refer to Paragraph and Character Styles which can include style groups. It doesn't refer to color swatches.
In addition to an ASE file, if you had a Creative Cloud subscription, you could use Creative Cloud Libraries to store and apply color groups.