Not sure if you want to implement this but I defintive will add this in my personal build of some OS program, you might consider to also provide:
all options apply when the mouse is hovering above bg or fg rectangle inside the toolbar:
mousewheel changes value
shift mousewheel changes saturation
ctrl mousewheel changes hue
Might add a reply if things will enhance worflow as I expect.
Retrieving data ...