It certainly is. You don't mention what version of RH you are
using or what your output is. If you let us know we'll advise you
further as the detail depends on your answer. I assume it compiled
HTML (.CHM) but just checking.
You can do this with your skin. If you open your skin editor,
click on the Navigation tab and there you can specify images for a
closed or open book. Just specify your chosen image, try one of the
ones in the RH Gallery, and compile.