I think I am as confused as you are, but I have been trying to understand what is happening.
I believe the problem occurs when ever you 'open' a book by clicking on the top menu for the Book module. For example- Add 10 images to a book layout (the 'first' edit), go back to the Grid view, select a folder of 100 images, Click the menu for Book module (the 'second' edit), : you now see 110 images on the filmstrip.
For the books I design, I always-
1. Create a 'Standard' Collection with the selected images I intend to use.
2. Start the Book design in the Book module
3. Immediately create a "Saved Book" from the tab above the main window.
4. For ALL future edit work on this book ALWAYS open it in the Book module by double-clicking on the Saved Book Collection.
I also then delete any remnant 'Standard' collections, and I keep my 'Book' Collections in a Book Collection 'Set' to be OC tidy.
If you need more photos added to the Book, simply drag them to the 'Book' Collection from Grid view.
The (saved) Book Collection is always constantly recording everything you do to your book design photos and layout.
Hint: (double-clicking on the Saved Book Collection.) OR a single-click on the white arrow that appears when you hover the mouse cursor over Book Collections.
I have the same problem - this looks like an annoying bug that compromises the usability of the book module at just the time when the introduction of customisable cells and templates had made it of interest again. The unwanted images added to the new books seem to be those those from the last book. (Another user has reported the bug at New book's pictures merge with previous ones! | Photoshop Family Customer Community and Adobe seems to have acknowledged it. They've suggested a workaround; however that workaround does not work for me.) I hope this gets fixed soon.
In the meantime the better workaraound suggested by WobertC above is invaluable: 1. Create a book based on selecting just one picture; 2. save the book; 3. go to the that book collection in the library module and remove all pictures that have now been wrongly added from the collection again; 4. now add all the pictures you really wanted to add in the first place by dragging them into the collection in the library module; 5. open the book via the arrow next to the collection. A little tedious, but so much better than manually removing the wrongly added pictures one by one in the book module after having them gotten mixed up with all the right ones. (Thanks, WobertC.)