if you are using simple buttons (and not movieclip buttons), then only the up animation will play when your button first appears and is not moused over.
if you have a movieclip button, you need a stop() in its first frame.
I think they are movieclip buttons, although I create them as simple buttons, they assume the movieclip icon when I am inside them
What first frame are you refering to: the first frame in the main timeline or the first frame inside the button?
you should see your button symbol in the library. if it has a movieclip icon, you need to add a stop() to that movieclip's first frame.