I've found the solution myself. Just gonna leave the solution here for other people who encounter this problem in the future.
Hold down the "CTRL" key and click on the format you want to have as the default.
Well I didn't know that, and it's a really useful tip. Nice one Vincent.
you're welcome =)