Wow, that's awful!
It is not unusual at all to have some fans on some PCs start initially at a very high RPM during initial boot, and something sounds terribly wrong with at least one fan. The good news is, your problem seems very repeatable. Having a fan that squeals only intermittently is much more difficult to trace down.
In your case, I would suggest:
1. Turning PC completely off
2. Completely and carefully block one fan from spinning
3. Start PC and let it run for about 2 seconds - that should be long enough to know if you are blocking the noisy fan and short enough to keep from cooking anything due to a stopped fan. You may need to do this with a power strip if your power supply does not have a quick-acting toggle switch to shut it down.
Repeat steps 2 and 3 until you are blocking the noisy fan. Here's a suggested order: CPU fan 1, CPU fan 2, power supply fan, motherboard chipset fan, any case fans that you have, GPU fans - dead last since changing the GPU did not seem to make any difference, and because it could be kind of difficult to block them carefully (with something long, non-conductive, and soft enough to not damage any of the fans)
4. Replace the bad fan, and/or fix what may be a mechanical interference problem with a fan
Good news! It was one of the case fans. Some interference with a cable or side cover. Of course, the least accessible fan, but I got it