This is penny wise, pound foolish. You reduce quad channel memory on the second bank to dual channel and you are inviting all kinds of trouble and/or incompatibilities by not using identical DIMM's for all slots. Performance issues arise when not fully populating banks. You may get by with 4 x 8GB sticks in the second bank fo 48 GB total, if they have the exact same timings as the first bank, but no guarantees that it will work.
Definitely do not run just 2 sticks of different density with the 4 sticks of 4GB. The boards currently support asynchronous timings but do not handle that well since the memory controller is in the CPU. Then on top of that ram models have different manufacturer chips with far different latency sub timings which will cause any number of issues. You are far better off replacing the 4Gb sticks with 8 GB sticks. At the very least you want 4x 4Gb sticks to add to what you have but you need to get the same model ram as the current and hope the chips are compatible with the chips in the current ram since the Dimm manufacturers change chips often from production run to production run. Over all the only reliable way to upgrade in this situation is replace the current sticks with all new ram of the same model and production. This is unfortunately the volatility of the ram chip production market.
Thanks to both contributers. Ensuring that the distribution was even was the way I was expecting to go based on my previous experience, but worth the conidersation to learn the pitfalls of such an approach and advance my understanding to include the mixing of manufacturing runs of RAM.