I could not reproduce the issue.
I tried creating a course on Connect with 3 attempts, and pool of questions(6 out of 8).
I gave two attempts. The order as well as the questions were different in both attempts.
Which Presenter version are you using . What LMS ?
Could you drop your presentation here: email@example.com
Thanks for the reply to the query.
The Mandate that the user must have unlimited number of attempts.
Presenter version is 8 and the LMS is "sum total".
I will also mail you the presentation on the given email id in a moment.
This is a expected behaviour and this is how it is designed.
Let me explain :
The shuffle option is valid when we have a number of different users. So each person who takes the course may get different questions in different order.
For the same person who is taking the course in multple attempts, this workflow is like a resume scenario.
So, let a person taking a course exit it in between. When he re-launches the course he should see the same set of questions but from the question he left.
Now let him completes the course and fail. When he comes back, it is again a resume scenario.
On all the LMS(accept connect) this is the expected behaviour.
On connect , in case of multiple attempts, the user sees a different set of questions whenever he fails.
Hope this helps.