The way it's working now is the correct way. What you're describing sounds like a bug, and it's good it's not doing it anymore, as it's a serious security flaw.
This is a Security Settings option:
Passwords and password time-outs can only be set for PKCS#12 IDs. Since a file can contain multiple IDs, passwords and time-outs are configured at the file level rather than for individual IDs.
Note:If the is read only, then the Change Password and Password Timeout options are disabled.
To change the password timeout:
1. Choose Advanced (Acrobat) or Document (Reader) > Security Settings.
2. Highlight Digital ID Files in the left-hand tree (Figure 13).
3. Select a file in the right-hand panel (Figure 13).
4. Choose Password Timeout.
Tip: The password timeout feature interacts with the Login/Logout feature as described in
“Logging in to PKCS#12 Files”
5. Configure the Password Timeout Policy dialog by specifying when a password prompt should
Always: A password is always required each time the digital ID is used regardless of whether or not you are logged in to a file.
After: Choose a value from the drop-down list to set a time frame.
Once per session: A password is asked for only once while the application is open.
Never: The password is not usually required when using this ID and you are logged into the file.
6. Enter the password.
7. Choose OK.