I create a simple file called 256encrypt.pdf and encrypted with aes256
I am using "Algorithm 2.B: Computing a hash" from ISO32000-2 to verify the user password
user password: password
User string from the PDF test file : f4 65 f1 69 9a e2 ea 71 ba e7 6b 48 bb 12 8f 1f 18 74 e3 d3 e2 97 7e b8 d6 fe 9f 7f 86 b0 6d 89 c9 38 40 c5 64 dc 5a 32 04 4d 9c 6f 28 d2 98 d0
User string hash value: f4 65 f1 69 9a e2 ea 71 ba e7 6b 48 bb 12 8f 1f 18 74 e3 d3 e2 97 7e b8 d6 fe 9f 7f 86 b0 6d 89
User Validatiaon salt: c9 38 40 c5 64 dc 5a 32
User Key salt:04 4d 9c 6f 28 d2 98 d0
The input for the "Algorithm 2.B: Computing a hash" is as follows:
user Validation Salt: c9 38 40 c5 64 dc 5a 32
password: 70 61 73 73 77 6f 72 64 (password)
step 1: SHA256(password+user Validation Salt)
the result is K = 9d 47 2d 4e f0 96 cd dd 7a 8c 04 8d b4 d2 b8 ee be fe b9 9f 7f cc e1 29 ea 63 ad f2 a3 d5 11 5b
step 2: Make a new string K1 with 64 repetitions of the input password and K
K1= 70 61 73 73 77 6f 72 64 9d 47 2d 4e f0 96 cd dd 7a 8c 04 8d b4 d2 b8 ee be fe b9 9f 7f cc e1 29 ea 63 ad f2 a3 d5 11 5b
70 61 73 73 77 6f 72 64 9d 47 2d 4e f0 96 cd dd 7a 8c 04 8d b4 d2 b8 ee be fe b9 9f 7f cc e1 29 ea 63 ad f2 a3 d5 11 5b
70 61 73 73 77 6f 72 64 9d 47 2d 4e f0 96 cd dd 7a 8c 04 8d b4 d2 b8 ee be fe b9 9f 7f cc e1 29 ea 63 ad f2 a3 d5 11 5b
.......
70 61 73 73 77 6f 72 64 9d 47 2d 4e f0 96 cd dd 7a 8c 04 8d b4 d2 b8 ee be fe b9 9f 7f cc e1 29 ea 63 ad f2 a3 d5 11 5b
Total 64 times , total length = 0xa00
step3: Encrypt K1 with AES_128(CBC)
AES_CBC_128_NOPADDING:
Key = 9d 47 2d 4e f0 96 cd dd 7a 8c 04 8d b4 d2 b8 ee
IV = be fe b9 9f 7f cc e1 29 ea 63 ad f2 a3 d5 11 5b
K1 64 repeat of the 70 61 73 73 77 6f 72 64 9d 47 2d 4e f0 96 cd dd 7a 8c 04 8d b4 d2 b8 ee be fe b9 9f 7f cc e1 29 ea 63 ad f2 a3 d5 11 5b
Result : Total length is 0xa00. The beginning part of the E is
E =
47 df 2a 7f 90 8a c4 d9 f2 8b a0 f1 49 f0 8e 09 51 c4 a3 ce fd 28 48 f3 d7 c1 04 76 1b 6b 5b f2 6d 3d 2c 3f 03 26 76 06 d5 67 44 c8 2a b6
10 02 a5 8d a7 93 4f 94 02 b9 bf 93 b5 2d 17 82 02 3b f7 8e 8a 07 0f 18 ed 19 b3 ba 55 8b 14 b7 45 16 80 47 4f 6e c3 b6 20 d2 72 cd d1 46
2c d3 88 f7 c4 f7 e3 3a 04 3d 72 4f e0 d2 66 63 c4 9c 77 7c c5 53 fd 69 81 f6 3b 3d f5 8e b2 bd 66 4e 0f c6 1e 96 5e 91 e2 3d 60 5c 60 75
a3 13 49 58 85 e8 bb 37 93 91 4c 4f 79 a5 80 f2 13 be 44 22 aa e5 ee 6c 29 2c 76 50 a3 15 85 69 5e e9 c5 29 13 2a f6 67 51 8e 1e 7f 23 8a
90 a7 fe 93 c7 ff 45 ee 2a f0 c0 70 f1 78 2e 80 bd be 06 4f ad 69 4d 47 e6 3f ae e2 6a 76 ef 3e 56 8f 2d f5 c9 49 26 f3 7e 6e 61 8b 5e e6
e6 2d dd 76 cd 30 33 1d fe bf 11 60 ce 33 35 43 da b7 33 9b b9 6a 86 cd 35 a0 ca 84 99 0c ca 71 28 b3 01 b9 23 b4 a0 87 4e fb ff af b6 bd
.......
step4:
The result of the first 16 bytes of E mod 3 is 1
step5:
Using SHA384 to get 48 bytes K
K = 29 de 28 c1 f0 17 c9 37 bd 93 97 e3 b5 51 b0 86 b9 0c 96 e0 77 28 87 1c 11 7b 41 ce 64 bf a8 7f f2 8b a2 7b 52 58 79 a9 63 c0 b2 31 f8 4e e4 6e
This is the end of round 1 and go back to step 2 using this new K
When round is equal or bigger than 64, check E[last byte], if E[last byte] > round -32, go back to step 2
The final round is 69. and the final result is
K = ab 7c c6 03 bc da 85 51 3f 3d 22 fb 58 8c 42 1d 45 67 55 92 9f 4f d2 41 b3 93 07 04 7d b1 30 6d
But this K does not match with the first 32 byte of the user string.
