15 Replies Latest reply on Jul 11, 2018 2:13 PM by dirckb

    Can't decrypt attachment with Crypt filter

    dirckb Level 1

      This question has been asked a couple of times previously and the answers are not helpful.

       

      A PDF encrypted with attachments only (attachments use the /Crypt filter, /StdCF) does not work the same as a PDF encrypted with all-data-encrypted.

       

      I have software that can read one but not the other.

       

      Conversely, I have two PDFs created (by me) with identical format except for this setting.

      The trailer ID is the same, the Crypt settings are the same (/EncryptMetadata false, /U, /O, /R /V, etc.,

      except one is set up with attachments only (/StmF /Identity /StrF /Identity, and /StdCF /AuthEvent /EFOpen)

      and one with everything (/StmF /StdCF /StrF /StdCF, and /StdCF /AuthEvent /DocOpen.)

       

      Let me stress: these files are *identical* except for these settings and the encrypted v non-encrypted data.

      All of the objects have the same object id's, etc.

       

      The contents of the encrypted attachment stream is *identical* byte for byte, between the two files,

      and Acrobat can extract the attachment from the fully encrypted file but generates a 0 byte file for the other.

       

      3rd party software: QPdf (5.1.3) cannot read the attachment from the Acrobat generated file, but *can* read it from the file I generated.

      Foxit reader (7.03.916) can read the one generated by Acrobat, but extracts garbage from the file I generated.

       

      I've been over the PDF reference 1.7 and the supplements and can't find any magic tricks.  Any help would be appreciated.

        • 1. Re: Can't decrypt attachment with Crypt filter
          lrosenth Adobe Employee

          Please post the files you mention below (one that works and one that does not).  Can’t really help w/o seeing real documents.

          • 2. Re: Can't decrypt attachment with Crypt filter
            dirckb Level 1

            Here is the fully encrypted file https://www.dropbox.com/s/asm1x7t7lh50b93/fullencrypt.pdf?dl=0

            Here is the crypt filter file I generated https://www.dropbox.com/s/zpo7tlj6uog76yb/cryptfilter.pdf?dl=0

            Here is an acrobat crypt filter file, just in case https://www.dropbox.com/s/99k6qrux99ladkd/acrobatX.pdf?dl=0

             

            All of them have user password "userpassword1234", owner password is blank.

            Thanks for taking a look at this, I appreciate it.

            • 3. Re: Can't decrypt attachment with Crypt filter
              dirckb Level 1

              FYI I posted a reply to this thread but I guess I replied to me and not to you, hopefully you got notified.

              Thanks again.

              • 4. Re: Can't decrypt attachment with Crypt filter
                dirckb Level 1

                Just wondering if you've had time to pull the files and reproduce the problem.

                • 5. Re: Can't decrypt attachment with Crypt filter
                  dirckb Level 1

                  After two years, my dropbox links have expired, I will re-up them:

                  Dropbox - fullencrypt.pdf

                  Dropbox - cryptfilter.pdf

                  https://www.dropbox.com/s/7k0boyf8cavncju/acrobatX.pdf?dl=0

                   

                  I will also link to two additional, considerably older versions of this question:'

                  2012 Re: File attachment only AES 128 encryption

                  2006 Re: Encrypt only file attachments

                   

                  PDF is supposed to be an ISO standard, but this behavior is undocumented, as is the behavior of encryption R 6.  Fortunately, a skilled third-party managed to reverse-engineer that in a way that actually works.  For this, I am at a loss.

                   

                  Any chance a software engineer might be able to look at this sometime this decade?

                  • 6. Re: Can't decrypt attachment with Crypt filter
                    lrosenth Adobe Employee

                    Have you looked at ISO 32000-2, the latest version of the PDF standard?  Are you still feeling as if this information is not clear?

                    • 7. Re: Can't decrypt attachment with Crypt filter
                      dirckb Level 1

                      Thank you for replying.

                       

                      I didn't realize that ISO-32000-2 had been published, I bought a copy and did some reading.  It looks like it documents /R 6, although I didn't step through the algorithm to see if it matches the (working) implementation.  It doesn't document anything unusual about "encrypt-attachments only".

                       

                      Today I did some additional testing for "encrypt-attachments only".

                       

                      The good:

                       

                      My software (CentraDoc) can read attachments from /R 5 and /R 6 files created by Acrobat X.

                      Acrobat X can read attachments from /R 5 and /R 6 files created by CentraDoc.

                       

                      The bad:

                       

                      CentraDoc cannot read attachments from /R 4 files created by Acrobat X.

                      Acrobat X cannot read attachments from /R 4 files create by CentraDoc.

                       

                      The ugly:

                       

                      As described previously, the /R 4 example files are *identical* except for the "attachments only" settings and the lack of encryption on most of cryptfilter.pdf.  Same file /ID (<00000000000000000000000000000000> <00000000000000000000000000000000>) same object id's (attachment stream is 16), same O and U keys, etc. etc.

                       

                      Here are the differences in the encryption dictionary:

                       

                      * fullencrypt.pdf ("encrypt everything except metadata"):

                       

                          /AuthEvent /DocOpen

                          /StmF /StdCF

                          /StrF /StdCF

                       

                      * cryptfilter.pdf ("encrypt-attachments only"):

                       

                          /AuthEvent /EFOpen

                          /EFF /StdCF

                          /StmF /Identity

                          /StrF /Identity

                       

                      cryptfilter.pdf includes the /Crypt filter on the relevant stream, and fullencrypt does not:

                       

                          16 0 obj

                          <<

                          /DecodeParms [ <<

                          /Name /StdCF

                          >> ]

                          /Filter [ /Crypt ]

                          /Length 17 0 R

                          >> stream

                          ...

                       

                      AcrobatX won't read the encrypted attachment from the "encrypt-attachments only" file, but it will from "encrypt everything except metadata".

                       

                      Once again: the bytes in the encrypted attachment streams are **identical** in the two files.  They both look like this:

                       

                          B7 37 E7 97 5B 18 62 EA

                          36 EC 6A 71 18 07 FA 35

                          B3 14 D0 9B 8E 6F 90 8C

                          3B A3 26 50 20 3E C8 2D

                          7C D0 7C C8 14 F2 4E C5

                          C4 25 99 6F D5 9D 97 38

                       

                      There's nothing in the documentation that would account for these two sets of bytes being different.

                       

                      This implies the calculated encryption key is somehow different for Acrobat in these two cases, since the algorithm (AES 128) should stay the same.

                       

                      What's am I missing?

                      • 8. Re: Can't decrypt attachment with Crypt filter
                        dirckb Level 1

                        Some minor clarifications:

                         

                        * The 16 byte random initializer for both files was set the same for testing

                        * There's nothing in the documentation that would explain why these these two sets of bytes *should* be different, when everything else is the same.

                        * R 5 and R 6 cases work if the stream bytes are the same for similar tests.

                        • 9. Re: Can't decrypt attachment with Crypt filter
                          lrosenth Adobe Employee

                          I’m having our security team take a look…

                          • 10. Re: Can't decrypt attachment with Crypt filter
                            dirckb Level 1

                            Thanks!  Please keep me posted.

                            • 11. Re: Can't decrypt attachment with Crypt filter
                              dirckb Level 1

                              Just a ping to see if there is any feedback....

                              • 12. Re: Can't decrypt attachment with Crypt filter
                                kingaling Level 1

                                Hello,
                                So I was reading the 32000-2 and saw something interesting that appears to conflict with the file you provided:
                                cryptfilter.pdf


                                From the spec:
                                If a security handler of revision 4 or 5 is specified, then the standard security handler supports crypt filters.
                                The support appears to be limited to the Identity crypt filter and crypt filters named StdCF whose dictionaries contain an AuthEvent value of DocOpen.

                                 

                                Your StdCF dictionary contains /AuthEvent /EFOpen

                                For Crypt filters, the limitation seems to imply that EFOpen is not supported.

                                 

                                If I have misread or missed something please let me know. It's possible.

                                • 13. Re: Can't decrypt attachment with Crypt filter
                                  dirckb Level 1

                                  Somehow I missed the notification of your response.  Just happened to be looking at the thread...  Anyway:

                                  /AuthEvent /EFOpen is how "encrypt attachments only" is specified.  (Perhaps the new spec is disallowing this case because it isn't documented and/or doesn't work as documented).  I just created a file with "encrypt attachments only" in Acrobat X Pro.  The /Encrypt dictionary looks like this:

                                  <<  /CF <<

                                              /StdCF <<

                                                  /AuthEvent /EFOpen

                                                  /CFM /AESV2

                                                  /Length 16

                                              >>

                                          >>

                                      /EFF /StdCF

                                      /EncryptMetadata false

                                      /Filter /Standard

                                      /Length 128

                                      /O(...)

                                      /P -1028

                                      /R 4

                                      /StmF /Identity

                                      /StrF /Identity

                                      /U(...)

                                      /V 4

                                  >>

                                  • 14. Re: Can't decrypt attachment with Crypt filter
                                    kingaling Level 1

                                    So I believe I have found the problem with the files you uploaded:

                                    cryptfilter.pdf

                                    fullencrypt.pdf

                                    password: userpassword1234

                                     

                                    I had the same issue you described. Extracting the embedded text file from cryptfilter.pdf did not error but it wrote a 0-length file to disk.

                                    I manually extracted and decrypted the file with no issue.

                                    After reading the spec ISO_32000-2_2017(en) "very carefully" I noticed something that's going to make me have to change my own code as well...

                                     

                                    In section 7.6.3.2.b

                                    "For all strings and streams without crypt filter..." etc.

                                    This section is the "usual" case. Most of the time, PDF objects are encrypted with the file level encryption dictionary.

                                    And under those circumstances the key you use will be modified by the object number, generation number and lastly, a 4-byte salt, increasing the size of the file key by 9 bytes before you hash it.

                                    That new key is used for decryption.

                                    But when you use a crypt filter in the objects definition like:

                                    16 0 obj

                                    <<

                                    /DecodeParms [ <<

                                    /Name /StdCF

                                    >> ]

                                    /Filter [ /Crypt ]

                                    etc...

                                    then section 7.6.3.2 no longer applies.

                                     

                                    To test this, I created a document with one embedded file and did what you did; "encrypt attachments only".

                                    I got the same structure you defined in your last post with the /EFOpen.

                                    Ignoring section 7.6.3.2 and using only the file key as the decryption key, I was able to get the original file.

                                     

                                    Long story short:

                                    Due to the above section of the spec, it is impossible to have the same encrypted data in cryptfilter.pdf:[16 0 obj] that you have in fullencrypt.pdf:[16 0 obj]

                                    2 different keys would have been used to encrypt the data.

                                    • 15. Re: Can't decrypt attachment with Crypt filter
                                      dirckb Level 1

                                      Thanks for your prompt reply.  Following your approach I have successfully decrypted an R 4 V 4 attachment created from Acrobat X with "encrypt attachments only".

                                       

                                      This info is in PDF 32000-1:2008 section 7.6.2, but not in the Adobe PDF reference sixth edition.  I didn't read the specs carefully enough.

                                       

                                      I appreciate your assistance, let me know where to send the beer.