17 Replies Latest reply on Sep 5, 2011 3:30 AM by Peter Spier

    Why does this DeviceGray PDF get lost by InDesign?

    John Hawkinson Level 5

      We turned up a fairly scary bug this week, that only got scarier as I started to track it down.

      I have a PDF file which, when placed, does not appear in InDesign's PDF export.

      This seems to be a pretty serious problem, so I'm pursuing it with Adobe, but I'm curious if, informally, anyone has any thoughts.

      It doesn't seem to be related to the export settings, though I certainly see it with PDF/X:4. This problem is present in CS4 and up through CS5.5, though does not seem to be present in CS3.

       

      The PDF was generated (I believe) with a Fujitsu ScanSnap S1500M under MacOS 10.6.8.

      The PDF is a little bit intriguing since it seems to have:

      . A DeviceGray ICC profile

      . An Indexed colorspace

      . The imaged stored as an XObject

       

      It runs through Acrobat X's Newspaper Ads preflight just fine.

       

      Anyone have any thoughts?

       

      There is a link to the PDF: r2.pdf

      (well, that's a cropped version that exhibits all the same symptoms).

       

      Interestingly, in Acro X, choosing Edit Image..., launches Photoshop which then says "Could not complete your request because this is not an Adobe Photoshop PDF image file."

       

      Here's what Acrobat's Browser Internal Structure has to say:

      r2struc1.png
      r2struc2.png

        • 1. Re: Why does this DeviceGray PDF get lost by InDesign?
          Peter Spier Most Valuable Participant (Moderator)

          I don't expect any of us are going to be able to tell you much. It shows up here in an exported PDF in Acrobat 9's preflight reports, but is invisible on the page. Curiously, when I double click to select in preflight it highlights a huge box that extends outside the page area on a couple of sides.

           

          I saved the original as .eps and palced that next to a re-distilled version from the eps, then exported again. The .eps is also invisible, but the re-distilled version shows some random vertical black bars. I don't know if any of that helps you.

          • 2. Re: Why does this DeviceGray PDF get lost by InDesign?
            Gernot Hoffmann Level 3

            It`s not really a Grayscale. It's an image with only two levels, black and white,

            like a Photoshop Bitmap. A Generic Gamma=2.2 ICC profile is embedded, but

            this is entirely meaningless for only two levels - black and white.

            DeviceGray is the alternative (it means 'use the actual color space of the output

            device without any color management'). ICC profile DeviceGray doesn't exist.

            It's an Indexed Color image with only two possible indices (0,1). This explains as

            well BitsPerComponent=1 (just on/off).There is no need to save such an image

            by Indexed Color. This mode uses normally 4 to 256 input entries in a  table with

            4 to 256 outputs triples R,G,B. In a Grayscale one wouldn't use Indexed Color,

            but numbers and a tone reproduction curve (TRC), for instance the mentioned

            profile.

             

            In my humble opinion it's not really surprising, that InDesign cannot handle this

            strange construct.

             

            Best regards -- Gernot Hoffmann

            1 person found this helpful
            • 3. Re: Why does this DeviceGray PDF get lost by InDesign?
              John Hawkinson Level 5

              Peter Spier:

              I don't expect any of us are going to be able to tell you much. It shows up here in an exported PDF in Acrobat 9's preflight reports, but is invisible on the page. Curiously, when I double click to select in preflight it highlights a huge box that extends outside the page area on a couple of sides.

              This is probably a result of the crop -- the original image is a letter-sized PDF.

              I saved the original as .eps and palced that next to a re-distilled version from the eps, then exported again. The .eps is also invisible, but the re-distilled version shows some random vertical black bars. I don't know if any of that helps you.

              Err...huh. I'm not sure why you would have thought to run it through the Acrobat distiller twice, but it is interesting with the bars. I assume you mean this:

              bars.png

              That is certainly...odd.

               

              Gernot Hoffmann:

              Thanks for clarifying that DeviceGray is the alternative; of course I meant to say that it has a Generic Gray profile...

              There is no need to save such an image by Indexed Color. This mode uses normally 4 to 256 input entries in a  table with

              4 to 256 outputs triples R,G,B. In a Grayscale one wouldn't use Indexed Color,

              but numbers and a tone reproduction curve (TRC), for instance the mentioned

              profile.

              Yeah, I agree, it doesn't make a lot of sense. Still, it's what I've got.

               

              In my humble opinion it's not really surprising, that InDesign cannot handle this

              strange construct.

              Well:

               

              #1: Why do you think so? Because it is a 1bpp image that is encoded as Indexed Color? Because it has a DeviceGray alternative? Because it has a gray profile?

              #2: If InDesign previews it on the screen, it should include it in the output. Anything else is really bad. I guess, maybe, it's OK to throw up an error (ideally as soon as possible, like on import, or in preflight) instead of exporting, but it should not export blankness. Much better for the export to fail.

              #3: Since it works in CS3 and fails in CS5, it's sure looks like a PDF Library regression.

              #4: I don't know how closely the PDF Library team works with the Acrobat Pro team (Dov, can you comment?), but I would expect that if there are weird PDFs that it doesn't support, then the preflights that ship with Acrobat Pro ought to detect them. Of course the problem might be a lot more subtle than that...

               

              p.s.: To me, given our recent discussion of EPS files, it is certainly ironic that the EPS file produced by Acrobat is much more difficult to read than the raw PDF file. Mostly because it contains thousands of lines of Adobe AGM prolog/setup/resources, but since I'm not really familar with them, it does make it awfully tough to tell what is going on. So far the best tool for interpretting this has been "pdftk r2.pdf output r2u.pdf uncompress" to generate human-readable PDF. I guess it's on par with the Acrobat BIS tool above, though it's a little easier to deal with since Acrobat can be slow interactively.

              • 4. Re: Why does this DeviceGray PDF get lost by InDesign?
                Peter Spier Most Valuable Participant (Moderator)

                John,

                 

                I'm not sure you understood what I meant about redistilled. Sometimes when PDFs are flakwy I find saving as eps, then running the eps through distiller, will give me a PDF that behaves a bit better. In my very brief testing I did that, then placed both the eps and the PDF I distilled from it on a page in ID.

                • 5. Re: Why does this DeviceGray PDF get lost by InDesign?
                  John Hawkinson Level 5

                  Yeah, I definitely confused myself there and what I said didn't make

                  much sense, sorry about that. Yes, running it through Distiller makes

                  plenty of sense as a way to remove some flakiness. Not sure how I

                  ended up with the twice -- I guess it was two Save As > EPS choices

                  that I was really getting confused about.  But saving as EPS (or

                  similar) is basically a requirement for feeding into Acrobat

                  Distiller, so that all makes sense too.

                   

                  Anyhow, not quite sure where this gets me. I mean, I'm not really seeking

                  a workaround here -- the right workaround appears to be opening it in

                  Photoshop and rasterizing it, since it's a raster PDF anyhow.

                  • 6. Re: Why does this DeviceGray PDF get lost by InDesign?
                    Stix Hart Level 5

                    John Hawkinson wrote:

                     

                    I mean, I'm not really seeking a workaround here --

                     

                    And there you get the difference between your approach and a lot of others!   Me, I'd just export it to a tiff from Acrobat and keep on working...  I'm not saying your approach is wrong, just that for some of us spending time on stuff like this is unaffordable.

                    • 7. Re: Why does this DeviceGray PDF get lost by InDesign?
                      John Hawkinson Level 5

                      I think maybe I need to stop speaking in public... When I said I wasn't looking for a workaround, it's because I already have one! As I told you, I can just rasterize it in Photoshop. (You're right, Save As > Image > TIFF in Acrobat is a better choice, since it saves one from wondering what resolution to use in Photoshop; for some reason I don't tend to trust Acrobat...).

                       

                      And there you get the difference between your approach and a lot of others!   Me, I'd just export it to a tiff from Acrobat and keep on working...  I'm not saying your approach is wrong, just that for some of us spending time on stuff like this is unaffordable.

                       

                      I don't spend time on every item like this, but I consider it part of my job to nail down these bugs and get them fixed so they don't bite someone else. Also, I do find them fascinating.

                       

                      The other problem in our workflow is that it's far too easy to fail to notice when this condition ocurrs, I had sort of meant to ask about this -- we print our proofs from ID, then we export PDFs that we send on to our printer's prepress department. How many people in this situation instead print proofs from exported PDFs? If we did that, we'd be more likely to catch these problems. It feels like it would be less convenient, but maybe that's naive. And a little automation here or there... Plus, we would gain background printing capability, which could be awesome.

                       

                      (In other news, Save As > TIFF on the latest of these files gives me a 2546x3321 image in Photoshop. Running pdfimages -j on it and then identify -verbose on the output gives me a 2547x3322 image. Inspection of the original PDF suggests that pdfimages is right and Acrobat/Photoshop are wrong:

                      << /Width 2547
                         /BitsPerComponent 8
                         /Name /X
                         /Height 3322
                         /Filter[/DCTDecode]
                         /Subtype
                         /Image
                         /Length 281788
                         /ColorSpace
                         /DeviceCMYK
                         /Type
                         /XObject
                      >> stream
                      

                      and that's mildly depressing as well. Fencepost errors! Hmm, this is DeviceCMYK, maybe it doesn't actually have the problem. )

                      • 8. Re: Why does this DeviceGray PDF get lost by InDesign?
                        Stix Hart Level 5

                        John Hawkinson wrote:

                         

                        I think maybe I need to stop speaking in public...

                         

                         

                        I wasn't meaning that at all!

                         

                        We never print proofs from InDesign, we always make a PDF using the final settings we we are going to create the proof from and use that, for a lot of reasons including I don't really trust InDesign's printing...  Also it just seems more fool proof to put it through that export stage sooner rather than later, over 80% of our proofs are emailed anyway, which is another reason for being consistent with a print proof, pdf proof and final pdf.

                        • 9. Re: Why does this DeviceGray PDF get lost by InDesign?
                          John Hawkinson Level 5

                          It turns out this is a bug in InDesign's downsampling of Indexed Gray images.

                          If one turns off downsampling of Color Images then the output PDF is just fine.

                          (I guess that makes it two bugs, since the Color Images section should not affect a Grayscale image.)

                          • 10. Re: Why does this DeviceGray PDF get lost by InDesign?
                            [Jongware] Most Valuable Participant

                            Perhaps ID takes a shortcut here and assumes "indexed color == RGB color".

                             

                            (There isn't such a thing as "indexed gray image", is there?)

                            • 11. Re: Why does this DeviceGray PDF get lost by InDesign?
                              Gernot Hoffmann Level 3

                              It's not even a Grayscale, because it's coded by 1 bit per pixel.

                               

                              Best regards --Gernot Hoffmann

                              • 12. Re: Why does this DeviceGray PDF get lost by InDesign?
                                John Hawkinson Level 5

                                Sure there is. It's just not common, because it's usually not worth the

                                effort. But if you have a grayscale image with only 4 possible values for

                                each pixel, then using 2-bit indexed color is an effective and efficient

                                representation. But as a practical matter, this is extremely rare.

                                 

                                (One could also imagine a 16-bit grayspace [colorspace?] where only

                                a smaller subset was used that would fit in 8 bits, which could then

                                be represented as indexed gray. This seems the sort of thing you

                                might get if you started with a 10-bit log format...)

                                • 13. Re: Why does this DeviceGray PDF get lost by InDesign?
                                  Peter Spier Most Valuable Participant (Moderator)

                                  John Hawkinson wrote:

                                  ...we print our proofs from ID, then we export PDFs that we send on to our printer's prepress department. How many people in this situation instead print proofs from exported PDFs?

                                  Always. And I expect my printer to send me proofs back made from RIPed data. You need to be looking at what the next step will be. If you send PDF, you proof the PDF you are sending. Period.

                                   

                                  I don't know about your systems, but here PDFs tend to spool and print much faster than native ID files, too.

                                  • 14. Re: Why does this DeviceGray PDF get lost by InDesign?
                                    Eugene Tyson Adobe Community Professional & MVP

                                    i don't see the point in printing a print ready pdf on our office printer. The issues that can arrise at RIP stage can be different to what I get here.

                                     

                                    I print directly from InDesign and proof the pages. that's what I expect in my print ready pdf. I analyse the print ready PDF onscreen and compare to the print outs here. Then after I've preflighted and checked the PDF the best I can here I send to the printers. Who then send me the proofs that have gone through their RIP.

                                     

                                    I can then compare what was RIPed in the PDF to what I have printed from InDesign.

                                     

                                    There's very rarely a problem.

                                     

                                    I just don't see the point of printing a print ready pdf here myself as I can't replicate the situations that happen at RIP.

                                    • 15. Re: Why does this DeviceGray PDF get lost by InDesign?
                                      Peter Spier Most Valuable Participant (Moderator)

                                      No, you can't replicate waht happens in the RIP, but that's why you get a proof back from the RIP data. In the meantime, printing the PDF helps catch the sort of problem that John encountered BEFORE it gets out the door.

                                       

                                      It's not wise to presume, especially in a newspaper situation where you get content from all over made with who knows what software and techniques, that what you see in ID is what you will get in the PDF.  We see reports of objects shifting, type misbehaving, and so forth all the time. Haven't seen it lately, but one of my other favorite problems is PDFs made in Quark where the default is to have OPI enabled and all the images drop out when re-exported from ID.

                                      • 16. Re: Why does this DeviceGray PDF get lost by InDesign?
                                        Eugene Tyson Adobe Community Professional & MVP

                                        That's very true. As why I proof my PDF on screen before sending.

                                         

                                        I dont' get an awful lot of supplied material, I usually generate content myself.

                                         

                                        But you make a good point. I should really print from the PDF.

                                        • 17. Re: Why does this DeviceGray PDF get lost by InDesign?
                                          Peter Spier Most Valuable Participant (Moderator)

                                          I find I catch a lot more of the stuff like typos on a printed proof rather than on screen.