20 Replies Latest reply: Mar 28, 2011 9:01 PM by Chris Cox RSS

    Invalid BMP Format with Alpha channel

    verpies Community Member

      This post is directed to Adobe technical personnel and requires the knowledge of internal bitmap format structures.

      CLAIM: Photoshop CS4 creates invalid file format when saving an image as  a Bitmap
      (BMP) with Alpha Channels, in the advanced format A8 R8 G8 B8.

       

      According to Microsoft documentation of the Bitmap (BMP) format, available at:

      http://msdn.microsoft.com/en-us/library/dd183376%28VS.85%29.aspx

       

      " If the biCompression member of the BITMAPINFOHEADER is BI_RGB, <...snip...> Each DWORD  in the bitmap array represents the relative intensities of blue, green,  and red, respectively, for a pixel. The high byte in each DWORD is not used..."

       

      Yet, Photoshop CS4 saves Bitmaps with Alpha Channels in the advanced format A8 R8 G8 B8 in violation of this documentation.

       

      Specifically, Photoshop CS4 writes:

      1) BITMAPINFOHEADER of the size 0x28 bytes which is equal to sizeof(BITMAPINFOHEADER)

      2) biCompression member of the BITMAPINFOHEADER is set to BI_RGB

      3) The high byte in each DWORD of the pixel array is USED and contains the Alpha Channel values

      4) There is no indication in the Bitmap headers of the presence of Alpha Channel data described in pt.3 above.

       

       

       

      Proposed Solution:

      1) Use BITMAPV4HEADER in lieu of BITMAPINFOHEADER because it contains a mandatory Alpha Channel bitmask.

          a) Alternatively use the proprietary Adobe header of the size 0x38 bytes, that Photoshop seems to be using in the wild.

      2) Indicate the presence of the Alpha Channel by setting the bV4AlphaMask member of the BITMAPV4HEADER to an appropriate non-zero bitmask.

          a) Optionally indicate the presence of a valid bV4AlphaMask by setting the bV4V4Compression member, to BI_ALPHABITFIELDS

          b) Optionally set the bV4RedMask and bV4GreenMask and bV4BlueMask members to appropriate bitmasks

              and indicate this fact by setting the bV4V4Compression member to BI_BITFIELDS or to BI_ALPHABITFIELDS

       

      NOTE: The Microsoft documentation, available at:

      http://msdn.microsoft.com/en-us/library/dd183380%28VS.85%29.aspx

      ...states that:

      "bV4AlphaMask - Color mask that specifies the alpha component of each pixel" irrespective of the value of bV4V4Compression member of the BITMAPV4HEADER.

      The same is not true about the bV4RedMask and bV4GreenMask and bV4BlueMask members of the BITMAPV4HEADER, because these members are valid only when bV4V4Compression member is set to BI_BITFIELDS (or to BI_ALPHABITFIELDS in the wild).

      Also, BITMAPV5HEADER can be used instead of BITMAPV4HEADER if advanced color profile needs to be embedded, but at the expense of increased file size.

        • 1. Re: BUG: Invalid Bitmap Format with Alpha channel
          Chris Cox Adobe Employee

          Have you talked to Microsoft about this?

          Microsoft requested the advanced formats, and verified the formats back when they were added (10 years ago, I've forgotten?).

           

          There is no "proprietary Adobe header" - everything we're writing is directly taken from the Microsoft documentation.

           

          Many other applications write BITMAPINFOHEADER, BI_RGB, and ARGB data.

          • 2. Re: BUG: Invalid Bitmap Format with Alpha channel
            verpies Community Member

            Yes, I did.  I sent them the bitmap files saved by Photoshop and they replied that these files are not compliant with the MSDN documentation.

             

            There are 5 different alternatives to  BITMAPINFOHEADER that exist in the wild.  See the attached image:

            BMP_Headers.gif

            BITMAPCOREHEADER, BITMAPINFOHEADER, BITMAPV4HEADER, BITMAPV5HEADER are all documented by Microsoft.

             

            However BITMAPADOBEHEADER and BITMAPADOBEALPHAHEADER are proprietary and undocumented by Microsoft, yet Adobe Photoshop creates them when saving images in certain advanced Bitmap (BMP) formats.

             

            For example when saving in these advanced bitmap formats, Photoshop creates proprietary headers:

            16bpp X1 R5 G6 B5

            16bpp A4 R4 G4 B4

            32bpp X4 R8 G8 B8

             

            I can send you these bitmap files or you can generate them yourself in Photoshop.

            Also, I can provide you with a MS-Winows utility that displays the header sizes of bitmaps and you will see that these files saved by Photoshop have a header sizes equal to 0x34 bytes or 0x38 bytes.  Ergo, PROPRIETARY and undocumented by Microsoft's MSDN.

             

            Are you convinced already that Photoshop uses proprietary bitmap headers?

             

            Also, the practice of "many other applications writing BITMAPINFOHEADER, BI_RGB, and ARGB data" does not make it legal and compliant with Microsft's documentation.

            BITMAPV4HEADER, BITMAPV5HEADER were especially invented to explicitly indicate the presence of the Alpha Channel in bitmaps and advanced color info.

             

            All they have to do, is use them correctly.

            • 3. Re: BUG: Invalid Bitmap Format with Alpha channel
              Chris Cox Adobe Employee

              Again, there is no Adobe header, and no proprietary header involved (unless you consider Microsoft to be proprietary) -- everything we're writing came straight out of Microsoft's documentation.   Though I notice that you skipped the V2 and V3 headers.

               

              No, I'm not convinced, because I wrote the code in question using Microsoft documentation, at their request.

              And then Microsoft verified the results of our BMP files (all variations).

               

              You might need to do a little more searching, I know that Microsoft has simplified some of their BMP/BITMAPINFOHEADER documentation over the years - but the full documentation should still be available.

               

              We are using the BMP format correctly, according to the experts in the graphics and games groups at Microsoft.

              • 5. Re: BUG: Invalid Bitmap Format with Alpha channel
                verpies Community Member

                Very well then, please let me know what is this header (of the size 0x38 bytes) in the attached file that I generated 5 minutes ago with Photoshop CS4 ?

                 

                http://1dollarsoftware.com/Photoshop32bpp_X8R8G8B8.bmp

                 

                This file has a standard

                BITMAPFILEHEADER that has 0x0e bytes and it is followed be a non-standard header that has 0x38 bytes (also its first DWORD member has the value 0x38).

                Normally the first member is biSize and BITMAPINFOHEADER.biSize == 0x28, but not here...

                 

                For your reference:

                sizeof(BITMAPCOREHEADER) == 0x0c

                sizeof(BITMAPINFOHEADER) == 0x28

                sizeof(BITMAPV4HEADER) == 0x6c

                sizeof(BITMAPV5HEADER) == 0x7c

                 

                So what is is this non-standard 0x38 byte header in the attached file ?

                 

                 

                P.S.

                I did not document the v2 and v3 headers because Microsoft does not document them in the MSDN and these headers do not appear in Microsoft's SDK. Also no other software generates them.  Google returns no hits about the v2 and v3 bitmap headers.

                • 6. Re: BUG: Invalid Bitmap Format with Alpha channel
                  verpies Community Member

                  You might want to check these files which I generated with Photoshop a while ago:

                   

                  http://1dollarsoftware.com/Photoshop16bpp_X1R5G6B5.bmp

                  http://1dollarsoftware.com/Photoshop16bpp_A4R4G4B4.bmp

                   

                  They also have the 0x38 bytes undocumented header....

                   

                  P.S.

                  What is YOUR name for the 0x34 and 0x38 byte headers?

                  • 7. Re: BUG: Invalid Bitmap Format with Alpha channel
                    Chris Cox Adobe Employee

                    From the Microsoft documentation (not sure if we changed the member names or not, but this is what is in my platform independent header):

                     

                    struct BITMAPV3INFOHEADER

                    {

                    uint32 biSize;

                    int32 biWidth;

                    int32 biHeight;

                    uint16 biPlanes;

                    uint16 biBitCount;

                    uint32 biCompression;

                    uint32 biSizeImage;

                    uint32 biXPelsPerMeter;

                    uint32 biYPelsPerMeter;

                    uint32 biClrUsed;

                    uint32 biClrImportant;

                    /* new for V3 */

                    uint32 biRedMask;

                    uint32 biGreenMask;

                    uint32 biBlueMask;

                    uint32 biAlphaMask;

                    };

                    And Photoshop only writes 2 values for the size  (well, unless you could OS2 format): sizeof(BITMAPINFOHEADER) and sizeof(BITMAPV3INFOHEADER).

                    • 8. Re: BUG: Invalid Bitmap Format with Alpha channel
                      verpies Community Member

                      Very well, so our discussion bears fruit

                      I was presumptious of me to state that the 0x38 sized header is an invention of Adobe, and my original post should be corrected as:

                       

                      WAS:

                      "a) Alternatively use the proprietary Adobe header of the size 0x38 bytes, that Photoshop seems to be using in the wild."

                       

                      SHOULD BE:

                      "a) Alternatively use the undocumented header of the size 0x38 bytes, that Photoshop seems to be using in the wild."

                       

                      What was named as BITMAPADOBEALPHAHEADER should have been named BITMAPV3INFOHEADER - their structure is the same.

                       

                      Nonetheless BITMAPV3INFOHEADER is still undocumented.  Google search yields NO HITS on it.

                      Also, I cannot find it in MSDN nor in officiel Microsoft SDK sources.

                      Also, no other software, that I know of, generates bitmap files with BITMAPV3INFOHEADER.

                       

                      Because of this undocumented header the following major software is incompatible with such bitmaps generated by Photoshop:

                      http://1dollarsoftware.com/Photoshop32bpp_X8R8G8B8.bmp

                      1) MS_Office - Does not display the image at all

                      2) Windows Picture and Fax Viewer v6.00.2900.5512 - Does not display the image at all

                      3) Firefox v3.6.12 (the most recent version) - Displays the image with corrupted colors

                      4) Opera v9.62 - Does not display the image at all
                      5) Chrome v7.0.517.44 - Does not display the image at all

                      6) Internet Explorer 8.0.6001.18702 and v6.0.2900.5512 - Displays corrupted image. Note the little black artifact in upper right corner of the image.

                       

                      For completeness, I am attaching an updated comparison between all the bitmap headers that I have encountered in the wild:BMP_Headers.png

                      • 9. Re: BUG: Invalid Bitmap Format with Alpha channel
                        verpies Community Member
                        FORMATEXAMPLEHEADERIEXPLOR
                        FIREFOXOPERACHROME Safari
                        16bpp A1 R5 G5 B5 SaveBITMAPV3INFOHEADERErrorOKErrorErrorError
                        16bpp A4 R4 G4 B4 SaveBITMAPV3INFOHEADERErrorOKErrorErrorError
                        16bpp      R5 G6 B5 SaveBITMAPV3INFOHEADERErrorOKErrorErrorOK
                        16bpp X1 R5 G5 B5 SaveBITMAPINFOHEADER
                        OKOKOKOK
                        16bpp X4 R4 G4 B4 SaveBITMAPV3INFOHEADERErrorOKErrorErrorError
                        24bpp      R8 G8 B8 SaveBITMAPINFOHEADEROKOKOKOKOK
                        32bpp A8 R8 G8 B8 SaveBITMAPINFOHEADER
                        OKOK
                        OK
                        32bpp X8 R8 G8 B8 SaveBITMAPV3INFOHEADERErrorErrorErrorErrorOK

                         

                        The empty fields mean that I did not have the time to test it yet.

                        <<safari test added by curious admin>>

                        • 10. Re: BUG: Invalid Bitmap Format with Alpha channel
                          Chris Cox Adobe Employee

                          The V3 header WAS documented on MSDN and in Microsoft header files included with MSVC.    Microsoft does have a history of changing their documentation and sometimes losing details in the process.  I would not be surprised if this were another example.

                           

                          We found the exact same header used in BMP files from Microsoft, and in BMP files used in commercial games (yeah, I scavanged a bunch to make sure we could read them all).

                           

                          Again, Microsoft asked for these formats and tested our results before we shipped the software.  It's not like we're working outside of Microsoft's realm in any of this.

                           

                          Yes, some programs may only implement a subset of BMP support (heck, we don't even implement ALL the possible permutations).

                          I'm not sure what you expect me to do about other applications not implementing parts of the BMP format that we do.

                          • 11. Re: BUG: Invalid Bitmap Format with Alpha channel
                            Chris Cox Adobe Employee

                            The advanced BMP formats are game/3D oriented -- I wouldn't expect the simple image parsers from web browsers to display them.

                            • 12. Re: BUG: Invalid Bitmap Format with Alpha channel
                              verpies Community Member

                              Dear Curious Admin,

                               

                              Thanks for adding the Safari column

                               

                              I cannot assign a bacground image to a table cell while posting on this forum

                              Could you allow the "background" keyword when inserting Raw HTML, so I can insert overlapping images in a table cell like this ? :

                              <table><tr><td background="checkerboard.bmp"><img src="foreground.bmp"/></td></tr></table>

                               

                              Now if I try to insert Raw HTML that contains this "background" keyword , your script simply cuts it out.

                               

                              This would help me illustrate to Chris more of the bugs with current implementation and demonstrate how proper formats behave.

                              • 13. Re: BUG: Invalid Bitmap Format with Alpha channel
                                Chris Cox Adobe Employee

                                Admins can't insert any more HTML than you can.  There are limitations on the HTML accepted in the forum for formatting and security reasons.

                                 

                                One more time: Photoshop's BMP implementation is working correctly as verified by Microsoft, and was written entirely using publicly available documentation from Microsoft.

                                 

                                So far you have yet to demonstrate even a single bug, but have shown how you can accuse other developers based on your own lack of research.

                                If you would like to ask questions and learn more, just ask.

                                • 14. Re: BUG: Invalid Bitmap Format with Alpha channel
                                  verpies Community Member

                                  Too bad the Admin cannot help me, because it limits my diagramming ability.

                                   

                                  The usage of undocumented bitmaps headers (yes they are undocumented today and I have never seen them documented in the past) is not the original point that I brought up. However this issue is also sigficant because it causes many incompatibilities with MAJOR applications.

                                   

                                  I illustrate this at the URL below:

                                  http://1dollarsoftware.com/bmp.html

                                   

                                  However my original point was not about undocumeted structures but that in a bitmap file Photoshop does not indicate the presence of an Alpha Channel in a consistent and in documented manner.

                                  Because of this whan users make a PowerPoint presentation, they loose the Alpha channel.  When programmers design icons or GUI elements in Photoshop, they also lose the Alpha channel.

                                   

                                  Not even Photoshop itself is certain when an Alpha channel is present in a bitmap file, and spuriously adds it, even when it was removed earlier.

                                   

                                  See the examples in the URL above, where I propose alternative encodings of the bitmap files using minimum documented header versions that are possible without loosing information.

                                   

                                   

                                  Regards,

                                  George

                                  • 15. Re: BUG: Invalid Bitmap Format with Alpha channel
                                    Chris Cox Adobe Employee

                                    Yes, Photoshop implements more of the BMP spec. than some other applications.

                                    The advanced format support was requested by Microsoft to help game developers who use those advanced BMP options.

                                     

                                    You are not documenting incompatibilities -- but that Photoshop implements more than the other applications.  That is a given.

                                    Photoshop also implements many file formats that those other applications cannot read at all, and variants of other file formats that those applications cannot read as well.

                                    If you wanted to file a bug about that, wouldn't you file a bug with PowerPoint because it cannot read all valid BMP files, even though it is a Microsoft product?

                                     

                                     

                                    Yes, there are some ambiguities when reading a bitfields format BMP.  Photoshop tries to implement the bitfields reading as generally as possible, and that can lead to some empty alpha channel values depending on the bitfields options.  Image wise, the extra channel does no harm.  For round tripping purposes, the extra channel does no harm.  So, what is your point?

                                     

                                     

                                    Also, you seem to have transparency and alpha channels confused.  Alpha channels are any extra channel, and the term generally applies to channels that are not transparency/opacity.  Transparency is a specific meaning for a specific subset of possible alpha channels.

                                     

                                    And you might want to check the BMP documentation - it supports an extra channel, but assigns no meaning to that channel.  It is not transparency, and using it as transparency will cause problems with interoperability -- some apps use the channel for things other than transparency, and some games use it for other masks.

                                     

                                     

                                     

                                    You still seem to be approaching this all wrong.

                                    Instead of screaming BUG all the time, you should be asking questions, learning about the formats, learning about imaging and interoperability.

                                    • 16. Re: BUG: Invalid Bitmap Format with Alpha channel
                                      verpies Community Member

                                      Photoshop is a flagship graphics editor and I expect only perfection from it and adherence to current documentation.

                                       

                                      The original intention of a feature is of no consequence.  It is a social argument - not technical.

                                      Many people in my company use Photoshop to create presentations - not games and it is up to the user what he/she uses the tool for.  The tool must work reliably, yet it does not.

                                       

                                      How can you imply that "Error opening" an image created by Photoshop by major software is not an incompatibility?

                                      I cannot file a bug with PowerPoint team because they are coding according to current Microsoft's documentation - Photoshop is not.

                                      The result of some software audit a decade ago is irrelevant to current situation, even if that audit was without a flaw, which I am not claiming that it was.

                                       

                                      Image wise, the extra channel does no harm.  For round tripping purposes, the extra channel does no harm.  So, what is your point?

                                      But it does!

                                      1) It harms performance.  Alpha blending is many times slower than normal blitting (sometimes requires a hard division by n^2-1), see:

                                      http://msdn.microsoft.com/en-us/library/dd183393%28VS.85%29.aspx

                                      ...and since Photoshop does not indicate the presence of an Alpha channel in bitmap headers it is hard to select the proper pixel operation routine.

                                      2) Also lack of reliable information in the headers prevents searching large collections of files for bitmaps containing an Alpha channel.

                                      3) Showing an Alpha channel in the Photoshop GUI after it has just been removed in previous operation, is confusing to the Photoshop user at best.

                                      3) Finally, showing an Alpha channel after opening a bitmap file in the format R5G6B5, that does not even have a space for an Alpha channel is plainly illogical.

                                       

                                      You are not documenting incompatibilities -- but that Photoshop implements more than the other applications.

                                      Yes, I am.  For example VisualStudio and PowerPoint can open bitmap files generated by other applications  (including translucency in the Alpha channel)  and of the same "non-openable" pixel format as generated by Photoshop.  The "Alternative" table in my last URL does not document files that have been generated by Photoshop, but by other software.

                                       

                                      You keep suggesting that I should learn about bitmaps and ask questions.

                                      Thus, what kind of questions do you suggest that I ask about the component structures of the bitmap files, such as: BITMAPFILEHEADER, BITEMAP*INFOHEADER, BITMAPV?HEADER, the BITFIELDs, Color Table, Pixel Array and Color Profile Data ?  Did I miss any?

                                       

                                       

                                      P.S.

                                      I did not use the word "transparent" or "transparency" even once.  Please do not split hairs on the contents of the Alpha channel...

                                      • 17. Re: BUG: Invalid Bitmap Format with Alpha channel
                                        Chris Cox Adobe Employee

                                        Photoshop is a flagship graphics editor and I expect only perfection from it and adherence to current documentation.

                                        We adhere to standards and documentation.

                                        If another company loses part of their documentation, that does not change the standard or the fact that we are implementing what WAS documented.  Nor does that change the fact that we are reading and writing files that can be found elsewhere in the world (again, check the BMP files in some games).

                                        If Microsoft contacts us and gives a really good explanation for why we should no longer support the file format options that they previously asked us to support, and are used in the wild, then maybe we would reconsider.

                                         

                                        Photoshop is working reliably.  The fact that other programs are not able to read everything that Photoshop writes is not always a problem with Photoshop - it can mean (as in this case) that the other programs do not implement the full file format specification.   If you want another program to read the file format variants that it cannot read, you should file a bug report or feature request with the makers of that application. Asking Photoshop to not write a valid file is just, well, silly.

                                         

                                        How can you imply that "Error opening" an image created by Photoshop by major software is not an incompatibility?

                                        It is an incompatibility.  The other application is not compatible with the full BMP specification.  Photoshop implements more of the specification than the other program.  Again, that is not a problem with Photoshop.

                                         

                                        I cannot file a bug with PowerPoint team because they are coding according to current Microsoft's documentation - Photoshop is not.

                                        There are BMP files written to the Microsoft spec. which Microsoft cannot read. That is a bug in PowerPoint.  Microsoft has lost part of their documentation on the BMP format, and that is another bug for the MSDN team.

                                         

                                        Alpha blending is many times slower than normal blitting (sometimes requires a hard division by n^2-1

                                        1) We're talking about alpha channels, not transparency blending.  Alpha channels have nothing to do with transparency blending.

                                        2) only a beginner would use a divide instruction when you can use repiprocal multiplication to get the same answer faster and without the serialization of a divide instruction (and some compilers will do that automatically, if the optimizer is turned on).

                                         

                                        User confusion is the only good argument there.  And it's funny how the users who use these files all the time, haven't complained....

                                        Yeah, if I have some time I might see about cleaning up that code and removing the spurious alpha channel when possible.  But it's a low priority, because, again, it isn't harming anything.

                                         

                                         

                                        I'm not splitting hairs -- you are confusing the term alpha channel with transparency.  You keep using the term alpha channel as if it WERE transparency, which hasn't been the case for 25 years.  (historically it was true after the original Porter-Duff paper, but then the term alpha channel was used to mean additional channels of any content and even in the mid 80s images with "alpha channels" had things other than transparency in them)

                                        • 18. Re: BUG: Invalid Bitmap Format with Alpha channel
                                          BMPFan007

                                          Greetings,

                                           

                                          I came across this thread while writing a BMP filter.  But I didn't come across this variant of bitmap via Photoshop; rather, I created one using GIMP 2.7.1 (development version), with its Advanced Options on the BMP export and choosing 32 bits -> X8 R8 G8 B8.  GIMP 2.6.11 also can generate such files.  This development release is dated June 29, 2010, so it predates this thread at least; I don't if GIMP or Photoshop added the format first.  But there are at least two programs which can generate such files.

                                           

                                          I was able to duplicate the image-viewing issues with Opera (11.01 Windows and 11.00 Linux), Windows Picture and Fax Viewer, Visual Studio 2010, and Powerpoint 2007, and the success with Safari for Windows.  Irfanview was also able to view the image; Konqueror 4.5.4 could not.  Interestingly, Microsoft Paint XP was the closest of any of the not-quites, but it was not quite correct.

                                           

                                          At any rate the genie's out of the bag by now.   It doesn't look to be difficult to implement, so most likely with time more software will support it.  For that matter it might be fairly easy to add support to it to Firefox, Konqueror, and other open-source projects.

                                           

                                          I do appreciate the documentation on Wikipedia, though.  I was surprised to find the 56-bit header, too, expecting the 52-bit header without the alpha channel, which incidently is mentioned as the Windows NT (3.1?) version of BMP version 3 here.

                                          • 19. Re: BUG: Invalid Bitmap Format with Alpha channel
                                            Noel Carboni Community Member

                                            verpies wrote:

                                             

                                            Because of this undocumented header the following major software is incompatible with such bitmaps generated by Photoshop:

                                            http://1dollarsoftware.com/Photoshop32bpp_X8R8G8B8.bmp

                                             

                                            6) Internet Explorer 8.0.6001.18702 and v6.0.2900.5512 - Displays corrupted image. Note the little black artifact in upper right corner of the image.

                                             

                                            FYI, Internet Explorer 9.0.8112.16421 displays it just fine, without any artifact I can find.

                                             

                                            IE9BMP.jpg

                                             

                                            I really don't understand what the problem is here, nor why it rates so much discussion...  It's clear Photoshop reads/writes a whole bunch of formats, not all of which are accepted by all other applications.

                                             

                                            If you need a format that works across the board, can you not just choose a format that works with all of them?

                                             

                                            -Noel

                                            • 20. Re: BUG: Invalid Bitmap Format with Alpha channel
                                              Chris Cox Adobe Employee

                                              I was surprised to find the 56-bit header, too, expecting the 52-bit header without the alpha channel, which incidently is mentioned as the Windows NT (3.1?) version of BMP version 3 here.

                                              Ah, good.  Someone else copied the MSDN documentation before Microsoft lost it.

                                              I found several similar documents on other web sites -- looks like they were copied verbatim from one source.

                                               

                                              Without the MSDN documentation, I suspect that many file readers are going to fail to support that variant of BMP.