8 Replies Latest reply on Apr 28, 2009 8:06 PM by -==cfSearching==-

    Using javax.mail.internet.MimeMessage etc. to create EML file

    EDUYork Level 1

      Hi,

       

      I have email messages stored in a MySQL database and their attachments stored as base64-encoded text.  I'm trying to loop through a query and write out a properly formatted .EML file for each message.  I started from some borrowed code that was used to send IMAP messages, and I had no prior experience with Java, but I've since worked through some tutorials and I now know how to write and compile a simple class. But I'm still having trouble (lots!) instantiating the correct Java objects in the correct way and order in ColdFusion.

       

      Here's what sort of works, with problems I'll detail below:

       

      <cfscript>
          clsSession = createObject("Java", "javax.mail.Session");
          objProperties = createObject("Java", "java.util.Properties");


          msg = CreateObject("Java", "javax.mail.internet.MimeMessage");
          mmp = CreateObject("Java", "javax.mail.internet.MimeMultipart");
          mbp = CreateObject("Java", "javax.mail.internet.MimeBodyPart");
          objRecipientType = CreateObject("Java", "javax.mail.Message$RecipientType");


          fds = CreateObject("Java", "javax.activation.FileDataSource");


          fos = CreateObject("Java", "java.io.FileOutputStream");
          oos = CreateObject("Java", "java.io.ObjectOutputStream");

      </cfscript>

       

      <cfloop query="getAllMessages">

      <!--- Here I also do a query "qGetMsgAtts" to get all attachments by getAllMessages.msg_id ,

            and use valueList(qGetMsgAtts.attIndex) to create "msgAttachmentsList" --->

       

      <!--- initialize a java MimeMessage object --->

      <cfscript>
          objProperties.init();

      //  (I was actually setting properties here, but I never retrieved them and code works w/o them)

       

          objSession = clssession.getInstance(objProperties);


      // Message
          msg.init(objSession);
          msg.addFrom(add.parse(getAllMessages.msgFrom, false));
          msg.setReplyTo(add.parse(getAllMessages.msgFrom, false));
          msg.addRecipients(objRecipientType.TO, add.parse(getAllMessages.msgTo, false));

      // ETC... no problems setting msg properties
          loopCount = loopCount + 1;


      // file name
          thisFileName = '#userId#_#myFun.doCountToken(loopCount)#.eml'; // my doCountToken(integer) function simply pads with zeros.


      // Message body parts
          mbp.init();
          if (not msgIsHTML)

          {
              mbp.setText(msgBody);
          }
          else
          {
              mbp.setContent(msgBody, "text/html");
          }
          mmp.addBodyPart(mbp);
          msg.setContent(mmp);


          if (len (msgAttachmentsList)) // message has attachments
          try // TRY TO ATTACH FILES
          {
            { // qGetMsgAtts.colnames: msg_Id, attIndex, fileName, fileMIMEType, fileMacCreator, fileMacType, fileContents
                writeOutput('Has attachments ' & msgAttachmentsList & '<br />');
                for (i=1; i is listLen(msgAttachmentsList); i++)
                {

                    fds.init(qGetMsgAtts.fileContents); // WORKS

       

                    mbp.init();

                    mbp.setDataHandler(dh.init(fds)); // WORKS

                    mbp.setFileName(qGetMsgAtts.fileName[i]); // WORKS

                    mbp.setContent(qGetMsgAtts.fileContents[i],qGetMsgAtts.fileMIMEType[i]); // WORKS

                    mmp.addBodyPart(mbp);          

                }
                msg.setContent(mmp);
            }
          }
          catch(Any excpt) {
                  writeOutput("#excpt.Type# error attaching content. #excpt.Message# #excpt.Detail#<br>");
          }


          aFile = thisFilePath & thisFileName;  // PATH AND FILE NAME
          fos.init(aFile); //
          oos.init(fos); //


      try// WRITE THE FILE TO DISK

      {
          msg.writeTo(oos); // WORKS ONLY IF MIME TYPE IS text/plain or text/html
          oos.flush(); 
          oos.close();
          fos.close();
      }
      catch (Any excpt) {
          writeOutput("#excpt.Type# error attaching content. #excpt.Message# #excpt.Detail#<br>");
          oos.flush();
          oos.close();
      }

      </cfscript>

      </cfloop>

       

      Although that writes the correct number of files with the correct file names, there are 2 major problems at this point:

       

      1. the contents of the file on disk appears to contain all the messages. I'm doing msg.init() atop every loop so I don't understand why. (Also the part boundary appears to be the same in every part in every message.)
      2. I get an error like the following for any MIME type other than text/plain or text/html :
        javax.activation.UnsupportedDataTypeException error attaching content. no object DCH for MIME type application/pdf no object DCH for MIME type application/pdf

       

      I Googled that and I understand that I probably need to use something like
          bads = createObject("java","javax.mail.util.ByteArrayDataSource");
          baos = createObject("Java", "java.io.ByteArrayOutputStream");

      ...somewhere in the mix, but all my attempts to instantiate such objects have so far failed, usually with "Method Not Found" or "Constructor not found".

       

      I hope that explanation makes sense. Anyone ever write EML files using ColdFusion? I'll be forever grateful for any suggestions or snippets.

       

      Richard

      York U CA

        • 1. Re: Using javax.mail.internet.MimeMessage etc. to create EML file
          EDUYork Level 1

          EDUYork wrote:

          <snip>

          there are 2 major problems at this point:

           

          1. the contents of the file on disk appears to contain all the messages. I'm doing msg.init() atop every loop so I don't understand why. (Also the part boundary appears to be the same in every part in every message.)
          2. I get an error like the following for any MIME type other than text/plain or text/html :
            javax.activation.UnsupportedDataTypeException error attaching content. no object DCH for MIME type application/pdf no object DCH for MIME type application/pdf

           

          I Googled that and I understand that I probably need to use something like
              bads = createObject("java","javax.mail.util.ByteArrayDataSource");
              baos = createObject("Java", "java.io.ByteArrayOutputStream");

          ...somewhere in the mix, but all my attempts to instantiate such objects have so far failed, usually with "Method Not Found" or "Constructor not found".

           

          I hope that explanation makes sense. Anyone ever write EML files using ColdFusion? I'll be forever grateful for any suggestions or snippets.

           

          Richard

          York U CA

           

          I've answered #1 for myself: I just needed to (proof-read more closely and) re-initialize the MimeMultipart object as well...

           

           

          // Message body parts
              mmp.init();
              mbp.init();
              if (not msgIsHTML){ // etc., etc.

           

          I'm still seeking the way to create the part object as ByteArrayOutputStream (or similar) so it will properly detect and encode the many other mime types amongst my attachments.

           

          Cheers!

          • 2. Re: Using javax.mail.internet.MimeMessage etc. to create EML file
            -==cfSearching==- Level 4

            I have never tried to reconstruct a message, so I don't know what other information you might need, like headers and such. But have you tried just setting the content?

             

            Though I am wondering if there is a simpler option  (like saving the original file), as this does seems like the long route. ...

            1 person found this helpful
            • 3. Re: Using javax.mail.internet.MimeMessage etc. to create EML file
              EDUYork Level 1

              -==cfSearching==- wrote:

               

              I have never tried to reconstruct a message, so I don't know what other information you might need, like headers and such. But have you tried just setting the content?

               

              Though I am wondering if there is a simpler option  (like saving the original file), as this does seems like the long route. ...

               

              Thanks for the link. I'm using setContent() successfully for text/plain and text/html content types, but it's throwing a DataContentHandler error "no object DCH for MIME type" for anything else.  I've Googled that and learned java.io.ByteArrayOutputStream and/or javax.mail.util.ByteArrayDataSource have the ability to do that correctly for other MIME types but I can't seem to figure out how to instantiate or initialize any working combination.

               

              What works up to a point is (msgBody comes from my query)


                  mmp.init(); // javax.mail.internet.MimeMultipart
                  mbp.init(); // javax.mail.internet.MimeBodyPart
                  if (not msgIsHTML){
                      mbp.setText(msgBody);
                  }
                  else
                  {
                      mbp.setContent(msgBody, "text/html");
                  }
                  mmp.addBodyPart(mbp);
                  msg.setContent(mmp);

               

              // part not working is here, where I loop through attachments list. //

               

              // file creation works
                  fos.init(aFile); // a string, the absolute path and file name: ../userId/eml/abc_xyz.eml
                  oos.init(fos); // java.io.ObjectOutputStream
                  msg.writeTo(oos); // writeTo method outputs a valid EML file

               

              The resulting EML file even opens in Outlook and/or gets appended properly to an IMAP folder on a live server, and the body content is what it's supposed to be. But attachments other than plain text or HTML appear and have correct file name, but they are 0k empty files.

              • 4. Re: Using javax.mail.internet.MimeMessage etc. to create EML file
                -==cfSearching==- Level 4

                If you are going to use a ByteArrayDataSource, just call its constructor by passing in the binary data and mime type:

                 

                        source = ByteArrayDataSource.init( binaryData, "image/gif");
                        handler = DataHandler.init(source);

                 

                 

                But the code is also using the java objects incorrectly.  You should capture the object returned from init() each time.  When you know you will be reusing an object, it is best to use two separate variables:  one used only to perform init() and the other to store the instance returned from init().  For example, instead of doing this within the loop

                 

                   mbp .init(); // javax.mail.internet.MimeBodyPart

                 

                Create a MimeBodyPart object outside the loop. [ Do not call init().]

                 

                    MimeBodyPart = createObject("java", "javax.mail.internet.MimeBodyPart");

                 

                Then use MimeBodyPart whenver you need to create a new instance. But store the results in a separate variable, an use the second variable in the rest of the code.

                 

                 

                    for (......)

                        {

                       // create a new instance of MimeBodyPart

                              newBodyPart = MimeBodyPart.init();

                              newBodyPart.setFileName("something.doc");

                             ...

                        }

                1 person found this helpful
                • 5. Re: Using javax.mail.internet.MimeMessage etc. to create EML file
                  EDUYork Level 1

                  This was very helpful, it looks as if I'm almost there --- thanks again! For the record I was using proper variable setting in some places, and most of the places I wasn't were in fact copied from someone else's application. Anyway among other things I changed the database I'm storing the attachments in to longblob fields rather than binaryEncode(file_contents,'Base64') into a longtext field.

                   

                  The code that follows now appears to do everything I wanted, except that when I open a resulting EML message 1) there's some corruption at the beginning of the headers and at the end of the body, and 2) the attachments don't actually open... Acrobat says they're "damaged" and probably not decoded properly, and similarly Word displays a dialog by which one may change the encoding, but any choice displays only gibberish. The header corruption at the beginning of the first line has the result that it makes the sent date unreadable, and at the end of the body it's usually a w- or a z followed by one or more unicode squares.

                   

                  So here's what now seems to get me very close to the solution (with extraneous stuff replaced by comments):

                   

                  Before anything...

                  <cfscript>

                      objProperties = createObject("Java", "java.util.Properties");

                   

                      MailSession = createObject("Java", "javax.mail.Session");

                      msg = CreateObject("Java", "javax.mail.internet.MimeMessage");
                      mmp = CreateObject("Java", "javax.mail.internet.MimeMultipart");
                      mbp = CreateObject("Java", "javax.mail.internet.MimeBodyPart");

                      objRecipientType = CreateObject("Java", "javax.mail.Message$RecipientType");


                      dh = CreateObject("Java", "javax.activation.DataHandler");


                      oos = CreateObject("Java", "java.io.ObjectOutputStream");
                      fos = CreateObject("Java", "java.io.FileOutputStream");

                      baos = CreateObject("Java", "java.io.ByteArrayOutputStream");

                  </cfscript>

                  <!--- Query the database to get all messages, then loop through the query: --->

                  <cfloop query="qGetAllMsgs">
                  <!--- initialize message field variables etc. --->
                  <!--- Query the database to get all attachment belonging to this message --->

                  <cfscript>

                      loopCount = loopCount + 1;
                      newObjProperties = objProperties.init();

                      objSession = newMailSession.getInstance(newObjProperties);

                      newMessage = msg.init(objSession);

                  // set message fields
                      newMessage.addFrom(add.parse(
                  qGetAllMsgs.msgFrom, false));
                      newMessage.setReplyTo(add.parse(
                  qGetAllMsgs.msgFrom, false));
                      newMessage.addRecipients(objRecipientType.TO, add.parse(
                  qGetAllMsgs.msgTo, false));
                      newMessage.addRecipients(objRecipientType.CC, add.parse(
                  qGetAllMsgs.msgCC, false));
                      newMessage.addRecipients(objRecipientType.BCC, add.parse(
                  qGetAllMsgs.msgBcc, false));
                      newMessage.setSubject(
                  qGetAllMsgs.msgSubject);
                      newMessage.setSentDate(
                  qGetAllMsgs.msgSentDate);

                      thisFileName = '#qGetAllMsgs.userId#_#myFun.doCountToken(loopCount)#.eml'; // pads loopCount with 0s


                  // Message body parts
                      newMimeMultipart = mmp.init();
                      newBodyPart = mbp.init();
                      if (not msgIsHTML){
                          newBodyPart.setText(msgBody);
                      }
                      else
                      {
                          newBodyPart.setContent(msgBody, "text/html");
                      }
                      newMimeMultipart.addBodyPart(newBodyPart);
                      newMessage.setContent(newMimeMultipart);
                      if (len (msgAttachmentsList)) // message has attachments
                      try // TRY TO ATTACH FILES
                      {
                        {
                           
                            for (i=1; i lte listLen(msgAttachmentsList); i++)
                            {   

                                newBodyPart = mbp.init();
                                source = bads.init(qGetMsgAtts.fileContents[i],qGetMsgAtts.fileMIMEType[i]);
                                handler = dh.init(source);
                                newBodyPart.setFileName(javaCast('string',qGetMsgAtts.fileName[i]));
                                newBodyPart.setContent(qGetMsgAtts.fileContents[i],qGetMsgAtts.fileMIMEType[i]);
                                newBodyPart.setDataHandler(handler);
                                newMimeMultipart.addBodyPart(newBodyPart);

                            }
                            newMessage.setContent(newMimeMultipart);
                        }
                      }
                      catch(Any excpt) {
                              writeOutput("#excpt.Type# error attaching content (#qGetMsgAtts.fileName[i]#). #excpt.Message# #excpt.Detail#<br>");
                      }

                     
                      // write eml file to disk
                      emlFile = emlFilePath & thisFileName;
                      myFileOutputStream = fos.init(
                  emlFile ); //
                      myObjectOutputStream = oos.init(myFileOutputStream); //
                  try

                         {
                      newMessage.writeTo(myObjectOutputStream); // WORKS: 'myObjectOutputStream'
                      myFileOutputStream.close(); //
                      myObjectOutputStream.flush(); //
                      myObjectOutputStream.close();
                      }
                  catch (Any excpt)

                         {
                      writeOutput("#excpt.Type# error attaching content. #excpt.Message# #excpt.Detail#<br>");
                      myFileOutputStream.close(); //
                      myObjectOutputStream.flush(); //
                      myObjectOutputStream.close();
                      }

                  </cfscript>

                  </cfloop>

                   

                  The result looks like an email message with the correct body and attachments, but all the files have 9 unicode characters at the top (I think they're triplets that refer to 3 single characters, and I can only paste a few of them in this text area: ¬í) and the attachments will not open in their helper applications.  I'm currently trying to get them on a mail server to see if the server's decoding changes anything.

                   

                  Thanks again for the help that got me this far!

                   

                  Richard

                  York U.

                  • 6. Re: Using javax.mail.internet.MimeMessage etc. to create EML file
                    -==cfSearching==- Level 4

                    EDUYork wrote:

                     

                    The code that follows now appears to do everything I wanted, except that when I open a resulting EML message 1) there's some corruption at the beginning of the headers and at the end of the body, and 2) the attachments don't actually open...


                           ...

                        myFileOutputStream = fos.init(emlFile ); //
                        myObjectOutputStream = oos.init(myFileOutputStream); //

                     

                    Hi Richard,

                     

                    It looks okay except for the part that writes it to a file.  You only need one output stream here, so just use FileOutputStream. I think ObjectOutputStream is used more for serialization, and is probably what is causing the strange output here.

                     

                    try
                    {
                        emlFile = emlFilePath & thisFileName;
                        outStream = CreateObject("Java", "java.io.FileOutputStream").init( emlFile );
                        newMessage.writeTo( outStream );
                        outStream.flush();
                        outStream.close();
                    }
                    catch (Any excpt)
                    {
                        writeOutput("#excpt.Type# error attaching content. #excpt.Message# #excpt.Detail#<br>");

                        // only close the stream if it exists
                        if ( IsDefined("outStream") )
                        {
                            outStream.close();
                        }
                    }

                     

                     

                    BTW, the best reference for javax.email is sun's: http://java.sun.com/developer/onlineTraining/JavaMail/index.html   .So, did you ever find out if there is an easier way to do this?  I am wondering if you could store the whole message (headers, etc..) without the attachements?

                    • 7. Re: Using javax.mail.internet.MimeMessage etc. to create EML file
                      EDUYork Level 1

                      Each of your suggestions has brought me one step closer, and I'm deeply in your debt!

                       

                      When I copy/pasted your most recent try/catch block I got an error like the one below... the first time it said line 255, but when I moved that above the try it just moved down one line and threw the exact same error.

                       

                      \u00a0\u00a0\u00a0, on line 256, column 1, is not a valid identifer name.


                      The CFML compiler was processing:


                          * A script statement beginning with \u00a0\u00a0\u00a0 on line 256, column 1.
                           * A script statement beginning with try on line 253, column 1.
                           * A cfscript tag beginning on line 115, column 4.
                           * A cfscript tag beginning on line 115, column 4.



                      The error occurred in E:\ColdFusion8\wwwroot\FCToIMAP2\write_eml.cfm: line 256


                      254 : {
                      255 : //    emlFile = emlFilePath & thisFileName;
                      256 :     outStream = CreateObject("Java", "java.io.FileOutputStream").init( emlFile );
                      257 :     newMessage.writeTo( outStream );
                      258 :     outStream.flush();

                       

                      But I had tried something along these lines before... I uncommented some earlier lines and removed the ObjectOutputStream as you suggested. I now have


                          emlFile = thisFilePath & thisFileName;
                           myFileOutputStream = fos.init(emlFile);


                      try
                      {
                           newMessage.writeTo(myFileOutputStream);
                           myFileOutputStream.flush();
                           myFileOutputStream.close();
                      }
                      catch (Any excpt)
                      {
                         writeOutput("#excpt.Type# error attaching content. #excpt.Message# #excpt.Detail#<br>");
                         if (isDefined('myFileOutputStream'))
                         {
                           myFileOutputStream.close();
                         }
                      }

                       

                      The bad characters are gone... the attachments open (all but one so far, which most likely really is damaged)... the dates are correct.  I now have some additional headers to write but I'm on track to finish my application, which is a migration tool. I don't think I have many options for storing the messages in another way as they come from a (superb, but according to my management unworthy) proprietary collaboration platform that due to its server/client architecture can't be fully migrated by more conventional means (e.g., IMAP... it fails to recognize most subfolders and can't capture Sent messages at all).  I can generate a script that contains all the details of every message and has all the attachments as attach1-attachN. I save the script as a text file, download the attachments to a folder and place that in a zip file. Using ColdFusion I unzip the files, parse the script for the details, and populate a MySQL database with everything including the attachments.  I got that far on my own. I found a CFX tag CFX_IMAP4, that can place it on our next system if it's in RFC-822 format.

                       

                      And now I have that!  Thanks to your invaluable input I also intend to continue reading the Java documentation and see if I can't use the imap.cfc that I borrowed the beginnings of all this from.

                       

                      Best regards,

                       

                      Richard

                      York U.

                      • 8. Re: Using javax.mail.internet.MimeMessage etc. to create EML file
                        -==cfSearching==- Level 4

                        Ah, okay.  It sounds like you have other issues to work with.

                         

                        EDUYork wrote:

                         

                        When I copy/pasted your most recent try/catch block I got an error like the one below... the first time it said line 255, but when I moved that above the try it just moved down one line and threw the exact same error.

                         

                        \u00a0\u00a0\u00a0, on line 256, column 1, is not a valid identifer name.


                         

                        Well.. it was not intended as copy and paste code ;-) Though I just tried and it works fine for me (no typos .. amazingly).  So I am not sure what the problem was. Anyway, I am glad you got it all working.