23 Replies Latest reply on Jul 4, 2014 7:15 PM by jadams602

    XFL and source control

    patrick_cyr

      Since FLA is binary you can't have multiple people editing assets in the same FLA at once because binary files can't be merged in source control systems.  At first glance XFL seems like a great solution to this problem, but I've run into a number of issues using it in practice. Mostly it is a matter of XML files changing when the data they contain is essentially the same.

       

      - The files listed in DOMDocument.xml don't seem to be listed in any deterministic order, and thus two of them can contain exactly the same file list but appear to be very different to source control diff's.  When multiple people are making changes and trying to merge them it can be impossible for diff to tell the difference between changed lines and inserted/removed lines.

      - The XML files are generated somewhat differently on PC vs. Mac.  At the very least the direction of slashes (/ vs \) in paths is different.  It also seems that some numbers get saved with different precision.

      - More?

       

      Has anyone had any luck with this?  If not, is it Adobe's intention for XFL files to be usable that way?  It sure would be helpful to us.

       

      - Patrick

        • 1. Re: XFL and source control
          J.Auboux

          I have the same problem!

          Big part of our decision to upgrade early our Flash CS4 licenses to CS5 was the XFL format, and the promise to be able to work efficiently with our source control (subversion alas SVN)

           

          "By working with uncompressed XFL format, you can allow each part of the Flash file to be worked on separately by different people. You can also use a source control system to manage the changes made to each subfile within your uncompressed XFL file. Together, these capabilities allow for much easier collaboration on larger projects with multiple designers and developers."

           

          But what a huge disapointment when I realized that even an "empty" change like opening a symbol in the library and closing it resulted in resaving the whole library files, with lots of diff's everywhere.

          This makes automatic and manual merging virtuallyimpossibe, unless you work on a tiny xfl with 2 bitmap symbols, and defeats the purpose of switching from closed, binary format (FLA) to an open, text/xml based format (XFL).

          When working concurrently on a XFL document, Instead of having one conflict like on a FLA document, you get so many conflicts that manually merging is even worse (you don't have your myfla.fla.mine & myfla.fla.rXXXX, myfla.fla.RYYYY anymore, now you have your xfl folder full of conflicts

          I've found that ranodm differences could be categorized in a few cases:

          • xml elements order changed (like the embedded fonts element <DOMFontItem> in DOMDOcument.xml, wich order is swapped at each save!)
          • rounding errors : floating point values, is modified even if nothing was modified in the document, the floating point value which has like 15 significant numbers, totally differs starting from the third number for no apparent reason this appears in alpha attributes, and <Matrix> or <Point >elements
          • the <Edge> element is really annoying  because not only the order changes, but its "edges" content changes too ! this represents 90+% of the total unexpected changes when you have lots of vector shapes, drawn with pen tablet, necause those elements are quite big and you have lots of them
          • DOMSymbolItem.itemID changes randoomly too.
          • + Maybe other changes that I missed...

           

          Strangely the only predictable change - the "lastModified" attribute - only changes in a few files, many files are modified,, but keeping the same timestamp....and other files seem unchanged, except for the lastModified timestamp.

           

          I really hope Adobe is going to provide an update for Flash Professional CS5 soon, able to save XFL's predictably and without random modifications !!

           

          Jeremie

          • 2. Re: XFL and source control
            jordan2k2 Level 1

            We're having exactly the same issues as well.....It would be great to hear from someone, just so we are aware that they know of the problem and are looking into it.  Like everyone else, I was looking forward to XFL as a way to actually keep track of FLA changes, but it looks like it isn't quite there yet.  It's an excellent step forward though.

             

            Jordan

            • 3. Re: XFL and source control
              J.Auboux Level 1

              I have a pending support request since monday (5th)..hopefully I'll get a (positive ?) answer shortly

              I don't see how adobe could overlook this problem, unless fixing it implies very deep changes in the code ??

              I'll keep you updated about the answer.

              • 4. Re: XFL and source control
                patrick_cyr Level 1

                This is a nice blog post which clearly illustrates one of the problems.

                 

                http://www.dz015.com/?p=202

                • 5. Re: XFL and source control
                  J.Auboux Level 1

                  Thanks for the link, though this looks more like a bug (or strange behavior at least) in git.

                  For me the real problem is flash not saving in a deterministic way

                  • 6. Re: XFL and source control
                    patrick_cyr Level 1

                    I don't think that behavior is limited to git alone, but after thinking about it some more I'm not sure that git did the wrong thing there.  Essentially, the guy was trying to merge two shapes onto the same scene, but there is no way for a source control system to know which one to put first and object order is important.  It's essentially the equivalent of two people adding a new line of code at the same location in a file: the source control conflict cannot know which one should come first and so it becomes a conflict.

                     

                    How source control systems handle these things is important, though.  I guess I worry that there could be some situation where svn could confuse a change with a move and end up introducing an error to the data, but I admit that it's just a worry at this point and I have no evidence it would do so.

                    • 7. Re: XFL and source control
                      J.Auboux Level 1

                      I was just trying to say that SVN is more conservatiive, not trying to auto merge modifications on the same lines, even if they are (almost) identical

                      SVN would rather raise a conflict and let the user solve it manually. Sometimes makes you lose time where you shouldn't, sometime prevents some bad merge

                      By the way, still no response from adobe support, it's been a month that I submitted the question.

                      • 8. Re: XFL and source control
                        rahulbotics

                        Same issues here between Mac and PC.  I mostly have issues around DOMBitmapItem changing both order and path separator.  Unbelievable that they didn't think to normalize both the order and path separator.  I think I'm going to write a quick script to fix the separator before check in (the re-ordering seems less important).  I think that would address the main issues with non-concurrent distributed work (ie. only one owner at a time, but the owner changes).

                        • 9. Re: XFL and source control
                          vvirgill

                          Hey guys,

                           

                          Sorry for the late answer but we have been looking at these issues and are currently looking into possible solutions. As someone mentioned earlier there are a few different classes of differences between XML files across saves. Also sometimes when you apparently did not change a specific movie clip in the library but still note differences in its XML file, it can be because Flash maintains dependencies and links between library items, so even a small change to one item can ripple and cause other items that reference it to be rewritten (which is why the date/time stamp changes).

                           

                          But the 2 main classes of differences we are looking into are the order of which items are written in the XML, so that we can achieve a consistent ordered list, and changes in shape data / rounding issues.

                           

                          Feel free to ping me directly if you have any further questions.

                           

                          thanks,

                          Valerio.

                          • 10. Re: XFL and source control
                            J.Auboux Level 1

                            Better late than never

                             

                            I'm really glad you have taken our feedback into account.

                            Fixing the order of the XML to make it deterministic and the floating point WEIRD roundings will be a great step forward for the usability of XFL format in source control software and should reduce the size of our "diff' by a factor 10 at least, making them much more meaningful...

                            I understand the problem of dependencies beetween library symbols, though in many of cases I can't understand how a change of one symbol implies resaving the whole library for no apparent reason. This happens on our production documents, it's not that easy to demonstrate with a basic sample, but if you have problem reproducing this issues, I can definitely spend some time trying to make a sample. Any way it seems to be a step in the right direction already, because changing timestamps is far from being my biggest complain.

                            I juste hope the 11.0.2 update that has just been announced and is still not downloadable already has this fixes !

                            • 11. Re: XFL and source control
                              jordan2k2 Level 1

                              Hi,

                               

                              Like everyone else, I would like to thank your for looking into these issues.  I'm not in any position to make any estimates regarding the size or scope of this issue, but I was wondering if you had any timeline as to when we can expect a resolution....even a progress update would be most appreciated.

                               

                              Thanks in advance,

                               

                              Jordan

                              • 12. Re: XFL and source control
                                ceineke

                                It would be great to get an update on this from you guys at Adobe.

                                • 13. Re: XFL and source control
                                  ceineke Level 1

                                  Please?

                                  • 14. Re: XFL and source control
                                    ceineke Level 1

                                    It has almost been 6 months since this discussion was started. This is a pretty serious problem. We originally bought licenses for Flash CS5 because we were looking forward to using the non-binary format. Now with the non-binary working not being saved consistently, that's pretty much NIL feature for us. I'd be great if I could get somebody from Adobe to report about the status of this defect.

                                    • 15. Re: XFL and source control
                                      Fardeen Ghulam

                                      Hi,

                                       

                                      we have the exact same issue with versioning an XFL.

                                       

                                      Is there any tool better than the Flash IDE to handle this issue ? Any JSFL ? ANT Task ?

                                       

                                      The XFL format was supposed to let several people work on a same file...

                                      • 16. Re: XFL and source control
                                        ceineke Level 1

                                        We went back to binary fla files and manual locking of the project file. It was too big task to merge about 190 files each time somebody only saved the project...

                                        • 17. Re: XFL and source control
                                          Fardeen Ghulam Level 1

                                          That's what i want to avoid. Goind back to the crappy FLA format...

                                           

                                          I was a pain to move from FLA to XFL (dozen of font issues as always with Flash IDE) and now the only solution would be to move back to FLA ?

                                           

                                          Adobe ? Any other solution ?

                                          • 18. Re: XFL and source control
                                            J.Auboux Level 1

                                            Hi all,

                                            I've almost sorted out the issues with uncompressed XFL format and source control

                                            the problem:

                                            Although adobe has fixed some random differences when saving in latest flash professional cs5 update (11.0.2), you still experience huge diffs when doing team work on xfl's

                                            Actually, the biggest issue is that XFL format uses a cache to prevent from resaving the whole XFL directory, but checks the validity of the cache based on the modification date of the files

                                            It means that updating or checkouting an xfl from a repository will get you "wron"g modification dates, resulting in Flash professional resaving all xml's even with the slightest modification, generating a huge (and useless) diff, which will propagate to other users when you commit your local modifications.

                                             

                                            the workaround for TortoiseSVN (tested on TortoiseSVN 1.6.10, Build 19898 - 64 Bit):

                                            -go into settings panel and check the 'Set file dates to the "last commit time"' checkbox

                                            -recheckout your local workcopies, (or if it really bothers you, juste make sure you have no local modification on xfl directories, delete and re-update them)

                                            -now all your xfl files have the actual revision date as modification date.

                                            little test

                                            -perform a modification on an XFL

                                            -notice the much faster saving process, and check your local modifications with tortoise : hurrah, flash only saved the relevant files (almost).

                                            -if each team member performs these steps, you are now ready to work collaboratively much more easily. your svn logs on xfls will be reduced by a huge factor if you do small modifications at each commit, resulting readable logs, much less conflicts that you'll be able to handle manually most of the time

                                             

                                            !!!WARNING!!!

                                            flash will still generate "random" modification on the resaved xmls, but it's less annoying than modifying all files

                                             

                                            Since this setting is a global svn setting, it can't be activated just on XFL files, so be aware that it may have side effects: http://tortoisesvn.net/docs/release/TortoiseSVN_en/tsvn-dug-settings.html

                                             

                                             

                                            Set file dates to the “last commit time

                                            This option tells TortoiseSVN to set the file dates to the last commit time when doing a checkout or an update. Otherwise TortoiseSVN will use the current date. If you are developing software it is generally best to use the current date because build systems normally look at the date stamps to decide which files need compiling. If you use “last commit time” and revert to an older file revision, your project may not compile as you expect it to.

                                             

                                             

                                             

                                             

                                            Also, this is a bit experimental and hasn't been tested extensively.

                                            To be safe,you should clean all temporary objects after any revert, or update to a previous revision, to force recompilation event though the modification date is older

                                            (with mxmlc, don't use incremental compilation, or delete the temps files, with flash, "delete ASO files")

                                             

                                            -If you really need finer control, you will probably need to use svn properties to store the modification date and hook scripts to the commit/checkout/update commands

                                            -I just googled the svn command line equivalent here: http://stackoverflow.com/questions/2171939/how-can-i-keep-original-file-timestamp-on-subve rsion

                                             

                                            You can set it in .subversion/config:

                                            [miscellany]
                                            use-commit-times = yes

                                             

                                            Feel free to contribute to this thread if you find a better solution/improvment to this method , or to simply report success or failure in applying this method

                                            For those who use git, I don't know how it handles modification dates, but maybe there's a similar solution.

                                             

                                            Jeremie

                                            • 19. Re: XFL and source control
                                              da4seen

                                              Has this been resolved in any updates from adobe or are there still extensive issues lingering?

                                               

                                              Thank you in advance!

                                              • 20. Re: XFL and source control
                                                J.Auboux Level 1

                                                Working with xfl format under svn is still very painful.

                                                svn doesn't keep track of the commited files modification dates (they are not even stored on the repository, only the commit dates are stored)

                                                This fools flash cs5 into thinking the files are not up to date (since their dates don't match those stored in symdepend.cache) and modifies and resaves them all.

                                                Basically, if you work alone and always in the same local workcopy, everything works fine, but if you work in team environment, working on different workcopies, branches , etc...this gets really messy (LOTS of conflicts)

                                                Eventually, I added a svn:needs-lock property on all our xf (and fla) files, to force developers to lock any xfl before editing. It worked pretty well to avoid conccurent modifications, as a result we don't have conflicts anymore, but flash cs5 still makes lots of unecessary modification to the xfls. for xfls, we only make basic use of svn (update / log / commit, no merge, no resolve conflicts...)

                                                • 21. Re: XFL and source control
                                                  Ziplinski

                                                  Has anything happened on this since February? Is XFL any better with CS5.5?

                                                   

                                                  Thanks

                                                  • 22. Re: XFL and source control
                                                    jadams602 Level 3

                                                    Even in the latest Flash CC 2014 there is still unnecessary font and persistedData reordering in in the XML of the DOMDocument.xml that places unnecessary changes in version control checkins and merging between different users.

                                                     

                                                    I see this conversation is over 3 years old but is there any known bugbase report on this that we can vote up? I would have expected this to have been fixed by now after all this time.

                                                    • 23. Re: XFL and source control
                                                      jadams602 Level 3

                                                      In case this happens to be useful to others in their workflow to try and minimize the non-deterministic reordering that happens with some unordered lists in DOMDocument.xml every time you save a .xfl, I have created an XSL transform that will canonicalize an input DOMDocument.xml into an output version that sorts the <fonts> and <persistedData> sections based on their name attributes, producing a consistent deterministic order of child elements:

                                                       

                                                      https://dl.dropboxusercontent.com/u/83837/CanonicalizeDOMDocument.xsl

                                                       

                                                      So if your team/or yourself wanted to hit the DOMDocument.xml with this transform before each commit to your version control, you should be able to minimize the unnecessary changes that happen in the <fonts> and <persistedData> sections.

                                                      If anyone knows of any other non-deterministic sections of the DOMDocument.xml that could use a fix like this, let me know and I can update the transform (or you can update the .xsl above and share as it is very short and straightforward to extend)

                                                       

                                                      You can use any flavor of XSL/XSLT tool you want with the above .xsl.

                                                      If you are on OS X there is a builtin 'xsltproc' commandline tool you can use in Terminal on the command line with a call like:

                                                      xsltproc -o DOMDocument-new.xml CanonicalizeDOMDocument.xsl  DOMDocument.xml

                                                       

                                                      If you use a text editor that lets you setup custom Text Filters like BBEdit, you can set up a .sh filter with the contents of the following two lines:

                                                      #!/bin/sh

                                                      xsltproc CanonicalizeDOMDocument.xsl -