10 Replies Latest reply on Aug 4, 2010 8:40 PM by deshartman

    Async Integration Testng

    deshartman

      I have looked through the forums and also search Google, but I cannot seem to find a solution that fits.

       

      I am doing integration testing using FlexUnit 4 with a Swiz framework for events. I ran my code and it works as expected, but I want to ensure it keeps working by setting up tests. I have set up unit tests to confirm each part of the program and stubbed out as required. This all works fine. Next part I am working on  (and struggling with) is the integration or end-2-end tests.

       

      My thinking is I start at a point in the program that then calls other parts via Events (Swiz dispatchEvents) and these in turn may call other events. It is very much in sequence and I want to confirm this sequence and the variables set at each point. So I did the following

       

      ========================

      [Test(async)]

      public function setInitialViewTest():void

      {

           Async.handleEvent(this, presenter.eventDispatcher, "event1", checkVars1, 500, null, noEventFired);

           Async.handleEvent(this, controller.eventDispatcher, "event2", checkVars2, 500, null, noEventFired);

           Async.handleEvent(this, presenter.eventDispatcher, "event3", checkVars3, 500, null, noEventFired);

           presenter.setInitialView();

      }

       

      public function checkVars1(someEvent:MyEvent, data:Object):void

      {

           // Check some variables

      }

       

      public function checkVars2(someEvent:MyEvent, data:Object):void

      {

           // Check some variables

      }

       

      public function checkVars3(someEvent:MyEvent, data:Object):void

      {

           // Check some variables

      }

       

      ======================

       

      Problem I run into is that it works fine for two events, but when I add a third or a fourth in the mix I keep getting Error:Asynchronous Event of of order. I then move event3 between event 1 and event2 and it works, but adding event4 check kills it again. The code being check is really simple and fires the events in sequence and the presenter and controller listen and then fire the next event.

       

      Question is:

       

      Can I use FlexUnit 4 for this kind of sequence related async testing? Am I missing something here?

       

      Thanks

      Des

        • 1. Re: Async Integration Testng
          mlabriola Level 4

          Des,

           

          You can certainly use FlexUnit 4 for this kind of testing.

           

          The error you are getting suggests the events are either not coming in the order you expect, or, possibly, one of these events is getting called more than once. Without using FlexUnit for the async stuff, have you just tried doing the same thing and adding some trace statements to the handlers to see when they get called?

           

          We do this exact type of testing with Adobe components and have thousands of tests running on it, so I know that part works.

           

          Mike

          • 2. Re: Async Integration Testng
            deshartman Level 1

            Michael

             

            Bugger, not quite what I wanted to hear. ;-)

             

            Here is what I have tried:

             

            1) I put trace statements in the code at all the points where I am about to dispatch the event and right after it is fired. When I check the output, the events happen in the order I predict.

             

            2) If I have two handleEvents, it works fine and I have tried is for 1,2 & 2,3 and even for event 1,3 at a time and it works.

             

            3) If I add a third Async test, whether proceedOnEvent or handleEvent, I get the error.

             

            I'll build a small test harness and post it on gitHub. Maybe you could help me out in the mean time. Is there a way I can trace what event is actually out of order in FlexUnit? It would be more useful if it reported something like "expected Event AAAA next instead of BBBB"

             

             

            Thanks

            Des

            • 3. Re: Async Integration Testng
              deshartman Level 1

              ok, as promised I built an example setup. I posted it on GitHub

               

              instruction to set up Git:  http://progit.org/book/ch1-4.html

               

              Then check out the project: git clone https://deshartman@github.com/deshartman/FlexUnit4AsyncTest.git//deshartman@github.com/deshartman/FlexUnit4AsyncTest.git

               

              This should create a complete Flex Project that you can test with.

               

              -------------------

               

              I ran the tests and as expected it fails with out of order.

               

              Here is the trace flow:

               

              Dispatching Event 1 from Presenter 1

              Event 1 caught in Controller 1

              Dispatching event 2 from Controller 1

              Event 2 caught in Controller 2

              Dispatching event 3 from Controller 2

              Event 3 caught in Presenter 2

               

              and the Test checks

               

              Async.proceedOnEvent(this, presenter1.eventDispatcher, CustomEvent.EVENT1, 500, noEvent);

              Async.proceedOnEvent(this, controller1.eventDispatcher, CustomEvent.EVENT2, 500, noEvent);

              Async.proceedOnEvent(this, controller2.eventDispatcher, CustomEvent.EVENT3, 500, noEvent);

               

               

              So based on this, it should pass, but it fails. I can get it to pass if I reverse the order

               

              Async.proceedOnEvent(this, controller2.eventDispatcher, CustomEvent.EVENT3, 500, noEvent);

              Async.proceedOnEvent(this, controller1.eventDispatcher, CustomEvent.EVENT2, 500, noEvent);

              Async.proceedOnEvent(this, presenter1.eventDispatcher, CustomEvent.EVENT1, 500, noEvent);

              but this is not what I am expecting.
              Will be glad to sort this out
              Thanks
              Des

              • 4. Re: Async Integration Testng
                mlabriola Level 4

                great. should be able to review in a few hours

                 

                appreciate the effort,

                mike

                • 5. Re: Async Integration Testng
                  mlabriola Level 4

                  Des,

                   

                  If I take your example and add this code:

                   

                  [Test(async)]

                  public function test1():void

                  {

                    Async.proceedOnEvent(this, presenter1.eventDispatcher, CustomEvent.EVENT1, 500, noEvent);

                    Async.proceedOnEvent(this, controller1.eventDispatcher, CustomEvent.EVENT2, 500, noEvent);

                    Async.proceedOnEvent(this, controller2.eventDispatcher, CustomEvent.EVENT3, 500, noEvent);

                   

                   

                    presenter1.eventDispatcher.addEventListener( CustomEvent.EVENT1, eventTest );

                    controller1.eventDispatcher.addEventListener( CustomEvent.EVENT2, eventTest );

                    controller2.eventDispatcher.addEventListener( CustomEvent.EVENT3, eventTest );

                   

                    presenter1.sendEvent1();

                  }

                   

                   

                  public function eventTest( event:CustomEvent ):void {

                    trace( (event as Event).type );

                  }

                   

                  Meaning I added my own listeners and traced them out. Here is what I get in the trace console:

                   

                  Event3

                  Event2

                  Event1

                   

                  Which is out of order from the 1,2,3 you are setting up in your test

                   

                  This is what I meant by tracing them out. Confirm this on your side.

                   

                  Mike

                  • 6. Re: Async Integration Testng
                    deshartman Level 1

                    Mike

                     

                    ok, I see what you are doing, but what I do not understand is why these are out of order. I am firing the events in order, so they should be picked up in order. Why would they be in reverse order? I am of course assuming that the same amount of time is taken to dispatch each event. Even if it were slightly different I would expect then to arrive close to order, i.e. 1, 2, 3 and worst case if 2 take longer, 1,3,2

                     

                    Is this an anomaly or normla behaviour? Am I missing something here?

                     

                    Thanks

                    Des

                    • 7. Re: Async Integration Testng
                      mlabriola Level 4

                      I am not a Swiz expert, but here is my guess. It is perfectly normal.

                       

                      Flash Player is single threaded.

                       

                      At the time when your test runs, your event listeners look something like this:

                       

                      Event 1:

                      1) Swiz

                      2) FlexUnit Async Code

                       

                      Event 2:

                      1) Swiz

                      2) FlexUnit Async Code

                      Event 3:

                      1) FlexUnit Async Code

                       

                      So, when Event 1 fires, Swiz catches it and does what it needs to... which causes Event 2 to be dispatched, The first listener on Event 2 is Swiz. Swiz catches it and does its business, which results in Event 3 being dispatched.

                       

                      Your FlexUnit listener for Event 3 fires. The stack pops

                      Now the next item in the listener list for Event 2 occurs, which is the FlexUnit code. The stack pops.

                      Now the next item in the listener list for Event 1 occurs, which is the FlexUnit code.

                       

                      Get me?

                      Mike

                      • 8. Re: Async Integration Testng
                        mlabriola Level 4

                        Another Visual of the call stack

                         

                        Dispatch Event 1

                             Swiz for Event 1

                                  Dispatch Event 2

                                       Swiz for Event 2

                                            Dispatch Event 3

                                                 FlexUnit Async for Event 3

                                       FlexUnit Async for Event 2

                             FlexUnit Async for Event 1

                         

                        HTH,

                        Mike

                        • 9. Re: Async Integration Testng
                          deshartman Level 1

                          ok, I see where you are going with this.

                           

                          So the issue here is that the events are piling up on the stack and FlexUnit cannot get to run it's checks until Swiz hands back control?

                           

                          If Flex were multi-threaded, I presume the events would be in sequence then?

                           

                          Thanks for your help. Now that I understand what is happening, I can work around it.

                           

                          PS. Is this how FlexUnit is used for sequence testing without Swiz?, i.e. normal event dispatching or do you do the events in order when doing normal Flex based event dispatching. The reason I ask, is maybe the Swiz guys can do something about this if Flex events can handle this in sequence?

                           

                           

                          Thanks

                          Des

                          • 10. Re: Async Integration Testng
                            mlabriola Level 4

                            This is how FlexUnit is used for normal sequencing. However, the normal use cases for sequencing are UIComponents and server calls, both of which tend to break across frames so the stack resolves itself.

                             

                            In 4.2 we will be introducing some new async methods which will allow you to ignore the order, but that feels hacky here too.

                             

                            Mike