FlexUnit 4 test cases running sequence problem

[Rupam_K]

Hi All,

I am facing some problem in running my test cases in sequence in FlexUnit 4. Suppose i am adding the test case class in my main MXML.

public function runMyTest():void
            {

                 flexUnitCore = new FlexUnitCore();

                 flexUnitCore.addListener( new UIListener( testRunner ));

              //My test suite class name is SvcTest

                 flexUnitCore.run(SvcTest);
                
            
             }

In my test suite class called SvcTest i simply defined public variables of all the classes which needs to be tested , I want all of them to be excuted in sequence. For examples the code is below:-

package com.testcase.svc
{
    import com.testcase.svc.cases.*;
    [Suite]
    [RunWith("org.flexunit.runners.Suite")]    
    public class SvcTest
    {
        public var a:LocaleSvcTest;
        public var b:AuthenticateSvcTest;
        public var c:NumberFormattingSvcTest;
        public var d:EngineSvcTest;
        public var e:ApplicationConfigSvcTest;
    }
}

Now i want my first test class to be executed is LocaleSvcTest, then AuthenticationSvcTest, so on and so forth. But i see it takes random class and executes. Can some1 help me out with these problem and tell me how to make it in a sequence and execute one by one.

Thanks in Advance,

Rupam

[mlabriola]

Rupam,

By definition unit tests are not supposed to have a dependent order. Doing so means that you are not testing a single 'unit' anymore as there are dependencies.

There are hooks in the code to allow you to do manually sorting on a request, but I doubt most of this works in the beta as it is not a high priority item for us. Eventually, the Order parameter that can be used on test methods will be extended to this level but I don't have a timeline on that.

I am sorry this continues to sound as though our project is incomplete, but you are trying to use it in a way that it was not intended at this point.

If you are feeling very adventerous, I can point you to a place where you could deal with this temporarily. The Suite class (org.flexunit.runners.Suite), which is the one specified in the metadata for your suite finds these classes. You could copy this class, and then specify your own runner using that same metadata and do whatever you would like in there (sort, filter, etc.).

We are going to be pushing some new bits to the SVN site today and releasing another beta soon. If it looks like a low impact change I can try to add this piece in, but I can't guarantee it.

Sorry,

Mike

[JasonHanson]

Michael or Rupam, has there been any progress on this front?  I am using this build:

FlexUnit 4 beta 2.0 (Turnkey Test Project)

24th August 2009

The specific issue that I have is that most of the unit tests I want to write are dependent on a 4 part async login being complete prior to running tests.  The login is complicated and was not designed with unit testing in mind.  I was able to write a test for login with the new [Before] meta data tag, and I chain together the different phases of the login process.  That took a while since our service layer uses callback methods and does not dispatch events when the async opperation is complete .... but that is a different topic.

What can I do to ensure that my login that requires 4 chained async operations is complete before running any other tests?  Specifying the order in my Test Suite did not do what I expected.  Setting the order of tests in different classes did not do what I expected either.  i.e.

TestLogin.as

[Test(order=1)]

public function testLogin() : void

TestSomethingElse.as

[Test(order=2)]

public function testSomethingElse():void

The "testLogin" always is the last test to run.

What am I doing wrong?

--jason

[mlabriola]

Jason,

The order meta data is just inside a single test case, so if testLogin and testSomethingElse were both in the same test case file, then they would run in that order. While it is clear from your post that you already know I wish you weren't trying to depend on order, you can also place an order metadata on the test case itself to dictate the order each of those run.

Mike

[JasonHanson]

Michael,

Thanks for the quick reply.  I suspected that order was specific to a individual test case, but was not certain how Flex Unit read the order meta data.  Thank you for the clarification.

Testing input / output algorithms is fairly straight forward with FlexUnit.  I rarely run into issue with that type of test.  I have struggled with async tests in the past and I must say that Flex Unit 4 is much better with async tests.  Someone is obviously listening .

So much of the code that I write, and the projects I work on, require established authentication with a server prior to running an async unit test.  Especially with RTMP push.  My Flash client needs to keep sync with the server.  The client makes a request to the server and then waits for an RTMP push with updated state (data).  These are the async transactions I need to write unit tests for and am having problems.

The application I am trying to write tests for is a MMO game.  The server waits to send an RTMP push of data until a certain time, as to not flood the client with a data update each time the state on the server changes (other player walks into scene, leaves scene, sends chat message, etc).  The data is collated and then pushed at a reasonable interval (500ms - 5000ms depending on load).  An authenticated connection with the server is required prior to making calls and pushing data back.

It is not as simple as making an aync call on a service object.

Most of our bugs have to do with the server pushing the wrong data at the wrong time; lots of race conditions.  It would be helpful to be able to have unit tests for the client server communication - I know that requires a dependency on a server working - but that is really part of what I would like to test.

What do you observe that others are doing to write unit tests that require complex authentication prior to making a call to a server that does not return async to a service object, but rather does a data push sometime later (withing 5000ms)?

I am open to rewriting the codebase to work better with unit tests, but I will still have the same requirements on the server side as I do now.

The usecase I am trying to write a test for right now is this:

User logs into client

Server pushes user data including inventory list

Client deletes an item from inventory (send 'delete item request' to server and wait for the next data push)

Server sends an RTMP push with a 'deleteItem' command to be executed by client

Client executes 'deleteItem' and updates model

Assert that the model was updated correctly

Thanks for any insight, ideas, suggestions, criticisms, you may have on my situation.

--jason

[Rupam_K]

Hi jason,

you can follow this thread

http://forums.adobe.com/thread/472342?tstart=0

Its a temporary solution for a time being. Please read the whole thread.

Thanks

Rupam

[JasonHanson]

Thanks Rupam, that does solve my problem with sequencing login prior to running tests.  Thanks

[Rupam_K]

you welcome

[Jens_Wegar]

Know this thread is old, but in case someone else has this issue in the future., here's how I solved the issue of needing to authenticate. In contrast to the solution outlined in the other thread, mine does not depend on a particular order of the tests. In terms of server load however it's probably heavier, but when testing that shouldn't matter too much. Anyway, in FlexUnit you can create an async startup sequence by defining the async metatag in front of the setUp function. Then, inside the function you call the Async.proceedOnEvent, which will cause the test case to wait with the rest of the tests until the defined event occurs. This way, you can make sure a user has been authenticated before proceeding with tests in the test case and your tests are not depending on a particular order.

[Before(async)]
          public function setUp():void
          {
               _auth = new AuthenticationManager();

               // create the mock login event
               var evt:AuthenticationEvent = new AuthenticationEvent(AuthenticationEvent.AUTHENTICATION_EVENT_INITIALIZE, false, false);
               evt.username = "testuser";
               evt.password = "testpassword";
               _auth.authenticate( evt );

               Async.proceedOnEvent( this, _auth, AuthenticationEvent.AUTHENTICATION_EVENT_SUCCESS, 5000 );
}

For convenience I created a base class that will handle the authentication for me. Whenever I write a test case that requires authentication I simply subclass my base class and just set the async metatag on the setUp function:

In my test case class:

[Before(async)]

          public function setUp():void

          {

               _service = new Shipments();

               setupAuthentication();

          }

And the base class:

package flexUnitTests

{

     import fi.wegar.events.AuthenticationEvent;

     import flowcontrol.models.AuthenticationManager;

     import org.flexunit.async.Async;

     public class AuthenticatedTestBase

     {     

          protected var _auth:AuthenticationManager;

          protected function setupAuthentication():void

          {

               _auth = new AuthenticationManager();

               // create the mock login event

               var evt:AuthenticationEvent = new AuthenticationEvent(AuthenticationEvent.AUTHENTICATION_EVENT_INITIALIZE, false, false);

               evt.username = "testuser";

               evt.password = "testpassword";

               _auth.authenticate( evt );

               Async.proceedOnEvent( this, _auth, AuthenticationEvent.AUTHENTICATION_EVENT_SUCCESS, 5000 );

          }

          protected function tearDownAuthentication():void

          {

               _auth = null;

          }

     }

}

The downside to this solution of course is that you're authenticating a user at each call to a test method in the test case, since setUp and tearDown are called for each one of the tests.

[mlabriola]

I consider your downside and upside

I would prefer to see this type of authentication and re-login on each test whenever possible.

Mike