7 Replies Latest reply on Jan 13, 2012 7:08 AM by Duck&Dive

    Captivate 5.5 pass/fail issues

    Duck&Dive
    Duck&Dive Dec 22, 2011 10:58 AM

      I have been working for several days in attempt to configure a Captivate 5.5 project to function correctly as SCORM 1.2 content deliverable from an LMS; this should not be a difficult task! Configuring the SCORM 1.2 report settings for content with a scored assessment is not rocket science.

       

      Process:

      -developed content

      -developed a scored assessment

      -define a pass/fail threshold for the assessment (completion requirements are simple, learner meets or exceeds threshold or not)

      -configure manifest

      -select pass/fail (pass/fail is for scored assessments, incomplete/complete is for % of pages viewed by learner; i.e. no assessment)

      -select quiz results only

      -report score (which obviously should include the learners pass or fail status)

       

      In the event there was a misunderstanding to the completion requirements for my content: complete assessment, meet or exceed threshold to obtain a pass status otherwise it’s a failed attempt. For the life of me I can’t understand why the Captivate 5.5 SCO issues a “Failed” status after initializing the API. Obviously at this point in the course the completion requirements, completing the assessment, have not been met by the learner!

       

      Information copied from the ADL SCORM 1.2 Conformance Requirements document:

      When setting (writing) the cmi.core.lesson_status element, the SCO shall set the value according to the following:

      “incomplete” – This value is used when the SCO is taken for no-credit or credit, when the SCO is exited prematurely (before a passed/failed/completed status can be determined).

       

      I’ve tweaked the settings to the point where I can get the SCO to send an incomplete status after the API has been initialized. In addition I can access the assessment, pass it and receive a passing transcript. I can also access the assessment, fail it and receive a failed transcript. However, if I view just a few pages and exit I receive a failed transcript. Again, how can a completion status, fail, be sent by the SCO when the completion requirements have not been met? (refer back to the ADL comformance requirement above)  It’s worthy to note bookmarks are recorded in my transcript…but what good are bookmarks if you have a failed transcript? A failed attempt is a valid completion. For certifications purposes our transcripts are locked once they’ve obtained a valid completion; i.e. pass, fail or complete.

       

      Here is a valid scenario:

      A 2 hour course has been developed for corporate. Every employee in the company must complete the course by passing the assessment. If any employee fails the course twice they must travel to corporate head quarters for a 3 day ILT training and coaching session. The employee logs into the LMS, launches the course and midway through the course the employee’s boss calls. The boss wants the employee in his office to discuss an issue with a customer. No problem right? The course utilizes bookmarks so at a later time the employee can re-launch and continue on where he left off. The employee exits the course, meets with the boss and returns to his desk. He logs back into the LMS, accesses his learning plan to discover there is a failed transcript for the corporate course rather than an incomplete transcript. Once again the employee’s boss calls and this time he is upset. Why? The LMS generated an email and sent it to the boss informing him the employee now has one failed attempt at completing the corporate course. One more failed attempt and the employee will have an expensive 3 day trip to corporate head quarters!

        The observed functionality of Captivate content not compiling with the SCORM standard is unacceptable!!

      • 2937 Views
      • Tags:
        • 1. Re: Captivate 5.5 pass/fail issues
          Duck&Dive Level 1
          Duck&Dive Jan 9, 2012 6:27 AM (in response to Duck&Dive)

          Is there a way to force Captivate 5.5 to function correctly based on ADL conformance?

          • 2. Re: Captivate 5.5 pass/fail issues
            Erik Lord Adobe Community Professional
            Erik Lord Jan 10, 2012 7:40 PM (in response to Duck&Dive)

            I may not grasp the situation exactly, but as a quick reply, you state,

            "pass/fail is for scored assessments, incomplete/complete is for % of pages viewed by learner"

            Are you mixing the two? If so, are you selecting SCORM 2004? Is your LMS SCORM 2004-compliant?

             

            If you must use SCORM 1.2, don't mix those two - report the score only, not the page views. Captivate can get wacky when trying to track both pass/fail and complete/incomplete at the same time...

             

            I'll try to revisit this tomorrow and reference a matrix I've got on various reporting combinations and results with Captivate...

            HTH

            Erik

            • 3. Re: Captivate 5.5 pass/fail issues
              Erik Lord Adobe Community Professional
              Erik Lord Jan 11, 2012 7:22 AM (in response to Duck&Dive)

              On revisiting, it looks like you are just setting pass/fail options, correct?

              What options are you selecting in the Quiz Preferences dialog?

              Are you setting a Mastery Score? If so, try NOT doing that as such a setting may be forcing the LMS to establish a pass/fail status when the SCO exits, regardless of whether the assessment was actually taken or not.

               

              Now keep in mind, as I understand it, that your scenario may well occur if the user takes an assessment (quiz) part-way through and exits the lesson. I believe when a quiz starts, a score is kept and updated as each question is answered; so if there are 4 questions and the user answers the first one correctly, they now have a 25%....and if they exit at that point, their current score is 25%....which Captivate or the LMS may then evaluate as a 'fail' as it does not meet the 'passing' requirements....

              Erik

              • 4. Re: Captivate 5.5 pass/fail issues
                Duck&Dive Level 1
                Duck&Dive Jan 11, 2012 7:58 AM (in response to Erik Lord)

                Erick,

                 

                Based on SCORM 1.2 standards a pass/fail status is for content containing a scored assessment and incomplete/complete status is for content which does not contain a scored assessment.

                We have content which contains a scored assessment and we’re publishing to SCORM 1.2. We have attempted to use pass/fail without success.

                This is a synopsis of our best attempt at using the pass/fail option:

                -upon accessing the SCO a status of incomplete is sent to the LMS

                -upon completing content and passing the assessment a passing status is sent to the LMS

                -upon completing content and failing the assessment a failed status is sent to the LMS

                -however, if exiting SCO prior to completing content and not accessing assessment, a failed status is sent to the LMS

                 

                The above synopsis is incorrect. If a learner exits the SCO without completing the content and not accessing the assessment the SCO should return an incomplete status to the LMS. When the learner re-launches the SCO then book marking will return the learner to the point in the content where he/she left off; Captivates book marking capabilities are excellent. A pass/fail status cannot be nor should it be determined until the learner has met the course completion requirements; i.e. pass or fail the assessment.


                As I noted in my initial post from the ADL Conformance requirements document; “incomplete” – This value is used when the SCO is taken for no-credit or credit, when the SCO is exited prematurely (before a passed/failed/completed status can be determined).


                With the above ADL statement in mind a final status cannot be determined until the course completion requirements are met by the learner. For a pass/fail the completion requirement is for the learner to complete the assessment. Even if the learner accesses the assessment and exits prior to completing it, which would result in a failed attempt. For incomplete/complete the requirement is for the learner to meet the % of content viewed. A status of incomplete should be maintained until the learner meets the course completion requirements.

                 

                We have also attempted to use incomplete/complete for the same course. The issue we have with this is that even if the learner fails the assessment the SCO sends a complete status to the LMS. Obviously our content with an assessment requires a pass/fail which is what we want.

                 

                Thanks for your feedback, 

                • 5. Re: Captivate 5.5 pass/fail issues
                  RodWard Adobe Community Professional & MVP
                  RodWard Jan 11, 2012 9:21 AM (in response to Duck&Dive)

                  Just out of curiosity, have you tried uploading your Captivate course to the SCORM Cloud LMS and tested what that does with it?

                   

                  I mention it because SCORM Cloud is built by Rustici Software, the people that make many of the SCORM players used by large LMS vendors.  They're the recognised experts when it comes to SCORM.  So it would be interesting to see if their SCORM Cloud LMS has an issue with your Captivate course too.  I've always found that my Cp courses set up for SCORM 1.2 worked pretty flawlessly and got a total thumbs up from SCORM Cloud.

                  • 6. Re: Captivate 5.5 pass/fail issues
                    Erik Lord Adobe Community Professional
                    Erik Lord Jan 12, 2012 6:46 PM (in response to Duck&Dive)

                    I like Rod's suggestion, just to see if the behavior changes. We test in the SCORM Cloud environment for such unexpected results as well.

                    Overall, however, how the CP development team has interpreted the SCORM spec is...what it is. Definitely feel free to make suggestions on how it should be changed:

                    https://www.adobe.com/cfusion/mmform/index.cfm?name=wishform

                     

                    For your immediate needs, the only solution is to either hit upon the proper combination of value settings for the lesson to act the way you need...and there's no guarantee that's possible. It sounds like you need to do pass/fail, no complete/incomplete, for your options...but I don't think  you'll be able to overcome the lesson marking 'failed' if the user doesn't finish taking the quiz...

                    ...but maybe you'll hit upon a combination of settings that works.

                     

                    If not, then the only solution is possibly altering the quiz results before it gets sent to the LMS, via some custom JS coding in the parent HTML document, or choosing a different tool...

                    Wish I had better guidance!

                    Erik

                    • 7. Re: Captivate 5.5 pass/fail issues
                      Duck&Dive Level 1
                      Duck&Dive Jan 13, 2012 7:08 AM (in response to Erik Lord)

                      Erik,

                       

                       

                       

                      Rod had a good suggestion. However, I can’t see any possible way the behavior of the content could change; it is what it is. The problem is not our LMS it’s the content. Inside the course start HTM file is an option to activate an alert box which I’ve done; and by the way the alert box displays pretty much the same data as the Java Console. As you can clearly see (Location = 3) I viewed 3 pages of the content prior to exiting, obviously not accessing the quiz, the content sent a failed status. Our LMS only records information sent by the course content. Therefore even testing with Rustici’s, who I’m very familiar with, would record exactly the same thing.

                       

                       

                       

                      cid:image001.jpg@01CCD1D7.D7B774C0