1、The Case of the Repeating Licensure Examination Candidate: Psychometric, Policy, and Legal Issues Moderator I. Leon Smith, President & CEO Professional Examination Service Presenters: Mary Browne, Program Director Professional Examination Service Julia C. Works Atkinson & Atkinson,Presented at the 2
2、004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,The Case of the Repeating Licensure Examination Candidate: Psychometric and Policy Issues Mary Browne Program Director, PES,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Presented at
3、 the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Why Retake?,Failure Exam Malfunction Suspicion of cheating Illness or other incident,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Second, ThirdChances,Legal rulings support pr
4、ograms that allow retakes The spirit behind most US education and law is to allow second chances How does this impact on public protection Should there be a cutoff on number of retakes allowed,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Instead of a Ret
5、ake,Alternative form of assessment Partial retake - Retaking only domains or sections failed,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Remediation,Should it be required Is it effective,Presented at the 2004 CLEAR Annual Conference September 30 October
6、 2 Kansas City, Missouri,Retake Concerns,Overlap with previous forms Timing What results patterns are expected? Suspicious? Practice effect False Positives Adverse Impact,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Overlap With Previous Forms,Repeated I
7、tems Equating Item Bank limitations. How do candidates perform on a repeated item? Repeat previous response Intervening learning based on recollection of the item?,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Retake Timing Considerations,How frequently i
8、s testing offered? How many forms of the test are available? How different are test forms?,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Reasons for Concern,False Positives False Negatives Errors of Measurement,Presented at the 2004 CLEAR Annual Conferenc
9、e September 30 October 2 Kansas City, Missouri,True Score and Standard Error,Obtained score = True Score + Random Error 83 = 83 3 or 80-86 Errors are extraneous variables such as the fatigue, the particular sample of questions chosen for the test, environmental conditions or luck in guessing. For ev
10、ery candidate there is a “best” and “worst” score possible.,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Some Examples of Score Changes in a Licensing Program.,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Fro
11、m One Exam to the Next,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Two Common Failing Score Patterns,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Score Patterns - Failing,Presented at the 2004 CLEAR Annual C
12、onference September 30 October 2 Kansas City, Missouri,Score Patterns - Failing,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Passing Score Patterns,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Score Patterns
13、- Passing,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Some Score History Illustrations,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Passpoints,Most passpoints are set to pass the minimally qualified candidat
14、e. Candidates whose qualifications are at a level just below the minimum may pass due to score error. With repeated testing, this becomes more likely.,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Score Patterns - Passing,Presented at the 2004 CLEAR Annua
15、l Conference September 30 October 2 Kansas City, Missouri,Dilemmas,Balance between fairness in allowing candidate retests and protecting the public from unqualified candidates. Eliminating candidate advantage gained from repeated retakes,Presented at the 2004 CLEAR Annual Conference September 30 Oct
16、ober 2 Kansas City, Missouri,Remedies,Adaptive Testing Set a Confidence Interval or Indifference Zone Change the passpoint depending on number of retakes Raise the passpoint to reflect retake advantage Average last two or three test scores,Presented at the 2004 CLEAR Annual Conference September 30 O
17、ctober 2 Kansas City, Missouri,Who Should Benefit?,Should decisions be weighted on the side of public protection or fairness to candidates? Where is the balance here? Will more stringent retake requirements result in adverse impact?,Presented at the 2004 CLEAR Annual Conference September 30 October
18、2 Kansas City, Missouri,Contact InformationMary BrownePES475 Riverside Drive212 367-4250 (ph) 212 367-4266 (fax)browneproexam.orghttp:/www.proexam.org,Case of the Repeating Licensure Examination Candidate: A Legal Perspective Julia C. Works Atkinson & Atkinson 1603 Orrington Avenue Suite 2080 Evanst
19、on, IL 60201 Tel. 847.864.0070 Fax.847.864.0588 E-mail: ,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Legal Perspective,Credential: Voluntary Private Sectorvs.High
20、 Stakes Licensure Examination: Mandatory Statutes and/or Rules Public Protection,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,High Stakes Licensure Examination,Who relies on examination results?Regulatory Boards For what purpose?Measure of Minimum Compet
21、enceProtection of the public from unqualified practitioners,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Legal Perspective,Limiting the number of attempts:How do regulatory boards look at this issue?Security exposure of exam items Validity of Results Doe
22、s the fact that it took an applicant X number of times to pass the licensing exam mean they are less than minimally competent?,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Currently,Majority of regulatory boards do not limit the number of examination att
23、empts. Those boards that do limit the number of attempts may require candidates to seek some type of remedial education before being permitted to sit for the examination again.,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Currently,Many high stakes licen
24、sure examinations may not be retaken within a 90-day window Examination attempts may be limited to a specified number within a specified time period For example: Limited to 3 attempts/retakes Limited to 4 times within open window of administration of examination,Presented at the 2004 CLEAR Annual Co
25、nference September 30 October 2 Kansas City, Missouri,How do you limit the number of attempts?,Statute Rule/Regulation Contractually Policies and Procedures,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Legality of Limitations,Constitutional Claims challe
26、nging limitation on number of attempts:Equal ProtectionDue Process,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Equal Protection,Argument may be asserted that limiting the number of attempts violates Equal Protection because such limitations, whether est
27、ablished through statute or regulation, are not rationally related to any legitimate state interest.,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Linton v. Missouri Veterinary Medical Board,Legislature changed law to limit number of attempts to pass lice
28、nsing exam to three(3). (Effective Aug. 1992)Facts: Following revision to law, applicant failed licensing exam 3 times in MO but passed on the 4th attempt in Illinois Applicant then applied to MO Board for license Application was denied Applicant appealed board decision asserting statute and regulat
29、ion violated her right to Equal ProtectionLinton v. Missouri Veterinary Medical Board, 988 S.W.2d 513 (Mo.banc 1999).,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Linton v. Missouri Veterinary Medical Board,Court held applicants right to Equal Protection
30、 not violated where three-exam limitation was rationally related to a legitimate state interest (i.e. the establishment of a high level of competence for veterinarians in Missouri).Court found the interest of the legislature here was the same interest as establishing the test in the first place.,Pre
31、sented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Linton v. Missouri Veterinary Medical Board,Next question under Equal Protection Analysis:Is the means chosen rationally related to achieving the legitimate state interest?,Presented at the 2004 CLEAR Annual Conf
32、erence September 30 October 2 Kansas City, Missouri,Linton v. Missouri Veterinary Medical Board,In other wordsAre those who must take the exam four or more times before passing of equal or greater competence than those who pass the test in three or fewer attempts? Is there a reasonable connection be
33、tween competence and those who pass the exam in three or fewer attempts?,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Linton v. Missouri Veterinary Medical Board,Following this decisionStatute amended to read, in part, as follows: If that passing score w
34、as not received within three attempts, the board may require the applicant to appear before the board or submit evidence that the applicant has completed at least thirty hours of board-approved continuing education.V.A.M.S. 340.234 Examination application and fees.,Presented at the 2004 CLEAR Annual
35、 Conference September 30 October 2 Kansas City, Missouri,Currently,Similar restriction in Alabama with regard to the state examination where board rule reads, in part, as follows:(4) A maximum of three retakes of the state written exam will be permitted. Alabama Admin. Code 930-X-1-.09.ExaminationsA
36、labama State Board of Veterinary Medicine.,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Due Process,Allegations that limitation on number of attempts violates Due Process clause of U.S. Constitution because it deprives individuals of a vested right to ta
37、ke the licensing examination.,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,State Board of Registration for the Healing Arts v. Boston,Statute limited the number of times physical therapist assistants could take licensing examinationFacts:- Applicant fail
38、ed physical therapy assistant exam three times prior to August 1999, when statute limiting number of attempts became effective.- Applicant applied to take physical therapy assistant exam for 4th time after the statute effective date.- Board denied application. Administrative Hearing Commission rever
39、sed Board decision and ordered Board to admit her to the examination.- Board sought review of Administrative Hearing Commission decision that found statute could not be applied retroactively.,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,State Board of Re
40、gistration for the Healing Arts v. Boston,Court found that the legislative intent of the statute clearly manifested that it be applied retroactively. Statute: The board shall not issue a license to practice as a physical therapy assistant or allow any person to sit for the Missouri state board exami
41、nation for physical therapist assistants who has failed three or more times any physical therapist assistant licensing examination administered in one or more states or territories of the United States or the District of ColumbiaV.M.S.A. Section 334.655.3, amended Aug. 28, 1999,Presented at the 2004
42、 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,State Board of Registration for the Healing Arts v. Boston,Court found that the Missouri Constitution bars retrospective application of a statute except where: 1) legislative intent is clearly manifested that the statute is to be
43、applied retrospectively; and 2) the statute is procedural only and does not affect any substantive or vested right.,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,State Board of Registration for the Healing Arts v. Boston,Court found that statutory languag
44、e unmistakably refers to past conduct Court further held that application of the statute did not deprive applicant of any vested right. As a general matter, professional licensing in a particular profession is a privilege granted by the state. Restrictions surrounding the practice of a profession ar
45、e for the benefit of society, not the practitioner Licensing statutes confer no substantive rights upon applicants seeking licensure,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Other situations to be considered,Licensure by Endorsement See Yap v. Zollar
46、, 691 N.E.2d 18 (Ill. App. 1 Dist. 1997) Statute requiring candidates for licensure as registered nurses to have passed nationwide exam within 3 years of first attempt rather than passing within 6 attempts within an unlimited period of time could only be applied prospectively because this was a subs
47、tantive change and retroactive application would deprive applicant of a vested right where applicant had reasonably relied on prior statute. See also John v. Dept. of Professional Regulation, 713 N.E.2d 673 (Ill. App. 1 Dist. 1999).,Presented at the 2004 CLEAR Annual Conference September 30 October
48、2 Kansas City, Missouri,Other situations to be considered,Foreign Graduate Equivalency Programs,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Contact Information,Julia C. Works Atkinson & Atkinson 1603 Orrington Avenue Suite 2080 Evanston, IL 60201 Tel.: 847.864.0070 Fax: 847.864.0588 E-mail: ,Presented at the 2004 CLEAR Annual Conference September 30 October 2 Kansas City, Missouri,Reference,Millman, J. (1989). If at First You Dont Succeed: Setting Passing Scores When More Than One Attempt is Permitted. Educational Researcher, 18(6), 5 9.,
