ANSI INCITS ISO IEC 23988-2007 Information technology A code of practice for the use of information technology (IT) in the delivery of assessments.pdf

1、INCITS/ISO/IEC 23988:20072008 (ISO/IEC 23988:2007, IDT) Information technology A code of practice for the use of information technology (IT) in thedelivery of assessmentsINCITS/ISO/IEC 23988:20072008(ISO/IEC 23988:2007, IDT)INCITS/ISO/IEC 23988:20072008 ii ITIC 2008 All rights reserved PDF disclaime

2、r This PDF file may contain embedded typefaces. In accordance with Adobes licensing policy, this file may be printed or viewed but shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading this file, parties accep

3、t therein the responsibility of not infringing Adobes licensing policy. The ISO Central Secretariat accepts no liability in this area. Adobe is a trademark of Adobe Systems Incorporated. Details of the software products used to create this PDF file can be found in the General Info relative to the fi

4、le; the PDF-creation parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below. Adopted by INCITS (

5、InterNational Committee for Information Technology Standards) as an American National Standard. Date of ANSI Approval: 7/1/2008 Published by American National Standards Institute, 25 West 43rd Street, New York, New York 10036 Copyright 2008 by Information Technology Industry Council (ITI). All right

6、s reserved. These materials are subject to copyright claims of International Standardization Organization (ISO), International Electrotechnical Commission (IEC), American National Standards Institute (ANSI), and Information Technology Industry Council (ITI). Not for resale. No part of this publicati

7、on may be reproduced in any form, including an electronic retrieval system, without the prior written permission of ITI. All requests pertaining to this standard should be submitted to ITI, 1250 Eye Street NW, Washington, DC 20005. Printed in the United States of America INCITS/ISO/IEC 23988:2007200

8、8 ITIC 2008 All rights reserved iii Contents Page Foreword iv Introduction v 1 Scope 1 2 Normative references 3 3 Terms and definitions 4 4 Guiding principles. 7 5 Interface between assessment content and IT delivery . 8 6 IT delivery of assessments general . 9 7 Software for IT delivery of assessme

9、nts navigation and usability issues . 15 8 Instant automated scoring, result determination and feedback . 18 9 Preparation and transmission of assessment content and correct responses 20 10 Software and procedures for recording and transmission of candidates details and responses and for their stora

10、ge by assessment distributors . 22 11 Provision of instructions and assessment-specific information for assessment centres . 24 12 Provision of information and practice material for candidates . 25 13 Equipment and facilities at assessment centre, including storage of responses . 27 14 Staffing of a

11、ssessment centre 30 15 Preparation for the assessment session 31 16 Conduct of assessment session 32 17 Procedures relating to emergencies, technical failures and irregularities 34 Annex A (informative) Scenarios. 35 Bibliography. 38 INCITS/ISO/IEC 23988:20072008 iv ITIC 2008 All rights reserved For

12、eword ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical

13、 committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in

14、 the work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. The main task of the joint technical committee is to prepare Internati

15、onal Standards. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote. Attention is drawn to the possibility that some of t

16、he elements of this document may be the subject of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights. ISO/IEC 23988 was prepared by BSI (as BS 7988) and was adopted, under a special fast-track procedure, by Joint Technical Committee ISO/IEC JTC 1,

17、 Information technology, in parallel with its approval by the national bodies of ISO and IEC. INCITS/ISO/IEC 23988:20072008 ITIC 2008 All rights reserved v Introduction Growth in the power and capabilities of information technology (IT) has led to the increasing use of IT to deliver, score and recor

18、d responses of tests and assessments in a wide range of educational and other contexts. Suitably used, IT delivery offers advantages of speed and efficiency, better feedback and improvements in validity and reliability, but its increased use has raised issues about the security and fairness of IT-de

19、livered assessments, as well as resulting in a wide range of different practices. The aims of this Standard are to provide a means of: showing that the delivery and scoring of the assessment are fair and do not disadvantage some groups of candidates, for example those who are not IT literate; showin

20、g that a summative assessment has been conducted under secure conditions and is the authentic work of the candidate; showing that the validity of the assessment is not compromised by IT delivery; providing evidence of the security of the assessment, which can be presented to regulatory and funding o

21、rganisations (including regulatory bodies in education and training, in industry or in financial services); establishing a consistent approach to the regulations for delivery, which should be of benefit to assessment centres who deal with more than one assessment distributor; giving an assurance of

22、quality to purchasers of off-the-shelf assessment software. It is envisaged that the achievement of these aims will enhance the status of IT-delivered assessments and assessment software and encourage their wider use in situations where they are beneficial. Users attention is drawn to any existing d

23、omain-specific legislation covering the subject matter of this Standard. This could include, but is not limited to legislation relating to disability, special educational needs, data protection, privacy, freedom of information, language and health and safety in relation to equipment or a broader wor

24、k environment context. This Standard takes the form of guidance and recommendations. It should not be quoted as if it were a specification, and particular care should be taken to ensure that claims of compliance are not misleading. A Standard does not purport to include all the necessary provisions

25、of a contract. Users of Standards are responsible for their correct application. Compliance with a Standard does not of itself confer immunity from legal obligations. This Standard is intended for organisations involved in the use of IT for the delivery of assessments, including (but not limited to)

26、: universities, colleges, learning centres, and schools who assess their students, for diagnostic, formative and summative (final test) purposes; organisations which deliver and/or award educational examinations; professional bodies, industry organisations and others who deliver and/or award examina

27、tions or assessments in vocational subjects; producers and distributors of IT certification tests; assessment centres which administer assessments locally; INCITS/ISO/IEC 23988:20072008 vi ITIC 2008 All rights reserved open, distance, or e-learning centres and other organisations which may provide a

28、ssessment facilities on an occasional basis; training companies, employers and governmental/military organisations providing assessments as part of vocational education and training; organisations providing assessments required for regulatory purposes, for example a financial services company might

29、be required to assess its sales people for product knowledge, or a pharmaceutical company might be required to assess its staff on safety procedures; open learning and distance learning material providers, including on-line universities and commercial publishers and distributors of learning material

30、s, who incorporate assessments in their material; producers of item banks, quizzes and tests available as revision aids. This Standard is also relevant to developers of software for delivering assessments, who will need to provide software which enables their clients to comply with this Standard. Th

31、is Standard distinguishes three main roles in the IT delivery of assessments: assessment sponsors, responsible for assessment content and award of certificates; assessment distributors, responsible for delivering assessments via IT including developing or specifying the delivery software; assessment

32、 centres, where the assessments are taken. The roles involved in the different stages of the assessment life cycle (see 1.3) are combined in different ways by the organisations involved and not all organisations are involved in all roles. Thus, a university may combine the roles of assessment sponso

33、r, assessment distributor and assessment centre, whilst the assessments distributed by an awarding body (as assessment sponsor and distributor) are normally taken in assessment centres which are separate organisations. This Standard therefore recommends the action to be taken in relation to each of

34、these three roles, irrespective of the type of organisation undertaking the role. The scenarios in Annex A give examples of how this can work in practice. The roles of the different organisations are often inter-related, especially in high-stakes assessment. For example, the assessment distributor o

35、r awarding organisation might have an overall responsibility for the integrity of the whole process, including monitoring assessment centres. Assessment centres need to comply with the general regulations of assessment distributors, as well as with those which are specific to the use of IT. It shoul

36、d be noted that some of the organisations to which this Standard is relevant may also need to comply with domain-specific requirements of regulatory authorities in relation to the design and conduct of assessments, including pedagogical aspects. Where the candidates are employees of the assessment c

37、entre, there may also be more specific legislative requirements. AMERICAN NATIONAL STANDARD INCITS/ISO/IEC 23988:20072008 ITIC 2008 All rights reserved 1 Information technology A code of practice for the use of information technology (IT) in the delivery of assessments 1 Scope 1.1 General This Stand

38、ard gives recommendations on the use of information technology (IT) to deliver assessments to candidates and to record and score their responses. Its scope is defined in terms of three dimensions: the types of assessment to which it applies, the stages of the assessment life cycle to which it applie

39、s and this Standards focus on specifically IT aspects. 1.2 Types of assessment This Standard is relevant to a wide range of assessments, including: assessments used in education (both compulsory and post-compulsory), training and compliance (e.g. compliance with legislation relating to health and sa

40、fety or financial services); assessments of knowledge, understanding and skills (i.e. achievement tests), but not psychological tests of aptitude and personality; high-stakes assessments and examinations and also low-stakes assessments used for feedback on progress, identification of learning needs,

41、 self-assessment and remediation; assessments which include feedback, as well as those which provide only a result; both fixed-date, test-windows and on-demand assessments; both items which can be scored by computer and the delivery (but not scoring) of items or tasks which are delivered using IT, b

42、ut have to be referred to a human marker for scoring; however, the use of IT for scoring lengthy free-text responses (essays and similar) is excluded; a wide range of computer-scorable assessments, including not only objective tests (multiple-choice and other item types), but also assessments of key

43、boarding skills and software use; NOTE Items can include graphics, multimedia and access to data, case studies, etc. This Standard does not, however, cover the use of specialist simulations (e.g. of industrial processes or driving/piloting). assessments taken in universities, academies, colleges, hi

44、gh-schools, training centres, schools and assessment centres and also assessments taken in less formal settings, including learning centres and in the workplace; assessments which are set, taken and scored within a single organisation (for example a university or a company) and assessments taken in

45、an organisation separate from the one responsible for the assessment content; assessments delivered in a variety of ways, including on-line, on local networks and on stand-alone computers. Clause 9 of this Standard is also applicable to the use of IT for the transmission of candidates coursework. IN

46、CITS/ISO/IEC 23988:20072008 2 ITIC 2008 All rights reserved 1.3 Assessment life cycle Although assessment procedures vary, the typical life cycle of assessment consists of the following steps: a) identification of need to assess; b) design of outcomes/assessment methodology; c) preparation and calib

47、ration; d) pre-registration (includes payment); e) distribution1); f) authentication (includes identification)1); g) delivery1); h) response return1); i) scoring, result determination and/or feedback1); j) data return1); k) analysis; l) appeals; m) certification. Preparation of assessment content is

48、 outside the scope of this Standard, but where pretesting is undertaken using IT for delivery, the relevant clauses of this Standard should be taken into consideration. 1.4 Focus on specifically IT aspects The emphasis throughout this Standard is on the additional or different measures which should

49、be taken as a result of the introduction of IT. Measures which are common to both paper and IT-delivered systems are either omitted or covered in only general terms. This Standard does not cover the purely pedagogical aspects of assessment preparation or scoring, but does cover the use of IT to implement pedagogical decisions and the interface between assessment content and IT delivery. The quality of the assessment content and the scoring procedures are of cruc