BS 8611-2016 Robots and robotic devices Guide to the ethical design and application of robots and robotic systems《机器人和机器人装置 机器人和机器人系统的伦理设计和应用指南》.pdf

1、BSI Standards PublicationBS 8611:2016Robots and robotic devicesGuide to the ethical design andapplication of robots and roboticsystemsPublishing and copyright informationThe BSI copyright notice displayed in this document indicates when the documentwas last issued. The British Standards Institution

2、2016Published by BSI Standards Limited 2016ISBN 978 0 580 89530 2ICS 25.040.30The following BSI references relate to the work on this document:Committee reference AMT/-/2Draft for comment 15/30320088 DCPublication historyFirst published April 2016Amendments issued since publicationDate Text affected

3、BS 8611:2016 BRITISH STANDARDContents1 Scope 12 Normative references 13 Terms and definitions 14 Ethical risk assessment 25 Ethical guidelines and measures 86 Ethics-related system design recommendations 137 Verification and validation 148 Information for use 16Bibliography 20List of tablesTable 1 E

4、thical issues, hazards and risks 3Summary of pagesThis document comprises a front cover, an inside front cover, pages i to ii,pages 1 to 22, an inside back cover and a back cover.BRITISH STANDARD BS 8611:2016 The British Standards Institution 2016 iForewordPublishing informationThis British Standard

5、 is published by BSI Standards Limited, under licence fromThe British Standards Institution, and came into effect on 30 April 2016. ThisBritish Standard has been prepared by Technical Subcommittee AMT/-/2, Robotsand robotic devices. A list of organizations represented on this committee canbe obtaine

6、d on request to its secretary.Information about this documentThis is the first edition of a new standard. It addresses issues in a field oftechnology that is rapidly changing and influencing society. Although effortshave been made to incorporate the anticipated effects of future developmentsin the s

7、tandard, it is not possible for all future circumstances to have beenconsidered or covered, for example non-embodied autonomous systems.Use of this documentAs a guide, this British Standard takes the form of guidance andrecommendations. It should not be quoted as if it were a specification or acode

8、of practice and claims of compliance cannot be made to it.Presentational conventionsThe guidance in this standard is presented in roman (i.e. upright) type. Anyrecommendations are expressed in sentences in which the principal auxiliaryverb is “should”.Commentary, explanation and general informative

9、material is presented insmaller italic type, and does not constitute a normative element.Contractual and legal considerationsThis publication does not purport to include all the necessary provisions of acontract. Users are responsible for its correct application.Compliance with a British Standard ca

10、nnot confer immunity from legalobligations.BRITISH STANDARDBS 8611:2016ii The British Standards Institution 20161 ScopeThis British Standard gives guidance on the identification of potential ethicalharm and provides guidelines on safe design, protective measures andinformation for the design and app

11、lication of robots. It builds on existing safetyrequirements for different types of robots; industrial, personal care and medical.This British Standard describes ethical hazards associated with the use of robotsand provides guidance to eliminate or reduce the risks associated with theseethical hazar

12、ds. Significant ethical hazards are presented and guidance given onhow they are to be dealt with for various robot applications.Ethical hazards are broader than physical hazards. Most physical hazards haveassociated psychological hazards due to fear and stress. Thus, physical hazardsimply ethical ha

13、zards and safety design features are part of ethical design.Safety elements are covered by safety standards; this British Standard isconcerned with ethical elements.This British Standard is intended for use by designers and managers, amongstothers.2 Normative referencesThe following documents, in wh

14、ole or in part, are normatively referenced in thisdocument and are indispensable for its application. For dated references, onlythe edition cited applies. For undated references, the latest edition of thereferenced document (including any amendments) applies.BS EN ISO 12100:2010, Safety of machinery

15、 General principles for design Riskassessment and risk reduction (ISO 12100:2010)BS ISO 8373, Robots and robotic devices - VocabularyBS ISO 31000, Risk management Principles and guidelines3 Terms and definitionsFor the purposes of this British Standard, the terms and definitions given inBS ISO 8373

16、and the following apply.3.1 autonomous systemsystem which has the ability to perform intended tasks based on current state,knowledge and sensing, without human intervention3.2 ethical harmanything likely to compromise psychological and/or societal and environmentalwell-beingNOTE Examples of ethical

17、harm include stress, embarrassment, anxiety, addiction,discomfort, deception, humiliation, being disregarded. This might be experienced inrelation to a persons gender, race, religion, age, disability, poverty or many otherfactors.3.3 ethical hazardpotential source of ethical harm3.4 ethical riskprob

18、ability of ethical harm occurring from the frequency and severity ofexposure to a hazard3.5 ethicscommon understanding of principles that constrain and guide human behaviourBRITISH STANDARD BS 8611:2016 The British Standards Institution 2016 13.6 harminjury or damage to health3.7 responsible researc

19、h and innovation (RRI)process that seeks to promote creativity and opportunities for science andinnovation that are socially desirable and undertaken in the public interestSOURCE: EPSRC framework for responsible innovation1)3.8 roboticistperson actively engaged in a process leading to the developmen

20、t, production,installation, deployment or maintenance of a robot3.9 systemset of parts which, when interoperating, has behaviour that is not present in anyof the parts themselvesNOTE This definition includes systems of fleeting duration, such as cars passingeach other and behaving appropriately duri

21、ng this moment, to long-lived systemssuch as Voyager II.3.10 vulnerable peoplepeople having reduced physical, sensory or mental capabilities (e.g. partiallydisabled, elderly having some reduction in their physical and mentalcapabilities), or lack of experience and knowledge (e.g. children)SOURCE: BS

22、 EN 60335-1:2012+A11:2014, 3.Z.54 Ethical risk assessment4.1 GeneralEthical issues can be divided into societal, application, commercial/financial andenvironmental, as described in Clause 5 to Clause 8. Table 1 describes examplesof the ethical hazards and risks associated with these issues. See also

23、BS ISO 31000 for further guidance.1)See last viewed 4 April 2016.BRITISH STANDARDBS 8611:20162 The British Standards Institution 2016Table1Ethicalissues,hazardsandrisksEthicalissueEthicalhazardEthicalriskMitigationCommentVerification/ValidationSocietalLossoftrust(humanrobot)Robotnolongerusedorismisu

24、sed,abusedDesigntoensurereliabilityinbehaviourIfunexpectedbehaviouroccurs,ensuretraceabilitytohelpexplainwhathappenedUservalidationDeception (intentionalorunintentional)Confusion,unintended(perhapsdelayed)consequences,eventuallossoftrustAvoiddeceptionduetothebehaviourand/orappearanceoftherobotandens

25、uretransparencyofroboticnatureSoftware verification;uservalidation;expertguidanceAnthropo- morphizationMisinterpretationAvoidunnecessaryanthropomorphization Clarificationofintenttosimulatehumanornot,orintendedorexpectedbehaviourSeedeception(above)Useanthropomorphizationonlyforwell-defined,limitedand

26、socially-acceptedpurposesUservalidation;expertguidancePrivacyandconfidentialityUnauthorizedaccess,collectionand/ordistributionofdata,ingintothepublicdomainortounauthorized, unwarrantedentitiesClarityoffunctionControlofdata,justificationofdatacollectionanddistributionEnsureuserawarenessofdatamanageme

27、ntandobtaininformedconsentinappropriatecontextsPrivacybydesignDataencryption,storagelocation,adherencetolegislationSoftware verificationLackofrespectforculturaldiversityandpluralismLossoftrustinthedevice,embarrassment,shame,offenceAwarenessofculturalnormsincorporatedintoprogrammingOrganizational, pr

28、ofessional,regionalSoftware verification;uservalidationRobotaddictionLossofhumancapability,dependency,reductioninwillingnesstoengagewithothers,isolationRaiseawarenessofdependencyAdifficultarea,particularlyinrelationtovulnerablepeople CarefulevaluationofpotentialapplicationsisneededUservalidation;exp

29、ertguidanceBRITISH STANDARD BS 8611:2016 The British Standards Institution 2016 3Table1Ethicalissues,hazardsandrisksEthicalissueEthicalhazardEthicalriskMitigationCommentVerification/ValidationSocietalEmploymentSocialdislocation,jobreplacement, unemployment,lossofskills/needtore-skillTargeteddeployme

30、ntofrobotstounsocial,hazardousactivities,organizational;designEconomicandsocialassessmentApplicationMisuseAllinthiscolumnMinimizepotentialforunethicaluseviaethicalriskassessment AdherencetostandardsandregulationsInnovateresponsiblySoftware verificationUnsuitable divergentuseUnsuitableorinappropriate

31、outcomesConsiderbalancebetweenuseful/positivedivergentuseandinappropriateusage,inherentdesignSoftware verificationDehumanizationofhumansintherelationshipwithrobotsInappropriatecontrolexercisedbytherobotLossofrespectforhumandignity,dehumanizationandhumanrightsInbuiltlimits,designandimplementationprin

32、ciplesandpracticeUservalidation;expertguidanceInappropriate “trust”ofahumanbyrobotMalignorinadequatehumancontrolModelofappropriatehumancontrolDesignrobottoreachsafestatewithrespecttoanyothertaskstherobotisexecutingSoftware verification;uservalidationSelf-learningsystemexceedingitsremitInappropriateu

33、seofresourcesLimitsonextentofself-learningwithouthumanmonitoringKeepingthehumanintheloopisusuallyadvisableSoftware verificationCommercial/ FinancialApprobationoflegalresponsibilityandauthorityFailuretomeetfaircontractconditionsleadingtoillegality,inappropriateactions,avoidanceofresponsibilitiesAtten

34、tiontocontracts,recognizingthecharacteristicsoftherobot,itsintendeddomainofapplicationanditsoperatorsAnissueinvolvingcorporatesocialresponsibility,entraininglegalandsocialconsiderationsSoftware verification;expertguidanceBRITISH STANDARDBS 8611:20164 The British Standards Institution 2016Table1Ethic

35、alissues,hazardsandrisksEthicalissueEthicalhazardEthicalriskMitigationCommentVerification/ValidationCommercial/ FinancialEmploymentissuesJobreplacement,jobchange,unemployment,lossoftaxrevenueAppropriatesupportnetworks,appropriatetaxation,retrainingopportunitiesNoteliteratureonroboteconomicsEconomica

36、ndsocialassessmentEqualityofaccessPropagationofthe“digitaldivide”,isolationofminorities,non-compliancewithhumanrightslegislationInclusivedesignofrobotbehaviourtoconformwithCorporateSocialResponsibility,andrecognitionofcharacteristicsofintendedapplicationdomainSupportnetworkstominimizerisksLegalasses

37、sment;software verificationLearningbyrobotsthathavesomedegreeofbehavioural autonomyRobotmightdevelopneworamendedactionplans,oromitstepsinprocesses,thatcouldhaveunforeseenconsequencesforsafetyand/orqualityofoutcomesDesignerscanenablerobotstoinformtheiroperatorswhenanewformofbehaviourhasbeendevelopedI

38、mportantconsiderationforhuman-robottrustSoftware verification;uservalidationInformedconsentUnawareoperatorscausingaccidents,unwantedconsequences,unfairandinequitableresponsibilitiesplaceduponconsenter,inabilitytorespondtosituationsDesigncontractstomakeexplicit,inplainlanguage,whattheconsenterisagree

39、ingto,asfarastherobotisconcerned,includingrisksAlso,contractstomakeplainwhatpersonalorprivateinformationcouldbepassedontothenetwork(s)towhichtherobotisconnectedInprinciple,theactionsofsetsofrobots,especiallywhennetworked,isnearlyunlimited Thereareprivacyandsecurityissuesthatmightariseasrobotsworking

40、incloseco-ordinationwithhumansbecomemoreinformedabouthumanbehaviourUservalidationBRITISH STANDARD BS 8611:2016 The British Standards Institution 2016 5Table1Ethicalissues,hazardsandrisksEthicalissueEthicalhazardEthicalriskMitigationCommentVerification/ValidationCommercial/ FinancialInformedcommandUs

41、erisunawareofextentorlegalityorsocialacceptanceofthetasksgiventotherobot,consequencesofthetasksmightramifyinunexpectedwaysandextentsAsaprinciple,robotsoughttoonlyactonthebasisofaproperly-constructed command.Iftherobotcannotconstruesuchacommandfromthehumanscommunication,itoughttoseekfurtherguidanceAg

42、enerallyacceptedprincipleisthat“thelastpersontoissueacommandisresponsibleforitsoutcome”.Forequity,safetyandsecurity,itisnecessarytoensurethatoperatorsareasinformedaspossibleaboutthecapabilitiesandlimitationsoftherobotsthattheyutilizeA)Software verificationEnvironmentalEnvironmental awareness(robotan

43、dappliances)CauseconcernsaboutwastageanddestructionoftheenvironmentFailuretoconformtoregulationsand/orcodesofpracticedesignedtoprotecttheenvironmentresultinginharmtothesocietyAdoptgood/bestmanagementpracticeindesign(seeBSENISO9000series)andsupportforrecyclingandothercirculareconomyactivitiesLikelyto

44、beofsteadily-increasing importanceLifecycle compliancetestsEnvironmental awareness (operationsandapplications)Executionofnon-sustainableactions,harmtolocalsituation,reputationalharmGood/bestmanagementpractice(seeBSENISO14000series)builtintocontractterms,designofrobotsensingtoensurerecognitionandlabe

45、llingofphysicalenvironment,and,particularlyinrelationtopick-and-placeactions,sourcesandsinksareknownCompliancetestsA)Forexamplesof“properly-constructedcommands”,designersandoperatorscanconsultNATOsC-BML(CoalitionBattleManagementLanguage).See5.1.12.BRITISH STANDARDBS 8611:20166 The British Standards

46、Institution 20164.2 Ethical hazard identificationThe concept for a new robot or application of an existing robot should bereviewed to identify which groups of humans or animals are likely to beaffected by it. There might be ethical harm to more groups than the immediateusers. The range of ethical ha

47、zards might vary depending on the intended useof the robot and the perception and background of the subjects. Newapplications for existing robots can result in new and different ethical hazards.The principles and techniques for responsible research and innovation (RRI) asdefined in BS ISO 31000 shou

48、ld be adapted and followed.BS EN ISO 12100:2010 and ISO/TR 14121-2 provide requirements and guidanceon performing risk assessments, including risk analysis which focuses on hazardidentification. The guidance contained in BS EN ISO 12100:2010, Clause 5 hasbeen derived from the iterative process of ap

49、plying safeguarding measures (inaccordance with the general principles for designing robots given in thatstandard) to hazards identified as ergonomic hazards. When considering theoverlap with ethical hazards, the same procedure should be applied. The sourceof an ergonomic hazard could be, for example, the design or location ofindicators and visual display units with the consequence of discomfort due tomisunderstanding of the robot.Ethical hazards could be considered as part of the ergonomic hazards that canbe present in mobile robots