1、BSI Standards PublicationBS 8611:2016Robots and robotic devicesGuide to the ethical design andapplication of robots and roboticsystemsPublishing and copyright informationThe BSI copyright notice displayed in this document indicates when the documentwas last issued. The British Standards Institution
2、2016Published by BSI Standards Limited 2016ISBN 978 0 580 89530 2ICS 25.040.30The following BSI references relate to the work on this document:Committee reference AMT/-/2Draft for comment 15/30320088 DCPublication historyFirst published April 2016Amendments issued since publicationDate Text affected
3、BS 8611:2016 BRITISH STANDARDContents1 Scope 12 Normative references 13 Terms and definitions 14 Ethical risk assessment 25 Ethical guidelines and measures 86 Ethics-related system design recommendations 137 Verification and validation 148 Information for use 16Bibliography 20List of tablesTable 1 E
4、thical issues, hazards and risks 3Summary of pagesThis document comprises a front cover, an inside front cover, pages i to ii,pages 1 to 22, an inside back cover and a back cover.BRITISH STANDARD BS 8611:2016 The British Standards Institution 2016 iForewordPublishing informationThis British Standard
5、 is published by BSI Standards Limited, under licence fromThe British Standards Institution, and came into effect on 30 April 2016. ThisBritish Standard has been prepared by Technical Subcommittee AMT/-/2, Robotsand robotic devices. A list of organizations represented on this committee canbe obtaine
6、d on request to its secretary.Information about this documentThis is the first edition of a new standard. It addresses issues in a field oftechnology that is rapidly changing and influencing society. Although effortshave been made to incorporate the anticipated effects of future developmentsin the s
7、tandard, it is not possible for all future circumstances to have beenconsidered or covered, for example non-embodied autonomous systems.Use of this documentAs a guide, this British Standard takes the form of guidance andrecommendations. It should not be quoted as if it were a specification or acode
8、of practice and claims of compliance cannot be made to it.Presentational conventionsThe guidance in this standard is presented in roman (i.e. upright) type. Anyrecommendations are expressed in sentences in which the principal auxiliaryverb is “should”.Commentary, explanation and general informative
9、material is presented insmaller italic type, and does not constitute a normative element.Contractual and legal considerationsThis publication does not purport to include all the necessary provisions of acontract. Users are responsible for its correct application.Compliance with a British Standard ca
10、nnot confer immunity from legalobligations.BRITISH STANDARDBS 8611:2016ii The British Standards Institution 20161 ScopeThis British Standard gives guidance on the identification of potential ethicalharm and provides guidelines on safe design, protective measures andinformation for the design and app
11、lication of robots. It builds on existing safetyrequirements for different types of robots; industrial, personal care and medical.This British Standard describes ethical hazards associated with the use of robotsand provides guidance to eliminate or reduce the risks associated with theseethical hazar
12、ds. Significant ethical hazards are presented and guidance given onhow they are to be dealt with for various robot applications.Ethical hazards are broader than physical hazards. Most physical hazards haveassociated psychological hazards due to fear and stress. Thus, physical hazardsimply ethical ha
13、zards and safety design features are part of ethical design.Safety elements are covered by safety standards; this British Standard isconcerned with ethical elements.This British Standard is intended for use by designers and managers, amongstothers.2 Normative referencesThe following documents, in wh
14、ole or in part, are normatively referenced in thisdocument and are indispensable for its application. For dated references, onlythe edition cited applies. For undated references, the latest edition of thereferenced document (including any amendments) applies.BS EN ISO 12100:2010, Safety of machinery
15、 General principles for design Riskassessment and risk reduction (ISO 12100:2010)BS ISO 8373, Robots and robotic devices - VocabularyBS ISO 31000, Risk management Principles and guidelines3 Terms and definitionsFor the purposes of this British Standard, the terms and definitions given inBS ISO 8373
16、and the following apply.3.1 autonomous systemsystem which has the ability to perform intended tasks based on current state,knowledge and sensing, without human intervention3.2 ethical harmanything likely to compromise psychological and/or societal and environmentalwell-beingNOTE Examples of ethical
17、harm include stress, embarrassment, anxiety, addiction,discomfort, deception, humiliation, being disregarded. This might be experienced inrelation to a persons gender, race, religion, age, disability, poverty or many otherfactors.3.3 ethical hazardpotential source of ethical harm3.4 ethical riskprob
18、ability of ethical harm occurring from the frequency and severity ofexposure to a hazard3.5 ethicscommon understanding of principles that constrain and guide human behaviourBRITISH STANDARD BS 8611:2016 The British Standards Institution 2016 13.6 harminjury or damage to health3.7 responsible researc
19、h and innovation (RRI)process that seeks to promote creativity and opportunities for science andinnovation that are socially desirable and undertaken in the public interestSOURCE: EPSRC framework for responsible innovation1)3.8 roboticistperson actively engaged in a process leading to the developmen
20、t, production,installation, deployment or maintenance of a robot3.9 systemset of parts which, when interoperating, has behaviour that is not present in anyof the parts themselvesNOTE This definition includes systems of fleeting duration, such as cars passingeach other and behaving appropriately duri
21、ng this moment, to long-lived systemssuch as Voyager II.3.10 vulnerable peoplepeople having reduced physical, sensory or mental capabilities (e.g. partiallydisabled, elderly having some reduction in their physical and mentalcapabilities), or lack of experience and knowledge (e.g. children)SOURCE: BS
22、 EN 60335-1:2012+A11:2014, 3.Z.54 Ethical risk assessment4.1 GeneralEthical issues can be divided into societal, application, commercial/financial andenvironmental, as described in Clause 5 to Clause 8. Table 1 describes examplesof the ethical hazards and risks associated with these issues. See also
23、BS ISO 31000 for further guidance.1)See last viewed 4 April 2016.BRITISH STANDARDBS 8611:20162 The British Standards Institution 2016Table1Ethicalissues,hazardsandrisksEthicalissueEthicalhazardEthicalriskMitigationCommentVerification/ValidationSocietalLossoftrust(humanrobot)Robotnolongerusedorismisu
24、sed,abusedDesigntoensurereliabilityinbehaviourIfunexpectedbehaviouroccurs,ensuretraceabilitytohelpexplainwhathappenedUservalidationDeception (intentionalorunintentional)Confusion,unintended(perhapsdelayed)consequences,eventuallossoftrustAvoiddeceptionduetothebehaviourand/orappearanceoftherobotandens
25、uretransparencyofroboticnatureSoftware verification;uservalidation;expertguidanceAnthropo- morphizationMisinterpretationAvoidunnecessaryanthropomorphization Clarificationofintenttosimulatehumanornot,orintendedorexpectedbehaviourSeedeception(above)Useanthropomorphizationonlyforwell-defined,limitedand
26、socially-acceptedpurposesUservalidation;expertguidancePrivacyandconfidentialityUnauthorizedaccess,collectionand/ordistributionofdata,ingintothepublicdomainortounauthorized, unwarrantedentitiesClarityoffunctionControlofdata,justificationofdatacollectionanddistributionEnsureuserawarenessofdatamanageme
27、ntandobtaininformedconsentinappropriatecontextsPrivacybydesignDataencryption,storagelocation,adherencetolegislationSoftware verificationLackofrespectforculturaldiversityandpluralismLossoftrustinthedevice,embarrassment,shame,offenceAwarenessofculturalnormsincorporatedintoprogrammingOrganizational, pr
28、ofessional,regionalSoftware verification;uservalidationRobotaddictionLossofhumancapability,dependency,reductioninwillingnesstoengagewithothers,isolationRaiseawarenessofdependencyAdifficultarea,particularlyinrelationtovulnerablepeople CarefulevaluationofpotentialapplicationsisneededUservalidation;exp
29、ertguidanceBRITISH STANDARD BS 8611:2016 The British Standards Institution 2016 3Table1Ethicalissues,hazardsandrisksEthicalissueEthicalhazardEthicalriskMitigationCommentVerification/ValidationSocietalEmploymentSocialdislocation,jobreplacement, unemployment,lossofskills/needtore-skillTargeteddeployme
30、ntofrobotstounsocial,hazardousactivities,organizational;designEconomicandsocialassessmentApplicationMisuseAllinthiscolumnMinimizepotentialforunethicaluseviaethicalriskassessment AdherencetostandardsandregulationsInnovateresponsiblySoftware verificationUnsuitable divergentuseUnsuitableorinappropriate
31、outcomesConsiderbalancebetweenuseful/positivedivergentuseandinappropriateusage,inherentdesignSoftware verificationDehumanizationofhumansintherelationshipwithrobotsInappropriatecontrolexercisedbytherobotLossofrespectforhumandignity,dehumanizationandhumanrightsInbuiltlimits,designandimplementationprin
32、ciplesandpracticeUservalidation;expertguidanceInappropriate “trust”ofahumanbyrobotMalignorinadequatehumancontrolModelofappropriatehumancontrolDesignrobottoreachsafestatewithrespecttoanyothertaskstherobotisexecutingSoftware verification;uservalidationSelf-learningsystemexceedingitsremitInappropriateu
33、seofresourcesLimitsonextentofself-learningwithouthumanmonitoringKeepingthehumanintheloopisusuallyadvisableSoftware verificationCommercial/ FinancialApprobationoflegalresponsibilityandauthorityFailuretomeetfaircontractconditionsleadingtoillegality,inappropriateactions,avoidanceofresponsibilitiesAtten
34、tiontocontracts,recognizingthecharacteristicsoftherobot,itsintendeddomainofapplicationanditsoperatorsAnissueinvolvingcorporatesocialresponsibility,entraininglegalandsocialconsiderationsSoftware verification;expertguidanceBRITISH STANDARDBS 8611:20164 The British Standards Institution 2016Table1Ethic
35、alissues,hazardsandrisksEthicalissueEthicalhazardEthicalriskMitigationCommentVerification/ValidationCommercial/ FinancialEmploymentissuesJobreplacement,jobchange,unemployment,lossoftaxrevenueAppropriatesupportnetworks,appropriatetaxation,retrainingopportunitiesNoteliteratureonroboteconomicsEconomica
36、ndsocialassessmentEqualityofaccessPropagationofthe“digitaldivide”,isolationofminorities,non-compliancewithhumanrightslegislationInclusivedesignofrobotbehaviourtoconformwithCorporateSocialResponsibility,andrecognitionofcharacteristicsofintendedapplicationdomainSupportnetworkstominimizerisksLegalasses
37、sment;software verificationLearningbyrobotsthathavesomedegreeofbehavioural autonomyRobotmightdevelopneworamendedactionplans,oromitstepsinprocesses,thatcouldhaveunforeseenconsequencesforsafetyand/orqualityofoutcomesDesignerscanenablerobotstoinformtheiroperatorswhenanewformofbehaviourhasbeendevelopedI
38、mportantconsiderationforhuman-robottrustSoftware verification;uservalidationInformedconsentUnawareoperatorscausingaccidents,unwantedconsequences,unfairandinequitableresponsibilitiesplaceduponconsenter,inabilitytorespondtosituationsDesigncontractstomakeexplicit,inplainlanguage,whattheconsenterisagree
39、ingto,asfarastherobotisconcerned,includingrisksAlso,contractstomakeplainwhatpersonalorprivateinformationcouldbepassedontothenetwork(s)towhichtherobotisconnectedInprinciple,theactionsofsetsofrobots,especiallywhennetworked,isnearlyunlimited Thereareprivacyandsecurityissuesthatmightariseasrobotsworking
40、incloseco-ordinationwithhumansbecomemoreinformedabouthumanbehaviourUservalidationBRITISH STANDARD BS 8611:2016 The British Standards Institution 2016 5Table1Ethicalissues,hazardsandrisksEthicalissueEthicalhazardEthicalriskMitigationCommentVerification/ValidationCommercial/ FinancialInformedcommandUs
41、erisunawareofextentorlegalityorsocialacceptanceofthetasksgiventotherobot,consequencesofthetasksmightramifyinunexpectedwaysandextentsAsaprinciple,robotsoughttoonlyactonthebasisofaproperly-constructed command.Iftherobotcannotconstruesuchacommandfromthehumanscommunication,itoughttoseekfurtherguidanceAg
42、enerallyacceptedprincipleisthat“thelastpersontoissueacommandisresponsibleforitsoutcome”.Forequity,safetyandsecurity,itisnecessarytoensurethatoperatorsareasinformedaspossibleaboutthecapabilitiesandlimitationsoftherobotsthattheyutilizeA)Software verificationEnvironmentalEnvironmental awareness(robotan
43、dappliances)CauseconcernsaboutwastageanddestructionoftheenvironmentFailuretoconformtoregulationsand/orcodesofpracticedesignedtoprotecttheenvironmentresultinginharmtothesocietyAdoptgood/bestmanagementpracticeindesign(seeBSENISO9000series)andsupportforrecyclingandothercirculareconomyactivitiesLikelyto
44、beofsteadily-increasing importanceLifecycle compliancetestsEnvironmental awareness (operationsandapplications)Executionofnon-sustainableactions,harmtolocalsituation,reputationalharmGood/bestmanagementpractice(seeBSENISO14000series)builtintocontractterms,designofrobotsensingtoensurerecognitionandlabe
45、llingofphysicalenvironment,and,particularlyinrelationtopick-and-placeactions,sourcesandsinksareknownCompliancetestsA)Forexamplesof“properly-constructedcommands”,designersandoperatorscanconsultNATOsC-BML(CoalitionBattleManagementLanguage).See5.1.12.BRITISH STANDARDBS 8611:20166 The British Standards
46、Institution 20164.2 Ethical hazard identificationThe concept for a new robot or application of an existing robot should bereviewed to identify which groups of humans or animals are likely to beaffected by it. There might be ethical harm to more groups than the immediateusers. The range of ethical ha
47、zards might vary depending on the intended useof the robot and the perception and background of the subjects. Newapplications for existing robots can result in new and different ethical hazards.The principles and techniques for responsible research and innovation (RRI) asdefined in BS ISO 31000 shou
48、ld be adapted and followed.BS EN ISO 12100:2010 and ISO/TR 14121-2 provide requirements and guidanceon performing risk assessments, including risk analysis which focuses on hazardidentification. The guidance contained in BS EN ISO 12100:2010, Clause 5 hasbeen derived from the iterative process of ap
49、plying safeguarding measures (inaccordance with the general principles for designing robots given in thatstandard) to hazards identified as ergonomic hazards. When considering theoverlap with ethical hazards, the same procedure should be applied. The sourceof an ergonomic hazard could be, for example, the design or location ofindicators and visual display units with the consequence of discomfort due tomisunderstanding of the robot.Ethical hazards could be considered as part of the ergonomic hazards that canbe present in mobile robots
