1、 ETSI GS NFV-REL 002 V1.1.1 (2015-09) Network Functions Virtualisation (NFV); Reliability; Report on Scalable Architectures for Reliability Management Disclaimer This document has been produced and approved by the Network Functions Virtualisation (NFV) ETSI Industry Specification Group (ISG) and rep
2、resents the views of those members who participated in this ISG. It does not necessarily represent the views of the entire ETSI membership. GROUP SPECIFICATION ETSI ETSI GS NFV-REL 002 V1.1.1 (2015-09)2 Reference DGS/NFV-REL002 Keywords architecture, NFV, reliability ETSI 650 Route des Lucioles F-06
3、921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N 348 623 562 00017 - NAF 742 C Association but non lucratif enregistre la Sous-Prfecture de Grasse (06) N 7803/88 Important notice The present document can be downloaded from: http:/www.etsi.org/standards-searc
4、h The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any existing or perceived difference in contents between
5、such versions and/or in print, the only prevailing document is the print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat. Users of the present document should be aware that the document may be subject to revision or change of status. Information
6、 on the current status of this and other ETSI documents is available at http:/portal.etsi.org/tb/status/status.asp If you find errors in the present document, please send your comment to one of the following services: https:/portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No
7、part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF version shall not be modified without the written authorization of ETSI. The copyright and the for
8、egoing restriction extend to reproduction in all media. European Telecommunications Standards Institute 2015. All rights reserved. DECTTM, PLUGTESTSTM, UMTSTMand the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members. 3GPPTM and LTE are Trade Marks of ETSI registered for the
9、 benefit of its Members and of the 3GPP Organizational Partners. GSM and the GSM logo are Trade Marks registered and owned by the GSM Association. ETSI ETSI GS NFV-REL 002 V1.1.1 (2015-09)3 Contents Intellectual Property Rights 4g3Foreword . 4g3Modal verbs terminology 4g31 Scope 5g32 References 5g32
10、.1 Normative references . 5g32.2 Informative references 6g33 Definitions and abbreviations . 7g33.1 Definitions 7g33.2 Abbreviations . 7g34 Scalable Architecture and NFV 8g34.1 Introduction 8g34.2 Overview of Current Adoption in Cloud Data Centres 9g34.3 Applicability to NFV 9g35 Scaling State . 10g
11、35.1 Context . 10g35.2 Categories of Dynamic State 13g35.3 Challenges 14g36 Methods for Achieving High Availability 15g36.1 High Availability Scenarios . 15g36.2 Dynamic Scaling with Migration Avoidance . 16g36.3 Lightweight Rollback Recovery . 20g36.3.1 Overview 20g36.3.2 Checkpointing . 21g36.3.3
12、Checkpointing with Buffering 22g36.3.4 Checkpointing with Replay 23g36.3.5 Summary Trade-offs of Rollback Approaches . 24g37 Recommendations 24g37.1 Conclusion 24g37.2 Guidelines for Scalable Architecture Components . 24g37.3 Future Work . 25g3Annex A (informative): Experimental Results . 26g3A.1 Mi
13、gration Avoidance Results . 26g3A.2 Lightweight Rollback Recovery Results 27g3A.2.1 Introduction 27g3A.2.2 Latency . 28g3A.2.3 Throughput . 29g3A.2.4 Replay Time . 29g3A.2.5 Conclusion 30g3Annex B (informative): Authors Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI st
14、andards“, which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (http:/ipr.etsi.org). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not r
15、eferenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Foreword This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Network Functions Virtualisation (NFV). Modal verbs termino
16、logy In the present document “shall“, “shall not“, “should“, “should not“, “may“, “need not“, “will“, “will not“, “can“ and “cannot“ are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). “must“ and “must not“ are NOT allowed in E
17、TSI deliverables except when used in direct citation. ETSI ETSI GS NFV-REL 002 V1.1.1 (2015-09)5 1 Scope The present document describes a study of how todays Cloud/Data Centre techniques can be adapted to achieve scalability, efficiency, and reliability in NFV environments. These techniques are desi
18、gned for managing shared processing state with low-latency and high-availability requirements. They are shown to be application-independent that can be applied generally, rather than have each VNF use its own idiosyncratic method for meeting these goals. Although an individual VNF could manage its o
19、wn scale and replication, the techniques described here require a single coherent manager, such as an orchestrator, to manage the scale and capacity of many disparate VNFs. Todays IT/Cloud Data Centres exhibit very high availability levels by limiting the amount of unique state in a single element a
20、nd creating a virtual network function from a number of small replicated components whose functional capacity can be scaled in and out by adjusting the running number of components. Reliability and availability for these type of VNFs is provided by a number of small replicated components. When an in
21、dividual component fails, little state is lost and the overall VNF experiences minimal change in functional capacity. Capacity failures can be recovered by instantiating additional components. The present document considers a variety of use cases, involving differing levels of shared state and diffe
22、rent reliability requirements; each case is explored for application-independent ways to manage state, react to failures, and respond to increased load. The intent of the present document is to demonstrate the feasibility of these techniques for achieving high availability for VNFs and provide guida
23、nce on Best Practices for scale out system architectures for the management of reliability. As such, the architectures described in the present document are strictly illustrative in nature. Accordingly, the scope of the present document is stated as follows: Provide an overview of how such architect
24、ures are currently deployed in Cloud/Data Centres. Describe various categories of state and how scaling state can be managed. Describe scale-out techniques for instantiating new VNFs in a single location where failures have occurred or unexpected traffic surges have been experienced. Scale-out may b
25、e done over multiple servers within a location or in a server in the same rack or cluster within any given location. Scaling out over servers in multiple locations can be investigated in follow-up studies. Develop guidelines for monitoring state such that suitable requirements for controlling elemen
26、ts (e.g. orchestrator) can be formalized in follow-up studies. 2 References 2.1 Normative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific r
27、eferences, the latest version of the referenced document (including any amendments) applies. Referenced documents which are not found to be publicly available in the expected location might be found at http:/docbox.etsi.org/Reference. NOTE: While any hyperlinks included in this clause were valid at
28、the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are necessary for the application of the present document. Not applicable. ETSI ETSI GS NFV-REL 002 V1.1.1 (2015-09)6 2.2 Informative references References are either specific (identified by d
29、ate of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. NOTE: While any hyperlinks included in this clause were
30、 valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. i.1 R. Strom and S. Yemini: “Optimistic Recovery in D
31、istributed Systems“, ACM Transactions on Computer Systems, 3(3):204-226, August 1985. i.2 Sangjin Han, Keon Jang, Dongsu Han and Sylvia Ratnasamy: “A Software NIC to Augment Hardware“, in Submission to 25thACM Symposium on Operating Systems Principles (2015). i.3 E.N. Elnozahy, Lorenzo Alvisi, Yi-Mi
32、n Wang, David Johnson: “A Survey of Rollback-Recovery Protocols in Message-Passing Systems“, ACM Computing Surveys, Vol. 34, Issue 3, September 2002, pages 375-408. i.4 B. Cully, G. Lefebvre, D. Meyer, M. Feeley, N. Hutchinson and A. Warfield: “Remus: High Availability via Asynchronous Virtual Machi
33、ne Replication“. In Proceedings USENIX NSDI, 2008. i.5 Kemari Project. NOTE: Available at http:/ i.6 J. Sherry, P. Gao, S. Basu, A. Panda, A. Krishnamurthy, C. Macciocco, M. Manesh, J. Martins, S. Ratnasamy, L. Rizzo and S. Shenker: “Rollback Recovery for Middleboxes“, Proceedings of the ACM, SIGCOM
34、M, 2015. i.7 ETSI NFV Reliability Working Group Work Item DGS/NFV-REL004 (V0.0.5), June 2015: “Report on active Monitoring and Failure Detection in NFV Environments“. i.8 OPNFV Wiki: “Project: Fault Management (Doctor)“. NOTE: Available at https:/wiki.opnfv.org/doctor. i.9 E. Kohler et al.: “Click M
35、odular Router“, ACM Transactions on Computer Systems, August 2000. i.10 “Riverbed Completes Acquisition of Mazu Networks“. NOTE: Available at: http:/ i.11 Digital Corpora: “2009-M57-Patents packet trace“. i.12 S. Rajagopalan et al.: “Pico Replication: A High Availability Framework for Middleboxes“,
36、Proceedings of ACM SoCC, 2013. i.13 Remus PV domU Requirements. NOTE: Available at http:/wiki.xen.org/wiki/Remus_PV_domU_requirements. i.14 B. Cully et al.: “Remus: High Availability via Asynchronous Virtual Machine Replication“, Proceedings USENIX NSDI, 2008. i.15 Lee D. and Brownlee N.: “Passive M
37、easurement of One-way and Two-way Flow Lifetimes“, ACM SIGCOMM Computer Communications Review 37, 3 (November 2007). ETSI ETSI GS NFV-REL 002 V1.1.1 (2015-09)7 3 Definitions and abbreviations 3.1 Definitions For the purposes of the present document, the following terms and definitions apply: affinit
38、y: for the purposes of the present document, property whereby a flow is always directed to the VNF instance that maintains the state needed to process that flow checkpoint: snapshot consisting of all state belonging to a VNF; required to make an identical “copy“ of the running VNF on another system
39、NOTE: One way to generate a checkpoint is by using memory snapshotting built in to the hypervisor. core: independent processing unit within a CPU which executes program instructions correct recovery: A system recovers correctly if its internal state after a failure is consistent with the observable
40、behaviour of the system before the failure. NOTE: See i.1 for further details. flow: sequence of packets that share the same 5-tuple: source port and IP address, destination port and IP address, and protocol non-determinism: A program is non-deterministic if two executions of the same code over the
41、same inputs may generate different outputs. NOTE: Programs which when given the same input are always guaranteed to produce the same output are called deterministic. stable storage: memory, SSD, or disk storage whose failure conditions are independent of the failure condition of the VNF; stable stor
42、age should provide the guarantee that even if the VNF fails, the stable storage will remain available state: contents of all memory required to execute the VNF, e.g. counters, timers, tables, protocol state machines thread: concurrent unit of execution, e.g. p-threads or process.h threads 3.2 Abbrev
43、iations For the purposes of the present document, the following abbreviations apply: CDF Cumulative Distribution Function CPU Central Processing Unit DDoS Distributed Denial of Service DHCP Dynamic Host Configuration Protocol DPDK Data Plane Development Kit DPI Deep Packet Inspection FTMB Fault Tole
44、rant MiddleBox Gbps Giga bits per second HA High AvailabilityIDS Intrusion Detection System IP Internet Protocol Kpps Kilo packets per second Mpps Mega packets per second NAT Network Address Translation NFV Network Function Virtualisation NFVI Network Function Virtualisation Infrastructure NIC Netwo
45、rk Interface Controller NUMA Non Uniform Memory Access QoS Quality of Service TCP Transmission Control Protocol VF Virtual FunctionVM Virtual Machine ETSI ETSI GS NFV-REL 002 V1.1.1 (2015-09)8 VNF Virtualised Network Function VPN Virtual Private Network WAN Wide Area Network 4 Scalable Architecture
46、and NFV 4.1 Introduction Traditional reliability management in telecommunications networks typically depends on a variety of redundancy schemes. For example, spare resources may be designated in some form of standby mode; these resources are activated in the event of network failures such that servi
47、ce outages are minimized. Alternately, over-provisioning of resources may also be considered (active-active mode) such that if one resource fails, the remaining resources can still process traffic loads. The advent of Network Functions Virtualisation (NFV) ushered in an environment where the focus o
48、f telecommunications network operations shifted from specialized and sophisticated hardware with potentially proprietary software functions residing on them towards commoditized and commercially available servers and standardized software that can be loaded up on them on an as needed basis. In such
49、an environment, Service Providers can enable dynamic loading of Virtual Network Functions (VNF) to readily available servers as and when needed - this is referred to as “scaling out“ (see note). Traffic loads can vary with bursts and spikes of traffic due to external events; alternately network resource failures may reduce the available resources to process existing load adequately. The management of high a
