1、 TECHNICAL REPORT ATIS-01000038 ANALYSIS OF LARGE DS3 FCC REPORTABLE OUTAGES As a leading technology and solutions development organization, ATIS brings together the top global ICT companies to advance the industrys most-pressing business priorities. Through ATIS committees and forums, nearly 200 co
2、mpanies address cloud services, device solutions, emergency services, M2M communications, cyber security, ehealth, network evolution, quality of service, billing support, operations, and more. These priorities follow a fast-track development lifecycle from design and innovation through solutions tha
3、t include standards, specifications, requirements, business use cases, software toolkits, and interoperability testing. ATIS is accredited by the American National Standards Institute (ANSI). ATIS is the North American Organizational Partner for the 3rd Generation Partnership Project (3GPP), a found
4、ing Partner of oneM2M, a member and major U.S. contributor to the International Telecommunication Union (ITU) Radio and Telecommunications sectors, and a member of the Inter-American Telecommunication Commission (CITEL). For more information, visit . Notice of Disclaimer Final Rule (FCC Outage Rules
5、).1The NRSC launched an investigation by opening Issue Statement 21 entitled Large DS3 Outage Investigation to address this concern. The Issue Statement defines the business need as: Quarterly reports presented to the NRSC and industry by the FCC Public and Homeland Security Bureau (PSHSB) indicated
6、 there was an upward trend in large DS3 outages, diminishing network reliability and raising questions regarding capacity and diversity. The NRSC, as part of its mission, agreed to analyze this trend and provide guidance to the industry to mitigate these outages. Figure 1 - Big DS3 Outages2During th
7、e August 17, 2011, NRSC meeting, the FCC PSHSB noted that the number of these events appeared to be on the rise. They speculated that the cause could be that facilities are getting very large, which would mean that the overall capacity of traffic on these facilities is also increasing. During the De
8、cember 7, 2011, NRSC meeting, the FCC PSHSB noted their data indicated that Big DS3 outages appeared to be increasing at a significant rate. In addition to asking the NRSC to investigate these types of outages, it was also noted that this upward trend resulted in the development of a number of quest
9、ions the FCC raised in FCC Docket #11-60, In the matter of reliability and continuity of Communications Networks, Including Broadband technologies effects on Broadband Communications Networks of Damage or Failure of Network equipment or severe overload Notice of 1Part II Federal Communications Commi
10、ssion 47 CFR Parts 0, 4, and 63 Disruptions to Communications; Final Rule December 3, 2004. 2FCC NORS Analysis presented at NRSC Public Meeting on December 7, 2011, Slide 15. ATIS-01000038 2 Inquiry. Specifically, the FCC noted that the numbers of large DS3 events were causing them to question wheth
11、er enough diversity is built into networks. Major points that the NRSC considered were: 1. Is the apparent increase due to network re-designs and larger capacity systems or is there a reliability issue? 2. Are these types of outages identifying a lack of diversity in portions of the network that are
12、 designed to have redundancy? 3. Is the number of large DS3 outages a significant percentage of the total number of DS3 outages? 4. Are the requirements of the existing Part 4 outage reporting rules adequate for DS3 failures given the evolution that has taken place in communications providers networ
13、ks, (e.g., increasing capacity of transport facilities)? 5. Is the standardized methodology for calculating the outage index (i.e., the level of customer impact), relevant for Large DS3 events? These major points are addressed below. 2 Introduction Communication providers are required by Part 4 Rule
14、s to report outages of at least 30 minutes duration that affect at least 1,350 DS3 minutes. “DS3 minutes” are defined as the mathematical result of multiplying the duration of an outage, expressed in minutes, by the number of previously operating DS3 circuits that were affected by the outage.3The NO
15、RS report form contains a field for the Number of Potentially Affected DS3s. This paper discusses the investigation process and findings of the NRSC from its examination of large DS3 outages as identified in NORS reports for which the reason reportable was 1,350 DS3 minutes from January 2010 through
16、 November 30, 2011. Large DS3 outages are described by the FCC as affecting greater than 1,000 DS3s. The FCCs concern about the high number of large DS3 outages occurring stems from the assumption that 666,000 customers are potentially affected, per the Outage Index equation described in ATIS-010002
17、1.2012, Analysis of FCC-Reportable Service Data. With the apparent increase in the number of large DS3 outage reports filed and the assumed customer impact, the NRSC understands that this would be of concern to the FCC. The NRSC, however, also considered these outages from a broader perspective, wit
18、h consideration being given to todays network realities when assessing customer impact and significance of the events. This paper also provides an analysis of NRSC member data which provides a significant, albeit partial, subset of the total number of outage reports filed by industry under the FCCs
19、Outage Rules and, where appropriate, makes recommendations to reduce or mitigate these types of outages. The NRSC considered relevant industry work related to capacity and diversity issues, changes in customer needs and purchasing decisions, and network evolution including large DS3 capacity systems
20、. 3 References At the time of publication, the editions indicated were valid. All standards are subject to revision, and the reader is encouraged to investigate the possibility of applying the most recent editions of the standards indicated below. 3Historically, the earliest capacity calculation of
21、a single Digital Signal 3 (DS3) was defined as 672 unique voice or data customers that are impacted by a failure of a DS3. This definition carries over the paradigm from the Outage Reporting rules prior to 2005 that focused on 30,000 potentially affected customers. The current calculation describes
22、a situation where each customer is allotted 64 Kilobits per second (Kbps) capacity on one of the 672 channels of available payload on the DS3.The result of this historical view of the usage of a DS3 overestimates the number of customers affected by the failed capacity of a single DS3 (i.e., 1 DS3 re
23、presents 672 customers). When this calculation is applied to a failure of 1,000 DS3s, the potential customer impact is implied to be 672,000 unique voice or data customers with a combined rate capacity of 5.333 GB. ATIS-01000038 3 1 Part II Federal Communications Commission 47 CFR Parts 0, 4, and 63
24、 Disruptions to Communications; Final Rule December 3, 2004.42 ATIS-0100021. 0100021, Analysis of FCC-Reportable Service Data, December 2012.53 FCC Network Outage Reporting System User Manual, Version 7, December 17, 2012.64 ATIS-I-0000041, National Diversity Assurance Initiative, February 2006.74 D
25、efinitions Final Rule DS0 Digital Signal 0 DS3 Digital Signal 3 Gbps Gigabit per second GB Gigabyte Kbps Kilobit per second Mbps Megabit per second NDAI National Diversity Assurance Initiative NORS Network Outage Reporting System NRSC Network Reliability Steering Committee OC3 Optical Carrier 3 PSHS
26、B Public Safety and Homeland Security Bureau TDM Time Division Multiplexing 5 Objective, Scope, If possible, member companies were asked to provide a complete data set of all 1,350 DS3 events regardless of the number of DS3s impacted; or Where complete data sets were not available, a subset of outag
27、es where: o Reason Reportable was 1,350 DS3s Minutes; and o DS3s reported equaled 1,000 or more. This data set was provided to the trusted third party entity under Non-Disclosure Agreement for analysis. This scope was adjusted later to include a month by month extension of the data range of January
28、2010 through July 2012 to continue investigation of ongoing trends in reporting related to the large DS3 events. 5.3 Methodology The team began work by using collected data to confirm a trend existed within NRSC member companies and to determine whether the trend aligned with the FCCs trend. Outage
29、data filed by service providers into the NORS system is protected and unavailable to the NRSC for routine evaluation. To perform any meaningful evaluation regarding these failures, member companies agree to provide their individual NORS filings to a trusted entity to preserve the protection of the r
30、eports and any sensitive information contained within the reports. The aggregated data was then used to develop the theories the team would explore and to further narrow the focus to the most prospective leads. Following the initial data review, each company performed an internal deep dive into the
31、top 80% of root and direct cause areas. This step in the investigation was essential in expanding on the details and trends behind the NORS reports to formulate the findings and recommendations. The deep dive was also used to discern if there was a specific product line, specific hardware element, o
32、r activity during a certain time of day that could support the causes of large DS3 outages. The team evaluated the outages from the perspective of network and technological changes to gauge how these might interrelate to the increases detected by the FCC over time. The theory is that larger capacity
33、 systems can rapidly exceed FCC reporting thresholds mathematically due to equivalent bandwidth calculations performed by service providers as they determine the quantity of DS3s. The theory postulates that as the network moves toward larger capacities, an increase in outages meeting FCC thresholds
34、can be expected to some magnitude. With the changes to technology, the issue of reliability and resiliency and the use of diversity were debated along with the ability of sophisticated enterprise customers to understand network design and associated risks. Prior work completed by other industry grou
35、ps was investigated to determine the relevance to this current issue and if that work could be incorporated into NRSC recommendations. One concept involved the time of day outages occur in the network and that through assessment it may be possible to estimate the most likely times a provider could p
36、otentially expect a large DS3 event to occur. The methodology of calculating the level of customer impact, or pain felt, for large DS3 events was also studied as it ATIS-01000038 6 relates to an outage index in use by industry and the FCC for several years. This calculation did not appear to scale c
37、orrectly with these larger capacity systems and overemphasized the actual impact. 6 Analysis others can be protected against single or multiple failures in various ways. Figure 3 below illustrates a mesh network where unprotected lightpaths are used to provide diversity9. In this example, an unprote
38、cted optical path is reserved and shared (S/T) and provides failover protection for bandwidth demands for optical paths A/B and C/D. Each primary path A/B and C/D also utilize unprotected wavelengths with path protection being provided through the use of the reserved unprotected path. 8using File Si
39、ze 100 GB and 0% overhead. 9ATIS-01000038 8 Figure 3 - Shared Backup Path Protection Before Failure Figure 4 illustrates the path diversity provided in the mesh arrangement if a failure occurs (C/D) and traffic is automatically rerouted across the reserved unprotected path (S/T)10. The mesh arrangem
40、ent reduces overall costs for customers while providing resilient and reliable network diversity. Figure 4 - Shared backup path protection after failure and recovery In diverse arrangements such as above, the failure of a single lightpath due to a planned or unplanned event does not impact customers
41、, but can qualify as a reportable condition under Part 4 rules. Without the understanding of this type of arrangement, the outage may appear to be significant in nature when in fact it is non-service affecting. The NRSC supports the dissemination of information to customers who purchase unprotected
42、lightpaths or circuits that outline risks and limitations related to network resiliency and diversity. A knowledgeable consumer is better able to evaluate their needs and risks for service and application diversity. The team also supports the view that customers with mission critical applications sh
43、ould have alternate or customer provided diversity and it is within their right to make the decision regarding the level of diversity deployed to support their services. 10ATIS-01000038 9 DIVERSITY ISSUES Part of the teams analysis focused on the discussion of diversity and whether the failure of a
44、large capacity system was causing significant customer impact. Three types or levels of diversity were defined relevant to the use of large capacity systems: Network Diversity Diversity Services Application Diversity The team reasoned that each type of diversity was distinctive and defined the respo
45、nsibility for ensuring that specific type or level of diversity as follows: Network Diversity This type of diversity references the practice of ensuring there are multiple physically diverse circuit routes to a critical facility. This level of diversity is exclusively within the responsibility of th
46、e service provider to manage and maintain. It refers to the network elements and facilities within the service providers span of control and ends at the demarcation point with other provider networks. Diversity Services This type of diversity references what a service provider sells to their custome
47、rs. The level of diversity is based on design, cost, and needs of the individual customer. The customer is responsible for deciding what level of diversity is utilized and/or built. The overall diversity is based on the customers decision to purchase or not purchase a certain service, decision to ut
48、ilize service from multiple providers, or to build a portion of the network themselves. Application Diversity This type of diversity is the diversity that the service providers customer provides to their customers. This level of diversity is exclusively within the responsibility of the customer and
49、is independent of the service provider. Customers develop their own diversity plan for applications that run over their physical networks. The large capacity system outages analyzed by the team were consistent with network diversity and diversity services. The team was also interested in reviewing the previous work by the ATIS National Diversity Assurance Initiative (NDAI) regarding their approach and recommendations for diversity. The NRSC was specifically interested in the following NDAI recommendation: “This guidance will provide customers with
