1、 TECHNICAL REPORT T1.TR.83-2003 Technical Report on Performance Parameters for IP-based Applications Prepared by T1A1.3 Working Group on Performance of Networks and Services Problem Solvers to the Telecommunications Industry A Word from ATIS and Committee T1 Established in February 1984, Committee T
2、1 develops technical standards, reports and requirements regarding interoperability of telecommunications networks at interfaces with end-user systems, carriers, information and enhanced-service providers, and customer premises equipment (CPE). Committee T1 is sponsored by ATIS and is accredited by
3、ANSI. T1.TR.83-2003 Published by Alliance for Telecommunications Industry Solutions 1200 G Street, NW, Suite 500 Washington, DC 20005 Committee T1 is sponsored by the Alliance for Telecommunications Industry Solutions (ATIS) and accredited by the American National Standards Institute (ANSI). Copyrig
4、ht 2004 by Alliance for Telecommunications Industry Solutions All rights reserved. No part of this publication may be reproduced in any form, in an electronic retrieval system or otherwise, without the prior written permission of the publisher. For information contact ATIS at 202.628.6380. ATIS is o
5、nline at . Printed in the United States of America. T1.TR.83-2003 Technical Report on Performance Parameters for IP-based Applications Secretariat Alliance for Telecommunications Industry Solutions Approved June 1, 2004 Abstract Many factors can affect the quality and the ultimate user experience of
6、 IP-based applications such as e-mail, web hosting, and streaming media. Some factors are transparent to the user, while others are not, so it is critical to identify and define parameters that are useful and meaningful. This Technical Report (TR) presents preliminary parameter definitions that capt
7、ure the user-oriented performance of several IP-based applications, with a goal of socializing the need for more detailed development and eventual standardization of such parameters. T1.TR.83-2003 ii FOREWORD This Technical Report (TR) identifies and defines parameters that capture the user-oriented
8、 performance of IP-based applications such as e-mail, web hosting, and streaming media. The goal is to promote the development and eventual standardization of appropriate measurements that include an agreed-upon list of parameters. This TR is intended for providers of IP-based telecommunications net
9、works and services, communications equipment suppliers, and users of such services and equipment. It was developed by the Performance of Networks and Services Working Group of the Network Performance, Reliability, and Quality of Service Committee (formerly the T1A1.3 Working Group of Technical Subco
10、mmittee T1A1, Committee T1). Feedback on this report is welcome and should be sent to the Alliance for Telecommunications Industry Solutions, Suite 500, 1200 G Street N.W., Washington, D.C. 20005. TABLE OF CONTENTS 1 INTRODUCTION, SCOPE, PURPOSE, AND APPLICATION. 1 1.1 INTRODUCTION 1 1.2 SCOPE 1 1.3
11、 PURPOSE 1 1.4 APPLICATION . 1 2 ACCESS TO IP-BASED APPLICATIONS 2 3 WEB HOSTING PERFORMANCE USER PERSPECTIVE 2 3.1 WEB SERVICE ACCESSIBILITY 2 3.1.1 Network/Server Accessibility . 3 3.1.2 Page Accessibility 3 3.2 WEB SERVICE CONTINUITY 3 3.3 WEB SERVICE FULFILLMENT 3 3.4 OPEN ISSUES WEB HOSTING (US
12、ER PERSPECTIVE). 4 3.4.1 Web Access Methods 4 3.4.2 Size 2) web hosting; 3) and streaming media. The parameters provided here may also prove useful when considering other applications, and while it is hoped that this report will lead to such consideration, the scope of this report is limited to the
13、three applications given here. 1.3 Purpose This report identifies and defines examples of parameters that capture the user-oriented performance of IP-based applications related to e-mail, web hosting, and streaming media. The goal is to promote the development and eventual standardization of appropr
14、iate measurements that use an agreed-upon list of well-defined and well-accepted parameters, leading to more consistent and thorough reporting of IP-based service performance. Accordingly, this initial report on the subject is quite general and it is recognized that a more detailed development (with
15、 better definitions) is needed for formal standardization. 1.4 Application The performance parameters identified and defined in this technical report are preliminary and intended to be used on a voluntary basis by providers of IP-based networks and services, T1.TR.83-2003 2 communications equipment
16、suppliers, and users of such services and equipment. Such use is intended primarily to provide the experiential basis for a future, more detailed standard on this subject. 2 ACCESS TO IP-BASED APPLICATIONS This clause applies to the access of all applications described subsequently in this report. C
17、urrently, the three most popular methods of connecting to an ISP for web access are: 1. Dialup: User is connected to an ISP via regular phone line at a maximum speed of 56Kbps. 2. DSL: A user with DSL access typically has a speed of around 640Kbps. 3. Broadband: This type of access includes cable mo
18、dem and Local Area Network (LAN), where the typical connection speed is 1Mbps and above. In the case of “always on” service it is assumed that access to IP-based applications is not an issue, except for any concerns described below associated with a specific application. In the case of “dial up,” th
19、ere are some specific aspects to access that require discussion. The three steps involved in accessing Internet service by dial-up are: 1. Line and modem accessibility: The line connection and the dial up modem connection are available. Performance Parameter: Failure rate for line and modem accessib
20、ility. 2. Modem Synchronization: The dialed modem and the users modem synchronized successfully. Performance Parameter: Failure rate for modem synchronization. 3. IP/PPP setup and user authentication: The IP and PPP setup is made, and the user is authenticated. Performance Parameter: Failure rate fo
21、r IP/PPP setup and authentication. 3 WEB HOSTING PERFORMANCE USER PERSPECTIVE The user perspective of web hosting performance focuses on accessing and obtaining content from a web site. A user is not concerned with where the web site is or how the web access request travels to the web server. There
22、are three main components of user perspective for web hosting performance. They are discussed in the following clauses. 3.1 Web Service Accessibility Web service accessibility is divided into two sub categories: 1) network/server accessibility; and 2) page accessibility. They are discussed below. T1
23、.TR.83-2003 3 3.1.1 Network/Server Accessibility There are two parameters under this category that either pass or fail when a user makes a web request: 1. DNS Lookup Failure Rate: Once the user makes a web request, the web browser attempts to resolve the web site name to an IP address by contacting
24、the Domain Name System (DNS). The failure to reach DNS or resolve the name within a specified time results in DNS lookup failure. 2. TCP Connection Timeout Rate: After the DNS request is fulfilled, the web browser attempts to make a connection to that web site. If the connection acknowledgement is n
25、ot received within the timeout window set by the browser, a timeout occurs and the web request is aborted. 3.1.2 Page Accessibility This category has two parameters that either pass or fail. They are defined as follows: 1. “Page Not Found” Rate: The server returns a “page not found” error when the b
26、rowsers request for the base page cannot be fulfilled. This could be caused by request for a nonexistent page or the expiration of cached content that could not be refreshed from the origin server. 2. Page Timeout Rate: The browser waits for a certain amount of time after issuing a request for a web
27、 page. If that request is not fulfilled within this time window, a page timeout occurs. 3.2 Web Service Continuity Web service continuity is defined as users ability to successfully receive the base page and all the content on that base page. This includes all the objects and images on that page. Th
28、ere are three components in this category. Connection Reset Rate: During the web page download, if the web server does not receive appropriate response from the browser, it resets and closes the TCP connection. The web request is aborted. Content Timeout Rate: Once the browser has successfully estab
29、lished a connection to a server and is downloading the web content, if the browser timeout threshold for server response is exceeded, there is a content timeout. NOTE - Page timeouts, which were discussed earlier, may occur before any web content has been received from the server. Content timeout ma
30、y happen only after some content has been received from the web server. Content Error Rate: A content error is defined as the failure of the browser to receive a complete web page without any errors. A partial page download without all the embedded objects or images in the base page results in a con
31、tent error. 3.3 Web Service Fulfillment Web service fulfillment tracks time associated with different components of a successful web request. The user fulfillment experience is directly impacted by each component and also the total time for all the components. These components are: DNS Lookup Time:
32、The time it takes to properly resolve a web site IP address from a Uniform Resource Locator (URL). T1.TR.83-2003 4 TCP Connect Time: The time needed for a host to connect to the web server, or for a web server to connect to a host. Time to Get First Byte: The time between the transmission of a reque
33、st for content from the browser, to the receipt of the first byte from the server is the time to first byte. It measures the web servers latency. Page Download Time: The amount of time needed to download the entire page of a web site including any embedded object and images. Response Time: The total
34、 time it takes to fulfill a web request. This is obtained by adding DNS lookup time, TCP connection time, time to get first byte, and page download time. 3.4 Open Issues Web Hosting (User Perspective) There are three issues that must be taken into account when considering web service fulfillment. Th
35、ese issues - web access methods, size & composition of a web page, and presence of a web proxy - are described below. 3.4.1 Web Access Methods A users web service fulfillment is directly related to the access method used to connect to the Internet (see clause 2). A users expectation for web service
36、fulfillment will be dependent on the connection method used. For a given web site, the web service fulfillment expectation for cable modem or a LAN user would be higher than that of a dialup user. A method could be defined to normalize the results based on access method used. 3.4.2 Size & Compositio
37、n of a Web Page Web service fulfillment would be affected by the size and composition of a web page. A large web page as measured by the total byte count of the page would have a larger page download time than a similarly composed page with a smaller size. Similarly, a web page with many embedded ob
38、jects and images would have a larger page download time than the one with few or no embedded objects and images. A method that normalizes the results based on web page size may address these concerns. 3.4.3 Presence of a Web Proxy When a web proxy is present in the path between a host and web server
39、, the proxy terminates the hosts TCP connections, and establishes a corresponding connection to the web server. The proxys intervention will improve some aspects of fulfillment, by quickly establishing TCP connections and completing some resource requests from information stored in its cache. As a r
40、esult, a test that repeats a request for a web page may only measure the proxys performance once the proxy retrieves the needed resources from the web server. The presence of a web proxy must be noted in any results reporting. T1.TR.83-2003 5 4 IP-BASED E-MAIL PERFORMANCE Each Internet Service Provi
41、der (ISP) has a set of computers running mail applications within its network. An Internet mail server is a software application that transmits and receives mail between the mail servers using TCP/IP. The mail server also works with client software on the users computers to send and receive messages
42、 from the users. The major steps involved in the transaction are: Accessing the Internet service. Sending a fixed sized e-mail. Receiving a fixed sized e-mail. Each of the three major steps is broken down into minor steps that the system performs, sometimes transparent to the user, which can be moni
43、tored. Listed below are the major and minor steps, and the associated performance parameters. 4.1 Accessing the E-Mail Service As noted in clause 2, access may be dial-up or broadband (always-on), with dial-up having the most performance issues that can be especially important to e-mail usage. 4.2 S
44、ending a Fixed Size E-Mail The three steps involved in sending a fixed size e-mail are: 1. Mail server connectivity: The client software residing on the users computer is able to connect to the mail server designated for sending outbound mail. Performance Parameter: Failure rate for Mail server conn
45、ectivity. 2. Completion of mail sending: Mail transfer from the client to the mail server is successful. Performance Parameter: Failure rate for mail sending completion. 3. Throughput for sending mail: Mail transfer from the client to the mail server occurs within a specified amount of time. The tim
46、e is measured from the beginning of message transmittal to the time the mail server confirmation is received. Performance Parameter: Failure rate for the mail transfer (sending), if it does not complete within the specified amount of time. 4.3 Receiving a Fixed Size E-Mail The four steps involved in
47、 receiving a fixed size e-mail are: 1. Mail server connectivity: The client software residing on the users computer is able to connect to the mail server designated for receiving mail. T1.TR.83-2003 6 Performance Parameter: Failure rate for mail server connectivity. 2. Mail server authentication: Th
48、e client software residing on the users computer is able to authenticate to the mail server designated for receiving mail. Performance Parameter: Failure rate for mail server authentication. 3. Completion of receiving mail: Mail transfer from the client to the mail server is successful. Performance
49、Parameter: Failure rate for mail receiving completion. 4. Throughput for receiving mail: Mail transfer from the mail server to the client occurs within a specified amount of time. The time is measured from the initial request transmittal to the time the last message bit is received. Performance Parameter: Failure rate for the mail transfer (receiving), if it does not complete within the specified amount of time. 4.4 Open Issues Internet E-Mail Performance There are three issues that must be taken into account when
