1、Bina Ramamurthy,10/10/2018,Copyright 2010 B. Ramamurthy,1,Distributed Systems,Introduction,10/10/2018,Copyright 2010 B. Ramamurthy,2,A distributed system is a network of autonomous computers cooperating to accomplish a task. Hardware and software components of a distributed system coordinate their a
2、ctivity by sharing resources such as data, computation, compute cycles, bandwidth and storage. Examples: Internet, intranet, grid and mobile computing systems.,Topics,10/10/2018,Copyright 2010 B. Ramamurthy,3,Some fundamental terms Advances in Client-side technologies Communication network Middlewar
3、e concept Server-side technology Application models (web, web2.0, web3.0),Evolution of Internet Computing,10/10/2018,4,CSE507 Introduction 2008,Publish,Inform,Interact,Integrate,Transact,Discover (intelligence),Automate (discovery),time,scale,?,Semantic discovery,Parallel HPC,web,deep web,Beyond Sea
4、rch Engines: Enabling Information Technology and Scientific Applications,10/10/2018,CSE507 Introduction 2008,5,Financial: Build Portfolio,Medicine: plan treatment,Environment: Plan Forestation,Biotech: drug discovery,Simple Search (stateless),Complex multi-organizational applications,TV/Remote,Wirel
5、ess device,Copyright 2010 B. Ramamurthy,10/10/2018,6,Terminologies,Protocol,10/10/2018,Copyright 2010 B. Ramamurthy,7,Protocol is a set of rules that end points in a telecommunication system use when exchanging information. IP: Internet protocol defines an unreliable packet transfer protocol. TCP: T
6、ransmission Control Protocol builds on IP to define a reliable data delivery protocol. LDAP: Lightweight Directory Access Protocol builds on TCP to define a query-response protocol for querying the state of a remote database. HTTP: Hyper Text Transfer Protocol builds on TCP to facilitate hyper-text
7、document exchange.,Service,10/10/2018,Copyright 2010 B. Ramamurthy,8,Service is a network-enabled entity that provides a specific capability. Service = Protocol + Behavior A service definition permits many implementations. Examples: ability to move files, create processes, verify access rights An FT
8、P server speaks File Transfer Protocol and supports remote read and write access to a collection of files.,API,10/10/2018,Copyright 2010 B. Ramamurthy,9,Application Program Interface (API) defines a standard interface for invoking a specified set of functionality. Examples: The Generic Security Serv
9、ice (GSS) API defines standard functions for verifying identity of communicating parties, encrypting messages and so forth.,SDK,10/10/2018,Copyright 2010 B. Ramamurthy,10,Software Development Kit (SDK) denotes a set of code designed to be linked with, and invoked from within, an application program
10、to provide specified functionality. An SDK typically implements an API. Example: Different SDKs implement GSS-API using the Kerberos or PKI protocols, respectively.,Internet,10/10/2018,Copyright 2010 B. Ramamurthy,11,Internet is a very large distributed system. Interconnection of a collection of het
11、erogeneous networks of computers. Protocols: IP, TCP, HTTP Services: world wide web (www), file transfers (ftp), email, etc.,Copyright 2010 B. Ramamurthy,10/10/2018,12,Advances in Client-side programming,Client programming,10/10/2018,Copyright 2010 B. Ramamurthy,13,Simple programs written as a singl
12、e module Single entry point typically in a “main” function Procedural, functional and object-oriented Applications, applets (web-based) More recently the focus is on rich content: Rich Internet Application (RIA) Adobe Flash, Adobe Flex, Microsoft Silverlight, Ajax Model-View-Controller (MVC) model f
13、or design and deployment flexibility. Ex: java swing, struts,Client/Server,10/10/2018,Copyright 2010 B. Ramamurthy,14,Server: refers to a process on a networked computer that accepts requests from other (local or remote) processes to perform a service and responds appropriately. Client: requesting p
14、rocess in the above is referred to as the client. Request and response are in the form of messages. Request/response model Client is said to invoke an operation on the server. Many distributed systems today are constructed out of interacting clients/servers. Issues: connectivity (speed and accessibi
15、lity), addressing, naming,Copyright 2010 B. Ramamurthy,10/10/2018,15,Advances in Networking,Internetworking,10/10/2018,Copyright 2010 B. Ramamurthy,16,Internet stack Standardization IPV4, IPV6: Internet protocol version 4, 6 Tremendous increase in network bandwidth: measured in bits per second From
16、few kilobits per second (56kb/s dial up lines to 1.5Mb/s T1 to 100Gb/s ethernet) What can you do with such fast delivery speed? Connect them up Networked application models,Communication Network,10/10/2018,Page 17,A communication middleware framework isolates the application developers from the deta
17、ils of the network protocol.,Application,Application,Network Protocol Stack,Network Protocol Stack,Copyright 2010 B. Ramamurthy,Client/server Issues,10/10/2018,Copyright 2010 B. Ramamurthy,18,Basic object-technology could not fulfill the promises such as reusability and interoperability fully in the
18、 context internet and enterprise level applications. Deployment was still a major problem and as a result portability and mobility were impaired.,Issues in networked systems,10/10/2018,Page 19,Heterogeneity in various aspects of a distributed systems Communication modes different Synchronous: RPC As
19、ynchronous: P2P, Publish and subscribe Variations in products Many vendors: IBM, IONA, TIBCO, Apache, Adobe Additional runtime features: fault tolerance, load balancing, transaction handling, usage metering, auditing, ,Copyright 2010 B. Ramamurthy,Copyright 2010 B. Ramamurthy,10/10/2018,20,Advances
20、in Middleware,Communication Middleware,10/10/2018,Page 21,Application,Application,Network Protocol Stack,Network Protocol Stack,Middleware,Middleware,Copyright 2010 B. Ramamurthy,Remote procedure call (RPC),10/10/2018,Page 22,Client Application,Server Application,Network Protocol Stack,Network Proto
21、col Stack,RPC Stub code,RPC stub code,RPC library/runtime,RPC library/runtime,Procedure call,Execute call,RPC stubs and runtime enable location transparency, encapsulate RPC communicationinfrastructure and provide a procedure call interface.,Copyright 2010 B. Ramamurthy,Distributed Objects,10/10/201
22、8,Page 23,Client Application,Server Application,Network Protocol Stack,Network Protocol Stack,Client proxies,Server skeletons,ORB,ORB,invoke method,Execute method,ORB (Object Request broker) enables client applications to remotely instantiate, locate, invoke methods and Delete server objects; Java R
23、MI, Microsofts DCOM; CORBA is meant to be platform independent.,Copyright 2010 B. Ramamurthy,24,Middleware (Eg. CORBA, Grid),“network”,client,server,middleware,middleware,client,server,“desktop”,BR,10/10/2018,Copyright 2010 B. Ramamurthy,Technical Layers,10/10/2018,Page 25,Participant A,Participant
24、C,Participant B,Technology Independent Interface Description,Technology Independent Interface Description,Technology Independent Interface Description,Core Assets,XML,ORB,Technology Adapters,Middleware Mapping,CORBA,XML Web services,Middleware Buses,Communication facilities,Business Logic,GRID,Copyr
25、ight 2010 B. Ramamurthy,10/10/2018,Copyright 2010 B. Ramamurthy,26,What is a grid?,Grid is a sophisticated framework that enables sharing of a variety of resources among distributed applications. Open standard Large scale operations Automatic Intelligent Spontaneous Interoperable Service-oriented,10
26、/10/2018,Copyright 2010 B. Ramamurthy,27,What is a grid? (A formal definition),Grid specifies a standard architecture, infrastructure, protocols and application program interface (API) for building an open enterprise system. It can provide Middleware supporting network of systems to facilitate shari
27、ng, standardization and openness. Infrastructure and application model dealing with sharing of compute cycles, data, storage and other resources. A framework for high reliability, availability and security. Interoperation of batch-oriented and service-based architectures. Standard service level feat
28、ure definitions and higher level concepts for inter and intra-business collaboration.,10/10/2018,Copyright 2010 B. Ramamurthy,28,Beginnings of The Grid,Beginnings of the grid in Search for Extra Terrestrial Intelligence (setihome project) http:/planetary.org/html/UPDATES/seti/index.html The Wow sign
29、al http:/planetary.org/html/UPDATES/seti/SETIhome/wowsignal.html,Message-oriented Middleware (MOM),10/10/2018,Page 29,Made famous by IBMs MQseries and TIBCOs Rendezvous products. Based on messages and queues. A message contains a header and a payload. A queue can store and distribute messages. Publi
30、sh/subscribe model of communication: A topic offers another model of communication between subscribers and publishers. MOM allows for loose coupling between message consumers and message producers enabling dynamic, reliable, flexible, high-performance systems to be built.,Copyright 2010 B. Ramamurth
31、y,Copyright 2010 B. Ramamurthy,10/10/2018,30,Server-side Advances,Server-side advances: Two-tier applications,10/10/2018,Copyright 2010 B. Ramamurthy,31,Server-side: Three-tier Applications,10/10/2018,Copyright 2010 B. Ramamurthy,32,Tremendous advances in DBMS: relational, query languages, Object-re
32、lationalACID property transactional systems,Programming Model for Web-based applications,10/10/2018,Copyright 2010 B. Ramamurthy,33,Web client,Business Logic,Web Service,Components and Application Servers,10/10/2018,34,An application server mediates between a web server and backend systems. Request
33、from a web client is passed onto an application server by the web server. Programmer productivity, cost-effective deployment, rapid time to market, seamless integration, application portability, scalability, security are some of the challenges that component technology tries to address head on. Ente
34、rprise Java Beans is Suns server component model that provides portability across application servers, and supports complex systems features such as transactions, security, etc. on behalf of the application components. EJB is a specification provided by Sun and many third party vendors have products
35、 compliant with this specification: BEA systems (bought out by oracle), IONA, IBM, Oracle, Sybase (bought out by IBM).,Copyright 2010 B. Ramamurthy,Application Programming Model for Three-tier Applications,10/10/2018,Copyright 2010 B. Ramamurthy,35,Expectations of a Distributed System,10/10/2018,Cop
36、yright 2010 B. Ramamurthy,36,Access transparency: enables local and remote resources to be accessed using identical operations. Location transparency: enables resources to be accessed without knowledge of their location. Concurrency transparency: enables several processes to operate concurrently usi
37、ng shared resources without interference between them. Replication transparency: enables multiple instances of resources to be used to increase reliability and performance without knowledge of the replicas by users or application programmers. Failure transparency: enables the concealment of faults,
38、allowing users and application programs to complete their tasks despite the failure of hardware or software components. Mobility transparency: allows the movement of resources and clients within a system without affecting the operation of users or programs. Performance transparency: allows the syste
39、m to be reconfigured to improve performance as loads vary. “Scalability” Expansion transparency: allows the system and applications to expand in scale without change to the system structure or the application algorithms.,Issues contd.,10/10/2018,Copyright 2010 B. Ramamurthy,37,Heterogeneity of compo
40、nents Scalability : ability to perform well under increased loads and data sizes Failure handling Concurrency Transparency Reliability Interoperability Performance Openness Security and protection,Copyright 2010 B. Ramamurthy,10/10/2018,38,Emerging Application Models,Large scale systems,10/10/2018,C
41、opyright 2010 B. Ramamurthy,39,Most emerging distributed applications are very large demanding large amounts of storage and data resources E-commerce systems and online businesses Applications connecting communities: from search to social networking See worlds 10 largest data bases Amount of data co
42、llected by various sources from terrorism monitoring to environmental monitoring Data deluge: most of the data is write once read many (WORM) Analytics resulting in data-intensive computing models and big-data computing (CSE487 material) We will look just one success story of distributed system: ,A,
43、10/10/2018,Copyright 2010 B. Ramamurthy,40,Werner Vogels talk “Order in the Chaos: Building the A Platform.“ 1995: Started out with a single web service on a single server. Today amazon has about 150 web services on its homepage alone. 1 million merchant partners; 60 million customers One server of
44、customers and inventory grew into two servers; more database servers were added as the business expanded 1999: A mistep during this exponential growth period was moving to mainframe from distributed server. Failed to meet scalability, reliability and performance; it was scratched in 2000.,Amazon (co
45、ntd.),10/10/2018,Copyright 2010 B. Ramamurthy,41,Robustness: Shopping cart is tested for 20000 items by a single customer, for example! Amazons secret sauce is “operating reliably at scale”. After “the denial of service” debacle in 1999, they decided to use Web services to insulate the databases fro
46、m being overwhelmed by direct interaction with online applications. Each web service is the responsibility of a team of developers: “And they are not just responsible for writing the service and then tossing it over the wall for testing and eventual entry into production where some poor maintenance
47、geek has to look after it. The Amazon CTO tells his Web services team members: “You build it. You own it.“ That means the team is responsible for its Web services on-going operation. If a Web service stops working in the middle of the night, team members are called to fix it.” Web services are kept
48、simple: complexity is the notorious enemy of reliability No attachment to one technology or standard: what ever customer wants, give it.,Copyright 2010 B. Ramamurthy,10/10/2018,42,On to more fundamental concepts,Synchrony,10/10/2018,43,Synchronous and asynchronous communications Synchronous: immedia
49、te response of communicating partners Server process/thread blocks until response is completed Follows request/response pattern Used when servers are available all the time Typically communicating partners are tightly coupled Examples: request from web client to a web browser for “search” or for “in
50、formation” CORBA procedure invocation Java RMI (remote method invocation) Traditional remote procedure call (RPC),Copyright 2010 B. Ramamurthy,Asynchronous communication,10/10/2018,44,Communicating partners are decoupled Message driven: sender creates a message and delivers it to a mediator who then sends it to “a” recipient Server need not be available all the time Sender and receiver loosely coupled Can facilitate high-performance message-based system Example: Any event-driven system Any messaging system (instant messenger) Publish-subscribe mode communications,
