Distributed Systems Fall 2009

Course introduction

Fall 2009 5DV020 3

Defining distributed systems• “A distributed system is one in which

components located at networked computers communicate and coordinate their actions by passing messages.”

(Coulouris, Dollimore, Kindberg, 2005)

• ”A distributed system is one in which the failure of computer you didn't even know existed can render your own computer unusable.”

(Leslie Lamport, 1987)

Fall 2009 5DV020 4

Outline

• Staff presentation• Course presentation• Lessons from last year• This year's course• Basics and challenges of distributed systems

• Summary

Fall 2009 5DV020 5

Staff presentation

• Daniel Henriksson (danielh@cs) • Lars Larsson (larsson@cs)

• Questions about the assignment?– Always send email to both teachers, even replies to replies!

• Questions about lectures?– Send email to the appropriate teacher!

Fall 2009 5DV020 6

Course presentation

• Students should obtain:– Knowledge of theoretical models for distributed systems

– Knowledge of problems and solutions in designing and implementation of distributed systems

Course presentation

• The course covers:– Architectural models of distributed systems

– Client-Server, peer-to-peer, transactions, transparency, naming, error handling, resource management, and synchronization

– Computer security in a broad perspective

– Distributed programming and middlewares

Fall 2009 5DV020 8

Course presentation

• Theoretical part (4.5 ECTS)– Theory, methods, algorithms, and principles

• Practical part (3 ECTS)– Practical obligatory assignment

Fall 2009 5DV020 9

Lessons from last year (positive)

• Students were very happy with the level of assistance with the assignment

• The anonymous feedback form was appreciated, as was the web site itself

• Students acquired a lot of new knowledge

• The average grade given to the course was “very good”

• The practical assignment was rewarding, albeit hard

Fall 2009 5DV020 10

Lessons from last year (negative)• The practical assignment was hard, albeit rewarding

• The book was hard to read, and did not cover ordering algorithms sufficiently

• Java RMI was hard to use

• First set of theory lectures should be allocated more time

• Lectures should all have custom made slides

• Exam too focused on details

• Interface used for practical was confusing

Fall 2009 5DV020 11

This year's course

• Same practical assignment, but:– No potentially confusing interface

– Any programming language allowed

• Added a group exercise on Java RMI

• All lectures will have custom made slides

• More time allocated to key topics

• Exam more focused on “broader” questions

• Keep up good work with assistance, listening to feedback, lecturing style, and web page

Fall 2009 5DV020 12

The practical assignment

• Group communication middleware– Group membership handling

– Message ordering guarantees

– (Reliable) Multicast communication

• Presentation of working implementation at the end of the course

• Solved in pairs (2 students per team)

• Deals with theory from the first set of lectures

http://www.cs.umu.se/kurser/5DV020/HT09/assignment.html

Fall 2009 5DV020 13

What to learn?

• Book is dense with information– See reading guide on web page – it is actually accurate

– Extremely good, but no easy read• Start now! You will be busy later...

• Understand the problems and solutions– Learn the general ideas of algorithms and how/why they work, not every minute step

• Definitions are very important!

Fall 2009 5DV020 14

Schedule is subject to change

• We want to be able to adapt the schedule to fit your needs• If our in-class discussion of topics take longer time, that’s fine (within reason)

• Consider the topics for the lectures to be a reading guide, not a guarantee• If this poses a problem, please give us feedback, and we will try to remedy the situation

Fall 2009 5DV020 15

Basics of distributed systems

• Architectural model– Placement of components across the network and their relationships, and the network itself

• System model– Client/Server

• HTTP, FTP, … countless others!

– Peer-to-Peer (P2P)• BitTorrent, Freenet, Direct Connect, …

– Combinations: multiple servers, mobile code, mobile agents, thin clients

Fall 2009 5DV020 16

Design of distributed systems

• Performance issues– Responsiveness– Throughput– Load balancing

• Quality of Service (QoS)– Reliability, security, performance (time-critical)

– Adaptability

Design of distributed systems

• Caching and replication– Web caching and Content Distribution Networks

• Dependability– Correctness, security, and fault tolerance

Fall 2009 5DV020 18

Distributed systems challenges

• Heterogeneous systems– Networks, hardware, OS, languages, protocols

• Openness• Security• Scalability

– Control physical resource costs and performance loss, resource conservation, bottleneck avoidance

Distributed systems challenges

• Failure handling– Detection, masking, tolerance, recovery, redundancy

• Concurrency– No global time, simultaneously running processes

• Transparency– Access, location, concurrency, replication, failure, mobility, performance, scaling

Fall 2009 5DV020 20

Distributed systems challenges

• How are parameters passed and how is data converted?– Marshalling, unmarshalling

• How are distributed resources (functions, methods, objects) published and discovered?

• How are errors handled?• Security!• Memory handling – distributed garbage collection?

Fall 2009 5DV020 21

Middleware

• Distributed systems often utilize middleware to aid development

• Offers layer of abstraction• Extends upon traditional programming models:– Local procedure call → Remote procedure call

– OOP → Remote Method Invocation– Event-based programming model

Fall 2009 5DV020 22

Middleware

Applications, Services

RMI, RPC

Request/Reply protocolMarshalling, Unmarshalling

UDP, TCP

Middleware

Fall 2009 5DV020 23

Operation invocation

• Data structures must be “flattened” and serialized for transport– External formats, e.g. XML, JSON, Java Object Serialization, ...

• Use interface– Procedures having either input, output, or both

– No pointers

– Service interface: provided services

– Remote interface: operations accessible from other processes

– Cross-language/platform interfaces: IDL, WSDL

Fall 2009 5DV020 24

Semantics

• (Local call = exactly once)

• Maybe once– Omission failures (lost packets, crashes)

• At-least-once– Crash failures, arbitrary failures (multiple executions)

– Used by Sun RPC

• At-most-once– Executed exactly once or not at all

– Used by Java RMI, Corba

Fall 2009 5DV020 25

Security

• Distributed system = increased exposure

• Client- and Server-authentication

• Client authorization– Is the client allowed to perform X?

• Proof of execution– Server must be able to prove that something has been executed

– Also, non-repudiation: it should not be possible to claim that something did not happen if it did

Fall 2009 5DV020 26

Distributed systems: a mess!

• Communication performance variations– Latency (delay), bandwidth (throughput), jitter (variation in time)

• Clocks and timing– Clock drift

• Interaction models– Asynchronous, synchronous

• Event ordering– Delays cause replies to arrive to some process before the request

Distributed systems: a mess!

• Failures– Distributed systems are much more likely to fail unexpectedly

– Lost packets, bit errors, local failures, no response, method does not exist, etc …

If you can write stable programs in spite of these difficulties,

you are a great programmer!

Fall 2009 5DV020 28

Summary

• Introduction to the course• What are distributed systems?• Consequences of distributed systems– Concurrency– No global time– Unrelated errors

Fall 2009 5DV020 29

Summary

• Challenges– Heterogeneity– Openness– Security– Scalability– Error handling– Concurrency– Transparency

• Middlewares

Fall 2009 5DV020 30

Next lecture

• Time, and the lack thereof– No global time, and its implications

– Algorithms for (roughly) synchronizing processes