Distributed Systems

Principles and Paradigms

Chapter 09
(version 27th November 2001)

Maarten van Steen

Vrije Universiteit Amsterdam, Faculty of Science
Dept. Mathematics and Computer Science
Room R4.20. Tel: (020) 444 7784
E-mail:[email protected], URL: www.cs.vu.nl/ steen/

01 Introduction
02 Communication
03 Processes
04 Naming
05 Synchronization
06 Consistency and Replication
07 Fault Tolerance
08 Security
09 Distributed Object-Based Systems
10 Distributed File Systems
11 Distributed Document-Based Systems
12 Distributed Coordination-Based Systems
00 – 1 /
Distributed Object-Based Systems

CORBA

DCOM

Globe

09 – 1 Distributed Object-Based Systems/

 CORBA

CORBA: Common Object Request Broker Architecture

Background:

Developed by the Object Management Group

(OMG) in response to industrial demands for object-
based middleware
Currently in version #2.4 with #3 (almost) done
CORBA is a specification: different implementa-
tions of CORBA exist
Very much the work of a committee: there are
over 800 members of the OMG and many of them
have a say in what CORBA should look like

Essence: CORBA provides a simple distributed-object

model, with specifications for many supporting ser-
vices it may be here to stay (for a couple of years)

09 – 2 Distributed Object-Based Systems/9.1 CORBA

 CORBA Overview (1/2)
Client machine Server machine

Client application Object implementation

Static Dynamic ORB Object Skeleton Dynamic ORB

IDL Invocation interface adapter Skeleton interface
proxy Interface Interface

Client ORB Server ORB

Local OS Local OS

Network

Object Request Broker (ORB): CORBA’s object bro-

ker that connects clients, objects, and services
Proxy/Skeleton: Precompiled code that takes care
of (un)marshaling invocations and results
Dynamic Invocation/Skeleton Interface (DII/DSI): To
allow clients to “construct” invocation requests at
runtime instead of calling methods at a proxy, and
having the server-side “reconstruct” those request
into regular method invocations
Object adapter: Server-side code that handles incom-
ing invocation requests.

09 – 3 Distributed Object-Based Systems/9.1 CORBA

 CORBA Overview (2/2)

Interface repository: Database containing interface

definitions and which can be queried at runtime

Implementation repository: Database containing the

implementation (code, and possibly also state) of
objects. Effectively: a server that can launch ob-
ject servers.

09 – 4 Distributed Object-Based Systems/9.1 CORBA

 CORBA Object Model
Essence: CORBA has a “traditional” remote-object
model in which an object residing at an object server
is remote accessible through proxies

Observation: All CORBA specifications are given by

means of interface descriptions, expressed in an IDL.
CORBA follows an interface-based approach to ob-
jects:

Not the objects, but interfaces are the really im-

portant entities

An object may implement one or more interfaces

Interface descriptions can be stored in an inter-

face repository, and looked up at runtime

Mappings from IDL to specific programming are

part of the CORBA specification (languages in-
clude C, C++, Smalltalk, Cobol, Ada, and Java.

09 – 5 Distributed Object-Based Systems/9.1 CORBA

 CORBA Services
Service Description
Collection Facilities for grouping objects into lists, queue, sets, etc.
Query Facilities for querying collections of objects in a declara-
tive manner
Concurrency Facilities to allow concurrent access to shared objects
Transaction Flat and nested transactions on method calls over multi-
ple objects
Event Facilities for asynchronous communication through
events
Notification Advanced facilities for event-based asynchronous com-
munication
Externalization Facilities for marshaling and unmarshaling of objects
Life cycle Facilities for creation, deletion, copying, and moving of
objects
Licensing Facilities for attaching a license to an object
Naming Facilities for systemwide naming of objects
Property Facilities for associating (attribute, value) pairs with ob-
jects
Trading Facilities to publish and find the services an object has
to offer
Persistence Facilities for persistently storing objects
Relationship Facilities for expressing relationships between objects
Security Mechanisms for secure channels, authorization, and au-
diting
Time Provides the current time within specified error margins

09 – 6 Distributed Object-Based Systems/9.1 CORBA

 Communication Models (1/2)
Object invocations: CORBA distinguishes three dif-
ferent forms of direct invocations:
Request type Failure sem. Description
Synchronous At-most-once Caller blocks
One-way Unreliable Nonblocking call
Deferred syn- At-most-once Nonblocking, but can pick-
chronous up results later

Event communication: There are also additional fa-

cilities by means of event channels:

Push event to consumers

Supplier
Consumer
Event channel Supplier
Consumer
Supplier

Ask suppliers for new event

Supplier
Consumer
Event channel Supplier
Consumer
Supplier

09 – 7 Distributed Object-Based Systems/9.1 CORBA

 Communication Models (2/2)

Messaging facilities: reliable asynchronous and per-

sistent method invocations:

Client application
1. Call by the
application
Client Callback 4. Call by the ORB
proxy interface
3. Response from server
Client
ORB

2. Request to server

Client application
1. Call by the 4. Call by the
application application
Client Polling
proxy interface
3. Response from server
Client
ORB

2. Request to server

09 – 8 Distributed Object-Based Systems/9.1 CORBA

 Processes
Most aspects of processes for in CORBA have been
discussed in previous classes. What remains is the
concept of interceptors:

Client application

Client
proxy Invocation request

Request-level
interceptor
Client
ORB
Message-level
interceptor

Local OS

To server

Request-level: Allows you to modify invocation se-

mantics (e.g., multicasting)
Message-level: Allows you to control message-passing
between client and server (e.g., handle reliability
and fragmentation)

09 – 9 Distributed Object-Based Systems/9.1 CORBA

 Naming
Important: In CORBA, it is essential to distinguish
specification-level and implementation-level object ref-
erences

Specification level: An object reference is considered

to be the same as a proxy for the referenced ob-
ject having an object reference means you can
directly invoke methods; there is no separate client-
to-object binding phase

Implementation level: When a client gets an object

reference, the implementation ensures that, one
way or the other, a proxy for the referenced object
is placed in the client’s address space:

  
  

  
 "!$#%&('*),+-.0/21 35476-8(9-8;:=<$/?>'"6@/BADCE

Conclusion: Object references in CORBA used to be

highly implementation dependent: different imple-
mentations of CORBA could normally not exchange
their references.
09 – 10 Distributed Object-Based Systems/9.1 CORBA
 Interoperable Object References
(1/2)
Observation: Recognizing that object references are
implementation dependent, we need a separate refer-
encing mechanism to cross ORB boundaries

Solution: Object references passed from one ORB to

another are transformed by the bridge through which
they pass (different transformation schemes can be
implemented)

Observation: Passing an object reference 
from
ORB A to ORB B circumventing the A-to-B bridge may

be useless if ORB B doesn’t understand 

09 – 11 Distributed Object-Based Systems/9.1 CORBA
 Interoperable Object References
(2/2)

Observation: To allow all kinds of different systems

to communicate, we standardize the reference that is
passed between bridges:

Tagged Profile
Interoperable Object Reference (IOR)

Repository Profile
Profile
identifier ID

IIOP Object key Components

Host Port
version

POA Object Other server-

identifier identifier specific information

09 – 12 Distributed Object-Based Systems/9.1 CORBA

 Naming Service
Essence: CORBA’s naming service allows servers to
associate a name to an object reference, and have
clients subsequently bind to that object by resolving
its name

Observation: In most CORBA implementations, ob-

ject references denote servers at specific hosts; nam-
ing makes it easier to relocate objects

Observation: In the naming graph all nodes are ob-

jects; there are no restrictions to binding names to ob-
jects CORBA allows arbitrary naming graphs

Question: How do you imagine cyclic name resolu-

tion stops?

Observation: There is no single root; an initial con-

text node is returned through a special call to the ORB.
Also: the naming service can operate across different
ORBs interoperable naming service
09 – 13 Distributed Object-Based Systems/9.1 CORBA
 Fault Tolerance

Essence: Mask failures through replication, by putting

objects into object groups. Object groups are trans-
parent to clients: they appear as “normal” objects.
This approach requires a separate type of object ref-
erence: Interoperable Object Group Reference:

Interoperable Object Group Reference (IOGR)

Repository Profile Profile

Profile-1 Profile-N
identifier ID ID

IIOP Object IIOP Object

Host-1 Port-1 Components Host-N Port-N Components
ver. key-1 ver. key-N

TAG Other group- TAG Other group-

PRIMARY specific information BACKUP specific information

Note: IOGRs have the same structure as IORs; the

main difference is that they are used differently. In
IORs an additional profile is used as an alternative; in
IOGR, it denotes another replica.

09 – 14 Distributed Object-Based Systems/9.1 CORBA

 Security
Essence: Allow the client and object to be mostly un-
aware of all the security policies, except perhaps at
binding time; the ORB does the rest. Specific poli-
cies are passed to the ORB as (local) objects and are
invoked when necessary:

Client application Object implementation

Set of
client-specific
Security Policy policy objects Policy Security
service object object service
Set of
Policy object-specific Policy
Security policy objects Security
object object
service service

Client ORB Set of relevant Server ORB

ORB security
Local OS services Local OS

Network
Invocation

Examples: Type of message protection, lists of trusted

parties.
09 – 15 Distributed Object-Based Systems/9.1 CORBA
 Distributed COM

DCOM: Distributed Component Object Model

Microsoft’s solution to establishing inter-process

communication, possibly across machine bound-
aries.

Supports a primitive notion of distributed objects

Evolved from early Windows versions to current

NT-based systems (including Windows 2000)

Comparable to CORBA’s object request broker

09 – 16 Distributed Object-Based Systems/9.2 Distributed COM

 DCOM Overview (1/2)

Somewhat confused? DCOM is related to many things

that have been introduced by Microsoft in the past
couple of years:

Grouping
ActiveX Documents Scripting
(Controls)
In-place
Document editing
OLE Embedding
linking Drag
and drop
Interprocess
data transfer Persistent Object
Persistent references activation
COM storage

Core COM library

DCOM: Adds facilities to communicate across pro-

cess and machine boundaries.

09 – 17 Distributed Object-Based Systems/9.2 Distributed COM

 DCOM Overview (2/2)

Client machine Object server

Class SCM
SCM Client application object Object

Proxy Client COM Proxy Object COM

marshaler proxy marshaler stub

Local OS Local OS
Registry Registry

Network
Microsoft RPC

SCM: Service Control Manager, responsible for ac-

tivating objects (cf., to CORBA’s implementation
repository).

Proxy marshaler: handles the way that object refer-

ences are passed between different machines

09 – 18 Distributed Object-Based Systems/9.2 Distributed COM

 COM Object Model

An interface is a collection of semantically related

operations

Each interface is typed, and therefore has a glob-

ally unique interface identifier

A client always requests an implementation of an

interface:

– Locate a class that implements the interface

– Instantiate that class, i.e., create an object
– Throw the object away when the client is done

09 – 19 Distributed Object-Based Systems/9.2 Distributed COM

 DCOM Services
CORBA DCOM/COM+ Windows 2000
Collection ActiveX Data Objects –
Query None –
Concurrency Thread concurrency –
Transaction COM+ Automatic Distributed Transac-
Transactions tion Coordinator
Event COM+ Events –
Notification COM+ Events –
Externalization Marshaling utilities –
Life cycle Class factories, JIT –
activation
Licensing Special class facto- –
ries
Naming Monikers Active Directory
Property None Active Directory
Trading None Active Directory
Persistence Structured storage Database access
Relationship None Database access
Security Authorization SSL, Kerberos
Time None None

Note: COM+ is effectively COM plus services that

were previously available in an ad-hoc fashion
09 – 20 Distributed Object-Based Systems/9.2 Distributed COM
 Communication Models
Object invocations: Synchronous remote-method calls
with at-most-once semantics. Asynchronous invoca-
tions are supported through a polling model, as in
CORBA.

Event communication: Similar to CORBA’s push-

style model:

Supplier Consumer

Event class m_event

object

Event Consumer
Interface object
containing m_event
m_event

Object
Invocation Invocation implementing
is stored is passed m_event
to consumer
Event
store

Messaging: Completely analogous to CORBA mes-

saging.
09 – 21 Distributed Object-Based Systems/9.2 Distributed COM
 Processes: Passing Object
References
Observation: Objects are referenced by means of
a local interface pointer. The question is how such
pointers can be passed between different machines:

Process A Process B
Marshaled
Client application client proxy Client application

Client Proxy Proxy Client

proxy (un)marshaler (un)marshaler proxy

Binding information Same binding

information

Object
Object
stub

Object server

Question: Where does the proxy marshaler come

from? Do we always need it?
09 – 22 Distributed Object-Based Systems/9.2 Distributed COM
 Naming: Monikers

Observation: DCOM can handle only objects as tem-

porary instances of a class. To accommodate objects
that can outlive their client, something else is needed.

Moniker: A hack to support real objects

A moniker associates data (e.g., a file), with an

application or program
Monikers can be stored
A moniker can contain a binding protocol, spec-
ifying how the associated program should be
“launched” with respect to the data.

1 Client Calls BindMoniker at moniker

2 Moniker Lookup CLSID and tell SCM to create object
3 SCM Loads class object
4 Class object Creates object, returns int. pointer
5 Moniker Instructs object to load previously stored state
6 Object Loads its state from file
7 Moniker Returns interface pointer of object to client

09 – 23 Distributed Object-Based Systems/9.2 Distributed COM

 Active Directory

Essence: a worldwide distributed directory service,

but one that does not provide location transparency.

Basics: Associate a directory service (called domain

controller) with each domain; look up the controller
using a normal DNS query:

DNS

1. Ask for host address 2. Requested address

of domain controller
in a given domain
3. LDAP query
Domain
Client
controller
4. LDAP reply

Note: Controller is implemented as an LDAP server

09 – 24 Distributed Object-Based Systems/9.2 Distributed COM

 Fault Tolerance

Automatic transactions: Each class object (from which

objects are created), has a transaction attribute that
determines how its objects behave as part of a trans-
action:
Attr. value Description
REQUIRES NEW A new transaction is always started at
each invocation
REQUIRED A new transaction is started if not al-
ready done so
SUPPORTED Join a transaction only if caller is al-
ready part of one
NOT SUPPORTED Never join a transaction (no transaction
support)
DISABLED Never join a transaction, even if told to
do so

Note: Transactions are essentially executed at the

level of a method invocation.

09 – 25 Distributed Object-Based Systems/9.2 Distributed COM

 Security (1/2)

Declarative security: Register per object what the

system should enforce with respect to authentication.
Authentication is associated with users and user groups.
There are different authentication levels:

Auth. level Description

NONE No authentication is required
CONNECT Authenticate client when first con-
nected to server
CALL Authenticate client at each invocation
PACKET Authenticate all data packets
PACKET INTEGRITY Authenticate data packets and do in-
tegrity check
PACKET PRIVACY Authenticate, integrity-check, and en-
crypt data packets

09 – 26 Distributed Object-Based Systems/9.2 Distributed COM

 Security (2/2)

Delegation: A server can impersonate a client de-

pending on a level:
Impersonation Description
ANONYMOUS The client is completely anonymous to
the server
IDENTIFY The server knows the client and can do
access control checks
IMPERSONATE The server can invoke local objects on
behalf of the client
DELEGATE The server can invoke remote objects
on behalf of the client

Note: There is also support for programmatic secu-

rity by which security levels can be set by an appli-
cation, as well as the required security services (see
book).

09 – 27 Distributed Object-Based Systems/9.2 Distributed COM

 Globe

Experimental wide-area system currently being de-

veloped at Vrije Universiteit

Unique for its focus on scalability by means of

truly distributed objects

Prototype version up and running across multi-

ple machines distributed in NL and across Europe
and the US.

09 – 28 Distributed Object-Based Systems/Globe

 Object Model (1/3)
Essence: A Globe object is a physically distributed
shared object: the object’s state may be physically
distributed across several machines
Distributed shared object

Local object

Process

Network

Interface

Local object: A nondistributed object residing a sin-

gle address space, often representing a distributed
shared object
Contact point: A point where clients can contact the
distributed object; each contact point is described
through a contact address

09 – 29 Distributed Object-Based Systems/Globe

 Object Model (2/3)
Observation: Globe attempts to separate function-
ality from distribution by distinguishing different local
subobjects:
Same interface as implemented
by semantics subobject

Control
subobject

Replication Semantics
subobject subobject

Communication
subobject

Communication
with other local
objects

Semantics subobject: Contains the methods that im-

plement the functionality of the distributed shared
object
Communication subobject: Provides a (relatively sim-
ple), network-independent interface for communi-
cation between local objects

09 – 30 Distributed Object-Based Systems/Globe

 Object Model (3/3)
Same interface as implemented
by semantics subobject

Control
subobject

Replication Semantics
subobject subobject

Communication
subobject

Communication
with other local
objects

Replication subobject: Contains the implementation

of an object-specific consistency protocol that
controls exactly when a method on the semantics
subobject may be invoked
Control subobject: Connects the user-defined inter-
faces of the semantics subobject to the generic,
predefined interfaces of the replication subobject

09 – 31 Distributed Object-Based Systems/Globe

 Client-to-Object Binding
1. Name
Client Naming service
Object handle
2. Object handle
Location service
3. Select Addresses &
address protocols
Register contact
address
Class object
Local object

5. Make contact

4. Load & instantiate

class(es)
Distributed shared object
(Trusted)
class repository

Observation: Globe’s contact addresses correspond

to CORBA’s object references

09 – 32 Distributed Object-Based Systems/Globe

 Globe Services
Service Possible implementation Av?
Collection Separate object that holds refer- No
ences to other objects
Concurrency Each object implements its own No
concurrency control strategy
Transaction Separate object representing a No
transaction manager
Event/Notif. Separate object per group of No
events (as in DCOM)
Externalization Each object implements its own Yes
marshaling routines
Life cycle Separate class objects combined Yes
with per-object implementations
Licensing Implemented by each object sepa- No
rately
Naming Separate service, implemented by Yes
a collection of naming objects
Property Separate service, implemented by No
a collection of directory objects
Persistence Implemented on a per-object basis Yes
Security Implemented per object, combined Yes
with (local) security services
Replication Implemented on a per-object basis Yes
Fault tolerance Implemented per object combined Yes
with fault-tolerant servers

09 – 33 Distributed Object-Based Systems/Globe

 Object References
Essence: Globe uses location-independent object han-
dles which are to be resolved to contact addresses
(which describes where and how an object can be
contacted):

Associated with a contact point of the distributed

object
Specifies (for example) a transport-level network
address to which the object will listen
Contains an implementation handle, specifying
exactly what the client should implement if it wants
to communicate through the contact point:

 
     !"#$%&''
–

''(*),+- +/0)21 #+-3)!,4) 4&#+56)21 879)

'.

– “slave/master-slave/tcp/ip”

Observation: Objects in Globe have their own object-

specific implementations; there is no “standard” proxy
that is implemented for all clients
09 – 34 Distributed Object-Based Systems/Globe
 Naming Objects

Observation: Globe separates naming from locating

objects (as described in Chapter 04). The current
naming service is based on DNS, using TXT records
for storing object handles

Observation: The location service is implemented as

a generic, hierarchical tree, similar to the approach
explained in Chapter 04.

09 – 35 Distributed Object-Based Systems/Globe

 Caching and Replication
Observation: Here’s where Globe differs from many
other systems:

The organization of a local object is such that

replication is inherently part of each distributed
shared object

All replication subobjects have the same interface:


0
Method Description
Called to synchronize replicas of the se-
mantics subobjects, obtain locks if neces-
! sary, etc.
Provide marshaled arguments of a specific
method, and pass invocation to local ob-
 E! jects in other address spaces
Called after the control subobject has in-
voked a specific method at the semantics
subobject

This approach allows to implement any object-

specific caching/replication strategy

09 – 36 Distributed Object-Based Systems/Globe

 Security

Essence: Additional security subobject checks for au-

thorized communication, invocation, and parameter val-
ues. Globe can be integrated with existing security
services:

Security domain of B
Principal object
of process B

Process B

7
Local object B
6

Distributed
shared 5
object Security domain of A

2 Kerberos
1
Local object A
4 3

Process A

Principal object
of process A

09 – 37 Distributed Object-Based Systems/Globe

 Comparison

09 – 38 Distributed Object-Based Systems/Globe

Issue CORBA DCOM Globe
Design goals Interoperability Functionality Scalability
Object model Remote objects Remote objects Distributed objects
Services Many of its own From environment Few
Interfaces IDL based Binary Binary
Sync. communication Yes Yes Yes
Async. communication Yes Yes No
Callbacks Yes Yes No
Events Yes Yes No
Messaging Yes Yes No
Object server Flexible (POA) Hard-coded Object dependent
Directory service Yes Yes No
Trading service Yes No No
Naming service Yes Yes Yes
Location service No No Yes
Object reference Object’s location Interface pointer True identifier
Synchronization Transactions Transactions Only intra-object
Replication support Separate server None Separate subobject
Transactions Yes Yes No
Fault tolerance By replication By transactions By replication
Recovery support Yes By transactions No