Module-1

Distributed System Models and Enabling Technologies: Scalable Computing Over the
Internet, Technologies for Network Based Systems, System Models for Distributed and
Cloud Computing, Software Environments for Distributed Systems and Clouds,
nPerformance, Security and Energy Efficiency.

1.1 SCALABLE COMPUTING OVER THE INTERNET

• From past 60 years, we have seen many changes in computer technology, including
advancements in machine architecture, operating system platforms, network
connectivity, and application workloads.

1.1.1 The Age of Internet Computing

• The increasing number of Internet users requires high-performance computing (HPC)
services from supercomputer sites and large data centers. Traditional performance
benchmarks like the Linpack Benchmark are no longer sufficient.
• Instead, high-throughput computing (HTC) systems, utilizing parallel and distributed
computing technologies, are needed.
• Upgrading data centers with faster servers, efficient storage, and high-bandwidth
networks is essential to support advanced network-based computing and web services.

1.1.1.1 The Platform Evolution

o Two Main Computing Systems: HTC and HPC

o HTC:These systems consist of (heterogeneous) made up of different elements

nodes that work independently.

o They focus on file sharing and tasks that can be executed over a long period.

Shruthi T S,KSIT
 o They rely on a P2P (Peer-to-Peer) network, which uses distributed control and
is geographically sparse (i.e., spread across different locations).

o HPC (High-Performance Computing) Systems:

o These systems consist of homogeneous nodes (similar hardware/software).

o They focus on high-speed processing for computationally intensive tasks.

o They use Clusters or MPPs (Massively Parallel Processors), which operate

under centralized control and can handle disparate clusters.

o By integrating P2P networks (HTC) and Clusters/MPPs (HPC),a

Computational and Data Grid is formed.

o Once computational and data grids are established, they enable three advanced
computing paradigms:
High-Performance Computing

o High-Performance Computing (HPC) processes data very fast and solves

complex problems at high speed. An example of HPC is a supercomputer.

o HPC systems have long focused on raw speed performance, advancing from
Gflops in the 1990s to Pflops by 2010, primarily driven by scientific,
engineering, and manufacturing needs.
o The Top 500 supercomputers are ranked based on floating-point performance
using the Linpack benchmark.
o However, supercomputer users make up less than 10% of all computer users.
Today, most users rely on desktop computers or large servers for tasks like
Internet searches and market-driven computing.

High-Performance Computing

o High-Performance Computing(HPC) uses distributed computing

facilities(nothing but multiple computers working together)to handle
applications that need a lot of computing power over a long period.
o The focus of high-end computing is shifting from High-Performance Computing
(HPC) to High-Throughput Computing (HTC), which prioritizes handling large
volumes of tasks efficiently.

Shruthi T S,KSIT
 o HTC is essential for applications like Internet searches and web services used
by millions simultaneously. Unlike HPC, which emphasizes raw speed, HTC
measures performance based on task completion rates.
o Besides improving batch processing, HTC must also address challenges like
cost, energy efficiency, security, and reliability in data centers.

1.1.1.4 Three New Computing Paradigms

• The evolution of computing has introduced new paradigms such as Service-

Oriented Architecture (SOA), Web 2.0, virtualization, cloud computing, and the
Internet of Things (IoT).
• These advancements are transforming how computing resources are accessed
and utilized. Since the early days of the Internet, experts have provide computing
as a utility, evolving from networks to data centers and now to cloud computing.
• It including clusters, massively parallel processors (MPPs), peer-to-peer (P2P)
networks, grids, clouds, web services, social networks, and IoT

1. Web 2.0 Services

• These are online applications that let people create, share, and interact over the
internet.
• They work using Service-Oriented Architecture (SOA), which helps connect
different services.
• Examples: Google Docs (for online documents), Facebook (for social
networking), YouTube (for sharing videos).
2. Internet Clouds
• This is a system where computers and data storage are available on demand
through the internet.
• It uses virtualization, meaning one physical server can act like many virtual
servers.
• Examples: Amazon Web Services (AWS), Microsoft Azure, and Google Cloud
help businesses store and process data online instead of on personal computers.
3. Internet of Things (IoT)
• This is when everyday devices are connected to the internet and can collect and
share data automatically.
• It uses RFID and sensors to track and monitor things in real-time.

Shruthi T S,KSIT
 • Examples: Smart home devices like Alexa and Nest (for home automation),
industrial IoT (for factories), and healthcare monitoring systems (for tracking
patients' health).
1.1.1.5 Computing Paradigm Distinctions

…>There are 4 types

o Centralized computing
o Parallel computing
o Distributed computing
o Cloud computing
1️⃣ Centralized Computing
• One central server controls all computing and data storage.
• Other devices (clients) connect to this central server for processing and
accessing files.
• Example: Google Drive, where all data is stored on Google's servers.
2️⃣ Parallel Computing
• Multiple processors work together at the same time to solve a problem.
• Speeds up processing by dividing a task into smaller parts and solving them
simultaneously.
• Example: Supercomputers used for weather forecasting or scientific research.
3️⃣ Distributed Computing
• Many computers (nodes) work together to process tasks, but they are in
different locations.
• Tasks are shared among multiple computers instead of relying on one central
system.
• Example: Bitcoin mining or SETI@home, where many computers share the
workload.
4️⃣ Cloud Computing
• Uses the internet to store and process data instead of a personal computer.
• Allows access to files and software from anywhere, anytime.
• Example: Netflix, where videos are streamed from cloud servers.

1️.1️.1️.6 Distributed System Families

• Since the mid-1990s, peer-to-peer (P2P) networks and cluster networks have
been integrated into large-scale computational and data grids, enabling wide-area
computing infrastructures.
• More recently, there has been a growing interest in leveraging Internet cloud
resources for data-intensive applications.

Shruthi T S,KSIT
 • Cloud computing has emerged as a shift from desktop computing to service-
oriented computing, utilizing server clusters and massive databases in data
centers.
• Both grids and clouds focus on resource sharing in terms of hardware, software,
and data, but they operate as distinct distributed systems.
➢ The design objectives for high-performance computing (HPC) and high-
throughput computing (HTC) systems include:
• Efficiency: Maximizing resource utilization through parallelism in HPC,
while focusing on job throughput, data access, storage, and power efficiency
in HTC.
• Dependability: Ensuring reliability and self-management across all levels,
with Quality of Service (QoS) assurance, even in failure conditions.
• Adaptation: Supporting billions of job requests across large datasets and
virtualized cloud resources under different workload and service models.
• Flexibility: Enabling distributed systems to efficiently handle both HPC
(scientific and engineering) and HTC (business-oriented) applications.

1.1.2 Scalable Computing Trends and New Paradigms

Includes,
✓ Degrees of Parallelism
✓ Innovative Applications
✓ The Trend toward Utility Computing
✓ The Hype Cycle of New Technologies
▪ Fifty years ago, when hardware was bulky and expensive, most computers
were designed in a bit-serial fashion.
• Data-level parallelism (DLP) was made popular through SIMD (single
instruction, multiple data) and vector machines using vector or array types
of instructions. DLP requires even more hardware support and compiler
assistance to work properly.
Innovative Applications :
✓ Both HPC and HTC systems desire transparency in many application aspects.

Shruthi T S,KSIT
 ✓ For example, data access, resource allocation, process location, concurrency
in execution, job replication, and failure recovery should be made transparent
to both users and system management.

✓ For example, distributed transaction processing is often practiced in the

banking and finance industry. Transactions represent 90 percent of the
existing market for reliable banking systems. Users must deal with multiple
database servers in distributed transactions.

1.1.2.3 The Trend toward Utility Computing

• Utility computing focuses on a business model in which customers receive

computing resources from a paid service provider. All grid/cloud platforms
are regarded as utility service providers.

Shruthi T S,KSIT
 • Figure 1.2 identifies major computing paradigms to facilitate the study of distributed
systems and their applications. These paradigms share some common characteristics.
• First, they are all ubiquitous in daily life. Reliability and scalability are two major
design objectives in these computing models.
• Second, they are aimed at autonomic operations that can be self- organized to support
dynamic discovery.

1.1.2.4 The Hype Cycle of New Technologies

• Any new and emerging computing and information technology may go through a
hype cycle, Generally illustrated in Figure 1.3. This cycle shows the expectations for
the technology at five different stages
• Also as shown in Figure 1.3, the cloud technology had just crossed the peak of the
expectation stage in 2010, and it was expected to take two to five more years to reach
the productivity stage.

1.1.3.2 Cyber-Physical Systems

• A Cyber-Physical System (CPS) integrates computational processes with the physical
world, combining computation, communication, and control (3C technologies) into an
intelligent feedback system.
• While the Internet of Things (IoT) focuses on networking connections among physical
objects, CPS explores virtual reality (VR) applications in the physical world. CPS has
the potential to revolutionize how we interact with the physical environment, just as
the Internet changed interactions in the virtual world.
• The relationship between IoT, CPS, and cloud computing will be further explored.

1.2 TECHNOLOGIES FOR NETWORK-BASED SYSTEMS

• Over the past 30 years, advancements in component and network technologies
have been vital for the development of High-Performance Computing (HPC)
and High-Throughput Computing (HTC) systems.

Shruthi T S,KSIT
 • Processor speed, measured in MIPS, and network bandwidth, measured in
Mbps or Gbps, have significantly increased.
• Figure 1.4 illustrates this growth, with "GE" representing 1 Gbps Ethernet
bandwidth.
• These improvements have enhanced computing efficiency, enabling faster data
processing and communication in distributed systems.

• Modern CPUs use a multicore architecture with dual, quad, six, or more cores,
exploiting instruction-level (ILP) and thread-level parallelism (TLP).
• Over the years, processor speed has grown significantly, from 1 MIPS in 1978
(VAX 780) to 22,000 MIPS in 2008 (Sun Niagara 2), aligning with Moore’s
Law. However, clock rate improvements have slowed due to power and heat
limitations, rarely exceeding 5 GHz.
• Advanced ILP techniques like superscalar architecture, dynamic branch
prediction, and speculative execution enhance performance.
• Additionally, GPUs leverage many-core architectures with hundreds to
thousands of simple cores to maximize data-level (DLP) and thread-level
parallelism.

Shruthi T S,KSIT
*Each core in a multicore processor acts like a separate processor.

*Each core has its own private L1 cache, which stores frequently used data for quick access.

*All cores share a common L2 cache, which allows them to exchange data efficiently.

*This design helps improve performance by allowing multiple tasks to run simultaneously
while reducing delays in accessing memory.

Multithreading Technology

➢ In this dig there are five independent threads of instructions with five processor and
four pipelined data paths (functional units) in each of the following five processor
categories

1️.four-issue superscalar processor

2️. a fine-grain multithreaded processor

3️. a coarse-grain multi threaded processor

4️.a two-core CMP

5. and a simultaneous multithreaded (SMT) processor.

Modern processors use different architectures to execute instructions efficiently.

1️. Superscalar Processor

o A single-threaded processor with four functional units.

o Executes multiple instructions from the same thread in parallel.

2️. Multithreaded Processors (Four-Way Multithreading)

Shruthi T S,KSIT
 o Each processor can handle four independent threads across four data paths.

o Helps improve CPU utilization by switching between threads.

3️. Dual-Core Processor

o Contains two processing cores, each acting as a two-way superscalar

processor.

o Each core handles different tasks independently.

4️. Fine-Grain Multithreading

o Switches execution between different threads every cycle.

o Helps reduce idle time but increases thread-switching overhead.

5. Coarse-Grain Multithreading

o Executes multiple instructions from the same thread before switching to

another.

o Reduces switching overhead but may cause delays if a thread stalls.

6. Chip Multiprocessing (CMP / Multicore Processing)

o Different threads execute on separate processor cores simultaneously.

o Greatly improves parallelism and workload distribution.

7. Simultaneous Multithreading (SMT)

o Allows multiple threads to execute instructions in the same cycle.

o Maximizes instruction scheduling efficiency but requires complex hardware.

GPU Computing to Exascale and Beyond

➢ A Graphics Processing Unit (GPU) is like a super-fast assistant for your computer. Its
job is to handle images, videos, and 3️D graphics, so the main processor (CPU) doesn’t
get overloaded.
➢ It is located inside the video card (also called a graphics card), which is a special part
of your computer.
➢ It acts as a coprocessor, meaning it works alongside the CPU.

➢ The first GPU was made by NVIDIA in 1999 and was called GeForce 256.

➢ It could handle 10 million shapes (polygons) per second, making games and
animations much smoother.

How GPUs Work

Shruthi T S,KSIT
 ➢ A CPU (brain of the computer) usually works on one task at a time or a few tasks.
But a GPU is like a team of workers that can do thousands of tasks at the same time—
this is called parallel processing.
➢ Early GPUs functioned as coprocessors for the CPU, but modern GPUs have evolved
into powerful parallel computing units. NVIDIA GPUs now feature up to 1️2️8 cores, with
each core handling 8 threads, enabling 1️,02️4️ concurrent threads on a single chip—far
surpassing traditional CPUs in parallelism.
➢ CPU: Optimized for low-latency cache-based computing and handles fewer threads.
GPU: Optimized for high-throughput parallel processing with efficient on-chip memory
management.
➢ Modern GPUs are widely used in high-performance computing (HPC) and
supercomputers for massively parallel floating-point calculations.
➢ GPUs offload data-intensive computations from CPUs, enabling faster processing in
applications beyond graphics, such as scientific simulations, AI, and deep learning.
➢ NVIDIA CUDA Tesla and Fermi GPUs are specialized for HPC clusters and parallel data
processing.

GPU Programming Model

This diagram represents the GPU architecture and its interaction with the CPU and memory.

• The GPU consists of multiple multiprocessors, each containing several processing

units (P1, P2, ..., Pn) for parallel processing.

• The device memory (GPU memory) stores data and instructions for computations.

• The CPU communicates with the GPU and main memory to send and receive data.

• The GPU is optimized for massively parallel tasks, making it efficient for graphics, AI,
and high-performance computing (HPC).

Shruthi T S,KSIT
 ➢ four challenges are identified for exascale computing:
(1️) energy and power, (2️) memory and storage, (3️) concurrency and locality, and (4️)
system resiliency
This diagram represents the architecture of an NVIDIA Fermi GPU, specifically
showing the structure of a Streaming Multiprocessor (SM).
• The Fermi GPU consists of 16 Streaming Multiprocessors (SMs), each containing 32
CUDA cores (only one SM is shown in the diagram).
• The CUDA cores perform computations in parallel, handling both floating-point (FP)
and integer (INT) operations.
• The warp scheduler and dispatch unit control the execution of multiple threads in
parallel.
• The register file (32,768 x 32-bit) stores temporary data for high-speed processing.
• Load/Store (LD/ST) units handle memory operations, moving data between shared
memory and caches.
• SFU (Special Function Units) execute complex mathematical functions like sine,
cosine, and square roots.
• Interconnect network, shared memory (64 KB L1 cache), and uniform cache facilitate
fast data access and communication within the GPU.

Power Efficiency of the GPU

Shruthi T S,KSIT
This figure compares the performance of GPUs (Graphics Processing Units) and CPUs
(Central Processing Units) in terms of GFLOPS/W (Giga Floating Point Operations Per Second
per Watt) over time, and projects the performance of Exascale systems in the future. Here's
a breakdown:

• GPU Performance (Green Line): Shows a significant increase in performance from

2️01️0 to 2️01️6. The data point labeled "GPU - 5 Gflops/W" indicates the measured
performance of a GPU core in 2️01️1️.

• CPU Performance (Blue Line): Demonstrates a much slower rate of improvement

compared to GPUs over the same period. The data point labeled "CPU" indicates the
measured performance of a CPU core in 2️01️1️ (0.8 GFLOPS/W/core).

• Exascale (EF) Performance (Yellow Line): Projects a much higher performance level
(60 GFLOPS/W/core) for Exascale systems in the future (estimated around 2️01️1️). This
line represents the anticipated performance needed for future supercomputers.

o Memory, Storage, and Wide-Area Networking

This graph shows the improvement in memory chip capacity (in Mbit) and disk storage
capacity (in GB) over 33 years, from 1978 to 2011.
• Memory chips (blue line) increased from 64KB in 1981 to 64GB in 2011, showing a
steady and exponential growth.
Shruthi T S,KSIT
 • Disk capacity (red line) also grew significantly, from the Seagate ST-506 (0.01GB) in
the early 1980s to the Seagate Barracuda XT (3TB) in 2011.
• Various key storage devices, such as Maxtor DiamondMax, WDC WD1200JB, and
Hitachi GST, are highlighted along the timeline.
• The graph uses a logarithmic scale, indicating that both memory and disk capacities
followed an exponential growth pattern.

➢ System-Area Interconnects

In small computing clusters, nodes are typically connected via Ethernet switches or a
Local Area Network (LAN). The main types of networking architectures used in clusters
are:

1️.Local Area Network (LAN) – Connects client hosts to large servers for general
networking.
2️.Storage Area Network (SAN) – Links servers to storage devices (e.g., disk arrays) for
efficient data access.
3️.Network-Attached Storage (NAS) – Allows clients to connect directly to shared disk
storage.

• For small clusters, a simple setup using a Gigabit Ethernet switch and copper cables
can be used. These network solutions are commercially available and commonly
used in large clusters to ensure efficient data sharing and communication.
Wide-Area Networking

• Ethernet bandwidth has grown rapidly from 10 Mbps (1979) to 1 Gbps (1999) and
later to 40–100 Gbps (2011), with expectations of reaching 1 Tbps soon. In 2006,
different network types had bandwidths ranging from 1 Gbps (copper desktop) to
1,000 Gbps (international links).
• Network performance has been doubling faster than Moore’s Law, leading to the
rise of massively distributed systems.

Shruthi T S,KSIT
 • According to the IDC 2010 report, InfiniBand and Ethernet are expected to be
the dominant interconnect technologies in high-performance computing (HPC),
with Gigabit Ethernet widely used in data centers.

➢ Virtual Machines and Virtualization Middleware

• A conventional computer has a single OS image.

• This offers a rigid architecture that tightly couples application software to a specific
hardware platform.
• Some software running well on one machine may not be executable on another
platform with a different instruction set under a fixed OS.
• Virtual Machines (VMs): These are like independent computers running inside a real
computer. Each VM has its own OS and applications.
• This approach makes computers more efficient, flexible, and scalable, especially for
cloud computing.
• A regular computer has only one operating system, so it’s not very flexible.
• Hypervisor (Virtualization Middleware): This is the software that creates and
manages VMs. It sits between the hardware and the virtual machines, making sure each
VM gets the necessary resources (CPU, memory, storage, etc.).
• Example: Popular hypervisors include VMware, VirtualBox, KVM, and Microsoft
Hyper-V.
•

• Physical Machine (a): One OS, no virtualization.

• Native VM (b): Hypervisor runs directly on hardware (fastest & most efficient).
• Hosted VM (c): Hypervisor runs inside an OS (easier to set up but slower).
• Dual-Mode VM (d): Hybrid approach, improving performance.

Shruthi T S,KSIT
 • A Virtual Machine (VM) is a software-based system that mimics physical
hardware, allowing multiple operating systems to run on the same physical
machine. The Virtual Machine Monitor (VMM), also known as a hypervisor,
manages virtual resources and enables this functionality.
• There are three main VM architectures, as shown in Figure 1.12:
1. Native (Bare-Metal) VM – The hypervisor runs directly on the hardware, managing
VMs without requiring a host OS. Example: XEN hypervisor.

2. Host VM – The hypervisor runs in nonprivileged mode, meaning the host OS remains
unchanged. This allows VMs to be installed easily without modifying the existing
system.

3. Dual-Mode VM – The VMM operates at both user and supervisor levels, requiring
some modifications to the host OS for better integration.

•
VMs provide hardware independence, meaning applications and operating
systems can be bundled into virtual appliances that can run on different
hardware platforms.
• This flexibility makes VMs essential for cloud computing, software testing, and
resource-efficient computing environments.
➢ VM Primitive Operations

Shruthi T S,KSIT
 ➢ Virtual Infrastructures
A virtual infrastructure allows computing, storage, and networking resources
to be dynamically allocated to different applications, making IT environments
more efficient, cost-effective, and scalable. Let's break it down step by step:
1️. Separation of Hardware and Software
2️. Mapping Resources to Applications
3️. Benefits of Virtual Infrastructures
4️. Virtualization in Clusters, Clouds, and Grids

➢ Data Center Virtualization for Cloud Computing

• A data center is a facility that houses a large number of servers that store and
process data.
• Cloud companies (like Google, Amazon, Microsoft) use x86 processors,
terabyte disks, and Gigabit Ethernet because they are:
✓ Affordable
✔Widelyavailable
✔ Efficient for handling large-scale cloud services

1. Data Center Growth and Cost Breakdown:

• A large data center may be built with thousands of servers. Smaller data hundreds
of servers. centers are typically built hundreds of servers.

2. Low-Cost Design Philosophy:

Shruthi T S,KSIT
 • High-end switches and routers are too expensive for data centers. Instead of using
costly high-bandwidth networks, cloud providers opt for cheaper, commodity
hardware to keep costs low while maintaining efficiency.

➢ Convergence of Technologies
• Essentially, cloud computing is enabled by the convergence of technologies in
four areas:
o hardware virtualization and multi-core chips
o utility and grid computing
o SOA, Web 2.0, and WS mashups
o atonomic computing and data center automation.
• SOA(Service-Oriented Architecture) enables modular cloud applications,
• while Web 2.0 and mashups combine services for dynamic and scalable
solutions.
1.3 SYSTEM MODELS FOR DISTRIBUTED AND CLOUD COMPUTING

Clusters of Cooperative Computers

• A computing cluster is a group of computers working together as if they were
a single system
• They are used to handle large workloads and big data efficiently.
• A computing cluster distributes tasks across multiple computers to improve
performance and reliability. For example, Google Search uses large clusters to
process billions of search queries quickly and efficiently.
Diagram

▪ The diagram represents a computing cluster, which is a group of servers

working together as one system. These servers are connected through
a fast network (like Ethernet, Myrinet, or InfiniBand).
▪ 2. Key Parts of the Diagram:
▪ Servers (S₀, S₁, S₂, … Sₙ)

Shruthi T S,KSIT
 ▪ These are the individual computers (nodes) that make up the cluster.
▪ Each server works on different tasks, but together, they act as one
system.
▪ High-Speed Network (Yellow Area)
▪ The servers are connected using high-bandwidth networks such as
SAN, LAN, or NAS (Storage Area Network, Local Area Network, Network
Attached Storage).
▪ This ensures fast data transfer between the computers.
▪ I/O Devices & Disk Arrays
▪ The cluster shares storage (disk arrays) and input/output devices (like
hard drives or network connections).
▪ This helps all the servers access shared data efficiently.
▪ Gateway
▪ The cluster connects to the Internet through a gateway.
▪ The gateway acts like a door, allowing external users or systems to
communicate with the cluster.
▪ The Internet
This is where the cluster connects to external systems, allowing users to access it
remotely.

Single-System Image

• SSI allows multiple computers in a cluster to work together and appear as one
powerful system to the user.
• It enables all the computers in the cluster to share resources, making the system
easier to use and manage.
• Software or hardware makes multiple computers work together as one system;
without it, they would function separately.

Grid Computing Infrastructures

• Grid computing is a technology that connects multiple computers across different

locations to work together as a single system.

• These computers share resources such as processing power, storage, and data to
solve complex problems efficiently.

• Grid computing helps in fields like weather forecasting, space research, and medical
simulations by using multiple computers together.

• This evolution has contributed to the rapid growth of the IT economy, which was
projected to rise from $1️ trillion in 2️001️ to $2️0 trillion by 2️01️5.

Over the last 30 years, computing has evolved in three stages:

Shruthi T S,KSIT
 1️. Internet Computing – Allowed computers to connect remotely (e.g., Telnet
command).
2️. Web Computing – Allowed people to access websites and data over the Internet
(e.g., HTTP for browsing).
3️. Grid Computing – Enables multiple computers in different locations to work
together on complex tasks at the same time.

• Grid computing connects distant computers so they can share resources and collaborate
on large computing tasks.
• This is useful for scientific research, data analysis, and solving complex problems.

Ex: Like multiple people working on a Google Doc together, sharing power and resources to
complete a task faster.

• 1.3.2.2 Grid Families

Grid computing requires new technologies, software, networks, and hardware to function
efficiently. Many national projects and industrial platforms have been developed by
companies like IBM, Microsoft, Sun, HP, Dell, and Cisco to support grid computing.

• Grid Service Providers (GSPs) have emerged, similar to how the Internet and web
services grew over time.

• Grids are classified into two main types:

Shruthi T S,KSIT
 1️. Computational/Data Grids – Used for large-scale computing and data
processing, often at the national level.

2️. Peer-to-Peer (P2️P) Grids – Decentralized networks where devices share

resources directly.

1️. Grid (Yellow Section):

o Contains powerful computing resources like servers, databases, and

expensive equipment (e.g., telescopes, supercomputers).

o These resources are connected through an IP broadband network to share

computing power and data.

2️. The Internet (Green Section):

o Connects users (individuals, organizations) to the grid.

o Acts as a bridge between the grid resources and end-users.

3️. End-User Devices (Blue Section):

o Various devices like laptops, desktop computers, mobile phones, and other
gadgets can access the grid services.

o Users can perform complex calculations, store data, or access powerful

computing resources remotely.

How It Works:

• Organizations and research institutes share their computing power to create a grid.

• Users from different locations connect via the internet to access these shared
resources.

Shruthi T S,KSIT
 • This setup allows users to perform high-performance computing tasks without
needing expensive local hardware.

1.3.3 Peer-to-Peer Network Families

o An example of a well-established distributed system is the client-server

architecture.
o In this scenario, client machines (PCs and workstations) are connected to a
central server for compute, e-mail, file access, and database applications. The
P2P architecture offers a distributed model of networked systems.
o First, a P2P network is client-oriented instead of server-oriented.
o In this section, P2P systems are introduced at the physical level and overlay
networks at the logical level.

1.3.3.1 P2P Systems

▪ In a P2P system, every node acts as both a client and a server, providing part
of the system resources.
▪ Peer machines are simply client computers connected to the Internet. All
client machines act autonomously to join or leave the system freely.
▪ This implies that no master-slave relationship exists among the peers.
▪ No central coordination or central database is needed. In other words, no
peer machine has a global view of the entire P2P system.
▪ The system is self-organizing with distributed control.
▪ Figure 1.17 shows the architecture of a P2P network at two abstraction levels.
Initially, the peers are totally unrelated. Each peer machine joins or leaves the
P2P network voluntarily.
▪ Only the participating peers form the physical network at any time. Unlike
the cluster or grid, a P2P network does not use a dedicated interconnection
network.
▪ The physical network is simply an ad hoc network formed at various Internet
domains randomly using the TCP/IP and NAI protocols. Thus, the physical
network varies in size and topology dynamically due to the free membership
in the P2P network.
1.3.3.2 Overlay Networks

Shruthi T S,KSIT
 ▪ Data items or files are distributed in the participating peers. Based on
communication or file-sharing needs, the peer IDs form an overlay network at
the logical level.
▪ In a P2P (Peer-to-Peer) system, data items or files are distributed among
participating peers.
▪ A logical overlay network is formed by mapping each peer's ID to a virtual
network, independent of the physical connections. This overlay network
dynamically updates when peers join or leave the system.
There are two types of overlay networks:
1. Unstructured Overlay Networks – Nodes connect randomly, with no fixed
routes for message or file transmission. Queries are sent through flooding,
leading to high network traffic and unpredictable search results.
2. Structured Overlay Networks – Follow predefined connectivity rules and
structured topology. Efficient routing mechanisms ensure better search
performance and reduced traffic.

➢ P2P Application Families

Peer-to-peer (P2P) networks are categorized into four main application families based on
their use cases:
1. Distributed File Sharing – This category includes P2P networks used for sharing
digital content such as music and videos. Popular examples include Gnutella, Napster,
and BitTorrent.

2. Collaboration Networks – These P2P applications facilitate real-time communication

and collaboration, such as MSN, Skype, and instant messaging platforms.

3. Distributed P2P Computing – These networks harness computing power from

multiple machines for specific applications. For example, SETI@home utilizes over 3
million Internet hosts to provide 25 Tflops of computing power.

Shruthi T S,KSIT
 4. P2P Platforms and Middleware – Platforms like JXTA, .NET, and
FightingAID@home support various P2P applications by providing essential services
such as naming, discovery, communication, security, and resource aggregation.

➢ Cloud Computing over the Internet

▪ Cloud computing is revolutionizing IT by shifting computing and data storage from
individual desktops to large data centers.
▪ Experts like Gordon Bell, Jim Gray, and Alex Szalay highlight that computational
science is becoming more data-intensive, requiring balanced supercomputing systems
with high-speed I/O and networking.
▪ Instead of moving large datasets to local machines, cloud computing brings
computations to the data, ensuring efficiency and scalability.
▪ IBM defines cloud computing as a pool of virtualized resources capable of hosting
diverse workloads, from backend jobs to interactive applications.
▪ Cloud environments enable rapid provisioning, scalability, self-recovery, and real-
time resource monitoring, ensuring seamless workload management and failure
recovery. This model enhances IT flexibility, efficiency, and resilience, making cloud
computing a key driver of modern data-intensive applications.

➢ Internet Clouds
▪ Cloud computing utilizes a virtualized platform with on-demand elastic
resources, dynamically provisioning hardware, software, and data.
▪ It shifts desktop computing to a service-oriented model using server clusters
and large databases in data centers.
▪ Cloud computing is cost-effective and benefits both users and providers by
leveraging machine virtualization. The cloud must be secure, trustworthy,
and dependable to support multiple applications.
▪ Some view the cloud as a centralized resource pool, while others see it as a
distributed computing network across multiple servers.

➢ The Cloud Landscape

Shruthi T S,KSIT
Traditional distributed computing systems face challenges like high maintenance, poor
resource utilization, and rising costs. Cloud computing addresses these issues by offering on-
demand computing through three key service models:

1. Infrastructure as a Service (IaaS) – Provides virtualized infrastructure (servers,

storage, networks) where users can deploy and run applications without managing the
underlying hardware.

2. Platform as a Service (PaaS) – Offers a virtualized platform with middleware,

databases, and development tools, enabling users to build and deploy applications
without managing infrastructure.

3. Software as a Service (SaaS) – Delivers cloud-hosted applications (CRM, ERP, HR,

etc.) via a browser, eliminating the need for upfront investments in servers or software
licenses.

Cloud computing reduces costs and improves scalability, making it a preferred model for
modern IT solutions.

The following list highlights eight reasons to adapt the cloud for upgraded Internet applications
and web services:

1. Desired location in areas with protected space and higher energy efficiency
2. Sharing of peak-load capacity among a large pool of users, improving overall utilization

3. Separation of infrastructure maintenance duties from domain-specific application

development

4. Significant reduction in cloud computing cost, compared with traditional computing

paradigms
5. Cloud computing programming and application development

Shruthi T S,KSIT
6. Service and data discovery and content/service distribution

7. Privacy, security, copyright, and reliability issues

8. Service agreements, business models, and pricing policies

➢ SOFTWARE ENVIRONMENTS FOR DISTRIBUTED SYSTEMS AND CLOUDS

There are 3️ different software environment used for distributed s/m and clouds they are

1️. Service-Oriented Architecture(SOA)-it is a s/w services

2️. Layered Architecture in web services-The layered architecture for web services and grids
is a structured way to organize different components in distributed computing ex for web
services is javascriptnand xml

3️. Java RMI and CORBA-rmi(remote method invocation)accessing client & server through
remote.cobra(common object request broker architecture)-one of the famous middleware
technology which can be used communication among various services

SOA-it is a s/w services,design approach consider an aws application or networking there

are many services are hosted in that so to access those services with the help of a particular
well developed or defined appln we can use one services called as soa,

Soa is used to build grids

• Bottom Layer: Bit-Level Internet (Networking Layer)

- Handles the actual data transmission over the internet.
Key components are tcp….
• Middle Layer: Base Hosting Environment
- Acts as a foundation for web services and grid applications.s upports running
services, whether they are web services, Java applications, or distributed objects
(CORBA).

Shruthi T S,KSIT
 • Top Layer: Higher-Level Services (Web Services & Grid Services)
- Provides specific application-level services for users and businesses.
➢ EVOLUTION OF SOA

Shruthi T S,KSIT
 Trends toward Distributed Operating Systems
• DCE is a middleware-based system for distributed computing environments.
• The Amoeba was academically developed at Free University in the Netherlands.
• The Open Software Foundation (OSF) has pushed the use of DCE for distributed
computing.
• However, the Amoeba, DCE, and MOSIX2 are still research prototypes that are
primarily used in academia.
No successful commercial OS products followed these research systems

➢ Parallel and Distributed Programming Models

• This diagram represents a cloud computing model where user data, applications,
operating systems, and hardware are separated from each other.

Shruthi T S,KSIT
 • It shows how cloud computing allows flexibility, enabling users to access their data
and applications from different devices and operating systems.
1️. Hardware (Bottom Layer)
• Includes laptops, tablets, and smartphones.
• Cloud computing allows users to access their apps and data from any device, like
laptops, tablets, or smartphones.
1️. Operating Systems Layer
• Includes operating systems like Windows, Linux, and MacOS.
• users can open their files and use applications on any operating system (OS),
such as Windows, Linux, or macOS. For example, if you save a document on
Google Drive using a Windows computer, you can still access and edit it later
on a Mac or a smartphone without any issues. Cloud computing removes the
limitation of being tied to a specific OS. users can switch between different
devices and operating systems
2️. Application Layer
• Includes various applications like Microsoft Word, web browsers, and open-source
software.
• traditional applications are designed to run on specific operating systems, like
Microsoft Word for Windows or macOS. However, cloud-based applications like
Google Docs work directly through a web browser, so they don’t depend on the OS.
Whether you use Windows, Linux, MacOS, or even a smartphone, you can access and
use Google Docs without needing to install anything special.
• d) User Data & Storage (Top Layer)
1️. Data storage is centralized in the cloud,
o Data can be accessed from any device, anytime, anywhere.
o Cloud computing advantage: Even if an application is lost or replaced, the
user’s data remains secure and accessible.

Hadoop Library Hadoop offers a software platform that was originally developed by a Yahoo!
group. The pack age enables users to write and run applications over vast amounts of
Shruthi T S,KSIT
 distributed data. Users can easily scale Hadoop to store and process petabytes of data in the
web space. Also, Hadoop is economical in that it comes with an open source version of
MapReduce that minimizes overhead

➢ Performance Metrics and Scalability Analysis

▪ CPU speed is measured in MIPS(MIPS – Million Instructions Per Second),
Tflops(Tera Floating Point Operations Per Second), or TPS(Transactions Per Second
).
▪ while network performance depends on bandwidth (Mbps) and latency. A good
system should have low latency, high bandwidth, and minimal overhead (e.g., fast OS
boot time and I/O speed).
➢ Dimensions of Scalability
• Scalability in distributed systems ensures performance improvements while
maintaining compatibility with existing resources.
• It can be costly, and system scaling may increase or decrease resources based on
practical needs. The key dimensions of scalability include:
• • Size Scalability: Enhancing performance by adding processors, memory, or
storage, though not all architectures scale equally.
• • Software Scalability: Upgrading OS, compilers, and applications, but ensuring
compatibility with large systems is challenging.
• • Application Scalability: Adjusting problem size to match system capabilities for
better efficiency.
• • Technology Scalability: Adapting to new hardware and networking technologies,
considering generation updates, space constraints, and component heterogeneity.
• scalable performance is estimated against the multiplicity of OS images in
distributed systems deployed up to 2️01️0.
• NUMA (nonuniform memory access) machines are often made out of SMP nodes
with distributed, shared memory.

Shruthi T S,KSIT
 • A NUMA machine can run with multiple operating systems, and can scale to a few
thousand processors communicating with the MPI library.
• the number of OS images in a large grid structure could be hundreds or thousands
fewer than the total number of processors in the grid.
• A P2️P network can easily scale to millions of independent peer nodes, essentially
desktop machines.

1.5.1.3 Scalability versus OS Image Count

• Scalability in distributed systems depends on the number of OS images. SMP has one
OS image but limited scalability, while NUMA uses multiple OS images and scales
better.
• Clusters and clouds scale higher, with clusters depending on active nodes and clouds
using VMs.
• Grids combine clusters, mainframes, or supercomputers with fewer OS images than
processors. P2️P networks can scale to millions of independent machines.

The image is a graph showing system scalability versus OS image count based on 2010
technology.
• The X-axis represents the number of OS images in a system.

• The Y-axis represents the number of processors or cores in a system (scalability).

Key Insights:

• SMP (Symmetric Multiprocessing) has low scalability with only one OS image and
a limited number of processors.

• NUMA (Non-Uniform Memory Access) has more OS images and scales better but is
still limited compared to clusters.

Shruthi T S,KSIT
 • Clusters and clouds have higher scalability, with cloud systems acting as virtualized
clusters.

• Grids have an even higher OS image count, as they connect multiple clusters or high-
end machines.

• P2P (Peer-to-Peer) networks can scale to millions of independent nodes, having the
highest OS image count but varying performance.

The diagram highlights how different system architectures balance scalability and OS image
multiplicity.

➢ Amdahl’s Law
• Amdahl’s Law describes the theoretical speedup of a program when executed on
multiple processors in a parallel computing environment.
• It is about parallel vs sequential work
• It helps to speed up the program when the multiple processors running in a
parallelparallel computing environment.

• It accounts for the sequential bottleneck (α)—the fraction of the code that must be
executed sequentially
• the parallelizable portion (1 - α), which can be distributed across multiple processors.
Acc to the formula
• αT represents the sequential execution time, (1 - α)T/n represents the parallel
execution time

• which means that even if an infinite number of processors are used, the speedup
cannot exceed 1/α. For example, if α = 0.25, the maximum speedup is 4, no matter

Shruthi T S,KSIT
 how many processors are added. α (Alpha) = The fraction of the program that must
run sequentially (cannot be parallelized).

• (1 - α) = The portion that can be parallelized.

• n = The number of processors (workers).

• The goal is to make α as small as possible to achieve better speedup.

Problem with Fixed Workload (Amdahl’s Law and Efficiency)

When we run a program on multiple processors, we assume the total work stays the same
(fixed workload). However, adding more processors does not always make the system
efficient.

Using too many processors is wasteful if some work must be done sequentially. It's better to
reduce the sequential part or increase the workload to use processors efficiently.

Gustafson’s Law
• Gustafson’s Law(Guftas law) addresses the efficiency problem in large
processor clusters by scaling the workload instead of keeping it fixed.

• where α is the sequential portion and n is the number of processors.

Shruthi T S,KSIT
 • Unlike Amdahl’s Law, which assumes a constant problem size, Gustafson’s Law
increases the parallelizable portion of the workload to match the number of
processors.
• Use Amdahl’s Law for fixed workloads (limited speedup due to sequential
bottleneck).
Use Gustafson’s Law for scaled workloads (better efficiency as workload
grows with processors).
Scaling the problem size helps maximize processor utilization in large
systems.

Shruthi T S,KSIT