CL - 3
CL - 3
Academic year-2023-24
Lab Manual
BE
AI & DS
Semester VIII
Prepared by,
Ms. Bormane P. D.
2
To be amongst the top artificial intelligent & data science programs for catering
to the changing needs of the industry and society.
Mission
PSO1 Problem Solving and Programming Skills: Graduates will be able to apply
programming skills to identify, modify and test algorithms that apply intelligence to
make realistic decisions in problem solving.
PSO2 Professional Skills: Graduates will be able to collect, analyze, interpret and
visualize data to solve problems in agriculture, automation, finance and medical
domains.
Index
Page
Sr. No Title Of Experiment CO PO PSO
No
Bridge the Gap. C406.1 PO1, PO2 1 6
Objective: To explain the key concepts of network segmentation and its benefits.
Theory:
Add here
Assignment 1
Objective: To design and implement a distributed application using Remote Procedure Call (RPC)
for remote computation.
Problem statement: Design a distributed application using RPC for remote computation where
client submits an integer value to the server and server calculates factorial and returns the result to
the client program.
Theory:
RPC is a communication paradigm that allows programs to execute procedures or functions on a
remote server as if they were local. It abstracts the complexities of network communication, making
it appear to the developer as a simple local function call. RPC involves two main components: a
client and a server.
Design Considerations:
- Interface Definition: Define the interface specifying the remote procedure to be called. In
this case, it includes a method for calculating the factorial.
- Serialization: Data serialization is essential for transmitting parameters and results between
the client and server. Common formats include JSON or XML.
- Server Implementation: The server implements the specified remote procedure (calculating
factorial) and exposes it to clients. It listens for incoming requests and responds accordingly.
- Client Implementation: The client initiates the RPC, sending the required parameters to the
server and handling the response.
Steps:
1. The client sends an RPC request to the server, including the integer for which the factorial
needs to be calculated.
2. The server receives the request, extracts the parameter, and computes the factorial.
4. The client receives the result, deserializes it, and processes the final output.
OUTPUT:
Assignment 2
Objective: To design and implement a distributed application using Remote Method Invocation
(RMI).
Problem statement: Design a distributed application using RMI for remote computation where
client submits two strings to the server and server returns the concatenation of the given strings.
Theory:
Remote Method Invocation (RMI) is a Java-based mechanism that allows objects in one Java Virtual
Machine (JVM) to invoke methods on objects in another JVM, making it a powerful tool for building
distributed applications. RMI abstracts the complexities of network communication and provides a
way for objects in different JVMs to interact seamlessly.
- RMI Architecture: In an RMI-based system, there are typically three main components: the
client, the server, and the RMI registry. The RMI registry acts as a lookup service, enabling clients
to find the remote objects they need to invoke.
- Remote Interface: Objects that are intended to be invoked remotely must implement a
remote interface. This interface defines the methods that can be called remotely, and its
implementation is provided by the server.
- Serialization in RMI: Like in RPC, data must be serialized for transmission between the
client and server. Java's built-in serialization mechanism is often used to serialize parameters and
return values.
Design Considerations:
- Remote Interface Definition: Define a remote interface that includes a method for
concatenating two strings. This interface should extend the `Remote` interface to signify that it is
intended for remote invocation.
- Server Implementation: Implement the server class that extends `Unicast Remote Object`
and implements the remote interface. This class will provide the actual implementation of the string
concatenation method.
Steps:
1. The client obtains a reference to the remote object from the RMI registry using a specified
name or URL.
2. The client invokes the remote method on the obtained remote object, passing two strings as
parameters.
3. The RMI runtime handles the details of parameter serialization, network communication, and
method invocation.
4. The server receives the remote method invocation, deserializes the parameters, performs the
string concatenation, and returns the result.
5. The RMI runtime handles the serialization of the result, sends it back to the client, and the
client deserializes the result for further processing.
OUTPUT:
Assignment 3
Problem Statement: Design a distributed application using MapReduce under Hadoop for: a)
Character counting in a given text file. b) Counting no. of occurrences of every word in a given text
file.
Theory:
MapReduce is a programming model and processing framework designed for parallel processing of
large datasets across distributed clusters. It consists of two main phases: the Map phase and the
Reduce phase.
- Map Phase: During the Map phase, input data is divided into chunks, and a map function is
applied to each chunk independently. The map function processes the input and emits key-value
pairs.
- Shuffling and Sorting: The emitted key-value pairs are shuffled and sorted based on keys.
This phase ensures that all values for a particular key end up together, enabling efficient processing
in the next phase.
- Reduce Phase: In the Reduce phase, a reduce function is applied to groups of key-value
pairs with the same key. The reduce function aggregates and processes the values for each key,
producing the final output.
Design Considerations:
a) Character Counting:
Map Function (Character Counting): The map function takes as input a chunk of text and emits
key-value pairs, where the key is the character, and the value is 1.
Reduce Function (Character Counting): The reduce function sums up the values for each
character, providing the total count of each character in the text.
b) Word Counting:
Map Function (Word Counting): The map function tokenizes the input text into words, emitting
key-value pairs with the word as the key and 1 as the value.
Reduce Function (Word Counting): The reduce function sums up the values for each word,
providing the total count of occurrences for each word in the text.
Hadoop is an open-source framework that supports the distributed processing of large datasets using
the MapReduce programming model. It includes the Hadoop Distributed File System (HDFS) for
distributed storage and the Hadoop MapReduce engine for parallel processing.
Input data is stored in HDFS, which divides the data into blocks and distributes them across the
nodes in the Hadoop cluster. Hadoop coordinates the execution of MapReduce jobs across the
cluster, managing the distribution of tasks to individual nodes.
Steps: add
OUTPUT:
Assignment 4
Title: Simulate requests between client and servers using the load balancing algorithms.
Objective: To design and implement a simulation of client requests and their distribution among
servers using various load balancing algorithms.
Problem Statement: Write code to simulate requests coming from clients and distribute them
among the servers using the load balancing algorithms.
Theory:
Load balancing is a critical aspect of distributed systems that involves distributing incoming requests
or tasks among multiple servers to ensure optimal resource utilization, improve system performance,
and prevent overloading of individual servers. Several load balancing algorithms exist, each with its
own approach to distributing the load.
a) Round Robin: In Round Robin, each incoming request is sequentially assigned to the next
available server in a circular manner. This ensures a fair distribution of requests among servers.
b) Random: The Random algorithm randomly selects a server from the available pool to handle
each incoming request. While simple, it may lead to uneven distribution in the long run.
c) Least Connections: The Least Connections algorithm assigns new requests to the server with the
fewest active connections. This aims to distribute load based on current server utilization.
d) Weighted Round Robin: Weighted Round Robin assigns weights to servers, influencing the
probability of selection. Servers with higher weights receive more requests, allowing for
proportional load distribution based on server capacity.
The simulation involves generating a stream of incoming client requests and applying the selected
load balancing algorithm to distribute these requests among a predefined set of servers. The
performance metrics, such as server load, response time, and request distribution, can be analyzed
to assess the effectiveness of each load balancing algorithm.
Conclusion: We have a hands-on exploration of load balancing in distributed systems through the
simulation of client requests and their distribution among servers. By implementing and comparing
various load balancing algorithms such as Round Robin, Random, Least Connections, and Weighted
Round Robin, we got insights into their impact on system performance and resource utilization.
OUTPUT:
Assignment 5
Theory:
The Clonal Selection Algorithm is an optimization algorithm inspired by the immune system's clonal
selection process. It is commonly used for solving optimization problems, particularly in the domain
of artificial immune systems. The key steps of the algorithm include:
- Affinity Maturation: Evaluate the affinity of each antibody to the problem at hand. Affinity
represents how well an antibody performs in the given context.
- Cloning: Clone antibodies in proportion to their affinities. Antibodies with higher affinities are
duplicated more, simulating the proliferation of effective solutions.
- Hypermutation: Introduce variations or mutations to the cloned antibodies to explore the solution
space further.
- Selection: Evaluate the performance of the mutated antibodies, and select the best among the
original and mutated antibodies.
- Replacement: Replace the less fit antibodies with the newly selected ones to form the next
generation.
Implementation Considerations:
- Objective Function: Implement the objective function that measures the affinity or fitness of an
antibody.
- Cloning and Hypermutation: Develop mechanisms for cloning antibodies based on their
affinities and introducing controlled variations through hypermutation.
- Selection and Replacement: Implement a selection mechanism to choose the most promising
antibodies and replace the less fit ones in each generation.
OUTPUT:
Assignment 6
Objective: To apply neural style transfer, a deep learning technique, to create an artwork based
on a given image.
Problem Statement: Create and Art with Neural style transfer on a given image using deep
learning.
Theory:
Deep Learning and Neural Networks:
Neural style transfer leverages deep learning, specifically convolutional neural networks (CNNs).
CNNs are powerful in extracting hierarchical features from images, making them suitable for tasks
like image recognition and style transfer.
CNNs consist of layers with learnable filters that capture different features in an image. In neural
style transfer, features from specific layers are utilized to separate content and style information.
Neural style transfer is a technique in deep learning that combines the content of one image with
the artistic style of another image. It involves the use of convolutional neural networks (CNNs) to
separate and recombine content and style features from two input images.
The content of an image is captured by the higher-level features in the CNN, typically deeper
layers, while the style is represented by the correlations between the different features at various
scales.
Loss Functions:
The optimization process involves defining content and style loss functions. The content loss
measures the difference between the generated image and the content image, while the style loss
quantifies the difference in style between the generated image and the style image.
Optimization:
- The objective is to minimize the combined loss function, adjusting the pixel values of the
generated image iteratively to match both the content and style of the input images.
Steps: Add
Conclusion: We have hands-on experience with neural style transfer, showcasing the application
of deep learning in creating artistic images. By combining the content of one image with the style of
another, it demonstrates the capability of neural networks to generate visually appealing and unique
artworks through the process of feature extraction, loss optimization, and image synthesis.
OUTPUT:
Assignment 7
Objective: To apply the principles of Artificial Immune Pattern Recognition (AIPR) to classify
different types of structural damage.
Problem Statement: To apply the artificial immune pattern recognition to perform a task of
structure damage Classification.
Theory:
Artificial immune pattern recognition draws inspiration from the natural immune system's ability to
identify and respond to foreign entities. In the context of structure damage classification, this
involves creating a computational model that mimics the key features of the immune system. The
theory encompasses the following key components:
Pattern Recognition: The system must be designed to recognize patterns associated with different
types of structural damage. This involves the extraction of relevant features from input data, such as
images or sensor readings.
Artificial Immune System (AIS): The AIS is a computational model that incorporates concepts
from immunology to perform pattern recognition. Key elements include antigens representing
structural damage patterns, antibodies for pattern recognition, and a learning mechanism for
adaptation.
Training and Adaptation: The system should be capable of learning and adapting over time. This
involves training the artificial immune system on labeled data to enhance its ability to accurately
classify structural damage patterns.
Performance Metrics: Evaluation metrics, such as accuracy, precision, recall, and F1 score, will be
employed to assess the effectiveness of the artificial immune pattern recognition system in the task
of structure damage classification.
Clonal Selection: Mimicking the clonal selection process in the immune system, the algorithm will
generate diverse and specialized detectors to enhance its ability to identify specific damage patterns.
Memory Cells and Learning: The incorporation of memory cells allows the algorithm to retain
information about previously encountered damage patterns, enabling continuous learning and
adaptation.
Self-Nonself Discrimination: Similar to the immune system's ability to distinguish between self
and nonself entities, the algorithm will differentiate between normal structural features and
anomalous patterns indicative of damage.
Feedback and Adaptation: The system will dynamically adapt based on feedback, refining its
recognition capabilities over time to improve accuracy and reduce false positives/negatives.
Diagram/Steps: Add
Conclusion: We have applied Artificial Immune Pattern Recognition for structural damage
classification which has the potential to greatly improve our ability to detect and categorize damage
to structures.
OUTPUT:
Assignment 8
Objective: To implement Distributed Evolutionary Algorithms using the DEAP library in Python.
Theory: Evolutionary Algorithms (EAs) are optimization techniques inspired by the process of
natural selection. They are particularly effective in solving complex optimization problems where
traditional methods may struggle. The theoretical foundation of DEAP and distributed evolutionary
algorithms involves several key concepts.
DEAP Library Overview: A detailed exploration of the DEAP library is crucial for understanding
its capabilities. This involves discussing the core components of DEAP, such as the Toolbox,
Operators, and Algorithms. Examples of implementing basic evolutionary algorithms using DEAP's
single-node capabilities should be covered.
Distributed Evolutionary Algorithms: Expanding on the foundation laid by DEAP, this section
delves into the modifications required for distributed computing. Discuss how the population is
divided among multiple nodes, how parallel fitness evaluations are conducted, and how information
is exchanged between nodes. Explore strategies for load balancing and synchronization in a
distributed EA.
Implementation Details: Provide a step-by-step guide on how to modify a basic DEAP algorithm
for distributed computing. This involves adapting the code to handle distributed populations, parallel
fitness evaluations, and communication between nodes. Highlight any challenges encountered
during the implementation and the strategies used to overcome them.
Challenges and Future Directions: Discuss the challenges faced during the implementation and
potential improvements or extensions to the distributed evolutionary algorithm. This could involve
exploring additional parallelization strategies, optimizing communication protocols, or adapting the
algorithm for specific distributed computing environments.
Diagram/Steps: Add
OUTPUT:
Assignment 9
Objective: To design and develop a distributed application that utilizes the MapReduce paradigm
to analyze weather data.
Problem Statement: Design and develop a distributed application to find the coolest/hottest
year from the available weather data. Use weather data from the Internet and process it using
MapReduce.
Theory:
The development and understanding of the distributed application for identifying the coolest/hottest
year from weather data using MapReduce involve several key concepts and components. This
comprehensive exploration covers various aspects, from the MapReduce paradigm to the integration
of weather data, and the overall design and implementation of the distributed system.
MapReduce Paradigm:
MapReduce is a programming model and parallel processing framework that enables the efficient
processing of large datasets across distributed clusters. The paradigm involves two main phases: the
Map phase, where input data is processed and transformed into intermediate key-value pairs, and the
Reduce phase, where these intermediate results are aggregated to produce the final output.
Understanding the fundamentals of MapReduce is crucial for developing scalable and distributed
applications.
Implementation Details:
The practical implementation of the MapReduce application involves choosing an appropriate
programming language, such as Java or Python, and selecting a suitable distributed computing
framework, such as Apache Hadoop. The design should consider factors such as data partitioning,
key-value pair definition, and task scheduling to ensure an efficient and effective execution of the
MapReduce algorithm.
Evaluation Metrics:
To assess the performance and accuracy of the distributed application, various metrics must be
considered. These metrics may include execution time, resource utilization, scalability under
increasing dataset sizes, and the correctness of identified coolest and hottest years. A comprehensive
evaluation framework will provide insights into the application's strengths and areas for
improvement.
Diagram/Steps: Add
Conclusion: We have developed a distributed application for identifying the coolest and hottest
years from weather data using the MapReduce paradigm is a complex and multifaceted task.
OUTPUT:
Assignment 10
Objective: To implement Ant Colony Optimization (ACO) to solve the Traveling Salesman
Problem (TSP).
Problem Statement: Implement Ant colony optimization by solving the Traveling salesman
problem using python. Problem statement- A salesman needs to visit a set of cities exactly once and
return to the original city. The task is to find the shortest possible route that the salesman can take
to visit all the cities and return to the starting city.
Theory:
Ant Colony Optimization (ACO) is a nature-inspired optimization algorithm based on the foraging
behavior of ants. It falls under the broader category of swarm intelligence, where a group of agents
collaborates to find solutions to complex problems. In the case of ACO, the algorithm mimics the
way real ants find the shortest path between their nest and a food source.
Ant Colony Optimization Basics: ACO is based on the concept of pheromones, chemical
substances that ants deposit on the ground to communicate with each other. The intensity of the
pheromone trail influences the probability of other ants choosing the same path. In the algorithm,
artificial ants traverse the solution space, depositing pheromones on the edges of the graph
representing possible routes.
Traveling Salesman Problem (TSP) The Traveling Salesman Problem involves finding the shortest
possible route that visits a set of cities exactly once and returns to the starting city. TSP is a classic
NP-hard problem with numerous applications in logistics, transportation, and network design.
Various heuristic and metaheuristic algorithms have been applied to solve TSP, and ACO is
particularly suitable due to its ability to handle combinatorial optimization problems efficiently.
Algorithm Steps: Initialization: Define the cities and distances between them, set pheromone
levels on edges. b. Ant Movement: Artificial ants construct solutions by probabilistically choosing
paths based on pheromone levels. c. Pheromone Update: Evaporate existing pheromones and
deposit new pheromones based on the quality of the solutions found. d. Termination: Repeat the
ant movement and pheromone update steps for a predefined number of iterations or until a
convergence criterion is met.
Parameter Tuning and Enhancements: Fine-tuning the parameters of the ACO algorithm, such as
the pheromone evaporation rate and exploration-exploitation balance, is crucial for achieving
optimal results. Additionally, enhancements like local search mechanisms can be incorporated to
further improve solution quality.
Diagram/Steps: Add
Conclusion: We have implemented Ant Colony Optimization for solving the Traveling Salesman
Problem using Python. ACO, inspired by the foraging behavior of ants, proves to be a powerful
metaheuristic for addressing combinatorial optimization problems.
OUTPUT:
Theory:
Add from VLab: -- introduction, theory, procedure/diagram
Conclusion: We have explored and simulated fuzzy control and its applications using VLab.
OUTPUT:
Screenshots