Parallel Computing(VisvesvarayaTechnologicalUniversity)

Department of ComputerScience &Engineering

LAB MANUAL

PARALLELCOMPUTING(BCS702)
(2024-2025)
 (BCS702) ParallelComputingLabExperiments
Sl. Experiments
No
1. Write a OpenMP program to sort an array onnelementsusing bothsequentialand parallelmergesort
(using Section). Record the difference in execution time.

What is OpenMp?
OpenMP is a set ofcompiler directives as wellas anAPI for programs written inC, C++, or
FORTRAN that provides support for parallel programming in shared- memoryenvironments.
OpenMPidentifiesparallelregionsas blocksofcodethat may run in parallel. Application developers
insert compiler directives into their code at
parallelregions,andthesedirectivesinstructtheOpenMPrun-timelibrarytoexecute the region in
parallel.
What is mergesort?
Merge sort is a divide and conquer algorithm that sortsthe input arrayby breakingit into subarrays
untiltheyhave 1 element, andthen merging the results back into a
sortedarray.ThetimeandspacecomplexitiesareO(NlogN)andO(n)respectively. Sequential Merge
Sort
SequentialmergesortinCisasortingalgorithmthatfollowsthedivide-and-conquer paradigm.
Itrecursivelydividesanarrayintotwohalvesuntilthesub-arrayscontain onlyoneelement (which is
considered sorted).Then, it merges these sub-arrays in a sorted manner to produce new sorted sub-
arrays,repeating this process until the entire array is sorted.
ParallelMergeSort
Parallel merge sort is an algorithm that sorts a list by recursively dividing it into sublists, sorting
them independently using multiple threads or processes, and then merging the sorted sublists. This
approach leverages parallel processing to improve sorting speed, especially for large datasets.
Section:
Sections are independent blocks of code, able to be assigned to separate threads if they are
available.

Code:
#include<stdio.h>
#include<stdlib.h>
#include <omp.h>
void merge(int arr[],int left,int mid,int right)
{
int n1 = mid - left + 1;
intn2=right-mid;
int L[n1], R[n2];
for(int i=0;i<n1;i++)
L[i] = arr[left + i];
for(int j=0;j<n2;j++) R[j]
arr[mid + 1 + j];
int i = 0, j = 0, k = left;
while(i<n1 &&j<n2)
{
if (L[i] <= R[j])
arr[k++]=L[i++];
else
arr[k++]=R[j++];
}
while (i < n1) arr[k+
+]=L[i++]; while (j <
n2) arr[k++]=R[j++];
}
//SequentialMerge Sort
void mergeSortSequential(intarr[],intleft,intright)
{
if (left < right)
{
int mid = left + (right - left) / 2;
mergeSortSequential(arr, left, mid);
mergeSortSequential(arr,mid+1,right);
merge(arr, left, mid, right);
}
}
//ParallelMerge Sort
voidmergeSortParallel(intarr[],intleft,intright){ if
(left < right) {
int mid=left+(right -left)/2; #pragma
omp parallel sections
{
#pragma omp section
mergeSortParallel(arr, left, mid);
#pragma omp section
mergeSortParallel(arr,mid+1,right);
}
merge(arr,left,mid,right);
}
}
int main()
{ int n;
printf("Enter number of elements:");
scanf("%d", &n);
int*arr1=(int*)malloc(n*sizeof(int));
int*arr2=(int*)malloc(n*sizeof(int));
printf("Enter %d elements: ", n);
for(int i=0;i<n;i++)
{ scanf("%d", &arr1[i]);
arr2[i] =arr1[i];//Copyforparallelsorting
}
doublestart,end;
//SequentialSort
start = omp_get_wtime();
mergeSortSequential(arr1,0,n-1);
end = omp_get_wtime();
printf("SequentialMergeSortTime:%fseconds\n",end-start);
//ParallelSort
start=omp_get_wtime();
mergeSortParallel(arr2,0,n-1);
end = omp_get_wtime();
 printf("ParallelMergeSortTime:%fseconds\n",end-start);
printf("Sorted array: ");
for(int i=0;i<n;i++)
printf("%d ", arr1[i]);
printf("\n");
free(arr1);
free(arr2);
return 0;
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$ gcc -fopenmp
mergesort.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$ ./a.out
Enter number of elements: 6
Enter6elements: 291 5104
SequentialMergeSortTime:0.000001seconds
ParallelMergeSortTime:0.000244seconds
Sorted array: 1 2 4 5 9 10
2. Write an OpenMP program that divides the Iterations into chunks containing 2
iterations,respectively(OMP_SCHEDULE=static,2).Its input should be the number
of iterations, and its output should be which iterations of a parallelized for loop are
executed by which thread. For example, if there are two threads and four iterations,
the output might be the following:
a.Thread 0:Iterations0–1
b. Thread 1: Iterations2–3

What is Scheduling in OpenMP?

Scheduling is a method in OpenMP to distribute iterations to different threads in for
loop.Describe show iterations of the loop are divided among the threads in the
team. The default schedule is implementation dependent.
Static:
Loop iterations are divided into pieces of size chunk and then statically assigned to
threads. If chunk is not specified, the iterations are evenly (if possible) divided
contiguously among the threads. If you do not specify chunk-size variable,OpenMP
will divides iterations into chunks that are approximately equal in size and it
distributes chunks to threads in order.

Code:

#include<stdio.h>
#include<omp.h>
int main() {
intnum_iterations;
printf("Enterthenumberofiterations:");
scanf("%d", &num_iterations);
#pragmaompparallel
{
#pragmaompfor schedule(static,2)
for(inti=0;i<num_iterations;i++) {
printf("Thread%d:Iteration%d\n",omp_get_thread_num(), i);
 }
}
return0;
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$ gcc -fopenmp
iterations.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$ ./a.out
Enter the number of iterations: 6
Thread1:Iteration2
Thread1:Iteration3
Thread2:Iteration4
Thread2:Iteration5
Thread0:Iteration0
Thread0:Iteration1
3. Write a OpenMP program tocalculate n Fibonacci numbers using tasks.

Tasks:
AnOpenMPtaskisasinglelineofcodeor astructuredblockwhichisimmediately
“writtendown”inalistoftasks.Thenewtaskcanbeexecutedimmediately,oritcan
bedeferred.If theif clauseisusedandtheargumentevaluatesto0,thenthetaskis
executedimmediately,supersedingwhateverelsethatthreadisdoing.Therehasto
beanexistingparallelthreadteamforthistowork.Otherwiseonethreadendsup
doingalltasksandyou don’tget anycontributionto parallelism.
Fibonaccisequence:
TheFibonacci sequence is a sequence where the next term is the sum of the previous
two terms.The first two terms of the Fibonacci sequence are 0 followed by 1.

Ex:
FibonacciSequenceis:0,1,1,2,3,5,8,13……….

Code:

#include<stdio.h>
#include<omp.h>
intfib(intn)
{
inti, j;
if(n<2)
returnn;
else
{
#pragmaomptaskshared(i)firstprivate(n)
i=fib(n-1);

#pragmaomptaskshared(j)firstprivate(n)
j=fib(n-2);

#pragmaomptaskwait
returni+j;
}
 }

int main()
{
intn=10;

omp_set_dynamic(0);
omp_set_num_threads(4);

#pragmaompparallelshared(n)
{
#pragmaompsingle
printf("fib(%d)=%d\n",n,fib(n));
}
}

OUTPUT:
bantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$ gcc -fopenmp
fibonacci.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$ ./a.out
fib(10) = 55
4. WriteaOpenMPprogramtofindtheprimenumbersfrom1tonemployingparallel for
directive. Record both serial and parallel execution times.

ParallelDirective:
The omp paralleldirective explicitly instructsthe compiler to parallelize the chosen
block of code.
ForDirective:
Causestheworkdoneinaforloopinsideaparallelregiontobedividedamong threads.
PrimeNumber:
Aprimenumberisapositiveintegerthatisdivisibleonlyby1anditself.Forexample: 2, 3, 5,
7, 11, 13, 17…..

Code:

1)
#include<stdio.h>
#include<omp.h>
void main()
{
intprime[1000],i,j,n;

//Promptuserforinput
printf("\nInordertofindprimenumbers from1ton, enterthevalueofn:"); scanf("%d",
&n);

//Initializeallnumbersasprime(set allto1) for(i =

1; i <= n; i++) {
prime[i]=1;
}

//1isnotaprimenumber
prime[1] = 0;

//SieveofEratostheneswithparallelization for(i
= 2; i * i <= n; i++) {
#pragma omp parallel for
for(j= i* i;j<=n;j= j+ i)
{ if(prime[j] == 1) {
prime[j]=0;
}
}
}

//Print primenumbers
printf("\nPrimenumbersfrom1to%dare\n",n);
for(i = 2; i <= n; i++) {
if(prime[i]==1) {
printf("%d\t",i);
}
}
printf("\n");
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$gcc-fopenmpprime.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$ ./a.out
Inorderto findprime numbers from1to n, enter the value of n:100
Prime numbers from 1 to 100 are
2 3 5 7 11 13 17 19 23 29 31
37 41 43 47 53 59 61 67 71 73
79 83 89 97

2)

#include<stdio.h>
#include<omp.h>

int main()
{
int prime[1000],i,j,n;

//Prompt userforinput
printf("\nInorder tofindprimenumbers from1ton, enterthevalueofn:");
scanf("%d", &n);

//Initializeallnumbersasprime(set allto1) f
for(i = 1; i <= n; i++) {
prime[i]=1;
 }

//1isnotaprimenumber
prime[1] = 0;

//SieveofEratostheneswithparallelization for(i
= 2; i * i <= n; i++) {
#pragma omp parallel for
for(j= i* i;j<=n;j= j+ i)
{ if(prime[j] == 1) {
prime[j]=0;
}
}
}

//Print primenumbers
printf("\nPrimenumbersfrom1to%dare\n",n); for(i =
2; i <= n; i++) {
if(prime[i]==1) {
printf("%d\t",i);
}
}
doublestart,end;
//Sequential
start=omp_get_wtime();
prime[i];
end=omp_get_wtime();
printf("SequentialTime:%fseconds\n",end-start);
// Parallel
start=omp_get_wtime();
prime[i];
end=omp_get_wtime();
printf("ParallelTime:%fseconds\n",end-start);
printf("\n");
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$gcc-fopenmpprime.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$ ./a.out
Inorderto findprime numbersfrom1to n,enterthe valueofn:50 Prime
numbers from 1 to 50 are
2 3 5 7 11 13 17 19 23 29 31
37 41 43 47
SequentialTime:0.000001seconds
Parallel Time: 0.000000 seconds
5. Write a MPI Program to demonstration of MPI_Send and MPI_Recv.
Message-PassingInterface(MPI)
Message-Passing is a communication model used on distributed-memory
architecture.MPI is a standard that specifies the message-passing libraries supporting
parallel programming inC/C++or Fortran.MPI_Send and MPI_Recv are the basic
building blocks for essentially all of the more specialized MPI commands in MPI.
They are also the basic communication tools in MPI application. Since MPI_Send
and MPI_Recv involve two ranks, theyare called “point-to-point” communication.
The process of communicating data follows a standard pattern. Rank A decides to
senddatatorankB.Itfirst packsthedataintoabuffer.Thisavoidssending multiple
messages,which wouldtakemore time.Rank A then callsMPI_Sendtocreatea message
for rank B. The communication device is then given the responsibility of routing the
message to the correct destination. Rank B must know that it is about to receive a
message and acknowledge this by calling MPI_Recv.MPI_RANK:
"Rank"referstoauniqueidentifierassignedtoeachprocesswithinacommunication
group.It'salogicalwayofnumberingprocessestofacilitatecommunicationbetween
them. MPI automatically assigns ranks in the MPI_COMM_WORLD group when
the MPI environment is initialized (MPI_Init ).

Code:

1)
#include<stdio.h>
#include "mpi.h"
int main(int argc,char **argv)
{
int my_rank,numbertoreceive[10],numbertosend[3]={73,2,-16};
int recv_count, i;
MPI_Status status;
MPI_Init(&argc,&argv);
MPI_Comm_rank(MPI_COMM_WORLD,&my_rank);
if (my_rank==0){
MPI_Recv(numbertoreceive,3,MPI_INT,MPI_ANY_SOURCE,
MPI_ANY_TAG, MPI_COMM_WORLD,
&status);
printf("status.MPI_SOURCE=%d\n",status.MPI_SOURCE);
printf("status.MPI_TAG = %d\n", status.MPI_TAG);
printf("status.MPI_ERROR = %d\n", status.MPI_ERROR);
MPI_Get_count(&status, MPI_INT, &recv_count);
printf("Receive %d data\n", recv_count);
for(i=0;i<recv_count;i++)
printf("recv[%d]=%d\n",i,numbertoreceive[i]);
}
elseMPI_Send(numbertosend,3,MPI_INT,0,10,MPI_COMM_WORLD);
MPI_Finalize();
return 0;
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpicc-g-osend
SendRcv.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np5./send
status.MPI_SOURCE = 3
status.MPI_TAG=10
status.MPI_ERROR=110
Receive3data
recv[0]=73
recv[1]=2
recv[2] =-16

2)
#include <mpi.h>
#include <stdio.h>
#include<stdlib.h>

intmain(intargc,char**argv){
//InitializetheMPIenvironment MPI_Init(NULL,
NULL);
//Findoutrank,size int
world_rank;
MPI_Comm_rank(MPI_COMM_WORLD,&world_rank);
int world_size;
MPI_Comm_size(MPI_COMM_WORLD,&world_size);

//Weareassumingatleast2processesforthistask if
(world_size < 2) {
fprintf(stderr,"Worldsizemustbegreaterthan1for%s\n",argv[0]);
MPI_Abort(MPI_COMM_WORLD, 1);
}

int number;
if(world_rank==0){
//Ifwearerank0,setthenumberto -1andsend it toprocess1 number = -1;
MPI_Send(
/*data =*/&number,
/* count =*/1,
/* datatype =*/MPI_INT,
/* destination= */1,
/*tag =*/0,
/* communicator=*/MPI_COMM_WORLD);
}elseif(world_rank==1){ MPI_Recv(
/*data =*/&number,
/* count =*/1,
/* datatype =*/MPI_INT,
/* source =*/0,
/*tag =*/0,
/* communicator=*/MPI_COMM_WORLD,
/* status =*/MPI_STATUS_IGNORE);
printf("Process1receivednumber%dfromprocess0\n",number);
}
MPI_Finalize();
}
 OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpicc-g-osend
sendrcv.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np5./send
Process 1 received number -1 from process 0
6. WriteaMPIprogramtodemonstrationofdeadlockusingpointtopoint communication and
avoidance of deadlock by altering the call sequence.
Deadlock:
Deadlockisanoften-encounteredsituationinparallelprocessing.Itresultswhentwo
ormoreprocessesareincontentionforthesamesetofresources.Incommunications, a
typical scenario involves two processes wishing to exchange messages: each is
tryingtogiveamessagetotheother,butneitherofthemisreadytoacceptamessage. What
is point-to-point communication in MPI?
MPIprocessescommunicatebyexplicitlysendingandreceiving messages.Inpoint- to-
point, messages are sent between two processes. Since MPI processes are
independent, in order to coordinate work, they need to communicate by explicitly
sending and receiving messages.

Demonstrationofdeadlock

Code:

#include <mpi.h>
#include<stdio.h>
intmain(intargc,char**argv){ int
rank, size, data = 0;

MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

if(size<2){
printf("Thisprogramrequiresatleast2processes.\n");
MPI_Abort(MPI_COMM_WORLD, 1);
}

if(rank==0){
//Rank0sendsfirst,thenreceives
MPI_Send(&data,1,MPI_INT,1,0,MPI_COMM_WORLD);
printf("Process0sentdatatoProcess1\n");

MPI_Recv(&data, 1, MPI_INT, 1, 0, MPI_COMM_WORLD,

MPI_STATUS_IGNORE);
printf("Process0receiveddatafromProcess1\n");
}elseif(rank==1) {
//Rank1sendsfirst,thenreceives
MPI_Send(&data,1,MPI_INT,0,0,MPI_COMM_WORLD);
printf("Process1sentdatatoProcess0\n");
 MPI_Recv(&data, 1, MPI_INT, 0, 0, MPI_COMM_WORLD,
MPI_STATUS_IGNORE);
printf("Process1receiveddatafromProcess0\n");
}

MPI_Finalize();
return 0;
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpicc-g-oMPI Mpi7.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np5./MPI
Process 0 sent data to Process 1
Process0receiveddatafromProcess1
Process 1 sent data to Process 0
Process1receiveddatafromProcess0

Deadlockavoidance
Tofixdeadlock,needtocoordinatethecommunicationbetweenpairsofprocesses so that
there is an ordering of sends and receives between them.

Code:

#include <mpi.h>
#include<stdio.h>

intmain(intargc,char**argv){ int
rank, size, data = 0;

MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

if(size<2){
printf("Thisprogramrequiresatleast2processes.\n");
MPI_Abort(MPI_COMM_WORLD, 1);
}

if(rank==0){
//Rank0sendsfirst
MPI_Send(&data,1,MPI_INT,1,0,MPI_COMM_WORLD);
printf("Process0sentdatatoProcess1\n");

MPI_Recv(&data, 1, MPI_INT, 1, 0, MPI_COMM_WORLD,

MPI_STATUS_IGNORE);
printf("Process0receiveddatafromProcess1\n");
}elseif(rank==1) {
//Rank1receivesfirst
 MPI_Recv(&data, 1, MPI_INT, 0, 0, MPI_COMM_WORLD,
MPI_STATUS_IGNORE);
printf("Process1receiveddatafromProcess0\n");

MPI_Send(&data,1,MPI_INT,0,0,MPI_COMM_WORLD);
printf("Process1sentdatatoProcess0\n");
}

MPI_Finalize();
return 0;
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpicc-g-oMPI
avoiddeadlock.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np2./MPI
Process 0 sent data to Process 1
Process0receiveddatafromProcess1
Process1receiveddatafromProcess0
Process 1 sent data to Process 0
7. WriteaMPI ProgramtodemonstrationofBroadcastoperation.
BroadcastingwithMPI_Bcast:
Abroadcast is one of the standard collective communication techniques. During a
broadcast, one process sends the same datato allprocesses ina communicator. One
of the main uses of broadcasting is to send out user input to a parallel program, or
sendoutconfigurationparameterstoallprocesses.Basicallyprocesszerois theroot
process, and it hasthe initialcopyofdata.Alloftheother processesreceive the copy of
data.

Code:

#include <mpi.h>
#include<stdio.h>
intmain(intargc,char**argv){ int
rank, size;
intdata;//Thedatatobroadcast

MPI_Init(&argc, &argv); // Initialize MPI

MPI_Comm_rank(MPI_COMM_WORLD, &rank); // Get process rank
MPI_Comm_size(MPI_COMM_WORLD, &size); //Gettotalnumberof
processes

if(rank==0){
data=42;//Root processsetsthe data
printf("Process%disbroadcastingdata=%d\n", rank, data);
}

// Broadcast the data from process 0 to all other processes

MPI_Bcast(&data,1,MPI_INT,0,MPI_COMM_WORLD);
 //Allprocessesprintthereceiveddata
printf("Process%dreceiveddata=%d\n", rank,data);

MPI_Finalize();//FinalizeMPI
return 0;
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpicc-g-oMPI
broadcast.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np2./MPI
Process 0 is broadcasting data = 42
Process0 received data=42
Process1 received data=42
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np5./MPI
Process 0 is broadcasting data = 42
Process0 received data=42
Process1 received data=42
Process2 received data=42
Process3 received data=42
Process4 received data=42

2)
#include <mpi.h>
#include <stdio.h>
#include<stdlib.h>
#include<assert.h>

#defineCONDUCTOR0

intmain(intargc,char**argv){ int
answer = 0;
intnumProcs=0,myRank=0;

MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD,&numProcs);
MPI_Comm_rank(MPI_COMM_WORLD, &myRank);

if(myRank==CONDUCTOR)
{ answer = 42;
}

printf("BEFOREthebroadcast,process%d'sanswer=%d\n",myRank,answer);
MPI_Bcast(&answer, 1, MPI_INT, 0, MPI_COMM_WORLD);
printf("AFTERthebroadcast,process%d'sanswer=%d\n",myRank,answer);
MPI_Finalize();

return0;
}
 OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpicc-g-oMPI
Broadcast.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np5./MPI
BEFORE the broadcast, process 0's answer = 42
AFTERthe broadcast, process0'sanswer = 42
BEFOREthebroadcast,process1'sanswer=0
AFTERthe broadcast, process1'sanswer = 42
BEFOREthebroadcast,process3'sanswer=0
BEFOREthebroadcast,process2'sanswer=0
AFTERthe broadcast, process2'sanswer = 42
BEFOREthebroadcast,process4'sanswer=0
AFTERthe broadcast, process3'sanswer = 42
AFTERthe broadcast, process4'sanswer = 42

8. WriteaMPIProgramdemonstrationofMPI_ScatterandMPI_Gather
MPI_Scatter:
MPI_Scatterisacollectiveroutinethatisverysimilarto MPI_Bcast.
MPI_Scatterinvolves a designated root process sending data to all processes in a
communicator.TheprimarydifferencebetweenMPI_Bcast andMPI_Scatteris
smallbutimportant. MPI_Bcast sendsthe same pieceofdatatoallprocesses while
MPI_Scatter sends chunks of an array to different processes.

MPI_Gather:
MPI_Gather is the inverse ofMPI_Scatter. Instead of spreading elements from one
process to many processes, MPI_Gathertakes elements from many processes and
gathers them to one single process. This routine is highly useful to many parallel
algorithms, such as parallel sorting and searching.

Code:

#include <mpi.h>
#include <stdio.h>
#include<stdlib.h>
intmain(intargc,char**argv){ int
size, rank; MPI_Init(&argc,
&argv);
MPI_Comm_size(MPI_COMM_WORLD, &size);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);

intglobaldata[4];/*wantstodeclarearraythisway*/ int
localdata[4]; /*without using pointers*/

inti;
if(rank==0){

for(i=0;i<size;i++) globaldata[i]
= i;

printf("1.Processor %dhasdata:",rank);
 for (i = 0; i < size; i++)
printf("%d",globaldata[i]);
printf("\n");
}
MPI_Scatter(globaldata, 1, MPI_INT, &localdata, 1, MPI_INT, 0,
MPI_COMM_WORLD);

printf("2.Processor%dhasdata%d\n",rank,localdata[rank]);
localdata[rank]= 5;
printf("3.Processor%dnowhas%d\n", rank, localdata[rank]);

MPI_Gather(&localdata, 1, MPI_INT, globaldata, 1, MPI_INT, 0,

MPI_COMM_WORLD);

if(rank==0){
printf("4.Processor%dhasdata:",rank); for
(i = 0; i < size; i++)
printf("%d",globaldata[i]);
printf("\n");
}
MPI_Finalize();
return 0;
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpicc-g-oMPI
scattergather.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np5./MPI
1. Processor0hasdata:01234
2. Processor 0hasdata0
3. Processor0nowhas5
2. Processor 1hasdata0
3. Processor1nowhas5
2. Processor 2hasdata0
3. Processor2nowhas5
2. Processor 3hasdata0
3. Processor3nowhas5
2. Processor 4hasdata255
3. Processor4nowhas5
4. Processor0hasdata:51234

2)
#include <mpi.h>
#include <stdio.h>
#include<stdlib.h>

intmain(intargc,char**argv){ int
size, rank;

MPI_Init(&argc,&argv);
 MPI_Comm_size(MPI_COMM_WORLD, &size);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);

intglobaldata[4];/*wantstodeclarearraythisway*/ int
localdata;/*without using pointers*/

inti;
if(rank==0){

for(i=0;i<size;i++)
globaldata[i] = i;

printf("1.Processor%dhasdata:",rank); for
(i=0; i<size; i++)
printf("%d",globaldata[i]);
printf("\n");
}

MPI_Scatter(globaldata, 1, MPI_INT, &localdata, 1, MPI_INT, 0,

MPI_COMM_WORLD);

printf("2.Processor%dhasdata%d\n",rank,localdata); localdata= 5;
printf("3.Processor%dnowhas%d\n",rank, localdata);

MPI_Gather(&localdata, 1, MPI_INT, globaldata, 1, MPI_INT, 0,

MPI_COMM_WORLD);

if(rank==0){
printf("4.Processor%dhasdata:",rank); for
(i=0; i<size; i++)
printf("%d",globaldata[i]);
printf("\n");
}
MPI_Finalize();
return 0;
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpicc-g-oMPI
scattergather.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np5./MPI
1. Processor0hasdata:01234
2. Processor 0hasdata0
3. Processor0nowhas5
2.Processor1hasdata1
3. Processor 1nowhas5
2.Processor2hasdata2
3. Processor 2nowhas5
2.Processor3hasdata3

3. Processor 3nowhas5
2. Processor 4hasdata4
3. Processor4nowhas5
4. Processor0hasdata:55555
9. Write a MPI Program to demonstration of MPI_Reduce and MPI_All reduce (MPI_MAX, MPI_MIN,
MPI_SUM, MPI_PROD)
MPI_Reduce:
Reduce is a classic concept from functional programming. Data reduction involves reducingaset
ofnumbersinto asmallerset ofnumbersviaa function.Forexample, let’s saywe have a list ofnumbers [1, 2,
3, 4, 5].
Reducing this list ofnumbers with thesumfunctionwouldproducesum([1,2,3,4,5])=15.Similarly,the multiplication
Reduction would yield multiply([1, 2, 3,4,5])=120.MPI_Reduce takes an arrayof input elements on each process and
returns an array of output elements to the root process.Theoutput elements contain the reduced result.
MPI_AllReduce:
Many parallel applications will require accessing the reduced results across all processes rather than the
root process. In a similar complementary style of MPI_All gather to MPI_Gather,MPI_All reduce will
reduce the values and distribute the results to all processes.

Code:

#include <mpi.h>

#include<stdio.h>

int main(int argc,char**argv)

int rank, size;

int value,sum,product,max,min;
int all_sum,all_product,all_max,all_min;

MPI_Init(&argc, &argv); // Initialize

MPI MPI_Comm_rank(MPI_COMM_WORLD, &rank); // Get rank
MPI_Comm_size(MPI_COMM_WORLD, &size); //Getnumberofprocesses

value=rank+1;// Eachprocesshasauniquevalue

//----MPI_Reduce----
MPI_Reduce(&value, &sum, 1,MPI_INT,MPI_SUM,0,
MPI_COMM_WORLD);
MPI_Reduce(&value,&product,1,MPI_INT,MPI_PROD,0, MPI_COMM_WORLD);
MPI_Reduce(&value, &max, 1,MPI_INT,MPI_MAX,0,
MPI_COMM_WORLD);
MPI_Reduce(&value, &min, 1,MPI_INT,MPI_MIN,0,
MPI_COMM_WORLD);

if(rank==0){
printf("===MPI_Reduceresultsatroot===\n");
printf("SUM = %d\n", sum);
printf("PRODUCT=%d\n", product);
 printf("MAX =%d\n",max);
printf("MIN = %d\n", min);
}

//----MPI_Allreduce----
MPI_Allreduce(&value,
&all_sum,1,MPI_INT,MPI_SUM,MPI_COMM_WORLD);
MPI_Allreduce(&value,&all_product,1,MPI_INT,MPI_PROD, MPI_COMM_WORLD);
MPI_Allreduce(&value, &all_max,1,MPI_INT,MPI_MAX,
MPI_COMM_WORLD);
MPI_Allreduce(&value, &all_min, 1,MPI_INT,MPI_MIN,
MPI_COMM_WORLD);

printf("Process%d:ALL_SUM=%d,ALL_PROD=%d,ALL_MAX=%d, ALL_MIN = %d\

n",rank,all_sum,all_product,all_max,all_min);

MPI_Finalize();//FinalizeMPI
return 0;
}

OUTPUT:
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpicc-g-oMPI reduceallreduce.c
ubantu@ubantu-HP-Pro-Tower-280-G9-PCI-Desktop-PC:~$mpirun-np5./MPI
===MPI_Reduce results at root=== SUM = 15
PRODUCT=120
MAX =5
MIN =1
Process0:ALL_SUM=15,ALL_PROD=120,ALL_MAX=5,ALL_MIN=1
Process1:ALL_SUM=15,ALL_PROD=120,ALL_MAX=5,ALL_MIN=1
Process2:ALL_SUM=15,ALL_PROD=120,ALL_MAX=5,ALL_MIN=1
Process3:ALL_SUM=15,ALL_PROD=120,ALL_MAX=5,ALL_MIN=1
Process4:ALL_SUM=15,ALL_PROD=120,ALL_MAX=5,ALL_MIN=1

Executed on– (Ubantu11.4.0-1ubantu1~22.04)11.4.0