Thanks to visit codestin.com
Credit goes to www.scribd.com

Scribd
Upload
19 views3 pages

Hadoop Codes

This document provides a step-by-step guide to install Hadoop using HomeBrew on a system. It includes instructions for configuring environment variables, modifying core, HDFS, MapReduce, and YARN configuration files. The user is advised to ensure Java JDK is installed and properly set up before proceeding with the Hadoop installation and configuration.

Uploaded by

22btrcn117
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
19 views3 pages

Hadoop Codes

This document provides a step-by-step guide to install Hadoop using HomeBrew on a system. It includes instructions for configuring environment variables, modifying core, HDFS, MapReduce, and YARN configuration files. The user is advised to ensure Java JDK is installed and properly set up before proceeding with the Hadoop installation and configuration.

Uploaded by

22btrcn117
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Install Hadoop using HomeBrew

STEP 1:brew install hadoop

Open the document containing the environment variable settings :

$ cd /opt/homebrew/Cellar/hadoop/3.3.6/libexec/etc/hadoop

then open hadoop-env.sh with any editor, I will use vscode

$code hadoop-env.sh

Make sure to have java jdk installed and JAVA_HOME setup on


your system. If you haven t follow below link ( Install Java 8 or 11
not newer versions)

Use this command to nd the path in terminal


$ /usr/libexec/java_home

and add the location on hadoop-env.sh

Step 2: Make changes to Core files

cd /opt/homebrew/Cellar/hadoop/3.3.6/libexec/etc/hadoop
code core-site.xml

and add the following con guration

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
fi
’
fi
Step 3 : Make changes to hdfs files :

cd /opt/homebrew/Cellar/hadoop/3.3.6/libexec/etc/hadoop
code hdfs-site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

Step 4 : Make changes to mapred files

cd /opt/homebrew/Cellar/hadoop/3.3.6/libexec/etc/hadoop
code mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.application.classpath</name>
<value>
$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:
$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*
</value>
</property>
</configuration>
Step 5: Make changes to yarn files :

cd /opt/homebrew/Cellar/hadoop/3.3.6/libexec/etc/hadoop
code yarn-site.xml

<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.env-whitelist</name>
<value>
JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADO
OP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YAR
N_HOME,HADOOP_MAPRED_HOME
</value>
</property>
</configuration>

You might also like