• Install brew (Homebrew)

  • Install hadoop

    brew install hadoop

  • cd into hadoop configuration files

    cd /opt/homebrew/Cellar/hadoop/3.3.4/libexec/etc/hadoop

  • Update hadoop-env.sh conf

    vim hadoop-env.sh

    Find export JAVA_HOME line and uncomment it and put your $JAVA_HOME path in there you can also find it using /usr/libexec/java_home

  • Update core-site.xml conf

    vim core-site.xml

    Add configuration

    <configuration>
 <property>
  <name>fs.defaultFS</name>
  <value>hdfs://localhost:9000</value>
 </property>
</configuration>

  • Update hdfs-site.xml

    vim hdfs-site.xml

    Add configuration

    <configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
</configuration>

  • Update mapred-site.xml

    vim mapred-site.xml

    Add configuration

    <configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
  <property>
    <name>mapreduce.application.classpath</name>   
    <value>
      $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*
    </value>
  </property>
</configuration>

  • Update yarn-site.xml

    vim yarn-site.xml

    Add configuration

    <configuration>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.env-whitelist</name>  
    <value>
      JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME
    </value>
  </property>
</configuration>
  • Enable remote login from “Sharing” in system preferences and allow Administrators from “Only these users:” radio button

  • Run

    cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

  • Run

    stop-all.sh
hadoop namenode -format
start-all.sh

  • Run jps to verify all the installation

    16641 NameNode
16882 SecondaryNameNode
17170 NodeManager
16741 DataNode
17279 Jps
17071 ResourceManager
  • Grab popcorns and work with Hadoop!