반응형
하둡은 자바 기반 오픈소스 프로그램이기 때문에, 자바를 우선적으로 설치해줘야 하는데 하둡 버전에 맞는 Java 버전은 이 사이트를 참고하면 된다.
Hadoop 3.3 이상부터는 Java 11을 사용해야 한다. 참고로 Hadoop 3.0 ~ 3.2는 Java 11이 지원되지 않으며, Java 8을 설치해야 한다.
wget https://downloads.apache.org/hadoop/common/hadoop-3.2.2/hadoop-3.2.2.tar.gz
Hadoop 3.3.4
# 다운로드
wget https://dlcdn.apache.org/hadoop/common/hadoop-3.3.4/hadoop-3.3.4.tar.gz
# 압축 풀기
tar xvzf hadoop-3.3.4.tar.gz
# 폴더 이동
sudo mv hadoop-3.3.4 /usr/local/hadoop
# Hadoop 디렉토리 사용자 그룹 변경
sudo chown -R $USER:$USER /usr/local/hadoop
# The java implementation to use. By default, this environment
# variable is REQUIRED on ALL platforms except OS X!
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-arm64
# Location of Hadoop. By default, Hadoop will attempt to determine
# this location based upon its execution path.
# export HADOOP_HOME=
참고
https://choosunsick.github.io/post/hadoop_install/
라즈베리파이 yarn 클러스터에 하둡(hadoop)과 스파크(spark) 설치하기
라즈베리파이 os 설치하기 raspberrypi imager를 통해 라즈베리파이의 os를 깔아줍니다. 해당 프로그램은 링크에서 자신의 컴퓨터 os에 맞게 설치할 수 있습니다. 설치
choosunsick.github.io
위 사이트 보고 실행하니 아주 잘 싱글 노드에서 그 과정은 아래 코드 참고
goodtech@pi050:~$
goodtech@pi050:~$ sudo chown goodtech:goodtech -R /opt/hadoop
[sudo] password for goodtech:
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ nano ~/.bashrc
goodtech@pi050:~$ source ~/.bashrc
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ nano /opt/hadoop/etc/hadoop/hadoop-env.sh
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ hadoop version | grep Hadoop
Hadoop 3.3.6
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ cd ~/.ssh
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$ nano config
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$ nano config
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$
goodtech@pi050:~/.ssh$ source ~/.bashrc
goodtech@pi050:~/.ssh$ cd ..
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ ls
Desktop Downloads Pictures Templates hadoop-3.3.6.tar.gz snap
Documents Music Public Videos hadoopdata
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ pwd
/home/goodtech
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ source ~/.bashrc
goodtech@pi050:~$ clusterscp ~/.bashrc
clusterscp: command not found
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ nano ~/.bashrc
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ cluster-scp ~/.bashrc
pi051 copying...
pi052 copying...
pi053 copying...
all files copied successfully
goodtech@pi050:~$ cluster-cmd source ~/.bashrc
pi051
pi052
pi053
pi050
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ nano ~/.bashrc
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ hadoop version | grep Hadoop
Hadoop 3.3.6
goodtech@pi050:~$ cd /opt/hadoop/etc/hadoop
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$ nano core-site.xml
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$ nano hdfs-site.xml
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$ sudo mkdir -p /opt/hadoop_tmp/hdfs/datanode
[sudo] password for goodtech:
goodtech@pi050:/opt/hadoop/etc/hadoop$ ls
capacity-scheduler.xml kms-log4j.properties
configuration.xsl kms-site.xml
container-executor.cfg log4j.properties
core-site.xml mapred-env.cmd
hadoop-env.cmd mapred-env.sh
hadoop-env.sh mapred-queues.xml.template
hadoop-metrics2.properties mapred-site.xml
hadoop-policy.xml shellprofile.d
hadoop-user-functions.sh.example ssl-client.xml.example
hdfs-rbf-site.xml ssl-server.xml.example
hdfs-site.xml user_ec_policies.xml.template
httpfs-env.sh workers
httpfs-log4j.properties yarn-env.cmd
httpfs-site.xml yarn-env.sh
kms-acls.xml yarn-site.xml
kms-env.sh yarnservice-log4j.properties
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$ sudo mkdir -p /opt/hadoop_tmp/hdfs/namenode
goodtech@pi050:/opt/hadoop/etc/hadoop$ sudo chown goodtech:goodtech -R /opt/hadoop_tmp
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$ nano mapred-site.xml
goodtech@pi050:/opt/hadoop/etc/hadoop$ nano yarn-site.xml
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$
goodtech@pi050:/opt/hadoop/etc/hadoop$ cd
goodtech@pi050:~$
goodtech@pi050:~$
goodtech@pi050:~$ hdfs namenode -format
2024-07-22 20:47:19,676 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = pi050/192.168.0.50
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 3.3.6
STARTUP_MSG: classpath = /opt/hadoop/etc/hadoop:/opt/hadoop/share/hadoop/common/lib/commons-compress-1.21.jar:/opt/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/opt/hadoop/share/hadoop/common/lib/jsr305-3.0.2.jar:/opt/hadoop/share/hadoop/common/lib/zookeeper-3.6.3.jar:/opt/hadoop/share/hadoop/common/lib/stax2-api-4.2.1.jar:/opt/hadoop/share/hadoop/common/lib/jetty-http-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/common/lib/kerb-server-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/zookeeper-jute-3.6.3.jar:/opt/hadoop/share/hadoop/common/lib/netty-resolver-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/commons-text-1.10.0.jar:/opt/hadoop/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-x86_64.jar:/opt/hadoop/share/hadoop/common/lib/httpcore-4.4.13.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-redis-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jetty-servlet-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/common/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/opt/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/hadoop/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-handler-proxy-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jsch-0.1.55.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-haproxy-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/netty-all-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/token-provider-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-resolver-dns-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/opt/hadoop/share/hadoop/common/lib/avro-1.7.7.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-memcache-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-stomp-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar:/opt/hadoop/share/hadoop/common/lib/nimbus-jose-jwt-9.8.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-smtp-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.89.Final-osx-aarch_64.jar:/opt/hadoop/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/opt/hadoop/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/kerb-core-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-common-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jersey-core-1.19.4.jar:/opt/hadoop/share/hadoop/common/lib/kerb-common-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-udt-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/httpclient-4.5.13.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-native-epoll-4.1.89.Final-linux-x86_64.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/commons-codec-1.15.jar:/opt/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop/share/hadoop/common/lib/jetty-util-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/opt/hadoop/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/opt/hadoop/share/hadoop/common/lib/failureaccess-1.0.jar:/opt/hadoop/share/hadoop/common/lib/re2j-1.1.jar:/opt/hadoop/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-socks-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/opt/hadoop/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/opt/hadoop/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jetty-security-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-sctp-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-rxtx-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/hadoop-shaded-guava-1.1.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jetty-util-ajax-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-aarch_64.jar:/opt/hadoop/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/curator-client-5.2.0.jar:/opt/hadoop/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/opt/hadoop/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/opt/hadoop/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/opt/hadoop/share/hadoop/common/lib/jackson-core-2.12.7.jar:/opt/hadoop/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/opt/hadoop/share/hadoop/common/lib/kerb-client-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/commons-net-3.9.0.jar:/opt/hadoop/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/opt/hadoop/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/opt/hadoop/share/hadoop/common/lib/jetty-io-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-handler-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/opt/hadoop/share/hadoop/common/lib/reload4j-1.2.22.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-mqtt-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jetty-webapp-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/common/lib/jetty-server-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/common/lib/curator-framework-5.2.0.jar:/opt/hadoop/share/hadoop/common/lib/kerby-util-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-http-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/commons-io-2.8.0.jar:/opt/hadoop/share/hadoop/common/lib/dnsjava-2.1.7.jar:/opt/hadoop/share/hadoop/common/lib/snappy-java-1.1.8.2.jar:/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/opt/hadoop/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/hadoop/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.89.Final-osx-x86_64.jar:/opt/hadoop/share/hadoop/common/lib/jersey-server-1.19.4.jar:/opt/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/opt/hadoop/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/opt/hadoop/share/hadoop/common/lib/metrics-core-3.2.4.jar:/opt/hadoop/share/hadoop/common/lib/jettison-1.5.4.jar:/opt/hadoop/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/hadoop/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-dns-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-xml-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/checker-qual-2.5.2.jar:/opt/hadoop/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/opt/hadoop/share/hadoop/common/lib/netty-transport-native-epoll-4.1.89.Final-linux-aarch_64.jar:/opt/hadoop/share/hadoop/common/lib/kerb-util-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/kerby-config-1.0.1.jar:/opt/hadoop/share/hadoop/common/lib/hadoop-annotations-3.3.6.jar:/opt/hadoop/share/hadoop/common/lib/jetty-xml-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/hadoop/share/hadoop/common/lib/netty-buffer-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/jersey-json-1.20.jar:/opt/hadoop/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/hadoop/share/hadoop/common/lib/gson-2.9.0.jar:/opt/hadoop/share/hadoop/common/lib/netty-codec-http2-4.1.89.Final.jar:/opt/hadoop/share/hadoop/common/lib/hadoop-auth-3.3.6.jar:/opt/hadoop/share/hadoop/common/hadoop-registry-3.3.6.jar:/opt/hadoop/share/hadoop/common/hadoop-kms-3.3.6.jar:/opt/hadoop/share/hadoop/common/hadoop-common-3.3.6-tests.jar:/opt/hadoop/share/hadoop/common/hadoop-nfs-3.3.6.jar:/opt/hadoop/share/hadoop/common/hadoop-common-3.3.6.jar:/opt/hadoop/share/hadoop/hdfs:/opt/hadoop/share/hadoop/hdfs/lib/commons-compress-1.21.jar:/opt/hadoop/share/hadoop/hdfs/lib/paranamer-2.3.jar:/opt/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/opt/hadoop/share/hadoop/hdfs/lib/zookeeper-3.6.3.jar:/opt/hadoop/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/kotlin-stdlib-common-1.4.10.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-http-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/zookeeper-jute-3.6.3.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-resolver-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-x86_64.jar:/opt/hadoop/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-redis-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-servlet-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-all-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-3.10.6.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/avro-1.7.7.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/opt/hadoop/share/hadoop/hdfs/lib/okio-2.8.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.8.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.89.Final-osx-aarch_64.jar:/opt/hadoop/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-common-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-udt-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.89.Final-linux-x86_64.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/opt/hadoop/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-util-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/opt/hadoop/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/re2j-1.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-socks-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-security-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.1.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.89.Final-osx-aarch_64.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/opt/hadoop/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/opt/hadoop/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/okhttp-4.9.3.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-io-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-handler-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/kotlin-stdlib-1.4.10.jar:/opt/hadoop/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-webapp-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-server-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-http-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-io-2.8.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/opt/hadoop/share/hadoop/hdfs/lib/snappy-java-1.1.8.2.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.89.Final-osx-x86_64.jar:/opt/hadoop/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/opt/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/hadoop/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/opt/hadoop/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/opt/hadoop/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/opt/hadoop/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-dns-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-xml-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.89.Final-linux-aarch_64.jar:/opt/hadoop/share/hadoop/hdfs/lib/HikariCP-java7-2.4.12.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/hadoop-annotations-3.3.6.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-xml-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-buffer-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/opt/hadoop/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/hadoop/share/hadoop/hdfs/lib/gson-2.9.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-codec-http2-4.1.89.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/hadoop-auth-3.3.6.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.6-tests.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.3.6-tests.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.3.6.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.3.6.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.3.6.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.3.6.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.6-tests.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.3.6.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.3.6-tests.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.3.6.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.3.6-tests.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.3.6.jar:/opt/hadoop/share/hadoop/yarn:/opt/hadoop/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/opt/hadoop/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/opt/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/hadoop/share/hadoop/yarn/lib/asm-commons-9.4.jar:/opt/hadoop/share/hadoop/yarn/lib/jline-3.9.0.jar:/opt/hadoop/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/asm-tree-9.4.jar:/opt/hadoop/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/opt/hadoop/share/hadoop/yarn/lib/jetty-jndi-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/websocket-api-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/opt/hadoop/share/hadoop/yarn/lib/bcprov-jdk15on-1.68.jar:/opt/hadoop/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/opt/hadoop/share/hadoop/yarn/lib/fst-2.50.jar:/opt/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/hadoop/share/hadoop/yarn/lib/websocket-client-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/java-util-1.9.0.jar:/opt/hadoop/share/hadoop/yarn/lib/jetty-plus-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/websocket-server-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/opt/hadoop/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/opt/hadoop/share/hadoop/yarn/lib/jetty-annotations-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/guice-4.0.jar:/opt/hadoop/share/hadoop/yarn/lib/objenesis-2.6.jar:/opt/hadoop/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/opt/hadoop/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/opt/hadoop/share/hadoop/yarn/lib/websocket-servlet-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/jna-5.2.0.jar:/opt/hadoop/share/hadoop/yarn/lib/json-io-2.5.1.jar:/opt/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/opt/hadoop/share/hadoop/yarn/lib/bcpkix-jdk15on-1.68.jar:/opt/hadoop/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/opt/hadoop/share/hadoop/yarn/lib/jetty-client-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/opt/hadoop/share/hadoop/yarn/lib/websocket-common-9.4.51.v20230217.jar:/opt/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/opt/hadoop/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-services-api-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-services-core-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-router-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-3.3.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.3.6.jar
STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1be78238728da9266a4f88195058f08fd012bf9c; compiled by 'ubuntu' on 2023-06-18T08:22Z
STARTUP_MSG: java = 11.0.23
************************************************************/
2024-07-22 20:47:19,712 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2024-07-22 20:47:19,882 INFO namenode.NameNode: createNameNode [-format]
2024-07-22 20:47:20,302 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2024-07-22 20:47:20,985 INFO namenode.NameNode: Formatting using clusterid: CID-704aa76e-b851-4979-85e8-32322d7c887c
2024-07-22 20:47:21,062 INFO namenode.FSEditLog: Edit logging is async:true
2024-07-22 20:47:21,121 INFO namenode.FSNamesystem: KeyProvider: null
2024-07-22 20:47:21,123 INFO namenode.FSNamesystem: fsLock is fair: true
2024-07-22 20:47:21,124 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2024-07-22 20:47:21,193 INFO namenode.FSNamesystem: fsOwner = goodtech (auth:SIMPLE)
2024-07-22 20:47:21,193 INFO namenode.FSNamesystem: supergroup = supergroup
2024-07-22 20:47:21,193 INFO namenode.FSNamesystem: isPermissionEnabled = true
2024-07-22 20:47:21,193 INFO namenode.FSNamesystem: isStoragePolicyEnabled = true
2024-07-22 20:47:21,193 INFO namenode.FSNamesystem: HA Enabled: false
2024-07-22 20:47:21,288 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2024-07-22 20:47:21,592 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit : configured=1000, counted=60, effected=1000
2024-07-22 20:47:21,592 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
2024-07-22 20:47:21,597 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2024-07-22 20:47:21,597 INFO blockmanagement.BlockManager: The block deletion will start around 2024 Jul 22 20:47:21
2024-07-22 20:47:21,600 INFO util.GSet: Computing capacity for map BlocksMap
2024-07-22 20:47:21,600 INFO util.GSet: VM type = 64-bit
2024-07-22 20:47:21,606 INFO util.GSet: 2.0% max memory 1.9 GB = 39.7 MB
2024-07-22 20:47:21,607 INFO util.GSet: capacity = 2^22 = 4194304 entries
2024-07-22 20:47:21,641 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled
2024-07-22 20:47:21,641 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
2024-07-22 20:47:21,650 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.999
2024-07-22 20:47:21,650 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
2024-07-22 20:47:21,650 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2024-07-22 20:47:21,651 INFO blockmanagement.BlockManager: defaultReplication = 1
2024-07-22 20:47:21,652 INFO blockmanagement.BlockManager: maxReplication = 512
2024-07-22 20:47:21,652 INFO blockmanagement.BlockManager: minReplication = 1
2024-07-22 20:47:21,652 INFO blockmanagement.BlockManager: maxReplicationStreams = 2
2024-07-22 20:47:21,652 INFO blockmanagement.BlockManager: redundancyRecheckInterval = 3000ms
2024-07-22 20:47:21,652 INFO blockmanagement.BlockManager: encryptDataTransfer = false
2024-07-22 20:47:21,652 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000
2024-07-22 20:47:21,718 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911
2024-07-22 20:47:21,719 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215
2024-07-22 20:47:21,719 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215
2024-07-22 20:47:21,719 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215
2024-07-22 20:47:21,744 INFO util.GSet: Computing capacity for map INodeMap
2024-07-22 20:47:21,745 INFO util.GSet: VM type = 64-bit
2024-07-22 20:47:21,745 INFO util.GSet: 1.0% max memory 1.9 GB = 19.9 MB
2024-07-22 20:47:21,745 INFO util.GSet: capacity = 2^21 = 2097152 entries
2024-07-22 20:47:21,754 INFO namenode.FSDirectory: ACLs enabled? true
2024-07-22 20:47:21,754 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
2024-07-22 20:47:21,754 INFO namenode.FSDirectory: XAttrs enabled? true
2024-07-22 20:47:21,754 INFO namenode.NameNode: Caching file names occurring more than 10 times
2024-07-22 20:47:21,767 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536
2024-07-22 20:47:21,770 INFO snapshot.SnapshotManager: SkipList is disabled
2024-07-22 20:47:21,777 INFO util.GSet: Computing capacity for map cachedBlocks
2024-07-22 20:47:21,777 INFO util.GSet: VM type = 64-bit
2024-07-22 20:47:21,778 INFO util.GSet: 0.25% max memory 1.9 GB = 5.0 MB
2024-07-22 20:47:21,778 INFO util.GSet: capacity = 2^19 = 524288 entries
2024-07-22 20:47:21,801 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2024-07-22 20:47:21,801 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2024-07-22 20:47:21,801 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2024-07-22 20:47:21,809 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
2024-07-22 20:47:21,809 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2024-07-22 20:47:21,817 INFO util.GSet: Computing capacity for map NameNodeRetryCache
2024-07-22 20:47:21,817 INFO util.GSet: VM type = 64-bit
2024-07-22 20:47:21,817 INFO util.GSet: 0.029999999329447746% max memory 1.9 GB = 610.1 KB
2024-07-22 20:47:21,817 INFO util.GSet: capacity = 2^16 = 65536 entries
2024-07-22 20:47:21,869 INFO namenode.FSImage: Allocated new BlockPoolId: BP-122981760-192.168.0.50-1721648841855
2024-07-22 20:47:21,929 INFO common.Storage: Storage directory /opt/hadoop_tmp/hdfs/namenode has been successfully formatted.
2024-07-22 20:47:21,986 INFO namenode.FSImageFormatProtobuf: Saving image file /opt/hadoop_tmp/hdfs/namenode/current/fsimage.ckpt_0000000000000000000 using no compression
2024-07-22 20:47:22,201 INFO namenode.FSImageFormatProtobuf: Image file /opt/hadoop_tmp/hdfs/namenode/current/fsimage.ckpt_0000000000000000000 of size 403 bytes saved in 0 seconds .
2024-07-22 20:47:22,227 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
2024-07-22 20:47:22,259 INFO namenode.FSNamesystem: Stopping services started for active state
2024-07-22 20:47:22,259 INFO namenode.FSNamesystem: Stopping services started for standby state
2024-07-22 20:47:22,268 INFO namenode.FSImage: FSImageSaver clean checkpoint: txid=0 when meet shutdown.
2024-07-22 20:47:22,269 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at pi050/192.168.0.50
************************************************************/
goodtech@pi050:~$ start-dfs.sh
Starting namenodes on [pi050]
Starting datanodes
Starting secondary namenodes [pi050]
2024-07-22 20:47:51,772 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
goodtech@pi050:~$ start-yarn.sh
Starting resourcemanager
Starting nodemanagers
goodtech@pi050:~$ jps
5424 Jps
4616 DataNode
5097 ResourceManager
5226 NodeManager
4474 NameNode
4812 SecondaryNameNode
goodtech@pi050:~$
참고자료
https://developer-woong.tistory.com/36
참고
Hadoop HA 구성하기 with Zookeeper (1) - Java, Hadoop, Zookeeper 설치
반응형
'Bigdata Hadoop' 카테고리의 다른 글
| 데이터 과학/빅데이터 실험실 - 4부 중 4편: 3노드 클러스터의 Ubuntu를 통한 Kafka와 Zookeeper (4) | 2024.07.24 |
|---|---|
| 데이터 과학/빅데이터 실험실 - 4부 중 3부: 3노드 클러스터의 Ubuntu를 통한 Hive 및 Postgres (6) | 2024.07.23 |
| AB BLE 게이트웨이 V4 사용자 가이드 (1) | 2024.07.22 |
| 데이터 과학/빅데이터 실험실 - 4부 중 2부: 3노드 클러스터에서 Ubuntu 20.04를 통한 Hadoop 3.2.1 및 Spark 3.0.0 사용하기 (1) | 2024.07.22 |
| Bigdata- 01 (1) | 2024.07.20 |
| Unit sshd.service could not be found 에러 해결 (3) | 2024.07.19 |
| 스파크 애플리케이션 기능 테스트를 위한 샘플 데이터 생성 방법 (0) | 2024.07.18 |
| 빅 데이터 - 클러스터 환경: Raspberry Pi-4, Hadoop 및 Spark 기반 (0) | 2024.07.17 |
취업, 창업의 막막함, 외주 관리, 제품 부재!
당신의 고민은 무엇입니까? 현실과 동떨어진 교육, 실패만 반복하는 외주 계약,
아이디어는 있지만 구현할 기술이 없는 막막함.
우리는 알고 있습니다. 문제의 원인은 '명확한 학습, 실전 경험과 신뢰할 수 있는 기술력의 부재'에서 시작됩니다.
이제 고민을 멈추고, 캐어랩을 만나세요!
코딩(펌웨어), 전자부품과 디지털 회로설계, PCB 설계 제작, 고객(시장/수출) 발굴과 마케팅 전략으로 당신을 지원합니다.
제품 설계의 고수는 성공이 만든 게 아니라 실패가 만듭니다. 아이디어를 양산 가능한 제품으로!
귀사의 제품을 만드세요. 교육과 개발 실적으로 신뢰할 수 있는 파트너를 확보하세요.
지난 30년 여정, 캐어랩이 얻은 모든 것을 함께 나누고 싶습니다.
귀사가 성공하기까지의 긴 고난의 시간을 캐어랩과 함께 하세요.
캐어랩
