Spark

快速开始

安装

wget https://archive.apache.org/dist/spark/spark-3.1.3/spark-3.1.3-bin-without-hadoop.tgz

配置

• conf/spark-env.sh

  JAVA_HOME=/opt/env/jdk-8
  SPARK_DIST_CLASSPATH=$(/opt/env/hadoop/bin/hadoop classpath)
  SPARK_MASTER_HOST=vm101
  SPARK_MASTER_PORT=7077
  

• conf/workers

  vm101
  

启动

sbin/start-all.sh

收藏

Spark 任务提交流程说明