zeppelin编译安装笔记

发表于 | 分类于 | | 阅读次数 |

clone工程

git clone --recursive https://github.com/apache/zeppelin.git

依赖软件安装

1
node/cnpm/R(evaluate包必须)

zeppelin-web编译

1
2
3
4
1. cnpm install -g
2. bower –-alow-root install
3. grunt –-force
若报错，需要先删掉zeppelin-web目录下的node_modules目录，再使用zeppelin编译命令编译

zeppelin编译

1
2
3
4
5
6
7
# update all pom.xml to use scala 2.11
./dev/change_scala_version.sh 2.11
# build zeppelin with all interpreters and include latest version of Apache spark support for local mode.
`mvn clean package -DskipTests -Pspark-2.1 -Phadoop-2.7 -Pyarn -Ppyspark -Psparkr -Pr -Pscala-2.11`
or 指定hadoop版本
`mvn clean package -DskipTests -Pspark-2.1 -Phadoop.version=2.7.3 -Pyarn -Ppyspark -Psparkr -Pr -Pscala-2.11`
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
ERROR [2017-08-24 14:06:12,405] ({pool-2-thread-2} Logging.scala[logError]:91) - Error initializing SparkContext.
org.apache.spark.SparkException: Found both spark.driver.extraClassPath and SPARK_CLASSPATH. Use only the former.
    at org.apache.spark.SparkConf$$anonfun$validateSettings$7$$anonfun$apply$8.apply(SparkConf.scala:543)
    at org.apache.spark.SparkConf$$anonfun$validateSettings$7$$anonfun$apply$8.apply(SparkConf.scala:541)
    at scala.collection.immutable.List.foreach(List.scala:381)
    at org.apache.spark.SparkConf$$anonfun$validateSettings$7.apply(SparkConf.scala:541)
    at org.apache.spark.SparkConf$$anonfun$validateSettings$7.apply(SparkConf.scala:529)
    at scala.Option.foreach(Option.scala:257)
    at org.apache.spark.SparkConf.validateSettings(SparkConf.scala:529)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:365)
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2258)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply(SparkSession.scala:831)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$8.apply(SparkSession.scala:823)
    at scala.Option.getOrElse(Option.scala:121)
	
	
解决办法：注释$SPARK_HOME/spark-env.sh中的SPARK_CLASSPATH变量
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.tracing.SpanReceiverHost.get(Lorg/apache/hadoop/conf/Configuration;Ljava/lang/String;)Lorg/apache/hadoop/tracing/SpanReceiverHost;
解决办法：hadoop版本不匹配
发现org.apache.hadoop.tracing.SpanReceiverHost.get(xxx)报错是由于Zeppelin提供的Hadoop版本和Spark编译时指定的版本不一致引起，只需要使用$SPARK_HOME/jars/hadoop-annotations-2.7.3.jar、hadoop-auth-2.7.3.jar、hadoop-common-2.7.3.jar替换掉$ZEPPELIN_HOME/lib下的对应文件即可。

###启停zeppelin

1
./bin/zeppelin-daemon.sh start/stop

Refrence

1. Apache Zeppelin安装及介绍