Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> NullPointerException in Spring Data Hadoop with CDH4


Copy link to this message
-
Re: NullPointerException in Spring Data Hadoop with CDH4
Hi,

For Spring Data Hadoop problems, it's best to use the designated forum
[1]. These being said I've tried to reproduce your error but I can't -
I've upgraded the build to CDH 4.1.3 which runs fine against the VM on
the CI (4.1.1).
Maybe you have some other libraries on the client classpath?

 From the stacktrace, it looks like the org.apache.hadoop.mapreduce.Job
class has no 'state' or 'info' fields...

Anyway, let's continue the discussion on the forum.

Cheers,
[1] http://forum.springsource.org/forumdisplay.php?87-Hadoop

On 02/12/13 2:51 PM, Christian Schneider wrote:
> Hi,
> I try to use Spring Data Hadoop with CDH4 to write a Map Reduce Job.
>
> On startup, I get the following exception:
>
> Exception in thread "SimpleAsyncTaskExecutor-1" java.lang.ExceptionInInitializerError
> at org.springframework.data.hadoop.mapreduce.JobExecutor$2.run(JobExecutor.java:183)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.NullPointerException
> at org.springframework.util.ReflectionUtils.makeAccessible(ReflectionUtils.java:405)
> at org.springframework.data.hadoop.mapreduce.JobUtils.<clinit>(JobUtils.java:123)
> ... 2 more
>
> I guess there is a problem with my Hadoop related dependencies. I couldn't find any reference showing how to configure Spring Data together with CDH4. But Costin showed, he is able to configure it: https://build.springsource.org/browse/SPRINGDATAHADOOP-CDH4-JOB1
>
>
> **Maven Setup**
>
> <properties>
> <spring.hadoop.version>1.0.0.BUILD-SNAPSHOT</spring.hadoop.version>
> <hadoop.version>2.0.0-cdh4.1.3</hadoop.version>
> </properties>
>
> <dependencies>
> ...
> <dependency>
> <groupId>org.springframework.data</groupId>
> <artifactId>spring-data-hadoop</artifactId>
> <version>${spring.hadoop.version}</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>hadoop-common</artifactId>
> <version>${hadoop.version}</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>hadoop-client</artifactId>
> <version>${hadoop.version}</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>hadoop-streaming</artifactId>
> <version>${hadoop.version}</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>hadoop-test</artifactId>
> <version>2.0.0-mr1-cdh4.1.3</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>hadoop-tools</artifactId>
> <version>2.0.0-mr1-cdh4.1.3</version>
> </dependency>
> ...
> </dependencies>
> ...
> <repositories>
> <repository>
> <id>cloudera</id>
> <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
> <snapshots>
> <enabled>false</enabled>
> </snapshots>
> </repository>
>
> <repository>
> <id>spring-snapshot</id>
> <name>Spring Maven SNAPSHOT Repository</name>
> <url>http://repo.springframework.org/snapshot</url>
> </repository>
> </repositories>
>
> **Application Context**
>
> <?xml version="1.0" encoding="UTF-8"?>
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns:hdp="http://www.springframework.org/schema/hadoop" xmlns:context="http://www.springframework.org/schema/context"
> xmlns:hadoop="http://www.springframework.org/schema/hadoop"
> xsi:schemaLocation="
> http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd
> http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/integration
> http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd">
>
> <context:property-placeholder location="classpath:hadoop.properties" />
>
> <hdp:configuration id="hadoopConfiguration">

Costin
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB