Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper, mail # user - problem running giraph example "shortest path"


Copy link to this message
-
problem running giraph example "shortest path"
Nir Harel 2012-12-05, 15:07
hello friend,

we have some problems in giraph that others may have too.
we have a problem running giraph example "shortest path" over hadoop.
we install:
1. debian OS.
2. hadoop (version 1.0.3) single node
and running word count on it (successfully)
3. mvn(3.0.4), and create a jar with it.
4.upload the input for the example to the hdfs.

and when we run the example, there is no output, only logs as follow:

thanks, Gil and Nir

Terminal:

hduser@beb-1:/usr/local/giraph-trunk$ hadoop jar /home/hduser/Desktop/giraph.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsVertex -if org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexInputFormat -ip /user/hduser/shortestPathsInputGraph/ -of org.apache.giraph.io.JsonLongDoubleFloatDoubleVertexOutputFormat -op shortestPathsOutputGraph20 -w 3

12/12/05 16:27:33 INFO graph.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)
12/12/05 16:27:34 INFO mapred.JobClient: Running job: job_201212051558_0002
12/12/05 16:27:35 INFO mapred.JobClient:  map 0% reduce 0%
12/12/05 16:27:52 INFO mapred.JobClient:  map 25% reduce 0%
12/12/05 16:27:55 INFO mapred.JobClient:  map 50% reduce 0%
12/12/05 16:28:01 INFO mapred.JobClient:  map 75% reduce 0%

___________

syslog logs:

obcache/job_201212051558_0002/attempt_201212051558_0002_m_000000_0/work
2012-12-05 16:27:45,482 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=beb-1.bgu.ac.il:22181 sessionTimeout=60000 watcher=org.apache.giraph.graph.BspServiceMaster@5567d7fb
2012-12-05 16:27:45,497 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server beb-1.bgu.ac.il/127.0.1.1:22181
2012-12-05 16:27:45,501 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to beb-1.bgu.ac.il/127.0.1.1:22181, initiating session
2012-12-05 16:27:45,512 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server beb-1.bgu.ac.il/127.0.1.1:22181, sessionid = 0x13b6b770d950001, negotiated timeout = 600000
2012-12-05 16:27:45,514 INFO org.apache.giraph.graph.BspService: process: Asynchronous connection complete.
2012-12-05 16:27:45,519 INFO org.apache.giraph.graph.GraphMapper: map: No need to do anything when not a worker
2012-12-05 16:27:45,519 INFO org.apache.giraph.graph.GraphMapper: cleanup: Starting for MASTER_ZOOKEEPER_ONLY
2012-12-05 16:27:45,548 INFO org.apache.giraph.graph.BspServiceMaster: becomeMaster: First child is '/_hadoopBsp/job_201212051558_0002/_masterElectionDir/beb-1.bgu.ac.il_00000000000' and my bid is '/_hadoopBsp/job_201212051558_0002/_masterElectionDir/beb-1.bgu.ac.il_00000000000'
2012-12-05 16:27:45,587 INFO org.apache.giraph.comm.netty.NettyServer: NettyServer: Using execution handler with 8 threads after requestFrameDecoder.
2012-12-05 16:27:45,612 INFO org.apache.giraph.graph.BspService: process: applicationAttemptChanged signaled
2012-12-05 16:27:45,624 INFO org.apache.giraph.comm.netty.NettyServer: start: Started server communication server: beb-1.bgu.ac.il/127.0.1.1:30000 with up to 16 threads on bind attempt 0 with sendBufferSize = 32768 receiveBufferSize = 524288 backlog = 3
2012-12-05 16:27:45,627 INFO org.apache.giraph.comm.netty.NettyClient: NettyClient: Using execution handler with 8 threads after requestEncoder.
2012-12-05 16:27:45,630 INFO org.apache.giraph.graph.BspServiceMaster: becomeMaster: I am now the master!
2012-12-05 16:27:45,634 INFO org.apache.giraph.graph.BspService: getApplicationAttempt: Node /_hadoopBsp/job_201212051558_0002/_applicationAttemptsDir already exists!
2012-12-05 16:27:51,985 FATAL org.apache.giraph.graph.GraphMapper: uncaughtException: OverrideExceptionHandler on thread org.apache.giraph.graph.MasterThread, msg = generateVertexInputSplits: Got IOException, exiting...
java.lang.IllegalStateException: generateVertexInputSplits: Got IOException at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:270) at org.apache.giraph.graph.BspServiceMaster.createInputSplits(BspServiceMaster.java:568) at org.apache.giraph.graph.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:633) at org.apache.giraph.graph.MasterThread.run(MasterThread.java:102)
Caused by: java.io.IOException: No input paths specified in job at org.apache.giraph.io.GiraphFileInputFormat.listStatus(GiraphFileInputFormat.java:191) at org.apache.giraph.io.GiraphFileInputFormat.listVertexStatus(GiraphFileInputFormat.java:251) at org.apache.giraph.io.GiraphFileInputFormat.getVertexSplits(GiraphFileInputFormat.java:322) at org.apache.giraph.io.TextVertexInputFormat.getSplits(TextVertexInputFormat.java:61) at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:268) ... 3 more
2012-12-05 16:27:51,986 INFO org.apache.giraph.zk.ZooKeeperManager: run: Shutdown hook started.
2012-12-05 16:27:51,986 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Forced a shutdown hook kill of the ZooKeeper process.
2012-12-05 16:27:52,301 INFO org.apache.zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x13b6b770d950001, likely server has closed socket, closing socket connection and attempting reconnect
2012-12-05 16:27:52,302 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143 typically means killed).
_____________________________

ErrorLogsattempt_201212051558_0002_m_000000_0task_201212051558_0002_m_000000beb-1.bgu.ac.ilFAILED
java.lang.Throwable: Child Error at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 1. at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)