Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> TaskStatus Exception using HFileOutputFormat


Copy link to this message
-
TaskStatus Exception using HFileOutputFormat

We're trying to use HFileOutputFormat for bulk hbase loading.   When using HFileOutputFormat's setOutputPath or configureIncrementalLoad, the job is unable to run.  The error I see in the jobtracker logs is: Trying to set finish time for task attempt_201301030046_123198_m_000002_0 when no start time is set, stackTrace is : java.lang.Exception

If I remove an references to HFileOutputFormat, and use FileOutputFormat.setOutputPath, things seem to run great.  Does anyone know what could be causing the TaskStatus error when using HFileOutputFormat?

Thanks,

Sean
What I see on the Job Tracker:

2013-02-06 00:17:33,685 ERROR org.apache.hadoop.mapred.TaskStatus: Trying to set finish time for task attempt_201301030046_123198_m_000002_0 when no start time is set, stackTrace is : java.lang.Exception
        at org.apache.hadoop.mapred.TaskStatus.setFinishTime(TaskStatus.java:145)
        at org.apache.hadoop.mapred.TaskInProgress.incompleteSubTask(TaskInProgress.java:670)
        at org.apache.hadoop.mapred.JobInProgress.failedTask(JobInProgress.java:2945)
        at org.apache.hadoop.mapred.JobInProgress.updateTaskStatus(JobInProgress.java:1162)
        at org.apache.hadoop.mapred.JobTracker.updateTaskStatuses(JobTracker.java:4739)
        at org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:3683)
        at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3378)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
What I see from the console:

391  [main] INFO  org.apache.hadoop.hbase.mapreduce.HFileOutputFormat  - Looking up current regions for table org.apache.hadoop.hbase.client.HTable@3a083b1b
1284 [main] INFO  org.apache.hadoop.hbase.mapreduce.HFileOutputFormat  - Configuring 41 reduce partitions to match current region count
1285 [main] INFO  org.apache.hadoop.hbase.mapreduce.HFileOutputFormat  - Writing partition information to file:/opt/webtrends/oozie/jobs/Lab/O/VisitorAnalytics.MapReduce/bin/partitions_1360109875112
1319 [main] INFO  org.apache.hadoop.util.NativeCodeLoader  - Loaded the native-hadoop library
1328 [main] INFO  org.apache.hadoop.io.compress.zlib.ZlibFactory  - Successfully loaded & initialized native-zlib library
1329 [main] INFO  org.apache.hadoop.io.compress.CodecPool  - Got brand-new compressor
1588 [main] INFO  org.apache.hadoop.hbase.mapreduce.HFileOutputFormat  - Incremental table output configured.
2896 [main] INFO  org.apache.hadoop.hbase.mapreduce.TableOutputFormat  - Created table instance for Lab_O_VisitorHistory
2910 [main] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat  - Total input paths to process : 1
Job Name:       job_201301030046_123199
Job Id: http://strack01.staging.dmz:50030/jobdetails.jsp?jobid=job_201301030046_123199
Job URL:        VisitorHistory MapReduce (soozie01.Lab.O)
3141 [main] INFO  org.apache.hadoop.mapred.JobClient  - Running job: job_201301030046_123199
4145 [main] INFO  org.apache.hadoop.mapred.JobClient  -  map 0% reduce 0%
10162 [main] INFO  org.apache.hadoop.mapred.JobClient  - Task Id : attempt_201301030046_123199_m_000002_0, Status : FAILED
10196 [main] WARN  org.apache.hadoop.mapred.JobClient  - Error reading task outputhttp://sdata01.staging.dmz:50060/tasklog?plaintext=true&attemptid=attempt_201301030046_123199_m_000002_0&filter=stdout
10199 [main] WARN  org.apache.hadoop.mapred.JobClient  - Error reading task outputhttp://sdata01.staging.dmz:50060/tasklog?plaintext=true&attemptid=attempt_201301030046_123199_m_000002_0&filter=stderr
10199 [main] INFO  org.apache.hadoop.mapred.JobClient  - Task Id : attempt_201301030046_123199_r_000042_0, Status : FAILED
10203 [main] WARN  org.apache.hadoop.mapred.JobClient  - Error reading task outputhttp://sdata01.staging.dmz:50060/tasklog?plaintext=true&attemptid=attempt_201301030046_123199_r_000042_0&filter=stdout
10205 [main] WARN  org.apache.hadoop.mapred.JobClient  - Error reading task outputhttp://sdata01.staging.dmz:50060/tasklog?plaintext=true&attemptid=attempt_201301030046_123199_r_000042_0&filter=stderr
10206 [main] INFO  org.apache.hadoop.mapred.JobClient  - Task Id : attempt_201301030046_123199_m_000002_1, Status : FAILED
10210 [main] WARN  org.apache.hadoop.mapred.JobClient  - Error reading task outputhttp://sdata05.staging.dmz:50060/tasklog?plaintext=true&attemptid=attempt_201301030046_123199_m_000002_1&filter=stdout
10213 [main] WARN  org.apache.hadoop.mapred.JobClient  - Error reading task outputhttp://sdata05.staging.dmz:50060/tasklog?plaintext=true&attemptid=attempt_201301030046_123199_m_000002_1&filter=stderr
10213 [main] INFO  org.apache.hadoop.mapred.JobClient  - Task Id : attempt_201301030046_123199_r_000042_1, Status : FAILED
10217 [main] WARN  org.apache.hadoop.mapred.JobClient  - Error reading task outputhttp://sdata05.staging.dmz:50060/tasklog?plaintext=true&attemptid=attempt_201301030046_123199_r_000042_1&filter=stdout
10219 [main] WARN  org.apache.hadoop.mapred.JobClient  - Error reading task outputhttp://sdata05.staging.dmz:50060/tasklog?plaintext=true&attemptid=attempt_201301030046_123199_r_000042_1&filter=stderr
10220 [main] INFO  org.apache.hadoop.mapred.JobClient  - Task Id : attempt_201301030046_123199_m_000002_2, Status : FAILED
10224 [main] WARN  org.apache.hadoop.mapred.JobClient  - Error reading task outputhttp://sdata03.staging.dmz:50060/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB