Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa, mail # user - do not write data to hbase


+
Dimboo Zhu 2012-06-27, 08:51
Copy link to this message
-
Re: do not write data to hbase
Eric Yang 2012-06-27, 14:53
Hi Mason,

There is a bug in SeqFileWriter that it doesn't forward data to next writer.  It should be the last writer in the pipeline stage for pipeline writer to work.

Sent from my iPhone

On Jun 27, 2012, at 1:51 AM, Dimboo Zhu <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I configed chukwa to write data into hdfs and hbase using the following lines.
>
> <property>
>     <name>chukwaCollector.pipeline</name>            
>      <value>
> org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter,
> org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,
> org.apache.hadoop.chukwa.datacollection.writer.hbase.HBaseWriter
>      </value>
> </property>
>
> But it looks like do nothing to hbase.
> Anybody can help? Thanks,
>
> Dianbau
>
>
> 2012-06-27 15:53:25,039 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 87% complete
> 2012-06-27 15:53:48,188 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
> 2012-06-27 15:53:48,192 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
> HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
> 1.0.3   0.10.0  akazam  2012-06-27 15:48:49     2012-06-27 15:53:48     GROUP_BY
>
> Success!
>
> Job Stats (time in seconds):
> JobId   Maps    Reduces MaxMapTime      MinMapTIme      AvgMapTime      MaxReduceTime   MinReduceTime   AvgReduceTime   Alias   Feature Outputs
> job_201206211450_9575   1       0       6       6       6       0       0       0       CleanseBuffer,ConcatBuffer,HDFSMetrics  MAP_ONLY        ClusterSummary,
> job_201206211450_9576   1       1       12      12      12      21      21      21      CleanseBuffer,GroupBuffer,MapReduceMetrics,MapReduceSummary,TimeSeries  GROUP_BY,COMBINER       ClusterSummary,
> job_201206211450_9578   1       1       18      18      18      12      12      12      CleanseBuffer,ComputeBuffer,ConcatBuffer,SystemMetrics,TimeSeries       GROUP_BY,COMBINER       ClusterSummary,
> job_201206211450_9580   1       1       9       9       9       12      12      12      CleanseBuffer,GroupBuffer,MapReduceMetrics,MapReduceSummary,TimeSeries  GROUP_BY,COMBINER       ClusterSummary,
>
> Input(s):
> Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
> Successfully read 0 records (326 bytes) from: "hbase://SystemMetrics"
> Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
> Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
>
> Output(s):
> Successfully stored 0 records in: "ClusterSummary"
> Successfully stored 0 records in: "ClusterSummary"
> Successfully stored 0 records in: "ClusterSummary"
> Successfully stored 0 records in: "ClusterSummary"
>
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0
> Total bags proactively spilled: 0
> Total records proactively spilled: 0
>
> Job DAG:
> job_201206211450_9580
> job_201206211450_9578
> job_201206211450_9576
> job_201206211450_9575
>
>