Hi all,

I configed chukwa to write data into hdfs and hbase using the following
lines.

<property>
    <name>chukwaCollector.pipeline</name>
     <value>
org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter,
org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,
org.apache.hadoop.chukwa.datacollection.writer.hbase.HBaseWriter
     </value>
</property>

But it looks like do nothing to hbase.
Anybody can help? Thanks,

Dianbau
2012-06-27 15:53:25,039 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 87% complete
2012-06-27 15:53:48,188 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2012-06-27 15:53:48,192 [main] INFO
org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:

HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt
Features
1.0.3   0.10.0  akazam  2012-06-27 15:48:49     2012-06-27 15:53:48
GROUP_BY

Success!

Job Stats (time in seconds):
JobId   Maps    Reduces MaxMapTime      MinMapTIme      AvgMapTime
MaxReduceTime   MinReduceTime   AvgReduceTime   Alias   Feature Outputs
job_201206211450_9575   1       0       6       6       6       0
0       0       CleanseBuffer,ConcatBuffer,HDFSMetrics  MAP_ONLY
ClusterSummary,
job_201206211450_9576   1       1       12      12      12      21
21      21
CleanseBuffer,GroupBuffer,MapReduceMetrics,MapReduceSummary,TimeSeries
GROUP_BY,COMBINER       ClusterSummary,
job_201206211450_9578   1       1       18      18      18      12
12      12
CleanseBuffer,ComputeBuffer,ConcatBuffer,SystemMetrics,TimeSeries
GROUP_BY,COMBINER       ClusterSummary,
job_201206211450_9580   1       1       9       9       9       12
12      12
CleanseBuffer,GroupBuffer,MapReduceMetrics,MapReduceSummary,TimeSeries
GROUP_BY,COMBINER       ClusterSummary,

Input(s):
Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
Successfully read 0 records (326 bytes) from: "hbase://SystemMetrics"
Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
Successfully read 0 records (319 bytes) from: "hbase://Hadoop"

Output(s):
Successfully stored 0 records in: "ClusterSummary"
Successfully stored 0 records in: "ClusterSummary"
Successfully stored 0 records in: "ClusterSummary"
Successfully stored 0 records in: "ClusterSummary"

Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0

Job DAG:
job_201206211450_9580
job_201206211450_9578
job_201206211450_9576
job_201206211450_9575

Hi Mason,

There is a bug in SeqFileWriter that it doesn't forward data to next writer.  It should be the last writer in the pipeline stage for pipeline writer to work.

Sent from my iPhone

On Jun 27, 2012, at 1:51 AM, Dimboo Zhu <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I configed chukwa to write data into hdfs and hbase using the following lines.
>
> <property>
>     <name>chukwaCollector.pipeline</name>            
>      <value>
> org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter,
> org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,
> org.apache.hadoop.chukwa.datacollection.writer.hbase.HBaseWriter
>      </value>
> </property>
>
> But it looks like do nothing to hbase.
> Anybody can help? Thanks,
>
> Dianbau
>
>
> 2012-06-27 15:53:25,039 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 87% complete
> 2012-06-27 15:53:48,188 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
> 2012-06-27 15:53:48,192 [main] INFO  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
> HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt      Features
> 1.0.3   0.10.0  akazam  2012-06-27 15:48:49     2012-06-27 15:53:48     GROUP_BY
>
> Success!
>
> Job Stats (time in seconds):
> JobId   Maps    Reduces MaxMapTime      MinMapTIme      AvgMapTime      MaxReduceTime   MinReduceTime   AvgReduceTime   Alias   Feature Outputs
> job_201206211450_9575   1       0       6       6       6       0       0       0       CleanseBuffer,ConcatBuffer,HDFSMetrics  MAP_ONLY        ClusterSummary,
> job_201206211450_9576   1       1       12      12      12      21      21      21      CleanseBuffer,GroupBuffer,MapReduceMetrics,MapReduceSummary,TimeSeries  GROUP_BY,COMBINER       ClusterSummary,
> job_201206211450_9578   1       1       18      18      18      12      12      12      CleanseBuffer,ComputeBuffer,ConcatBuffer,SystemMetrics,TimeSeries       GROUP_BY,COMBINER       ClusterSummary,
> job_201206211450_9580   1       1       9       9       9       12      12      12      CleanseBuffer,GroupBuffer,MapReduceMetrics,MapReduceSummary,TimeSeries  GROUP_BY,COMBINER       ClusterSummary,
>
> Input(s):
> Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
> Successfully read 0 records (326 bytes) from: "hbase://SystemMetrics"
> Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
> Successfully read 0 records (319 bytes) from: "hbase://Hadoop"
>
> Output(s):
> Successfully stored 0 records in: "ClusterSummary"
> Successfully stored 0 records in: "ClusterSummary"
> Successfully stored 0 records in: "ClusterSummary"
> Successfully stored 0 records in: "ClusterSummary"
>
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0
> Total bags proactively spilled: 0
> Total records proactively spilled: 0
>
> Job DAG:
> job_201206211450_9580
> job_201206211450_9578
> job_201206211450_9576
> job_201206211450_9575
>
>