Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> hicc start problem:"Unable to load dashboard"


Copy link to this message
-
Re: hicc start problem:"Unable to load dashboard"
On Mon, Jun 18, 2012 at 7:37 PM, scott <[EMAIL PROTECTED]> wrote:
> Thanks,Eric.
>
> I still have some doubts about the process of writing logs to hbase and
> hdfs.
>
> 1. In my project, i need to collect and record some metrics for near
> real-time monitoring, and also store some logs for later analysis. For
> monitoring, hbase can be used, and for later log analysis, logs should be
> stored into hdfs.  Chukwa has provided writting to both hbase and hdfs.
> which can be set in
> chukwa-collector-conf.xml  using chukwaCollector.pipeline
>
> <name>chukwaCollector.pipeline</name>
> <value>org.apache.hadoop.chukwa.datacollection.writer.hbase.HBaseWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value>
> ....
> However, in chukwaCollector.pipeline, we must write hbase.HBaseWriter ahead
> of writer.SeqFileWriter,  for in SeqFileWriter source code, i found
> that chunks will not pass to next writer. please verify that.

SeqFileWriter does not pass to next writer, this is the reason that it
has to be the last writer in the pipeline as a workaround.

> 2. For HDFS, I want to store the data categorized by
> [dataType]/[yyyyMMdd]/[HH]/[mm]/.  in your last letter, you said that it's
> old design to have PostProcessor to extract metrics to store in DB and it
> should not be used now, then, what will we do to achieve aggregating the
> data into such category. Is there any new code to check in to solve it ?

There is no new code written for inject MR data to DB or HBase,
patches are welcome.

regards,
Eric
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB