Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> hicc start problem:"Unable to load dashboard"

Copy link to this message
Re: hicc start problem:"Unable to load dashboard"
Hi, Eric

could you please give me some suggestions on continuously tailing a file if
any rotates happened?

In our project, the log mechasim is as follows:
when the file (eg file A )is over the size, it will copy all data to
another backup file ( which always with file name ***bak00) ,  then the
file A will truncate and resetted.

it seems that there are some code in CHUKWA to detect the rotation, and
some solution on that. Could you plz give some detailed on that, and any
advice in consideration of our log mechanism, to make little revision to
fully collect the log data?


Scott Huan
2012/6/19 Eric Yang-3 [via Apache Chukwa] <

> On Mon, Jun 18, 2012 at 7:37 PM, scott <[hidden email]<http://user/SendEmail.jtp?type=node&node=4025023&i=0>>
> wrote:
> > Thanks,Eric.
> >
> > I still have some doubts about the process of writing logs to hbase and
> > hdfs.
> >
> > 1. In my project, i need to collect and record some metrics for near
> > real-time monitoring, and also store some logs for later analysis. For
> > monitoring, hbase can be used, and for later log analysis, logs should
> be
> > stored into hdfs.  Chukwa has provided writting to both hbase and hdfs.
> > which can be set in
> > chukwa-collector-conf.xml  using chukwaCollector.pipeline
> >
> > <name>chukwaCollector.pipeline</name>
> >
> <value>org.apache.hadoop.chukwa.datacollection.writer.hbase.HBaseWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value>
> > ....
> > However, in chukwaCollector.pipeline, we must write hbase.HBaseWriter
> ahead
> > of writer.SeqFileWriter,  for in SeqFileWriter source code, i found
> > that chunks will not pass to next writer. please verify that.
> SeqFileWriter does not pass to next writer, this is the reason that it
> has to be the last writer in the pipeline as a workaround.
> > 2. For HDFS, I want to store the data categorized by
> > [dataType]/[yyyyMMdd]/[HH]/[mm]/.  in your last letter, you said that
> it's
> > old design to have PostProcessor to extract metrics to store in DB and
> it
> > should not be used now, then, what will we do to achieve aggregating the
> > data into such category. Is there any new code to check in to solve it ?
> There is no new code written for inject MR data to DB or HBase,
> patches are welcome.
> regards,
> Eric
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
> http://apache-chukwa.679492.n3.nabble.com/hicc-start-problem-Unable-to-load-dashboard-tp4013192p4025023.html
>  To unsubscribe from hicc start problem:"Unable to load dashboard", click
> here<http://apache-chukwa.679492.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4013192&code=c2Nvb3RodWFuQGdtYWlsLmNvbXw0MDEzMTkyfC0xNDMwOTk2Njc=>
> .
> NAML<http://apache-chukwa.679492.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
System.out.println("hello world again!")
View this message in context: http://apache-chukwa.679492.n3.nabble.com/hicc-start-problem-Unable-to-load-dashboard-tp4013192p4025057.html
Sent from the Chukwa - Users mailing list archive at Nabble.com.