Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa, mail # user - hicc start problem:"Unable to load dashboard"


+
scott 2012-05-24, 09:27
+
scott 2012-05-25, 04:03
+
Eric Yang 2012-06-08, 04:36
+
scott 2012-06-14, 10:26
+
Eric Yang 2012-06-15, 03:33
Copy link to this message
-
Re: hicc start problem:"Unable to load dashboard"
scott 2012-06-19, 02:37
Thanks,Eric.

I still have some doubts about the process of writing logs to hbase and
hdfs.

1. In my project, i need to collect and record some metrics for near
real-time monitoring, and also store some logs for later analysis. For
monitoring, hbase can be used, and for later log analysis, logs should be
stored into hdfs.  Chukwa has provided writting to both hbase and hdfs.
which can be set in *chukwa-collector-conf.xml  *
using chukwaCollector.pipeline
*
*
*<name>chukwaCollector.pipeline</name>*
*
<value>org.apache.hadoop.chukwa.datacollection.writer.hbase.HBaseWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value>
*
....
However, in chukwaCollector.pipeline, we must write *hbase.HBaseWriter *ahead
of* writer.SeqFileWriter,  for in **SeqFileWriter source code,* i found
that chunks will not pass to next writer. please verify that.

2. For HDFS, I want to store the data categorized by
*[dataType]/[yyyyMMdd]/[HH]/[mm]/.
 in *your last letter, you said that it's old design to have PostProcessor
to extract metrics to store in DB and it should not be used now, then, what
will we do to achieve aggregating the data into such category. Is there any
new code to check in to solve it ?
Regards
Scott
2012/6/15 Eric Yang-3 [via Apache Chukwa] <
ml-node+[EMAIL PROTECTED]>

> Hi Scott,
>
> Demux with HDFS and PostProcessor store data to mysql.  This is the
> old design before HBase came into existence.  The idea was to cleanse
> the data and extract metrics to store in database for hicc to query.
> PostProcessor was legacy code and it should not be used.
>
> regards,
> Eric
>
> On Thu, Jun 14, 2012 at 3:26 AM, scott <[hidden email]<http://user/SendEmail.jtp?type=node&node=4025017&i=0>>
> wrote:
>
> > Hi  Eric,
> >
> > Thanks for your reply, now the web interface work fine ( also for my
> usage,
> > i monitor other logs, not only Hadoop cluster)
> >
> > Then, i came to store the data into HDFS, and run Demux and PostPocessor
> to
> > divide the logs into datatype/datetime categories,
> > Here i found some running exceptions as belows: it says jdbc.conf is not
> > found.
> > Does it mean in Demux and PostProcess, the results will be written into
> > databases?  can you give some hints on what will be written for any
> reasons?
> >
> >
> > Thanks~
> >
> >
> > Best Regards!
> > Scott
> >
> >
> > Error logs sends to the console , not in Demux, postProcessor log file
> >
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> >
> > hadoop@master:~/hadoop/chukwa$ tailjava.io.FileNotFoundException:
> > /home/hadoop/hadoop/chukwa/conf/jdbc.conf (No such file or directory)
> >         at java.io.FileInputStream.open(Native Method)
> >         at java.io.FileInputStream.<init>(FileInputStream.java:120)
> >         at java.io.FileReader.<init>(FileReader.java:55)
> >         at
> >
> org.apache.hadoop.chukwa.util.ClusterConfig.getContents(ClusterConfig.java:36)
>
> >         at
> >
> org.apache.hadoop.chukwa.util.ClusterConfig.<init>(ClusterConfig.java:60)
> >         at
> >
> org.apache.hadoop.chukwa.dataloader.MetricDataLoader.initEnv(MetricDataLoader.java:96)
>
> >         at
> >
> org.apache.hadoop.chukwa.dataloader.MetricDataLoader.run(MetricDataLoader.java:202)
>
> >         at
> >
> org.apache.hadoop.chukwa.dataloader.MetricDataLoader.call(MetricDataLoader.java:577)
>
> >         at
> >
> org.apache.hadoop.chukwa.dataloader.MetricDataLoader.call(MetricDataLoader.java:48)
>
> >         at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >         at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> >         at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
厚积薄发
System.out.println("hello world again!")
View this message in context: http://apache-chukwa.679492.n3.nabble.com/hicc-start-problem-Unable-to-load-dashboard-tp4013192p4025022.html
Sent from the Chukwa - Users mailing list archive at Nabble.com.
+
Eric Yang 2012-06-19, 05:07
+
scott 2012-06-28, 10:19
+
Eric Yang 2012-06-29, 05:24