Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa, mail # user - Unable to use Swimlanes visualization.


+
Jesse Chang 2012-10-03, 08:24
+
Jesse Chang 2012-10-03, 21:05
+
Jesse Chang 2012-10-07, 22:07
Copy link to this message
-
Re: Unable to use Swimlanes visualization.
Eric Yang 2012-10-10, 03:56
In some implementations of Swimlanes, it doesn't work because of
incompatibilities with Hadoop >= 0.21 due to a change in the
JobHistory log format.  The parser may need to be updated for this
to work with newer version of Hadoop.

regards,
Eric

On Sun, Oct 7, 2012 at 3:07 PM, Jesse Chang <[EMAIL PROTECTED]> wrote:
> I'm getting the following error message from postprocess.log after I
> switched to Chukwa 0.4.0.
>
> 2012-10-07 14:17:41,166 ERROR main FSMDataLoader -
> java.io.FileNotFoundException: File
> /chukwa/temp/fsm_1349644589617_org.apache.hadoop.chukwa.analysis.salsa.fsm.JobHistoryTaskDataMapper
> does not exist.
>
> I got the same kind of error messages when I used Chukwa 0.5.0. What am I
> missing?
>
> Thanks,
> Jesse
>
> On Wed, Oct 3, 2012 at 2:05 PM, Jesse Chang <[EMAIL PROTECTED]> wrote:
>>
>> OK. I added org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter
>> to chukwaCollector.pipeline, and I am getting some data such as
>> /chukwa/postProcess/demuxOutputDir_1349295946720/MyCluster/mapred_tasktracker.
>>
>> But it cannot process data further because of the following errors.
>>
>> 2012-10-03 13:10:45,023 ERROR pool-1-thread-1 MetricDataLoader -
>> Initialization failed for: MyCluster. Please check jdbc configuration.
>>
>> I guess I need MySQL and follow the guide for the version 0.4.
>>
>> Jesse
>>
>>
>> On Wed, Oct 3, 2012 at 1:24 AM, Jesse Chang <[EMAIL PROTECTED]>
>> wrote:
>>>
>>> Hi,
>>>
>>> I am using Chukwa version 0.5.0 and the CDH4 package - "Version: 4.0.4
>>> (#79 built by jenkins on 20120821-1039 git:
>>> 119fe7452f0c356dcde763a1d22eda9363d24ad4)". I re-built Chukwa 0.5.0 with
>>> cdh4 jar files - hbase-0.92.1-cdh4.0.0.jar,
>>> hadoop-common-2.0.0-cdh4.0.1.jar, hadoop-core-2.0.0-mr1-cdh4.0.1.jar, and
>>> hadoop-hdfs-2.0.0-cdh4.0.1.jar.
>>>
>>> I could see some data such as shuffle_output_bytes was collected at
>>> http://myhost:4080/hicc/jsp/graph_explorer.jsp. I ran the aggregation script
>>> - share/chukwa/script/pig/ClusterSummary.pig - successfully to see Cluster
>>> Status graphs and MapReduce graphs at http://myhost:4080/hicc/#. There are
>>> no HDFS metrics graphs at the HDFS tab though.
>>>
>>> First of all, is the Swimlanes visualization available at Chukwa 0.5.0?
>>> If not, what is the current status?
>>>
>>> How can I check if Demux is happening? I can see
>>> org.apache.hadoop.chukwa.extraction.demux.DemuxManager is running, but I
>>> don't see anything is really happening. What would be the cause? How can I
>>> debug it? Is Demux still necessary to see a swimlanes visualization of
>>> Mapreduce stages?
>>>
>>> Basically I followed the guide at
>>> http://incubator.apache.org/chukwa/docs/r0.5.0/Quick_Start_Guide.html and
>>> http://incubator.apache.org/chukwa/docs/r0.5.0/admin.html. I just found that
>>> Chukwa Demux was removed from the picture when I compared it to the 0.4.0
>>> guide at http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html. Still
>>> does Chukwa 0.5.0 support Demux?
>>>
>>> What would be the best way to see swimlanes of mapreduce stages?
>>>
>>> Thanks,
>>> Jesse
>>>
>>>
>>>
>>>
>>>
>>
>
+
Eric Yang 2012-10-03, 15:01