Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> Data not visible in Hbase


Copy link to this message
-
Re: Data not visible in Hbase
Hi,

Do you have writer.hdfs.filesystem in chukwa-collector-conf.xml
pointing to your namenode?
>From the collector log file, it indicates that there are data written
to HBase, and HDFS.

HICC dashboard does not show any data unless you run
ClusterSummary.pig aggregation script.
You can debug if there is data in HBase by running HBase shell like this:

scan "SystemMetrics", {LIMIT=>100}

If everything is working as expected, there should be some data
appears on the console.
For data sending to HDFS, the data is normally written to:

hdfs://namenode:8020/chukwa/logs/*.done files

The data on HDFS will only get splitted apart by running chukwa demux ETL job.
After demux, the data would be stored to hdfs://namenode:8020/chukwa/repos.

Hope this helps.
regards,
Eric

On Sun, Dec 18, 2011 at 2:39 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
> Hello list,
>
>   I am using Chukwa-0.5.0 to collect data to HDFS and Hbase.All the
> processes seem to be running fine but I am not able to see anything
> happening on the HDFA and Hbase web interface.Need some guidance.Here
> is the content of agent.log file -
>
>  2011-12-19 03:56:59,936 INFO main Log4JMetricsContext -
> Log4JMetricsContext.chukwaAgent.period=60
> 2011-12-19 03:57:00,052 INFO main ChukwaAgent - Config - CHUKWA_HOME:
> [/home/saif/chukwa-incubating-0.5.0]
> 2011-12-19 03:57:00,052 INFO main ChukwaAgent - Config -
> CHUKWA_CONF_DIR: [/home/saif/chukwa-incubating-0.5.0/etc/chukwa]
> 2011-12-19 03:57:00,354 INFO main ChukwaAgent - Config -
> CHECKPOINT_BASE_NAME: [chukwa_agent_checkpoint]
> 2011-12-19 03:57:00,355 INFO main ChukwaAgent - Config -
> checkpointDir: [/home/saif/chukwa/log]
> 2011-12-19 03:57:00,355 INFO main ChukwaAgent - Config -
> CHECKPOINT_INTERVAL_MS: [5000]
> 2011-12-19 03:57:00,355 INFO main ChukwaAgent - Config -
> DO_CHECKPOINT_RESTORE: [true]
> 2011-12-19 03:57:00,355 INFO main ChukwaAgent - Config -
> STATS_INTERVAL_MS: [10000]
> 2011-12-19 03:57:00,355 INFO main ChukwaAgent - Config - tags:
> [cluster="chukwa"]
> 2011-12-19 03:57:00,355 INFO main ChukwaAgent - checkpoints are
> enabled, period is 5000
> 2011-12-19 03:57:00,356 INFO main ChukwaAgent - No checkpoints found
> in /home/saif/chukwa/log
> 2011-12-19 03:57:00,356 INFO main ChukwaAgent - starting adaptors
> listed in /home/saif/chukwa-incubating-0.5.0/etc/chukwa/initial_adaptors
> 2011-12-19 03:57:00,450 INFO main ChukwaAgent - started a new adaptor,
> id = adaptor_c79bf882974a14286cffe29d3d4cf0d6
> function=[org.apache.hadoop.chukwa.datacollection.adaptor.sigar.SystemMetrics@56c163f]
> 2011-12-19 03:57:00,454 INFO main ChukwaAgent - started a new adaptor,
> id = adaptor_93df4746476c9a4b624f6755b122f9dc
> function=[org.apache.hadoop.chukwa.datacollection.adaptor.SocketAdaptor@7d95d4fe]
> 2011-12-19 03:57:00,455 INFO main ChukwaAgent - started a new adaptor,
> id = adaptor_79203ebcc495e11b747254e660380620
> function=[org.apache.hadoop.chukwa.datacollection.adaptor.SocketAdaptor@177f409c]
> 2011-12-19 03:57:00,456 INFO main ChukwaAgent - started a new adaptor,
> id = adaptor_c6fd1e5021ad8023477ef0b56a24a29f
> function=[org.apache.hadoop.chukwa.datacollection.adaptor.SocketAdaptor@2b8bbc5a]
> 2011-12-19 03:57:00,457 INFO main ChukwaAgent - started a new adaptor,
> id = adaptor_e2b39549cac6e966126df286a79ae6ad
> function=[org.apache.hadoop.chukwa.datacollection.adaptor.SocketAdaptor@513c6a22]
> 2011-12-19 03:57:00,460 INFO main FileAdaptor - adaptor id:
> adaptor_0e36ff3fa49e813872f20a93305db668 started file adaptor on file
> /home/saif/hbase
> 2011-12-19 03:57:00,460 INFO main ChukwaAgent - started a new adaptor,
> id = adaptor_0e36ff3fa49e813872f20a93305db668
> function=[org.apache.hadoop.chukwa.datacollection.adaptor.FileAdaptor@7455d93d]
> 2011-12-19 03:57:00,461 INFO Thread-8 FileAdaptorTailer - calling
> sendFile for /home/saif/hbase
> 2011-12-19 03:57:00,462 INFO main AgentControlSocketListener -
> AgentControlSocketListerner ask for port: 9093
> 2011-12-19 03:57:00,462 INFO main AgentControlSocketListener - socket