Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> The collector data sink file path


+
IvyTang 2012-03-15, 03:33
+
IvyTang 2012-03-15, 04:05
Copy link to this message
-
Re: The collector data sink file path
Hi Ivy,

This is because the configuration is using SeqFileWriter instead of LocalWriter.

You should change your configuration to:

<property>
    <name>chukwaCollector.pipeline</name>
    <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.LocalWriter</value>
</property>

to use write data locally.  By using SeqFileWriter, the data is
written to hdfs://hostname:port/chukwa/logs.

regards,
Eric

On Wed, Mar 14, 2012 at 9:05 PM, IvyTang <[EMAIL PROTECTED]> wrote:
> Sorry  ,  got it wrong.
>
> My question is:
>
>
> The the wiki says
>
> Configuring and starting the Collector
>
> Edit etc/chukwa/chukwa-collector-conf.xml and comment out the default
> properties for chukwaCollector.writerClass, and chukwaCollector.pipeline.
> Uncomment block for HBaseWriter parameters, and save.
>
> But my conf is
>   <property>
>     <name>chukwaCollector.writerClass</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value>
>   </property>
>
>   <property>
>     <name>chukwaCollector.pipeline</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value>
>   </property>
>
> however , the hadoop indeed received the log .
>
> 2. The chukwaCollector.localOutputDir and chukwaCollector.writerClass has
> been comment.
> So where is the local data sink  file ?
>
>
> On Thu, Mar 15, 2012 at 11:33 AM, IvyTang <[EMAIL PROTECTED]> wrote:
>>
>> the chukwaCollector.localOutputDir in chukwa-collector-conf.xml  is set to
>> be /tmp/chukwa/dataSink/.
>>
>> when i want to see the sink file , i  found there is no such directory.
>>
>> but the log is being transfered to hadoop.
>>
>> Where the data sink file?
>>
>> --
>> Best regards,
>>
>> Ivy Tang
>>
>>
>>
>
>
>
> --
> Best regards,
>
> Ivy Tang
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB