Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa, mail # user - The collector data sink file path


+
IvyTang 2012-03-15, 03:33
+
IvyTang 2012-03-15, 04:05
Copy link to this message
-
Re: The collector data sink file path
Eric Yang 2012-03-16, 19:40
Hi Ivy,

This is because the configuration is using SeqFileWriter instead of LocalWriter.

You should change your configuration to:

<property>
    <name>chukwaCollector.pipeline</name>
    <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.LocalWriter</value>
</property>

to use write data locally.  By using SeqFileWriter, the data is
written to hdfs://hostname:port/chukwa/logs.

regards,
Eric

On Wed, Mar 14, 2012 at 9:05 PM, IvyTang <[EMAIL PROTECTED]> wrote:
> Sorry  ,  got it wrong.
>
> My question is:
>
>
> The the wiki says
>
> Configuring and starting the Collector
>
> Edit etc/chukwa/chukwa-collector-conf.xml and comment out the default
> properties for chukwaCollector.writerClass, and chukwaCollector.pipeline.
> Uncomment block for HBaseWriter parameters, and save.
>
> But my conf is
>   <property>
>     <name>chukwaCollector.writerClass</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.PipelineStageWriter</value>
>   </property>
>
>   <property>
>     <name>chukwaCollector.pipeline</name>
>
> <value>org.apache.hadoop.chukwa.datacollection.writer.SocketTeeWriter,org.apache.hadoop.chukwa.datacollection.writer.SeqFileWriter</value>
>   </property>
>
> however , the hadoop indeed received the log .
>
> 2. The chukwaCollector.localOutputDir and chukwaCollector.writerClass has
> been comment.
> So where is the local data sink  file ?
>
>
> On Thu, Mar 15, 2012 at 11:33 AM, IvyTang <[EMAIL PROTECTED]> wrote:
>>
>> the chukwaCollector.localOutputDir in chukwa-collector-conf.xml  is set to
>> be /tmp/chukwa/dataSink/.
>>
>> when i want to see the sink file , i  found there is no such directory.
>>
>> but the log is being transfered to hadoop.
>>
>> Where the data sink file?
>>
>> --
>> Best regards,
>>
>> Ivy Tang
>>
>>
>>
>
>
>
> --
> Best regards,
>
> Ivy Tang
>
>
>