Hello,

I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
Hbase. And I want to see flume-ng logs. But I can not find log under
flume-ng root directory. My log4j.properties configure the log directory as
default "./logs".

And I start flume-ng as following:
> Flume-ng agent ‹name host1 ‹conf-file /home/hadoop/flume-ng/conf/razor.conf

Thanks a lot
Regards

Hi.

"./logs" means in the directory, from where you start flume, in the subdirectory "logs".

Best,
 Alex

On Jan 2, 2013, at 6:45 AM, GuoWei <[EMAIL PROTECTED]> wrote:

> Hello,
>
> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
> Hbase. And I want to see flume-ng logs. But I can not find log under
> flume-ng root directory. My log4j.properties configure the log directory as
> default "./logs".
>
> And I start flume-ng as following:
>> Flume-ng agent ‹name host1 ‹conf-file /home/hadoop/flume-ng/conf/razor.conf
>
> Thanks a lot
> Regards
>
>

Hello, Alexander,

I start flume from the root directory ex: /home/hadoop/flume-ng.
And I also make logs directory in flume-mg directory. And give the logs
folder 777 permission.
But I still can not see the log files under logs directory.

I can view the log from the console. But never get the log file.

Any wrong configurations ?

Thanks
On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <[EMAIL PROTECTED]> wrote:

>Hi.
>
>"./logs" means in the directory, from where you start flume, in the
>subdirectory "logs".
>
>Best,
> Alex
>
>On Jan 2, 2013, at 6:45 AM, GuoWei <[EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>> Hbase. And I want to see flume-ng logs. But I can not find log under
>> flume-ng root directory. My log4j.properties configure the log
>>directory as
>> default "./logs".
>>
>> And I start flume-ng as following:
>>> Flume-ng agent ‹name host1 ‹conf-file
>>>/home/hadoop/flume-ng/conf/razor.conf
>>
>> Thanks a lot
>> Regards
>>
>>
>
>

Hi,

You've to add the log4j.properties into the classpath:

bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>

<config directory> means the directory where log4j.properties is located

In log4j.properties be sure you've the correct path:

flume.root.logger=INFO,LOGFILE
flume.log.dir=./logs
flume.log.file=flume.log

- Alex

On Jan 2, 2013, at 11:00 AM, GuoWei <[EMAIL PROTECTED]> wrote:

> Hello, Alexander,
>
> I start flume from the root directory ex: /home/hadoop/flume-ng.
> And I also make logs directory in flume-mg directory. And give the logs
> folder 777 permission.
> But I still can not see the log files under logs directory.
>
> I can view the log from the console. But never get the log file.
>
> Any wrong configurations ?
>
> Thanks
>
>
>
>
> On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <[EMAIL PROTECTED]> wrote:
>
>> Hi.
>>
>> "./logs" means in the directory, from where you start flume, in the
>> subdirectory "logs".
>>
>> Best,
>> Alex
>>
>> On Jan 2, 2013, at 6:45 AM, GuoWei <[EMAIL PROTECTED]> wrote:
>>
>>> Hello,
>>>
>>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>>> Hbase. And I want to see flume-ng logs. But I can not find log under
>>> flume-ng root directory. My log4j.properties configure the log
>>> directory as
>>> default "./logs".
>>>
>>> And I start flume-ng as following:
>>>> Flume-ng agent ‹name host1 ‹conf-file
>>>> /home/hadoop/flume-ng/conf/razor.conf
>>>
>>> Thanks a lot
>>> Regards
>>>
>>>
>>
>>
>
>

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

Using that method, if we have multiple agents, they all append to the same flume.log file.

Is there way to create separate log file for each agent without creating separate folder for each flume-Ng configuration?

Sent from my Windows Phone
________________________________
From: Alexander Alten-Lorenz
Sent: 02-01-2013 16:48
To: [EMAIL PROTECTED]
Subject: Re: where to Find flume-ng logs

Hi,

You've to add the log4j.properties into the classpath:

bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>

<config directory> means the directory where log4j.properties is located

In log4j.properties be sure you've the correct path:

flume.root.logger=INFO,LOGFILE
flume.log.dir=./logs
flume.log.file=flume.log

- Alex

On Jan 2, 2013, at 11:00 AM, GuoWei <[EMAIL PROTECTED]> wrote:

> Hello, Alexander,
>
> I start flume from the root directory ex: /home/hadoop/flume-ng.
> And I also make logs directory in flume-mg directory. And give the logs
> folder 777 permission.
> But I still can not see the log files under logs directory.
>
> I can view the log from the console. But never get the log file.
>
> Any wrong configurations ?
>
> Thanks
>
>
>
>
> On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <[EMAIL PROTECTED]> wrote:
>
>> Hi.
>>
>> "./logs" means in the directory, from where you start flume, in the
>> subdirectory "logs".
>>
>> Best,
>> Alex
>>
>> On Jan 2, 2013, at 6:45 AM, GuoWei <[EMAIL PROTECTED]> wrote:
>>
>>> Hello,
>>>
>>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>>> Hbase. And I want to see flume-ng logs. But I can not find log under
>>> flume-ng root directory. My log4j.properties configure the log
>>> directory as
>>> default "./logs".
>>>
>>> And I start flume-ng as following:
>>>> Flume-ng agent ‹name host1 ‹conf-file
>>>> /home/hadoop/flume-ng/conf/razor.conf
>>>
>>> Thanks a lot
>>> Regards
>>>
>>>
>>
>>
>
>

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

You could pass per agent a different log4j.property file as well as a different flume-env.sh by setting up different directories and point each agent into.

On Jan 4, 2013, at 11:01 AM, Abhijeet Pathak <[EMAIL PROTECTED]> wrote:

> Using that method, if we have multiple agents, they all append to the same flume.log file.
>
> Is there way to create separate log file for each agent without creating separate folder for each flume-Ng configuration?
>
> Sent from my Windows Phone
> ________________________________
> From: Alexander Alten-Lorenz
> Sent: 02-01-2013 16:48
> To: [EMAIL PROTECTED]
> Subject: Re: where to Find flume-ng logs
>
> Hi,
>
> You've to add the log4j.properties into the classpath:
>
> bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>
>
> <config directory> means the directory where log4j.properties is located
>
> In log4j.properties be sure you've the correct path:
>
> flume.root.logger=INFO,LOGFILE
> flume.log.dir=./logs
> flume.log.file=flume.log
>
> - Alex
>
> On Jan 2, 2013, at 11:00 AM, GuoWei <[EMAIL PROTECTED]> wrote:
>
>> Hello, Alexander,
>>
>> I start flume from the root directory ex: /home/hadoop/flume-ng.
>> And I also make logs directory in flume-mg directory. And give the logs
>> folder 777 permission.
>> But I still can not see the log files under logs directory.
>>
>> I can view the log from the console. But never get the log file.
>>
>> Any wrong configurations ?
>>
>> Thanks
>>
>>
>>
>>
>> On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <[EMAIL PROTECTED]> wrote:
>>
>>> Hi.
>>>
>>> "./logs" means in the directory, from where you start flume, in the
>>> subdirectory "logs".
>>>
>>> Best,
>>> Alex
>>>
>>> On Jan 2, 2013, at 6:45 AM, GuoWei <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hello,
>>>>
>>>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>>>> Hbase. And I want to see flume-ng logs. But I can not find log under
>>>> flume-ng root directory. My log4j.properties configure the log
>>>> directory as
>>>> default "./logs".
>>>>
>>>> And I start flume-ng as following:
>>>>> Flume-ng agent ‹name host1 ‹conf-file
>>>>> /home/hadoop/flume-ng/conf/razor.conf
>>>>
>>>> Thanks a lot
>>>> Regards
>>>>
>>>>
>>>
>>>
>>
>>
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
>
>

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

Here's the command you should use to run flume:

*/path/to/flume-ng agent —name host1 **--conf /home/hadoop/flume-ng/conf **
-f /home/hadoop/flume-ng/conf/razor.conf *
And if you want to manually specify the location of the logs, you can
either hard code it into the log4j.properties or you can remove the line
"flume.log.dir=./log" from that file (don't know if that's necessary) and
then add the following to the end of your flume command:

-Dflume.log.dir=/log/dir

- Connor

On Wed, Jan 2, 2013 at 3:17 AM, Alexander Alten-Lorenz
<[EMAIL PROTECTED]>wrote:

> Hi,
>
> You've to add the log4j.properties into the classpath:
>
> bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>
>
> <config directory> means the directory where log4j.properties is located
>
> In log4j.properties be sure you've the correct path:
>
> flume.root.logger=INFO,LOGFILE
> flume.log.dir=./logs
> flume.log.file=flume.log
>
> - Alex
>
> On Jan 2, 2013, at 11:00 AM, GuoWei <[EMAIL PROTECTED]> wrote:
>
> > Hello, Alexander,
> >
> > I start flume from the root directory ex: /home/hadoop/flume-ng.
> > And I also make logs directory in flume-mg directory. And give the logs
> > folder 777 permission.
> > But I still can not see the log files under logs directory.
> >
> > I can view the log from the console. But never get the log file.
> >
> > Any wrong configurations ?
> >
> > Thanks
> >
> >
> >
> >
> > On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <[EMAIL PROTECTED]> wrote:
> >
> >> Hi.
> >>
> >> "./logs" means in the directory, from where you start flume, in the
> >> subdirectory "logs".
> >>
> >> Best,
> >> Alex
> >>
> >> On Jan 2, 2013, at 6:45 AM, GuoWei <[EMAIL PROTECTED]> wrote:
> >>
> >>> Hello,
> >>>
> >>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
> >>> Hbase. And I want to see flume-ng logs. But I can not find log under
> >>> flume-ng root directory. My log4j.properties configure the log
> >>> directory as
> >>> default "./logs".
> >>>
> >>> And I start flume-ng as following:
> >>>> Flume-ng agent ‹name host1 ‹conf-file
> >>>> /home/hadoop/flume-ng/conf/razor.conf
> >>>
> >>> Thanks a lot
> >>> Regards
> >>>
> >>>
> >>
> >>
> >
> >
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>

Thanks Connor and Alexandra.
I got the log file.

Regards
GuoWei

From:  Connor Woodson <[EMAIL PROTECTED]>
Reply-To:  <[EMAIL PROTECTED]>
Date:  Wed, 2 Jan 2013 14:41:12 -0800
To:  <[EMAIL PROTECTED]>
Subject:  Re: where to Find flume-ng logs

Here's the command you should use to run flume:

/path/to/flume-ng agent —name host1 --conf /home/hadoop/flume-ng/conf -f
/home/hadoop/flume-ng/conf/razor.conf
And if you want to manually specify the location of the logs, you can either
hard code it into the log4j.properties or you can remove the line
"flume.log.dir=./log" from that file (don't know if that's necessary) and
then add the following to the end of your flume command:

-Dflume.log.dir=/log/dir

- Connor

On Wed, Jan 2, 2013 at 3:17 AM, Alexander Alten-Lorenz <[EMAIL PROTECTED]>
wrote:
> Hi,
>
> You've to add the log4j.properties into the classpath:
>
> bin/flume-ng agent -n <agent> -f <configfile> --conf <config directory>
>
> <config directory> means the directory where log4j.properties is located
>
> In log4j.properties be sure you've the correct path:
>
> flume.root.logger=INFO,LOGFILE
> flume.log.dir=./logs
> flume.log.file=flume.log
>
> - Alex
>
> On Jan 2, 2013, at 11:00 AM, GuoWei <[EMAIL PROTECTED]> wrote:
>
>> > Hello, Alexander,
>> >
>> > I start flume from the root directory ex: /home/hadoop/flume-ng.
>> > And I also make logs directory in flume-mg directory. And give the logs
>> > folder 777 permission.
>> > But I still can not see the log files under logs directory.
>> >
>> > I can view the log from the console. But never get the log file.
>> >
>> > Any wrong configurations ?
>> >
>> > Thanks
>> >
>> >
>> >
>> >
>> > On 13-1-2 下午3:23, "Alexander Alten-Lorenz" <[EMAIL PROTECTED]> wrote:
>> >
>>> >> Hi.
>>> >>
>>> >> "./logs" means in the directory, from where you start flume, in the
>>> >> subdirectory "logs".
>>> >>
>>> >> Best,
>>> >> Alex
>>> >>
>>> >> On Jan 2, 2013, at 6:45 AM, GuoWei <[EMAIL PROTECTED]> wrote:
>>> >>
>>>> >>> Hello,
>>>> >>>
>>>> >>> I use flume-mg 1.3 to collect logs and user Hbase Sink to store data to
>>>> >>> Hbase. And I want to see flume-ng logs. But I can not find log under
>>>> >>> flume-ng root directory. My log4j.properties configure the log
>>>> >>> directory as
>>>> >>> default "./logs".
>>>> >>>
>>>> >>> And I start flume-ng as following:
>>>>> >>>> Flume-ng agent ‹name host1 ‹conf-file
>>>>> >>>> /home/hadoop/flume-ng/conf/razor.conf
>>>> >>>
>>>> >>> Thanks a lot
>>>> >>> Regards
>>>> >>>
>>>> >>>
>>> >>
>>> >>
>> >
>> >
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>