Kumar, Deepak8
2012-09-18, 18:02
Brock Noland
2012-09-18, 18:05
Shumin Wu
2012-09-18, 18:10
Kumar, Deepak8
2012-10-03, 10:42
Shumin Wu
2012-10-03, 14:10
Kumar, Deepak8
2012-10-05, 11:19
Brock Noland
2012-10-05, 14:35
Kumar, Deepak8
2012-10-05, 15:08
Brock Noland
2012-10-05, 15:17
Kumar, Deepak8
2012-10-05, 15:28
Brock Noland
2012-10-08, 16:02
Kumar, Deepak8
2012-10-09, 06:03
Brock Noland
2012-10-09, 14:51
-
Moving from Flume OG to Flume NGKumar, Deepak8 2012-09-18, 18:02
Hi,
I am trying to switch from Flume 0.94 to Flume NG. In previous version we do have the static class EventSource.Base through which we can create our own Source or Sink. I don't find anything as such here in flume NG.
Could you please guide me how do we create our own Sources here in Flume NG?
Regards,
Deepak
I am trying to switch from Flume 0.94 to Flume NG. In previous version we do have the static class EventSource.Base through which we can create our own Source or Sink. I don't find anything as such here in flume NG.
Could you please guide me how do we create our own Sources here in Flume NG?
Regards,
Deepak
+
Kumar, Deepak8 2012-09-18, 18:02
-
Re: Moving from Flume OG to Flume NGBrock Noland 2012-09-18, 18:05
Hi,
The flume developer guide as these details:
http://flume.apache.org/FlumeDeveloperGuide.html
Brock
On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Hi,
> I am trying to switch from Flume 0.94 to Flume NG. In previous version we do have the static class EventSource.Base through which we can create our own Source or Sink. I don't find anything as such here in flume NG.
>
> Could you please guide me how do we create our own Sources here in Flume NG?
>
> Regards,
> Deepak
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
The flume developer guide as these details:
http://flume.apache.org/FlumeDeveloperGuide.html
Brock
On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Hi,
> I am trying to switch from Flume 0.94 to Flume NG. In previous version we do have the static class EventSource.Base through which we can create our own Source or Sink. I don't find anything as such here in flume NG.
>
> Could you please guide me how do we create our own Sources here in Flume NG?
>
> Regards,
> Deepak
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Brock Noland 2012-09-18, 18:05
-
Re: Moving from Flume OG to Flume NGShumin Wu 2012-09-18, 18:10
Hi Deepark,
I had hard time switching from OG to NG in the beginning, too. The new way
in NG to make your custom source would look like this:
public class CustomSource extends AbstractSource implements PollableSource,
Configurable
or
public class CustomSource extends AbstractSource implements
EventDrivenSource, Configurable
, depending on your source type.
Brock's link is a good reading resource.
Shumin
On Tue, Sep 18, 2012 at 11:05 AM, Brock Noland <[EMAIL PROTECTED]> wrote:
> Hi,
>
> The flume developer guide as these details:
>
> http://flume.apache.org/FlumeDeveloperGuide.html
>
> Brock
>
> On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> > I am trying to switch from Flume 0.94 to Flume NG. In previous version
> we do have the static class EventSource.Base through which we can create
> our own Source or Sink. I don't find anything as such here in flume NG.
> >
> > Could you please guide me how do we create our own Sources here in Flume
> NG?
> >
> > Regards,
> > Deepak
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>
I had hard time switching from OG to NG in the beginning, too. The new way
in NG to make your custom source would look like this:
public class CustomSource extends AbstractSource implements PollableSource,
Configurable
or
public class CustomSource extends AbstractSource implements
EventDrivenSource, Configurable
, depending on your source type.
Brock's link is a good reading resource.
Shumin
On Tue, Sep 18, 2012 at 11:05 AM, Brock Noland <[EMAIL PROTECTED]> wrote:
> Hi,
>
> The flume developer guide as these details:
>
> http://flume.apache.org/FlumeDeveloperGuide.html
>
> Brock
>
> On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> > I am trying to switch from Flume 0.94 to Flume NG. In previous version
> we do have the static class EventSource.Base through which we can create
> our own Source or Sink. I don't find anything as such here in flume NG.
> >
> > Could you please guide me how do we create our own Sources here in Flume
> NG?
> >
> > Regards,
> > Deepak
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>
+
Shumin Wu 2012-09-18, 18:10
-
RE: Moving from Flume OG to Flume NGKumar, Deepak8 2012-10-03, 10:42
Hi Shumin,
I have gone through the documentation, but it don't have much. Do you have any use case in which you are using CustomSource? As in flume 0.9.4, we do have flume master url where we can see the source/sinks/decorators etc. How we can verify here whether our custom source is fine & up or have any errors?
It would be much help if you please guide me a bit.
Regards,
Deepak
From: Shumin Wu [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, September 18, 2012 2:11 PM
To: [EMAIL PROTECTED]
Subject: Re: Moving from Flume OG to Flume NG
Hi Deepark,
I had hard time switching from OG to NG in the beginning, too. The new way in NG to make your custom source would look like this:
public class CustomSource extends AbstractSource implements PollableSource, Configurable
or
public class CustomSource extends AbstractSource implements EventDrivenSource, Configurable
, depending on your source type.
Brock's link is a good reading resource.
Shumin
On Tue, Sep 18, 2012 at 11:05 AM, Brock Noland <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi,
The flume developer guide as these details:
http://flume.apache.org/FlumeDeveloperGuide.html
Brock
On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
> Hi,
> I am trying to switch from Flume 0.94 to Flume NG. In previous version we do have the static class EventSource.Base through which we can create our own Source or Sink. I don't find anything as such here in flume NG.
>
> Could you please guide me how do we create our own Sources here in Flume NG?
>
> Regards,
> Deepak
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
I have gone through the documentation, but it don't have much. Do you have any use case in which you are using CustomSource? As in flume 0.9.4, we do have flume master url where we can see the source/sinks/decorators etc. How we can verify here whether our custom source is fine & up or have any errors?
It would be much help if you please guide me a bit.
Regards,
Deepak
From: Shumin Wu [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, September 18, 2012 2:11 PM
To: [EMAIL PROTECTED]
Subject: Re: Moving from Flume OG to Flume NG
Hi Deepark,
I had hard time switching from OG to NG in the beginning, too. The new way in NG to make your custom source would look like this:
public class CustomSource extends AbstractSource implements PollableSource, Configurable
or
public class CustomSource extends AbstractSource implements EventDrivenSource, Configurable
, depending on your source type.
Brock's link is a good reading resource.
Shumin
On Tue, Sep 18, 2012 at 11:05 AM, Brock Noland <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi,
The flume developer guide as these details:
http://flume.apache.org/FlumeDeveloperGuide.html
Brock
On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
> Hi,
> I am trying to switch from Flume 0.94 to Flume NG. In previous version we do have the static class EventSource.Base through which we can create our own Source or Sink. I don't find anything as such here in flume NG.
>
> Could you please guide me how do we create our own Sources here in Flume NG?
>
> Regards,
> Deepak
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Kumar, Deepak8 2012-10-03, 10:42
-
Re: Moving from Flume OG to Flume NGShumin Wu 2012-10-03, 14:10
Our company has some BI that of course cannot be available in the default
sources provided by flume-ng. So yes, I am using the custom source, in a
way that is pretty much like the code snippet I have pasted in last email.
So far I have found the flume log a good resource to scrub error/exception
informations. You may configure the log4j property file to dump any many
details as you can.
AFAIK, there is no master/node notion in NG any longer. If you want to set
up a distributed cluster, you may take a look at the "setting multi-agent
flow" at
https://people.apache.org/~mpercy/flume/flume-1.2.0-incubating-SNAPSHOT/docs/FlumeUserGuide.html
Good luck!
Shumin
On Wed, Oct 3, 2012 at 3:42 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>wrote:
> Hi Shumin,****
>
> I have gone through the documentation, but it don’t have much. Do you have
> any use case in which you are using CustomSource? As in flume 0.9.4, we do
> have flume master url where we can see the source/sinks/decorators etc. How
> we can verify here whether our custom source is fine & up or have any
> errors?****
>
> ** **
>
> It would be much help if you please guide me a bit.****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Shumin Wu [mailto:[EMAIL PROTECTED]]
> *Sent:* Tuesday, September 18, 2012 2:11 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Moving from Flume OG to Flume NG****
>
> ** **
>
> Hi Deepark,
>
> I had hard time switching from OG to NG in the beginning, too. The new way
> in NG to make your custom source would look like this:
>
> public class CustomSource extends AbstractSource implements
> PollableSource, Configurable
>
> or
>
> public class CustomSource extends AbstractSource implements
> EventDrivenSource, Configurable
>
> , depending on your source type.
>
> Brock's link is a good reading resource.
>
> Shumin****
>
> On Tue, Sep 18, 2012 at 11:05 AM, Brock Noland <[EMAIL PROTECTED]> wrote:
> ****
>
> Hi,
>
> The flume developer guide as these details:
>
> http://flume.apache.org/FlumeDeveloperGuide.html
>
> Brock****
>
>
> On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> > I am trying to switch from Flume 0.94 to Flume NG. In previous version
> we do have the static class EventSource.Base through which we can create
> our own Source or Sink. I don't find anything as such here in flume NG.
> >
> > Could you please guide me how do we create our own Sources here in Flume
> NG?
> >
> > Regards,
> > Deepak
>
>
> ****
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/****
>
> ** **
>
sources provided by flume-ng. So yes, I am using the custom source, in a
way that is pretty much like the code snippet I have pasted in last email.
So far I have found the flume log a good resource to scrub error/exception
informations. You may configure the log4j property file to dump any many
details as you can.
AFAIK, there is no master/node notion in NG any longer. If you want to set
up a distributed cluster, you may take a look at the "setting multi-agent
flow" at
https://people.apache.org/~mpercy/flume/flume-1.2.0-incubating-SNAPSHOT/docs/FlumeUserGuide.html
Good luck!
Shumin
On Wed, Oct 3, 2012 at 3:42 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>wrote:
> Hi Shumin,****
>
> I have gone through the documentation, but it don’t have much. Do you have
> any use case in which you are using CustomSource? As in flume 0.9.4, we do
> have flume master url where we can see the source/sinks/decorators etc. How
> we can verify here whether our custom source is fine & up or have any
> errors?****
>
> ** **
>
> It would be much help if you please guide me a bit.****
>
> ** **
>
> Regards,****
>
> Deepak****
>
> ** **
>
> *From:* Shumin Wu [mailto:[EMAIL PROTECTED]]
> *Sent:* Tuesday, September 18, 2012 2:11 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Moving from Flume OG to Flume NG****
>
> ** **
>
> Hi Deepark,
>
> I had hard time switching from OG to NG in the beginning, too. The new way
> in NG to make your custom source would look like this:
>
> public class CustomSource extends AbstractSource implements
> PollableSource, Configurable
>
> or
>
> public class CustomSource extends AbstractSource implements
> EventDrivenSource, Configurable
>
> , depending on your source type.
>
> Brock's link is a good reading resource.
>
> Shumin****
>
> On Tue, Sep 18, 2012 at 11:05 AM, Brock Noland <[EMAIL PROTECTED]> wrote:
> ****
>
> Hi,
>
> The flume developer guide as these details:
>
> http://flume.apache.org/FlumeDeveloperGuide.html
>
> Brock****
>
>
> On Tue, Sep 18, 2012 at 1:02 PM, Kumar, Deepak8 <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> > I am trying to switch from Flume 0.94 to Flume NG. In previous version
> we do have the static class EventSource.Base through which we can create
> our own Source or Sink. I don't find anything as such here in flume NG.
> >
> > Could you please guide me how do we create our own Sources here in Flume
> NG?
> >
> > Regards,
> > Deepak
>
>
> ****
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/****
>
> ** **
>
+
Shumin Wu 2012-10-03, 14:10
-
Moving from Flume OG to Flume NGKumar, Deepak8 2012-10-05, 11:19
Hi Adam,
I am just trying to use hbase sink with flume. It is a sort of Hello World. My configuration file looks something like this. When I execute the node for hbase-agent, the log shows following error:
2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test from HBase
It seems, it looks for hbase at localhost, but the hbase is running on another VM. How should I provide the hbase VM details here? For testing purpose, I have created a hbase-site.xml at location flume/lib/ but it doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is 1.2.
Configuration File:
hbase-agent.sources = tail
hbase-agent.sinks = sink1
hbase-agent.channels = ch1
hbase-agent.sources.tail.type = exec
hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
hbase-agent.sources.tail.channels = ch1
hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
hbase-agent.sinks.sink1.channel = ch1
hbase-agent.sinks.sink1.table = test
hbase-agent.sinks.sink1.columnFamily = f1
hbase-agent.sinks.sink1.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
hbase-agent.sinks.sink1.serializer.payloadColumn = col1
hbase-agent.sinks.sink1.serializer.keyType = timestamp
hbase-agent.sinks.sink1.serializer.rowPrefix = 1
hbase-agent.sinks.sink1.serializer.suffix = timestamp
hbase-agent.channels.ch1.type=memory
Regards,
Deepak
I am just trying to use hbase sink with flume. It is a sort of Hello World. My configuration file looks something like this. When I execute the node for hbase-agent, the log shows following error:
2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test from HBase
It seems, it looks for hbase at localhost, but the hbase is running on another VM. How should I provide the hbase VM details here? For testing purpose, I have created a hbase-site.xml at location flume/lib/ but it doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is 1.2.
Configuration File:
hbase-agent.sources = tail
hbase-agent.sinks = sink1
hbase-agent.channels = ch1
hbase-agent.sources.tail.type = exec
hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
hbase-agent.sources.tail.channels = ch1
hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
hbase-agent.sinks.sink1.channel = ch1
hbase-agent.sinks.sink1.table = test
hbase-agent.sinks.sink1.columnFamily = f1
hbase-agent.sinks.sink1.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
hbase-agent.sinks.sink1.serializer.payloadColumn = col1
hbase-agent.sinks.sink1.serializer.keyType = timestamp
hbase-agent.sinks.sink1.serializer.rowPrefix = 1
hbase-agent.sinks.sink1.serializer.suffix = timestamp
hbase-agent.channels.ch1.type=memory
Regards,
Deepak
+
Kumar, Deepak8 2012-10-05, 11:19
-
Re: Moving from Flume OG to Flume NGBrock Noland 2012-10-05, 14:35
Hi,
When start the flume agent it prints out the command it is running to
start the agent. Can you send that along?
Brock
On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Hi Adam,
>
> I am just trying to use hbase sink with flume. It is a sort of Hello World.
> My configuration file looks something like this. When I execute the node for
> hbase-agent, the log shows following error:
>
>
>
> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
> from HBase
>
>
>
> It seems, it looks for hbase at localhost, but the hbase is running on
> another VM. How should I provide the hbase VM details here? For testing
> purpose, I have created a hbase-site.xml at location flume/lib/ but it
> doesn’t makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is
> 1.2.
>
>
>
>
>
> Configuration File:
>
>
>
> hbase-agent.sources = tail
>
> hbase-agent.sinks = sink1
>
> hbase-agent.channels = ch1
>
> hbase-agent.sources.tail.type = exec
>
> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
>
> hbase-agent.sources.tail.channels = ch1
>
> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
> hbase-agent.sinks.sink1.channel = ch1
>
> hbase-agent.sinks.sink1.table = test
>
> hbase-agent.sinks.sink1.columnFamily = f1
>
> hbase-agent.sinks.sink1.serializer > org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>
> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>
> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>
> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>
> hbase-agent.channels.ch1.type=memory
>
>
>
> Regards,
>
> Deepak
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
When start the flume agent it prints out the command it is running to
start the agent. Can you send that along?
Brock
On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Hi Adam,
>
> I am just trying to use hbase sink with flume. It is a sort of Hello World.
> My configuration file looks something like this. When I execute the node for
> hbase-agent, the log shows following error:
>
>
>
> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
> from HBase
>
>
>
> It seems, it looks for hbase at localhost, but the hbase is running on
> another VM. How should I provide the hbase VM details here? For testing
> purpose, I have created a hbase-site.xml at location flume/lib/ but it
> doesn’t makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is
> 1.2.
>
>
>
>
>
> Configuration File:
>
>
>
> hbase-agent.sources = tail
>
> hbase-agent.sinks = sink1
>
> hbase-agent.channels = ch1
>
> hbase-agent.sources.tail.type = exec
>
> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
>
> hbase-agent.sources.tail.channels = ch1
>
> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
> hbase-agent.sinks.sink1.channel = ch1
>
> hbase-agent.sinks.sink1.table = test
>
> hbase-agent.sinks.sink1.columnFamily = f1
>
> hbase-agent.sinks.sink1.serializer > org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>
> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>
> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>
> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>
> hbase-agent.channels.ch1.type=memory
>
>
>
> Regards,
>
> Deepak
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Brock Noland 2012-10-05, 14:35
-
RE: Moving from Flume OG to Flume NGKumar, Deepak8 2012-10-05, 15:08
Broke, this is the command it prints.
[root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f ./conf/deepakConf/flume.conf -name hbase-agent
+ exec /opt/sponge/java//bin/java -Xmx20m -cp '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -f ./conf/deepakConf/flume.conf -name hbase-agent
Regards,
Deepak
-----Original Message-----
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Friday, October 05, 2012 10:35 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Moving from Flume OG to Flume NG
Hi,
When start the flume agent it prints out the command it is running to
start the agent. Can you send that along?
Brock
On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Hi Adam,
>
> I am just trying to use hbase sink with flume. It is a sort of Hello World.
> My configuration file looks something like this. When I execute the node for
> hbase-agent, the log shows following error:
>
>
>
> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
> from HBase
>
>
>
> It seems, it looks for hbase at localhost, but the hbase is running on
> another VM. How should I provide the hbase VM details here? For testing
> purpose, I have created a hbase-site.xml at location flume/lib/ but it
> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is
> 1.2.
>
>
>
>
>
> Configuration File:
>
>
>
> hbase-agent.sources = tail
>
> hbase-agent.sinks = sink1
>
> hbase-agent.channels = ch1
>
> hbase-agent.sources.tail.type = exec
>
> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
>
> hbase-agent.sources.tail.channels = ch1
>
> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
> hbase-agent.sinks.sink1.channel = ch1
>
> hbase-agent.sinks.sink1.table = test
>
> hbase-agent.sinks.sink1.columnFamily = f1
>
> hbase-agent.sinks.sink1.serializer > org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>
> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>
> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>
> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>
> hbase-agent.channels.ch1.type=memory
>
>
>
> Regards,
>
> Deepak
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
[root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f ./conf/deepakConf/flume.conf -name hbase-agent
+ exec /opt/sponge/java//bin/java -Xmx20m -cp '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -f ./conf/deepakConf/flume.conf -name hbase-agent
Regards,
Deepak
-----Original Message-----
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Friday, October 05, 2012 10:35 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Moving from Flume OG to Flume NG
Hi,
When start the flume agent it prints out the command it is running to
start the agent. Can you send that along?
Brock
On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Hi Adam,
>
> I am just trying to use hbase sink with flume. It is a sort of Hello World.
> My configuration file looks something like this. When I execute the node for
> hbase-agent, the log shows following error:
>
>
>
> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
> from HBase
>
>
>
> It seems, it looks for hbase at localhost, but the hbase is running on
> another VM. How should I provide the hbase VM details here? For testing
> purpose, I have created a hbase-site.xml at location flume/lib/ but it
> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is
> 1.2.
>
>
>
>
>
> Configuration File:
>
>
>
> hbase-agent.sources = tail
>
> hbase-agent.sinks = sink1
>
> hbase-agent.channels = ch1
>
> hbase-agent.sources.tail.type = exec
>
> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
>
> hbase-agent.sources.tail.channels = ch1
>
> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
> hbase-agent.sinks.sink1.channel = ch1
>
> hbase-agent.sinks.sink1.table = test
>
> hbase-agent.sinks.sink1.columnFamily = f1
>
> hbase-agent.sinks.sink1.serializer > org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>
> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>
> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>
> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>
> hbase-agent.channels.ch1.type=memory
>
>
>
> Regards,
>
> Deepak
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Kumar, Deepak8 2012-10-05, 15:08
-
Re: Moving from Flume OG to Flume NGBrock Noland 2012-10-05, 15:17
Does either of
'/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
contain the hbase jars? Generally you want the hbase command to be in
the path of the user starting flume. In the hbase conf folder would be
the hbase-site.xml which would get picked up by the flume agent.
Brock
On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Broke, this is the command it prints.
> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f ./conf/deepakConf/flume.conf -name hbase-agent
> + exec /opt/sponge/java//bin/java -Xmx20m -cp '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -f ./conf/deepakConf/flume.conf -name hbase-agent
>
> Regards,
> Deepak
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Friday, October 05, 2012 10:35 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
> Hi,
>
> When start the flume agent it prints out the command it is running to
> start the agent. Can you send that along?
>
> Brock
>
> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
>> Hi Adam,
>>
>> I am just trying to use hbase sink with flume. It is a sort of Hello World.
>> My configuration file looks something like this. When I execute the node for
>> hbase-agent, the log shows following error:
>>
>>
>>
>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>> from HBase
>>
>>
>>
>> It seems, it looks for hbase at localhost, but the hbase is running on
>> another VM. How should I provide the hbase VM details here? For testing
>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is
>> 1.2.
>>
>>
>>
>>
>>
>> Configuration File:
>>
>>
>>
>> hbase-agent.sources = tail
>>
>> hbase-agent.sinks = sink1
>>
>> hbase-agent.channels = ch1
>>
>> hbase-agent.sources.tail.type = exec
>>
>> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
>>
>> hbase-agent.sources.tail.channels = ch1
>>
>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>>
>> hbase-agent.sinks.sink1.channel = ch1
>>
>> hbase-agent.sinks.sink1.table = test
>>
>> hbase-agent.sinks.sink1.columnFamily = f1
>>
>> hbase-agent.sinks.sink1.serializer >> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>>
>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>>
>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>>
>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>>
>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>>
>> hbase-agent.channels.ch1.type=memory
>>
>>
>>
>> Regards,
>>
>> Deepak
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
'/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
contain the hbase jars? Generally you want the hbase command to be in
the path of the user starting flume. In the hbase conf folder would be
the hbase-site.xml which would get picked up by the flume agent.
Brock
On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Broke, this is the command it prints.
> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f ./conf/deepakConf/flume.conf -name hbase-agent
> + exec /opt/sponge/java//bin/java -Xmx20m -cp '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -f ./conf/deepakConf/flume.conf -name hbase-agent
>
> Regards,
> Deepak
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Friday, October 05, 2012 10:35 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
> Hi,
>
> When start the flume agent it prints out the command it is running to
> start the agent. Can you send that along?
>
> Brock
>
> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
>> Hi Adam,
>>
>> I am just trying to use hbase sink with flume. It is a sort of Hello World.
>> My configuration file looks something like this. When I execute the node for
>> hbase-agent, the log shows following error:
>>
>>
>>
>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>> from HBase
>>
>>
>>
>> It seems, it looks for hbase at localhost, but the hbase is running on
>> another VM. How should I provide the hbase VM details here? For testing
>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is
>> 1.2.
>>
>>
>>
>>
>>
>> Configuration File:
>>
>>
>>
>> hbase-agent.sources = tail
>>
>> hbase-agent.sinks = sink1
>>
>> hbase-agent.channels = ch1
>>
>> hbase-agent.sources.tail.type = exec
>>
>> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
>>
>> hbase-agent.sources.tail.channels = ch1
>>
>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>>
>> hbase-agent.sinks.sink1.channel = ch1
>>
>> hbase-agent.sinks.sink1.table = test
>>
>> hbase-agent.sinks.sink1.columnFamily = f1
>>
>> hbase-agent.sinks.sink1.serializer >> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>>
>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>>
>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>>
>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>>
>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>>
>> hbase-agent.channels.ch1.type=memory
>>
>>
>>
>> Regards,
>>
>> Deepak
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Brock Noland 2012-10-05, 15:17
-
RE: Moving from Flume OG to Flume NGKumar, Deepak8 2012-10-05, 15:28
Brock,
Apologize miss spelled your name last time. Hbase.jar as well as hbase-site.xml is there in /opt/sponge/fluming/flume/lib directory. Could you please guide me, how to provide hbase command while starting flume?
Regards,
Deepak
-----Original Message-----
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Friday, October 05, 2012 11:17 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Moving from Flume OG to Flume NG
Does either of
'/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
contain the hbase jars? Generally you want the hbase command to be in
the path of the user starting flume. In the hbase conf folder would be
the hbase-site.xml which would get picked up by the flume agent.
Brock
On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Broke, this is the command it prints.
> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f ./conf/deepakConf/flume.conf -name hbase-agent
> + exec /opt/sponge/java//bin/java -Xmx20m -cp '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -f ./conf/deepakConf/flume.conf -name hbase-agent
>
> Regards,
> Deepak
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Friday, October 05, 2012 10:35 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
> Hi,
>
> When start the flume agent it prints out the command it is running to
> start the agent. Can you send that along?
>
> Brock
>
> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
>> Hi Adam,
>>
>> I am just trying to use hbase sink with flume. It is a sort of Hello World.
>> My configuration file looks something like this. When I execute the node for
>> hbase-agent, the log shows following error:
>>
>>
>>
>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>> from HBase
>>
>>
>>
>> It seems, it looks for hbase at localhost, but the hbase is running on
>> another VM. How should I provide the hbase VM details here? For testing
>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is
>> 1.2.
>>
>>
>>
>>
>>
>> Configuration File:
>>
>>
>>
>> hbase-agent.sources = tail
>>
>> hbase-agent.sinks = sink1
>>
>> hbase-agent.channels = ch1
>>
>> hbase-agent.sources.tail.type = exec
>>
>> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
>>
>> hbase-agent.sources.tail.channels = ch1
>>
>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>>
>> hbase-agent.sinks.sink1.channel = ch1
>>
>> hbase-agent.sinks.sink1.table = test
>>
>> hbase-agent.sinks.sink1.columnFamily = f1
>>
>> hbase-agent.sinks.sink1.serializer
>> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>>
>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>>
>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>>
>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>>
>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>>
>> hbase-agent.channels.ch1.type=memory
>>
>>
>>
>> Regards,
>>
>> Deepak
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Apologize miss spelled your name last time. Hbase.jar as well as hbase-site.xml is there in /opt/sponge/fluming/flume/lib directory. Could you please guide me, how to provide hbase command while starting flume?
Regards,
Deepak
-----Original Message-----
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Friday, October 05, 2012 11:17 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Moving from Flume OG to Flume NG
Does either of
'/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
contain the hbase jars? Generally you want the hbase command to be in
the path of the user starting flume. In the hbase conf folder would be
the hbase-site.xml which would get picked up by the flume agent.
Brock
On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Broke, this is the command it prints.
> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f ./conf/deepakConf/flume.conf -name hbase-agent
> + exec /opt/sponge/java//bin/java -Xmx20m -cp '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -f ./conf/deepakConf/flume.conf -name hbase-agent
>
> Regards,
> Deepak
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Friday, October 05, 2012 10:35 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
> Hi,
>
> When start the flume agent it prints out the command it is running to
> start the agent. Can you send that along?
>
> Brock
>
> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
>> Hi Adam,
>>
>> I am just trying to use hbase sink with flume. It is a sort of Hello World.
>> My configuration file looks something like this. When I execute the node for
>> hbase-agent, the log shows following error:
>>
>>
>>
>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>> from HBase
>>
>>
>>
>> It seems, it looks for hbase at localhost, but the hbase is running on
>> another VM. How should I provide the hbase VM details here? For testing
>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is
>> 1.2.
>>
>>
>>
>>
>>
>> Configuration File:
>>
>>
>>
>> hbase-agent.sources = tail
>>
>> hbase-agent.sinks = sink1
>>
>> hbase-agent.channels = ch1
>>
>> hbase-agent.sources.tail.type = exec
>>
>> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
>>
>> hbase-agent.sources.tail.channels = ch1
>>
>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>>
>> hbase-agent.sinks.sink1.channel = ch1
>>
>> hbase-agent.sinks.sink1.table = test
>>
>> hbase-agent.sinks.sink1.columnFamily = f1
>>
>> hbase-agent.sinks.sink1.serializer
>> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>>
>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>>
>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>>
>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>>
>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>>
>> hbase-agent.channels.ch1.type=memory
>>
>>
>>
>> Regards,
>>
>> Deepak
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Kumar, Deepak8 2012-10-05, 15:28
-
Re: Moving from Flume OG to Flume NGBrock Noland 2012-10-08, 16:02
Deepak,
What you have should work. Could you post your full log file on
pastebin.com or send it to me privately? (List doesn't take it)
Brock
On Fri, Oct 5, 2012 at 10:28 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Brock,
>
> Apologize miss spelled your name last time. Hbase.jar as well as
> hbase-site.xml is there in /opt/sponge/fluming/flume/lib directory. Could
> you please guide me, how to provide hbase command while starting flume?
>
> Regards,
>
> Deepak
>
>
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Friday, October 05, 2012 11:17 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
>
>
> Does either of
>
>
>
> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>
>
>
> contain the hbase jars? Generally you want the hbase command to be in
>
> the path of the user starting flume. In the hbase conf folder would be
>
> the hbase-site.xml which would get picked up by the flume agent.
>
>
>
> Brock
>
>
>
> On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
> wrote:
>
>> Broke, this is the command it prints.
>
>> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f
>> ./conf/deepakConf/flume.conf -name hbase-agent
>
>> + exec /opt/sponge/java//bin/java -Xmx20m -cp
>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>> -Djava.library.path= org.apache.flume.node.Application -f
>> ./conf/deepakConf/flume.conf -name hbase-agent
>
>>
>
>> Regards,
>
>> Deepak
>
>>
>
>> -----Original Message-----
>
>> From: Brock Noland [mailto:[EMAIL PROTECTED]]
>
>> Sent: Friday, October 05, 2012 10:35 AM
>
>> To: [EMAIL PROTECTED]
>
>> Cc: [EMAIL PROTECTED]
>
>> Subject: Re: Moving from Flume OG to Flume NG
>
>>
>
>> Hi,
>
>>
>
>> When start the flume agent it prints out the command it is running to
>
>> start the agent. Can you send that along?
>
>>
>
>> Brock
>
>>
>
>> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
>> wrote:
>
>>> Hi Adam,
>
>>>
>
>>> I am just trying to use hbase sink with flume. It is a sort of Hello
>>> World.
>
>>> My configuration file looks something like this. When I execute the node
>>> for
>
>>> hbase-agent, the log shows following error:
>
>>>
>
>>>
>
>>>
>
>>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>
>>> from HBase
>
>>>
>
>>>
>
>>>
>
>>> It seems, it looks for hbase at localhost, but the hbase is running on
>
>>> another VM. How should I provide the hbase VM details here? For testing
>
>>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>
>>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume
>>> is
>
>>> 1.2.
>
>>>
>
>>>
>
>>>
>
>>>
>
>>>
>
>>> Configuration File:
>
>>>
>
>>>
>
>>>
>
>>> hbase-agent.sources = tail
>
>>>
>
>>> hbase-agent.sinks = sink1
>
>>>
>
>>> hbase-agent.channels = ch1
>
>>>
>
>>> hbase-agent.sources.tail.type = exec
>
>>>
>
>>> hbase-agent.sources.tail.command = tail -F
>>> /opt/sponge/flumeNG/jongkook.txt
>
>>>
>
>>> hbase-agent.sources.tail.channels = ch1
>
>>>
>
>>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
>>>
>
>>> hbase-agent.sinks.sink1.channel = ch1
>
>>>
>
>>> hbase-agent.sinks.sink1.table = test
>
>>>
>
>>> hbase-agent.sinks.sink1.columnFamily = f1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer >
>>> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>
>>>
>
>>> hbase-agent.channels.ch1.type=memory
>
>>>
>
>>>
>
>>>
>
>>> Regards,
>
>>>
>
>>> Deepak
>
>>
>
>>
>
>>
>
>> --
>
>> Apache MRUnit - Unit testing MapReduce -
>> http://incubator.apache.org/mrunit/
>
>
>
>
>
>
>
> --
>
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
What you have should work. Could you post your full log file on
pastebin.com or send it to me privately? (List doesn't take it)
Brock
On Fri, Oct 5, 2012 at 10:28 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Brock,
>
> Apologize miss spelled your name last time. Hbase.jar as well as
> hbase-site.xml is there in /opt/sponge/fluming/flume/lib directory. Could
> you please guide me, how to provide hbase command while starting flume?
>
> Regards,
>
> Deepak
>
>
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Friday, October 05, 2012 11:17 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
>
>
> Does either of
>
>
>
> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>
>
>
> contain the hbase jars? Generally you want the hbase command to be in
>
> the path of the user starting flume. In the hbase conf folder would be
>
> the hbase-site.xml which would get picked up by the flume agent.
>
>
>
> Brock
>
>
>
> On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
> wrote:
>
>> Broke, this is the command it prints.
>
>> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f
>> ./conf/deepakConf/flume.conf -name hbase-agent
>
>> + exec /opt/sponge/java//bin/java -Xmx20m -cp
>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>> -Djava.library.path= org.apache.flume.node.Application -f
>> ./conf/deepakConf/flume.conf -name hbase-agent
>
>>
>
>> Regards,
>
>> Deepak
>
>>
>
>> -----Original Message-----
>
>> From: Brock Noland [mailto:[EMAIL PROTECTED]]
>
>> Sent: Friday, October 05, 2012 10:35 AM
>
>> To: [EMAIL PROTECTED]
>
>> Cc: [EMAIL PROTECTED]
>
>> Subject: Re: Moving from Flume OG to Flume NG
>
>>
>
>> Hi,
>
>>
>
>> When start the flume agent it prints out the command it is running to
>
>> start the agent. Can you send that along?
>
>>
>
>> Brock
>
>>
>
>> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
>> wrote:
>
>>> Hi Adam,
>
>>>
>
>>> I am just trying to use hbase sink with flume. It is a sort of Hello
>>> World.
>
>>> My configuration file looks something like this. When I execute the node
>>> for
>
>>> hbase-agent, the log shows following error:
>
>>>
>
>>>
>
>>>
>
>>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>
>>> from HBase
>
>>>
>
>>>
>
>>>
>
>>> It seems, it looks for hbase at localhost, but the hbase is running on
>
>>> another VM. How should I provide the hbase VM details here? For testing
>
>>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>
>>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume
>>> is
>
>>> 1.2.
>
>>>
>
>>>
>
>>>
>
>>>
>
>>>
>
>>> Configuration File:
>
>>>
>
>>>
>
>>>
>
>>> hbase-agent.sources = tail
>
>>>
>
>>> hbase-agent.sinks = sink1
>
>>>
>
>>> hbase-agent.channels = ch1
>
>>>
>
>>> hbase-agent.sources.tail.type = exec
>
>>>
>
>>> hbase-agent.sources.tail.command = tail -F
>>> /opt/sponge/flumeNG/jongkook.txt
>
>>>
>
>>> hbase-agent.sources.tail.channels = ch1
>
>>>
>
>>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
>>>
>
>>> hbase-agent.sinks.sink1.channel = ch1
>
>>>
>
>>> hbase-agent.sinks.sink1.table = test
>
>>>
>
>>> hbase-agent.sinks.sink1.columnFamily = f1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer >
>>> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>
>>>
>
>>> hbase-agent.channels.ch1.type=memory
>
>>>
>
>>>
>
>>>
>
>>> Regards,
>
>>>
>
>>> Deepak
>
>>
>
>>
>
>>
>
>> --
>
>> Apache MRUnit - Unit testing MapReduce -
>> http://incubator.apache.org/mrunit/
>
>
>
>
>
>
>
> --
>
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Brock Noland 2012-10-08, 16:02
-
RE: Moving from Flume OG to Flume NGKumar, Deepak8 2012-10-09, 06:03
Brock,
I copied hbase-site.xml in conf & is working fine now.
Thanks,
Deepak
-----Original Message-----
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Monday, October 08, 2012 12:03 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Moving from Flume OG to Flume NG
Deepak,
What you have should work. Could you post your full log file on
pastebin.com or send it to me privately? (List doesn't take it)
Brock
On Fri, Oct 5, 2012 at 10:28 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Brock,
>
> Apologize miss spelled your name last time. Hbase.jar as well as
> hbase-site.xml is there in /opt/sponge/fluming/flume/lib directory. Could
> you please guide me, how to provide hbase command while starting flume?
>
> Regards,
>
> Deepak
>
>
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Friday, October 05, 2012 11:17 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
>
>
> Does either of
>
>
>
> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>
>
>
> contain the hbase jars? Generally you want the hbase command to be in
>
> the path of the user starting flume. In the hbase conf folder would be
>
> the hbase-site.xml which would get picked up by the flume agent.
>
>
>
> Brock
>
>
>
> On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
> wrote:
>
>> Broke, this is the command it prints.
>
>> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f
>> ./conf/deepakConf/flume.conf -name hbase-agent
>
>> + exec /opt/sponge/java//bin/java -Xmx20m -cp
>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>> -Djava.library.path= org.apache.flume.node.Application -f
>> ./conf/deepakConf/flume.conf -name hbase-agent
>
>>
>
>> Regards,
>
>> Deepak
>
>>
>
>> -----Original Message-----
>
>> From: Brock Noland [mailto:[EMAIL PROTECTED]]
>
>> Sent: Friday, October 05, 2012 10:35 AM
>
>> To: [EMAIL PROTECTED]
>
>> Cc: [EMAIL PROTECTED]
>
>> Subject: Re: Moving from Flume OG to Flume NG
>
>>
>
>> Hi,
>
>>
>
>> When start the flume agent it prints out the command it is running to
>
>> start the agent. Can you send that along?
>
>>
>
>> Brock
>
>>
>
>> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
>> wrote:
>
>>> Hi Adam,
>
>>>
>
>>> I am just trying to use hbase sink with flume. It is a sort of Hello
>>> World.
>
>>> My configuration file looks something like this. When I execute the node
>>> for
>
>>> hbase-agent, the log shows following error:
>
>>>
>
>>>
>
>>>
>
>>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>
>>> from HBase
>
>>>
>
>>>
>
>>>
>
>>> It seems, it looks for hbase at localhost, but the hbase is running on
>
>>> another VM. How should I provide the hbase VM details here? For testing
>
>>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>
>>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume
>>> is
>
>>> 1.2.
>
>>>
>
>>>
>
>>>
>
>>>
>
>>>
>
>>> Configuration File:
>
>>>
>
>>>
>
>>>
>
>>> hbase-agent.sources = tail
>
>>>
>
>>> hbase-agent.sinks = sink1
>
>>>
>
>>> hbase-agent.channels = ch1
>
>>>
>
>>> hbase-agent.sources.tail.type = exec
>
>>>
>
>>> hbase-agent.sources.tail.command = tail -F
>>> /opt/sponge/flumeNG/jongkook.txt
>
>>>
>
>>> hbase-agent.sources.tail.channels = ch1
>
>>>
>
>>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
>>>
>
>>> hbase-agent.sinks.sink1.channel = ch1
>
>>>
>
>>> hbase-agent.sinks.sink1.table = test
>
>>>
>
>>> hbase-agent.sinks.sink1.columnFamily = f1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer >
>>> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
I copied hbase-site.xml in conf & is working fine now.
Thanks,
Deepak
-----Original Message-----
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Monday, October 08, 2012 12:03 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Moving from Flume OG to Flume NG
Deepak,
What you have should work. Could you post your full log file on
pastebin.com or send it to me privately? (List doesn't take it)
Brock
On Fri, Oct 5, 2012 at 10:28 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Brock,
>
> Apologize miss spelled your name last time. Hbase.jar as well as
> hbase-site.xml is there in /opt/sponge/fluming/flume/lib directory. Could
> you please guide me, how to provide hbase command while starting flume?
>
> Regards,
>
> Deepak
>
>
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Friday, October 05, 2012 11:17 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
>
>
> Does either of
>
>
>
> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>
>
>
> contain the hbase jars? Generally you want the hbase command to be in
>
> the path of the user starting flume. In the hbase conf folder would be
>
> the hbase-site.xml which would get picked up by the flume agent.
>
>
>
> Brock
>
>
>
> On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
> wrote:
>
>> Broke, this is the command it prints.
>
>> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f
>> ./conf/deepakConf/flume.conf -name hbase-agent
>
>> + exec /opt/sponge/java//bin/java -Xmx20m -cp
>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>> -Djava.library.path= org.apache.flume.node.Application -f
>> ./conf/deepakConf/flume.conf -name hbase-agent
>
>>
>
>> Regards,
>
>> Deepak
>
>>
>
>> -----Original Message-----
>
>> From: Brock Noland [mailto:[EMAIL PROTECTED]]
>
>> Sent: Friday, October 05, 2012 10:35 AM
>
>> To: [EMAIL PROTECTED]
>
>> Cc: [EMAIL PROTECTED]
>
>> Subject: Re: Moving from Flume OG to Flume NG
>
>>
>
>> Hi,
>
>>
>
>> When start the flume agent it prints out the command it is running to
>
>> start the agent. Can you send that along?
>
>>
>
>> Brock
>
>>
>
>> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
>> wrote:
>
>>> Hi Adam,
>
>>>
>
>>> I am just trying to use hbase sink with flume. It is a sort of Hello
>>> World.
>
>>> My configuration file looks something like this. When I execute the node
>>> for
>
>>> hbase-agent, the log shows following error:
>
>>>
>
>>>
>
>>>
>
>>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>
>>> from HBase
>
>>>
>
>>>
>
>>>
>
>>> It seems, it looks for hbase at localhost, but the hbase is running on
>
>>> another VM. How should I provide the hbase VM details here? For testing
>
>>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>
>>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume
>>> is
>
>>> 1.2.
>
>>>
>
>>>
>
>>>
>
>>>
>
>>>
>
>>> Configuration File:
>
>>>
>
>>>
>
>>>
>
>>> hbase-agent.sources = tail
>
>>>
>
>>> hbase-agent.sinks = sink1
>
>>>
>
>>> hbase-agent.channels = ch1
>
>>>
>
>>> hbase-agent.sources.tail.type = exec
>
>>>
>
>>> hbase-agent.sources.tail.command = tail -F
>>> /opt/sponge/flumeNG/jongkook.txt
>
>>>
>
>>> hbase-agent.sources.tail.channels = ch1
>
>>>
>
>>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>
>>>
>
>>> hbase-agent.sinks.sink1.channel = ch1
>
>>>
>
>>> hbase-agent.sinks.sink1.table = test
>
>>>
>
>>> hbase-agent.sinks.sink1.columnFamily = f1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer >
>>> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>
>>>
>
>>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Kumar, Deepak8 2012-10-09, 06:03
-
Re: Moving from Flume OG to Flume NGBrock Noland 2012-10-09, 14:51
Great to hear, that's interesting it did not work in the lib directory.
On Tue, Oct 9, 2012 at 1:03 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Brock,
> I copied hbase-site.xml in conf & is working fine now.
>
> Thanks,
>
> Deepak
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Monday, October 08, 2012 12:03 PM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
> Deepak,
>
> What you have should work. Could you post your full log file on
> pastebin.com or send it to me privately? (List doesn't take it)
>
> Brock
>
> On Fri, Oct 5, 2012 at 10:28 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
>> Brock,
>>
>> Apologize miss spelled your name last time. Hbase.jar as well as
>> hbase-site.xml is there in /opt/sponge/fluming/flume/lib directory. Could
>> you please guide me, how to provide hbase command while starting flume?
>>
>> Regards,
>>
>> Deepak
>>
>>
>>
>> -----Original Message-----
>> From: Brock Noland [mailto:[EMAIL PROTECTED]]
>> Sent: Friday, October 05, 2012 11:17 AM
>> To: [EMAIL PROTECTED]
>> Cc: [EMAIL PROTECTED]
>> Subject: Re: Moving from Flume OG to Flume NG
>>
>>
>>
>> Does either of
>>
>>
>>
>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>>
>>
>>
>> contain the hbase jars? Generally you want the hbase command to be in
>>
>> the path of the user starting flume. In the hbase conf folder would be
>>
>> the hbase-site.xml which would get picked up by the flume agent.
>>
>>
>>
>> Brock
>>
>>
>>
>> On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
>> wrote:
>>
>>> Broke, this is the command it prints.
>>
>>> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f
>>> ./conf/deepakConf/flume.conf -name hbase-agent
>>
>>> + exec /opt/sponge/java//bin/java -Xmx20m -cp
>>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>>> -Djava.library.path= org.apache.flume.node.Application -f
>>> ./conf/deepakConf/flume.conf -name hbase-agent
>>
>>>
>>
>>> Regards,
>>
>>> Deepak
>>
>>>
>>
>>> -----Original Message-----
>>
>>> From: Brock Noland [mailto:[EMAIL PROTECTED]]
>>
>>> Sent: Friday, October 05, 2012 10:35 AM
>>
>>> To: [EMAIL PROTECTED]
>>
>>> Cc: [EMAIL PROTECTED]
>>
>>> Subject: Re: Moving from Flume OG to Flume NG
>>
>>>
>>
>>> Hi,
>>
>>>
>>
>>> When start the flume agent it prints out the command it is running to
>>
>>> start the agent. Can you send that along?
>>
>>>
>>
>>> Brock
>>
>>>
>>
>>> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
>>> wrote:
>>
>>>> Hi Adam,
>>
>>>>
>>
>>>> I am just trying to use hbase sink with flume. It is a sort of Hello
>>>> World.
>>
>>>> My configuration file looks something like this. When I execute the node
>>>> for
>>
>>>> hbase-agent, the log shows following error:
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>>
>>>> from HBase
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> It seems, it looks for hbase at localhost, but the hbase is running on
>>
>>>> another VM. How should I provide the hbase VM details here? For testing
>>
>>>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>>
>>>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume
>>>> is
>>
>>>> 1.2.
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> Configuration File:
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> hbase-agent.sources = tail
>>
>>>>
>>
>>>> hbase-agent.sinks = sink1
>>
>>>>
>>
>>>> hbase-agent.channels = ch1
>>
>>>>
>>
>>>> hbase-agent.sources.tail.type = exec
>>
>>>>
>>
>>>> hbase-agent.sources.tail.command = tail -F
>>>> /opt/sponge/flumeNG/jongkook.txt
>>
>>>>
>>
>>>> hbase-agent.sources.tail.channels = ch1
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.channel = ch1
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.table = test
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
On Tue, Oct 9, 2012 at 1:03 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Brock,
> I copied hbase-site.xml in conf & is working fine now.
>
> Thanks,
>
> Deepak
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Monday, October 08, 2012 12:03 PM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Moving from Flume OG to Flume NG
>
> Deepak,
>
> What you have should work. Could you post your full log file on
> pastebin.com or send it to me privately? (List doesn't take it)
>
> Brock
>
> On Fri, Oct 5, 2012 at 10:28 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
>> Brock,
>>
>> Apologize miss spelled your name last time. Hbase.jar as well as
>> hbase-site.xml is there in /opt/sponge/fluming/flume/lib directory. Could
>> you please guide me, how to provide hbase command while starting flume?
>>
>> Regards,
>>
>> Deepak
>>
>>
>>
>> -----Original Message-----
>> From: Brock Noland [mailto:[EMAIL PROTECTED]]
>> Sent: Friday, October 05, 2012 11:17 AM
>> To: [EMAIL PROTECTED]
>> Cc: [EMAIL PROTECTED]
>> Subject: Re: Moving from Flume OG to Flume NG
>>
>>
>>
>> Does either of
>>
>>
>>
>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>>
>>
>>
>> contain the hbase jars? Generally you want the hbase command to be in
>>
>> the path of the user starting flume. In the hbase conf folder would be
>>
>> the hbase-site.xml which would get picked up by the flume agent.
>>
>>
>>
>> Brock
>>
>>
>>
>> On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
>> wrote:
>>
>>> Broke, this is the command it prints.
>>
>>> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f
>>> ./conf/deepakConf/flume.conf -name hbase-agent
>>
>>> + exec /opt/sponge/java//bin/java -Xmx20m -cp
>>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>>> -Djava.library.path= org.apache.flume.node.Application -f
>>> ./conf/deepakConf/flume.conf -name hbase-agent
>>
>>>
>>
>>> Regards,
>>
>>> Deepak
>>
>>>
>>
>>> -----Original Message-----
>>
>>> From: Brock Noland [mailto:[EMAIL PROTECTED]]
>>
>>> Sent: Friday, October 05, 2012 10:35 AM
>>
>>> To: [EMAIL PROTECTED]
>>
>>> Cc: [EMAIL PROTECTED]
>>
>>> Subject: Re: Moving from Flume OG to Flume NG
>>
>>>
>>
>>> Hi,
>>
>>>
>>
>>> When start the flume agent it prints out the command it is running to
>>
>>> start the agent. Can you send that along?
>>
>>>
>>
>>> Brock
>>
>>>
>>
>>> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <[EMAIL PROTECTED]>
>>> wrote:
>>
>>>> Hi Adam,
>>
>>>>
>>
>>>> I am just trying to use hbase sink with flume. It is a sort of Hello
>>>> World.
>>
>>>> My configuration file looks something like this. When I execute the node
>>>> for
>>
>>>> hbase-agent, the log shows following error:
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>>
>>>> from HBase
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> It seems, it looks for hbase at localhost, but the hbase is running on
>>
>>>> another VM. How should I provide the hbase VM details here? For testing
>>
>>>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>>
>>>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume
>>>> is
>>
>>>> 1.2.
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> Configuration File:
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> hbase-agent.sources = tail
>>
>>>>
>>
>>>> hbase-agent.sinks = sink1
>>
>>>>
>>
>>>> hbase-agent.channels = ch1
>>
>>>>
>>
>>>> hbase-agent.sources.tail.type = exec
>>
>>>>
>>
>>>> hbase-agent.sources.tail.command = tail -F
>>>> /opt/sponge/flumeNG/jongkook.txt
>>
>>>>
>>
>>>> hbase-agent.sources.tail.channels = ch1
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.channel = ch1
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.table = test
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Brock Noland 2012-10-09, 14:51
