Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - flume-cassandra


Copy link to this message
-
Re: flume-cassandra
Alexander Alten-Lorenz 2013-01-29, 07:31
Hi,

As the error message says, you haven't a valid log4j config. Start your flume-ng agent with the additional option "--conf" and point this into the directory which have a log4j.properties file

-Alex

On Jan 29, 2013, at 6:15 AM, Sri Ramya <[EMAIL PROTECTED]> wrote:

> HI
>
> i am unable to configure flume-ng, i am geting a warning
>
> + exec /home/sriramya/project/jdk1.7.0_07/bin/java -Xmx20m
> -Dflume.root.logger=DEBUG,console -cp
> './conf/:/home/sriramya/Desktop/apache-flume-1.3.1-bin/lib/*'
> -Djava.library.path= org.apache.flume.node.Application -f conf/flume.conf
> -n agent1
> log4j:WARN No appenders could be found for logger
> (org.apache.flume.node.Application).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
>
>
> and its get terminated.  Please tell me how to start
>
>
> On Tue, Jan 29, 2013 at 9:24 AM, Sri Ramya <[EMAIL PROTECTED]> wrote:
>
>> hi Ralph
>>
>> I am able to integrate flume with Cassandra. But the problem is ' if i
>> send a single event to Cassandra to store, it is storing multiple row in
>> Cassnadra. Thats why i thought of shifting to logsandra. But its very
>> difficult to to use that logsandra plug in. Any way by your suggestionsNow
>> i am staring work with Flume-ng.
>>
>> Make me clear in one thing, Is there any separation of storing data in
>> Cassandra based on flume-ng agent(IP address)???
>>
>> Thank you
>>
>>
>> On Mon, Jan 28, 2013 at 10:26 PM, Ralph Goers <[EMAIL PROTECTED]>wrote:
>>
>>> This is absolutely NOT correct.   We are using Flume NG to log to
>>> Cassandra.   We fought with Flume OG for months trying to get it to work
>>> but it was never stable.  The architecture of Flume OG had serious
>>> problems, which is why Flume NG was created.
>>>
>>> Ralph
>>>
>>>
>>>
>>> On Jan 23, 2013, at 9:46 PM, Priyanka Jain wrote:
>>>
>>> Hi ,
>>> Can you give details like your input format and all ...
>>> I think going with flume 0.9.4 for cassandra is better.
>>>
>>> On Thu, Jan 24, 2013 at 10:43 AM, Mike Percy <[EMAIL PROTECTED]> wrote:
>>>
>>>> What do you mean by "collector"?
>>>>
>>>>
>>>> On Wed, Jan 23, 2013 at 9:05 PM, Sri Ramya <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Thank you very much. But I need a collector in my application,
>>>>> flume-ng  does not have any collector. Thats why i started with Cloudeera
>>>>> flum. Now i am getting a problem with integrating flume collector with
>>>>> cassandra.  If any body tried it before please help me.
>>>>> thank in advance.
>>>>>
>>>>>
>>>>> On Thu, Jan 24, 2013 at 10:26 AM, Mike Percy <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> Hi Sri,
>>>>>> Cloudera originally created Flume, then contributed it to the Apache
>>>>>> Software Foundation (ASF), and continues to invest heavily into it under
>>>>>> the auspices of the ASF. The current generation of Flume is called Flume
>>>>>> NG. I encourage you to use the latest "NG" generation of Flume (version
>>>>>> 1.x) over the so-called "OG" generation (version 0.9.x).
>>>>>>
>>>>>> Regards,
>>>>>> Mike
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 23, 2013 at 7:30 PM, Sri Ramya <[EMAIL PROTECTED]>wrote:
>>>>>>
>>>>>>> Thank you very much.
>>>>>>> Can u tell me whats the difference between cloudera flume and
>>>>>>> flume-ng . And which is the better option to use.
>>>>>>> Thank you.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 24, 2013 at 7:23 AM, shekhar sharma <
>>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>>
>>>>>>>> You can use develop your own customized sink which can do
>>>>>>>> pre-processing in flume and then send the events to cassandra
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Som
>>>>>>>>
>>>>>>>> On Thu, Jan 24, 2013 at 12:22 AM, Camp, Roy <[EMAIL PROTECTED]> wrote:
>>>>>>>>
>>>>>>>>> You may also want to check out
>>>>>>>>>
>>>>>>>>> https://github.com/btoddb/flume-ng-cassandra-sink
>>>>>>>>>
>>>>>>>>> Roy
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -----Original Message-----

Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF