hey

I used flume-ng(1.3.0) to collecting webserver log, memory as channel, hdfs as sink, and workd good.
        But this time i come up against a series of problems because that i want use mongodb as a final storage , however  i can not find any ideas
what use mongodb or other nosql as a sink via google and official guide except a plugin named mongo-hadoop,anyone can help me? thanks very much !

--
lancexxx
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

Hi,

we haven't a mongo sink, so you have to write your own. But you could use a avro sink and import the contents into mongo via http://datasyndrome.com/post/27075009640/from-avro-to-mongo-in-one-line-with-avro-to-mongo-pig?

cheers,
 Alex

On Oct 23, 2012, at 9:58 AM, lancexxx <[EMAIL PROTECTED]> wrote:

>
> hey
>
> I used flume-ng(1.3.0) to collecting webserver log, memory as channel, hdfs as sink, and workd good.
>        But this time i come up against a series of problems because that i want use mongodb as a final storage , however  i can not find any ideas
> what use mongodb or other nosql as a sink via google and official guide except a plugin named mongo-hadoop,anyone can help me? thanks very much !
>
> --
> lancexxx
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

hi，I found a solution that use mongodb on githua,this is the url:
https://github.com/leonlee/flume-ng-mongodb-sink  

what do you think of it with the avro sink?
thanks!
--  
lancexxx
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On 2012年10月23日Tuesday at 下午4:40, Alexander Lorenz wrote:

> Hi,
>  
> we haven't a mongo sink, so you have to write your own. But you could use a avro sink and import the contents into mongo via http://datasyndrome.com/post/27075009640/from-avro-to-mongo-in-one-line-with-avro-to-mongo-pig?
>  
> cheers,
> Alex  
>  
> On Oct 23, 2012, at 9:58 AM, lancexxx <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
>  
> >  
> > hey
> >  
> > I used flume-ng(1.3.0) to collecting webserver log, memory as channel, hdfs as sink, and workd good.  
> > But this time i come up against a series of problems because that i want use mongodb as a final storage , however i can not find any ideas  
> > what use mongodb or other nosql as a sink via google and official guide except a plugin named mongo-hadoop,anyone can help me? thanks very much !
> >  
> > --  
> > lancexxx
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >  
>  
>  
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>  
>  

Yes, would work with FlumeNG. You've to change in the example the source into avro, and give them a try.

Cheers,
 Alex

On Oct 23, 2012, at 4:20 PM, lancexxx <[EMAIL PROTECTED]> wrote:

> hi，I found a solution that use mongodb on githua,this is the url:
> https://github.com/leonlee/flume-ng-mongodb-sink  
>
> what do you think of it with the avro sink?
> thanks!
> --  
> lancexxx
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>
>
> On 2012年10月23日Tuesday at 下午4:40, Alexander Lorenz wrote:
>
>> Hi,
>>
>> we haven't a mongo sink, so you have to write your own. But you could use a avro sink and import the contents into mongo via http://datasyndrome.com/post/27075009640/from-avro-to-mongo-in-one-line-with-avro-to-mongo-pig?
>>
>> cheers,
>> Alex  
>>
>> On Oct 23, 2012, at 9:58 AM, lancexxx <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
>>
>>>
>>> hey
>>>
>>> I used flume-ng(1.3.0) to collecting webserver log, memory as channel, hdfs as sink, and workd good.  
>>> But this time i come up against a series of problems because that i want use mongodb as a final storage , however i can not find any ideas  
>>> what use mongodb or other nosql as a sink via google and official guide except a plugin named mongo-hadoop,anyone can help me? thanks very much !
>>>
>>> --  
>>> lancexxx
>>> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>>>
>>
>>
>> --
>> Alexander Alten-Lorenz
>> http://mapredit.blogspot.com
>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>>
>
>

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

Here is a good sink for Flume-NG & Cassandra.

https://github.com/btoddb/flume-ng-cassandra-sink

From: lancexxx [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, October 23, 2012 12:58 AM
To: [EMAIL PROTECTED]
Subject: flume-ng use mongdb as sink
hey

            I used flume-ng(1.3.0) to collecting webserver log, memory as channel, hdfs as sink, and workd good.
        But this time i come up against a series of problems because that i want use mongodb as a final storage , however  i can not find any ideas
what use mongodb or other nosql as a sink via google and official guide except a plugin named mongo-hadoop,anyone can help me? thanks very much !
--
lancexxx
Sent with Sparrow<http://www.sparrowmailapp.com/?sig>