Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> RE: How to extract data from MySQL using Flume


+
shouvanik.haldar@... 2013-06-24, 11:16
Copy link to this message
-
Re: How to extract data from MySQL using Flume
Flume is a collection/aggregation framework in broader prospect .

So if you write an application which reads from your database and dumps
into a file time to time or streaming, then flume will read them and store
in wherever you want them to be stored.

Where as sqoop will just import your dataset from most of teh RDBMS and
puts in hdfs. and reverse is true as well
On Mon, Jun 24, 2013 at 4:46 PM, <[EMAIL PROTECTED]> wrote:

>  Hi,****
>
> ** **
>
> If I am not wrong, we cannot use flume to extract data from Oracle also.
> Am I correct?****
>
> ** **
>
> Thanks,****
>
> Shouvanik****
>
> ** **
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Monday, June 24, 2013 4:19 PM
> *To:* Haldar, Shouvanik
> *Subject:* Re: How to extract data from MySQL using Flume****
>
> ** **
>
> Flume is not the tool to extract data from mysql****
>
> ** **
>
> use sqoop ****
>
> ** **
>
> On Mon, Jun 24, 2013 at 3:07 PM, <[EMAIL PROTECTED]> wrote:**
> **
>
> Hi,****
>
>  ****
>
> Does anybody have idea how to extract data from MYSQL/Oracle using Flume?*
> ***
>
>  ****
>
> Regards,****
>
> Shouvanik****
>
>  ****
>
> *From:* nagesh [mailto:[EMAIL PROTECTED] <[EMAIL PROTECTED]>]
> *Sent:* Monday, June 24, 2013 11:44 AM
> *To:* [EMAIL PROTECTED]; [EMAIL PROTECTED]
> *Subject:* Re: Flume Async HBase Sink not working****
>
>  ****
>
>  ****
>
> Hi,
> I created new table in the HBase and I inserted rows and cols successfully
> using normal HBasesink. When I changed HBasesink to AsyncHBase sink no rows
> or cols inserting. My requirement is I have 100's of text files which are
> generated by some process. I want to read those text files and insert into
> according tables. For example If that files contains both customer and
> product details. I need to insert customer details to customer table and
> product details to product table. I am new to Flume, could you please give
> any suggestion to solve this problem. I want to do all these in JAVA
> technology.
>
> Thanks
> Nagesh Thati.
>
> On Thursday 20 June 2013 11:44 PM, Sravya Tirukkovalur wrote:****
>
>  Hi Nagesh,  ****
>
>  ****
>
> Thanks for attaching the log.****
>
>  ****
>
> Are there missing rows in the table or are there no rows at all? If it is
> the former, I think I see a reason. Your row keys are "1+timestamp", so if
> multiple rows are written in the same time stamp, the last one would
> overwrite previous ones. ****
>
>  ****
>
> Cheers,****
>
> Sravya****
>
>  ****
>
>  ****
>
> On Wed, Jun 19, 2013 at 9:34 PM, nagesh <[EMAIL PROTECTED]> wrote:**
> **
>
> Hi,
> Thanks for the reply. I attached my conf file to this mail.
> I am using Flume latest version *flume-1.3.1*, and HBase 0.94.1. ****
>
>
>
>
> On Tuesday 18 June 2013 09:57 PM, iain wright wrote:****
>
> Can you please paste your config? What version of flume and hbase? ****
>
> On Jun 18, 2013 1:24 AM, "nagesh" <[EMAIL PROTECTED]> wrote:****
>
>
> Hi,
> I am new to flume. I have tried HBaseSink it is working and hbase puts
> also working. But, when I changed it to Async HBase Sink flume not
> putting any info. to HBase. I am not getting any warnings or error while
> using Async Hbase Sink. But, still puts not working.
> Can any one please help to get out of this issue.****
>
>  ****
>
> -- ****
>
> Thanks****
>
> Nagesh Thati****
>
>
>
> ****
>
>  ****
>
> -- ****
>
> Sravya Tirukkovalur****
>
> ** **
>
> -- ****
>
> Thanks****
>
> Nagesh Thati****
>
>  ****
>
>  ****
>
> ** **
>  ------------------------------
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited.
>
> Where allowed by local law, electronic communications with Accenture and
> its affiliates, including e-mail and instant messaging (including content),
> may be scanned by our systems for the purposes of information security and

Nitin Pawar
+
Srinivasan Ramalingam 2013-06-24, 11:29
+
shouvanik.haldar@... 2013-06-24, 11:39
+
shouvanik.haldar@... 2013-06-24, 09:37