Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Mysql Load Data Infile with Hadoop?


Copy link to this message
-
RE: Mysql Load Data Infile with Hadoop?
You might also try using something like Fuse-dfs (http://wiki.apache.org/hadoop/MountableHDFS) to "mount" the HDFS file system on the mysql machine.  You could then use a standard unix path to specify the loadfile.

Marc

-----Original Message-----
From: tim robertson [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, May 19, 2009 1:03 PM
To: [EMAIL PROTECTED]
Subject: Re: Mysql Load Data Infile with Hadoop?

So you are using a java program to execute a "load data infile"
command on mysql through JDBC?
If so I *think* you would have to copy it onto the mysql machine from
HDFS first, or the machine running the command and then try a 'load
data local infile'.

Or pehaps use the
http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/lib/db/DBOutputFormat.html

Cheers

Tim

On Tue, May 19, 2009 at 9:54 PM, dealmaker <[EMAIL PROTECTED]> wrote:
>
> Hi,
>  I am want to load data in mysql using a hadoop file similar to following:
> LOAD DATA INFILE 'test.txt' INTO TABLE test
>  FIELDS TERMINATED BY ','  LINES STARTING BY 'xxx';
>
> But how do I load the hdfs file into the mysql comand above?  Do I start the
> file name with hdfd://test.txt?  I am using java.
> Thanks.
>
> --
> View this message in context: http://www.nabble.com/Mysql-Load-Data-Infile-with-Hadoop--tp23623256p23623256.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>

PRIVATE AND CONFIDENTIAL - NOTICE TO RECIPIENT: THIS E-MAIL IS MEANT FOR ONLY THE INTENDED RECIPIENT OF THE TRANSMISSION, AND MAY BE A COMMUNICATION PRIVILEGE BY LAW. IF YOU RECEIVED THIS E-MAIL IN ERROR, ANY REVIEW, USE, DISSEMINATION, DISTRIBUTION, OR COPYING OF THIS EMAIL IS STRICTLY PROHIBITED. PLEASE NOTIFY US IMMEDIATELY OF THE ERROR BY RETURN E-MAIL AND PLEASE DELETE THIS MESSAGE FROM YOUR SYSTEM.