Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Mysql Load Data Infile with Hadoop?


Copy link to this message
-
RE: Mysql Load Data Infile with Hadoop?
You might also try using something like Fuse-dfs (http://wiki.apache.org/hadoop/MountableHDFS) to "mount" the HDFS file system on the mysql machine.  You could then use a standard unix path to specify the loadfile.

Marc

-----Original Message-----
From: tim robertson [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, May 19, 2009 1:03 PM
To: [EMAIL PROTECTED]
Subject: Re: Mysql Load Data Infile with Hadoop?

So you are using a java program to execute a "load data infile"
command on mysql through JDBC?
If so I *think* you would have to copy it onto the mysql machine from
HDFS first, or the machine running the command and then try a 'load
data local infile'.

Or pehaps use the
http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/lib/db/DBOutputFormat.html

Cheers

Tim

On Tue, May 19, 2009 at 9:54 PM, dealmaker <[EMAIL PROTECTED]> wrote:
>
> Hi,
>  I am want to load data in mysql using a hadoop file similar to following:
> LOAD DATA INFILE 'test.txt' INTO TABLE test
>  FIELDS TERMINATED BY ','  LINES STARTING BY 'xxx';
>
> But how do I load the hdfs file into the mysql comand above?  Do I start the
> file name with hdfd://test.txt?  I am using java.
> Thanks.
>
> --
> View this message in context: http://www.nabble.com/Mysql-Load-Data-Infile-with-Hadoop--tp23623256p23623256.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>

PRIVATE AND CONFIDENTIAL - NOTICE TO RECIPIENT: THIS E-MAIL IS MEANT FOR ONLY THE INTENDED RECIPIENT OF THE TRANSMISSION, AND MAY BE A COMMUNICATION PRIVILEGE BY LAW. IF YOU RECEIVED THIS E-MAIL IN ERROR, ANY REVIEW, USE, DISSEMINATION, DISTRIBUTION, OR COPYING OF THIS EMAIL IS STRICTLY PROHIBITED. PLEASE NOTIFY US IMMEDIATELY OF THE ERROR BY RETURN E-MAIL AND PLEASE DELETE THIS MESSAGE FROM YOUR SYSTEM.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB