Sqoop, mail # user - RE: Complex free form queries - 2014-09-18, 20:24
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Plain View
Copy link to this message
-
RE: Complex free form queries
Pratik,

Are you facing a problem or trying to make a recommendation?
Regards,
Sambit.
From: pratik khadloya [mailto:[EMAIL PROTECTED]]
Sent: Thursday, September 18, 2014 1:09 PM
To: [EMAIL PROTECTED]
Subject: Complex free form queries

The sqoop docs say:

The facility of using free-form query in the current version of Sqoop is limited to simple queries where there are no ambiguous projections and no OR conditions in the WHERE clause. Use of complex queries such as queries that have sub-queries or joins leading to ambiguous projections can lead to unexpected results.

Does anyone know why such is case is not supported and can it be avoided by:

a) Using only 1 mapper
or
b) Creating a view out of the complex query

I have tested a hive textfile import for a very complex query and verified the data and it seems to be correct. I checked the number of words, number of lines and file sizes of the dump from mysql vs the text file imported onto hdfs by sqoop.
My query does have OR conditions. I have attached an obfuscated version of the query, and that screenprint is still 1/2 of the complete query.

Any info on this will be helpful.

Thanks,
Pratik

 
+
pratik khadloya 2014-09-19, 00:18
+
pratik khadloya 2014-09-18, 20:09
+
pratik khadloya 2014-09-18, 21:41
+
Venkat Ranganathan 2014-09-18, 22:44
+
pratik khadloya 2014-09-18, 23:13
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB