Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Sqoop and more than one partition key


+
Santosh Achhra 2013-01-04, 17:13
Copy link to this message
-
Re: Sqoop and more than one partition key
Hi Santosh,
I'm afraid that Sqoop is currently supporting only tables that are using maximally one column for partitioning.

I was playing around to see if there is a chance to at least partially get working multiple partitions. I was able to get it running when I've created the table manually prior Sqoop import and then run Sqoop with following arguments:

  --hive-partition-key "f1" --hive-partition-value "1', f2='1"

However I would not advise to use that in production because it's ugly hack that might stop working in future Sqoop versions.

Jarcec

On Sat, Jan 05, 2013 at 01:13:30AM +0800, Santosh Achhra wrote:
> Hello,
>
> I am trying to import and create hive table which has more than one
> partition key using below mentioned command however command
> fails giving below mentioned error. Is more than one partition key
> supported in Sqoop.
> 13/01/04 17:08:01 INFO hive.HiveImport: FAILED: ParseException line 1:403
> cannot recognize input near ',' '
>
> sqoop --options-file connect.parm --table TABLEA --columns  "f1, f2,f3"
> --split-by "f4"  --hive-import   -m 2 --hive-partition-key "f4,f5"
> --hive-partition-value "v1,v2"
>
>
> Good wishes,always !
> Santosh
+
Santosh Achhra 2013-01-06, 02:32
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB