Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Does sqoop 1/2 support import as Parquet file?


Copy link to this message
-
Re: Does sqoop 1/2 support import as Parquet file?
"Cloudera connector Powered by Teradata" seems not to support HCatalog.

If using GenericJdbcManager, I could create hcatalog table using ORC, but
failed with this error to write into a parquet table:

2014-06-18 17:11:38,881 WARN [main]
org.apache.hadoop.mapred.YarnChild: Exception running child :
java.lang.RuntimeException: Should never be used
at org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getRecordWriter(MapredParquetOutputFormat.java:76)
at org.apache.hcatalog.mapreduce.FileOutputFormatContainer.getRecordWriter(FileOutputFormatContainer.java:109)
at org.apache.hcatalog.mapreduce.HCatOutputFormat.getRecordWriter(HCatOutputFormat.java:245)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:624)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:744)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

According to this post
https://groups.google.com/forum/#!topic/parquet-dev/xi28AoDJMJs, it won't
work until Hive 0.13.
On Wed, Jun 18, 2014 at 3:25 PM, Benyi Wang <[EMAIL PROTECTED]> wrote: