Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Issue while quering Hive


Copy link to this message
-
Re: Issue while quering Hive
Nitin Pawar 2013-09-16, 12:07
As per my understanding, hadoop 1.x does not provide you any help on
processing compressing files in parallel manner. (Atleast this was the case
few months back).

This bzip2 splitting etc is added in hadoop2.x as per my understanding.
On Mon, Sep 16, 2013 at 5:18 PM, Garg, Rinku <[EMAIL PROTECTED]>wrote:

>  Thanks Nitin,****
>
> ** **
>
> That way it worked, But in that case Hadoop will not be able to split my
> file into chunks/blocks and run multiple maps in parallel. This can cause
> under-utilization of my cluster's 'mapping' power. Is that rue??****
>
> ** **
>
> Thanks & Regards,****
>
> *Rinku Garg*
>
> ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* 16 September 2013 15:57
>
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Issue while quering Hive****
>
> ** **
>
> Does your .gz file contains the data in sequencefile ? or its a plain csv?
> ****
>
> ** **
>
> I think looking at the filename its a plain csv file, so I would recommend
> that you create a normal table with TextInputFormat (the default) and load
> data in the new table and give it a try. ****
>
> ** **
>
> ** **
>
> On Mon, Sep 16, 2013 at 3:36 PM, Garg, Rinku <[EMAIL PROTECTED]>
> wrote:****
>
> Hi Nitin,****
>
>  ****
>
> Yes, I created the table with sequencefile.****
>
>  ****
>
> Thanks & Regards,****
>
> *Rinku Garg*****
>
>  ****
>
>  ****
>
>  ****
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* 16 September 2013 14:19
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Issue while quering Hive****
>
>  ****
>
> Look at the error message ****
>
>  ****
>
> Caused by: java.io.IOException:
> hdfs://localhost:54310/user/hive/warehouse/cpj_tbl/cpj.csv.gz not a
> SequenceFile****
>
>  ****
>
> Did you create table with sequencefile ? ****
>
>  ****
>
> On Mon, Sep 16, 2013 at 1:33 PM, Garg, Rinku <[EMAIL PROTECTED]>
> wrote:****
>
> Hi All,****
>
>  ****
>
> I have setup Hadoop, hive setup and trying to load gzip file in hadoop
> cluster. Files are loaded successfully and can be view on web UI. While
> executing Select query it gives me the below mentioned error.****
>
>  ****
>
> ERROR org.apache.hadoop.security.UserGroupInformation:
> PriviledgedActionException as:nxtbig (auth:SIMPLE)
> cause:java.io.IOException: java.lang.reflect.InvocationTargetException****
>
> 2013-09-16 09:11:18,971 WARN org.apache.hadoop.mapred.Child: Error running
> child****
>
> java.io.IOException: java.lang.reflect.InvocationTargetException****
>
>         at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> ****
>
>         at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> ****
>
>         at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:369)
> ****
>
>         at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:316)
> ****
>
>         at
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:430)
> ****
>
>         at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:540)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:395)
> ****
>
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)****
>
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:268)****
>
>         at java.security.AccessController.doPrivileged(Native Method)****
>
>         at javax.security.auth.Subject.doAs(Subject.java:396)****
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1407)
> ****
>
>         at org.apache.hadoop.mapred.Child.main(Child.java:262)****
>
> Caused by: java.lang.reflect.InvocationTargetException****
>
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native

Nitin Pawar