Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive 0.11.0 | Issue with ORC Tables


Copy link to this message
-
Re: Hive 0.11.0 | Issue with ORC Tables
Keshav,

Owen has provided the solution already. Thats the easiest of the the lot
and from the master who wrote ORC himself :)

to put it in simple words what he has suggested is,

create a staging table which will be based on default text data format.
>From the staging data load data into a ORC file backed table.

you can refer Owen's mail for the respective queries.

On Fri, Sep 20, 2013 at 4:46 PM, Savant, Keshav <
[EMAIL PROTECTED]> wrote:

>  Hi Nitin,****
>
> ** **
>
> Thanks for your reply,  we were in an impression that the codec will be
> responsible for ORC format conversion also.****
>
> However as per your reply it seems that a conversion from normal CSV to
> ORC is required before hive upload.****
>
> ** **
>
> We got some leads from following URLs****
>
> https://cwiki.apache.org/Hive/languagemanual-orc.html****
>
> http://www.math.uic.edu/t3m/SnapPy/installing.html****
>
> ** **
>
> Please suggest how it can be done using some already available libraries,
> or we need to write our own converter.****
>
> ** **
>
> Kind Regards,****
>
> Keshav****
>
> ** **
>
> *From:* Nitin Pawar [mailto:[EMAIL PROTECTED]]
> *Sent:* Thursday, September 19, 2013 5:56 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Hive 0.11.0 | Issue with ORC Tables****
>
> ** **
>
> How did you create "test.txt" as ORC file? ****
>
> ** **
>
> ** **
>
> On Thu, Sep 19, 2013 at 5:34 PM, Savant, Keshav <
> [EMAIL PROTECTED]> wrote:****
>
> Hi All,****
>
>  ****
>
> We have setup apache “hive 0.11.0” services on Hadoop cluster (apache
> version 0.20.203.0). Hive is showing expected results when tables are
> stored as *TextFile*. ****
>
> Whereas, Hive 0.11.0’s new feature ORC(*Optimized Row Columnar*) is
> throwing an exception while running a select query, when we run select
> queries on tables stored as “*ORC*”.****
>
> Stacktrace of the exception :****
>
>  ****
>
> 2013-09-19 20:33:38,095 ERROR CliDriver
> (SessionState.java:printError(386)) - Failed with exception
> java.io.IOException:com.google.protobuf.InvalidProtocolBufferException:
> While parsing a protocol message, the input ended unexpectedly in the
> middle of a field.  This could mean either than the input has been
> truncated or that an embedded message misreported its own length.****
>
> java.io.IOException: com.google.protobuf.InvalidProtocolBufferException:
> While parsing a protocol message, the input ended unexpectedly in the
> middle of a field.  This could mean either than the input has been
> truncated or that an embedded message misreported its own length.****
>
>         at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:544)
> ****
>
>         at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:488)
> ****
>
>         at
> org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)****
>
>         at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1412)**
> **
>
>         at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)**
> **
>
>         at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)****
>
>         at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)****
>
>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)***
> *
>
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)**
> **
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> ****
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> ****
>
>         at java.lang.reflect.Method.invoke(Method.java:597)****
>
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)****
>
> Caused by: com.google.protobuf.InvalidProtocolBufferException: While
> parsing a protocol message, the input ended unexpectedly in the middle of a
> field.  This could mean either than the input has been truncated or that an

Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB