Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Migrating to hive 8.1 on EMR


Copy link to this message
-
Re: Migrating to hive 8.1 on EMR
Sorry, missed the link:
https://forums.aws.amazon.com/thread.jspa?threadID=95988&tstart=25

----- Original Message -----
From: "Mark Grover" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, June 19, 2012 1:31:25 PM
Subject: Re: Migrating to hive 8.1 on EMR

Rajan,
On a somewhat related note, I had issues when using Amazon's MySQL server (on the master node) as metastore on a Hive 0.8.1 cluster.
Related thread:
----- Original Message -----
From: "Carl Steinbach" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Monday, June 18, 2012 5:07:12 PM
Subject: Re: Migrating to hive 8.1 on EMR

Hi Ranjan,
Looks like the NPE is getting generated here:

if (0 == getCols().size()) {
throw new HiveException(
"at least one column must be specified for the table");
}
Which would seem to indicate that the table record which was fetched doesn't have any columns.
Did you run the metastore upgrade scripts for 0.8.0 (located in metastore/scripts/upgrade)? Hive 0.8.0 includes HIVE-2246 which changed the name and schema of the metastore table used to store column information.
Thanks.
Carl

On Mon, Jun 18, 2012 at 12:56 PM, Ranjan Bagchi < [EMAIL PROTECTED] > wrote:
Hi,

I've built a datastore using Hive 7.1 backed by S3 using persistent metadata. Now that hive 8.1 is available, I'd like to migrate to the new version.

However, I'm having trouble reading tables with the persistent schema. Looking in the logs, I'm getting stack traces like the following:
2012-06-18 19:51:23,176 ERROR org.apache.hadoop.hive.ql.exec.Task (SessionState.
java:printError(380)) - Failed with exception null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162
)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:946)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:860)
at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:252
7)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:298)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.ja
va:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:261)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:218)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:409)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:567)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

2012-06-18 19:51:23,180 ERROR org.apache.hadoop.hive.ql.Driver (SessionState.java:printError(380)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

Does anyone have experience with this?

Thanks,

Ranjan