Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> hive json serde


Copy link to this message
-
RE: hive json serde
A few things that should help...

- There are three ways to handle JSON within a Hive table, get_json_object, json_tuple, and JSON SerDe. These are separate mechanisms. So if you want to use get_json_object, you don't need to load a SerDe.

- With a SerDe, Hive understands JSON fields natively, so you can just use the plain field names, no special function call at all.

- I'm not sure if auxpath accepts a relative path, but I know the full path works.

Chuck

________________________________
From: Mark Golden [[EMAIL PROTECTED]]
Sent: Monday, September 17, 2012 5:58 PM
To: Connell, Chuck
Cc: [EMAIL PROTECTED]
Subject: Re: hive json serde

It works now. Looks like there is a bug in the code.

if you do hive --auxpath ./serde then I get an error but if I get the full path as
hive --auxpath /var/lib/hdfs/serde/ then get_json_object() works.

Thanks for your help. Appreciate it.

CCing hive user group as well.
On Mon, Sep 17, 2012 at 2:54 PM, Mark Golden <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
:-) Thanks. I downloaded and got it compiled as well.

My table has nested JSON values so when I tried to do get_json_object(<column>,'$.<key>') this serde doesn't work in that case. It throws an error. I'll email the developer too.

When you tried did you try this on a cluster of nodes or on a single instance?
On Mon, Sep 17, 2012 at 2:38 PM, Connell, Chuck <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Go to the Downloads link on the project page, near the upper right.
________________________________
From: Mark Golden [[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>]
Sent: Monday, September 17, 2012 5:32 PM
To: Connell, Chuck
Subject: Re: hive json serde

Hey Chuck,

Where did you download the prebuild jar from?

On Mon, Sep 17, 2012 at 12:10 PM, Connell, Chuck <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
I used his pre-built jar. No need to compile anything.

Be sure to add “ --auxpath /path/to/jar/dir “ to the Hive command line.

Chuck

From: Connell, Chuck [mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>]
Sent: Monday, September 17, 2012 3:06 PM
To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
Subject: RE: hive json serde

I just finished testing this one. No problems found. The developer is also quite responsive to issues raised.  I encouraged him to submit it to the Hive dev team as core code.

https://github.com/rcongiu/Hive-JSON-Serde/

Chuck Connell
Nuance R&D Data Team
Burlington, MA
From: Mark Golden [mailto:[EMAIL PROTECTED]]
Sent: Monday, September 17, 2012 2:56 PM
To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
Subject: hive json serde

Hi,

I'm looking for a hive json serde that works well. Any suggestions? I did a quick google and found a few but none of them are complete.

Thanks,
Mark.

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB