Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> casting complex data types for outputs of custom scripts


+
rohan monga 2014-01-13, 23:22
+
Andre Araujo 2014-01-14, 23:11
Copy link to this message
-
Re: casting complex data types for outputs of custom scripts
@OP - first thing i'd ask is does your python script obey the ^A,^B,^C,^D
etc. nesting delimiter pattern.  give that your create table does not
specify delimiters those are the defaults.  nb. ^A == control-A == \001

Cheers,
Stephen.
On Tue, Jan 14, 2014 at 3:11 PM, Andre Araujo <[EMAIL PROTECTED]> wrote:

> I had a similar issue in the past when trying to cast an empty array to
> array(<bigint>). By default Hive assumes it's an array(<string>).
> I don't think there's currently a Hive syntax to cast values to complex
> data types. If there's one, I'd love to know what it is :)
>
>
> On 14 January 2014 10:22, rohan monga <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> I have a table that is of the following format
>>
>> create table t1 ( f1 int, f2 array<struct<a1:int, a2:int>> );
>>
>> Now I have a custom script that does some computation and generates
>> the value for f2
>> like so
>>
>> from (
>>     from randomtable r
>>     map r.g1, r.g2, r.g3
>>     using '/bin/cat' as g1, g2, g3
>>     cluster by g1 ) m
>>     insert overwrite table t1
>>     reduce m.g1, m.g2, m.g3
>>     using 'python customScript.py' as ( f1 , f2 );
>>
>> however f2 is not being loaded properly into t1, it comes up broken or
>> null. What should I do so that f2 is loaded as an array of structs.
>>
>>
>> Thanks,
>>
>> --
>> Rohan Monga
>>
>
>
>
> --
> André Araújo
> Big Data Consultant/Solutions Architect
> The Pythian Group - Australia - www.pythian.com
>
> Office (calls from within Australia): 1300 366 021 x1270
> Office (international): +61 2 8016 7000  x270 *OR* +1 613 565 8696   x1270
> Mobile: +61 410 323 559
> Fax: +61 2 9805 0544
> IM: pythianaraujo @ AIM/MSN/Y! or [EMAIL PROTECTED] @ GTalk
>
> “Success is not about standing at the top, it's the steps you leave behind.”
> — Iker Pou (rock climber)
>
> --
>
>
>
>
+
rohan monga 2014-01-20, 00:38
+
Bogala, Chandra Reddy 2014-01-21, 05:15
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB