Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Re: Converting Array to a String


Copy link to this message
-
Re: Converting Array to a String
As currently implemented, the parameters passed to the reflect() UDF must
be primitives (and I'm guessing that "collectedSet" is a list).  I think
this is primarily due to the fact that Hive only supports a limited number
of complex types (specifically: list, map, struct, union) and these types
are abstracted in a way which makes it difficult to convert to concrete
Java types and vice versa (for a good discussion of the abstraction used to
represent complex types you may want to refer to the following post:
http://www.congiu.com/articles/json_serde).

Long story short, you'll probably want to write your own UDF or possibly
reuse the existing reflect() UDF to handle your specific case.

Matt

P.S. You can see the source code for the reflect() UDF on GitHub:
https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFReflect.java.
 The exception you are seeing is generated by the following block of code:

      if (arguments[i].getCategory() != ObjectInspector.Category.PRIMITIVE) {
        throw new UDFArgumentTypeException(i,
            "The parameters of GenericUDFReflect(class,method[,arg1[,arg2]...])"
            + " must be primitive (int, double, string, etc).");
      }
At a very high level, I think you would want to remove the for loop where
that exception is generated and instead have something like:

ListObjectInspector listOI = (ListObjectInspector) arguments[2];

Then replace the following lines:

    // Get the parameter values
    for (int i = 2; i < arguments.length; i++) {
      parameterJavaValues[i - 2] argumentOIs[i].getPrimitiveJavaObject(arguments[i].get());
    }
with something like:

Object test[] = new Object[listOI.getListLength(arguments[2].get())];
for (int i = 0; i < test.length; i++) {
  test[i] = listOI.getListElement(arguments[2].get(), i);
}
parameterJavaValues[i - 2] = test

I've left out a lot of details and probably missing some "gotchas," but
hopefully that helps you get the wheels turning...

On Thu, Nov 17, 2011 at 6:41 AM, Tucker, Matt <[EMAIL PROTECTED]>wrote:

> I’m running into the same issue, and I see that it’s addressed in
> HIVE-2223.****
>
> ** **
>
> In the meantime, I’m getting an error when trying to use the reflect()
> function :****
>
> SELECT reflect("org.apache.commons.lang.StringUtils", "join",
> collectedSet), …****
>
> ** **
>
> FAILED: Error in semantic analysis: Line 1:69 Argument type mismatch
> collectedSet: The parameters of
> GenericUDFReflect(class,method[,arg1[,arg2]...]) must be primitive (int,
> double, string, etc).****
>
> ** **
>
> Matt Tucker****
>
> Associate eBusiness Analyst****
>
> Walt Disney Parks and Resorts Online****
>
> Ph: 407-566-2545****
>
> Tie: 8-296-2545****
>
> ** **
>

--
Matt Martin
Think Big Analytics
[EMAIL PROTECTED]
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB