Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - GenericUDF Testing in Hive

Copy link to this message
GenericUDF Testing in Hive
Raj Hadoop 2014-01-20, 22:19
The following is a an example for a GenericUDF. I wanted to test this through a Hive query. Basically want to pass parameters some thing like "select ComplexUDFExample('a','b','c') from employees limit 10".

class ComplexUDFExample extends GenericUDF {
  ListObjectInspector listOI;
  StringObjectInspector elementOI;
  public String getDisplayString(String[] arg0) {
    return "arrayContainsExample()"; // this should probably be better
  public ObjectInspector initialize(ObjectInspector[] arguments) throws UDFArgumentException {
    if (arguments.length != 2) {
      throw new UDFArgumentLengthException("arrayContainsExample only takes 2 arguments: List<T>, T");
    // 1. Check we received the right object types.
    ObjectInspector a = arguments[0];
    ObjectInspector b = arguments[1];
    if (!(a instanceof ListObjectInspector) || !(b instanceof StringObjectInspector)) {
      throw new UDFArgumentException("first argument must be a list / array, second argument must be a string");
    this.listOI = (ListObjectInspector) a;
    this.elementOI = (StringObjectInspector) b;
    // 2. Check that the list contains strings
    if(!(listOI.getListElementObjectInspector() instanceof StringObjectInspector)) {
      throw new UDFArgumentException("first argument must be a list of strings");
    // the return type of our function is a boolean, so we provide the correct object inspector
    return PrimitiveObjectInspectorFactory.javaBooleanObjectInspector;
  public Object evaluate(DeferredObject[] arguments) throws HiveException {
    // get the list and string from the deferred objects using the object inspectors
    List<String> list = (List<String>) this.listOI.getList(arguments[0].get());
    String arg = elementOI.getPrimitiveJavaObject(arguments[1].get());
    // check for nulls
    if (list == null || arg == null) {
      return null;
    // see if our list contains the value we need
    for(String s: list) {
      if (arg.equals(s)) return new Boolean(true);
    return new Boolean(false);
hive> select ComplexUDFExample('a','b','c') from email_list_1 limit 10;
FAILED: SemanticException [Error 10015]: Line 1:7 Arguments length mismatch ''c'': arrayContainsExample only takes 2 arguments: List<T>, T
How to test this example in Hive query. I know I am invoking it wrong. But how can I invoke it correctly.
My requirement is to pass a String of arrays as first argument and another string as second argument in Hive like below.
Select col1, ComplexUDFExample( collectset(col2) , 'xyz')
Group By col1;
How do i do that?
Thanks in advance.