Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - slow performance when using udf


Copy link to this message
-
slow performance when using udf
wd 2011-08-15, 08:49
hi,

I create a udf to decode urlencoded things, but found the speed for
mapred is 3 times(73sec -> 213 sec) as before. How to optimize it?

package com.test.hive.udf;

import org.apache.hadoop.hive.ql.exec.UDF;
import java.net.URLDecoder;

public final class urldecode extends UDF {

    public String evaluate(final String s) {
        if (s == null) { return null; }
        return getString(s);
    }

    public static String getString(String s) {
        String a;
        try {
            a = URLDecoder.decode(s);
        } catch ( Exception e) {
            a = "";
        }
        return a;
    }

    public static void main(String args[]) {
        String t = "%E5%A4%AA%E5%8E%9F-%E4%B8%89%E4%BA%9A";
        System.out.println( getString(t) );
    }
}