Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Date format - any easier way


Copy link to this message
-
Re: Date format - any easier way
I knocked up the following when we were experimenting with Hive. I've been
meaning to go and tidy it up for a while, but using it with a separator of
"" (empty string) should have the desired effect. (Obviously the UDF throws
an exception if the array is empty, been meaning to fix that for a while...)

Cheers,

Phil.

import java.util.List;

import org.apache.hadoop.hive.ql.exec.Description;
import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDF;
import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
import
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector.Category;
import
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector;
import
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;

@Description(name = "implode", value = "_FUNC_(list,separator) - joins the
elements of list together, separated by the given string."
+ " Returns a string. [Add an example here]")
public class GenericUDFImplode extends GenericUDF {
private ListObjectInspector listOI = null;
private PrimitiveObjectInspector glueOI = null;

@Override
public Object evaluate(DeferredObject[] params) throws HiveException {
return join(listOI.getList(params[0].get()),
glueOI.getPrimitiveJavaObject(params[1].get()).toString());
}

private String join(List<?> list, String separator) {
if (list == null) {
return null;
}
if (list.size() == 0) {
return "";
}

StringBuffer buf = new StringBuffer();

for (Object o : list) {
buf.append(o);
buf.append(separator);
}

return buf.substring(0, buf.length() - separator.length());
}

@Override
public String getDisplayString(String[] args) {
return "implode(" + args[0] + "," + args[1] + ")";
}

@Override
public ObjectInspector initialize(ObjectInspector[] params)
throws UDFArgumentException {
if (params[0].getCategory() != Category.LIST) {
throw new UDFArgumentException("Expected: List as argument 1 to implode()");
}
if (params[1].getCategory() != Category.PRIMITIVE) {
throw new UDFArgumentException("Expected: Primitive as argument 2 to
implode()");
}

listOI = (ListObjectInspector) params[0];
glueOI = (PrimitiveObjectInspector) params[1];

return PrimitiveObjectInspectorFactory.javaStringObjectInspector;
}

}
On 15 May 2012 15:33, Nitin Pawar <[EMAIL PROTECTED]> wrote:

> I will write an UDF for array concatenation and upload on GIT if anyone
> does not have it already
>
>
> On Tue, May 15, 2012 at 7:24 PM, Zoltán Tóth-Czifra <
> [EMAIL PROTECTED]> wrote:
>
>>  Matt, thanks!
>>
>>  Luckily the order of the parts of the date is correct (reordering them
>> would bet he same craziness).
>>
>>  Finally it is:
>>
>>  regexp_replace(
>> date_sub(
>> to_date(
>> from_unixtime(
>> unix_timestamp()
>> )
>> ), 1
>> ), "[-]", ""
>> )
>>
>>  Nitin, concat apparently doesn't take arrays, and I did not find any
>> other way to join arrays in HQL. However, it would be very handy.
>>
>>  Thanks guys!
>>
>>  ------------------------------
>> *From:* Tucker, Matt [[EMAIL PROTECTED]]
>> *Sent:* Tuesday, May 15, 2012 3:33 PM
>>
>> *To:* [EMAIL PROTECTED]
>> *Subject:* RE: Date format - any easier way
>>
>>   What about wrapping it in regexp_replace(…, “[-]”, “”) ?  It may not
>> be the cleanest, but I’d recommend passing variables from the shell :)
>>
>>
>>
>> Matt Tucker
>>
>>
>>
>> *From:* Zoltán Tóth-Czifra [mailto:[EMAIL PROTECTED]]
>> *Sent:* Tuesday, May 15, 2012 9:27 AM
>> *To:* [EMAIL PROTECTED]
>> *Subject:* RE: Date format - any easier way
>>
>>
>>
>> Nitin,
>>
>>
>>
>> Thank you. As you see below I know and use this function. My problem is
>> that it doesn't give YYYYMMDD format, but YYYY-MM-DD instead, and
>> formatting is not trivial as you can see it too.
>>
>>
>>  ------------------------------
>>
>> *From:* Nitin Pawar [[EMAIL PROTECTED]]
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB