Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> UDAF terminatePartial structure


Copy link to this message
-
Re: UDAF terminatePartial structure
I found this Cloudera example helpful:
http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/org.apache.hadoop.hive/hive-contrib/0.7.0-cdh3u0/org/apache/hadoop/hive/contrib/udaf/example/UDAFExampleMaxMinNUtil.java#UDAFExampleMaxMinNUtil.Evaluator

igor
decide.com

On Mon, Jul 29, 2013 at 4:32 PM, Ritesh Agrawal <[EMAIL PROTECTED]>wrote:

> Hi Robin,
>
> Thanks for the suggestion. I did find such an example in Hadoop The
> definitive guide book. However I am not total confused.
>
> The book extends UDAF instead of AbstractGenericUDAFResolver. Which one is
> recommended ?
>
> Also the example in the book uses DoubleWritable as a return type for the
> "terminate" function. However, I will be returning an arraylist of double.
> Do I always need to written objects that are derived from
> WritableComponents.
>
> Ritesh
> On Jul 29, 2013, at 4:15 PM, Robin Morris wrote:
>
> > I believe a map will be passed correctly from the terminatePartial to the
> > merge functions.  But it seems a bit of overkill.
> >
> > Why not define a class within your UDAF which has 4 public data members,
> > and return instances of that class from terminatePartial()?
> >
> > Robin
> >
> >
> > On 7/29/13 3:19 PM, "Ritesh Agrawal" <[EMAIL PROTECTED]> wrote:
> >
> >> Hi all,
> >>
> >> I am writing my first UDAF. In my terminatePartial() function, I need to
> >> store different data having different data types. Below is a list of
> >> items that I need to store
> >> 1. C1 : list of doubles
> >> 2. C2: list of doubles
> >> 3. C3: double
> >> 4. Show: list of strings
> >>
> >>
> >> I am wondering can I use simple HashMap and store these different
> objects
> >> into it. Will it automatically serialize or will I need to write my own
> >> serializiable method. Also is there any example of a UDAF that shows how
> >> to use map type structure for storing partial results.
> >>
> >> Thanks
> >>
> >> Ritesh
> >
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB