Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Dynamically generating load/store path


Copy link to this message
-
Re: Dynamically generating load/store path
You could write a custom load/storefunc  and override setLocation(String
location, Job job). The logic you want the UDF for could go in there.
On Tue, Nov 13, 2012 at 4:45 PM, Jonathan Coveney <[EMAIL PROTECTED]>wrote:

> If it's a parameter, it could just be passed in as a $var
>
>
> 2012/11/13 Miki Tebeka <[EMAIL PROTECTED]>
>
> > Greetings,
> >
> > Is there a way to dynamically generate (maybe via UDF) the path to
> > load/store data? (something like "A = LOAD InputPath() USING
> > PigStorage();")
> >
> > Currently we calculate the load/store path outside the pig script and
> pass
> > it as parameter. However we'd like to have a UDF that does that.
> >
> > Thanks,
> > --
> > Miki
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB