Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Question on custom store function


+
felix gao 2011-11-02, 01:07
Copy link to this message
-
Re: Question on custom store function
Hey Felix,

>> The only problem is that in the setStoreLocation function we have to call
>> FileOutputFormat.setOutputPath(job, new Path(location));

Cant you massage location to appropriate string you want to?

Ashutosh

On Tue, Nov 1, 2011 at 18:07, felix gao <[EMAIL PROTECTED]> wrote:

> I have wrote a custom store function that primarily based on the
> multi-storage store function.  They way I use it is
>
>
> store load_log INTO
> '/Users/felix/Documents/pig/multi_store_output/ns_{0}/site_{1}' using
> MyMultiStorage('2,1', '1,2');
> where {0} and {1} will be substituted with the tuple index at 0 and index
> at 1.  Everything is fine and all the data is written to the correct place.
>  The only problem is that in the setStoreLocation function we have to call
> FileOutputFormat.setOutputPath(job, new Path(location)); i have
> 'Users/felix/Documents/pig/multi_store_output/ns_{0}/site_{1}' as my output
> location so there is actually a folder created in my fs with ns_{0}
> and site_{1}.  Is there a way to tell hadoop not to create those output
> directory?
>
> Thanks,
>
> Felix
>
+
felix gao 2011-11-02, 21:10
+
Ashutosh Chauhan 2011-11-02, 22:00
+
felix gao 2011-11-02, 22:19
+
Dmitriy Ryaboy 2011-11-03, 16:45
+
Raghu Angadi 2011-11-04, 22:31
+
felix gao 2011-11-08, 18:41
+
Raghu Angadi 2011-11-08, 19:57
+
felix gao 2011-11-08, 20:32
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB