Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Question on custom store function


+
felix gao 2011-11-02, 01:07
Copy link to this message
-
Re: Question on custom store function
Hey Felix,

>> The only problem is that in the setStoreLocation function we have to call
>> FileOutputFormat.setOutputPath(job, new Path(location));

Cant you massage location to appropriate string you want to?

Ashutosh

On Tue, Nov 1, 2011 at 18:07, felix gao <[EMAIL PROTECTED]> wrote:

> I have wrote a custom store function that primarily based on the
> multi-storage store function.  They way I use it is
>
>
> store load_log INTO
> '/Users/felix/Documents/pig/multi_store_output/ns_{0}/site_{1}' using
> MyMultiStorage('2,1', '1,2');
> where {0} and {1} will be substituted with the tuple index at 0 and index
> at 1.  Everything is fine and all the data is written to the correct place.
>  The only problem is that in the setStoreLocation function we have to call
> FileOutputFormat.setOutputPath(job, new Path(location)); i have
> 'Users/felix/Documents/pig/multi_store_output/ns_{0}/site_{1}' as my output
> location so there is actually a folder created in my fs with ns_{0}
> and site_{1}.  Is there a way to tell hadoop not to create those output
> directory?
>
> Thanks,
>
> Felix
>
+
felix gao 2011-11-02, 21:10
+
Ashutosh Chauhan 2011-11-02, 22:00
+
felix gao 2011-11-02, 22:19
+
Dmitriy Ryaboy 2011-11-03, 16:45
+
Raghu Angadi 2011-11-04, 22:31
+
felix gao 2011-11-08, 18:41
+
Raghu Angadi 2011-11-08, 19:57
+
felix gao 2011-11-08, 20:32