Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - store less files


Copy link to this message
-
store less files
Jameson Li 2011-04-01, 07:57
Hi,

When I run the below pig codes:
a = load '/logs/2011-03-31';
b = filter a by $1=='a' and $2=='b';
store b into '20110331-ab';

It runs a M/R that have thousands maps, and then create a output store
directory that have the same number so many files.

I have a doubt that how I could store less files when I use pig to store
files in the HDFS.
Thanks,
Jameson Li.