Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - save several 64MB files in Pig Latin


+
Pedro Sá da Costa 2013-06-07, 07:56
Copy link to this message
-
Re: save several 64MB files in Pig Latin
Johnny Zhang 2013-06-07, 08:14
Pedro, you can try Piggybank MultiStorage, which split results into
different dir/files by specific index attribute. But not sure how it can
make sure the file size is 64MB. Why 64MB specifically? what's the
connection between your data and 64MB?

Johnny
On Fri, Jun 7, 2013 at 12:56 AM, Pedro Sá da Costa <[EMAIL PROTECTED]>wrote:

> I am using the instruction:
>
> store A into 'result-australia-0' using PigStorage('\t');
>
> to store the data in HDFS. But the problem is that, this creates 1 file
> with 500MB of size. Instead, want to save several 64MB files. How I do
> this?
>
> --
> Best regards,
>
+
Pedro Sá da Costa 2013-06-10, 04:53
+
Bertrand Dechoux 2013-06-10, 05:29
+
Pedro Sá da Costa 2013-06-10, 05:42
+
Johnny Zhang 2013-06-10, 06:58
+
Bertrand Dechoux 2013-06-10, 09:21
+
Pedro Sá da Costa 2013-06-10, 09:36
+
Ruslan Al-Fakikh 2013-06-10, 11:34
+
Bertrand Dechoux 2013-06-10, 11:42