| clear query|facets|time |
Search criteria: .
Results from 91 to 100 from
24985 (0.225s).
|
|
|
Loading phrases to help you
refine your search...
|
|
GROUP BY Issue - Pig - [mail # user]
|
|
...Hi, On running the following query I am getting multiple records with same value of F1 SELECT F1, COUNT(*) FROM ( SELECT F1, F2, COUNT(*) FROM TABLE1 GROUP BY F1, F2 ) a GROUP BY...
|
|
|
Author: Gourav Sengupta,
2013-06-10, 14:39
|
|
|
Re: save several 64MB files in Pig Latin - Pig - [mail # user]
|
|
...The hadoop wiki give a brief explanation : http://wiki.apache.org/hadoop/HowManyMapsAndReduces The logic is indeed the same for Pig because, under the hood, Pig will generated and opti...
|
|
|
Author: Bertrand Dechoux,
2013-06-10, 11:42
|
|
|
Re: save several 64MB files in Pig Latin - Pig - [mail # user]
|
|
...Hi Pedro, Yes, Pig Latin is always compiled to MapReduce. Usually you don't have to specify the number of mappers (I am not sure whether you really can). If you have a file of 500MB an...
|
|
|
Author: Ruslan Al-Fakikh,
2013-06-10, 11:34
|
|
|
Re: save several 64MB files in Pig Latin - Pig - [mail # user]
|
|
...Yes, I understand the previous answers now. The reason of my question is because I was trying to "split" a file with pig latin by loading the file and writing portions of the file again in H...
|
|
|
Author: Pedro Sá da Costa,
2013-06-10, 09:36
|
|
|
Re: save several 64MB files in Pig Latin - Pig - [mail # user]
|
|
...I wasn't clear. Specifying the size of the files is not your real aim, I guess. But you think that's what is needed in order to solve your problem that we don't know about. 500MB is not a re...
|
|
|
Author: Bertrand Dechoux,
2013-06-10, 09:21
|
|
|
Re: save several 64MB files in Pig Latin - Pig - [mail # user]
|
|
...Hi, Pedro: Basically how many splits of files depends on how many reducer you have in your Pig job. So if total result data size is 100MB, and you have 10 reducers, you will get 10 files and...
|
|
|
Author: Johnny Zhang,
2013-06-10, 06:58
|
|
|
Re: save several 64MB files in Pig Latin - Pig - [mail # user]
|
|
...I don't understand why my purpose is not clear. The previous e-mails explain it very clearly. I want to split a 500MB single txt in HDFS into multiple files using Pig latin. Is it poss...
|
|
|
Author: Pedro Sá da Costa,
2013-06-10, 05:42
|
|
|
Re: save several 64MB files in Pig Latin - Pig - [mail # user]
|
|
...The purpose is not really clear. But if you are looking for how to specify multiple Reducer task, it is well explained in the documentation. http://pig.apache.org/docs/r0.11.1/perf.html#para...
|
|
|
Author: Bertrand Dechoux,
2013-06-10, 05:29
|
|
|
Re: save several 64MB files in Pig Latin - Pig - [mail # user]
|
|
...I said 64MB, but it can be 128MB, or 5KB. It doesn't matter the number. I just want to extract data and put into several files with specific size. Basically, I am doing a cat to a big txt fi...
|
|
|
Author: Pedro Sá da Costa,
2013-06-10, 04:53
|
|
|
[PIG-3250] Pig dryrun generates wrong output in .expanded file for 'SPLIT....OTHERWISE...' command - Pig - [issue]
|
|
...step to reproduce it:1. input files 'users'123452. pig script split.pigdefine group_and_count (A,key) returns B { SPLIT $A INTO $B IF $key<7, Y IF $key==5, Z OTHERWISE;}...
|
|
|
http://issues.apache.org/jira/browse/PIG-3250
Author: Johnny Zhang,
2013-06-10, 01:31
|
|
|
|
