| clear query|facets|time |
Search criteria: .
Results from 31 to 40 from
64 (0.159s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: debug feature?? - Pig - [mail # user]
|
|
...Hi, Basically it would be perfect if you first test with a small amount of data in local mode and then run the script on the big data to verify the correctness. If this is not possible...
|
|
|
Author: Ruslan Al-Fakikh,
2012-10-19, 13:04
|
|
|
Re: Decide if function is algebraic at planning phase - Pig - [mail # user]
|
|
...Hi! Out of curiosity: what for? Algebraic works faster in most cases. Possible solutions: 1) Maybe you can disable the use of combiner or something else that is related. Maybe if...
|
|
|
Author: Ruslan Al-Fakikh,
2012-10-09, 07:34
|
|
|
Re: Input and output path - Pig - [mail # user]
|
|
...MiaoMiao, Mohit, If we are talking about embedding Pig into Python, I'd like to add that we can also embed Pig into Java using PigServer http://wiki.apache.org/pig/EmbeddedPig Mi...
|
|
|
Author: Ruslan Al-Fakikh,
2012-09-13, 21:04
|
|
|
Re: Input and output path - Pig - [mail # user]
|
|
...Mohit, I am suggesting setting up a whole Hive warehouse. This way your folders will look like /user/hive/warehouse/yourdataset/date=2012-09-11 /user/hive/warehouse/yourdataset/date=20...
|
|
|
Author: Ruslan Al-Fakikh,
2012-09-11, 15:12
|
|
|
Re: Input and output path - Pig - [mail # user]
|
|
...Mohit, I guess you could use parameters substitution here http://wiki.apache.org/pig/ParameterSubstitution Also, a note about your architecture: You can consider using Hive parti...
|
|
|
Author: Ruslan Al-Fakikh,
2012-09-10, 23:17
|
|
|
Re: loading data to mysql using pig - Pig - [mail # user]
|
|
...Hi, Probably DBStorage is more convenient (I haven't tried it), but you can also you Sqoop if you are ok with storing data to HDFS first and then using Sqoop to insert data to MySql &n...
|
|
|
Author: Ruslan Al-Fakikh,
2012-09-10, 23:12
|
|
|
Re: Storing field in a bag - Pig - [mail # user]
|
|
...Hi, Mohit, http://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#STORE I guess you can only STORE relations, not fields, etc Ruslan On Mon, Sep 10, 2012 at 9:53 PM, Mohit An...
|
|
|
Author: Ruslan Al-Fakikh,
2012-09-10, 23:05
|
|
|
Re: Custom DB Loader UDF - Pig - [mail # user]
|
|
...Terry, Russell, Just a proposal: maybe it should be added to DBStorage? http://pig.apache.org/docs/r0.8.1/api/org/apache/pig/piggybank/storage/DBStorage.html As far as I know it only s...
|
|
|
Author: Ruslan Al-Fakikh,
2012-08-31, 23:50
|
|
|
Re: Custom DB Loader UDF - Pig - [mail # user]
|
|
...Terry, Probably I can mislead you in some way, I haven't implemented our loader myself but what we have is something like @Override public void setLocation(String string, Job job) thr...
|
|
|
Author: Ruslan Al-Fakikh,
2012-08-31, 22:55
|
|
|
Re: Custom DB Loader UDF - Pig - [mail # user]
|
|
...Hi Terry, I am not sure whether you architecture is correct, but what we do in my team: we override setLocation in LoadFunc so that it caches db data to hdfs. Basically the simplest wa...
|
|
|
Author: Ruslan Al-Fakikh,
2012-08-31, 21:44
|
|
|
|
