| clear query|facets|time |
Search criteria: .
Results from 11 to 20 from
64 (0.078s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Trying to store a bag of tuples using AvroStorage. - Pig - [mail # user]
|
|
...Hi Dan, Could you attach your script and sample input files for both cases (with and without the schema). In the case where no schema is provided, the stack trace shows that the ...
|
|
|
Author: Stan Rosenberg,
2012-03-26, 15:11
|
|
|
Re: What should storefuncs do on parse errors while reading? - Pig - [mail # user]
|
|
...I typically increment a counter and have a bounded log of randomly sampled erroneous data. stan On Mar 24, 2012 6:50 PM, "[EMAIL PROTECTED]" wrote: ...
|
|
|
Author: Stan Rosenberg,
2012-03-25, 14:44
|
|
|
Re: Trying to store a bag of tuples using AvroStorage. - Pig - [mail # user]
|
|
...Hi Dan, This looks like an avro bug. I'll have a look later tonight unless someone else has a more immediate answer. Best, stan On Mar 25, 2012 12:36 AM, "Dan Young" ...
|
|
|
Author: Stan Rosenberg,
2012-03-25, 14:41
|
|
|
Re: Globbing several AVRO files with different (extended) schemes - Pig - [mail # user]
|
|
...There is a patch for AvroStorage which computes a union schema thereby allowing input avro files having different schemas, specifically (un-nested) records with different fields. https...
|
|
|
Author: Stan Rosenberg,
2012-03-22, 02:13
|
|
|
Re: config/reference data files for UDFS - Pig - [mail # user]
|
|
...Hi Alan, I am also curious to see how the distributed cache is used in a UDF. However, the code you reference in the patch doesn't appear to contain such an example. What is the ...
|
|
|
Author: Stan Rosenberg,
2012-03-13, 20:55
|
|
|
Re: Understanding LoadFunc sequence - Pig - [mail # user]
|
|
...Hi Bill, I've used the following in my UDFs: public static boolean isBackend(JobContext ctx) { // HACK borrowed from HCatLoader: this property should only be set on the backend...
|
|
|
Author: Stan Rosenberg,
2012-02-04, 04:04
|
|
|
Re: Passing schema inside Load functionc - Pig - [mail # user]
|
|
...Hi Praveenesh, Maybe this will get you started. Suppose we have the desired schema parsed and stored in 'map' of type LinkedHashMap. The key is your field name, and the val...
|
|
|
Author: Stan Rosenberg,
2012-02-04, 02:40
|
|
|
Re: Passing schema inside Load functionc - Pig - [mail # user]
|
|
...Hi Praveenesh, Assuming you have already read these: http://ofps.oreilly.com/titles/9781449302641/load_and_store_funcs.html http://pig.apache.org/docs/r0.9.2/udf.html#load-store-...
|
|
|
Author: Stan Rosenberg,
2012-02-03, 22:42
|
|
|
Re: Passing schema inside Load functionc - Pig - [mail # user]
|
|
...My hunch is you'll have to write a custom loader, but I'll let the experts chime in. E.g., AvroStorage loader can parse the schema from a json file passed to it via the constructor. &n...
|
|
|
Author: Stan Rosenberg,
2012-02-03, 22:32
|
|
|
Re: Pig/Avro Question - Pig - [mail # user]
|
|
...Check the code in PigAvroInputFormat; it overrides 'listStatus' from FileInputFormat so that files not ending in .avro are filtered. stan On Fri, Feb 3, 2012 at 1:58 PM, Russell ...
|
|
|
Author: Stan Rosenberg,
2012-02-03, 21:17
|
|
|
|
