| clear query|facets|time |
Search criteria: .
Results from 91 to 100 from
333 (0.204s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Changing the schema before Storing - Pig - [mail # user]
|
|
...Thanks Younos for catching that and sorry that you got bit by it. That is in fact a javadoc bug. I've just opened a JIRA for it: https://issues.apache.org/jira/browse/PIG-3092 http://p...
|
|
|
Author: Bill Graham,
2012-12-12, 06:37
|
|
|
Re: Question regarding a custom LoadFunc implementation - Pig - [mail # user]
|
|
...We had a yml file that mapped physical datasources to the loader that the generic one serves as a facade to. Now we're moving to an HCatalog based solution that handles that as well as the l...
|
|
|
Author: Bill Graham,
2012-12-11, 23:06
|
|
|
Re: Question regarding a custom LoadFunc implementation - Pig - [mail # user]
|
|
...I think the latter would be better. Since the LoadFunc would be decoupled from the data exporter you could schedule the exporting independent of the loading. We do something similar, without...
|
|
|
Author: Bill Graham,
2012-12-11, 16:12
|
|
|
Re: Changing the schema before Storing - Pig - [mail # user]
|
|
...The STORE command doesn't take the AS clause, that's to define the schema at LOAD time. When storing, just prepare your relation with the the desired schema and then STORE it without the AS....
|
|
|
Author: Bill Graham,
2012-12-11, 07:27
|
|
|
Re: Do we have any plan for "Cost based optimizer"? - Pig - [mail # user]
|
|
...Duke has done work around CBOs for Hadoop as part of Starfish and there was some discussion back in the spring about doing the same for Pig, but I'm not aware of any concrete plans or JIRAs ...
|
|
|
Author: Bill Graham,
2012-12-06, 16:45
|
|
|
Re: Reducer estimation - Pig - [mail # dev]
|
|
...We do basically what you're describing. Each of our scripts has a logical name which defines the workflow. For each job in the workflow we persist the job stats, counters and conf in HBase v...
|
|
|
Author: Bill Graham,
2012-12-04, 07:01
|
|
|
Re: How to perfom a logical diff on two PigStorage files - Pig - [mail # user]
|
|
...I've done this in two passes. First I do an intersection test and determine the outer misses by join key on each side, similar to what you've done. I then store the left_only and right_only ...
|
|
|
Author: Bill Graham,
2012-11-30, 23:14
|
|
|
[PIG-3062] Change HBaseStorage to permit overriding pushProjection - Pig - [issue]
|
|
...In some cases it's useful to subclass HBaseStorage and override the logic in pushProjection. To do that we need to create the following protected methods:protected void setColumnInfoList(Lis...
|
|
|
http://issues.apache.org/jira/browse/PIG-3062
Author: Bill Graham,
2012-11-28, 06:23
|
|
|
[PIG-3063] Projection push down should support nested schemas - Pig - [issue]
|
|
...LoadPushDown.pushProjection(..) should support nested schemas. Currently the RequiredFieldList passed to this method only include the top-level fields so more granular projection at a lower ...
|
|
|
http://issues.apache.org/jira/browse/PIG-3063
Author: Bill Graham,
2012-11-20, 22:26
|
|
|
[PIG-2934] HBaseStorage filter optimizations - Pig - [issue]
|
|
...Our HBase pal/guru Gary Helmling was kind enough to do a code review of HBaseStorage. He suggested some good filter optimizations: when using the "lt*" and "gt*" options, set the start/stop ...
|
|
|
http://issues.apache.org/jira/browse/PIG-2934
Author: Bill Graham,
2012-11-20, 07:47
|
|
|
|
