| clear
query|
facets|
time |
Search criteria: .
Results from 1 to 10 from
11 (0.0s).
|
|
|
Loading phrases to help you
refine your search...
|
|
[expand - 1 more]
[collapse]
-
Reading from HCat as Avro -
Crunch - [mail # dev]
|
| ...The problem I was trying to solve was not wanting to deal with HCatRecords(which are basically GenericRecords) in the M/R code. Particularly with allthe code we (my team/org) have around tod... |
|
|
|
|
| |
|
Reading from HCat as Avro - Crunch - [mail # dev]
|
| ...I wasn't sure if this was the place to discuss this or on a JIRA. Tofollowup on the work for the HCatSource, I wrote some code to convert anHCatRecord into a specific avro model. This way yo... |
|
|
|
|
|
|
|
[CRUNCH-659] Upgrade to Hive 2.x -
Crunch - [issue]
|
| ...I've been working on CRUNCH-340 to finish implementing the HCatSource and HCatTarget. It seems to be in a better place now that crunch only supports hadoop 2. I was looking to target as high... |
|
|
|
|
|
|
|
HCatalog and Crunch named outputs -
Crunch - [mail # dev]
|
| ...I've recently taken up the work efforts on CRUNCH-340 [1] to get afunctioning source and target for going against HCatalog. One of the issuesI've ran into is around named outputs being added... |
|
|
|
|
|
|
|
Processing many map only collections in single pipeline with spark -
Crunch - [mail # user]
|
| ...Instead of using readTextFile on the pipeline, try using the read method and use the TextFileSource, which can accept in a collection of paths. https://github.com/apache/crunch/blob/master/c... |
|
|
|
|
|
|
|
[CRUNCH-588] Modify HFileUtils to flex on affected regions for hfiles, rather than all regions -
Crunch - [issue]
|
| ...HFileUtils when preparing for writing HFiles sets the number of reducers equal to the number of regions in the table, and then writes out the start keys for each region to a sequence f... |
|
|
|
|
|
|
|
[CRUNCH-591] Swap out deprecated usage of HTable and HTableInterface for Table and RegionLocator where applicable -
Crunch - [issue]
|
| ...HTable and HTableInterface have been deprecated, and should be replaced, but passivity should be maintained. Currently, HTable is only used in HFileUtils and several ITs.... |
|
|
|
|
|
|
|
[expand - 1 more]
[collapse]
-
Cleaning up after exceptions -
Crunch - [mail # user]
|
| ...Thanks. I’ll look into that. Also, I just noticed that as of 0.8.2, crunch has a public cleanup() on the Pipeline interface. I should be able to use that, as my code was just updated to that... |
|
|
|
|
| |
|
Cleaning up after exceptions - Crunch - [mail # user]
|
| ...If I have a scenario where I have already called Pipeline#run (and some temporary directories were created by Crunch during the run), and have continued on to do some additional processing (... |
|
|
|
|
|
|
|
[expand - 1 more]
[collapse]
-
crunch 0.8.2+6-cdh4.4.0 -
Crunch - [mail # user]
|
| ...Attached is a unit test that emits a null avro record that causes this stack trace to occur. The file that is being read in doesn't matter as the contents are ignored. I just needed to read ... |
|
|
|
|
| |
|
crunch 0.8.2+6-cdh4.4.0 - Crunch - [mail # user]
|
| ...This is the full stack trace. I removed some parts of the stack trace that emits the null values: org.apache.crunch.CrunchRuntimeException: org.apache.avro.file.DataFileWriter$AppendWr... |
|
|
|
|
|
|
|
|