Joseph K. Bradley
mail # user
mail # dev
last 7 days (1)
last 30 days (1)
last 90 days (1)
last 6 months (1)
last 9 months (27)
newest on top
oldest on top
. Results from
Loading phrases to help you
refine your search...
[SPARK-24610] wholeTextFiles broken for small files
...Spark is unable to read small files using the wholeTextFiles method when split size related configs are specified - either explicitly or if they are contained in other config files like hive...
, 2018-07-12, 20:37
[SPARK-21243] Limit the number of maps in a single shuffle fetch
...Right now spark can limit the # of parallel fetches and also limits the amount of data in one fetch, but one fetch to a host could be for 100's of blocks. In one instance we saw 450+ blocks....
, 2017-07-21, 19:46
[SPARK-21500] Update Shuffle Fetch bookkeeping immediately after receiving remote block
...Currently we update any remote shuffle fetch related parameters only when we start processing the FetchResult rather than updating them immediately after receiving them. This would spe...
, 2017-07-21, 15:36
[SPARK-21181] Suppress memory leak errors reported by netty
...We are seeing netty report memory leak erros like the one below after switching to 2.1. ERROR ResourceLeakDetector: LEAK: ByteBuf.release() was not called before it's garbage-collected. Enab...
, 2017-07-03, 11:15
Where is yarn-shuffle.jar in maven?
- [mail # user]
...Hi Neal,From my understanding, the reason I think the shuffle is not available inmaven central is that it is an external component and dependent on thecluster manager version - atleast in ca...
, 2016-12-13, 18:05
Checkpointing in Spark - Cleaning files and Support across app attempts
- [mail # dev]
...While checkpointing RDDs as a part of an application that doesn't usespark-streaming, I observed that the checkpointed files are not beingcleaned up even after the application completes succ...
, 2016-10-11, 22:13
[SPARK-17417] Fix sorting of part files while reconstructing RDD/partition from checkpointed files.
...Spark currently assumes # of partitions to be less than 100000 and uses %05d padding. If we exceed this no., the sort logic in ReliableCheckpointRDD gets messed up and fails. This is because...
, 2016-10-10, 15:57
[SPARK-17365] Kill multiple executors together to reduce lock contention
...To regulate pending and running executors we determine the executors which are eligible to kill and kill them iteratively rather than a loop. This does an RPC call and is synchronized leadin...
, 2016-09-22, 17:11
Setting YARN executors' JAVA_HOME
- [mail # dev]
...Hi Ryan,You can get more info on this here: Spark documentation.The page addresses what you need. You can look forspark.executorEnv.[EnvironmentVariableName]and set your java home assp...
, 2016-08-18, 18:51
Spark 2.0.0 JaninoRuntimeException
- [mail # user]
...I see a similar issue being resolved recently:https://issues.apache.org/jira/browse/SPARK-15285On Fri, Aug 12, 2016 at 3:33 PM, Aris wrote:-Dhruve Ashar...
, 2016-08-12, 21:30
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by