Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
clear query|facets|time Search criteria: .   Results from 1 to 10 from 640 (0.268s).
Loading phrases to help you
refine your search...
Re: Running beyond physical memory limits - Spark - [mail # user]
...The setting to increase is spark.yarn.executor.memoryOverheadOn Wed, Apr 15, 2015 at 6:35 AM, Brahma Reddy Battula <[EMAIL PROTECTED]> wrote: ...
   Author: Sandy Ryza, 2015-04-15, 16:42
[SPARK-5112] Expose SizeEstimator as a developer API - Spark - [issue]
..."The best way to size the amount of memory consumption your dataset will require is to create an RDD, put it into cache, and look at the SparkContext logs on your driver program. The logs wi...
http://issues.apache.org/jira/browse/SPARK-5112    Author: Sandy Ryza, 2015-04-15, 14:04
Re: Spark: Using "node-local" files within functions? - Spark - [mail # user]
...Hi Tobias,It should be possible to get an InputStream from an HDFS file.  However, ifyour libraries only work directly on files, then maybe that wouldn't work?If that's the case and dif...
   Author: Sandy Ryza, 2015-04-14, 16:16
Re: Using memory mapped file for shuffle - Spark - [mail # dev]
...Hi Kannan,Both in MapReduce and Spark, the amount of shuffle data a task produces canexceed the tasks memory without risk of OOM.-SandyOn Tue, Apr 14, 2015 at 6:47 AM, Imran Rashid  wro...
   Author: Sandy Ryza, 2015-04-14, 16:07
Re: Rack locality - Spark - [mail # user]
...Hi Riya,As far as I know, that is correct, unless Mesos fine-grained mode handlesthis in some mysterious way.-SandyOn Mon, Apr 13, 2015 at 2:09 PM, rcharaya  wrote: ...
   Author: Sandy Ryza, 2015-04-13, 22:08
[MAPREDUCE-5612] Add javadoc for TaskCompletionEvent.Status - MapReduce - [issue]
...What's the difference between FAILED and TIPFAILED?  What is OBSOLETE?...
http://issues.apache.org/jira/browse/MAPREDUCE-5612    Author: Sandy Ryza, 2015-04-10, 21:19
Re: Spark Job Run Resource Estimation ? - Spark - [mail # user]
...Hi Deepak,I'm going to shamelessly plug my blog post on tuning Spark:http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/It talks about tuning executor size as we...
   Author: Sandy Ryza, 2015-04-09, 17:03
[expand - 1 more] - Re: [VOTE] Release Apache Spark 1.3.1 (RC2) - Spark - [mail # dev]
...+1Built against Hadoop 2.6 and ran some jobs against a pseudo-distributedYARN cluster.-SandyOn Wed, Apr 8, 2015 at 12:49 PM, Patrick Wendell  wrote: ...
   Author: Sandy Ryza, 2015-04-08, 21:20
[expand - 1 more] - Re: Strategy regarding maximum number of executor's failure for log running jobs/ spark streaming jobs - Spark - [mail # user]
...What's the advantage of killing an application for lack of resources?I think the rationale behind killing an app based on executor failures isthat, if we see a lot of them in a short span of...
   Author: Sandy Ryza, 2015-04-06, 20:34
[YARN-642] Fix up /nodes REST API to have 1 param and be consistent with the Java API - YARN - [issue]
...The code behind the /nodes RM REST API is unnecessarily muddled, logs the same misspelled INFO message repeatedly, and does not return unhealthy nodes, even when asked....
http://issues.apache.org/jira/browse/YARN-642    Author: Sandy Ryza, 2015-04-04, 03:09
Sort:
project
Spark (225)
YARN (176)
MapReduce (144)
Hadoop (70)
HDFS (22)
Avro (2)
Pig (1)
type
issue (377)
mail # user (148)
mail # dev (111)
mail # general (4)
date
last 7 days (4)
last 30 days (33)
last 90 days (89)
last 6 months (144)
last 9 months (640)
author
Ted Yu (2012)
Harsh J (1316)
Jun Rao (1101)
Todd Lipcon (1013)
Andrew Purtell (1005)
Stack (1001)
GitHub Import (895)
Jonathan Ellis (861)
Josh Elser (851)
stack (825)
Jarek Jarcec Cecho (813)
Yusaku Sako (791)
Hitesh Shah (782)
Siddharth Seth (764)
Jean-Daniel Cryans (752)
Eric Newton (734)
Brock Noland (724)
Steve Loughran (711)
Jonathan Hsieh (701)
Roman Shaposhnik (687)
James Taylor (680)
Namit Jain (648)
Hyunsik Choi (646)
Owen O'Malley (619)
Bikas Saha (582)
Sandy Ryza
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB