Home | About | Sematext search-lucene.com search-hadoop.com
clear query|facets|time Search criteria: .   Results from 1 to 10 from 522 (0.117s).
Loading phrases to help you
refine your search...
A couple questions about shared variables - Spark - [mail # dev]
...Hey All,A couple questions came up about shared variables recently, and I wanted toconfirm my understanding and update the doc to be a little more clear.*Broadcast variables*Now that tasks d...
   Author: Sandy Ryza, 2014-09-20, 15:50
Re: Problem with giving memory to executors on YARN - Spark - [mail # user]
...I'm actually surprised your memory is that high. Spark only allocatesspark.storage.memoryFraction for storing RDDs.  This defaults to .6, so 32GB * .6 * 10 executors should be a total o...
   Author: Sandy Ryza, 2014-09-20, 15:10
Re: spark-submit: fire-and-forget mode? - Spark - [mail # user]
...Hi Tobias,YARN cluster mode should have the behavior you're looking for.  The clientprocess will stick around to report on things, but should be able to bekilled without affecting the a...
   Author: Sandy Ryza, 2014-09-18, 14:37
Re: Problem with pyspark command line invocation -- option truncation... (Spark v1.1.0) ... - Spark - [mail # user]
...Hi team didata,This doesn't directly answer your question, but with Spark 1.1, instead ofuser the driver options, it's better to pass your spark properties usingthe "conf" option.E.g.pyspark...
   Author: Sandy Ryza, 2014-09-17, 00:03
Re: Yarn Over-allocating Containers - Spark - [mail # user]
...Hi Praveen,I believe you are correct.  I noticed this a little while ago and had a fixfor it as part of SPARK-1714, but that's been delayed.  I'll look into thisa little deeper and...
   Author: Sandy Ryza, 2014-09-12, 18:51
Re: Spark authenticate enablement - Spark - [mail # dev]
...Hi Jun,I believe that's correct that Spark authentication only works against YARN.-SandyOn Thu, Sep 11, 2014 at 2:14 AM, Jun Feng Liu  wrote: ...
   Author: Sandy Ryza, 2014-09-12, 18:48
[expand - 2 more] - Re: Reporting serialized task size after task broadcast change? - Spark - [mail # dev]
...Hmm, well I can't find it now, must have been hallucinating.  Do you knowoff the top of your head where I'd be able to find the size to log it?On Thu, Sep 11, 2014 at 6:33 PM, Reynold X...
   Author: Sandy Ryza, 2014-09-12, 01:48
[expand - 3 more] - Re: [VOTE] Release Apache Hadoop 2.5.1 RC0 - HDFS - [mail # dev]
...+1 (binding)Ran a pseudo-distributed cluster with the Fair Scheduler and ran someMapReduce example jobs.-SandyOn Wed, Sep 10, 2014 at 12:15 PM, Alejandro Abdelnur wrote: ...
   Author: Sandy Ryza, 2014-09-10, 20:56
[expand - 3 more] - Re: Lost executor on YARN ALS iterations - Spark - [mail # dev]
...That's rightOn Tue, Sep 9, 2014 at 2:04 PM, Debasish Das wrote: ...
   Author: Sandy Ryza, 2014-09-10, 16:59
[SPARK-3082] yarn.Client.logClusterResourceDetails throws NPE if requested queue doesn't exist - Spark - [issue]
http://issues.apache.org/jira/browse/SPARK-3082    Author: Sandy Ryza, 2014-09-05, 23:24
Sort:
project
YARN (175)
MapReduce (144)
Spark (111)
Hadoop (67)
HDFS (22)
Avro (2)
Pig (1)
type
issue (339)
mail # user (91)
mail # dev (89)
mail # general (3)
date
last 7 days (4)
last 30 days (30)
last 90 days (105)
last 6 months (176)
last 9 months (522)
author
Ted Yu (1644)
Harsh J (1293)
Jun Rao (1031)
Todd Lipcon (1002)
Stack (974)
Jonathan Ellis (842)
Andrew Purtell (797)
Jean-Daniel Cryans (755)
jacques@... (738)
stack (716)
Yusaku Sako (708)
Jarek Jarcec Cecho (699)
Eric Newton (696)
Jonathan Hsieh (675)
Roman Shaposhnik (659)
Brock Noland (657)
Namit Jain (649)
Neha Narkhede (647)
Hitesh Shah (626)
Owen O'Malley (625)
Steve Loughran (617)
Siddharth Seth (614)
Josh Elser (563)
Eli Collins (545)
Arun C Murthy (543)
Sandy Ryza