Home | About | Sematext search-lucene.com search-hadoop.com
clear query|facets|time Search criteria: .   Results from 1 to 10 from 28 (0.218s).
Loading phrases to help you
refine your search...
Re: All of the tasks have been completed but the Stage is still shown as "Active"? - Spark - [mail # user]
...History Server is also very helpful.On Thu, Jul 10, 2014 at 7:37 AM, Haopu Wang  wrote:SUREN HIRAMAN, VP TECHNOLOGYVelosAccelerating Machine Learning440 NINTH AVENUE, 11TH FLOORNEW YORK...
   Author: Surendranauth Hiraman, 2014-07-10, 12:16
Re: CPU/Disk/network performance instrumentation - Spark - [mail # dev]
...+1 on advanced tab.On Wed, Jul 9, 2014 at 5:20 PM, Mridul Muralidharan wrote:SUREN HIRAMAN, VP TECHNOLOGYVelosAccelerating Machine Learning440 NINTH AVENUE, 11TH FLOORNEW YORK, NY 10001O: (9...
   Author: Surendranauth Hiraman, 2014-07-09, 21:44
Re: Purpose of spark-submit? - Spark - [mail # user]
...Are there any gaps beyond convenience and code/config separation in usingspark-submit versus SparkConf/SparkContext if you are willing to set yourown config?If there are any gaps, +1 on havi...
   Author: Surendranauth Hiraman, 2014-07-09, 12:31
[expand - 5 more] - Re: Comparative study - Spark - [mail # user]
...Aaron,I don't think anyone was saying Spark can't handle this data size, giventestimony from the Spark team, Bizo, etc., on large datasets. This has keptus trying different things to get our...
   Author: Surendranauth Hiraman, 2014-07-08, 20:42
[expand - 1 more] - Re: Spark memory optimization - Spark - [mail # user]
...Using persist() is a sort of a "hack" or a hint (depending on yourperspective :-)) to make the RDD use disk, not memory. As I mentionedthough, the disk io has consequences, mainly (I think) ...
   Author: Surendranauth Hiraman, 2014-07-07, 12:31
Re: Enable Parsing Failed or Incompleted jobs on HistoryServer (YARN mode) - Spark - [mail # user]
...I've had some odd behavior with jobs showing up in the history server in1.0.0. Failed jobs do show up but it seems they can show up minutes orhours later. I see in the history server logs me...
   Author: Surendranauth Hiraman, 2014-07-03, 10:57
PySpark Driver from Jython - Spark - [mail # dev]
...Has anyone tried running pyspark driver code in Jython, preferably bycalling python code within Java code?I know CPython is the only interpreter tested because of the need tosupport C extens...
   Author: Surendranauth Hiraman, 2014-07-01, 18:32
Re: Changing log level of spark - Spark - [mail # user]
...One thing we ran into was that there was another log4j.properties earlierin the classpath. For us, it was in our MapR/Hadoop conf.If that is the case, something like the following could help...
   Author: Surendranauth Hiraman, 2014-07-01, 13:41
Re: Spark executor error - Spark - [mail # user]
...I unfortunately haven't seen this directly. But some typical things I trywhen debugging are as follows.Do you see a corresponding error on the other side of that connection(alpinenode7.alpin...
   Author: Surendranauth Hiraman, 2014-06-26, 11:27
[expand - 2 more] - Re: Trailing Tasks Saving to HDFS - Spark - [mail # dev]
...I've created an issue for this but if anyone has any advice, please let meknow.Basically, on about 10 GBs of data, saveAsTextFile() to HDFS hangs on tworemaining tasks (out of 320). Those ta...
   Author: Surendranauth Hiraman, 2014-06-19, 18:19
Spark (22)
Kafka (6)
mail # user (20)
mail # dev (8)
last 7 days (0)
last 30 days (0)
last 90 days (6)
last 6 months (23)
last 9 months (28)
Ted Yu (1650)
Harsh J (1293)
Jun Rao (1035)
Todd Lipcon (1001)
Stack (973)
Jonathan Ellis (842)
Andrew Purtell (798)
Jean-Daniel Cryans (753)
jacques@... (738)
Yusaku Sako (716)
stack (716)
Jarek Jarcec Cecho (699)
Eric Newton (697)
Jonathan Hsieh (675)
Roman Shaposhnik (660)
Brock Noland (656)
Neha Narkhede (651)
Namit Jain (649)
Hitesh Shah (626)
Owen O'Malley (625)
Steve Loughran (615)
Siddharth Seth (614)
Josh Elser (569)
Eli Collins (545)
Arun C Murthy (543)
Surendranauth Hiraman