Home | About | Sematext search-lucene.com search-hadoop.com
clear query|facets|time Search criteria: .   Results from 1 to 10 from 29 (0.15s).
Loading phrases to help you
refine your search...
Re: UpdateStateByKey - How to improve performance? - Spark - [mail # user]
...Depending on the density of your keys, the alternative signaturedef updateStateByKey[S](updateFunc: (Iterator[(K, Seq[V], Option[S])]) ? Iterator[(K, S)], partitioner: Partitioner, rememberP...
   Author: Michael Malak, 2014-08-06, 21:07
Re: relationship of RDD[Array[String]] to Array[Array[String]] - Spark - [mail # user]
...It's really more of a Scala question than a Spark question, but the standard OO (not Scala-specific) way is to create your own custom supertype (e.g. MyCollectionTrait), inherited/implemente...
   Author: Michael Malak, 2014-07-21, 17:21
15 new MLlib algorithms - Spark - [mail # dev]
...At Spark Summit, Patrick Wendell indicated the number of MLlib algorithms would "roughly double" in 1.1 from the current approx. 15.http://spark-summit.org/wp-content/uploads/2014/07/Future-...
   Author: Michael Malak, 2014-07-09, 18:43
Re: parallel Reduce within a key - Spark - [mail # user]
...How about a treeReduceByKey? :-)On Friday, June 20, 2014 11:55 AM, DB Tsai  wrote: Currently, the reduce operation combines the result from mappersequentially, so it's O(n).Xiangru...
   Author: Michael Malak, 2014-06-20, 18:09
[SPARK-1817] RDD zip erroneous when partitions do not divide RDD count - Spark - [issue]
...Example:scala> sc.parallelize(1L to 2L,4).zip(sc.parallelize(11 to 12,4)).collectres1: Array[(Long, Int)] = Array((2,11))But more generally, it's whenever the number of partitions...
http://issues.apache.org/jira/browse/SPARK-1817    Author: Michael Malak, 2014-06-04, 17:16
GraphX triplets on 5-node graph - Spark - [mail # dev]
...Shouldn't I be seeing N2 and N4 in the output below? (Spark 0.9.0 REPL) Or am I missing something fundamental?val nodes = sc.parallelize(Array((1L, "N1"), (2L, "N2"), (3L, "N3"), (4L, "N4"),...
   Author: Michael Malak, 2014-05-29, 06:48
[SPARK-1836] REPL $outer type mismatch causes lookup() and equals() problems - Spark - [issue]
...Anand Avati partially traced the cause to REPL wrapping classes in $outer classes. There are at least two major symptoms:1. equals()=========In REPL equals() (required in custom classes used...
http://issues.apache.org/jira/browse/SPARK-1836    Author: Michael Malak, 2014-05-28, 20:40
Re: rdd ordering gets scrambled - Spark - [mail # user]
...Mohit Jaggi:A workaround is to use zipWithIndex (to appear in Spark 1.0, but if you're still on 0.9x you can swipe the code from https://github.com/apache/spark/blob/master/core/src/main/sca...
   Author: Michael Malak, 2014-05-28, 15:44
[SPARK-1857] map() with lookup() causes exception - Spark - [issue]
...Using map() and lookup() in conjunction throws an exceptionval a = sc.parallelize(Array(11))val m = sc.parallelize(Array((11,21)))a.map(m.lookup(_)(0)).collect14/05/14 15:03:35 ERROR Executo...
http://issues.apache.org/jira/browse/SPARK-1857    Author: Michael Malak, 2014-05-25, 22:01
Re: [VOTE] Release Apache Spark 1.0.0 (rc5) - Spark - [mail # dev]
...While developers may appreciate "1.0 == API stability," I'm not sure that will be the understanding of the VP who gives the green light to a Spark-based development effort.I fear a bug that ...
   Author: Michael Malak, 2014-05-17, 23:10
Sort:
project
Spark (14)
Hive (13)
Avro (1)
Pig (1)
type
mail # user (19)
issue (5)
mail # dev (5)
date
last 7 days (0)
last 30 days (0)
last 90 days (3)
last 6 months (14)
last 9 months (29)
author
Ted Yu (1650)
Harsh J (1292)
Jun Rao (1035)
Todd Lipcon (1001)
Stack (973)
Jonathan Ellis (842)
Andrew Purtell (799)
Jean-Daniel Cryans (753)
jacques@... (738)
Yusaku Sako (717)
stack (716)
Jarek Jarcec Cecho (699)
Eric Newton (697)
Jonathan Hsieh (675)
Roman Shaposhnik (660)
Brock Noland (656)
Neha Narkhede (650)
Namit Jain (649)
Hitesh Shah (626)
Owen O'Malley (625)
Steve Loughran (615)
Siddharth Seth (614)
Josh Elser (570)
Eli Collins (545)
Arun C Murthy (543)
Michael Malak