Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
clear query|facets|time Search criteria: .   Results from 1 to 10 from 162 (0.151s).
Loading phrases to help you
refine your search...
Is there a way (in Java) to turn Java Iterable into a JavaRDD? - Spark - [mail # user]
...I notice new methods such as JavaSparkContext makeRDD (with few usefulexamples) - It takes a Seq but while there are ways to turn a list into aSeq I see nothing that uses an Iterable ...
   Author: Steve Lewis, 2014-12-19, 18:26
Who is using Spark and related technologies for bioinformatics applications? - Spark - [mail # user]
...I am aware of the ADAM project in Berkeley and I am working on Proteomicsearches -anyone else working in this space ...
   Author: Steve Lewis, 2014-12-17, 16:28
[expand - 2 more] - Re: how to convert an rdd to a single output file - Spark - [mail # user]
...what would good spill settings be?On Fri, Dec 12, 2014 at 2:45 PM, Sameer Farooqui wrote:Steven M. Lewis PhD4221 105th Ave NEKirkland, WA 98033206-384-1340 (cell)Skype lordjoe_com ...
   Author: Steve Lewis, 2014-12-12, 23:07
In Java how can I create an RDD with a large number of elements - Spark - [mail # user]
...assume I don't care about values which may be created in a later map - inscala I can sayval rdd = sc.parallelize(1 to 1000000000, numSlices = 1000)but in Java JavaSparkContext can only paral...
   Author: Steve Lewis, 2014-12-09, 02:18
[expand - 1 more] - Re: How can I create an RDD with millions of entries created programmatically - Spark - [mail # user]
...looks good but how do I say that in Javaas far as I can see sc.parallelize (in Java)  has only one implementationwhich takes a List - requiring an in memory representationOn Mon, Dec 8,...
   Author: Steve Lewis, 2014-12-08, 21:12
Problems creating and reading a large test file - Spark - [mail # user]
...I am trying to look at problems reading a data file over 4G. In my testingI am trying to create such a file.My plan is to create a fasta file (a simple format used in biology)looking likeTCC...
   Author: Steve Lewis, 2014-12-06, 01:21
I am having problems reading files in the 4GB range - Spark - [mail # user]
...I am using a custom hadoop input format which works well on smaller filesbut fails with a file at about 4GB size - the format is generating about800 splits and all variables in my code are l...
   Author: Steve Lewis, 2014-12-05, 18:53
How can a function get a TaskContext - Spark - [mail # user]
...https://github.com/apache/spark/blob/master/core/src/main/java/org/apache/spark/TaskContext.javahas a Java implementation if TaskContext wit a very useful method/** * Return the currently ac...
   Author: Steve Lewis, 2014-12-04, 17:52
[expand - 2 more] - Re: Any ideas why a few tasks would stall - Spark - [mail # user]
...Thanks - I found the same thing -calling       boolean forceShuffle = true;        myRDD =   myRDD.coalesce(120,forceShuffle );worked - ther...
   Author: Steve Lewis, 2014-12-04, 17:16
Failed to read chunk exception - Spark - [mail # user]
...I am running a large job using 4000 partitions - after running for fourhours on a 16 node cluster it fails with the following message.The errors are in spark code and seem address unreliabil...
   Author: Steve Lewis, 2014-12-04, 16:50
Sort:
project
MapReduce (74)
Spark (44)
Hadoop (39)
HDFS (5)
type
mail # user (162)
date
last 7 days (3)
last 30 days (13)
last 90 days (36)
last 6 months (46)
last 9 months (162)
author
Ted Yu (1830)
Harsh J (1303)
Jun Rao (1014)
Todd Lipcon (994)
Stack (986)
Andrew Purtell (875)
Jonathan Ellis (854)
stack (757)
Jean-Daniel Cryans (750)
Jarek Jarcec Cecho (747)
Yusaku Sako (742)
Eric Newton (707)
Jonathan Hsieh (683)
Hitesh Shah (677)
Roman Shaposhnik (677)
Josh Elser (674)
Steve Loughran (651)
Namit Jain (648)
Siddharth Seth (643)
Brock Noland (633)
Owen O'Malley (623)
Hyunsik Choi (582)
Neha Narkhede (566)
Arun C Murthy (548)
Eli Collins (545)
Steve Lewis
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB