Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
clear query|facets|time Search criteria: .   Results from 1 to 10 from 120 (0.356s).
Loading phrases to help you
refine your search...
[expand - 3 more] - Re: upgrade from spark 1.2.1 to 1.3 on EC2 cluster and problems - Spark - [mail # user]
...You could stop the running the processes and run the same processes usingthe new version, starting with the master and then the slaves. You wouldhave to snoop around a bit to get the command...
   Author: Dean Wampler, 2015-03-26, 02:46
Re: Date and decimal datatype not working - Spark - [mail # user]
...Recall that the input isn't actually read until to do something that forcesevaluation, like call saveAsTextFile. You didn't show the whole stack tracehere, but it probably occurred while par...
   Author: Dean Wampler, 2015-03-25, 20:19
[expand - 1 more] - Re: newbie quesiton - spark with mesos - Spark - [mail # user]
...I think the problem is the use the loopback address:export SPARK_LOCAL_IP=127.0.0.1In the stack trace from the slave, you see this:...  Reason: Connection refused: localhost/127.0.0.1:5...
   Author: Dean Wampler, 2015-03-25, 19:44
[expand - 1 more] - Re: Optimal solution for getting the header from CSV with Spark - Spark - [mail # user]
...Good point. There's no guarantee that you'll get the actual firstpartition. One reason why I wouldn't allow a CSV header line in a real datafile, if I could avoid it.Back to Spark, a safer a...
   Author: Dean Wampler, 2015-03-24, 16:20
Re: How to deploy binary dependencies to workers? - Spark - [mail # user]
...Both spark-submit and spark-shell have a --jars option for passingadditional jars to the cluster. They will be added to the appropriateclasspaths.Dean Wampler, Ph.D.Author: Programming Scala...
   Author: Dean Wampler, 2015-03-24, 15:16
Re: Getting around Serializability issues for types not in my control - Spark - [mail # user]
...Well, it's complaining about trait OptionInstances which is defined inOption.scala in the std package. Use scalap or javap on the scalaz libraryto find out which member of the trait is the p...
   Author: Dean Wampler, 2015-03-23, 19:27
Re: Converting SparkSQL query to Scala query - Spark - [mail # user]
...There isn't any automated way. Note that as the DataFrame implementationimproves, it will probably do a better job with query optimization thanhand-rolled Scala code. I don't know if that's ...
   Author: Dean Wampler, 2015-03-23, 19:09
Re: Saving Dstream into a single file - Spark - [mail # user]
...You can use the coalesce method to reduce the number of partitions. You canreduce to one if the data is not too big. Then write the output.Dean Wampler, Ph.D.Author: Programming Scala, 2nd E...
   Author: Dean Wampler, 2015-03-23, 13:37
Re: registerTempTable is not a member of RDD on spark 1.2? - Spark - [mail # user]
...In 1.2 it's a member of SchemaRDD and it becomes available on RDD (throughthe "type class" mechanism) when you add a SQLContext, like so.val sqlContext = new SQLContext(sc)import sqlContext....
   Author: Dean Wampler, 2015-03-23, 13:08
Re: Error while installing Spark 1.3.0 on local machine - Spark - [mail # user]
...Any particular reason you're not just downloading a build fromhttp://spark.apache.org/downloads.html Even if you aren't using Hadoop, anyof those builds will work.If you want to build from s...
   Author: Dean Wampler, 2015-03-22, 15:35
Sort:
project
Hive (94)
Spark (26)
type
mail # user (112)
mail # dev (5)
issue (3)
date
last 7 days (3)
last 30 days (12)
last 90 days (16)
last 6 months (21)
last 9 months (120)
author
Ted Yu (1986)
Harsh J (1314)
Jun Rao (1089)
Todd Lipcon (1011)
Stack (1000)
Andrew Purtell (975)
GitHub Import (895)
Jonathan Ellis (859)
Josh Elser (822)
stack (819)
Jarek Jarcec Cecho (807)
Yusaku Sako (785)
Hitesh Shah (767)
Jean-Daniel Cryans (753)
Siddharth Seth (743)
Eric Newton (733)
Brock Noland (725)
Jonathan Hsieh (700)
Steve Loughran (693)
Roman Shaposhnik (686)
Namit Jain (648)
Hyunsik Choi (640)
James Taylor (638)
Owen O'Malley (619)
Neha Narkhede (579)
Dean Wampler
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB