Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
clear query|facets|time Search criteria: .   Results from 1 to 10 from 134 (0.205s).
Loading phrases to help you
refine your search...
[expand - 4 more] - Re: MappedStream vs Transform API - Spark - [mail # user]
...Hi, Sorry for the wrong formatting in the earlier mail.On Tue, Mar 17, 2015 at 2:31 PM, Tathagata Das  wrote:  Ok. When I was going through source code it confused me to ...
   Author: madhu phatak, 2015-03-17, 09:25
[expand - 1 more] - Re: why generateJob is a private API? - Spark - [mail # user]
...Hi, Thank you for the response.Regards,Madhukara Phatakhttp://datamantra.io/On Tue, Mar 17, 2015 at 5:50 AM, Tathagata Das  wrote: ...
   Author: madhu phatak, 2015-03-17, 08:58
Re: Need Advice about reading lots of text files - Spark - [mail # user]
...Hi,Internally Spark uses HDFS api to handle file data. Have a look at HAR,Sequence file input format. More information on this cloudera blog.Regards,Madhukara Phatakhttp://datamantra.io/On S...
   Author: madhu phatak, 2015-03-16, 06:27
Re: Streaming: getting data from Cassandra based on input stream values - Spark - [mail # user]
...Hi,In that case, you can try the following.val joinRDD = kafkaStream.transform( streamRDD => {val ids = streamRDD.map(_._2).collect();ids.map(userId =>  ctable.select("user_name")...
   Author: madhu phatak, 2015-01-24, 07:58
Re: save a histogram to a file - Spark - [mail # user]
...Hi,histogram method return normal scala types not a RDD. So you will nothave saveAsTextFile.You can use makeRDD method make a rdd out of the data and saveAsObject fileval hist = a.histogram(...
   Author: madhu phatak, 2015-01-23, 09:25
Re: DAG info - Spark - [mail # user]
...Hi,You can turn off these messages using log4j.properties.On Fri, Jan 2, 2015 at 1:51 PM, Robineast  wrote:Regards,Madhukara Phatakhttp://www.madhukaraphatak.com ...
   Author: madhu phatak, 2015-01-03, 08:35
Re: Joins in Spark - Spark - [mail # user]
...Hi, You can map your vertices rdd as followval pairVertices = verticesRDD.map(vertice => (vertice,null))the above gives you a pairRDD. After join make sure that you removesuperfluous...
   Author: madhu phatak, 2014-12-23, 05:22
Re: broadcasting object issue - Spark - [mail # user]
...Hi, Just ran your code on spark-shell.  If you replace val bcA = sc.broadcast(a)withval bcA = sc.broadcast(new B().getA)it seems to work. Not sure why.On Tue, Dec 23, 2014 at ...
   Author: madhu phatak, 2014-12-23, 05:17
Re: reading files recursively using spark - Spark - [mail # user]
...Hi,You can use FileInputformat API of Hadoop and newApiHadoopFile of spark toget recursion. More on the topic you can refer herehttp://stackoverflow.com/questions/8114579/using-fileinputform...
   Author: madhu phatak, 2014-12-19, 11:28
Re: When will spark 1.2 released? - Spark - [mail # user]
...It’s on Maven Central already http://search.maven.org/#browse%7C717101892On Fri, Dec 19, 2014 at 11:17 AM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:Regards,Madhukara Phatakhttp://ww...
   Author: madhu phatak, 2014-12-19, 06:02
Sort:
project
Hadoop (95)
Spark (14)
MapReduce (12)
HDFS (9)
Hive (4)
type
mail # user (119)
mail # dev (15)
date
last 7 days (0)
last 30 days (3)
last 90 days (6)
last 6 months (14)
last 9 months (134)
author
Ted Yu (1983)
Harsh J (1313)
Jun Rao (1089)
Todd Lipcon (1011)
Stack (1000)
Andrew Purtell (973)
GitHub Import (895)
Jonathan Ellis (858)
Josh Elser (820)
stack (818)
Jarek Jarcec Cecho (807)
Yusaku Sako (783)
Hitesh Shah (765)
Jean-Daniel Cryans (753)
Siddharth Seth (739)
Eric Newton (733)
Brock Noland (725)
Jonathan Hsieh (700)
Steve Loughran (690)
Roman Shaposhnik (686)
Namit Jain (648)
Hyunsik Choi (640)
James Taylor (636)
Owen O'Malley (619)
Neha Narkhede (580)
madhu phatak
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB