Home | About | Sematext search-lucene.com search-hadoop.com
clear query|facets|time Search criteria: .   Results from 1 to 10 from 36 (0.159s).
Loading phrases to help you
refine your search...
Re: Spark SQL and Hive tables - Spark - [mail # user]
...I have ran into the same issue. I understand with the new assembly builtwith -Phive, I can run a spark job in yarn-cluster mode. But is there a wayfor me to run spark-shell with support of h...
   Author: Chen Song, 2014-09-30, 15:56
[expand - 1 more] - Re: spark time out - Spark - [mail # user]
...I am running the job on 500 executors, each with 8G and 1 core.See lots of fetch failures on reduce stage, when running a simplereduceByKeymap tasks -> 4000reduce tasks -> 200On Mon, S...
   Author: Chen Song, 2014-09-23, 18:14
[expand - 1 more] - Re: exception in spark 1.1.0 - Spark - [mail # user]
...snappy native lib was installed as part of hadoop (cdh5.1.0)the location is in /usr/lib/hadoop/lib/-rwxr-xr-x 1 hdfs hadoop  23904 Jul 12 14:11 libsnappy.so.1.1.3lrwxrwxrwx 1 hdfs hadoo...
   Author: Chen Song, 2014-09-22, 13:31
spark job hung up - Spark - [mail # user]
...I am testing my spark job on yarnspark: 1.0.0-cdh5.1.0yarn: cdh5.1.0Once a while the spark job hung up (stuck in some stage without anyprogress on driver and executors) after some failures. ...
   Author: Chen Song, 2014-09-20, 09:08
[expand - 2 more] - Re: hive auto join conversion - Hive - [mail # user]
...Yeah, I was trying the same thing, though a little big ugly.My query needs to LJ/J with multiple tables. When there are 1 or 2 LJ/Js,rewriting works but when there are > 3 tables, the got...
   Author: Chen Song, 2014-08-12, 15:38
[expand - 4 more] - Re: saveAsTextFiles file not found exception - Spark - [mail # user]
...Thanks for putting this together, Andrew.On Tue, Aug 12, 2014 at 2:11 AM, Andrew Ash  wrote:Chen Song ...
   Author: Chen Song, 2014-08-12, 15:26
[expand - 1 more] - Re: increase parallelism of reading from hdfs - Spark - [mail # user]
...Thanks Paul. I will give a try.On Mon, Aug 11, 2014 at 1:11 PM, Paul Hamilton wrote:Chen Song ...
   Author: Chen Song, 2014-08-11, 19:55
spark streaming multiple file output paths - Spark - [mail # user]
...In Spark Streaming, is there a way to write output to different paths basedon the partition key? The saveAsTextFiles method will write output in thesame directory.For example, if the partiti...
   Author: Chen Song, 2014-08-07, 15:39
question on HIVE-5891 - Hive - [mail # user]
...I am using cdh5 distribution and It doesn't look like this jirahttps://issues.apache.org/jira/browse/HIVE-5891is backported into cdh 5.1.0.Is there a workaround to modify the query that is s...
   Author: Chen Song, 2014-08-04, 15:00
yarn container memory setting - Hadoop - [mail # user]
...I read a bit on documentation on yarn memory tuning and found thatIt is suggested to set mapreduce.map.java.opts = 0.8 *mapreduce.map.memory.mb.I am wondering why is 0.8, but not 0.9 or high...
   Author: Chen Song, 2014-07-21, 20:45
Sort:
project
Hive (14)
Spark (14)
HBase (3)
Hadoop (2)
Flume (1)
Kafka (1)
MapReduce (1)
type
mail # user (36)
date
last 7 days (1)
last 30 days (4)
last 90 days (15)
last 6 months (21)
last 9 months (36)
author
Ted Yu (1650)
Harsh J (1293)
Jun Rao (1037)
Todd Lipcon (1001)
Stack (973)
Jonathan Ellis (842)
Andrew Purtell (801)
Jean-Daniel Cryans (753)
jacques@... (738)
Yusaku Sako (719)
stack (716)
Jarek Jarcec Cecho (699)
Eric Newton (697)
Jonathan Hsieh (675)
Roman Shaposhnik (660)
Brock Noland (656)
Neha Narkhede (652)
Namit Jain (649)
Hitesh Shah (626)
Owen O'Malley (625)
Steve Loughran (615)
Siddharth Seth (614)
Josh Elser (570)
Eli Collins (545)
Arun C Murthy (543)
Chen Song