Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
clear query|facets|time Search criteria: .   Results from 1 to 10 from 64 (0.233s).
Loading phrases to help you
refine your search...
UNION two RDDs - Spark - [mail # user]
...Hi Spark users,I wonder if val resultRDD = RDDA.union(RDDB) will always have records inRDDA before records in RDDB.Also, will resultRDD.coalesce(1) change this ordering?Best Regards,Jerry&nb...
   Author: Jerry Lam, 2014-12-18, 20:53
[expand - 2 more] - Re: Spark SQL 1.1.1 reading LZO compressed json files - Spark - [mail # user]
...Hi Michael,This is what I did. I was thinking if there is a more efficient way toaccomplish this.I was doing a very simple benchmark: Convert lzo compressed json files toparquet files using ...
   Author: Jerry Lam, 2014-12-17, 19:02
[expand - 1 more] - Re: Accessing rows of a row in Spark - Spark - [mail # user]
...Hi Mark,Thank you for helping out.The items I got back from Spark SQL has the type information as follows:scala> itemsres16: org.apache.spark.sql.Row = [WrappedArray([1,orange],[2,apple])...
   Author: Jerry Lam, 2014-12-15, 19:49
Filtering nested data using Spark SQL - Spark - [mail # user]
...Hi spark users,I'm trying to filter a json file that has the following schema using SparkSQL:root |-- user_id: string (nullable = true) |-- item: array (nullable = true) | &nb...
   Author: Jerry Lam, 2014-12-10, 23:12
[expand - 2 more] - Re: How to change MAX_FILES_PER_REGION_PER_FAMILY in LoadIncrementalHFiles? - HBase - [mail # user]
...Hi Matteo,Thank you for addressing the issue. For now, I will just set the variablein hbase-site.xml.Best Regards,JerryOn Wed, Aug 20, 2014 at 12:33 PM, Matteo Bertozzi wrote: ...
   Author: Jerry Lam, 2014-08-20, 17:24
[expand - 1 more] - Re: Spark SQL and Hive tables - Spark - [mail # user]
...Hi Sameer,The blog post you referred to is about Spark SQL. I don't think the intentof the article is meant to guide you how to read data from Hive via SparkSQL. So don't worry too much abou...
   Author: Jerry Lam, 2014-07-25, 21:48
[expand - 2 more] - Re: Repeated data item search with Spark SQL(1.0.1) - Spark - [mail # user]
...Hi Michael,Thank you for the explanation. Can you validate the following statement istrue/incomplete/false:"hql uses Hive to parse and to construct the logical plan whereas sql ispure spark ...
   Author: Jerry Lam, 2014-07-16, 18:14
[expand - 2 more] - Re: Need help on spark Hbase - Spark - [mail # user]
...Hi Rajesh,I saw : Warning: Local jar /home/rajesh/hbase-0.96.1.1-hadoop2/lib/hbase-client-0.96.1.1-hadoop2.jar, does not exist, skipping.in your log.I believe this jar contains the HBaseConf...
   Author: Jerry Lam, 2014-07-16, 17:56
[expand - 3 more] - Re: How to kill running spark yarn application - Spark - [mail # user]
...For your information, the SparkSubmit runs at the host you executed thespark-submit shell script (which in turns invoke the SparkSubmit program).Since you are running in yarn-cluster mode, t...
   Author: Jerry Lam, 2014-07-15, 14:56
[SPARK-2448] Table name is not getting applied to their attributes after "registerAsTable" - Spark - [issue]
...The following sample code will fail:val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)import hiveContext._hql("USE test")hql("select id from m").registerAsTable("m")hql("select ...
http://issues.apache.org/jira/browse/SPARK-2448    Author: Jerry Lam, 2014-07-14, 20:57
Sort:
project
HBase (35)
Spark (15)
Pig (11)
MapReduce (3)
type
mail # user (62)
issue (2)
date
last 7 days (3)
last 30 days (4)
last 90 days (4)
last 6 months (16)
last 9 months (64)
author
Ted Yu (1835)
Harsh J (1303)
Jun Rao (1014)
Todd Lipcon (994)
Stack (987)
Andrew Purtell (875)
Jonathan Ellis (854)
stack (758)
Jean-Daniel Cryans (751)
Jarek Jarcec Cecho (747)
Yusaku Sako (743)
Eric Newton (706)
Jonathan Hsieh (683)
Hitesh Shah (680)
Roman Shaposhnik (677)
Josh Elser (673)
Steve Loughran (651)
Namit Jain (648)
Siddharth Seth (643)
Brock Noland (634)
Owen O'Malley (623)
Hyunsik Choi (582)
Neha Narkhede (566)
Arun C Murthy (548)
Eli Collins (545)
Jerry Lam
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB