Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> load data stored as sequencefiles


+
Artem Ervits 2013-09-20, 15:17
+
Artem Ervits 2013-09-24, 14:21
+
Swagatika Tripathy 2013-09-24, 17:06
Copy link to this message
-
Re: load data stored as sequencefiles
If you look at your load command,

LOAD DATA INPATH '/TEST/SeqFiles/201308300700/part-m-00001' INTO TABLE
tblname;
you are loading a part file which does not look correct.

Secondly,
Why can't you just import using sqoop. Why you have to do load data?
If you are importing to hdfs using sqoop, and then loading data into hive
table, then you may want to give complete file name instead of part file in
load command
On Tue, Sep 24, 2013 at 7:51 PM, Artem Ervits <[EMAIL PROTECTED]> wrote:

>  Anyone?****
>
> ** **
>
> *From:* Artem Ervits [mailto:[EMAIL PROTECTED]]
> *Sent:* Friday, September 20, 2013 11:18 AM
> *To:* [EMAIL PROTECTED]
> *Subject:* load data stored as sequencefiles****
>
> ** **
>
> Hello all,****
>
>  ****
>
> I’m a bit lost with using Hive and SequenceFiles. I loaded data using
> Sqoop from a RDBMS and stored as sequencefile. I jarred the class generated
> by sqoop and added it to my create table script. Now I create a table in
> hive and specify “STORED AS SEQUENCEFILE”, I also “ADD JAR
> SQOOP_GENERATED.JAR”. Then I try to insert data with the same generated jar
> added. I also specify ****
>
>  ****
>
> SET hive.exec.compress.output=true;****
>
> SET io.seqfile.compression.type=BLOCK;****
>
>  ****
>
> LOAD DATA INPATH '/TEST/SeqFiles/201308300700/part-m-00001' INTO TABLE
> tblname;****
>
>  ****
>
> When the query executes, I see this “[num_partitions: 0, num_files: 2,
> num_rows: 0, total_size: 478662618, raw_data_size: 0]”****
>
>  ****
>
> When I select on the table,  I get org.apache.hadoop.hive.serde2.SerDeException:
> class org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe: expects either
> BytesWritable or Text object! ****
>
>  ****
>
> So my question is, how do I specify my generated class along with
> SequenceFileInputFormat in my create statement? How do I specify the
> inputformats?****
>
>
> This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
> If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited. If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message. Thank you.****
>  ------------------------------
>
>
> Confidential Information subject to NYP's (and its affiliates')
> information management and security policies (
> http://infonet.nyp.org/QA/HospitalManual).****
>
> This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
> If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited. If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message. Thank you.
>

--
Nitin Pawar
+
Artem Ervits 2013-09-24, 16:59
+
Artem Ervits 2013-09-24, 17:01
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB