Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Hive error when loading csv data.


+
Sandeep Reddy P 2012-06-26, 19:07
+
Harsh J 2012-06-26, 19:51
+
Michael Segel 2012-06-26, 20:00
+
Sandeep Reddy P 2012-06-26, 21:30
Copy link to this message
-
Re: Hive error when loading csv data.
Michael's suggestion was to change your data to:

c|zxy|xyz
d|abc,def|abcd

and then use "|" as the delimiter.

-- Hitesh

On Jun 26, 2012, at 2:30 PM, Sandeep Reddy P wrote:

> Thanks for the reply.
> I didnt get that Michael. My f2 should be "abc,def"
>
> On Tue, Jun 26, 2012 at 4:00 PM, Michael Segel <[EMAIL PROTECTED]>wrote:
>
>> Alternatively you could write a simple script to convert the csv to a pipe
>> delimited file so that "abc,def" will be abc,def.
>>
>> On Jun 26, 2012, at 2:51 PM, Harsh J wrote:
>>
>>> Hive's delimited-fields-format record reader does not handle quoted
>>> text that carry the same delimiter within them. Excel supports such
>>> records, so it reads it fine.
>>>
>>> You will need to create your table with a custom InputFormat class
>>> that can handle this (Try using OpenCSV readers, they support this),
>>> instead of relying on Hive to do this for you. If you're successful in
>>> your approach, please also consider contributing something back to
>>> Hive/Pig to help others.
>>>
>>> On Wed, Jun 27, 2012 at 12:37 AM, Sandeep Reddy P
>>> <[EMAIL PROTECTED]> wrote:
>>>>
>>>>
>>>> Hi all,
>>>> I have a csv file with 46 columns but i'm getting error when i do some
>>>> analysis on that data type. For simplification i have taken 3 columns
>> and
>>>> now my csv is like
>>>> c,zxy,xyz
>>>> d,"abc,def",abcd
>>>>
>>>> i have created table for this data using,
>>>> hive> create table test3(
>>>>> f1 string,
>>>>> f2 string,
>>>>> f3 string)
>>>>> row format delimited
>>>>> fields terminated by ",";
>>>> OK
>>>> Time taken: 0.143 seconds
>>>> hive> load data local inpath '/home/training/a.csv'
>>>>> into table test3;
>>>> Copying data from file:/home/training/a.csv
>>>> Copying file: file:/home/training/a.csv
>>>> Loading data to table default.test3
>>>> OK
>>>> Time taken: 0.276 seconds
>>>> hive> select * from test3;
>>>> OK
>>>> c       zxy     xyz
>>>> d       "abc    def"
>>>> Time taken: 0.156 seconds
>>>>
>>>> When i do select f2 from test3;
>>>> my results are,
>>>> OK
>>>> zxy
>>>> "abc
>>>> but this should be abc,def
>>>> When i open the same csv file with Microsoft Excel i got abc,def
>>>> How should i solve this error??
>>>>
>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> sandeep
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>>
>
>
> --
> Thanks,
> sandeep
+
Michel Segel 2012-06-26, 22:48
+
Sandeep Reddy P 2012-06-27, 01:58
+
Michel Segel 2012-06-27, 02:13
+
Sandeep Reddy P 2012-06-27, 02:52
+
Michel Segel 2012-06-27, 02:11
+
ramakanth reddy 2012-06-27, 12:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB