Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> How to load /t /n file to Hive


Copy link to this message
-
Re: How to load /t /n file to Hive
Hi Gabo,

Are you suggesting to use java.net.URLEncoder ? Can you be more specific ? I have lot of fields in the file which are not only URL related but some text fields which has new line characters.

Thanks,
Raj
________________________________
 From: Gabriel Eisbruch <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; Raj Hadoop <[EMAIL PROTECTED]>
Sent: Friday, September 20, 2013 4:43 PM
Subject: Re: How to load /t /n file to Hive
 
Hi 
 One way that we used to solve that problem it's to transform the data when you are creating/loading it, for example we've applied UrlEncode to each field on create time.

Thanks,
Gabo.

2013/9/20 Raj Hadoop <[EMAIL PROTECTED]>

Hi Nitin,

>Thanks for the reply. I have a huge file in unix.

>As per the file definition, the file is a tab separated file of fields. But I am sure that within some field's I have some new line character.

>How should I find a record? It is a huge file. Is there some command?

>Thanks,

>
>
>From: Nitin Pawar <[EMAIL PROTECTED]>
>To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; Raj Hadoop <[EMAIL PROTECTED]>
>Sent: Friday, September 20, 2013 3:15 PM
>Subject: Re: How to load /t /n file to Hive
>
>
>
>If your data contains new line chars, its better you write a custom map reduce job and convert the data into a single line removing all unwanted chars in column separator as well just having single new line char per line 
>
>
>
>On Sat, Sep 21, 2013 at 12:38 AM, Raj Hadoop <[EMAIL PROTECTED]> wrote:
>
>Please note that there is an escape chacter in the fields where the /t and /n are present.
>>
>>
>>
>>From: Raj Hadoop <[EMAIL PROTECTED]>
>>To: Hive <[EMAIL PROTECTED]>
>>Sent: Friday, September 20, 2013 3:04 PM
>>Subject: How to load /t /n file to Hive
>>
>>
>>
>>Hi,
>> 
>>I have a file which is delimted by a tab. Also, there are some fields in the file which has a tab /t character and a new line /n character in some fields.
>> 
>>Is there any way to load this file using Hive load command? Or do i have to use a Custom Map Reduce (custom) Input format with java ? Please advise.
>> 
>>Thanks,
>>Raj
>>
>>
>
>
>
>--
>Nitin Pawar
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB