Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> How to delete Specific date data using hive QL?


+
Hamza Asad 2013-06-03, 05:50
+
Nitin Pawar 2013-06-03, 06:19
+
Hamza Asad 2013-06-03, 07:06
+
Nitin Pawar 2013-06-03, 08:27
+
Hamza Asad 2013-06-03, 09:19
+
Stephen Sprague 2013-06-03, 19:09
+
Hamza Asad 2013-06-04, 06:07
+
Nitin Pawar 2013-06-04, 06:40
+
Hamza Asad 2013-06-04, 07:11
+
Nitin Pawar 2013-06-04, 07:18
+
Hamza Asad 2013-06-04, 07:52
Copy link to this message
-
Re: How to delete Specific date data using hive QL?
Adding my two cents
If you are having an unpartitioned data/table and would like to partition it on some specific columns in source table, Use dynamic partition insert.
That would get the source data in separate partitions on a partitioned target table.

http://kickstarthadoop.blogspot.com/2011/06/how-to-speed-up-your-hive-queries-in.html

Regards
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: Hamza Asad <[EMAIL PROTECTED]>
Date: Tue, 4 Jun 2013 12:52:49
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: How to delete Specific date data using hive QL?

Thank u soooo much nitin for your help.. :)
On Tue, Jun 4, 2013 at 12:18 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:

> 1- Does partitioning improve performance?
> --Only if you make use of partitions in your queries (mostly in where
> clause to limit data to your query for a specific value of partitioned
> column)
>
> 2- Do i have to create partition table new or i can create partition on
> existing table by renaming that date column and add partition column
> event_date (the actual column name) ?
> you can not create partitions on already existing data unless the data is
> in partitioned directories on hdfs.
> I would recommend create a new table with partitioned columns.
> load data from old table into partitioned table
> dump old table
>
> 3- can i import data directly into partition table using sqoop command?
> you can import data directly into a partition.
>
> for exported data, you don't have to worry. it remains as it is
>
>
> On Tue, Jun 4, 2013 at 12:41 PM, Hamza Asad <[EMAIL PROTECTED]>wrote:
>
>> No i don't want to change my queries. I want that my queries work on same
>> table and partition does not change its schema.
>> and from schema i means schema on mysql (exported data).
>>
>> Few more things
>> 1- Does partitioning improve performance?
>> 2- Do i have to create partition table new or i can create partition on
>> existing table by renaming that date column and add partition column
>> event_date (the actual column name) ?
>> 3- can i import data directly into partition table using sqoop command?
>>
>>
>>
>>
>> On Tue, Jun 4, 2013 at 11:40 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>>
>>> partitioning of data in hive is more for the reasons on how you layout
>>> data in a well defined manner so that when you access your data , you
>>> request only for specific data by specifying the partition columns in where
>>> clause.
>>>
>>> to answer your question,
>>> do you have to change your queries? out of the box the queries should
>>> work as it is unless and until you are changing the table schema by
>>> removing/adding new columns.
>>> does the format change when you export data? if your select statement is
>>> not changing it will not change
>>> will table schema change? do you mean schema on hive or mysql ?
>>>
>>>
>>> On Tue, Jun 4, 2013 at 11:37 AM, Hamza Asad <[EMAIL PROTECTED]>wrote:
>>>
>>>> thats far more better :) ..
>>>> Please tell me few more things. Do i have to change my query if i
>>>> create table with partition on date? rest of the columns would be same as
>>>> it is? Also if i export that partitioned table to mysql, does schema of
>>>> that table would same as it was before partition?
>>>>
>>>>
>>>> On Tue, Jun 4, 2013 at 12:09 AM, Stephen Sprague <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> there is no delete semantic.
>>>>>
>>>>> you either partition on the data you want to drop and use drop
>>>>> partition (or drop table for the whole shebang) or you can do as Nitin
>>>>> suggests by selecting the inverse of the data you want to delete and store
>>>>> it back into the table itself.  Not ideal but maybe it could work for your
>>>>> situation.
>>>>>
>>>>> Now here's another idea.  This was just _recently_ discussed on this
>>>>> group as coincidence would have it.  if you were to have scanned just a
>>>>> little of the groups messages you would have seen that and could then have

*Muhammad Hamza Asad*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB