Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - How to delete Specific date data using hive QL?


Copy link to this message
-
Re: How to delete Specific date data using hive QL?
Hamza Asad 2013-06-04, 07:11
No i don't want to change my queries. I want that my queries work on same
table and partition does not change its schema.
and from schema i means schema on mysql (exported data).

Few more things
1- Does partitioning improve performance?
2- Do i have to create partition table new or i can create partition on
existing table by renaming that date column and add partition column
event_date (the actual column name) ?
3- can i import data directly into partition table using sqoop command?
On Tue, Jun 4, 2013 at 11:40 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:

> partitioning of data in hive is more for the reasons on how you layout
> data in a well defined manner so that when you access your data , you
> request only for specific data by specifying the partition columns in where
> clause.
>
> to answer your question,
> do you have to change your queries? out of the box the queries should work
> as it is unless and until you are changing the table schema by
> removing/adding new columns.
> does the format change when you export data? if your select statement is
> not changing it will not change
> will table schema change? do you mean schema on hive or mysql ?
>
>
> On Tue, Jun 4, 2013 at 11:37 AM, Hamza Asad <[EMAIL PROTECTED]>wrote:
>
>> thats far more better :) ..
>> Please tell me few more things. Do i have to change my query if i create
>> table with partition on date? rest of the columns would be same as it is?
>> Also if i export that partitioned table to mysql, does schema of that table
>> would same as it was before partition?
>>
>>
>> On Tue, Jun 4, 2013 at 12:09 AM, Stephen Sprague <[EMAIL PROTECTED]>wrote:
>>
>>> there is no delete semantic.
>>>
>>> you either partition on the data you want to drop and use drop partition
>>> (or drop table for the whole shebang) or you can do as Nitin suggests by
>>> selecting the inverse of the data you want to delete and store it back into
>>> the table itself.  Not ideal but maybe it could work for your situation.
>>>
>>> Now here's another idea.  This was just _recently_ discussed on this
>>> group as coincidence would have it.  if you were to have scanned just a
>>> little of the groups messages you would have seen that and could then have
>>> added to the discussion! :)
>>>
>>>
>>> On Mon, Jun 3, 2013 at 2:19 AM, Hamza Asad <[EMAIL PROTECTED]>wrote:
>>>
>>>> Thanx for your response nitin. Anybody else have any better solution?
>>>>
>>>>
>>>> On Mon, Jun 3, 2013 at 1:27 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> hive does not give you a record level deletion as of now.
>>>>>
>>>>> so unless you have partitioned, other option is you overwrite the
>>>>> table with data which you want
>>>>> please wait for others to suggest you more options. this one is just
>>>>> mine and can be costly too
>>>>>
>>>>>
>>>>> On Mon, Jun 3, 2013 at 12:36 PM, Hamza Asad <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> no, its not partitioned by date.
>>>>>>
>>>>>>
>>>>>> On Mon, Jun 3, 2013 at 11:19 AM, Nitin Pawar <[EMAIL PROTECTED]
>>>>>> > wrote:
>>>>>>
>>>>>>> how is the data laid out?
>>>>>>> is it partitioned data by the date?
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 3, 2013 at 11:20 AM, Hamza Asad <[EMAIL PROTECTED]>wrote:
>>>>>>>
>>>>>>>> Dear all,
>>>>>>>>             How can i remove data of specific dates from HDFS using
>>>>>>>> hive query language?
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Muhammad Hamza Asad*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Nitin Pawar
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Muhammad Hamza Asad*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nitin Pawar
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Muhammad Hamza Asad*
>>>>
>>>
>>>
>>
>>
>> --
>> *Muhammad Hamza Asad*
>>
>
>
>
> --
> Nitin Pawar
>

--
*Muhammad Hamza Asad*