Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Problem with the windowing function ntile (Exceptions)


Copy link to this message
-
Re: Problem with the windowing function ntile (Exceptions)
Hi Navis,

that's awesome! Thank you for the quick response and the patch. I just
tested it against my query which used to fail and it just worked. So
you have at least my thumbs-up.

Thanks again for your help.

Cheers,
Lars

On Thu, Jul 25, 2013 at 10:01 AM, Navis류승우 <[EMAIL PROTECTED]> wrote:
> I've booked this and attached patch for it.
>
> https://issues.apache.org/jira/browse/HIVE-4932
>
> Could you test with that? thanks.
>
> 2013/7/25 Lars Francke <[EMAIL PROTECTED]>:
>> We're still being bitten by this problem without a workaround. Does
>> anyone have an idea?
>>
>> Thanks,
>> Lars
>>
>> On Wed, Jul 17, 2013 at 11:24 PM, Lars Francke <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>> I'm running a query like this:
>>>
>>> CREATE TABLE foo
>>>   STORED AS ORC
>>> AS
>>> SELECT
>>>   id,
>>>   season,
>>>   amount,
>>>   ntile(10)
>>> OVER (
>>>   PARTITION BY season
>>>   ORDER BY amount DESC
>>> )
>>> FROM bar;
>>>
>>> On a small enough dataset that works fine but when switching to a
>>> larger sample we're seeing exceptions like this:
>>>
>>> "Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Reset on
>>> PersistentByteBasedList not supported"
>>>
>>> Looking at the code (without really understanding it) we tried setting:
>>> SET hive.ptf.partition.persistence='org.apache.hadoop.hive.ql.exec.PTFPersistence$PartitionedByteBasedList';
>>>
>>> because that List supports reset but we are seeing a
>>> ClassNotFoundException so we're doing that wrong.
>>>
>>> Next try was setting hive.ptf.partition.persistence.memsize higher
>>> which worked but first of all we don't really understand what all of
>>> that stuff is doing and second of all we fear that it just might break
>>> down again.
>>>
>>> Any hints as to what that error really means and how to deal with it
>>> would be greatly appreciated.
>>>
>>> Thanks!
>>>
>>> Lars