Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Re: Percentile calculation


Copy link to this message
-
Re: Percentile calculation
MiaoMiao 2012-10-02, 03:10
More info, please.

On Mon, Oct 1, 2012 at 4:50 PM, Mayank Bansal
<[EMAIL PROTECTED]> wrote:
> Hi,
>
>
>
> I am trying to run the hive udf percentile, I am trying to run it on a
> column with something around 116 million unique values.
>
> The maximum space that I can give to the reducer is 12 GB, the job keeps on
> failing due to java heap space error.
>
> Is there a way to optimize this, so that I don’t encounter this error?
>
> Or any other suggestion or solution which could help me out?
>
>
>
> Thanks,
>
> Mayank
>
>
> ________________________________
> This email message may contain proprietary, private and confidential
> information. The information transmitted is intended only for the person(s)
> or entities to which it is addressed. Any review, retransmission,
> dissemination or other use of, or taking of any action in reliance upon,
> this information by persons or entities other than the intended recipient is
> prohibited and may be illegal. If you received this in error, please contact
> the sender and delete the message from your system.
>
> Mu Sigma takes all reasonable steps to ensure that its electronic
> communications are free from viruses. However, given Internet accessibility,
> the Company cannot accept liability for any virus introduced by this e-mail
> or any attachment and you are advised to use up-to-date virus checking
> software.