Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Review Request 14250: HIVE-4957: Restrict number of bit vectors, to prevent out of Java heap memory


Copy link to this message
-
Re: Review Request 14250: HIVE-4957: Restrict number of bit vectors, to prevent out of Java heap memory


> On Sept. 21, 2013, 2:27 a.m., Carl Steinbach wrote:
> > ql/src/test/results/clientnegative/compute_stats_long.q.out, line 29
> > <https://reviews.apache.org/r/14250/diff/2/?file=354858#file354858line29>
> >
> >     The error message isn't making it back to the user because it's getting generated at runtime on the cluster. Is it possible to bounds check this parameter at compile time instead?

Today, the UDF framework doesn't support validation of inputs at compile time. Given the current framework, this is the best we can do.
- Shreepadma
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14250/#review26313
-----------------------------------------------------------
On Sept. 20, 2013, 8:02 p.m., Shreepadma Venugopalan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14250/
> -----------------------------------------------------------
>
> (Updated Sept. 20, 2013, 8:02 p.m.)
>
>
> Review request for hive and Brock Noland.
>
>
> Bugs: HIVE-4957
>     https://issues.apache.org/jira/browse/HIVE-4957
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Restricts the number of bit vectors used by Flajolet-Martin distinct value estimator to 1024.
>
>
> Diffs
> -----
>
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java 63110bb
>   ql/src/test/queries/clientnegative/compute_stats_long.q PRE-CREATION
>   ql/src/test/results/clientnegative/compute_stats_long.q.out PRE-CREATION
>
> Diff: https://reviews.apache.org/r/14250/diff/
>
>
> Testing
> -------
>
> Adds a new negative test case.
>
>
> Thanks,
>
> Shreepadma Venugopalan
>
>