Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Distinct errors


Copy link to this message
-
Re: Distinct errors
I have found that this error also occurs when running in pseudo-distributed
mode. The reason most of the time is the default value of the scratch
directory configuration which is /tmp/hive-${user.name}. Since /tmp gets
automatically cleared up on some systems it leads to this problem.

The patch provided with this
Jira<https://issues.apache.org/jira/browse/HIVE-1277>addresses this by
attempting to automatically create the scratch directory
if it does not yet exist.

Arvind

On Thu, Mar 25, 2010 at 9:58 AM, Yongqiang He <
[EMAIL PROTECTED]> wrote:

>  This error often occurs when you are running job in local mode.
> If you see this error, you can first do a “set
> hive.exec.scratchdir=/tmp/create_this_dir_first;”.
>
> Thanks
> Yongqiang
>
> On 2/26/10 10:13 AM, "Peter Sankauskas" <[EMAIL PROTECTED]> wrote:
>
> Looks like the same error as before... the temp directory is missing.
>
> Kind regards,
> Peter Sankauskas
>
> Motally, Inc
> Office: +1 (415) 932-6898
>
>
> On Fri, Feb 26, 2010 at 5:33 AM, Aryeh Berkowitz <[EMAIL PROTECTED]>
> wrote:
>
> When I try to do a SELECT DISTINCT, I get “No such file” errors.
>
> hive> SELECT DISTINCT URL FROM URLS;
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> java.io.IOException: No such file or directory
>         at java.io.UnixFileSystem.createFileExclusively(Native Method)
>         at java.io.File.checkAndCreate(File.java:1704)
>         at java.io.File.createTempFile(File.java:1792)
>         at
> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:87)
>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
>         at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:630)
>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
>         at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
>         at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
>
>
>
>