Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Distinct errors


Copy link to this message
-
Re: Distinct errors
I have found that this error also occurs when running in pseudo-distributed
mode. The reason most of the time is the default value of the scratch
directory configuration which is /tmp/hive-${user.name}. Since /tmp gets
automatically cleared up on some systems it leads to this problem.

The patch provided with this
Jira<https://issues.apache.org/jira/browse/HIVE-1277>addresses this by
attempting to automatically create the scratch directory
if it does not yet exist.

Arvind

On Thu, Mar 25, 2010 at 9:58 AM, Yongqiang He <
[EMAIL PROTECTED]> wrote:

>  This error often occurs when you are running job in local mode.
> If you see this error, you can first do a “set
> hive.exec.scratchdir=/tmp/create_this_dir_first;”.
>
> Thanks
> Yongqiang
>
> On 2/26/10 10:13 AM, "Peter Sankauskas" <[EMAIL PROTECTED]> wrote:
>
> Looks like the same error as before... the temp directory is missing.
>
> Kind regards,
> Peter Sankauskas
>
> Motally, Inc
> Office: +1 (415) 932-6898
>
>
> On Fri, Feb 26, 2010 at 5:33 AM, Aryeh Berkowitz <[EMAIL PROTECTED]>
> wrote:
>
> When I try to do a SELECT DISTINCT, I get “No such file” errors.
>
> hive> SELECT DISTINCT URL FROM URLS;
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> java.io.IOException: No such file or directory
>         at java.io.UnixFileSystem.createFileExclusively(Native Method)
>         at java.io.File.checkAndCreate(File.java:1704)
>         at java.io.File.createTempFile(File.java:1792)
>         at
> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:87)
>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
>         at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:55)
>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:630)
>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:504)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:382)
>         at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:138)
>         at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:197)
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:303)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB