Anyone using Hadoop running on Isilon NAS? I am trying to submit a job with a user other than the one running Hadoop and I'm getting the following error:

Exception in thread "main" java.io.IOException: Permission denied
        at java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.io.File.checkAndCreate(File.java:1717)
        at java.io.File.createTempFile0(File.java:1738)
        at java.io.File.createTempFile(File.java:1815)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:115)
Any ideas?
Artem Ervits
Data Analyst
New York Presbyterian Hospital

--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

Hi Artem,

I don't think this is a Isilon issue as much as its a /tmp directory
permission issue. Ensure that /tmp has proper world+rw permissions, OR
if thats not doable, set the JVM system property of -Djava.io.tmpdir
to a local user writable directory and also re-point config
hadoop.tmp.dir to such a thing thats non /tmp but writable by the user
(/home/${user.name}/.tmp maybe).

Of course, easiest is just to fix the /tmp to be drwxrwxrwt,

On Thu, Oct 18, 2012 at 12:48 AM, Artem Ervits <[EMAIL PROTECTED]> wrote:
> Anyone using Hadoop running on Isilon NAS? I am trying to submit a job with
> a user other than the one running Hadoop and I’m getting the following
> error:
>
>
>
> Exception in thread "main" java.io.IOException: Permission denied
>
>         at java.io.UnixFileSystem.createFileExclusively(Native Method)
>
>         at java.io.File.checkAndCreate(File.java:1717)
>
>         at java.io.File.createTempFile0(File.java:1738)
>
>         at java.io.File.createTempFile(File.java:1815)
>
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:115)
>
>
>
>
>
> Any ideas?
>
>
>
>
>
> Artem Ervits
>
> Data Analyst
>
> New York Presbyterian Hospital
>
>
>
> --------------------
>
> This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
> If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited.  If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message.  Thank you.
>
> --------------------
>
> This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
> If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited.  If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message.  Thank you.
>
>

--
Harsh J

Look at the directory permissions?

On Wed, Oct 17, 2012 at 12:18 PM, Artem Ervits <[EMAIL PROTECTED]> wrote:

>  Anyone using Hadoop running on Isilon NAS? I am trying to submit a job
> with a user other than the one running Hadoop and I’m getting the following
> error:****
>
> ** **
>
> Exception in thread "main" java.io.IOException: Permission denied****
>
>         at java.io.UnixFileSystem.createFileExclusively(Native Method)****
>
>         at java.io.File.checkAndCreate(File.java:1717)****
>
>         at java.io.File.createTempFile0(File.java:1738)****
>
>         at java.io.File.createTempFile(File.java:1815)****
>
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:115)****
>
> ** **
>
> ** **
>
> Any ideas?****
>
> ** **
>
> ** **
>
> Artem Ervits****
>
> Data Analyst****
>
> New York Presbyterian Hospital****
>
> ** **
>
> --------------------
>
> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
>
>
> --------------------
>
> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
>
>
>
>

out of curiosity, what does running HDFS give you when running thru an
Isilon cluster?
On Wed, Oct 17, 2012 at 3:59 PM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> Look at the directory permissions?
>
>
> On Wed, Oct 17, 2012 at 12:18 PM, Artem Ervits <[EMAIL PROTECTED]> wrote:
>
>>  Anyone using Hadoop running on Isilon NAS? I am trying to submit a job
>> with a user other than the one running Hadoop and I’m getting the following
>> error:****
>>
>> ** **
>>
>> Exception in thread "main" java.io.IOException: Permission denied****
>>
>>         at java.io.UnixFileSystem.createFileExclusively(Native Method)***
>> *
>>
>>         at java.io.File.checkAndCreate(File.java:1717)****
>>
>>         at java.io.File.createTempFile0(File.java:1738)****
>>
>>         at java.io.File.createTempFile(File.java:1815)****
>>
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:115)****
>>
>> ** **
>>
>> ** **
>>
>> Any ideas?****
>>
>> ** **
>>
>> ** **
>>
>> Artem Ervits****
>>
>> Data Analyst****
>>
>> New York Presbyterian Hospital****
>>
>> ** **
>>
>> --------------------
>>
>> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
>>
>>
>> --------------------
>>
>> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
>>
>>
>>
>>
>
--
--- Get your facts first, then you can distort them as you please.--

I will try that first thing in the morning, thank you very much!
Artem Ervits
Data Analyst
New York Presbyterian Hospital

----- Original Message -----
From: Harsh J [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, October 17, 2012 03:36 PM
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Subject: Re: Hadoop on Isilon problem

Hi Artem,

I don't think this is a Isilon issue as much as its a /tmp directory
permission issue. Ensure that /tmp has proper world+rw permissions, OR
if thats not doable, set the JVM system property of -Djava.io.tmpdir
to a local user writable directory and also re-point config
hadoop.tmp.dir to such a thing thats non /tmp but writable by the user
(/home/${user.name}/.tmp maybe).

Of course, easiest is just to fix the /tmp to be drwxrwxrwt,

On Thu, Oct 18, 2012 at 12:48 AM, Artem Ervits <[EMAIL PROTECTED]> wrote:
> Anyone using Hadoop running on Isilon NAS? I am trying to submit a job with
> a user other than the one running Hadoop and I’m getting the following
> error:
>
>
>
> Exception in thread "main" java.io.IOException: Permission denied
>
>         at java.io.UnixFileSystem.createFileExclusively(Native Method)
>
>         at java.io.File.checkAndCreate(File.java:1717)
>
>         at java.io.File.createTempFile0(File.java:1738)
>
>         at java.io.File.createTempFile(File.java:1815)
>
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:115)
>
>
>
>
>
> Any ideas?
>
>
>
>
>
> Artem Ervits
>
> Data Analyst
>
> New York Presbyterian Hospital
>
>
>
> --------------------
>
> This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
> If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited.  If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message.  Thank you.
>
> --------------------
>
> This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
> If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited.  If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message.  Thank you.
>
>

--
Harsh J
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

With Isilon, there is no need for hdfs-site configuration file. Isilon takes care of replication, although you can certainly add hadoop replication. The biggest plus is the scalability for storage layer. We keep a lot of our data in Isilon and importing into hdfs will result in two locations of massive amounts of data. With this solution, we can read and write in one location. Performance wise, I cannot answer yet, so far our standard cluster runs jobs 3x faster than Isilon but we have many factors why it is so and results are inconclusive.
Artem Ervits
Data Analyst
New York Presbyterian Hospital

From: Rita [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, October 17, 2012 08:23 PM
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Subject: Re: Hadoop on Isilon problem

out of curiosity, what does running HDFS give you when running thru an Isilon cluster?
On Wed, Oct 17, 2012 at 3:59 PM, Mohit Anchlia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Look at the directory permissions?
On Wed, Oct 17, 2012 at 12:18 PM, Artem Ervits <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Anyone using Hadoop running on Isilon NAS? I am trying to submit a job with a user other than the one running Hadoop and I’m getting the following error:

Exception in thread "main" java.io.IOException: Permission denied
        at java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.io.File.checkAndCreate(File.java:1717)
        at java.io.File.createTempFile0(File.java:1738)
        at java.io.File.createTempFile(File.java:1815)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:115)
Any ideas?
Artem Ervits
Data Analyst
New York Presbyterian Hospital
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

--
--- Get your facts first, then you can distort them as you please.--
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

Yup, that was it. I confused this tmp with another tmp we created before.

Thank you.

-----Original Message-----
From: Harsh J [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, October 17, 2012 3:36 PM
To: [EMAIL PROTECTED]
Subject: Re: Hadoop on Isilon problem

Hi Artem,

I don't think this is a Isilon issue as much as its a /tmp directory permission issue. Ensure that /tmp has proper world+rw permissions, OR if thats not doable, set the JVM system property of -Djava.io.tmpdir to a local user writable directory and also re-point config hadoop.tmp.dir to such a thing thats non /tmp but writable by the user (/home/${user.name}/.tmp maybe).

Of course, easiest is just to fix the /tmp to be drwxrwxrwt,

On Thu, Oct 18, 2012 at 12:48 AM, Artem Ervits <[EMAIL PROTECTED]> wrote:
> Anyone using Hadoop running on Isilon NAS? I am trying to submit a job
> with a user other than the one running Hadoop and I'm getting the
> following
> error:
>
>
>
> Exception in thread "main" java.io.IOException: Permission denied
>
>         at java.io.UnixFileSystem.createFileExclusively(Native Method)
>
>         at java.io.File.checkAndCreate(File.java:1717)
>
>         at java.io.File.createTempFile0(File.java:1738)
>
>         at java.io.File.createTempFile(File.java:1815)
>
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:115)
>
>
>
>
>
> Any ideas?
>
>
>
>
>
> Artem Ervits
>
> Data Analyst
>
> New York Presbyterian Hospital
>
>
>
> --------------------
>
> This electronic message is intended to be for the use only of the
> named recipient, and may contain information that is confidential or privileged.
> If you are not the intended recipient, you are hereby notified that
> any disclosure, copying, distribution or use of the contents of this
> message is strictly prohibited.  If you have received this message in
> error or are not the named recipient, please notify us immediately by
> contacting the sender at the electronic mail address noted above, and
> delete and destroy all copies of this message.  Thank you.
>
> --------------------
>
> This electronic message is intended to be for the use only of the
> named recipient, and may contain information that is confidential or privileged.
> If you are not the intended recipient, you are hereby notified that
> any disclosure, copying, distribution or use of the contents of this
> message is strictly prohibited.  If you have received this message in
> error or are not the named recipient, please notify us immediately by
> contacting the sender at the electronic mail address noted above, and
> delete and destroy all copies of this message.  Thank you.
>
>

--
Harsh J
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

No problem, thanks for closing the loop!

On Thu, Oct 18, 2012 at 8:41 PM, Artem Ervits <[EMAIL PROTECTED]> wrote:
> Yup, that was it. I confused this tmp with another tmp we created before.
>
> Thank you.
>
> -----Original Message-----
> From: Harsh J [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, October 17, 2012 3:36 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Hadoop on Isilon problem
>
> Hi Artem,
>
> I don't think this is a Isilon issue as much as its a /tmp directory permission issue. Ensure that /tmp has proper world+rw permissions, OR if thats not doable, set the JVM system property of -Djava.io.tmpdir to a local user writable directory and also re-point config hadoop.tmp.dir to such a thing thats non /tmp but writable by the user (/home/${user.name}/.tmp maybe).
>
> Of course, easiest is just to fix the /tmp to be drwxrwxrwt,
>
> On Thu, Oct 18, 2012 at 12:48 AM, Artem Ervits <[EMAIL PROTECTED]> wrote:
>> Anyone using Hadoop running on Isilon NAS? I am trying to submit a job
>> with a user other than the one running Hadoop and I'm getting the
>> following
>> error:
>>
>>
>>
>> Exception in thread "main" java.io.IOException: Permission denied
>>
>>         at java.io.UnixFileSystem.createFileExclusively(Native Method)
>>
>>         at java.io.File.checkAndCreate(File.java:1717)
>>
>>         at java.io.File.createTempFile0(File.java:1738)
>>
>>         at java.io.File.createTempFile(File.java:1815)
>>
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:115)
>>
>>
>>
>>
>>
>> Any ideas?
>>
>>
>>
>>
>>
>> Artem Ervits
>>
>> Data Analyst
>>
>> New York Presbyterian Hospital
>>
>>
>>
>> --------------------
>>
>> This electronic message is intended to be for the use only of the
>> named recipient, and may contain information that is confidential or privileged.
>> If you are not the intended recipient, you are hereby notified that
>> any disclosure, copying, distribution or use of the contents of this
>> message is strictly prohibited.  If you have received this message in
>> error or are not the named recipient, please notify us immediately by
>> contacting the sender at the electronic mail address noted above, and
>> delete and destroy all copies of this message.  Thank you.
>>
>> --------------------
>>
>> This electronic message is intended to be for the use only of the
>> named recipient, and may contain information that is confidential or privileged.
>> If you are not the intended recipient, you are hereby notified that
>> any disclosure, copying, distribution or use of the contents of this
>> message is strictly prohibited.  If you have received this message in
>> error or are not the named recipient, please notify us immediately by
>> contacting the sender at the electronic mail address noted above, and
>> delete and destroy all copies of this message.  Thank you.
>>
>>
>
>
>
> --
> Harsh J
>
>
> --------------------
>
> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
>
>
>
>
> --------------------
>
> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

Harsh J