Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Hadoop on Isilon problem


+
Artem Ervits 2012-10-17, 19:18
+
Mohit Anchlia 2012-10-17, 19:59
+
Rita 2012-10-18, 00:23
Copy link to this message
-
Re: Hadoop on Isilon problem
Artem Ervits 2012-10-18, 02:27
With Isilon, there is no need for hdfs-site configuration file. Isilon takes care of replication, although you can certainly add hadoop replication. The biggest plus is the scalability for storage layer. We keep a lot of our data in Isilon and importing into hdfs will result in two locations of massive amounts of data. With this solution, we can read and write in one location. Performance wise, I cannot answer yet, so far our standard cluster runs jobs 3x faster than Isilon but we have many factors why it is so and results are inconclusive.
Artem Ervits
Data Analyst
New York Presbyterian Hospital

From: Rita [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, October 17, 2012 08:23 PM
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Subject: Re: Hadoop on Isilon problem

out of curiosity, what does running HDFS give you when running thru an Isilon cluster?
On Wed, Oct 17, 2012 at 3:59 PM, Mohit Anchlia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Look at the directory permissions?
On Wed, Oct 17, 2012 at 12:18 PM, Artem Ervits <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Anyone using Hadoop running on Isilon NAS? I am trying to submit a job with a user other than the one running Hadoop and I’m getting the following error:

Exception in thread "main" java.io.IOException: Permission denied
        at java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.io.File.checkAndCreate(File.java:1717)
        at java.io.File.createTempFile0(File.java:1738)
        at java.io.File.createTempFile(File.java:1815)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:115)
Any ideas?
Artem Ervits
Data Analyst
New York Presbyterian Hospital
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

--
--- Get your facts first, then you can distort them as you please.--
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
+
Harsh J 2012-10-17, 19:36
+
Artem Ervits 2012-10-18, 02:21
+
Artem Ervits 2012-10-18, 15:11
+
Harsh J 2012-10-18, 15:24