Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> REPOST: How to adapt bin/hdfs for executing from outside $HADOOP_HOME/bin?


Copy link to this message
-
Re: REPOST: How to adapt bin/hdfs for executing from outside $HADOOP_HOME/bin?
The hadoop script in 0.20.x probably has all of the features that the
hdfs script in 0.21.x has (plus the functionality of the mapred
script). Give this a try and see if you get any errors.

-Joey

On Thu, May 19, 2011 at 8:31 AM, Gabriele Kahlout
<[EMAIL PROTECTED]> wrote:
> that simple? No changes to hdfs-config.sh?
>
> What about all the other stuff in the hdfs?
> For example the script calls hdfs dfs , like that won't it crash?
>
> elif [ "$COMMAND" = "dfs" ] ; then
>  CLASS=org.apache.hadoop.fs.FsShell
>
> On Thu, May 19, 2011 at 5:26 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
>
>> I would just write your own hdfs script that has the following:
>>
>> #!/bin/sh
>>
>> export HADOOP_HOME=/path/to/hadoop
>> exec ${HADOOP_HOME}/bin/hadoop "$@"
>>
>> -Joey
>>
>> On Thu, May 19, 2011 at 8:10 AM, Gabriele Kahlout
>> <[EMAIL PROTECTED]> wrote:
>> > because I've an immutable script written for hadoop that uses hdfs.
>> >
>> > On Thu, May 19, 2011 at 5:02 PM, Joey Echeverria <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> Why do you need the hdfs script? Typically 0.20.x is used with just the
>> >> hadoop script.
>> >>
>> >> -Joey
>> >> On May 19, 2011 8:00 AM, "Gabriele Kahlout" <[EMAIL PROTECTED]>
>> >> wrote:
>> >> > $ hadoop version
>> >> > Hadoop 0.20.3-SNAPSHOT
>> >> > Subversion
>> >> >
>> >>
>> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-append-r
>> >> > 1041718
>> >> > Compiled by hammer on Mon Dec 6 17:38:16 CET 2010
>> >> >
>> >> >
>> >> > On Thu, May 19, 2011 at 4:55 PM, Joey Echeverria <[EMAIL PROTECTED]>
>> >> wrote:
>> >> >
>> >> >> What version of hadoop is installed?
>> >> >>
>> >> >> -Joey
>> >> >> On May 19, 2011 7:49 AM, "Gabriele Kahlout" <
>> [EMAIL PROTECTED]>
>> >> >> wrote:
>> >> >> > I said i don't have write access (~ the administrator with write
>> >> access
>> >> >> will
>> >> >> > not place the script for me).
>> >> >> >
>> >> >> > On Thu, May 19, 2011 at 3:56 PM, Niels Basjes <[EMAIL PROTECTED]>
>> >> wrote:
>> >> >> >
>> >> >> >> So why don't you ask for someone with write access to put the file
>> >> >> there?
>> >> >> >>
>> >> >> >> 2011/5/19 Gabriele Kahlout <[EMAIL PROTECTED]>:
>> >> >> >> > so your question is, why do you have the problem in the first
>> >> place?
>> >> >> >> > because it's not in $HADOOP_HOME/bin (older hadoop) and I don't
>> >> have
>> >> >> >> wrtie
>> >> >> >> > access.
>> >> >> >> >
>> >> >> >> > On Thu, May 19, 2011 at 3:33 PM, Joey Echeverria <
>> >> [EMAIL PROTECTED]>
>> >> >> >> wrote:
>> >> >> >> >
>> >> >> >> >> Why do you need to move the script from $HADOOP_HOME/bin?
>> >> >> >> >>
>> >> >> >> >> Can't you just symlink it or write a script which runs the
>> >> original?
>> >> >> >> >>
>> >> >> >> >> -Joey
>> >> >> >> >>
>> >> >> >> >> On May 19, 2011, at 4:15, Gabriele Kahlout <
>> >> [EMAIL PROTECTED]
>> >> >> >
>> >> >> >> >> wrote:
>> >> >> >> >>
>> >> >> >> >> > I'm still having the following problem, any suggestions?
>> >> >> >> >> >
>> >> >> >> >> > I'm trying to modify the
>> >> >> >> >> > hdfs<
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >> >>
>> >>
>> >>
>> http://svn.apache.org/viewvc/hadoop/hdfs/tags/release-0.21.0/bin/hdfs?view=markup
>> >> >> >> >> >script
>> >> >> >> >> > so that it still functions although not located in
>> >> $HADOOP_HOME/bin
>> >> >> >> >> > anymore, but when I execute the modified hdfs I get:
>> >> >> >> >> >
>> >> >> >> >> > hdfs: line 110: exec: org.apache.hadoop.fs.FsShell: not found
>> >> >> >> >> >
>> >> >> >> >> > line 110 is:
>> >> >> >> >> >
>> >> >> >> >> > exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
>> >> >> >> >> >
>> >> >> >> >> > I've highlighted the changes I made to the script:
>> >> >> >> >> >
>> >> >> >> >> > bin=*"$HADOOP_HOME"/bin # was* dirname "$0" bin=cd "$bin";
>> pwd
>> >> >> >> >> >
>> >> >> >> >> > ./*hdfs-config.sh # was .* "$bin"/hdfs-config.sh
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > On Mon, May 16, 2011 at 12:20 PM, Gabriele Kahlout <

Joseph Echeverria
Cloudera, Inc.
443.305.9434