-Re: pig installation - hbase hadoop on remote system
Jacob Perkins 2011-02-22, 13:08
Your mac needs to have the hadoop configuration (eg. hdfs-site.xml,
mapred-site.xml, core-site.xml, depending on the version of hadoop)
files available somewhere in pig's classpath. It may do to simply copy
them directly from one of the remote machines.
On Tue, 2011-02-22 at 17:12 +0530, rashmi behera wrote:
> I am new to Hbase/Hadoop concept. Following is the scenario -:
> 1) Our Hadoop is installed in a remote system. Data is loaded in HBase
> through HBase writer.
> 2) I am trying to install pig on my local mac OS X( version 10.6.5) so that
> i will fetch data from that remote system. I downloaded Pig latest release
> from http://pig.apache.org/releases.html ( 17 December, 2010: release 0.8.0
> I did the following things - :
> supp:~ rashmi$ export PATH=/Users/rashmi/Desktop/pig-0.8.0/bin:$PATH
> supp:~ rashmi$ pig -help
> Error: JAVA_HOME is not set.
> supp:~ rashmi$ export
> when i ran pig -help i got the following output -:
> supp:~ rashmi$ pig -help
> Apache Pig version 0.8.0 (r1043805)
> compiled Dec 08 2010, 17:26:09
> USAGE: Pig [options] [-] : Run interactively in grunt shell.
> Pig [options] -e[xecute] cmd [cmd ...] : Run cmd(s).
> Pig [options] [-f[ile]] file : Run cmds found in file.
> options include:
> -4, -log4jconf - Log4j configuration file, overrides log conf
> -b, -brief - Brief logging (no timestamps)
> -c, -check - Syntax check
> -d, -debug - Debug level, INFO is default
> -e, -execute - Commands to execute (within quotes)
> -f, -file - Path to the script to execute
> -h, -help - Display this message. You can specify topic to get help for
> that topic.
> properties is the only topic currently supported: -h properties.
> -i, -version - Display version information
> -l, -logfile - Path to client side log file; default is current working
> -m, -param_file - Path to the parameter file
> -p, -param - Key value pair of the form param=val
> -r, -dryrun - Produces script with substituted parameters. Script is not
> -t, -optimizer_off - Turn optimizations off. The following values are
> SplitFilter - Split filter conditions
> MergeFilter - Merge filter conditions
> PushUpFilter - Filter as early as possible
> PushDownForeachFlatten - Join or explode as late as possible
> ColumnMapKeyPrune - Remove unused data
> LimitOptimizer - Limit as early as possible
> AddForEach - Add ForEach to remove unneeded columns
> MergeForEach - Merge adjacent ForEach
> LogicalExpressionSimplifier - Combine multiple expressions
> All - Disable all optimizations
> All optimizations are enabled by default. Optimization values are
> case insensitive.
> -v, -verbose - Print all error messages to screen
> -w, -warning - Turn warning logging on; also turns warning aggregation
> -x, -exectype - Set execution mode: local|mapreduce, default is
> -F, -stop_on_failure - Aborts execution on the first failed job; default
> is off
> -M, -no_multiquery - Turn multiquery optimization off; default is on
> -P, -propertyFile - Path to property file
> when i ran pig command i got the following error -:
> supp:~ rashmi$ pig
> 2011-02-22 12:48:26,319 [main] INFO org.apache.pig.Main - Logging error
> messages to: /Users/rashmi/pig_1298359106317.log
> 2011-02-22 12:48:26,474 [main] ERROR org.apache.pig.Main - ERROR 4010:
> Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor
> core-site.xml was found in the classpath).If you plan to use local mode,
> please put -x local option in command line
> Details at logfile: /Users/rashmi/pig_1298359106317.log
> My Question is
> 1) What all i need to do , so that i could connect to remote hadoop system