Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Passing parameters to Pig Script using Java


Copy link to this message
-
Re: Passing parameters to Pig Script using Java
Oops, sorry
Yes this is old code.
Thanks for pointing this out.
This should give a hint though.
Julien
On 10/7/10 6:16 PM, "Jeff Zhang" <[EMAIL PROTECTED]> wrote:

Hi Julien,

You did what we have done in Pig 0.8. There's a little difference
between your api and ours. We encapsulates the parameter in Map rather
than List.
On Fri, Oct 8, 2010 at 4:43 AM, Julien Le Dem <[EMAIL PROTECTED]> wrote:
> Here's my workaround:
> I extend PigServer with the following code copy/pasted from other places in Pig.
> Parameters are in the form: "foo=bar"
> Julien
>
> import org.apache.pig.ExecType;
> import org.apache.pig.backend.executionengine.ExecException;
> import org.apache.pig.backend.executionengine.ExecJob;
> import org.apache.pig.impl.PigContext;
> import org.apache.pig.tools.grunt.GruntParser;
> import org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor;
>
>
>    /**
>     * adapted from org.apache.pig.Main
>     * returns the stream of final pig script to be passed to Grunt
>     */
>    private Reader runParamPreprocessor(InputStream origPigScript, List<String> params,
>                                               List<String> paramFiles, String scriptFile)
>                                    throws org.apache.pig.tools.parameters.ParseException, IOException{
>        ParameterSubstitutionPreprocessor psp = new ParameterSubstitutionPreprocessor(50);
>        String[] type1 = new String[1];
>        String[] type2 = new String[1];
>
>        StringWriter writer = new StringWriter();
>        psp.genSubstitutedFile (new BufferedReader(new InputStreamReader(origPigScript)), writer,  params.size() > 0 ? params.toArray(type1) : null,
>          paramFiles.size() > 0 ? paramFiles.toArray(type2) : null);
>
>        return new BufferedReader(new StringReader(writer.toString()));
>
>    }
>
>    /**
>  * adapted original code from file to apply the preprocessor
>  * @param filePath relative path of the pig script
>  * @param params parameters to be applied by the preprocessor
>  * @throws IOException
>  */
>    public void registerScript(String filePath, List<String> params) throws IOException {
>        try {
>            InputStream stream = new FileInputStream(filePath);
>            try {
>             GruntParser grunt = new GruntParser(runParamPreprocessor(stream, params, new ArrayList<String>(), filePath));
>             grunt.setInteractive(false);
>             grunt.setParams(this);
>             grunt.parseStopOnError(true);
>            } finally {
>             try {
>              stream.close();
>             } catch (IOException e) {
>              e.printStackTrace();
>             }
>            }
>        } catch (org.apache.pig.tools.pigscript.parser.ParseException e) {
>            throw new IOException("Error while parsing script: "+filePath,e);
>        } catch (org.apache.pig.tools.parameters.ParseException e) {
>         throw new IOException("Error while parsing parameters: "+params,e);
>  }
>    }
>
>    /**
>  * adapted original code from file to apply the preprocessor
>  * @param filePath relative path of the pig script
>  * @param params parameters to be applied by the preprocessor
>  * @throws IOException
>  */
>    public void registerScript(String filePath, String... params) throws IOException {
>     registerScript(filePath, Arrays.asList(params));
>    }
>
>
>
>
>
> On 10/7/10 1:27 PM, "Olga Natkovich" <[EMAIL PROTECTED]> wrote:
>
> Not at this point as parameter substitution is implemented a s preprocessor on the script.
>
> Olga
>
> -----Original Message-----
> From: rakesh kothari [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, October 07, 2010 11:47 AM
> To: [EMAIL PROTECTED]
> Subject: Passing parameters to Pig Script using Java
>
>
> Hi,
>
> I have a pig script that needs certain parameters (passed using "-p" in pig shell) to execute. Is there a way to pass these parameters if I want to execute this script using "PigServer" after registering the script using PigServer.registerScript() ?

Best Regards

Jeff Zhang
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB