Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Passing parameters to Pig Script using Java


Copy link to this message
-
Re: Passing parameters to Pig Script using Java
Oops, sorry
Yes this is old code.
Thanks for pointing this out.
This should give a hint though.
Julien
On 10/7/10 6:16 PM, "Jeff Zhang" <[EMAIL PROTECTED]> wrote:

Hi Julien,

You did what we have done in Pig 0.8. There's a little difference
between your api and ours. We encapsulates the parameter in Map rather
than List.
On Fri, Oct 8, 2010 at 4:43 AM, Julien Le Dem <[EMAIL PROTECTED]> wrote:
> Here's my workaround:
> I extend PigServer with the following code copy/pasted from other places in Pig.
> Parameters are in the form: "foo=bar"
> Julien
>
> import org.apache.pig.ExecType;
> import org.apache.pig.backend.executionengine.ExecException;
> import org.apache.pig.backend.executionengine.ExecJob;
> import org.apache.pig.impl.PigContext;
> import org.apache.pig.tools.grunt.GruntParser;
> import org.apache.pig.tools.parameters.ParameterSubstitutionPreprocessor;
>
>
>    /**
>     * adapted from org.apache.pig.Main
>     * returns the stream of final pig script to be passed to Grunt
>     */
>    private Reader runParamPreprocessor(InputStream origPigScript, List<String> params,
>                                               List<String> paramFiles, String scriptFile)
>                                    throws org.apache.pig.tools.parameters.ParseException, IOException{
>        ParameterSubstitutionPreprocessor psp = new ParameterSubstitutionPreprocessor(50);
>        String[] type1 = new String[1];
>        String[] type2 = new String[1];
>
>        StringWriter writer = new StringWriter();
>        psp.genSubstitutedFile (new BufferedReader(new InputStreamReader(origPigScript)), writer,  params.size() > 0 ? params.toArray(type1) : null,
>          paramFiles.size() > 0 ? paramFiles.toArray(type2) : null);
>
>        return new BufferedReader(new StringReader(writer.toString()));
>
>    }
>
>    /**
>  * adapted original code from file to apply the preprocessor
>  * @param filePath relative path of the pig script
>  * @param params parameters to be applied by the preprocessor
>  * @throws IOException
>  */
>    public void registerScript(String filePath, List<String> params) throws IOException {
>        try {
>            InputStream stream = new FileInputStream(filePath);
>            try {
>             GruntParser grunt = new GruntParser(runParamPreprocessor(stream, params, new ArrayList<String>(), filePath));
>             grunt.setInteractive(false);
>             grunt.setParams(this);
>             grunt.parseStopOnError(true);
>            } finally {
>             try {
>              stream.close();
>             } catch (IOException e) {
>              e.printStackTrace();
>             }
>            }
>        } catch (org.apache.pig.tools.pigscript.parser.ParseException e) {
>            throw new IOException("Error while parsing script: "+filePath,e);
>        } catch (org.apache.pig.tools.parameters.ParseException e) {
>         throw new IOException("Error while parsing parameters: "+params,e);
>  }
>    }
>
>    /**
>  * adapted original code from file to apply the preprocessor
>  * @param filePath relative path of the pig script
>  * @param params parameters to be applied by the preprocessor
>  * @throws IOException
>  */
>    public void registerScript(String filePath, String... params) throws IOException {
>     registerScript(filePath, Arrays.asList(params));
>    }
>
>
>
>
>
> On 10/7/10 1:27 PM, "Olga Natkovich" <[EMAIL PROTECTED]> wrote:
>
> Not at this point as parameter substitution is implemented a s preprocessor on the script.
>
> Olga
>
> -----Original Message-----
> From: rakesh kothari [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, October 07, 2010 11:47 AM
> To: [EMAIL PROTECTED]
> Subject: Passing parameters to Pig Script using Java
>
>
> Hi,
>
> I have a pig script that needs certain parameters (passed using "-p" in pig shell) to execute. Is there a way to pass these parameters if I want to execute this script using "PigServer" after registering the script using PigServer.registerScript() ?

Best Regards

Jeff Zhang