Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # user - Re: Issue in Oozie sqoop action with <command></command> and --query


Copy link to this message
-
Re: Issue in Oozie sqoop action with <command></command> and --query
Jarek Jarcec Cecho 2012-10-22, 15:40
Hi Nitin,
it can actually get quite tricky to correctly split the command line when Oozie would allow all soft of different escaping inside <command> argument. That's why it fits for simple use cases and that's why there is tag <arg> for more advance use cases.

I can see why you need to generate your whole command line in your use case. I've actually done something similar (conditional sqoop command line generation) in the past, but I've used <arg> tags and just filled argument values (password, username, query). Is something like that possible in your use case?

Jarcec

On Mon, Oct 22, 2012 at 11:11:21AM -0400, Nitin kak wrote:
> Okay. So the thing is that my pre-compiler could have generated the whole
> command(which requires the --query "select a, b as c from .... " ) as a
> parameter. While what I now need is many more parameter values for
> --connect, --query, --username, --password and all.
>
> What I feel is that if sqoop command supports "--query" construct, then we
> should be able to use that construct in <command></command> in Oozie.
>
> Please let me know what you think.
>
> Regards,
> Nitin
>
> On Mon, Oct 22, 2012 at 11:01 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>
> > Hi Nitin,
> > I personally do not see that as an issue. Tag <command> is meant for
> > simple sqoop command lines and there are tags <arg> for more advance use
> > cases. You happen to be in advance use case and therefore you need <arg>
> > tags. Seems a perfectly valid solution to me.
> >
> > Would you mind describe why do you necessarily need <command> tag and why
> > you can't use <arg> tags?
> >
> > Jarcec
> >
> > On Mon, Oct 22, 2012 at 10:40:18AM -0400, Nitin kak wrote:
> > > Hi Jarcec,
> > >
> > >      Thanks for ur help. Isn't that kind of a issue that we can't use
> > > "--query" in <command><command> in Oozie. If that is, can we raise this
> > > issue somewhere?
> > >
> > > Regards,
> > > Nitin Kak
> > >
> > > On Fri, Oct 19, 2012 at 5:39 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]
> > >wrote:
> > >
> > > > That is correct - you can't use <command> tag and <arg> tags together.
> > You
> > > > can use either <command> or <arg>.
> > > >
> > > > Jarcec
> > > >
> > > > On Fri, Oct 19, 2012 at 05:32:12PM -0400, Nitin kak wrote:
> > > > > Ahh...that sucks..I hope they will come up with something in future
> > > > > releases...
> > > > >
> > > > > One follow up question...We can't use <command></command> in
> > combination
> > > > > with <arg></arg> right?
> > > > >
> > > > > On Fri, Oct 19, 2012 at 5:25 PM, Jarek Jarcec Cecho <
> > [EMAIL PROTECTED]
> > > > >wrote:
> > > > >
> > > > > > Hi Nikin,
> > > > > > workaround to use --query parameter inside <command> tag in Oozie
> > is to
> > > > > > use multiple <arg> tags instead :-)
> > > > > >
> > > > > > Basically Oozie will blindly divide value supplied in <command>
> > tag by
> > > > > > spaces without taking into account any escaping. You are required
> > to
> > > > use
> > > > > > <arg> tags instead of <command> in case that you need to specify
> > > > arguments
> > > > > > with spaces. For more information please consider Oozie Sqoop
> > action
> > > > > > documentation [1].
> > > > > >
> > > > > > Jarcec
> > > > > >
> > > > > > Links:
> > > > > > 1:
> > > > > >
> > > >
> > http://oozie.apache.org/docs/3.2.0-incubating/DG_SqoopActionExtension.html
> > > > > >
> > > > > > On Fri, Oct 19, 2012 at 05:10:07PM -0400, Nitin kak wrote:
> > > > > > > Hi guys,
> > > > > > >
> > > > > > >       Just wanted to know how we can use "--query" attribute in
> > > > > > > <command></command> of sqoop action in Oozie. The issue is that
> > if we
> > > > > > have
> > > > > > > something like
> > > > > > >
> > > > > > > * <command>blah blah ....  --query "select * from
> > table"</command>
> > > > *is
> > > > > > > converted to
> > > > > > >
> > > > > > > <arg>blah</arg>
> > > > > > > <arg>blah</arg>
> > > > > > > <arg>--query</arg>
> > > > > > > <arg>"select</arg>
> > > > > > > <arg>*</arg>
> > >
+
Nitin kak 2012-10-22, 17:03
+
Jarek Jarcec Cecho 2012-10-22, 17:27
+
Nitin kak 2012-10-19, 21:10
+
Jarek Jarcec Cecho 2012-10-19, 21:25
+
Nitin kak 2012-10-19, 21:32
+
Jarek Jarcec Cecho 2012-10-19, 21:39