Pig, mail # user - Re: Running pig script as different user

Re: Running pig script as different user

Miki Tebeka 2012-11-29, 22:42

As in -Dhadoop.user.ugi=bugs ?
On Wed, Nov 28, 2012 at 5:02 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:

> Try using "hadoop.job.ugi" to run the jobs as a different user on the
> cluster. I'm not sure if that works on the more recent versions of
> Hadoop though.
>
> Sent from my iPhone
>
> On Nov 29, 2012, at 6:28 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:
>
> > Greetings,
> >
> > Is there a way, other than "su", to run pig script as a different user?
> >
> > Thanks,
> > --
> > Miki
>

Re: Running pig script as different user

Miki Tebeka 2012-11-30, 21:12

I tried running
PIG_OPTS=-Dhadoop.user.ugi=bugs pig /path/to/script.pig

Still ran as the original user.
On Thu, Nov 29, 2012 at 2:42 PM, Miki Tebeka <[EMAIL PROTECTED]> wrote:

> As in -Dhadoop.user.ugi=bugs ?
>
>
> On Wed, Nov 28, 2012 at 5:02 PM, Prashant Kommireddi <[EMAIL PROTECTED]>wrote:
>
>> Try using "hadoop.job.ugi" to run the jobs as a different user on the
>> cluster. I'm not sure if that works on the more recent versions of
>> Hadoop though.
>>
>> Sent from my iPhone
>>
>> On Nov 29, 2012, at 6:28 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:
>>
>> > Greetings,
>> >
>> > Is there a way, other than "su", to run pig script as a different user?
>> >
>> > Thanks,
>> > --
>> > Miki
>>
>
>

Re: Running pig script as different user

Prashant Kommireddi 2012-12-01, 01:52

Hi Miki,

What version of hadoop are you on? I can confirm this works on 0.20.2 but
never tried this on the newer versions.

Try hadoop.job.ugi=<username,group> where the group must be the name of the
group associated with the username.

For eg, -Dhadoop.job.ugi=bugs,bugs
On Sat, Dec 1, 2012 at 2:42 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:

> I tried running
> PIG_OPTS=-Dhadoop.user.ugi=bugs pig /path/to/script.pig
>
> Still ran as the original user.
>
>
> On Thu, Nov 29, 2012 at 2:42 PM, Miki Tebeka <[EMAIL PROTECTED]>
> wrote:
>
> > As in -Dhadoop.user.ugi=bugs ?
> >
> >
> > On Wed, Nov 28, 2012 at 5:02 PM, Prashant Kommireddi <
> [EMAIL PROTECTED]>wrote:
> >
> >> Try using "hadoop.job.ugi" to run the jobs as a different user on the
> >> cluster. I'm not sure if that works on the more recent versions of
> >> Hadoop though.
> >>
> >> Sent from my iPhone
> >>
> >> On Nov 29, 2012, at 6:28 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:
> >>
> >> > Greetings,
> >> >
> >> > Is there a way, other than "su", to run pig script as a different
> user?
> >> >
> >> > Thanks,
> >> > --
> >> > Miki
> >>
> >
> >
>

Re: Running pig script as different user

Dmitriy Ryaboy 2012-12-03, 23:35

This should not work in versions of hadoop that support security for
fairly obvious reasons.

On Fri, Nov 30, 2012 at 5:52 PM, Prashant Kommireddi
<[EMAIL PROTECTED]> wrote:
> Hi Miki,
>
> What version of hadoop are you on? I can confirm this works on 0.20.2 but
> never tried this on the newer versions.
>
> Try hadoop.job.ugi=<username,group> where the group must be the name of the
> group associated with the username.
>
> For eg, -Dhadoop.job.ugi=bugs,bugs
>
>
> On Sat, Dec 1, 2012 at 2:42 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:
>
>> I tried running
>> PIG_OPTS=-Dhadoop.user.ugi=bugs pig /path/to/script.pig
>>
>> Still ran as the original user.
>>
>>
>> On Thu, Nov 29, 2012 at 2:42 PM, Miki Tebeka <[EMAIL PROTECTED]>
>> wrote:
>>
>> > As in -Dhadoop.user.ugi=bugs ?
>> >
>> >
>> > On Wed, Nov 28, 2012 at 5:02 PM, Prashant Kommireddi <
>> [EMAIL PROTECTED]>wrote:
>> >
>> >> Try using "hadoop.job.ugi" to run the jobs as a different user on the
>> >> cluster. I'm not sure if that works on the more recent versions of
>> >> Hadoop though.
>> >>
>> >> Sent from my iPhone
>> >>
>> >> On Nov 29, 2012, at 6:28 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:
>> >>
>> >> > Greetings,
>> >> >
>> >> > Is there a way, other than "su", to run pig script as a different
>> user?
>> >> >
>> >> > Thanks,
>> >> > --
>> >> > Miki
>> >>
>> >
>> >
>>

Re: Running pig script as different user

UMBC 2012-12-11, 12:17

Miki, Prashant,


   Curiously I've got the same need as you do. I've started to work on this problem few weeks ago by patching Hadoop client code. Here is what I have so far https://github.com/vladistan/hadoop-common/commits/cloudera-cdh3u5

   I only tried it on with the Cloudera's CDH3 hadoop, because this is the only version I run on all my clusters. To use the patch, checkout the branch above, compile with ant ( make sure build the 'tar' file ).

   Then unpack the tar somewhere, set your HADOOP_HOME variable to the new location and try to use Pig as is.

   I've been testing it for the past few days, it works okay with locally launched hadoop and pig jobs. Also works with Hamake and map reduce jobs, but for some reason it fails with hamake and pig scripts. I am currently investigating why is that.

   So you are welcome to try the patch above, let me know how it works for you. Just bear in mind that the patch above is still work in progress and it is just a quick hack to get things going fast. For proper production use I recommend you install your cluster with Kerberos security and use kerberos mechanism to authenticate as different users.

   -- Vlad

Hi Miki,

What version of hadoop are you on? I can confirm this works on 0.20.2 but
never tried this on the newer versions.

Try hadoop.job.ugi=<username,group> where the group must be the name of the
group associated with the username.

For eg, -Dhadoop.job.ugi=bugs,bugs
On Sat, Dec 1, 2012 at 2:42 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:

> I tried running
> PIG_OPTS=-Dhadoop.user.ugi=bugs pig /path/to/script.pig
>
> Still ran as the original user.
>
>
> On Thu, Nov 29, 2012 at 2:42 PM, Miki Tebeka <[EMAIL PROTECTED]>
> wrote:
>
> > As in -Dhadoop.user.ugi=bugs ?
> >
> >
> > On Wed, Nov 28, 2012 at 5:02 PM, Prashant Kommireddi <
> [EMAIL PROTECTED]>wrote:
> >
> >> Try using "hadoop.job.ugi" to run the jobs as a different user on the
> >> cluster. I'm not sure if that works on the more recent versions of
> >> Hadoop though.
> >>
> >> Sent from my iPhone
> >>
> >> On Nov 29, 2012, at 6:28 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:
> >>
> >> > Greetings,
> >> >
> >> > Is there a way, other than "su", to run pig script as a different
> user?
> >> >
> >> > Thanks,
> >> > --
> >> > Miki
> >>
> >
> >
>

Re: Running pig script as different user

Miki Tebeka 2012-12-11, 17:06

Thanks, I'll have a look and see if this is applicable to CDH4 as well.
On Tue, Dec 11, 2012 at 4:17 AM, UMBC <[EMAIL PROTECTED]> wrote:

> Miki, Prashant,
>
>
> Curiously I've got the same need as you do. I've started to work
> on this problem few weeks ago by patching Hadoop client code. Here is
> what I have so far
> https://github.com/vladistan/hadoop-common/commits/cloudera-cdh3u5
>
> I only tried it on with the Cloudera's CDH3 hadoop, because this is
> the only version I run on all my clusters. To use the patch, checkout
> the branch above, compile with ant ( make sure build the 'tar' file ).
>
> Then unpack the tar somewhere, set your HADOOP_HOME variable to the
> new location and try to use Pig as is.
>
> I've been testing it for the past few days, it works okay with
> locally launched hadoop and pig jobs. Also works with Hamake and map
> reduce jobs, but for some reason it fails with hamake and pig scripts. I
> am currently investigating why is that.
>
> So you are welcome to try the patch above, let me know how it
> works for you. Just bear in mind that the patch above is still work in
> progress and it is just a quick hack to get things going fast. For proper
> production use I recommend you install your cluster with Kerberos security
> and use kerberos mechanism to authenticate as different users.
>
> -- Vlad
>
>
>
>
>
>
>
> Hi Miki,
>
> What version of hadoop are you on? I can confirm this works on 0.20.2 but
> never tried this on the newer versions.
>
> Try hadoop.job.ugi=<username,group> where the group must be the name of the
> group associated with the username.
>
> For eg, -Dhadoop.job.ugi=bugs,bugs
>
>
> On Sat, Dec 1, 2012 at 2:42 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:
>
> > I tried running
> > PIG_OPTS=-Dhadoop.user.ugi=bugs pig /path/to/script.pig
> >
> > Still ran as the original user.
> >
> >
> > On Thu, Nov 29, 2012 at 2:42 PM, Miki Tebeka <[EMAIL PROTECTED]>
> > wrote:
> >
> > > As in -Dhadoop.user.ugi=bugs ?
> > >
> > >
> > > On Wed, Nov 28, 2012 at 5:02 PM, Prashant Kommireddi <
> > [EMAIL PROTECTED]>wrote:
> > >
> > >> Try using "hadoop.job.ugi" to run the jobs as a different user on the
> > >> cluster. I'm not sure if that works on the more recent versions of
> > >> Hadoop though.
> > >>
> > >> Sent from my iPhone
> > >>
> > >> On Nov 29, 2012, at 6:28 AM, Miki Tebeka <[EMAIL PROTECTED]>
> wrote:
> > >>
> > >> > Greetings,
> > >> >
> > >> > Is there a way, other than "su", to run pig script as a different
> > user?
> > >> >
> > >> > Thanks,
> > >> > --
> > >> > Miki
> > >>
> > >
> > >
> >
>
>