Hi All,
Currently MR job spilled his counters into file at the end of the run.

Is there any built-in configuration/plug-in to make it store these counters
into HBase as well?

Sounds to me like a great feature!

Does anybody did something similar?
If you did, how did you do it? Run on directory and load it into HBase?

Thanks,
Ophir

Hi Ophir,

That sounds like a useful feature, maybe file a jira?

I've never tried to save counters form the MR job into HBase, but you
could pull it from the file as you said or from the Job object after
waitForCompletion() returns by calling getCounters().

-Joey

On Wed, May 18, 2011 at 8:21 AM, Ophir Cohen <[EMAIL PROTECTED]> wrote:
> Hi All,
> Currently MR job spilled his counters into file at the end of the run.
>
> Is there any built-in configuration/plug-in to make it store these counters
> into HBase as well?
>
> Sounds to me like a great feature!
>
> Does anybody did something similar?
> If you did, how did you do it? Run on directory and load it into HBase?
>
> Thanks,
> Ophir
>

--
Joseph Echeverria
Cloudera, Inc.
443.305.9434

For jobs you create it sounds as a great idea.
What about other jobs such as Hive/Pig jobs?

Can anyone as any idea how it can be done on all MR jobs in a cluster no
matter the triggering of the jobs?
Ophir

On Wed, May 18, 2011 at 7:09 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:

> Hi Ophir,
>
> That sounds like a useful feature, maybe file a jira?
>
> I've never tried to save counters form the MR job into HBase, but you
> could pull it from the file as you said or from the Job object after
> waitForCompletion() returns by calling getCounters().
>
> -Joey
>
> On Wed, May 18, 2011 at 8:21 AM, Ophir Cohen <[EMAIL PROTECTED]> wrote:
> > Hi All,
> > Currently MR job spilled his counters into file at the end of the run.
> >
> > Is there any built-in configuration/plug-in to make it store these
> counters
> > into HBase as well?
> >
> > Sounds to me like a great feature!
> >
> > Does anybody did something similar?
> > If you did, how did you do it? Run on directory and load it into HBase?
> >
> > Thanks,
> > Ophir
> >
>
>
>
> --
> Joseph Echeverria
> Cloudera, Inc.
> 443.305.9434
>