Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - HiveHistoryViewer concurrency problem


Copy link to this message
-
Re: HiveHistoryViewer concurrency problem
Qiang Wang 2013-01-05, 04:59
Hi Jie:

As I know, hive history log is structured and class *HiveHistory* is used
to write and read hive history log.

*HiveHistoryViewer* serves as a listener to listen and store parsed log
data. It has two members:

HashMap<String, QueryInfo> *jobInfoMap*, which stores QueryInfo related
with hive query

and

HashMap<String, TaskInfo> *taskInfoMap*, which stores TaskInfo related with
hadoop map/red job

you can dump the two maps and find what you want.

Hope these info helps

Qiang
2013/1/5 Jie Li <[EMAIL PROTECTED]>

> Hi Qiang,
>
> Could you describe how HiveHistoryViewer is used? I'm also looking for
> a tool to understand the Hive log.
>
> Thanks,
> Jie
>
> On Sat, Jan 5, 2013 at 9:54 AM, Qiang Wang <[EMAIL PROTECTED]> wrote:
> > Does Anybody have an idea about this?
> >
> > https://issues.apache.org/jira/browse/HIVE-3857
> >
> >
> > 2013/1/4 Qiang Wang <[EMAIL PROTECTED]>
> >>
> >> new HiveHistoryViewer() throws ConcurrentModificationException when
> called
> >> concurrently by several threads.
> >>
> >> According to the stack trace, HiveHistory.parseLine use private static
> >> Map<String, String> parseBuffer to store parsed data and this caused the
> >> exception.
> >>
> >> I don't know why a static buffer rather than a local buffer is used!
> >> Anybody have an idea about this?
> >
> >
>