new HiveHistoryViewer() throws ConcurrentModificationException when called
concurrently by several threads.

According to the stack trace, HiveHistory.parseLine use *private static
Map<String, String> parseBuffer* to store parsed data and this caused the
exception.

I don't know why a static buffer rather than a local buffer is used!
Anybody have an idea about this?

Does Anybody have an idea about this?

https://issues.apache.org/jira/browse/HIVE-3857
2013/1/4 Qiang Wang <[EMAIL PROTECTED]>

> new HiveHistoryViewer() throws ConcurrentModificationException when called
> concurrently by several threads.
>
> According to the stack trace, HiveHistory.parseLine use *private static
> Map<String, String> parseBuffer* to store parsed data and this caused the
> exception.
>
> I don't know why a static buffer rather than a local buffer is used!
> Anybody have an idea about this?
>

Hi Qiang,

Could you describe how HiveHistoryViewer is used? I'm also looking for
a tool to understand the Hive log.

Thanks,
Jie

On Sat, Jan 5, 2013 at 9:54 AM, Qiang Wang <[EMAIL PROTECTED]> wrote:
> Does Anybody have an idea about this?
>
> https://issues.apache.org/jira/browse/HIVE-3857
>
>
> 2013/1/4 Qiang Wang <[EMAIL PROTECTED]>
>>
>> new HiveHistoryViewer() throws ConcurrentModificationException when called
>> concurrently by several threads.
>>
>> According to the stack trace, HiveHistory.parseLine use private static
>> Map<String, String> parseBuffer to store parsed data and this caused the
>> exception.
>>
>> I don't know why a static buffer rather than a local buffer is used!
>> Anybody have an idea about this?
>
>

It is likely an oversight. The Majority of hive code was not written to be
multi-threaded.
On Fri, Jan 4, 2013 at 10:41 PM, Jie Li <[EMAIL PROTECTED]> wrote:

> Hi Qiang,
>
> Could you describe how HiveHistoryViewer is used? I'm also looking for
> a tool to understand the Hive log.
>
> Thanks,
> Jie
>
> On Sat, Jan 5, 2013 at 9:54 AM, Qiang Wang <[EMAIL PROTECTED]> wrote:
> > Does Anybody have an idea about this?
> >
> > https://issues.apache.org/jira/browse/HIVE-3857
> >
> >
> > 2013/1/4 Qiang Wang <[EMAIL PROTECTED]>
> >>
> >> new HiveHistoryViewer() throws ConcurrentModificationException when
> called
> >> concurrently by several threads.
> >>
> >> According to the stack trace, HiveHistory.parseLine use private static
> >> Map<String, String> parseBuffer to store parsed data and this caused the
> >> exception.
> >>
> >> I don't know why a static buffer rather than a local buffer is used!
> >> Anybody have an idea about this?
> >
> >
>

Hi Jie:

As I know, hive history log is structured and class *HiveHistory* is used
to write and read hive history log.

*HiveHistoryViewer* serves as a listener to listen and store parsed log
data. It has two members:

HashMap<String, QueryInfo> *jobInfoMap*, which stores QueryInfo related
with hive query

and

HashMap<String, TaskInfo> *taskInfoMap*, which stores TaskInfo related with
hadoop map/red job

you can dump the two maps and find what you want.

Hope these info helps

Qiang
2013/1/5 Jie Li <[EMAIL PROTECTED]>

> Hi Qiang,
>
> Could you describe how HiveHistoryViewer is used? I'm also looking for
> a tool to understand the Hive log.
>
> Thanks,
> Jie
>
> On Sat, Jan 5, 2013 at 9:54 AM, Qiang Wang <[EMAIL PROTECTED]> wrote:
> > Does Anybody have an idea about this?
> >
> > https://issues.apache.org/jira/browse/HIVE-3857
> >
> >
> > 2013/1/4 Qiang Wang <[EMAIL PROTECTED]>
> >>
> >> new HiveHistoryViewer() throws ConcurrentModificationException when
> called
> >> concurrently by several threads.
> >>
> >> According to the stack trace, HiveHistory.parseLine use private static
> >> Map<String, String> parseBuffer to store parsed data and this caused the
> >> exception.
> >>
> >> I don't know why a static buffer rather than a local buffer is used!
> >> Anybody have an idea about this?
> >
> >
>

Maybe it's not.

But this exception happens when I create an *HiveHistoryViewer* instance,
in which case only reading, parsing file is invloved and it's not intended
to be shared between threads.

So the exception surprised me and I wonder why a static buffer was used
instead of a local buffer which has no concurrent issue.
2013/1/5 Edward Capriolo <[EMAIL PROTECTED]>

> It is likely an oversight. The Majority of hive code was not written to be
> multi-threaded.
>
>
>
> On Fri, Jan 4, 2013 at 10:41 PM, Jie Li <[EMAIL PROTECTED]> wrote:
>
>> Hi Qiang,
>>
>> Could you describe how HiveHistoryViewer is used? I'm also looking for
>> a tool to understand the Hive log.
>>
>> Thanks,
>> Jie
>>
>> On Sat, Jan 5, 2013 at 9:54 AM, Qiang Wang <[EMAIL PROTECTED]> wrote:
>> > Does Anybody have an idea about this?
>> >
>> > https://issues.apache.org/jira/browse/HIVE-3857
>> >
>> >
>> > 2013/1/4 Qiang Wang <[EMAIL PROTECTED]>
>> >>
>> >> new HiveHistoryViewer() throws ConcurrentModificationException when
>> called
>> >> concurrently by several threads.
>> >>
>> >> According to the stack trace, HiveHistory.parseLine use private static
>> >> Map<String, String> parseBuffer to store parsed data and this caused
>> the
>> >> exception.
>> >>
>> >> I don't know why a static buffer rather than a local buffer is used!
>> >> Anybody have an idea about this?
>> >
>> >
>>
>
>