Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Ivory (per term indexing) with Hadoop


Copy link to this message
-
Re: Ivory (per term indexing) with Hadoop
Luiz Antonio Falaguasta B... 2012-01-26, 11:53
I don't know, Kumar. I started to handle Hadoop now.

2012/1/26 Ashwanth Kumar <[EMAIL PROTECTED]>

> Don't we have Lucene plugin kind of thing for Hadoop?
>
>  - Ashwanth
>
>
> On Thu, Jan 26, 2012 at 9:27 AM, Leonardo Gamas <[EMAIL PROTECTED]
> > wrote:
>
>> Luiz,
>>
>> You could create a class that implements the Writable interface and holds
>> Doc-ID and TF, let's say DocTF. Then you emit it with context.write(term,
>> DocTF) in your Mapper class.
>> Another option is to use a generic Pair<A,B>, that implements Writable,
>> to hold your data.
>>
>> P.S.: It's not mandatory to implement Writable. You could use another
>> serialization framework, but Writable will work without any additional
>> configuration.
>>
>> 2012/1/25 Luiz Antonio Falaguasta Barbosa <[EMAIL PROTECTED]>
>>
>>> People,
>>>
>>> Only for explain it better, this is figure 4:
>>>
>>> [image: image.png]
>>>
>>> It seems to be difficult to implement lines 9 to 11 of map method.
>>>
>>> Does anybody how to do this? I'd tried to find it in Ivory (
>>> http://lintool.github.com/Ivory/) and Cloud9 (
>>> https://github.com/lintool/**Cloud9 <https://github.com/lintool/Cloud9>)
>>> but I didn't get.
>>>
>>> Regards,
>>>
>>> Luiz
>>>
>>>
>>> 2012/1/25 Luiz Antonio Falaguasta Barbosa <[EMAIL PROTECTED]>
>>>
>>>> Hi people,
>>>>
>>>> Please, does somebody know where could I find an implementation of per
>>>> term inverted indexing (Ivory), like that showed in figure 4 of paper
>>>> http://www.dcs.gla.ac.uk/~richardm/papers/IPM_MapReduce.pdf ?
>>>>
>>>> I just would like to take some source code like that
>>>> http://developer.yahoo.com/hadoop/tutorial/module4.html and change it
>>>> with the per term indexing.
>>>>
>>>> Does somebody have it?
>>>>
>>>> Thanks in advance!
>>>>
>>>> Regards,
>>>>
>>>> Luiz
>>>
>>>
>>>
>>>
>>> --
>>> []s,
>>>
>>> Luiz
>>>
>>
>>
>>
>> --
>>
>> *Leonardo Gamas*
>> Software Engineer
>> T +55 (71) 3494-3514
>> C +55 (75) 8134-7440
>> [EMAIL PROTECTED]
>> www.jusbrasil.com.br
>>
>>
>
--
[]s,

Luiz