Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> Chain Jobs and Accumulo.


+
Juan Moreno 2012-07-16, 21:02
+
Jim Klucar 2012-07-16, 21:05
+
Marc Parisi 2012-07-16, 21:11
+
Billie J Rinaldi 2012-07-16, 21:22
+
Ed Kohlwey 2012-07-16, 21:27
+
Billie J Rinaldi 2012-07-16, 21:39
+
William Slacum 2012-07-16, 21:48
+
Juan Moreno 2012-07-16, 21:49
+
William Slacum 2012-07-16, 21:52
+
Juan Moreno 2012-07-16, 22:13
Copy link to this message
-
Re: Chain Jobs and Accumulo.
You'd basically be doing a copy of the getInputSplits and getRecordReader
methods, except they'd be returning the mapred version of those classes.

On Mon, Jul 16, 2012 at 3:13 PM, Juan Moreno
<[EMAIL PROTECTED]>wrote:

> How hard would it be to implement own version using the mapred API ?
>
> Would I have to do something as complex as InputFormatBase ? (It's a
> mammoth class)
> On Jul 16, 2012 5:53 PM, "William Slacum" <[EMAIL PROTECTED]>
> wrote:
>
>> mapred was deprecated as of 0.20.0 (
>> http://hadoop.apache.org/common/docs/r0.20.0/api/org/apache/hadoop/mapred/InputFormat.html)
>> :)
>>
>> On Mon, Jul 16, 2012 at 2:49 PM, Juan Moreno <
>> [EMAIL PROTECTED]> wrote:
>>
>>> The hadoop API is very confusing in that regard. Currently Accumulo runs
>>> atop 0.20 and in that version , mapred is the new one and mapreduce is the
>>> old one. InputFormat currently makes use of mapreduce.
>>>  On Jul 16, 2012 5:40 PM, "Billie J Rinaldi" <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>> On Monday, July 16, 2012 5:27:09 PM, "Ed Kohlwey" <[EMAIL PROTECTED]>
>>>> wrote:
>>>> > I would suggest spending the effort porting chainmapper to the new API
>>>> > (mapreduce) since the old API will eventually be removed.
>>>>
>>>> I assumed that would be true since the old API was deprecated, and that
>>>> is why we no longer support it.  However, the old API has been undeprecated
>>>> since 0.20.205.0 and 1.0.0, which seems to indicate it's not going away.
>>>>  Does anyone know what the plan for it is?
>>>>
>>>> Billie
>>>>
>>>>
>>>> > Sent from my smartphone. Please excuse any typos or shorthand.
>>>> > On Jul 16, 2012 5:22 PM, "Billie J Rinaldi" <
>>>> > [EMAIL PROTECTED] > wrote:
>>>> >
>>>> >
>>>> > On Monday, July 16, 2012 5:02:52 PM, "Juan Moreno" <
>>>> > [EMAIL PROTECTED] > wrote:
>>>> > > Hi there, I have a use case where I need to use a Chain Mapper
>>>> > > and/or
>>>> > > Reducer. The problem is that
>>>> > > the AccumuloInputFormat extends hadoop.mapreduce.InputFormat rather
>>>> > > than implementing hadoop.mapred.InputFormat
>>>> > >
>>>> > >
>>>> > > Trying to make use of org.apache.hadoop.mapred.lib.ChainMapper
>>>> > > does not work because it requires the use of the mapred package. Is
>>>> > > there a version of the AccumuloInputFormat which uses
>>>> > > the hadoop.mapred package instead? Can InputFormatBase be rewritten
>>>> > > with the newer API?
>>>> > >
>>>> > >
>>>> > > Thanks!
>>>> > > Juan
>>>> >
>>>> > I opened ACCUMULO-695 to add support for the old mapred API.
>>>> >
>>>> > Billie
>>>>
>>>
>>
+
John Vines 2012-07-16, 22:36
+
Ed Kohlwey 2012-07-16, 23:20
+
John Armstrong 2012-07-17, 15:45
+
Juan Moreno 2012-07-17, 20:12
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB