Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # general - [VOTE] Direction for Hadoop development


+
Owen OMalley 2010-11-29, 22:30
+
Doug Cutting 2010-11-29, 23:14
+
Owen OMalley 2010-11-30, 00:22
+
Doug Cutting 2010-12-01, 15:21
+
Owen OMalley 2010-12-01, 15:40
+
Doug Cutting 2010-12-06, 19:30
+
Chris Douglas 2010-12-06, 22:40
+
Doug Cutting 2010-12-06, 23:45
+
Roy T. Fielding 2010-12-07, 01:09
+
Arun C Murthy 2010-12-07, 16:45
+
Doug Cutting 2010-12-07, 17:18
+
Roy T. Fielding 2010-12-07, 22:37
+
Doug Cutting 2010-12-08, 18:12
+
Owen OMalley 2010-12-14, 03:08
+
Eric Sammer 2010-12-14, 04:49
+
Owen OMalley 2010-12-14, 05:43
+
Eric Sammer 2010-12-14, 07:14
+
Owen OMalley 2010-12-14, 19:08
+
Jay Booth 2010-12-01, 16:29
+
Scott Carey 2010-12-08, 03:33
+
Konstantin Shvachko 2010-12-01, 01:57
+
Owen OMalley 2010-12-01, 19:11
+
Owen OMalley 2010-12-06, 17:16
+
Chris Douglas 2010-12-06, 18:40
+
Arun C Murthy 2010-12-06, 18:46
+
Tom White 2010-12-06, 21:14
+
Konstantin Shvachko 2010-12-07, 11:27
+
Doug Cutting 2010-12-07, 17:22
+
Konstantin Shvachko 2010-12-07, 18:26
+
Doug Cutting 2010-12-08, 18:55
+
Steve Loughran 2010-12-01, 12:25
+
Eric Sammer 2010-12-07, 03:36
+
Owen OMalley 2010-12-07, 08:13
+
Jeff Hammerbacher 2010-12-07, 10:23
+
Arun C Murthy 2010-12-07, 16:12
+
Doug Cutting 2010-12-07, 17:26
+
Owen OMalley 2010-12-07, 18:25
Copy link to this message
-
Re: [VOTE] Direction for Hadoop development
Doug Cutting 2010-12-08, 19:20
On 12/07/2010 10:25 AM, Owen O'Malley wrote:
> The new code reads the new or old versions of SequenceFile seamlessly
> using auto-detection of the version. The old code fails with an explicit
> message saying that it can't read this version. This is the only
> mechanism available when upgrading a file format with a single version
> number and is the mechanism that we've used 6 times in the past.

The last such change was nearly four years ago, in:

https://issues.apache.org/jira/browse/HADOOP-732

The quantity of data stored in SequenceFiles has greatly increased over
the past four years.  The project's concern for compatibility has also
correspondingly increased over that time.

The new format version might not be written when folks are using
Writable or some other serialization currently supported by
SequenceFile.  The only situation in your patch where the new version is
required is for Avro.  You might simply drop support for Avro and leave
the file version number alone since Avro already includes a container
file format.  Or you might only use the new format version for
non-class-determined serializations like Avro.  Or you might use
SequenceFile's existing metadata for non-class-determined serializations
like Avro and leave the file version number alone.

Doug
+
Eric Sammer 2010-12-07, 18:08
+
Arun C Murthy 2010-12-07, 15:55
+
Jay Booth 2010-12-07, 16:06