Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # dev - Setting Charset in getBytes() call.


+
David Medinets 2012-10-28, 21:50
+
Ed Kohlwey 2012-10-28, 22:18
+
William Slacum 2012-10-29, 15:39
+
David Medinets 2012-10-29, 16:00
+
Josh Elser 2012-10-29, 16:21
+
Benson Margulies 2012-10-29, 16:24
+
John Vines 2012-10-29, 16:42
+
Josh Elser 2012-10-29, 16:57
+
David Medinets 2012-10-29, 17:00
+
William Slacum 2012-10-29, 17:13
Copy link to this message
-
Re: Setting Charset in getBytes() call.
Mike Drob 2012-10-29, 17:16
One specific use case is when creating a new connection, the password is
passed as a byte[], when I expect most sane applications will treat it as a
String (either via reading it from a file, or reading it from a terminal
input). If somebody creates the password with a different platform encoding
than what the programmer expects, then it will cause a lock out that is
very difficult to debug.

On topic to the original question, if anybody is brave enough to use Java
7, then there are predefined constants in the JDK -
http://docs.oracle.com/javase/7/docs/api/java/nio/charset/StandardCharsets.html

UTF-8 should always be present (according to the JLS), and as a multi-byte
format should be able to encode any character that you would need to. I've
had this conversation with Keith before, so hopefully he can weigh in on
this.

Mike

On Mon, Oct 29, 2012 at 12:57 PM, Josh Elser <[EMAIL PROTECTED]> wrote:

> Benson, perhaps "contrived" would have been better than "hypothetical" :).
> That being said, I also hadn't thought about other JVM implementations.
>
> I wonder if leaving a commented note in the accumulo-env.sh script for
> alternative namings for the "file.encoding" name and the JVM it applies to
> would be sufficient?
>
> David, can you give some sort of feel for the usages of the getBytes()
> calls? Since most of the API deals with things in terms of Text and byte[]
> (Key and Value decomposed), are most of the usages configuration/user-input
> based as your initial snippet from InputFormatBase showed?
>
>
> On 10/29/2012 12:42 PM, John Vines wrote:
>
>> Are there any experts when it comes to character encodings? First of all,
>> I
>> would like to make sure there are no sacrifices being made by forcing
>> UTF-8.
>>
>>  From there, if I think JVM properties is the way to go. Should there be
>> ANY
>> sort of shortfall with UTF-8, we should allow users to switch the encoding
>> to the type of their pleasure. We can tweak the scripts to set the jvm
>> property but still allow users to override should they need it in their
>> setup. This allows us to not only avoid a massive code change, it also
>> makes it easier for users to switch to an encoding should they have a need
>> to.
>>
>> John
>>
>> On Mon, Oct 29, 2012 at 12:24 PM, Benson Margulies <[EMAIL PROTECTED]
>> >wrote:
>>
>>  On Mon, Oct 29, 2012 at 12:21 PM, Josh Elser <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>> David, I beg to differ.
>>>>
>>>> Setting it via the JVM property is a single change to make, whereas if
>>>>
>>> you
>>>
>>>> change every single usage of getBytes(), you now forced the next person
>>>>
>>> to
>>>
>>>> branch the code, change everything to UTF16 (hypothetical use case) and
>>>> continue a diverged codebase forever.
>>>>
>>>
>>> Typically, the reason(s) that people don't take this approach are:
>>>
>>> a: a fear that other JVMs don't have this parameter, or don't have it
>>> under the same name.
>>> b: a desire to read or write files for uses in 'the platform encoding'
>>> whatever it is, in addition to whatever needs to be done in UTF-8.
>>>
>>> I'd be very surprised if Accumulo ever decided to do this sort of
>>> thing in UTF-16.
>>>
>>>
>>>
>>>> I would say that the reason that such a JVM property exists is to
>>>>
>>> alleviate
>>>
>>>> you from having to make these code changes in the first place.
>>>>
>>>> On 10/29/2012 12:00 PM, David Medinets wrote:
>>>>
>>>>>
>>>>> I like the idea of making the change explicit in the source code.
>>>>> Setting the encoding in the jvm property would be easier but not as
>>>>> explicit. I have a few dozen of the files changed. Today I have free
>>>>> time since Hurricane Sandy has closed offices.
>>>>>
>>>>> On Mon, Oct 29, 2012 at 11:39 AM, William Slacum
>>>>> <wilhelm.von.cloud@accumulo.**net <[EMAIL PROTECTED]>>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> Isn't it easier to just set the JVM property `file.encoding`?
>>>>>>
>>>>>> On Sun, Oct 28, 2012 at 3:18 PM, Ed Kohlwey <[EMAIL PROTECTED]>
+
Michael Flester 2012-10-29, 19:14
+
John Vines 2012-10-29, 19:18
+
Benson Margulies 2012-10-29, 20:02
+
David Medinets 2012-10-29, 20:29
+
Michael Flester 2012-10-30, 00:27
+
Josh Elser 2012-10-30, 00:46
+
Benson Margulies 2012-10-30, 00:54
+
Josh Elser 2012-10-30, 01:57
+
John Vines 2012-10-30, 02:08
+
David Medinets 2012-10-30, 02:47
+
Josh Elser 2012-10-30, 22:27
+
David Medinets 2012-10-30, 23:47
+
Josh Elser 2012-10-31, 00:21
+
Benson Margulies 2012-10-31, 00:31
+
William Slacum 2012-10-31, 00:41
+
David Medinets 2012-10-31, 02:29
+
John Vines 2012-10-31, 02:35
+
Christopher Tubbs 2012-10-31, 18:02
+
Marc Parisi 2012-11-02, 12:24
+
Benson Margulies 2012-11-02, 19:56
+
John Vines 2012-11-02, 20:18
+
Christopher Tubbs 2012-11-03, 01:54
+
David Medinets 2012-11-03, 03:34
+
Josh Elser 2012-11-02, 23:34
+
Drew Farris 2012-10-30, 01:22
+
Adam Fuchs 2012-10-30, 20:26
+
Ed Kohlwey 2012-10-30, 01:44
+
Ed Kohlwey 2012-10-30, 01:54
+
Eric Newton 2012-10-30, 20:02
+
Marc Parisi 2012-10-30, 22:28
+
Marc Parisi 2012-10-30, 22:31
+
Benson Margulies 2012-10-30, 23:26