Hello,

I wrote a single program to directly write data content to HFile.
Hbase is installed as pseudo-mode.

here is my code:
public static void putData() throws Exception{
FileSystem fs = new RawLocalFileSystem();
fs.setConf(new Configuration());
//block size is 2K
HFile.Writer hwriter = new HFile.Writer(fs, new
Path("hdfs://localhost:8020/test"), 2, (Compression.Algorithm)null,
null);
//value size is 1K, it means one block can contain 2 key/value pairs
byte[] key1 = "tom".getBytes();
byte[] value1 = new byte[1024];
for(int i=0;i<1024;i++){
value1[i] = 'a';
}
hwriter.append(key1, value1);

byte[] key2 = "jim".getBytes();
byte[] value2 = new byte[1024];
for(int i=0;i<1024;i++){
value2[i] = 'b';
}
}
after I run this method, it returned error information as follows:
Exception in thread "main" java.lang.IllegalArgumentException: Wrong
FS: hdfs://localhost:8020/test, expected: file:///.

How can I set the second parameter in the HFile.Writer constructor?

Thanks!

Yong

yonghu <yongyong313@...> writes:

>
> Hello,
>...

try something like that:

Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);

HFile.Writer hwriter = new HFileWriterV2(conf, new CacheConfig(conf), fs, new
Path(fs.getWorkingDirectory() + "/foo"));

Thanks for your reply. But I am using hbase 0.90.2. There is no
HFileWriterV2 class. Can you show me how to use HFileWriter
constructor?

Thanks

Yong

On Tue, Mar 6, 2012 at 3:19 PM, Konrad Tendera <[EMAIL PROTECTED]> wrote:
> yonghu <yongyong313@...> writes:
>
>>
>> Hello,
>>...
>
> try something like that:
>
> Configuration conf = new Configuration();
> FileSystem fs = FileSystem.get(conf);
>
> HFile.Writer hwriter = new HFileWriterV2(conf, new CacheConfig(conf), fs, new
> Path(fs.getWorkingDirectory() + "/foo"));
>

On Tue, Mar 6, 2012 at 6:48 AM, yonghu <[EMAIL PROTECTED]> wrote:
> Thanks for your reply. But I am using hbase 0.90.2. There is no
> HFileWriterV2 class. Can you show me how to use HFileWriter
> constructor?
>

Find the equivalent constructor in your code base.

You should upgrade to 0.90.5 at least if not 0.92.0.  A bunch has been
fixed since 0.90.2.

St.Ack

Thanks for your reply. I have already solve the problem.

Yong

On Tue, Mar 6, 2012 at 5:02 PM, Stack <[EMAIL PROTECTED]> wrote:
> On Tue, Mar 6, 2012 at 6:48 AM, yonghu <[EMAIL PROTECTED]> wrote:
>> Thanks for your reply. But I am using hbase 0.90.2. There is no
>> HFileWriterV2 class. Can you show me how to use HFileWriter
>> constructor?
>>
>
> Find the equivalent constructor in your code base.
>
> You should upgrade to 0.90.5 at least if not 0.92.0.  A bunch has been
> fixed since 0.90.2.
>
> St.Ack

On Tue, Mar 6, 2012 at 8:07 AM, yonghu <[EMAIL PROTECTED]> wrote:
> Thanks for your reply. I have already solve the problem.
>

What did you do to fix it?
Thanks,
St.Ack

As shown in my code, I used RawLocalFileSystem() to return a
FileSystem reference. But when I create a HFile.Writer, in the path
parameter, I try to create a path in HDFS. There are two ways to
change the code into correct.

The first one is looks like:

FileSystem fs = new RawLocalFileSystem();
fs.setConf(new Configuration());
HFile.Writer hwriter = new HFile.Writer(fs, new Path("test"), 2,
(Compression.Algorithm)null, null);
the data is stored in the file:///.

The second one is looks like:

Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);
HFile.Writer hwriter = new HFile.Writer(fs, new
Path("hdfs://localhost:8020/test"), 2, (Compression.Algorithm)null,
null);
the data is stored in the hdfs.

regards!

Yong
On Tue, Mar 6, 2012 at 5:10 PM, Stack <[EMAIL PROTECTED]> wrote:
> On Tue, Mar 6, 2012 at 8:07 AM, yonghu <[EMAIL PROTECTED]> wrote:
>> Thanks for your reply. I have already solve the problem.
>>
>
> What did you do to fix it?
> Thanks,
> St.Ack