Hi,

I am trying to use importtsv map-reduce job to load data into HBase.

I am creating TSV file after fetching data from MySQL DB using the
following java code.

But Hadoop complains that my TSV file has bad lines and no data is being
imported into HBase

Am i doing something wrong here? Any inputs would be greatly appreciated.

public class MySQLTest {

    public static void main(String[] args) {

        String jdbcURL = "jdbc:mysql://199.65.32.78/Report";
        String dbClass = "org.gjt.mm.mysql.Driver";
        String user="user";
        String passwd="password^";
        String query = "Select
reports.report_ID,reports.report_path,reports.report_time FROM reports";
        try {
            File f = new File("./XYZ.tsv");
            f.createNewFile();
            FileOutputStream fos = new FileOutputStream(f);
            Class.forName(dbClass);
            Connection con = (Connection) DriverManager.getConnection
(jdbcURL,user,passwd);
            java.sql.Statement stmt = con.createStatement();
            java.sql.ResultSet rs = stmt.executeQuery(query);
            while (rs.next()) {

                //report id
                fos.write(String.valueOf(rs.getInt(1)).getBytes());
                fos.write("\t".getBytes());
                System.out.println(rs.getInt(1));

                //report path
                fos.write(rs.getString(2).getBytes());
                fos.write("\t".getBytes());
                System.out.println(rs.getString(2));

                //report time
                fos.write(rs.getString(3).getBytes());
                fos.write("\t".getBytes());
                System.out.println(rs.getString(3));
                fos.write("\n".getBytes());

                } //end while
            fos.close();
            con.close();
        } catch (ClassNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (SQLException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

    }
}
--
Regards,
Anand

Hi,

Can you also share how exactly you invoke the import-tsv command?

On Tue, Jun 19, 2012 at 9:02 PM, AnandaVelMurugan Chandra Mohan
<[EMAIL PROTECTED]> wrote:
> Hi,
>
> I am trying to use importtsv map-reduce job to load data into HBase.
>
> I am creating TSV file after fetching data from MySQL DB using the
> following java code.
>
> But Hadoop complains that my TSV file has bad lines and no data is being
> imported into HBase
>
> Am i doing something wrong here? Any inputs would be greatly appreciated.
>
> public class MySQLTest {
>
>    public static void main(String[] args) {
>
>        String jdbcURL = "jdbc:mysql://199.65.32.78/Report";
>        String dbClass = "org.gjt.mm.mysql.Driver";
>        String user="user";
>        String passwd="password^";
>        String query = "Select
> reports.report_ID,reports.report_path,reports.report_time FROM reports";
>        try {
>            File f = new File("./XYZ.tsv");
>            f.createNewFile();
>            FileOutputStream fos = new FileOutputStream(f);
>            Class.forName(dbClass);
>            Connection con = (Connection) DriverManager.getConnection
> (jdbcURL,user,passwd);
>            java.sql.Statement stmt = con.createStatement();
>            java.sql.ResultSet rs = stmt.executeQuery(query);
>            while (rs.next()) {
>
>                //report id
>                fos.write(String.valueOf(rs.getInt(1)).getBytes());
>                fos.write("\t".getBytes());
>                System.out.println(rs.getInt(1));
>
>                //report path
>                fos.write(rs.getString(2).getBytes());
>                fos.write("\t".getBytes());
>                System.out.println(rs.getString(2));
>
>                //report time
>                fos.write(rs.getString(3).getBytes());
>                fos.write("\t".getBytes());
>                System.out.println(rs.getString(3));
>                fos.write("\n".getBytes());
>
>                } //end while
>            fos.close();
>            con.close();
>        } catch (ClassNotFoundException e) {
>            // TODO Auto-generated catch block
>            e.printStackTrace();
>        } catch (SQLException e) {
>            // TODO Auto-generated catch block
>            e.printStackTrace();
>        } catch (IOException e) {
>            // TODO Auto-generated catch block
>            e.printStackTrace();
>        }
>
>    }
> }
> --
> Regards,
> Anand

--
Harsh J

hi,

I am running the following command from hadoop bin folder.

 ./hadoop jar /usr/local/hbase-0.92.1-security/hbase-0.92.1-security.jar
importtsv -Dimporttsv.columns=HBASE_ROW_KEY,report:path,report:time
tempptmd  hdfs://<namenode>:9000/user/hadoop/temp/cbm/XYZ.tsv

My TSV file has three columns. I want first column to be used as row key.

Am i doing anything wrong? Thanks.

On Tue, Jun 19, 2012 at 9:43 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Can you also share how exactly you invoke the import-tsv command?
>
> On Tue, Jun 19, 2012 at 9:02 PM, AnandaVelMurugan Chandra Mohan
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I am trying to use importtsv map-reduce job to load data into HBase.
> >
> > I am creating TSV file after fetching data from MySQL DB using the
> > following java code.
> >
> > But Hadoop complains that my TSV file has bad lines and no data is being
> > imported into HBase
> >
> > Am i doing something wrong here? Any inputs would be greatly appreciated.
> >
> > public class MySQLTest {
> >
> >    public static void main(String[] args) {
> >
> >        String jdbcURL = "jdbc:mysql://199.65.32.78/Report";
> >        String dbClass = "org.gjt.mm.mysql.Driver";
> >        String user="user";
> >        String passwd="password^";
> >        String query = "Select
> > reports.report_ID,reports.report_path,reports.report_time FROM reports";
> >        try {
> >            File f = new File("./XYZ.tsv");
> >            f.createNewFile();
> >            FileOutputStream fos = new FileOutputStream(f);
> >            Class.forName(dbClass);
> >            Connection con = (Connection) DriverManager.getConnection
> > (jdbcURL,user,passwd);
> >            java.sql.Statement stmt = con.createStatement();
> >            java.sql.ResultSet rs = stmt.executeQuery(query);
> >            while (rs.next()) {
> >
> >                //report id
> >                fos.write(String.valueOf(rs.getInt(1)).getBytes());
> >                fos.write("\t".getBytes());
> >                System.out.println(rs.getInt(1));
> >
> >                //report path
> >                fos.write(rs.getString(2).getBytes());
> >                fos.write("\t".getBytes());
> >                System.out.println(rs.getString(2));
> >
> >                //report time
> >                fos.write(rs.getString(3).getBytes());
> >                fos.write("\t".getBytes());
> >                System.out.println(rs.getString(3));
> >                fos.write("\n".getBytes());
> >
> >                } //end while
> >            fos.close();
> >            con.close();
> >        } catch (ClassNotFoundException e) {
> >            // TODO Auto-generated catch block
> >            e.printStackTrace();
> >        } catch (SQLException e) {
> >            // TODO Auto-generated catch block
> >            e.printStackTrace();
> >        } catch (IOException e) {
> >            // TODO Auto-generated catch block
> >            e.printStackTrace();
> >        }
> >
> >    }
> > }
> > --
> > Regards,
> > Anand
>
>
>
> --
> Harsh J
>

--
Regards,
Anand

Hi,

I got this fixed, by changing my delimter to | (pipe) instead of tab. Now
it loads data into Hbase. Thanks!!

On Wed, Jun 20, 2012 at 10:30 AM, AnandaVelMurugan Chandra Mohan <
[EMAIL PROTECTED]> wrote:

>
> hi,
>
> I am running the following command from hadoop bin folder.
>
>  ./hadoop jar /usr/local/hbase-0.92.1-security/hbase-0.92.1-security.jar
> importtsv -Dimporttsv.columns=HBASE_ROW_KEY,report:path,report:time
> tempptmd  hdfs://<namenode>:9000/user/hadoop/temp/cbm/XYZ.tsv
>
> My TSV file has three columns. I want first column to be used as row key.
>
> Am i doing anything wrong? Thanks.
>
> On Tue, Jun 19, 2012 at 9:43 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Can you also share how exactly you invoke the import-tsv command?
>>
>> On Tue, Jun 19, 2012 at 9:02 PM, AnandaVelMurugan Chandra Mohan
>> <[EMAIL PROTECTED]> wrote:
>> > Hi,
>> >
>> > I am trying to use importtsv map-reduce job to load data into HBase.
>> >
>> > I am creating TSV file after fetching data from MySQL DB using the
>> > following java code.
>> >
>> > But Hadoop complains that my TSV file has bad lines and no data is being
>> > imported into HBase
>> >
>> > Am i doing something wrong here? Any inputs would be greatly
>> appreciated.
>> >
>> > public class MySQLTest {
>> >
>> >    public static void main(String[] args) {
>> >
>> >        String jdbcURL = "jdbc:mysql://199.65.32.78/Report";
>> >        String dbClass = "org.gjt.mm.mysql.Driver";
>> >        String user="user";
>> >        String passwd="password^";
>> >        String query = "Select
>> > reports.report_ID,reports.report_path,reports.report_time FROM reports";
>> >        try {
>> >            File f = new File("./XYZ.tsv");
>> >            f.createNewFile();
>> >            FileOutputStream fos = new FileOutputStream(f);
>> >            Class.forName(dbClass);
>> >            Connection con = (Connection) DriverManager.getConnection
>> > (jdbcURL,user,passwd);
>> >            java.sql.Statement stmt = con.createStatement();
>> >            java.sql.ResultSet rs = stmt.executeQuery(query);
>> >            while (rs.next()) {
>> >
>> >                //report id
>> >                fos.write(String.valueOf(rs.getInt(1)).getBytes());
>> >                fos.write("\t".getBytes());
>> >                System.out.println(rs.getInt(1));
>> >
>> >                //report path
>> >                fos.write(rs.getString(2).getBytes());
>> >                fos.write("\t".getBytes());
>> >                System.out.println(rs.getString(2));
>> >
>> >                //report time
>> >                fos.write(rs.getString(3).getBytes());
>> >                fos.write("\t".getBytes());
>> >                System.out.println(rs.getString(3));
>> >                fos.write("\n".getBytes());
>> >
>> >                } //end while
>> >            fos.close();
>> >            con.close();
>> >        } catch (ClassNotFoundException e) {
>> >            // TODO Auto-generated catch block
>> >            e.printStackTrace();
>> >        } catch (SQLException e) {
>> >            // TODO Auto-generated catch block
>> >            e.printStackTrace();
>> >        } catch (IOException e) {
>> >            // TODO Auto-generated catch block
>> >            e.printStackTrace();
>> >        }
>> >
>> >    }
>> > }
>> > --
>> > Regards,
>> > Anand
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Regards,
> Anand
>

--
Regards,
Anand