Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Scribe Source .thrift file


+
Hari Shreedharan 2013-02-03, 00:14
+
Juhani Connolly 2013-02-05, 02:30
+
Denny Ye 2013-02-05, 02:46
+
Hari Shreedharan 2013-02-05, 04:05
+
Hari Shreedharan 2013-02-05, 06:57
Copy link to this message
-
Re: Scribe Source .thrift file
hi Hari, the original version of generated code from that Thrift template
has only support message with 'String' type, I think it's ineffective that
deserialized from byte stream to String again. That's CPU sensitive work
and useless in progress of data flow. Thus, I modified code of LogEntry to
support byte array incoming from socket except deserializing.

What should I do next?
2013/2/5 Hari Shreedharan <[EMAIL PROTECTED]>

> Denny,
>
> I modified the file's namespace and the final file looks like this:
>
> namespace java org.apache.flume.source.scribe
>
> enum ResultCode
> {
> OK,
> TRY_LATER
> }
>
> struct LogEntry
> {
> 1: string category,
> 2: string message
> }
>
> service Scribe
> {
> ResultCode Log(1: list<LogEntry> messages);
> }
>
>
>
> But it looks like the ScribeSource.java (line 154) expects
> LogEntry.message to be "binary" and not "string." Since there are no unit
> tests, I cannot verify that making message binary will not break something.
> Could you give me some fix for this?
>
>
> Thanks
> Hari
>
> --
> Hari Shreedharan
>
>
> On Monday, February 4, 2013 at 8:05 PM, Hari Shreedharan wrote:
>
> > Hi Denny, Juhani,
> >
> > Thanks! I will add this file to the compileThrift profile, so we can
> update the thrift generated code when we upgrade thrift.
> >
> > Thanks!
> > Hari
> >
> > --
> > Hari Shreedharan
> >
> >
> > On Monday, February 4, 2013 at 6:46 PM, Denny Ye wrote:
> >
> > > Sorry for late response
> > >
> > > Here is the scribe.thrift file :
> > > ************************* file separator *********************
> > > namespace java scribe.thrift
> > >
> > > enum ResultCode
> > > {
> > > OK,
> > > TRY_LATER
> > > }
> > >
> > > struct LogEntry
> > > {
> > > 1: string category,
> > > 2: string message
> > > }
> > >
> > > service scribe
> > > {
> > > ResultCode Log(1: list<LogEntry> messages);
> > > }
> > > ************************* file separator *********************
> > >
> > > To Juhani, I haven't use the facebook template, there have so many
> > > additional interfaces that I don't like it.
> > >
> > >
> > >
> > > 2013/2/5 Juhani Connolly <[EMAIL PROTECTED] (mailto:
> [EMAIL PROTECTED])>
> > >
> > > > This is the file we want:
> > > >
> > > > https://github.com/facebook/**scribe/blob/master/if/scribe.**thrift<
> https://github.com/facebook/scribe/blob/master/if/scribe.thrift>
> > > >
> > > > It's apache 2 licensed but also has a facebooke copyright header. I
> think
> > > > this is why we didn't originally include it.
> > > >
> > > >
> > > > On 02/03/2013 09:14 AM, Hari Shreedharan wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > The scribe source has some thrift generated code committed, but no
> > > > > ".thrift" file. This makes upgrading to a new version of libthrift
> > > > > impossible.
> > > > >
> > > > > I believe Denny Ye contributed this code, so Denny, it'd be nice
> if you
> > > > > could submit the ".thrift" file as soon as possible.
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Hari
> > > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
>
>
+
Hari Shreedharan 2013-02-05, 07:53
+
Denny Ye 2013-02-05, 08:08
+
Hari Shreedharan 2013-02-05, 08:10
+
Denny Ye 2013-02-05, 08:16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB