Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request: Flume adopt message from existing local Scribe

Copy link to this message
Re: Review Request: Flume adopt message from existing local Scribe

This is an automatically generated e-mail. To reply, visit:

(Updated July 30, 2012, 2:49 a.m.)
Review request for Flume and Hari Shreedharan.

hi Hari,
   Great thanks for your advice. I changed the code template and deleted additional blanks and whitespaces.
   ScribeSource is responsibility for existing Scribe, importantly, it can adopt the user interface with Scribe.
   User who wants to use ScribeScource may have the Scribe instance in there old ingest system.

   How can we test ScribeSource from Java code? Here is example:

   TSocket socket = new TSocket("Flume server address", 1499, 10000);

   TTransport transport = new TFramedTransport(socket);
   TBinaryProtocol protocol = new TBinaryProtocol(transport, false, false);

   Client client = new Client(protocol);

   List<LogEntry> list = new ArrayList<LogEntry>(n);
   for (int j = 0; j < 100; j++) {
     list.add(new LogEntry("catogory", "message"));

   client.Log(list);//Sending Scribe entry to Flume with Thrift


There may someone like me that want to replace central Scribe with Flume to adopt existing ingest system, using smooth changes for application user.
Here is the ScribeSource put into legacy folder without deserializing.
This addresses bug https://issues.apache.org/jira/browse/FLUME-1382.
Diffs (updated)

  trunk/flume-ng-dist/pom.xml 1363210
  trunk/flume-ng-sources/flume-scribe-source/pom.xml PRE-CREATION
  trunk/flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/LogEntry.java PRE-CREATION
  trunk/flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ResultCode.java PRE-CREATION
  trunk/flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java PRE-CREATION
  trunk/flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ScribeSource.java PRE-CREATION
  trunk/flume-ng-sources/pom.xml PRE-CREATION
  trunk/pom.xml 1363210

Diff: https://reviews.apache.org/r/6089/diff/

I already used ScribeSource into local environment and tested in past week. It can use the existing local Scribe interface

Denny Ye