Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Unable to compile a namespace-less schema


Copy link to this message
-
Re: Unable to compile a namespace-less schema
Thanks Sean.

Unfortunately, I already have massive amounts of data that were written
without a namespace (from Apache Pig's AvroStorage). Is there any way to
read it using specific records in real code (i.e. one that has packages)?

Vitaly
On Fri, Oct 11, 2013 at 2:15 PM, Sean Busbey <[EMAIL PROTECTED]> wrote:

> Hi Vitaly,
>
> In general, Java does not allow classes outside of the default package to
> import classes from within the default package.
>
> I think this means that this is expected behavior, given that Avro says
> non-namespaced schemas generate code in the default package.
>
> For your particular issue, this means either restructuring your
> application so that Main is in the default package or giving the schemas a
> namespace so they'll be in some package. The latter is preferable; in your
> example com.company or com.company.serialization would be appropriate.
>
> HTH
>
>
> On Fri, Oct 11, 2013 at 3:59 PM, Vitaly Gordon <[EMAIL PROTECTED]> wrote:
>
>> Hi Doug,
>> I've attached a maven project that contains code that shows the problem.
>> The code is basically the same one from the Avro guide, but what is
>> important to observe is that since the Main class has a package define, it
>> cannot import the classes generated from the namespace-less schema.
>>
>> just run mvn:compile to get the compilation errors
>>
>> Thanks,
>> Vitaly
>>
>>
>> On Thu, Oct 10, 2013 at 1:58 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>>
>>> I encourage you to please provide a complete test, code that fails.
>>> If maven is involved, create a simple, Maven project that illustrates
>>> the problem.
>>>
>>> Thanks,
>>>
>>> Doug
>>>
>>> On Wed, Oct 9, 2013 at 11:21 PM, Vitaly Gordon <[EMAIL PROTECTED]>
>>> wrote:
>>> > Hi Doug,
>>> > You are right, the code does compile with javac. Apparently it is some
>>> maven
>>> > error, where it doesn't like to compile package-less files.
>>> >
>>> > Having said that, I still have the issue of not being able to use
>>> these java
>>> > files in my code, because there is no way to import them. One thing I
>>> tried
>>> > that sometimes work is adding some arbitrary namespace to the avro
>>> schema.
>>> > However, when I try to read records using the new (with namespace)
>>> schema, I
>>> > get in return a generic record instead of a specific one. This
>>> behavior can
>>> > be observed in the same file I attached by adding an arbitrary
>>> namespace to
>>> > the schema before generating the Java classes from it.
>>> >
>>> > Is there any way to read specific records when the schema that was
>>> used to
>>> > write them contains no namespace?
>>> >
>>> > Thanks,
>>> > Vitaly
>>> >
>>> >
>>> > On Wed, Oct 9, 2013 at 6:07 PM, Doug Cutting <[EMAIL PROTECTED]>
>>> wrote:
>>> >>
>>> >> Using the current trunk of Avro I am able to:
>>> >>  - extract the schema from the data file you provided (using
>>> >> avro-tools schema command)
>>> >>  - generate Java classes for this schema (using the avro-tools compile
>>> >> command)
>>> >>  - compile these generated Java classes (using the javac command)
>>> >>
>>> >> Can you provide a complete case of what fails for you?
>>> >>
>>> >> Thanks,
>>> >>
>>> >> Doug
>>> >>
>>> >> On Wed, Oct 9, 2013 at 4:56 PM, Vitaly Gordon <[EMAIL PROTECTED]>
>>> wrote:
>>> >> > Does anyone else might have an idea how I can resolve this
>>> >> > "namespace-less"
>>> >> > Avro schema code generation?
>>> >> >
>>> >> > Thanks,
>>> >> > Vitaly
>>> >> >
>>> >> >
>>> >> > On Mon, Oct 7, 2013 at 2:04 PM, Vitaly Gordon <[EMAIL PROTECTED]>
>>> wrote:
>>> >> >>
>>> >> >> Hi Sean,
>>> >> >> Here is a file that contains a single record that I cannot read
>>> using a
>>> >> >> specific reader.
>>> >> >>
>>> >> >> It's hard for me to add code because the problem is a compilation
>>> >> >> problem
>>> >> >> with the generated Java files.
>>> >> >>
>>> >> >> So to recreate the problem:
>>> >> >> 1. Extract the schema from the record
>>> >> >> 2. Generate the code from the schema
>>> >> >> 3. Compile
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB