Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Unable to compile a namespace-less schema


Copy link to this message
-
Re: Unable to compile a namespace-less schema
Thanks Sean.

Unfortunately, I already have massive amounts of data that were written
without a namespace (from Apache Pig's AvroStorage). Is there any way to
read it using specific records in real code (i.e. one that has packages)?

Vitaly
On Fri, Oct 11, 2013 at 2:15 PM, Sean Busbey <[EMAIL PROTECTED]> wrote:

> Hi Vitaly,
>
> In general, Java does not allow classes outside of the default package to
> import classes from within the default package.
>
> I think this means that this is expected behavior, given that Avro says
> non-namespaced schemas generate code in the default package.
>
> For your particular issue, this means either restructuring your
> application so that Main is in the default package or giving the schemas a
> namespace so they'll be in some package. The latter is preferable; in your
> example com.company or com.company.serialization would be appropriate.
>
> HTH
>
>
> On Fri, Oct 11, 2013 at 3:59 PM, Vitaly Gordon <[EMAIL PROTECTED]> wrote:
>
>> Hi Doug,
>> I've attached a maven project that contains code that shows the problem.
>> The code is basically the same one from the Avro guide, but what is
>> important to observe is that since the Main class has a package define, it
>> cannot import the classes generated from the namespace-less schema.
>>
>> just run mvn:compile to get the compilation errors
>>
>> Thanks,
>> Vitaly
>>
>>
>> On Thu, Oct 10, 2013 at 1:58 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>>
>>> I encourage you to please provide a complete test, code that fails.
>>> If maven is involved, create a simple, Maven project that illustrates
>>> the problem.
>>>
>>> Thanks,
>>>
>>> Doug
>>>
>>> On Wed, Oct 9, 2013 at 11:21 PM, Vitaly Gordon <[EMAIL PROTECTED]>
>>> wrote:
>>> > Hi Doug,
>>> > You are right, the code does compile with javac. Apparently it is some
>>> maven
>>> > error, where it doesn't like to compile package-less files.
>>> >
>>> > Having said that, I still have the issue of not being able to use
>>> these java
>>> > files in my code, because there is no way to import them. One thing I
>>> tried
>>> > that sometimes work is adding some arbitrary namespace to the avro
>>> schema.
>>> > However, when I try to read records using the new (with namespace)
>>> schema, I
>>> > get in return a generic record instead of a specific one. This
>>> behavior can
>>> > be observed in the same file I attached by adding an arbitrary
>>> namespace to
>>> > the schema before generating the Java classes from it.
>>> >
>>> > Is there any way to read specific records when the schema that was
>>> used to
>>> > write them contains no namespace?
>>> >
>>> > Thanks,
>>> > Vitaly
>>> >
>>> >
>>> > On Wed, Oct 9, 2013 at 6:07 PM, Doug Cutting <[EMAIL PROTECTED]>
>>> wrote:
>>> >>
>>> >> Using the current trunk of Avro I am able to:
>>> >>  - extract the schema from the data file you provided (using
>>> >> avro-tools schema command)
>>> >>  - generate Java classes for this schema (using the avro-tools compile
>>> >> command)
>>> >>  - compile these generated Java classes (using the javac command)
>>> >>
>>> >> Can you provide a complete case of what fails for you?
>>> >>
>>> >> Thanks,
>>> >>
>>> >> Doug
>>> >>
>>> >> On Wed, Oct 9, 2013 at 4:56 PM, Vitaly Gordon <[EMAIL PROTECTED]>
>>> wrote:
>>> >> > Does anyone else might have an idea how I can resolve this
>>> >> > "namespace-less"
>>> >> > Avro schema code generation?
>>> >> >
>>> >> > Thanks,
>>> >> > Vitaly
>>> >> >
>>> >> >
>>> >> > On Mon, Oct 7, 2013 at 2:04 PM, Vitaly Gordon <[EMAIL PROTECTED]>
>>> wrote:
>>> >> >>
>>> >> >> Hi Sean,
>>> >> >> Here is a file that contains a single record that I cannot read
>>> using a
>>> >> >> specific reader.
>>> >> >>
>>> >> >> It's hard for me to add code because the problem is a compilation
>>> >> >> problem
>>> >> >> with the generated Java files.
>>> >> >>
>>> >> >> So to recreate the problem:
>>> >> >> 1. Extract the schema from the record
>>> >> >> 2. Generate the code from the schema
>>> >> >> 3. Compile