This has probably changed with the Java code refactor, but I've posted
some answers inline, to the best of my understanding.



On 12/16/2017 12:17 PM, Animesh Trivedi wrote:
The write/reader interfaces are parallel implementations that make some
things easier, but don't encompass all available functionality (for
example, fixed size lists, nested lists, some dictionary operations,
etc). However, you should be able to accomplish everything using
The safe methods ensure that the vector is large enough to set the
value. You can use the unsafe versions if you know that your vector has
already allocated enough space for your data.
> 4. What are MinorTypes?
Minor types are a representation of the different vector types. I
believe they are being de-emphasized in favor of FieldTypes, as minor
types don't contain enough information to represent all vectors.
> 5. For a writer, what is a dictionary provider? For example in the
> code, the reader is given as the dictionary provider for
> the writer. But, is it something more than just:
> DictionaryProvider.MapDictionaryProvider provider = new
> DictionaryProvider.MapDictionaryProvider();
> ArrowFileWriter arrowWriter = new ArrowFileWriter(root, provider,
> fileOutputStream.getChannel());
The dictionary provider is an interface for looking up dictionary
values. When reading a file, the reader itself has already read the
dictionaries and thus serves as the provider.
> 6. I am not clearly sure about the sequence of call that one needs to do
> write on mutators. For example, if I code something like
> NullableIntVector intVector = (NullableIntVector) fieldVector;
> NullableIntVector.Mutator mutator = intVector.getMutator();
> [.write num values]
> mutator.setValueCount(num)
> then this works for primitive types, but not for VarBinary type. There I
> have to set the capacity first,
> NullableVarBinaryVector varBinaryVector = (NullableVarBinaryVector)
> fieldVector;
> varBinaryVector.setInitialCapacity(items);
> varBinaryVector.allocateNew();
> NullableVarBinaryVector.Mutator mutator = varBinaryVector.getMutator();
The method calls are not very well documented - I would suggest looking
at the reader/writer implementations to see what calls are required for
which vector types. Generally variable length vectors (lists, var
binary, etc) behave differently than fixed width vectors (ints, longs, etc).
> Example of these are here:
> (writeField[???] functions).
> Thank you very much,
> --
> Animesh
> On Thu, Dec 14, 2017 at 6:15 PM, Wes McKinney <[EMAIL PROTECTED]> wrote:
>> hi Animesh,
>> I suggest you try the ArrowStreamReader/Writer or
>> ArrowFileReader/Writer classes. See
>> src/main/java/org/apache/arrow/tools/
>> for example working code for this
>> - Wes
>> On Thu, Dec 14, 2017 at 8:30 AM, Animesh Trivedi
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB