Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Is there a java implementation to serialize a large record that does not fit in-memory?

$
0
0
I'm pretty new to avro and am trying to evaluate its potential as a
"container" for hdfs file contents.
I do know about SequenceFiles but would like a language agnostic file format
if possible.

From the java api it seems like all object serialization occurs in-memory,
which seems to bound potential record size to the amount of available
memory. Is there another java workaround or implementation that I'm not
seeing? Explanations about feasibility (or lack of) would be very helpful.

Thanks!

Viewing all articles
Browse latest Browse all 5648

Trending Articles