Hey guys,
I am using AvroStorage like this:
STORE alias INTO '$OUTPUT'
USING org.apache.pig.piggybank.storage.avro.AvroStorage('{
"index" : 1,
"schema_uri": "file://path/schema.avsc"}');
so, it is explicit to take the schema.avsc from the local file system, not
HDFS.
It works in a pseudo-distributed cluster, but fails on a normal cluster
with java.io.FileNotFoundException for the schema file
Looks like this is happening in backend.
I assume this is because the backend invocation of AvroStorage on a node,
different from the node I am running the pig script from, cannot find the
file in the local file system.
Why can't it use the schema file from front-end invocation?
Does it mean that I am only limited to either HDFS locations for schema_uri
or using embedding the schema string in AvroStorage parameters?
Thanks in advance
Ruslan Al-Fakikh
I am using AvroStorage like this:
STORE alias INTO '$OUTPUT'
USING org.apache.pig.piggybank.storage.avro.AvroStorage('{
"index" : 1,
"schema_uri": "file://path/schema.avsc"}');
so, it is explicit to take the schema.avsc from the local file system, not
HDFS.
It works in a pseudo-distributed cluster, but fails on a normal cluster
with java.io.FileNotFoundException for the schema file
Looks like this is happening in backend.
I assume this is because the backend invocation of AvroStorage on a node,
different from the node I am running the pig script from, cannot find the
file in the local file system.
Why can't it use the schema file from front-end invocation?
Does it mean that I am only limited to either HDFS locations for schema_uri
or using embedding the schema string in AvroStorage parameters?
Thanks in advance
Ruslan Al-Fakikh