Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Shading all modules in a project

$
0
0
Hello,

Over in Apache HBase, we have an interesting problem that I'm hoping the
shade plugin can help us solve. It all started with a performance
enhancement to reduce garbage by creating a subclass of a protobuf type
[0]. It works fine for us most of the time. However, when using this code
from a Hadoop mapreduce job, things get strange. Due to the permissions on
the parent object and the way Hadoop uses "fat" jars, we now have
classloader troubles [1]. We were able to work around the issue by
documenting the required classpath constructions. However, this is not
possible for all users. This is where the shade plugin comes in.

I gave shade a shot [2], though I admit I'm not terribly familiar with it,
nor the inner workings of maven. As far as I can tell, the trouble is the
plugin invocation is limited to the single module in our multi-module
project. I think what's happening is shade is doing what I want for the
hbase-protocol module, but I also need it to do bytecode rewriting for the
modules that consume this module as well. Luckily, this detail is not part
of our public API so it should be transparent to our users.

My patch is posted on [2]. Any pointers/advice is greatly appreciated.

Thanks in advance,
Nick

[0]: https://issues.apache.org/jira/browse/HBASE-9867
[1]: https://issues.apache.org/jira/browse/HBASE-10304
[2]: https://issues.apache.org/jira/browse/HBASE-11118

Viewing all articles
Browse latest Browse all 5648

Trending Articles