Hello Jena-Users,
In the jena-dev mailing list, on March 31, Andy announced a preliminary
build of Fuseki 2:
http://mail-archives.apache.org/mod_mbox/jena-dev/201403.mbox/%3C53394253.1070506%40apache.org%3E
He ended that post with the following invitation: “If you integrate
Fuseki with your own systems,
then it would be good to discuss how to make that easier because now is
a good time to make
changes.”
Here I will address that topic from my team’s point of view. I see
this topic as somewhere near the
boundary of “jena-users” and “jena-dev”, so perhaps we can
start the discussion here in the user
space, and then promote to the dev space as warranted.
We use RDF and Jena as the symbolic data backbone of our Glue-AI social
robotics platform,
http://glue.ai. It contains a sub-project called
http://appdapter.org which largely consists
of wrappers of Jena features.
There are several technology connection issues we face that I think may
be of general interest.
To boil down our platform as succinctly as I can:
1) Code is in Java and Scala
2) Build, deploy, and distribute using Maven
3) Semantic data access using Jena (ARQ, Fuseki), persistence using TDB
and AllegroGraph.
4) Packaging of components using OSGi bundles (run under Felix or other
container).
5) Webapp deployment using the Ops4j PAX-Web OSGi wrapper for Jetty.
(This component is quite spiffy, and demonstrates some of the power of
the OSGi platform).
https://ops4j1.jira.com/wiki/display/paxweb/Pax+Web
6) Semantic data change notification and other (non-semantic) data
plumbing using AMQP pub-sub Topics, implemented by Apache QPid, which
is compatible with JMS 1.1.
http://qpid.apache.org/
The interesting intersections occur between Jena/Fuseki #3, OSGi
PAX-Web webapp deployment #5,
and AMQP/QPid/JMS topics #6.
The component we really want to run is an HTTP-enabled SPARQL(+Update)
engine with TDB persistence
(i.e. Fuseki), deployed as a bundle to be discovered and exposed to the
web by PAX-Web,
and able to supply AMQP notifications when graphs change. Our initial
approach to the latter is to
publish to a single Topic called “GraphChangeNotice”, using a range
of QPid “subjects” corresponding
to the different graphs in our datasets. Thus any consumer agent in
our network can easily subscribe
to notices on the graphs it is interested in. An unfinished design
area is the exact representation
of graph change notices; many forms are possible but they are naturally
all similar to a bag of Quads
plus some metadata indicating when, where, and how the change occurred.
There is also a natural
relationship between these notifications and the transaction model used
by our ARQ datasets.
We have some prototypes of each aspect of this component I have
described, but we do not yet have a
complete implementation, in part because of challenges stemming from
the structure of older
versions of Fuseki and Joseki. Of course we can always hack the
features we want on top of
Fuseki source, but then we worry about the cost of maintaining
compatibility with future versions
of the original.
The new ability of Fuseki 2 to compile into a deployable .WAR webapp
will help us somewhat. From
there we will be looking for ways to layer on the kind of graph
monitoring feature that Andy kindly
sketched out for us in this thread from May 2014:
“Notification of Graph Changes by ARQ+TDB, during or after SPARQL Update”
http://mail-archives.apache.org/mod_mbox/jena-users/201405.mbox/%3C5378C885.3030306%40apache.org%3E
...and then connect those notifications to our AMQP topics as I
described above.
Given all this context, I could make suggestions about what I would
most like to see done with the
Fuseki codebase going forward, but I think first it would be good to
find some consensus about how
valuable these two feature categories are seen to be (or not to be) by
the Jena community:
A) AMQP / JMS publishing of graph change notices
B) OSGi component bundling
The purpose and utility of #A is presumably obvious, while the benefits
of #B are more complex and
subtle. Overall, I think the entire Jena stack would benefit from
some attention to the OSGi
universe (which does not require breaking compatibility with the
non-OSGi universe), but I don’t want
to launch into a manifest-o about that unless other folks are really
interested in the subject.
I look forward to hearing what other Jena users and developers think
about these topics!
peace,
Stu B22
In the jena-dev mailing list, on March 31, Andy announced a preliminary
build of Fuseki 2:
http://mail-archives.apache.org/mod_mbox/jena-dev/201403.mbox/%3C53394253.1070506%40apache.org%3E
He ended that post with the following invitation: “If you integrate
Fuseki with your own systems,
then it would be good to discuss how to make that easier because now is
a good time to make
changes.”
Here I will address that topic from my team’s point of view. I see
this topic as somewhere near the
boundary of “jena-users” and “jena-dev”, so perhaps we can
start the discussion here in the user
space, and then promote to the dev space as warranted.
We use RDF and Jena as the symbolic data backbone of our Glue-AI social
robotics platform,
http://glue.ai. It contains a sub-project called
http://appdapter.org which largely consists
of wrappers of Jena features.
There are several technology connection issues we face that I think may
be of general interest.
To boil down our platform as succinctly as I can:
1) Code is in Java and Scala
2) Build, deploy, and distribute using Maven
3) Semantic data access using Jena (ARQ, Fuseki), persistence using TDB
and AllegroGraph.
4) Packaging of components using OSGi bundles (run under Felix or other
container).
5) Webapp deployment using the Ops4j PAX-Web OSGi wrapper for Jetty.
(This component is quite spiffy, and demonstrates some of the power of
the OSGi platform).
https://ops4j1.jira.com/wiki/display/paxweb/Pax+Web
6) Semantic data change notification and other (non-semantic) data
plumbing using AMQP pub-sub Topics, implemented by Apache QPid, which
is compatible with JMS 1.1.
http://qpid.apache.org/
The interesting intersections occur between Jena/Fuseki #3, OSGi
PAX-Web webapp deployment #5,
and AMQP/QPid/JMS topics #6.
The component we really want to run is an HTTP-enabled SPARQL(+Update)
engine with TDB persistence
(i.e. Fuseki), deployed as a bundle to be discovered and exposed to the
web by PAX-Web,
and able to supply AMQP notifications when graphs change. Our initial
approach to the latter is to
publish to a single Topic called “GraphChangeNotice”, using a range
of QPid “subjects” corresponding
to the different graphs in our datasets. Thus any consumer agent in
our network can easily subscribe
to notices on the graphs it is interested in. An unfinished design
area is the exact representation
of graph change notices; many forms are possible but they are naturally
all similar to a bag of Quads
plus some metadata indicating when, where, and how the change occurred.
There is also a natural
relationship between these notifications and the transaction model used
by our ARQ datasets.
We have some prototypes of each aspect of this component I have
described, but we do not yet have a
complete implementation, in part because of challenges stemming from
the structure of older
versions of Fuseki and Joseki. Of course we can always hack the
features we want on top of
Fuseki source, but then we worry about the cost of maintaining
compatibility with future versions
of the original.
The new ability of Fuseki 2 to compile into a deployable .WAR webapp
will help us somewhat. From
there we will be looking for ways to layer on the kind of graph
monitoring feature that Andy kindly
sketched out for us in this thread from May 2014:
“Notification of Graph Changes by ARQ+TDB, during or after SPARQL Update”
http://mail-archives.apache.org/mod_mbox/jena-users/201405.mbox/%3C5378C885.3030306%40apache.org%3E
...and then connect those notifications to our AMQP topics as I
described above.
Given all this context, I could make suggestions about what I would
most like to see done with the
Fuseki codebase going forward, but I think first it would be good to
find some consensus about how
valuable these two feature categories are seen to be (or not to be) by
the Jena community:
A) AMQP / JMS publishing of graph change notices
B) OSGi component bundling
The purpose and utility of #A is presumably obvious, while the benefits
of #B are more complex and
subtle. Overall, I think the entire Jena stack would benefit from
some attention to the OSGi
universe (which does not require breaking compatibility with the
non-OSGi universe), but I don’t want
to launch into a manifest-o about that unless other folks are really
interested in the subject.
I look forward to hearing what other Jena users and developers think
about these topics!
peace,
Stu B22