Modularity is the spawn of Lucifer and a stinking donkey

classic Classic list List threaded Threaded
37 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Modularity is the spawn of Lucifer and a stinking donkey

Emmanuel Bernard
Now that I have your attention, I'd like to discuss issues we are experiencing when trying to modularize the Hibernate portfolio and make it work in AS 7.1.

## Disclaimer

I perfectly understand all the coolness about modularity (speed, easier dependency isolation etc). I have also carefully read :

- https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
- https://community.jboss.org/wiki/ModularSerialization

But these tend to avoid the more complex cases of portable libraries that ought to run even outside AS 7 but have a wide variety of class and resource loading needs.
I am not a complete modularity bozo but I am definitely not familiar with JBoss Modules nor similar solution.

## Requirements / Landscape

Hibernate ORM uses the notion of service registry and integrator object that help during the integration or customization of the engine behavior by third-party frameworks.
Enlistment of Integrators is done via the service locator pattern (a service file in META-INF/services/ that is looked up and contain the implementation class(es) at stake.

Hibernate Envers is one of those customizer that depends on Hibernate ORM. Note that the core of Hibernate ORM does not depend on Hibernate Envers. The service locator file is contained in Hibernate Envers JAR.
Hibernate OGM likewise, heavily customizes ORM and depends on Hibernate ORM classes - the reverse is not true. The service locator file is contained in Hibernate OGM JAR.
Hibernate Search optionally depend on Hibernate ORM and JPA. The core of Hibernate Search is independent but an Hibernate Search ORM module has an integrator implementation. On top of that, Hibernate Search optionally depend on some JPA classes and behaves differently if they are there - we look them up in the classpath by reflection.

On top of that, these projects do load resources (config files, classes):

- from what Jason calls a Deployment classloader (the user application classes and resources really) - entities, custom analyzer implementations, resources files etc. A user could even write a custom Integrator and use the service locator pattern from his application.
- from direct dependencies (Lucene is a declared dependency of Hibernate Search)
- from dependencies of the deployment: for example an app developer adds the phonetic analyzer as a dependency of his application and ask Hibernate Search to use it
- from modules that use these projects. Modeshape and Capedwarf are being modularized and are making use of Hibernate Search as a module. Properly loading the necessary classes located in Modeshape or Capedwarf's module but from Hibernate Search's engine proves to be very hard in our current approach.

All of these projects should be able to run outside JBoss AS 7, so a modular friendly solution should somehow be abstracted and generic enough.

## What solution?

More and more projects are being modularized including ones with complex resource loading dependencies like the ones I have described. AFAIK Infinispan is even in a worse situation as clustering and late class binding is at stake but let's put this one aside.
I'd love to get a reference design outcome from this thread that could be copied across all these projects and future ones like Bean Validation.

Today, we mostly use the good old and simple TCCL model which works fine if the jars are directly embedded in the app but fail the minute we start to move these dependencies into modules. Sanne, Strong, Scott Marlow and I are using dangerous amount of Advil to try and make everything work as expected. Some help would be awesome.

To sum up:

- can the Hibernate portfolio be supported within JBoss Module and how?
- what kind of ClassloaderService contract should we use within these projects to be modular friendly (JBoss Modules and others)?
- would such contract be generic enough to be extrapolated to JSRs in need of modular friendliness?
- how to solve the chicken and egg issue of the bootstrapping: if we need to pass a ClassloaderService impl? How do we do that best in a modular environment without forcing the application developer to implement such godforsaken ClassloaderService contract or even worse pass directly to us the right classloader for each call.

Thanks,

Emmanuel
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

David Lloyd-2
OK I admit I LOL'ed.

On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:

> Now that I have your attention, I'd like to discuss issues we are experiencing when trying to modularize the Hibernate portfolio and make it work in AS 7.1.
>
> ## Disclaimer
>
> I perfectly understand all the coolness about modularity (speed, easier dependency isolation etc). I have also carefully read :
>
> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
> - https://community.jboss.org/wiki/ModularSerialization
>
> But these tend to avoid the more complex cases of portable libraries that ought to run even outside AS 7 but have a wide variety of class and resource loading needs.
> I am not a complete modularity bozo but I am definitely not familiar with JBoss Modules nor similar solution.
>
> ## Requirements / Landscape
>
> Hibernate ORM uses the notion of service registry and integrator object that help during the integration or customization of the engine behavior by third-party frameworks.
> Enlistment of Integrators is done via the service locator pattern (a service file in META-INF/services/ that is looked up and contain the implementation class(es) at stake.
>
> Hibernate Envers is one of those customizer that depends on Hibernate ORM. Note that the core of Hibernate ORM does not depend on Hibernate Envers. The service locator file is contained in Hibernate Envers JAR.
> Hibernate OGM likewise, heavily customizes ORM and depends on Hibernate ORM classes - the reverse is not true. The service locator file is contained in Hibernate OGM JAR.
> Hibernate Search optionally depend on Hibernate ORM and JPA. The core of Hibernate Search is independent but an Hibernate Search ORM module has an integrator implementation. On top of that, Hibernate Search optionally depend on some JPA classes and behaves differently if they are there - we look them up in the classpath by reflection.
>
> On top of that, these projects do load resources (config files, classes):
>
> - from what Jason calls a Deployment classloader (the user application classes and resources really) - entities, custom analyzer implementations, resources files etc. A user could even write a custom Integrator and use the service locator pattern from his application.
> - from direct dependencies (Lucene is a declared dependency of Hibernate Search)
> - from dependencies of the deployment: for example an app developer adds the phonetic analyzer as a dependency of his application and ask Hibernate Search to use it
> - from modules that use these projects. Modeshape and Capedwarf are being modularized and are making use of Hibernate Search as a module. Properly loading the necessary classes located in Modeshape or Capedwarf's module but from Hibernate Search's engine proves to be very hard in our current approach.
>
> All of these projects should be able to run outside JBoss AS 7, so a modular friendly solution should somehow be abstracted and generic enough.
>
> ## What solution?
>
> More and more projects are being modularized including ones with complex resource loading dependencies like the ones I have described. AFAIK Infinispan is even in a worse situation as clustering and late class binding is at stake but let's put this one aside.
> I'd love to get a reference design outcome from this thread that could be copied across all these projects and future ones like Bean Validation.
>
> Today, we mostly use the good old and simple TCCL model which works fine if the jars are directly embedded in the app but fail the minute we start to move these dependencies into modules. Sanne, Strong, Scott Marlow and I are using dangerous amount of Advil to try and make everything work as expected. Some help would be awesome.
>
> To sum up:
>
> - can the Hibernate portfolio be supported within JBoss Module and how?
> - what kind of ClassloaderService contract should we use within these projects to be modular friendly (JBoss Modules and others)?
> - would such contract be generic enough to be extrapolated to JSRs in need of modular friendliness?
> - how to solve the chicken and egg issue of the bootstrapping: if we need to pass a ClassloaderService impl?
How do we do that best in a modular environment without forcing the
application developer to implement such godforsaken ClassloaderService
contract or even worse pass directly to us the right classloader for
each call.

I'll just start at the beginning and you can skip over the background if
you like.

The key starting concept is that a class' (or package's) identity is not
just its name but also its class loader.  This is the underlying
(existing) truth that modularity brings to the fore.  Corollary to this
are the fact that a single VM may have more than one class or package
with the same name, as well as the fact that not all classes/packages
are always directly visible from a given class loader.

This problem (as you've seen) manifests itself primarily when you're
locating a class or a resource by name.  You basically have two options.
  You can search *relative* to a class loader (most commonly, TCCL,
though using a single explicit class loader or the caller's class loader
also fall into this category).  Or, you can use the *absolute* identity
of a class.

Using relative resolution is often a perfectly adequate solution when
you're loading a single class or resource; in fact for some cases (like
ServiceLoader for example) it's a perfect fit in conjunction with TCCL
(in its capacity as an identifier for the "current" application).  You
want the user to be able to specify their implementation of something,
and you want it to be reasonably transparent; ServiceLoader+TCCL does
this fairly well.

ServiceLoader also does well from the perspective of APIs with a static,
fixed number of implementations.  In this case, it is appropriate for a
framework to have ServiceLoader use the class loader of the framework
itself.  The framework would then be sure to import the implementations
in question (including their service descriptors); in our modular
environment, which we call a "service import".  Note that this often
means there is a circular dependency between API and implementation:
that's OK!

A third ServiceLoader option is of course to simply accept a class
loader argument when looking up an implementation.  This grants the most
flexibility and tends to work well in just about any environment, though
it may be somewhat lacking aesthetically, if you care about such things.

In any case, ServiceLoader already fits in very well to modularity; it's
just a question of understanding your use case to know the appropriate
way to apply it.  The key characteristic however of such a fit is that
it is trying to load a single resource of some kind.  Once you move into
object graph territory, things become a hell of a lot more complex when
it comes to the relative resolution game.

A good example is serialization.  Having a single class loader for all
resolution needs often simply doesn't cut it.  It works to an extent,
iff the object graph in question never "escapes" what the application
(or current class loader) is cognizant of.  However it may well be the
case that an implementation class isn't "visible" to the single class
loader.  In this case, especially if more than one class with a given
name is existent in the system, there's no unambiguous solution to load
a class relative to an application, unless you explicitly add the
desired class loaders to the resolution path of the application's class
loader.

The two solutions to this problem are to either enforce (at serialize
time) a policy which prevents serializing objects of non-visible
classes, or to give up relative resolution and go to "absolute" identities.

When you use an absolute identity, you're persisting not only the class'
name but also the identity of its initiating class loader.  Back in the
RMI and applet days, this would have been done (rather clumsily) by
extension name or perhaps code source URL.  In our container environment
we tend to use module identifier.  But either way, this identity is
useless unless you have a mechanism to resolve it back to a class
loader.  This mechanism (at least as of today) is going to vary
substantially from one runtime environment to another though.  Thus
being able to plug in to the process is critical [1].

There's another tricky dimension to this problem though.  Say you want
everything - the ability to use the "current" application to resolve a
class (i.e. to disambiguate a class of a given name from a neighbor
application which might want to execute the exact same code but get its
own relatively visible class), but also the ability to absolutely
resolve "invisible" classes.  This would be common in the case where you
have two EAR deployments in an app server, each bundling their own copy
of an EJB JAR (for whatever reason) but which both link against common
modules.

There's no silver bullet here, but this problem can be solved to a
significant extent if you know which class loaders are candidates for
relative resolution and marking them as such in the target externalized
class data.   In AS for example, we could use information about the
deployment class-path graph to distinguish what was bundled with the app
and what is external to it, because we know that deployment classes are
very likely to be visible from the TCCL (and if not, we would not have a
very large class loader landscape over which to search).

Of course as an end user you don't really want to think about any of
this crap, you just want it to work.  The user can't avoid having some
knowledge of this though - they have to know what policy they apply to
the data they're accessing.  Sometimes it's simple - if they have a flat
class loader, and they're reading modularized serial data, they can just
discard the class loader identification because if a class isn't found
in their class loader, it's not going to be found anywhere else either.

Sometimes it's more complex though.  Writing from one environment and
reading from a completely differently structured environment can get
extremely hairy when using absolute resolution, requiring various
degrees of translation.  The best advice would be to have users strive
to use the same kind of environment when dealing with serialized data.
For example, modular writers should be consumed by modular readers.

Note that though I use serialization as my main example, these concepts
should apply as well (at a certain level) to Hibernate and friends,
Infinispan, etc.

[1] As an example, see
https://github.com/dmlloyd/jboss-marshalling/blob/master/api/src/main/java/org/jboss/marshalling/ClassResolver.java#L34
--
- DML
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

David Bosschaert
In OSGi the Service Registry is used for this purpose. Generally what is
done is referred to as the 'Whiteboard Pattern'. A certain component
that can be extended checks the Service Registry for services that
implement the interface that plug-ins implement. This interface is
typically exported by the main component. So plug-ins register
themselves in the Service Registry and become usable by other modules
that way.

However I understand that you need to come up with a solution that also
works outside of OSGi, in that case java.util.ServiceLoader should work
(as David L) suggests. It has been problematic inside OSGi in the past,
but there is a spec now that resolves that issue [1].
I would personally stay away from depending too much on the TCCL.
Passing the class loader to be used in the APIs is generally more
portable across various modularity systems.

Best regards,

David

[1] Chapter 133 in Enterprise R5 http://www.osgi.org/News/20120326

On 09/05/2012 19:01, David M. Lloyd wrote:

> OK I admit I LOL'ed.
>
> On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:
>> Now that I have your attention, I'd like to discuss issues we are experiencing when trying to modularize the Hibernate portfolio and make it work in AS 7.1.
>>
>> ## Disclaimer
>>
>> I perfectly understand all the coolness about modularity (speed, easier dependency isolation etc). I have also carefully read :
>>
>> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
>> - https://community.jboss.org/wiki/ModularSerialization
>>
>> But these tend to avoid the more complex cases of portable libraries that ought to run even outside AS 7 but have a wide variety of class and resource loading needs.
>> I am not a complete modularity bozo but I am definitely not familiar with JBoss Modules nor similar solution.
>>
>> ## Requirements / Landscape
>>
>> Hibernate ORM uses the notion of service registry and integrator object that help during the integration or customization of the engine behavior by third-party frameworks.
>> Enlistment of Integrators is done via the service locator pattern (a service file in META-INF/services/ that is looked up and contain the implementation class(es) at stake.
>>
>> Hibernate Envers is one of those customizer that depends on Hibernate ORM. Note that the core of Hibernate ORM does not depend on Hibernate Envers. The service locator file is contained in Hibernate Envers JAR.
>> Hibernate OGM likewise, heavily customizes ORM and depends on Hibernate ORM classes - the reverse is not true. The service locator file is contained in Hibernate OGM JAR.
>> Hibernate Search optionally depend on Hibernate ORM and JPA. The core of Hibernate Search is independent but an Hibernate Search ORM module has an integrator implementation. On top of that, Hibernate Search optionally depend on some JPA classes and behaves differently if they are there - we look them up in the classpath by reflection.
>>
>> On top of that, these projects do load resources (config files, classes):
>>
>> - from what Jason calls a Deployment classloader (the user application classes and resources really) - entities, custom analyzer implementations, resources files etc. A user could even write a custom Integrator and use the service locator pattern from his application.
>> - from direct dependencies (Lucene is a declared dependency of Hibernate Search)
>> - from dependencies of the deployment: for example an app developer adds the phonetic analyzer as a dependency of his application and ask Hibernate Search to use it
>> - from modules that use these projects. Modeshape and Capedwarf are being modularized and are making use of Hibernate Search as a module. Properly loading the necessary classes located in Modeshape or Capedwarf's module but from Hibernate Search's engine proves to be very hard in our current approach.
>>
>> All of these projects should be able to run outside JBoss AS 7, so a modular friendly solution should somehow be abstracted and generic enough.
>>
>> ## What solution?
>>
>> More and more projects are being modularized including ones with complex resource loading dependencies like the ones I have described. AFAIK Infinispan is even in a worse situation as clustering and late class binding is at stake but let's put this one aside.
>> I'd love to get a reference design outcome from this thread that could be copied across all these projects and future ones like Bean Validation.
>>
>> Today, we mostly use the good old and simple TCCL model which works fine if the jars are directly embedded in the app but fail the minute we start to move these dependencies into modules. Sanne, Strong, Scott Marlow and I are using dangerous amount of Advil to try and make everything work as expected. Some help would be awesome.
>>
>> To sum up:
>>
>> - can the Hibernate portfolio be supported within JBoss Module and how?
>> - what kind of ClassloaderService contract should we use within these projects to be modular friendly (JBoss Modules and others)?
>> - would such contract be generic enough to be extrapolated to JSRs in need of modular friendliness?
>> - how to solve the chicken and egg issue of the bootstrapping: if we need to pass a ClassloaderService impl?
> How do we do that best in a modular environment without forcing the
> application developer to implement such godforsaken ClassloaderService
> contract or even worse pass directly to us the right classloader for
> each call.
>
> I'll just start at the beginning and you can skip over the background if
> you like.
>
> The key starting concept is that a class' (or package's) identity is not
> just its name but also its class loader.  This is the underlying
> (existing) truth that modularity brings to the fore.  Corollary to this
> are the fact that a single VM may have more than one class or package
> with the same name, as well as the fact that not all classes/packages
> are always directly visible from a given class loader.
>
> This problem (as you've seen) manifests itself primarily when you're
> locating a class or a resource by name.  You basically have two options.
>    You can search *relative* to a class loader (most commonly, TCCL,
> though using a single explicit class loader or the caller's class loader
> also fall into this category).  Or, you can use the *absolute* identity
> of a class.
>
> Using relative resolution is often a perfectly adequate solution when
> you're loading a single class or resource; in fact for some cases (like
> ServiceLoader for example) it's a perfect fit in conjunction with TCCL
> (in its capacity as an identifier for the "current" application).  You
> want the user to be able to specify their implementation of something,
> and you want it to be reasonably transparent; ServiceLoader+TCCL does
> this fairly well.
>
> ServiceLoader also does well from the perspective of APIs with a static,
> fixed number of implementations.  In this case, it is appropriate for a
> framework to have ServiceLoader use the class loader of the framework
> itself.  The framework would then be sure to import the implementations
> in question (including their service descriptors); in our modular
> environment, which we call a "service import".  Note that this often
> means there is a circular dependency between API and implementation:
> that's OK!
>
> A third ServiceLoader option is of course to simply accept a class
> loader argument when looking up an implementation.  This grants the most
> flexibility and tends to work well in just about any environment, though
> it may be somewhat lacking aesthetically, if you care about such things.
>
> In any case, ServiceLoader already fits in very well to modularity; it's
> just a question of understanding your use case to know the appropriate
> way to apply it.  The key characteristic however of such a fit is that
> it is trying to load a single resource of some kind.  Once you move into
> object graph territory, things become a hell of a lot more complex when
> it comes to the relative resolution game.
>
> A good example is serialization.  Having a single class loader for all
> resolution needs often simply doesn't cut it.  It works to an extent,
> iff the object graph in question never "escapes" what the application
> (or current class loader) is cognizant of.  However it may well be the
> case that an implementation class isn't "visible" to the single class
> loader.  In this case, especially if more than one class with a given
> name is existent in the system, there's no unambiguous solution to load
> a class relative to an application, unless you explicitly add the
> desired class loaders to the resolution path of the application's class
> loader.
>
> The two solutions to this problem are to either enforce (at serialize
> time) a policy which prevents serializing objects of non-visible
> classes, or to give up relative resolution and go to "absolute" identities.
>
> When you use an absolute identity, you're persisting not only the class'
> name but also the identity of its initiating class loader.  Back in the
> RMI and applet days, this would have been done (rather clumsily) by
> extension name or perhaps code source URL.  In our container environment
> we tend to use module identifier.  But either way, this identity is
> useless unless you have a mechanism to resolve it back to a class
> loader.  This mechanism (at least as of today) is going to vary
> substantially from one runtime environment to another though.  Thus
> being able to plug in to the process is critical [1].
>
> There's another tricky dimension to this problem though.  Say you want
> everything - the ability to use the "current" application to resolve a
> class (i.e. to disambiguate a class of a given name from a neighbor
> application which might want to execute the exact same code but get its
> own relatively visible class), but also the ability to absolutely
> resolve "invisible" classes.  This would be common in the case where you
> have two EAR deployments in an app server, each bundling their own copy
> of an EJB JAR (for whatever reason) but which both link against common
> modules.
>
> There's no silver bullet here, but this problem can be solved to a
> significant extent if you know which class loaders are candidates for
> relative resolution and marking them as such in the target externalized
> class data.   In AS for example, we could use information about the
> deployment class-path graph to distinguish what was bundled with the app
> and what is external to it, because we know that deployment classes are
> very likely to be visible from the TCCL (and if not, we would not have a
> very large class loader landscape over which to search).
>
> Of course as an end user you don't really want to think about any of
> this crap, you just want it to work.  The user can't avoid having some
> knowledge of this though - they have to know what policy they apply to
> the data they're accessing.  Sometimes it's simple - if they have a flat
> class loader, and they're reading modularized serial data, they can just
> discard the class loader identification because if a class isn't found
> in their class loader, it's not going to be found anywhere else either.
>
> Sometimes it's more complex though.  Writing from one environment and
> reading from a completely differently structured environment can get
> extremely hairy when using absolute resolution, requiring various
> degrees of translation.  The best advice would be to have users strive
> to use the same kind of environment when dealing with serialized data.
> For example, modular writers should be consumed by modular readers.
>
> Note that though I use serialization as my main example, these concepts
> should apply as well (at a certain level) to Hibernate and friends,
> Infinispan, etc.
>
> [1] As an example, see
> https://github.com/dmlloyd/jboss-marshalling/blob/master/api/src/main/java/org/jboss/marshalling/ClassResolver.java#L34

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Scott Marlow
In reply to this post by David Lloyd-2
On 05/09/2012 02:01 PM, David M. Lloyd wrote:

> OK I admit I LOL'ed.
>
> On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:
>> Now that I have your attention, I'd like to discuss issues we are experiencing when trying to modularize the Hibernate portfolio and make it work in AS 7.1.
>>
>> ## Disclaimer
>>
>> I perfectly understand all the coolness about modularity (speed, easier dependency isolation etc). I have also carefully read :
>>
>> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
>> - https://community.jboss.org/wiki/ModularSerialization
>>
>> But these tend to avoid the more complex cases of portable libraries that ought to run even outside AS 7 but have a wide variety of class and resource loading needs.
>> I am not a complete modularity bozo but I am definitely not familiar with JBoss Modules nor similar solution.
>>
>> ## Requirements / Landscape
>>
>> Hibernate ORM uses the notion of service registry and integrator object that help during the integration or customization of the engine behavior by third-party frameworks.
>> Enlistment of Integrators is done via the service locator pattern (a service file in META-INF/services/ that is looked up and contain the implementation class(es) at stake.
>>
>> Hibernate Envers is one of those customizer that depends on Hibernate ORM. Note that the core of Hibernate ORM does not depend on Hibernate Envers. The service locator file is contained in Hibernate Envers JAR.
>> Hibernate OGM likewise, heavily customizes ORM and depends on Hibernate ORM classes - the reverse is not true. The service locator file is contained in Hibernate OGM JAR.
>> Hibernate Search optionally depend on Hibernate ORM and JPA. The core of Hibernate Search is independent but an Hibernate Search ORM module has an integrator implementation. On top of that, Hibernate Search optionally depend on some JPA classes and behaves differently if they are there - we look them up in the classpath by reflection.
>>
>> On top of that, these projects do load resources (config files, classes):
>>
>> - from what Jason calls a Deployment classloader (the user application classes and resources really) - entities, custom analyzer implementations, resources files etc. A user could even write a custom Integrator and use the service locator pattern from his application.
>> - from direct dependencies (Lucene is a declared dependency of Hibernate Search)
>> - from dependencies of the deployment: for example an app developer adds the phonetic analyzer as a dependency of his application and ask Hibernate Search to use it
>> - from modules that use these projects. Modeshape and Capedwarf are being modularized and are making use of Hibernate Search as a module. Properly loading the necessary classes located in Modeshape or Capedwarf's module but from Hibernate Search's engine proves to be very hard in our current approach.
>>
>> All of these projects should be able to run outside JBoss AS 7, so a modular friendly solution should somehow be abstracted and generic enough.
>>
>> ## What solution?
>>
>> More and more projects are being modularized including ones with complex resource loading dependencies like the ones I have described. AFAIK Infinispan is even in a worse situation as clustering and late class binding is at stake but let's put this one aside.
>> I'd love to get a reference design outcome from this thread that could be copied across all these projects and future ones like Bean Validation.
>>
>> Today, we mostly use the good old and simple TCCL model which works fine if the jars are directly embedded in the app but fail the minute we start to move these dependencies into modules. Sanne, Strong, Scott Marlow and I are using dangerous amount of Advil to try and make everything work as expected. Some help would be awesome.
>>
>> To sum up:
>>
>> - can the Hibernate portfolio be supported within JBoss Module and how?
>> - what kind of ClassloaderService contract should we use within these projects to be modular friendly (JBoss Modules and others)?
>> - would such contract be generic enough to be extrapolated to JSRs in need of modular friendliness?
>> - how to solve the chicken and egg issue of the bootstrapping: if we need to pass a ClassloaderService impl?
> How do we do that best in a modular environment without forcing the
> application developer to implement such godforsaken ClassloaderService
> contract or even worse pass directly to us the right classloader for
> each call.
>
> I'll just start at the beginning and you can skip over the background if
> you like.
>
> The key starting concept is that a class' (or package's) identity is not
> just its name but also its class loader.  This is the underlying
> (existing) truth that modularity brings to the fore.  Corollary to this
> are the fact that a single VM may have more than one class or package
> with the same name, as well as the fact that not all classes/packages
> are always directly visible from a given class loader.
>
> This problem (as you've seen) manifests itself primarily when you're
> locating a class or a resource by name.  You basically have two options.
>    You can search *relative* to a class loader (most commonly, TCCL,
> though using a single explicit class loader or the caller's class loader
> also fall into this category).  Or, you can use the *absolute* identity
> of a class.
>
> Using relative resolution is often a perfectly adequate solution when
> you're loading a single class or resource; in fact for some cases (like
> ServiceLoader for example) it's a perfect fit in conjunction with TCCL
> (in its capacity as an identifier for the "current" application).  You
> want the user to be able to specify their implementation of something,
> and you want it to be reasonably transparent; ServiceLoader+TCCL does
> this fairly well.
>
> ServiceLoader also does well from the perspective of APIs with a static,
> fixed number of implementations.  In this case, it is appropriate for a
> framework to have ServiceLoader use the class loader of the framework
> itself.  The framework would then be sure to import the implementations
> in question (including their service descriptors); in our modular
> environment, which we call a "service import".  Note that this often
> means there is a circular dependency between API and implementation:
> that's OK!

We currently use this for envers but that doesn't seem as desirable for
other members of the Hibernate portfolio that may be on a separate
lifecycle.  For example, the Hibernate OGM is a persistence provider
that depends on Hibernate ORM.  If we have Hibernate ORM depend on OGM,
that limits the number of OGM versions that can be in use on AS7.

Would it be possible, to add a MSC enhancement, that allows an inverse
dependency service loader dependency to be expressed?  Such that it
would be enough to only have OGM depend on ORM (with an inverse service
dependency specified).  I'm thinking that the OGM module would need to
exchange the service dependency information with the ORM module and
clear it, when OGM goes away.

If this is possible, it would make a nice future enhancement IMO.

>
> A third ServiceLoader option is of course to simply accept a class
> loader argument when looking up an implementation.  This grants the most
> flexibility and tends to work well in just about any environment, though
> it may be somewhat lacking aesthetically, if you care about such things.

I'm not sure of how OGM would make its presence known to ORM currently.
  Probably via a custom SPI that allows the inverse service loader
dependency to be expressed (so that
https://github.com/hibernate/hibernate-orm/blob/master/hibernate-core/src/main/java/org/hibernate/service/classloading/internal/ClassLoaderServiceImpl.java 
can know about Hibernate search/envers/ogm/...).

If we cannot have a MSC way to express the inverse service loader
dependency, this sounds like the next best option.

>
> In any case, ServiceLoader already fits in very well to modularity; it's
> just a question of understanding your use case to know the appropriate
> way to apply it.  The key characteristic however of such a fit is that
> it is trying to load a single resource of some kind.  Once you move into
> object graph territory, things become a hell of a lot more complex when
> it comes to the relative resolution game.
>
> A good example is serialization.  Having a single class loader for all
> resolution needs often simply doesn't cut it.  It works to an extent,
> iff the object graph in question never "escapes" what the application
> (or current class loader) is cognizant of.  However it may well be the
> case that an implementation class isn't "visible" to the single class
> loader.  In this case, especially if more than one class with a given
> name is existent in the system, there's no unambiguous solution to load
> a class relative to an application, unless you explicitly add the
> desired class loaders to the resolution path of the application's class
> loader.
>
> The two solutions to this problem are to either enforce (at serialize
> time) a policy which prevents serializing objects of non-visible
> classes, or to give up relative resolution and go to "absolute" identities.
>
> When you use an absolute identity, you're persisting not only the class'
> name but also the identity of its initiating class loader.  Back in the
> RMI and applet days, this would have been done (rather clumsily) by
> extension name or perhaps code source URL.  In our container environment
> we tend to use module identifier.  But either way, this identity is
> useless unless you have a mechanism to resolve it back to a class
> loader.  This mechanism (at least as of today) is going to vary
> substantially from one runtime environment to another though.  Thus
> being able to plug in to the process is critical [1].
>
> There's another tricky dimension to this problem though.  Say you want
> everything - the ability to use the "current" application to resolve a
> class (i.e. to disambiguate a class of a given name from a neighbor
> application which might want to execute the exact same code but get its
> own relatively visible class), but also the ability to absolutely
> resolve "invisible" classes.  This would be common in the case where you
> have two EAR deployments in an app server, each bundling their own copy
> of an EJB JAR (for whatever reason) but which both link against common
> modules.
>
> There's no silver bullet here, but this problem can be solved to a
> significant extent if you know which class loaders are candidates for
> relative resolution and marking them as such in the target externalized
> class data.   In AS for example, we could use information about the
> deployment class-path graph to distinguish what was bundled with the app
> and what is external to it, because we know that deployment classes are
> very likely to be visible from the TCCL (and if not, we would not have a
> very large class loader landscape over which to search).
>
> Of course as an end user you don't really want to think about any of
> this crap, you just want it to work.  The user can't avoid having some
> knowledge of this though - they have to know what policy they apply to
> the data they're accessing.  Sometimes it's simple - if they have a flat
> class loader, and they're reading modularized serial data, they can just
> discard the class loader identification because if a class isn't found
> in their class loader, it's not going to be found anywhere else either.
>
> Sometimes it's more complex though.  Writing from one environment and
> reading from a completely differently structured environment can get
> extremely hairy when using absolute resolution, requiring various
> degrees of translation.  The best advice would be to have users strive
> to use the same kind of environment when dealing with serialized data.
> For example, modular writers should be consumed by modular readers.
>
> Note that though I use serialization as my main example, these concepts
> should apply as well (at a certain level) to Hibernate and friends,
> Infinispan, etc.
>
> [1] As an example, see
> https://github.com/dmlloyd/jboss-marshalling/blob/master/api/src/main/java/org/jboss/marshalling/ClassResolver.java#L34

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Ales Justin
In reply to this post by David Lloyd-2
Since I think it's exactly on the subject, I'll just ask you (David) here. :-)

loader.  This mechanism (at least as of today) is going to vary
substantially from one runtime environment to another though.  Thus
being able to plug in to the process is critical [1].

CapeDwarf is using Infinispan to store some data.
We always wrap Cache usage with app's CL - cache::getAdvancedCache::with(cl).
So when that data is persisted (mashalled), it is app's CL / module that gets into serialized info.

But it cannot get deserialized:

Full stack-trace: http://pastebin.com/5SWvahNV

Caused by: org.jboss.modules.ModuleNotFoundException: deployment.29bc6d63-8cc8-4a4e-a626-94f3a9dca7ed.war:main
at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:206) [jboss-modules.jar:1.1.2.GA]
at org.jboss.marshalling.ModularClassResolver.resolveClass(ModularClassResolver.java:106)

The problem as you can see is that deployment module cannot be found by plain ModuleLoader,
but - I guess?- only by AS7' ServiceModuleLoader -- which has the deployment module info.

Do we already have such (based off ServiceML) ClassResolver?
If yes, why it's not used? 
As I don't see how apps that use Infinispan (EDG?) can work w/o it (aka not having the same problem).

-Ales


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

jtgreene
Administrator
On 5/10/12 10:59 AM, Ales Justin wrote:
> The problem as you can see is that deployment module cannot be found by
> plain ModuleLoader,
> but - I guess?- only by AS7' ServiceModuleLoader -- which has the
> deployment module info.
>
> Do we already have such (based off ServiceML) ClassResolver?
> If yes, why it's not used?
> As I don't see how apps that use Infinispan (EDG?) can work w/o it (aka
> not having the same problem).

Yes it should be using the SML, it looks like its set up correctly in
CacheContainerAdd.java:265. So not sure how it's getting the wrong value?

This area was changed quite a bit, so maybe your patch is based on an
older version?


--
Jason T. Greene
JBoss AS Lead / EAP Platform Architect
JBoss, a division of Red Hat
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Ales Justin
>> The problem as you can see is that deployment module cannot be found by
>> plain ModuleLoader,
>> but - I guess?- only by AS7' ServiceModuleLoader -- which has the
>> deployment module info.
>>
>> Do we already have such (based off ServiceML) ClassResolver?
>> If yes, why it's not used?
>> As I don't see how apps that use Infinispan (EDG?) can work w/o it (aka
>> not having the same problem).
>
> Yes it should be using the SML, it looks like its set up correctly in CacheContainerAdd.java:265. So not sure how it's getting the wrong value?

Ah, ok, I'll check.

> This area was changed quite a bit, so maybe your patch is based on an older version?

Hmm, not really, just rebased it yday ...

Anyway, I'll have a closer look.

-Ales


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

David Lloyd-2
In reply to this post by Scott Marlow
On 05/10/2012 10:24 AM, Scott Marlow wrote:

> On 05/09/2012 02:01 PM, David M. Lloyd wrote:
>> OK I admit I LOL'ed.
>>
>> On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:
>>> Now that I have your attention, I'd like to discuss issues we are
>>> experiencing when trying to modularize the Hibernate portfolio and
>>> make it work in AS 7.1.
>>>
>>> ## Disclaimer
>>>
>>> I perfectly understand all the coolness about modularity (speed,
>>> easier dependency isolation etc). I have also carefully read :
>>>
>>> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
>>> - https://community.jboss.org/wiki/ModularSerialization
>>>
>>> But these tend to avoid the more complex cases of portable libraries
>>> that ought to run even outside AS 7 but have a wide variety of class
>>> and resource loading needs.
>>> I am not a complete modularity bozo but I am definitely not familiar
>>> with JBoss Modules nor similar solution.
>>>
>>> ## Requirements / Landscape
>>>
>>> Hibernate ORM uses the notion of service registry and integrator
>>> object that help during the integration or customization of the
>>> engine behavior by third-party frameworks.
>>> Enlistment of Integrators is done via the service locator pattern (a
>>> service file in META-INF/services/ that is looked up and contain the
>>> implementation class(es) at stake.
>>>
>>> Hibernate Envers is one of those customizer that depends on Hibernate
>>> ORM. Note that the core of Hibernate ORM does not depend on Hibernate
>>> Envers. The service locator file is contained in Hibernate Envers JAR.
>>> Hibernate OGM likewise, heavily customizes ORM and depends on
>>> Hibernate ORM classes - the reverse is not true. The service locator
>>> file is contained in Hibernate OGM JAR.
>>> Hibernate Search optionally depend on Hibernate ORM and JPA. The core
>>> of Hibernate Search is independent but an Hibernate Search ORM module
>>> has an integrator implementation. On top of that, Hibernate Search
>>> optionally depend on some JPA classes and behaves differently if they
>>> are there - we look them up in the classpath by reflection.
>>>
>>> On top of that, these projects do load resources (config files,
>>> classes):
>>>
>>> - from what Jason calls a Deployment classloader (the user
>>> application classes and resources really) - entities, custom analyzer
>>> implementations, resources files etc. A user could even write a
>>> custom Integrator and use the service locator pattern from his
>>> application.
>>> - from direct dependencies (Lucene is a declared dependency of
>>> Hibernate Search)
>>> - from dependencies of the deployment: for example an app developer
>>> adds the phonetic analyzer as a dependency of his application and ask
>>> Hibernate Search to use it
>>> - from modules that use these projects. Modeshape and Capedwarf are
>>> being modularized and are making use of Hibernate Search as a module.
>>> Properly loading the necessary classes located in Modeshape or
>>> Capedwarf's module but from Hibernate Search's engine proves to be
>>> very hard in our current approach.
>>>
>>> All of these projects should be able to run outside JBoss AS 7, so a
>>> modular friendly solution should somehow be abstracted and generic
>>> enough.
>>>
>>> ## What solution?
>>>
>>> More and more projects are being modularized including ones with
>>> complex resource loading dependencies like the ones I have described.
>>> AFAIK Infinispan is even in a worse situation as clustering and late
>>> class binding is at stake but let's put this one aside.
>>> I'd love to get a reference design outcome from this thread that
>>> could be copied across all these projects and future ones like Bean
>>> Validation.
>>>
>>> Today, we mostly use the good old and simple TCCL model which works
>>> fine if the jars are directly embedded in the app but fail the minute
>>> we start to move these dependencies into modules. Sanne, Strong,
>>> Scott Marlow and I are using dangerous amount of Advil to try and
>>> make everything work as expected. Some help would be awesome.
>>>
>>> To sum up:
>>>
>>> - can the Hibernate portfolio be supported within JBoss Module and how?
>>> - what kind of ClassloaderService contract should we use within these
>>> projects to be modular friendly (JBoss Modules and others)?
>>> - would such contract be generic enough to be extrapolated to JSRs in
>>> need of modular friendliness?
>>> - how to solve the chicken and egg issue of the bootstrapping: if we
>>> need to pass a ClassloaderService impl?
>> How do we do that best in a modular environment without forcing the
>> application developer to implement such godforsaken ClassloaderService
>> contract or even worse pass directly to us the right classloader for
>> each call.
>>
>> I'll just start at the beginning and you can skip over the background if
>> you like.
>>
>> The key starting concept is that a class' (or package's) identity is not
>> just its name but also its class loader. This is the underlying
>> (existing) truth that modularity brings to the fore. Corollary to this
>> are the fact that a single VM may have more than one class or package
>> with the same name, as well as the fact that not all classes/packages
>> are always directly visible from a given class loader.
>>
>> This problem (as you've seen) manifests itself primarily when you're
>> locating a class or a resource by name. You basically have two options.
>> You can search *relative* to a class loader (most commonly, TCCL,
>> though using a single explicit class loader or the caller's class loader
>> also fall into this category). Or, you can use the *absolute* identity
>> of a class.
>>
>> Using relative resolution is often a perfectly adequate solution when
>> you're loading a single class or resource; in fact for some cases (like
>> ServiceLoader for example) it's a perfect fit in conjunction with TCCL
>> (in its capacity as an identifier for the "current" application). You
>> want the user to be able to specify their implementation of something,
>> and you want it to be reasonably transparent; ServiceLoader+TCCL does
>> this fairly well.
>>
>> ServiceLoader also does well from the perspective of APIs with a static,
>> fixed number of implementations. In this case, it is appropriate for a
>> framework to have ServiceLoader use the class loader of the framework
>> itself. The framework would then be sure to import the implementations
>> in question (including their service descriptors); in our modular
>> environment, which we call a "service import". Note that this often
>> means there is a circular dependency between API and implementation:
>> that's OK!
>
> We currently use this for envers but that doesn't seem as desirable for
> other members of the Hibernate portfolio that may be on a separate
> lifecycle. For example, the Hibernate OGM is a persistence provider that
> depends on Hibernate ORM. If we have Hibernate ORM depend on OGM, that
> limits the number of OGM versions that can be in use on AS7.

Cases where implementations are pluggable could be handled using TCCL or
by specifying a class loader, or by explicit registration at run time.
As I said above, you would only use static dependencies if the
implementations are more or less fixed.

> Would it be possible, to add a MSC enhancement, that allows an inverse
> dependency service loader dependency to be expressed? Such that it would
> be enough to only have OGM depend on ORM (with an inverse service
> dependency specified). I'm thinking that the OGM module would need to
> exchange the service dependency information with the ORM module and
> clear it, when OGM goes away.

MSC has absolutely nothing to do with any of this.  Module loading is
static and is not affected by services in any way.

In your case you probably want a registration system.  But be aware that
if you have more than one OGM implementation, you're not going to be
able to use the ORM API solely to load them anyway (how would you tell
it which one you want?); instead you'd need some mechanism to specify,
so why not use TCCL or explicit class loader specification, as appropriate?
--
- DML
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Ales Justin
In reply to this post by jtgreene
>> The problem as you can see is that deployment module cannot be found by
>> plain ModuleLoader,
>> but - I guess?- only by AS7' ServiceModuleLoader -- which has the
>> deployment module info.
>>
>> Do we already have such (based off ServiceML) ClassResolver?
>> If yes, why it's not used?
>> As I don't see how apps that use Infinispan (EDG?) can work w/o it (aka
>> not having the same problem).
>
> Yes it should be using the SML, it looks like its set up correctly in CacheContainerAdd.java:265. So not sure how it's getting the wrong value?

I'm cloning configurations from pre-defined configs (in infinispan.xml), for each app.
Perhaps there is where it gets lost? <------- Paul
(I've seen other configs get lost before :-))

Although, I don't know why plain ModuleLoader would then be used ...

-Ales


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Scott Marlow
In reply to this post by David Lloyd-2
On 05/10/2012 12:21 PM, David M. Lloyd wrote:

> On 05/10/2012 10:24 AM, Scott Marlow wrote:
>> On 05/09/2012 02:01 PM, David M. Lloyd wrote:
>>> OK I admit I LOL'ed.
>>>
>>> On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:
>>>> Now that I have your attention, I'd like to discuss issues we are
>>>> experiencing when trying to modularize the Hibernate portfolio and
>>>> make it work in AS 7.1.
>>>>
>>>> ## Disclaimer
>>>>
>>>> I perfectly understand all the coolness about modularity (speed,
>>>> easier dependency isolation etc). I have also carefully read :
>>>>
>>>> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
>>>> - https://community.jboss.org/wiki/ModularSerialization
>>>>
>>>> But these tend to avoid the more complex cases of portable libraries
>>>> that ought to run even outside AS 7 but have a wide variety of class
>>>> and resource loading needs.
>>>> I am not a complete modularity bozo but I am definitely not familiar
>>>> with JBoss Modules nor similar solution.
>>>>
>>>> ## Requirements / Landscape
>>>>
>>>> Hibernate ORM uses the notion of service registry and integrator
>>>> object that help during the integration or customization of the
>>>> engine behavior by third-party frameworks.
>>>> Enlistment of Integrators is done via the service locator pattern (a
>>>> service file in META-INF/services/ that is looked up and contain the
>>>> implementation class(es) at stake.
>>>>
>>>> Hibernate Envers is one of those customizer that depends on Hibernate
>>>> ORM. Note that the core of Hibernate ORM does not depend on Hibernate
>>>> Envers. The service locator file is contained in Hibernate Envers JAR.
>>>> Hibernate OGM likewise, heavily customizes ORM and depends on
>>>> Hibernate ORM classes - the reverse is not true. The service locator
>>>> file is contained in Hibernate OGM JAR.
>>>> Hibernate Search optionally depend on Hibernate ORM and JPA. The core
>>>> of Hibernate Search is independent but an Hibernate Search ORM module
>>>> has an integrator implementation. On top of that, Hibernate Search
>>>> optionally depend on some JPA classes and behaves differently if they
>>>> are there - we look them up in the classpath by reflection.
>>>>
>>>> On top of that, these projects do load resources (config files,
>>>> classes):
>>>>
>>>> - from what Jason calls a Deployment classloader (the user
>>>> application classes and resources really) - entities, custom analyzer
>>>> implementations, resources files etc. A user could even write a
>>>> custom Integrator and use the service locator pattern from his
>>>> application.
>>>> - from direct dependencies (Lucene is a declared dependency of
>>>> Hibernate Search)
>>>> - from dependencies of the deployment: for example an app developer
>>>> adds the phonetic analyzer as a dependency of his application and ask
>>>> Hibernate Search to use it
>>>> - from modules that use these projects. Modeshape and Capedwarf are
>>>> being modularized and are making use of Hibernate Search as a module.
>>>> Properly loading the necessary classes located in Modeshape or
>>>> Capedwarf's module but from Hibernate Search's engine proves to be
>>>> very hard in our current approach.
>>>>
>>>> All of these projects should be able to run outside JBoss AS 7, so a
>>>> modular friendly solution should somehow be abstracted and generic
>>>> enough.
>>>>
>>>> ## What solution?
>>>>
>>>> More and more projects are being modularized including ones with
>>>> complex resource loading dependencies like the ones I have described.
>>>> AFAIK Infinispan is even in a worse situation as clustering and late
>>>> class binding is at stake but let's put this one aside.
>>>> I'd love to get a reference design outcome from this thread that
>>>> could be copied across all these projects and future ones like Bean
>>>> Validation.
>>>>
>>>> Today, we mostly use the good old and simple TCCL model which works
>>>> fine if the jars are directly embedded in the app but fail the minute
>>>> we start to move these dependencies into modules. Sanne, Strong,
>>>> Scott Marlow and I are using dangerous amount of Advil to try and
>>>> make everything work as expected. Some help would be awesome.
>>>>
>>>> To sum up:
>>>>
>>>> - can the Hibernate portfolio be supported within JBoss Module and how?
>>>> - what kind of ClassloaderService contract should we use within these
>>>> projects to be modular friendly (JBoss Modules and others)?
>>>> - would such contract be generic enough to be extrapolated to JSRs in
>>>> need of modular friendliness?
>>>> - how to solve the chicken and egg issue of the bootstrapping: if we
>>>> need to pass a ClassloaderService impl?
>>> How do we do that best in a modular environment without forcing the
>>> application developer to implement such godforsaken ClassloaderService
>>> contract or even worse pass directly to us the right classloader for
>>> each call.
>>>
>>> I'll just start at the beginning and you can skip over the background if
>>> you like.
>>>
>>> The key starting concept is that a class' (or package's) identity is not
>>> just its name but also its class loader. This is the underlying
>>> (existing) truth that modularity brings to the fore. Corollary to this
>>> are the fact that a single VM may have more than one class or package
>>> with the same name, as well as the fact that not all classes/packages
>>> are always directly visible from a given class loader.
>>>
>>> This problem (as you've seen) manifests itself primarily when you're
>>> locating a class or a resource by name. You basically have two options.
>>> You can search *relative* to a class loader (most commonly, TCCL,
>>> though using a single explicit class loader or the caller's class loader
>>> also fall into this category). Or, you can use the *absolute* identity
>>> of a class.
>>>
>>> Using relative resolution is often a perfectly adequate solution when
>>> you're loading a single class or resource; in fact for some cases (like
>>> ServiceLoader for example) it's a perfect fit in conjunction with TCCL
>>> (in its capacity as an identifier for the "current" application). You
>>> want the user to be able to specify their implementation of something,
>>> and you want it to be reasonably transparent; ServiceLoader+TCCL does
>>> this fairly well.
>>>
>>> ServiceLoader also does well from the perspective of APIs with a static,
>>> fixed number of implementations. In this case, it is appropriate for a
>>> framework to have ServiceLoader use the class loader of the framework
>>> itself. The framework would then be sure to import the implementations
>>> in question (including their service descriptors); in our modular
>>> environment, which we call a "service import". Note that this often
>>> means there is a circular dependency between API and implementation:
>>> that's OK!
>>
>> We currently use this for envers but that doesn't seem as desirable for
>> other members of the Hibernate portfolio that may be on a separate
>> lifecycle. For example, the Hibernate OGM is a persistence provider that
>> depends on Hibernate ORM. If we have Hibernate ORM depend on OGM, that
>> limits the number of OGM versions that can be in use on AS7.
>
> Cases where implementations are pluggable could be handled using TCCL or
> by specifying a class loader, or by explicit registration at run time.
> As I said above, you would only use static dependencies if the
> implementations are more or less fixed.
>
>> Would it be possible, to add a MSC enhancement, that allows an inverse
>> dependency service loader dependency to be expressed? Such that it would
>> be enough to only have OGM depend on ORM (with an inverse service
>> dependency specified). I'm thinking that the OGM module would need to
>> exchange the service dependency information with the ORM module and
>> clear it, when OGM goes away.
>
> MSC has absolutely nothing to do with any of this. Module loading is
> static and is not affected by services in any way.

Yeah, I took a wrong turn there into MSC.  I should of asked the same
question about MCS but I think you already answered below (Hibernate ORM
needs to know which services are appropriate for the current deployment).

>
> In your case you probably want a registration system. But be aware that
> if you have more than one OGM implementation, you're not going to be
> able to use the ORM API solely to load them anyway (how would you tell
> it which one you want?); instead you'd need some mechanism to specify,
> so why not use TCCL or explicit class loader specification, as appropriate?

If the registration system is per deployment based (really Hibernate
session factory level), that might be as good as TCCL but would also
work with naked (unmanaged threads) as well.

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Paul Ferraro
In reply to this post by Ales Justin
On Thu, 2012-05-10 at 18:33 +0200, Ales Justin wrote:

> >> The problem as you can see is that deployment module cannot be found by
> >> plain ModuleLoader,
> >> but - I guess?- only by AS7' ServiceModuleLoader -- which has the
> >> deployment module info.
> >>
> >> Do we already have such (based off ServiceML) ClassResolver?
> >> If yes, why it's not used?
> >> As I don't see how apps that use Infinispan (EDG?) can work w/o it (aka
> >> not having the same problem).
> >
> > Yes it should be using the SML, it looks like its set up correctly in CacheContainerAdd.java:265. So not sure how it's getting the wrong value?
>
> I'm cloning configurations from pre-defined configs (in infinispan.xml), for each app.
> Perhaps there is where it gets lost? <------- Paul
> (I've seen other configs get lost before :-))
>
> Although, I don't know why plain ModuleLoader would then be used ...
>
> -Ales

If you're loading cache configurations from infinispan.xml - they will
not be wired using the SML.  Only cache configurations defined in the
infinispan subsystem - or via the management API - will use this add
operation.  When you clone your cache configurations, you'll want to
wire them to use the ModularClassResolver via
ConfigurationBuilder.classResolver(...) method.

Paul

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Ales Justin
>>> Yes it should be using the SML, it looks like its set up correctly in CacheContainerAdd.java:265. So not sure how it's getting the wrong value?
>>
>> I'm cloning configurations from pre-defined configs (in infinispan.xml), for each app.
>> Perhaps there is where it gets lost? <------- Paul
>> (I've seen other configs get lost before :-))
>>
>> Although, I don't know why plain ModuleLoader would then be used ...
>>
>> -Ales
>
> If you're loading cache configurations from infinispan.xml - they will
> not be wired using the SML.  Only cache configurations defined in the
> infinispan subsystem - or via the management API - will use this add
> operation.  When you clone your cache configurations, you'll want to
> wire them to use the ModularClassResolver via
> ConfigurationBuilder.classResolver(...) method.

Nah, missleading "infinispan.xml" usage. :-)
I'm using the Infinispan subsystem (I meant AS7' infinispan.xml build file).
Shouldn't then SML be used automatically -- after the config clone?

-Ales


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Ales Justin
>>>> Yes it should be using the SML, it looks like its set up correctly in CacheContainerAdd.java:265. So not sure how it's getting the wrong value?
>>>
>>> I'm cloning configurations from pre-defined configs (in infinispan.xml), for each app.
>>> Perhaps there is where it gets lost? <------- Paul
>>> (I've seen other configs get lost before :-))
>>>
>>> Although, I don't know why plain ModuleLoader would then be used ...
>>>
>>> -Ales
>>
>> If you're loading cache configurations from infinispan.xml - they will
>> not be wired using the SML.  Only cache configurations defined in the
>> infinispan subsystem - or via the management API - will use this add
>> operation.  When you clone your cache configurations, you'll want to
>> wire them to use the ModularClassResolver via
>> ConfigurationBuilder.classResolver(...) method.
>
> Nah, missleading "infinispan.xml" usage. :-)
> I'm using the Infinispan subsystem (I meant AS7' infinispan.xml build file).
> Shouldn't then SML be used automatically -- after the config clone?

Or, you're saying I need to explicitly call this while cloning?

btw: any idea where the plain ML comes from?

-Ales


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Ales Justin
>>>>> Yes it should be using the SML, it looks like its set up correctly in CacheContainerAdd.java:265. So not sure how it's getting the wrong value?
>>>>
>>>> I'm cloning configurations from pre-defined configs (in infinispan.xml), for each app.
>>>> Perhaps there is where it gets lost? <------- Paul
>>>> (I've seen other configs get lost before :-))
>>>>
>>>> Although, I don't know why plain ModuleLoader would then be used ...
>>>>
>>>> -Ales
>>>
>>> If you're loading cache configurations from infinispan.xml - they will
>>> not be wired using the SML.  Only cache configurations defined in the
>>> infinispan subsystem - or via the management API - will use this add
>>> operation.  When you clone your cache configurations, you'll want to
>>> wire them to use the ModularClassResolver via
>>> ConfigurationBuilder.classResolver(...) method.
>>
>> Nah, missleading "infinispan.xml" usage. :-)
>> I'm using the Infinispan subsystem (I meant AS7' infinispan.xml build file).
>> Shouldn't then SML be used automatically -- after the config clone?
>
> Or, you're saying I need to explicitly call this while cloning?
>
> btw: any idea where the plain ML comes from?

This is cache config cloning, not cache container config.

Where I would expect the cache config to get the ClassResolver info from its container' config.

-Ales


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

jtgreene
Administrator
In reply to this post by David Lloyd-2
On 5/10/12 11:21 AM, David M. Lloyd wrote:

> On 05/10/2012 10:24 AM, Scott Marlow wrote:
>> On 05/09/2012 02:01 PM, David M. Lloyd wrote:
>>> OK I admit I LOL'ed.
>>>
>>> On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:
>>>> Now that I have your attention, I'd like to discuss issues we are
>>>> experiencing when trying to modularize the Hibernate portfolio and
>>>> make it work in AS 7.1.
>>>>
>>>> ## Disclaimer
>>>>
>>>> I perfectly understand all the coolness about modularity (speed,
>>>> easier dependency isolation etc). I have also carefully read :
>>>>
>>>> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
>>>> - https://community.jboss.org/wiki/ModularSerialization
>>>>
>>>> But these tend to avoid the more complex cases of portable libraries
>>>> that ought to run even outside AS 7 but have a wide variety of class
>>>> and resource loading needs.
>>>> I am not a complete modularity bozo but I am definitely not familiar
>>>> with JBoss Modules nor similar solution.
>>>>
>>>> ## Requirements / Landscape
>>>>
>>>> Hibernate ORM uses the notion of service registry and integrator
>>>> object that help during the integration or customization of the
>>>> engine behavior by third-party frameworks.
>>>> Enlistment of Integrators is done via the service locator pattern (a
>>>> service file in META-INF/services/ that is looked up and contain the
>>>> implementation class(es) at stake.
>>>>
>>>> Hibernate Envers is one of those customizer that depends on Hibernate
>>>> ORM. Note that the core of Hibernate ORM does not depend on Hibernate
>>>> Envers. The service locator file is contained in Hibernate Envers JAR.
>>>> Hibernate OGM likewise, heavily customizes ORM and depends on
>>>> Hibernate ORM classes - the reverse is not true. The service locator
>>>> file is contained in Hibernate OGM JAR.
>>>> Hibernate Search optionally depend on Hibernate ORM and JPA. The core
>>>> of Hibernate Search is independent but an Hibernate Search ORM module
>>>> has an integrator implementation. On top of that, Hibernate Search
>>>> optionally depend on some JPA classes and behaves differently if they
>>>> are there - we look them up in the classpath by reflection.
>>>>
>>>> On top of that, these projects do load resources (config files,
>>>> classes):
>>>>
>>>> - from what Jason calls a Deployment classloader (the user
>>>> application classes and resources really) - entities, custom analyzer
>>>> implementations, resources files etc. A user could even write a
>>>> custom Integrator and use the service locator pattern from his
>>>> application.
>>>> - from direct dependencies (Lucene is a declared dependency of
>>>> Hibernate Search)
>>>> - from dependencies of the deployment: for example an app developer
>>>> adds the phonetic analyzer as a dependency of his application and ask
>>>> Hibernate Search to use it
>>>> - from modules that use these projects. Modeshape and Capedwarf are
>>>> being modularized and are making use of Hibernate Search as a module.
>>>> Properly loading the necessary classes located in Modeshape or
>>>> Capedwarf's module but from Hibernate Search's engine proves to be
>>>> very hard in our current approach.
>>>>
>>>> All of these projects should be able to run outside JBoss AS 7, so a
>>>> modular friendly solution should somehow be abstracted and generic
>>>> enough.
>>>>
>>>> ## What solution?
>>>>
>>>> More and more projects are being modularized including ones with
>>>> complex resource loading dependencies like the ones I have described.
>>>> AFAIK Infinispan is even in a worse situation as clustering and late
>>>> class binding is at stake but let's put this one aside.
>>>> I'd love to get a reference design outcome from this thread that
>>>> could be copied across all these projects and future ones like Bean
>>>> Validation.
>>>>
>>>> Today, we mostly use the good old and simple TCCL model which works
>>>> fine if the jars are directly embedded in the app but fail the minute
>>>> we start to move these dependencies into modules. Sanne, Strong,
>>>> Scott Marlow and I are using dangerous amount of Advil to try and
>>>> make everything work as expected. Some help would be awesome.
>>>>
>>>> To sum up:
>>>>
>>>> - can the Hibernate portfolio be supported within JBoss Module and how?
>>>> - what kind of ClassloaderService contract should we use within these
>>>> projects to be modular friendly (JBoss Modules and others)?
>>>> - would such contract be generic enough to be extrapolated to JSRs in
>>>> need of modular friendliness?
>>>> - how to solve the chicken and egg issue of the bootstrapping: if we
>>>> need to pass a ClassloaderService impl?
>>> How do we do that best in a modular environment without forcing the
>>> application developer to implement such godforsaken ClassloaderService
>>> contract or even worse pass directly to us the right classloader for
>>> each call.
>>>
>>> I'll just start at the beginning and you can skip over the background if
>>> you like.
>>>
>>> The key starting concept is that a class' (or package's) identity is not
>>> just its name but also its class loader. This is the underlying
>>> (existing) truth that modularity brings to the fore. Corollary to this
>>> are the fact that a single VM may have more than one class or package
>>> with the same name, as well as the fact that not all classes/packages
>>> are always directly visible from a given class loader.
>>>
>>> This problem (as you've seen) manifests itself primarily when you're
>>> locating a class or a resource by name. You basically have two options.
>>> You can search *relative* to a class loader (most commonly, TCCL,
>>> though using a single explicit class loader or the caller's class loader
>>> also fall into this category). Or, you can use the *absolute* identity
>>> of a class.
>>>
>>> Using relative resolution is often a perfectly adequate solution when
>>> you're loading a single class or resource; in fact for some cases (like
>>> ServiceLoader for example) it's a perfect fit in conjunction with TCCL
>>> (in its capacity as an identifier for the "current" application). You
>>> want the user to be able to specify their implementation of something,
>>> and you want it to be reasonably transparent; ServiceLoader+TCCL does
>>> this fairly well.
>>>
>>> ServiceLoader also does well from the perspective of APIs with a static,
>>> fixed number of implementations. In this case, it is appropriate for a
>>> framework to have ServiceLoader use the class loader of the framework
>>> itself. The framework would then be sure to import the implementations
>>> in question (including their service descriptors); in our modular
>>> environment, which we call a "service import". Note that this often
>>> means there is a circular dependency between API and implementation:
>>> that's OK!
>>
>> We currently use this for envers but that doesn't seem as desirable for
>> other members of the Hibernate portfolio that may be on a separate
>> lifecycle. For example, the Hibernate OGM is a persistence provider that
>> depends on Hibernate ORM. If we have Hibernate ORM depend on OGM, that
>> limits the number of OGM versions that can be in use on AS7.
>
> Cases where implementations are pluggable could be handled using TCCL or
> by specifying a class loader, or by explicit registration at run time.
> As I said above, you would only use static dependencies if the
> implementations are more or less fixed.
>
>> Would it be possible, to add a MSC enhancement, that allows an inverse
>> dependency service loader dependency to be expressed? Such that it would
>> be enough to only have OGM depend on ORM (with an inverse service
>> dependency specified). I'm thinking that the OGM module would need to
>> exchange the service dependency information with the ORM module and
>> clear it, when OGM goes away.
>
> MSC has absolutely nothing to do with any of this.  Module loading is
> static and is not affected by services in any way.
>
> In your case you probably want a registration system.  But be aware that
> if you have more than one OGM implementation, you're not going to be
> able to use the ORM API solely to load them anyway (how would you tell
> it which one you want?); instead you'd need some mechanism to specify,
> so why not use TCCL or explicit class loader specification, as appropriate?

In case it's helpful. The way he handle JAXP is that we have a container
default and we also support TCCL.

The container default is defined by a special module:
https://github.com/jbossas/jboss-as/blob/master/build/src/main/resources/modules/javax/xml/jaxp-provider/main/module.xml


The rules are

1. Use service loader on TCCL (users deployment first), if an impl is
found use that
2. If not result is find use the default implementation, loaded from the
classloader of the "jaxp-provider" default module.
3. If the default jaxp-provider module doesn't contain it, we use the
JVM default, whatever it may be.

The way we implement this relies on installing a false, delegating JAXP
provider. Although in our own code we should define SPIs of some sort.

So for example, Hibernate OGM could provide a ProviderLocator interface,
which AS would then implement. Our implementation would follow similar
logic as above, although maybe you want slightly different rules. OGM
would ship with a predefined ProviderLocator which would simply search
TCCL.

--
Jason T. Greene
JBoss AS Lead / EAP Platform Architect
JBoss, a division of Red Hat
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Thomas Diesler
In reply to this post by Emmanuel Bernard
The standard (i.e. OSGi) works like this:

* There is an external ServiceRegistry that both the service provider
and service consumer can depend on.
* The service producer binds a service instance typically using an API
interface to the registry
* A service consumer has a class loading dependency (wire) on the API
and uses the registry to do a lookup.
* Multiple service implementations are allowed with well defined
ordering rules
* In contrary to jboss-modules, dependencies can be expressed at package
level (version ranges are supported)
* Similar to jboss-modules, explicit dependencies are also supported
(require bundle semantic) - discouraged!

If I understand correctly your current service lookup approach works
well with a flat class space. It should hence be possible to come up
with a simple service registry abstraction that can use an underlying
OSGi registry, a jboss-msc layer or some other implementation. An
initial registry impl that works with ServiceLoader is feasible. There
is also a project that brings the OSGi service registry to non-osgi
environments (i.e. without the modularity part)

For the module dependency part you could:

* Also use the standard (i.e. OSGi metadata in your jars) - this should
then work in any OSGi runtime (felix, equinoix, virgo, etc), but also in
a flat class space if the bundles can be treated as jars.
  * Please also note that today most appservers build on top of OSGi.
Hibernate bundles would work in WebSphere, WebLogic, Glassfish, etc. out
of the box. It would not work out of the box in AS7 however
* For AS7 Hibernate bundlescould be treated as as ordinary modules with
their dependencies expressed in modules.xml

In future the distinction between modules/bundles should go away in AS7
when we manage to align the lifecycle of module/bundle deployments.

In short - going with the standard is a viable option IMHO.

cheers
-thomas

On 05/09/2012 06:50 PM, Emmanuel Bernard wrote:

> Now that I have your attention, I'd like to discuss issues we are experiencing when trying to modularize the Hibernate portfolio and make it work in AS 7.1.
>
> ## Disclaimer
>
> I perfectly understand all the coolness about modularity (speed, easier dependency isolation etc). I have also carefully read :
>
> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
> - https://community.jboss.org/wiki/ModularSerialization
>
> But these tend to avoid the more complex cases of portable libraries that ought to run even outside AS 7 but have a wide variety of class and resource loading needs.
> I am not a complete modularity bozo but I am definitely not familiar with JBoss Modules nor similar solution.
>
> ## Requirements / Landscape
>
> Hibernate ORM uses the notion of service registry and integrator object that help during the integration or customization of the engine behavior by third-party frameworks.
> Enlistment of Integrators is done via the service locator pattern (a service file in META-INF/services/ that is looked up and contain the implementation class(es) at stake.
>
> Hibernate Envers is one of those customizer that depends on Hibernate ORM. Note that the core of Hibernate ORM does not depend on Hibernate Envers. The service locator file is contained in Hibernate Envers JAR.
> Hibernate OGM likewise, heavily customizes ORM and depends on Hibernate ORM classes - the reverse is not true. The service locator file is contained in Hibernate OGM JAR.
> Hibernate Search optionally depend on Hibernate ORM and JPA. The core of Hibernate Search is independent but an Hibernate Search ORM module has an integrator implementation. On top of that, Hibernate Search optionally depend on some JPA classes and behaves differently if they are there - we look them up in the classpath by reflection.
>
> On top of that, these projects do load resources (config files, classes):
>
> - from what Jason calls a Deployment classloader (the user application classes and resources really) - entities, custom analyzer implementations, resources files etc. A user could even write a custom Integrator and use the service locator pattern from his application.
> - from direct dependencies (Lucene is a declared dependency of Hibernate Search)
> - from dependencies of the deployment: for example an app developer adds the phonetic analyzer as a dependency of his application and ask Hibernate Search to use it
> - from modules that use these projects. Modeshape and Capedwarf are being modularized and are making use of Hibernate Search as a module. Properly loading the necessary classes located in Modeshape or Capedwarf's module but from Hibernate Search's engine proves to be very hard in our current approach.
>
> All of these projects should be able to run outside JBoss AS 7, so a modular friendly solution should somehow be abstracted and generic enough.
>
> ## What solution?
>
> More and more projects are being modularized including ones with complex resource loading dependencies like the ones I have described. AFAIK Infinispan is even in a worse situation as clustering and late class binding is at stake but let's put this one aside.
> I'd love to get a reference design outcome from this thread that could be copied across all these projects and future ones like Bean Validation.
>
> Today, we mostly use the good old and simple TCCL model which works fine if the jars are directly embedded in the app but fail the minute we start to move these dependencies into modules. Sanne, Strong, Scott Marlow and I are using dangerous amount of Advil to try and make everything work as expected. Some help would be awesome.
>
> To sum up:
>
> - can the Hibernate portfolio be supported within JBoss Module and how?
> - what kind of ClassloaderService contract should we use within these projects to be modular friendly (JBoss Modules and others)?
> - would such contract be generic enough to be extrapolated to JSRs in need of modular friendliness?
> - how to solve the chicken and egg issue of the bootstrapping: if we need to pass a ClassloaderService impl? How do we do that best in a modular environment without forcing the application developer to implement such godforsaken ClassloaderService contract or even worse pass directly to us the right classloader for each call.
>
> Thanks,
>
> Emmanuel
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Scott Marlow
In reply to this post by jtgreene
On 05/10/2012 03:04 PM, Jason T. Greene wrote:

> On 5/10/12 11:21 AM, David M. Lloyd wrote:
>> On 05/10/2012 10:24 AM, Scott Marlow wrote:
>>> On 05/09/2012 02:01 PM, David M. Lloyd wrote:
>>>> OK I admit I LOL'ed.
>>>>
>>>> On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:
>>>>> Now that I have your attention, I'd like to discuss issues we are
>>>>> experiencing when trying to modularize the Hibernate portfolio and
>>>>> make it work in AS 7.1.
>>>>>
>>>>> ## Disclaimer
>>>>>
>>>>> I perfectly understand all the coolness about modularity (speed,
>>>>> easier dependency isolation etc). I have also carefully read :
>>>>>
>>>>> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
>>>>> - https://community.jboss.org/wiki/ModularSerialization
>>>>>
>>>>> But these tend to avoid the more complex cases of portable libraries
>>>>> that ought to run even outside AS 7 but have a wide variety of class
>>>>> and resource loading needs.
>>>>> I am not a complete modularity bozo but I am definitely not familiar
>>>>> with JBoss Modules nor similar solution.
>>>>>
>>>>> ## Requirements / Landscape
>>>>>
>>>>> Hibernate ORM uses the notion of service registry and integrator
>>>>> object that help during the integration or customization of the
>>>>> engine behavior by third-party frameworks.
>>>>> Enlistment of Integrators is done via the service locator pattern (a
>>>>> service file in META-INF/services/ that is looked up and contain the
>>>>> implementation class(es) at stake.
>>>>>
>>>>> Hibernate Envers is one of those customizer that depends on Hibernate
>>>>> ORM. Note that the core of Hibernate ORM does not depend on Hibernate
>>>>> Envers. The service locator file is contained in Hibernate Envers JAR.
>>>>> Hibernate OGM likewise, heavily customizes ORM and depends on
>>>>> Hibernate ORM classes - the reverse is not true. The service locator
>>>>> file is contained in Hibernate OGM JAR.
>>>>> Hibernate Search optionally depend on Hibernate ORM and JPA. The core
>>>>> of Hibernate Search is independent but an Hibernate Search ORM module
>>>>> has an integrator implementation. On top of that, Hibernate Search
>>>>> optionally depend on some JPA classes and behaves differently if they
>>>>> are there - we look them up in the classpath by reflection.
>>>>>
>>>>> On top of that, these projects do load resources (config files,
>>>>> classes):
>>>>>
>>>>> - from what Jason calls a Deployment classloader (the user
>>>>> application classes and resources really) - entities, custom analyzer
>>>>> implementations, resources files etc. A user could even write a
>>>>> custom Integrator and use the service locator pattern from his
>>>>> application.
>>>>> - from direct dependencies (Lucene is a declared dependency of
>>>>> Hibernate Search)
>>>>> - from dependencies of the deployment: for example an app developer
>>>>> adds the phonetic analyzer as a dependency of his application and ask
>>>>> Hibernate Search to use it
>>>>> - from modules that use these projects. Modeshape and Capedwarf are
>>>>> being modularized and are making use of Hibernate Search as a module.
>>>>> Properly loading the necessary classes located in Modeshape or
>>>>> Capedwarf's module but from Hibernate Search's engine proves to be
>>>>> very hard in our current approach.
>>>>>
>>>>> All of these projects should be able to run outside JBoss AS 7, so a
>>>>> modular friendly solution should somehow be abstracted and generic
>>>>> enough.
>>>>>
>>>>> ## What solution?
>>>>>
>>>>> More and more projects are being modularized including ones with
>>>>> complex resource loading dependencies like the ones I have described.
>>>>> AFAIK Infinispan is even in a worse situation as clustering and late
>>>>> class binding is at stake but let's put this one aside.
>>>>> I'd love to get a reference design outcome from this thread that
>>>>> could be copied across all these projects and future ones like Bean
>>>>> Validation.
>>>>>
>>>>> Today, we mostly use the good old and simple TCCL model which works
>>>>> fine if the jars are directly embedded in the app but fail the minute
>>>>> we start to move these dependencies into modules. Sanne, Strong,
>>>>> Scott Marlow and I are using dangerous amount of Advil to try and
>>>>> make everything work as expected. Some help would be awesome.
>>>>>
>>>>> To sum up:
>>>>>
>>>>> - can the Hibernate portfolio be supported within JBoss Module and
>>>>> how?
>>>>> - what kind of ClassloaderService contract should we use within these
>>>>> projects to be modular friendly (JBoss Modules and others)?
>>>>> - would such contract be generic enough to be extrapolated to JSRs in
>>>>> need of modular friendliness?
>>>>> - how to solve the chicken and egg issue of the bootstrapping: if we
>>>>> need to pass a ClassloaderService impl?
>>>> How do we do that best in a modular environment without forcing the
>>>> application developer to implement such godforsaken ClassloaderService
>>>> contract or even worse pass directly to us the right classloader for
>>>> each call.
>>>>
>>>> I'll just start at the beginning and you can skip over the
>>>> background if
>>>> you like.
>>>>
>>>> The key starting concept is that a class' (or package's) identity is
>>>> not
>>>> just its name but also its class loader. This is the underlying
>>>> (existing) truth that modularity brings to the fore. Corollary to this
>>>> are the fact that a single VM may have more than one class or package
>>>> with the same name, as well as the fact that not all classes/packages
>>>> are always directly visible from a given class loader.
>>>>
>>>> This problem (as you've seen) manifests itself primarily when you're
>>>> locating a class or a resource by name. You basically have two options.
>>>> You can search *relative* to a class loader (most commonly, TCCL,
>>>> though using a single explicit class loader or the caller's class
>>>> loader
>>>> also fall into this category). Or, you can use the *absolute* identity
>>>> of a class.
>>>>
>>>> Using relative resolution is often a perfectly adequate solution when
>>>> you're loading a single class or resource; in fact for some cases (like
>>>> ServiceLoader for example) it's a perfect fit in conjunction with TCCL
>>>> (in its capacity as an identifier for the "current" application). You
>>>> want the user to be able to specify their implementation of something,
>>>> and you want it to be reasonably transparent; ServiceLoader+TCCL does
>>>> this fairly well.
>>>>
>>>> ServiceLoader also does well from the perspective of APIs with a
>>>> static,
>>>> fixed number of implementations. In this case, it is appropriate for a
>>>> framework to have ServiceLoader use the class loader of the framework
>>>> itself. The framework would then be sure to import the implementations
>>>> in question (including their service descriptors); in our modular
>>>> environment, which we call a "service import". Note that this often
>>>> means there is a circular dependency between API and implementation:
>>>> that's OK!
>>>
>>> We currently use this for envers but that doesn't seem as desirable for
>>> other members of the Hibernate portfolio that may be on a separate
>>> lifecycle. For example, the Hibernate OGM is a persistence provider that
>>> depends on Hibernate ORM. If we have Hibernate ORM depend on OGM, that
>>> limits the number of OGM versions that can be in use on AS7.
>>
>> Cases where implementations are pluggable could be handled using TCCL or
>> by specifying a class loader, or by explicit registration at run time.
>> As I said above, you would only use static dependencies if the
>> implementations are more or less fixed.
>>
>>> Would it be possible, to add a MSC enhancement, that allows an inverse
>>> dependency service loader dependency to be expressed? Such that it would
>>> be enough to only have OGM depend on ORM (with an inverse service
>>> dependency specified). I'm thinking that the OGM module would need to
>>> exchange the service dependency information with the ORM module and
>>> clear it, when OGM goes away.
>>
>> MSC has absolutely nothing to do with any of this. Module loading is
>> static and is not affected by services in any way.
>>
>> In your case you probably want a registration system. But be aware that
>> if you have more than one OGM implementation, you're not going to be
>> able to use the ORM API solely to load them anyway (how would you tell
>> it which one you want?); instead you'd need some mechanism to specify,
>> so why not use TCCL or explicit class loader specification, as
>> appropriate?
>
> In case it's helpful. The way he handle JAXP is that we have a container
> default and we also support TCCL.
>
> The container default is defined by a special module:
> https://github.com/jbossas/jboss-as/blob/master/build/src/main/resources/modules/javax/xml/jaxp-provider/main/module.xml
>
>
>
> The rules are
>
> 1. Use service loader on TCCL (users deployment first), if an impl is
> found use that
> 2. If not result is find use the default implementation, loaded from the
> classloader of the "jaxp-provider" default module.
> 3. If the default jaxp-provider module doesn't contain it, we use the
> JVM default, whatever it may be.
>
> The way we implement this relies on installing a false, delegating JAXP
> provider. Although in our own code we should define SPIs of some sort.
>
> So for example, Hibernate OGM could provide a ProviderLocator interface,
> which AS would then implement. Our implementation would follow similar
> logic as above, although maybe you want slightly different rules. OGM
> would ship with a predefined ProviderLocator which would simply search
> TCCL.
>

David/Jason,

It might be helpful if we discussed the Hibernate ORM case on IRC
(either in #Hibernate-dev or #jboss-as7 (or a private room perhaps).  Do
you guys have any time this week for that?

Scott
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Steve Ebersole
Hey all, just to tie this up with a bow I wanted to send my take away
from the discussions we had on IRC and make sure everyone had the same
take away.  Jason and David, wanted to thank you guys again for your
time and input.  It really helped out.  I also went ahead and
copied/pasted the transcript so we had record of discussion.

The basic idea is to control this via a specific deployment descriptor
(jboss-hibernate.xml or somesuch).  It would, in the general case,
provide 2 pieces of information:
1) Which Hibernate version are we talking about.  This is a very
high-level concept of version, mainly 3 or 4.
2) Which "modules" do you want included.  OGM?  Envers?  Etc.  It won't
name modules per-se in terms of AS modules.  But more what Hibernate
integrations do you want included.  We will also allow for custom AS
modules to be named here, since they could also be "Hibernate
integrations" (provide service loader locatable impls of Integrator).  
Additionally, the inclusion of a module here will export the classes
from the corresponding AS module into the app classloader.

To account for app deployments that contain Hibernate jars, the
suggestion was to allow a flag in this deployment descriptor to disable
the inclusion of AS provided Hibernate modules/jars.  This could also
be leveraged to, for example, bundle an alternate JPA provider.

There is still some gray area around what to do when the app deployment
contains Hibernate jars but has no deployment descriptor.  Its kind of
a 50/50 proposition and seems like there is not a good solution.


On Mon 21 May 2012 02:25:47 PM CDT, Scott Marlow wrote:

> On 05/10/2012 03:04 PM, Jason T. Greene wrote:
>> On 5/10/12 11:21 AM, David M. Lloyd wrote:
>>> On 05/10/2012 10:24 AM, Scott Marlow wrote:
>>>> On 05/09/2012 02:01 PM, David M. Lloyd wrote:
>>>>> OK I admit I LOL'ed.
>>>>>
>>>>> On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:
>>>>>> Now that I have your attention, I'd like to discuss issues we are
>>>>>> experiencing when trying to modularize the Hibernate portfolio and
>>>>>> make it work in AS 7.1.
>>>>>>
>>>>>> ## Disclaimer
>>>>>>
>>>>>> I perfectly understand all the coolness about modularity (speed,
>>>>>> easier dependency isolation etc). I have also carefully read :
>>>>>>
>>>>>> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
>>>>>> - https://community.jboss.org/wiki/ModularSerialization
>>>>>>
>>>>>> But these tend to avoid the more complex cases of portable libraries
>>>>>> that ought to run even outside AS 7 but have a wide variety of class
>>>>>> and resource loading needs.
>>>>>> I am not a complete modularity bozo but I am definitely not familiar
>>>>>> with JBoss Modules nor similar solution.
>>>>>>
>>>>>> ## Requirements / Landscape
>>>>>>
>>>>>> Hibernate ORM uses the notion of service registry and integrator
>>>>>> object that help during the integration or customization of the
>>>>>> engine behavior by third-party frameworks.
>>>>>> Enlistment of Integrators is done via the service locator pattern (a
>>>>>> service file in META-INF/services/ that is looked up and contain the
>>>>>> implementation class(es) at stake.
>>>>>>
>>>>>> Hibernate Envers is one of those customizer that depends on
>>>>>> Hibernate
>>>>>> ORM. Note that the core of Hibernate ORM does not depend on
>>>>>> Hibernate
>>>>>> Envers. The service locator file is contained in Hibernate Envers
>>>>>> JAR.
>>>>>> Hibernate OGM likewise, heavily customizes ORM and depends on
>>>>>> Hibernate ORM classes - the reverse is not true. The service locator
>>>>>> file is contained in Hibernate OGM JAR.
>>>>>> Hibernate Search optionally depend on Hibernate ORM and JPA. The
>>>>>> core
>>>>>> of Hibernate Search is independent but an Hibernate Search ORM
>>>>>> module
>>>>>> has an integrator implementation. On top of that, Hibernate Search
>>>>>> optionally depend on some JPA classes and behaves differently if
>>>>>> they
>>>>>> are there - we look them up in the classpath by reflection.
>>>>>>
>>>>>> On top of that, these projects do load resources (config files,
>>>>>> classes):
>>>>>>
>>>>>> - from what Jason calls a Deployment classloader (the user
>>>>>> application classes and resources really) - entities, custom
>>>>>> analyzer
>>>>>> implementations, resources files etc. A user could even write a
>>>>>> custom Integrator and use the service locator pattern from his
>>>>>> application.
>>>>>> - from direct dependencies (Lucene is a declared dependency of
>>>>>> Hibernate Search)
>>>>>> - from dependencies of the deployment: for example an app developer
>>>>>> adds the phonetic analyzer as a dependency of his application and
>>>>>> ask
>>>>>> Hibernate Search to use it
>>>>>> - from modules that use these projects. Modeshape and Capedwarf are
>>>>>> being modularized and are making use of Hibernate Search as a
>>>>>> module.
>>>>>> Properly loading the necessary classes located in Modeshape or
>>>>>> Capedwarf's module but from Hibernate Search's engine proves to be
>>>>>> very hard in our current approach.
>>>>>>
>>>>>> All of these projects should be able to run outside JBoss AS 7, so a
>>>>>> modular friendly solution should somehow be abstracted and generic
>>>>>> enough.
>>>>>>
>>>>>> ## What solution?
>>>>>>
>>>>>> More and more projects are being modularized including ones with
>>>>>> complex resource loading dependencies like the ones I have
>>>>>> described.
>>>>>> AFAIK Infinispan is even in a worse situation as clustering and late
>>>>>> class binding is at stake but let's put this one aside.
>>>>>> I'd love to get a reference design outcome from this thread that
>>>>>> could be copied across all these projects and future ones like Bean
>>>>>> Validation.
>>>>>>
>>>>>> Today, we mostly use the good old and simple TCCL model which works
>>>>>> fine if the jars are directly embedded in the app but fail the
>>>>>> minute
>>>>>> we start to move these dependencies into modules. Sanne, Strong,
>>>>>> Scott Marlow and I are using dangerous amount of Advil to try and
>>>>>> make everything work as expected. Some help would be awesome.
>>>>>>
>>>>>> To sum up:
>>>>>>
>>>>>> - can the Hibernate portfolio be supported within JBoss Module and
>>>>>> how?
>>>>>> - what kind of ClassloaderService contract should we use within
>>>>>> these
>>>>>> projects to be modular friendly (JBoss Modules and others)?
>>>>>> - would such contract be generic enough to be extrapolated to
>>>>>> JSRs in
>>>>>> need of modular friendliness?
>>>>>> - how to solve the chicken and egg issue of the bootstrapping: if we
>>>>>> need to pass a ClassloaderService impl?
>>>>> How do we do that best in a modular environment without forcing the
>>>>> application developer to implement such godforsaken
>>>>> ClassloaderService
>>>>> contract or even worse pass directly to us the right classloader for
>>>>> each call.
>>>>>
>>>>> I'll just start at the beginning and you can skip over the
>>>>> background if
>>>>> you like.
>>>>>
>>>>> The key starting concept is that a class' (or package's) identity is
>>>>> not
>>>>> just its name but also its class loader. This is the underlying
>>>>> (existing) truth that modularity brings to the fore. Corollary to
>>>>> this
>>>>> are the fact that a single VM may have more than one class or package
>>>>> with the same name, as well as the fact that not all classes/packages
>>>>> are always directly visible from a given class loader.
>>>>>
>>>>> This problem (as you've seen) manifests itself primarily when you're
>>>>> locating a class or a resource by name. You basically have two
>>>>> options.
>>>>> You can search *relative* to a class loader (most commonly, TCCL,
>>>>> though using a single explicit class loader or the caller's class
>>>>> loader
>>>>> also fall into this category). Or, you can use the *absolute*
>>>>> identity
>>>>> of a class.
>>>>>
>>>>> Using relative resolution is often a perfectly adequate solution when
>>>>> you're loading a single class or resource; in fact for some cases
>>>>> (like
>>>>> ServiceLoader for example) it's a perfect fit in conjunction with
>>>>> TCCL
>>>>> (in its capacity as an identifier for the "current" application). You
>>>>> want the user to be able to specify their implementation of
>>>>> something,
>>>>> and you want it to be reasonably transparent; ServiceLoader+TCCL does
>>>>> this fairly well.
>>>>>
>>>>> ServiceLoader also does well from the perspective of APIs with a
>>>>> static,
>>>>> fixed number of implementations. In this case, it is appropriate
>>>>> for a
>>>>> framework to have ServiceLoader use the class loader of the framework
>>>>> itself. The framework would then be sure to import the
>>>>> implementations
>>>>> in question (including their service descriptors); in our modular
>>>>> environment, which we call a "service import". Note that this often
>>>>> means there is a circular dependency between API and implementation:
>>>>> that's OK!
>>>>
>>>> We currently use this for envers but that doesn't seem as desirable
>>>> for
>>>> other members of the Hibernate portfolio that may be on a separate
>>>> lifecycle. For example, the Hibernate OGM is a persistence provider
>>>> that
>>>> depends on Hibernate ORM. If we have Hibernate ORM depend on OGM, that
>>>> limits the number of OGM versions that can be in use on AS7.
>>>
>>> Cases where implementations are pluggable could be handled using
>>> TCCL or
>>> by specifying a class loader, or by explicit registration at run time.
>>> As I said above, you would only use static dependencies if the
>>> implementations are more or less fixed.
>>>
>>>> Would it be possible, to add a MSC enhancement, that allows an inverse
>>>> dependency service loader dependency to be expressed? Such that it
>>>> would
>>>> be enough to only have OGM depend on ORM (with an inverse service
>>>> dependency specified). I'm thinking that the OGM module would need to
>>>> exchange the service dependency information with the ORM module and
>>>> clear it, when OGM goes away.
>>>
>>> MSC has absolutely nothing to do with any of this. Module loading is
>>> static and is not affected by services in any way.
>>>
>>> In your case you probably want a registration system. But be aware that
>>> if you have more than one OGM implementation, you're not going to be
>>> able to use the ORM API solely to load them anyway (how would you tell
>>> it which one you want?); instead you'd need some mechanism to specify,
>>> so why not use TCCL or explicit class loader specification, as
>>> appropriate?
>>
>> In case it's helpful. The way he handle JAXP is that we have a container
>> default and we also support TCCL.
>>
>> The container default is defined by a special module:
>> https://github.com/jbossas/jboss-as/blob/master/build/src/main/resources/modules/javax/xml/jaxp-provider/main/module.xml
>>
>>
>>
>>
>> The rules are
>>
>> 1. Use service loader on TCCL (users deployment first), if an impl is
>> found use that
>> 2. If not result is find use the default implementation, loaded from the
>> classloader of the "jaxp-provider" default module.
>> 3. If the default jaxp-provider module doesn't contain it, we use the
>> JVM default, whatever it may be.
>>
>> The way we implement this relies on installing a false, delegating JAXP
>> provider. Although in our own code we should define SPIs of some sort.
>>
>> So for example, Hibernate OGM could provide a ProviderLocator interface,
>> which AS would then implement. Our implementation would follow similar
>> logic as above, although maybe you want slightly different rules. OGM
>> would ship with a predefined ProviderLocator which would simply search
>> TCCL.
>>
>
> David/Jason,
>
> It might be helpful if we discussed the Hibernate ORM case on IRC
> (either in #Hibernate-dev or #jboss-as7 (or a private room perhaps).
> Do you guys have any time this week for that?
>
> Scott
--
[hidden email]
http://hibernate.org

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

jboss-hibernate-irc-discussion.txt (48K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

Scott Marlow

On 05/21/2012 06:40 PM, Steve Ebersole wrote:
> Hey all, just to tie this up with a bow I wanted to send my take away
> from the discussions we had on IRC and make sure everyone had the same
> take away. Jason and David, wanted to thank you guys again for your time
> and input. It really helped out. I also went ahead and copied/pasted the
> transcript so we had record of discussion.
>
> The basic idea is to control this via a specific deployment descriptor
> (jboss-hibernate.xml or somesuch).

My understanding is that this jboss-hibernate.xml is an optional
deployment descriptor that would be used by a deployment (EE or non-EE)
that wants to use any of the following technologies:

* Hibernate Search
* Hibernate OGM
* Hibernate Integrator module implementation

The result of including a jboss-hibernate.xml, would be that the
deployment includes the needed Hibernate modules (Hibernate
Search/OGM/other custom modules).

If there is no jboss-hibernate.xml, there would be no change from
current AS behaviour (OGM could be packaged with the application but
cannot be used as a module like other persistence providers).

This in effect will allow Hibernate ORM to use the application
deployment as a composite module that also contains the Hibernate
Search/OGM/Other Integrator module implementation(s).

This might require some Hibernate ORM changes or not (depending on
whether its service loader searches in the TCCL currently).  We can
probably use the jboss-deployment-structure.xml as an alternative (short
term solution) before we have this new Hibernate deployer that looks for
the presence of the jboss-hibernate.xml file.  I think more discussion
is needed about the contents of jboss-hibernate.xml before we can
implement that.

How does that sound?

It would, in the general case,

> provide 2 pieces of information:
> 1) Which Hibernate version are we talking about. This is a very
> high-level concept of version, mainly 3 or 4.
> 2) Which "modules" do you want included. OGM? Envers? Etc. It won't name
> modules per-se in terms of AS modules. But more what Hibernate
> integrations do you want included. We will also allow for custom AS
> modules to be named here, since they could also be "Hibernate
> integrations" (provide service loader locatable impls of Integrator).
> Additionally, the inclusion of a module here will export the classes
> from the corresponding AS module into the app classloader.
>
> To account for app deployments that contain Hibernate jars, the
> suggestion was to allow a flag in this deployment descriptor to disable
> the inclusion of AS provided Hibernate modules/jars. This could also be
> leveraged to, for example, bundle an alternate JPA provider.
>
> There is still some gray area around what to do when the app deployment
> contains Hibernate jars but has no deployment descriptor. Its kind of a
> 50/50 proposition and seems like there is not a good solution.
>
>
> On Mon 21 May 2012 02:25:47 PM CDT, Scott Marlow wrote:
>> On 05/10/2012 03:04 PM, Jason T. Greene wrote:
>>> On 5/10/12 11:21 AM, David M. Lloyd wrote:
>>>> On 05/10/2012 10:24 AM, Scott Marlow wrote:
>>>>> On 05/09/2012 02:01 PM, David M. Lloyd wrote:
>>>>>> OK I admit I LOL'ed.
>>>>>>
>>>>>> On 05/09/2012 11:50 AM, Emmanuel Bernard wrote:
>>>>>>> Now that I have your attention, I'd like to discuss issues we are
>>>>>>> experiencing when trying to modularize the Hibernate portfolio and
>>>>>>> make it work in AS 7.1.
>>>>>>>
>>>>>>> ## Disclaimer
>>>>>>>
>>>>>>> I perfectly understand all the coolness about modularity (speed,
>>>>>>> easier dependency isolation etc). I have also carefully read :
>>>>>>>
>>>>>>> - https://community.jboss.org/wiki/ModuleCompatibleClassloadingGuide
>>>>>>> - https://community.jboss.org/wiki/ModularSerialization
>>>>>>>
>>>>>>> But these tend to avoid the more complex cases of portable libraries
>>>>>>> that ought to run even outside AS 7 but have a wide variety of class
>>>>>>> and resource loading needs.
>>>>>>> I am not a complete modularity bozo but I am definitely not familiar
>>>>>>> with JBoss Modules nor similar solution.
>>>>>>>
>>>>>>> ## Requirements / Landscape
>>>>>>>
>>>>>>> Hibernate ORM uses the notion of service registry and integrator
>>>>>>> object that help during the integration or customization of the
>>>>>>> engine behavior by third-party frameworks.
>>>>>>> Enlistment of Integrators is done via the service locator pattern (a
>>>>>>> service file in META-INF/services/ that is looked up and contain the
>>>>>>> implementation class(es) at stake.
>>>>>>>
>>>>>>> Hibernate Envers is one of those customizer that depends on
>>>>>>> Hibernate
>>>>>>> ORM. Note that the core of Hibernate ORM does not depend on
>>>>>>> Hibernate
>>>>>>> Envers. The service locator file is contained in Hibernate Envers
>>>>>>> JAR.
>>>>>>> Hibernate OGM likewise, heavily customizes ORM and depends on
>>>>>>> Hibernate ORM classes - the reverse is not true. The service locator
>>>>>>> file is contained in Hibernate OGM JAR.
>>>>>>> Hibernate Search optionally depend on Hibernate ORM and JPA. The
>>>>>>> core
>>>>>>> of Hibernate Search is independent but an Hibernate Search ORM
>>>>>>> module
>>>>>>> has an integrator implementation. On top of that, Hibernate Search
>>>>>>> optionally depend on some JPA classes and behaves differently if
>>>>>>> they
>>>>>>> are there - we look them up in the classpath by reflection.
>>>>>>>
>>>>>>> On top of that, these projects do load resources (config files,
>>>>>>> classes):
>>>>>>>
>>>>>>> - from what Jason calls a Deployment classloader (the user
>>>>>>> application classes and resources really) - entities, custom
>>>>>>> analyzer
>>>>>>> implementations, resources files etc. A user could even write a
>>>>>>> custom Integrator and use the service locator pattern from his
>>>>>>> application.
>>>>>>> - from direct dependencies (Lucene is a declared dependency of
>>>>>>> Hibernate Search)
>>>>>>> - from dependencies of the deployment: for example an app developer
>>>>>>> adds the phonetic analyzer as a dependency of his application and
>>>>>>> ask
>>>>>>> Hibernate Search to use it
>>>>>>> - from modules that use these projects. Modeshape and Capedwarf are
>>>>>>> being modularized and are making use of Hibernate Search as a
>>>>>>> module.
>>>>>>> Properly loading the necessary classes located in Modeshape or
>>>>>>> Capedwarf's module but from Hibernate Search's engine proves to be
>>>>>>> very hard in our current approach.
>>>>>>>
>>>>>>> All of these projects should be able to run outside JBoss AS 7, so a
>>>>>>> modular friendly solution should somehow be abstracted and generic
>>>>>>> enough.
>>>>>>>
>>>>>>> ## What solution?
>>>>>>>
>>>>>>> More and more projects are being modularized including ones with
>>>>>>> complex resource loading dependencies like the ones I have
>>>>>>> described.
>>>>>>> AFAIK Infinispan is even in a worse situation as clustering and late
>>>>>>> class binding is at stake but let's put this one aside.
>>>>>>> I'd love to get a reference design outcome from this thread that
>>>>>>> could be copied across all these projects and future ones like Bean
>>>>>>> Validation.
>>>>>>>
>>>>>>> Today, we mostly use the good old and simple TCCL model which works
>>>>>>> fine if the jars are directly embedded in the app but fail the
>>>>>>> minute
>>>>>>> we start to move these dependencies into modules. Sanne, Strong,
>>>>>>> Scott Marlow and I are using dangerous amount of Advil to try and
>>>>>>> make everything work as expected. Some help would be awesome.
>>>>>>>
>>>>>>> To sum up:
>>>>>>>
>>>>>>> - can the Hibernate portfolio be supported within JBoss Module and
>>>>>>> how?
>>>>>>> - what kind of ClassloaderService contract should we use within
>>>>>>> these
>>>>>>> projects to be modular friendly (JBoss Modules and others)?
>>>>>>> - would such contract be generic enough to be extrapolated to
>>>>>>> JSRs in
>>>>>>> need of modular friendliness?
>>>>>>> - how to solve the chicken and egg issue of the bootstrapping: if we
>>>>>>> need to pass a ClassloaderService impl?
>>>>>> How do we do that best in a modular environment without forcing the
>>>>>> application developer to implement such godforsaken
>>>>>> ClassloaderService
>>>>>> contract or even worse pass directly to us the right classloader for
>>>>>> each call.
>>>>>>
>>>>>> I'll just start at the beginning and you can skip over the
>>>>>> background if
>>>>>> you like.
>>>>>>
>>>>>> The key starting concept is that a class' (or package's) identity is
>>>>>> not
>>>>>> just its name but also its class loader. This is the underlying
>>>>>> (existing) truth that modularity brings to the fore. Corollary to
>>>>>> this
>>>>>> are the fact that a single VM may have more than one class or package
>>>>>> with the same name, as well as the fact that not all classes/packages
>>>>>> are always directly visible from a given class loader.
>>>>>>
>>>>>> This problem (as you've seen) manifests itself primarily when you're
>>>>>> locating a class or a resource by name. You basically have two
>>>>>> options.
>>>>>> You can search *relative* to a class loader (most commonly, TCCL,
>>>>>> though using a single explicit class loader or the caller's class
>>>>>> loader
>>>>>> also fall into this category). Or, you can use the *absolute*
>>>>>> identity
>>>>>> of a class.
>>>>>>
>>>>>> Using relative resolution is often a perfectly adequate solution when
>>>>>> you're loading a single class or resource; in fact for some cases
>>>>>> (like
>>>>>> ServiceLoader for example) it's a perfect fit in conjunction with
>>>>>> TCCL
>>>>>> (in its capacity as an identifier for the "current" application). You
>>>>>> want the user to be able to specify their implementation of
>>>>>> something,
>>>>>> and you want it to be reasonably transparent; ServiceLoader+TCCL does
>>>>>> this fairly well.
>>>>>>
>>>>>> ServiceLoader also does well from the perspective of APIs with a
>>>>>> static,
>>>>>> fixed number of implementations. In this case, it is appropriate
>>>>>> for a
>>>>>> framework to have ServiceLoader use the class loader of the framework
>>>>>> itself. The framework would then be sure to import the
>>>>>> implementations
>>>>>> in question (including their service descriptors); in our modular
>>>>>> environment, which we call a "service import". Note that this often
>>>>>> means there is a circular dependency between API and implementation:
>>>>>> that's OK!
>>>>>
>>>>> We currently use this for envers but that doesn't seem as desirable
>>>>> for
>>>>> other members of the Hibernate portfolio that may be on a separate
>>>>> lifecycle. For example, the Hibernate OGM is a persistence provider
>>>>> that
>>>>> depends on Hibernate ORM. If we have Hibernate ORM depend on OGM, that
>>>>> limits the number of OGM versions that can be in use on AS7.
>>>>
>>>> Cases where implementations are pluggable could be handled using
>>>> TCCL or
>>>> by specifying a class loader, or by explicit registration at run time.
>>>> As I said above, you would only use static dependencies if the
>>>> implementations are more or less fixed.
>>>>
>>>>> Would it be possible, to add a MSC enhancement, that allows an inverse
>>>>> dependency service loader dependency to be expressed? Such that it
>>>>> would
>>>>> be enough to only have OGM depend on ORM (with an inverse service
>>>>> dependency specified). I'm thinking that the OGM module would need to
>>>>> exchange the service dependency information with the ORM module and
>>>>> clear it, when OGM goes away.
>>>>
>>>> MSC has absolutely nothing to do with any of this. Module loading is
>>>> static and is not affected by services in any way.
>>>>
>>>> In your case you probably want a registration system. But be aware that
>>>> if you have more than one OGM implementation, you're not going to be
>>>> able to use the ORM API solely to load them anyway (how would you tell
>>>> it which one you want?); instead you'd need some mechanism to specify,
>>>> so why not use TCCL or explicit class loader specification, as
>>>> appropriate?
>>>
>>> In case it's helpful. The way he handle JAXP is that we have a container
>>> default and we also support TCCL.
>>>
>>> The container default is defined by a special module:
>>> https://github.com/jbossas/jboss-as/blob/master/build/src/main/resources/modules/javax/xml/jaxp-provider/main/module.xml
>>>
>>>
>>>
>>>
>>>
>>> The rules are
>>>
>>> 1. Use service loader on TCCL (users deployment first), if an impl is
>>> found use that
>>> 2. If not result is find use the default implementation, loaded from the
>>> classloader of the "jaxp-provider" default module.
>>> 3. If the default jaxp-provider module doesn't contain it, we use the
>>> JVM default, whatever it may be.
>>>
>>> The way we implement this relies on installing a false, delegating JAXP
>>> provider. Although in our own code we should define SPIs of some sort.
>>>
>>> So for example, Hibernate OGM could provide a ProviderLocator interface,
>>> which AS would then implement. Our implementation would follow similar
>>> logic as above, although maybe you want slightly different rules. OGM
>>> would ship with a predefined ProviderLocator which would simply search
>>> TCCL.
>>>
>>
>> David/Jason,
>>
>> It might be helpful if we discussed the Hibernate ORM case on IRC
>> (either in #Hibernate-dev or #jboss-as7 (or a private room perhaps).
>> Do you guys have any time this week for that?
>>
>> Scott
>
> --
> [hidden email]
> http://hibernate.org

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Modularity is the spawn of Lucifer and a stinking donkey

David Lloyd-2
On 05/22/2012 09:10 AM, Scott Marlow wrote:

>
> On 05/21/2012 06:40 PM, Steve Ebersole wrote:
>> Hey all, just to tie this up with a bow I wanted to send my take away
>> from the discussions we had on IRC and make sure everyone had the same
>> take away. Jason and David, wanted to thank you guys again for your time
>> and input. It really helped out. I also went ahead and copied/pasted the
>> transcript so we had record of discussion.
>>
>> The basic idea is to control this via a specific deployment descriptor
>> (jboss-hibernate.xml or somesuch).
>
> My understanding is that this jboss-hibernate.xml is an optional
> deployment descriptor that would be used by a deployment (EE or non-EE)
> that wants to use any of the following technologies:
>
> * Hibernate Search
> * Hibernate OGM
> * Hibernate Integrator module implementation
>
> The result of including a jboss-hibernate.xml, would be that the
> deployment includes the needed Hibernate modules (Hibernate
> Search/OGM/other custom modules).
>
> If there is no jboss-hibernate.xml, there would be no change from
> current AS behaviour (OGM could be packaged with the application but
> cannot be used as a module like other persistence providers).

It sounds like you're implying that a regular JavaEE application with no
JBoss-specific descriptors in it cannot use JPA.  Is this correct?

> This in effect will allow Hibernate ORM to use the application
> deployment as a composite module that also contains the Hibernate
> Search/OGM/Other Integrator module implementation(s).

That's fine assuming that we have JPA out of the box.

> This might require some Hibernate ORM changes or not (depending on
> whether its service loader searches in the TCCL currently). We can
> probably use the jboss-deployment-structure.xml as an alternative (short
> term solution) before we have this new Hibernate deployer that looks for
> the presence of the jboss-hibernate.xml file. I think more discussion is
> needed about the contents of jboss-hibernate.xml before we can implement
> that.

The rule could probably be, if there's a jboss-hibernate.xml then use
that to configure, else use TCCL.  That should work in most cases
(though it might break if they bundle certain combinations of JARs; this
should be tested).

--
- DML
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
12