Virtual Bundles

Overview¶

The OSGi framework supports deploying and managing bundles, which are reusable units of code and resources. Although the OSGi specification does not explicitly require bundles to be packaged as JAR files, it does require that bundles look like JAR files (i.e., they have a manifest and named byte entries). For the most part, this abstraction as worked well and has allowed framework implementations to support other archive formats and even install-by-reference semantics (to some degree). However, there are limitations limitations as to what can be achieved by this approach.

The “JAR abstraction” employed by the OSGi specification for bundles requires that the framework is always in control of all aspects of class loading (e.g., searching for bytes, defining classes from bytes, etc.). While this makes sense since the framework must enforce consistency, it limits the framework to scenarios where a bundle can be modeled as a JAR file. Overall, this limitation might not seem so onerous, but the result is that every new requirement that comes along at the bundle level results in modifications (and bloating) of the core framework specification.

To avoid bloat and added conceptual complexity, this proposal introduces the primitive concept of a virtual bundle to the OSGi framework. A virtual bundle can quite succinctly be described as a bundle whose content is not managed by the framework. More specifically, the idea is to weaken the “JAR abstraction” where the framework expects to have access to byte entries in an archive and instead allow another entity to manage entry access and more importantly bundle class loading. The ultimate goal is to reduce the need to modify the framework to support niche requirements by enabling their support in upper layers.

Use cases¶

Some potential use cases for virtual bundles:

• Composite bundles – virtual bundles could wrap a composite bundle concept (i.e., a bundle composed of other bundles).
• Byte code weaving – virtual bundles could enable byte code weaving by allowing an external entity to manage the class loading for on-the-fly weaving.
• Module system integration – foreign module systems could wrap their modules as virtual bundles for OSGi interoperability.
• Marshaling – temporary marshaling bundles could be modeled as virtual bundles, providing better control over class loader monitoring for garbage collection purposes.
• Install by reference – an install-by-reference mechanism could be modeled on top of the framework as a virtual bundle.
• Exotic use cases – for example, supporting bundles whose class path refers to external content.

This is not a complete list of use cases, but provides some potential examples.

Terminology¶

The following terms are used in this document:

• bundle – a normal, framework-managed bundle.
• virtual bundle – a bundle with content managed by a third party.
• third party – typically this will likely refer to another bundle, but not necessarily since a virtual bundle could be installed externally using the standard launching and embedding API.
• virtual module – third-party managed bundle content (naming explained further below).
• content – in the context of normal bundles, this generally is referring to a bundle's bytes, but for a virtual module it refers to bytes and classes.
• manager – the third-party responsible for providing access to virtual module content; mostly likely managers will be an installed bundle following some form of extender-like pattern.
• management object – the object injected by the manager into the framework to manage the virtual bundle's content (this is an implementation of VirtualModule as will be described shortly).
• manager-owned class loader – this term is used to indicate that the associated class loader comes from a third party.

Technical approach¶

To support a virtual bundle concept, we need to introduce a few new interfaces to manage access to the virtual bundle's content and to provide it access to its own wiring state. This section discusses these interfaces as well as other technical issues.

Proposed API¶

This proposal defines the following new interfaces and/or augmentations to existing interfaces to support adding a virtual bundle concept to the framework.

NOTE 1: The package name used for this prototype is org.apache.felix.framework.ext to avoid an IP issues regarding the development of the prototype in the open at Apache Felix. If we want to change it to the org.osgi.framework namespace, we need some way of making timely updates of the proposed packages available to the public.

NOTE 2: All names in this document (e.g., interfaces, methods, etc.) are subject to change and were merely chosen for the purposes of making progress on the prototype.

VirtualModule¶

The VirtualModule interface abstracts access to the virtual bundle's content and is the management object handling content access. The name might seem confusing, but results from how framework implementations must handle bundles. Normally in the OSGi framework, a bundle is not necessarily associated with a single bundle archive; instead, it may have multiple archives associated with it at run time depending on whether it has been updated or not. In the Felix framework implementation, we call these things modules and this naming was chosen for that reason:

    public interface VirtualModule {
        void resolve(Wire bootWire, List<Wire> wires) throws BundleException;

        Class loadClass(String name) throws ClassNotFoundException;
        URL getResource(String name);
        Enumeration getResources(String name);

        URL getEntry(String entry);
        Enumeration<String> getEntryPaths(String path);
        Enumeration<URL> findEntries(String path, String filePattern, boolean recurse);
    }

The VirtualModule interface is implemented by a manager wishing to provide a virtual bundle. The core of the interface provides access to the content (i.e., both classes and entries). Third-party management of class access is the main differentiator between a normal bundle and a virtual bundle; this means the manager owns the class loader associated with the VirtualModule.

Most of the methods on this interface are self explanatory; however, the resolve() method is not. The resolve() method provides the virtual module its wiring context in the form of Wire objects. This allows the manager-owned class loader to implement proper OSGi class delegation. The bootWire parameter is a special wire that performs boot delegation, while the wires parameter performs normal delegation for imported packages and required bundles. The resolve() method is technically a setter method and is called by the framework when resolving the virtual module to inject its wires; however, the method may throw an exception if it cannot resolve at that time.

NOTE 1: Technically, we could merge bootWire into wires or we could eliminate wires and just have a single delegateWire that wraps the actual wires.

For clarity, the class- and resource-related methods take into account wiring delegation; the entry-related methods do not. This is similar for normal bundles.

NOTE 2: We could potentially use a dispose() method that is called when the framework is really done with the virtual module (i.e., when it is refreshed).

Wire¶

The Wire interface provides a delegation mechanism to the manager of virtual module class loaders:

    public interface Wire {
        Class loadClass(String name) throws ClassNotFoundException;
        URL getResource(String name) throws ResourceNotFoundException;
        Enumeration getResources(String name) throws ResourceNotFoundException;
    }

The methods are reasonable self explanatory, since they perform the actions normally associated with the methods of the same name on a class loader. However, their behavior is defined to help managers support proper OSGi class and resource delegation. The result of each method and its meaning are:

• If the method returns a result, then this result should be returned by the manager-owned class loader (with the possible exception of getResources()) and delegation should stop.
• If the method returns null, then the manager-owned class loader should continue its search.
• If the method throws an exception, then the manager-owned class loader should stop its search and throw an exception.

Injection of wires into a virtual module does not compel the manager-owned class loader to obey proper OSGi delegation patterns. It is recommended to do so to ensure consistency, but the third-party provider has the flexibility to deviate as it sees fit, but it must live with the consequences.

FelixBundleContext¶

The framework needs to provide explicit support for installing virtual bundles and currently this happens via two new methods on the BundleContext interface. For the prototype, these methods are added to a specialization of BundleContext:

    public interface FelixBundleContext extends BundleContext {
        VirtualModuleContext installBundle(String location, Map headers, VirtualModule vm)
            throws BundleException;
        VirtualModuleContext reinstallBundle(Bundle bundle, VirtualModule vm)
            throws BundleException;
    }

The installBundle() method is how a manager installs a virtual bundle for the first time. The location parameter is the normal bundle location string, the headers parameter is the virtual bundle's manifest, and the vm parameter is the virtual bundle's VirtualModule implementation. The reinstallBundle() method is used by a manager to reinstall or reattach a VirtualModule implementation to a previously installed virtual bundle, such as on framework restart.

NOTE 1: Technically, it would be possible to avoid passing in the VirtualModule instance to installBundle() and force the manager to always attach VirtualModule implementations using reinstallBundle(), but this approach at least makes the first install atomic.

NOTE 2: Perhaps reinstallBundle() should be on the Bundle interface.

VirtualModuleContext¶

When a manager installs or reinstalls a virtual bundle, it receives a VirtualModuleContext:

    public interface VirtualModuleContext {
        Bundle getBundle();
        File getDataFile();
    }

The sole purpose of a VirtualModuleContext is to provide the manager with access to the virtual bundle's private data area. The VirtualModuleContext is valid even when the virtual bundle is not ACTIVE, but becomes invalid once the virtual bundle is UNINSTALLED.

NOTE: This could be implemented as a super interface of BundleContext.

Virtual bundle lifecycle¶

In an effort to minimize the impact to the framework, the lifecycle handling for virtual bundles has been kept purposely simplistic. There was a conscious decision to avoid making the framework responsible for reifying the relationship between a virtual bundle and its manager; instead, this is solely the manager's responsibility. This does have some have some potential ramifications on issues like ordering, which will be discussed shortly along with other lifecycle-related issues.

Persistence of virtual bundles¶

When a virtual bundle is installed, it is installed persistently; however, this has a different meaning than for normal bundles. A virtual bundle is recorded persistently in the bundle cache and its specified headers are cached for it; this means the headers cannot change after installation unless updated, like a normal bundle. The managed object (i.e., the VirtualModule instance) associated with a virtual bundle is not persisted. This means on subsequent framework restarts, the framework is able to reconstitute a virtual bundle and maintains its private data area, but the reconstituted virtual bundle is merely an empty shell. It is the managers responsibility to reinstall the virtual bundle's associated VirtualModule.

Manager/virtual bundle ordering¶

In many cases it will be important for the manager to start before anyone attempts to use a virtual bundle. If so, the manager should be placed in a lower start level than its virtual bundles. Although not optimal, this is acceptable since virtual bundles are quite low level and are effectively extending the framework. This may not be necessary in all cases and could potentially be alleviated to some degree if the framework were proactive during the reinstall phase of a virtual bundle (e.g., it could immediately try to restart persistently started bundles after a reinstall).

NOTE: This is also related to the “active dependencies” topic of RFC-154; if the framework managed some active dependencies then this could be resolved that way, but that opens another whole can of worms.

Refreshing a virtual bundle¶

When a normal bundle is refreshed, the framework throws away the class loader associated with the bundle and will ultimately create a new one when needed. For virtual bundles, the first part is the similar, but the second is not since the framework has no way to create the needed VirtualModule instance. Thus, when a virtual bundle is refreshed, the framework throws away the associated VirtualModule instance and sets the associated state of the virtual bundle to INSTALLED. It is the manager's responsibility to detect this situation and reinstall the needed VirtualModule instance.

NOTE: Technically, I think it may be possible to achieve this somewhat atomically with a synchronous bundle listener.

Refreshing a virtual bundle does not necessarily have a direct impact on the manager. In other words, the virtual bundle does not necessarily have an implicit dependency on its manager.

Refreshing a manager¶

The framework must maintain dependencies from a manager to its installed virtual bundles so when a manager is refreshed, then all of its virtual bundles will be refreshed too. If the class implementing the VirtualModule instance comes from a bundle other than the manager, then the framework should associate an implicit dependency between this other bundle and the virtual module too so it is refreshed when this other bundle is refreshed, in addition to the manager.

NOTE: It is not necessarily clear that we need to directly support this last case.

Effective time of a virtual module¶

The effective time of a virtual module instance is related to the lifecycle of the virtual bundle itself and the virtual bundle's manager. It seems obvious that a virtual module instance should be valid (i.e., used by the framework) while the virtual bundle state is RESOLVED, STARTING, ACTIVE, STOPPING, and UNINSTALLED (until refreshed); this mimics normal bundle behavior. With respect to the manager's lifecycle, the prototype currently assumes the virtual module is valid during these same lifecycle states in the manager. In other words, the manager does not need to be active after the fact for the virtual bundles to continue to function, it just needs to be active to install them initially.

NOTE: The alternative is to treat this as some sort of “active dependency” where if the manager is stopped, its associated virtual bundles are refreshed immediately.

Open issues¶

This section documents open and/or unaddressed issues.

Installation interception¶

Some form of bundle installation interception is necessary to integrate cleanly with existing management agents that use BundleContext.installBundle() to deploy bundles. One possibility is to introduce a new service interface used by the framework, such as:

    public interface InstallHook {
        boolean installBundle(String location);
        boolean installBundle(String location, InputStream is);
    }

Managers could register such a service which would be used by the framework during bundle installation to call out to the managers to given them an opportunity to process the bundle installation instead of using the default handling. This is somewhat analogous to resource processors in the Deployment Admin specification.

Updating a virtual bundle¶

No issues are foreseen in the normal update scenario (i.e., updating a bundle to a completely new version of a bundle whether it is virtual or not). It should be possible to update a virtual bundle to a new virtual module (and headers), as well as updating a normal bundle to a virtual bundle or vice versa. This will likely require adding another update() method to Bundle that accepts the appropriate parameters (e.g., Bundle.update(Map headers, VirtualModule vm)).

A more complicated case is related to ordering, which is how to deal with bundles that were installed before the manager was present and/or activated. In this case, a normal update is not completely sufficient since the manager really wants to update the bundle to a virtual bundle, but keep its existing content. Technically, this is possible with the current API by using the entry-related Bundle methods to reconstruct the installed bundle, then performing an update on it to convert it to a virtual bundle.

Resource handling¶

Typically, a framework implementation has to know something about the content of a bundle to create resource URLs. For example, both Felix and Equinox create resource URLs something like this:

    bundle://<framework-id><bundle-id>:<classpath-idx>/path/to/resource.txt

This sort of approach is necessary since the specification requires that resource URLs can be used as the context to create other resource URLs. Unfortunately, this breaks the module's encapsulation of its content (i.e., the framework must be aware that there is a bundle class path concept).

Currently, a manager must manager register a URL stream handler to provide a protocol to access its virtual modules' content as resources if it cannot be handled via an existing protocol. The downside of this approach is that, for now, a manager has to be active to provide a stream handler service, which means resource access will stop working if the manager is stopped.

A potential solution to this is to inject the virtual module with a resource URL factory which allows the manager to inject its own “opaque” index integer into the framework's normal resource URL.

Dynamic imports¶

It is possible to add support for dynamic imports through the injection of a special type of wire in the VirtualModule.resolve() method. Like the boot wire, this dynamic wire would be special and would be searched by the manager-owned class loader after its own content and would potentially result in a dynamic import.

Fragments¶

It may technically be possible to support fragments. Currently, a virtual module is injected with wires that provide access to classes and resources. Conceptually, we could handle fragments by injecting the virtual module with a set of fragment bundles from which it could load entries. The only trick is that the injected bundles could not be a normal bundle since a normal bundle can have multiple bundle revisions associated with it; the injected fragment bundle would need to be a wrapper around a specific revision. Other approach it to create a new wire-like interface for fragment access which could be injected into the virtual module.

Security¶

One possible approach to deal with security is to inject the virtual module with a protection domain for it to use when defining classes. For virtual modules using predefined classes, then it won't be possible to assign additional permissions to those classes.

Lazy activation¶

Too support lazy activation across normal bundles and virtual bundles, API would need to be defined for them to participate in this process. Mostly likely, the virtual module would need to be injected with some object for keep track of which classes are being created and which bundles need to be activated.

Considered alternatives¶

TBD