Jetspeed TODO

Last Modified: $Date$
Latest version at: http://www.working-dogs.com/jetspeed/cvsweb/index.cgi/jetspeed/docs
Author:  burton@apache.org

*************************************** TODO ***********************************
          
- remove PortletFactory from CVS
- remove xml.peer.registrymarkup classes

****** TEST SUITE TODO *******

- write a test to make sure OCS stream processing works

- make sure that jetspeed-config gets processed again when I modify file settings.

****** PSML.NEXT support *****

- DO NOT USE THE SAME INSTANCE OF THE REGISTRIES EVERYWHERE.

- Simply most of the factories... there is a *lot* of duplicate code here.  Make
  sure that we clean this up.  Especially PortletControlRegistry

- make sure that all factories *only* use either *Ref classes or *Entry classes.
  do not allow classnames, etc.

- remove PortletExport from CVS.

- org.apache.jetspeed.portal.PortletSet should go away in favor of .psml.peer.PortletSet

- write docs on the new jetspeed-config and PSML.  This should be how to use this
  as an API and also how to configure it.

- rework JetspeedConfig:

    - RegistryManager becomes a Singleton based on URL.
      all REGISTRY classes move to point to JetspeedConfig.getInstance().getRegistryManager()

    - essentially we need to make all instances of the RegistryManager tied under 
      JetspeedConfig so that we can have multiple JetspeedConfigs... one for each OCS 
      stream.

- remove LayoutFactory from .portal.factory

- convert the .xml.api.portletmarkup package to .psml

- make sure BasePortletSet.init is always called

- move around the org.apache.jetspeed.registry package.  This should become an
  org.apache.jetspeed.config.registry package.  The org.apache.jetspeed.config 
  should now have a JetspeedConfig object which handles initializing the registry.
  This way we can further extend the Jetspeed configuration mechanism without 
  having to modify the registry.

- move the org.apache.jetspeed.registry.backend package to .factory

- setup org.apache.jetspeed.psml package:
    - .factory ... all factories for PSML operation
    - move PortletSetFactory to .factory
    - setup the peer package
        - put PortletSet within this package

- remove registry.xsd from CVS... obsolete

- implement profile support
    - ProfileRegistry support (and put all object in there).
    - section 4 correctly implemented    

- implement all easy registry reference mechanisms (PortletRegistry class, etc)

- reimplement skins and make sure they work

- remove registry.xsd from CVS.
- remove portletregistry.psml from CVS

- change the PSML implementation around correctly:
    - <psml> root (use namespaces correctly)
    - portlets -> <portlet-set>
    - home dir should be /users/burton
    - xmlns set correctly
    - section 4 correctly implemented    

- (DONE)port the old PortletRegistry code over to use the new PortletRegistry

- (DONE)implement PortletControlRegistry and PortletControllerRegistry
    - have PSML use the entries within the registry
    - have the PortletControlFactory and PortletControllerFactory use the registry
      reference and *not* the classname.


****** HIGH PRIORITY ****** 

- BUG/FEATURE:  In the CocoonPortlet make sure that we put PortletConfig within the 
  ServletRequest.  This way an XSP page can get it without creating its own instance
  of it (which will end up being buggy anyway).

- ARCHITECTURE:  see if it is possible to remove all explicit package declarations
  within the API.  IE do not refer to any classes with the full name.. org.apache.jetspeed.etc.

- MAJOR BUG: Currently Jetspeed doesn't support more than 2 level on PortletEntry
  inheritence.  Since it just normalizes from ref -> ref -> abstract only the 
  first ref will get any values.  The second ref will not.  The correct way to 
  do this is from abstrace -> ref -> ref on down the chain

- BUG in FeedDaemon:
    > There remains a problem when the FeedDaemon runs twice (it should also be in
    > the cvs version). The Feeder can not know if a portlet is in the registry,
    > and both the getPortlet (if it is not) and the addPortlet (if it is) throw
    > PortletExceptions. So, the second time the feeder runs, it removes all
    > entries from the registry that are already there. The third time, it puts
    > them again, and so on...
    
- TODO: Add a test for the DiskCache.  Should test local URLs so that NT/UNIX 
  and all JDKs work.  IE file:// mapping.

- FEATURE: convert all XML parser requests to use JAXP.

- BUG: make sure that I can logon with the Hypersonic SQL configuration

- Should I create a CapabilityMap entry in PortletConfig???  This would 
  certainly increase performance and since the CapabilityMap never changes...

- Fix the Turbine infinite redirect problem.  This is still within CVS and it 
  appears to manifest itself when the client can't establish (or doesn't 
  support) session management.  This is very important to get this fixed within
  Jetspeed because the WAP functionality will break if it can't establish a new 
  session as this will infinitely redirect.  We will also loose performance here
  because a new ClientCapabilityMap will have to be created for every new client
  request

- FEATURE: Rework the Channel Browser and Channel Search so that MimeType 
  content is used correctly.

- BUG: Many content engines are NOT using the correct getLastModification() or 
  'Last-Modified' header impl.  PHP is a major offender here.  It returns the 
  last modification date of the .php file on the filesystem.  Therefore this 
  Portlet will *never* get updated.  We need to use the OCS XSLT mechanism to 
  build out a deployment expiration mechanism.

- We should use the OCS refresh mechanism to dynamically reload this info.  There
  was a PROPOSAL sent to the mailing list.  Provide an impl of this if no other
  is provided.
  
- BUG: if Jetspeed is serving a local URL that is a CGI/dynamic content, it tries
  to read this from the disk instead of HTTP.  We need a failover if the content
  isn't on disk:

    This was done for performance reasons.  It is much cheaper to fetch this URL 
    from the filesystem instead of the HTTP server.  There needs to be some 
    detection code here.  

    - if file exists
       - use local file
    - if file does not exist
       - assume URL is some type of CGI/dynamic content.
           - if url does not exist allow the webserver to report an error.

- BUG: FileServerPortlet doesn't not expire itself when running under Tomcat?

- BUG: When running in a new Servlet Context under Tomcat (reported by Neeme Praks):

> I have context for jetspeed set to "/jetspeed" and I have my XSP file in
> http://myhost/jetspeed/myfile.xml, the physical path is
> root-to-tomcat\webapps\jetspeed\myfile.xml (Win2K system). When
> resolving the URL to physical, JetspeedDiskCache returns
> file://root-to-tomcat/webapps/jetspeed/jetspeed/myfile.xml.
> As much as I understand, EngineContext.getInstance().getDocumentRoot()
> returns root-to-tomcat\webapps\jetspeed and then the
> /jetspeed/myfile.xml from URL is appended to this, resulting in one
> extra "jetspeed". Haven't figured a way out yet how to fix this...
  
    - Add support to that Jetspeed can resolve its own document root mapping.
  
****** NORMAL PRIORITY ****** 

- FEATURE:  In MetaData (meta-data) add a "default=boolean" attribute.  This
  is necessary because sometimes you want to use metadata to provide a 
  default title/description but still let a portlet define its own.  OCS 
  should use the dublic core metatdata as defaults but still let portlets
  provide their own... if they don't then we can fall back on the defaults.
  
  - We should apply the following patch to ocs.xsl:
  
    Index: src/content/xsl/ocs.xsl
    ==================================================================
    RCS file: /products/cvs/jetspeed/jetspeed/src/content/xsl/ocs.xsl,v
    retrieving revision 1.6
    diff -r1.6 ocs.xsl
    28a29,30
    >         <xsl:variable name="tit" select="./dc:title"/>        
    >         <xsl:variable name="desc" select="./dc:description"/>
    38a41,44
    >             <metainfo>
    >               <title><xsl:value-of select="$tit"/></title>
    >               <description><xsl:value-of select="$desc"/></description>
    >             </metainfo>
    46a53,56
    >             <metainfo>
    >               <title><xsl:value-of select="$tit"/></title>
    >               <description><xsl:value-of select="$desc"/></description>
    >             </metainfo>
    

- BUG:  If the PortletRegistry receives another Application then the 
  Applications Portlet isn't smart enough to realize that its content has
  changed.  It needs to expire itself when another application has been added 
  to PSML

- Improve Portlet subcription.  You should be able to pick from a couple of
  generic portlets.  Then you should also be able to search from additional
  portlets using keywords (from OCS) or domain name.

    - Portlets should have the the ability to be added to a users profile so 
      they when they logon they get their own profile and their own portlets.

- Under scripts/sql create an external/turbine and external/jyve directories.  
  Jyve and Turbine should go under both.

- FEATURE: Move ClientCapabilityMap string matches into the 
  JetspeedResources.properties file as regexp

- BUG: Make the Portlet CapabilityMap have inheritance. This is important so 
  that we can have an abstract RSS-WML Portlet

    - Take this out into the PortletRegistry... replace all the Mozilla-WML 
      entries, etc with this.
        
- BUG:  Make JCM more WAP/XSL/Cocoon friendly by not including the stylesheet
  within JetspeedResources.properties.  Instead, put this within the PSML 
  registry
        
- For 1.2b2 move this over to xml.apache.org

- FEATURE:  Put portlet editing and maximization in PSML.
  
- FEATURE:  Do documentation on JCM for 1.2
 
- FEATURE:  Build in a daemon to syndicate JCM via RSS and OCS

******* REQUIRED DOCUMENTATION FOR 1.2 ********

- Better documentation on the control panel and daemons is required for 1.2

- Document how the PortletRegistry works.

- Document properties files better

- Document the Daemon package

- Document the Admin console

- Document how the updated FeedDaemon works

- Document how user customization works

- ... on JCM, what it is and how to use it.
    - describe what the bookmarklet does
    - tell them that it stores the password in clear text

- Documentation on how to setup user authentication

- Docs and a dist of Turbine for setting up SQL


********************************** 1.2b2 ***************************************

- (DONE) ARCHITECTURE:  try to move all castor generated code references into one single
  peer package.  The peer-factory package should be a factory and used to obtain
  references to objects in the registry.peer package

- (DONE) in the PortletRegistry... addEntry and removeEntry should throw 
  PortletExceptions

- (DONE) FEATURE/BUG: Jetspeed internals (PortletRegistry.getEntry()) was 
  throwing a RuntimeException if default.psml was specifying an entry that was
  never registered.  Rework the PortletRegistry so that it throws a *real* 
  Exception

- (DONE) REWORK: rework the way Portlets are viewed:  
    - (DONE) Remove the PortletViewerScreen
    - (DONE) The Home screen should except:
        /portlet
        /portlet-controller
        
        as path info.

    - (DONE) PortletControllerFactory should have a new method:
        getPortletController( MimeType mimeType )... this will return the 
        default PortletController for the specified mimetype.

- (DONE) (BUG): The URLFetcherDownloader wasn't adding content to the DiskCache

- (DONE) in Home.java rework WAP/HTML support so that:
  - (DONE) if non-HTML MimeTypes are used
        - (DONE) at the begining: turn off Layout (IE: use EmptyLayout)
        - (DONE) at the end: instead of returning an element, output directly to the
          Servlet output stream

- (DONE) build out a MOTD for Jetspeed so that after users start it they know exactly
  how to run it.

- (DONE) Fix the 'tomcat-build' so that we don't have to go to /servlet/jetspeed but 
  just /jetspeed

- (DONE) remove need to build classpath by putting files in WEB-INF/lib

- (DONE) BUG: Make sure that portlet property inheritance works.

- (DONE) Moreover links seem to fail within Jetspeed.  The reason seems to be that if can't
  figure out that the URL is really in the cache.

- (DONE) FEATURE:  Fix my Ant build script for the Jetspeed/Tomcat distribution.  There
  should be a target for 'jetspeed-tomcat-dist'

- (DONE) Provide a CapabilityMap for Portlets
    - (DONE) Provide a <capability-map> in PSML.

- (DONE) BUG: remove enable.userauthentication.  If this is set to false it causes
  the User object to be null which then throws NPEs during Jetspeeds operation
  Since a database is now a required component within Jetspeed this shouldn't
  be a problem.

- (DONE) FEATURE:  Put all PROPOSALS into CVS as /docs/proposals with STATUS, 
  AUTHOR, etc as a header

- (DONE) FEATURE:  upgrade to Xerces 1.1.2 and Xalan 1.1
    - (DONE) BUG: Make absolute SURE that the DTD for my.netscape.com is NOT getting 
      fetched by Xerces.  Try this with the latest version of Xerces and Xalan.

- (DONE) FEATURE: ECS performance improvement... is it a waste to continually call output() 
  within ECS.  Should I buffer this up?  Should ECS internally handle buffering 
  its output with a String?  If I have AbstractPortlet have a String named 'content'
  and a boolean flag named 'dirty' I can have setContent() dynamically generate the 
  correct output. 

- (DONE) BUG: Have the public.cache.url resolve from a Jetspeed URL resolver (ServerContext?)
  and remove public.cache.directory... just one more thing to break

- (DONE) Put the XML.com jetspeed article within sitebook
  
********************************** 1.3 *****************************************
- Support the refreshRate in OCS.  This should map to the expire mechanism in
  Jetspeed.  We should put this in PSML and we can get the data via XSLT

- implement support to dynamically reload: 

  TurbineResources.properties 
  JetspeedResources.properties

- When using the DiskCache to fetch a URL... we have been serving it from the  
  local filesystem if this is on LOCALHOST.  This is done so that the URL isn't
  slowed down by being placed in the cache.  However it doesn't work for dynamic
  content.. IE .jsp, .asp, etc.  Implement this:

    We need a regular expression entry within JetspeedResources.properties

    http.server.dynamic.content=*.jsp
    http.server.dynamic.content=*.cfm
    http.server.dynamic.content=*/servlet/*

    This way we can say that if these URLs are dynamic not to read them from the filesystem. 

- In PSML rename <portlets> to <portlet-set> 

- Make all methods that aren't inherited either static or final.  This should 
  improve performance on some JVMs... This way the JIT can inline the code
  because it knows that it doesn't have to worry about inheritance.

- Add the ability for a URL to bypass the cache.  Maybe:
  <url cache="false">http://www.cnn.com</url>

- Add the ability to set a mime type for a url.  Maybe:

  <url mime-type="image/gif">http://localhost/test.gif</url>

  This would work well for a FileServerPortlet so that it can have an <img> tag
  
  For WAP this would work for text/wap or something

- Support a configuration interface for Portlets.  Provide a mechanism so that
  you can edit a Portlets init parameters.  Use PortletConfig.getInitParameterNames()
  to figure out what to allow for editing.

- Rework the way navigations are used with Turbine/Jetspeed.  Make it possible
  to kill these off via specifying them in the URI.

- Write a portlet that allows the USER to input a remote URL into Jetspeed.
  This URL should have some tags stripped.  <html> <body> <script>
 
- Serve PSML and user XML files from /content/users/[USERNAME]

- CocoonPortlet.  Use the stylesheet URL in the XSP document but if the Portlet
  defines a "stylesheet" property this should override the XSP stylesheet.  
  The CocoonPortlet should be enhanced to support multiple content types though
  in most situations (99%) the default stylesheets will be used.

- Move all Util.globalStringReplace() functionality to use StringUtil.replace

- When the user goes to the PortletViewerScreen put some standard Portlets 
  around the content.  For example also put the Applications Portlet, Polling 
  portlet, etc

- The ability to publish HTML documents on a specific URL as content within 
  Jetspeed and through the Admin interface.  Mark these in some way so that they
  become "editorial" pieces.  This has the benefit that these pieces can be 
  discussed in the Forum code.  These should appear under a "Features" portlet.
  
- Write an "uptime" portlet so that it shows how long the Virtual Machine has 
  been running.
  
- Write a "bookmark/favorites" Portlet.
  
- Add polling/voting support as a Portlet

- Add support for submiting an Article to the site maintainers (these should be
  the standard JCM maintainers)

- The RSSPortlet should only expire itself when its on-disk content actually CHANGES
  right not it is just auto-expiring after a fixed interval. Consider putting 
  this code into the AbstractPortlet as the default behavior.

- In the PortletRegistry add/remove/get methods should throw a RegistryException
  to tell the user if the operation worked.  This should fix a bug with the 
  MemoryStore expiration code that causes a RuntimeException

- JCM: Send an e-mail off to users when content is published.

- JCM: add a maximum topic count so that the end of the list get's chopped of 
  and web pages don't keep going on forever.

- Put the remote image disk buffering code back in.  This was removed due to a 
  bug and the fact that we migrated over to XSP.  This should be a trivial tweak

- On the JCM Admin page add a section for posting this URL to XMLTree.

- Add some statistics to the top navigation:
    - current number of portlets
    - current number of users

- Fix the bug with the threadpool.  I should be able to create as many threads
  as I want and there should be *NO* stall points.  Also, all VMs should not
  crash/stall/etc.

- JCM: improve content administration:  
    - ability to add/remove/modify topics
    - ability to add/remove/modity items
    
- XML: - pretty print all XML when you save it do disk
       - make backup when you save XML to disk

- in JCM automatically add the additional fields (pubDate,etc

- Have an attribute on "entry" named "hidden" and if true this Portlet/Entry
  is never browsable.

- (DONE) rework the PortletBrowsing support:
    - (DONE) should allow you to browse by title and description (remove URL).  When
      (DONE) you click on the title it should allow you to "preview" this item.  

    - Should allow back and forward navigation and handle the begin and end points
      OK.
    - Should handle a situation where there are no portlets and give the user
      a decent message

- Meerkat (http://www.oreillynet.com/meerkat) style content functionality.

- Ability to dock Portlets into the Mozilla sidebar.

- Portlet content delivery via e-mail.

- Don't call init() within Home.  Instead make the DaemonFactory a Turbine Service
  that should be initialized.  
    - we will need to use the latest Turbine from CVS for this feature.
    - this is also necessary because jetspeed will totally screw up if requests
      come in while it is trying to be initialized.  Try to make this 
      synchronized so that multiple things don't happen at once.

- Improve the DiskCacheDaemon.  Don't constantly pull down resources.  Instead
  only download resources that have changed.  This should improve performance
  and reduce network activity.
    - Use the OCS meta information to put in the minimum download interval and 
      don't download a new version even if this has changed.

- JCM:  Support a <from> element on an item which should be the server name
  where this URL was published (EX: java.apache.org).  This way we can have an 
  item come out with:
  
             blah blah blah.  From: java.apache.org
             
             - and in this situation java.apache.org would be an HREF

- JCM:  Under the /content/channel/item/description element add some formating 
 options:
    <b>
    <i>
    <p>
    <br>
    
- Create a "Logon" Portlet that should replace the current Logon screen.  The 
  link on the navigation screen to logon should point to the new Portlet and 
  also put this new portlet into the default PSML under Applications

- The DiskCache and RSSPortlet are not correctly putting the Jetspeed images from
  RSS into the public cache.  The image isn't comming up in the web browser but
  is coming from a remote HTTP server.  When the RSSPortlet is parsing the DOM
  make sure it get's the URL from the remote server.

- Manage PSML better.  
    - Do xerces schema validation when parsing the contents.
    - Cache the documents after they are parsed out so that there is no 
      performance hit when someone wants their preferences
    - If documents are modified on disk outside of Jetspeed reparse these out.

- Build in a way to keep rendered Portlets persistent so that they survive 
  server restarts.  The PortletFactory should check a separate disk cache after a 
  MemoryStore cache fetch fails.  The content should be parsed out and stuck 
  back into the Portlet with a ClearElement.  Portlets should be able to 
  override this so that if they don't want the data the can veto this.
    - This is a very important feature.  The FeedDaemon should instantiate all
      portlets so that their HTML can be saved to disk.  This way the can 
      be swapped into memory very easy.  The FeedDameon will then instantiate 
      all Portlets and assume they are in memory... some may be swapped out but
      the PortletFactory can swap them back in if necessary.
    - How is this integrated into Cocoon/Cocoon 2.0?

    - See if this actually provides a speed improvment.  Meta info will have to 
      be stored on this disk and this basically means we have to use Castor which
      would negate any performance improvement we would achieve.

- integrate the Javacorporate Forum code.
    - Discussion portlet which obtains it's information based on another portlet.  

- Track the last time a daemon has run and also the total time it took the 
  daemon to process.

- Provide a Portlet expiration mechanism in XML.  This should markup when a 
  portlet should be expired and the PortletFactory should override the Portlets
  schedule with the content in the XML.  This should be placed in the queue when
  it expires for redownloading.

- Add support for a Portlet "state"

    Include the state in the cache engine so that different cache states are 
    correctly displayed.
    
    this.getPortletConfig().getState()
    
    - STATE_NORMAL
    - STATE_MINIMIZED
    - STATE_MAXIMIZED
    
    This is necessary so that portlets can determine how much information 
    to display when the user changes their state.  For example a maximized
    portlet might display extra graphics, etc.

- Also consider having meta-info for the width and height of the portlet 
  (narrow, wide, tall, short, etc)

- Redo the stylesheet so that everything has an entry (portlet-background, etc)
  Then with a tool the user could generate their own stylesheet.

- Ability to set the refresh rate within portlet markup.  This would override 
  the refresh rate on portlets that specify their own refresh rate.

- Improve RSS support.  Only use 1 stylesheet this should be *very* smart and 
  work across RSS 0.90, 0.91, and 0.92.  
    - it is important here that we still use the RSS parser to pull the images
    locally but use an XSLT for the UI
  
- require all portlets to use XHTML.  Then we can process the <form> data
  and add namespaces within the Factory and pull them out when we call
  service() on a Portlet