-------------------------------------------------------------------------- T O R Q U E -------------------------------------------------------------------------- o XML as configuration o Use the digester to turn the datamodel into an object and get rid of the dependency on xerces. It would be nice to use the digester with MinML so that the distribution can be tiny o Some analysis on what we actually have would be prudent. Torque is somewhat based on the work of Scott Ambler and I would like to know where it is the same and where it diverges so we know what we're dealing with. o find all other OSS persistence layer projects and see if we can join forces. o using cursors to allow controlled access to a large set of data. typically there is no need to see a massive set of records all at once. [JSS] Not really database agnostic though. Must be implemented conditionally. [JVZ] But it has to be made to work transparently. The specifics should be in the RDB specific object. The adapter probably. o using proxies for large objects. if you had a user object that had 50 attributes it would more efficient to create a proxy object of that user that might have a few attributes, enough to identify the object. you might use the proxy objects in a list and when it is selected the real user object is transparently retrieved. [JSS] I consider this custom code and can be implemented in the base objects quite easily. o would it be possible to support native connection methods for high efficiency. an easy to access C APIs, would it even be useful? [JSS] in this case, the JDBC driver can be made native and is probably not a worry/optimization of ours because this is handled in the JDBC layer. o application programmers should be able to save, update, and delete objects without any knowledge of the database schema. if the attribute selection is based on columns in tables than the object hierarchy and database schema cannot vary independently. it is likely that the database schema may need to change but you don't want this to affect application programmers who may know little about databases. reasonably so, the persistence layer should take care of everything. [JSS] YES! Questions --------- These are really just questions I'm jotting down as I go through the source. Not really important that they be answered here, I'll post the whole slew to the list eventually. o Why is there a save(DBConnection) in the BaseObject? The objects should know nothing about databases, and if knowledge of a specific database is required it should probably be in some mapping mechanism. The BaseObject should be retrievable from any store so the BaseObject shouldn't contain information about any specific store type. XML persistence would be nice to add with one of myriad tools that exist. o Same question for the the Persistent interface. Would be nice to get rid of the DB specifics. o Why do we have so many key types. I thought the surrogate key idea was better? That the keys for objects should have no business meaning under any circumstance. This practice is strongly warned against in Ambler's papers. The object key should be a number or a string. o Why does the TableMap make reference to the IDGenerator? I realize the id method can be specifed on a table basis. Maybe the datamodel object could be used at run time as the source of information about the database setup. Than a lot of the objects that are coupled could be decoupled because all the meta info would be present in the datamodel object. o Same goes for the DatabaseMap. We definitely have information duplicated across objects. Most of this info is already present in the datamodel used to produce the OM. Could use the datamodel to decouple which is what would be need to have pluggable connection pools. o Could the information in the adapters be coupled with information in the db.props file and combined into some XML descriptor for database specifics? ---- Get rid of ${torque.home} in the standard build.xml files and define all the possible values that can be set and document them so that people can use the standard build.xml file in a known way. Things are a little loosy-goosy and fragile right now. What I do could still potentially break scarab's build because I'm not sure how things are being used ... but I'm getting there :-) Scott Eade has found that Monitor threads appear to be living on past servlet restarts. Need to add a test for that. ---- Problems with Large Schema Files In some cases, large schema files can cause Torque to fail to generate the object model. There appears to be a memory leak which affects only the generation of the object model. It does NOT present any problems at runtime, only during generation and only for especially large or complex schema files. (FIXME: measure what "large or complex" really means -- in units of tables or foreign keys or something.) An attempt has been made (unsuccessfully) to allow a schema to be split across multiple files. These notes are intended to record the problems in hopes that solutions can be found later without having to rediscover the problems. A schema can be successfully split into multiple files provided that all foreign keys defined in one file refer only to tables that are also defined in the same file. A schema could be sufficiently complex that even the split up files would be too big for Torque to generate. One solution would be to find a way for Torque to generate code using keys to tables defined outside of the schema file. As of March 8, 2002, this doesn't work. Object.vm includes some complex logic for generating methods to get related objects, and even more complex logic for methods that get related objects by joining multiple tables. In order to generate these methods, Object.vm currently goes to the corresponding Table objects to get their foreign keys. That is what constrains the generation. The ForeignKey object does not contain sufficient information for Object.vm to generate those methods without support from the Table objects. Therefore the Tables must also be defined and in memory. Hence the requirement for a single file. A couple possible areas for investigation. Perhaps the graph package in Commons could be employed to capture the foreign key structure in a more memory efficient way. Right now Torque uses Texen tasks and velocity templates to transform the xml schema into object models. Might be worth investigating DVSL to see if it can do what Texen does but with less memory (or with less complexity).