___ 1. on all cache accessors lazy load cache if necessary. this way we don't have to call init code in tests. OK_ 2. determine why in cachetest if we ask memtable to clean up and there are two allocations, mem.first is OK and then mem.next wraps. DONE was my bug. OK_ 3. jenkins_clear uses free when asked why were keys and values not etch_malloc'ed? maybe this is OK. DONE we solved this issue by using installable callback on free. OK_ 4. write a true iterator for hashtable interface and all collections. this will be a struct object that when instantiated, calls first() and stores the current position as instance data. the iterator has three methods, next(), is_more(?), and destroy(). Each time next() is called it returns the current position, and calls next internally, storing the new current pos. it should probably permit a startat also. current position instance data will probably be specific to the collection and therefore would be a heap allocated member of the iterator object. To do this entirely generally all collections would want to be objectized with vtables similarly to the way instance data is done, and implement at least first() and next(). Add a destroy() handler as well, which might want to have freekey and freeval arguments. DONE OK_ 4.1 install iterator in struct value DONE OK_ 4.2 install iterator in array value DONE OK 5. cache the etch_object vtable in a global variable, and reference this in new_etchobject, in order that we don't have to do a cache lookup on every object construction. DONE. OK_ 6. write tests for boxed objects ___ 7. configure etch_free to not necessarily complain if allocation untracked. not sure how atm, we definitely can't have another parameter. ___ 8. deal with the discontinuity between memory tracking table and the vtable cache. when we clear the memtable after each test, we compromise the final cleanup of the cache, which now has dangling pointers into the memtable. we deal with this currently by not etch_malloc'ing vtables. maybe this is OK, but there is something less than satisying about this arrangement. ___ 9. make shallow copy non-virtual in etchobject. OK_ 10. write ctor with dedicated vtable for boxed arraylist type, that permits destruction of the boxed object with or without freeing memory for the list contents. since we can't pass a "don't free content" parameter with the etchobjects's destroy(), we'll need to be able to set a flag somewhere in the object itself to so indicate, prior to calling destroy(). One way we can do this is by adding a parameter to new_etch_arraylist, which will set the is_readonly flag on the underlying etch_arraylist, causing arraylist_destroy() to ignore its is_free_content parameter. PARTLY DONE 11/1. we added the ctor flag to prevent memory cleanup attempt. we did not yet add a custom ctor, so clone and destroy cleanup is still invoked from the default clone and destroy. OK_ 11. add an optional content destroy handler callback to collections. this would be made simpler given item (4), objectizing collections with first() and next(), since we would now have a vtable to which we would only need to add another virtual method. PARTLY DONE installable callback on hashtables, still need to do arraylist. DONE. OK_ 12. add a content type property to collections differentiating between content for which memory can be freed as a unit, and content consisting of multiple heap allocations in particular etchobject. DONE. OK_ 13. etchobject.size is not accurate for objects which have dynamic underlying content. WON'T FIX -- the way it works is the way it must be, in order for anonymous copying of objects to work the size must be the memory footprint of the object itself. OK_ 14. change new_xxxxx ctors which construct aliased etchobject, to syntax new_boxed_xxxxx(). this is not pretty but it is descriptive of what is happening, which the current ctor names are not. DONE. ___ 15. add __LINE, __FILE, to exception object ctor ___ 16. consider moving impl* and impl_type to etchobject, meaning we could always call destroy_instancedata on the impl, and would no longer need to subclass objects simply because they have instance data. ___ 17. define default exceptions -- STARTED OK? 18. create startup hashtable of exceptions, augmentable at runtime. this is likely NOT AN ISSUE, exceptions likely do not have to be looked up, but rather are static. OK? 19. determine how we are going to hash using complex objects as keys. we need to add hashtable functionality to supply hashkey in advance. we could make this much easier but somewhat less efficient by doing a lookup() first to get the key, storing the key, and then doing the insert. the downside is that we would compute the hashkey twice. PARTLY DONE. the only objects as keys are currently id_name and related. we hash these by declaring object length 4, since the hash key is in first 4 bytes. we'll put off the rest of this until we need to hash other object keys, if indeed we do. N/A 20. consider do-over as raw com. DECIDED NO. OK_ 21. allow destroy hashtable to etch_free content configurably. currently it free's meaning we can't etch_malloc content. possibly a system hashtable should free, others should etch_free. see also 21.1. DONE by installable callback. ___ 21.1 change memtable to a system hashtable. lose is_memtable_instance. this means changing memtable code to use direct jenk calls ___ 22. include ref problems with id_name and typedef'ed derivatives. problem arises from putting the wrapped object defs in with the native object defs, meaning we have to include all the object headers and code, along with hash, etc. Move this stuff out and into another file, but do this only after a clean check-in, so we can easily back out the code if need be. OK_ 23. generalize structval catch/throw code so we can use it on all objects DONE ___ 24. add code file and code line number to exceptions. need throw macros to do so. OK_ 25. unfubar code module file name caching - possibly use existing global cache hashed by path name. DONE used global cache, adding code for text keys, and for key as value. OK_ 26. add serial number to allocations. DONE OK_ 27. standardize cloning of keys - currently sometimes caller clones, sometimes not, copy should be made at as low a level as possible to facilitate clean code. WON'T FIX ___ 28. change short_type calls to constant enums ___ 29. enumerate class IDs, change assignments accordingly ___ 30. do an impl_mask inherited from obj_mask. why? OK_ 31. add synchronization to arraylist DONE OK_ 32. add synchronization to hashtable DONE OK_ 33. unit test for arraylist synchronization DONE ___ 34. unit test for hashtable synchronization OK_ 35. fix bug in arraylist_remove DONE see test_threadpool::test_synched_arraylist OK_ 36. implement a global last_result in order to communicate exception from methods with void returns. is_exception would also query last_result. OK_ 37. add code to reference count objects. DONE but see (50) below. ___ 38. unit test for reference counting of objects. also see (50) below. ___ 39. replace non-portable unicode conversion with portable. this is a significant task, possibly best to ifdef the operating system and use OS facilities. OK_ 40. previously thought that hash value was dependent on encoding. however scott informs that identifiers always ascii. they are unicode in current code. need to change these accordingly. so change id_name.name and similar. ALTERNATIVE: continue to maintain an 16-bit version of the name, but add an 8-bit version as well, or convert on the fly. DONE. implementation temporarily converts to 8-bit before computing id value. OK_ 41. change flexbuffer get methods to return a result code and accept a pointer to a returned value. since we can't throw exception, and any returned value can be valid, we need a separate result for these methods. also change all spots in the code which call these gets. DONE ___ 42. convert dedicated global cache code to use generic etchmap code. only difference is passing the hashmap and possibly alpha key buffer to methods. eliminates the one-off code but fixes nothing otherwise. ___ 43. change nativearray to permit more dimensions than 3. possibly add a fourth dimension to simple syntax. add additional nativearray APIs to permit n dimensions, by passing arrays as parameters. OK_ 44. code another method similar to etchobj_assign_to(0) which returns the object assigned to, or make etchobj_assign_to() return the address. we may need this if (a) we use the inheritance model where the child points at its instantiated parent object, and (b) we do child = parent assignments on those objects. DONE ___ 45. Populate etch_arraylist's and etch_hashtable's etch_collection_mask items content_obj_type and content_class_id. ___ 46. Mark all objects in collections as static, thus destroy() has no effect. WTF: not sure why we want to do this. Perhaps this means support the ability to do so. Of course we already can mark the collection as having static content; however if we assign one collection to another we can't do this since both shells would be marked static content. So this todo is apparently to address this scenario. ___ 47. Change etch_arraylist to permit etch_nativearray as backing store. Since natarray get() always returns a newly instantiated object, and arraylist get() always returns a non-disposable, we would need to resolve the inconsistencies of access in some manner. ___ 48. string object needs a static method to ensure one encoding or the other, as opposed to conversion methods which we have. this would return the existing buffer if already in the desired encoding, otherwise convert it. For example void* etchstring_getvalue(etch_string* s, const int encoding); or wchar_t* etchstring_get_wvalue(), char* etchstring_get_nvalue(); ___ 50. add methods to etch_syncobj.h, .c, to atomically operate on an objmask* refcount. change all destructors accordingly. we would probably use the APR atomic method to do this, so no need for explicit lock in our code. note that we need a global method to sync on increment refcount also, which uses the same lock as decrement. ___ 51. add logic to create a vtable without caching it, and use this logic in the construction of objects that will only be instantiated once. OK_ 52. do not serialize a string's null terminator, and add a null terminator when deserializing a string. OK_ 53. convert to wire encoding from unicode when serializing strings, and convert from wire encoding to unicode when deserializing strings. ___ 54. make etch c date object work the same as java, i.e. a date should serialize and deserialize to the same values as the java version. ___ 55. implement an internal arrayvalue representation which does not require individual items to be objectized. this specifically addresses the byte array, where we can't continue to render each byte as an object. ___ 56. implement custom validators. ___ 57. plug in custom validator to examples value factories, e.g. xmpl_perf_valufact, parameters for dist(). ___ 58. override set_session on all classes whose session interface is sessionmessage, ala etch_transport.c in which we already did so. ___ 59. make the generated value factory static type and fields more consistent with those of the default vf. if they can actually be static, that would be best. ___ 60. change early complex inheritance objects to be simpler, e.g., masked. if they need obj.parent, change them. then get rid of unneeded object fields. ___ 61. change various object parameters to constant, e.g. etch_event. OK_ 62. determine if we can link all the generated but unedited files into a static lib. there are only four editable .c files, leaving 17 loose .c and .h files on the client and 16 on the server. many of these are shared by each side, so a common lib would be useful and cleaner. ___ 63. eliminate i_deliveryservice instantiation, instead embedding i_deliveryservice as the first part of deliveryservice. i_deliveryservice then becomes a mask. ___ 64. server hangs on shutdown if a client is still open. for example, if a long sleep on the client prevents closing the connection, then server hangs and eventually crashes when it attempts to clean up client sessions.