These notes describe the difference between Derby release 10.5.1.1 and the preceding release 10.4.2.0.
Derby is a pure Java relational database engine using standard SQL and JDBC as its APIs.
Derby functionality includes:
- Embedded engine with JDBC drivers
- Network Server
- Network client JDBC drivers
- Command line tools: ij (SQL scripting), dblook (schema dump) and sysinfo (system info)
This is a feature release. The following features were added.
- SQL Roles - SQL roles (as described in SQL 2003 and errata) are useful for administering privileges for groups of users. Administering roles is generally less error-prone than administering privileges for individual users. SQL Roles are defined in Feature T331 "Basic roles" and Feature T332 "Extended roles". Derby 10.5 implements a subset of T331, plus support for CURRENT_ROLE, which is a part of T332.
- Generated Columns - Generated Columns is a feature which instructs Derby to fill a column with an expression built out of other columns in the row. Derby constructs these column values at INSERT and UPDATE time. The user declares indexes on these columns. This in turn improves SELECT performance since it lets users declare indexes on pre-computed pieces of the WHERE clause. This feature satisfies support for SQL Feature T175.
- LOB Improvements - There were many performance and usability improvements for BLOBs and CLOBs.
- Replication of encrypted databases - With 10.5 it is possible to replicate encrypted databases.
- OFFSET/FETCH FIRST - SQL 2008 has added new syntax to support a direct way to limit the returned set of rows in a result set, through the fetch first and offset clauses.
- In-memory back end - Initial implementation of a storage engine for Derby where all data is kept in memory. There is no documentation for this feature. This functionality itself is not yet fully implemented, but users are welcome to experiment with it. For details, see the Primer for In-memory Back Ends.
- Standard ALTER COLUMN syntax - Allow standard SQL "SET" keyword in ALTER COLUMN syntax, like so: "ALTER TABLE ALTER [COLUMN] columnname SET DEFAULT default-value"
- SYSCS_UTIL.SYSCS_UPDATE_STATISTICS - New system procedure that updates cardinality statistics (or creates them if they do not exist) for a table's index or for all the indexes on a table, allowing a user to ensure that a query plan based on the most recent state of the table can be created.
The following issues are addressed by Derby release 10.5.1.1. These issues are not addressed in the preceding 10.4.2.0 release.
Issue Id Description DERBY-4148 NullPointerException in system.sttest run with 10.5.1.0 DERBY-4146 Cannot insert row when a generated column references an identity column DERBY-4145 NullPointerException on CREATE TABLE when generated column depends on another generated column DERBY-4141 XAExceptions thrown by Derby can have errorCode 0 DERBY-4135 Failure in BlobClob4BlobTest due to wrong results being returned DERBY-4128 Failure in ServerPropertiesTest due to java.security.AccessControlException on the server side, in 10.4 to 10.5.1. soft upgrade mode DERBY-4127 ArrayIndexOutOfBoundsException in DatabaseMetaDataTest when run in Soft Upgrade mode DERBY-4126 You can't invoke a table function which is stored in a jar file inside the database DERBY-4125 The in-memory storage back end doesn't work on Windows DERBY-4122 ClassCastException in SQLClob when running in soft upgrade mode (10.4.2.0 -> 10.5.1.0) DERBY-4121 Documentation: more UPDATE_STATISTICS fixes needed for Reference Manual and Tuning Derby DERBY-4119 Compress on a large table fails with IllegalArgumentException - Illegal Capacity DERBY-4117 dblook script fails when URL contains special characters DERBY-4116 SYSCS_UTIL.SYSCS_UPDATE_STATISTICS should update the store estimated row count for the table DERBY-4104 Documentation: update documentation files to 10.5 DERBY-4103 Allow unlimited growth for in-memory conglomerates DERBY-4096 'testRunTimeStatistics(org.apache.derbyTesting.functionTests.tests.lang.OffsetFetchNextTest)junit.framework.AssertionFailedError' on Windows DERBY-4095 Trigger fails with ERROR 38000: The exception 'java.sql.SQLException: ResultSet not open during VTIResultSet.getNextRowCore() DERBY-4094 Enforce absolute paths in the in-memory back end internal storage structure DERBY-4093 Improve handling of the database temporary directory DERBY-4088 DDMReader readBytes ArrayIndexOutOfBoundsException DERBY-4084 Determine the subSubProtocol name for the in-memory back end DERBY-4079 Add support for SQL:2008 <result offset clause> and <fetch first clause> to limit result set cardinality DERBY-4078 Document the SQL Standard routine matching rules which Derby now supports DERBY-4073 Creation/configuration of ClientXDataSource fails because of two setSsl methods DERBY-4072 shutdown with incorrect permission on log files shows java.lang.NullPointerException at org.apache.derby.impl.store.raw.log.LogToFile.flush(LogToFile.java:3964). Should give bettter message. DERBY-4071 AssertFailure when selecting rows from a table with CHARACTER and VARCHAR columns DERBY-4063 Constraint causes wrong query result when using exists DERBY-4061 InputStream returned from Blob.getBinaryStream(long, long) terminates the stream by returning 0, should return -1 DERBY-4060 Blob.getBinaryStream(long,long) is off by one for the pos+len check DERBY-4059 If space reclamation cannot obtain container lock it will not retry getting the lock DERBY-4050 Multithreaded clob update causes growth in table that does not get reclaimed DERBY-4049 SELECT statement topic in Reference Manual should reflect current WHERE clause behavior DERBY-4048 Update Statistics code in AlterTableConstantAction should use execute transaction rather than compile transaction DERBY-4042 org.apache.derby.impl.load.Import needs to escape single quotes DERBY-4040 SQLChar.getLength returns wrong length for some data values DERBY-4028 two rows can be inserted with the same value in a column that a unique constraint on that column should prevent DERBY-4027 An attempt was made to access an out of range slot on a page DERBY-4025 CHAR data type documentation fails to describe the maximum allowed length DERBY-4024 Replace felix.jar file with the felix sources DERBY-4022 Remove dependency of junit targets on deprecated jdk16 variable DERBY-4017 Removed deprecated jdk16 variable from the machinery which builds javadoc DERBY-4016 Document syntax change for ALTER TABLE in Reference Manual DERBY-4014 Update Derby Jira Report XML VTI to access more fields DERBY-4013 Allow standard SQL syntax: ALTER TABLE ALTER [COLUMN] <col> SET DEFAULT <default> DERBY-4012 'largedata/LobLimits.java' fails with "EXPECTED SQL Exception: (XJ001) Java exception: 'A truncation error was encountered trying to shrink CLOB 'XXXX' to length 104857600.: org.apache.derby.iapi.services.io.DerbyIOException'." DERBY-4010 PassThroughException should not reimplement initCause()/getCause() DERBY-4008 Only send RDBNAM on ACCSEC if EBCDIC conversion is possible DERBY-4006 ALTER COLUMN ... WITH DEFAULT NULL does not change the default DERBY-4004 Remove required RDBNAM from ACCSEC. Use SECCHK RDBNAM if none is provided on ACCSEC DERBY-3997 ORDER BY causes column to be returned DERBY-3996 Change eol style to native on jsr169 stubs DERBY-3990 ResultSetStreamTest fails on case sensitive file systems, and isn't run as part of any suite DERBY-3989 Set the Java 5 compilation path based on the Java 6 compilation path if a Java 5 environment can't be found but a Java 6 environment can be DERBY-3988 Set the Java 6 complilation path based on the Java 5 compilation path so that the JDBC4 support is always built DERBY-3985 Replace BUILDING.txt with a simlified up-to-date BUILDING.html DERBY-3984 Separate out the small number of build targets which really require JDK1.4 features from the bulk of the targets which ought to compile cleanly against small device libraries DERBY-3983 User Guide documentation on the limitations of small-device support is stale DERBY-3982 Add a facility to override the default set of old versions to be tested in the upgrade tests. DERBY-3981 Improve distribution of hash codes in SQLBinary and SQLChar DERBY-3978 Clob.truncate(long) in the client driver doesn't update the cached Clob length DERBY-3977 Clob.truncate with a value greater than the Clob length raises different exceptions in embedded and client driver DERBY-3975 SELECT DISTINCT may return duplicates with territory-based collation DERBY-3972 Update test harness to run with DesktopEE JRE DERBY-3970 PositionedStoreStream doesn't initialize itself properly DERBY-3969 NPE if you declare a constraint on a generated column and omit the datatype DERBY-3966 Do not require JRE 1.4 libraries at compile time since SUN's JDK 1.4 went out of support and can be used on JRE 1.4 without any problem DERBY-3964 NullPointerException when re-evaluating generated column during ON DELETE SET NULL referential action DERBY-3959 Reference Manual needs information on generated columns DERBY-3958 ImportExportProcedures imports ./testData/ImportExport/position_info.del using US-ASCII but the file contains non-ascii characters DERBY-3956 Remove method TemplateRow.checkPartialColumnTypes DERBY-3950 You should get an error if you try to override a generated column via an INSERT driven by an inner ResultSet DERBY-3948 Don't allow references to generated columns in the NEW transition variables of triggered actions of BEFORE triggers DERBY-3945 Generation clauses which mention user-coded functions may produce different resuls depending on who performs the triggering INSERT/UPDATE DERBY-3943 testReplication_Local_3_p2_StateTests_smallInsert_immediateStopMaster_DISABLED() causes hang in teardown() DERBY-3938 Document that you can change the length of varbinary as well as varchar columns DERBY-3936 Add CharacterStreamDescriptor DERBY-3935 Introduce interface for a position aware stream DERBY-3934 Improve performance of reading modified Clobs DERBY-3932 Add tests of generated columns when sql authorization is turned on DERBY-3931 GeneratedColumnsTest fails with "'F_MINUS' is not recognized as a function or procedure." on cvm/phoneME DERBY-3930 SQL roles: Add VTI for CONTAINED_ROLES DERBY-3923 Make the datatype optional when declaring columns with generation clauses DERBY-3922 Enable the adding of generated columns via ALTER TABLE DERBY-3917 RolesConferredPrivilegesTest fails with 'java.lang.NoClassDefFoundError: java.sql.DriverManager' with weme 6.1 DERBY-3911 BTreeController.reclaim_deleted_rows() may hide exceptions DERBY-3909 Race condition in NetXAResource.removeXaresFromSameRMchain() DERBY-3907 Save useful length information for Clobs in store DERBY-3905 Failed tests should save the database off to the fail directory DERBY-3904 NPE on left join with aggregate DERBY-3903 'The class 'org.apache.derby.diag.EnabledRoles' does not exist or is inaccessible.' in new roles tests (DERBY-3886) DERBY-3902 some messages are referenced but never used - MessageBundleTest output DERBY-3901 SQL roles: Move upgrade logic from 10.4 to 10.5 DERBY-3897 SQLSessionContext not correctly initialized in some non-method call nested contexts DERBY-3895 Needs update of master for 'derbyall/derbynetmats/DerbyNet/derbynetmats/holdCursorJDBC30'? DERBY-3894 Reference Manual: supported functions should be listed in JDBC escape syntax topic DERBY-3890 Replication: NPE for startSlave of encrypted database DERBY-3889 LOBStreamControl.truncate() doesn't delete temporary files DERBY-3886 SQL roles: ij show enabled and settable roles DERBY-3883 LOBStreamControl.replaceBytes() leaves temporary files open DERBY-3880 NPE on a query with having clause involving a join DERBY-3879 Getting Started Guide has typo and version problems DERBY-3878 Replication: stopSlave does not close serversocket when master has crashed. DERBY-3877 SQL roles: build support for dblook DERBY-3875 Derby cannot replace a database after encountering corruption DERBY-3874 Document numeric function ATAN2(float1,float2) DERBY-3872 NullPoinerException thrown when INTEGER function used as a predicate in a WHERE clause of a SELECT .. GROUP BY .. HAVING statement DERBY-3871 EmbedBlob.setBytes returns incorrect insertion count DERBY-3869 intermittent hang pinging server on Linux DERBY-3865 NPE in testSemantics(.....lang.RolesTest) on JavaME / CVM DERBY-3864 Remove utility class PrivilegedFileOps DERBY-3863 improve test importExportIJ.sql DERBY-3859 Reset the current role when EmbedPooledConnection creates a new logical connection DERBY-3855 The Tuning guide should mention that LIKE transformations/optimizations are disabled when using territory-based collations DERBY-3850 Remove unneeded workarounds for DERBY-177 and DERBY-3693 DERBY-3847 Tuning Guide lacks information on turning off Derby property settings DERBY-3845 Problems running org.apache.derbyTesting.system.optimizer.RunOptimizerTest DERBY-3841 'testTimeSlice_*(NetworkServerControlApiTest.java)' fails on Jvm1.5/vista, Jvm1.4/SunOS 5.11 DERBY-3840 The test code executes java processes by just executing java instead of using a full path. This may cause the wrong java to be picked up. DERBY-3837 need to test getting appropriate error message when os files are read-only DERBY-3835 Convert derbynet/getCurrentProperties to JUnit DERBY-3832 Convert derbynet/timeslice.java to Junit DERBY-3831 RuntimeStatisticsParser can not distinguish names of table or index well and truly DERBY-3828 Convert jdbcapi/SURTest_ij.sql to JUnit. DERBY-3827 Add Apache 2.0 license headers to a number of files DERBY-3826 Remove the update-with-jdbc4 script, which is dead code now DERBY-3825 StoreStreamClob.getReader(charPos) performs poorly DERBY-3818 client Insert/retrieval of 18MB Clob is extremely slow in MultiByteClobTest DERBY-3816 Administration Guide topics on unsupported DB2 driver should be removed DERBY-3813 Derby tests for the existance of BigDecimal methods toPlainString and bdPrecison but does not check if they were found before using them. DERBY-3812 failure in testAttributeDrdaStreamOutBufferSize(org.apache.derbyTesting.functionTests.tests.management.NetworkServerMBeanTest)junit.framework.AssertionFailedError: expected:<0> but was:<131072> DERBY-3810 Create a simple Clob performance regression test DERBY-3807 traces of removed ij functionality of ij.unicodeEscape should be removed DERBY-3805 Create a simple Blob performance regression test DERBY-3803 'org.apache.derbyTesting.functionTests.tests.lang.XMLTypeAndOpsTest' failures on JVM 1.5 on trunk, 10.4 and 10.3 DERBY-3802 Convert "org.apache.derbyTesting.functionTests.tests.lang.optimizerOverrides.sql" to junit. DERBY-3800 Convert derbynet/maxthreads to JUnit DERBY-3799 NullPointerException when accessing a clob through a pooled connection DERBY-3798 Remove unnecessary call to Hashtable.get() in TableScanResultSet.getNextRowCore() DERBY-3797 Convert jdbcapi/metadataMultiConn to JUnit. DERBY-3796 Convert derbynet/OutBufferedStream to JUnit DERBY-3793 Remove unnecessary methods from InternalClob interface DERBY-3791 Excessive memory usage when fetching small Clobs DERBY-3787 Document the new system stored procedure SYSCS_UTIL.SYSCS_UPDATE_STATISTICS which was added as part of DERBY-269 DERBY-3786 Assert failure in CacheEntry.unkeepForRemove when running stress.multi DERBY-3784 ImportExportTest doesn't detect lack of expected errors DERBY-3783 LOBStreamControl shouldn't throw SQLException DERBY-3782 Client Configuration.java imports engine class org.apache.derby.iapi.services.info.JVMInfo DERBY-3781 PositionedStoreStream.reposition(pos) with pos greater than length leaves the stream object in an inconsistent state DERBY-3780 Run junit tests with -Dfile.encoding="UTF-16" to expose encoding issues and analyze failures DERBY-3779 Add client side JDBC statement pool documentation DERBY-3777 SecureServerTest, SSLTest are failed on Zos with exit code 143 starting network server DERBY-3776 testGetBytes under BlobClob4BlobTest failed on Zos with encoding issue DERBY-3775 BlobStoredProcedureTest failed on Zos: AssertionFailedError: Error SYSIBM.BLOBGETPOSITIONFROMLOCATOR returns the wrong value for the position of the Blob expected:<8> but was:<-1> DERBY-3774 jdbc4/ClobTest fails on Zos with AssertionFailedError: Streams differ at index 0 expected:<200> but was:<72> DERBY-3773 ImportExportLobTest failed on Zos Clobs differ at index 1 expected:<99> but was:<196> DERBY-3771 testClasspathChecker under SysinfoCPCheckTest failed on Zos DERBY-3770 Create a utility class for skipping data in an InputStream DERBY-3769 Make LOBStoredProcedure on the server side smarter about the read buffer size DERBY-3768 Make EmbedBlob.length use skip instead of read DERBY-3767 Convert "org.apache.derbyTesting.functionTests.tests.lang.nulls.sql" to junit. DERBY-3766 EmbedBlob.setPosition is highly ineffective for streams DERBY-3764 Union Query fail on Derby 10.4.1.3 DERBY-3763 Rename BaseJDBCTestCase.usingDerbyNet DERBY-3762 Convert "org.apache.derbyTesting.functionTests.tests.lang.arithmetic.sql" to junit. DERBY-3760 Convert "org.apache.derbyTesting.functionTests.tests.lang.miscerrors.sql" to junit. DERBY-3759 Convert "org.apache.derbyTesting.functionTests.tests.lang.ungroupedAggregatesNegative.sql" to junit. DERBY-3758 Convert "org.apache.derbyTesting.functionTests.tests.lang.precedence.sql" to junit DERBY-3755 ij's help text lacks the optional [HOLD | NOHOLD] syntax for GET CURSOR DERBY-3754 Convert "org.apache.derbyTesting.functionTests.tests.lang.connect.sql" to junit DERBY-3751 Convert case.sql to junit DERBY-3750 Convert "org.apache.derbyTesting.functionTests.tests.lang.constant Expression.sql" to junit DERBY-3745 Derby can leak classloaders in an app server environment DERBY-3743 Revoking EXECUTE privilege on a function if used in a CHECK constraint: implementation problem DERBY-3742 Move test code from CompressedNumber to a unit test DERBY-3741 SQL LENGTH function materializes CLOB into memory DERBY-3739 Skip and read methods in ArrayInputStream may overflow DERBY-3738 Add more tests for legal/illegal commands in the different replication states DERBY-3736 Revoking a column level privilege from a user, a prepared statement relying on that privilege can still be executed DERBY-3735 Incorrect position calculation in PositionedStoreStream with read(byte[],...) DERBY-3734 Maximum value allowed for derby.storage.fileCacheSize (100) is too low for large system. Increase the maximum value and redocument the property. DERBY-3732 SQL Length function materializes BLOB into memory DERBY-3731 Improve calculation of refSize in ClassSize.java DERBY-3730 Bundle-SymbolicName: needed in Derby manifest for OSGi 4 environment DERBY-3728 Fix error handling in PrivilegedFileOpsForTests DERBY-3726 Don't call RAFContainer.padFile() from instances of RAFContainer4 DERBY-3725 add more information to the XSDB1:ERROR XSDB1: Unknown page format at page error DERBY-3723 Reset current schema to default (user name) when creating a new logical connection in the client driver DERBY-3722 Add circularity check for the GRANT role statement DERBY-3718 NPE when firing a trigger DERBY-3717 convert derbynet/getCurrentProperties.java to junit DERBY-3715 java/testing/README.htm refers to a non-existing test DERBY-3711 convert store/aes.sql to junit test & add unrestricted test cases. DERBY-3708 setting tracedirectory from the command line does not work DERBY-3707 STATUS file changes June 2008 DERBY-3706 NetworkServer console messages should print a time stamp DERBY-3705 In Net Client mode, negative values for stream length are accepted without an exception for PreparedStatement.setAsciiStream() DERBY-3704 If an IOException is encountered during establishment of the connection, Network Server should print the root exception to the console instead of a generic message DERBY-3703 Make it possible to build the JSR169 support with the jdk1.4 libraries DERBY-3702 Remove xml-apis.jar because we no longer have to compile against jdk1.3 DERBY-3701 java.lang.Exception: DRDA_UnableToAccept.S:Unable to accept connections and client hang if tracing is turned on but traceDirectory does not exist DERBY-3700 Get rid of tools/java/empty.jar DERBY-3696 Delete superfluous test lang/longStringColumn.java DERBY-3695 NullPointerException when invoking statement event listeners if one of the listeners is null DERBY-3694 'Method resolution for signature java.lang.String.valueOf(int) was ambiguous. (No single maximally specific method.)' in 'testFunctionNullHandling(org.apache.derbyTesting.functionTests.tests.lang.RoutineTest)' on JavaME DERBY-3693 Deadlocks accessing DB metadata DERBY-3692 'javax.transaction.xa.XAException' ++ in 'J2EEDataSourceTest' DERBY-3690 EmbedPooledConnection doesn't reset schema when creating a new logical connection DERBY-3686 convert predicatePushdown.sql to JUnit DERBY-3685 weme 6.1 Base connection pooling suitejunit.framework.AssertionFailedError: java.lang.NullPointerException DERBY-3684 Wire the vti demo into the build. DERBY-3682 SYSCS_BULK_INSERT doesn't quote identifiers or strings properly DERBY-3681 When authenticating a user at connect time, verify that the user provided is not also a defined role name. DERBY-3678 StackOverflowException in deadlock trace DERBY-3674 'ALTER TABLE' is not allowed on the System table '"SYSCS_DIAG"."TRANSACTION_TABLE"' in derbyall - store/TransactionTable.sql DERBY-3673 Add checks that a new role isn't already a user authorization id DERBY-3668 Remove JDBC 3.0-specific topics from Reference Manual and merge implementation notes as needed DERBY-3667 SQL roles: Make CURRENT_ROLE check that the role is still valid DERBY-3666 Make role descriptor a dependency Provider DERBY-3665 Master file for derbynet/testProperties.java needs update for 10.3.2.2 to 10.3.3.0 version change. DERBY-3663 Convert store/streamingColumn to JUnit DERBY-3661 Typo yields redundant test case in GrantRevokeTest DERBY-3659 revive tests with ldap references to be runnable as junit tests DERBY-3658 LOBStateTracker should not use SYSIBM.CLOBRELEASELOCATOR when the database is soft-upgraded from 10.2 DERBY-3657 Comment in template security policy incorrectly says that JMX is not enabled by default DERBY-3655 errror in nightly regression test: LobStreamsTest:encryptedjunit.framework.AssertionFailedError: f:\jartest\JarResults.2008-04-29\ibm16_suites.All\system\singleUse\oneuse1e\tmp\lob6165.tmp DERBY-3653 Convert datetime.sql to JUnit DERBY-3652 Derby does not follow the SQL Standard when trying to map SQL routines to Java methods. DERBY-3649 can't call a stored function with an aggregate argument without getting the following error: ERROR 42Y29 DERBY-3642 Update and rectify the Derby tutorial DERBY-3640 Add an example for table function in Derby Reference Manual DERBY-3638 java/testing/Readme.htm location of derbyTesting.jar DERBY-3633 tools/derbyrunjartest.java fails with 'No command given.' (expecting 'No arguments given.') DERBY-3631 UDF used with aggregate arguments results in error 30000 DERBY-3629 Tools Guide should document continuation marker for ij DERBY-3625 XSDA3 error in concateTests in lang.LangHarnessJavaTest caused by bug in SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE() DERBY-3623 Convert deadlockMode.java to JUnit DERBY-3622 SYSCS_UTIL.SYSCS_EMPTY_STATEMENT_CACHE needs a better description in the reference manual DERBY-3619 Implement more load types for org.apache.derbyTesting.perf.clients.Runner DERBY-3618 Perform thread dump with ASSERTS with jdk 1.5 or higher DERBY-3615 Making the instructions more clear in documentation(BUILDING.txt) regarding setting JRE lib path for a Windows system DERBY-3613 SELECT DISTINCT field FROM TABLE_NAME GROUP BY field, field2 DERBY-3612 Developer's Guide needs correction on garbage collection DERBY-3602 If derbytesting.jar is in a different directory than the derby jars SystemPrivilegesPermissionTest fails with java.security.AccessControlException DERBY-3601 Optimize LOBStateTracker for non-locator servers DERBY-3597 Incorporate DERBY-3310 and DERBY-3494 write-ups into NormalizeResultSetNode javadoc DERBY-3596 Creation of logical connections from a pooled connection causes resource leak on the server DERBY-3589 AllocPage.createPage() doesn't initialize minimumRecordSize correctly DERBY-3588 suites.All fails to run on Jvm 1.5 when built with JDK 1.5 (Failed to invoke suite(): .jdbc4._Suite) DERBY-3587 Convert jdbcapi/testRelative.java to JUnit DERBY-3586 Remove am.Connection.reset(LogWriter,ClientBaseDataSource,boolean) and called methods DERBY-3584 Replication tests shall not be run on Java ME DERBY-3582 IndexOutOfBoundsError in ClockPolicy.moveHand DERBY-3581 Changing certain properties on client DataSource objects causes existing connections to reflect the new values DERBY-3580 Remove unused method Connection.resetConnection(LogWriter, String, Properties) DERBY-3579 The Developer's Guide incorrectly describes the behavior of transactions inside procedures and functions DERBY-3578 DrdaStreamOutBufferSize attribute of NetworkServerMBean should be of type int instead of String DERBY-3576 Merge EngineBlob and EngineClob into a single interface DERBY-3574 With client, attempting to get the lob length after commit or connection close if there was a call to length() before commit does not throw an exception DERBY-3572 directory error in the file "java/testing/README.htm" DERBY-3571 LOB locators are not released if the LOB columns are not accessed by the client DERBY-3570 Implement DETERMINISTIC keyword for procedures and functions DERBY-3568 Convert jdbcapi/savepointJdbc30_JSR169.java and jdbcapi/savepointJdbc30_XA.java to JUnit DERBY-3567 AsynchronousLogShipper#forceFlush should time out DERBY-3566 Alter column set data type not allowed in soft upgrade with unique constraint DERBY-3562 Number of log files (and log dir size) on the slave increases continuously DERBY-3561 testStartStopManagementFromApplication(org.apache.derbyTesting.functionTests.tests.management.ManagementMBeanTest)junit.framework.AssertionFailedError: expected:<2> but was:<5> DERBY-3558 regression test failure in testDerbyJarAttributeAlpha - Security Exception DERBY-3556 change derby.tests.trace property to print the name of the test before it runs it DERBY-3554 Change Collation test to run DatabaseMetaDataTest, BatchUpdateTest,GroupByExpressionTest, and UpdateableResultSetTest for only one locale DERBY-3551 Implement procedure SYSCS_UTIL.SYSCS_PREPARE_REPLICATION() DERBY-3549 Unable to start slave mode after authentication failure on a previous startSlave attempt DERBY-3548 NoClassDefFoundError failure in SystemPrivilegesPermissionTest for weme6.1 DERBY-3546 Failed to get database schemas of a JAR database DERBY-3544 If NetworkServer fails to shutdown when run as a separate process by NetworkServerTestSetup, test run will hang DERBY-3543 NetworkServerControl with options but no command does not give usage message DERBY-3538 NullPointerException during execution for query with LEFT OUTER JOIN whose inner table selects all constants. DERBY-3531 DatabasePermission uses a JDK 1.4 method that is not present in J2ME/CDC/Foundation. DERBY-3527 The slave will not notice that a network cable is unplugged and will therefore reject failover/stopSlave commands DERBY-3526 AsynchronousLogShipper#workToDo is blocked while the log shipper sends a log chunk DERBY-3525 Remove unneeded code to get JDBC level in BrokeredConnection and BrokeredStatement classes DERBY-3521 Functionality for skipping testsuites on certain platforms fails for the new management testsuite on phoneME advanced DERBY-3520 convert views.sql to junit DERBY-3515 23 timeouts encountered while running management junit suite as part of full nightly runs. DERBY-3509 The replication log shipper is not notified when a new replication transmitter is instantiated in MC#handleException. DERBY-3508 Log receiver thread fails with NPE at failover when master has died DERBY-3506 Add JMX testing with a security manager and investigate testing with JMX authentication. DERBY-3503 Change stress.multi to dump thread stacks before killing off testers with jdk 1.5 and higher DERBY-3494 Move the setup of NormalizeResultSetNode into the NormalizeResultSetNode DERBY-3489 Error message XRE04 does not include the right port number. DERBY-3469 Clob.length() doesn't detect a closed underlying connection in a consistent way DERBY-3462 Require new permissions in o.a.d.security.SystemPermission to allow control to Derby's JMX management and to ensure information is not leaked through JMX DERBY-3447 Shutdown on a database without stopping replication hangs DERBY-3446 Make ResultSet.getStatement return the correct statement when created by a logical statement DERBY-3431 DatabaseMetaData.getConnection returns the wrong connection when using connection pooling DERBY-3420 The -ca option should be removed from ij and from the ij command usage line DERBY-3409 Remove JDBC 2.0-specific topics from Reference Manual and merge implementation notes as needed DERBY-3408 Wrong message when using SHOW in ij. DERBY-3401 Removing a ConnectionEventListener from a PooledConnection during its connectionClosed() callback causes other ConnectionEventListener callbacks to be missed DERBY-3397 Derby 10.3.1.4 and 10.3.2.1 break scrollable result sets? Hibernate Query.setFirstResult and/or setMaxResults DERBY-3390 SQLException thrown from user function kills network connection DERBY-3385 Tests for JMX Management and Monitoring DERBY-3381 "ERROR XSDA3: Limitation: Record cannot be updated or inserted due to lack of space on the page...." in suites.All DERBY-3379 "No Current connection" on PooledConnection.getConnection() if pooled connection is reused during connectionClosed processing DERBY-3373 SQL "distinct" and "order by" needed together DERBY-3360 Invalid method java.lang.Integer >> void <init>(short) because java.lang.NoSuchMethodException: java.lang.Integer.<init>(short) DERBY-3354 Select from large lob table with embedded gives OutOfMemoryError DERBY-3347 ERROR XSDB3: Container information cannot change once written DERBY-3338 CancelQueryTask.forgetContext() could be simplified. DERBY-3327 SQL roles: Implement authorization stack (and SQL session context to hold it) DERBY-3320 Database creation and boot should fail if collation=TERRITORY_BASED and the selected locale is not supported DERBY-3319 Logical connections do not check if a transaction is active on close DERBY-3310 ASSERT in MergeSort.checkColumnTypes() disallow legal type conversions DERBY-3307 NPE in PooledConnction event notification handling if a null listener is added DERBY-3270 Delayed (on-demand) creation of current user schema makes select from view belonging to other schema fail. DERBY-3266 Not possible for non-db-owner to create a temporary table. "Get ERROR 42507: User 'USERB' can not perform the operation in schema 'SESSION'." DERBY-3227 Remove final from all getConnection() methods in EmbeddedDataSource DERBY-3223 SQL roles: make use of privileges granted to roles in actual privilege checking DERBY-3219 Group by query with many aggregate columns and case statements fails with: ERROR XSDA7: Restore of a serializable or SQLData object of class , attempted to read more data than was originally stored DERBY-3200 Developer's Guide: Add examples showing use of SQL authorization with user authentication DERBY-3193 SQL roles: Add documentation DERBY-3180 error XSDA3 when test is executing SYSCS_INPLACE_COMPRESS_TABLE in specific situation DERBY-3151 Reduce dependency on NetworkServerControl in TestConfiguration DERBY-3137 SQL roles: add catalog support DERBY-3130 Reduce memory footprint of StoredRecordHeader DERBY-3116 totalSpace not properly initialized in AllocPage DERBY-3097 Unnecessary if statement can be removed from BaseActivation.getColumnFromNow DERBY-3063 Extend the system function mechanism to support functions with multiple arguments DERBY-3043 'Schema <schemaname> does not exist' when constraint used in table definition DERBY-3035 Cannot restore backups without BACKUP.HISTORY file DERBY-2991 Index split deadlock DERBY-2958 need to mention ClientDriver In Derby reference manual's java.sql.Driver interface section DERBY-2892 Closing a resultset after retrieving a large > 32665 bytes value with Network Server does not release locks DERBY-2861 Thread safety issue in TableDescriptor DERBY-2822 Add caching of store stream length in StoreStreamClob, if appropriate DERBY-2750 Convert lang/declareGlobalTempTableJava.java to JUnit DERBY-2747 Reply incorrectly handles read() returning -1 DERBY-2514 convert lang/closed.java to junit DERBY-2498 NullPointerException on ClientDataSource.getConnection() when ds.setdatabaseName was invalid DERBY-2389 DOCS - Move Derby system and properties info from Tuning Guide into Reference Manual DERBY-2388 DOCS - Reorder JDBC Reference section in Ref Manual DERBY-2353 intermittent NPEs during DELETE ops in a reasonably large transaction DERBY-2351 ORDER BY with expression with distinct in the select list returns incorrect result DERBY-2320 Example in DEGREES function is not appropriate. DERBY-2207 Improve usability of Derby's client/server security by implementing ANSI Roles DERBY-2085 Misleading error message for non-matching ORDER BY clause in queries with GROUP BY. DERBY-1944 jdbcapi/ParameterMappingTest.java test does not execute test for setObject(Blob/Clob) in DerbyNetClient DERBY-1932 Reference Manual updates - JDBC Reference section DERBY-1848 jdbcapi/SetQueryTimeoutTest.java fails on IBM wctme 5.7 DERBY-1764 Rewrite stress.multi as a JUnit test DERBY-1726 Make i18n/LocalizedDisplay.sql and i18n/LocalizedConnectionAttribute.sql behave equally on different platforms DERBY-1670 Remove optional osgi.jar download by including equivalent jar from the apache incubator felix project in the Derby's svn DERBY-1458 Add MessageBundleTest to build script to verify that there are no orphaned message ids DERBY-1416 Message id J029 (CONN_RESTORE_FROM) does not have a matching message text DERBY-1415 Message id J030 in reference.MessageId.java does not have a matching message DERBY-1412 Possible values for derby.storage.rowLocking Derby engine is not currently documented eventhough the property is DERBY-1411 DRDAConnThread uses SQLState.NO_SUCH_DATABASE, which has no matching message DERBY-1387 Add JMX extensions to Derby DERBY-1331 Derby's "set schema" behavior is not compliant with SQL 2003 Foundation spec. DERBY-1107 For existing databases JDBC metadata queries do not get updated properly between maintenance versions. DERBY-1062 Change the internal implemetation of SYSCS_INPLACE_COMPRESS_TABLE to share existing alter table DERBY-961 Error 22001 can include 'XX-RESOLVE-XX' which should be removed DERBY-719 Calling SYSCS_INPLACE_COMPRESS_TABLE on SYS.SYSTABLES results in a lock timeout, with only a single connection active DERBY-576 xaHelper in ij creates global id that is not the same across platforms DERBY-503 Documentation should recommend using .newInstance() to instantiate JDBC driver DERBY-481 implement SQL generated columns DERBY-269 Provide some way to update index cardinality statistics (e.g. reimplement update statistics) DERBY-48 A connection request that has a default schema that is being created by another transaction will fail to connect
Compared with the previous release (10.4.2.0), Derby release 10.5.1.1 introduces the following incompatibilities. These merit your special attention.
Note for DERBY-4073: The method setSsl(int) has been removed from the client data source classes.
Note for DERBY-4050: Table growth may result from multi-threaded Clob updates before the fix for DERBY-4050.
Note for DERBY-4042: Format of file name arguments to the import procedures changed for files with single quotes (') in their names.
Note for DERBY-4008: After the change for DERBY-4008, applications may see a different error message when attempting to connect a 10.5 client to older revision servers (e.g. 10.4 and 10.3) with multibyte database names.
Note for DERBY-3977: A different exception is thrown by the embedded driver when trying to truncate a Clob with a too large length argument.
Note for DERBY-3701: An error message will be logged to derby.log if the Network Server tracing file cannot be created. Starting with version 10.5, the Network Server will attempt to create the trace directory if it does not exist. Any intervening directories in the given path will also be created if possible.
Note for DERBY-3652: Derby now follows the SQL Standard rules for matching functions and procedures to Java methods.
Note for DERBY-3420: The -ca command line option has been removed from ij.
Note for DERBY-3347: A bug that could cause unrecoverable database corruption has been fixed.
Note for DERBY-3327: The effect of setting the current default schema (
SET SCHEMA schemaname
) inside nested connection of a stored procedure or function has been changed to comply with SQL standard semantics.Note for DERBY-3319: Exception is thrown when connection with uncommitted operations is closed.
Note for DERBY-2351: An ORDER BY clause of a DISTINCT query which specifies to order by a column which was not in the DISTINCT list is now rejected, because the intent of the query is ambiguous. Previously, Derby instead produced non-distinct results. Also, an ORDER BY clause which specifies a table-name-qualified column alias is now rejected as invalid, where previously it was accepted.
Note for DERBY-2085: Derby has improved the error message which is issued when an invalid column reference is found in a grouped query. Derby now issues message 42Y36 instead of 42Y30. Furthermore, the wording of message 42Y36 has been expanded to further explain the behavior.
Note for DERBY-1062: Applications will see a different error code and message when they attempt to call SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE on a table that does not exist.
Note for DERBY-48: In Derby, a user's initial default schema is named the same as the user name, or APP if a user is not provided at connect time. This schema is implicitly auto-created the first time a schema object is created in that schema.
Note for DERBY-4073
Summary of Change
The method setSsl(int) has been removed from the client data source classes.
Symptoms Seen by Applications Affected by Change
The application will either fail to compile, or experience a NoSuchMethodError. Only applications invoking the removed method are affected.
Incompatibilities with Previous Release
The method setSsl(int) can no longer be used to configure SSL with the client data sources.
Rationale for Change
Having two setSsl-methods caused choice problems for some applications using Derby data sources and also configuring the data source through introspection, since there were two methods called setSsl; setSsl(int) and setSsl(String).
Application Changes Required
Use setSsl(String) instead of setSsl(int).
Note for DERBY-4050
Summary of Change
Table growth may result from multi-threaded Clob updates before the fix for DERBY-4050.
Symptoms Seen by Applications Affected by Change
Before the fix for DERBY-4050, applications may have seen growth in the size of the database, particularly the size of the dat files for Clob tables, when doing updates to the Clob tables from multiple threads. The fix for DERBY-4050 will prevent future growth but will not reclaim space consumed by the bug in the past. To reclaim the space, a full offline compress of the clob table is needed. See the Derby documentation for usage of SYSCS_UTIL.SYSCS_COMPRESS_TABLE
Incompatibilities with Previous Release
none
Rationale for Change
DERBY-4050 fixes a bug in space reclamation.
Application Changes Required
The compress table should only need to be run once if the fix for DERBY-4050 has been applied. No application changes should be required.
Note for DERBY-4042
Summary of Change
Format of file name arguments to the import procedures changed for files with single quotes (') in their names.
Symptoms Seen by Applications Affected by Change
In the previous release, applications that imported data from a file whose file name contained a single quote (') character would fail unless they replaced each single quote in the file name argument to the import procedure with two single quotes. Now this has changed and Derby does not understand the file name argument unless it has the same number of single quote characters as the actual file name. Applications that worked around the problems in earlier releases by adding extra single quotes, will now see exceptions like the following:
ERROR XIE04: Data file not found: Rock''n''roll.csvApplications that do not import files whose names contain single quotes will not be affected by this change.
Incompatibilities with Previous Release
Previous releases required file name arguments to the import procedures to double each occurrence of a single quote in the file name, like this when importing a file called C:/Chip's/TERMS.dat:
PreparedStatement ps = conn.prepareStatement( "CALL SYSCS_UTIL.SYSCS_IMPORT_TABLE(null,'MY_TABLE',?,null,null,null,0)"); ps.setString(1, "C:/Chip''s/TERMS.dat"); ps.execute();In this release the above code would result in the following exception:
ERROR XIE04: Data file not found: C:/Chip''s/TERMS.datIn this release, applications must use the actual name of the file, like this:
PreparedStatement ps = conn.prepareStatement( "CALL SYSCS_UTIL.SYSCS_IMPORT_TABLE(null,'MY_TABLE',?,null,null,null,0)"); ps.setString(1, "C:/Chip's/TERMS.dat"); ps.execute();Rationale for Change
It is more intuitive if the file name arguments match the actual file names. Also, in the previous releases there were some combinations of single quotes and other special characters (like double quotes) that it was not possible to get to work even with the workaround.
Application Changes Required
Applications that work around the issue in previous releases by adding extra single quotes to the file name arguments, must be changed so that they do not add extra single quote characters.
Note for DERBY-4008
Summary of Change
After the change for DERBY-4008, applications may see a different error message when attempting to connect a 10.5 client to older revision servers (e.g. 10.4 and 10.3) with multibyte database names.
Symptoms Seen by Applications Affected by Change
In previous releases when attempting to connect to network server using a multibyte database name the connect would fail with the the exception:
SQLSTATE:22005 java.sql.SQLDataException: Unicode string cannot convert to Ebcdic string
This message will not change for 10.5 clients connecting to 10.5 servers. After the change for DERBY-4008 the error message may change when connecting 10.5 clients to older revision server. Instead the message will be:
SQLSTATE:08006 java.sql.SQLNonTransientConnectionException: A network protocol error was encountered and the connection has been terminated: A PROTOCOL Data Stream Syntax Error was detected. Reason: 0x8,464. Plaintext connection attempt to an SSL enabled server?
with a chained SQLException:
SQLSTATE:22005 java.sql.SQLDataException: Unicode string cannot convert to Ebcdic string
The protocol error will also show on the server console.Incompatibilities with Previous Release
The client error message when attempting to connect with multibyte database names will change with mixed revision client/server as described in the previous section.
Rationale for Change
This change was made in preparation for fixing DERBY-728 to allow multibyte characters in the database name. The change in error message with back revision servers was considered an acceptable change and necessary to fix DERBY-728.
Application Changes Required
To reveert to the previous error message, users can upgrade their server to the latest on the 10.3 or 10.4 branch, or upgrade their server to 10.5. Otherwise applications expecting the old error message should upon gettting the SQLState: 08006 message retrieve the chained exception with SQLExcepiton.getNextMessage() and then process the message as before.
Note for DERBY-3977
Summary of Change
A different exception is thrown by the embedded driver when trying to truncate a Clob with a too large length argument.
Symptoms Seen by Applications Affected by Change
In the previous release, the embedded driver would throw an exception with SQLState XJ076 when trying to truncate a Clob with a length argument bigger than the Clob length. Now, it will throw the same exception as the client driver, XJ079. The error messages also differ:
Before: XJ076: The position argument '18149' exceeds the size of the BLOB/CLOB. Now: XJ079: The length specified '18149' exceeds the size of the BLOB/CLOB.Incompatibilities with Previous Release
Applications catching a specific exception based on SQLState may behave differently. The incompatibility can only be seen if the application calls
Clob.truncate
with the embedded driver.Rationale for Change
To make the embedded and the client driver consistent.
Application Changes Required
Look for SQLState XJ079 instead of XJ076 when
Clob.truncate()
is called.
Note for DERBY-3701
Summary of Change
An error message will be logged to derby.log if the Network Server tracing file cannot be created. Starting with version 10.5, the Network Server will attempt to create the trace directory if it does not exist. Any intervening directories in the given path will also be created if possible.
Symptoms Seen by Applications Affected by Change
Before the fix for DERBY-3110, if derby.drda.traceAll was set to true and the derby.drda.traceDirectory was set to a non-existent directory, no tracing would occur and no error would occur. After the fix for DERBY-3110, an error "java.lang.Exception: DRDA_UnableToAccept.S:Unable to accept connections" would occur and the client would hang and no tracing would occur. With this fix for version 10.5 and higher, the Network Server will attempt to create the trace directory if possible. For 10.4.2 (and the next release on the 10.3 branch), the Network Server will still not try to create the directory. For all these releases the Network Server will print an error on session connect if there is any problem creating the trace file, but the Network Server will not cause the session connection to fail. Users who have trace turned on and the trace directory set to a non-existent directory may now see exceptions in the derby.log on connect indicating that the trace file is not found or with 10.5 or higher they may see tracing occur where it did not before.
Incompatibilities with Previous Release
Tracing properties will not be ignored or cause the client to hang if the trace directory is set to a non-existent directory.
Rationale for Change
The tracing properties should not be summarily ignored or cause the client to hang if the trace directory does not exist.
Application Changes Required
Applications that counted on the derby.drda.traceAll property being ignored if derby.drda.traceDirectory was set to a non-existent directory, need to turn tracing off or they may now see many errors in the derby.log or large amounts of tracing.
Note for DERBY-3652
Summary of Change
Derby now follows the SQL Standard rules for matching functions and procedures to Java methods.
Symptoms Seen by Applications Affected by Change
In previous releases, Derby matched SQL routines to Java methods using an irregular set of rules which were hard to describe. At the same time, the user documentation falsely claimed that Derby followed the signature matching rules in the ANSI/ISO SQL Standard. Derby now conforms to the Standard behavior. In general, the Standard behavior is easier to understand and it is now possible to be confident that a function or procedure definition will match the desired Java method. In certain corner cases, however, methods which used to resolve will no longer resolve. The Standard resolution rules are described in the Derby Reference Guide.
Incompatibilities with Previous Release
Routines may resolve differently. Here are the differences listed in declining order of likelihood:
- Ambiguity - Some SQL routines may match multiple Java methods now. This raises an error according to the Standard. This is most likely to affect the following matches:
- Matching INTEGER to Java int and Integer.
- Failure - Some SQL routines may fail to match Java methods. This is most likely to affect the following matches:
- SMALLINT no longer matches int.
- INTEGER no longer matches long.
- BIGINT no longer matches float.
- REAL no longer matches double.
- VARCHAR no longer matches Object.
- NUMERIC no longer matches Object.
- Change - Some SQL routines may resolve to different Java methods now.
- Success - Some SQL routines may now resolve even though previously they did not resolve to Java methods. This is most likely to affect the following matches:
- BLOB now matches Blob.
- CLOB now matches Clob.
Rationale for Change
The previous behavior violated the SQL Standard and was very hard to explain. The new behavior is correct and easy to describe.
Application Changes Required
This release includes a lint tool, SignatureChecker. If your application uses SQL functions and/or procedures, you should run this tool against your databases in order to find routines which no longer match. To run the tool, make sure that your classpath contains the 10.5 jar files, including derbytools.jar.
On a J2SE platform, run the lint tool as follows (where CONNECTION_URL_TO_DATABASE is the connection URL you would use in order to obtain a connection via DriverManager.getConnection()):
java org.apache.derby.tools.SignatureChecker CONNECTION_URL_TO_DATABASEAlternatively, you can invoke the tool via derbyrun.jar. E.g.:
java -jar derbyrun.jar SignatureChecker "jdbc:derby:myDB"On a J2ME platform, run the lint tool as follows (where DATABASE_NAME is the database name you would set via EmbeddedSimpleDataSource.setDatabaseName()):
java org.apache.derby.tools.SignatureChecker DATABASE_NAMEThe tool examines every routine registered in the database and prints out results like the following:
Found a matching method for: "APP"."DOINSERT"( ) Found a matching method for: "APP"."DOINSERTANDCOMMIT"( ) Found a matching method for: "APP"."APPENDFOOANDBAR"( VARCHAR ) Unresolvable routine: "APP"."IDONTEXIST"( VARCHAR , INTEGER ). Detailed reason: No method was found that matched the method call z.iDontExist(java.lang.String, int), tried all combinations of object and primitive types and any possible type conversion for any parameters the method call may have. The method might exist but it is not public and/or static, or the parameter types are not method invocation convertible. Found a matching method for: "APP"."RUNDDL"( VARCHAR ) Unresolvable routine: "APP"."TABFUNCDOESNTEXIST"( VARCHAR , BIGINT ). Detailed reason: No method was found that matched the method call org.apache.derbyTesting.functionTests.tests.lang.TableFunctionTest.appendFooAndBar(java.lang.String, long), tried all combinations of object and primitive types and any possible type conversion for any parameters the method call may have. The method might exist but it is not public and/or static, or the parameter types are not method invocation convertible.In the example above, the lint tool found matches for all routines except for the functions app.iDontExit and app.tabFuncDoesntExist. If the lint tool cannot find a match for one of your functions or procedures, it tells you what signature it expected to find. You need to adjust your application in one of the following ways:
- Method - Change the signature of your Java method to match the signature suggested by the lint tool.
- Routine - Drop and recreate your function/procedure so that its arguments and return type match your Java method according to the Standard rules described in the Reference Guide.
Note for DERBY-3420
Summary of Change
The -ca command line option has been removed from ij.
Symptoms Seen by Applications Affected by Change
Applications which attempt to pass JDBC connection attributes using the -ca flag to ij will be rejected, with a usage message such as:
Usage: java org.apache.derby.tools.ij [-p propertyfile] [inputfile]Incompatibilities with Previous Release
Applications which attempt to pass JDBC connection attributes using the -ca flag to ij will fail, as that flag is no longer recognized by ij.
Rationale for Change
It used to be that you could not specify both client and embedded attributes in the connection URL. Since Derby now supports such combinations of attributes in the connection URL, the community decided that the -ca option doesn't really add value and so it has been removed.
Application Changes Required
Applications should include connection attributes in the connection URL. For example:
connect 'jdbc:derby:myDB;territory=no_NO;collation=TERRITORY_BASED;create=true';
Note for DERBY-3347
Summary of Change
A bug that could cause unrecoverable database corruption has been fixed.
Symptoms Seen by Applications Affected by Change
A bug that could cause database corruption was introduced in the 10.3 codeline and affects the following releases:
- Apache Derby 10.3.1.4
- Apache Derby 10.3.2.1
Users who are hit by this bug may experience exceptions at various times during execution of SQL statements, booting or shutdown of a database, or during checkpointing. It may result in a number of different error messages, including any of these:
ERROR XSDB3: Container information cannot change once written: was 0, now 80 ERROR XSDG1: Page Page(1039,Container(0, 5856)) could not be written to disk, please check if disk is full. ERROR XSDG2: Invalid checksum on Page Page(0,Container(0, 1313)) ERROR XSDG3: Meta-data for Container org.apache.derby.impl.store.raw.data.RAFContainer4@1afb0c7 could not be accessed ERROR XSLA1: Log Record has been sent to the stream, but it cannot be applied to the store (Object null). This may cause recovery problems also.Incompatibilities with Previous Release
None.
Rationale for Change
Database corruption is bad.
Application Changes Required
No changes are required. However, since the database corruption may go unnoticed for a while, users may want to check the consistency of their databases after upgrading Derby. The process is described on the following wiki page: http://wiki.apache.org/db-derby/DatabaseConsistencyCheck . If a corruption is detected, restoring the database from backup is the only reliable way to recover.
Note for DERBY-3327
Summary of Change
The effect of setting the current default schema (
SET SCHEMA schemaname
) inside nested connection of a stored procedure or function has been changed to comply with SQL standard semantics.Previously, setting the schema in a nested connection would have an effect on the current default schema also in the SQL connection of the caller. The SQL standard requires that the value of the default current schema at the time of the call be reestablished when the call is completed. This is now implemented.
If the current schema is dropped by a stored procedure or function, the current schema as well as the saved values of the callers will be reset to the initial default schema of the root connection.
Symptoms Seen by Applications Affected by Change
The current default schema of a connection which calls a stored procedure or function which sets the current schema may be different after the return.Rationale for Change
SQL standard compliance.
Application Changes Required
If the application relies on a stored procedure to change the current default schema, this must now be done in the calling connection instead.
Note for DERBY-3319
Summary of Change
Exception is thrown when connection with uncommitted operations is closed.
Symptoms Seen by Applications Affected by Change
In the previous release, applications could close a connection obtained from Derby's implementations of
javax.sql.DataSource
orjavax.sql.ConnectionPoolDataSource
, even if the connection had uncommitted operations. Now, Derby raises anSQLException
when an attempt to close a connection with an active transaction is made. The exception will have SQLState 25001, and its message will say the following:java.sql.SQLException: Cannot close a connection while a transaction is still active.Incompatibilities with Previous Release
Applications that close connections with active transactions now fail.
Rationale for Change
The previous behaviour was not consistent with the behaviour of connection objects obtained from
java.sql.DriverManager
, which already raise an exception in such a situation. The previous behaviour could also cause resource leaks because there is no way to free the resources held by an active transaction once its connection object has been closed.Application Changes Required
Users must call
commit()
orrollback()
, or use auto-commit, before attempting to close a connection with uncommitted operations.
Note for DERBY-2351
Summary of Change
An ORDER BY clause of a DISTINCT query which specifies to order by a column which was not in the DISTINCT list is now rejected, because the intent of the query is ambiguous. Previously, Derby instead produced non-distinct results. Also, an ORDER BY clause which specifies a table-name-qualified column alias is now rejected as invalid, where previously it was accepted.
Symptoms Seen by Applications Affected by Change
New rules for DISTINCT and ORDER BY
Applications which specify certain combinations of SELECT DISTINCT with ORDER BY will now receive an error message, whereas formerly such applications received non-distinct results.
As an example, take the following:
create table person (name varchar(10), age int);
insert into person values ('John', 10);
insert into person values ('John', 30);
insert into person values ('Mary', 20);
SELECT DISTINCT name FROM person ORDER BY age;
The query above is now rejected, with the error message:
ERROR 42879: The ORDER BY clause may not contain column 'AGE', since the query specifies DISTINCT and that column does not appear in the query result.If the AGE column is included in the DISTINCT list in the above query, there is no ambiguity
New column alias rules
Applications which specify a column alias for a column in the SELECT statement, and which specify an ORDER BY clause which specifies that column alias qualified by the table name, will now receive an error indicating that the ORDER BY clause is invalid.
As an example, take the following:
create table t1 (i int, j int);
select t1.id as idcolumn1, t1.id as idcolumn2 from t1 order by t1.idcolumn1, t1.idcolumn2;This query is now rejected, as there is no column named 'idcolumn1' in table 't1'. The error message is:
ERROR 42X04: Column 'T1.IDCOLUMN1' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE statement then 'T1.IDCOLUMN1' is not a column in the target table.Valid forms of the query above are:
select t1.id as idcolumn1, t1.id as idcolumn2 from t1 order by idcolumn1, idcolumn2;
or
select t1.id as idcolumn1, t1.id as idcolumn2 from t1 order by t1.id, t1.id;
Rationale for Change
When the query ambiguously specifies both DISTINCT and ORDER BY, Derby was unsure whether to return the rows properly ordered, but non-distinct, or to return a distinct set of rows, but in an unknown order. Since no clear resolution of the ambiguity could be found, we chose instead to reject the query.
The rules for resolving column references in ORDER BY clauses have been enhanced to consider column aliases and column names more fully. Derby now uses different resolution rules depending on whether the ORDER BY column reference is table.column, or just column:
- if the table name is provided, we match against the underlying table name, and don't consider any aliases
- if the table name is NOT provided, we first match against the alias name, if present, and if no alias name matches then we match against the underlying source column name.
Application Changes Required
A query which specifies ordering by a non-distinct column should instead include the ORDER BY column in the DISTINCT list, to resolve the ambiguity about which values of that column should be used to distinctly identify the resulting rows.
A query which specifies table-name.alias-name should be rewritten to specify either simply alias-name, or table-name.column-name.
Note for DERBY-2085
Summary of Change
Derby has improved the error message which is issued when an invalid column reference is found in a grouped query. Derby now issues message 42Y36 instead of 42Y30. Furthermore, the wording of message 42Y36 has been expanded to further explain the behavior.
Consider the table
CREATE TABLE t (i INT, j INT)
and consider the following query:
SELECT i FROM t GROUP BY i ORDER BY j
In Derby releases 10.2, 10.3, and 10.4, this query gives the following error message:
ERROR 42Y30: The SELECT list of a grouped query contains at least one invalid expression. If a SELECT list has a GROUP BY, the list may only contain valid grouping expressions and valid aggregate expressions.
This is misleading since there is no invalid expression in the SELECT list. It is the ORDER BY clause that is wrong. In Derby 10.5, this query will now give the following error message, which is a revised version of the message issued by 10.1 (that is, Derby 10.5 behaves more like Derby 10.1 in this respect):
ERROR 42Y36: Column reference 'J' is invalid, or is part of an invalid expression. For a SELECT list with a GROUP BY, the columns and expressions being selected may only contain valid grouping expressions and valid aggregate expressions.
Symptoms Seen by Applications Affected by Change
An application which issued a grouped select statement with an invalid column reference in the order by clause may now receive a different error message than it did in 10.4.
Rationale for Change
We feel that message 42Y36 is more helpful to the user in explaining the problem with the invalid query.
Note for DERBY-1062
Summary of Change
Applications will see a different error code and message when they attempt to call SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE on a table that does not exist.
Symptoms Seen by Applications Affected by Change
Applications will see a different error code and message when they attempt to call SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE on a table that does not exist. The new error code and message will be ERROR 42Y55: 'ALTER TABLE' cannot be performed on 'MissingTableName' because it does not exist. The old error code and message used to be ERROR 42X05: Table/View 'MissingTableName' does not exist.
Incompatibilities with Previous Release
The SQL exception code in the previous release was 42X05 and error message was "Table/View 'MissingTableName' does not exist."
Rationale for Change
To avoid duplication of code, the error handling is now done by a generic routine in ALTER TABLE rather than a routine specific to SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE. This change to use ALTER TABLE code to implement SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE was made in 10.5 release and that is the reason behind the change in error code behavior.
Application Changes Required
If the application is looking for 42X05, it should be changed to look for 42Y55.
Note for DERBY-48
Summary of Change
In Derby, a user's initial default schema is named the same as the user name, or APP if a user is not provided at connect time. This schema is implicitly auto-created the first time a schema object is created in that schema.
Previously, this auto-creation would be performed as part of the user transaction. This would sometimes lead to locking issues as described in this issue. With this change, the auto-creation is now performed and committed immediately in a separate sub-transaction.
Symptoms Seen by Applications Affected by Change
The initial default schema will be present in cases where it previously would not yet have been created: If the user transaction that creates a schema object leading to auto-creation of the initial default schema rolls back for some reason after having created the schema, up till now the auto-creation of the initial default schema would be rolled back as well. Since it is now created and committed in a sub-transaction, the schema creation will not be rolled back: the default schema will persist.
Incompatibilities with Previous Release
Most applications should not be impacted by this change, but there are some corner cases as described below:
If the application tests for the existence of the initial default schema by querying Derby system tables, the results could now be different than in earlier releases, if the test is made after a rollback as described in the previous section.
Since the initial default schema will now potentially exist in cases where it would previously not exist, schema operations may be impacted, e.g. where before a DROP SCHEMA <default schema name> RESTRICT would fail due to it not yet existing, it could now work (if empty), depending on when the drop attempt is made.
Rationale for Change
Implicit schema creation is now performed in its own transaction to avoid deadlocks with other connections accessing the same schema.
Doing this is a separate transaction avoids holding dictionary locks longer than necessary, cf. DERBY-48 and thus reduces the chance for deadlocks.
Application Changes Required
Verify that the application code does not rely on the initial default schema being absent after a rollback.
Derby release 10.5.1.1 was built using the following environment:
- Branch - Source code came from the 10.5 branch.
- Machine - Microsoft Windows XP Professional Version 2002 Service Pack 2.
- Ant - Apache Ant version 1.7.0 compiled on December 13 2006.
- JDK 1.4 - Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2) Classic VM (build 1.4.2, J2RE 1.4.2 IBM Windows 32 build cn142-20061124 (SR7)
- Java 6 - Java(TM) SE Runtime Environment (build pwi3260sr3-20081106_07(SR3)) IBM J9 VM (build 2.4, J2RE 1.6.0 IBM J9 2.4 Windows XP x86-32 jvmwi3260-20081105_25433
- Compiler - The 1.6 compiler was used to compile all classes, but java 1.4.2 classes were used to compile appropriate 1.4.2 support
- JSR 169 - J2ME support was built using IBM's j9 jvm from WEME6.1