These notes describe the difference between Apache Derby release 10.7.1.1 and the preceding release 10.6.2.1.
The most up to date information about Derby releases can be found on the Derby download page.
Apache Derby is a pure Java relational database engine using standard SQL and JDBC as its APIs. More information about Derby can be found on the Apache web site. Derby functionality includes:
- Embedded engine with JDBC drivers
- Network Server
- Network client JDBC drivers
- Command line tools: ij (SQL scripting), dblook (schema dump) and sysinfo (system info)
This is a feature release. The following new features were added:
- Definer's rights - Procedures and functions can now run with their creators' privileges, rather than with the current user's permissions.
- BOOLEAN data type - Boolean is now a legal data type for columns, routine arguments, and function return values.
- Table truncation - A new TRUNCATE TABLE command drops all rows in a table quickly.
- Query plan browsing - A new PlanExporter tool helps developers visualize query plans better.
- Unicode database names - Remote clients can now use database names which include Unicode characters outside the ASCII codeset.
The following issues are addressed by Derby release 10.7.1.1. These issues are not addressed in the preceding 10.6.2.1 release.
Issue Id Description DERBY-4911 restoreIntrFlagIfSeen may throw ShutdownException causing confusing console stack trace at server shutdown DERBY-4908 Instability in CheckConstraintTest.testBuiltInFunctions DERBY-4904 Plan exporter doesn't work if XPLAIN schema has special characters DERBY-4903 Plan exporter tool produces broken output if query contains less-than operator DERBY-4902 Encoding problem in output from plan exporter DERBY-4899 Refactor access to primary structures in AlterTableConstantAction DERBY-4898 testGetURL test fails in nightly runs. DERBY-4895 Thread local TableDescriptor#referencedColumnMap is not removed from app threads when Derby is shut down DERBY-4893 Allow Java 5 language constructs in build code DERBY-4892 Unsafe use of BigDecimal constructors DERBY-4891 Enable testing of boolean type in ParameterMappingTest DERBY-4890 Different string to boolean conversion on embedded and client DERBY-4889 Different byte to boolean conversion on embedded and client DERBY-4888 ClassCastException when calling boolean meta-data functions with 10.7 client against 10.6 server DERBY-4885 Reduce lock timeout in SavepointJdbc30Test DERBY-4884 DatabasePropertyTestSetup cannot change static properties in encrypted databases DERBY-4881 Deadlock accessing SYS.SYSSTATISTICS DERBY-4880 Unchecked warnings in JMXManagementService and VTI demos DERBY-4879 Lookahead warning from ij.jj DERBY-4878 Silence sanity info message when running "ant -q" DERBY-4877 Unchecked warnings in jdbc4 test suite DERBY-4876 Deprecation warnings in VTITemplate DERBY-4873 NullPointerException in testBoundaries with ibm jvm 1.6 DERBY-4870 Reference Manual map file issues with out-of-order/duplicate topics DERBY-4868 Remove Types.BIT/Types.BOOLEAN switch in JVMInfo DERBY-4867 Update BaseTestCase.isSunJVM() to accept Oracle DERBY-4866 Simplify SYSCS_BULK_INSERT with IdUtil.mkQualifiedName() DERBY-4865 Remove Maven artifact history section from maven2/README.txt DERBY-4860 RunSuite on derbylang and other subsuites fails with java.lang.NoClassDefFoundError because of prepended ^ DERBY-4859 Missing doPrivileged in DatabaseClasses.WriteClassFile() DERBY-4858 Remove deprecated code for generating CHANGES.html DERBY-4857 Utilize the SOAP API to fetch JIRA issue list for release notes generation DERBY-4840 Reference topic example elements lack titles DERBY-4836 many failures with IBM 1.5 and 1.6 on windows after test InternationalConnectTest DERBY-4835 Trigger plan does not recompile with upgrade from 10.5.3.0 to 10.6.1.0 causing java.lang.NoSuchMethodError DERBY-4829 dblook fails if there is a column privilege on a table whose name contains a quote DERBY-4828 Import fails if column names contain double quotes DERBY-4827 Modify the documentation for the 10.7 release regarding the UTF-8 CCSID manager DERBY-4826 Documentation tasks for new minor release of Derby DERBY-4825 Assert failure in LargeDataLocksTest.testGetCharacterStream() because of wrong number of locks DERBY-4822 DropSchemaConstantAction: could reuse the current connection provided by the available activation object DERBY-4815 Override mechanism for modules.properties works backwards DERBY-4812 ReplicationMessageTransmit run does not unwrap PrivilegedActionException which can lead to failure of replicationTests.ReplicationRun_Local_StateTest_part1_1 DERBY-4810 setTimestamp() methods don't agree on trailing zeros DERBY-4806 SysinfoCPCheckTest fail on different Windows platforms on 10.6.2.0 release candidate DERBY-4804 Make database used in store.OSReadOnlyTest fully read-only DERBY-4803 Sequences do not work in INSERT/SELECT DERBY-4802 Document the TRUNCATE command DERBY-4799 IllegalArgumentException when generating error message on server DERBY-4796 Missing escape for apostrophes in many messages DERBY-4791 LIKE operator optimizations and concatenation DERBY-4786 Shutdown command without username and password should work with mixed client and network server releases. DERBY-4781 suites.All fails completely with Failed to invoke suite():java.lang.reflect.InvocationTargetException because of XplainStatisticsTest DERBY-4777 Different behavior in Client and Embedded modes when update on an invalid cursor DERBY-4774 Investigate character set and encoding issues of PlanExporter Tool DERBY-4772 Data truncation error with XPLAIN-functionality enabled DERBY-4767 Detailed prompt for Error XCL16 is different between Client and Embed DERBY-4765 Convert "org.apache.derbyTesting.functionTests.tests.derbynet.DerbyNetNewServer" to junit. DERBY-4764 Files with missing ASF license headers DERBY-4760 test failure in derbynet/testProtocol.java with ibm jvms DERBY-4758 Adding documentation about the PlanExporter Tool DERBY-4756 Silence deprecation warnings for JDBC 4 stubs DERBY-4755 DERBY-4601 added a new message to derby.log during engine shutdown and it changed the database shutdown message. Need to update docs with that information DERBY-4752 CheapDateFormatter returns incorrect and invalid date strings DERBY-4746 (Server) Implement UTF8 support on DRDA DERBY-4743 Attempt to access a DECLARE GLOBAL TEMPORARY TABLE after commit in an XA transaction in Network server fails. DERBY-4738 correct error prompt for nonpositive length in Blob.setBytesX() DERBY-4737 Document the BOOLEAN datatype DERBY-4732 Release system resources in CanonTestCase thoroughly DERBY-4730 Add BOOLEAN to the list of datatypes returned by DatabaseMetaData.getTypeInfo() DERBY-4723 Using an instance lock to protect static shared data in EmbedPooledConnection DERBY-4720 Allow boolean-valued expressions in select lists DERBY-4718 Correct wrong main() method in LocalizedDisplayScriptTest and LocalizedAttributeScriptTest DERBY-4716 Enable BOOLEAN typed columns DERBY-4714 Clean up some edge-case uses of the BOOLEAN keyword DERBY-4711 Hung thread after another thread is interrupted DERBY-4709 Create test that parse client trace file to detect round-trips for Derby-4653 DERBY-4707 convert ShutdownDatabase.java into junit DERBY-4704 Incorrect nullability when casting non-nullable VARCHAR to BOOLEAN DERBY-4703 Make it possible for table functions to return BOOLEAN columns DERBY-4696 Remove the outdated content about "frameworks subdirectory" in "getstartderby" DERBY-4695 Internal assignment of tablenumer, columnnumber looks wrong in query tree, although no ill effects are seen. DERBY-4692 Unions between BOOLEAN and non-BOOLEAN datatypes should be rejected DERBY-4690 Too few arguments when generating AM_RECORD_NOT_FOUND error DERBY-4685 Dead/unreachable code in OpenConglomerate.lockPositionForWrite() DERBY-4684 Correct the implicit casting of other types to BOOLEAN DERBY-4680 Add documentation for routines running with definer's rights DERBY-4674 BooleanValuesTest fails if Xalan is not available DERBY-4667 BootLockTest.testBootLock() sometimes fails with connection refused. DERBY-4665 Unidiomatic error handling in TimestampArithTest DERBY-4658 Allow explicit casts of string values to BOOLEAN DERBY-4653 Avoid unnecessary round-trip for commit in the client driver DERBY-4626 Timestamp truncated when converted to string with explicit calendar DERBY-4625 TIMESTAMP function doesn't accept nanoseconds DERBY-4624 Broken logic for avoiding testing across midnight in TimestampArithTest DERBY-4613 Make client driver treat BOOLEAN columns the same way as the embedded driver does. DERBY-4601 Shutting down just a single database should log a different message than shutting down the system DERBY-4587 Add tools for improved analysis and understanding of query plans and execution statistics DERBY-4583 TRUE by itself is not accepted in WHERE DERBY-4577 An expanding update fails with an nospc.U error. DERBY-4551 Allow database user to execute stored procedures with same permissions as database owner and/or routine definer DERBY-4531 Client setCharacterStream closes its Reader argument stream in finalizer DERBY-4459 Verification error at execute-time when an outer function which takes a primitive arg is wrapped around an inner function which returns the corresponding Java wrapper object DERBY-4449 ArrayIndexOutOfBoundsException when inserting DEFAULT into unspecified column DERBY-4371 Non-selected columns for SELECT DISTINCT allowed in ORDER BY clause if ordered by expression DERBY-4343 ASSERT FAILED calling setTransactionIsolation checking isolation_ == level on pooled connection DERBY-4315 Attempt to reuse streams in client on insert gives protocol exception and inserts wrong data DERBY-4314 With derby client setTransactionIsolation executes and commits even if isolation has not changed DERBY-4241 Improve transition from read-only to writable Clob representation DERBY-4083 BTreeScan.delete() throws AM_RECORD_NOT_FOUND if record is found DERBY-4067 ClientConnectionPoolDataSource.getPooledConnection and ClientXADataSource.getXAConnection ignore connection attributes DERBY-4052 SQLChar.copyState() doesn't copy localeFinder DERBY-3953 VIEWS which invoke user-coded functions may return different results depending on who SELECTs from them DERBY-3944 CHECK constraints involving user-coded functions may return different results depending on who performs the triggering INSERT/UPDATE DERBY-3898 Blob.setBytes differs between embedded and client driver when the specified length is invalid DERBY-3801 Convert "org.apache.derbyTesting.functionTests.tests.lang.holdCursorIJ.sql" to junit. DERBY-3727 Improve error information reported when encountering "ERROR XSDB3: Container information cannot change once written: was 103, now 80" DERBY-3644 NetworkServerControl.ping() on trunk against 10.3.2.1 and older servers give 'DRDA_InvalidReplyHeader2.S:Invalid reply header from network server: Invalid string . Plaintext connection attempt to an SSL enabled server?' DERBY-3173 Removed cached String objects from SQLDate, SQLTime and SQLTimestamp DERBY-2785 ij "describe" built in command cannot describe a table named "run" DERBY-2419 Tighten encapsulation of state in TestConfiguration DERBY-2156 message XSDB8 and 42Y32 have references to db2j DERBY-2011 DROP INDEX, TRIGGER and VIEW contain examples in the syntax section in the reference manual. DERBY-1963 Routine parameter names displayed by dblook are not escaped. DERBY-1958 improve XSDG3 error to print container, actual i/o operation, and file name. DERBY-1938 Add support for setObject(<arg>, null) DERBY-1511 SELECT clause without a WHERE, causes an Exception when extracting a Blob from a database DERBY-1482 Update triggers on tables with blob columns stream blobs into memory even when the blobs are not referenced/accessed. DERBY-1439 Investigate removing the antiGC thread in embedded Derby DERBY-1425 testProtocol hangs with 10.1 client talking to 10.2 server DERBY-499 Expose BOOLEAN datatype to end users DERBY-462 DatabaseMetaData method supportsConvert() returns true
Compared with the previous release (10.6.2.1), Derby release 10.7.1.1 introduces the following new features and incompatibilities. These merit your special attention.
Note for DERBY-4786: A shutdown request with no user credentials to a 10.3 (or earlier) server from a newer client will give multiple messages on both the server and client sides.
Note for DERBY-4777: When a network client attempts to update through an invalid cursor, Derby now throws 'ERROR 42X30' instead of 'ERROR XJ202'.
Note for DERBY-4772: Column types in the XPLAIN tables changed to accommodate more data.
Note for DERBY-4577: UPDATE statements should not raise the following error: "ERROR nospc: nospc.U"
Note for DERBY-4314: Derby client Connection.setTransactionIslation will not commit if isolation level does not change.
Note for DERBY-1511: Queries will not bulk fetch rows with large object columns if the cursor is holdable.
Note for DERBY-4786
Summary of Change
A shutdown request with no user credentials to a 10.3 (or earlier) server from a newer client will give multiple messages on both the server and client sides.
Symptoms Seen by Applications Affected by Change
A shutdown request from a 10.4 (or higher) client with no user credentials to a 10.3 (or earlier) server shows the following messages on the server side. (Note that the version info about the product changes depending on the release being used for the server.)
java.lang.Throwable: DRDA_UnknownProtocol.S, 2 Execution failed because of a Distributed Protocol Error: DRDA_Proto_SYNTAXRM;CODPNT arg = 0; Error Code Value = 1. Plaintext connection attempt from an SSL enabled client? Apache Derby Network Server - 10.3.3.1 - (1) shutdown at {2}Messages on the client side appear as follows. (Note that the version info about the product changes depending on the release being used for the client.)
Invalid reply header from network server: Invalid string . Plaintext connection attempt to an SSL enabled server? Apache Derby Network Server - 10.7.1.0 - (1) shutdownRationale for Change
10.3 (and earlier) versions do not support shutdown commands with credentials. The credential-bearing shutdown command was introduced in 10.4 and requires network protocol level 2. 10.3 and earlier versions understand protocol level 1 and do not recognize the credential-bearing shutdown command. Because of this, a 10.4 (or higher) client can't shutdown a 10.3 (or lower) server. With DERBY-4786, a 10.7 client will first try the shutdown at protocol level 2. If that does not succeed, then the 10.7 client will send the shutdown at protocol level 1. This fix has been backported to the 10.4, 10.5, and 10.6 branches also. Future releases on those branches will enjoy this ability to successfully shutdown 10.3 (and earlier) servers.
Note for DERBY-4777
Summary of Change
When a network client attempts to update through an invalid cursor, Derby now throws 'ERROR 42X30' instead of 'ERROR XJ202'.
Symptoms Seen by Applications Affected by Change
In previous releases, Derby threw 'ERROR XJ202' when a network client tried to update through a closed or nonexistent cursor. For a cursor named "JDK4", the detailed message was "ERROR XJ202: Invalid cursor name 'JDK4'." If you are running 10.7, however, Derby now throws 'ERROR 42X30' instead, and the detailed message is "ERROR 42X30: Cursor 'JDK4' not found. Verify that autocommit is OFF."
From now on, DERBY performs identically in Client and Embedded modes when you attempt to update through a closed or nonexistent cursor.
Incompatibilities with Previous Release
Applications may fail if they expect 'ERROR XJ202' to be thrown in this situation.
Rationale for Change
This change was made so that the Client and Embedded behavior would be similar.
Application Changes Required
If applications need to track updates through invalid cursors, those applications should look for 'ERROR 42X30' regardless of whether they run in Client or Embedded mode.
Note for DERBY-4772
Summary of Change
Column types in the XPLAIN tables changed to accommodate more data.
Symptoms Seen by Applications Affected by Change
None.
Incompatibilities with Previous Release
None, but keeping the old table definitions for the XPLAIN tables may result in data truncation errors (see DERBY-4772 and DERBY-4673).
Rationale for Change
Amount of recorded data was too large to fit into the old column definitions.
Application Changes Required
It is recommended to drop existing XPLAIN tables and have them recreated with the new table definitions. This can be done by disabling the XPLAIN feature, dropping the tables, and then enable the XPLAIN feature again. Alternatively, if you want to keep existing data, specify a different schema to save XPLAIN data in.
Note for DERBY-4577
Summary of Change
UPDATE statements should not raise the following error: "ERROR nospc: nospc.U"
Symptoms Seen by Applications Affected by Change
An UPDATE of a row which spans multiple pages can fail, raising "ERROR nospc: nospc.U". This error should never be returned to the client. The error is very timing and data dependent, and has been encountered in only a few applications.
Rationale for Change
UPDATES of rows may fail in some cases, returning this error which clients should never see.
Application Changes Required
After applying the fix, clients should not see this error. To ensure that updates of existing rows no longer see this error, run the SYSCS_UTIL.SYSCS_COMPRESS_TABLE() system procedure on your tables. You may want to defensively compress all of your tables. Alternatively, it is also ok to wait and see if the error occurs and then only compress the tables that encounter the error.
Note for DERBY-4314
Summary of Change
Derby client Connection.setTransactionIslation will not commit if isolation level does not change.
Symptoms Seen by Applications Affected by Change
Applications relying on Connection.setTransactionIsolation to commit transactions, may see either errors related to transactions in progress or transactions that fail to commit.
Incompatibilities with Previous Release
Before 10.6.1.0, derby client Connection.setTransactionIsolation would commit even if isolation has not changed. Starting with 10.6.1.0, derby client Connection.setTransactionIsolatin does not commit if the isolation has not changed.
Rationale for Change
The change was made to client to match behavior of the embedded driver and to improve client performance.
Application Changes Required
Applications should not rely on transactions change based on calling Connection.setTransactionIsolation. An explicit commit should be called to ensure transactions commit.
Note for DERBY-1511
Summary of Change
Queries will not bulk fetch rows with large object columns if the cursor is holdable.
Symptoms Seen by Applications Affected by Change
Queries that perform full table scans in order to fetch relatively small BLOB or CLOB values, may need longer time to complete than they did in previous releases. This only affects queries that are executed on a
Statement
(orPreparedStatement
orCallableStatement
) that has theResultSet.HOLD_CURSORS_OVER_COMMIT
flag set.Rationale for Change
The bulk fetching that happened in previous releases, was not a safe optimization if the fetched rows contained large object columns (BLOB or CLOB) and the cursor was holdable. If the transaction was committed in the middle of a scan, the pre-fetched large objects cached inside the scan would be closed, and they would not be valid for use when they were later returned to the application. To prevent certain ways to access large objects from failing, the optimization was disabled for the cases where it was not known to be safe.
Application Changes Required
The queries affected by this change will run with no performance degradation compared to previous releases if the application can be rewritten to use non-holdable cursors. This can be achieved by calling
Connection.setHoldability()
with theResultSet.CLOSE_CURSORS_AT_COMMIT
flag before creating the statement instance, or by supplying that flag toConnection
'screateStatement()
,prepareStatement()
orprepareCall()
method in theresultSetHoldability
parameter.Alternatively, if all the BLOB and CLOB values in the table in question are small enough to fit in a VARCHAR FOR BIT DATA column or in a VARCHAR column, one may change the type of the LOB columns to one of these other data types, and the query will again be allowed to use the bulk fetch optimization.
Derby release 10.7.1.1 was built using the following environment:
- Branch - Source code came from the 10.7 branch.
- Machine - Mac OS X 10.5.8.
- Ant - Apache Ant version 1.7.1 compiled on June 27 2008.
- JDK 1.4 - Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_22-b02-329).
- Java 6 - Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125).
- Compiler - The 1.6.0_17-b04-248-9M3125 javac was used to compile all classes.
- JSR 169 - J2ME support was built using libraries from phoneME Advanced Milestone Release 2.
It is essential that you verify the integrity of the downloaded files using the PGP and MD5 signatures. MD5 verification ensures the file was not corrupted during the download process. PGP verification ensures that the file came from a certain person.
The PGP signatures can be verified using PGP or GPG. First download the Apache Derby KEYS as well as the
asc
signature file for the particular distribution. It is important that you get these files from the ultimate trusted source - the main ASF distribution site, rather than from a mirror. Then verify the signatures using ...% pgpk -a KEYS % pgpv db-derby-X.Y.tar.gz.asc or % pgp -ka KEYS % pgp db-derby-X.Y.tar.gz.asc or % gpg --import KEYS % gpg --verify db-derby-X.Y.tar.gz.ascTo verify the MD5 signature on the files, you need to use a program called
md5
ormd5sum
, which is included in many unix distributions. It is also available as part of GNU Textutils. Windows users can get binary md5 programs from here, here, or here.We strongly recommend that you verify your downloads with both PGP and MD5.