Release Notes - Hive - Version 1.1.0 ** Sub-task * [HIVE-7073] - Implement Binary in ParquetSerDe * [HIVE-8121] - Create micro-benchmarks for ParquetSerde and evaluate performance * [HIVE-8122] - Make use of SearchArgument classes for Parquet SERDE * [HIVE-8130] - Support Date in Avro * [HIVE-8131] - Support timestamp in Avro * [HIVE-8362] - Investigate flaky test parallel.q * [HIVE-8651] - CBO: sort column changed in infer_bucket_sort test * [HIVE-8707] - Fix ordering differences due to Java 8 HashMap function * [HIVE-8718] - Refactoring: move mapLocalWork field from MapWork to BaseWork * [HIVE-8773] - Fix TestWebHCatE2e#getStatus for Java8 * [HIVE-8862] - Fix ordering diferences on TestParse tests due to Java8 * [HIVE-8922] - CBO: assorted date and timestamp issues * [HIVE-8923] - HIVE-8512 needs to be fixed also for CBO * [HIVE-8936] - Add SORT_QUERY_RESULTS for join tests that do not guarantee order * [HIVE-8962] - Add SORT_QUERY_RESULTS for join tests that do not guarantee order #2 * [HIVE-9030] - CBO: Plans with comparison of values with different types * [HIVE-9033] - Fix ordering differences due to Java8 (part 2) * [HIVE-9034] - CBO: type change in literal_ints.q * [HIVE-9035] - CBO: Disable PPD when functions are non-deterministic (ppd_random.q - non-deterministic udf rand() pushed above join) * [HIVE-9043] - HiveException: Conflict on row inspector for {table} * [HIVE-9066] - temporarily disable CBO for non-deterministic functions * [HIVE-9104] - windowing.q failed when mapred.reduce.tasks is set to larger than one * [HIVE-9109] - Add support for Java 8 specific q-test out files * [HIVE-9127] - Improve CombineHiveInputFormat.getSplit performance * [HIVE-9133] - CBO (Calcite Return Path): Refactor Semantic Analyzer to Move CBO code out * [HIVE-9153] - Perf enhancement on CombineHiveInputFormat and HiveInputFormat * [HIVE-9161] - Fix ordering differences on UDF functions due to Java8 * [HIVE-9181] - Fix SkewJoinOptimizer related Java 8 ordering differences * [HIVE-9193] - Fix ordering differences due to Java 8 (Part 3) * [HIVE-9194] - Support select distinct * * [HIVE-9200] - CBO (Calcite Return Path): Inline Join, Properties * [HIVE-9206] - Fix Desc Formatted related Java 8 ordering differences * [HIVE-9222] - Fix ordering differences due to Java 8 (Part 4) * [HIVE-9224] - CBO (Calcite Return Path): Inline Table, Properties * [HIVE-9239] - Fix ordering differences due to Java 8 (Part 5) * [HIVE-9241] - Fix TestCliDriver.testCliDriver_subquery_multiinsert * [HIVE-9257] - Merge from spark to trunk January 2015 * [HIVE-9259] - Fix ClassCastException when CBO is enabled for HOS [Spark Branch] * [HIVE-9264] - Merge encryption branch to trunk * [HIVE-9292] - CBO (Calcite Return Path): Inline GroupBy, Properties * [HIVE-9315] - CBO (Calcite Return Path): Inline FileSinkOperator, Properties * [HIVE-9352] - Merge from spark to trunk (follow-up of HIVE-9257) * [HIVE-9409] - Avoid ser/de loggers as logging framework can be incompatible on driver and workers * [HIVE-9410] - ClassNotFoundException occurs during hive query case execution with UDF defined [Spark Branch] * [HIVE-9428] - LocalSparkJobStatus may return failed job as successful [Spark Branch] * [HIVE-9431] - CBO (Calcite Return Path): Removing AST from ParseContext * [HIVE-9434] - Shim the method Path.getPathWithoutSchemeAndAuthority * [HIVE-9444] - CBO (Calcite Return Path): Rewrite GlobalLimitOptimizer * [HIVE-9449] - Push YARN configuration to Spark while deply Spark on YARN[Spark Branch] * [HIVE-9450] - [Parquet] Check all data types work for Parquet in Group By operator * [HIVE-9477] - No error thrown when global limit optimization failed to find enough number of rows [Spark Branch] * [HIVE-9487] - Make Remote Spark Context secure [Spark Branch] * [HIVE-9493] - Failed job may not throw exceptions [Spark Branch] ** Bug * [HIVE-1344] - error in select disinct * [HIVE-1654] - select distinct should allow column name regex * [HIVE-1869] - TestMTQueries failing on jenkins * [HIVE-3781] - Index related events should be delivered to metastore event listener * [HIVE-4009] - CLI Tests fail randomly due to MapReduce LocalJobRunner race condition * [HIVE-5536] - Incorrect Operation Name is passed to hookcontext * [HIVE-5631] - Index creation on a skew table fails * [HIVE-5664] - Drop cascade database fails when the db has any tables with indexes * [HIVE-5865] - AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8' * [HIVE-6165] - Unify HivePreparedStatement from jdbc:hive and jdbc:hive2 * [HIVE-6198] - ORC file and struct column names are case sensitive * [HIVE-6308] - COLUMNS_V2 Metastore table not populated for tables created without an explicit column list. * [HIVE-6421] - abs() should preserve precision/scale of decimal input * [HIVE-6468] - HS2 & Metastore using SASL out of memory error when curl sends a get request * [HIVE-6623] - Add "owner" tag to ptest2 created instances * [HIVE-6679] - HiveServer2 should support configurable the server side socket timeout and keepalive for various transports types where applicable * [HIVE-6683] - Beeline does not accept comments at end of line * [HIVE-6914] - parquet-hive cannot write nested map (map value is map) * [HIVE-7024] - Escape control characters for explain result * [HIVE-7069] - Zookeeper connection leak * [HIVE-7270] - SerDe Properties are not considered by show create table Command * [HIVE-7932] - It may cause NP exception when add accessed columns to ReadEntity * [HIVE-7951] - InputFormats implementing (Job)Configurable should not be cached * [HIVE-7997] - Potential null pointer reference in ObjectInspectorUtils#compareTypes() * [HIVE-8099] - IN operator for partition column fails when the partition column type is DATE * [HIVE-8182] - beeline fails when executing multiple-line queries with trailing spaces * [HIVE-8257] - Accumulo introduces old hadoop-client dependency * [HIVE-8266] - create function using statement compilation should include resource URI entity * [HIVE-8284] - Equality comparison is done between two floating point variables in HiveRelMdUniqueKeys#getUniqueKeys() * [HIVE-8295] - Add batch retrieve partition objects for metastore direct sql * [HIVE-8308] - Acid related table properties should be defined in one place and should be case insensitive * [HIVE-8317] - WebHCat pom should explicitly depend on jersey-core * [HIVE-8326] - Using DbTxnManager with concurrency off results in run time error * [HIVE-8330] - HiveResultSet.findColumn() parameters are case sensitive * [HIVE-8338] - Add ip and command to semantic analyzer hook context * [HIVE-8345] - q-test for Avro date support * [HIVE-8359] - Map containing null values are not correctly written in Parquet files * [HIVE-8374] - schematool fails on Postgres versions < 9.2 * [HIVE-8381] - Update hive version on trunk to 0.15 * [HIVE-8387] - add retry logic to ZooKeeperStorage in WebHCat * [HIVE-8448] - Union All might not work due to the type conversion issue * [HIVE-8450] - Create table like does not copy over table properties * [HIVE-8485] - HMS on Oracle incompatibility * [HIVE-8491] - Fix build name in ptest pre-commit message * [HIVE-8500] - beeline does not need to set hive.aux.jars.path * [HIVE-8512] - queries with star and gby produce incorrect results * [HIVE-8518] - Compile time skew join optimization returns duplicated results * [HIVE-8523] - Potential null dereference in DDLSemanticAnalyzer#addInputsOutputsAlterTable() * [HIVE-8532] - return code of "source xxx" clause is missing * [HIVE-8556] - introduce overflow control and sanity check to BytesBytesMapJoin * [HIVE-8564] - DROP TABLE IF EXISTS throws exception if the table does not exist. * [HIVE-8565] - beeline may go into an infinite loop when using EOF * [HIVE-8576] - Guaranteed NPE in StatsRulesProcFactory * [HIVE-8594] - Wrong condition in SettableConfigUpdater#setHiveConfWhiteList() * [HIVE-8600] - Add option to log explain output for query * [HIVE-8610] - Compile time skew join optimization doesn't work with auto map join * [HIVE-8611] - grant/revoke syntax should support additional objects for authorization plugins * [HIVE-8612] - Support metadata result filter hooks * [HIVE-8613] - percentile_approx raise a comparator error * [HIVE-8627] - Compute stats on a table from impala caused the table to be corrupted * [HIVE-8634] - HiveServer2 fair scheduler queue mapping doesn't handle the secondary groups rules correctly * [HIVE-8636] - CBO: split cbo_correctness test * [HIVE-8666] - hive.metastore.server.max.threads default is too high * [HIVE-8680] - Set Max Message for Binary Thrift endpoints * [HIVE-8693] - Separate out fair scheduler dependency from hadoop 0.23 shim * [HIVE-8706] - Table statistic collection on counter failed due to table name character case. * [HIVE-8708] - Add query id to explain log option * [HIVE-8715] - Hive 14 upgrade scripts can fail for statistics if database was created using auto-create * [HIVE-8720] - Update orc_merge tests to make it consistent across OS'es * [HIVE-8728] - Fix ptf.q determinism * [HIVE-8730] - schemaTool failure when date partition has non-date value * [HIVE-8736] - add ordering to cbo_correctness to make result consistent * [HIVE-8739] - handle Derby and Oracle errors with joins and filters in Direct SQL in a invalid-DB-specific path * [HIVE-8757] - YARN dep in scheduler shim should be optional * [HIVE-8762] - HiveMetaStore.BooleanPointer should be replaced with an AtomicBoolean * [HIVE-8784] - Querying partition does not work with JDO enabled against PostgreSQL * [HIVE-8791] - Hive permission inheritance throws exception S3 * [HIVE-8796] - TestCliDriver acid tests with decimal needs benchmark to be updated * [HIVE-8797] - Simultaneous dynamic inserts can result in "partition already exists" error * [HIVE-8803] - DESC SCHEMA is not working * [HIVE-8805] - CBO skipped due to SemanticException: Line 0:-1 Both left and right aliases encountered in JOIN 'avg_cs_ext_discount_amt' * [HIVE-8808] - HiveInputFormat caching cannot work with all input formats * [HIVE-8811] - Dynamic partition pruning can result in NPE during query compilation * [HIVE-8812] - TestMinimrCliDriver failure if run in the same command as TestHBaseNegativeCliDriver * [HIVE-8825] - SQLCompletor catches Throwable and ignores it * [HIVE-8827] - Remove SSLv2Hello from list of disabled protocols * [HIVE-8830] - hcatalog process don't exit because of non daemon thread * [HIVE-8845] - Switch to Tez 0.5.2 * [HIVE-8847] - Fix bugs in jenkins scripts * [HIVE-8848] - data loading from text files or text file processing doesn't handle nulls correctly * [HIVE-8850] - ObjectStore:: rollbackTransaction() needs to be looked into further. * [HIVE-8863] - Cannot drop table with uppercase name after "compute statistics for columns" * [HIVE-8866] - Vectorization on partitioned table throws ArrayIndexOutOfBoundsException when partitions are not of same #of columns * [HIVE-8869] - RowSchema not updated for some ops when columns are pruned * [HIVE-8870] - errors when selecting a struct field within an array from ORC based tables * [HIVE-8872] - Hive view of HBase range scan intermittently returns incorrect data. * [HIVE-8873] - Switch to calcite 0.9.2 * [HIVE-8874] - Error Accessing HBase from Hive via Oozie on Kerberos 5.0.1 cluster * [HIVE-8875] - hive.optimize.sort.dynamic.partition should be turned off for ACID * [HIVE-8876] - incorrect upgrade script for Oracle (13->14) * [HIVE-8877] - improve context logging during job submission via WebHCat * [HIVE-8879] - Upgrade derby version to address race candition * [HIVE-8880] - non-synchronized access to split list in OrcInputFormat * [HIVE-8881] - Receiving json "{"error":"Could not find job job_1415748506143_0002"}" when web client tries to fetch all jobs from webhcat where HDFS does not have the data. * [HIVE-8886] - Some Vectorized String CONCAT expressions result in runtime error Vectorization: Unsuported vector output type: StringGroup * [HIVE-8888] - Mapjoin with LateralViewJoin generates wrong plan in Tez * [HIVE-8889] - JDBC Driver ResultSet.getXXXXXX(String columnLabel) methods Broken * [HIVE-8890] - HiveServer2 dynamic service discovery: use persistent ephemeral nodes curator recipe * [HIVE-8891] - Another possible cause to NucleusObjectNotFoundException from drops/rollback * [HIVE-8893] - Implement whitelist for builtin UDFs to avoid untrused code execution in multiuser mode * [HIVE-8901] - increase retry attempt, interval on metastore database errors * [HIVE-8909] - Hive doesn't correctly read Parquet nested types * [HIVE-8914] - HDFSCleanup thread holds reference to FileSystem * [HIVE-8916] - Handle user@domain username under LDAP authentication * [HIVE-8917] - HIVE-5679 adds two thread safety problems * [HIVE-8926] - Projections that only swap input columns are identified incorrectly as identity projections * [HIVE-8938] - Compiler should save the transform URI as input entity * [HIVE-8944] - TestCompactor fails with IncompatibleClassChangeError * [HIVE-8947] - HIVE-8876 also affects Postgres < 9.2 * [HIVE-8948] - TestStreaming is flaky * [HIVE-8964] - Some TestMiniTezCliDriver tests taking two hours * [HIVE-8965] - Enhance PTest to kill all processes between tests and to report when a TEST*.xml file is not generated * [HIVE-8967] - Fix bucketmapjoin7.q determinism * [HIVE-8975] - Possible performance regression on bucket_map_join_tez2.q * [HIVE-8978] - Fix test determinism issue for qfile: smb_mapjoin_1.q etc * [HIVE-8990] - mapjoin_mapjoin.q is failing on Tez (missed golden file update) * [HIVE-9001] - Ship with log4j.properties file that has a reliable time based rolling policy * [HIVE-9003] - Vectorized IF expr broken for the scalar and scalar case * [HIVE-9006] - hiveserver thrift api version is still 6 * [HIVE-9011] - Fix parquet_join.q determinism * [HIVE-9024] - NullPointerException when starting webhcat server if templeton.hive.properties is not set * [HIVE-9025] - join38.q (without map join) produces incorrect result when testing with multiple reducers * [HIVE-9032] - Help for orcfiledump script does not reflect new options * [HIVE-9048] - Hive build failed on hadoop-1 after HIVE-8828. * [HIVE-9051] - TezJobMonitor in-place updates logs too often to logfile * [HIVE-9053] - select constant in union all followed by group by gives wrong result * [HIVE-9055] - Tez: union all followed by group by followed by another union all gives error * [HIVE-9060] - Fix child operator references after NonBlockingOpDeDupProc * [HIVE-9067] - OrcFileMergeOperator may create merge file that does not match properties of input files * [HIVE-9077] - Set completer in CliDriver is not working * [HIVE-9090] - Rename "Tez File Merge Work" to smaller name * [HIVE-9096] - GenericUDF may be left unclosed in PartitionPrune#visitCall() * [HIVE-9108] - Fix for HIVE-8735 is incorrect (stats with long paths) * [HIVE-9111] - Potential NPE in OrcStruct for list and map types * [HIVE-9112] - Query may generate different results depending on the number of reducers * [HIVE-9113] - Explain on query failed with NPE * [HIVE-9114] - union all query in cbo test has undefined ordering * [HIVE-9120] - Hive Query log does not work when hive.exec.parallel is true * [HIVE-9122] - Need to remove additional references to hive-shims-common-secure, hive-shims-0.20 * [HIVE-9126] - Backport HIVE-8827 (Remove SSLv2Hello from list of disabled protocols) to 0.14 branch * [HIVE-9129] - Migrate to newer Calcite snapshot, where ByteString is now in org.apache.calcite.avatica.util * [HIVE-9130] - vector_partition_diff_num_cols result is not updated after CBO upgrade * [HIVE-9131] - MiniTez optimize_nullscan test is unstable * [HIVE-9141] - HiveOnTez: mix of union all, distinct, group by generates error * [HIVE-9149] - Add unit test to test implicit conversion during dynamic partitioning/distribute by * [HIVE-9150] - Unrelated types are compared in GenTezWork#getFollowingWorkIndex() * [HIVE-9154] - Cache pathToPartitionInfo in context aware record reader * [HIVE-9155] - HIVE_LOCKS uses int instead of bigint hive-txn-schema-0.14.0.mssql.sql * [HIVE-9162] - stats19 test is environment-dependant * [HIVE-9166] - Place an upper bound for SARG CNF conversion * [HIVE-9168] - Vectorized Coalesce for strings is broken * [HIVE-9177] - Fix child operator references after NonBlockingOpDeDupProc (II) * [HIVE-9195] - CBO changes constant to column type * [HIVE-9197] - fix lvj_mapjoin.q diff in trunk * [HIVE-9199] - Excessive exclusive lock used in some DDLs with DummyTxnManager * [HIVE-9203] - CREATE TEMPORARY FUNCTION hangs trying to acquire lock * [HIVE-9205] - Change default tez install directory to use /tmp instead of /user and create the directory if it does not exist * [HIVE-9215] - Some mapjoin queries broken with IdentityProjectRemover with PPD * [HIVE-9221] - Remove deprecation warning for hive.metastore.local * [HIVE-9234] - HiveServer2 leaks FileSystem objects in FileSystem.CACHE * [HIVE-9242] - Many places in CBO code eat exceptions * [HIVE-9243] - Static Map in IOContext is not thread safe * [HIVE-9249] - java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.HiveVarcharWritable cannot be cast to org.apache.hadoop.hive.common.type.HiveVarchar when joining tables * [HIVE-9255] - Fastpath for limited fetches from unpartitioned tables * [HIVE-9296] - Need to add schema upgrade changes for queueing events in the database * [HIVE-9299] - Reuse Configuration in AvroSerdeUtils * [HIVE-9300] - Make TCompactProtocol configurable * [HIVE-9301] - Potential null dereference in MoveTask#createTargetPath() * [HIVE-9309] - schematool fails on Postgres 8.1 * [HIVE-9310] - CLI JLine does not flush history back to ~/.hivehistory * [HIVE-9316] - TestSqoop tests in WebHCat testsuite hardcode libdir path to hdfs * [HIVE-9317] - move Microsoft copyright to NOTICE file * [HIVE-9321] - Notification message size can be arbitrarily long, DbNotificationListener limits to 1024 * [HIVE-9330] - DummyTxnManager will throw NPE if WriteEntity writeType has not been set * [HIVE-9331] - get rid of pre-optimized-hashtable memory optimizations * [HIVE-9344] - Fix flaky test optimize_nullscan * [HIVE-9347] - Bug with max() together with rank() and grouping sets * [HIVE-9351] - Running Hive Jobs with Tez cause templeton to never report percent complete * [HIVE-9353] - make TABLE keyword optional in INSERT INTO TABLE foo... * [HIVE-9359] - Export of a large table causes OOM in Metastore and Client * [HIVE-9361] - Intermittent NPE in SessionHiveMetaStoreClient.alterTempTable * [HIVE-9366] - wrong date in description annotation in date_add() and date_sub() udf * [HIVE-9369] - fix arguments length checking in Upper and Lower UDF * [HIVE-9377] - UDF in_file() in WHERE predicate causes NPE. * [HIVE-9381] - HCatalog hardcodes maximum append limit to 1000. * [HIVE-9382] - Query got rerun with Global Limit optimization on and Fetch optimization off * [HIVE-9386] - FileNotFoundException when using in_file() * [HIVE-9390] - Enhance retry logic wrt DB access in TxnHandler * [HIVE-9393] - reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG * [HIVE-9396] - date_add()/date_sub() should allow tinyint/smallint/bigint arguments in addition to int * [HIVE-9401] - Backport: Fastpath for limited fetches from unpartitioned tables * [HIVE-9404] - NPE in org.apache.hadoop.hive.metastore.txn.TxnHandler.determineDatabaseProduct() * [HIVE-9414] - Fixup post HIVE-9264 - Merge encryption branch to trunk * [HIVE-9437] - Beeline does not add any existing HADOOP_CLASSPATH * [HIVE-9440] - Folders may not be pruned for Hadoop 2 * [HIVE-9441] - Remove call to deprecated Calcite method * [HIVE-9443] - ORC PPD - fix fuzzy case evaluation of IS_NULL * [HIVE-9445] - Revert HIVE-5700 - enforce single date format for partition column storage * [HIVE-9446] - JDBC DatabaseMetadata.getColumns() does not work for temporary tables * [HIVE-9448] - Merge spark to trunk 1/23/15 * [HIVE-9454] - Test failures due to new Calcite version * [HIVE-9462] - HIVE-8577 - breaks type evolution * [HIVE-9473] - sql std auth should disallow built-in udfs that allow any java methods to be called * [HIVE-9475] - HiveMetastoreClient.tableExists does not work * [HIVE-9476] - Beeline fails to start on trunk * [HIVE-9502] - Parquet cannot read Map types from files written with Hive <= 0.12 * [HIVE-9514] - schematool is broken in hive 1.0.0 * [HIVE-9519] - Bump up spark client connection timeout * [HIVE-9593] - ORC Reader should ignore unknown metadata streams * [HIVE-9610] - Continuation of HIVE-9438 - The standalone-jdbc jar missing some classes * [HIVE-9611] - Allow SPARK_HOME as well as spark.home to define sparks location * [HIVE-9621] - HiveServer2 http mode - embedded jetty should use SynchronousQueue * [HIVE-9646] - Beeline doesn't show Spark job progress info [Spark Branch] * [HIVE-9651] - FileSinkOperator does not pass in conf to serde.initialize ** Improvement * [HIVE-2828] - make timestamp accessible in the hbase KeyValue * [HIVE-3187] - support ISO-2012 timestamp literals * [HIVE-3280] - Make HiveMetaStoreClient a public API * [HIVE-4639] - Add has null flag to ORC internal index * [HIVE-4766] - Support HS2 client login timeout when the thrift thread max# is reached * [HIVE-4809] - ReduceSinkOperator of PTFOperator can have redundant key columns * [HIVE-5718] - Support direct fetch for lateral views, sub queries, etc. * [HIVE-6148] - Support arbitrary structs stored in HBase * [HIVE-7032] - Remove Triple Negative in Error 10129 * [HIVE-7313] - Allow in-memory/ssd session-level temp-tables * [HIVE-7408] - HCatPartition needs getPartCols method * [HIVE-7550] - Extend cached evaluation to multiple expressions * [HIVE-7605] - add more javadoc to HivePrivilegeObject * [HIVE-7685] - Parquet memory manager * [HIVE-7858] - Parquet compression should be configurable via table property * [HIVE-7868] - AvroSerDe error handling could be improved * [HIVE-7896] - orcfiledump should be able to dump data * [HIVE-7977] - Avoid creating serde for partitions if possible in FetchTask * [HIVE-8094] - add LIKE keyword support for SHOW FUNCTIONS * [HIVE-8155] - In select statement after * any random characters are allowed in hive but in RDBMS its not allowed * [HIVE-8337] - Change default of hive.warehouse.subdir.inherit.perms to true * [HIVE-8347] - Use base-64 encoding instead of custom encoding for serialized objects * [HIVE-8357] - Path type entities should use qualified path rather than string * [HIVE-8395] - CBO: enable by default * [HIVE-8424] - Support fair scheduler user queue mapping in non-impersonation mode * [HIVE-8449] - webhcat startup does not create the log directory * [HIVE-8469] - Add parquet.compression as a Serde Property * [HIVE-8552] - Remove hard code of offline and make addHost api public * [HIVE-8642] - Hive stack() UDTF Doesn't Support NULL Insert Values * [HIVE-8661] - JDBC MinimizeJAR should be configurable in pom.xml * [HIVE-8710] - Add more tests for transactional inserts * [HIVE-8813] - Allow tests to be excluded based on pattern/regex * [HIVE-8823] - Add additional serde properties for parquet * [HIVE-8829] - Upgrade to Thrift 0.9.2 * [HIVE-8839] - Support "alter table .. add/replace columns cascade" * [HIVE-8910] - Refactoring of PassThroughOutputFormat * [HIVE-8933] - Check release builds for SNAPSHOT dependencies * [HIVE-8971] - HIVE-8965 exposed some classes which start with Test but are not tests * [HIVE-8976] - Make nine additional tests deterministic * [HIVE-8977] - TestParquetDirect should be abstract * [HIVE-8983] - PTest Backup spark.log in addition to hive.log * [HIVE-8988] - Support advanced aggregation in Hive to Calcite path * [HIVE-8996] - Rename getUGIForConf * [HIVE-9037] - Improve explain plan to show joining keys for shuffle join * [HIVE-9049] - Metastore should use TCompactProtocol as opposed to TBinaryProtocol * [HIVE-9062] - Explain plan doesn't print join keys for Tez shuffle join * [HIVE-9106] - improve the performance of null scan optimizer when several table scans share a physical path * [HIVE-9119] - ZooKeeperHiveLockManager does not use zookeeper in the proper way * [HIVE-9140] - Add Calcite's ReduceExpressionRules to Hive * [HIVE-9158] - Multiple LDAP server URLs in hive.server2.authentication.ldap.url * [HIVE-9176] - Delegation token interval should be configurable in HadoopThriftAuthBridge * [HIVE-9189] - Add ProjectRemove rule on CBO path * [HIVE-9226] - Beeline interweaves the query result and query log sometimes * [HIVE-9244] - Upgrade 0.23 hadoop-shims to latest stable hadoop-2.6.0 * [HIVE-9314] - Writable object inspector should use Writable classes from Hadoop for get * [HIVE-9318] - Add UnionMerge rule on cbo path * [HIVE-9341] - Apply ColumnPrunning for noop PTFs * [HIVE-9357] - Create ADD_MONTHS UDF * [HIVE-9358] - Create LAST_DAY UDF * [HIVE-9367] - CombineFileInputFormatShim#getDirIndices is expensive * [HIVE-9372] - Parallel checking non-combinable paths in CombineHiveInputFormat * [HIVE-9383] - Improve schema verification error message * [HIVE-9538] - Exclude thirdparty directory from tarballs * [HIVE-9549] - Include missing directories in source tarball * [HIVE-9586] - Too verbose log can hurt performance, we should always check log level first * [HIVE-9608] - Define SPARK_HOME if not defined automagically ** New Feature * [HIVE-3405] - UDF initcap to obtain a string with the first letter of each word in uppercase other letters in lowercase * [HIVE-7122] - Storage format for create like table * [HIVE-8435] - Add identity project remover optimization * [HIVE-9174] - Enable queuing of HCatalog notification events in metastore DB * [HIVE-9175] - Add alters to list of events handled by NotificationListener * [HIVE-9184] - Modify HCatClient to support new notification methods in HiveMetaStoreClient ** Task * [HIVE-6977] - Delete Hiveserver1 * [HIVE-7111] - Extend join transitivity PPD to non-column expressions * [HIVE-8351] - Allow testing of both hadoop-1 and hadoop-2 for precommits * [HIVE-8370] - Fix tabs in DefaultHBaseValueFactory * [HIVE-8465] - Fix some minor test fails on trunk * [HIVE-8608] - Move jenkins scripts to source control * [HIVE-8609] - Move beeline to jline2 * [HIVE-8826] - Remove jdbm from top level license file * [HIVE-8828] - Remove hadoop 20 shims * [HIVE-8894] - Move calcite.version to root pom * [HIVE-8935] - Add debug logging around token stores * [HIVE-8974] - Upgrade to Calcite 1.0.0-SNAPSHOT (with lots of renames) * [HIVE-8979] - Merge shims/common-secure into shims/common * [HIVE-8989] - Make groupby_multi_single_reducer.q and smb_mapjoin_3.q deterministic * [HIVE-9183] - Compilation against HBase 1.0.0 fails due to missing dependency on high_scale_lib.Counter * [HIVE-9304] - [Refactor] remove unused method in SemAly * [HIVE-9345] - Add encryption branch to pre-commit script * [HIVE-9402] - Create GREATEST and LEAST udf * [HIVE-9407] - Fix spacing in pom * [HIVE-9408] - Add hook interface so queries can be redacted before being placed in job.xml * [HIVE-9527] - Include dot files in tarball * [HIVE-9554] - Rename 0.15 upgrade scripts to 1.1 * [HIVE-9672] - Update RELEASE_NOTES on trunk to reflect the 1.0.0 release ** Test * [HIVE-7924] - auto_sortmerge_join_8 sometimes fails with OOM * [HIVE-8256] - Add SORT_QUERY_RESULTS for test that doesn't guarantee order #2 * [HIVE-8327] - mvn site -Pfindbugs * [HIVE-8573] - Fix some non-deterministic vectorization tests * [HIVE-8601] - Fix auto_sortmerge_join_16 determinism * [HIVE-8602] - Add SORT_QUERY_RESULTS for skewjoinopt2 * [HIVE-8753] - TestMiniTezCliDriver.testCliDriver_vector_mapjoin_reduce failing on trunk * [HIVE-8801] - Make orc_merge_incompat1.q deterministic across platforms * [HIVE-8929] - incorrect error message for cbo path * [HIVE-9091] - Add additional unit tests for HiveSQLException * [HIVE-9173] - Fix auto_join29.q, auto_join_without_localtask.q determinism * [HIVE-9290] - Make some test results deterministic * [HIVE-9360] - TestSparkClient throws Timeoutexception * [HIVE-9403] - File tests determinism with multiple reducers * [HIVE-9417] - Fix failing test groupby_grouping_window.q on trunk * [HIVE-9435] - Fix auto_join21.q for Tez Release Notes - Hive - Version 1.0.0 ** Bug * [HIVE-5631] - Index creation on a skew table fails * [HIVE-5664] - Drop cascade database fails when the db has any tables with indexes * [HIVE-6198] - ORC file and struct column names are case sensitive * [HIVE-6468] - HS2 & Metastore using SASL out of memory error when curl sends a get request * [HIVE-7270] - SerDe Properties are not considered by show create table Command * [HIVE-8099] - IN operator for partition column fails when the partition column type is DATE * [HIVE-8295] - Add batch retrieve partition objects for metastore direct sql * [HIVE-8374] - schematool fails on Postgres versions < 9.2 * [HIVE-8485] - HMS on Oracle incompatibility * [HIVE-8706] - Table statistic collection on counter failed due to table name character case. * [HIVE-8715] - Hive 14 upgrade scripts can fail for statistics if database was created using auto-create * [HIVE-8739] - handle Derby and Oracle errors with joins and filters in Direct SQL in a invalid-DB-specific path * [HIVE-8784] - Querying partition does not work with JDO enabled against PostgreSQL * [HIVE-8805] - CBO skipped due to SemanticException: Line 0:-1 Both left and right aliases encountered in JOIN 'avg_cs_ext_discount_amt' * [HIVE-8807] - Obsolete default values in webhcat-default.xml * [HIVE-8811] - Dynamic partition pruning can result in NPE during query compilation * [HIVE-8827] - Remove SSLv2Hello from list of disabled protocols * [HIVE-8830] - hcatalog process don't exit because of non daemon thread * [HIVE-8845] - Switch to Tez 0.5.2 * [HIVE-8866] - Vectorization on partitioned table throws ArrayIndexOutOfBoundsException when partitions are not of same #of columns * [HIVE-8870] - errors when selecting a struct field within an array from ORC based tables * [HIVE-8873] - Switch to calcite 0.9.2 * [HIVE-8876] - incorrect upgrade script for Oracle (13->14) * [HIVE-8880] - non-synchronized access to split list in OrcInputFormat * [HIVE-8886] - Some Vectorized String CONCAT expressions result in runtime error Vectorization: Unsuported vector output type: StringGroup * [HIVE-8888] - Mapjoin with LateralViewJoin generates wrong plan in Tez * [HIVE-8947] - HIVE-8876 also affects Postgres < 9.2 * [HIVE-8966] - Delta files created by hive hcatalog streaming cannot be compacted * [HIVE-9003] - Vectorized IF expr broken for the scalar and scalar case * [HIVE-9025] - join38.q (without map join) produces incorrect result when testing with multiple reducers * [HIVE-9038] - Join tests fail on Tez * [HIVE-9051] - TezJobMonitor in-place updates logs too often to logfile * [HIVE-9053] - select constant in union all followed by group by gives wrong result * [HIVE-9067] - OrcFileMergeOperator may create merge file that does not match properties of input files * [HIVE-9090] - Rename "Tez File Merge Work" to smaller name * [HIVE-9108] - Fix for HIVE-8735 is incorrect (stats with long paths) * [HIVE-9111] - Potential NPE in OrcStruct for list and map types * [HIVE-9112] - Query may generate different results depending on the number of reducers * [HIVE-9114] - union all query in cbo test has undefined ordering * [HIVE-9126] - Backport HIVE-8827 (Remove SSLv2Hello from list of disabled protocols) to 0.14 branch * [HIVE-9141] - HiveOnTez: mix of union all, distinct, group by generates error * [HIVE-9155] - HIVE_LOCKS uses int instead of bigint hive-txn-schema-0.14.0.mssql.sql * [HIVE-9162] - stats19 test is environment-dependant * [HIVE-9166] - Place an upper bound for SARG CNF conversion * [HIVE-9168] - Vectorized Coalesce for strings is broken * [HIVE-9205] - Change default tez install directory to use /tmp instead of /user and create the directory if it does not exist * [HIVE-9234] - HiveServer2 leaks FileSystem objects in FileSystem.CACHE * [HIVE-9249] - java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.HiveVarcharWritable cannot be cast to org.apache.hadoop.hive.common.type.HiveVarchar when joining tables * [HIVE-9278] - Cached expression feature broken in one case * [HIVE-9317] - move Microsoft copyright to NOTICE file * [HIVE-9359] - Export of a large table causes OOM in Metastore and Client * [HIVE-9361] - Intermittent NPE in SessionHiveMetaStoreClient.alterTempTable * [HIVE-9390] - Enhance retry logic wrt DB access in TxnHandler * [HIVE-9401] - Backport: Fastpath for limited fetches from unpartitioned tables * [HIVE-9404] - NPE in org.apache.hadoop.hive.metastore.txn.TxnHandler.determineDatabaseProduct() * [HIVE-9436] - RetryingMetaStoreClient does not retry JDOExceptions * [HIVE-9473] - sql std auth should disallow built-in udfs that allow any java methods to be called * [HIVE-9514] - schematool is broken in hive 1.0.0 ** Improvement * [HIVE-3280] - Make HiveMetaStoreClient a public API * [HIVE-8933] - Check release builds for SNAPSHOT dependencies ** Task * [HIVE-6977] - Delete Hiveserver1 Release Notes - Hive - Version 0.14.0 ** Sub-task * [HIVE-4629] - HS2 should support an API to retrieve query logs * [HIVE-5176] - Wincompat : Changes for allowing various path compatibilities with Windows * [HIVE-5179] - Wincompat : change script tests from bash to sh * [HIVE-5579] - Update statistics rules for different types of joins * [HIVE-5760] - Add vectorized support for CHAR/VARCHAR data types * [HIVE-5804] - Support column statistics for expressions in GBY attributes, JOIN condition etc. when annotating operator tree with statistics * [HIVE-5805] - Support for operators like PTF, Script, Extract etc. in statistics annotation. * [HIVE-5868] - Add statistics rule for Union operator * [HIVE-5949] - In statistics annotation add flag to say if statistics is estimated or accurate * [HIVE-5998] - Add vectorized reader for Parquet files * [HIVE-6031] - explain subquery rewrite for where clause predicates * [HIVE-6123] - Implement checkstyle in maven * [HIVE-6252] - sql std auth - support 'with admin option' in revoke role metastore api * [HIVE-6290] - Add support for hbase filters for composite keys * [HIVE-6367] - Implement Decimal in ParquetSerde * [HIVE-6394] - Implement Timestmap in ParquetSerde * [HIVE-6445] - Add qop support for kerberos over http in HiveServer2 * [HIVE-6626] - Hive does not expand the DOWNLOADED_RESOURCES_DIR path * [HIVE-6627] - HiveServer2 should handle scratch dir permissions / errors in a better way * [HIVE-6714] - Fix getMapSize() of LazyMap * [HIVE-6735] - Make scalable dynamic partitioning work in vectorized mode * [HIVE-6760] - Scalable dynamic partitioning should bail out properly for list bucketing * [HIVE-6761] - Hashcode computation does not use maximum parallelism for scalable dynamic partitioning * [HIVE-6798] - Update column stats based on filter expression in stats annotation * [HIVE-6815] - Version of the HIVE-6374 for Hive 0.13 * [HIVE-6982] - Export all .sh equivalent for windows (.cmd files) in bin, bin/ext * [HIVE-6993] - Update hive for Tez VertexLocationHint and getAVailableResource API changes * [HIVE-7029] - Vectorize ReduceWork * [HIVE-7078] - Need file sink operators that work with ACID * [HIVE-7094] - Separate out static/dynamic partitioning code in FileRecordWriterContainer * [HIVE-7156] - Group-By operator stat-annotation only uses distinct approx to generate rollups * [HIVE-7184] - TestHadoop20SAuthBridge no longer compiles after HADOOP-10448 * [HIVE-7204] - Use NULL vertex location hint for Prewarm DAG vertices * [HIVE-7262] - Partitioned Table Function (PTF) query fails on ORC table when attempting to vectorize * [HIVE-7286] - Parameterize HCatMapReduceTest for testing against all Hive storage formats * [HIVE-7291] - Refactor TestParser to understand test-property file * [HIVE-7350] - Changes related to TEZ-692, TEZ-1169, TEZ-1234 * [HIVE-7357] - Add vectorized support for BINARY data type * [HIVE-7398] - Parent GBY of MUX is removed even it's not for semijoin * [HIVE-7404] - Revoke privilege should support revoking of grant option * [HIVE-7405] - Vectorize GROUP BY on the Reduce-Side (Part 1 – Basic) * [HIVE-7420] - Parameterize tests for HCatalog Pig interfaces for testing against all storage formats * [HIVE-7427] - Changes for EdgeConfigurations * [HIVE-7457] - Minor HCatalog Pig Adapter test clean up * [HIVE-7491] - Stats annotation fails to evaluate constant expressions in filter operator * [HIVE-7513] - Add ROW__ID VirtualColumn * [HIVE-7535] - Make use of number of nulls column statistics in filter rule * [HIVE-7536] - Make use of decimal column statistics in statistics annotation * [HIVE-7544] - Changes related to TEZ-1288 (FastTezSerialization) * [HIVE-7548] - Precondition checks should not fail the merge task in case of automatic trigger * [HIVE-7571] - RecordUpdater should read virtual columns from row * [HIVE-7589] - Some fixes and improvements to statistics annotation rules * [HIVE-7601] - Bring up tez-branch upto the API changes from TEZ-1058, TEZ-1303, TEZ-1346, TEZ-1041 * [HIVE-7639] - Bring tez-branch upto api changes in TEZ-1379, TEZ-1057, TEZ-1382 * [HIVE-7646] - Modify parser to support new grammar for Insert,Update,Delete * [HIVE-7655] - CBO: Reading of partitioned table stats slows down explain * [HIVE-7656] - Bring tez-branch up-to the API changes made by TEZ-1372 * [HIVE-7663] - OrcRecordUpdater needs to implement getStats * [HIVE-7679] - JOIN operator should update the column stats when number of rows changes * [HIVE-7734] - Join stats annotation rule is not updating columns statistics correctly * [HIVE-7735] - Implement Char, Varchar in ParquetSerDe * [HIVE-7788] - Generate plans for insert, update, and delete * [HIVE-7790] - Update privileges to check for update and delete * [HIVE-7808] - Changes to work against Tez-0.5 RC * [HIVE-7809] - Fix ObjectRegistry to work with Tez 0.5 * [HIVE-7820] - union_null.q is not deterministic * [HIVE-7825] - Bring tez-branch up-to the API changes made by TEZ-1472, TEZ-1469 * [HIVE-7836] - Ease-out denominator for multi-attribute join case in statistics annotation * [HIVE-7864] - [CBO] Query fails if it refers only partitioning column * [HIVE-7869] - Build long running HS2 test framework * [HIVE-7904] - Missing null check cause NPE when updating join column stats in statistics annotation * [HIVE-7905] - CBO: more cost model changes * [HIVE-7907] - Bring up tez branch to changes in TEZ-1038, TEZ-1500 * [HIVE-7935] - Support dynamic service discovery for HiveServer2 * [HIVE-7979] - Fix testconfiguration.property file in Tez branch * [HIVE-7990] - With fetch column stats disabled number of elements in grouping set is not taken into account * [HIVE-7991] - Incorrect calculation of number of rows in JoinStatsRule.process results in overflow * [HIVE-7992] - StatsRulesProcFactory should gracefully handle overflows * [HIVE-7994] - BMJ test fails on tez * [HIVE-7995] - Column statistics from expression does not handle fields within complex types * [HIVE-8006] - CBO Trunk Merge: Test fail that includes Table Sample, rows(), query hints * [HIVE-8016] - CBO: PPD to honor hive Join Cond, Casting fixes, Add annotations for IF, Code cleanup * [HIVE-8021] - CBO: support CTAS and insert ... select * [HIVE-8046] - CBO: fix issues with Windowing queries * [HIVE-8069] - CBO: RowResolver after SubQuery predicate handling should be reset to outer query block RR * [HIVE-8111] - CBO trunk merge: duplicated casts for arithmetic expressions in Hive and CBO * [HIVE-8125] - CBO Trunk Merge: On Failure Fall Back to Non CBO * [HIVE-8144] - CBO: HiveProjectRel factory should create RelSubSets * [HIVE-8145] - CBO: bail from Optiq planning if a Select list contains multiple references to the same name * [HIVE-8159] - CBO: bail from Optiq planning if a Select list contains multiple references to the same name * [HIVE-8168] - With dynamic partition enabled fact table selectivity is not taken into account when generating the physical plan (Use CBO cardinality using physical plan generation) * [HIVE-8172] - HiveServer2 dynamic service discovery should let the JDBC client use default ZooKeeper namespace * [HIVE-8173] - HiveServer2 dynamic service discovery: figure out best ZooKeeper ACLs for security * [HIVE-8186] - Self join may fail if one side have virtual column(s) and other doesn't * [HIVE-8193] - Hook HiveServer2 dynamic service discovery with session time out * [HIVE-8194] - CBO: bail for having clause referring select expr aliases * [HIVE-8199] - CBO Trunk Merge: quote2 test fails due to incorrect literal translation * [HIVE-8223] - CBO Trunk Merge: partition_wise_fileformat2 select result depends on ordering * [HIVE-8228] - CBO: fix couple of issues with partition pruning * [HIVE-8237] - CBO: Use Fully qualified table name (db.tablename in ReloptHiveTable) * [HIVE-8288] - HiveServer2 dynamic discovery should create znodes organized by version number & add support for removing server uri's of a particular version from the server script. * [HIVE-8309] - CBO: Fix OB by removing constraining DT, Use external names for col Aliases, Remove unnecessary Selects, Make DT Name counter query specific * [HIVE-8377] - Enable Kerberized SSL for HiveServer2 in http mode * [HIVE-8454] - Select Operator does not rename column stats properly in case of select star * [HIVE-8522] - CBO: Update Calcite Version to 0.9.2-incubating-SNAPSHOT * [HIVE-8530] - CBO: Preserve types of literals * [HIVE-8549] - NPE in PK-FK inference when one side of join is complex tree * [HIVE-8580] - Support LateralViewJoinOperator and LateralViewForwardOperator in stats annotation * [HIVE-8582] - CBO: Outer Join Simplification is broken * [HIVE-8653] - CBO: Push Semi Join through, Project/Filter/Join * [HIVE-8654] - CBO: parquet_ctas test returns incorrect results * [HIVE-8655] - CBO: ppr_pushdown, udf_substr produces incorrect results due to broken tablesample handling * [HIVE-8656] - CBO: auto_join_filters fails * [HIVE-8657] - CBO: inputddl5, udf_reverse tests fail * [HIVE-8662] - CBO: tez_dml fails * [HIVE-8768] - CBO: Fix filter selectivity for "in clause" & "<>" ** Bug * [HIVE-1363] - 'SHOW TABLE EXTENDED LIKE' command does not strip single/double quotes * [HIVE-1608] - use sequencefile as the default for storing intermediate results * [HIVE-1879] - Remove hive.metastore.metadb.dir property from hive-default.xml and HiveConf * [HIVE-2137] - JDBC driver doesn't encode string properly. * [HIVE-2390] - Add UNIONTYPE serialization support to LazyBinarySerDe * [HIVE-2597] - Repeated key in GROUP BY is erroneously displayed when using DISTINCT * [HIVE-3392] - Hive unnecessarily validates table SerDes when dropping a table * [HIVE-3685] - TestCliDriver (script_pipe.q) failed with IBM JDK * [HIVE-3925] - dependencies of fetch task are not shown by explain * [HIVE-4064] - Handle db qualified names consistently across all HiveQL statements * [HIVE-4118] - ANALYZE TABLE ... COMPUTE STATISTICS FOR COLUMNS fails when using fully qualified table name * [HIVE-4274] - Table created using HCatalog java client doesn't set the owner * [HIVE-4561] - Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0.0000 ,if all the column values larger than 0.0 (or if all column values smaller than 0.0) * [HIVE-4576] - templeton.hive.properties does not allow values with commas * [HIVE-4577] - hive CLI can't handle hadoop dfs command with space and quotes. * [HIVE-4723] - DDLSemanticAnalyzer.addTablePartsOutputs eats several exceptions * [HIVE-4795] - Delete/Alter/Describe actions fail when SerDe is not on class path * [HIVE-4965] - Add support so that PTFs can stream their output; Windowing PTF should do this * [HIVE-5077] - Provide an option to run local task in process * [HIVE-5092] - Fix hiveserver2 mapreduce local job on Windows * [HIVE-5150] - UnsatisfiedLinkError when running hive unit tests on Windows * [HIVE-5268] - HiveServer2 accumulates orphaned OperationHandle objects when a client fails while executing query * [HIVE-5315] - Cannot attach debugger to Hiveserver2 * [HIVE-5336] - HCatSchema.remove(HCatFieldSchema hcatFieldSchema) should renumber the fieldPositionMap and the fieldPositionMap should not be cached by the end user * [HIVE-5376] - Hive does not honor type for partition columns when altering column type * [HIVE-5434] - Creating a new HiveConnection does not handle the case when properties are supplied separately from connection string * [HIVE-5456] - Queries fail on avro backed table with empty partition * [HIVE-5607] - Hive fails to parse the "%" (mod) sign after brackets. * [HIVE-5631] - Index creation on a skew table fails * [HIVE-5664] - Drop cascade database fails when the db has any tables with indexes * [HIVE-5677] - Beeline warns about unavailable files if HIVE_OPTS is set * [HIVE-5789] - WebHCat E2E tests do not launch on Windows * [HIVE-5847] - DatabaseMetadata.getColumns() doesn't show correct column size for char/varchar/decimal * [HIVE-5870] - Move TestJDBCDriver2.testNewConnectionConfiguration to TestJDBCWithMiniHS2 * [HIVE-6035] - Windows: percentComplete returned by job status from WebHCat is null * [HIVE-6093] - table creation should fail when user does not have permissions on db * [HIVE-6149] - TestJdbcDriver2 is unable to drop a database created from previous runs ("hbasedb") * [HIVE-6176] - Beeline gives bogus error message if an unaccepted command line option is given * [HIVE-6187] - Add test to verify that DESCRIBE TABLE works with quoted table names * [HIVE-6195] - Create unit tests to exercise behaviour when creating a HBase Table in Hive * [HIVE-6200] - Hive custom SerDe cannot load DLL added by "ADD FILE" command * [HIVE-6245] - HS2 creates DBs/Tables with wrong ownership when HMS setugi is true * [HIVE-6305] - test use of quoted identifiers in user/role names * [HIVE-6313] - Minimr tests in hadoop-1 hangs on shutdown * [HIVE-6321] - hiveserver2 --help says Unrecognized option: -h * [HIVE-6322] - Fix file_with_header_footer_negative.q * [HIVE-6331] - HIVE-5279 deprecated UDAF class without explanation/documentation/alternative * [HIVE-6374] - Hive job submitted with non-default name node (fs.default.name) doesn't process locations properly * [HIVE-6437] - DefaultHiveAuthorizationProvider should not initialize a new HiveConf * [HIVE-6446] - Ability to specify hadoop.bin.path from command line -D * [HIVE-6447] - Bucket map joins in hive-tez * [HIVE-6480] - Metastore server startup script ignores ENV settings * [HIVE-6487] - PTest2 do not copy failed source directories * [HIVE-6508] - Mismatched results between vector and non-vector mode with decimal field * [HIVE-6511] - casting from decimal to tinyint,smallint, int and bigint generates different result when vectorization is on * [HIVE-6515] - Custom vertex in hive-tez should be able to accept multiple MR-inputs * [HIVE-6521] - WebHCat cannot fetch correct percentComplete for Hive jobs * [HIVE-6531] - Runtime errors in vectorized execution. * [HIVE-6538] - yet another annoying exception in test logs * [HIVE-6549] - remove templeton.jar from webhcat-default.xml, remove hcatalog/bin/hive-config.sh * [HIVE-6550] - SemanticAnalyzer.reset() doesn't clear all the state * [HIVE-6555] - TestSchemaTool is failing on trunk after branching * [HIVE-6560] - varchar and char types cannot be cast to binary * [HIVE-6563] - hdfs jar being pulled in when creating a hadoop-2 based hive tar ball * [HIVE-6564] - WebHCat E2E tests that launch MR jobs fail on check job completion timeout * [HIVE-6569] - HCatalog still has references to deprecated property hive.metastore.local * [HIVE-6570] - Hive variable substitution does not work with the "source" command * [HIVE-6571] - query id should be available for logging during query compilation * [HIVE-6583] - wrong sql comments : ----... instead of -- ---... * [HIVE-6586] - Update parameters in HiveConf.java after commit HIVE-6037 * [HIVE-6592] - WebHCat E2E test abort when pointing to https url of webhdfs * [HIVE-6594] - UnsignedInt128 addition does not increase internal int array count resulting in corrupted values during serialization * [HIVE-6597] - WebHCat E2E tests doAsTests_6 and doAsTests_7 need to be updated * [HIVE-6598] - Importing the project into eclipse as maven project have some issues * [HIVE-6601] - alter database commands should support schema synonym keyword * [HIVE-6602] - Multi-user HiveServer2 throws error * [HIVE-6612] - Misspelling "schemaTool completeted" * [HIVE-6620] - UDF printf doesn't take either CHAR or VARCHAR as the first argument * [HIVE-6622] - UDF translate doesn't take either CHAR or VARCHAR as any of its arguments * [HIVE-6637] - UDF in_file() doesn't take CHAR or VARCHAR as input * [HIVE-6648] - Permissions are not inherited correctly when tables have multiple partition columns * [HIVE-6652] - Beeline gives evasive error message for any unrecognized command line arguement * [HIVE-6669] - sourcing txn-script from schema script results in failure for mysql & oracle * [HIVE-6683] - Beeline does not accept comments at end of line * [HIVE-6684] - Beeline does not accept comments that are preceded by spaces * [HIVE-6695] - bin/hcat should include hbase jar and dependencies in the classpath [followup/clone of HCATALOG-621] * [HIVE-6698] - hcat.py script does not correctly load the hbase storage handler jars * [HIVE-6707] - Lazy maps are broken (LazyMap and LazyBinaryMap) * [HIVE-6709] - HiveServer2 help command is not recognizing properly. * [HIVE-6711] - ORC maps uses getMapSize() from MapOI which is unreliable * [HIVE-6715] - Hive JDBC should include username into open session request for non-sasl connection * [HIVE-6724] - HCatStorer throws ClassCastException while storing tinyint/smallint data * [HIVE-6726] - Hcat cli does not close SessionState * [HIVE-6727] - Table level stats for external tables are set incorrectly * [HIVE-6741] - HiveServer2 startup fails in secure (kerberos) mode due to backward incompatible hadoop change * [HIVE-6745] - HCat MultiOutputFormat hardcodes DistributedCache keynames * [HIVE-6756] - alter table set fileformat should set serde too * [HIVE-6768] - remove hcatalog/webhcat/svr/src/main/config/override-container-log4j.properties * [HIVE-6773] - Update readme for ptest2 framework * [HIVE-6782] - HiveServer2Concurrency issue when running with tez intermittently, throwing "org.apache.tez.dag.api.SessionNotRunning: Application not running" error * [HIVE-6783] - Incompatible schema for maps between parquet-hive and parquet-pig * [HIVE-6784] - parquet-hive should allow column type change * [HIVE-6785] - query fails when partitioned table's table level serde is ParquetHiveSerDe and partition level serde is of different SerDe * [HIVE-6788] - Abandoned opened transactions not being timed out * [HIVE-6792] - hive.warehouse.subdir.inherit.perms doesn't work correctly in CTAS * [HIVE-6793] - DDLSemanticAnalyzer.analyzeShowRoles() should use HiveAuthorizationTaskFactory * [HIVE-6807] - add HCatStorer ORC test to test missing columns * [HIVE-6811] - LOAD command does not work with relative paths on Windows * [HIVE-6817] - Some hadoop2-only tests need diffs to be updated * [HIVE-6820] - HiveServer(2) ignores HIVE_OPTS * [HIVE-6822] - TestAvroSerdeUtils fails with -Phadoop-2 * [HIVE-6824] - Hive HBase query fails on Tez due to missing jars - part 2 * [HIVE-6826] - Hive-tez has issues when different partitions work off of different input types * [HIVE-6828] - Hive tez bucket map join conversion interferes with map join conversion * [HIVE-6835] - Reading of partitioned Avro data fails if partition schema does not match table schema * [HIVE-6843] - INSTR for UTF-8 returns incorrect position * [HIVE-6847] - Improve / fix bugs in Hive scratch dir setup * [HIVE-6853] - show create table for hbase tables should exclude LOCATION * [HIVE-6858] - Unit tests decimal_udf.q, vectorization_div0.q fail with jdk-7. * [HIVE-6861] - more hadoop2 only golden files to fix * [HIVE-6862] - add DB schema DDL and upgrade 12to13 scripts for MS SQL Server * [HIVE-6868] - Create table in HCatalog sets different SerDe defaults than what is set through the CLI * [HIVE-6870] - Fix maven.repo.local setting in Hive build * [HIVE-6871] - Build fixes to allow Windows to run TestCliDriver * [HIVE-6877] - TestOrcRawRecordMerger is deleting test.tmp.dir * [HIVE-6880] - TestHWISessionManager fails with -Phadoop-2 * [HIVE-6883] - Dynamic partitioning optimization does not honor sort order or order by * [HIVE-6884] - HiveLockObject and enclosed HiveLockObjectData override equal() method but didn't do so for hashcode() * [HIVE-6888] - Hive leaks MapWork objects via Utilities::gWorkMap * [HIVE-6890] - Bug in HiveStreaming API causes problems if hive-site.xml is missing on streaming client side * [HIVE-6891] - Alter rename partition Perm inheritance and general partition/table group inheritance * [HIVE-6893] - out of sequence error in HiveMetastore server * [HIVE-6898] - Functions in hive are failing with java.lang.ClassNotFoundException on Tez * [HIVE-6900] - HostUtil.getTaskLogUrl signature change causes compilation to fail * [HIVE-6901] - Explain plan doesn't show operator tree for the fetch operator * [HIVE-6908] - TestThriftBinaryCLIService.testExecuteStatementAsync has intermittent failures * [HIVE-6910] - Invalid column access info for partitioned table * [HIVE-6913] - Hive unable to find the hashtable file during complex multi-staged map join * [HIVE-6915] - Hive Hbase queries fail on secure Tez cluster * [HIVE-6916] - Export/import inherit permissions from parent directory * [HIVE-6919] - hive sql std auth select query fails on partitioned tables * [HIVE-6921] - index creation fails with sql std auth turned on * [HIVE-6922] - NullPointerException in collect_set() UDAF * [HIVE-6927] - Add support for MSSQL in schematool * [HIVE-6928] - Beeline should not chop off "describe extended" results by default * [HIVE-6931] - Windows unit test fixes * [HIVE-6932] - hive README needs update * [HIVE-6934] - PartitionPruner doesn't handle top level constant expression correctly * [HIVE-6936] - Provide table properties to InputFormats * [HIVE-6937] - Fix test reporting url's after jenkins move from bigtop * [HIVE-6939] - TestExecDriver.testMapRedPlan3 fails on hadoop-2 * [HIVE-6944] - WebHCat e2e tests broken by HIVE-6432 * [HIVE-6945] - issues with dropping partitions on Oracle * [HIVE-6946] - Make it easier to run WebHCat e2e tests * [HIVE-6947] - More fixes for tests on hadoop-2 * [HIVE-6952] - Hive 0.13 HiveOutputFormat breaks backwards compatibility * [HIVE-6954] - After ALTER FILEFORMAT, DESCRIBE throwing exception * [HIVE-6955] - ExprNodeColDesc isSame doesn't account for tabAlias: this affects trait Propagation in Joins * [HIVE-6956] - Duplicate partitioning column for union when dynamic partition sort optimization is enabled * [HIVE-6957] - SQL authorization does not work with HS2 binary mode and Kerberos auth * [HIVE-6959] - Enable Constant propagation optimizer for Hive Vectorization * [HIVE-6960] - Set Hive pom to use Hadoop-2.4 * [HIVE-6961] - Drop partitions treats partition columns as strings * [HIVE-6965] - Transaction manager should use RDBMS time instead of machine time * [HIVE-6966] - More fixes for TestCliDriver on Windows * [HIVE-6967] - Hive transaction manager fails when SQLServer is used as an RDBMS * [HIVE-6968] - list bucketing feature does not update the location map for unpartitioned tables * [HIVE-6972] - jdbc HTTP configuration options should be part of sessionConf part of connection string * [HIVE-6976] - Show query id only when there's jobs on the cluster * [HIVE-6978] - beeline always exits with 0 status, should exit with non-zero status on error * [HIVE-6979] - Hadoop-2 test failures related to quick stats not being populated correctly * [HIVE-6984] - Analyzing partitioned table with NULL values for the partition column failed with NPE * [HIVE-6985] - sql std auth - privileges grants to public role not being honored * [HIVE-6986] - MatchPath fails with small resultExprString * [HIVE-6987] - Metastore qop settings won't work with Hadoop-2.4 * [HIVE-6989] - Error with arithmetic operators with javaXML serialization * [HIVE-6990] - Direct SQL fails when the explicit schema setting is different from the default one * [HIVE-6994] - parquet-hive createArray strips null elements * [HIVE-6995] - GenericUDFBridge should log exception when it is unable to instantiate UDF object * [HIVE-6996] - FS based stats broken with indexed tables * [HIVE-7001] - fs.permissions.umask-mode is getting unset when Session is started * [HIVE-7003] - Fix typo in README * [HIVE-7004] - Fix more unit test failures on hadoop-2 * [HIVE-7005] - MiniTez tests have non-deterministic explain plans * [HIVE-7006] - Fix ql_rewrite_gbtoidx.q output file * [HIVE-7009] - HIVE_USER_INSTALL_DIR could not bet set to non-HDFS filesystem * [HIVE-7011] - HiveInputFormat's split generation isn't thread safe * [HIVE-7012] - Wrong RS de-duplication in the ReduceSinkDeDuplication Optimizer * [HIVE-7015] - Failing to inherit group/permission should not fail the operation * [HIVE-7016] - Hive returns wrong results when execute UDF on top of DISTINCT column * [HIVE-7017] - Insertion into Parquet tables fails under Tez * [HIVE-7021] - HiveServer2 memory leak on failed queries * [HIVE-7023] - Bucket mapjoin is broken when the number of small aliases is two or more * [HIVE-7027] - Hive job fails when referencing a view that explodes an array * [HIVE-7030] - Remove hive.hadoop.classpath from hiveserver2.cmd * [HIVE-7031] - Utiltites.createEmptyFile uses File.Separator instead of Path.Separator to create an empty file in HDFS * [HIVE-7033] - grant statements should check if the role exists * [HIVE-7035] - Templeton returns 500 for user errors - when job cannot be found * [HIVE-7037] - Add additional tests for transform clauses with Tez * [HIVE-7041] - DoubleWritable/ByteWritable should extend their hadoop counterparts * [HIVE-7042] - Fix stats_partscan_1_23.q and orc_createas1.q for hadoop-2 * [HIVE-7043] - When using the tez session pool via hive, once sessions time out, all queries go to the default queue * [HIVE-7045] - Wrong results in multi-table insert aggregating without group by clause * [HIVE-7050] - Display table level column stats in DESCRIBE FORMATTED TABLE * [HIVE-7051] - Display partition level column stats in DESCRIBE FORMATTED PARTITION * [HIVE-7052] - Optimize split calculation time * [HIVE-7053] - Unable to fetch column stats from decimal columns * [HIVE-7055] - config not propagating for PTFOperator * [HIVE-7057] - webhcat e2e deployment scripts don't have x bit set * [HIVE-7060] - Column stats give incorrect min and distinct_count * [HIVE-7061] - sql std auth - insert queries without overwrite should not require delete privileges * [HIVE-7062] - Support Streaming mode in Windowing * [HIVE-7063] - Optimize for the Top N within a Group use case * [HIVE-7065] - Hive jobs in webhcat run in default mr mode even in Hive on Tez setup * [HIVE-7066] - hive-exec jar is missing avro core * [HIVE-7067] - Min() and Max() on Timestamp and Date columns for ORC returns wrong results * [HIVE-7071] - Use custom Tez split generator to support schema evolution * [HIVE-7072] - HCatLoader only loads first region of hbase table * [HIVE-7075] - JsonSerde raises NullPointerException when object key is not lower case * [HIVE-7076] - Plugin (exec hook) to log to application timeline data to Yarn * [HIVE-7077] - Hive contrib compilation maybe broken with removal of org.apache.hadoop.record * [HIVE-7079] - Hive logs errors about missing tables when parsing CTE expressions * [HIVE-7080] - In PTest framework, Add logs URL to the JIRA comment * [HIVE-7082] - Vectorized parquet reader should create assigners only for the columns it assigns, not for scratch columns * [HIVE-7083] - Fix test failures on trunk * [HIVE-7087] - Remove lineage information after query completion * [HIVE-7092] - Insert overwrite should not delete the original directory * [HIVE-7096] - Support grouped splits in Tez partitioned broadcast join * [HIVE-7099] - Add Decimal datatype support for Windowing * [HIVE-7104] - Unit tests are disabled * [HIVE-7105] - Enable ReduceRecordProcessor to generate VectorizedRowBatches * [HIVE-7107] - Fix HiveServer1 JDBC Driver spec compliancy issue * [HIVE-7109] - Resource leak in HBaseStorageHandler * [HIVE-7112] - Tez processor swallows errors * [HIVE-7114] - Extra Tez session is started during HiveServer2 startup * [HIVE-7116] - HDFS FileSystem object cache causes permission issues in creating tmp directories * [HIVE-7117] - Partitions not inheriting table permissions after alter rename partition * [HIVE-7118] - Oracle upgrade schema scripts do not map Java long datatype columns correctly for transaction related tables * [HIVE-7119] - Extended ACL's should be inherited if warehouse perm inheritance enabled * [HIVE-7123] - Follow-up of HIVE-6367 * [HIVE-7130] - schematool is broken for minor version upgrades (eg 0.13.x) * [HIVE-7131] - Dependencies of fetch task for tez are not shown properly * [HIVE-7135] - Fix test fail of TestTezTask.testSubmit * [HIVE-7143] - Add Streaming support in Windowing mode for more UDAFs (min/max, lead/lag, fval/lval) * [HIVE-7144] - GC pressure during ORC StringDictionary writes * [HIVE-7146] - posexplode() UDTF fails with a NullPointerException on NULL columns * [HIVE-7147] - ORC PPD should handle CHAR/VARCHAR types * [HIVE-7149] - Parquet not able to handle negative decimal numbers * [HIVE-7154] - TestMetrics fails intermittently on the trunk * [HIVE-7155] - WebHCat controller job exceeds container memory limit * [HIVE-7159] - For inner joins push a 'is not null predicate' to the join sources for every non nullSafe join condition * [HIVE-7161] - TestMetastoreVersion fails intermittently on trunk * [HIVE-7162] - hadoop-1 build broken by HIVE-7071 * [HIVE-7165] - Fix hive-default.xml.template errors & omissions * [HIVE-7167] - Hive Metastore fails to start with SQLServerException * [HIVE-7169] - HiveServer2 in Http Mode should have a configurable IdleMaxTime timeout * [HIVE-7170] - Fix display_colstats_tbllvl.q in trunk * [HIVE-7173] - Support HIVE-4867 on mapjoin of MR Tasks * [HIVE-7174] - Do not accept string as scale and precision when reading Avro schema * [HIVE-7176] - FileInputStream is not closed in Commands#properties() * [HIVE-7182] - ResultSet is not closed in JDBCStatsPublisher#init() * [HIVE-7183] - Size of partColumnGrants should be checked in ObjectStore#removeRole() * [HIVE-7187] - Reconcile jetty versions in hive * [HIVE-7188] - sum(if()) returns wrong results with vectorization * [HIVE-7190] - WebHCat launcher task failure can cause two concurent user jobs to run * [HIVE-7191] - optimized map join hash table has a bug when it reaches 2Gb * [HIVE-7192] - Hive Streaming - Some required settings are not mentioned in the documentation * [HIVE-7199] - Cannot alter table to parquet * [HIVE-7200] - Beeline output displays column heading even if --showHeader=false is set * [HIVE-7201] - Fix TestHiveConf#testConfProperties test case * [HIVE-7202] - DbTxnManager deadlocks in hcatalog.cli.TestSematicAnalysis.testAlterTblFFpart() * [HIVE-7209] - allow metastore authorization api calls to be restricted to certain invokers * [HIVE-7210] - NPE with "No plan file found" when running Driver instances on multiple threads * [HIVE-7213] - COUNT(*) returns out-dated count value after TRUNCATE * [HIVE-7220] - Empty dir in external table causes issue (root_dir_external_table.q failure) * [HIVE-7225] - Unclosed Statement's in TxnHandler * [HIVE-7226] - Windowing Streaming mode causes NPE for empty partitions * [HIVE-7228] - StreamPrinter should be joined to calling thread * [HIVE-7229] - String is compared using equal in HiveMetaStore#HMSHandler#init() * [HIVE-7232] - VectorReduceSink is emitting incorrect JOIN keys * [HIVE-7234] - Select on decimal column throws NPE * [HIVE-7235] - TABLESAMPLE on join table is regarded as alias * [HIVE-7236] - Tez progress monitor should indicate running/failed tasks * [HIVE-7237] - hive.exec.parallel=true w/ Hive 0.13/Tez causes application to linger forever * [HIVE-7241] - Wrong lock acquired for alter table rename partition * [HIVE-7242] - alter table drop partition is acquiring the wrong type of lock * [HIVE-7245] - Fix parquet_columnar * [HIVE-7246] - Hive transaction manager hardwires bonecp as the JDBC pooling implementation * [HIVE-7247] - Fix itests using hadoop-1 profile * [HIVE-7249] - HiveTxnManager.closeTxnManger() throws if called after commitTxn() * [HIVE-7251] - Fix StorageDescriptor usage in unit tests * [HIVE-7257] - UDF format_number() does not work on FLOAT types * [HIVE-7263] - Missing fixes from review of parquet-timestamp * [HIVE-7265] - BINARY columns use BytesWritable::getBytes() without ::getLength() * [HIVE-7268] - On Windows Hive jobs in Webhcat always run on default MR mode * [HIVE-7271] - Speed up unit tests * [HIVE-7274] - Update PTest2 to JClouds 1.7.3 * [HIVE-7279] - UDF format_number() does not work on DECIMAL types * [HIVE-7281] - DbTxnManager acquiring wrong level of lock for dynamic partitioning * [HIVE-7282] - HCatLoader fail to load Orc map with null key * [HIVE-7287] - hive --rcfilecat command is broken on Windows * [HIVE-7294] - sql std auth - authorize show grant statements * [HIVE-7298] - desc database extended does not show properties of the database * [HIVE-7302] - Allow Auto-reducer parallelism to be turned off by a logical optimizer * [HIVE-7303] - IllegalMonitorStateException when stmtHandle is null in HiveStatement * [HIVE-7304] - Transitive Predicate Propagation doesn't happen properly after HIVE-7159 * [HIVE-7314] - Wrong results of UDF when hive.cache.expr.evaluation is set * [HIVE-7317] - authorization_explain.q fails when run in sequence * [HIVE-7323] - Date type stats in ORC sometimes go stale * [HIVE-7325] - Support non-constant expressions for ARRAY/MAP type indices. * [HIVE-7326] - Hive complains invalid column reference with 'having' aggregate predicates * [HIVE-7339] - hive --orcfiledump command is not supported on Windows * [HIVE-7342] - support hiveserver2,metastore specific config files * [HIVE-7344] - Add streaming support in Windowing mode for FirstVal, LastVal * [HIVE-7345] - Beeline changes its prompt to reflect successful database connection even after failing to connect * [HIVE-7346] - Wrong results caused by hive ppd under specific join condition * [HIVE-7352] - Queries without tables fail under Tez * [HIVE-7353] - HiveServer2 using embedded MetaStore leaks JDOPersistanceManager * [HIVE-7354] - windows:Need to set hbase jars in hadoop classpath explicitly * [HIVE-7356] - Table level stats collection fail for partitioned tables * [HIVE-7359] - Stats based compute query replies fail to do simple column transforms * [HIVE-7363] - VectorExpressionWriterDecimal is missing null check in setValue() * [HIVE-7366] - getDatabase using direct sql * [HIVE-7373] - Hive should not remove trailing zeros for decimal numbers * [HIVE-7374] - SHOW COMPACTIONS fail with remote metastore when there are no compations * [HIVE-7376] - add minimizeJar to jdbc/pom.xml * [HIVE-7385] - Optimize for empty relation scans * [HIVE-7389] - Reduce number of metastore calls in MoveTask (when loading dynamic partitions) * [HIVE-7393] - Tez jobs sometimes fail with NPE processing input splits * [HIVE-7394] - ORC writer logging fails when the padding is < 0.01 * [HIVE-7396] - BucketingSortingReduceSinkOptimizer throws NullPointException during ETL * [HIVE-7397] - Set the default threshold for fetch task conversion to 1Gb * [HIVE-7399] - Timestamp type is not copied by ObjectInspectorUtils.copyToStandardObject * [HIVE-7409] - Add workaround for a deadlock issue of Class.getAnnotation() * [HIVE-7412] - column stats collection throws exception if all values for a column is null * [HIVE-7414] - Update golden file for MiniTez temp_table.q * [HIVE-7415] - Test TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx failing * [HIVE-7417] - select count(1) from ... where true; fails in optimizer * [HIVE-7419] - Missing break in SemanticAnalyzer#getTableDescFromSerDe() * [HIVE-7421] - Make VectorUDFDateString use the same date parsing and formatting as GenericUDFDate * [HIVE-7422] - Array out of bounds exception involving ql.exec.vector.expressions.aggregates.gen.VectorUDAFAvgDouble * [HIVE-7423] - produce hive-exec-core.jar from ql module * [HIVE-7424] - HiveException: Error evaluating concat(concat(' ', str2), ' ') in ql.exec.vector.VectorSelectOperator.processOp * [HIVE-7426] - ClassCastException: ...IntWritable cannot be cast to ...Text involving ql.udf.generic.GenericUDFBasePad.evaluate * [HIVE-7429] - Set replication for archive called before file exists * [HIVE-7433] - ColumnMappins.ColumnMapping should expose public accessors for its fields * [HIVE-7441] - Custom partition scheme gets rewritten with hive scheme upon concatenate * [HIVE-7450] - Database should inherit perms of warehouse dir * [HIVE-7451] - pass function name in create/drop function to authorization api * [HIVE-7452] - Boolean comparison is done through reference equality rather than using equals * [HIVE-7459] - Fix NPE when an empty file is included in a Hive query that uses CombineHiveInputFormat * [HIVE-7470] - Wrong Thrift declaration for {{ShowCompactResponseElement}} * [HIVE-7472] - CLONE - Import fails for tables created with default text, sequence and orc file formats using HCatalog API * [HIVE-7473] - Null values in DECIMAL columns cause serialization issues with HCatalog * [HIVE-7475] - Beeline requires newline at the end of each query in a file * [HIVE-7481] - The planning side changes for SMB join on hive-tez * [HIVE-7482] - The execution side changes for SMB join in hive-tez * [HIVE-7486] - Delete jar should close current classloader * [HIVE-7488] - pass column names being used for inputs to authorization api * [HIVE-7490] - Revert ORC stripe size * [HIVE-7494] - ORC returns empty rows for constant folded date queries * [HIVE-7508] - Kerberos support for streaming * [HIVE-7514] - Vectorization does not handle constant expression whose value is NULL * [HIVE-7521] - Reference equality is used on Boolean in NullScanOptimizer#WhereFalseProcessor#process() * [HIVE-7522] - Update .q.out for cluster_tasklog_retrieval.q test * [HIVE-7529] - load data query fails on hdfs federation + viewfs * [HIVE-7531] - auxpath parameter does not handle paths relative to current working directory. * [HIVE-7533] - sql std auth - set authorization privileges for tables when created from hive cli * [HIVE-7538] - Fix eclipse:eclipse after HIVE-7496 * [HIVE-7539] - streaming windowing UDAF seems to be broken without Partition Spec * [HIVE-7553] - avoid the scheduling maintenance window for every jar change * [HIVE-7557] - When reduce is vectorized, dynpart_sort_opt_vectorization.q under Tez fails * [HIVE-7558] - HCatLoader reuses credentials across jobs * [HIVE-7563] - ClassLoader should be released from LogFactory * [HIVE-7574] - CommonJoinOperator.checkAndGenObject calls LOG.Trace per row from probe side in a HashMap join consuming 4% of the CPU * [HIVE-7576] - Add PartitionSpec support in HCatClient API * [HIVE-7579] - error message for 'drop admin role' in sql std auth mode is not informative * [HIVE-7583] - Use FileSystem.access() if available to check file access for user * [HIVE-7592] - List Jars or Files are not supported by Beeline * [HIVE-7595] - isKerberosMode() does a case sensitive comparison * [HIVE-7599] - NPE in MergeTask#main() when -format is absent * [HIVE-7600] - ConstantPropagateProcFactory uses reference equality on Boolean * [HIVE-7618] - TestDDLWithRemoteMetastoreSecondNamenode unit test failure * [HIVE-7620] - Hive metastore fails to start in secure mode due to "java.lang.NoSuchFieldError: SASL_PROPS" error * [HIVE-7623] - hive partition rename fails if filesystem cache is disabled * [HIVE-7629] - Problem in SMB Joins between two Parquet tables * [HIVE-7634] - Use Configuration.getPassword() if available to eliminate passwords from hive-site.xml * [HIVE-7635] - Query having same aggregate functions but different case throws IndexOutOfBoundsException * [HIVE-7637] - Change throws clause for Hadoop23Shims.ProxyFileSystem23.access() * [HIVE-7638] - Disallow CREATE VIEW when created with a temporary table * [HIVE-7645] - Hive CompactorMR job set NUM_BUCKETS mistake * [HIVE-7647] - Beeline does not honor --headerInterval and --color when executing with "-e" * [HIVE-7648] - authorization check api should provide table for create table,drop/create index, and db for create/switch db * [HIVE-7649] - Support column stats with temporary tables * [HIVE-7658] - Hive search order for hive-site.xml when using --config option * [HIVE-7664] - VectorizedBatchUtil.addRowToBatchFrom is not optimized for Vectorized execution and takes 25% CPU * [HIVE-7666] - Join selectivity calculation should use exponential back-off for conjunction predicates * [HIVE-7667] - handle cast for long in get_aggr_stats() api for metastore for mysql * [HIVE-7669] - parallel order by clause on a string column fails with IOException: Split points are out of order * [HIVE-7673] - Authorization api: missing privilege objects in create table/view * [HIVE-7676] - JDBC: Support more DatabaseMetaData, ResultSetMetaData methods * [HIVE-7678] - add more test cases for tables qualified with database/schema name * [HIVE-7680] - Do not throw SQLException for HiveStatement getMoreResults and setEscapeProcessing(false) * [HIVE-7681] - qualified tablenames usage does not work with several alter-table commands * [HIVE-7682] - HadoopThriftAuthBridge20S should not reset configuration unless required * [HIVE-7683] - Test TestMinimrCliDriver.testCliDriver_ql_rewrite_gbtoidx is still failing * [HIVE-7694] - SMB join on tables differing by number of sorted by columns with same join prefix fails * [HIVE-7695] - hive stats issue when insert query is appending data into table * [HIVE-7700] - authorization api - HivePrivilegeObject for permanent function should have database name set * [HIVE-7701] - Upgrading tez to 0.4.1 causes metadata only query to fail. * [HIVE-7704] - Create tez task for fast file merging * [HIVE-7710] - Rename table across database might fail * [HIVE-7712] - hive-exec-0.13.0.2.1.2.0-402.jar contains avro classes compiled against hadoop-v1 * [HIVE-7722] - TestJdbcDriver2.testDatabaseMetaData fails after HIVE-7676 * [HIVE-7723] - Explain plan for complex query with lots of partitions is slow due to in-efficient collection used to find a matching ReadEntity * [HIVE-7730] - Extend ReadEntity to add accessed columns from query * [HIVE-7733] - Ambiguous column reference error on query * [HIVE-7738] - tez select sum(decimal) from union all of decimal and null throws NPE * [HIVE-7741] - Don't synchronize WriterImpl.addRow() when dynamic.partition is enabled * [HIVE-7744] - In Windowing Streaming mode Avg and Sum give incorrect results when Wdw size is same as partition size * [HIVE-7753] - Same operand appears on both sides of > in DataType#compareByteArray() * [HIVE-7759] - document hive cli authorization behavior when SQL std auth is enabled * [HIVE-7760] - Constants in VirtualColumn should be final * [HIVE-7764] - Support all JDBC-HiveServer2 authentication modes on a secure cluster * [HIVE-7769] - add --SORT_BEFORE_DIFF to union all .q tests * [HIVE-7770] - Undo backward-incompatible behaviour change introduced by HIVE-7341 * [HIVE-7771] - ORC PPD fails for some decimal predicates * [HIVE-7774] - Issues with location path for temporary external tables * [HIVE-7777] - Add CSV Serde based on OpenCSV * [HIVE-7784] - Created the needed indexes on Hive.PART_COL_STATS for CBO * [HIVE-7786] - add --SORT_BEFORE_DIFF to union all tez .q.out files * [HIVE-7787] - Reading Parquet file with enum in Thrift Encoding throws NoSuchFieldError * [HIVE-7800] - Parquet Column Index Access Schema Size Checking * [HIVE-7807] - Refer to umask property using FsPermission.UMASK_LABEL. * [HIVE-7812] - Disable CombineHiveInputFormat when ACID format is used * [HIVE-7813] - Hive join key not null shouldn't be generated for partition column * [HIVE-7823] - HIVE-6185 removed Partition.getPartition * [HIVE-7824] - CLIServer.getOperationStatus eats ExceutionException * [HIVE-7828] - TestCLIDriver.parquet_join.q is failing on trunk * [HIVE-7829] - Entity.getLocation can throw an NPE * [HIVE-7834] - Use min, max and NDV from the stats to better estimate many to many vs one to many inner joins * [HIVE-7840] - Generated hive-default.xml.template mistakenly refers to property "name"s as "key"s * [HIVE-7841] - Case, When, Lead, Lag UDF is missing annotation * [HIVE-7846] - authorization api should support group, not assume case insensitive role names * [HIVE-7847] - query orc partitioned table fail when table column type change * [HIVE-7851] - Fix NPE in split generation on Tez 0.5 * [HIVE-7857] - Hive query fails after Tez session times out * [HIVE-7859] - Tune zlib compression in ORC to account for the encoding strategy * [HIVE-7863] - Potential null reference in TxnDbUtil#prepareDb() * [HIVE-7865] - Extend TestFileDump test case to printout ORC row index information * [HIVE-7878] - add -- SORT_BEFORE_DIFF to optimize_nullscan.q test * [HIVE-7883] - DBTxnManager trying to close already closed metastore client connection * [HIVE-7887] - VectorFileSinkOp does not publish the stats correctly * [HIVE-7889] - Query fails with char partition column * [HIVE-7890] - SessionState creates HMS Client while not impersonating * [HIVE-7891] - Table-creation fails through HCatClient for Oracle-based metastore. * [HIVE-7892] - Thrift Set type not working with Hive * [HIVE-7895] - Storage based authorization should consider sticky bit for drop actions * [HIVE-7897] - ObjectStore not using getPassword() for JDO connection string * [HIVE-7899] - txnMgr should be session specific * [HIVE-7901] - CLONE - pig -useHCatalog with embedded metastore fails to pass command line args to metastore (org.apache.hive.hcatalog version) * [HIVE-7902] - Cleanup hbase-handler/pom.xml dependency list * [HIVE-7911] - Guaranteed ClassCastException in AccumuloRangeGenerator * [HIVE-7913] - Simplify filter predicates for CBO * [HIVE-7914] - Simplify join predicates for CBO to avoid cross products * [HIVE-7915] - Expose High and Low value in plan.ColStatistics * [HIVE-7919] - sql std auth: user with 'admin option' for role should be able to list all users in the role * [HIVE-7927] - Checking sticky bit needs shim * [HIVE-7936] - Support for handling Thrift Union types * [HIVE-7943] - hive.security.authorization.createtable.owner.grants is ineffective with Default Authorization * [HIVE-7944] - current update stats for columns of a partition of a table is not correct * [HIVE-7946] - CBO: Merge CBO changes to Trunk * [HIVE-7949] - Create table LIKE command doesn't set new owner * [HIVE-7950] - StorageHandler resources aren't added to Tez Session if already Session is already Open * [HIVE-7957] - Revisit event version handling in dynamic partition pruning on Tez * [HIVE-7971] - Support alter table change/replace/add columns for existing partitions * [HIVE-7972] - hiveserver2 specific configuration file is not getting used * [HIVE-7976] - Merge tez branch into trunk (tez 0.5.0) * [HIVE-7982] - Regression in explain with CBO enabled due to issuing query per K,V for the stats * [HIVE-7984] - AccumuloOutputFormat Configuration items from StorageHandler not re-set in Configuration in Tez * [HIVE-7985] - With CBO enabled cross product is generated when a subquery is present * [HIVE-7987] - Storage based authorization - NPE for drop view * [HIVE-7993] - With CBO enabled Q75 fails with RuntimeException: cannot find field _col69 from [0:_col18,...] * [HIVE-8002] - Deprecate the use of JDBC client only parameters from the map used to transfer HiveConf configs to the server. * [HIVE-8008] - NPE while reading null decimal value * [HIVE-8012] - TestHiveServer2Concurrency is not implemented * [HIVE-8018] - Fix typo in config var name for dynamic partition pruning * [HIVE-8019] - Missing hive 0.13.1 commit in trunk : export/import statement authorization - CVE-2014-0228 * [HIVE-8022] - Recursive root scratch directory creation is not using hdfs umask properly * [HIVE-8023] - Code in HIVE-6380 eats exceptions * [HIVE-8031] - CBO needs to scale down NDV with selectivity to avoid underestimating * [HIVE-8034] - Don't add colon when no port is specified * [HIVE-8041] - Hadoop-2 build is broken with JDK6 * [HIVE-8044] - Container size and hash table size should be taken into account before deciding to do a MapJoin * [HIVE-8045] - SQL standard auth with cli - Errors and configuration issues * [HIVE-8047] - Lazy char/varchar are not using escape char defined in serde params * [HIVE-8051] - Some union queries fail with dynamic partition pruning on tez * [HIVE-8052] - Vectorization: min() on TimeStamp datatype fails with error "Vector aggregate not implemented: min for type: TIMESTAMP" * [HIVE-8056] - SessionState.dropSessionPaths should use FileSystem.getLocal(conf) to delete local files * [HIVE-8062] - Stats collection for columns fails on a partitioned table with null values in partitioning column * [HIVE-8071] - hive shell tries to write hive-exec.jar for each run * [HIVE-8078] - ORC Delta encoding corrupts data when delta overflows long * [HIVE-8081] - "drop index if exists" fails if table specified does not exist * [HIVE-8082] - generateErrorMessage doesn't handle null ast properly * [HIVE-8083] - Authorization DDLs should not enforce hive identifier syntax for user or group * [HIVE-8085] - stats optimizer should not use Description annotation to figure out function mapping (because FunctionRegistry doesn't) * [HIVE-8090] - Potential null pointer reference in WriterImpl#StreamFactory#createStream() * [HIVE-8092] - Vectorized Tez count(*) returns NULL instead of 0 when result is empty * [HIVE-8095] - Tez and Vectorized GROUP BY: ClassCastException: ...HiveDecimal cannot be cast to ...HiveDecimalWritable * [HIVE-8099] - IN operator for partition column fails when the partition column type is DATE * [HIVE-8101] - Hive on spark Issue during compiling hive source * [HIVE-8102] - Partitions of type 'date' behave incorrectly with daylight saving time. * [HIVE-8103] - Read ACID tables with FetchOperator returns no rows * [HIVE-8104] - Insert statements against ACID tables NPE when vectorization is on * [HIVE-8105] - booleans and nulls not handled properly in insert/values * [HIVE-8107] - Bad error message for non-existent table in update and delete * [HIVE-8112] - Change reporting string to reflect update in Tez * [HIVE-8114] - Type resolution for udf arguments of Decimal Type results in error * [HIVE-8115] - Hive select query hang when fields contain map * [HIVE-8126] - Standalone hive-jdbc jar is not packaged in the Hive distribution * [HIVE-8138] - Global Init file should allow specifying file name not only directory * [HIVE-8139] - Upgrade commons-lang from 2.4 to 2.6 * [HIVE-8142] - Add merge operators to queryplan.thrift instead of generated source file * [HIVE-8143] - Create root scratch dir with 733 instead of 777 perms * [HIVE-8146] - Test TestTempletonUtils.testFindContainingJar failing * [HIVE-8148] - HDFS Path named with file:// instead of file:/// results in Unit test failures in Windows * [HIVE-8149] - hive.optimize.reducededuplication should be set to false for IUD ops * [HIVE-8151] - Dynamic partition sort optimization inserts record wrongly to partition when used with GroupBy * [HIVE-8152] - Update with expression in set fails * [HIVE-8153] - Reduce the verbosity of debug logs in ORC record reader * [HIVE-8154] - HadoopThriftAuthBridge20S.getHadoopSaslProperties is incompatible with Hadoop 2.4.1 and later * [HIVE-8156] - Vectorized reducers need to avoid memory build-up during a single key * [HIVE-8158] - Optimize writeValue/setValue in VectorExpressionWriterFactory (in VectorReduceSinkOperator codepath) * [HIVE-8162] - Dynamic sort optimization propagates additional columns even in the absence of order by * [HIVE-8167] - mvn install command broken by HIVE-8126 commit * [HIVE-8169] - Windows: alter table ..set location from hcatalog failed with NullPointerException * [HIVE-8170] - Hive Metastore schema script missing for mssql for v0.14.0 * [HIVE-8171] - Tez and Vectorized Reduce doesn't create scratch columns * [HIVE-8175] - Hive metastore upgrade from v0.13.0 to v0.14.0 script for Oracle is missing an upgrade step * [HIVE-8178] - OrcNewInputFormat::getSplits() calls OrcInputFormat.generateSplitsInfo twice * [HIVE-8179] - Fetch task conversion: Remove some dependencies on AST * [HIVE-8184] - inconsistence between colList and columnExprMap when ConstantPropagate is applied to subquery * [HIVE-8185] - hive-jdbc-0.14.0-SNAPSHOT-standalone.jar fails verification for signatures in build * [HIVE-8188] - ExprNodeGenericFuncEvaluator::_evaluate() loads class annotations in a tight loop * [HIVE-8189] - A select statement with a subquery is failing with HBaseSerde * [HIVE-8191] - Update and delete on tables with non Acid output formats gives runtime error * [HIVE-8196] - Joining on partition columns with fetch column stats enabled results it very small CE which negatively affects query performance * [HIVE-8200] - Make beeline use the hive-jdbc standalone jar * [HIVE-8201] - Remove hardwiring to HiveInputFormat in acid qfile tests * [HIVE-8203] - ACID operations result in NPE when run through HS2 * [HIVE-8205] - Using strings in group type fails in ParquetSerDe * [HIVE-8210] - TezJobMonitor should print time spent in Application (RUNNING) * [HIVE-8212] - Regression for hcat commandline alter view set tblproperties * [HIVE-8217] - WebHCat 'jobs' endpoint fails if it runs into issues with any of the jobs * [HIVE-8221] - authorize additional metadata read operations in metastore storage based authorization * [HIVE-8225] - CBO trunk merge: union11 test fails due to incorrect plan * [HIVE-8226] - Vectorize dynamic partitioning in VectorFileSinkOperator * [HIVE-8227] - NPE w/ hive on tez when doing unions on empty tables * [HIVE-8229] - Add multithreaded tests for the Hive Writable data types * [HIVE-8231] - Error when insert into empty table with ACID * [HIVE-8235] - Insert into partitioned bucketed sorted tables fails with "this file is already being created by" * [HIVE-8236] - VectorHashKeyWrapper allocates too many zero sized arrays * [HIVE-8239] - MSSQL upgrade schema scripts does not map Java long datatype columns correctly for transaction related tables * [HIVE-8240] - VectorColumnAssignFactory throws "Incompatible Bytes vector column and primitive category VARCHAR" * [HIVE-8241] - With vectorization enabled count(distinct)) fails with ClassCastException * [HIVE-8246] - HiveServer2 in http-kerberos mode is restrictive on client usernames * [HIVE-8248] - TestHCatLoader.testReadDataPrimitiveTypes() occasionally fails * [HIVE-8250] - Truncating table doesnt invalidate stats * [HIVE-8257] - Accumulo introduces old hadoop-client dependency * [HIVE-8258] - Compactor cleaners can be starved on a busy table or partition. * [HIVE-8260] - CBO : Query query has date_dim d1,date_dim d2 and date_dim d3 but the explain has d1, d1 and d1 * [HIVE-8261] - CBO : Predicate pushdown is removed by Optiq * [HIVE-8263] - CBO : TPC-DS Q64 is item is joined last with store_sales while it should be first as it is the most selective * [HIVE-8269] - Revert HIVE-8200 (Make beeline use the hive-jdbc standalone jar) * [HIVE-8270] - JDBC uber jar is missing some classes required in secure setup. * [HIVE-8271] - Jackson incompatibility between hadoop-2.4 and hive-14 * [HIVE-8272] - Query with particular decimal expression causes NPE during execution initialization * [HIVE-8273] - Beeline doesn't print applicationID for submitted DAG * [HIVE-8277] - IP address string in HS2, metastore have a "/" prefix * [HIVE-8279] - sql std auth - additional test cases * [HIVE-8280] - CBO : When filter is applied on dimension table PK/FK code path is not in effect. * [HIVE-8281] - NPE with dynamic partition pruning on Tez * [HIVE-8283] - Missing break in FilterSelectivityEstimator#visitCall() * [HIVE-8284] - Equality comparison is done between two floating point variables in HiveRelMdUniqueKeys#getUniqueKeys() * [HIVE-8287] - Metadata action errors don't have information about cause * [HIVE-8290] - With DbTxnManager configured, all ORC tables forced to be transactional * [HIVE-8291] - ACID : Reading from partitioned bucketed tables has high overhead, 50% of time is spent in OrcInputFormat.getReader * [HIVE-8292] - Reading from partitioned bucketed tables has high overhead in MapOperator.cleanUpInputFileChangedOp * [HIVE-8296] - Tez ReduceShuffle Vectorization needs 2 data buffers (key and value) for adding rows * [HIVE-8298] - Incorrect results for n-way join when join expressions are not in same order across joins * [HIVE-8299] - HiveServer2 in http-kerberos & doAs=true is failing with org.apache.hadoop.security.AccessControlException * [HIVE-8304] - Tez Reduce-Side GROUP BY Vectorization doesn't copy NULL keys correctly * [HIVE-8306] - Map join sizing done by auto.convert.join.noconditionaltask.size doesn't take into account Hash table overhead and results in OOM * [HIVE-8310] - RetryingHMSHandler is not used when kerberos auth enabled * [HIVE-8311] - Driver is encoding transaction information too late * [HIVE-8313] - Optimize evaluation for ExprNodeConstantEvaluator and ExprNodeNullEvaluator * [HIVE-8314] - Restore thrift string interning of HIVE-7975 * [HIVE-8315] - CBO : Negate condition underestimates selectivity which results in an in-efficient plan * [HIVE-8316] - CBO : cardinality estimation for filters is much lower than actual row count * [HIVE-8318] - Null Scan optimizer throws exception when no partitions are selected * [HIVE-8321] - Fix serialization of TypeInfo for qualified types * [HIVE-8322] - VectorReduceSinkOperator: ClassCastException: ~StandardUnionObjectInspector$StandardUnion cannot be cast to ~IntWritable * [HIVE-8324] - Shim KerberosName (causes build failure on hadoop-1) * [HIVE-8328] - MapJoin implementation in Tez should not reload hashtables * [HIVE-8332] - Reading an ACID table with vectorization on results in NPE * [HIVE-8335] - TestHCatLoader/TestHCatStorer failures on pre-commit tests * [HIVE-8336] - Update pom, now that Optiq is renamed to Calcite * [HIVE-8340] - HiveServer2 service doesn't stop backend jvm process, which prevents follow-up service start. * [HIVE-8341] - Transaction information in config file can grow excessively large * [HIVE-8344] - Hive on Tez sets mapreduce.framework.name to yarn-tez * [HIVE-8348] - Fix Hive to match changes introduced by TEZ-1510 * [HIVE-8349] - DISTRIBUTE BY should work with tez auto-parallelism enabled * [HIVE-8354] - HIVE-7156 introduced required dependency on tez * [HIVE-8361] - NPE in PTFOperator when there are empty partitions * [HIVE-8363] - AccumuloStorageHandler compile failure hadoop-1 * [HIVE-8364] - We're not waiting for all inputs in MapRecordProcessor on Tez * [HIVE-8366] - CBO fails if there is a table sample in subquery * [HIVE-8367] - delete writes records in wrong order in some cases * [HIVE-8368] - compactor is improperly writing delete records in base file * [HIVE-8372] - Potential NPE in Tez MergeFileRecordProcessor * [HIVE-8378] - NPE in TezTask due to null counters * [HIVE-8380] - NanoTime class serializes and deserializes Timestamp incorrectly * [HIVE-8386] - HCAT api call is case sensitive on fields in struct column * [HIVE-8387] - add retry logic to ZooKeeperStorage in WebHCat * [HIVE-8389] - Fix CBO when indexes are used * [HIVE-8390] - CBO produces annoying exception message and wraps exceptions too much * [HIVE-8391] - Comparion between TIMESTAMP and Integer types goes to STRING as "common comparison denominator" instead of a numeric type * [HIVE-8392] - HiveServer2 Operation.close fails on windows * [HIVE-8393] - Handle SIGINT on Tez * [HIVE-8394] - HIVE-7803 doesn't handle Pig MultiQuery, can cause data-loss. * [HIVE-8399] - Build failure on trunk & 14 branch * [HIVE-8400] - Fix building and packaging hwi war file * [HIVE-8401] - OrcFileMergeOperator only close last orc file it opened, which resulted in stale data in table directory * [HIVE-8402] - Orc pushing SARGs into delta files causing ArrayOutOfBoundsExceptions * [HIVE-8403] - Build broken by datanucleus.org being offline * [HIVE-8404] - ColumnPruner doesnt prune columns from limit operator * [HIVE-8407] - [CBO] Handle filters with non-boolean return type * [HIVE-8408] - hcat cli throws NPE when authorizer using new api is enabled * [HIVE-8409] - SMB joins fail intermittently on tez * [HIVE-8411] - Support partial partition spec for certain ALTER PARTITION statements * [HIVE-8413] - [CBO] Handle ill-formed queries which have distinct, having in incorrect context * [HIVE-8415] - Vectorized comparison of timestamp and integer needs to treat integer as seconds since epoch * [HIVE-8417] - round(decimal, negative) errors out/wrong results with reduce side vectorization * [HIVE-8421] - [CBO] Use OptiqSemanticException in error conditions * [HIVE-8427] - Hive Streaming : secure streaming hangs leading to time outs. * [HIVE-8429] - Add records in/out counters * [HIVE-8433] - CBO loses a column during AST conversion * [HIVE-8434] - Vectorization logic using wrong values for DATE and TIMESTAMP partitioning columns in vectorized row batches... * [HIVE-8442] - Revert HIVE-8403 * [HIVE-8444] - update pom to junit 4.11 * [HIVE-8445] - TestColumnAccess, TestReadEntityDirect use same table names * [HIVE-8452] - Cleanup handling of resource configuration for tez * [HIVE-8460] - ORC SARG literal creation for double from float may lead to wrong evaluation of SARG * [HIVE-8461] - Make Vectorized Decimal query results match Non-Vectorized query results with respect to trailing zeroes... .0000 * [HIVE-8462] - CBO duplicates columns * [HIVE-8464] - Vectorized reducer nested group by query returns wrong results * [HIVE-8474] - Vectorized reads of transactional tables fail when not all columns are selected * [HIVE-8475] - add test case for use of index from not-current database * [HIVE-8476] - JavaDoc updates to HiveEndPoint.newConnection() for secure streaming with Kerberos * [HIVE-8478] - Vectorized Reduce-Side Group By doesn't handle Decimal type correctly * [HIVE-8479] - Tez sessions cannot change queues once assigned to one within a CLI session * [HIVE-8484] - HCatalog throws an exception if Pig job is of type 'fetch' * [HIVE-8489] - Add sanity check to dynamic partition pruning * [HIVE-8495] - Add progress bar for Hive on Tez queries * [HIVE-8497] - StatsNoJobTask doesn't close RecordReader, FSDataInputStream of which keeps open to prevent stale data clean * [HIVE-8498] - Insert into table misses some rows when vectorization is enabled * [HIVE-8510] - HIVE-8462 didn't update tez test output * [HIVE-8511] - fix build failure: cbo_correctness on tez * [HIVE-8514] - TestCliDriver.testCliDriver_index_in_db fails in trunk * [HIVE-8517] - When joining on partition column NDV gets overridden by StatsUtils.getColStatisticsFromExpression * [HIVE-8524] - When table is renamed stats are lost as changes are not propagated to metastore tables TAB_COL_STATS and PART_COL_STATS * [HIVE-8526] - Hive : CBO incorrect join order in TPC-DS Q45 as self join selectivity has incorrect CE * [HIVE-8534] - sql std auth : update configuration whitelist for 0.14 * [HIVE-8543] - Compactions fail on metastore using postgres * [HIVE-8546] - Handle "add archive scripts.tar.gz" in Tez * [HIVE-8547] - CBO and/or constant propagation breaks partition_varchar2 test * [HIVE-8550] - Hive cannot load data into partitioned table with Unicode key * [HIVE-8551] - NPE in FunctionRegistry (affects CBO in negative tests) * [HIVE-8555] - Too many casts results in loss of original string representation for constant * [HIVE-8557] - automatically setup ZooKeeperTokenStore to use kerberos authentication when kerberos is enabled * [HIVE-8558] - CBO: enable n-way joins after CBO join reordering * [HIVE-8560] - SerDes that do not inherit AbstractSerDe do not get table properties during initialize() * [HIVE-8562] - ResultSet.isClosed sometimes doesn't work with mysql * [HIVE-8563] - Running annotate_stats_join_pkfk.q in TestMiniTezCliDriver is causing NPE * [HIVE-8566] - Vectorized queries output wrong timestamps * [HIVE-8567] - Vectorized queries output extra stuff for Binary columns * [HIVE-8575] - CBO: decimal_udf is broken by recent changes (and other tests have type changes) * [HIVE-8576] - Guaranteed NPE in StatsRulesProcFactory * [HIVE-8577] - Cannot deserialize Avro schema with a map with null values * [HIVE-8579] - Guaranteed NPE in DDLSemanticAnalyzer * [HIVE-8586] - Record counters aren't updated correctly for vectorized queries * [HIVE-8587] - Vectorized Extract operator needs to update the Vectorization Context column map * [HIVE-8588] - sqoop REST endpoint fails to send appropriate JDBC driver to the cluster * [HIVE-8596] - HiveServer2 dynamic service discovery: ZK throws too many connections error * [HIVE-8603] - auto_sortmerge_join_5 is getting stuck on tez * [HIVE-8604] - Re-enable auto_sortmerge_join_5 on tez * [HIVE-8605] - HIVE-5799 breaks backward compatibility for time values in config * [HIVE-8611] - grant/revoke syntax should support additional objects for authorization plugins * [HIVE-8612] - Support metadata result filter hooks * [HIVE-8614] - Upgrade hive to use tez version 0.5.2-SNAPSHOT * [HIVE-8615] - beeline csv,tsv outputformat needs backward compatibility mode * [HIVE-8619] - CBO causes some more type problems * [HIVE-8620] - CBO: HIVE-8433 RowResolver check is too stringent * [HIVE-8624] - Record counters don't work with Tez container reuse * [HIVE-8625] - Some union queries result in plans with many unions with CBO on * [HIVE-8628] - NPE in case of shuffle join in tez * [HIVE-8629] - Streaming / ACID : hive cli session creation takes too long and times out if execution engine is tez * [HIVE-8631] - Compressed transaction list cannot be parsed in job.xml * [HIVE-8632] - VectorKeyHashWrapper::duplicateTo allocates too many zero sized arrays * [HIVE-8634] - HiveServer2 fair scheduler queue mapping doesn't handle the secondary groups rules correctly * [HIVE-8635] - CBO: ambiguous_col negative test no longer fails * [HIVE-8641] - Disable skew joins in tez. * [HIVE-8643] - DDL operations via WebHCat with doAs parameter in secure cluster fail * [HIVE-8646] - Hive class loading failure when executing Hive action via oozie workflows * [HIVE-8647] - HIVE-8186 causes addition of same child operator multiple times * [HIVE-8660] - sql std auth: property missing from whitelist - hive.exec.dynamic.partition.mode * [HIVE-8663] - Fetching Vectorization scratch column map in Reduce-Side stop working * [HIVE-8664] - Use Apache Curator in JDBC Driver and HiveServer2 for better reliability * [HIVE-8665] - Fix misc unit tests on Windows * [HIVE-8668] - mssql sql script has carriage returns * [HIVE-8671] - Overflow in estimate row count and data size with fetch column stats * [HIVE-8675] - Increase thrift server protocol test coverage * [HIVE-8677] - TPC-DS Q51 : fails with "init not supported" exception in GenericUDAFStreamingEvaluator.init * [HIVE-8683] - User name and group name cannot be the same when grant role * [HIVE-8685] - DDL operations in WebHCat set proxy user to "null" in unsecure mode * [HIVE-8687] - Support Avro through HCatalog * [HIVE-8688] - serialized plan OutputStream is not being closed * [HIVE-8689] - handle overflows in statistics better * [HIVE-8697] - Vectorized round(decimal, negative) produces wrong results * [HIVE-8698] - default log4j.properties not included in jar files anymore * [HIVE-8703] - More Windows unit test fixes * [HIVE-8704] - HivePassThroughOutputFormat cannot proxy more than one kind of OF (in one process) * [HIVE-8705] - Support using pre-authenticated subject in kerberized HiveServer2 HTTP mode * [HIVE-8711] - DB deadlocks not handled in TxnHandler for Postgres, Oracle, and SQLServer * [HIVE-8713] - Unit test TestParquetTimestampUtils.testTimezone failing * [HIVE-8714] - getDatabase reports direct SQL error when database is missing * [HIVE-8715] - Hive 14 upgrade scripts can fail for statistics if database was created using auto-create * [HIVE-8720] - Update orc_merge tests to make it consistent across OS'es * [HIVE-8723] - Set reasonable connection timeout for CuratorFramework ZooKeeper clients in Hive * [HIVE-8724] - Right outer join produces incorrect result on Tez * [HIVE-8727] - Dag summary has incorrect row counts and duration per vertex * [HIVE-8732] - ORC string statistics are not merged correctly * [HIVE-8733] - HiveServer2 dynamic service discovery not picking correct IP address when hive.server2.thrift.bind.host is not set * [HIVE-8735] - statistics update can fail due to long paths * [HIVE-8736] - add ordering to cbo_correctness to make result consistent * [HIVE-8737] - setEnv is not portable, which fails TestCliDriverMethods#testprocessInitFiles on Windows * [HIVE-8740] - Sorted dynamic partition does not work correctly with constant folding * [HIVE-8745] - Joins on decimal keys return different results whether they are run as reduce join or map join * [HIVE-8747] - Estimate number of rows for table with 0 rows overflows resulting in an in-efficient plan * [HIVE-8752] - Disjunction cardinality estimation has selectivity of 1 * [HIVE-8754] - Sqoop job submission via WebHCat doesn't properly localize required jdbc jars in secure cluster * [HIVE-8759] - HiveServer2 dynamic service discovery should add hostname instead of ipaddress to ZooKeeper * [HIVE-8764] - Windows: HiveServer2 TCP SSL cannot recognize localhost * [HIVE-8766] - Hive RetryHMSHandler should be retrying the metastore operation in case of NucleusException * [HIVE-8771] - Abstract merge file operator does not move/rename incompatible files correctly * [HIVE-8772] - zookeeper info logs are always printed from beeline with service discovery mode * [HIVE-8778] - ORC split elimination can cause NPE when column statistics is null * [HIVE-8781] - Nullsafe joins are busted on Tez * [HIVE-8782] - HBase handler doesn't compile with hadoop-1 * [HIVE-8785] - HiveServer2 LogDivertAppender should be more selective for beeline getLogs * [HIVE-8794] - Hive on Tez leaks AMs when killed before first dag is run * [HIVE-8798] - Some Oracle deadlocks not being caught in TxnHandler * [HIVE-8799] - boatload of missing apache headers ** Improvement * [HIVE-538] - make hive_jdbc.jar self-containing * [HIVE-860] - Persistent distributed cache * [HIVE-2365] - SQL support for bulk load into HBase * [HIVE-3005] - Skip execution phase for queries that contain "LIMIT 0" clause * [HIVE-3006] - Skip execution of queries with always false WHERE clauses * [HIVE-3595] - Hive should adapt new FsShell commands since Hadoop 2 has changed FsShell argument structures * [HIVE-3635] - allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type * [HIVE-3684] - Add support for filter pushdown for composite keys * [HIVE-3907] - Hive should support adding multiple resources at once * [HIVE-4867] - Deduplicate columns appearing in both the key list and value list of ReduceSinkOperator * [HIVE-4997] - HCatalog doesn't allow multiple input tables * [HIVE-5072] - [WebHCat]Enable directly invoke Sqoop job through Templeton * [HIVE-5160] - HS2 should support .hiverc * [HIVE-5298] - AvroSerde performance problem caused by HIVE-3833 * [HIVE-5370] - format_number udf should take user specifed format as argument * [HIVE-5408] - Method matching needs to be improved in NumericOpMethodResolver to suport decimal type with parameters * [HIVE-5447] - HiveServer2 should allow secure impersonation over LDAP or other non-kerberos connection * [HIVE-5652] - Improve JavaDoc of UDF class * [HIVE-5771] - Constant propagation optimizer for Hive * [HIVE-5799] - session/operation timeout for hiveserver2 * [HIVE-5821] - Evaluate the usefulness of UNKNOWN state in HiveServer2 state transitions and get rid of it if not required * [HIVE-5871] - Use multiple-characters as field delimiter * [HIVE-5961] - Add explain authorize for checking privileges * [HIVE-6024] - Load data local inpath unnecessarily creates a copy task * [HIVE-6037] - Synchronize HiveConf with hive-default.xml.template and support show conf * [HIVE-6089] - Add metrics to HiveServer2 * [HIVE-6132] - Support hbase filters for Hive HBase Integration * [HIVE-6147] - Support avro data stored in HBase columns * [HIVE-6304] - Update HCatReader/Writer docs to reflect recent changes * [HIVE-6325] - Enable using multiple concurrent sessions in tez * [HIVE-6410] - Allow output serializations separators to be set for HDFS path as well. * [HIVE-6411] - Support more generic way of using composite key for HBaseHandler * [HIVE-6430] - MapJoin hash table has large memory overhead * [HIVE-6438] - Sort query result for test, removing order by clause * [HIVE-6473] - Allow writing HFiles via HBaseStorageHandler table * [HIVE-6510] - Clean up math based UDFs * [HIVE-6561] - Beeline should accept -i option to Initializing a SQL file * [HIVE-6584] - Add HiveHBaseTableSnapshotInputFormat * [HIVE-6593] - Create a maven assembly for hive-jdbc * [HIVE-6657] - Add test coverage for Kerberos authentication implementation using Hadoop's miniKdc * [HIVE-6677] - HBaseSerDe needs to be refactored * [HIVE-6691] - support decimals for optimized hashmap keys in MapJoin * [HIVE-6694] - Beeline should provide a way to execute shell command as Hive CLI does * [HIVE-6799] - HiveServer2 needs to map kerberos name to local name before proxy check * [HIVE-6899] - Add an ability to specify the type of execution to use (async/sync execution) on JDBC client * [HIVE-6920] - Parquet Serde Simplification * [HIVE-6923] - Use slf4j For Logging Everywhere * [HIVE-6924] - MapJoinKeyBytes::hashCode() should use Murmur hash * [HIVE-6938] - Add Support for Parquet Column Rename * [HIVE-6973] - HiveServer2 should support a non-kerberos user authentication on a secure cluster. * [HIVE-6999] - Add streaming mode to PTFs * [HIVE-7000] - Several issues with javadoc generation * [HIVE-7022] - Replace BinaryWritable with BytesWritable in Parquet serde * [HIVE-7026] - Support newly added role related APIs for v1 authorizer * [HIVE-7048] - CompositeKeyHBaseFactory should not use FamilyFilter * [HIVE-7100] - Users of hive should be able to specify skipTrash when dropping tables. * [HIVE-7127] - Handover more details on exception in hiveserver2 * [HIVE-7136] - Allow Hive to read hive scripts from any of the supported file systems in hadoop eco-system * [HIVE-7137] - Add progressable to writer interfaces so they could report progress while different operations are in progress * [HIVE-7138] - add row index dump capability to ORC file dump * [HIVE-7140] - Bump default hive.metastore.client.socket.timeout to 5 minutes * [HIVE-7142] - Hive multi serialization encoding support * [HIVE-7168] - Don't require to name all columns in analyze statements if stats collection is for all columns * [HIVE-7196] - Configure session by single open session call * [HIVE-7208] - move SearchArgument interface into serde package * [HIVE-7211] - Throws exception if the name of conf var starts with "hive." does not exists in HiveConf * [HIVE-7219] - Improve performance of serialization utils in ORC * [HIVE-7222] - Support timestamp column statistics in ORC and extend PPD for timestamp * [HIVE-7223] - Support generic PartitionSpecs in Metastore partition-functions * [HIVE-7230] - Add Eclipse formatter file for Hive coding conventions * [HIVE-7231] - Improve ORC padding * [HIVE-7243] - Print padding information in ORC file dump * [HIVE-7250] - Adaptive compression buffer size for wide tables in ORC * [HIVE-7361] - using authorization api for RESET, DFS, ADD, DELETE, COMPILE commands * [HIVE-7386] - PTest support non-spot instances and higher cpu masters * [HIVE-7432] - Remove deprecated Avro's Schema.parse usages * [HIVE-7445] - Improve LOGS for Hive when a query is not able to acquire locks * [HIVE-7495] - Print dictionary size in orc file dump * [HIVE-7519] - Refactor QTestUtil to remove its duplication with QFileClient for qtest setup and teardown * [HIVE-7532] - allow disabling direct sql per query with external metastore * [HIVE-7543] - Cleanup of org.apache.hive.service.auth package * [HIVE-7549] - Code cleanup of Task.java and HiveInputFormat.java * [HIVE-7554] - Parquet Hive should resolve column names in case insensitive manner * [HIVE-7562] - Cleanup ExecReducer * [HIVE-7596] - Cleanup OperatorFactory, ReduceSinkOperator, and reportStats * [HIVE-7609] - Collect partition level stats by default * [HIVE-7615] - Beeline should have an option for user to see the query progress * [HIVE-7616] - pre-size mapjoin hashtable based on statistics * [HIVE-7689] - Fix wrong lower case table names in Postgres Metastore back end * [HIVE-7697] - PlanUtils.getTableDesc uses printStackTrace and returns null * [HIVE-7705] - there's a useless threadlocal in LBUtils that shows up in perf profiles * [HIVE-7736] - improve the columns stats update speed for all the partitions of a table * [HIVE-7737] - Hive logs full exception for table not found * [HIVE-7757] - PTest2 separates test files with spaces while QTestGen uses commas * [HIVE-7818] - Support boolean PPD for ORC * [HIVE-7832] - Do ORC dictionary check at a finer level and preserve encoding across stripes * [HIVE-7833] - Remove unwanted allocation in ORC RunLengthIntegerWriterV2 determine encoding function * [HIVE-7876] - further improve the columns stats update speed for all the partitions of a table * [HIVE-7885] - CLIServer.openSessionWithImpersonation logs as if it were openSession * [HIVE-7912] - Don't add is not null filter for partitioning column * [HIVE-7921] - Fix confusing dead assignment in return statement (JavaHiveVarcharObjectInspector) * [HIVE-7923] - populate stats for test tables * [HIVE-7925] - extend current partition status extrapolation to support all DBs * [HIVE-7931] - Convert all tabs to spaces [code cleanup] * [HIVE-7947] - Add message at the end of each testcase with timestamp in Webhcat system tests * [HIVE-7975] - HS2 memory optimization: Internalizing instance fields of Thrift-generated metastore API classes * [HIVE-8036] - PTest SSH Options * [HIVE-8038] - Decouple ORC files split calculation logic from Filesystem's get file location implementation * [HIVE-8042] - Optionally allow move tasks to run in parallel * [HIVE-8096] - Fix a few small nits in TestExtendedAcls * [HIVE-8100] - Add QTEST_LEAVE_FILES to QTestUtil * [HIVE-8137] - Empty ORC file handling * [HIVE-8245] - Collect table read entities at same time as view read entities * [HIVE-8320] - Error in MetaException(message:Got exception: org.apache.thrift.transport.TTransportException java.net.SocketTimeoutException: Read timed out) * [HIVE-8350] - Constant folding should happen before group-by optimization * [HIVE-8358] - Constant folding should happen before PCR * [HIVE-8385] - UNION Operator in Hive * [HIVE-8428] - PCR doesnt remove filters involving casts * [HIVE-8490] - Constant folding should happen before partition pruning * [HIVE-8492] - Enhance Constant Folding to propagate constants for simple expressions * [HIVE-8501] - Fix CBO to use indexes when GenericUDFBridge is applied * [HIVE-8585] - Constant folding should happen before ppd * [HIVE-8597] - SMB join small table side should use the same set of serialized payloads across tasks * [HIVE-8598] - Push constant filters through joins * [HIVE-8748] - jdbc uber jar is missing commons-logging * [HIVE-8779] - Tez in-place progress UI can show wrong estimated time for sub-second queries ** New Feature * [HIVE-5317] - Implement insert, update, and delete in Hive with full ACID support * [HIVE-5823] - Support for DECIMAL primitive type in AvroSerDe * [HIVE-5908] - Use map-join hint to cache intermediate result * [HIVE-6100] - Introduce basic set operations as UDFs * [HIVE-6455] - Scalable dynamic partitioning and bucketing optimization * [HIVE-6469] - skipTrash option in hive command line * [HIVE-6806] - CREATE TABLE should support STORED AS AVRO * [HIVE-7036] - get_json_object bug when extract list of list with index * [HIVE-7054] - Support ELT UDF in vectorized mode * [HIVE-7068] - Integrate AccumuloStorageHandler * [HIVE-7090] - Support session-level temporary tables in Hive * [HIVE-7122] - Storage format for create like table * [HIVE-7158] - Use Tez auto-parallelism in Hive * [HIVE-7203] - Optimize limit 0 * [HIVE-7233] - File hive-hwi-0.13.1 not found on lib folder * [HIVE-7255] - Allow partial partition spec in analyze command * [HIVE-7299] - Enable metadata only optimization on Tez * [HIVE-7341] - Support for Table replication across HCatalog instances * [HIVE-7390] - Make quote character optional and configurable in BeeLine CSV/TSV output * [HIVE-7416] - provide context information to authorization checkPrivileges api call * [HIVE-7430] - Implement SMB join in tez * [HIVE-7446] - Add support to ALTER TABLE .. ADD COLUMN to Avro backed tables * [HIVE-7506] - MetadataUpdater: provide a mechanism to edit the statistics of a column in a table (or a partition of a table) * [HIVE-7509] - Fast stripe level merging for ORC * [HIVE-7547] - Add ipAddress and userName to ExecHook * [HIVE-7587] - Fetch aggregated stats from MetaStore * [HIVE-7604] - Add Metastore API to fetch one or more partition names * [HIVE-7654] - A method to extrapolate columnStats for partitions of a table * [HIVE-7826] - Dynamic partition pruning on Tez * [HIVE-8267] - Exposing hbase cell latest timestamp through hbase columns mappings to hive columns. * [HIVE-8376] - Umbrella Jira for HiveServer2 dynamic service discovery * [HIVE-8531] - Fold is not null filter if there are other comparison filter present on same column * [HIVE-8690] - Move Avro dependency to 1.7.7 ** Task * [HIVE-2974] - Add online docs for from_utc_timestamp() and to_utc_timestamp() * [HIVE-5342] - Remove pre hadoop-0.20.0 related codes * [HIVE-5976] - Decouple input formats from STORED as keywords * [HIVE-6338] - Improve exception handling in createDefaultDb() in Metastore * [HIVE-6432] - Remove deprecated methods in HCatalog * [HIVE-6543] - TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync is failing sometimes * [HIVE-6836] - Upgrade parquet to 1.4.0 * [HIVE-6869] - Golden file updates for tez tests. * [HIVE-6903] - Change default value of hive.metastore.execute.setugi to true * [HIVE-6988] - Hive changes for tez-0.5.x compatibility * [HIVE-7008] - Clean-up some old dead code * [HIVE-7034] - Explain result of TezWork is not deterministic * [HIVE-7095] - Fix test fails for both hadoop-1 and hadoop-2 * [HIVE-7108] - Cleanup HBaseStorageHandler * [HIVE-7126] - Cleanup build warnings while building hive projects * [HIVE-7194] - authorization_ctas.q failing on trunk * [HIVE-7206] - Duplicate declaration of build-helper-maven-plugin in root pom * [HIVE-7240] - Add classifier for avro-mapred jar * [HIVE-7289] - revert HIVE-6469 * [HIVE-7301] - Restore constants moved to HiveConf by HIVE-7211 * [HIVE-7364] - Trunk cannot be built on -Phadoop1 after HIVE-7144 * [HIVE-7365] - Explain authorize for auth2 throws exception * [HIVE-7381] - Class TezEdgeProperty missing license header * [HIVE-7456] - HIVE-7361 missed updating authorization_fail_8 * [HIVE-7496] - Exclude conf/hive-default.xml.template in version control and include it dist profile * [HIVE-7497] - Fix some default values in HiveConf * [HIVE-7498] - NPE on show grant for global privilege * [HIVE-7524] - Enable auto conversion of SMBjoin in presence of constant propagate optimization * [HIVE-7789] - Documentation for AccumuloStorageHandler * [HIVE-7960] - Upgrade to Hadoop 2.5 * [HIVE-8072] - TesParse_union is failing on trunk * [HIVE-8224] - Support Char, Varchar in AvroSerDe * [HIVE-8265] - Build failure on hadoop-1 * [HIVE-8637] - In insert into X select from Y, table properties from X are clobbering those from Y ** Test * [HIVE-7084] - TestWebHCatE2e is failing on trunk * [HIVE-7085] - TestOrcHCatPigStorer.testWriteDecimal tests are failing on trunk * [HIVE-7086] - TestHiveServer2.testConnection is failing on trunk * [HIVE-7252] - TestTezTask.testSubmit fails in trunk * [HIVE-7343] - Update committer list * [HIVE-7684] - Avoid validating hook EnforceReadOnlyTable during test driver cleanup. * [HIVE-8035] - Add SORT_QUERY_RESULTS for test that doesn't guarantee order * [HIVE-8070] - TestHWIServer failed due to wrong references to war and properties file * [HIVE-8213] - TestHWISessionManager failed due to miss hadoop2 dependencies * [HIVE-8360] - Add cross cluster support for webhcat E2E tests * [HIVE-8420] - TestHadoop20SAuthBridge broken with hadoop-1 * [HIVE-8553] - Add a hive smoke test for Apache Bigtop * [HIVE-8618] - Add SORT_QUERY_RESULTS for test that doesn't guarantee order #3 ** Wish * [HIVE-6241] - Remove direct reference of Hadoop23Shims inQTestUtil Release Notes - Hive - Version 0.13.0 ** Sub-task * [HIVE-2436] - Update project naming and description in Hive website * [HIVE-3746] - Fix HS2 ResultSet Serialization Performance Regression * [HIVE-4282] - Implement vectorized column-scalar expressions * [HIVE-4283] - Implement vectorized logical expressions. * [HIVE-4284] - Implement class for vectorized row batch * [HIVE-4370] - Change ORC tree readers to return batches of rows instead of a row * [HIVE-4379] - Implement Vectorized Column-Column expressions * [HIVE-4380] - Implement Vectorized Scalar-Column expressions * [HIVE-4381] - Implement vectorized aggregation expressions * [HIVE-4383] - Implement vectorized string column-scalar filters * [HIVE-4384] - Implement vectorized string functions UPPER(), LOWER(), LENGTH() * [HIVE-4385] - Implement vectorized LIKE filter * [HIVE-4404] - Vectorized filter and select operators * [HIVE-4405] - Generate vectorized execution plan * [HIVE-4416] - Vectorized expression for unary minus. * [HIVE-4431] - Implement vectorized string concatenation * [HIVE-4450] - Extend Vector Aggregates to support GROUP BY * [HIVE-4451] - Add support for string column type vector aggregates: COUNT, MIN and MAX * [HIVE-4452] - Add support for COUNT(*) in vector aggregates * [HIVE-4453] - Input format to read vector data from ORC * [HIVE-4457] - Queries not supported by vectorized code path should fall back to non vector path. * [HIVE-4462] - Finish support for modulo (%) operator for vectorized arithmetic * [HIVE-4472] - OR, NOT Filter logic can lose an array, and always takes time O(VectorizedRowBatch.DEFAULT_SIZE) * [HIVE-4477] - remove redundant copy of arithmetic filter unit test testColOpScalarNumericFilterNullAndRepeatingLogic * [HIVE-4479] - Child expressions are not being evaluated hierarchically in a few templates. * [HIVE-4480] - Implement partition support for vectorized query execution * [HIVE-4481] - Vectorized row batch should be initialized with additional columns to hold intermediate output. * [HIVE-4483] - Input format to read vector data from RC file * [HIVE-4493] - Implement vectorized filter for string column compared to string column * [HIVE-4495] - Implement vectorized string substr * [HIVE-4509] - Integer division should be cast to double. * [HIVE-4511] - Vectorized reader support for Byte Boolean and Timestamp. * [HIVE-4512] - The vectorized plan is not picking right expression class for string concatenation. * [HIVE-4514] - Handle constants in projection * [HIVE-4529] - Add partition support for vectorized ORC Input format * [HIVE-4534] - IsNotNull and NotCol incorrectly handle nulls. * [HIVE-4537] - select * fails on orc table when vectorization is enabled * [HIVE-4539] - Move test utils and fix build to remove false test failures * [HIVE-4541] - Run check-style on the branch and fix style issues. * [HIVE-4544] - VectorizedRowBatchCtx::CreateVectorizedRowBatch should create only the projected columns and not all columns * [HIVE-4548] - Speed up vectorized LIKE filter for special cases abc%, %abc and %abc% * [HIVE-4552] - Vectorized RecordReader for ORC does not set the ColumnVector.IsRepeating correctly * [HIVE-4553] - Column Column, and Column Scalar vectorized execution tests * [HIVE-4592] - fix failure to set output isNull to true and other NULL propagation issues; update arithmetic tests * [HIVE-4595] - Support strings in GROUP BY keys * [HIVE-4596] - Fix serialization exceptions in VectorGroupByOperator * [HIVE-4599] - VectorGroupByOperator steals the non-vectorized children and crashes query if vectorization fails * [HIVE-4602] - Enable running all hive e2e tests under vectorization * [HIVE-4603] - VectorSelectOperator projections change the index of columns for subsequent operators. * [HIVE-4606] - Implement vector group by hash spill * [HIVE-4608] - Vectorized UDFs for Timestamp in nanoseconds * [HIVE-4612] - Vectorized aggregates do not emit proper rows in presence of GROUP BY * [HIVE-4624] - Integrate Vectorized Substr into Vectorized QE * [HIVE-4637] - Fix VectorUDAFSum.txt to honor the expected vector column type * [HIVE-4640] - CommonOrcInputFormat should be the default input format for Orc tables. * [HIVE-4642] - Implement vectorized RLIKE and REGEXP filter expressions * [HIVE-4649] - Unit test failure in TestColumnScalarOperationVectorExpressionEvaluation * [HIVE-4651] - TestVectorGroupByOperator causes asserts in StandardStructObjectInspector.init * [HIVE-4652] - VectorHashKeyWrapperBatch.java should be in vector package (instead of exec) * [HIVE-4655] - Vectorization not working with negative constants, hive doesn't fold constants. * [HIVE-4665] - error at VectorExecMapper.close in group-by-agg query over ORC, vectorized * [HIVE-4666] - Count(*) over tpch lineitem ORC results in Error: Java heap space * [HIVE-4667] - tpch query 1 fails with java.lang.ClassCastException * [HIVE-4668] - wrong results for query with modulo (%) in WHERE clause filter * [HIVE-4673] - Use VectorExpessionWriter to write column vectors into Writables. * [HIVE-4678] - second clause of AND, OR filter not applied for vectorized execution * [HIVE-4680] - second clause of OR filter not applied in vectorized query execution * [HIVE-4681] - Fix ORC TimestampTreeReader.nextVector() to handle milli-nano math corectly * [HIVE-4684] - Query with filter constant on left of "=" and column expression on right does not vectorize * [HIVE-4685] - query using LIKE does not vectorize * [HIVE-4686] - Max on float returning wrong results * [HIVE-4688] - NPE in writing null values. * [HIVE-4694] - Fix ORC TestVectorizedORCReader testcase for Timestamps * [HIVE-4695] - Unit test failure in TestColumnColumnOperationVectorExpressionEvaluation * [HIVE-4699] - Integrate basic UDFs for Timesamp * [HIVE-4702] - Unit test failure TestVectorSelectOperator * [HIVE-4704] - TestCase FakeVectorRowBatchFromObjectIterables error * [HIVE-4706] - Query on Table with partition columns fail with AlreadyBeingCreatedException * [HIVE-4714] - Vectorized Sum of scalar subtract column returns negative result when positive exected * [HIVE-4716] - Classcast exception with two group by keys of types string and tinyint. * [HIVE-4718] - array out of bounds exception near VectorHashKeyWrapper.getBytes() with 2 column GROUP BY * [HIVE-4727] - Optimize ORC StringTreeReader::nextVector to not create dictionary of strings for each call to nextVector * [HIVE-4744] - Unary Minus Expression Throwing java.lang.NullPointerException * [HIVE-4745] - java.lang.RuntimeException: Hive Runtime Error while closing operators: java.lang.ClassCastException: org.apache.hadoop.io.NullWritable cannot be cast to org.apache.hadoop.hive.serde2.io.DoubleWritable * [HIVE-4754] - OrcInputFormat should be enhanced to provide vectorized input. * [HIVE-4758] - NULLs and record separators broken with vectorization branch intermediate outputs * [HIVE-4764] - Support Kerberos HTTP authentication for HiveServer2 running in http mode * [HIVE-4769] - Vectorized ORC reader does not handle absence of column present stream correctly. * [HIVE-4770] - Null Pointer Exception in Group By Operator * [HIVE-4774] - Hive Runtime Error while closing operators: java.lang.NullPointerException * [HIVE-4786] - Incorrect aggregate results * [HIVE-4787] - make vectorized LOWER(), UPPER(), LENGTH() work end-to-end; support expression input for vectorized LIKE * [HIVE-4794] - Unit e2e tests for vectorization * [HIVE-4821] - Implement vectorized type casting for all types * [HIVE-4822] - implement vectorized math functions * [HIVE-4823] - implement vectorized TRIM(), LTRIM(), RTRIM() * [HIVE-4828] - Make vectorization branch compile under JDK 7 * [HIVE-4846] - Implement Vectorized Limit Operator * [HIVE-4848] - std, stddev and stddev_pop aggregates on double/float fail to vectorize * [HIVE-4850] - Implement vectorized JOIN operators * [HIVE-4859] - String column comparison classes should be renamed. * [HIVE-4884] - ORC TimestampTreeReader.nextVector() off by a second when time in fractional * [HIVE-4887] - sql standard auth should disable commands that impose security risk * [HIVE-4898] - make vectorized math functions work end-to-end (update VectorizationContext.java) * [HIVE-4909] - Vectorized ORC reader does not set isRepeating flag correctly when 1’s are present is the input stream * [HIVE-4922] - create template for string scalar compared with string column * [HIVE-4925] - Modify Hive build to enable compiling and running Hive with JDK7 * [HIVE-4931] - MAX/MIN aggregates yield incorrect results * [HIVE-4945] - Make RLIKE/REGEXP run end-to-end by updating VectorizationContext * [HIVE-4959] - Vectorized plan generation should be added as an optimization transform. * [HIVE-4961] - Create bridge for custom UDFs to operate in vectorized mode * [HIVE-4971] - Unit test failure in TestVectorTimestampExpressions * [HIVE-4989] - Consolidate and simplify vectorization code and test generation * [HIVE-5126] - Make vector expressions serializable. * [HIVE-5146] - FilterExprOrExpr changes the order of the rows * [HIVE-5152] - Vector operators should inherit from non-vector operators for code re-use. * [HIVE-5173] - Wincompat : Add .cmd/text/crlf to .gitattributes * [HIVE-5174] - Wincompat : junit.file.schema configurability * [HIVE-5178] - Wincompat : QTestUtil changes * [HIVE-5187] - Enhance explain to indicate vectorized execution of operators. * [HIVE-5217] - Add long polling to asynchronous execution in HiveServer2 * [HIVE-5226] - orc_create.q and other orc tests fail on the branch. * [HIVE-5229] - Better thread management for HiveServer2 async threads * [HIVE-5230] - Better error reporting by async threads in HiveServer2 * [HIVE-5232] - Make JDBC use the new HiveServer2 async execution API by default * [HIVE-5252] - Add ql syntax for inline java code creation * [HIVE-5253] - Create component to compile and jar dynamic code * [HIVE-5282] - Some tests don't use ${system:test.dfs.mkdir} for mkdir * [HIVE-5308] - The code generation should be part of the build process. * [HIVE-5309] - Update hive-default.xml.template for vectorization flag; remove unused imports from MetaStoreUtils.java * [HIVE-5314] - Commit vectorization test data, comment/rename vectorization tests. * [HIVE-5332] - Milestone 1: Compile source code under maven * [HIVE-5333] - Milestone 2: Generate tests under maven * [HIVE-5334] - Milestone 3: Some tests pass under maven * [HIVE-5335] - Milestone 4: Most tests pass under maven * [HIVE-5365] - Boolean constants in the query are not handled correctly. * [HIVE-5371] - Root pom is malformed * [HIVE-5400] - Allow admins to disable compile and other commands * [HIVE-5430] - Refactor VectorizationContext and handle NOT expression with nulls. * [HIVE-5435] - Milestone 5: PTest2 maven support * [HIVE-5494] - Vectorization throws exception with nested UDF. * [HIVE-5503] - TopN optimization in VectorReduceSink * [HIVE-5517] - Implement end-to-end tests for vectorized string and math functions, and casts * [HIVE-5525] - Vectorized query failing for partitioned tables. * [HIVE-5556] - Pushdown join conditions * [HIVE-5557] - Push down qualifying Where clause predicates as join conditions * [HIVE-5558] - Support alternate join syntax * [HIVE-5566] - Milestone 6: All tests pass under hadoop 1 * [HIVE-5581] - Implement vectorized year/month/day... etc. for string arguments * [HIVE-5582] - Implement BETWEEN filter in vectorized mode * [HIVE-5583] - Implement support for IN (list-of-constants) filter in vectorized mode * [HIVE-5594] - Exception in vectorized map join. * [HIVE-5595] - Implement vectorized SMB JOIN * [HIVE-5597] - Temporary fix for HIVE-5172 (possible NPE in TUGIContainingTransport) * [HIVE-5600] - Fix PTest2 Maven support * [HIVE-5604] - Fix validation of nested expressions. * [HIVE-5610] - Merge maven branch into trunk * [HIVE-5611] - Add assembly (i.e.) tar creation to pom * [HIVE-5612] - Ability to compile odbc and re-generate generated code stored in source control * [HIVE-5616] - fix saveVersion.sh to work on mac * [HIVE-5624] - Create script for removing ant artifacts after merge * [HIVE-5627] - Document 'usehcatalog' parameter on WebHCat calls * [HIVE-5642] - Exception in UDFs with large number of arguments. * [HIVE-5653] - Vectorized Shuffle Join produces incorrect results * [HIVE-5673] - Create profile to generate protobuf * [HIVE-5674] - Merge latest trunk into branch and fix resulting tests * [HIVE-5675] - Ensure all artifacts are prefixed with hive- * [HIVE-5681] - Validation doesn't catch SMBMapJoin * [HIVE-5687] - Streaming support in Hive * [HIVE-5691] - Intermediate columns are incorrectly initialized for partitioned tables. * [HIVE-5697] - Correlation Optimizer may generate wrong plans for cases involving outer join * [HIVE-5699] - Add unit test for vectorized BETWEEN for timestamp inputs * [HIVE-5711] - Fix eclipse:eclipse maven goal * [HIVE-5713] - Verify versions of libraries post maven merge * [HIVE-5716] - Fix broken tests after maven merge (1) * [HIVE-5717] - Generate javadoc and source jars * [HIVE-5739] - Cleanup transitive dependencies * [HIVE-5740] - Tar files should extract to the directory of the same name minus tar.gz * [HIVE-5741] - Fix binary packaging build eg include hcatalog, resolve pom issues * [HIVE-5752] - log4j properties appear to have been lost in maven upgrade * [HIVE-5755] - Fix hadoop2 execution environment Milestone 1 * [HIVE-5756] - Implement vectorization support for IF conditional expression for long, double, timestamp, boolean and string inputs * [HIVE-5757] - Implement vectorized support for CASE * [HIVE-5758] - Implement vectorized support for NOT IN filter * [HIVE-5759] - Implement vectorized support for COALESCE conditional expression * [HIVE-5761] - Implement vectorized support for the DATE data type * [HIVE-5806] - Implement vectorization support for IF conditional expression for boolean and timestamp inputs * [HIVE-5807] - Implement vectorization support for IF conditional expression for string inputs * [HIVE-5826] - Add https support to HiveServer2 http mode * [HIVE-5842] - Fix issues with new paths to jar in hcatalog * [HIVE-5843] - Transaction manager for Hive * [HIVE-5849] - Improve the stats of operators based on heuristics in the absence of any column statistics * [HIVE-5877] - Implement vectorized support for IN as boolean-valued expression * [HIVE-5880] - Rename HCatalog HBase Storage Handler artifact id * [HIVE-5897] - Fix hadoop2 execution environment Milestone 2 * [HIVE-5898] - Make fetching of column statistics configurable * [HIVE-5914] - PTest2 should support build-only args * [HIVE-5915] - Shade Kryo dependency * [HIVE-5920] - Implement vectorized support for CASE WHEN a THEN b [WHEN c THEN d]* [ELSE e] END * [HIVE-5921] - Better heuristics for worst case statistics estimates for join, limit and filter operator * [HIVE-5923] - SQL std auth - parser changes * [HIVE-5928] - Add a hive authorization plugin api that does not assume privileges needed * [HIVE-5929] - SQL std auth - Access control statement updates * [HIVE-5930] - SQL std auth - implement set roles, show current roles * [HIVE-5931] - SQL std auth - add metastore get_principals_in_role api, support SHOW PRINCIPALS role_name * [HIVE-5941] - SQL std auth - support 'show roles' * [HIVE-5942] - SQL std auth - support DESCRIBE ROLE * [HIVE-5943] - SQL std auth - authorize DESCRIBE ROLE role * [HIVE-5944] - SQL std auth - authorize show all roles, create role, drop role * [HIVE-5952] - SQL std auth - authorize grant/revoke roles * [HIVE-5953] - SQL std auth - authorize grant/revoke on table * [HIVE-5954] - SQL std auth - get_privilege_set should check role hierarchy * [HIVE-5955] - SQL std auth - metastore api support for get_privilege_set api that checks specific role * [HIVE-5958] - SQL std auth - authorize statements that work with paths * [HIVE-5959] - SQL std auth - bootstrap SUPERUSER roles * [HIVE-5960] - SQL std auth - special handling of PUBLIC role * [HIVE-5966] - Fix eclipse:eclipse post shim aggregation changes * [HIVE-5978] - Rollups not supported in vector mode. * [HIVE-5979] - Failure in cast to timestamps. * [HIVE-6004] - Fix statistics annotation related test failures in hadoop2 * [HIVE-6017] - Contribute Decimal128 high-performance decimal(p, s) package from Microsoft to Hive * [HIVE-6051] - Create DecimalColumnVector and a representative VectorExpression for decimal * [HIVE-6060] - Define API for RecordUpdater and UpdateReader * [HIVE-6064] - Wincompat: windows path substitutions overridden by MiniMrShim.getConfiguration() on hadoop-2 * [HIVE-6066] - Wincompat: describe_comment_nonascii.q failing on windows * [HIVE-6067] - Implement vectorized decimal comparison filters * [HIVE-6112] - SQL std auth - support new privileges INSERT, DELETE * [HIVE-6124] - Support basic Decimal arithmetic in vector mode (+, -, *) * [HIVE-6139] - Implement vectorized decimal division and modulo * [HIVE-6156] - Implement vectorized reader for Date datatype for ORC format. * [HIVE-6161] - Fix TCLIService duplicate thrift definition for TColumn * [HIVE-6167] - Allow user-defined functions to be qualified with database name * [HIVE-6178] - Implement vectorized reader for DECIMAL datatype for ORC format. * [HIVE-6181] - support grant/revoke on views - parser changes * [HIVE-6183] - Implement vectorized type cast from/to decimal(p, s) * [HIVE-6186] - error in vectorized Column-Column comparison filter for repeating case * [HIVE-6199] - sql std auth - authorize 'show roles' * [HIVE-6207] - Integrate HCatalog with locking * [HIVE-6222] - Make Vector Group By operator abandon grouping if too many distinct keys * [HIVE-6232] - allow user to control out-of-range values in HCatStorer * [HIVE-6243] - error in high-precision division for Decimal128 * [HIVE-6250] - sql std auth - view authorization should not underlying table. More tests and fixes. * [HIVE-6254] - sql standard auth - use admin option specified in grant/revoke role statement * [HIVE-6257] - Add more unit tests for high-precision Decimal128 arithmetic * [HIVE-6258] - sql std auth - disallow cycles between roles * [HIVE-6300] - Add documentation for stats configs to hive-default.xml.template * [HIVE-6303] - VectorExpressionWriter for date and decimal datatypes. * [HIVE-6306] - HiveServer2 running in http mode should support for doAs functionality * [HIVE-6318] - Document SSL support added to HiveServer2 * [HIVE-6319] - Insert, update, delete functionality needs a compactor * [HIVE-6330] - Metastore support for permanent UDFs * [HIVE-6333] - Generate vectorized plan for decimal expressions. * [HIVE-6334] - sql std auth - pass username from sessionstate to v2 authorization interface * [HIVE-6344] - Add DECIMAL support to vectorized group by operator * [HIVE-6345] - Add DECIMAL support to vectorized JOIN operators * [HIVE-6349] - Column name map is broken * [HIVE-6350] - Support LDAP authentication for HiveServer2 in http mode * [HIVE-6351] - Support Pluggable Authentication Modules for HiveServer2 running in http mode * [HIVE-6380] - Specify jars/files when creating permanent UDFs * [HIVE-6386] - sql std auth - database should have an owner * [HIVE-6396] - Implement vectorized unary minus for decimal * [HIVE-6399] - bug in high-precision Decimal128 multiply * [HIVE-6406] - Introduce immutable-table table property and if set, disallow insert-into * [HIVE-6407] - Test authorization_revoke_table_priv.q is failing on trunk * [HIVE-6415] - Disallow transform clause in sql std authorization mode * [HIVE-6416] - Vectorized mathematical functions for decimal type. * [HIVE-6417] - sql std auth - new users in admin role config should get added * [HIVE-6422] - SQL std auth - revert change for view keyword in grant statement * [HIVE-6433] - SQL std auth - allow grant/revoke roles if user has ADMIN OPTION * [HIVE-6434] - Restrict function create/drop to admin roles * [HIVE-6440] - sql std auth - add command to change owner of database * [HIVE-6458] - Add schema upgrade scripts for metastore changes related to permanent functions * [HIVE-6460] - Need new "show" functionality for transactions * [HIVE-6474] - SQL std auth - only db owner should be allowed to create table within a db * [HIVE-6475] - Implement support for appending to mutable tables in HCatalog * [HIVE-6478] - SQL std auth - pass username from hiveserver2 to sessionstate * [HIVE-6496] - Queries fail to Vectorize. * [HIVE-6514] - TestExecDriver/HCat Pig tests fails with -Phadoop-2 * [HIVE-6541] - Need to write documentation for ACID work * [HIVE-6567] - "show grant ... on all" fails with NPE * [HIVE-6604] - Fix vectorized input to work with ACID * [HIVE-6625] - HiveServer2 running in http mode should support trusted proxy access * [HIVE-6673] - sql std auth - show grant statement for all principals throws NPE * [HIVE-6688] - Fix groupby_* qfile failures in hadoop-2 * [HIVE-6714] - Fix getMapSize() of LazyMap * [HIVE-6735] - Make scalable dynamic partitioning work in vectorized mode * [HIVE-6759] - Fix reading partial ORC files while they are being written * [HIVE-6760] - Scalable dynamic partitioning should bail out properly for list bucketing * [HIVE-6761] - Hashcode computation does not use maximum parallelism for scalable dynamic partitioning * [HIVE-6797] - Add protection against divide by zero in stats annotation * [HIVE-6798] - Update column stats based on filter expression in stats annotation * [HIVE-6815] - Version of the HIVE-6374 for Hive 0.13 ** Bug * [HIVE-1922] - semantic analysis error, when using group by and order by together * [HIVE-2055] - Hive should add HBase classpath dependencies when available * [HIVE-2137] - JDBC driver doesn't encode string properly. * [HIVE-2752] - Index names are case sensitive * [HIVE-2817] - Drop any table even without privilege * [HIVE-2818] - Create table should check privilege of target database, not default database * [HIVE-3272] - RetryingRawStore will perform partial transaction on retry * [HIVE-3455] - ANSI CORR(X,Y) is incorrect * [HIVE-3617] - Predicates pushed down to hbase is not handled properly when constant part is shown first * [HIVE-3815] - hive table rename fails if filesystem cache is disabled * [HIVE-3936] - Remote debug failed with hadoop 0.23X, hadoop 2.X * [HIVE-3938] - Hive MetaStore should send a single AddPartitionEvent for atomically added partition-set. * [HIVE-3949] - Some test failures in hadoop 23 * [HIVE-3969] - Session state for hive server should be cleaned-up * [HIVE-4003] - NullPointerException in exec.Utilities * [HIVE-4113] - Optimize select count(1) with RCFile and Orc * [HIVE-4116] - Can't use views using map datatype. * [HIVE-4144] - Add "select database()" command to show the current database * [HIVE-4200] - Consolidate submodule dependencies using ivy inheritance * [HIVE-4216] - TestHBaseMinimrCliDriver throws weird error with HBase 0.94.5 and Hadoop 23 and test is stuck infinitely * [HIVE-4247] - Filtering on a hbase row key duplicates results across multiple mappers * [HIVE-4256] - JDBC2 HiveConnection does not use the specified database * [HIVE-4257] - java.sql.SQLNonTransientConnectionException on JDBCStatsAggregator * [HIVE-4293] - Predicates following UDTF operator are removed by PPD * [HIVE-4388] - Upgrade HBase to 0.96 * [HIVE-4417] - Make unit tests pass on Windows which use Linux script in query files * [HIVE-4485] - beeline prints null as empty strings * [HIVE-4501] - HS2 memory leak - FileSystem objects in FileSystem.CACHE * [HIVE-4519] - JDBC getColumnTypeName() should respond with the Hive-specifc type name for ARRAY, STRUCT and MAP * [HIVE-4545] - HS2 should return describe table results without space padding * [HIVE-4619] - Hive 0.11.0 is not working with pre-cdh3u6 and hadoop-0.23 * [HIVE-4669] - Make username available to semantic analyzer hooks * [HIVE-4703] - Describe on a table returns "from deserializer" for column comments instead of values supplied in Create Table * [HIVE-4778] - hive.server2.authentication CUSTOM not working * [HIVE-4831] - QTestUtil based test exiting abnormally on windows fails startup of other QTestUtil tests * [HIVE-4837] - Union on void type fails with NPE * [HIVE-4872] - merge hive/build.properties and hive/hcatalog/build.properties * [HIVE-4910] - Hadoop 2 archives broken * [HIVE-4957] - Restrict number of bit vectors, to prevent out of Java heap memory * [HIVE-4974] - JDBC2 statements and result sets are not able to return their parents * [HIVE-4975] - Reading orc file throws exception after adding new column * [HIVE-4996] - unbalanced calls to openTransaction/commitTransaction * [HIVE-5002] - Loosen readRowIndex visibility in ORC's RecordReaderImpl to package private * [HIVE-5022] - Decimal Arithmetic generates NULL value * [HIVE-5032] - Enable hive creating external table at the root directory of DFS * [HIVE-5070] - Implement listLocatedStatus() in ProxyFileSystem for 0.23 shim * [HIVE-5071] - Address thread safety issues with HiveHistoryUtil * [HIVE-5082] - Beeline usage is printed twice when "beeline --help" is executed * [HIVE-5086] - Fix scriptfile1.q on Windows * [HIVE-5088] - Fix udf_translate.q on Windows * [HIVE-5090] - Remove unwanted file from the trunk. * [HIVE-5098] - Fix metastore for SQL Server * [HIVE-5099] - Some partition publish operation cause OOM in metastore backed by SQL Server * [HIVE-5102] - ORC getSplits should create splits based the stripes * [HIVE-5115] - Webhcat e2e tests TestMapReduce_1 and TestHeartbeat_2 require changes for Hadoop 2 * [HIVE-5132] - Can't access to hwi due to "No Java compiler available" * [HIVE-5133] - webhcat jobs that need to access metastore fails in secure mode * [HIVE-5150] - UnsatisfiedLinkError when running hive unit tests on Windows * [HIVE-5166] - TestWebHCatE2e is failing intermittently on trunk * [HIVE-5181] - RetryingRawStore should not retry on logical failures (e.g. from commit) * [HIVE-5202] - Support for SettableUnionObjectInspector and implement isSettable/hasAllFieldsSettable APIs for all data types. * [HIVE-5216] - Need to annotate public API in HCatalog * [HIVE-5218] - datanucleus does not work with MS SQLServer in Hive metastore * [HIVE-5224] - When creating table with AVRO serde, the "avro.schema.url" should be about to load serde schema from file system beside HDFS * [HIVE-5225] - There is no database named nonexistentdatabaseusedforhealthcheck * [HIVE-5231] - Remove TestSerDe.jar from data/files * [HIVE-5240] - Column statistics on a partitioned column should fail early with proper error message * [HIVE-5242] - Trunk fails to compile * [HIVE-5248] - WebHCat Proxy User Support (HIVE-4601) requires documentation * [HIVE-5249] - Missing test file for HIVE-5199 * [HIVE-5256] - A map join operator may have in-consistent output row schema with the common join operator which it will replace * [HIVE-5257] - Add kryo into eclipse template * [HIVE-5264] - SQL generated by MetaStoreDirectSql.java not compliant with Postgres. * [HIVE-5266] - HCatalog checkstyle failure due to HIVE-5225 * [HIVE-5268] - HiveServer2 accumulates orphaned OperationHandle objects when a client fails while executing query * [HIVE-5270] - Enable hash joins using tez * [HIVE-5272] - Column statistics on a invalid column name results in IndexOutOfBoundsException * [HIVE-5279] - Kryo cannot instantiate GenericUDAFEvaluator in GroupByDesc * [HIVE-5283] - Merge vectorization branch to trunk * [HIVE-5286] - Negative test date_literal1.q fails on java7 because the syntax is valid * [HIVE-5295] - HiveConnection#configureConnection tries to execute statement even after it is closed * [HIVE-5311] - TestHCatPartitionPublish can fail randomly * [HIVE-5315] - Cannot attach debugger to Hiveserver2 * [HIVE-5318] - Import Throws Error when Importing from a table export Hive 0.9 to Hive 0.10 * [HIVE-5327] - Potential leak and cleanup in utilities.java * [HIVE-5345] - Operator::close() leaks Operator::out, holding reference to buffers * [HIVE-5349] - QTestutil does not properly set UTF-8 * [HIVE-5350] - Cleanup exception handling around parallel orderby * [HIVE-5353] - job submission that requires access to metastore should not require additional jars to be shipped to target node * [HIVE-5360] - fix hcatalog checkstyle issue introduced in HIVE-5223 * [HIVE-5368] - Changes to work creation for tez * [HIVE-5377] - the error handling in serialize/deserializeExpression is insufficient, some test may pass in error * [HIVE-5379] - NoClassDefFoundError is thrown when using lead/lag with kryo serialization * [HIVE-5380] - Non-default OI constructors should be supported for backwards compatibility * [HIVE-5384] - hcatloadstore.pig should be upgraded to org.apache.hive.hcatalog.pig.HCatLoader * [HIVE-5385] - StringUtils is not in commons codec 1.3 * [HIVE-5403] - Move loading of filesystem, ugi, metastore client to hive session * [HIVE-5404] - Remove changes from HIVE-5184 * [HIVE-5407] - show create table creating unusable DDL when some reserved keywords exist * [HIVE-5411] - Migrate expression serialization to Kryo * [HIVE-5412] - HivePreparedStatement.setDate not implemented * [HIVE-5414] - The result of show grant is not visible via JDBC * [HIVE-5415] - Remove "System.err.println" from vectorization optimization * [HIVE-5418] - Integer overflow bug in ConditionalResolverCommonJoin.AliasFileSizePair * [HIVE-5420] - HiveServer2 async thread timeout should be cast to long * [HIVE-5422] - Upgrade Kyro to 2.22 now that it is released * [HIVE-5425] - Provide a configuration option to control the default stripe size for ORC * [HIVE-5428] - Direct SQL check fails during tests * [HIVE-5440] - HiveServer2 doesn't apply SQL operation's config property * [HIVE-5441] - Async query execution doesn't return resultset status * [HIVE-5446] - Hive can CREATE an external table but not SELECT from it when file path have spaces * [HIVE-5448] - webhcat duplicate test TestMapReduce_2 should be removed * [HIVE-5449] - Hive schematool info option incorrectly reports error for Postgres metastore * [HIVE-5452] - HCatalog e2e test Pig_HBase_1 and Pig_HBase_2 are failing with ClassCastException * [HIVE-5453] - jobsubmission2.conf should use 'timeout' property * [HIVE-5454] - HCatalog runs a partition listing with an empty filter * [HIVE-5455] - Add build/ql/gen/vector to source folder in eclipse template * [HIVE-5459] - Add --version option to hive script * [HIVE-5460] - invalid offsets in lag lead should return an exception (per ISO-SQL) * [HIVE-5474] - drop table hangs when concurrency=true * [HIVE-5478] - WebHCat e2e testsuite for hcat authorization tests needs some fixes * [HIVE-5479] - SBAP restricts hcat -e 'show databases' * [HIVE-5480] - WebHCat e2e tests for doAs feature are failing * [HIVE-5481] - WebHCat e2e test: TestStreaming -ve tests should also check for job completion success * [HIVE-5482] - JDBC should depend on httpclient.version and httpcore.version 4.1.3 to be consistent with other modules * [HIVE-5484] - TestSchemaTool failures when Hive version has more than 3 revision numbers * [HIVE-5485] - SBAP errors on null partition being passed into partition level authorization * [HIVE-5486] - HiveServer2 should create base scratch directories at startup * [HIVE-5490] - SUBSTR(col, 1, 0) returns wrong result in vectorized mode * [HIVE-5492] - Explain query fails with NPE if a client doesn't call getResultSetSchema() * [HIVE-5496] - hcat -e "drop database if exists" fails on authorizing non-existent null db * [HIVE-5504] - OrcOutputFormat honors compression properties only from within hive * [HIVE-5506] - Hive SPLIT function does not return array correctly * [HIVE-5507] - [WebHCat] test.other.user.name parameter is missing from build.xml in e2e harness * [HIVE-5508] - [WebHCat] ignore log collector e2e tests for Hadoop 2 * [HIVE-5509] - [WebHCat] TestDriverCurl to use string comparison for jobid * [HIVE-5510] - [WebHCat] GET job/queue return wrong job information * [HIVE-5511] - percentComplete returned by job status from WebHCat is null * [HIVE-5513] - Set the short version directly via build script * [HIVE-5515] - Writing to an HBase table throws IllegalArgumentException, failing job submission * [HIVE-5519] - Use paging mechanism for templeton get requests. * [HIVE-5521] - Remove CommonRCFileInputFormat * [HIVE-5524] - Unwanted delay in getting Hive metastore connection with METASTORE_CLIENT_CONNECT_RETRY_DELAY/ * [HIVE-5526] - NPE in ConstantVectorExpression.evaluate(vrg) * [HIVE-5528] - hive log file name in local is ".log" * [HIVE-5531] - Hiverserver2 doesn't honor command line argument when initializing log4j * [HIVE-5535] - [WebHCat] Webhcat e2e test JOBS_2 fail due to permission when hdfs umask setting is 022 * [HIVE-5539] - remove hcatalog wiki tab from hive site * [HIVE-5540] - webhcat e2e test failures: "Expect 1 jobs in logs, but get 1" * [HIVE-5541] - [WebHCat] Log collector does not work since we don't close the hdfs status file * [HIVE-5542] - Webhcat is failing to run ddl command on a secure cluster * [HIVE-5543] - Running the mini tez cluster for tez unit tests * [HIVE-5546] - A change in ORCInputFormat made by HIVE-4113 was reverted by HIVE-5391 * [HIVE-5547] - webhcat pig job submission should ship hive tar if -usehcatalog is specified * [HIVE-5548] - Tests under common directory don't run as part of 'ant test' * [HIVE-5550] - Import fails for tables created with default text, sequence and orc file formats using HCatalog API * [HIVE-5552] - Merging of QBJoinTrees doesn't handle filter pushdowns correctly * [HIVE-5559] - Stats publisher fails for list bucketing when IDs are too long * [HIVE-5560] - Hive produces incorrect results on multi-distinct query * [HIVE-5563] - Skip reading columns in ORC for count(*) * [HIVE-5567] - Add better protection code for SARGs * [HIVE-5572] - Fails of non-sql command are not propagated to jdbc2 client * [HIVE-5574] - Unnecessary newline at the end of message of ParserException * [HIVE-5575] - ZooKeeper connection closed when unlock with retry * [HIVE-5576] - Blank lines missing from .q.out files created on Windows for testcase=TestCliDriver * [HIVE-5577] - Remove TestNegativeCliDriver script_broken_pipe1 * [HIVE-5578] - hcat script doesn't include jars from HIVE_AUX_JARS_PATH * [HIVE-5580] - push down predicates with an and-operator between non-SARGable predicates will get NPE * [HIVE-5596] - hive-default.xml.template is invalid * [HIVE-5598] - Remove dummy new line at the end of non-sql commands * [HIVE-5599] - Change default logging level to INFO * [HIVE-5601] - NPE in ORC's PPD when using select * from table with where predicate * [HIVE-5618] - Hive local task fails to run when run from oozie in a secure cluster * [HIVE-5620] - Tez job progress printing stops after a specific amount of time * [HIVE-5625] - Fix issue with metastore version restriction test. * [HIVE-5628] - ListBucketingPrunnerTest and DynamicMultiDimeCollectionTest should start with Test not end with it * [HIVE-5629] - Fix two javadoc failures in HCatalog * [HIVE-5631] - Index creation on a skew table fails * [HIVE-5633] - Perflogger broken due to HIVE-5403 * [HIVE-5635] - WebHCatJTShim23 ignores security/user context * [HIVE-5636] - Introduce getPartitionColumns() functionality from HCatInputFormat * [HIVE-5641] - BeeLineOpts ignores Throwable * [HIVE-5643] - ZooKeeperHiveLockManager.getQuorumServers incorrectly appends the custom zk port to quorum hosts * [HIVE-5646] - Cannot compile tests on tez branch * [HIVE-5648] - error when casting partition column to varchar in where clause * [HIVE-5649] - error when casting partition column to varchar in where clause * [HIVE-5656] - Hive produces unclear, confusing SemanticException when dealing with mod or pmod by zero * [HIVE-5657] - TopN produces incorrect results with count(distinct) * [HIVE-5664] - Drop cascade database fails when the db has any tables with indexes * [HIVE-5665] - Update PMC status for navis * [HIVE-5667] - ThriftCLIService log messages jumbled up * [HIVE-5668] - path normalization in MapOperator is expensive * [HIVE-5676] - Cleanup test cases as done during mavenization * [HIVE-5680] - Hive writes to HBase table throws NullPointerException * [HIVE-5682] - can not display the table's comment in chinese * [HIVE-5683] - JDBC support for char * [HIVE-5684] - Serde support for char * [HIVE-5685] - partition column type validation doesn't work in some cases * [HIVE-5686] - partition column type validation doesn't quite work for dates * [HIVE-5692] - Make VectorGroupByOperator parameters configurable * [HIVE-5694] - on hbase table, function's result like 'count' or 'sum' is multiplied by the number of map tasks. * [HIVE-5695] - PTest2 fix shutdown, duplicate runs, and add client retry * [HIVE-5696] - WebHCat e2e tests/jobsubmission.conf file is malformed and loosing tests * [HIVE-5698] - A few test files missing apache license header * [HIVE-5702] - add logging on WebHCat startup to print all env var and Configuration * [HIVE-5703] - While using tez, Qtest needs to close session before creating a new one * [HIVE-5715] - HS2 should not "start" a session for every command * [HIVE-5729] - Beeline displays version as ???? after mavenization * [HIVE-5730] - Beeline throws non-terminal NPE upon starting, after mavenization * [HIVE-5732] - HiveServer2: Duplicated new OperationManager in SessionManager * [HIVE-5743] - Maven pom reverts hadoop 23 version to 2.0.5-alpha * [HIVE-5745] - TestHiveLogging is failing (at least on mac) * [HIVE-5749] - Classpath issue with maven profile hadoop-2 * [HIVE-5763] - ExprNodeGenericFuncDesc.toString() generating unbalanced parenthesises * [HIVE-5765] - Beeline throws NPE when -e option is used * [HIVE-5767] - in SemanticAnalyzer#doPhase1, handling for TOK_UNION falls thru into TOK_INSERT * [HIVE-5768] - Beeline connection cannot be closed with !close command * [HIVE-5773] - Fix build due to conflict between HIVE-5711 and HIVE-5713 * [HIVE-5777] - PTest2 IllegalArgumentException when creating hosts * [HIVE-5779] - Subquery in where clause with distinct fails with mapjoin turned on with serialization error. * [HIVE-5780] - Add the missing declaration of HIVE_CLI_SERVICE_PROTOCOL_V4 in TCLIService.thrift * [HIVE-5782] - PTest2 should be able to ride out price spikes * [HIVE-5784] - Group By Operator doesn't carry forward table aliases in its RowResolver * [HIVE-5786] - Remove HadoopShims methods that were needed for pre-Hadoop 0.20 * [HIVE-5788] - select * fails for table after adding new columns using rcfile storage format * [HIVE-5790] - maven test build failure shows wrong error message * [HIVE-5791] - TestUseDatabase in hcategory failed to pass when illegal filename in /tmp * [HIVE-5800] - Hive is inconsistent when handling strings in arithmetic operations * [HIVE-5809] - incorrect stats in some cases with hive.stats.autogather=true * [HIVE-5812] - HiveServer2 SSL connection transport binds to loopback address by default * [HIVE-5813] - Multi-way Left outer join fails in vectorized mode * [HIVE-5816] - maven test failure shows ant message when running single qfile test * [HIVE-5817] - column name to index mapping in VectorizationContext is broken * [HIVE-5825] - Case statement type checking too restrictive for parameterized types * [HIVE-5827] - Incorrect location of logs for failed tests. * [HIVE-5828] - Make shims dependecny on specific hadoop hive shims optional * [HIVE-5829] - Rewrite Trim and Pad UDFs based on GenericUDF * [HIVE-5830] - SubQuery: Not In subqueries should check if subquery contains nulls in matching column * [HIVE-5835] - Null pointer exception in DeleteDelegator in templeton code * [HIVE-5839] - BytesRefArrayWritable compareTo violates contract * [HIVE-5844] - dynamic_partition_skip_default.q test fails on trunk * [HIVE-5845] - CTAS failed on vectorized code path * [HIVE-5846] - Analyze command fails with vectorization on * [HIVE-5858] - Alter table fails with exception when fs.hdfs.impl.disable.cache is set false * [HIVE-5859] - Create view does not captures inputs * [HIVE-5862] - While running some queries on large data using tez, we OOM. * [HIVE-5863] - INSERT OVERWRITE TABLE fails in vectorized mode for ORC format target table * [HIVE-5866] - Hive divide operator generates wrong results in certain cases * [HIVE-5876] - Split elimination in ORC breaks for partitioned tables * [HIVE-5887] - metastore direct sql doesn't work with oracle * [HIVE-5891] - Alias conflict when merging multiple mapjoin tasks into their common child mapred task * [HIVE-5893] - hive-schema-0.13.0.mysql.sql contains reference to nonexistent column * [HIVE-5894] - Fix minor PTest2 issues * [HIVE-5895] - vectorization handles division by zero differently from normal execution * [HIVE-5896] - Insert queries in ORC fails with vectorization on * [HIVE-5899] - NPE during explain extended with char/varchar columns * [HIVE-5904] - HiveServer2 JDBC connect to non-default database * [HIVE-5906] - TestGenericUDFPower should use delta to compare doubles * [HIVE-5909] - locate and instr throw java.nio.BufferUnderflowException when empty string as substring * [HIVE-5910] - In HiveConf, the name of mapred.min.split.size.per.rack is MAPREDMINSPLITSIZEPERNODE and the name of mapred.min.split.size.per.node is MAPREDMINSPLITSIZEPERRACK * [HIVE-5911] - Recent change to schema upgrade scripts breaks file naming conventions * [HIVE-5912] - Show partition command doesn't support db.table * [HIVE-5913] - HIVE-5880 - (Rename HCatalog HBase Storage Handler artifact id) breaks packaging * [HIVE-5916] - No need to aggregate statistics collected via counter mechanism * [HIVE-5917] - Hive packaging build is broken * [HIVE-5926] - Load Data OverWrite Into Table Throw org.apache.hadoop.hive.ql.metadata.HiveException * [HIVE-5935] - hive.query.string is not provided to FetchTask * [HIVE-5936] - analyze command failing to collect stats with counter mechanism * [HIVE-5939] - HCatalog hadoop-2 execution environment needs to be addressed. * [HIVE-5945] - ql.plan.ConditionalResolverCommonJoin.resolveMapJoinTask also sums those tables which are not used in the child of this conditional task. * [HIVE-5947] - Fix test failure in decimal_udf.q * [HIVE-5950] - ORC SARG creation fails with NPE for predicate conditions with decimal/date/char/varchar datatypes * [HIVE-5957] - Fix HCatalog Unit tests on Windows * [HIVE-5964] - Hive missing a filter predicate causing wrong results joining tables after sort by * [HIVE-5973] - SMB joins produce incorrect results with multiple partitions and buckets * [HIVE-5975] - [WebHCat] templeton mapreduce job failed if provide "define" parameters * [HIVE-5981] - Add hive-unit back to itests pom * [HIVE-5983] - Fix name of ColumnProjectionUtils.appendReadColumnIDs * [HIVE-5986] - ORC SARG evaluation fails with NPE for UDFs or expressions in predicate condition * [HIVE-5989] - Hive metastore authorization check is not threadsafe * [HIVE-5991] - ORC RLEv2 fails with ArrayIndexOutOfBounds exception for PATCHED_BLOB encoding * [HIVE-5992] - Hive inconsistently converts timestamp in AVG and SUM UDAF's * [HIVE-5994] - ORC RLEv2 encodes wrongly for large negative BIGINTs (64 bits ) * [HIVE-5995] - decimal_precision.q test case fails in trunk * [HIVE-6000] - Hive build broken on hadoop2 * [HIVE-6003] - bin/hive --debug should not append HIVE_CLIENT_OPTS to HADOOP_OPTS * [HIVE-6005] - BETWEEN is broken after using KRYO * [HIVE-6012] - restore backward compatibility of arithmetic operations * [HIVE-6015] - vectorized logarithm produces results for 0 that are different from a non-vectorized one * [HIVE-6016] - Hadoop23Shims has a bug in listLocatedStatus impl. * [HIVE-6018] - FetchTask should not reference metastore classes * [HIVE-6021] - Problem in GroupByOperator for handling distinct aggrgations * [HIVE-6022] - Load statements with incorrect order of partitions put input files to unreadable places * [HIVE-6027] - non-vectorized log10 has rounding issue * [HIVE-6028] - Partition predicate literals are not interpreted correctly. * [HIVE-6034] - vectorized % doesn't handle zeroes the same way as non-vectorized * [HIVE-6036] - A test case for embedded beeline - with URL jdbc:hive2:///default * [HIVE-6039] - Round, AVG and SUM functions reject char/varch input while accepting string input * [HIVE-6041] - Incorrect task dependency graph for skewed join optimization * [HIVE-6044] - webhcat should be able to return detailed serde information when show table using "fromat=extended" * [HIVE-6045] - Beeline hivevars is broken for more than one hivevar * [HIVE-6047] - Permanent UDFs in Hive * [HIVE-6048] - Hive load data command rejects file with '+' in the name * [HIVE-6052] - metastore JDO filter pushdown for integers may produce unexpected results with non-normalized integer columns * [HIVE-6054] - HiveServer2 does not log the output of LogUtils.initHiveLog4j(); * [HIVE-6065] - orc/FileDump and testRCFile should close on appropriate resources * [HIVE-6068] - HiveServer2 client on windows does not handle the non-ascii characters properly * [HIVE-6072] - With HCatalog refactoring, Hadoop_HBase e2e will fail * [HIVE-6082] - Certain KeeperException should be ignored in ZooKeeperHiveLockManage.unlockPrimitive * [HIVE-6083] - User provided table properties are not assigned to the TableDesc of the FileSinkDesc in a CTAS query * [HIVE-6085] - Tez changed test parse tests output * [HIVE-6104] - Join-key logging in join operator * [HIVE-6105] - LongWritable.compareTo needs shimming * [HIVE-6110] - schematool errors out when HIVE_OPTS is set * [HIVE-6117] - mapreduce.RecordReader instance needs to be initialized * [HIVE-6125] - Tez: Refactoring changes * [HIVE-6129] - alter exchange is implemented in inverted manner * [HIVE-6142] - Beeline ignores HIVE_OPTS * [HIVE-6151] - CLIService should use debug log level instead of info * [HIVE-6152] - insert query fails on hdfs federation + viewfs * [HIVE-6154] - HiveServer2 returns a detailed error message to the client only when the underlying exception is a HiveSQLException * [HIVE-6157] - Fetching column stats slower than the 101 during rush hour * [HIVE-6159] - Hive uses deprecated hadoop configuration in Hadoop 2.0 * [HIVE-6162] - "multiple SLF4J bindings" warning messages when running hive CLI on Hadoop 2.0 * [HIVE-6164] - Hive build on Windows failed with datanucleus enhancer error "command line is too long" * [HIVE-6166] - JsonSerDe is too strict about table schema * [HIVE-6170] - Upgrade to the latest version of bonecp * [HIVE-6174] - Beeline "set varible" doesn't show the value of the variable as Hive CLI * [HIVE-6177] - Fix keyword KW_REANME which was intended to be KW_RENAME * [HIVE-6180] - Hive JDBC Driver connection fail when no default database passed in the connection string * [HIVE-6184] - Bug in SessionManager.stop() in HiveServer2 * [HIVE-6185] - DDLTask is inconsistent in creating a table and adding a partition when dealing with location * [HIVE-6189] - Support top level union all statements * [HIVE-6195] - Create unit tests to exercise behaviour when creating a HBase Table in Hive * [HIVE-6203] - Privileges of role granted indrectily to user is not applied * [HIVE-6205] - alter partition column throws NPE in authorization * [HIVE-6208] - user-defined aggregate functions cannot be used as windowing function * [HIVE-6209] - 'LOAD DATA INPATH ... OVERWRITE ..' doesn't overwrite current data * [HIVE-6211] - WebHCat job status E2E tests fail in presence of other jobs * [HIVE-6216] - Update datanucleus.connectionPoolingType to BoneCP in hive-default.xml.template * [HIVE-6218] - Stats for row-count not getting updated with Tez insert + dbclass=counter * [HIVE-6224] - Remove unneeded tez dependencies from hive * [HIVE-6227] - WebHCat E2E test JOBS_7 fails * [HIVE-6229] - Stats are missing sometimes (regression from HIVE-5936) * [HIVE-6231] - NPE when switching to Tez execution mode after session has been initialized * [HIVE-6233] - JOBS testsuite in WebHCat E2E tests does not work correctly in secure mode * [HIVE-6237] - Webhcat e2e test JOBS_2 fail due to permission when hdfs umask setting is 022 * [HIVE-6240] - Update jetty to the latest stable (9.x) in the service module * [HIVE-6246] - Sign(a) UDF is not supported for decimal type * [HIVE-6249] - HiveServer2 JDBC SSL binary client transport should not call a TTransport#open * [HIVE-6251] - Add ability to specify delimiter in HCatalog Java API to create tables - HCatCreateTableDesc * [HIVE-6260] - Compress plan when sending via RPC (Tez) * [HIVE-6261] - Update metadata.q.out file for tez (after change to .q file) * [HIVE-6262] - Remove unnecessary copies of schema + table desc from serialized plan * [HIVE-6263] - Avoid sending input files multiple times on Tez * [HIVE-6264] - Unbalanced number of HiveParser msgs.push/msgs.pop calls when doing lookahead * [HIVE-6267] - Explain explain * [HIVE-6268] - Network resource leak with HiveClientCache when using HCatInputFormat * [HIVE-6287] - batchSize computation in Vectorized ORC reader can cause BufferUnderFlowException when PPD is enabled * [HIVE-6298] - Add config flag to turn off fetching partition stats * [HIVE-6301] - get_json_object throw java.lang.IllegalStateException: No match found exception. * [HIVE-6309] - Hive incorrectly removes TaskAttempt output files if MRAppMaster fails once * [HIVE-6310] - Fix a few minimr test failures * [HIVE-6312] - doAs with plain sasl auth should be session aware * [HIVE-6315] - MetaStoreDirectSql ctor should not throw * [HIVE-6320] - Row-based ORC reader with PPD turned on dies on BufferUnderFlowException * [HIVE-6323] - Fix unit test file_with_header_footer_negative.q in TestNegativeMinimrCliDriver * [HIVE-6326] - Split generation in ORC may generate wrong split boundaries because of unaccounted padded bytes * [HIVE-6328] - Hive script should not overwrite AUX_CLASSPATH with HIVE_AUX_JARS_PATH if the latter is set * [HIVE-6340] - Beeline outputs error message when HIVE_AUX_JARS_PATH is set * [HIVE-6353] - Update hadoop-2 golden files after HIVE-6267 * [HIVE-6354] - Some index test golden files produce non-deterministic stats in explain * [HIVE-6356] - Dependency injection in hbase storage handler is broken * [HIVE-6358] - filterExpr not printed in explain for tablescan operators (ppd) * [HIVE-6359] - beeline -f fails on scripts with tabs in them. * [HIVE-6360] - Hadoop 2.3 + Tez 0.3 * [HIVE-6364] - HiveServer2 - Request serving thread should get class loader from existing SessionState * [HIVE-6369] - ORC Writer (int RLE v2) fails with ArrayIndexOutOfBounds * [HIVE-6372] - getDatabaseMajor/Minor version returns wrong values * [HIVE-6373] - TestCliDriverMethods test can cause entire build to fail * [HIVE-6375] - Fix CTAS for parquet * [HIVE-6377] - make HADOOP_HOME setting consistent between hive and webhcat_config.sh * [HIVE-6381] - bin/hcat script won't launch - uses invalid $HIVE_HOME * [HIVE-6382] - PATCHED_BLOB encoding in ORC will corrupt data in some cases * [HIVE-6383] - Newly added tests in TestJdbcDriver2 from HIVE-4395 is not running * [HIVE-6389] - LazyBinaryColumnarSerDe-based RCFile tables break when looking up elements in null-maps. * [HIVE-6392] - Hive (and HCatalog) don't allow super-users to add partitions to tables. * [HIVE-6395] - multi-table insert from select transform fails if optimize.ppd enabled * [HIVE-6398] - MapRedTask.configureDebugVariablesForChildJVM mixes HIVE_CHILD_CLIENT_DEBUG_OPTS and HIVE_MAIN_CLIENT_DEBUG_OPTS in env check * [HIVE-6400] - Add Apache Rat plugin to pom.xml * [HIVE-6403] - uncorrelated subquery is failing with auto.convert.join=true * [HIVE-6409] - FileOutputCommitterContainer::commitJob() cancels delegation tokens too early. * [HIVE-6414] - ParquetInputFormat provides data values that do not match the object inspectors * [HIVE-6420] - upgrade script for Hive 13 is missing for Derby * [HIVE-6447] - Bucket map joins in hive-tez * [HIVE-6461] - Run Release Audit tool, fix missing license issues * [HIVE-6464] - Test configuration: reduce the duration for which lock attempts are retried * [HIVE-6467] - metastore upgrade script 016-HIVE-6386.derby.sql uses char rather than varchar * [HIVE-6472] - JDBC cancel will not work with current HiveServer2 * [HIVE-6477] - Aggregation functions for tiny/smallint broken with parquet * [HIVE-6479] - Few .q.out files need to be updated post HIVE-5958 * [HIVE-6482] - Fix NOTICE file: pre release task * [HIVE-6485] - Downgrade to httpclient-4.2.5 in JDBC from httpclient-4.3.2 * [HIVE-6488] - Investigate TestBeeLineWithArgs * [HIVE-6491] - ClassCastException in AbstractParquetMapInspector * [HIVE-6495] - TableDesc.getDeserializer() should use correct classloader when calling Class.forName() * [HIVE-6499] - Using Metastore-side Auth errors on non-resolvable IF/OF/SerDe * [HIVE-6503] - document pluggable authentication modules (PAM) in template config, wiki * [HIVE-6505] - Make stats optimizer more robust in presence of distinct clause * [HIVE-6506] - hcatalog should automatically work with new tableproperties in ORC * [HIVE-6507] - OrcFile table property names are specified as strings * [HIVE-6508] - Mismatched results between vector and non-vector mode with decimal field * [HIVE-6511] - casting from decimal to tinyint,smallint, int and bigint generates different result when vectorization is on * [HIVE-6512] - HiveServer2 ThriftCLIServiceTest#testDoAs is an invalid test * [HIVE-6513] - Most hcatalog pig tests fail when building for hadoop-2 * [HIVE-6518] - Add a GC canary to the VectorGroupByOperator to flush whenever a GC is triggered * [HIVE-6519] - Allow optional "as" in subquery definition * [HIVE-6522] - AVG() failure with decimal type * [HIVE-6528] - Add maven compiler plugin to ptest2 pom * [HIVE-6529] - Tez output files are out of date * [HIVE-6530] - JDK 7 trunk build fails after HIVE-6418 patch * [HIVE-6531] - Runtime errors in vectorized execution. * [HIVE-6537] - NullPointerException when loading hashtable for MapJoin directly * [HIVE-6539] - Couple of issues in fs based stats collection * [HIVE-6542] - build error with jdk 7 * [HIVE-6545] - analyze table throws NPE for non-existent tables. * [HIVE-6546] - WebHCat job submission for pig with -useHCatalog argument fails on Windows * [HIVE-6547] - normalize struct Role in metastore thrift interface * [HIVE-6548] - Missing owner name and type fields in schema script for DBS table * [HIVE-6551] - group by after join with skew join optimization references invalid task sometimes * [HIVE-6552] - Failed to generate new mapJoin operator by exception : Big Table Alias is null * [HIVE-6555] - TestSchemaTool is failing on trunk after branching * [HIVE-6557] - TestSchemaTool tests are failing * [HIVE-6558] - HiveServer2 Plain SASL authentication broken after hadoop 2.3 upgrade * [HIVE-6559] - sourcing txn-script from schema script results in failure for mysql & oracle * [HIVE-6562] - Protection from exceptions in ORC predicate evaluation * [HIVE-6563] - hdfs jar being pulled in when creating a hadoop-2 based hive tar ball * [HIVE-6566] - Incorrect union-all plan with map-joins on Tez * [HIVE-6568] - Vectorized cast of decimal to string and timestamp produces incorrect result. * [HIVE-6571] - query id should be available for logging during query compilation * [HIVE-6572] - Use shimmed version of hadoop conf names for mapred.{min,max}.split.size{.*} * [HIVE-6573] - Oracle metastore doesnt come up when hive.cluster.delegation.token.store.class is set to DBTokenStore * [HIVE-6574] - Type in ql/pom.xml prevents jobs from parquet queries from running on a cluster * [HIVE-6575] - select * fails on parquet table with map datatype * [HIVE-6576] - sending user.name as a form parameter in POST doesn't work post HADOOP-10193 * [HIVE-6580] - Refactor ThriftBinaryCLIService and ThriftHttpCLIService tests. * [HIVE-6585] - bucket map join fails in presence of _SUCCESS file * [HIVE-6591] - Importing a table containing hidden dirs fails * [HIVE-6592] - WebHCat E2E test abort when pointing to https url of webhdfs * [HIVE-6594] - UnsignedInt128 addition does not increase internal int array count resulting in corrupted values during serialization * [HIVE-6597] - WebHCat E2E tests doAsTests_6 and doAsTests_7 need to be updated * [HIVE-6598] - Importing the project into eclipse as maven project have some issues * [HIVE-6599] - Document new jdbc url connection parameters for HiveServer2 * [HIVE-6605] - Hive does not set the environment correctly when running in Tez mode * [HIVE-6606] - Stand alone metastore fails to start if new transaction values not defined in config * [HIVE-6607] - describe extended on a view fails with NPE * [HIVE-6608] - Add apache pom as parent pom * [HIVE-6609] - Doing Ctrl-C on hive cli doesn't kill running MR jobs on hadoop-2 * [HIVE-6610] - Unit test log needs to reflect DB Name * [HIVE-6611] - Joining multiple union all outputs fails on Tez * [HIVE-6618] - assertion when getting reference key from loader with byte-array mapjoin key * [HIVE-6621] - Beeline backward incompatible change * [HIVE-6630] - FS based stats collection have issues for list bucketing case * [HIVE-6633] - pig -useHCatalog with embedded metastore fails to pass command line args to metastore * [HIVE-6635] - Heartbeats are not being sent when DbLockMgr is used and an operation holds locks * [HIVE-6636] - /user/hive is a bad default for HDFS jars path for Tez * [HIVE-6639] - Vectorization: Partition column names are not picked up. * [HIVE-6640] - Change hive.version.shortname in hive 0.13 branch to '0.13.0' * [HIVE-6641] - optimized HashMap keys won't work correctly with decimals * [HIVE-6642] - Query fails to vectorize when a non string partition column is part of the query expression * [HIVE-6643] - Add a check for cross products in plans and output a warning * [HIVE-6644] - document TestStreaming_2 e2e test case for webhcat * [HIVE-6645] - to_date()/to_unix_timestamp() fail with NPE if input is null * [HIVE-6646] - Error in txn handler SQL * [HIVE-6647] - Bump the thrift api version to V7 for HiveServer2 * [HIVE-6649] - Vectorization: some date expressions throw exception. * [HIVE-6650] - hive.optimize.index.filter breaks non-index where with HBaseStorageHandler * [HIVE-6653] - WebHCat E2E test JOBS_7 and JOBS_9 fail as profile.url in job details is being returned as null * [HIVE-6656] - Bug in ORC Timestamp reader returns wrong nanoseconds * [HIVE-6658] - Modify Alter_numbuckets* test to reflect hadoop2 changes * [HIVE-6659] - Update log for list_bucket_* to add pre/post DB * [HIVE-6660] - HiveServer2 running in non-http mode closes server socket for an SSL connection after the 1st request * [HIVE-6661] - WebHCat E2E test TestPig_10 fails (Hadoop 2) * [HIVE-6662] - Vector Join operations with DATE columns fail * [HIVE-6663] - remove TUGIContainingProcessor class as it is not used anymore * [HIVE-6664] - Vectorized variance computation differs from row mode computation. * [HIVE-6666] - Metastore init scripts should always populate the version information at the end * [HIVE-6668] - When auto join convert is on and noconditionaltask is off, ConditionalResolverCommonJoin fails to resolve map joins. * [HIVE-6670] - ClassNotFound with Serde * [HIVE-6671] - WebHCat Job Submission API 'enablelog' parameter is only supported with Hadoop 1 * [HIVE-6672] - JARs loaded by permanent functions don't work properly with HiveServer2 * [HIVE-6674] - "show grant on all" throws NPE * [HIVE-6676] - hcat cli fails to run when running with hive on tez * [HIVE-6680] - Decimal128#update(Decimal128 o, short scale) should adjust the unscaled value. * [HIVE-6681] - Describe table sometimes shows "from deserializer" for column comments * [HIVE-6682] - nonstaged mapjoin table memory check may be broken * [HIVE-6686] - webhcat does not honour -Dlog4j.configuration=$WEBHCAT_LOG4J of log4j.properties file on local filesystem. * [HIVE-6687] - JDBC ResultSet fails to get value by qualified projection name * [HIVE-6689] - Provide an option to not display partition columns separately in describe table output * [HIVE-6690] - NPE in tez session state * [HIVE-6700] - In some queries inputs are closed on Tez before the operator pipeline is flushed * [HIVE-6701] - Analyze table compute statistics for decimal columns. * [HIVE-6702] - TezMergedLogicalInput needs to inform the framework when it is ready * [HIVE-6704] - date_add()/date_sub()/datediff() fail with NPE with null input * [HIVE-6706] - Tez queries fail when there are no input paths * [HIVE-6707] - Lazy maps are broken (LazyMap and LazyBinaryMap) * [HIVE-6708] - ConstantVectorExpression should create copies of data objects rather than referencing them * [HIVE-6710] - Deadlocks seen in transaction handler using mysql * [HIVE-6711] - ORC maps uses getMapSize() from MapOI which is unreliable * [HIVE-6716] - ORC struct throws NPE for tables with inner structs having null values * [HIVE-6721] - Streaming ingest needs to be able to send many heartbeats together * [HIVE-6724] - HCatStorer throws ClassCastException while storing tinyint/smallint data * [HIVE-6728] - Missing file override-container-log4j.properties in Hcatalog * [HIVE-6732] - Update Release Notes for Hive 0.13 * [HIVE-6733] - Driver context logs every query in the "warn" level * [HIVE-6734] - DDL locking too course grained in new db txn manager * [HIVE-6739] - Hive HBase query fails on Tez due to missing jars and then due to NPE in getSplits * [HIVE-6740] - Typo in src/ql/pom.xml * [HIVE-6742] - Tez Outputs need to be started before accessing the writer * [HIVE-6744] - Permanent UDF lookup fails when current DB has uppercase letters * [HIVE-6748] - FileSinkOperator needs to cleanup held references for container reuse * [HIVE-6749] - Turn hive.auto.convert.join.use.nonstaged off by default * [HIVE-6750] - Hive printing debug information in stdout after the end of CLI session * [HIVE-6752] - Vectorized Between and IN expressions don't work with decimal, date types. * [HIVE-6753] - Unions on Tez NPE when there's a mapjoin the union work * [HIVE-6757] - Remove deprecated parquet classes from outside of org.apache package * [HIVE-6763] - HiveServer2 in http mode might send same kerberos client ticket in case of concurrent requests resulting in server throwing a replay exception * [HIVE-6765] - ASTNodeOrigin unserializable leads to fail when join with view * [HIVE-6766] - HCatLoader always returns Char datatype with maxlength(255) when table format is ORC * [HIVE-6771] - Update WebHCat E2E tests now that comments is reported correctly in "describe table" output * [HIVE-6778] - ql/src/test/queries/clientpositive/pcr.q covers the test which generate 1.0 =1 predicate in partition pruner. * [HIVE-6779] - Hive cli may get into inconsistent state when Ctrl-C is hit on hadoop2 * [HIVE-6780] - Set tez credential file property along with MR conf property for Tez jobs * [HIVE-6781] - Hive JDBC in http mode is using HiveConf - should be removed * [HIVE-6782] - HiveServer2Concurrency issue when running with tez intermittently, throwing "org.apache.tez.dag.api.SessionNotRunning: Application not running" error * [HIVE-6786] - Off by one error in ORC PPD * [HIVE-6787] - ORC+ACID assumes all missing buckets are in ACID structure * [HIVE-6789] - HiveStatement client transport lock should unlock in finally block. * [HIVE-6795] - metastore initialization should add default roles with default, SBA * [HIVE-6796] - Create/drop roles is case-sensitive whereas 'set role' is case insensitive * [HIVE-6800] - HiveServer2 is not passing proxy user setting through hive-site * [HIVE-6802] - Fix metastore.thrift: add partition_columns.types constant * [HIVE-6804] - sql std auth - granting existing table privilege to owner should result in error * [HIVE-6808] - sql std auth - describe table, show partitions are not being authorized * [HIVE-6812] - show compactions returns error when there are no compactions * [HIVE-6816] - jar upload path w/o schema is not handled correctly * [HIVE-6818] - Array out of bounds when ORC is used with ACID and predicate push down * [HIVE-6819] - Correctness issue with Hive limit operator & predicate push down * [HIVE-6821] - Fix some non-deterministic tests * [HIVE-6823] - sql std auth - database authorization does not check for role ownership * [HIVE-6829] - alter table foo compact gives an error * [HIVE-6830] - After major compaction unable to read from partition with MR job * [HIVE-6834] - Dynamic partition optimization bails out after removing file sink operator * [HIVE-6837] - HiveServer2 thrift/http mode & binary mode proxy user check fails reporting IP null for client * [HIVE-6838] - q.out files need correction for stats properties - sample8,transform_ppr1,transform_ppr2,union_ppr * [HIVE-6840] - Use Unordered Output for Bucket Map Joins on Tez * [HIVE-6841] - Vectorized execution throws NPE for partitioning columns with __HIVE_DEFAULT_PARTITION__ * [HIVE-6845] - TestJdbcDriver.testShowRoleGrant can fail if TestJdbcDriver/TestJdbcDriver2 run together * [HIVE-6846] - allow safe set commands with sql standard authorization * [HIVE-6848] - Importing into an existing table fails * [HIVE-6850] - For FetchOperator, Driver uses the valid transaction list from the previous query * [HIVE-6855] - A couple of errors in MySQL db creation script for transaction tables * [HIVE-6856] - ddl commands fail with permissions issue when running using webhcat in secure Tez cluster * [HIVE-6860] - Issue with FS based stats collection on Tez * [HIVE-6863] - HiveServer2 binary mode throws exception with PAM * [HIVE-6864] - HiveServer2 concurrency uses incorrect user information in unsecured mode * [HIVE-6873] - DISTINCT clause in aggregates is handled incorrectly by vectorized execution * [HIVE-6875] - Make sure resources in Tez resource list are unique * [HIVE-6878] - MetaStoreDirectSql may not handle empty filter correctly * [HIVE-6879] - Vectorization: IsNull returns incorrect output column. * [HIVE-6881] - Postgres Upgrade script for hive 0.13 is broken * [HIVE-6882] - Make upgrade script schemaTool friendly * [HIVE-6887] - Add missing params to hive-default.xml.template * [HIVE-6894] - hcatalog streaming version in 0.13 branch should be "0.13" * [HIVE-6906] - Fix assembly/src.xml so that sr tar ball contains top level pom.xml * [HIVE-6907] - HiveServer2 - wrong user gets used for metastore operation with embedded metastore ** Improvement * [HIVE-860] - Persistent distributed cache * [HIVE-1180] - Support Common Table Expressions (CTEs) in Hive * [HIVE-1511] - Hive plan serialization is slow * [HIVE-2365] - SQL support for bulk load into HBase * [HIVE-2599] - Support Composit/Compound Keys with HBaseStorageHandler * [HIVE-2983] - Hive ant targets for publishing maven artifacts can be simplified * [HIVE-3107] - Improve semantic analyzer to better handle column name references in group by/sort by clauses * [HIVE-3181] - getDatabaseMajor/Minor version does not return values * [HIVE-3420] - Inefficiency in hbase handler when process query including rowkey range scan * [HIVE-3611] - Hive JDBC doesn't support BINARY column * [HIVE-3616] - Use Paths consistently * [HIVE-3777] - add a property in the partition to figure out if stats are accurate * [HIVE-3959] - Update Partition Statistics in Metastore Layer * [HIVE-3990] - Provide input threshold for direct-fetcher (HIVE-2925) * [HIVE-4340] - ORC should provide raw data size * [HIVE-4395] - Support TFetchOrientation.FIRST for HiveServer2 FetchResults * [HIVE-4449] - show create table method returns only "COMMENT 'from deserializer'" * [HIVE-4518] - Counter Strike: Operation Operator * [HIVE-4523] - round() function with specified decimal places not consistent with mysql * [HIVE-4568] - Beeline needs to support resolving variables * [HIVE-4632] - Use hadoop counter as a stat publisher * [HIVE-4693] - If you set hive.optimize.skewjoin=true, and number of identical keys is < hive.skewjoin.key don't fail with FileNotFoundException * [HIVE-4732] - Reduce or eliminate the expensive Schema equals() check for AvroSerde * [HIVE-4739] - Create public parallel test environment * [HIVE-4741] - Add Hive config API to modify the restrict list * [HIVE-4880] - Rearrange explain order of stages simpler * [HIVE-4888] - listPartitionsByFilter doesn't support lt/gt/lte/gte * [HIVE-4904] - A little more CP crossing RS boundaries * [HIVE-4914] - filtering via partition name should be done inside metastore server (implementation) * [HIVE-4997] - HCatalog doesn't allow multiple input tables * [HIVE-5112] - Upgrade protobuf to 2.5 from 2.4 * [HIVE-5114] - add a target to run tests without rebuilding them * [HIVE-5155] - Support secure proxy user access to HiveServer2 * [HIVE-5193] - Columnar Pushdown for RC/ORC File not happening in HCatLoader * [HIVE-5204] - Change type compatibility methods to use PrimitiveCategory rather than TypeInfo * [HIVE-5220] - Add option for removing intermediate directory for partition, which is empty * [HIVE-5223] - explain doesn't show serde used for table * [HIVE-5234] - partition name filtering uses suboptimal datastructures * [HIVE-5263] - Query Plan cloning time could be improved by using Kryo * [HIVE-5276] - Skip redundant string encoding/decoding for hiveserver2 * [HIVE-5288] - Perflogger should log under single class * [HIVE-5298] - AvroSerde performance problem caused by HIVE-3833 * [HIVE-5300] - MapredLocalTask logs success message twice * [HIVE-5306] - Use new GenericUDF instead of basic UDF for UDFAbs class * [HIVE-5351] - Secure-Socket-Layer (SSL) support for HiveServer2 * [HIVE-5361] - PTest2 should allow a different JVM for compilation versus execution * [HIVE-5366] - Refactor a few object inspector class to eliminate redundant information * [HIVE-5370] - format_number udf should take user specifed format as argument * [HIVE-5372] - Refactor TypeInfo and PrimitiveTypeEntry class hierachy to eliminate info repetition * [HIVE-5383] - PTest2 should allow you to specify ant properties which will only be added to the command when a test is executed * [HIVE-5391] - make ORC predicate pushdown work with vectorization * [HIVE-5395] - Various cleanup in ptf code * [HIVE-5408] - Method matching needs to be improved in NumericOpMethodResolver to suport decimal type with parameters * [HIVE-5423] - Speed up testing of scalar UDFS * [HIVE-5445] - PTest2 should use testonly target * [HIVE-5483] - use metastore statistics to optimize max/min/etc. queries * [HIVE-5512] - metastore filter pushdown should support "between" * [HIVE-5514] - webhcat_server.sh foreground option does not work as expected * [HIVE-5520] - Use factory methods to instantiate HiveDecimal instead of constructors * [HIVE-5554] - add more comments to CombineHiveInputFormat.java, BucketizedHiveInputFormat.java * [HIVE-5602] - Micro optimize select operator * [HIVE-5605] - AddResourceOperation, DeleteResourceOperation, DfsOperation, SetOperation should be removed from org.apache.hive.service.cli.operation * [HIVE-5619] - Allow concat() to accept mixed string/binary args * [HIVE-5626] - enable metastore direct SQL for drop/similar queries * [HIVE-5632] - Eliminate splits based on SARGs using stripe statistics in ORC * [HIVE-5655] - Hive incorrecly handles divide-by-zero case * [HIVE-5663] - Refactor ORC RecordReader to operate on direct & wrapped ByteBuffers * [HIVE-5666] - use Path instead of String for IOContext.inputPath * [HIVE-5679] - add date support to metastore JDO/SQL * [HIVE-5700] - enforce single date format for partition column storage * [HIVE-5707] - Validate values for ConfVar * [HIVE-5708] - PTest2 should trim long logs when posting to jira * [HIVE-5721] - Incremental build is disabled by MCOMPILER-209 * [HIVE-5722] - Skip generating vectorization code if possible * [HIVE-5726] - The DecimalTypeInfo instance associated with a decimal constant is not in line with the precision/scale of the constant * [HIVE-5728] - Make ORC InputFormat/OutputFormat usable outside Hive * [HIVE-5731] - Use new GenericUDF instead of basic UDF for UDFDate* classes * [HIVE-5737] - Provide StructObjectInspector for UDTFs rather than ObjectInspect[] * [HIVE-5750] - Queue name should not be hardcoded to 'default' when running Tez jobs * [HIVE-5753] - Remove collector from Operator base class * [HIVE-5793] - Update hive-default.xml.template for HIVE-4002 * [HIVE-5794] - HiveServer2 should retry startup * [HIVE-5802] - Hive UDF pow limits the second input to integer when the first input is a decimal * [HIVE-5855] - Add deprecated methods back to ColumnProjectionUtils * [HIVE-5872] - Make UDAFs such as GenericUDAFSum report accurate precision/scale for decimal types * [HIVE-5879] - Fix spelling errors in hive-default.xml * [HIVE-5901] - Query cancel should stop running MR tasks * [HIVE-5919] - COALESCE does not vectorize * [HIVE-5946] - DDL authorization task factory should be better tested * [HIVE-5951] - improve performance of adding partitions from client * [HIVE-5982] - Remove redundant filesystem operations and methods in FileSink * [HIVE-5985] - Make qfile_regex to accept multiple patterns * [HIVE-5993] - JDBC Driver should not hard-code the database name * [HIVE-6013] - Supporting Quoted Identifiers in Column Names * [HIVE-6023] - Numeric Data Type Support * [HIVE-6037] - Synchronize HiveConf with hive-default.xml.template and support show conf * [HIVE-6070] - document HIVE-6052 * [HIVE-6089] - Add metrics to HiveServer2 * [HIVE-6095] - Use paths consistently II * [HIVE-6109] - Support customized location for EXTERNAL tables created by Dynamic Partitioning * [HIVE-6115] - Remove redundant code in HiveHBaseStorageHandler * [HIVE-6116] - Use Paths consistently III * [HIVE-6118] - PTest2 is banned periodically by AWS because JClouds is too agressive * [HIVE-6121] - Use Paths Consistently - IV * [HIVE-6122] - Implement show grant on * [HIVE-6144] - Implement non-staged MapJoin * [HIVE-6171] - Use Paths consistently - V * [HIVE-6173] - Beeline doesn't accept --hiveconf option as Hive CLI does * [HIVE-6182] - LDAP Authentication errors need to be more informative * [HIVE-6188] - Document hive.metastore.try.direct.sql & hive.metastore.try.direct.sql.ddl * [HIVE-6193] - change partition pruning request to metastore to use list instead of set * [HIVE-6201] - Print failed query for qfile tests * [HIVE-6204] - The result of show grant / show role should be tabular format * [HIVE-6217] - Refactor Beeline's JDBC connection to use property map instead of long url * [HIVE-6221] - Optimize stats based queries in presence of filter * [HIVE-6228] - Use paths consistently - VII * [HIVE-6248] - HCatReader/Writer should hide Hadoop and Hive classes * [HIVE-6256] - add batch dropping of partitions to Hive metastore (as well as to dropTable) * [HIVE-6288] - MSCK can be slow when adding partitions * [HIVE-6304] - Update HCatReader/Writer docs to reflect recent changes * [HIVE-6325] - Enable using multiple concurrent sessions in tez * [HIVE-6327] - A few mathematic functions don't take decimal input * [HIVE-6339] - Implement new JDK7 schema management APIs in java.sql.Connection * [HIVE-6342] - hive drop partitions should use standard expr filter instead of some custom class * [HIVE-6378] - HCatClient::createTable() doesn't allow SerDe class to be specified * [HIVE-6385] - UDF degrees() doesn't take decimal as input * [HIVE-6393] - Support unqualified column references in Joining conditions * [HIVE-6410] - Allow output serializations separators to be set for HDFS path as well. * [HIVE-6418] - MapJoinRowContainer has large memory overhead in typical cases * [HIVE-6429] - MapJoinKey has large memory overhead in typical cases * [HIVE-6456] - Implement Parquet schema evolution * [HIVE-6457] - Ensure Parquet integration has good error messages for data types not supported * [HIVE-6459] - Change the precison/scale for intermediate sum result in the avg() udf * [HIVE-6481] - Add .reviewboardrc file * [HIVE-6484] - HiveServer2 doAs should be session aware both for secured and unsecured session implementation. * [HIVE-6486] - Support secure Subject.doAs() in HiveServer2 JDBC client. * [HIVE-6587] - allow specifying additional Hive classpath for Hadoop * [HIVE-6613] - Control when spcific Inputs / Outputs are started * [HIVE-6697] - HiveServer2 secure thrift/http authentication needs to support SPNego * [HIVE-6703] - Tez should store SHA of the jar when uploading to cache * [HIVE-6738] - HiveServer2 secure Thrift/HTTP needs to accept doAs parameter from proxying intermediary * [HIVE-6743] - Allow specifying the log level for Tez tasks * [HIVE-6825] - custom jars for Hive query should be uploaded to scratch dir per query; and/or versioned * [HIVE-6876] - Logging information should include thread id ** New Feature * [HIVE-784] - Support uncorrelated subqueries in the WHERE clause * [HIVE-1466] - Add NULL DEFINED AS to ROW FORMAT specification * [HIVE-2093] - create/drop database should populate inputs/outputs and check concurrency and user permission * [HIVE-3976] - Support specifying scale and precision with Hive decimal type * [HIVE-4660] - Let there be Tez * [HIVE-4943] - An explode function that includes the item's position in the array * [HIVE-5191] - Add char data type * [HIVE-5294] - Create collect UDF and make evaluator reusable * [HIVE-5324] - Extend record writer and ORC reader/writer interfaces to provide statistics * [HIVE-5325] - Implement statistics providing ORC writer and reader interfaces * [HIVE-5369] - Annotate hive operator tree with statistics from metastore * [HIVE-5562] - Provide stripe level column statistics in ORC * [HIVE-5613] - Subquery support: disallow nesting of SubQueries * [HIVE-5614] - Subquery support: allow subquery expressions in having clause * [HIVE-5615] - Subquery support: more tests * [HIVE-5783] - Native Parquet Support in Hive * [HIVE-5795] - Hive should be able to skip header and footer rows when reading data file for a table * [HIVE-5814] - Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat * [HIVE-5908] - Use map-join hint to cache intermediate result * [HIVE-6006] - Add UDF to calculate distance between geographic coordinates * [HIVE-6098] - Merge Tez branch into trunk * [HIVE-6100] - Introduce basic set operations as UDFs * [HIVE-6192] - Optimize sum(1) to answer query using metadata * [HIVE-6226] - It should be possible to get hadoop, hive, and pig version being used by WebHCat * [HIVE-6324] - Determine hive version using hive API or hive CLI * [HIVE-6455] - Scalable dynamic partitioning and bucketing optimization * [HIVE-6466] - Add support for pluggable authentication modules (PAM) in Hive * [HIVE-6492] - limit partition number involved in a table scan * [HIVE-6500] - Stats collection via filesystem * [HIVE-6578] - Use ORC file footer statistics through StatsProvidingRecordReader interface for analyze command ** Task * [HIVE-2974] - Add online docs for from_utc_timestamp() and to_utc_timestamp() * [HIVE-4583] - Make Hive compile and run with JDK7 * [HIVE-4856] - Upgrade HCat to 2.0.5-alpha * [HIVE-4862] - Create automatic Precommit testing * [HIVE-5154] - Remove unnecessary array creation in ReduceSinkOperator * [HIVE-5354] - Decimal precision/scale support in ORC file * [HIVE-5355] - JDBC support for decimal precision/scale * [HIVE-5356] - Move arithmatic UDFs to generic UDF implementations * [HIVE-5443] - Few hadoop2 .q.out needs to be updated * [HIVE-5450] - pTest2 TestReportParser is failing due to .svn directory * [HIVE-5500] - Update Brock Noland's username on credits page * [HIVE-5564] - Need to accomodate table decimal columns that were defined prior to HIVE-3976 * [HIVE-5565] - Limit Hive decimal type maximum precision and scale to 38 * [HIVE-5644] - Update PMC status for Thejas and Brock * [HIVE-5670] - annoying ZK exceptions are annoying * [HIVE-5704] - A couple of generic UDFs are not in the right folder/package * [HIVE-5706] - Move a few numeric UDFs to generic implementations * [HIVE-5885] - Add myself and Jitendra to committer list * [HIVE-5886] - [Refactor] Remove unused class JobCloseFeedback * [HIVE-6197] - Use paths consistently - VI * [HIVE-6368] - Document parquet on hive wiki * [HIVE-6629] - Change SET ROLE NONE to SET ROLE ALL * [HIVE-6723] - Tez golden files need to be updated * [HIVE-6767] - Golden file updates for hadoop-2 * [HIVE-6827] - Disable insecure commands with std sql auth * [HIVE-6849] - Golden files update for hadoop-2 ** Test * [HIVE-5289] - PTest2 should disable checking of libraries during batch exec * [HIVE-5290] - Some HCatalog tests have been behaving flaky * [HIVE-5537] - Add a test for ThriftCLIService async execution to ThriftCLIServiceTest * [HIVE-5637] - Sporadic minimr test failure * [HIVE-5693] - Rewrite some tests to reduce test time * [HIVE-6010] - create a test that would ensure vectorization produces same results as non-vectorized execution * [HIVE-6196] - Incorrect package name for few tests. ** Wish * [HIVE-3015] - org.apache.hadoop.hive.jdbc.HiveResultSetMetaData.getColumnClassName Method not supported * [HIVE-5938] - Remove apache.mina dependency for test Release Notes - Hive - Version 0.12.0 ** Sub-task * [HIVE-2304] - Support PreparedStatement.setObject * [HIVE-4055] - add Date data type * [HIVE-4266] - Refactor HCatalog code to org.apache.hive.hcatalog * [HIVE-4324] - ORC Turn off dictionary encoding when number of distinct keys is greater than threshold * [HIVE-4355] - HCatalog test TestPigHCatUtil might fail on JDK7 * [HIVE-4460] - Publish HCatalog artifacts for Hadoop 2.x * [HIVE-4478] - In ORC, add boolean noNulls flag to column stripe metadata * [HIVE-4626] - join_vc.q is not deterministic * [HIVE-4646] - skewjoin.q is failing in hadoop2 * [HIVE-4690] - stats_partscan_1.q makes different result with different hadhoop.mr.rev * [HIVE-4708] - Fix TestCliDriver.combine2.q on 0.23 * [HIVE-4711] - Fix TestCliDriver.list_bucket_query_oneskew_{1,2,3}.q on 0.23 * [HIVE-4712] - Fix TestCliDriver.truncate_* on 0.23 * [HIVE-4713] - Fix TestCliDriver.skewjoin_union_remove_{1,2}.q on 0.23 * [HIVE-4715] - Fix TestCliDriver.{recursive_dir.q,sample_islocalmode_hook.q,input12.q,input39.q,auto_join14.q} on 0.23 * [HIVE-4717] - Fix non-deterministic TestCliDriver on 0.23 * [HIVE-4721] - Fix TestCliDriver.ptf_npath.q on 0.23 * [HIVE-4746] - Fix TestCliDriver.list_bucket_dml_{2,4,5,9,12,13}.q on 0.23 * [HIVE-4750] - Fix TestCliDriver.list_bucket_dml_{6,7,8}.q on 0.23 * [HIVE-4756] - Upgrade Hadoop 0.23 profile to 2.0.5-alpha * [HIVE-4761] - ZooKeeperHiveLockManage.unlockPrimitive has race condition with threads * [HIVE-4762] - HMS cannot handle concurrent requests * [HIVE-4763] - add support for thrift over http transport in HS2 * [HIVE-4767] - ObjectStore.getPMF has concurrency problems * [HIVE-4871] - Apache builds fail with Target "make-pom" does not exist in the project "hcatalog". * [HIVE-4894] - Update maven coordinates of HCatalog artifacts * [HIVE-4895] - Move all HCatalog classes to org.apache.hive.hcatalog * [HIVE-4896] - create binary backwards compatibility layer hcatalog 0.12 and 0.11 * [HIVE-4908] - rename templeton to webhcat? * [HIVE-4940] - udaf_percentile_approx.q is not deterministic * [HIVE-4980] - Fix the compiling error in TestHadoop20SAuthBridge * [HIVE-5013] - [HCatalog] Create hcat.py, hcat_server.py to make HCatalog work on Windows * [HIVE-5014] - [HCatalog] Fix HCatalog build issue on Windows * [HIVE-5015] - [HCatalog] Fix HCatalog unit tests on Windows * [HIVE-5028] - Some tests with fail OutOfMemoryError PermGen Space on Hadoop2 * [HIVE-5035] - [WebHCat] Hardening parameters for Windows * [HIVE-5036] - [WebHCat] Add cmd script for WebHCat * [HIVE-5063] - Fix some non-deterministic or not-updated tests * [HIVE-5066] - [WebHCat] Other code fixes for Windows * [HIVE-5069] - Tests on list bucketing are failing again in hadoop2 * [HIVE-5078] - [WebHCat] Fix e2e tests on Windows plus test cases for new features * [HIVE-5163] - refactor org.apache.hadoop.mapred.HCatMapRedUtil * [HIVE-5213] - remove hcatalog/shims directory * [HIVE-5233] - move hbase storage handler to org.apache.hcatalog package * [HIVE-5236] - Change HCatalog spacing from 4 spaces to 2 * [HIVE-5260] - Introduce HivePassThroughOutputFormat that allows Hive to use general purpose OutputFormats instead of HiveOutputFormats in StorageHandlers * [HIVE-5261] - Make the Hive HBase storage handler work from HCatalog, and use HiveStorageHandlers instead of HCatStorageHandlers ** Bug * [HIVE-2015] - Eliminate bogus Datanucleus.Plugin Bundle ERROR log messages * [HIVE-2379] - Hive/HBase integration could be improved * [HIVE-2473] - Hive throws an NPE when $HADOOP_HOME points to a tarball install directory that contains a build/ subdirectory. * [HIVE-2702] - Enhance listPartitionsByFilter to add support for integral types both for equality and non-equality * [HIVE-2905] - Desc table can't show non-ascii comments * [HIVE-3189] - cast ( as bigint) returning null values * [HIVE-3191] - timestamp - timestamp causes null pointer exception * [HIVE-3253] - ArrayIndexOutOfBounds exception for deeply nested structs * [HIVE-3256] - Update asm version in Hive * [HIVE-3264] - Add support for binary dataype to AvroSerde * [HIVE-3475] - INLINE UDTF doesn't convert types properly * [HIVE-3562] - Some limit can be pushed down to map stage * [HIVE-3588] - Get Hive to work with hbase 94 * [HIVE-3632] - Upgrade datanucleus to support JDK7 * [HIVE-3691] - TestDynamicSerDe failed with IBM JDK * [HIVE-3756] - "LOAD DATA" does not honor permission inheritence * [HIVE-3772] - Fix a concurrency bug in LazyBinaryUtils due to a static field * [HIVE-3810] - HiveHistory.log need to replace '\r' with space before writing Entry.value to historyfile * [HIVE-3846] - alter view rename NPEs with authorization on. * [HIVE-3891] - physical optimizer changes for auto sort-merge join * [HIVE-3926] - PPD on virtual column of partitioned table is not working * [HIVE-3953] - Reading of partitioned Avro data fails because of missing properties * [HIVE-3957] - Add pseudo-BNF grammar for RCFile to Javadoc * [HIVE-3978] - HIVE_AUX_JARS_PATH should have : instead of , as separator since it gets appended to HADOOP_CLASSPATH * [HIVE-4003] - NullPointerException in exec.Utilities * [HIVE-4051] - Hive's metastore suffers from 1+N queries when querying partitions & is slow * [HIVE-4057] - LazyHBaseRow may return cache data if the field is null and make the result wrong * [HIVE-4089] - javax.jdo : jdo2-api dependency not in Maven Central * [HIVE-4106] - SMB joins fail in multi-way joins * [HIVE-4171] - Current database in metastore.Hive is not consistent with SessionState * [HIVE-4181] - Star argument without table alias for UDTF is not working * [HIVE-4194] - JDBC2: HiveDriver should not throw RuntimeException when passed an invalid URL * [HIVE-4214] - OVER accepts general expression instead of just function * [HIVE-4222] - Timestamp type constants cannot be deserialized in JDK 1.6 or less * [HIVE-4233] - The TGT gotten from class 'CLIService' should be renewed on time * [HIVE-4251] - Indices can't be built on tables whose schema info comes from SerDe * [HIVE-4290] - Build profiles: Partial builds for quicker dev * [HIVE-4295] - Lateral view makes invalid result if CP is disabled * [HIVE-4299] - exported metadata by HIVE-3068 cannot be imported because of wrong file name * [HIVE-4300] - ant thriftif generated code that is checkedin is not up-to-date * [HIVE-4322] - SkewedInfo in Metastore Thrift API cannot be deserialized in Python * [HIVE-4339] - build fails after branch (hcatalog version not updated) * [HIVE-4343] - HS2 with kerberos- local task for map join fails * [HIVE-4344] - CREATE VIEW fails when redundant casts are rewritten * [HIVE-4347] - Hcatalog build fail on Windows because javadoc command exceed length limit * [HIVE-4348] - Unit test compile fail at hbase-handler project on Windows becuase of illegal escape character * [HIVE-4350] - support AS keyword for table alias * [HIVE-4351] - Thrift code generation fails due to hcatalog * [HIVE-4364] - beeline always exits with 0 status, should exit with non-zero status on error * [HIVE-4369] - Many new failures on hadoop 2 * [HIVE-4375] - Single sourced multi insert consists of native and non-native table mixed throws NPE * [HIVE-4377] - Add more comment to https://reviews.facebook.net/D1209 (HIVE-2340) * [HIVE-4392] - Illogical InvalidObjectException throwed when use mulit aggregate functions with star columns * [HIVE-4403] - Running Hive queries on Yarn (MR2) gives warnings related to overriding final parameters * [HIVE-4406] - Missing "/" or "/" in hs2 jdbc uri switches mode to embedded mode * [HIVE-4407] - TestHCatStorer.testStoreFuncAllSimpleTypes fails because of null case difference * [HIVE-4418] - TestNegativeCliDriver failure message if cmd succeeds is misleading * [HIVE-4421] - Improve memory usage by ORC dictionaries * [HIVE-4422] - Test output need to be updated for Windows only unit test in TestCliDriver * [HIVE-4424] - MetaStoreUtils.java.orig checked in mistakenly by HIVE-4409 * [HIVE-4428] - Misspelling in describe extended output * [HIVE-4430] - Semantic analysis fails in presence of certain literals in on clause * [HIVE-4433] - Fix C++ Thrift bindings broken in HIVE-4322 * [HIVE-4435] - Column stats: Distinct value estimator should use hash functions that are pairwise independent * [HIVE-4436] - hive.exec.parallel=true doesn't work on hadoop-2 * [HIVE-4438] - Remove unused join configuration parameter: hive.mapjoin.size.key * [HIVE-4439] - Remove unused join configuration parameter: hive.mapjoin.cache.numrows * [HIVE-4440] - SMB Operator spills to disk like it's 1999 * [HIVE-4441] - [HCatalog] WebHCat does not honor user home directory * [HIVE-4442] - [HCatalog] WebHCat should not override user.name parameter for Queue call * [HIVE-4465] - webhcat e2e tests succeed regardless of exitvalue * [HIVE-4466] - Fix continue.on.failure in unit tests to -well- continue on failure in unit tests * [HIVE-4471] - Build fails with hcatalog checkstyle error * [HIVE-4474] - Column access not tracked properly for partitioned tables * [HIVE-4475] - Switch RCFile default to LazyBinaryColumnarSerDe * [HIVE-4486] - FetchOperator slows down SMB map joins by 50% when there are many partitions * [HIVE-4487] - Hive does not set explicit permissions on hive.exec.scratchdir * [HIVE-4489] - beeline always return the same error message twice * [HIVE-4492] - Revert HIVE-4322 * [HIVE-4496] - JDBC2 won't compile with JDK7 * [HIVE-4497] - beeline module tests don't get run by default * [HIVE-4502] - NPE - subquery smb joins fails * [HIVE-4510] - HS2 doesn't nest exceptions properly (fun debug times) * [HIVE-4513] - disable hivehistory logs by default * [HIVE-4516] - Fix concurrency bug in serde/src/java/org/apache/hadoop/hive/serde2/io/TimestampWritable.java * [HIVE-4521] - Auto join conversion fails in certain cases (empty tables, empty partitions, no partitions) * [HIVE-4525] - Support timestamps earlier than 1970 and later than 2038 * [HIVE-4535] - hive build fails with hadoop 0.20 * [HIVE-4540] - JOIN-GRP BY-DISTINCT fails with NPE when mapjoin.mapreduce=true * [HIVE-4542] - TestJdbcDriver2.testMetaDataGetSchemas fails because of unexpected database * [HIVE-4543] - Broken link in HCat 0.5 doc (Reader and Writer Interfaces) * [HIVE-4546] - Hive CLI leaves behind the per session resource directory on non-interactive invocation * [HIVE-4547] - A complex create view statement fails with new Antlr 3.4 * [HIVE-4550] - local_mapred_error_cache fails on some hadoop versions * [HIVE-4554] - Failed to create a table from existing file if file path has spaces * [HIVE-4559] - hcatalog/webhcat scripts in tar.gz don't have execute permissions set * [HIVE-4562] - HIVE-3393 brought in Jackson library,and these four jars should be packed into hive-exec.jar * [HIVE-4566] - NullPointerException if typeinfo and nativesql commands are executed at beeline before a DB connection is established * [HIVE-4572] - ColumnPruner cannot preserve RS key columns corresponding to un-selected join keys in columnExprMap * [HIVE-4573] - Support alternate table types for HiveServer2 * [HIVE-4578] - Changes to Pig's test harness broke HCat e2e tests * [HIVE-4580] - Change DDLTask to report errors using canonical error messages rather than http status codes * [HIVE-4581] - HCat e2e tests broken by changes to Hive's describe table formatting * [HIVE-4585] - Remove unused MR Temp file localization from Tasks * [HIVE-4586] - [HCatalog] WebHCat should return 404 error for undefined resource * [HIVE-4589] - Hive Load command failed when inpath contains space or any restricted characters * [HIVE-4591] - Making changes to webhcat-site.xml have no effect * [HIVE-4593] - ErrorMsg has several messages that reuse the same error code * [HIVE-4611] - SMB joins fail based on bigtable selection policy. * [HIVE-4615] - Invalid column names allowed when created dynamically by a SerDe * [HIVE-4618] - show create table creating unusable DDL when field delimiter is \001 * [HIVE-4619] - Hive 0.11.0 is not working with pre-cdh3u6 and hadoop-0.23 * [HIVE-4638] - Thread local PerfLog can get shared by multiple hiveserver2 sessions * [HIVE-4650] - Getting Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask on auto convert to MapJoin after upgrade to Hive-0.11.0.x from hive-0.10.0.x * [HIVE-4657] - HCatalog checkstyle violation after HIVE-2670 * [HIVE-4677] - [HCatalog] WebHCat e2e tests fail on Hadoop 2 * [HIVE-4679] - WebHCat can deadlock Hadoop if the number of concurrently running tasks if higher or equal than the number of mappers * [HIVE-4683] - fix coverage org.apache.hadoop.hive.cli * [HIVE-4689] - For outerjoins, joinEmitInterval might make wrong result * [HIVE-4691] - orc_createas1.q has minor inconsistency * [HIVE-4692] - Constant agg parameters will be replaced by ExprNodeColumnDesc with single-sourced multi-gby cases * [HIVE-4696] - WebHCat e2e test framework is missing files and instructions * [HIVE-4707] - Support configurable domain name for HiveServer2 LDAP authentication using Active Directory * [HIVE-4710] - ant maven-build -Dmvn.publish.repo=local fails * [HIVE-4724] - ORC readers should have a better error detection for non-ORC files * [HIVE-4730] - Join on more than 2^31 records on single reducer failed (wrong results) * [HIVE-4733] - HiveLockObjectData is not compared properly * [HIVE-4740] - HIVE-2379 is missing hbase.jar itself * [HIVE-4742] - A useless CAST makes Hive fail to create a VIEW based on an UNION * [HIVE-4748] - Fix TempletonUtilsTest failure on Windows * [HIVE-4757] - LazyTimestamp goes into irretrievable NULL mode once inited with NULL once * [HIVE-4781] - LEFT SEMI JOIN generates wrong results when the number of rows belonging to a single key of the right table exceed hive.join.emit.interval * [HIVE-4784] - ant testreport doesn't include any HCatalog tests * [HIVE-4785] - Implement isCaseSensitive for Hive JDBC driver * [HIVE-4789] - FetchOperator fails on partitioned Avro data * [HIVE-4798] - NPE when we call isSame from an instance of ExprNodeConstantDesc with null value * [HIVE-4802] - Fix url check for missing "/" or "/ after hostname in jdb uri * [HIVE-4804] - parallel order by fails for small datasets * [HIVE-4807] - Hive metastore hangs * [HIVE-4808] - WebHCat job submission is killed by TaskTracker since it's not sending a heartbeat properly * [HIVE-4810] - Refactor exec package * [HIVE-4811] - (Slightly) break up the SemanticAnalyzer monstrosity * [HIVE-4812] - Logical explain plan * [HIVE-4814] - Adjust WebHCat e2e tests until HIVE-4703 is addressed * [HIVE-4818] - SequenceId in operator is not thread safe * [HIVE-4820] - webhcat_config.sh should set default values for HIVE_HOME and HCAT_PREFIX that work with default build tree structure * [HIVE-4829] - TestWebHCatE2e checkstyle violation causes all tests to fail * [HIVE-4830] - Test clientnegative/nested_complex_neg.q got broken due to 4580 * [HIVE-4833] - Fix eclipse template classpath to include the correct jdo lib * [HIVE-4836] - make checkstyle ignore IntelliJ files and templeton e2e files * [HIVE-4838] - Refactor MapJoin HashMap code to improve testability and readability * [HIVE-4839] - build-common.xml has * [HIVE-4840] - Fix eclipse template classpath to include the BoneCP lib * [HIVE-4843] - Refactoring MapRedTask and ExecDriver for better re-usability (for tez) and readability * [HIVE-4845] - Correctness issue with MapJoins using the null safe operator * [HIVE-4852] - -Dbuild.profile=core fails * [HIVE-4853] - junit timeout needs to be updated * [HIVE-4854] - testCliDriver_load_hdfs_file_with_space_in_the_name fails on hadoop 2 * [HIVE-4865] - HiveLockObjects: Unlocking retries/times out when query contains ":" * [HIVE-4869] - Clean up HCatalog build post Hive integration * [HIVE-4870] - Explain Extended to show partition info for Fetch Task * [HIVE-4875] - hive config template is not parse-able due to angle brackets in description * [HIVE-4876] - Beeling help text do not contain -f and -e parameters * [HIVE-4878] - With Dynamic partitioning, some queries would scan default partition even if query is not using it. * [HIVE-4883] - TestHadoop20SAuthBridge tests fail sometimes because of race condition * [HIVE-4891] - Distinct includes duplicate records * [HIVE-4892] - PTest2 cleanup after merge * [HIVE-4893] - [WebHCat] HTTP 500 errors should be mapped to 400 for bad request * [HIVE-4899] - Hive returns non-meanful error message for ill-formed fs.default.name * [HIVE-4900] - Fix the mismatched column names in package.jdo * [HIVE-4915] - unit tests fail on windows because of difference in input file size * [HIVE-4927] - When we merge two MapJoin MapRedTasks, the TableScanOperator of the second one should be removed * [HIVE-4928] - Date literals do not work properly in partition spec clause * [HIVE-4929] - the type of all numeric constants is changed to double in the plan * [HIVE-4930] - Classes of metastore should not be included MR-task * [HIVE-4932] - PTFOperator fails resetting PTFPersistence * [HIVE-4935] - Potential NPE in MetadataOnlyOptimizer * [HIVE-4942] - Fix eclipse template files to use correct datanucleus libs * [HIVE-4951] - combine2_win.q.out needs update for HIVE-3253 (increasing nesting levels) * [HIVE-4952] - When hive.join.emit.interval is small, queries optimized by Correlation Optimizer may generate wrong results * [HIVE-4955] - serde_user_properties.q.out needs to be updated * [HIVE-4962] - fix eclipse template broken by HIVE-3256 * [HIVE-4964] - Cleanup PTF code: remove code dealing with non standard sql behavior we had original introduced * [HIVE-4968] - When deduplicating multiple SelectOperators, we should update RowResolver accordinly * [HIVE-4970] - BinaryConverter does not respect nulls * [HIVE-4972] - update code generated by thrift for DemuxOperator and MuxOperator * [HIVE-4987] - Javadoc can generate argument list too long error * [HIVE-4990] - ORC seeks fails with non-zero offset or column projection * [HIVE-4991] - hive build with 0.20 is broken * [HIVE-4995] - select * may incorrectly return empty fields with hbase-handler * [HIVE-4998] - support jdbc documented table types in default configuration * [HIVE-5010] - HCatalog maven integration doesn't override mvn.local.repo in two locations * [HIVE-5012] - [HCatalog] Make HCatalog work on Windows * [HIVE-5017] - DBTokenStore gives compiler warnings * [HIVE-5023] - Hive get wrong result when partition has the same path but different schema or authority * [HIVE-5026] - HIVE-3926 is committed in the state of not rebased to trunk * [HIVE-5034] - [WebHCat] Make WebHCat work for Windows * [HIVE-5046] - Hcatalog's bin/hcat script doesn't respect HIVE_HOME * [HIVE-5047] - Hive client filters partitions incorrectly via pushdown in certain cases involving "or" * [HIVE-5048] - StorageBasedAuthorization provider causes an NPE when asked to authorize from client side. * [HIVE-5049] - Create an ORC test case that has a 0.11 ORC file * [HIVE-5051] - StorageBasedAuthorizationProvider masks lower level exception with IllegalStateException * [HIVE-5055] - SessionState temp file gets created in history file directory * [HIVE-5056] - MapJoinProcessor ignores order of values in removing RS * [HIVE-5060] - JDBC driver assumes executeStatement is synchronous * [HIVE-5061] - Row sampling throws NPE when used in sub-query * [HIVE-5075] - bug in ExprProcFactory.genPruner * [HIVE-5079] - Make Hive compile under Windows * [HIVE-5084] - Fix newline.q on Windows * [HIVE-5085] - Hive Metatool errors out if HIVE_OPTS is set * [HIVE-5087] - Rename npath UDF to matchpath * [HIVE-5089] - Non query PreparedStatements are always failing on remote HiveServer2 * [HIVE-5091] - ORC files should have an option to pad stripes to the HDFS block boundaries * [HIVE-5100] - RCFile::sync(long) missing 1 byte in System.arraycopy() * [HIVE-5104] - HCatStorer fails to store boolean type * [HIVE-5105] - HCatSchema.remove(HCatFieldSchema hcatFieldSchema) does not clean up fieldPositionMap * [HIVE-5106] - HCatFieldSchema overrides equals() but not hashCode() * [HIVE-5120] - document what hive.server2.thrift.sasl.qop values mean in hive-default.xml.template * [HIVE-5122] - Add partition for multiple partition ignores locations for non-first partitions * [HIVE-5123] - group by on a same key producing wrong result * [HIVE-5127] - Upgrade xerces and xalan for WebHCat * [HIVE-5128] - Direct SQL for view is failing * [HIVE-5129] - Multiple table insert fails on count(distinct) * [HIVE-5131] - JDBC client's hive variables are not passed to HS2 * [HIVE-5137] - A Hive SQL query should not return a ResultSet when the underlying plan does not include a FetchTask * [HIVE-5144] - HashTableSink allocates empty new Object[] arrays & OOMs - use a static emptyRow instead * [HIVE-5145] - Fix TestCliDriver.list_bucket_query_multiskew_2.q on hadoop 0.23 * [HIVE-5149] - ReduceSinkDeDuplication can pick the wrong partitioning columns * [HIVE-5156] - HiveServer2 jdbc ResultSet.close should free up resources on server side * [HIVE-5161] - Additional SerDe support for varchar type * [HIVE-5167] - webhcat_config.sh checks for env variables being set before sourcing webhcat-env.sh * [HIVE-5196] - ThriftCLIService.java uses stderr to print the stack trace, it should use the logger instead. * [HIVE-5198] - WebHCat returns exitcode 143 (w/o an explanation) * [HIVE-5199] - Custom SerDe containing a nonSettable complex data type row object inspector throws cast exception with HIVE 0.11 * [HIVE-5203] - FunctionRegistry.getMethodInternal() should prefer method arguments with closer affinity to the original argument types * [HIVE-5210] - WebHCatJTShim implementations are missing Apache license headers * [HIVE-5239] - LazyDate goes into irretrievable NULL mode once inited with NULL once * [HIVE-5241] - Default log4j log level for WebHCat should be INFO not DEBUG * [HIVE-5246] - Local task for map join submitted via oozie job fails on a secure HDFS * [HIVE-5255] - Missing metastore schema files for version 0.11 * [HIVE-5265] - Direct SQL fallback broken on Postgres * [HIVE-5274] - HCatalog package renaming backward compatibility follow-up * [HIVE-5285] - Custom SerDes throw cast exception when there are complex nested structures containing NonSettableObjectInspectors. * [HIVE-5292] - Join on decimal columns fails to return rows * [HIVE-5296] - Memory leak: OOM Error after multiple open/closed JDBC connections. * [HIVE-5297] - Hive does not honor type for partition columns * [HIVE-5301] - Add a schema tool for offline metastore schema upgrade * [HIVE-5322] - FsPermission is initialized incorrectly in HIVE 5513 * [HIVE-5329] - Date and timestamp type converts invalid strings to '1970-01-01' * [HIVE-5337] - org.apache.hcatalog.common.HCatUtil is used by org.apache.hive.hcatalog.templeton.tool * [HIVE-5352] - cast('1.0' as int) returns null * [HIVE-5357] - ReduceSinkDeDuplication optimizer pick the wrong keys in pRS-cGBYm-cRS-cGBYr scenario when there are distinct keys in child GBY * [HIVE-5362] - TestHCatHBaseInputFormat has a bug which will not allow it to run on JDK7 and RHEL 6 * [HIVE-5364] - NPE on some queries from partitioned orc table * [HIVE-5374] - hive-schema-0.13.0.postgres.sql doesn't work * [HIVE-5375] - Bug in Hive-0.12 branch with parameterized types due to merge conflict with HIVE-5199 * [HIVE-5394] - ObjectInspectorConverters.getConvertedOI() does not return the correct object inspector for primitive type. * [HIVE-5401] - Array Out Of Bounds in OrcRecordReader * [HIVE-5402] - StorageBasedAuthorizationProvider is not correctly able to determine that it is running from client-side * [HIVE-5405] - Need to implement PersistenceDelegate for org.antlr.runtime.CommonToken * [HIVE-5410] - Hive command line option --auxpath still does not work post HIVE-5363 * [HIVE-5413] - StorageDelegationAuthorizationProvider uses non-existent org.apache.hive.hcatalog.hbase.HBaseHCatStorageHandler * [HIVE-5416] - templeton/tests/jobsubmission2.conf erroneously removed * [HIVE-5419] - Fix schema tool issues with Oracle metastore * [HIVE-5426] - TestThriftBinaryCLIService tests fail on branch 0.12 * [HIVE-5429] - HiveVarcharWritable length not reset when value is changed * [HIVE-5431] - PassthroughOutputFormat SH changes causes IllegalArgumentException * [HIVE-5433] - Fix varchar unit tests to work with hadoop-2.1.1 * [HIVE-5476] - Authorization-provider tests fail in sequential run * [HIVE-5477] - maven-publish fails because it can't find hive-metastore-0.12.0.pom * [HIVE-5488] - some files are missing apache license headers * [HIVE-5489] - NOTICE copyright dates are out of date, README needs update * [HIVE-5493] - duplicate jars with different versions for guava, commons-logging * [HIVE-5497] - Hive trunk broken against hadoop 0.20.2 ** Improvement * [HIVE-2084] - Upgrade datanucleus from 2.0.3 to a more recent version (3.?) * [HIVE-2608] - Do not require AS a,b,c part in LATERAL VIEW * [HIVE-2906] - Support providing some table properties by user via SQL * [HIVE-3603] - Enable client-side caching for scans on HBase * [HIVE-3725] - Add support for pulling HBase columns with prefixes * [HIVE-3764] - Support metastore version consistency check * [HIVE-3807] - Hive authorization should use short username when Kerberos authentication * [HIVE-4002] - Fetch task aggregation for simple group by query * [HIVE-4068] - Size of aggregation buffer which uses non-primitive type is not estimated correctly * [HIVE-4172] - JDBC2 does not support VOID type * [HIVE-4209] - Cache evaluation result of deterministic expression and reuse it * [HIVE-4228] - Bump up hadoop2 version in trunk * [HIVE-4241] - optimize hive.enforce.sorting and hive.enforce bucketing join * [HIVE-4268] - Beeline should support the -f option * [HIVE-4294] - Single sourced multi query cannot handle lateral view * [HIVE-4310] - optimize count(distinct) with hive.map.groupby.sorted * [HIVE-4393] - Make the deleteData flag accessable from DropTable/Partition events * [HIVE-4409] - Prevent incompatible column type changes * [HIVE-4423] - Improve RCFile::sync(long) 10x * [HIVE-4443] - [HCatalog] Have an option for GET queue to return all job information in single call * [HIVE-4444] - [HCatalog] WebHCat Hive should support equivalent parameters as Pig * [HIVE-4459] - Script hcat is overriding HIVE_CONF_DIR variable * [HIVE-4530] - Enforce minmum ant version required in build script * [HIVE-4549] - JDBC compliance change TABLE_SCHEMA to TABLE_SCHEM * [HIVE-4579] - Create a SARG interface for RecordReaders * [HIVE-4588] - Support session level hooks for HiveServer2 * [HIVE-4601] - WebHCat needs to support proxy users * [HIVE-4609] - Allow hive tests to specify an alternative to /tmp * [HIVE-4610] - HCatalog checkstyle violation after HIVE-4578 * [HIVE-4617] - Asynchronous execution in HiveServer2 to run a query in non-blocking mode * [HIVE-4620] - MR temp directory conflicts in case of parallel execution mode * [HIVE-4647] - RetryingHMSHandler logs too many error messages * [HIVE-4658] - Make KW_OUTER optional in outer joins * [HIVE-4675] - Create new parallel unit test environment * [HIVE-4682] - Temporary files are not closed in PTFPersistence on jvm reuse. * [HIVE-4772] - Enable parallel execution of various E2E tests * [HIVE-4825] - Separate MapredWork into MapWork and ReduceWork * [HIVE-4827] - Merge a Map-only task to its child task * [HIVE-4858] - Sort "show grant" result to improve usability and testability * [HIVE-4873] - Sort candidate functions in case of UDFArgumentException * [HIVE-4874] - Identical methods PTFDeserializer.addOIPropertiestoSerDePropsMap(), PTFTranslator.addOIPropertiestoSerDePropsMap() * [HIVE-4877] - In ExecReducer, remove tag from the row which will be passed to the first Operator at the Reduce-side * [HIVE-4879] - Window functions that imply order can only be registered at compile time * [HIVE-4885] - Alternative object serialization for execution plan in hive testing * [HIVE-4913] - Put deterministic ordering in the top-K ngrams output of UDF context_ngrams() * [HIVE-4920] - PTest2 handle Spot Price increases gracefully and improve rsync paralllelsim * [HIVE-4948] - WriteLockTest and ZNodeNameTest do not follow test naming pattern * [HIVE-4954] - PTFTranslator hardcodes ranking functions * [HIVE-4960] - lastAlias in CommonJoinOperator is not used * [HIVE-4967] - Don't serialize unnecessary fields in query plan * [HIVE-4985] - refactor/clean up partition name pruning to be usable inside metastore server * [HIVE-4992] - add ability to skip javadoc during build * [HIVE-5006] - Re-factor HiveServer2 JDBC PreparedStatement to avoid duplicate code * [HIVE-5027] - Upgrade Ivy to 2.3 * [HIVE-5031] - [WebHCat] GET job/:jobid to return userargs for a job in addtion to status information * [HIVE-5062] - Insert + orderby + limit does not need additional RS for limiting rows * [HIVE-5111] - ExprNodeColumnDesc doesn't distinguish partition and virtual columns, causing partition pruner to receive the latter * [HIVE-5121] - Remove obsolete code on SemanticAnalyzer#genJoinTree * [HIVE-5158] - allow getting all partitions for table to also use direct SQL path * [HIVE-5182] - log more stuff via PerfLogger * [HIVE-5206] - Support parameterized primitive types * [HIVE-5209] - JDBC support for varchar * [HIVE-5267] - Use array instead of Collections if possible in DemuxOperator * [HIVE-5278] - Move some string UDFs to GenericUDFs, for better varchar support * [HIVE-5363] - HIVE-3978 broke the command line option --auxpath ** New Feature * [HIVE-305] - Port Hadoop streaming's counters/status reporters to Hive Transforms * [HIVE-1402] - Add parallel ORDER BY to Hive * [HIVE-2206] - add a new optimizer for query correlation discovery and optimization * [HIVE-2482] - Convenience UDFs for binary data type * [HIVE-2517] - Support group by on struct type * [HIVE-2655] - Ability to define functions in HQL * [HIVE-2670] - A cluster test utility for Hive * [HIVE-3255] - Add DBTokenStore to store Delegation Tokens in DB * [HIVE-4005] - Column truncation * [HIVE-4095] - Add exchange partition in Hive * [HIVE-4123] - The RLE encoding for ORC can be improved * [HIVE-4246] - Implement predicate pushdown for ORC * [HIVE-4531] - [WebHCat] Collecting task logs to hdfs * [HIVE-4614] - Support outer lateral view * [HIVE-4844] - Add varchar data type * [HIVE-4911] - Enable QOP configuration for Hive Server 2 thrift transport * [HIVE-4963] - Support in memory PTF partitions ** Task * [HIVE-4331] - Integrated StorageHandler for Hive and HCat using the HiveStorageHandler * [HIVE-4819] - Comments in CommonJoinOperator for aliasTag is not valid * [HIVE-4886] - beeline code should have apache license headers * [HIVE-4999] - Shim class HiveHarFileSystem does not have a hadoop2 counterpart * [HIVE-5059] - Meaningless warning message from TypeCheckProcFactory * [HIVE-5116] - HIVE-2608 didn't removed udtf_not_supported2.q test * [HIVE-5219] - Move VerifyingObjectStore into ql package * [HIVE-5313] - HIVE-4487 breaks build because 0.20.2 is missing FSPermission(string) ** Test * [HIVE-4526] - auto_sortmerge_join_9.q throws NPE but test is succeeded * [HIVE-4636] - Failing on TestSemanticAnalysis.testAddReplaceCols in trunk * [HIVE-4645] - Stat information like numFiles and totalSize is not correct when sub-directory is exists * [HIVE-4743] - Improve test coverage of package org.apache.hadoop.hive.ql.io * [HIVE-4779] - Enhance coverage of package org.apache.hadoop.hive.ql.udf * [HIVE-4791] - improve test coverage of package org.apache.hadoop.hive.ql.udf.xml * [HIVE-4796] - Increase coverage of package org.apache.hadoop.hive.common.metrics * [HIVE-4805] - Enhance coverage of package org.apache.hadoop.hive.ql.exec.errors * [HIVE-4813] - Improve test coverage of package org.apache.hadoop.hive.ql.optimizer.pcr * [HIVE-5029] - direct SQL perf optimization cannot be tested well * [HIVE-5096] - Add q file tests for ORC predicate pushdown * [HIVE-5117] - orc_dictionary_threshold is not deterministic * [HIVE-5147] - Newly added test TestSessionHooks is failing on trunk * [HIVE-5197] - TestE2EScenerios.createTaskAttempt should use MapRedUtil Release Notes - Hive - Version 0.11.0 ** Sub-task * [HIVE-2340] - optimize orderby followed by a groupby * [HIVE-4075] - TypeInfoFactory is not thread safe and is access by multiple threads * [HIVE-4141] - InspectorFactories contains static HashMaps which can cause infinite loop * [HIVE-4162] - disable TestBeeLineDriver * [HIVE-4176] - disable TestBeeLineDriver in ptest util * [HIVE-4238] - Integrate HCatalog site into Hive site * [HIVE-4263] - Adjust build.xml package command to move all hcat jars and binaries into build * [HIVE-4264] - Move HCatalog trunk code from trunk/hcatalog/historical to trunk/hcatalog * [HIVE-4265] - HCatalog branches need to move out of trunk/hcatalog/historical * [HIVE-4278] - HCat needs to get current Hive jars instead of pulling them from maven repo * [HIVE-4325] - Merge HCat NOTICE file with Hive NOTICE file * [HIVE-4326] - Clean up remaining items in hive/hcatalog/historical/trunk ** Bug * [HIVE-4820] - webhcat_config.sh should set default values for HIVE_HOME and HCAT_PREFIX that work with default build tree structure * [HIVE-2264] - Hive server is SHUTTING DOWN when invalid queries beeing executed. * [HIVE-2332] - If all of the parameters of distinct functions are exists in group by columns, query fails in runtime * [HIVE-2689] - ObjectInspectorConverters cannot convert Void types to Array/Map/Struct types. * [HIVE-2723] - should throw "Ambiguous column reference key" Exception in particular join condition * [HIVE-2794] - Aggregations without grouping should return NULL when applied to partitioning column of a partitionless table * [HIVE-2820] - Invalid tag is used for MapJoinProcessor * [HIVE-2839] - Filters on outer join with mapjoin hint is not applied correctly * [HIVE-3084] - Hive CI failing due to script_broken_pipe1.q * [HIVE-3140] - Comment indenting is broken for "describe" in CLI * [HIVE-3179] - HBase Handler doesn't handle NULLs properly * [HIVE-3197] - Hive compile errors under Java 7 (JDBC 4.1) * [HIVE-3297] - change hive.auto.convert.join's default value to true * [HIVE-3300] - LOAD DATA INPATH fails if a hdfs file with same name is added to table * [HIVE-3308] - Mixing avro and snappy gives null values * [HIVE-3348] - semi-colon in comments in .q file does not work * [HIVE-3381] - Result of outer join is not valid * [HIVE-3384] - HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification * [HIVE-3403] - user should not specify mapjoin to perform sort-merge bucketed join * [HIVE-3428] - Fix log4j configuration errors when running hive on hadoop23 * [HIVE-3446] - PrimitiveObjectInspector doesn't handle timestamps properly * [HIVE-3464] - Merging join tree may reorder joins which could be invalid * [HIVE-3490] - Implement * or a.* for arguments to UDFs * [HIVE-3528] - Avro SerDe doesn't handle serializing Nullable types that require access to a Schema * [HIVE-3537] - release locks at the end of move tasks * [HIVE-3582] - NPE in union processing followed by lateral view followed by 2 group bys * [HIVE-3594] - When Group by Partition Column Type is Timestamp or STRING Which Format contains "HH:MM:SS", It will occur URISyntaxException * [HIVE-3622] - reflect udf cannot find method which has arguments of primitive types and String, Binary, Timestamp types mixed * [HIVE-3631] - script_pipe.q fails when using JDK7 * [HIVE-3645] - RCFileWriter does not implement the right function to support Federation * [HIVE-3648] - HiveMetaStoreFsImpl is not compatible with hadoop viewfs * [HIVE-3665] - Allow URIs without port to be specified in metatool * [HIVE-3697] - External JAR files on HDFS can lead to race condition with hive.downloaded.resources.dir * [HIVE-3698] - enhanceModel.notRequired is incorrectly determined * [HIVE-3699] - Multiple insert overwrite into multiple tables query stores same results in all tables * [HIVE-3702] - Renaming table changes table location scheme/authority * [HIVE-3703] - Hive Query Explain Plan JSON not being created properly * [HIVE-3714] - Patch: Hive's ivy internal resolvers need to use sourceforge for sqlline * [HIVE-3717] - Hive won't compile with -Dhadoop.mr.rev=20S * [HIVE-3728] - make optimizing multi-group by configurable * [HIVE-3729] - Error in groupSetExpression rule in Hive grammar * [HIVE-3735] - PTest doesn't work due to hive snapshot version upgrade to 11 * [HIVE-3741] - Driver.validateConfVariables() should perform more validations * [HIVE-3747] - Provide hive operation name for hookContext * [HIVE-3750] - JDBCStatsPublisher fails when ID length exceeds length of ID column * [HIVE-3757] - union_remove_9.q fails in trunk (hadoop 23) * [HIVE-3760] - TestNegativeMinimrCliDriver_mapreduce_stack_trace.q fails on hadoop-1 * [HIVE-3766] - Enable adding hooks to hive meta store init * [HIVE-3767] - BucketizedHiveInputFormat should be automatically used with Bucketized Map Joins also * [HIVE-3771] - HIVE-3750 broke TestParse * [HIVE-3774] - Sort merge join should work if join cols are a prefix of sort columns for each partition * [HIVE-3775] - Unit test failures due to unspecified order of results in "show grant" command * [HIVE-3778] - Add MapJoinDesc.isBucketMapJoin() as part of explain plan * [HIVE-3782] - testCliDriver_sample_islocalmode_hook fails on hadoop-1 * [HIVE-3783] - stats19.q is failing on trunk * [HIVE-3787] - Regression introduced from HIVE-3401 * [HIVE-3788] - testCliDriver_repair fails on hadoop-1 * [HIVE-3789] - Patch HIVE-3648 causing the majority of unit tests to fail on branch 0.9 * [HIVE-3795] - NPE in SELECT when WHERE-clause is an and/or/not operation involving null * [HIVE-3800] - testCliDriver_combine2 fails on hadoop-1 * [HIVE-3801] - testCliDriver_loadpart_err fails on hadoop-1 * [HIVE-3802] - testCliDriver_input39 fails on hadoop-1 * [HIVE-3803] - explain dependency should show the dependencies hierarchically in presence of views * [HIVE-3806] - Ptest failing due to "Argument list too long" errors * [HIVE-3809] - Concurrency issue in RCFile: multiple threads can use the same decompressor * [HIVE-3817] - Adding the name space for the maven task for the maven-publish target. * [HIVE-3820] - Consider creating a literal like "D" or "BD" for representing Decimal type constants * [HIVE-3824] - bug if different serdes are used for different partitions * [HIVE-3826] - Rollbacks and retries of drops cause org.datanucleus.exceptions.NucleusObjectNotFoundException: No such database row) * [HIVE-3828] - insert overwrite fails with stored-as-dir in cluster * [HIVE-3829] - Hive CLI needs UNSET TBLPROPERTY command * [HIVE-3832] - Insert overwrite doesn't create a dir if the skewed column position doesnt match * [HIVE-3839] - adding .gitattributes file for normalizing line endings during cross platform development * [HIVE-3840] - hive cli null representation in output is inconsistent * [HIVE-3847] - ppd.remove.duplicatefilters removing filters too aggressively * [HIVE-3849] - Aliased column in where clause for multi-groupby single reducer cannot be resolved * [HIVE-3850] - hour() function returns 12 hour clock value when using timestamp datatype * [HIVE-3852] - Multi-groupby optimization fails when same distinct column is used twice or more * [HIVE-3858] - Normalize left over CRLF files * [HIVE-3861] - Upgrade hbase dependency to 0.94 * [HIVE-3862] - testHBaseNegativeCliDriver_cascade_dbdrop fails on hadoop-1 * [HIVE-3872] - MAP JOIN for VIEW thorws NULL pointer exception error * [HIVE-3873] - lot of tests failing for hadoop 23 * [HIVE-3875] - negative value for hive.stats.ndv.error should be disallowed * [HIVE-3888] - wrong mapside groupby if no partition is being selected * [HIVE-3893] - something wrong with the hive-default.xml * [HIVE-3899] - Partition pruning fails on = expression * [HIVE-3908] - create view statement's outputs contains the view and a temporary dir. * [HIVE-3909] - Wrong data due to HIVE-2820 * [HIVE-3912] - table_access_keys_stats.q fails with hadoop 0.23 * [HIVE-3913] - Possible deadlock in ZK lock manager * [HIVE-3915] - Union with map-only query on one side and two MR job query on the other produces wrong results * [HIVE-3916] - For outer joins, when looping over the rows looking for filtered tags, it doesn't report progress * [HIVE-3918] - Normalize more CRLF line endings * [HIVE-3920] - Change test for HIVE-2332 * [HIVE-3921] - recursive_dir.q fails on 0.23 * [HIVE-3923] - join_filters_overlap.q fails on 0.23 * [HIVE-3924] - join_nullsafe.q fails on 0.23 * [HIVE-3927] - Potential overflow with new RCFileCat column sizes options * [HIVE-3931] - Add Oracle metastore upgrade script for 0.9 to 10.0 * [HIVE-3932] - Hive release tarballs don't contain PostgreSQL metastore scripts * [HIVE-3943] - Skewed query fails if hdfs path has special characters * [HIVE-3947] - MiniMR test remains pending after test completion * [HIVE-3948] - avro_nullable_fields.q is failing in trunk * [HIVE-3977] - Hive 0.10 postgres schema script is broken * [HIVE-3980] - Cleanup after HIVE-3403 * [HIVE-3984] - Maintain a clear separation between Windowing & PTF at the specification level. * [HIVE-3985] - Update new UDAFs introduced for Windowing to work with new Decimal Type * [HIVE-3986] - Fix select expr processing in PTF Operator * [HIVE-3987] - Update PTF invocation and windowing grammar * [HIVE-3992] - Hive RCFile::sync(long) does a sub-sequence linear search for sync blocks * [HIVE-3995] - PostgreSQL upgrade scripts are not valid * [HIVE-3998] - Oracle metastore update script will fail when upgrading from 0.9.0 to 0.10.0 * [HIVE-3999] - Mysql metastore upgrade script will end up with different schema than the full schema load * [HIVE-4000] - Hive client goes into infinite loop at 100% cpu * [HIVE-4004] - Incorrect status for AddPartition metastore event if RawStore commit fails * [HIVE-4018] - MapJoin failing with Distributed Cache error * [HIVE-4021] - PostgreSQL upgrade scripts are creating column with incorrect name * [HIVE-4024] - Derby metastore update script will fail when upgrading from 0.9.0 to 0.10.0 * [HIVE-4027] - Thrift alter_table api doesnt validate column type * [HIVE-4028] - Bring paranthesis handling in windowing specification in compliance with sql standard * [HIVE-4029] - Hive Profiler dies with NPE * [HIVE-4030] - Name windowing function in consistence with sql standard * [HIVE-4033] - NPE at runtime while selecting virtual column after joining three tables on different keys * [HIVE-4034] - Should be able to specify windowing spec without needing Between * [HIVE-4035] - Column Pruner for PTF Op * [HIVE-4036] - remove use of FunctionRegistry during PTF Op initialization * [HIVE-4039] - Hive compiler sometimes fails in semantic analysis / optimisation stage when boolean variable appears in WHERE clause. * [HIVE-4040] - fix ptf negative tests * [HIVE-4041] - Support multiple partitionings in a single Query * [HIVE-4052] - Disallow partition/sort and distribute/order combinations in windowing and partitioning spec * [HIVE-4056] - Extend rcfilecat to support (un)compressed size and no. of row * [HIVE-4067] - Followup to HIVE-701: reduce ambiguity in grammar * [HIVE-4071] - Map-join outer join produces incorrect results. * [HIVE-4072] - Hive eclipse build path update for string template jar * [HIVE-4073] - Make partition by optional in over clause * [HIVE-4077] - alterPartition and alterPartitions methods in ObjectStore swallow exceptions * [HIVE-4078] - Delay the serialize-deserialize pair in CommonJoinTaskDispatcher * [HIVE-4079] - Altering a view partition fails with NPE * [HIVE-4080] - Add Lead & Lag UDAFs * [HIVE-4081] - allow expressions with over clause * [HIVE-4082] - Break up ptf tests in PTF, Windowing and Lead/Lag tests * [HIVE-4083] - PTF ColumnPruner doesn't account for Partition & Order expressions * [HIVE-4084] - Generated aliases for windowing expressions is broken * [HIVE-4090] - Use of hive.exec.script.allow.partial.consumption can produce partial results * [HIVE-4092] - Store complete names of tables in column access analyzer * [HIVE-4093] - Remove sprintf from PTFTranslator and use String.format() * [HIVE-4094] - decimal_3.q & decimal_serde.q fail on hadoop 2 * [HIVE-4096] - problem in hive.map.groupby.sorted with distincts * [HIVE-4097] - ORC file doesn't properly interpret empty hive.io.file.readcolumn.ids * [HIVE-4098] - OrcInputFormat assumes Hive always calls createValue * [HIVE-4103] - Remove System.gc() call from the map-join local-task loop * [HIVE-4104] - Hive localtask does not buffer disk-writes or reads * [HIVE-4105] - Hive MapJoinOperator unnecessarily deserializes values for all join-keys * [HIVE-4107] - Update Hive 0.10.0 RELEASE_NOTES.txt * [HIVE-4108] - Allow over() clause to contain an order by with no partition by * [HIVE-4109] - Partition by column does not have to be in order by * [HIVE-4111] - Default value in lag is not handled correctly * [HIVE-4112] - Window range specification should be more flexible * [HIVE-4119] - ANALYZE TABLE ... COMPUTE STATISTICS FOR COLUMNS fails with NPE if the table is empty * [HIVE-4122] - Queries fail if timestamp data not in expected format * [HIVE-4126] - remove support for lead/lag UDFs outside of UDAF args * [HIVE-4130] - Bring the Lead/Lag UDFs interface in line with Lead/Lag UDAFs * [HIVE-4131] - Fix eclipse template classpath to include new packages added by ORC file patch * [HIVE-4138] - ORC's union object inspector returns a type name that isn't parseable by TypeInfoUtils * [HIVE-4139] - MiniDFS shim does not work for hadoop 2 * [HIVE-4140] - Specifying alias for windowing function * [HIVE-4142] - Remove inferring partition specification behavior * [HIVE-4143] - Incorrect column mappings with over clause * [HIVE-4146] - bug with hive.auto.convert.join.noconditionaltask with outer joins * [HIVE-4148] - Cleanup aisle "ivy" * [HIVE-4149] - wrong results big outer joins with array of ints * [HIVE-4151] - HiveProfiler NPE with ScriptOperator * [HIVE-4154] - NPE reading column of empty string from ORC file * [HIVE-4156] - need to add protobuf classes to hive-exec.jar * [HIVE-4159] - RetryingHMSHandler doesn't retry in enough cases * [HIVE-4167] - Hive converts bucket map join to SMB join even when tables are not sorted * [HIVE-4169] - union_remove_*.q fail on hadoop 2 * [HIVE-4170] - [REGRESSION] FsShell.close closes filesystem, removing temporary directories * [HIVE-4174] - Round UDF converts BigInts to double * [HIVE-4178] - ORC fails with files with different numbers of columns * [HIVE-4179] - NonBlockingOpDeDup does not merge SEL operators correctly * [HIVE-4180] - Filter getting dropped with PTFOperator * [HIVE-4182] - doAS does not work with HiveServer2 in non-kerberos mode with local job * [HIVE-4184] - Document HiveServer2 setup under the admin documentation on hive wiki * [HIVE-4185] - Document HiveServer2 JDBC and Beeline CLI in the user documentation * [HIVE-4186] - NPE in ReduceSinkDeDuplication * [HIVE-4187] - QL build-grammar target fails after HIVE-4148 * [HIVE-4188] - TestJdbcDriver2.testDescribeTable failing consistently * [HIVE-4189] - ORC fails with String column that ends in lots of nulls * [HIVE-4190] - OVER clauses with ORDER BY not getting windowing set properly * [HIVE-4191] - describe table output always prints as if formatted keyword is specified * [HIVE-4197] - Bring windowing support inline with SQL Standard * [HIVE-4202] - reuse Partition objects in PTFOperator processing * [HIVE-4208] - Clientpositive test parenthesis_star_by is non-deteministic * [HIVE-4217] - Fix show_create_table_*.q test failures * [HIVE-4219] - explain dependency does not capture the input table * [HIVE-4235] - CREATE TABLE IF NOT EXISTS uses inefficient way to check if table exists * [HIVE-4252] - hiveserver2 string representation of complex types are inconsistent with cli * [HIVE-4254] - Code cleanup : debug methods, having clause associated with Windowing * [HIVE-4255] - update show_functions.q.out for functions added for windowing * [HIVE-4259] - SEL operator created with missing columnExprMap for unions * [HIVE-4260] - union_remove_12, union_remove_13 are failing on hadoop2 * [HIVE-4261] - union_remove_10 is failing on hadoop2 with assertion (root task with non-empty set of parents) * [HIVE-4262] - fix last_value UDAF behavior * [HIVE-4269] - fix handling of binary type in hiveserver2, jdbc driver * [HIVE-4270] - bug in hive.map.groupby.sorted in the presence of multiple input partitions * [HIVE-4271] - Limit precision of decimal type * [HIVE-4272] - partition wise metadata does not work for text files * [HIVE-4275] - Hive does not differentiate scheme and authority in file uris * [HIVE-4280] - TestRetryingHMSHandler is failing on trunk. * [HIVE-4288] - Add IntelliJ project files files to .gitignore * [HIVE-4289] - HCatalog build fails when behind a firewall * [HIVE-4292] - hiveserver2 should support -hiveconf commandline parameter * [HIVE-4296] - ant thriftif fails on hcatalog * [HIVE-4302] - Fix how RowSchema and RowResolver are set on ReduceSinkOp that precedes PTFOp * [HIVE-4303] - 2 empty java files in hcatalog * [HIVE-4308] - Newly added test TestCliDriver.hiveprofiler_union0 is failing on trunk * [HIVE-4311] - DOS line endings in auto_join26.q * [HIVE-4315] - enable doAs in unsecure mode for hive server2, when MR job runs locally * [HIVE-4318] - OperatorHooks hit performance even when not used * [HIVE-4319] - Revert changes checked-in as part of HIVE-1953 * [HIVE-4320] - Consider extending max limit for precision to 38 * [HIVE-4323] - sqlline dependency is not required * [HIVE-4327] - NPE in constant folding with decimal * [HIVE-4332] - orc*.q tests fail on hadoop 2 * [HIVE-4333] - most windowing tests fail on hadoop 2 * [HIVE-4334] - ctas test on hadoop 2 has outdated golden file * [HIVE-4335] - serde_regex test fails on hadoop 2 * [HIVE-4336] - Selecting from a view, and another view that also selects from that view fails * [HIVE-4342] - NPE for query involving UNION ALL with nested JOIN and UNION ALL * [HIVE-4352] - Guava not getting included in build package * [HIVE-4356] - remove duplicate impersonation parameters for hiveserver2 * [HIVE-4358] - Check for Map side processing in PTFOp is no longer valid * [HIVE-4365] - wrong result in left semi join * [HIVE-4371] - some issue with merging join trees * [HIVE-4373] - Hive Version returned by HiveDatabaseMetaData.getDatabaseProductVersion is incorrect * [HIVE-4378] - Counters hit performance even when not used * [HIVE-4387] - ant maven-build fails because hcatalog doesn't have a make-pom target * [HIVE-4394] - test leadlag.q fails * [HIVE-4398] - HS2 Resource leak: operation handles not cleaned when originating session is closed * [HIVE-4407] - TestHCatStorer.testStoreFuncAllSimpleTypes fails because of null case difference * [HIVE-4412] - PTFDesc tries serialize transient fields like OIs, etc. * [HIVE-4419] - webhcat - support ${WEBHCAT_PREFIX}/conf/ as config directory * [HIVE-4420] - HCatalog unit tests stop after a failure * [HIVE-4421] - Improve memory usage by ORC dictionaries * [HIVE-4447] - hcatalog version numbers need to be updated * [HIVE-4455] - HCatalog build directories get included in tar file produced by "ant tar" * [HIVE-4461] - hcatalog jars not getting published to maven repo * [HIVE-4494] - ORC map columns get class cast exception in some context * [HIVE-4498] - TestBeeLineWithArgs.testPositiveScriptFile fails * [HIVE-4500] - HS2 holding too many file handles of hive_job_log_hive_*.txt files * [HIVE-4505] - Hive can't load transforms added using 'ADD FILE' * [HIVE-4527] - Fix eclipse project template ** Improvement * [HIVE-581] - improve group by syntax * [HIVE-948] - more query plan optimization rules * [HIVE-1953] - Hive should process comments in CliDriver * [HIVE-2439] - Upgrade antlr version to 3.4 * [HIVE-2477] - Use name of original expression for name of CAST output * [HIVE-3004] - RegexSerDe should support other column types in addition to STRING * [HIVE-3231] - msck repair should find partitions already containing data files * [HIVE-3252] - Add environment context to metastore Thrift calls * [HIVE-3401] - Diversify grammar for split sampling * [HIVE-3431] - Avoid race conditions while downloading resources from non-local filesystem * [HIVE-3492] - Provide ALTER for partition changing bucket number * [HIVE-3527] - Allow CREATE TABLE LIKE command to take TBLPROPERTIES * [HIVE-3531] - Simple lock manager for dedicated hive server * [HIVE-3564] - hivetest.py: revision number and applied patch * [HIVE-3628] - Provide a way to use counters in Hive through UDF * [HIVE-3633] - sort-merge join does not work with sub-queries * [HIVE-3672] - Support altering partition column type in Hive * [HIVE-3708] - Add mapreduce workflow information to job configuration * [HIVE-3709] - Stop storing default ConfVars in temp file * [HIVE-3710] - HiveConf.ConfVars.HIVE_STATS_COLLECT_RAWDATASIZE should not be checked in FileSinkOperator * [HIVE-3762] - Minor fix for 'tableName' in Hive.g * [HIVE-3784] - de-emphasize mapjoin hint * [HIVE-3793] - Print number of fetched rows after query in CliDriver * [HIVE-3796] - Multi-insert involving bucketed/sorted table turns off merging on all outputs * [HIVE-3799] - Better error message if metalisteners or hookContext cannot be loaded/instantiated * [HIVE-3805] - Resolve TODO in TUGIBasedProcessor * [HIVE-3833] - object inspectors should be initialized based on partition metadata * [HIVE-3853] - UDF unix_timestamp is deterministic if an argument is given, but it treated as non-deterministic preventing PPD * [HIVE-3874] - Create a new Optimized Row Columnar file format for Hive * [HIVE-3884] - Better align columns in DESCRIBE table_name output to make more human-readable * [HIVE-3904] - Replace hashmaps in JoinOperators to array * [HIVE-3917] - Support noscan operation for analyze command * [HIVE-3950] - Remove code for merging files via MR job * [HIVE-3952] - merge map-job followed by map-reduce job * [HIVE-3958] - support partial scan for analyze command - RCFile * [HIVE-3970] - Clean up/fix PartitionNameWhitelistPreEventListener * [HIVE-3996] - Correctly enforce the memory limit on the multi-table map-join * [HIVE-4001] - Add o.a.h.h.serde.Constants for backward compatibility * [HIVE-4007] - Create abstract classes for serializer and deserializer * [HIVE-4015] - Add ORC file to the grammar as a file format * [HIVE-4016] - Remove init(fname) from TestParse.vm for each test * [HIVE-4020] - Swap applying order of CP and PPD * [HIVE-4023] - Improve Error Logging in MetaStore * [HIVE-4025] - Add reflect UDF for member method invocation of column * [HIVE-4042] - ignore mapjoin hint * [HIVE-4045] - Modify PreDropPartitionEvent to pass Table parameter * [HIVE-4076] - Refactor code for finding windowing expressions * [HIVE-4125] - Expose metastore JMX metrics * [HIVE-4128] - Support avg(decimal) * [HIVE-4129] - Window handling dumps debug info on console, instead should use logger. * [HIVE-4157] - ORC runs out of heap when writing * [HIVE-4206] - Sort merge join does not work for outer joins for 7 inputs * [HIVE-4212] - sort merge join should work for outer joins for more than 8 inputs * [HIVE-4240] - optimize hive.enforce.bucketing and hive.enforce sorting insert * [HIVE-4258] - Log logical plan tree for debugging * [HIVE-4281] - add hive.map.groupby.sorted.testmode * [HIVE-4304] - Remove unused builtins and pdk submodules * [HIVE-4306] - PTFDeserializer should reconstruct OIs based on InputOI passed to PTFOperator * [HIVE-4328] - Change default bigtable selection policy for sort-merge joins ** New Feature * [HIVE-446] - Implement TRUNCATE * [HIVE-701] - lots of reserved keywords in hive * [HIVE-896] - Add LEAD/LAG/FIRST/LAST analytical windowing functions to Hive. * [HIVE-933] - Infer bucketing/sorting properties * [HIVE-2288] - Adding the oracle nvl function to the UDF * [HIVE-2691] - Specify location of log4j configuration files via configuration properties * [HIVE-2693] - Add DECIMAL data type * [HIVE-2935] - Implement HiveServer2 * [HIVE-3073] - Hive List Bucketing - DML support * [HIVE-3552] - HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys * [HIVE-3646] - Add 'IGNORE PROTECTION' predicate for dropping partitions * [HIVE-3682] - when output hive table to file,users should could have a separator of their own choice * [HIVE-3825] - Add Operator level Hooks * [HIVE-3834] - Support ALTER VIEW AS SELECT in Hive * [HIVE-3897] - Add a way to get the uncompressed/compressed sizes of columns from an RC File * [HIVE-3898] - getReducersBucketing in SemanticAnalyzer may return more than the max number of reducers * [HIVE-3903] - Allow updating bucketing/sorting metadata of a partition through the CLI * [HIVE-3937] - Hive Profiler * [HIVE-3951] - Allow Decimal type columns in Regex Serde * [HIVE-4019] - Ability to create and drop temporary partition function * [HIVE-4048] - Allow partition by/order by in partitioning spec in over clause and partition function * [HIVE-4120] - Implement decimal encoding for ORC * [HIVE-4127] - Testing with Hadoop 2.x causes test failure for ORC's TestFileDump * [HIVE-4155] - Expose ORC's FileDump as a service * [HIVE-4248] - Implement a memory manager for ORC ** Task * [HIVE-3679] - Unescape partition names returned by show partitions * [HIVE-3718] - Add check to determine whether partition can be dropped at Semantic Analysis time * [HIVE-3721] - ALTER TABLE ADD PARTS should check for valid partition spec and throw a SemanticException if part spec is not valid * [HIVE-3838] - Add input table name to MetaStoreEndFunctionContext for logging purposes * [HIVE-3940] - Track columns accessed in each table in a query * [HIVE-3981] - Split up tests in ptf_general_queries.q * [HIVE-3982] - Merge PTFDesc and PTFDef classes * [HIVE-4031] - Add apache headers in new files * [HIVE-4145] - Create hcatalog stub directory and add it to the build ** Test * [HIVE-3571] - add a way to run a small unit quickly * [HIVE-3842] - Remove redundant test codes * [HIVE-3944] - Make accept qfile argument for miniMR tests * [HIVE-3956] - TestMetaStoreAuthorization always uses the same port * [HIVE-4124] - Add more tests for windowing * [HIVE-4298] - add tests for distincts for hive.map.groutp.sorted * [HIVE-4337] - Update list bucketing test results ** Wish * [HIVE-4314] - Result of mapjoin_test_outer.q is not deterministic Release Notes - Hive - Version 0.10.0 ** Sub-task * [HIVE-1362] - Optimizer statistics on columns in tables and partitions * [HIVE-3146] - Support external hive tables whose data are stored in Azure blob store/Azure Storage Volumes (ASV) * [HIVE-3172] - Remove the duplicate JAR entries from the (“test.classpath”) to avoid command line exceeding char limit on windows * [HIVE-3204] - Windows: Fix the unit tests which contains “!” commands (Unix shell commands) * [HIVE-3207] - FileUtils.tar does not close input files * [HIVE-3317] - Fix “TestDosToUnix” unit tests on Windows by closing the leaking file handle in DosToUnix.java. * [HIVE-3319] - Fix the “TestHiveHistory”, “TestHiveConf”, & “TestExecDriver” unit tests on Windows by fixing the path related issues. * [HIVE-3320] - Handle “CRLF” line endings to avoid the extra spacing in generated test outputs in Windows. (Utilities.Java :: readColumn) * [HIVE-3327] - Remove the Unix specific absolute path of “Cat” utility in several .q files to make them run on Windows with CygWin in path. * [HIVE-3397] - PartitionPruner should log why it is not pushing the filter down to JDO ** Bug * [HIVE-1367] - cluster by multiple columns does not work if parenthesis is present * [HIVE-1399] - Nested UDAFs cause Hive Internal Error (NullPointerException) * [HIVE-1977] - DESCRIBE TABLE syntax doesn't support specifying a database qualified table name * [HIVE-2101] - mapjoin sometimes gives wrong results if there is a filter in the on condition * [HIVE-2372] - java.io.IOException: error=7, Argument list too long * [HIVE-2498] - Group by operator does not estimate size of Timestamp & Binary data correctly * [HIVE-2540] - LATERAL VIEW with EXPLODE produces ConcurrentModificationException * [HIVE-2542] - DROP DATABASE CASCADE does not drop non-native tables. * [HIVE-2544] - Nullpointer on registering udfs. * [HIVE-2646] - Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs * [HIVE-2711] - Make the header of RCFile unique * [HIVE-2715] - Upgrade Thrift dependency to 0.9.0 * [HIVE-2721] - ability to select a view qualified by the database / schema name * [HIVE-2732] - Reduce Sink deduplication fails if the child reduce sink is followed by a join * [HIVE-2736] - Hive UDFs cannot emit binary constants * [HIVE-2757] - hive can't find hadoop executor scripts without HADOOP_HOME set * [HIVE-2788] - When integrating into MapReduce2, Hive is unable to handle corrupt rcfile archive * [HIVE-2789] - query_properties.q contains non-deterministic queries * [HIVE-2800] - NPE in "create index" without comment clause in external metastore * [HIVE-2803] - utc_from_timestamp and utc_to_timestamp returns incorrect results. * [HIVE-2804] - Task log retrieval fails on Hadoop 0.23 * [HIVE-2860] - TestNegativeCliDriver autolocal1.q fails on 0.23 * [HIVE-2874] - Renaming external partition changes location * [HIVE-2904] - ant gen-test failed * [HIVE-2907] - Hive error when dropping a table with large number of partitions * [HIVE-2918] - Hive Dynamic Partition Insert - move task not considering 'hive.exec.max.dynamic.partitions' from CLI * [HIVE-2929] - race condition in DAG execute tasks for hive * [HIVE-2933] - analyze command throw NPE when table doesn't exists * [HIVE-2941] - Hive should expand nested structs when setting the table schema from thrift structs * [HIVE-2942] - substr on string containing UTF-8 characters produces StringIndexOutOfBoundsException * [HIVE-2955] - Queries consists of metadata-only-query returns always empty value * [HIVE-2957] - Hive JDBC doesn't support TIMESTAMP column * [HIVE-2963] - metastore delegation token is not getting used by hive commandline * [HIVE-2971] - GET_JSON_OBJECT fails on some valid JSON keys * [HIVE-2975] - Filter parsing does not recognize '!=' as operator and silently ignores invalid tokens * [HIVE-2976] - Fix maven-build Ant target * [HIVE-2984] - Fix test failure in TestNegativeCliDriver.dyn_part_max caused by HIVE-2918 * [HIVE-2990] - Remove hadoop-source Ivy resolvers and Ant targets * [HIVE-2999] - Offline build is not working * [HIVE-3000] - Potential infinite loop / log spew in ZookeeperHiveLockManager * [HIVE-3008] - Memory leak in TUGIContainingTransport * [HIVE-3013] - TestCliDriver cannot be debugged with eclipse since hadoop_home is set incorrectly * [HIVE-3014] - Fix metastore test failures caused by HIVE-2757 * [HIVE-3019] - Add JUnit to list of test dependencies managed by Ivy * [HIVE-3021] - Tests failing for me * [HIVE-3028] - Fix javadoc again * [HIVE-3029] - Update ShimLoader to work with Hadoop 2.x * [HIVE-3030] - escape more chars for script operator * [HIVE-3031] - hive docs target does not work * [HIVE-3035] - Modify clean target to remove ~/.ivy2/local/org.apache.hive ~/.ivy2/cache/org.apache.hive * [HIVE-3045] - Partition column values are not valid if any of virtual columns is selected * [HIVE-3049] - setup classpath for templates correctly for eclipse * [HIVE-3052] - TestHadoop20SAuthBridge always uses the same port * [HIVE-3057] - metastore.HiveMetaStore$HMSHandler should set the thread local raw store to null in shutdown() * [HIVE-3058] - hive.transform.escape.input breaks tab delimited data * [HIVE-3059] - revert HIVE-2703 * [HIVE-3062] - Insert into table overwrites existing table if table name contains uppercase character * [HIVE-3063] - drop partition for non-string columns is failing * [HIVE-3069] - Drop partition problem * [HIVE-3070] - Filter on outer join condition removed while merging join tree * [HIVE-3076] - drop partition does not work for non-partition columns * [HIVE-3079] - Revert HIVE-2989 * [HIVE-3081] - ROFL Moment. Numberator and denaminator typos * [HIVE-3082] - Oracle Metastore schema script doesn't include DDL for DN internal tables * [HIVE-3085] - make parallel tests work * [HIVE-3090] - Timestamp type values not having nano-second part breaks row * [HIVE-3092] - Hive tests should load Hive classes from build directory, not Ivy cache * [HIVE-3098] - Memory leak from large number of FileSystem instances in FileSystem.CACHE * [HIVE-3100] - Add HiveCLI that runs over JDBC * [HIVE-3101] - dropTable will all ways excute hook.rollbackDropTable whether drop table success or faild. * [HIVE-3112] - clear hive.metastore.partition.inherit.table.properties till HIVE-3109 is fixed * [HIVE-3120] - make copyLocal work for parallel tests * [HIVE-3123] - Hadoop20Shim. CombineFileRecordReader does not report progress within files * [HIVE-3124] - Error in Removing ProtectMode from a Table * [HIVE-3125] - sort_array doesn't work with LazyPrimitive * [HIVE-3126] - Generate & build the velocity based Hive tests on windows by fixing the path issues * [HIVE-3127] - Pass hconf values as XML instead of command line arguments to child JVM * [HIVE-3128] - use commons-compress instead of forking tar process * [HIVE-3134] - Drop table/index/database can result in orphaned locations * [HIVE-3135] - add an option in ptest to run on a single machine * [HIVE-3140] - Comment indenting is broken for "describe" in CLI * [HIVE-3142] - Bug in parallel test for singlehost flag * [HIVE-3149] - Dynamically generated paritions deleted by Block level merge * [HIVE-3161] - drop the temporary function at end of autogen_colalias.q * [HIVE-3164] - Fix non-deterministic testcases failures when running Hive0.9.0 on MapReduce2 * [HIVE-3165] - Hive thrift code doesnt generate quality hashCode() * [HIVE-3168] - LazyBinaryObjectInspector.getPrimitiveJavaObject copies beyond length of underlying BytesWritable * [HIVE-3171] - Bucketed sort merge join doesn't work when multiple files exist for small alias * [HIVE-3178] - retry not honored in RetryingRawMetastore * [HIVE-3180] - Fix Eclipse classpath template broken in HIVE-3128 * [HIVE-3203] - Drop partition throws NPE if table doesn't exist * [HIVE-3205] - Bucketed mapjoin on partitioned table which has no partition throws NPE * [HIVE-3206] - FileUtils.tar assumes wrong directory in some cases * [HIVE-3215] - JobDebugger should use RunningJob.getTrackingURL * [HIVE-3218] - Stream table of SMBJoin/BucketMapJoin with two or more partitions is not handled properly * [HIVE-3221] - HiveConf.getPositionFromInternalName does not support more than sinle digit column numbers * [HIVE-3225] - NPE on a join query with authorization enabled * [HIVE-3226] - ColumnPruner is not working on LateralView * [HIVE-3230] - Make logging of plan progress in HadoopJobExecHelper configurable * [HIVE-3232] - Resource Leak: Fix the File handle leak in EximUtil.java * [HIVE-3240] - Fix non-deterministic results in newline.q and timestamp_lazy.q * [HIVE-3242] - Fix cascade_dbdrop.q when building hive on hadoop0.23 * [HIVE-3243] - ignore white space between entries of hive/hbase table mapping * [HIVE-3246] - java primitive type for binary datatype should be byte[] * [HIVE-3247] - Sorted by order of table not respected * [HIVE-3248] - lack of semi-colon in .q file leads to missing the next statement * [HIVE-3249] - Upgrade guava to 11.0.2 * [HIVE-3251] - Hive doesn't remove scrach directories while killing running MR job * [HIVE-3257] - Fix avro_joins.q testcase failure when building hive on hadoop0.23 * [HIVE-3261] - alter the number of buckets for a non-empty partitioned table should not be allowed * [HIVE-3262] - bucketed mapjoin silently ignores mapjoin hint * [HIVE-3265] - HiveHistory.printRowCount() throws NPE * [HIVE-3267] - escaped columns in cluster/distribute/order/sort by are not working * [HIVE-3268] - expressions in cluster by are not working * [HIVE-3273] - Add avro jars into hive execution classpath * [HIVE-3275] - Fix autolocal1.q testcase failure when building hive on hadoop0.23 MR2 * [HIVE-3276] - optimize union sub-queries * [HIVE-3279] - Table schema not being copied to Partitions with no columns * [HIVE-3282] - Convert runtime exceptions to semantic exceptions for missing partitions/tables in show/describe statements * [HIVE-3283] - bucket information should be used from the partition instead of the table * [HIVE-3289] - sort merge join may not work silently * [HIVE-3291] - fix fs resolvers * [HIVE-3293] - Load file into a table does not update table statistics * [HIVE-3295] - HIVE-3128 introduced bug causing dynamic partitioning to fail * [HIVE-3301] - Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23 * [HIVE-3302] - Race condition in query plan for merging at the end of a query * [HIVE-3303] - Fix error code inconsistency bug in mapreduce_stack_trace.q and mapreduce_stack_trace_turnoff.q when running hive on hadoop23 * [HIVE-3306] - SMBJoin/BucketMapJoin should be allowed only when join key expression is exactly matches with sort/cluster key * [HIVE-3310] - [Regression] TestMTQueries test is failing on trunk * [HIVE-3311] - Convert runtime exceptions to semantic exceptions for validation of alter table commands * [HIVE-3338] - Archives broken for hadoop 1.0 * [HIVE-3339] - Change the rules in SemanticAnalyzer to use Operator.getName() instead of hardcoded names * [HIVE-3340] - shims unit test failures fails further test progress * [HIVE-3341] - Making hive tests run against different MR versions * [HIVE-3343] - Hive: Query misaligned result for Group by followed by Join with filter and skip a group-by result * [HIVE-3345] - Add junit exclude utility to disable testcases * [HIVE-3365] - Upgrade Hive's Avro dependency to version 1.7 * [HIVE-3375] - bucketed map join should check that the number of files match the number of buckets * [HIVE-3379] - stats are not being collected correctly for analyze table with dynamic partitions * [HIVE-3385] - fpair on creating external table * [HIVE-3443] - Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key * [HIVE-3448] - GenMRSkewJoinProcessor uses File.Separator instead of Path.Separator * [HIVE-3451] - map-reduce jobs does not work for a partition containing sub-directories * [HIVE-3452] - Missing column causes null pointer exception * [HIVE-3458] - Parallel test script doesnt run all tests * [HIVE-3459] - Dynamic partition queries producing no partitions fail with hive.stats.reliable=true * [HIVE-3461] - hive unit tests fail to get lock using zookeeper on windows * [HIVE-3465] - insert into statement overwrites if target table is prefixed with database name * [HIVE-3477] - Duplicate data possible with speculative execution for dynamic partitions * [HIVE-3478] - Remove the specialized logic to handle the file schemas in windows vs unix from build.xml * [HIVE-3479] - Bug fix: Return the child JVM exit code to the parent process to handle the error conditions * [HIVE-3480] - : Fix the file handle leaks in Symbolic & Symlink related input formats. * [HIVE-3481] - : Hiveserver is not closing the existing driver handle before executing the next command. It results in to file handle leaks. * [HIVE-3483] - joins using partitioned table give incorrect results on windows * [HIVE-3484] - RetryingRawStore logic needs to be significantly reworked to support retries within transactions * [HIVE-3485] - Hive List Bucketing - Skewed DDL doesn't support skewed value with string quote * [HIVE-3486] - CTAS in database with location on non-default name node fails * [HIVE-3487] - Some of the Metastore unit tests failing on Windows because of the static variables initialization problem in HiveConf class. * [HIVE-3493] - aggName of SemanticAnalyzer.getGenericUDAFEvaluator is generated in two different ways * [HIVE-3494] - Some of the JDBC test cases are failing on Windows because of the longer class path. * [HIVE-3495] - For UDAFs, when generating a plan without map-side-aggregation, constant agg parameters will be replaced by ExprNodeColumnDesc * [HIVE-3496] - Query plan for multi-join where the third table joined is a subquery containing a map-only union with hive.auto.convert.join=true is wrong * [HIVE-3497] - Avoid NPE in skewed information read * [HIVE-3498] - hivetest.py fails with --revision option * [HIVE-3505] - log4j template has logging threshold that hides all audit logs * [HIVE-3507] - Some of the tests are not deterministic * [HIVE-3515] - metadata_export_drop.q causes failure of other tests * [HIVE-3518] - QTestUtil side-effects * [HIVE-3519] - partition to directory comparison in CombineHiveInputFormat needs to accept partitions dir without scheme * [HIVE-3520] - ivysettings.xml does not let you override .m2/repository * [HIVE-3522] - Make separator for Entity name configurable * [HIVE-3523] - Hive info logging is broken * [HIVE-3525] - Avro Maps with Nullable Values fail with NPE * [HIVE-3529] - Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table * [HIVE-3533] - ZooKeeperHiveLockManager does not respect the option to keep locks alive even after the current session has closed * [HIVE-3535] - derby metastore upgrade script throw errors when updating from 0.7 to 0.8 * [HIVE-3536] - Output of sort merge join is no longer bucketed * [HIVE-3544] - union involving double column with a map join subquery will fail or give wrong results * [HIVE-3556] - Test "Path -> Alias" for explain extended * [HIVE-3560] - Hive always prints a warning message when using remote metastore * [HIVE-3563] - Drop database cascade fails when there are indexes on any tables * [HIVE-3581] - get_json_object and json_tuple return null in the presence of new line characters * [HIVE-3596] - Regression - HiveConf static variable causes issues in long running JVM insname of some metastore scripts are not per convention * [HIVE-3712] - Use varbinary instead of longvarbinary to store min and max column values in column stats schema * [HIVE-3713] - Metastore: Sporadic unit test failures * [HIVE-3722] - Create index fails on CLI using remote metastore * [HIVE-3723] - Hive Driver leaks ZooKeeper connections * [HIVE-3724] - Metastore tests use hardcoded ports * [HIVE-3729] - Error in groupSetExpression rule in Hive grammar * [HIVE-3732] - Multiple aggregates in query fail the job * [HIVE-3735] - PTest doesn't work due to hive snapshot version upgrade to 11 * [HIVE-3736] - hive unit test case build failure. * [HIVE-3742] - The derby metastore schema script for 0.10.0 doesn't run * [HIVE-3769] - Must publish new Hive-0.10 artifacts to apache repository. * [HIVE-3780] - RetryingMetaStoreClient Should Log the Caught Exception * [HIVE-3792] - hive pom file has missing conf and scope mapping for compile configuration. * [HIVE-3794] - Oracle upgrade script for Hive is broken * [HIVE-3814] - Cannot drop partitions on table when using Oracle metastore * [HIVE-3975] - Hive JIRA still shows 0.10 as unreleased in "Affects Version/s" dropdown * [HIVE-3978] - HIVE_AUX_JARS_PATH should have : instead of , as separator since it gets appended to HADOOP_CLASSPATH * [HIVE-3989] - TestCase TestMTQueries fails with Non-Sun Java * [HIVE-4074] - Doc update for .8, .9 and .10 * [HIVE-4166] - closeAllForUGI causes failure in hiveserver2 when fetching large amount of data ** Improvement * [HIVE-1653] - Ability to enforce correct stats * [HIVE-2021] - Add a configuration property that sets the variable substitution max depth * [HIVE-2529] - metastore 0.8 upgrade script for PostgreSQL * [HIVE-2585] - Collapse hive.metastore.uris and hive.metastore.local * [HIVE-2796] - Support auto completion for hive configs in CliDriver * [HIVE-2848] - Add validation to HiveConf ConfVars * [HIVE-2910] - Improve the HWI interface * [HIVE-2911] - Move global .hiverc file * [HIVE-2925] - Support non-MR fetching for simple queries with select/limit/filter operations only * [HIVE-2956] - [hive] Provide error message when using UDAF in the place of UDF instead of throwing NPE * [HIVE-2994] - pass a environment context to metastore thrift APIs * [HIVE-3012] - hive custom scripts do not work well if the data contains new lines * [HIVE-3018] - Make the new header for RC Files introduced in HIVE-2711 optional * [HIVE-3048] - Collect_set Aggregate does uneccesary check for value. * [HIVE-3051] - JDBC cannot find metadata for tables/columns containing uppercase character * [HIVE-3075] - Improve HiveMetaStore logging * [HIVE-3099] - add findbugs in build.xml * [HIVE-3106] - Add option to make multi inserts more atomic * [HIVE-3153] - Release codecs and output streams between flushes of RCFile * [HIVE-3195] - Typo in dynamic partitioning code bits, says "genereated" instead of "generated" in some places. * [HIVE-3202] - Add hive command for resetting hive confs * [HIVE-3210] - Support Bucketed mapjoin on partitioned table which has two or more partitions * [HIVE-3219] - BucketizedHiveInputFormat should be automatically used with SMBJoin * [HIVE-3234] - getting the reporter in the recordwriter * [HIVE-3277] - Enable Metastore audit logging for non-secure connections * [HIVE-3315] - Propagates filters which are on the join condition transitively * [HIVE-3323] - enum to string conversions * [HIVE-3337] - Create Table Like should copy configured Table Parameters * [HIVE-3380] - As a follow up for HIVE-3276, optimize union for dynamic partition queries * [HIVE-3391] - Keep the original query in HiveDriverRunHookContextImpl * [HIVE-3393] - get_json_object and json_tuple should use Jackson library * [HIVE-3395] - 0.23 compatibility: shim job.tracker.address * [HIVE-3400] - Add Retries to Hive MetaStore Connections * [HIVE-3406] - Yet better error message in CLI on invalid column name * [HIVE-3410] - All operators's conf should inherit from a common class * [HIVE-3422] - Support partial partition specifications in when enabling/disabling protections in Hive * [HIVE-3432] - perform a map-only group by if grouping key matches the sorting properties of the table * [HIVE-3447] - Provide backward compatibility for AvroSerDe properties * [HIVE-3450] - Hive maven-publish ant task should be configurable * [HIVE-3500] - To add instrumentation to capture if there is skew in reducers * [HIVE-3512] - Log client IP address with command in metastore's startFunction method * [HIVE-3513] - Allow Partition Offline Enable/Disable command to be specified at the ds level even when Partition is based on more columns than ds * [HIVE-3514] - Refactor Partition Pruner so that logic can be reused. * [HIVE-3524] - Storing certain Exception objects thrown in HiveMetaStore.java in MetaStoreEndFunctionContext * [HIVE-3550] - Early skipping for limit operator at reduce stage * [HIVE-3557] - Access to external URLs in hivetest.py * [HIVE-3570] - Add/fix facility to collect operator specific statisticsin hive + add hash-in/hash-out counter for GroupBy Optr * [HIVE-3573] - Revert HIVE-3268 * [HIVE-3590] - TCP KeepAlive and connection timeout for the HiveServer * [HIVE-3621] - Make prompt in Hive CLI configurable * [HIVE-3623] - Reset operator-id before executing parse tests * [HIVE-3626] - RetryingHMSHandler should wrap JDOException inside MetaException * [HIVE-3636] - Catch the NPe when using ^D to exit from CLI * [HIVE-3706] - getBoolVar in FileSinkOperator can be optimized * [HIVE-3707] - Round map/reduce progress down when it is in the range [99.5, 100) ** New Feature * [HIVE-887] - Allow SELECT without a mapreduce job * [HIVE-895] - Add SerDe for Avro serialized data * [HIVE-967] - Implement "show create table" * [HIVE-2397] - Support with rollup option for group by * [HIVE-2418] - replace or translate function in hive * [HIVE-2530] - Implement SHOW TBLPROPERTIES * [HIVE-2549] - Support standard cross join syntax * [HIVE-2694] - Add FORMAT UDF * [HIVE-2767] - Optionally use framed transport with metastore * [HIVE-2909] - SHOW COLUMNS table_name; to provide a comma-delimited list of columns. * [HIVE-2928] - Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo) * [HIVE-3001] - Returning Meaningful Error Codes & Messages * [HIVE-3056] - Create a new metastore tool to bulk update location field in Db/Table/Partition records * [HIVE-3066] - Add the option -database DATABASE in hive cli to specify a default database to use for the cli session. * [HIVE-3068] - Add ability to export table metadata as JSON on table drop * [HIVE-3072] - Hive List Bucketing - DDL support * [HIVE-3086] - Skewed Join Optimization * [HIVE-3152] - Disallow certain character patterns in partition names * [HIVE-3238] - A table generating, table generating function * [HIVE-3304] - sort merge join should work if both the tables are sorted in descending order * [HIVE-3433] - Implement CUBE and ROLLUP operators in Hive * [HIVE-3471] - Implement grouping sets in hive * [HIVE-3554] - Hive List Bucketing - Query logic * [HIVE-3610] - Add a command "Explain dependency ..." * [HIVE-3643] - Hive List Bucketing - set hive.mapred.supports.subdirectories * [HIVE-3649] - Hive List Bucketing - enhance DDL to specify list bucketing table * [HIVE-3705] - Adding authorization capability to the metastore * [HIVE-4053] - Add support for phonetic algorithms in Hive ** Task * [HIVE-1719] - Move RegexSerDe out of hive-contrib and over to hive-serde * [HIVE-2940] - RCFileMergeMapper Prints To Standard Output Even In Silent Mode * [HIVE-2979] - Implement INCLUDE_HADOOP_MAJOR_VERSION test macro * [HIVE-3002] - Revert HIVE-2986 * [HIVE-3022] - Add hive.exec.rcfile.use.explicit.header to hive-default.xml.template * [HIVE-3061] - hive.binary.record.max.length is a magic string * [HIVE-3314] - Extract global limit configuration to optimizer * [HIVE-3388] - Improve Performance of UDF PERCENTILE_APPROX() * [HIVE-3501] - Track table and keys used in joins and group bys for logging * [HIVE-3679] - Unescape partition names returned by show partitions * [HIVE-3689] - Update website with info on how to report security bugs ** Test * [HIVE-2937] - TestHiveServerSessions hangs when executed directly * [HIVE-2959] - TestRemoteHiveMetaStoreIpAddress always uses the same port * [HIVE-2960] - Stop testing concat of partitions containing control characters. * [HIVE-3427] - Newly added test testCliDriver_metadata_export_drop is consistently failing on trunk * [HIVE-3438] - Add tests for 'm' bigs tables sortmerge join with 'n' small tables where both m,n>1 * [HIVE-3499] - add tests to use bucketing metadata for partitions * [HIVE-3551] - Add more tests where output of sort merge join is sorted * [HIVE-3749] - New test cases added by HIVE-3676 in insert1.q is not deterministic ** Wish * [HIVE-2969] - Log Time To Submit metric with PerfLogger Release Notes - Hive - Version 0.9.0 ** Sub-task * [HIVE-2433] - add DOAP file for Hive * [HIVE-2600] - Enable/Add type-specific compression for rcfile * [HIVE-2716] - Move retry logic in HiveMetaStore to a separe class * [HIVE-2771] - Add support for filter pushdown for key ranges in hbase for keys of type string ** Bug * [HIVE-727] - Hive Server getSchema() returns wrong schema for "Explain" queries * [HIVE-1444] - "hdfs" is hardcoded in few places in the code which inhibits use of other file systems * [HIVE-1892] - show functions also returns internal operators * [HIVE-2329] - Not using map aggregation, fails to execute group-by after cluster-by with same key * [HIVE-2503] - HiveServer should provide per session configuration * [HIVE-2504] - Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory * [HIVE-2520] - left semi join will duplicate data * [HIVE-2543] - Compact index table's files merged in creation * [HIVE-2616] - Passing user identity from metastore client to server in non-secure mode * [HIVE-2617] - Insert overwrite table db.tname fails if partition already exists * [HIVE-2618] - Describe partition returns table columns but should return partition columns * [HIVE-2629] - Make a single Hive binary work with both 0.20.x and 0.23.0 * [HIVE-2631] - Make Hive work with Hadoop 1.0.0 * [HIVE-2632] - ignore exception for external jars via reflection * [HIVE-2635] - wrong class loader used for external jars * [HIVE-2647] - Force Bash shell on parallel test slave nodes * [HIVE-2648] - Parallel tests fail if master directory is not present * [HIVE-2649] - Allow multiple ptest runs by the same person * [HIVE-2650] - Parallel test commands that include cd fail * [HIVE-2654] - "hive.querylog.location" requires parent directory to be exist or else folder creation fails * [HIVE-2657] - builtins JAR is not being published to Maven repo & hive-cli POM does not depend on it either * [HIVE-2660] - Need better exception handling in RCFile tolerate corruptions mode * [HIVE-2666] - StackOverflowError when using custom UDF in map join * [HIVE-2673] - Eclipse launch configurations fail due to unsatisfied builtins JAR dependency * [HIVE-2674] - get_partitions_ps throws TApplicationException if table doesn't exist * [HIVE-2681] - SUCESS is misspelled * [HIVE-2690] - a bug in 'alter table concatenate' that causes filenames getting double url encoded * [HIVE-2705] - SemanticAnalyzer twice swallows an exception it shouldn't * [HIVE-2706] - StackOverflowError when using custom UDF after adding archive after adding jars * [HIVE-2714] - Lots of special characters are not handled in LIKE * [HIVE-2718] - NPE in union followed by join * [HIVE-2724] - Remove unused lib/log4j-1.2.15.jar * [HIVE-2725] - Fix flaky testing infrastructure * [HIVE-2734] - Fix some nondeterministic test output * [HIVE-2735] - PlanUtils.configureTableJobPropertiesForStorageHandler() is not called for partitioned table * [HIVE-2741] - Single binary built against 0.20 and 0.23, does not work against 0.23 clusters. * [HIVE-2746] - Metastore client doesn't log properly in case of connection failure to server * [HIVE-2749] - CONV returns incorrect results sometimes * [HIVE-2750] - Hive multi group by single reducer optimization causes invalid column reference error * [HIVE-2753] - Remove empty java files * [HIVE-2754] - NPE in union with lateral view * [HIVE-2755] - union follwowed by union_subq does not work if the subquery union has reducers * [HIVE-2758] - Metastore is caching too aggressively * [HIVE-2759] - Change global_limit.q into linux format file * [HIVE-2761] - Remove lib/javaewah-0.3.jar * [HIVE-2762] - Alter Table Partition Concatenate Fails On Certain Characters * [HIVE-2769] - union with a multi-table insert is not working * [HIVE-2772] - make union31.q deterministic * [HIVE-2778] - Fail on table sampling * [HIVE-2782] - New BINARY type produces unexpected results with supported UDFS when using MapReduce2 * [HIVE-2791] - filter is still removed due to regression of HIVE-1538 althougth HIVE-2344 * [HIVE-2792] - SUBSTR(CAST( AS BINARY)) produces unexpected results * [HIVE-2793] - Disable loadpart_err.q on 0.23 * [HIVE-2811] - Export LANG=en_US.UTF-8 to environment while running tests * [HIVE-2824] - typo in configuration parameter * [HIVE-2831] - TestContribCliDriver.dboutput and TestCliDriver.input45 fail on 0.23 * [HIVE-2833] - Fix test failures caused by HIVE-2716 * [HIVE-2837] - insert into external tables should not be allowed * [HIVE-2838] - cleanup readentity/writeentity * [HIVE-2840] - INPUT__FILE__NAME virtual column returns unqualified paths on Hadoop 0.23 * [HIVE-2856] - Fix TestCliDriver escape1.q failure on MR2 * [HIVE-2857] - QTestUtil.cleanUp() fails with FileNotException on 0.23 * [HIVE-2863] - Ambiguous table name or column reference message displays when table and column names are the same * [HIVE-2875] - Renaming partition changes partition location prefix * [HIVE-2883] - Metastore client doesnt close connection properly * [HIVE-2901] - Hive union with NULL constant and string in same column returns all null * [HIVE-2913] - BlockMergeTask Doesn't Honor Job Configuration Properties when used directly * [HIVE-2920] - TestStatsPublisherEnhanced throws NPE on JDBC connection failure * [HIVE-2923] - testAclPositive in TestZooKeeperTokenStore failing in clean checkout when run on Mac * [HIVE-2948] - HiveFileFormatUtils should use Path.SEPARATOR instead of File.Separator * [HIVE-2958] - GROUP BY causing ClassCastException [LazyDioInteger cannot be cast LazyInteger] * [HIVE-2970] - several jars in hive tar generated are not required * [HIVE-2992] - JOIN + LATERAL VIEW + MAPJOIN fails to return result (seems to stop halfway through and no longer do the final reduce part) * [HIVE-3596] - Regression - HiveConf static variable causes issues in long running JVM instances with /tmp/ data * [HIVE-3685] - TestCliDriver (script_pipe.q) failed with IBM JDK * [HIVE-4074] - Doc update for .8, .9 and .10 ** Improvement * [HIVE-1040] - use sed rather than diff for masking out noise in diff-based tests * [HIVE-1487] - parallelize test query runs * [HIVE-1877] - Add java_method() as a synonym for the reflect() UDF * [HIVE-2203] - Extend concat_ws() UDF to support arrays of strings * [HIVE-2249] - When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double * [HIVE-2471] - Add timestamp column to the partition stats table. * [HIVE-2518] - pull junit jar from maven repos via ivy * [HIVE-2559] - Add target to install Hive JARs/POMs in the local Maven cache * [HIVE-2577] - Expose the HiveConf in HiveConnection API * [HIVE-2589] - Newly created partition should inherit properties from table * [HIVE-2611] - Make index table output of create index command if index is table based * [HIVE-2628] - move one line log from MapOperator to HiveContextAwareRecordReader * [HIVE-2640] - Add alterPartition to AlterHandler interface * [HIVE-2642] - fix Hive-2566 and make union optimization more aggressive * [HIVE-2651] - The variable hive.exec.mode.local.auto.tasks.max should be changed * [HIVE-2652] - Change arc config to hide generated files from Differential by default * [HIVE-2662] - Add Ant configuration property for dumping classpath of tests * [HIVE-2665] - Support for metastore service specific HADOOP_OPTS environment setting * [HIVE-2676] - The row count that loaded to a table may not right * [HIVE-2700] - Add 'ivy-clean-cache' and 'very-clean' Ant targets * [HIVE-2712] - Make ZooKeeper token store ACL configurable * [HIVE-2756] - Views should be added to the inputs of queries. * [HIVE-2760] - TestCliDriver should log elapsed time * [HIVE-2764] - Obtain delegation tokens for MR jobs in secure hbase setup * [HIVE-2765] - hbase handler uses ZooKeeperConnectionException which is not compatible with HBase versions other than 0.89 * [HIVE-2773] - HiveStorageHandler.configureTableJobProperites() should let the handler know wether it is configuration for input or output * [HIVE-2779] - Improve hooks run in Driver * [HIVE-2781] - HBaseSerDe should allow users to specify the timestamp passed to Puts * [HIVE-2795] - View partitions do not have a storage descriptor * [HIVE-2797] - Make the IP address of a Thrift client available to HMSHandler. * [HIVE-2823] - Add logging of total run time of Driver * [HIVE-2825] - Concatenating a partition does not inherit location from table * [HIVE-2827] - Implement nullsafe equi-join * [HIVE-2832] - Cache error messages for additional logging * [HIVE-2835] - Change default configuration for hive.exec.dynamic.partition * [HIVE-2841] - Fix javadoc warnings * [HIVE-2850] - Remove zero length files * [HIVE-2853] - Add pre event listeners to metastore * [HIVE-2858] - Cache remote map reduce job stack traces for additional logging * [HIVE-2861] - Support eventual constant expression for filter pushdown for key ranges in hbase * [HIVE-2864] - If hive history file's directory doesn't exist don't crash * [HIVE-2865] - hive-config.sh should honor HIVE_HOME env * [HIVE-2866] - Cache local map reduce job errors for additional logging * [HIVE-2871] - Add a new hook to run at the beginning and end of the Driver.run method * [HIVE-2872] - Store which configs the user has explicitly changed * [HIVE-2879] - Add "rat" target to build to look for missing license headers * [HIVE-2881] - Remove redundant key comparing in SMBMapJoinOperator * [HIVE-2891] - TextConverter for UDF's is inefficient if the input object is already Text or Lazy * [HIVE-2908] - Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators * [HIVE-2930] - Add license to the Hive files * [HIVE-2946] - Hive metastore does not have any log messages while shutting itself down. * [HIVE-2961] - Remove need for storage descriptors for view partitions * [HIVE-3684] - Add support for filter pushdown for composite keys ** New Feature * [HIVE-1634] - Allow access to Primitive types stored in binary format in HBase * [HIVE-2005] - Implement BETWEEN operator * [HIVE-2279] - Implement sort_array UDF * [HIVE-2490] - Add reset operation and average time attribute to Metrics MBean. * [HIVE-2602] - add support for insert partition overwrite(...) if not exists * [HIVE-2612] - support hive table/partitions exists in more than one region * [HIVE-2621] - Allow multiple group bys with the same input data and spray keys to be run on the same reducer. * [HIVE-2695] - Add PRINTF() Udf * [HIVE-2698] - Enable Hadoop-1.0.0 in Hive * [HIVE-2810] - Implement NULL-safe equality operator <=> * [HIVE-2815] - Filter pushdown in hbase for keys stored in binary format * [HIVE-2819] - Closed range scans on hbase keys * [HIVE-2822] - Add JSON output to the hive ddl commands * [HIVE-2894] - RCFile Reader doesn't provide access to Metadata * [HIVE-2898] - Add nicer helper functions for adding and reading metadata from RCFiles * [HIVE-2936] - Warehouse table subdirectories should inherit the group permissions of the warehouse parent directory ** Task * [HIVE-2645] - Hive Web Server startup messages logs incorrect path it is searching for WAR * [HIVE-2719] - Fix test failures caused by HIVE-2589 * [HIVE-2748] - Upgrade Hbase and ZK dependcies * [HIVE-2768] - Add a getAuthorizationProvider to HiveStorageHandler * [HIVE-2805] - Move metastore upgrade scripts labeled 0.10.0 into scripts labeled 0.9.0 * [HIVE-2962] - Remove unnecessary JAR dependencies * [HIVE-2965] - Revert HIVE-2612 * [HIVE-2966] - Revert HIVE-2795 * [HIVE-3804] - Row number issue in hive ** Test * [HIVE-2686] - Test ppr_pushdown.q is failing on trunk * [HIVE-2727] - add a testcase for partitioned view on union and base tables have index ** Wish * [HIVE-2682] - Clean-up logs Release Notes - Hive - Version 0.8.0 ** New Feature * [HIVE-192] - Add TIMESTAMP column type for thrift dynamic_type * [HIVE-306] - Support "INSERT [INTO] destination" * [HIVE-788] - Triggers when a new partition is created for a table * [HIVE-818] - Create a Hive CLI that connects to hive ThriftServer * [HIVE-872] - Allow type widening on COALESCE/UNION ALL * [HIVE-956] - Add support of columnar binary serde * [HIVE-1003] - optimize metadata only queries * [HIVE-1310] - Partitioning columns should be of primitive types only * [HIVE-1343] - add an interface in RCFile to support concatenation of two files without (de)compression * [HIVE-1537] - Allow users to specify LOCATION in CREATE DATABASE statement * [HIVE-1694] - Accelerate GROUP BY execution using indexes * [HIVE-1734] - Implement map_keys() and map_values() UDFs * [HIVE-1735] - Extend Explode UDTF to handle Maps * [HIVE-1803] - Implement bitmap indexing in Hive * [HIVE-1918] - Add export/import facilities to the hive system * [HIVE-1941] - support explicit view partitioning * [HIVE-1950] - Block merge for RCFile * [HIVE-2090] - Add "DROP DATABASE ... CASCADE/RESTRICT" * [HIVE-2121] - Input Sampling By Splits * [HIVE-2185] - extend table statistics to store the size of uncompressed data (+extend interfaces for collecting other types of statistics) * [HIVE-2188] - Add get_table_objects_by_name() to Hive MetaStore * [HIVE-2215] - Add api for marking / querying set of partitions for events * [HIVE-2223] - support grouping on complex types in Hive * [HIVE-2225] - Purge expired events * [HIVE-2236] - Cli: Print Hadoop's CPU milliseconds * [HIVE-2244] - Add a Plugin Developer Kit to Hive * [HIVE-2272] - add TIMESTAMP data type * [HIVE-2278] - Support archiving for multiple partitions if the table is partitioned by multiple columns * [HIVE-2380] - Add Binary Datatype in Hive * [HIVE-2500] - Allow Hive to be debugged remotely * [HIVE-2509] - Literal bigint * [HIVE-2561] - Allow UDFs to specify additional FILE/JAR resources necessary for execution ** Bug * [HIVE-11] - better error code from Hive describe command * [HIVE-106] - Join operation fails for some queries * [HIVE-619] - Improve the error messages for missing/incorrect UDF/UDAF class * [HIVE-1218] - CREATE TABLE t LIKE some_view should create a new empty base table, but instead creates a copy of view * [HIVE-1302] - describe parse_url throws an error * [HIVE-1342] - Predicate push down get error result when sub-queries have the same alias name * [HIVE-1461] - Clean up references to 'hive.metastore.local' * [HIVE-1538] - FilterOperator is applied twice with ppd on. * [HIVE-1592] - ProxyFileSystem.close calls super.close twice. * [HIVE-1595] - job name for alter table archive partition

is not correct * [HIVE-1631] - JDBC driver returns wrong precision, scale, or column size for some data types * [HIVE-1675] - SAXParseException on plan.xml during local mode. * [HIVE-1825] - Different defaults for hive.metastore.local * [HIVE-1850] - alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?) * [HIVE-1884] - Potential risk of resource leaks in Hive * [HIVE-1937] - DDLSemanticAnalyzer won't take newly set Hive parameters * [HIVE-1943] - Metastore operations (like drop_partition) could be improved in terms of maintaining consistency of metadata and data * [HIVE-1959] - Potential memory leak when same connection used for long time. TaskInfo and QueryInfo objects are getting accumulated on executing more queries on the same connection. * [HIVE-1963] - Don't set ivy.home in build-common.xml * [HIVE-1965] - Auto convert mapjoin should not throw exception if the top operator is union operator. * [HIVE-1973] - Getting error when join on tables where name of table has uppercase letters * [HIVE-1974] - In error scenario some opened streams may not closed in ScriptOperator.java, Utilities.java * [HIVE-1975] - "insert overwrite directory" Not able to insert data with multi level directory path * [HIVE-1976] - Exception should be thrown when invalid jar,file,archive is given to add command * [HIVE-1980] - Merging using mapreduce rather than map-only job failed in case of dynamic partition inserts * [HIVE-1987] - HWI admin_list_jobs JSP page throws exception * [HIVE-1988] - Make the delegation token issued by the MetaStore owned by the right user * [HIVE-2001] - Add inputs and outputs to authorization DDL commands * [HIVE-2003] - LOAD compilation does not set the outputs during semantic analysis resulting in no authorization checks being done for it. * [HIVE-2008] - keyword_1.q is failing * [HIVE-2022] - Making JDO thread-safe by default * [HIVE-2024] - In Driver.execute(), mapred.job.tracker is not restored if one of the task fails. * [HIVE-2025] - Fix TestEmbeddedHiveMetaStore and TestRemoteHiveMetaStore broken by HIVE-2022 * [HIVE-2031] - Correct the exception message for the better traceability for the scenario load into the partitioned table having 2 partitions by specifying only one partition in the load statement. * [HIVE-2032] - create database does not honour warehouse.dir in dbproperties * [HIVE-2033] - A database's warehouse.dir is not used for tables created in it. * [HIVE-2034] - Backport HIVE-1991 after overridden by HIVE-1950 * [HIVE-2037] - Merge result file size should honor hive.merge.size.per.task * [HIVE-2040] - the retry logic in Hive's concurrency is not working correctly. * [HIVE-2042] - In error scenario some opened streams may not closed * [HIVE-2045] - TCTLSeparatedProtocol.SimpleTransportTokenizer.nextToken() throws Null Pointer Exception in some cases * [HIVE-2054] - Exception on windows when using the jdbc driver. "IOException: The system cannot find the path specified" * [HIVE-2060] - CLI local mode hit NPE when exiting by ^D * [HIVE-2061] - Create a hive_contrib.jar symlink to hive-contrib-{version}.jar for backward compatibility * [HIVE-2062] - HivePreparedStatement.executeImmediate always throw exception * [HIVE-2069] - NullPointerException on getSchemas * [HIVE-2080] - Few code improvements in the ql and serde packages. * [HIVE-2083] - Bug: RowContainer was set to 1 in JoinUtils. * [HIVE-2086] - Add test coverage for external table data loss issue * [HIVE-2095] - auto convert map join bug * [HIVE-2096] - throw a error if the input is larger than a threshold for index input format * [HIVE-2098] - Make couple of convenience methods in EximUtil public * [HIVE-2100] - virtual column references inside subqueries cause execution exceptions * [HIVE-2107] - Log4J initialization info should not be printed out if -S is specified * [HIVE-2113] - In shell mode, local mode continues if a local-mode task throws exception in pre-hooks * [HIVE-2117] - insert overwrite ignoring partition location * [HIVE-2120] - auto convert map join may miss good candidates * [HIVE-2122] - Remove usage of deprecated methods from org.apache.hadoop.io package * [HIVE-2125] - alter table concatenate fails and deletes data * [HIVE-2131] - Bitmap Operation UDF doesn't clear return list * [HIVE-2138] - Exception when no splits returned from index * [HIVE-2142] - Jobs do not get killed even when they created too many files. * [HIVE-2145] - NPE during parsing order-by expression * [HIVE-2146] - Block Sampling should adjust number of reducers accordingly to make it useful * [HIVE-2151] - Too many open files in running negative cli tests * [HIVE-2153] - Stats JDBC LIKE queries should escape '_' and '%' * [HIVE-2157] - NPE in MapJoinObjectKey * [HIVE-2159] - TableSample(percent ) uses one intermediate size to be int, which overflows for large sampled size, making the sampling never triggered. * [HIVE-2160] - Few code improvements in the metastore,hwi and ql packages. * [HIVE-2176] - Schema creation scripts are incomplete since they leave out tables that are specific to DataNucleus * [HIVE-2178] - Log related Check style Comments fixes * [HIVE-2181] - Clean up the scratch.dir (tmp/hive-root) while restarting Hive server. * [HIVE-2182] - Avoid null pointer exception when executing UDF * [HIVE-2183] - In Task class and its subclasses logger is initialized in constructor * [HIVE-2184] - Few improvements in org.apache.hadoop.hive.ql.metadata.Hive.close() * [HIVE-2186] - Dynamic Partitioning Failing because of characters not supported globStatus * [HIVE-2192] - Stats table schema incompatible after HIVE-2185 * [HIVE-2196] - Ensure HiveConf includes all properties defined in hive-default.xml * [HIVE-2197] - SessionState used before ThreadLocal set * [HIVE-2198] - While using Hive in server mode, HiveConnection.close() is not cleaning up server side resources * [HIVE-2199] - incorrect success flag passed to jobClose * [HIVE-2204] - unable to get column names for a specific table that has '_' as part of its table name * [HIVE-2211] - Fix a bug caused by HIVE-243 * [HIVE-2214] - CommandNeedRetryException.java is missing ASF header * [HIVE-2222] - runnable queue in Driver and DriverContext is not thread safe * [HIVE-2237] - hive fails to build in eclipse due to syntax error in BitmapIndexHandler.java * [HIVE-2243] - Can't publish maven release artifacts to apache repository * [HIVE-2248] - Comparison Operators convert number types to common type instead of double if possible * [HIVE-2253] - Merge failing of join tree in exceptional case * [HIVE-2257] - Enable TestHadoop20SAuthBridge * [HIVE-2259] - Skip comments in hive script * [HIVE-2260] - ExecDriver::addInputPaths should pass the table properties to the record writer * [HIVE-2275] - Revert HIVE-2219 and apply correct patch to improve the efficiency of dropping multiple partitions * [HIVE-2276] - Fix Inconsistency between RB and JIRA patches for HIVE-2194 * [HIVE-2281] - Regression introduced from HIVE-2155 * [HIVE-2286] - ClassCastException when building index with security.authorization turned on * [HIVE-2287] - Error during UNARCHIVE of a partition * [HIVE-2292] - Comment clause should immediately follow identifier field in CREATE DATABASE statement * [HIVE-2294] - Allow ShimLoader to work with Hadoop 0.20-append * [HIVE-2296] - bad compressed file names from insert into * [HIVE-2298] - Fix UDAFPercentile to tolerate null percentiles * [HIVE-2303] - files with control-A,B are not delimited correctly. * [HIVE-2307] - Schema creation scripts for PostgreSQL use bit(1) instead of boolean * [HIVE-2309] - Incorrect regular expression for extracting task id from filename * [HIVE-2315] - DatabaseMetadata.getColumns() does not return partition column names for a table * [HIVE-2319] - Calling alter_table after changing partition comment throws an exception * [HIVE-2322] - Add ColumnarSerDe to the list of native SerDes * [HIVE-2326] - Turn off bitmap indexing when map-side aggregation is turned off * [HIVE-2328] - hive.zookeeper.session.timeout is set to null in hive-default.xml * [HIVE-2331] - Turn off compression when generating index intermediate results * [HIVE-2334] - DESCRIBE TABLE causes NPE when hive.cli.print.header=true * [HIVE-2335] - Indexes are still automatically queried when out of sync with their source tables * [HIVE-2337] - Predicate pushdown erroneously conservative with outer joins * [HIVE-2338] - Alter table always throws an unhelpful error on failure * [HIVE-2342] - mirror.facebook.net is 404ing * [HIVE-2343] - stats not updated for non "load table desc" operations * [HIVE-2344] - filter is removed due to regression of HIVE-1538 * [HIVE-2356] - Fix udtf_explode.q and udf_explode.q test failures * [HIVE-2358] - JDBC DatabaseMetaData and ResultSetMetaData need to match for particular types * [HIVE-2362] - HiveConf properties not appearing in the output of 'set' or 'set -v' * [HIVE-2366] - Metastore upgrade scripts for HIVE-2246 do not migrate indexes nor rename the old COLUMNS table * [HIVE-2368] - Slow dropping of partitions caused by full listing of storage descriptors * [HIVE-2369] - Minor typo in error message in HiveConnection.java (JDBC) * [HIVE-2382] - Invalid predicate pushdown from incorrect column expression map for select operator generated by GROUP BY operation * [HIVE-2383] - Incorrect alias filtering for predicate pushdown * [HIVE-2384] - import of multiple partitions from a partitioned table with external location overwrites files * [HIVE-2386] - Add Mockito to LICENSE file * [HIVE-2391] - published POMs in Maven repo are incorrect * [HIVE-2393] - Fix whitespace test diff accidentally introduced in HIVE-1360 * [HIVE-2398] - Hive server doesn't return schema for 'set' command * [HIVE-2402] - Function like with empty string is throwing null pointer exception * [HIVE-2405] - get_privilege does not get user level privilege * [HIVE-2407] - File extensions not preserved in Hive.checkPaths when renaming new destination file * [HIVE-2411] - Metastore server tries to connect to NN without authenticating itself * [HIVE-2412] - Update Eclipse configuration to include Mockito dependency * [HIVE-2413] - BlockMergeTask ignores client-specified jars * [HIVE-2417] - Merging of compressed rcfiles fails to write the valuebuffer part correctly * [HIVE-2429] - skip corruption bug that cause data not decompressed * [HIVE-2431] - upgrading thrift version didn't upgrade libthrift.jar symlink correctly * [HIVE-2451] - TABLESAMBLE(BUCKET xxx) sometimes doesn't trigger input pruning as regression of HIVE-1538 * [HIVE-2455] - Pass correct remoteAddress in proxy user authentication * [HIVE-2459] - remove all @author tags from source * [HIVE-2463] - fix Eclipse for javaewah upgrade * [HIVE-2465] - Primitive Data Types returning null if the data is out of range of the data type. * [HIVE-2466] - mapjoin_subquery dump small table (mapjoin table) to the same file * [HIVE-2472] - Metastore statistics are not being updated for CTAS queries. * [HIVE-2474] - Hive PDK needs an Ivy configuration file * [HIVE-2481] - HadoopJobExecHelper does not handle null counters well * [HIVE-2486] - Phabricator for code review * [HIVE-2487] - Bug from HIVE-2446, the code that calls client stats publishers run() methods is in wrong place, should be in the same method but inside of while (!rj.isComplete()) {} loop * [HIVE-2488] - PDK tests failing on Hudson because HADOOP_HOME is not defined * [HIVE-2492] - PDK PluginTest failing on Hudson * [HIVE-2497] - partition pruning prune some right partition under specific conditions * [HIVE-2499] - small table filesize for automapjoin is not consistent in HiveConf.java and hive-default.xml * [HIVE-2501] - When new instance of Hive (class) is created, the current database is reset to default (current database shouldn't be changed). * [HIVE-2510] - Hive throws Null Pointer Exception upon CREATE TABLE . .... if the given doesn't exist * [HIVE-2516] - cleaunup QTestUtil: use test.data.files as current directory if one not specified * [HIVE-2519] - Dynamic partition insert should enforce the order of the partition spec is the same as the one in schema * [HIVE-2522] - HIVE-2446 bug (next one) - If constructor of ClientStatsPublisher throws runtime exception it will be propagated to HadoopJobExecHelper's progress method and beyond, whereas it shouldn't * [HIVE-2531] - Allow people to use only issue numbers without 'HIVE-' prefix with `arc diff --jira`. * [HIVE-2532] - Evaluation of non-deterministic/stateful UDFs should not be skipped even if constant oi is returned. * [HIVE-2534] - HiveIndexResult creation fails due to file system issue * [HIVE-2536] - Support scientific notation for Double literals * [HIVE-2548] - How to submit documentation fixes * [HIVE-2550] - Provide jira_base_url for improved arc commit workflow * [HIVE-2556] - upgrade script 008-HIVE-2246.mysql.sql contains syntax errors * [HIVE-2562] - HIVE-2247 Changed the Thrift API causing compatibility issues. * [HIVE-2565] - Add Java linter to Hive * [HIVE-2568] - HIVE-2246 upgrade script needs to drop foreign key in COLUMNS_OLD * [HIVE-2571] - eclipse template .classpath is broken * [HIVE-2572] - HIVE-2246 upgrade script changed the COLUMNS_V2.COMMENT length * [HIVE-2574] - ivy offline mode broken by changingPattern and checkmodified attributes * [HIVE-2578] - Debug mode in some situations doesn't work properly when child JVM is started from MapRedLocalTask * [HIVE-2580] - Hive build fails with error "java.io.IOException: Not in GZIP format" * [HIVE-2581] - explain task: getJSONPlan throws a NPE if the ast is null * [HIVE-2583] - bug in ivy 2.2.0 breaks build * [HIVE-2588] - Update arcconfig to include commit listener * [HIVE-2590] - HBase bulk load wiki page improvements * [HIVE-2598] - Update README.txt file to use description from wiki * [HIVE-2613] - HiveCli eclipse launch configuration hangs * [HIVE-2622] - Hive POMs reference the wrong Hadoop artifacts * [HIVE-2624] - Fix eclipse classpath template broken in HIVE-2523 * [HIVE-2625] - Fix maven-build Ant target * [HIVE-2630] - TestHiveServer doesn't produce a JUnit report file * [HIVE-2634] - revert HIVE-2566 * [HIVE-2643] - Recent patch prevents Hadoop confs from loading in 0.20.204 ** Improvement * [HIVE-1078] - CREATE VIEW followup: CREATE OR REPLACE * [HIVE-1360] - Allow UDFs to access constant parameter values at compile time * [HIVE-1567] - increase hive.mapjoin.maxsize to 10 million * [HIVE-1644] - use filter pushdown for automatically accessing indexes * [HIVE-1690] - HivePreparedStatement.executeImmediate(String sql) is breaking the exception stack * [HIVE-1731] - Improve miscellaneous error messages * [HIVE-1740] - support NOT IN and NOT LIKE syntax * [HIVE-1741] - HiveInputFormat.readFields should print the cause when there's an exception * [HIVE-1784] - Ctrl+c should kill currently running query, but not exit the CLI * [HIVE-1815] - The class HiveResultSet should implement batch fetching. * [HIVE-1833] - Task-cleanup task should be disabled * [HIVE-1887] - HIVE-78 Followup: group partitions by tables when do authorizations and there is no partition level privilege * [HIVE-1916] - Change Default Alias For Aggregated Columns (_c1) * [HIVE-1966] - mapjoin operator should not load hashtable for each new inputfile if the hashtable to be loaded is already there. * [HIVE-1989] - recognize transitivity of predicates on join keys * [HIVE-1991] - Hive Shell to output number of mappers and number of reducers * [HIVE-1994] - Support new annotation @UDFType(stateful = true) * [HIVE-2000] - adding comments to Hive Stats JDBC queries * [HIVE-2002] - Expand exceptions caught for metastore operations * [HIVE-2018] - avoid loading Hive aux jars in CLI remote mode * [HIVE-2020] - Create a separate namespace for Hive variables * [HIVE-2028] - Performance instruments for client side execution * [HIVE-2030] - isEmptyPath() to use ContentSummary cache * [HIVE-2035] - Use block-level merge for RCFile if merging intermediate results are needed * [HIVE-2036] - Update bitmap indexes for automatic usage * [HIVE-2038] - Metastore listener * [HIVE-2039] - remove hadoop version check from hive cli shell script * [HIVE-2051] - getInputSummary() to call FileSystem.getContentSummary() in parallel * [HIVE-2052] - PostHook and PreHook API to add flag to indicate it is pre or post hook plus cache for content summary * [HIVE-2056] - Generate single MR job for multi groupby query if hive.multigroupby.singlemr is enabled. * [HIVE-2068] - Speed up query "select xx,xx from xxx LIMIT xxx" if no filtering or aggregation * [HIVE-2070] - SHOW GRANT grantTime field should be a human-readable timestamp * [HIVE-2082] - Reduce memory consumption in preparing MapReduce job * [HIVE-2106] - Increase the number of operator counter * [HIVE-2109] - No lock for some non-mapred tasks config variable hive.lock.mapred.only.operation added * [HIVE-2119] - Optimizer on partition field * [HIVE-2126] - Hive's symlink text input format should be able to work with ComineHiveInputFormat * [HIVE-2127] - Improve stats gathering reliability by retries on failures with hive.stats.retries.max and hive.stats.retries.wait * [HIVE-2128] - Automatic Indexing with multiple tables * [HIVE-2133] - DROP TABLE IF EXISTS should not fail if a view of that name exists * [HIVE-2134] - Remove System.exit * [HIVE-2139] - Enables HiveServer to accept -hiveconf option * [HIVE-2144] - reduce workload generated by JDBCStatsPublisher * [HIVE-2147] - Add api to send / receive message to metastore * [HIVE-2148] - Add interface classification in Hive. * [HIVE-2154] - add exception handling to hive's record reader * [HIVE-2155] - Improve error messages emitted during semantic analysis * [HIVE-2156] - Improve error messages emitted during task execution * [HIVE-2171] - Allow custom serdes to set field comments * [HIVE-2191] - Allow optional [inner] on equi-join. * [HIVE-2194] - Add actions for alter table and alter partition events for metastore event listeners * [HIVE-2201] - reduce name node calls in hive by creating temporary directories * [HIVE-2208] - create a new API in Warehouse where the root directory is specified * [HIVE-2209] - Provide a way by which ObjectInspectorUtils.compare can be extended by the caller for comparing maps which are part of the object * [HIVE-2210] - ALTER VIEW RENAME * [HIVE-2213] - Optimize partial specification metastore functions * [HIVE-2217] - add Query text for debugging in lock data * [HIVE-2218] - speedup addInputPaths * [HIVE-2219] - Make "alter table drop partition" more efficient * [HIVE-2221] - Provide metastore upgarde script for HIVE-2215 * [HIVE-2224] - Ability to add partitions atomically * [HIVE-2226] - Add API to retrieve table names by an arbitrary filter, e.g., by owner, retention, parameters, etc. * [HIVE-2233] - Show current database in hive prompt * [HIVE-2245] - Make CombineHiveInputFormat the default hive.input.format * [HIVE-2246] - Dedupe tables' column schemas from partitions in the metastore db * [HIVE-2252] - Display a sample of partitions created when Fatal Error occurred due to too many partitioned created * [HIVE-2256] - Better error message in CLI on invalid column name * [HIVE-2282] - Local mode needs to work well with block sampling * [HIVE-2284] - bucketized map join should allow join key as a superset of bucketized columns * [HIVE-2290] - Improve error messages for DESCRIBE command * [HIVE-2299] - Optimize Hive query startup time for multiple partitions * [HIVE-2346] - Add hooks to run when execution fails. * [HIVE-2347] - Make Hadoop Job ID available after task finishes executing * [HIVE-2350] - Improve RCFile Read Speed * [HIVE-2354] - Support automatic rebuilding of indexes when they go stale * [HIVE-2364] - Make performance logging configurable. * [HIVE-2370] - Improve RCFileCat performance significantly * [HIVE-2378] - Warn user that precision is lost when bigint is implicitly cast to double. * [HIVE-2385] - Local Mode can be more aggressive if LIMIT optimization is on * [HIVE-2396] - RCFileReader Buffer Reuse * [HIVE-2404] - Allow RCFile Reader to tolerate corruptions * [HIVE-2440] - make hive mapper initialize faster when having tons of input files * [HIVE-2445] - The PerfLogger should log the full name of hooks, not just the simple name. * [HIVE-2446] - Introduction of client statistics publishers possibility * [HIVE-2447] - Add job ID to MapRedStats * [HIVE-2448] - Upgrade JavaEWAH to 0.3 * [HIVE-2450] - move lock retry logic into ZooKeeperHiveLockManager * [HIVE-2453] - Need a way to categorize queries in hooks for improved logging * [HIVE-2456] - JDBCStatsAggregator DELETE STATEMENT should escape _ and % * [HIVE-2457] - Files in Avro-backed Hive tables do not have a ".avro" extension * [HIVE-2458] - Group-by query optimization Followup: add flag in conf/hive-default.xml * [HIVE-2461] - Add method to PerfLogger to perform cleanup/final steps. * [HIVE-2462] - make INNER a non-reserved keyword * [HIVE-2467] - HA Support for Metastore Server * [HIVE-2470] - Improve support for Constant Object Inspectors * [HIVE-2479] - Log more Hadoop task counter values in the MapRedStats class. * [HIVE-2484] - Enable ALTER TABLE SET SERDE to work on partition level * [HIVE-2505] - Update junit jar in testlibs * [HIVE-2506] - Get ConstantObjectInspectors working in UDAFs * [HIVE-2515] - Make Constant OIs work with UDTFs. * [HIVE-2523] - add a new builtins subproject * [HIVE-2527] - Consecutive string literals should be combined into a single string literal. * [HIVE-2535] - Use sorted nature of compact indexes * [HIVE-2545] - Make metastore log4j configuration file configurable again. * [HIVE-2546] - add explain formatted * [HIVE-2553] - Use hashing instead of list traversal for IN operator for primitive types * [HIVE-2566] - reduce the number map-reduce jobs for union all * [HIVE-2569] - Too much debugging info on console if a job failed * [HIVE-2593] - avoid referencing /tmp in tests * [HIVE-2605] - Setting no_drop on a table should cascade to child partitions * [HIVE-2607] - Add caching to json_tuple * [HIVE-2619] - Add hook to run in metastore's endFunction which can collect more fb303 counters ** Task * [HIVE-1095] - Hive in Maven * [HIVE-2076] - Provide Metastore upgrade scripts and default schemas for PostgreSQL * [HIVE-2161] - Remaining patch for HIVE-2148 * [HIVE-2239] - Use the version commons-codec from Hadoop * [HIVE-2376] - Upgrade Hive's Thrift dependency to version 0.7.0 * [HIVE-2441] - Metastore upgrade scripts for schema change introduced in HIVE-2215 * [HIVE-2442] - Metastore upgrade script and schema DDL for Hive 0.8.0 * [HIVE-2468] - Make Hive compile against Hadoop 0.23 * [HIVE-2491] - Add pdk, hbase-handler etc as source dir in eclipse * [HIVE-2521] - Update wiki links in README file * [HIVE-2552] - Omit incomplete Postgres upgrade scripts from release tarball ** Sub-task * [HIVE-559] - Support JDBC ResultSetMetadata * [HIVE-1983] - Bundle Log4j configuration files in Hive JARs * [HIVE-2049] - Push down partition pruning to JDO filtering for a subset of partition predicates * [HIVE-2050] - batch processing partition pruning process * [HIVE-2114] - Backward incompatibility introduced from HIVE-2082 in MetaStoreUtils.getPartSchemaFromTableSchema() * [HIVE-2118] - Partition Pruning bug in the case of hive.mapred.mode=nonstrict * [HIVE-2140] - Return correct Major / Minor version numbers for Hive Driver * [HIVE-2158] - add the HivePreparedStatement implementation based on current HIVE supported data-type * [HIVE-2434] - add a TM to Hive logo image * [HIVE-2435] - Update project naming and description in Hive wiki * [HIVE-2436] - Update project naming and description in Hive website * [HIVE-2437] - update project website navigation links * [HIVE-2438] - add trademark attributions to Hive homepage * [HIVE-2476] - Update project description and wiki link in ivy.xml files ** Test * [HIVE-2426] - Test that views with joins work properly * [HIVE-2493] - TestLazySimpleSerde fails randomly * [HIVE-2513] - create a test to verify that partition pruning works for partitioned views with a union ** Wish * [HIVE-243] - ^C breaks out of running query, but not whole CLI Release Notes - Hive - Version 0.7.0 ** New Feature * [HIVE-78] - Authorization infrastructure for Hive * [HIVE-417] - Implement Indexing in Hive * [HIVE-471] - Add reflect() UDF for reflective invocation of Java methods * [HIVE-537] - Hive TypeInfo/ObjectInspector to support union (besides struct, array, and map) * [HIVE-842] - Authentication Infrastructure for Hive * [HIVE-1096] - Hive Variables * [HIVE-1293] - Concurrency Model for Hive * [HIVE-1304] - add row_sequence UDF * [HIVE-1405] - hive command line option -i to run an init file before other SQL commands * [HIVE-1408] - add option to let hive automatically run in local mode based on tunable heuristics * [HIVE-1413] - bring a table/partition offline * [HIVE-1438] - sentences() UDF for natural language tokenization * [HIVE-1481] - ngrams() UDAF for estimating top-k n-gram frequencies * [HIVE-1514] - Be able to modify a partition's fileformat and file location information. * [HIVE-1518] - context_ngrams() UDAF for estimating top-k contextual n-grams * [HIVE-1528] - Add json_tuple() UDTF function * [HIVE-1529] - Add ANSI SQL covariance aggregate functions: covar_pop and covar_samp. * [HIVE-1549] - Add ANSI SQL correlation aggregate function CORR(X,Y). * [HIVE-1609] - Support partition filtering in metastore * [HIVE-1624] - Patch to allows scripts in S3 location * [HIVE-1636] - Implement "SHOW TABLES {FROM | IN} db_name" * [HIVE-1659] - parse_url_tuple: a UDTF version of parse_url * [HIVE-1661] - Default values for parameters * [HIVE-1779] - Implement GenericUDF str_to_map * [HIVE-1790] - Patch to support HAVING clause in Hive * [HIVE-1792] - track the joins which are being converted to map-join automatically * [HIVE-1818] - Call frequency and duration metrics for HiveMetaStore via jmx * [HIVE-1819] - maintain lastAccessTime in the metastore * [HIVE-1820] - Make Hive database data center aware * [HIVE-1827] - Add a new local mode flag in Task. * [HIVE-1835] - Better auto-complete for Hive * [HIVE-1840] - Support ALTER DATABASE to change database properties * [HIVE-1856] - Implement DROP TABLE/VIEW ... IF EXISTS * [HIVE-1858] - Implement DROP {PARTITION, INDEX, TEMPORARY FUNCTION} IF EXISTS * [HIVE-1881] - Make the MetaStore filesystem interface pluggable via the hive.metastore.fs.handler.class configuration property * [HIVE-1889] - add an option (hive.index.compact.file.ignore.hdfs) to ignore HDFS location stored in index files. * [HIVE-1971] - Verbose/echo mode for the Hive CLI ** Improvement * [HIVE-138] - Provide option to export a HEADER * [HIVE-474] - Support for distinct selection on two or more columns * [HIVE-558] - describe extended table/partition output is cryptic * [HIVE-1126] - Missing some Jdbc functionality like getTables getColumns and HiveResultSet.get* methods based on column name. * [HIVE-1211] - Tapping logs from child processes * [HIVE-1226] - support filter pushdown against non-native tables * [HIVE-1229] - replace dependencies on HBase deprecated API * [HIVE-1235] - use Ivy for fetching HBase dependencies * [HIVE-1264] - Make Hive work with Hadoop security * [HIVE-1378] - Return value for map, array, and struct needs to return a string * [HIVE-1394] - do not update transient_lastDdlTime if the partition is modified by a housekeeping operation * [HIVE-1414] - automatically invoke .hiverc init script * [HIVE-1415] - add CLI command for executing a SQL script * [HIVE-1430] - serializing/deserializing the query plan is useless and expensive * [HIVE-1441] - Extend ivy offline mode to cover metastore downloads * [HIVE-1443] - Add support to turn off bucketing with ALTER TABLE * [HIVE-1447] - Speed up reflection method calls in GenericUDFBridge and GenericUDAFBridge * [HIVE-1456] - potentail NullPointerException * [HIVE-1463] - hive output file names are unnecessarily large * [HIVE-1469] - replace isArray() calls and remove LOG.isInfoEnabled() in Operator.forward() * [HIVE-1495] - supply correct information to hooks and lineage for index rebuild * [HIVE-1497] - support COMMENT clause on CREATE INDEX, and add new command for SHOW INDEXES * [HIVE-1498] - support IDXPROPERTIES on CREATE INDEX * [HIVE-1512] - Need to get hive_hbase-handler to work with hbase versions 0.20.4 0.20.5 and cloudera CDH3 version * [HIVE-1513] - hive starter scripts should load admin/user supplied script for configurability * [HIVE-1517] - ability to select across a database * [HIVE-1533] - Use ZooKeeper from maven * [HIVE-1536] - Add support for JDBC PreparedStatements * [HIVE-1546] - Ability to plug custom Semantic Analyzers for Hive Grammar * [HIVE-1581] - CompactIndexInputFormat should create split only for files in the index output file. * [HIVE-1605] - regression and improvements in handling NULLs in joins * [HIVE-1611] - Add alternative search-provider to Hive site * [HIVE-1616] - Add ProtocolBuffersStructObjectInspector * [HIVE-1617] - ScriptOperator's AutoProgressor can lead to an infinite loop * [HIVE-1622] - Use CombineHiveInputFormat for the merge job if hive.merge.mapredfiles=true * [HIVE-1638] - convert commonly used udfs to generic udfs * [HIVE-1641] - add map joined table to distributed cache * [HIVE-1642] - Convert join queries to map-join based on size of table/row * [HIVE-1645] - ability to specify parent directory for zookeeper lock manager * [HIVE-1655] - Adding consistency check at jobClose() when committing dynamic partitions * [HIVE-1660] - Change get_partitions_ps to pass partition filter to database * [HIVE-1692] - FetchOperator.getInputFormatFromCache hides causal exception * [HIVE-1701] - drop support for pre-0.20 Hadoop versions * [HIVE-1704] - remove Hadoop 0.17 specific test reference logs * [HIVE-1738] - Optimize Key Comparison in GroupByOperator * [HIVE-1743] - Group-by to determine equals of Keys in reverse order * [HIVE-1746] - Support for using ALTER to set IDXPROPERTIES * [HIVE-1749] - ExecMapper and ExecReducer: reduce function calls to l4j.isInfoEnabled() * [HIVE-1750] - Remove Partition Filtering Conditions when Possible * [HIVE-1751] - Optimize ColumnarStructObjectInspector.getStructFieldData() * [HIVE-1754] - Remove JDBM component from Map Join * [HIVE-1757] - test cleanup for Hive-1641 * [HIVE-1758] - optimize group by hash map memory * [HIVE-1761] - Support show locks for a particular table * [HIVE-1765] - Add queryid while locking * [HIVE-1768] - Update transident_lastDdlTime only if not specified * [HIVE-1782] - add more debug information for hive locking * [HIVE-1783] - CommonJoinOperator optimize the case of 1:1 join * [HIVE-1785] - change Pre/Post Query Hooks to take in 1 parameter: HookContext * [HIVE-1786] - Improve documentation for str_to_map() UDF * [HIVE-1787] - optimize the code path when there are no outer joins * [HIVE-1796] - dumps time at which lock was taken along with the queryid in show locks extended * [HIVE-1797] - Compressed the hashtable dump file before put into distributed cache * [HIVE-1798] - Clear empty files in Hive * [HIVE-1801] - HiveInputFormat or CombineHiveInputFormat always sync blocks of RCFile twice * [HIVE-1811] - Show the time the local task takes * [HIVE-1824] - create a new ZooKeeper instance when retrying lock, and more info for debug * [HIVE-1831] - Add a option to run task to check map-join possibility in non-local mode * [HIVE-1834] - more debugging for locking * [HIVE-1843] - add an option in dynamic partition inserts to throw an error if 0 partitions are created * [HIVE-1852] - Reduce unnecessary DFSClient.rename() calls * [HIVE-1855] - Include Process ID in the log4j log file name * [HIVE-1865] - redo zookeeper hive lock manager * [HIVE-1899] - add a factory method for creating a synchronized wrapper for IMetaStoreClient * [HIVE-1900] - a mapper should be able to span multiple partitions * [HIVE-1907] - Store jobid in ExecDriver * [HIVE-1910] - Provide config parameters to control cache object pinning * [HIVE-1923] - Allow any type of stats publisher and aggregator in addition to HBase and JDBC * [HIVE-1929] - Find a way to disable owner grants * [HIVE-1931] - Improve the implementation of the METASTORE_CACHE_PINOBJTYPES config * [HIVE-1948] - Have audit logging in the Metastore * [HIVE-1956] - "Provide DFS initialization script for Hive * [HIVE-1961] - Make Stats gathering more flexible with timeout and atomicity * [HIVE-1962] - make a libthrift.jar and libfb303.jar in dist package for backward compatibility * [HIVE-1970] - Modify build to run all tests regardless of subproject failures * [HIVE-1978] - Hive SymlinkTextInputFormat does not estimate input size correctly ** Bug * [HIVE-307] - "LOAD DATA LOCAL INPATH" fails when the table already contains a file of the same name * [HIVE-741] - NULL is not handled correctly in join * [HIVE-1203] - HiveInputFormat.getInputFormatFromCache "swallows" cause exception when throwing IOExcpetion * [HIVE-1305] - add progress in join and groupby * [HIVE-1376] - Simple UDAFs with more than 1 parameter crash on empty row query * [HIVE-1385] - UDF field() doesn't work * [HIVE-1416] - Dynamic partition inserts left empty files uncleaned in hadoop 0.17 local mode * [HIVE-1422] - skip counter update when RunningJob.getCounters() returns null * [HIVE-1440] - FetchOperator(mapjoin) does not work with RCFile * [HIVE-1448] - bug in 'set fileformat' * [HIVE-1453] - Make Eclipse launch templates auto-adjust to Hive version number changes * [HIVE-1462] - Reporting progress in FileSinkOperator works in multiple directory case * [HIVE-1465] - hive-site.xml ${user.name} not replaced for local-file derby metastore connection URL * [HIVE-1470] - percentile_approx() fails with more than 1 reducer * [HIVE-1471] - CTAS should unescape the column name in the select-clause. * [HIVE-1473] - plan file should have a high replication factor * [HIVE-1475] - .gitignore files being placed in test warehouse directories causing build failure * [HIVE-1489] - TestCliDriver -Doverwrite=true does not put the file in the correct directory * [HIVE-1491] - fix or disable loadpart_err.q * [HIVE-1494] - Index followup: remove sort by clause and fix a bug in collect_set udaf * [HIVE-1501] - when generating reentrant INSERT for index rebuild, quote identifiers using backticks * [HIVE-1508] - Add cleanup method to HiveHistory class * [HIVE-1509] - Monitor the working set of the number of files * [HIVE-1510] - HiveCombineInputFormat should not use prefix matching to find the partitionDesc for a given path * [HIVE-1520] - hive.mapred.local.mem should only be used in case of local mode job submissions * [HIVE-1523] - ql tests no longer work in miniMR mode * [HIVE-1532] - Replace globStatus with listStatus inside Hive.java's replaceFiles. * [HIVE-1534] - Join filters do not work correctly with outer joins * [HIVE-1535] - alter partition should throw exception if the specified partition does not exist. * [HIVE-1547] - Unarchiving operation throws NPE * [HIVE-1548] - populate inputs and outputs for all statements * [HIVE-1556] - Fix TestContribCliDriver test * [HIVE-1561] - smb_mapjoin_8.q returns different results in miniMr mode * [HIVE-1563] - HBase tests broken * [HIVE-1564] - bucketizedhiveinputformat.q fails in minimr mode * [HIVE-1570] - referencing an added file by it's name in a transform script does not work in hive local mode * [HIVE-1578] - Add conf. property hive.exec.show.job.failure.debug.info to enable/disable displaying link to the task with most failures * [HIVE-1580] - cleanup ExecDriver.progress * [HIVE-1583] - Hive should not override Hadoop specific system properties * [HIVE-1584] - wrong log files in contrib client positive * [HIVE-1589] - Add HBase/ZK JARs to Eclipse classpath * [HIVE-1593] - udtf_explode.q is an empty file * [HIVE-1598] - use SequenceFile rather than TextFile format for hive query results * [HIVE-1600] - need to sort hook input/output lists for test result determinism * [HIVE-1601] - Hadoop 0.17 ant test broken by HIVE-1523 * [HIVE-1606] - For a null value in a string column, JDBC driver returns the string "NULL" * [HIVE-1607] - Reinstate and deprecate IMetaStoreClient methods removed in HIVE-675 * [HIVE-1614] - UDTF json_tuple should return null row when input is not a valid JSON string * [HIVE-1628] - Fix Base64TextInputFormat to be compatible with commons codec 1.4 * [HIVE-1629] - Patch to fix hashCode method in DoubleWritable class * [HIVE-1630] - bug in NO_DROP * [HIVE-1633] - CombineHiveInputFormat fails with "cannot find dir for emptyFile" * [HIVE-1639] - ExecDriver.addInputPaths() error if partition name contains a comma * [HIVE-1647] - Incorrect initialization of thread local variable inside IOContext ( implementation is not threadsafe ) * [HIVE-1650] - TestContribNegativeCliDriver fails * [HIVE-1656] - All TestJdbcDriver test cases fail in Eclipse unless a property is added in run config * [HIVE-1657] - join results are displayed wrongly for some complex joins using select * * [HIVE-1658] - Fix describe * [extended] column formatting * [HIVE-1663] - ql/src/java/org/apache/hadoop/hive/ql/parse/SamplePruner.java is empty * [HIVE-1664] - Eclipse build broken * [HIVE-1670] - MapJoin throws EOFExeption when the mapjoined table has 0 column selected * [HIVE-1671] - multithreading on Context.pathToCS * [HIVE-1673] - Create table bug causes the row format property lost when serde is specified. * [HIVE-1674] - count(*) returns wrong result when a mapper returns empty results * [HIVE-1678] - NPE in MapJoin * [HIVE-1688] - In the MapJoinOperator, the code uses tag as alias, which is not always true * [HIVE-1691] - ANALYZE TABLE command should check columns in partition spec * [HIVE-1699] - incorrect partition pruning ANALYZE TABLE * [HIVE-1707] - bug when different partitions are present in different dfs * [HIVE-1711] - CREATE TABLE LIKE should not set stats in the new table * [HIVE-1712] - Migrating metadata from derby to mysql thrown NullPointerException * [HIVE-1713] - duplicated MapRedTask in Multi-table inserts mixed with FileSinkOperator and ReduceSinkOperator * [HIVE-1716] - make TestHBaseCliDriver use dynamic ports to avoid conflicts with already-running services * [HIVE-1717] - ant clean should delete stats database * [HIVE-1720] - hbase_stats.q is failing * [HIVE-1737] - Two Bugs for Estimating Row Sizes in GroupByOperator * [HIVE-1742] - Fix Eclipse templates (and use Ivy metadata to generate Eclipse library dependencies) * [HIVE-1748] - Statistics broken for tables with size in excess of Integer.MAX_VALUE * [HIVE-1753] - HIVE 1633 hit for Stage2 jobs with CombineHiveInputFormat * [HIVE-1756] - failures in fatal.q in TestNegativeCliDriver * [HIVE-1759] - Many important broken links on Hive web page * [HIVE-1760] - Mismatched open/commit transaction calls in case of connection retry * [HIVE-1767] - Merge files does not work with dynamic partition * [HIVE-1769] - pcr.q output is non-deterministic * [HIVE-1771] - ROUND(infinity) chokes * [HIVE-1775] - Assertation on inputObjInspectors.length in Groupy operator * [HIVE-1776] - parallel execution and auto-local mode combine to place plan file in wrong file system * [HIVE-1777] - Outdated comments for GenericUDTF.close() * [HIVE-1780] - Typo in hive-default.xml * [HIVE-1781] - outputs not populated for dynamic partitions at compile time * [HIVE-1794] - GenericUDFOr and GenericUDFAnd cannot receive boolean typed object * [HIVE-1795] - outputs not correctly populated for alter table * [HIVE-1804] - Mapjoin will fail if there are no files associating with the join tables * [HIVE-1806] - The merge criteria on dynamic partitons should be per partiton * [HIVE-1807] - No Element found exception in BucketMapJoinOptimizer * [HIVE-1808] - bug in auto_join25.q * [HIVE-1809] - Hive comparison operators are broken for NaN values * [HIVE-1812] - spurious rmr failure messages when inserting with dynamic partitioning * [HIVE-1828] - show locks should not use getTable()/getPartition * [HIVE-1829] - Fix intermittent failures in TestRemoteMetaStore * [HIVE-1830] - mappers in group followed by joins may die OOM * [HIVE-1844] - Hanging hive client caused by TaskRunner's OutOfMemoryError * [HIVE-1845] - Some attributes in the Eclipse template file is deprecated * [HIVE-1846] - change hive assumption that local mode mappers/reducers always run in same jvm * [HIVE-1848] - bug in MAPJOIN * [HIVE-1849] - add more logging to partition pruning * [HIVE-1853] - downgrade JDO version * [HIVE-1854] - Temporarily disable metastore tests for listPartitionsByFilter() * [HIVE-1857] - mixed case tablename on lefthand side of LATERAL VIEW results in query failing with confusing error message * [HIVE-1860] - Hive's smallint datatype is not supported by the Hive JDBC driver * [HIVE-1861] - Hive's float datatype is not supported by the Hive JDBC driver * [HIVE-1862] - Revive partition filtering in the Hive MetaStore * [HIVE-1863] - Boolean columns in Hive tables containing NULL are treated as FALSE by the Hive JDBC driver. * [HIVE-1864] - test load_overwrite.q fails * [HIVE-1867] - Add mechanism for disabling tests with intermittent failures * [HIVE-1870] - TestRemoteHiveMetaStore.java accidentally deleted during commit of HIVE-1845 * [HIVE-1871] - bug introduced by HIVE-1806 * [HIVE-1873] - Fix 'tar' build target broken in HIVE-1526 * [HIVE-1874] - fix HBase filter pushdown broken by HIVE-1638 * [HIVE-1878] - Set the version of Hive trunk to '0.7.0-SNAPSHOT' to avoid confusing it with a release * [HIVE-1896] - HBase and Contrib JAR names are missing version numbers * [HIVE-1897] - Alter command execution "when HDFS is down" results in holding stale data in MetaStore * [HIVE-1902] - create script for the metastore upgrade due to HIVE-78 * [HIVE-1903] - Can't join HBase tables if one's name is the beginning of the other * [HIVE-1908] - FileHandler leak on partial iteration of the resultset. * [HIVE-1912] - Double escaping special chars when removing old partitions in rmr * [HIVE-1913] - use partition level serde properties * [HIVE-1914] - failures in testhbaseclidriver * [HIVE-1915] - authorization on database level is broken. * [HIVE-1917] - CTAS (create-table-as-select) throws exception when showing results * [HIVE-1927] - Fix TestHadoop20SAuthBridge failure on Hudson * [HIVE-1928] - GRANT/REVOKE should handle privileges as tokens, not identifiers * [HIVE-1934] - alter table rename messes the location * [HIVE-1936] - hive.semantic.analyzer.hook cannot have multiple values * [HIVE-1939] - Fix test failure in TestContribCliDriver/url_hook.q * [HIVE-1944] - dynamic partition insert creating different directories for the same partition during merge * [HIVE-1951] - input16_cc.q is failing in testminimrclidriver * [HIVE-1952] - fix some outputs and make some tests deterministic * [HIVE-1964] - add fully deterministic ORDER BY in test union22.q and input40.q * [HIVE-1969] - TestMinimrCliDriver merge_dynamic_partition2 and 3 are failing on trunk * [HIVE-1979] - fix hbase_bulk.m by setting HiveInputFormat * [HIVE-1981] - TestHadoop20SAuthBridge failed on current trunk * [HIVE-1995] - Mismatched open/commit transaction calls when using get_partition() * [HIVE-1998] - Update README.txt and add missing ASF headers * [HIVE-2007] - Executing queries using Hive Server is not logging to the log file specified in hive-log4j.properties * [HIVE-2010] - Improve naming and README files for MetaStore upgrade scripts * [HIVE-2011] - upgrade-0.6.0.mysql.sql script attempts to increase size of PK COLUMNS.TYPE_NAME to 4000 * [HIVE-2059] - Add datanucleus.identifierFactory property to HiveConf to avoid unintentional MetaStore Schema corruption * [HIVE-2064] - Make call to SecurityUtil.getServerPrincipal unambiguous ** Sub-task * [HIVE-1361] - table/partition level statistics * [HIVE-1696] - Add delegation token support to metastore * [HIVE-1810] - a followup patch for changing the description of hive.exec.pre/post.hooks in conf/hive-default.xml * [HIVE-1823] - upgrade the database thrift interface to allow parameters key-value pairs * [HIVE-1836] - Extend the CREATE DATABASE command with DBPROPERTIES * [HIVE-1842] - Add the local flag to all the map red tasks, if the query is running locally. ** Task * [HIVE-1526] - Hive should depend on a release version of Thrift * [HIVE-1817] - Remove Hive dependency on unreleased commons-cli 2.0 Snapshot * [HIVE-1876] - Update Metastore upgrade scripts to handle schema changes introduced in HIVE-1413 * [HIVE-1882] - Remove CHANGES.txt * [HIVE-1904] - Create MetaStore schema upgrade scripts for changes made in HIVE-417 * [HIVE-1905] - Provide MetaStore schema upgrade scripts for changes made in HIVE-1823 ** Test * [HIVE-1464] - improve test query performance * [HIVE-1755] - JDBM diff in test caused by Hive-1641 * [HIVE-1774] - merge_dynamic_part's result is not deterministic * [HIVE-1942] - change the value of hive.input.format to CombineHiveInputFormat for tests Release Notes - Hive - Version 0.6.0 ** New Feature * [HIVE-259] - Add PERCENTILE aggregate function * [HIVE-675] - add database/schema support Hive QL * [HIVE-705] - Hive HBase Integration (umbrella) * [HIVE-801] - row-wise IN would be useful * [HIVE-862] - CommandProcessor should return DriverResponse * [HIVE-894] - add udaf max_n, min_n to contrib * [HIVE-917] - Bucketed Map Join * [HIVE-972] - support views * [HIVE-1002] - multi-partition inserts * [HIVE-1027] - Create UDFs for XPath expression evaluation * [HIVE-1032] - Better Error Messages for Execution Errors * [HIVE-1087] - Let user script write out binary data into a table * [HIVE-1121] - CombinedHiveInputFormat for hadoop 19 * [HIVE-1127] - Add UDF to create struct * [HIVE-1131] - Add column lineage information to the pre execution hooks * [HIVE-1132] - Add metastore API method to get partition by name * [HIVE-1134] - bucketing mapjoin where the big table contains more than 1 big partition * [HIVE-1178] - enforce bucketing for a table * [HIVE-1179] - Add UDF array_contains * [HIVE-1193] - ensure sorting properties for a table * [HIVE-1194] - sorted merge join * [HIVE-1197] - create a new input format where a mapper spans a file * [HIVE-1219] - More robust handling of metastore connection failures * [HIVE-1238] - Get partitions with a partial specification * [HIVE-1255] - Add mathematical UDFs PI, E, degrees, radians, tan, sign, and atan * [HIVE-1270] - Thread pool size in Thrift metastore server should be configurable * [HIVE-1272] - Add SymlinkTextInputFormat to Hive * [HIVE-1278] - Partition name to values conversion conversion method * [HIVE-1307] - More generic and efficient merge method * [HIVE-1332] - Archiving partitions * [HIVE-1351] - Tool to cat rcfiles * [HIVE-1397] - histogram() UDAF for a numerical column * [HIVE-1401] - Web Interface can ony browse default * [HIVE-1410] - Add TCP keepalive option for the metastore server * [HIVE-1439] - Alter the number of buckets for a table ** Bug * [HIVE-287] - support count(*) and count distinct on multiple columns * [HIVE-763] - getSchema returns invalid column names, getThriftSchema does not return old style string schemas * [HIVE-1011] - GenericUDTFExplode() throws NPE when given nulls * [HIVE-1022] - desc Table should work * [HIVE-1029] - typedbytes does not support nulls * [HIVE-1042] - function in a transform with more than 1 argument fails * [HIVE-1056] - Predicate push down does not work with UDTF's * [HIVE-1064] - NPE when operating HiveCLI in distributed mode * [HIVE-1066] - TestContribCliDriver failure in serde_typedbytes.q, serde_typedbytes2.q, and serde_typedbytes3.q * [HIVE-1075] - Make it possible for users to recover data when moveTask fails * [HIVE-1085] - ColumnarSerde should not be the default Serde when user specified a fileformat using 'stored as'. * [HIVE-1086] - Add "-Doffline=true" option to ant * [HIVE-1090] - Skew Join does not work in distributed env. * [HIVE-1092] - Conditional task does not increase finished job counter when filter job out. * [HIVE-1094] - Disable streaming last table if there is a skew key in previous tables. * [HIVE-1116] - bug with alter table rename when table has property EXTERNAL=FALSE * [HIVE-1124] - create view should expand the query text consistently * [HIVE-1125] - Hive CLI shows 'Ended Job=' at the beginning of the job * [HIVE-1129] - Assertion in ExecDriver.execute when assertions are enabled in HADOOP_OPTS * [HIVE-1142] - "datanucleus" typos in conf/hive-default.xml * [HIVE-1167] - Use TreeMap instead of Property to make explain extended deterministic * [HIVE-1174] - Job counter error if "hive.merge.mapfiles" equals true * [HIVE-1176] - 'create if not exists' fails for a table name with 'select' in it * [HIVE-1184] - Expression Not In Group By Key error is sometimes masked * [HIVE-1185] - Fix RCFile resource leak when opening a non-RCFile * [HIVE-1195] - Increase ObjectInspector[] length on demand * [HIVE-1200] - Fix CombineHiveInputFormat to work with multi-level of directories in a single table/partition * [HIVE-1204] - typedbytes: writing to stderr kills the mapper * [HIVE-1205] - RowContainer should flush out dummy rows when the table desc is null * [HIVE-1207] - ScriptOperator AutoProgressor does not set the interval * [HIVE-1242] - CombineHiveInputFormat does not work for compressed text files * [HIVE-1247] - hints cannot be passed to transform statements * [HIVE-1252] - Task breaking bug when breaking after a filter operator * [HIVE-1253] - date_sub() function returns wrong date because of daylight saving time difference * [HIVE-1257] - joins between HBase tables and other tables (whether HBase or not) are broken * [HIVE-1258] - set merge files to files when bucketing/sorting is being enforced * [HIVE-1261] - ql.metadata.Hive#close() should check for null metaStoreClient * [HIVE-1268] - Cannot start metastore thrift server on a specific port * [HIVE-1271] - Case sensitiveness of type information specified when using custom reducer causes type mismatch * [HIVE-1273] - UDF_Percentile NullPointerException * [HIVE-1274] - bug in sort merge join if the big table does not have any row * [HIVE-1275] - TestHBaseCliDriver hangs * [HIVE-1277] - Select query with specific projection(s) fails if the local file system directory for ${hive.user.scratchdir} does not exist. * [HIVE-1280] - problem in combinehiveinputformat with nested directories * [HIVE-1281] - Bucketing column names in create table should be case-insensitive * [HIVE-1286] - error/info message being emitted on standard output * [HIVE-1290] - sort merge join does not work with bucketizedhiveinputformat * [HIVE-1291] - Fix UDAFPercentile ndexOutOfBoundsException * [HIVE-1294] - HIVE_AUX_JARS_PATH interferes with startup of Hive Web Interface * [HIVE-1298] - unit test symlink_text_input_format.q needs ORDER BY for determinism * [HIVE-1308] - = throws NPE * [HIVE-1311] - bug is use of hadoop supports splittable * [HIVE-1312] - hive trunk does not compile with hadoop 0.17 any more * [HIVE-1315] - bucketed sort merge join breaks after dynamic partition insert * [HIVE-1317] - CombineHiveInputFormat throws exception when partition name contains special characters to URI * [HIVE-1320] - NPE with lineage in a query of union alls on joins. * [HIVE-1321] - bugs with temp directories, trailing blank fields in HBase bulk load * [HIVE-1322] - Cached FileSystem can lead to persistant IOExceptions * [HIVE-1323] - leading dash in partition name is not handled properly * [HIVE-1325] - dynamic partition insert should throw an exception if the number of target table columns + dynamic partition columns does not equal to the number of select columns * [HIVE-1326] - RowContainer uses hard-coded '/tmp/' path for temporary files * [HIVE-1327] - Group by partition column returns wrong results * [HIVE-1330] - fatal error check omitted for reducer-side operators * [HIVE-1331] - select * does not work if different partitions contain different formats * [HIVE-1338] - Fix bin/ext/jar.sh to work with hadoop 0.20 and above * [HIVE-1341] - Filter Operator Column Pruning should preserve the column order * [HIVE-1345] - TypedBytesSerDe fails to create table with multiple columns. * [HIVE-1350] - hive.query.id is not unique * [HIVE-1352] - rcfilecat should use '\t' to separate columns and print '\r\n' at the end of each row. * [HIVE-1353] - load_dyn_part*.q tests need ORDER BY for determinism * [HIVE-1354] - partition level properties honored if it exists * [HIVE-1364] - Increase the maximum length of various metastore fields, and remove TYPE_NAME from COLUMNS primary key * [HIVE-1365] - Bug in SMBJoinOperator which may causes a final part of the results in some cases. * [HIVE-1366] - inputFileFormat error if the merge job takes a different input file format than the default output file format * [HIVE-1371] - remove blank in rcfilecat * [HIVE-1373] - Missing connection pool plugin in Eclipse classpath * [HIVE-1377] - getPartitionDescFromPath() in CombineHiveInputFormat should handle matching by path * [HIVE-1388] - combinehiveinputformat does not work if files are of different types * [HIVE-1403] - Reporting progress to JT during closing files in FileSinkOperator * [HIVE-1407] - Add hadoop-*-tools.jar to Eclipse classpath * [HIVE-1409] - File format information is retrieved from first partition * [HIVE-1411] - DataNucleus throws NucleusException if core-3.1.1 JAR appears more than once on CLASSPATH * [HIVE-1412] - CombineHiveInputFormat bug on tablesample * [HIVE-1417] - Archived partitions throw error with queries calling getContentSummary * [HIVE-1418] - column pruning not working with lateral view * [HIVE-1420] - problem with sequence and rcfiles are mixed for null partitions * [HIVE-1421] - problem with sequence and rcfiles are mixed for null partitions * [HIVE-1425] - hive.task.progress should be added to conf/hive-default.xml * [HIVE-1428] - ALTER TABLE ADD PARTITION fails with a remote Thrift metastore * [HIVE-1435] - Upgraded naming scheme causes JDO exceptions * [HIVE-1448] - bug in 'set fileformat' * [HIVE-1454] - insert overwrite and CTAS fail in hive local mode * [HIVE-1455] - lateral view does not work with column pruning * [HIVE-1492] - FileSinkOperator should remove duplicated files from the same task based on file sizes * [HIVE-1524] - parallel execution failed if mapred.job.name is set * [HIVE-1594] - Typo of hive.merge.size.smallfiles.avgsize prevents change of value * [HIVE-1613] - hive --service jar looks for hadoop version but was not defined * [HIVE-1615] - Web Interface JSP needs Refactoring for removed meta store methods * [HIVE-1681] - ObjectStore.commitTransaction() does not properly handle transactions that have already been rolled back * [HIVE-1697] - Migration scripts should increase size of PARAM_VALUE in PARTITION_PARAMS ** Improvement * [HIVE-543] - provide option to run hive in local mode * [HIVE-964] - handle skewed keys for a join in a separate job * [HIVE-990] - Incorporate CheckStyle into Hive's build.xml * [HIVE-1047] - Merge tasks in GenMRUnion1 * [HIVE-1068] - CREATE VIEW followup: add a "table type" enum attribute in metastore's MTable, and also null out irrelevant attributes for MTable instances which describe views * [HIVE-1069] - CREATE VIEW followup: find and document current expected version of thrift, and regenerate code to match * [HIVE-1093] - Add a "skew join map join size" variable to control the input size of skew join's following map join job. * [HIVE-1102] - make number of concurrent tasks configurable * [HIVE-1108] - QueryPlan to be independent from BaseSemanticAnalyzer * [HIVE-1109] - Structured temporary directories * [HIVE-1110] - add counters to show that skew join triggered * [HIVE-1117] - Make QueryPlan serializable * [HIVE-1118] - Add hive.merge.size.per.task to HiveConf * [HIVE-1119] - Make all Tasks and Works serializable * [HIVE-1120] - In ivy offline mode, don't delete downloaded jars * [HIVE-1122] - Make ql/metadata/Table and Partition serializable * [HIVE-1128] - Let max/min handle complex types like struct * [HIVE-1136] - add type-checking setters for HiveConf class to match existing getters * [HIVE-1144] - CREATE VIEW followup: support ALTER TABLE SET TBLPROPERTIES on views * [HIVE-1150] - Add comment to explain why we check for dir first in add_partitions(). * [HIVE-1152] - Add metastore API method to drop partition / append partition by name * [HIVE-1164] - drop_partition_by_name() should use drop_partition_common() * [HIVE-1190] - Configure build to download Hadoop tarballs from Facebook mirror instead of Apache * [HIVE-1198] - When checkstyle is activated for Hive in Eclipse environment, it shows all checkstyle problems as errors. * [HIVE-1212] - Explicitly say "Hive Internal Error" to ease debugging * [HIVE-1216] - Show the row with error in mapper/reducer * [HIVE-1220] - accept TBLPROPERTIES on CREATE TABLE/VIEW * [HIVE-1228] - allow HBase key column to be anywhere in Hive table * [HIVE-1241] - add pre-drops in bucketmapjoin*.q * [HIVE-1244] - add backward-compatibility constructor to HiveMetaStoreClient * [HIVE-1246] - mapjoin followed by another mapjoin should be performed in a single query * [HIVE-1260] - from_unixtime should implment a overloading function to accept only bigint type * [HIVE-1276] - optimize bucketing * [HIVE-1295] - facilitate HBase bulk loads from Hive * [HIVE-1296] - CLI set and set -v commands should dump properties in alphabetical order * [HIVE-1297] - error message in Hive.checkPaths dumps Java array address instead of path string * [HIVE-1300] - support: alter table touch partition * [HIVE-1306] - cleanup the jobscratchdir * [HIVE-1316] - Increase the memory limit for CLI client * [HIVE-1328] - make mapred.input.dir.recursive work for select * * [HIVE-1329] - for ALTER TABLE t SET TBLPROPERTIES ('EXTERNAL'='TRUE'), change TBL_TYPE attribute from MANAGED_TABLE to EXTERNAL_TABLE * [HIVE-1335] - DataNucleus should use connection pooling * [HIVE-1348] - Moving inputFileChanged() from ExecMapper to where it is needed * [HIVE-1349] - Do not pull counters of non initialized jobs * [HIVE-1355] - Hive should use NullOutputFormat for hadoop jobs * [HIVE-1357] - CombineHiveInputSplit should initialize the inputFileFormat once for a single split * [HIVE-1372] - New algorithm for variance() UDAF * [HIVE-1383] - allow HBase WAL to be disabled * [HIVE-1387] - Add PERCENTILE_APPROX which works with double data type * [HIVE-1531] - Make Hive build work with Ivy versions < 2.1.0 * [HIVE-1543] - set abort in ExecMapper when Hive's record reader got an IOException * [HIVE-1693] - Make the compile target depend on thrift.home ** Task * [HIVE-1081] - Automated source code cleanup * [HIVE-1084] - Cleanup Class names * [HIVE-1103] - Add .gitignore file * [HIVE-1104] - Suppress Checkstyle warnings for generated files * [HIVE-1112] - Replace instances of StringBuffer/Vector with StringBuilder/ArrayList * [HIVE-1123] - Checkstyle fixes * [HIVE-1135] - Use Anakia for version controlled documentation * [HIVE-1137] - build references IVY_HOME incorrectly * [HIVE-1147] - Update Eclipse project configuration to match Checkstyle * [HIVE-1163] - Eclipse launchtemplate changes to enable debugging * [HIVE-1256] - fix Hive logo img tag to avoid stretching * [HIVE-1427] - Provide metastore schema migration scripts (0.5 -> 0.6) * [HIVE-1709] - Provide Postgres metastore schema migration scripts (0.5 -> 0.6) * [HIVE-1725] - Include metastore upgrade scripts in release tarball * [HIVE-1726] - Update README file for 0.6.0 release * [HIVE-1729] - Satisfy ASF release management requirements ** Sub-task * [HIVE-1340] - checking VOID type for NULL in LazyBinarySerde ** Test * [HIVE-1188] - NPE when running TestJdbcDriver/TestHiveServer * [HIVE-1236] - test HBase input format plus CombinedHiveInputFormat * [HIVE-1279] - temporarily disable HBase test execution * [HIVE-1359] - Unit test should be shim-aware