Log Message: |
HIVE-2666 [jira] StackOverflowError when using custom UDF in map join
(Kevin Wilfong via Yongqiang He)
Summary:
Resource files are now added to the class path as soon as they are added via the
CLI. This fixes the stack overflow error mentioned in the JIRA by ensuring a
consistent class loader between serializers and deserializers for the same
query.
Note that now serdes which contain a static block to register themselves are now
registered twice, once when adding the file to the class loader, and once when
an instance of the class is created. Previously, registering a serde twice
resulted in an exception, to avoid this, I have downgraded it to a warning.
When a custom UDF is used as part of a join which is converted to a map join,
the XMLEncoder enters an infinite loop when serializing the map reduce task for
the second time, as part of sending it to be executed. This results in a stack
overflow error.
Test Plan:
I ran the unit tests to verify nothing was broken.
I ran several queries which used custom UDFs and involved a join which was
converted to a map join. I verified these completed successfully consistently
Reviewers: JIRA, heyongqiang
Reviewed By: heyongqiang
CC: heyongqiang, kevinwilfong
Differential Revision: 957
|