Table of Contents

Copyright/License Notice
Preface
Chapter 1: Introduction
Chapter 2: Procedure for Portlet 1.0 Bridge for JavaServer™ Faces 1.2 Certification
Chapter 3: Requirements
Chapter 4: Installation
Chapter 5: TCK Project Structure
Chapter 6: Testing a Vendor Implementation
Chapter 7: Testing a Vendor Implementation in a Pluto Environment
Chapter 8: Verifying the Reference Implementation in your Environment
Chapter 9: TCK Command Reference
Chapter 10: Debugging Test Problems
Appendix A: Frequently Asked Questions





Copyright/License Notice





Portlet 1.0 Bridge for JavaServer™ Faces 1.2
Compatibility Kit User’s Guide
For Technology Licensees
Version1.0
May 2010

Copyright 2010 Apache Software Foundation

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

[return to TOC]






Preface





This guide describes how to install, configure, and run the Technology Compatibility Kit (TCK) that provides tests for the Portlet 1.0 Bridge for JavaServer™ Faces 1.2 (Bridge TCK). The Bridge TCK is designed as a portable, configurable automated test suite for verifying the compliance of a licensee’s implementation of the Portlet 1.0 Bridge for JavaServer™ Faces 1.2 Specification (hereafter referred to as a licensee implementation). It may also be used to verify that any compatible version of the bridge including the Reference Implementation runs correctly in a given JavaEE, Portlet container and Faces environment.  The Bridge TCK runs as a Maven project located under Subversion source control in  the Apache MyFaces PortletBridge project.  It relies on JUnit and Selenium to provide the necessary test harness and reporting tools to run the test suite.

Who Should Use This Book

This guide is for licensees of Oracle Corporation's Portlet 1.0 Bridge for JavaServer™ Faces 1.2 technology to assist them in running the test suite that verifies compliance of their
implementation of the Portlet 1.0 Bridge for JavaServer™ Faces 1.2 Specification.

NOTE All references to specific Web URLs are given for your convenience in locating the resources quickly. These references are always subject to changes that are in many cases beyond the control of the authors of this guide.

Before You Read This Book

Before reading this guide, you should familiarize yourself with the Java programming language, Portlet 1.0 standard (JSR 168),  JavaServer Faces 1.2 standard (JSR 252)  and the Portlet 1.0 Bridge for JavaServer™ Faces 1.2 Specification ( JSR 301). The Bridge TCK 1.0 is based on the JSR 301 Specification. Links to the specification and other product information can be found on the Web at: http://www.jcp.org/en/jsr/detail?id=301

How This Book Is Organized

If you are installing and using the Bridge TCK for the first time, it is recommended that you read chapters 1, 2, and 3 completely for the necessary background information, and then perform the steps outlined in chapters 4, 5 or 6, and 7, while referring to chapters 8, 9, 10, and the appendix as necessary.

Chapter 1, “Introduction,” gives an overview of the principles that apply generally to all Technology Compatibility Kits (TCKs) and describes the Bridge TCK. It also
includes a listing of the basic steps needed to get up and running with the Bridge TCK.

Chapter 2, “Procedure for Portlet 1.0 Bridge for JavaServertm Faces 1.2 Certification,” describes the conformance testing procedure and testing requirements.

Chapter 3, “Requirements,” lists the hardware and software requirements that must be met before the Bridge Compatibility Test Suite can be run.

Chapter 4, “Installation,” explains how to install Bridge TCK on machines that run the Solaris, Linux, and Windows XP/2000 operating systems.

Chapter 5, “TCK Project Structure,” explains how the TCK is organized in its various directories.

Chapter 6, “Testing a Vendor Implementation,” explains how to setup and run test suite using your bridge implementation in your non-Pluto based test server.

Chapter 7, “Testing a Vendor Implementation in a Pluto Environment,” explains how to setup and run test suite using your bridge implementation in a Pluto based test server.

Chapter 8, “Verifying the Reference Implementation in your Environment,” explains how to setup and run test suite using bridge reference implementation in your non-Pluto based test server

Chapter 9, "TCK Command Reference," lists each of the properties which can be set to control the behavior of a particular test run.

Chapter 10, “Debugging Test Problems,” describes several approaches for dealing with tests that fail to execute properly.

Appendix A, “Frequently Asked Questions,” provides answers to frequently asked questions.

[return to TOC]

Chapter 1  

Introduction  




        
This chapter provides an overview of the principles that apply generally to all Technology Compatibility Kits (TCKs) and describes the Portlet 1.0 Bridge for JavaServer™ Faces 1.2
Compatibility Kit (Bridge TCK). It also includes a listing of what is needed to get up and running with the Bridge TCK.

Compatibility Testing

Compatibility testing differs from traditional product testing in a number of ways. The focus of compatibility testing is to test those features and areas of an implementation that are likely to differ across implementations. It attempts to ensure that users of any given implementation will receive the same results regardless of the underlying implementation strategy. Compatibility test development for a given feature relies on a complete specification and reference implementation for that feature. Compatibility testing is not primarily concerned with robustness, performance, or ease of use.

Why Compatibility Testing is Important

Java™ platform compatibility is important to different groups involved with Java technologies for different reasons:

TCK Compatibility Rules

Compatibility criteria for all technology implementations are embodied in the TCK Compatibility Rules that apply to a specified technology. Each TCK tests for adherence to these Rules as described in Chapter 2, “Procedure for Portlet 1.0 Bridge for JavaServer™ Faces 1.2 Certification.”

TCK Overview

A TCK is a set of tools and tests used to verify that a licensee’s implementation of Oracle Corporation's  Portlet 1.0 Bridge for JavaServer™ Faces 1.2 technology conforms to the applicable
specification. All tests in the TCK are based on the written specifications for the Java platform. Compatibility testing is a means of ensuring correctness, completeness, and consistency across all implementations. The set of tests included with each TCK is called the “test suite.” All tests in the Bridge TCK’s test suite are self-checking and do not require tester interaction. Most
tests return either a Pass or Fail status. For a given licensee’s implementation to be certified, all of the required tests must pass. The definition of required tests will change over time. Before your final
certification test pass, be sure to download the latest Excludes File for the TCK you are using from the TCK site (http://wiki.apache.org/myfaces/PortletBridge).

Java Community Process (JCP) Program and Compatibility Testing

The Java Community Process(SM) (JCP(SM)) program is the formalization of the open process that Sun Microsystems, Inc. has been using since 1995 to develop and
revise Java technology specifications in cooperation with the international Java community. The JCP program specifies that the following three major components must be included as deliverables in a final Java technology release under the direction of the responsible Expert Group:
For further information on the JCP program see this URL: http://www.jcp.org.

The Bridge TCK

The Bridge TCK 1.0 is designed as a portable, configurable, automated test suite for verifying the compliance of a licensee’s implementation with Oracle Corporation's
Portlet 1.0 Bridge for JavaServer™ Faces 1.2 specification.

Bridge TCK Specifications and Requirements

This section lists the applicable requirements and specifications.
• Specification Requirements – Software requirements for a Bridge
implementation are described in detail in the Portlet 1.0 Bridge for JavaServer™ Faces 1.2 Specification.
Links to the JSR 301 specification and other product information can be found at http://www.jcp.org/en/jsr/detail?id=301.
• Reference Implementation – The designated Reference Implementation for
conformance testing of implementations based upon JSR 301 Specification
is Apache Open Source Project, MyFaces Portlet Bridge 1.0 (http://myfaces.apache.org/portlet-bridge/1.0/index.html).

The Bridge TCK

The Bridge TCK is composed of :
The Bridge TCK 1.0 is a maven project maintained in Subversion source control as part of the Apache MyFaces Portlet Bridge project.  The TCK test stuite is built and run using maven commands.  The test harness, used to automate executing the TCK tests uses JUnit and Selenium.  Maven automatically accesses the necessary versions and runs them during the appropriate phase of the lifecycle. More information can be found on the web:

Typical Usage

Each TCK test is implemented in its own specific portlet built from the test suite source.  Most test portlets are grouped into a single portlet application.  However, because one test involves varying values in the application's web.xml, the entire test suite is comprised of two portlet applications:  a main testsuite portlet application containing most of the test portlets, and an additional testsuite portlet application containing an individual test portlet with a distinct (web.xml) configuration.  Each of these test suite portlet applications is built using a maven command.  As described in detail later, the command used to generate the war files includes flexibility for:
Once generated, the testsuite wars are deployed to the host application server.  The test server(s) must not only contain a Portlet 1.0 (compatible) container to host the portlet application but must also include a technology for accessing and executing each portlet in these applications by URL.  Typically, this involves a portal application hosting a set of portal pages where each page contains a distinct test portlet.

Note:  The test harness requires a distinct URL to execute each test.  In most cases this translates into placing each test portlet on a distinct page.

Once the test applications are deployed and made accesible by URL, the TCK is executed by running a second maven command.  This command reads a test file containing the test URLs (provided by you) and executes each test in the file via the JUnit/Selenium harness.  Upon completion a report is generated providing summary and detailed information of the test results.

Note:  Prior to running the automated test harness you are encouraged to verify your portal/test suite configuration by invoking some of the test pages directly from a browser.  Thes tests are designed to ouput valid HTML indicating the test status along with the internal tags used by the harness.

TCK Compatibility Test Suite

The test suite is the collection of tests used by the test harness to test the compliance of a given Bridge in a given runtime environment. The tests are designed to verify that a licensee’s implementation of the technology complies with the appropriate specification. The individual tests correspond to assertions of the specification. The descriptions of tests that make up the TCK compatibility test suite are
included in the projects root directory in a file called "TCK Tests.html".

The Bridge TCK 1.0 test suite comprises two test categories:

Exclude Lists

Each version of a TCK includes an Exclude List file. This file identifies tests which do not have to be run as their results may not be repeatable in all environments. Whenever tests are run, the test harness automatically excludes any test on the Exclude List from being executed.

A licensee is not required to pass any test—or even run any test—on the Exclude List.  However, as many of these tests are excluded because of specific environmental limitations, licensee's are encouraged to review the comments in the excluded file to determine if the test can be successfully run in he or her environment, and if so, to include the test to verify proper behavior.


Note: Licensees are not permitted to alter or modify the Exclude Lists.  Changes to an Exclude List can only be made by using the procedure described in “Portlet Test Appeals Process”.

Bridge TCK—Getting Started

This section provides a general overview of what needs to be done to install, set up, test, and use the Bridge TCK:
  1. Make sure that the following software has been correctly installed:
    Note: Consult the documentation for each of these software applications for installation instructions.
  2. Install the  TCK 1.0 software.  
  3. Build the TCK portlet applications using Maven.
  4. Deploy the generated applications to your application server.
  5. Construct individual test pages, one for each test, using the installed portal.
  6. Verify the test pages/tests by manually accessing a few pages.
  7. Generate a test.xml test file containing the URLs used to access each test.
  8. (Optionally) generate a login.properties file containing the authentication name/password the harness passes to the portal when challenged.
  9. Run the TCK test harness by issuing the appropriate maven command.
  10. Review the results.
  11. Debug problems, fix and retry above until all tests pass.
[return to TOC]











Chapter 2

Procedure for Portlet 1.0 Bridge for JavaServer™ Faces 1.2 Certification





This chapter describes the compatibility testing procedure and compatibility requirements for Portlet 1.0 Bridge for JavaServer™ Faces 1.2 certification.

Certification Overview

Compatibility Requirements

Definitions

These definitions are for use only with these compatibility requirements and are not intended for any other purpose.

Term Definition
Computational Resource A piece of hardware or software that may vary in quantity, existence, or version, which may be required to exist in a minimum quantity and/or at a specific or minimum revision level so as to satisfy the requirements of the Test Suite. Examples of computational resources that may vary in quantity are RAM and file descriptors. Examples of computational resources that may vary in existence (this is, may exist or not) are graphics cards and device drivers. Examples of computational resources that may vary in version are operating systems and device drivers.
Conformance Tests All tests in the Test Suite for an indicated Technology Under Test, as distributed by the Maintenance Lead, excluding those tests on the Exclude List for the Technology Under Test.
Container An implementation of the associated Libraries, as specified in the Specifications, and a version of a J2SE Runtime Product, as specified in the Specifications, or a later version of a J2SE Runtime Product that also meets these compatibility requirements.
Documented Made technically accessible and made known to users, typically by means such as marketing materials, product documentation, usage messages, or developer support programs.
Exclude List  The most current list of tests, distributed by the Maintenance Lead, that are not required to be passed to certify conformance. The Maintenance Lead may add to the Exclude List for that Test Suite as needed at any time, in which case the updated Exclude List supplants any previous Exclude Lists for that Test Suite.
Libraries The class libraries, as specified through the Java Community ProcessSM (JCPSM), for the Technology Under Test.

The Libraries for Portlet 1.0 Bridge are listed at the end of this chapter.
Location Resource A location of classes or native libraries that are components of the test tools or tests, such that these classes or libraries may be required to exist in a certain location in order to satisfy the requirements of the test suite. For example, classes may be required to exist in directories named in a CLASSPATH variable, or native libraries may be required to exist in directories named in a PATH variable.
Maintenance Lead The JCP member responsible for maintaining the Specification, reference implementation, and TCK for the Technology. Oracle Corportation is the Maintenance Leads for Portlet Bridge 1.0.
Operating Mode Any Documented option of a Product that can be changed by a user in order to modify the behavior of the Product. For example, an Operating Mode of a Runtime can be binary (enable/disable optimization), an enumeration (select from a list of localizations), or a range (set the initial Runtime heap size).  
Product A licensee product in which the Technology Under Test is implemented or incorporated, and that is subject to compatibility testing.
Product Configuration A specific setting or instantiation of an Operating Mode.

For example, a Runtime supporting an Operating Mode that permits selection of an initial heap size might have a Product Configuration that sets the initial heap size to 1 Mb.
Resource A Computational Resource, a Location Resource, or a Security Resource.  
Rules These definitions and rules in this Compatibility Requirements section of this User’s Guide.
Runtime The Containers specified in the Specifications.
Security Resource A security privilege or policy necessary for the proper execution of the Test Suite.

For example, the user executing the Test Suite will need the privilege to access the files and network resources necessary for use of the Product.
Specifications The documents produced through the JCP that define a particular Version of a Technology.

The Specifications for the Technology Under Test can be found later in this chapter.
Technology Specifications and a reference implementation produced through the JCP.
Technology Under Test Specifications and the reference implementation for Portlet 1.0.
Test Suite The requirements, tests, and testing tools distributed by the Maintenance Lead as applicable to a given Version of the Technology.
Version A release of the Technology, as produced through the JCP.

 

 Rules for Bridge Products

PLT1 The Product must be able to satisfy all applicable compatibility requirements, including passing all Conformance Tests, in every Product Configuration and in every combination of Product Configurations, except only as specifically exempted by these Rules.

For example, if a Product provides distinct Operating Modes to optimize performance, then that Product must satisfy all applicable compatibility requirements for a Product in each Product Configuration, and combination of Product Configurations, of those Operating Modes.
PLT1.1 If an Operating Mode controls a Resource necessary for the basic execution of the Test Suite, testing may always use a Product Configuration of that Operating Mode providing that Resource, even if other Product Configurations do not provide that Resource. Notwithstanding such exceptions, each Product must have at least one set of Product Configurations of such Operating Modes that is able to pass all the Conformance Tests.

For example, a Product with an Operating Mode that controls a security policy (i.e., Security Resource) which has one or more Product Configurations that cause Conformance Tests to fail may be tested using a Product Configuration that allows all Conformance Tests to pass.
PLT1.2 A Product Configuration of an Operating Mode that causes the Product to report only version, usage, or diagnostic information is exempted from these compatibility rules.
PLT2 Some Conformance Tests may have properties that may be changed (currently there are no such tests in this TCK). Apart from changing such properties and other allowed modifications described in this User’s Guide, no source or binary code for a Conformance Test may be altered in any way without prior written permission. Any such allowed alterations to the Conformance Tests would be posted to the Java Partner Engineering web site and apply to all licensees.
PLT3 The testing tools supplied as part of the Test Suite or as updated by the Maintenance Lead must be used to certify compliance.
PLT4 The Exclude List associated with the Test Suite cannot be modified.
PLT5 The Maintenance Lead can define exceptions to these Rules. Such exceptions would be made available to and apply to all licensees.
PLT6 All hardware and software component additions, deletions, and modifications to a Documented supporting hardware/software platform, that are not part of the Product but required for the Product to satisfy the compatibility requirements, must be Documented and available to users of the Product.

For example, if a patch to a particular version of a supporting operating system is required for the Product to pass the Conformance Tests, that patch must be Documented and available to users of the Product.
PLT7 The Product must contain the full set of public and protected classes and interfaces for all the Libraries. Those classes and interfaces must contain exactly the set of public and protected methods, constructors, and fields defined in the Specifications for those Libraries. No subsetting, supersetting, or modifications of the public and protected API of the Libraries are allowed except only as specifically exempted by these Rules.
PLT7.1 If a Product includes Technologies in addition to the Technology Under Test, then it must contain the full set of combined public and protected classes and interfaces. The API of the Product must contain the union of the included Technologies. No further subsetting, supersetting, or modifications to the APIs of the included Technologies are allowed.
PLT8 The functional programmatic behavior of any binary class or interface must be that defined by the Specifications.

Portlet Test Appeals Process

Oracle Corportation has a well established process for managing challenges to its Portlet 1.0 Bridge Test Suite and plans to continue using a similar process in the future. Oracle Corporation, as Maintenance Lead, will authorize representatives from its engineering team to be the point of contact for all test challenges.

Process Used to Manage Challenges to Bridge Tests:

The following process will be used to manage challenges to Portlet 1.0 Bridge tests:
  1. Who can make challenges to the  Portlet 1.0 Bridge tests?

    Anyone, though you must create an Apache JIRA account to submit challenges.

  2. What challenges to the Portlet 1.0 Bridge tests may be submitted?

    Individual (or related) tests may be challenged for reasons such as:
  3. How are challenges submitted?

    By filing an Apache JIRA ticket (https://issues.apache.org/jira/secure/Dashboard.jspa).
    The JIRA form must be filed against the MyFaces Portlet Bridge project.  The "Issue Type" must be "TCK Challenge".  The Component should be set to "Testing".  The "Affects Version" should be set to to version of the TCK that contains the test being challenged.  The description field must contain the information in the form below.

  4. How and by whom are challenges addressed?

    The Apache committers responsible for the MyFaces Portlet Bridge project coordinate the review and decisions made by test development and specification development engineers.

    See the Portlet 1.0 Bridge TCK Test Appeals Steps below.

  5. How are approved changes to the Portlet 1.0 Bridge tests managed?

    All tests found to be invalid are placed on the Exclude List for that version of the Portlet 1.0 Bridge TCK within 1 business day. The JIRA is updated with all pertinent information including providing a description of the change and where/how to acquire it.  In addition, an announcement is sent to jsr-301-public@jcp.org describing the change and where/how to acquire it.

    Apache in conjunction with Oracle as Maintenance Lead, has the option of creating an alternative test to address any challenge. Alternative tests (and criteria for their use) will be updated in the SVN source repository. Note that passing an alternative test is deemed equivalent with passing the original test.

Portlet TCK Test Appeals Steps

  1. A test challenge is submitted using the Apache JIRA system contesting the validity of one or a related set of Portlet tests.

    A detailed justification for why each test should be invalidated must be included with the challenge as described by the Test Challenge form below.

  2. The Maintenance Lead evaluates the challenge.

    If the appeal is incomplete or unclear, the JIRA is updated request further information or correction. If all is in order, the Maintenance Lead will check with the test developers to review the purpose and validity of the test before writing a response. The Maintenance Lead will attempt to complete the response within 5 business days. If the challenge is similar to a previously rejected test challenge (i.e., same test and justification), the Maintenance Lead will update the JIRA accordingly.

  3. The challenge and any supporting materials from test developers is sent to the specification engineers for evaluation.

    A decision of test validity or invalidity is normally made within 15 working days of receipt of the challenge. All decisions will be documented with an explanation of why test validity was maintained or rejected.

  4. The licensee is informed of the decision and proceeds accordingly.

    If the test challenge is approved and one or more tests are invalidated, the Maintenance Lead places the tests on the Exclude List for that version of the Portlet (effectively removing the test(s) from the Test Suite). Details concerning the exlcusion will be added into the submitted JIRA challenge in the form below. If the test is valid but difficult to pass due to hardware or operating system limitations, the Maintenance Lead may choose to provide an alternate test to use in place of the original test (all alternate tests are made available to the licensee community).

  5. If the test challenge is rejected, the licensee may choose to escalate the decision to the Executive Committee (EC), however, it is expected that the licensee would continue to work with the Maintenance Lead to resolve the issue and only involve the EC as a last resort.


Test Challenge Form

Replace the values in [] with an appropriate response:
[Test Challenger Name and Company]
[Specification Name(s) and Version(s)]
[Test Suite Name and Version]
[Exclude List Version]
[Test Name]
[Complaint (argument for why test is invalid)]


Test Challenge Response Form

[Test Defender Name and Company]
[Test Defender Role in Defense (e.g., test developer, Maintenance Lead, etc.)]
[Specification Name(s) and Version(s)]
[Test Suite Name and Version]
[Test Name]
[Defense (argument for why test is valid)]
[Implications of test invalidity (e.g., other affected tests and test framework code, creation or exposure of ambiguities in spec (due to unspecified requirements), invalidation of the reference implementation, creation of serious holes in test suite)]
[Alternatives (e.g., are alternate test appropriate?)]

Reference Implementation for Portlet 1.0 Bridge

The Designated Reference Implementation for compatibility testing of Portlet 1.0 Bridge is as follows:

Specification for Portlet 1.0 Bridge for JavaServerTM Faces 1.2

The following web sites contain the Specifications for Portlet 1.0 Bridge for JavaServerTM Faces 1.2:
http://www.jcp.org/en/jsr/detail?id=301

Libraries for Portlet 1.0 Bridge for JavaServerTM Faces 1.2

The following packages constitute the required class libraries for Java™ Portlet Bridge 1.0:
[return to TOC]



















Chapter 3

Requirements





This chapter lists the required hardware configurations and prerequisite software that must be present before you can run the Bridge TCK.

Hardware Requirements

You can run the Portlet TCK software on compatible Java™ platforms on both Sun workstations and on personal computers. The following section lists the hardware requirements for both the TCK and the reference implementation. Hardware requirements for other implementations will vary.

All systems must meet the following recommended and minimum hardware requirements:

Software Requirements

The Bridge TCK Testharness relies on Selenium Remote Control to execute the tests/process the results.  Click here for the specific OS/browser matrix supported by Selenium. In summary, you can access run the Bridge TCK software on Sun Solaris™ operating system, Linux, and Windows XP/2000 platforms that meet the following minimum software requirements:
Operating Systems:
Browsers:

In addition to build and run the TCK you must have the following JavaTM software installed:
Finally, as the TCK is built in run within a local version of the Apache project, you will need to install:
[return to TOC]


















Chapter 4

Installation





This chapter explains how to install the Bridge TCK on a system running the Sun Solaris™, Redhat Linux, or Microsoft Windows operating system.

Obtaining the needed TCK Software

Obtain needed software from:


Installing the Software

  1. Install the JavaEE™ 5 Platform, JDK: Version 1.5 (any)
    Download the  JavaEE™ 5 Platform, JDK software from the Java Software web site and install. See the installation instructions that accompany the software for additional information.
  2. Install Maven
  3. Install Subversion
  4. Install an application server.  
  5. Install a Portlet 1.0-compliant portlet container, such as the reference implementation done as an Apache Open Source Project, Pluto 1.0 or Pluto 2.0.
    If the portlet container software doesn't come configured with consumer portal software that is used as the application that exposes portlets via page (resource) URLs, then also install such consumer software.

    Verify your installation by creating and accessing a page with a portlet on it.
  6. If the application server you are using isn't a fully compliant JavaEE server (i.e. it doesn't contain a Faces implementation) there is no need to download/install your own version.  You will be able to leverage Maven when it builds the TCK test suite wars to package/bundle the Faces version/impl of your choice directly into the war (as long as the version is available in an external  Maven respository  -- which is the case for both Mojarra and MyFaces).
  7. Do a Subversion checkout of the TCK project in a working directory of your choice:
    cd \myTCK
    svn checkout http://svn.apache.org/repos/asf/myfaces/portlet-bridge/tck/tags/jsr301-1.0.0 
[return to TOC]









Chapter 5

TCK Project Structure





Once you have successfully checked out the TCK project using subversion. You will see the following structure:

\Bridge301TCK
pom.xml
Portlet 1.0 Bridge for JavaServer Faces 1.2 TCK User's Guide.html
TCK Tests.html
tck-generate-test-wars.cmd
tck-run-tests.cmd
\portlet-bridge-tck-client
\portlet-bridge-tck-main
\portlet-bridge-tck-section3-2-lifecycle-set
\portlet-bridge-tck-section3-2-render-policy-always-delegate
\portlet-bridge-tck-section3-2-render-policy-default
\portlet-bridge-tck-section3-2-render-policy-never-delegate
\src

\src contains all the shared sources between the various test apps.  This includes the tests themselves, plus the non-app specific resource files (like the exclusions file which is located in src\test\resources\test-exclusions.xml).

\portlet-bridge-tck-main holds the main test application.  99% of the tests are represented within this application.
\portlet-bridge-tck-section3-2-lifecycle-set holds an additional test application.  The main test application includes a test to verify that when using the default configuration the Bridge uses the default Faces Lifecycle.  This application runs the same test however the web application (web.xml) is configured so that Faces/Bridge will use a specific Lifecycle implementation.

The three render-policy subdirectories build distinct web applications each of which represent the same (render-policy) test but with a distinct configuration.  However because the render-policy test is an excluded test, the TCK will not build or run these test applications by default.  To manually run these tests, if your environment can support them, you will need to execute the tck using the "-Pinclude-render-policy-tests" option on the maven command line.  See the exclusions file in src\test\resources\test-exclusions.xml for further information.

\portlet-bridge-tck-client contains project definitions for using the TCK to generate test definition files and for running the TCK against an external server.  When you run the TCK against an external (non-Jetty) server, the run/results are output into the \portlet-bridge-tck-client\target subdirectory.

pom.xml

The pom.xml file contains the primary Maven definitions for building and running the TCK.  Though each of the subdirectories also contain pom.xml's these are child pom's that all rely/reference this one.  In general you should not modify any of the pom.xml files.  The one exception is if you decide to customize this root pom.xml by changing its property definitions that relate to the particular instance of the Bridge or Faces that the TCK relies on/uses by default.

Portlet 1.0 Bridge for JavaServer Faces 1.2 TCK User's Guide.html

This User Manual.

TCK Tests.html

Provides a short description of each test in the TCK.

tck-generate-test-wars.cmd

The tck-generate-test-wars.cmd file is a simple (shell) script that executes the default Maven command for generating the test war files modfiied by whatever command line arguments you include.  Once you figure out your common usage, modify this script so you don't have to keep reentering the (lengthy) set of arguments.

tck-run-tests.cmd

 The tck-run-tests.cmd file is a simple (shell) script that executes the default Maven command for running the test harness to execute the tests installed in your target test application server.  To accomplish this test harness will need the following information from you:
  1. location of the file containing the description of the tests (URLs) to be run.
  2. location of the file containing authentication/credential information (used if the test server's portal challenges the test harness).
  3. The host and port the test server is listening on.
When using this script, these are preset in environment variables:
  1. TCK_CONFIG_HOME:  This variable defines a home directory containing a test.xml and login.properties file used to configure the harness.  Use this variable instead of the TCK_TEST_FILE and TCK_LOGIN_FILE environment variables if you use the standard names.
  2. TCK_TEST_FILE: This variable defines the path to the file containing the URLs for each of the tests to be run in your test server.  See XXXX for where/how this file is generated.
  3. TCK_LOGIN_FILE:  This variable defines the path to the file containing the username and password to be used by the test harness to authenicate itself when its challenged by the portal when it runs the tests.
  4. TCK_CLIENT_HOST:  This variable defines the host where the test server is located.
  5. TCK_CLIENT_PORT: This variable defines the port on the hsot that the test server is listening on.
[return to TOC]











Chapter 6

Testing a Vendor Implementation





This chapter describes a generic Bridge TCK configuration to work with a vendor implementation. See Chapter 7, "Testing a Vendor Implementation in a Pluto Environment" if you are testing your Bridge implementation using Apache Pluto.  See Chapter 8, Verifying the Reference
Implementation in Your Environment” for information on running Bridge TCK against the reference implementation from Apache MyFaces PortletBridge Project.

Before proceeeding, you may want to review the typical usage description from chapter 1.

Note: Much of the following relies on Maven commands to build and execute the TCK.  Maven commands are typically configured by using command line arguments.  As you will see this can often get unwieldly.  It is recommended that you either create templates for your often used commands, (shell) script files, modify the property definitions in the pom.xml file in the project's root directory directly, or use profiles to define groups of properties.  http://maven.apache.org/guides/introduction/introduction-to-profiles.html.

Publish your Bridge to the (local) Maven repository

Because the TCK is controlled by running Maven, prior to building and running the Bridge TCK you need to have published your Bridge implementation to a maven repository.  If the one you want to test isn't already in an (accessible) external repository, then you need to publish it to your local repository.  It is assumed your bridge implementation is contained in two jars, one containing the required implementations for the public/specified API interfaces and classes corresponding to the JavaDoc included with the specification and one containing your vendor specific bridge implementation. Both jars are published using the following Maven commands:
 mvn deploy:deploy-file -DgroupId=<your bridge groupId> -DartifactId=<your artifactId for the bridge api> -Dversion=<your version number> -Dpackaging=jar -Dfile=<location of bridge API jar> -Durl=<local file path to root of maven repository>

mvn deploy:deploy-file -DgroupId=<your bridge groupId> -DartifactId=<your artifactId for the bridge impl> -Dversion=<your version number> -Dpackaging=jar -Dfile=<location of bridge Impl jar> -Durl=<local file path to root of maven repository>

For example:  If I wanted to publish Oracle's bridge to my local repository using a maven groupid of oracle.portlet.bridge I might use:
mvn deploy:deploy-file -DgroupId=oracle.portlet.bridge -DartifactId=bridge-api -Dversion=1.0.0 -Dpackaging=jar -Dfile=c:\bridge\oracle\portlet-bridge-api-1.0.0.jar -Durl=file://"c:\Documents and Settings\myloginname\.m2"

mvn deploy:deploy-file -DgroupId=oracle.portlet.bridge -DartifactId=bridge-impl -Dversion=1.0.0 -Dpackaging=jar -Dfile=c:\bridge\oracle\portlet-bridge-impl-1.0.0.jar -Durl=file://"c:\Documents and Settings\myloginname\.m2"
 

Generate the TCK TestSuite Portlet Applications (.wars)

The TCK TestSuite Portlet Application .wars are built using a Maven command.  Two test applications are built: portlet-bridge-tck-main-jsr301-1.0.0 is generated in \portlet-bridge-tck-main\target while portlet-bridge-tck-section3-2-lifecycle-set-jsr301-1.0.0.war is generated in \portlet-bridge-tck-section3-2-lifecycle-set\target.

Maven command used when the Bridge and Faces are already deployed in the Test AppServer

The following command will build the two test war files without packaging either the Bridge jars or the Faces jars into these wars.  (i.e. use this command if the Bridge and Faces software is already installed/available in your test application server.

mvn clean install -Dtck.generate-war -Dportlet-bridge.groupId=<your bridge groupId>  -Dportlet-bridge.api.artifactId=<your artifactId for the bridge api> -Dportlet-bridge.version=<your version number>

For example
mvn clean install -Dtck.generate-war -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.version=1.0.0

Maven command which additionally includes the Bridge in generated wars.

The following command will build the two test war files which will include the Bridge jars.

mvn clean install -Dtck.generate-war -DincludeBridge -Dportlet-bridge.groupId=<your bridge groupId>  -Dportlet-bridge.api.artifactId=<your artifactId for the bridge api> -Dportlet-bridge.impl.artifactId=<your artifactId for the bridge impl> -Dportlet-bridge.version=<your version number>

For example
mvn clean install -Dtck.generate-war -DincludeBridge -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.api.artifactId=bridge-impl -Dportlet-bridge.version=1.0.0


Maven command which additionally includes JSF in generated wars.

The following command will build the two test war files including both the JSF jars and the Bridge jars.

mvn clean install -Dtck.generate-war -DincludeJSF=[mojarra|myfaces] -DincludeBridge -Dportlet-bridge.groupId=<your bridge groupId>  -Dportlet-bridge.api.artifactId=<your artifactId for the bridge api> -Dportlet-bridge.impl.artifactId=<your artifactId for the bridge impl> -Dportlet-bridge.version=<your version number>

For example to include the Mojarra JSF jars in your wars:
mvn clean install -Dtck.generate-war -DincludeJSF=mojarra -DincludeBridge -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.impl.artifactId=bridge-impl -Dportlet-bridge.version=1.0.0

Or to include the MyFaces JSF jars in your wars:
mvn clean install -Dtck.generate-war -DincludeJSF=myfaces -DincludeBridge -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.impl.artifactId=bridge-impl -Dportlet-bridge.version=1.0.0

Note:  Each of the above commands includes the default version of either Mojarra or MyFaces in the resulting wars.  The default version is controlled by the the Maven poroperty mojarra.version or myfaces.version.  Currently these are the minimum versions demanded by the specification: 1.2_03 and 1.2.2, respectively.  To override these with a later version include -Dmojarra.version or -Dmyfaces.version set to the intended version in the above command line.

Configure Test Server


Once the test applications have been built, deploy them to the test server.  Once deployed, expose each test via a distinct browser accessible URL.  I.e. place each portlet on a separate portal page.  At this point you should be able to verify your deployment/setup by running test(s) manually by accessing the specific portal pages.  

Run the TestHarness to Execute the TCK

The TestHarness runs the TCK by processing a test description file.  The test description file is an XML file called test.xml.  It contains one entry for each test the harness is to run.  The XML syntax for a test description has the form:
<entry key="testURL">testName</entry>

where testURL is the full URL path to the (page containing the) test and testName is the short name of the test.  A test's short name is derived from the test's portlet name.  A test's portlet name has the form: SpecificationSection-Test(short)Name-portlet.

Generating the Test Description File

Prior to running the TestHarness you will need to configure/generate an appropriate version of this test.xml for your specific configuration.  Though the project contains a pre-generated template of the test.xml file (src\test\resources\test.xml) its best to generate your own template file to ensure absolute consistency with the current verion of the TCK.  The following command will generate a template test.xml.

mvn generate-test-resources -Dtck.external-server=generate-test-file-template

The command uses a stylesheet included in the TCK src directory to transform the portlet.xml for the main Test App into a test.xml template (plus its adds the additional test in the other Test app).  One test.xml entry is created for each portlet in the portlet.xml. The resulting test.xml will be located in the /portlet-bridge-tck-client/target and contain N test entries  in the form <entry key="page-servlet-path">XXXTestname</entry> where XXXTestname varies for each test name that is to be run.  To transform this template into one that works for your server, merely replace the "page-servlet-path" value in each entry with the appropriate URL to the page containing the specific test named by XXXTestname.  For example in a Pluto environment you might end up with the following entry:
<entry key="pluto/portal/TestPage001">bridgeVersionTest</entry>

Alternatively you can write an xsl script to transform this file and specify it as an alternative on the command line:
mvn generate-test-resources -Dtck.external-server=generate-test-file-template -Dbridge.tck.stylesheet=/pathtostylsheet/yoursytlesheet.xsl

Note: because using the above command(s) utilizes Maven, any following Maven command that cleans the project (includes clean in its command line) will delete all the project's target directories including the one containing this generated test.xml file.  As one commonly only needs to generate this file once, after it has been generated it is recommended you move this file to a location outside the target directory.  For example merely move it to TCK_PROJECT_HOME/portlet-bridge-tck-client.

Configuring a login file

As most portals require user authentication to access pages, in addition to configuring a test description file, its (often) necessary to also create and configure a login.properties file so the harness can properly authenticate itself when challenged.  The TestHarness, as its running Selenium, handles authentication as it does all page requests, it looks for specific named fields/buttons in the page response to determine what it should do/send in the next request.  The login.properties file therefore contains the name/value pairs of the form fields and button for the portal's login page.  I.e. the login.properties file has the form:
field1Id=field1Value
field2Id-field2Value
SubmitButtonId

Note:  input fields are donoted as name/value pairs while the submit button is represented by only its name (id).

For example the login.properties file for Pluto is:
j_username=pluto
j_password=pluto
j_login

In every request sent by the TestHarness the harness looks at the response and if the first field from the login.properties file is present it sets each of the fields identified in the login.properties file with their associated value and submits the form (via the indicated submit button).

If your test server will present a login screen to the TestHarness when it executes page requests from the test.xml file, create a login.properties file with the appropriate information using the above syntax.  It is recommended you locate this file in the same location as the test.xml file.

Running the TestHarness

After properly configuring and deploying the TCK tests to your test server and creating an appropriate test.xml and login.properites (if necessary) file, run the TestHarness to execute the tests and generate the test reports by using the following command:

mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file=<path to test.xml> -Dbridge.tck.test.host=<host name> -Dbridge.tck.test.port=<port> [-Dbridge.tck.login-file=<path to login file>]


For example to run the tests and generate the test report in a situation where the test server is on the bridge-testserver port 80 and the test.xml and login.properties file both live in the TCK_HOME/portlet-bridge-tck-client directory:

mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file=c:\bridgeTCK\portlet-bridge-tck-client\test.xml -Dbridge.tck.test.host=bridge-testserver -Dbridge.tck.test.port=80 -Dbridge.tck.login-file=c:\bridgeTCK\portlet-bridge-tck-client\login.properties
 

Note:  the default value for bridge.tck.test.host is localhost and the default value for bridge.tck.test.port is 8080. You only need to define these properties on the command line if you are not using the default(s).

The above command builds the TestHarness client and then executes it by launching JUnit and Selenium.  Two browser windows will be activated. The first runs Selenium Remote Control and the second the execution of each test.  When the harness completes JUnit is used to generate the test reports.  The reports are output to TCK_HOME/portlet-bridge-tck-client/target/portlet-bridge-tck-client-report.html.  The raw data from which this report is generated is contained in TCK_HOME/portlet-bridge-tck-client/target/surefire-reports

Note: One test, destroyActionTest, can only be run successfully once per portlet activation (i.e. the application containing this test and/or the server must be bounced before this test can run successfully again).

Alternative to lengthy Maven command lines

As Maven doesn't provide a way to pull its properties directly from the environment, command lines including the variety of property definitions needed to operate the TCK can become quite lengthy.  Once you determine those commands that satisfy your needs, you might consider writing (shell) scripts to make execution easier.  The TCK_HOME directory contains two .cmd files (tck-generate-test-wars.cmd and tck-run-tests.cmd) which you can either use directly are alter to better suit your needs.

[return to TOC]














Chapter 7

Testing a Vendor Implementation in a Pluto Environment





As Apache Pluto is the reference implementation for the Java Portlet container, and the Bridge reference implementation is also an Apache project, efforts have been made in the TCK to simplify testing your vendor implementation running on Pluto.

Publish your Bridge to the (local) Maven repository

Because the TCK is controlled by running Maven, prior to building and running the Bridge TCK you need to have published your Bridge implementation to a maven repository.  If the one you want to test isn't already in an (accessible) external repository, then you need to publish it to your local repository.  It is assumed your bridge implementation is contained in two jars, one containing the required implementations for the public/specified API interfaces and classes corresponding to the JavaDoc included with the specification and one containing your vendor specific bridge implementation. Both jars are published using the following Maven commands:
 mvn deploy:deploy-file -DgroupId=<your bridge groupId> -DartifactId=<your artifactId for the bridge api> -Dversion=<your version number> -Dpackaging=jar -Dfile=<location of bridge API jar> -Durl=<local file path to root of maven repository>

mvn deploy:deploy-file -DgroupId=<your bridge groupId> -DartifactId=<your artifactId for the bridge impl> -Dversion=<your version number> -Dpackaging=jar -Dfile=<location of bridge Impl jar> -Durl=<local file path to root of maven repository>

For example:  If I wanted to publish Oracle's bridge to my local repository using a maven groupid of oracle.portlet.bridge I might use:
mvn deploy:deploy-file -DgroupId=oracle.portlet.bridge -DartifactId=bridge-api -Dversion=1.0.0 -Dpackaging=jar -Dfile=c:\bridge\oracle\portlet-bridge-api-1.0.0.jar -Durl=file://"c:\Documents and Settings\myloginname\.m2"

mvn deploy:deploy-file -DgroupId=oracle.portlet.bridge -DartifactId=bridge-impl -Dversion=1.0.0 -Dpackaging=jar -Dfile=c:\bridge\oracle\portlet-bridge-impl-1.0.0.jar -Durl=file://"c:\Documents and Settings\myloginname\.m2"
 

Generate the TCK TestSuite Portlet Applications (.wars)

The TCK TestSuite Portlet Application .wars are built using a Maven command.  Two test applications are built: portlet-bridge-tck-main-jsr301-1.0.0 is generated in \portlet-bridge-tck-main\target while portlet-bridge-tck-section3-2-lifecycle-set-jsr301-1.0.0.war is generated in \portlet-bridge-tck-section3-2-lifecycle-set\target.

Portlet Applications built to be deployed on Pluto require additional configuration in the application's web.xml to relate the portlets to the Pluto servlet.  This configuration information differs for a Pluto 1.0 container and a Pluto 2.0. Using any of the following commands will generate Pluto deployable .wars (with web.xml's already properly configured).

Maven command used when the Bridge and Faces are already deployed in the Test AppServer

The following command will build the two test war files without packaging either the Bridge jars or the Faces jars into these wars.  (i.e. use this command if the Bridge and Faces software is already installed/available in your test application server.

mvn clean install -Dtck.generate-war=[pluto1.0|pluto2.0] -Dportlet-bridge.groupId=<your bridge groupId>  -Dportlet-bridge.api.artifactId=<your artifactId for the bridge api> -Dportlet-bridge.version=<your version number>

For example to build the TCK wars for a Pluto 2.0 environment:
mvn clean install -Dtck.generate-war=pluto2.0 -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.version=1.0.0

Maven command which additionally includes the Bridge in generated wars.

The following command will build the two test war files including the Bridge jars.

mvn clean install -Dtck.generate-war=[pluto1.0|pluto2.0] -DincludeBridge -Dportlet-bridge.groupId=<your bridge groupId>  -Dportlet-bridge.api.artifactId=<your artifactId for the bridge api> -Dportlet-bridge.impl.artifactId=<your artifactId for the bridge impl> -Dportlet-bridge.version=<your version number>

For example
mvn clean install -Dtck.generate-war=pluto2.0 -DincludeBridge -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.impl.artifactId=bridge-impl -Dportlet-bridge.version=1.0.0


Maven command which additionally includes JSF in generated wars.

The following command will build the two test war files including both the JSF jars and the Bridge jars.

mvn clean install -Dtck.generate-war=[pluto1.0|pluto2.0] -DincludeJSF=[mojarra|myfaces] -DincludeBridge -Dportlet-bridge.groupId=<your bridge groupId>  -Dportlet-bridge.api.artifactId=<your artifactId for the bridge api> -Dportlet-bridge.impl.artifactId=<your artifactId for the bridge impl> -Dportlet-bridge.version=<your version number>

For example to include the Mojarra JSF jars in your wars:
mvn clean install -Dtck.generate-war=pluto2.0 -DincludeJSF=mojarra -DincludeBridge -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.impl.artifactId=bridge-impl -Dportlet-bridge.version=1.0.0

Or to include the MyFaces JSF jars in your wars:
mvn clean install -Dtck.generate-war=pluto2.0 -DincludeJSF=myfaces -DincludeBridge -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.impl.artifactId=bridge-impl -Dportlet-bridge.version=1.0.0

Note:  Each of the above commands includes the default version of either Mojarra or MyFaces in the resulting wars.  The default version is controlled by the the Maven poroperty mojarra.version or myfaces.version.  Currently these are the minimum versions demanded by the specification: 1.2_03 and 1.2.2, respectively.  To override these with a later version include -Dmojarra.version or -Dmyfaces.version set to the intended version in the above command line.

Configure Test Server


Once the test applications have been built, deploy them to the test server.  Once deployed, expose each test via a distinct browser accessible URL.  I.e. place each portlet on a separate portal page.  At this point you should be able to verify your deployment/setup by running test(s) manually by accessing the specific portal pages.  

To avoid the labor intensive step of manually creating the test pages copy TCK_HOME/src/test/resources/tomcat_pluto/pluto-portal-driver-config.xml to your applications server's deployed Pluto application's WEB-INF directory replacing the one that exists there.  This file maintains Pluto's repository of its portal pages.  It has been preconfigured to represent/hold all of the tests.

Run the TestHarness to Execute the TCK

The TestHarness runs the TCK by processing a test description file.  The test description file is an XML file called test.xml.  It contains one entry for each test the harness is to run.  The XML syntax for a test description has the form:
<entry key="testURL">testName</entry>

where testURL is the full URL path to the (page containing the) test and testName is the short name of the test.  A test's short name is derived from the test's portlet name.  A test's portlet name has the form: SpecificationSection-Test(short)Name-portlet.

Generating the Test Description File

Normally you would use the commands describes in Chapter 5 for generating the test description file.  The TCK project however contains a preconfigured version for Pluto.  This version corresponds to the portal structure defined by the pluto-portal-driver-config.xml file described in the previous section.  Copy TCK_HOME/src/test/resources/tomcat_pluto/test.xml to TCK_HOME/portlet-bridge-tck-client (or merely reference this file from this location directly).

Note: this file works for both versions of Pluto.

Configuring a login file

Pluto will challenge the caller/client on first access.  The TestHarness responds to this challenge based on information it pulls from the login.properties file.  See Chapter 5 for more details concerning the structure of this file.  However, as with the test.xml, the TCK project contains a preconfigured login.properties for Pluto.  Copy TCK_HOME/src/test/resources/tomcat_pluto/login.properties to TCK_HOME/portlet-bridge-tck-client (or merely reference this file from this location directly).

Note: this file works for both versions of Pluto.

Running the TestHarness

After properly configuring and deploying the TCK tests to your test server and creating an appropriate test.xml and login.properites (if necessary) file, run the TestHarness to execute the tests and generate the test reports by using the following command:

mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file=<path to test.xml> -Dbridge.tck.test.host=<host name> -Dbridge.tck.test.port=<port> [-Dbridge.tck.login-file=<path to login file>]


For example to run the tests and generate the test report in a situation where the test server is on the bridge-testserver port 80 and the test.xml and login.properties file both live in the TCK_HOME/portlet-bridge-tck-client directory:

mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file=c:\bridgeTCK\portlet-bridge-tck-client\test.xml -Dbridge.tck.test.host=bridge-testserver -Dbridge.tck.test.port=80 -Dbridge.tck.login-file=c:\bridgeTCK\portlet-bridge-tck-client\login.properties
 

Note:  the default value for bridge.tck.test.host is localhost and the default value for bridge.tck.test.port is 8080. You only need to define these properties on the command line if you are not using the default(s).

The above command builds the TestHarness client and then executes it by launching JUnit and Selenium.  Two browser windows will be activated. The first runs Selenium Remote Control and the second the execution of each test.  When the harness completes JUnit is used to generate the test reports.  The reports are output to TCK_HOME/portlet-bridge-tck-client/target/portlet-bridge-tck-client-report.html.  The raw data from which this report is generated is contained in TCK_HOME/portlet-bridge-tck-client/target/surefire-reports

Note: One test, destroyActionTest, can only be run successfully once per portlet activation (i.e. the application containing this test and/or the server must be bounced before this test can run successfully again).


Alternative to lengthy Maven command lines

As Maven doesn't provide a way to pull its properties directly from the environment, command lines including the variety of property definitions needed to operate the TCK can become quite lengthy.  Once you determine those commands that satisfy your needs, you might consider writing (shell) scripts to make execution easier.  The TCK_HOME directory contains two .cmd files (tck-generate-test-wars.cmd and tck-run-tests.cmd) which you can either use directly are alter to better suit your needs.

[return to TOC]

















Chapter 8

Verifying the Reference Implementation in your Environment





In addition to its main purpose of validating a vendor's Bridge implementation, the TCK can also be used to validate an existing Bridge implementation in a new application server/JSF/portlet container environment.  For example one might want to check/verify that the Reference Implementation runs correctly in a Liferay portal.  This chapter describes running the TCK to test the Reference Implementation in your application server environment.

Generate the TCK TestSuite Portlet Applications (.wars)

The TCK TestSuite Portlet Application .wars are built using a Maven command.  Two test applications are built: portlet-bridge-tck-main-jsr301-1.0.0 is generated in \portlet-bridge-tck-main\target while portlet-bridge-tck-section3-2-lifecycle-set-jsr301-1.0.0.war is generated in \portlet-bridge-tck-section3-2-lifecycle-set\target.

Maven command used when the Bridge and Faces are already deployed in the Test AppServer

The following command will build the two test war files without packaging either the Bridge jars or the Faces jars into these wars.  (i.e. use this command if the Bridge and Faces software is already installed/available in your test application server.

mvn clean install -Dtck.generate-war

For example
mvn clean install -Dtck.generate-war

Maven command which additionally includes the Bridge in generated wars.

The following command will build the two test war files including the Bridge jars.

mvn clean install -Dtck.generate-war -DincludeBridge

For example
mvn clean install -Dtck.generate-war -DincludeBridge


Maven command which additionally includes JSF in generated wars.

The following command will build the two test war files including both the JSF jars and the Bridge jars.

mvn clean install -Dtck.generate-war -DincludeJSF=[mojarra|myfaces] -DincludeBridge

For example to include the Mojarra JSF jars in your wars:
mvn clean install -Dtck.generate-war -DincludeJSF=mojarra -DincludeBridge

Or to include the MyFaces JSF jars in your wars:
mvn clean install -Dtck.generate-war -DincludeJSF=myfaces -DincludeBridge

Note:  Each of the above commands includes the default version of either Mojarra or MyFaces in the resulting wars.  The default version is controlled by the the Maven poroperty mojarra.version or myfaces.version.  Currently these are the minimum versions demanded by the specification: 1.2_03 and 1.2.2, respectively.  To override these with a later version include -Dmojarra.version or -Dmyfaces.version set to the intended version in the above command line.

Configure Test Server


Once the test applications have been built, deploy them to the test server.  Once deployed, expose each test via a distinct browser accessible URL.  I.e. place each portlet on a separate portal page.  At this point you should be able to verify your deployment/setup by running test(s) manually by accessing the specific portal pages.  

Run the TestHarness to Execute the TCK

The TestHarness runs the TCK by processing a test description file.  The test description file is an XML file called test.xml.  It contains one entry for each test the harness is to run.  The XML syntax for a test description has the form:
<entry key="testURL">testName</entry>

where testURL is the full URL path to the (page containing the) test and testName is the short name of the test.  A test's short name is derived from the test's portlet name.  A test's portlet name has the form: SpecificationSection-Test(short)Name-portlet.

Generating the Test Description File

Prior to running the TestHarness you will need to configure/generate an appropriate version of this test.xml for your specific configuration.  Though the project contains a pre-generated template of the test.xml file (src\test\resources\test.xml) its best to generate your own template file to ensure absolute consistency with the current verion of the TCK.  The following command will generate a template test.xml.

mvn generate-test-resources -Dtck.external-server=generate-test-file-template

The command uses a stylesheet included in the TCK src directory to transform the portlet.xml for the main Test App into a test.xml template (plus its adds the additional test in teh other Test app).  One test.xml entry is created for each portlet in the portlet.xml. The resulting test.xml will be located in the /portlet-bridge-tck-client/target and contain N test entries  in the form <entry key="page-servlet-path">XXXTestname</entry> where XXXTestname varies for each test name that is to be run.  To transform this template into one that works for your server, merely replace the "page-servlet-path" value in each entry with the appropriate URL to the page containing the specific test named by XXXTestname.  For example in a Pluto environment you might end up with the following entry:
<entry key="pluto/portal/TestPage001">bridgeVersionTest</entry>

Alternatively you can write an xsl script to transform this file and specify it as an alternative on the command line:
mvn generate-test-resources -Dtck.external-server=generate-test-file-template -Dbridge.tck.stylesheet=/pathtostylsheet/yoursytlesheet.xsl

Note: because using the above command(s) utilizes Maven, any following Maven command that cleans the project (includes clean in its command line) will delete all the project's target directories including the one containing this generated test.xml file.  As one commonly only needs to generate this file once, after it has been generated it is recommended you move this file to a location outside the target directory.  For example merely move it to TCK_PROJECT_HOME/portlet-bridge-tck-client.

Configuring a login file

As most portals require user authentication to access pages, in addition to configuring a test description file, its (often) necessary to also create and configure a login.properties file so the harness can properly authenticate itself when challenged.  The TestHarness, as its running Selenium, handles authentication as it does all page requests, it looks for specific named fields/buttons in the page response to determine what it should do/send in the next request.  The login.properties file therefore contains the name/value pairs of the form fields and button for the portal's login page.  I.e. the login.properties file has the form:
field1Id=field1Value
field2Id-field2Value
SubmitButtonId

Note:  input fields are donoted as name/value pairs while the submit button is represented by only its name (id).

For example the login.properties file for Pluto is:
j_username=pluto
j_password=pluto
j_login

In every request sent by the TestHarness the harness looks at the response and if the first field from the login.properties file is present it sets each of the fields identified in the login.properties file with their associated value and submits the form (via the indicated submit button).

If your test server will present a login screen to the TestHarness when it executes page requests from the test.xml file, create a login.properties file with the appropriate information using the above syntax.  It is recommended you locate this file in the same location as the test.xml file.

Running the TestHarness

After properly configuring and deploying the TCK tests to your test server and creating an appropriate test.xml and login.properites (if necessary) file, run the TestHarness to execute the tests and generate the test reports by using the following command:

mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file=<path to test.xml> -Dbridge.tck.test.host=<host name> -Dbridge.tck.test.port=<port> [-Dbridge.tck.login-file=<path to login file>]


For example to run the tests and generate the test report in a situation where the test server is on the bridge-testserver port 80 and the test.xml and login.properties file both live in the TCK_HOME/portlet-bridge-tck-client directory:

mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file=c:\bridgeTCK\portlet-bridge-tck-client\test.xml -Dbridge.tck.test.host=bridge-testserver -Dbridge.tck.test.port=80 -Dbridge.tck.login-file=c:\bridgeTCK\portlet-bridge-tck-client\login.properties
 

Note:  the default value for bridge.tck.test.host is localhost and the default value for bridge.tck.test.port is 8080. You only need to define these properties on the command line if you are not using the default(s).

The above command builds the TestHarness client and then executes it by launching JUnit and Selenium.  Two browser windows will be activated. The first runs Selenium Remote Control and the second the execution of each test.  When the harness completes JUnit is used to generate the test reports.  The reports are output to TCK_HOME/portlet-bridge-tck-client/target/portlet-bridge-tck-client-report.html.  The raw data from which this report is generated is contained in TCK_HOME/portlet-bridge-tck-client/target/surefire-reports

Note: One test, destroyActionTest, can only be run successfully once per portlet activation (i.e. the application containing this test and/or the server must be bounced before this test can run successfully again).

Alternative to lengthy Maven command lines

As Maven doesn't provide a way to pull its properties directly from the environment, command lines including the variety of property definitions needed to operate the TCK can become quite lengthy.  Once you determine those commands that satisfy your needs, you might consider writing (shell) scripts to make execution easier.  The TCK_HOME directory contains two .cmd files (tck-generate-test-wars.cmd and tck-run-tests.cmd) which you can either use directly are alter to better suit your needs.

[return to TOC]


















Chapter 9

TCK Command Reference





As has been shown in the previous chapters, the Maven commands used to build and run the TCK are highly configurable. Commands are driven/modified by setting Maven properties.  Maven properties are set in the command line using the following syntax: mvn install -DpropertyName=value.  In certain instances the mere existence of a property signifies behavior.  I.e. It is both valid and there is meaning to defining a property without a value.  The following is a complete reference of the properties recognized in the TCK.  Properties that have meaning/use without a value are indicated in the table as accepting an <emptyValue>.  Properties that require a value to be set and don't have a predefined default value use <none> in the Default Value column.

The most common properties that pertain to generating the Test wars (i.e. mvn install):


Property Name Default Value Allowed Values Description
tck.generate-war <emptyValue> <emptyValue>|pluto1.0|pluto2.0 This property instructs the Maven command to build and generate the Test wars. When defined without a value (<emptyValue>) the generated wars will contain a standard portlet application web.xml.  When defined with the value pluto1.0 the generated wars will contain a web.xml preconfigured for deployment in a Pluto 1.0 container.  When defined with a value of pluto2.0 the generated wars will contain a web.xml preconfigured for deployment in a Pluto 2.0 container.

Usage: mvn install -Dtck.generate-war=pluto2.0
includeBridge <emptyValue> <emptyValue> Setting this property instructs the Maven command to include the Bridge jars in the generated Test wars. By default this property is not set.  

Usage: mvn install -Dtck.generate-war -DincludeBridge
includeJSF <none> mojarra|myfaces Setting this property instructs the Maven command to include the JSF jars in the generated Test wars. When set with a value mojarra, the Mojarra (RI) implementation is included. When set with a value myffaces, the MyFaces implementation is included. By default this property is not set.

Usage: mvn install -Dtck.generate-war -DincludeJSF=myfaces
mojarra.version 1.2_03 1.2_03 or greater (mojarra)
 This property defines the Mojarra version used to include in the generated Test wars.  By default the TCK doesn't include the JSF jars in its Test wars. (See includeJSF above).  When used in conjunction with -DincludeJSF=mojarra this property is consulted to define the version to include.  By default (if not set) Mojarra 1.2_03 is used.

Usage mvn install -Dtck.generate-war -DincludeJSF=mojarra -Dmojarra-version=1.2_09
myfaces.version 1.2.2 1.2.2 or greater (myfaces) This property defines the MyFaces version used to include in the generated Test wars.  By default the TCK doesn't include the JSF jars in its Test wars. (See includeJSF above).  When used in conjunction with -DincludeJSF=myfaces this property is consulted to define the version to include.  By default (if not set) yFaces 1.2.2 is used.

Usage mvn install -Dtck.generate-war -DincludeJSF=myfaces -Dmyfaces-version=1.2.7
portlet-bridge.groupId Bridge RI This property defines the Maven atrifact group the bridge lies in and is used in part to locate the bridge in the Maven respository.  By default this property references the Reference Implementation.  To compile and/or include your bridge set this property to reference your bridge.

Usage: mvn install -Dtck.generate-war -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.version=1.0.0 
portlet-bridge.api.artifactId Bridge RI This property defines the Maven atrifact id for the portlet bridge api jar and is used in part to locate the bridge api jar in the Maven respository.  By default this property references the Reference Implementation.  To compile and/or include your bridge set this property to reference your bridge.

Usage: mvn install -Dtck.generate-war -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.version=1.0.0 
portlet-bridge.version Bridge RI This property defines the Maven version number for the portlet bridge api/impl jars and is used in part to locate the bridge api/impl jar in the Maven respository.  By default this property references the Reference Implementation.  To compile and/or include your bridge set this property to reference your bridge version.

Usage: mvn install -Dtck.generate-war -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.version=1.0.0 
portlet-bridge.impl.artifactId Bridge RI This property defines the Maven atrifact id for the portlet bridge impl jar and is used in part to locate the bridge impl jar in the Maven respository.  By default this property references the Reference Implementation.  Typically this property is only set when you want to include your bridge in the generate Test wars..

Usage: mvn install -Dtck.generate-war -DincludeBridge -Dportlet-bridge.groupId=oracle.portlet.bridge  -Dportlet-bridge.api.artifactId=bridge-api -Dportlet-bridge.impl.artifactId=bridge-impl -Dportlet-bridge.version=1.0.0 

Other less commonly set properties exist that aren't covered here.  In general they provide finer control on the ids/versions referenced above.  Consult the TCK's root .pom file for more information.

The most common properties that pertain to generating the Test Description File (i.e. mvn generate-test-resources):


Property Name Default Value Allowed Values Description
tck.external-server <none> generate-test-file-template When this property is set with a value of generate-test-file-template it instructs the Maven command to only generate the test description file (template) to be used with an external server.

Usage: mvn generate-test-resources -Dtck.external-server=generate-test-file-template
bridge.tck.test.file.stylesheet src/test/resources/test-template.xsl
When defined with the above, this property defines the location of the stylesheet that is used to transform portlet.xml and generate a test.xml template

Usage: mvn generate-test-resources -Dtck.external-server=generate-test-file-template -Dbridge.tck.test.file.stylesheet=c:\bridgeTCK\portlet-bridge-tck-client\generate_test_xml.xsl
bridge.tck.test.file.servlet.path page-servlet-path
 any string
 This property defines the string used to represent the page path in the generated test.xml when the test.xml is built using the standard (provided) stylesheet.  Change this string to something else if its easier for you to manually update the template afterwards.

Usage: mvn generate-test-resources -Dtck.external-server=generate-test-file-template -Dbridge.tck.tst.file.servlet.path=/pluto/portal/TestPage0000


The most common properties that pertain to running the Test Harness (i.e. mvn surefire-report:report):


Property Name Default Value Allowed Values Description
tck.external-server <none> run-test When this property is set with a value of run-test it instructs the Maven command to only run the test harness with an external server.

Usage: mvn surefire-report:report -Dtck.external-server=run-test
bridge.tck.test.file ${bridge.tck.test.file.dir}/${bridge.tck.test.file.name}
When defined with the above, this property defines the location of the test description file (test.xml) that drives the set of tests the harness runs.

Usage: mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file= c:\bridgeTCK\portlet-bridge-tck-client\test.xml
bridge.tck.login-file <none>
When used in conjunction with -Dtck.external-server=run-test, this property defines the location of the login.properties file that provides the information the harness needs to authenticate itself with the test server when challenged.

Usage: mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file= c:\bridgeTCK\portlet-bridge-tck-client\login.properties
bridge.tck.test.host localhost This property defines the name of the host used to access the test portal server.

Usage: mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file=c:\bridgeTCK\portlet-bridge-tck-client\test.xml -Dbridge.tck.test.host=bridge-testserver
bridge.tck.test.port 8080 This property defines the port the test portal server host is lisenting on.

Usage: mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file=c:\bridgeTCK\portlet-bridge-tck-client\test.xml -Dbridge.tck.test.host=bridge-testserver -Dbridge.tck.test.port=80
bridge.tck.browser *firefox [[*firefox|*iexplore] Define this property to have Selenium (Test harness) run in a browser other than Mozilla Firefox.

mvn surefire-report:report -Dtck.external-server=run-test -Dbridge.tck.test.file= c:\bridgeTCK\portlet-bridge-tck-client\test.xml -Bbridge.tck.browser=*iexplore

[return to TOC]

















Chapter 10

Debugging Test Problems





If any of the tests should fail descriptive information describing the nature of the failure is output by the test and included in the generated report.  If this description doesn't provide enough information to determine the cause of the failure, the bridge will need to be debugged.  As the bridge is part of a portlet application, debugging occurs using exactly the same procedure used to debug a portlet application in the test server environment.  As different test servers may have different procedures consult the documentation of yours for details.

Often when tracking down an issue its helpful to isolate the tests run to only the test at issue.  If using the TCK Test harness to execute the tests, you can accomplish this by commenting out or removing those tests in your test.xml file that you don't want to have run.  Another, often easier, alternative is to merely to run the test manually by invoking the page containing the test at issue directly from your browser (and then clicking on any of the indicated buttons if provided).

Because the primary function of the Bridge is to translate between the Portlet and JSF environments, one common cause for test failures is a bug either in the portlet container or the Faces implementation (more commonly the portlet container).  If you isolate the issue to a source outside the bridge then to pass the TCK you will need to verify bridge correctness (for the failed tests) on a different platform. For example, the Reference Implementation doesn't pass the JSP_EL Test when run in the embedded Jetty server (with pluto 1.0) because this server doesn't properly set up/transfer the EL Context across included JSPs.  However because the Reference Implementation passes the test when run in a Tomcat/Pluto environment, it passes the TCK.  (Likewise there is a test that fails in the Pluto 2.0 environment due to a Pluto 2.0 bug but it passes in the Pluto 1.0 environment).

[return to TOC]


















Appendix A

Frequently Asked Questions




Q: Where do I start to debug a test failure?
A: Check the detailed description concerning the failure contained in the test run report. If this doesn't provide enough infomration to resolve the problem you will need to attach a debugger to the server and inspect its behavior. See Chapter 10, “Debugging Test Problems” for more information.



Q: Where is the TCK's text exclusion list located?
A.  It is in TCK_HOME/src/test/resources/test-exclusions.xml


Q: Why are there tests in the Excludes File?
A:  The test exclusions file contains all tests that are not required to be run to certify your Bridge. In general excluded tests are those which were found for one reason or another not to operate correctly in certain specific environments due either to a JSF implementation dependency in the test or because it verifies behavior that the underlying portlet container isn't required to support..  Each test in the test exclusions file contains a comment detailing under what circumstance the test can and can't be run.  If you are running in an environment which supports the test, you are strongly encouraged to manually run the test to ensure this portion of the bridge function is operating correctly.


Q:  Should I run any tests that are in the Excludes File?
A:  Each test in the test exclusions file contains a comment detailing under what circumstance the test can and can't be run.  If you are running in an environment which supports the test, you are strongly encouraged to manually run the test to ensure this portion of the bridge function is operating correctly.


Q: What does the basic mvn install command do?
A:  The TCK has not only been built for external use to test (other) bridge implementations, but it is also used by the Reference Implementation to both verify its behavior and for regression testing.  The simpliest/basic form of the Maven commands are oriented towards this purpose.  The mvn install command will build the test wars, construct the appropriate server configuration files,  and automatically launch a Jetty server containing Pluto and run the tests.  As its anticipated bridge implementors will likely be targeting specific server environments, the details of using the built-in Jetty server haven't been provided in this documentation.  If you would like more information on this, consult the projects .pom files.

Q: A test fails because of a portlet container bug. Should I challenge the test and request it be excluded?
A:  No.  Its likely the test is valid and hence you will need to verify your Bridge passes in a different environment.  For example, the Reference Implementation doesn't pass the JSP_EL Test when run in the embedded Jetty server (with Pluto 1.0) because this server doesn't properly set up/transfer the EL Context across included JSPs.  However because the Reference Implementation passes the test when run in a Tomcat/Pluto environment, it passes the TCK.  (Likewise there is a test that fails in the Pluto 2.0 environment due to a Pluto 2.0 bug but it passes in the Pluto 1.0 environment).

[return to TOC]