ClearQuest Installation & User Guide
1.0 Introduction
The Rally Connector for ClearQuest allows customers to use Rally for Agile lifecycle management while still using ClearQuest for management of issues (or other types of objects).
The connector provides synchronization capability between ClearQuest and Rally through four services, described below.
2.0 Overview of features
The Rally Connector for ClearQuest runs as a Ruby script on a machine inside your network. The connector is configured via an XML file. A log file is created to track all changes made in Rally and ClearQuest by the connector. The connector requires a custom field exists in each system to store the unique id of the linked objects in the other system. The connector copies fields from Rally or ClearQuest based on a field mapping that the user has specified in their configuration file. Standard and custom fields can be mapped between the two systems.
The connector provides four services to synchronize objects between Rally and ClearQuest:
- Copy entities created in ClearQuest to Rally work items
- Copy work items created in Rally to ClearQuest entities
- Update Rally work items based on changes made to ClearQuest entities
- Update ClearQuest entities based on changes made to Rally work items
The configuration file specifies which services to run, and in what order.
It is important to recognize that the data models of Rally and ClearQuest are not identical. The data models of the two connected systems are only partially compatible. We recommend that you take some time to identify the key data items that you want to track in both systems and what you want the policy to be as far as having a primary system of record. You'll see specific information later in this document discussing some of the trade-offs to be considered and some potential approaches.
3.0 Software and hardware requirements
The Rally Connector for ClearQuest is available for Unlimited Edition customers Integration Hub partners.
Click on the Contact Support link at the bottom of any page in Rally to open a support case to obtain the Rally Connector for Clear Quest. Once you have received a response and obtained the installation package, proceed with the following tasks.
The following are the software and hardware requirements for the ClearQuest connector:
- A Rally subscription.
- A Rally account with administrator privileges (for setup only, but not needed to run the connector).
- The URL for the Rally server hosting your subscription (typically rally1.rallydev.com (excluding the HTTPS prefix, as the connector adds "https://" to whatever is specified).
- The ClearQuest user specified to run the connector needs to have "SQL Editor" privileges and needs to subscribe to the ClearQuest database (if not already subscribed). "ClearQuest Designer" privileges are needed to initially setup the connector but not to run the connector.
- Windows XP or Windows 7
- If using a proxy, the server and port number of the server.
- ClearQuest version 7.0.1.0 or higher. (We only tested with version 7.0.1.0)
- ClearQuest for Windows client installed on the Windows machine where you plan to run the connector.
4.0 Installation
4.1 Basic installation steps
- Install the connector (run the installer or extract the zip).
- Make configuration changes in Rally and the other system.
- Edit the cg_config.xml file and make changes to reflect your environment.
- Run rally2_cq_connector.exe to start the synchronization process.
4.2 Install the connector
(Optional) If you use a proxy server to access Rally, you will need to set the http_proxy environment variable. On Windows:
- right-click My Computer and select Properties
- select the Advanced tab and then click Environment Variables
- click New under System Variables
- enter http_proxy as the full HTTP URL, such as http://www-cache:8000.
- You may need to restart your system in order for the change to take effect.
Installation for Windows
Double-click the RallyConnectorforClearQuestInstall-<version>.exe to bring up the install wizard. Follow the prompts to complete the installation.
The Windows installer will install the following files (by default in C:\Program Files\RallyConnectorfor<connector_name>):
- rally2_cq_connector.exe - Executable to run the connector.
- cq_config.xml - Sample configuration file.
- WinTail.exe* - Utility to tail the log file output for debugging purposes.
- startcq.bat - Batch file that automatically runs the connector and shows the log file; useful for testing.
*WinTail is a free text file tail utility for Windows, generously made available by Bare Metal Software Pty Ltd. for redistribution without restriction. We have found it to be far superior to Notepad and other text editors for watching log files in real time as they are updated by the connector.
4.3 Make configuration changes in Rally and ClearQuest
Create an External ID field in Rally
- Log into Rally as a Workspace or Subscription administrator.
- Go to Setup → Workspaces & Projects.
- Click on the appropriate Workspace that you wish to map to ClearQuest. This will take you to the Detail page for the given Workspace.
- Click on Work Products & Fields and ensure the Work Product Type selected is Defect.
- Click Actions → New Field.
- Enter a Name of ExternalID, Display Name of ExternalID (name and display name must match), and type of String.
Make note of the name of this field. Once you start using the connector, this will contain the external id of the ClearQuest defect.
Create an External ID field in ClearQuest
Contact your ClearQuest administrator to perform these updates.
- Log into ClearQuest Designer and checkout the appropriate schema.
- Navigate to Record Types → Defect.
- Right-click on an empty row and select Add Field....
- In the dialog that appears, enter a Field Name of ExternalID, DB Column Name of ExternalID and select Type SHORT_STRING.
- Commit the changes to the database schema by clicking File → Check In.
- If the check-in doesn't produce any errors, commit the changes to the defect repository by clicking Database → Upgrade Database.
Make note of the display name of this field. This will add a string field to the ClearQuest form that will contain the Rally id. If you wish to view this field from your defect form, then modify the database schema to include this field.
4.4 Edit the configuration file
The configuration file was part of the delivered zip file and should be in the same directory where you extracted the Ruby gem. For ClearQuest, the default configuration file has a name of cq_config.xml. We recommend making a backup copy of the cq_config.xml in case you need to reference a valid configuration file later.
Edit cq_config.xml by entering the appropriate values between each begin and end tag.
Each major section is surrounded by a tag to delineate that section. In the example above, here is a description of each section:
- RallyConnection
- Defines the connection information for Rally, including Rally URL, user, password, and so on.
- CQConnection
- Defines the connection information for ClearQuest, including the database name, artifact type, and so on.
- Connector
- Defines the field mapping between the two systems. Generally, strings should be mapped to strings, integers to integers, and so on. The order of the fields in the mapping section determines the order in which those fields get updated in ClearQuest.
- ConnectorRunner
- Specifies a run interval for how often the connector will query each system to trigger creates/updates and what services should run.
Definition of tags in configuration file
Below is a more detailed explanation of each tag section in the configuration file:
Specification of elements contained in the RallyConnection section.
<RallyConnection>
| Tag Name | Description | Sample Values |
|---|---|---|
| <Url> Required |
Server used to connect to Rally (excluding the HTTPS prefix, as the connector adds "https://" to whatever is specified). | sandbox.rallydev.com rally1.rallydev.com myRally.mydomain.com 192.168.23.24 |
| <User> Required |
Login name for user to make the Web Services requests to create/update work items in Rally. | user@company.com |
| <Password> Required |
Password associated with the above <User>. Note: The connector will encode (not encrypt) this password string and update it in the configuration file as to avoid clear-text passwords being stored. | mypassword |
| <WorkspaceName> Required |
Workspace in Rally you want to copy/update work items. | My Workspace MyWorkspace |
| <Projects> Required
|
Contains a list of Project tags. Each tag refers to one Rally project that will be used when finding new Rally work items to copy to the other system. For updating work items from Rally to the other system, all projects in <WorkspaceName> are considered. At least one Rally project must be specified in this tag. | <Project>Rally1</Project> <Project>Rally2</Project> |
| <ArtifactType> Required |
Type of artifact you want to create/update in Rally. | (Defect, defect), (Story, UserStory, HierarchicalRequirement), TestCase |
| <ExternalIDField> Required
|
Rally custom string field (name and display name must be identical) that stores the unique id for the other system. Refer to the Create an External ID Field in Rally section above. | CQID |
| <CopySelectors> | Criteria to use when finding new Rally issues to copy to ClearQuest. Multiple criteria are ANDed together. An individual selector specification has the form: <field><relation><value>, where: <field> is the name of a ClearQuest defect field <relation> is one of =, !=, gt, lt, gte, lte <value> is a valid value for the <field>. |
<CopySelector>Status = Open</CopySelector> <CopySelector>Priority = Low</CopySelector> |
| <UpdateSelectors> | Criteria to use when finding existing Rally issues to be updated in ClearQuest. Multiple criteria are ANDed together. An individual selector specification has the form: <field> <relation> <value> where: <field> is the name of a ClearQuest defect field <relation> is either = or != <value> is a valid value for the <field>. |
<UpdateSelector>Release != alpha</UpdateSelector> |
Tip: Escape special characters contained in a Rally Workspace or Project name that are markup sensitive. For example:
">" greater than becomes ">"
"<" less than becomes "<"
Example: "Research & Development" → "Research & Development"
ClearQuest connection
<CQConnection>
| Tag name | Description | Sample values |
|---|---|---|
| <User>
Required |
User to make the API requests to create/update entities in ClearQuest. User needs SQL editor permissions. | myuser |
| <Password>
Required |
Password for user to make the API request to create/update entities in ClearQuest. Note: The first time the connector runs, it will encode the password so it is not saved in plain text. | mypassword |
|
<Database>
Required |
Database schema name for ClearQuest. | SAMPL |
| <DatabaseSet> | Name of the ClearQuest connection that you have setup with the ClearQuest client for Windows. This name may differ based on the how the user named the connection. This tag is not needed if you have only one connection setup. | 7.0.0 MyConnection |
|
<ArtifactType>
Required |
Database table name for entity you want to create/update in ClearQuest. | Defect |
| <Entity> | Use the Entity tag if the ClearQuest system is setup where the record type name and database name are different. | Defect |
|
<ExternalIDField>
Required |
ClearQuest custom string database column name that stores the unique id for the Rally work item. Refer to the Create an External ID Field in ClearQuest section above. | rallyid rally_object_id |
| <ExternalEndUserIDField> | ClearQuest custom string database column name that stores the formatted id (DExx or USxx) for the Rally work item. Refer to the Create an External ID Field in ClearQuest section above. | rallyid rally_formatted_id |
| <TempTimezoneOffset> | Offset from UTC (in hours) of the time zone that the server is running in. For example, for a server running in the Mountain time zone during daylight savings time (7 hours earlier than UTC), the entry would be <TempTimezoneOffset>-7</TempTimezoneOffset>. This optional tag is only needed if entries in the action_timestamp column in the ClearQuest History database are not in UTC. We eventually intend to remove this tag if it becomes possible to query the ClearQuest server directly for its time zone. | -5 |
</CQConnection>
Connector tags
<Connector>
| Tag Name | Description | Sample Values |
|---|---|---|
| <FieldMapping>
Required |
Determine what fields map between the two systems. | See Field Mapping section below. |
| <OtherFieldHandlers> <OtherEnumFieldHandler> | Allows a user to map non-alike drop-down values between the two systems. | See Mapping Drop-Down Values section below. |
</Connector>
Connector Runner tags
<ConnectorRunner>
| Tag Name | Description | Sample Values |
|---|---|---|
| <Preview> | Allows the user to enable a preview mode for testing where NO objects are copied/updated in either system. | false true |
| <Services>
Required |
Determine what type of services to run. Be careful if you update in both directions because whichever system runs the update first will overwrite the updates of the other system. | COPY_OTHER_TO_RALLY, COPY_RALLY_TO_OTHER, UPDATE_OTHER_TO_RALLY, UPDATE_RALLY_TO_OTHER |
| <LogLevel> | Determines what type of messages are written to the log file. The highest level is Debug where all messages are displayed. The default log level is Info. | Fatal, Error, Warn, Info, Debug |
</ConnectorRunner>
Tip: Incrementally set up the connector! Start with a basic configuration file, test that you can connect to ClearQuest and Rally in a "test" environment. Once you validate this is setup correctly, then start customizing the field mapping and field handler sections.
Field mapping
The field mapping section is located between the <Connector> tags in the configuration file and defines what fields map between the two systems.
For example, this definition sets up a mapping between the Rally field Name with the field Headline in the other system. On a create or update, the connector will only update the Name field in Rally and the Headline field in the other system.
<Connector>
<FieldMapping>
<Field><Rally>Name</Rally><Other>Headline</Other></Field>
</FieldMapping>
</Connector>
When you set up your mapping between the two systems, ensure the fields are compatible between the two systems (an integer field should map to an integer field in the other system, a rich text should map to a rich text in the other system).
Otherwise, you might experience situations where information is not created/updated between the two systems and you will see an error in the log file. For example, the connector will post an error for a particular work item if you try to post a string to a custom field of type integer in Rally.
You can add subsequent mappings by appending to this list. For example, this sets up a mapping to Rally Name → Headline, Rally Description → Description, Rally Priority → Priority:
<Connector>
<FieldMapping>
<Field><Rally>Name</Rally><Other>Headline</Other></Field>
<Field><Rally>Description</Rally><Other>Description</Other></Field>
<Field><Rally>Priority</Rally><Other>Priority</Other></Field>
</FieldMapping>
</Connector>
If you are mapping a drop-down value in Rally to the other system, this assumes that the drop-down values match. Otherwise, the connector will throw an error letting you know the value was not found in the list.
If your drop-down values are different between the two systems, see the next section Mapping Drop-Down Values.
Field handlers
Mapping Drop-Down Values
Mapping Users
Mapping Project Fields
4.5 Run the connector
Once the cq_config.xml file is setup, you can start running the connector. On Windows, you can simply open a DOS shell and type:
rally2_cq_connector.exe cq_config.xml
to start the service.
Double-click on WinTail.exe to run WinTail. In WinTail, open the file rallylog.log to watch the log messages generated by the connector. This is an easy way to confirm that the connector is working properly, or to discover any errors.
Tip: We ship a startcq.bat file to make it easier to test. Double-click on the batch file to automatically run the connector and open the log file in WinTail.
To stop the service, use Control-C in the command shell.
Multiple configuration files
For users who want to map to more than one workspace in Rally, need to map multiple artifact types and/or need to map to multiple containers (such as domain/projects in Quality Center or ClearQuest databases) in the other system, setting up multiple configuration files may make sense. Rally recommends naming the configuration files using descriptive names for easier troubleshooting.
To run multiple instances of the connector for different configuration files, pass in the list of config files as arguments to the rally2*_connector.rb script or rally2*_connector.exe.
ruby rally2*_connector.rb config_workspaceA.xml config_workspaceB.xml
or
rally2*_connector.exe config_workspaceA.xml config_workspaceB.xml
The connector will process the configuration files based on the command line argument order, and processes one file at at time.
Once it processes every configuration file, the connector will sleep based on the sleep value. The default is 15 minutes. To change the sleep value between runs, set this value (in minutes) as the last command line argument.
ruby rally2*_connector.rb config_workspaceA.xml config_workspaceB.xml 10
or
rally2*_connector.exe config_workspaceA.xml config_workspaceB.xml 10
*We recommend a sleep value of 10 minutes or more and advise against anything less.
5.1 Troubleshooting the connector
Once the connector is running, all errors are written to a log file in the directory where the rally2_* file resides. Informational messages, warning and errors are written to the log file depending on the value of the <LogLevel> tag in the <ConnectorRunner> section of the config file.
To see the most recent log messages on Windows, rename the extension of the WinTail file from .dat to .exe and then double-click on the WinTail.exe and browse to the rallylog.log file.
Before the connector starts synchronizing objects between the two systems, it performs a validation that confirms that 1) Connecting to Rally is successful, 2) Connecting to the other system is successful, 3) Fields in the field mapping exist in Rally and the other system, and 4) Ensures that each field handler has a corresponding field mapping section in the configuration file.
To confirm the validation is successful without moving objects between the two systems, ensure you set the <Preview> tag in the configuration file to true. This acts as a good trick if you want to experiment with some different configuration options to debug an issue.
If you still cannot determine the root cause of an issue, then please email the entire log file and config file to Rally Support.
6.1 Tips
- Mapping Rally projects: If you are copying defects from ClearQuest to Rally using the RallyReferenceFieldHandler with Projects, you may see a "Could not find Rally Project with name ..." error in the log file if the connector user does not have "Editor" permissions to that project in Rally.
- ClearQuest state workflow: Why am I not seeing my ClearQuest state updates when I update the Defect state in Rally? The connector supports a field mapping between Rally and ClearQuest states. However, if you have setup rules where a ClearQuest state can only be set if certain required fields are set (such as ClearQuest state of Closed requires the Resolution field set), updates may not occur when updating the state from Rally to ClearQuest. ClearQuest will update all other fields on the defect except the state and fail silently. To workaround this issue, you can setup a default value for the ClearQuest required fields in your configuration file. Contact Rally Support for more information on setting up this configuration. Due to the high customization of ClearQuest workflows, another solution we recommend is to use a custom field in ClearQuest that maps to the Rally state. Someone at your organization would then need to create a reconciliation report in ClearQuest that shows any defects that have different states for Rally and the custom ClearQuest field.
7.1 Known issues
- Mapping release/iterations with multiple projects: If you are using a <RallyReferenceHandler> to map Releases/Iterations with the same name and multiple projects, you may receive an error if the Release/Iteration you are updating on the Defect references a different Project than the given Defect is assigned. The connector only returns the FIRST Release/Iteration with the matching name in Rally so it could have a reference to a Release/Iteration in a different project.
- New lines in ClearQuest: Rich text fields like description do not transfer the new lines to Rally. Rally uses HTML whereas ClearQuest uses its own markup language. We plan to support transferring new lines in the future.