The ClearCase Bridge Administration Guide

Introduction

Welcome to Clearvision's ClearCase Bridge: Integrating Subversion and Git with ClearCase.

The ClearCase Bridge lets users of Subversion and Git carry on using their repositories while at the same time, automatically sharing code with Clearcase teams who are using Clearcase tools. Changes made at either side are seamlessly transported across the bridge to the other side. This way teams can collaborate across version control systems allowing Enterprise centralization and control while allowing localized freedom for other parts of the team.

This can be used as a permanent 2-way bridge, or to support a strategy of slowly migrating to another SCM System.

The ClearCase Bridge (also known as CC2X) implements both CC2SVN and CC2GIT functionality. The distribution package and executables are therefore named using the cc2x prefix. Note that there is only one distribution for the ClearCase bridge that contains both products. Provided you have licences for both products, you can use the same installation to create Git and Subversion bridges.

DISCLAIMER: Please read this guide carefully and plan your bridges well. Clearvision cannot be held liable for any corrupted data. Evaluations of the ClearCase Bridge must not be performed on live systems.

Overview

The ClearCase Bridge provides a bridging mechanism that allows you to create a transparent link between your Subversion and Git repositories on one side and ClearCase on the other.

Data checked in to either end of a bridge will automatically be transferred to the opposite end. The ClearCase Bridge operates asynchronously, meaning that when changes are committed in Subversion or Git or checked in to ClearCase, they are transferred in the background to the associated SCM system. The ClearCase bridge checks each bridged repository location on a regular basis to scan for new changes, which are then transferred to the other end of the bridge automatically.

Linking between SCM systems is achieved through bridges. Internally, each bridge maintains two workspaces: a ClearCase workspace (i.e. view) and a Subversion or Git workspace. The bridge synchronizes between the two workspaces and deals with updating and commiting changes to and from these workspaces.

The current release of the ClearCase Bridge focuses on transferring the main content that is held in files. It ensures that a consistent set of files is available at either end of the bridge. (Note that it does not aim to ensure that every revision is available in both repositories, nor does it transfer any revision metadata such as revision properties or tags). The main aim of the ClearCase Bridge is to make file contents available at both ends of a bridge.

Some of the basic meta-data such as information about who made a change and why is bridged by using check-in comments.

Planning Bridges

Before setting up bridges, please read through this guide carefully to allow you to make an informed decision about your bridge setup. You can save time (and effort) by planning your bridges well and creating a suitable setup.

Note that if you later discover that a setup was not suitable for your needs, you may need to perform manual modifications in one or more of the repositories in order to rectify the situation. Time spent on planning bridges up-front can save you time in the long run.

Setting up a New Bridge

When you first set up a new bridge, it is strongly recommended that one of the repositories is used as the master for the initial synchronization between ClearCase and Subversion or Git. Being the master means that only one side contains data in the location specified for the bridge, and the other side points to an empty directory. CC2X automatically detects data to be moved accross the bridge

If you were to bridge between two repository locations that already contain data, you would effectively merge the two locations together, which can result in conflicts that you would need to resolve manually. If you do need to merge two such repositories together, it is recommended that you first perform this merge manually on one side and make the merged data the master for the initial transfer. This will reduce the risk of the operation not having the desired result, and it will make it easier to undo any undesired results.

When you define a bridge, please note that both of the ends of the bridge must exist, i.e. if you set up a bridge from http://host.domain.com/svn/myrepository/trunk/mycomponent (on the Subversion side) to M:\\my_view\myproject\mycomponent (on the ClearCase side), then Subversion must contain a directory called mycomponent at the specified location, and the ClearCase element mycomponent must also exist and be accessible via the specified path. In terms of masters for the initial transfer, if Subversion is the master for this bridge, then ClearCase Bridge would expect to find data for mycomponent in the Subversion repository and an empty directory mycomponent in ClearCase. Any sub-directories will be coppied over the bridge as part of the initial transfer so do not need to be manually created.

Note that once the initial transfer has been performed, neither of the two ends of the bridges retain any particular ownership (or master relationship). This means that you can then edit changes in either location and they will be transferred to the other end of the bridge automatically.

In order to reduce conflicts on the ClearCase side, and in order to highlight the origin of the data, you could set up a special subversion or branch to import Subversion data into. In addition to reducing synchronization conflicts, using a branch will also give you flexibility on how to use the data in views. What branch to use for the ClearCase Bridge is defined through the config spec of the view used for the bridge.

On the Subversion side, you could use the ClearCase Bridge to only transfer particular baselines of a product using a bridge from the tags directory, or you could point to the trunk or particular branches in order to transfer development data. A combination of branches, tags and the trunk may also be useful. However, since the ClearCase Bridge does not translate between ClearCase and Subversion tagging concepts, you need to take care when setting up bridges for multiple tags.

Multiple Bridges for the Same Repository

There are situations where it is useful to install multiple bridges between the same two databases. For example, you may choose to only bridge certain development branches of the ClearCase database with Subversion. This can be achieved by installing bridges on all branches to be bridged, configuring the ClearCase config specs for the workspaces to choose versions from specific branches.

When installing multiple bridges, it is strongly recommended to avoid overlap between the targets of different bridges. Take the following example:-

In the example above, the src data tree from the Subversion trunk as well as branch1 from the branches area would both be bridged into the same view, i.e. the same branch in ClearCase. The result would be a mix of branch data and trunk data with no particular value.

What the user intended to do in the example is a valid use case. You can achieve the underlying goal of the example by creating two views with config specs for two different branches. For example you could bridge .../trunk/src into a view configured for the main line and then bridge .../branches/branch1/src into a view configured for a branch. This will keep main line (i.e. trunk) and branches in line between ClearCase and Subversion, i.e. data checked in to the ClearCase branch will be transferred into a Subversion branch, and the main line in ClearCase is transferred into the trunk in Subversion.

The config spec for bridge2_view would be configured as follows in order for the view to operate on the branch:-

element * CHECKEDOUT
element * /main/branch1/LATEST
element * /main/LATEST -mkbranch branch1

Please bear in mind the following implications of creating a bridge that operates on a branch:-

• If the bridge is set up with no data in ClearCase and a branch in Subversion, the bridge will create branches for every element checked in to ClearCase. This is due to the branching model in Subversion effectively creating a copy of every element in the branch.

• If, however, the bridge is set up with no data in Subversion and a branch workspace in ClearCase, you are more likely to get the desired result, i.e. you only have branches in ClearCase for elements that have been changed on the branch.

Installation Considerations

A ClearCase Bridge system has four key elements:

1. ClearCase

2. Subversion and/or Git

3. ClearCase Bridge Server

4. ClearCase Bridge Webadmin Server

You need to consider the following points before setting up CC2SVN or CC2GIT:

• The ClearCase Bridge server must be run on a supported platform.

• Both a ClearCase and a Subversion client must be installed on the ClearCase Bridge server host. cleartool, git and svn must be in the PATH of the user id used for running the ClearCase Bridge server.

• On Unix systems, please set the user's umask to a suitable value so that ClearCase creates directories with suitable permissions. This is a common issue with ClearCase. If in doubt, please set the umask for the user id used for running the ClearCase Bridge server to '000'.

• The Subversion server can run on any machine accessible through the network from the ClearCase Bridge server host. There is no need to run the ClearCase Bridge server on the same host as the Subversion server.

• ClearCase VOBs can be hosted by any host within your network, provided they are accessible from the ClearCase Bridge server host.

• Minimal Subversion hooks must be installed in the Subversion repository on the Subversion server host; see below for details.

• A special ClearCase attribute type must be created and applied to any VOBs being bridged

• There is no need for any ClearCase Bridge client software to be installed on ClearCase and Subversion client machines.

All ClearCase, Subversion server and ClearCase Bridge server may reside on the same host. However, provided the points above are followed, you can set up the ClearCase Bridge as a standalone machine within your network.

Constraints and Limitations

The ClearCase Bridge has a number of constraints and limitations outlined below:

• It does not translate between ClearCase and Subversion conventions. For example, Subversion uses branches and tags locations in the repository for management of branches and tags. When transferring data to ClearCase, these areas are treated as ordinary repository data with no special meaning attached.

• SVN externals are not allowed within bridges.
• It does not transfer checkin/commit log messages, branches, tags or other metadata. Only raw file data is transferred.

• The ClearCase Bridge requires ClearCase dynamic views and currently does not work with snapshot views.

• The account used for running the ClearCase Bridge server must have adequate access permissions to all areas to be bridged in ClearCase.

• The Subversion account specified in the bridge configuration must have adequate access permissions to all areas to be bridged in Subversion.

• On the ClearCase side any folder called lost+found at the top of a bridge will be ignored.

Supported platforms

CC2SVN and CC2GIT are currently supported on the following platforms (please note that we plan to support other platforms, contact sales@clearvision-cm.com for more info).

• Linux CentOS 5.3 32bit
• Windows XP 32-bit

• Git version 1.6.3 or later.
• Various Subversion versions (please see the website downloads page for more details).

Licence

The ClearCase Bridge requires a license, which can be obtained from sales@clearvision-cm.com. Note that ClearCase Bridge licenses are bound to particular machine and support up to a maximum number of bridges. Licenses are also time-limited.

To request a license you can use the get_license_info executable and run it on the machine to be used as the ClearCase Bridge server host. The get_license_info tool can be found within the tools directory of the zip package. This will retrieve system information from the machine and present you with a license request to return to Clearvision at the below address. Alternatively, you can fill in the details below and send them to sales@clearvision-cm.com:

License Request for Clearvision CC2SVN and /or CC2GIT:
     Evaluation: yes/no
     IP Address: 192.168.64.1
           Port: 54329
    MAC Address: 00-50-56-C0-00-05
          Users: 50
        Company: You Company name here
  Email Address: yourco@yourco.com
    Expiry Date: 2010-03-31
        Product: CC2X
 Num of Bridges: 5
    Svn Bridges: yes/no
    Git Bridges: yes/no

The get_license_info executable is run from a command prompt, e.g. on Windows:

Cut and paste the information printed by get_license_info, check all the fields, fill in the empty ones (Company Name, etc) and send to sales@clearvision-cm.com.

Installation

To install the Clearcase Bridge products, you will need to do the following on your CC2X Server machine:

• take case of prerequisites (user ids, basic configuration of !Clearcase, Subversion and Git)
• install CC2SVN or CC2GIT (whcih has 2 parts: the main Server and the administration GUI Server)

Prerequisites

The following prerequisites need to be in place before you can start using the ClearCase Bridge

CC2X Server User Account (on Unix)

User Id (and Group)

If you are running on Unix, we recommend setting up a user id (eg cc2x) to run the CC2X Server and contain the workspace and config files, etc. You can use this to control access to ClearCase, Subversion and Git appropriately, and as the user that runs the server process.

Note: Please make sure that you set the human-readable name of the user (ie the 5th field in /etc/passwd, see the chfn command) otherwise you may have problems with Git.

PATH considerations

On Unix, make sure that ClearCase, Subversion and Git client software is usable by the CC2X User Id.

ClearCase

ClearCase client software must be installed on the CC2X Server.

The ClearCase Bridge does not automatically start ClearCase VOBs and views, so ClearCase VOBs must be mounted and the relevant ClearCase views started before the ClearCase Bridge server is started.

• The installation consists of three parts: the ClearCase Bridge server and Webadmin Server, Subversion hooks and ClearCase attribute type creation.

ClearCase Attribute

The ClearCase bridge requires a special attribute type to exist for all ClearCase vobs being bridged. Please create the attribute type by changing your current working directory into the vob directory under a suitable view and run the following command:-

cleartool mkattype -vtype integer clearvision_cc2x

( Note that you will have to do this for ALL Vobs that are bridged, so please check that this has already been done before you add a new bridge!)

Subversion (for CC2SVN Bridges)

The ClearCase Bridge does not automatically start ClearCase VOBs and views or Subversion servers. Subversion must be up and running, ClearCase VOBs must be mounted and ClearCase views started before the ClearCase Bridge server is started.

• Subversion client software has been installed on the CC2X Server host
• The Subversion server is up and running and access has been configured for the CC2X user id

Subversion Hooks

The ClearCase bridge requires pre-revprop-change and post-revprop-change hooks to be installed on Subversion repositories. If you do not currently have such hooks, please create hooks in the hooks directory as follows:-

On Unix platforms:

 #!/bin/sh
 exit 0

Make sure that the script is executable, i.e. chmod 755 pre-revprop-change post-revprop-change.

On Windows platforms please create pre-revprop-change.bat and post-revprop-change.bat hooks in the hooks directory with the following content:

 exit 0

Git (for CC2GIT Bridges)

Git software must be installed on the CC2X Server host.

The central (bare) Git repository is up and running; it has a master branch in it and access has been configured for the CC2X user id.

ClearCase Bridge Installation

Start by unpacking the zip file on the machine to be used as the ClearCase Bridge server host. Note that files can be extracted in any location. However, recommended locations are C:\Program Files on Windows and /opt on Linux hosts. The following directory structure will be created:

• /opt/Clearvision/cc2x/
  • admin
  • docs
  • media
  • server
  • templates
  • tools

• On Linux, adjust permissions using chmod -R 755 Clearvision or chmod -R a+x Clearvision

Please take care when editing any configuration files. Text editors such as Vim or Wordpad or Emacs can be used for editing. However, avoid Notepad or MS Word as extra control characters may be introduced, causing CC2SVN to fail.

(Note: It is recommended to install and run the ClearCase Bridge server and the Webadmin server under the same user id. If you do need to use different user id's, then please make sure file permissions are set appropriately on the workspace directory, in particular the webadmin subdirectory and the files it contains.)

Configuration

Copy the template config file /opt/Clearvision/cc2svn/server/cc2x_server.cfg to an appropriate place, eg on Unix you might use /home/cc2x/.cc2x.cfg, on Windows, c:\cc2x\cc2x.cfg

Edit it to configure the following parameters:

(Note: avoid Notepad or MS Word as these may introduce unwanted characters, corrupting config files)

[General]
; Working area for the Clearcase Bridge
workspace=/opt/Clearvision/cc2x/workspace

; The location the product is installed into
install_location=/opt/Clearvision/cc2x

; The amount of delay between each bridge sync
update_delay=60

; Email server information to allow the !ClearCase Bridge to send email notifications on problems
smtp_host=mail.server.com
smtp_username=user
smtp_password=password
from_address=user@localhost
; Default recipient for email notifications. Can be overridden in webadmin interface
default_resolvers=user@localhost

[License]
company_name=Your Company Plc
licence_email=admin@yourcompany.com
expiry_date=2008-12-31
bridges=999
allow_svn_bridges=1
allow_git_bridges=1
license_key=PLEASE_ADD_YOUR_LICENCE_key_HERE
license_certificate=PLEASE_ADD_YOUR_LICENCE_certificate_HERE

[Server]
; Define this ClearCase Bridge server machine as per your license
server_ip=192.168.1.2

[WebAdmin]
; Port for webadmin server
webadmin_port=8181

The License and Server sections must match the licence received from Clearvision. Any changes to the License or Server section will invalidate your licence.

The workspace defined in the config file must point to a local file system and not a network drive. Also, on older Windows systems, if the workspace lives on a FAT file system, you will need share.exe running (for file locking to work correctly).

Logging

The ClearCase Bridge keeps log files for both the ClearCase Bridge server and the Webadmin server. You can configure the location of the log files using the log_file and web_log_file options, and the level of detail using the log_level option. The level can be one of the following:

• INFO: This provides verbose output about the operations of the ClearCase Bridge, which is particularly useful when investigating issues. Please bear in mind that due to the amount of output generated, the log file will grow quickly on this setting. INFO is a good log level to choose when you first install the product and configure bridges as it will give you the opportunity to review progress.

• WARNING: (the default) This shows warning and error messages, including details of possible issues. You may want to operate on WARNING level for a few days until you are satisfied that the product has been configured correctly, and switch to ERROR when you are happy.

• ERROR: This shows only error messages (minimal output), keeping the log file small.

For example:

log_file=/var/log/cc2x/cc2x_server.log
web_log_file=/var/log/cc2x/cc2x_web.log
log_level=INFO

ClearCase Bridge Webadmin Server

The Webadmin server must be started before the ClearCase Bridge server. This will create and configure important data structures required for the ClearCase Bridge server.

Please note that for the Webadmin server to operate correctly, you currently need to change your current working directory to the location of the Webadmin server executable and start the server from there (this will be fixed in a future release). You will also need to point to the location of the config file using the -f option:

cd /opt/Clearvision/cc2x/admin
./cc2x_webadmin -f /home/cc2x/.cc2x.cfg

ClearCase Bridge Server

Start the main server as follows:-

/opt/Clearvision/cc2x/server/cc2x_server -f /home/cc2x/.cc2x.cfg

The ClearCase Bridge server and Webadmin server must share the same config file to ensure that they are both configured to use the same workspace. It is via this workspace that the ClearCase Bridge server and Webadmin server exchange information.

Note that after you have added your bridges, you can safely stop the Webadmin server and leave the main server running (you only need the Webadmin Server running if you want to make changes).

Bridge Configuration

Bridges connect ClearCase and Subversion/Git through a Subversion/Git workspace and a ClearCase view. Note that whilst the ClearCase view must be created manually, the ClearCase Bridge will automatically create internal Subversion/Git workspaces in the location specified by the workspace configuration option.

Note that ClearCase views need to have a suitable Config Spec. For Example, if you are bridging with the main line in ClearCase, the following config spec would be suitable:

element * CHECKEDOUT
element * /main/LATEST

Every ClearCase view used for a bridge must have element * CHECKEDOUT as the first rule to ensure correct operation.

Also, please use dedicated views for bridges, i.e. do not use another user's view for a bridge as otherwise the bridge could inadvertently transfer checked out data or view-private files.

Bridges are configured in the web-based graphical user interface. Use a web browser to browse to the location of the admin server, e.g.: http://my.server.com:8181.

You will then be presented with the following login page:-

The default user name and password for the ClearCase Bridge Webadmin server are admin and admin. Once logged in, you can change your password using the Change Password link at the top right of the screen.

If you typed the correct user name and password, you are presented with the Bridge Configuration view. In order to add a new bridge, fill in the form at the bottom of the page. The following table has some additional information on the fields to be filled in:

Please note that while both CC2SVN and CC2GIT are configured through the same interface, they are two separately licensed products. You will only see the configuration options for the types of bridges you are licensed to use. Depending on your licence, you may therefore only see one of the two tables in the ClearCase Bridge configuration view.

In order to add new bridge, fill form at the bottom of the table and, when you are finished, press Apply. The newly configured bridge will then appear in the table as shown in the following screenshot.

You can make certain changes to existing bridges. However, the only settings that you are currently allowed to change is the list of resolver email addresses and the Subversion user name and password. For Git, the only setting that can be changed is the resolver emails.

You update information for a bridge by ticking the box in the Change column of the table, changing the information directly in the table and clicking Apply at the bottom of the screen.

Note: Both CC2GIT and CC2SVN can use whole repositories or a subdirectory within one for a bridge. It is recommened that whole subversion repositories are NOT used as Subversion uses branches and tags locations in the repository for management of branches and tags. When transferring data to ClearCase, these areas are treated as ordinary repository data with no special meaning attached.

Should you need to delete a bridge, you can click on the icon to the right of the bridge. Please note that deleting a bridge will not only stop the bridge but also reset internal metadata. If you recreate a bridge for the same view and repository location at a later stage, the ClearCase Bridge server will retransfer any data previously transferred. If you do need to delete and recreate a bridge, it is advised that you do not make any changes at either end of the bridge until the bridge has settled again (i.e. until the initial transfer has been completed).

Dealing with Merge Conflicts and Other Error Notifications

You are strongly advised not to ignore any error messages emailed to the list of resolver email addresses. If you receive such an email, it is very likely that the issues is permanent until resolved, even if you do not get any error messages again.

The most common issues are related to merge conflicts, which can occur if the same file is changed in Subversion/Git and ClearCase at the same time, and when the bridge comes to transfer changes across, it realises the conflict and informs the user.

If you get an email about a merge conflict, you can resolve the issue by checking in a fix changeset at either end of the bridge. The fixed files will then be transferred in the usual fashion with the next regular update and the two ends of the bridge will be back in sync.

Please refer to the User Guide for more information.

Troubleshooting

If you experience any issues during the installation please report your issues via the Clearvision Support Centre, which can be found at http://www.clearvision-cm.com/support-home/support-center.html.

This troubleshooting guide provides information on dealing with the most common issues that you may encounter.

Workspace Clean-up

Under rare circumstances it may become necessary to clean up the Subversion/Git working copies inside the ClearCase Bridge workspace. You can find the working copy for a bridge inside the workspace directory specified in the config file under svn (or git respectively), within which you should see a subdirectory with the name of the bridge. Please do not delete the subdirectory but only inspect the working copy itself using svn status or git status and fix any problems you may see.

Forcing a Resynchronisation

If a large number of errors have occurred, and if you have already fixed the cause of the errors in the repository, it may be necessary to force a complete resynchronisation of a bridge. At the moment, this can only be achieved by deleting the bridge and recreating it with the same settings.

Please note that during a resynchronisation all data will be sent across again from both ends of the bridge, and it is therefore advisable to leave for the bridge to settle before resuming work on the areas bridged. This is to prevent user changes from being overwritten by the bridge update.

Database Access Fails

If you see issues with locking of the cc2x.db database file, then consider the following:-

• The workspace setting in the config file may have been configured to point to a network drive. There are known issues with network drives and file locking, which is required to work for database access. Please reconfigure this setting to point to a local file system.

• On Windows this may be caused by the workspace setting pointing to a drive with a FAT file system. If this is the case, please make sure that the windows share process share.exe is running in the task manager.

• On Unix, you may have started the ClearCase Bridge server and the Webadmin server under a different user id. If this is the case, please adjust file permissions on cc2x.db accordingly.

Appendix A: CC2Git on Windows

This section describes how to set up a remote Git repository hosted on Windows.

Prerequisites

There are two flavours of Git released for Windows Msysgit and Cygwin Git.

Msysgit is a native Microsoft Windows port for the Git client, which does not come with host services to allow the sharing of public repositories.

In order to use ssh or the git protocol to share a public repository you can use Cygwin. Although it is larger and slower, it does come with an ssh server and the git-daemon.

Note that you can use cygwin on the server and run git-daemon or ssh and still use msysgit on the clients.

Steps

This example will use the ssh protocol.

Server or remote repository

Setup

• Install Cygwin
  • Select Packages

    • Devel > git

    • Devel > git-completion

    • Devel > git-gui

    • Devel > gitk

    • Net > openssh

    • Python > python

• run Cygwin
• Configure openSSH

  • run ssh-host-config

  • Follow onscreen instructions
• Start SSH

  • net start sshd

• Open Windows Firewall and create an exception to allow TCP traffic on port 22.

Create Repo

When pushing to a remote git repository, it is easier if you push to a bare repository $ git --bare init . If you want to push to a non-bare repository, you need to do a $ git reset --hard HEAD and $ git checkout -f on the remote server to sync the heads up. The latter is only recommended if you need to interact directly with the files on the remote server. The issue seems to be designed to prevent changes made locally on the remote repository being lost when changes are pushed from elsewhere.

• git --bare init

Client or local repository

Setup

• Install Cygwin or Msysgit
• Setup public keys
• Navigate to directory you wish to store repository

• Clone Git repository git clone ssh://hostname.domain/path/to/repo.

If you recieve "warnging: remote HEAD refers to nonexistent ref, unable to checkout." the first time you clone a remote repository, this is likely to be becasue you are cloneing a bare repository. Adding, commmiting and pushing a file should resolve any problems.