IBM DevOps Code ClearCase

The Rational ClearCase family consists of several software tools for supporting software configuration management (SCM) of source code and other software development assets. It is developed by the Rational Software division of IBM. ClearCase forms the base for configuration management for many large and medium sized businesses and can handle projects with hundreds or thousands of developers.

A part of Rational ClearCase is revision control system, which is a feature for end users.

ClearCase supports two kinds of use models, UCM (Unified Change Management), and base ClearCase. UCM provides an out-of-the-box model while base ClearCase provides a basic infrastructure (upon which UCM is built). Both can be configured to support a wide variety of needs. UCM is part of RUP and therefore all process templates and roles can be used from RUP.

ClearCase can run on a number of platforms including AIX, z/OS, Linux, HP-UX, Solaris, and Windows. It can handle large binary files, large numbers of files, and large repository sizes. It handles branching, labeling, and versioning of directories.

History

ClearCase was developed by Atria Software and first released in 1992 on Unix and later on Windows. Some of the Atria developers had worked on an earlier system: DSEE (Domain Software Engineering Environment) from Apollo Computer. After Hewlett-Packard bought Apollo Computer in 1989, they left to form Atria. Atria later merged with Pure Software to form PureAtria. That firm merged with Rational Software, which was purchased by IBM in 2003. IBM continues to develop and market ClearCase.

DSEE

DSEE introduced many concepts now used in ClearCase. The Apollo Domain file system allowed special handler programs to intervene during file access, and DSEE made use of this to invisibly substitute a versioned copy when a particular file was opened. With the versioning specification resident in the user environment, all accesses to versioned files were redirected, including such mundane accesses as printing, viewing in a generic text editor etc.

DSEE relied heavily on a file which described all the software modules and their dependencies. This file had to be generated manually, which was a major impediment to its use in large systems. However, once generated it allowed DSEE to calculate the optimum way to perform a build, re-using all modules that had previously been processed and whose version specifications matched the specifications for the current build.

DSEE also introduced the "version spec" which was then called a "thread". This was a list of possible versions that could be in the user environment, or in a build. The major innovation was using build signatures and software release signatures in the thread. The items in a thread might thus be:

• Any copies reserved for editing (i.e. checked out)
• Latest version (usually for developers only)
• A branched version of a file (a version on an alternate line of development).
• A labeled version (for developers working on a particular revision level)
• The version used in build XYZ.
• The version used in software release x.y.z.

Threads were processed from top to bottom for each file. A developer thread might have "reserved" at the top, followed by a labeled version. For a quick fix to an existing release, the thread would be "reserved", then the release signature.

In the absence of the invisible file redirection of the Apollo Domain file system, Rational ClearCase uses the virtual file system provided by the MVFS feature described below. The "thread" concept corresponds to the dynamic view. Support for derived objects in a view is similar to DSEE's concept.

ClearCase releases

• Version 7.1 (December 2008) introduced many changes and improvements for both the CCRC clients, a new CM Server platform (a generic WebSphere application server platform for ClearQuest), a generic programming model for both CC/CQ, a new multi-site monitoring system and more.
  • Version 7.1.1 (December 2009) introduced a number of new features and improvements including atomic commits.
  • Version 7.1.2 (October 2010) introduced a number of new features and improvements including supporting of Ubuntu, Windows 7 User Account Control (UAC) and smart card authentication.
    • Version 7.1.2.2 (April 2011) introduced a number of new features and improvements including supporting of Visual Studio 2010.
    • Version 7.1.2.3 (June 2011) introduced a number of new features and improvements including supporting of Windows 7 and 2008 R2 Service Pack 1 and also Visual Studio 2010 SP1.
• Version 8.0 (October 2011) The highlights of this release are a new dashboard component called ClearTeam Explorer (CTE), improvements for both ClearCase Remote Clients (CCRC) and native clients, and a new VOB schema version (80). The CTE client supports both dynamic view and web view access to ClearCase through a single, unified GUI.

Infrastructure

Database layer

The database system that ClearCase uses is RDM Embedded from Raima. In ClearCase terminology, an individual database is called a VOB (Versioned Object Base). On this layer, maintenance takes place using Raima tooling. Around this layer, a set of interfaces with accompanying tools is available to manage the physical database system. This requires specific Database administrator skills.

Server layer

Before version 7 every system, both clients and servers, would run the same ClearCase services or daemons. The most important service is called the Atria location Broker Daemon (ALBD) which takes care of all the communication between stations. After version 7 the server platform runs Websphere Application Server with a server application called the "CM Server" which at the same time serves Rational ClearQuest clients and Rational ClearCase Clients via the HTTP Protocol. Before version 7 there was a webservice by which users could access ClearCase via their browser but this has been discontinued. The older client service architecture is still supported. Before version 7 load balancing would be more difficult (specific services would have to run on specific servers) to implement after version 7 it is just a matter of putting multiple servers in sequence.

In a multi-site scenario every server runs ClearCase MultiSite Server and sends packages back and forth (via any protocol available) to synchronize.

Clients

Traditionally ClearCase supported full/fat clients. Since version 7 the CCRC 'ClearCase Remote Client' is available which is based on Eclipse (software). It is supplied in both fully packaged Eclipse versions as well as a plugin-in for Eclipse and other environment such as Visual Studio. Communication takes place via the HTTP Client protocol.

The CCRC client is now the preferred client.

Views

A distinguishing feature of ClearCase is a proprietary networked filesystem (MVFS: MultiVersion File System), which can be used to mount VOBs as a virtual file system through a dynamic view, selecting a consistent set of versions and allowing for the production of derived objects. The dynamic view allows this to map to a Software Configuration. This was a departure from the repository/sandbox model, allowing for the early management of artifacts (before they are checked in, and not limited to these first order configuration items).

Alternatively, ClearCase supports snapshot views which are just copies of repository data specified by a config-spec which spans one or several VOBs. As opposed to dynamic views, snapshot views are maintained on a local (OS-specific) filesystem and do not require network access. Instead, a snapshot view stores a copy of the VOB data locally on the user's computer. Snapshot views can be used while disconnected from the network and later resynchronized to the VOB when a connection is reestablished. This mode of operation is similar to how the widely-used CVS (Concurrent Versions System) software works.

From the perspective of software on the client computer, a view appears as just another file system. If new data is created in a ClearCase view then ClearCase will refer to the new data (files, directories, ...) as view-private to indicate it is specific to the view being used and not being version controlled (private) by ClearCase. This allows build systems to operate on the same file system structure as the source code, and ensures that each developer can build independently of each other. A view-private object can be added to source control and become a versioned element at any time, making it visible to other users.

Each developer typically has one or more views at their disposal. It is sometimes practical to share views between developers, but sharing branches is usually used instead. Having a branch hierarchy is often useful, so an entire development project shares a common development branch, while a smaller team shares a sub-branch, and each developer has his or her own private branch. Whenever some change is deemed stable enough for a larger group, it can be merged to the parent branch.

Configuration specifications

Under base ClearCase, each view is controlled by its associated configuration specification, commonly referred to as a config spec. This is a collection of rules (stored internally in a text file, but compiled to a more efficient form before use) that specifies what elements (files or directories) should be visible in a view, and which versions of these elements. When deciding which version, if any, of an element should be visible, ClearCase traverses the configuration specification line-by-line from top to bottom, stopping when a match is found. Earlier rules always take priority over later ones.

A typical configuration specification could look like this:

# Show all elements that are checked out to this view, regardless any other rules.
element * CHECKEDOUT

# If an element has a version on the 'module2_dev_branch', then the latest
# version of this branch shall be the visible version in this view.
element * .../module2_dev_branch/LATEST

# For all files named 'somefile', regardless of location, always show the latest version
# on the main branch.
element .../somefile /main/LATEST

# Use a specific version of a specific file. Note: This rule must appear before
# the next rule to have any effect!
element /vobs/project1/module1/a_header.h /main/proj_dev_branch/my_dev_branch1/14

# For other files in the 'project1/module1' directory, show versions
# labeled  'PROJ1_MOD2_LABEL_1'. Furthermore, don't allow any checkouts in this path.
element /vobs/project1/module1/... PROJ1_MOD2_LABEL_1 -nocheckout

# Show the 'ANOTHER_LABEL' version of all elements under the 'project1/module2' path.
# If an element is checked out, then branch that element from the currently
# visible version, and add it to the 'module2_dev_branch' branch.
element /vobs/project1/module2/... ANOTHER_LABEL -mkbranch module2_dev_branch

A configuration specification can also reference other configuration specifications using the 'include' statement.

Under ClearCase UCM these config specs still exist but do not need to be maintained or edited anymore by end-users or administrators.

Features

• Build auditing: The networked filesystem provided by MVFS allows for build auditing. Builds in views which use the MVFS are able to monitor and record file I/O operations performed during the build process, and associate each such event with the command which triggered it. This allows ClearCase to produce a bill-of-materials which it calls a Configuration Record (CR) for all builds and enable traceability for either software configuration management purposes or as part of a larger Application Lifecycle Management process. Build auditing is performed with command-line tools such as a built-in make tool (omake, clearmake) or by using the clearaudit command which can invoke a separate build tool.
• VOB (Versioned Object Base): A repository that stores versions of file elements, directory elements, derived objects, and metadata associated with these objects. With MultiSite, a VOB can have multiple replicas, at different sites.
• Configuration Record: The bill-of-materials artifact produced as the result of build auditing that contains:
  • Build procedure: The method (script, makefile, ...) that invoked the build
  • Inputs: All files (and their specific versions) that were used for a particular build.
  • Outputs: All derived object (DO) files (and any dependent DOs) produced as a result of the build.

The dependency information is stored in a hidden configuration record which can be shown for each derived object. The configuration record can be used to set up another view showing all files that have been read before during the build time. Alternatively, you can use the configuration record to apply a label to the files (and versions) that were read during the build.

• Build Avoidance: Use of MVFS (MultiVersion File System) allows derived objects built in one dynamic view to be automatically "copied over" to another dynamic view requiring "exactly the same" derived object. Two derived objects are deemed to be "exactly same" if they have the same configuration record (ClearCase terminology, also called bill of materials). Shared derived objects will be physically present on the VOB server, and not in the views that reference them. The process of "copying over" is called winking in in ClearCase terminology. This feature requires that the clearmake or omake tools are used instead of other build systems.
• Unix/Windows Interoperability: VOBs hosted on *nix (Solaris, Linux, AIX, HP-UX, IRIX primarily) servers can be accessed with dynamic views, snapshot views, or the new web protocol based client: the CCRC on Windows clients. VOBs hosted on Windows servers can be accessed with snapshot views or CCRC from Unix clients, but not dynamic views due to the Windows server returning file paths with backslashes as the path delimiter. There is a 3rd party patch for Linux to enable dynamic views from Windows VOBs.
• Integration With Other Products: Other products (originally) from Rational Software, notably ClearQuest and Rational Rose, integrate with ClearCase. ClearCase also integrates with TextPad, Microsoft Visual Studio, Code::Blocks, NetBeans and the Eclipse IDE through a plugin. There are also Emacs and Vim plugins available.
• Space Saving: ClearCase dynamic views can save huge amounts of space and allow for some measure of a shared work environment.

Weaknesses

• Speed: ClearCase dynamic and snapshot views are slower than local filesystems, even with good network infrastructure. Repeated subsequent builds may run dramatically faster, due to build avoidance using ClearCase's make substitute. Because MVFS requires server access every time a file is accessed, the performance of the file system depends on server capacity.
• Sensitivity to network problems: Because MVFS is essentially an online file system, failures of the server or network render the dynamic views unavailable. Offline access to the MVFS requires working in a snapshot view. A snapshot view may be created with an identical configuration specification. If the view data for dynamic views is stored on a client host (as may be done for performance reasons or to reduce the need for servers), ClearCase does not automatically handle host IP address changes. (2007)
• Cost of Ownership: ClearCase costs more than free, open-source, and some other commercial SCMs.

See also

• List of revision control software
• Comparison of revision control software
• Rational ClearQuest
• IBM Rational ClearCase UCM

References

• Posner, John (1994). "CASEVision/ClearCase User's Guide, Document Number: 007-2369-001". Silicon Graphics, Inc.

External links

• Rational ClearCase v7.1 Information Center
• Rational ClearCase Commands (official)
• Rational Clearcase Commands (not official)
• Rational ClearCase Commands Reference (not official)
• IBM Rational ClearCase product description