Git & Atlassian in a Hybrid Application platform
Approximately 70% of all development teams implement a development platform using ‘best in class’ tools.
Others utilise solutions from single vendors to provide a complete end to end development environment.
In this whitepaper we’ll explore some of the issues around software development infrastructure and look at the customer research Clearvision has carried out to better understand these pain points.
The reason many development teams prefer ‘best in class’ compared to a single vendor solution often comes down to four reasons:
Tie in – how many 3rd party companies add value to Collabnet solution verses JIRA or Jenkins;
Compromise on tool performance or functionality – no single vendor has ever created best in class functionality or performance across the whole end-to-end development environment;
Limited vendor bandwidth – if the vendor is not supported by sufficient 3rd party partners, you are limited to the capacity of the vendor; and
Lack of suitably skilled resources in the employment marketplace – how many graduates know how to use IBM RTC verses using git?
To be fair, ‘best in class’ solutions are not without their challenges:
Multiple procurement, support & upgrade points; and
Complexity of integrating multiple tools together.
In 2012, and again in 2013, Clearvision carried out market research regarding ‘best in class’ solutions. The primary goal was to clearly understand the pain points surrounding the implementation and support of these solutions.
What did we discover? How did this shape our product offering; Spectrum?
In order of importance, the pain points we uncovered were:
Maintaining continuity between all the independent tool following upgrades or the introduction of a new tool;
Suitably skilled support staff with the expertise and knowledge to maintain and enhance the integrations between the various tools;
Continuity of processes which flow from one tool to another;
Obtaining a reliable, consistent and accurate truth when constructing reports across the suite of tools;
Adding new users and groups to an environment where each tool has its own user / group / role database;
Time taken to instantiate a new environment; and
Consistent look and feel across all the tools.
As part of the research we asked clients to specify which tools they use across seven disciplines of software development.
The seven key disciplines they came up with were:
Manage – Planning, Review, Reporting;
Design – Requirements capture, Story writing;
Control – Manage change, sprints, Kanban, etc;
Develop – Development lifecycle;
Version – Software version control;
Test – Automated / Managed Testing; and
Release – Artefact build & deployment.
As you would expect, the majority of ‘best in class’ tools span a number of the above seven disciplines. As an example; dashboards and reporting tools need to gather data from many of the tools in the platform.
Of the hundreds of tools in the development marketplace, certain tools stood out as market leaders.
JIRA/JIRA Agile – Addressing aspects of change control and agile planning;
Jenkins & Bamboo – Addressing continuous building and testing; and
Git – distributed version control.
(Contact email@example.com for more details on the results of the market research)
The Growth of Git
The growth of the version control tool git is an excellent illustration of how those in charge of development environments are challenged with removing a legacy tool such as ClearCase, Perforce or Subversion (amongst others) for a more modern distributed tool such as git.
There are many reason for git’s rapid growth in popularity but the underlying point is that git functions differently from any other version control tool on the market. This means it is not a straight substitution for legacy centralised version control tools. You can not simply remove Subversion and replace it with git. Ignoring the challenges surrounding education and the migration of data to a new tool, the impact of transitioning to git affects areas such as workflows, automation (hooks/triggers) and reporting, to name a few.
This might sound like an argument not to make the transition to git. Far from it. The benefits in productivity, if implemented correctly, far outweigh the cost of migrating. The reality is simple – you have to embrace newer, modern tooling in order to stay ahead of the competition and to attract the finest, brightest and highly skilled workforce. Who wants to work on hardware and development tools which are 20 years old?
Summing up some of the challenges:
Refreshing the tools within your development environment is essential in order to retain an effective and efficient environment;
A replacement or new ‘best in class’ tool can often have features which overlap an existing tool;
What looks like a one for one replacement will impact the process and connectivity of existing tools in the stack;
Continuity of data for reports and dashboards can often be skewed following the introduction or replacement of a tool; and
IT staff need to learn the administration of a new tool such that they can support the whole platform.
Breaking down the pain points
If you have got this far, it’s mostly likely because you already operate a ‘best in class’ development environment or are thinking of moving away from closed vendor solutions. The chances are you either have personal experience or have heard of the following pain points:
Continuity of processes & workflows through the tools;
Lack of skilled IT staff to support all the tools;
Single truth of information for reports and dashboards;
Zero to running takes too long; and
Scaling and overall performance.
Continuity of processes & workflows through the tools
Efficient development environments depend on data (information) freely flowing from one tool to another. In order for this to happen, not only must the tools have a physical connection but there must be overarching processes and workflows which extract or interact with each tool. Maintaining the links and continuity of data following an upgrade or a tool replacement often requires maintenance to any hooks or triggers which support the workflow.
Lack of skilled IT staff to support all the tools
The number of major tools which constitute a ‘best in class’ environment generally range from seven to fifteen (not including small plugins or home grown integrations). The vast majority of organisations utilise their own staff to support development tools. Our survey discovered tool support was often shared between:
IT Department – looked after the installation, upgrades, backups, etc; and
Each project having a suitable skilled resource who understood the interdependencies of each tool and responsible for customising the look, feel, workflows, reporting etc.
Occasionally however, both of the above responsibilities would be managed by a centralised team. The common thread and complaint from each company was the misappropriation of resources and effort. Understandably business and development teams want to focus their efforts on building software not building and maintaining infrastructure. Not only were employed resources being poorly utilised but they either lacked sufficient skills to look after the suite of tools or, there were insufficient technically skilled staff to support the number of projects. We have all been in the expensive situation where a project teams stops working because they are waiting on a tool expert to fix a particular problem.
Single truth of information for reports and dashboards
Individual tools are often very good at providing reports and dashboards based on the data stored within their respective database. The problem is such data in isolation can be misleading unless it balanced against data from another tool. Maintaining data consistency following a tool upgrade or the complete replacement of a development tool often consumed tremendous amounts of time. The result was managers and team leaders were using reports which relied upon a certain amount of massaging or manual manipulation.
Zero to running takes too long
Organisations reported there were two different scenarios where company time was wasted:
initial implementation of the development tool environment; and
replication of an environment or provisioning of an environment for each new project.
Typically, an organisation’s first attempt to construct an end-to-end suite of ‘best in class’ tools would consume in excess of 30 man days, even after this period of time the environment would have gaps in the process, reports or other areas.
Once the suite of tools were running, the second challenge related to a project team requiring either their own independent implementation of the tools suit or a secure isolated partition of a centralised environment.
Scaling and overall performance
Connecting a suite of ‘best in class’ tools together is an initial challenge. Understanding where two different tools from two different vendors can cause a performance bottleneck as load or scale is applied is a very different challenge. For example, git and an open source permissions tool under normal load work in perfect harmony. If extreme load is applied (cloning) git works fine, however the permission controlling tool hits a threshold limit thus negatively affecting git cloning.
As with both options (best in class and single vendor), there are always challenges. For the 70% of the market who have weighed up their options and selected a ‘best in class’ route;
Imagine a plug & play framework where the core communication channels between the tools remain constant but the individual tools can be upgraded or replaced without affecting the application development or delivery.
Imagine having access to external support staff who focus is to ensure the latest versions of all the ‘best in class’ tools continue to function after an upgrade, thus freeing up your own staff to develop product not infrastructure.
Imagine having access to a suite of processes (workflows) and reports with dashboards which leverage meaningful data from across all the tools.
Imagine walking out of a meeting with the greenlight to start a new project and in less than 24 hours having a team of developers producing code as a result of using ‘best in class’ tools.
Imagine having the reassurance that someone, other than yourself, has load tested the very same configured environment and identified or resolved potential performance bottlenecks.
Last but not least, imagine not being tied in to any vendor (including Clearvision). The freedom to upgrade and move to the latest and greatest ‘best in class’ tools with all the added benefits of a recruiting staff with matching tool skills.
With Spectrum you’re not wedded to a narrow list of tools. You have the option to mix and match whatever you want, use the best tools available in the market. In other words, go into the shed, pick the best tools out, go out in the yard, and get the job done. That’s really the approach to the whole ecosystem of Spectrum Platform as a Service.
This approach also helps standardize the way that you do things, rather than forcing you to be completely ad hoc, trying to glue stuff together as a one-off every time. After all, the most time and effort in software development happens after it’s been hacked together the first time. It’s the maintenance and upgrades over time that require most of the time and effort.
With Spectrum, you’re not stuck with a one-off solution that’s a beast to maintain, because the process has been standardized. The functionality is exposed in a way that makes sense. So the add, modify, and remove functionality is very simple.
You can run Spectrum behind your firewall, in your private cloud, or wherever you want. On your public server, on a secure IL3 cloud, on IBM Blue Cloud, there really is no limit. When the next new kid on the block comes out with cool new infrastructure, you’ll also be able to run on that.
Spectrum is more than a flexible, integrated group of market leading tools, it’s a complete service with optional training and support. Learn more about Spectrum at http://spectrum.clearvision-cm.com/
For a complete list of open source, Atlassian and other third party tools supported in Spectrum, please contact firstname.lastname@example.org
Don’t stop your learning here! Check out Git 101, your free one stop guide to the basics of Git, including a Git cheat sheet to help you on your way!