Open Source and Vendor Tools – What makes sense?

It’s been a long time (Jazz CLM and Open Source in Sept. 2011 and Jazz Compare with Open Source in Feb 2009) since I posted on the subject of open source tools and vendor supplied tools.  Since most people don’t have the patience to look that far back in my blog’s history, and some things have changed over the interim, it’s time for a new post.  For anyone new to this blog, and in the interests of full disclosure, I work helping enable customers with tools and technologies from IBM/Rational (a vendor).  Hopefully you will find this a useful and objective discussion of the relative merits of these approaches.

The Simple Case for Open Source

So let’s look at an example of the typical argument that most people trot out in support of open source, “Open source is free, the support is much better than what you get with vendor supplied tools, and you avoid vendor lock in“.  Um…. yeah.  Let’s take a closer look at this statement, and see where it falls apart.

Nothing is Free

Open source tools are not free.  People make this false assumption because there are no licensing costs associated with open source tools.  However, you still have the costs associated with provisioning hardware for them, setting them up, and administering and maintaining them.  To be fair, you also have these costs when using vendor supplied tools, however vendor supplied tools tend to focus on administrative concerns, and thus tend to require a little less effort.  Open source tools are getting better in this respect, but there is a whole industry of companies that have a business model based on the administration and upkeep of these types of environments.  In some cases, an approach with one of these companies may be a more cost effective solution for you.

The point to be made here is that you need to evaluate these things with ALL of these costs in mind.  Think to yourself, do I want to maintain these tools and development environments myself, or would I prefer to pay someone else to do it for me?

Nothing is Easy

Open source tools will often make broad and sweeping claims for support.  They point to Git or Maven as examples of open source tools with great user support from the other users, in the forums and the self help sections dedicated to this purpose.  I really admire this type of support, because it can be quite good.  I find the various Linux forums quite useful when debugging my own Linux issues.  I try to get our support areas and forums to adhere to this type of standard (but to do so with a little bit of class and respect – sometimes those forums can be a little rough).  The thing to keep in mind is that while these are examples of good user support through forums, this only really holds true for open source tools that have reached a certain period of longevity, and a critical mass of users and administrators.  Some open source tools do not have this level of maturity or this size of user base.  Those tools may grow to achieve that level of support at some time in the future, but it all depends on the popularity of the tools over a period of time.

When evaluating support, pay attention to how long it takes for the forum to provide answers to complex issues.  Don’t forget, it “counts” even if the original submitter ends up posting the solution.  That is an example of good behavior in these environments, as administrators share solutions with others who might have a similar issue at a later date.  Vendors typically have teams dedicated to addressing issues as soon as you alert them.  Forums tend to be less time critical.

Transparency is Good (or is it?)

Some people claim that, “since I can see the source code, I can be sure of what is going on”.  They have a point.  It also means that anyone who wants to exploit weaknesses in the tool code can easily do so.  The other thing to keep in mind is that many vendors are shipping source code with their tools now anyhow.  Vendors would LOVE it if you made a suggestion to improve their products (“Hey look!  Free developers!”).  And modifying open source is not as easy as some would have you believe.  Often you will need to maintain your modifications to the open source code base, and then port and test these changes with each new release.  Wouldn’t it be easier to just have someone else maintain these fixes for you?

This argument is about code transparency, and should not be confused with business transparency (which I am a BIG proponent for).  It’s not bad or good, it’s just something else that you need to be aware of.  The fact that you can see (and modify) the code has implications that can impact the overall strategic value of your solution.  Be aware of those implications.

The Simple Case for Vendor Supplied Tools

Now let’s look at the reasons that people give for going with vendor supplied software development tools, “Vendor tools have better functionality, they do the code maintenance for us, and they deal with incorporating new technology.  We’ll always have support!”.  Um…. OK.  Let’s take a closer look at this statement and see where it falls apart.

The Cost of Functionality

Vendors love to claim that their tools are “best of breed”.  Often the functionality that can be found in vendor tools is specialized and more advanced than what you will find in an open source tool.  Vendors will provide functionality and usage patterns that will support certain industry standards or regulatory standards.  Often with open source tools, the administrator has to create these type of mechanisms as either a “plugin” or some “bolt on” addition to the open source tool.  This is one of the general strengths of the vendor supplied tools.  As a consumer of these tools you need to evaluate the relative worth of this additional functionality.  If you are doing some simple software development project, with a small co-located team, then you may never even use these capabilities.  Why pay for something that you will never take advantage of?  It’s like buying a sports car in a city where you cannot drive faster than 30 mph.  You are paying a premium for performance that you will never utilize.

However, if you need that additional functionality, then the availability of this specialized capability makes the licensing costs associated with vendor supplied software extremely attractive.  Why should you come up with a custom solution for this issue when somebody has already done all of the thinking, development, and testing for you?  You are not in the software tools business, so why should you dedicate any time or effort into developing software development tools?  You don’t collect rainwater on the roof, you pay for (and take advantage of) infrastructure and work that has already been done by an organization that specializes in delivering water to your office.  Water isn’t that complicated.  The costs associated with having some else worry about it, so you can focus on where you bring value, are seen as an acceptable trade off.

Incorporation of New Technology

Vendors will claim that they will provide the “latest and greatest” technology in their products.  For many vendor products this may be true.  Successful vendor products are like the support of successful open source products in this respect  For vendor tools, once a critical sized user base has been established, there is enough revenue to justify the constant upgrading and incorporation of new technology into the tool.  Vendors have entire teams dedicated to improving their tools, funded with maintenance contracts.  However, if a vendor tool does not have enough revenue associated with it, often the vendor will leave these tools on “life support”, with a skeleton staff tasked with maintaining the tools, but with no real innovation going on.  Just because something is a vendor tool does not ensure that it will be supported for a long time, or that it will continue to incorporate new technologies and innovate.  Have your eyes open, and know what you are getting into.

The Reality

Neither of these approaches seem to address all of the concerns and risks involved with software development tools by themselves.  It leads some to ask the question, “Why use software development tools at all?”.  If you think that you don’t need tools, then spend some time trying to develop a real product without them.  I have, and it’s not very pretty.

What About Integration?

The actual difference between vendor tool functionality and support, and the functionality and support provided by open source tools is not as vast as many vendors would like you to believe.  There is one notable exception to this.  That is the area of integration.

Integration should be a vital part of your strategy when deciding on tools to support your software development environment.  You should be seeking to break down barriers between business analysts, developers, testers, and IT administrators.  That is the whole idea behind the DevOps movement, which uses automation to help break down these barriers and improve communications.

Vendor tools tend to focus on integrations, and will usually have a supported API for integrations to their tools.  Open source also makes use of API’s, but the stability of these API’s isn’t the same as it is with vendor supplied tools.  With open source tools, the focus is commonly on providing extension points for the consumer to write the integrations themselves.  With vendor supplied tools, these integration points exist, but vendors will often make the implementation of these integrations available with a simple configuration change.  It is often how vendors will differentiate themselves from their open source alternatives.

Integrations can be tricky, and they often come with unadvertised consequences.  Is your tool going to integrate with some other tool in your environment?  If the other tool becomes unavailable, will your tool be able to work without a loss of functionality?  Will performance suffer?  Often the best integrations introduce a high degree of inter-dependency between the tools, which can provide a nice user experience, but makes your overall environment tougher to upgrade and maintain.

Probably the most important integration to consider is the one that you don’t know about yet.  You have other tools and automations that you may incorporate in the future.  Does the tool that you are considering provide an easy interface for sharing it’s information, and for linking it’s data to other (at this time unknown) repositories and systems?  You can never really know what the future holds, so it’s important that your tools adhere to your philosophy with respect to sharing data and integrations.  If you plan on using a centralized data warehouse for this, will the new tool be able to populate that data warehouse with the necessary information?  If you like the concept of linked data (which I really like), do the new tools support the idea of leaving data in place and creating linkages between data in different repositories?

The approach that you choose will be up to you, just make sure that your tools are consistent in applying this approach.

What Your Strategy Should Be

So what is a software development organization supposed to do?  Are there no good answers?

The More That Things Change….

…. the more they stay the same.  In my earlier blog post on this subject (Jazz CLM and Open Source) I argued for treating your software development tools like an investment portfolio.  I see no reason to change my thinking.  Many things have changed in the interim, and there are different factors like cloud computing, DevOps, and emerging standards for data sharing (like OSLC).

I still think that treating your investment in tools like a portfolio is important.  Don’t invest in a single vendor’s tools, or in an all open source tool environment.  Either one of these approaches has the potential to leave you vulnerable in different ways.  Financial investors spread their investments around to limit risk, and you should think the same way about your tools.  It is more important to have a software development environment philosophy.  Have some general standards that you consider to be important, and choose the tools that provide the best fit for your philosophy and environment.  Don’t fall into the trap of having a single tools vendor, or insisting on an all open source solution.  A mixture of these will probably be the path that provides the best fit for you, and imposes the least amount of risk.

For example, I would insist on adherence to open standards.  I think that having open standards allows you to more easily transition into and out of new technologies, since huge interfaces do not need to be written (or re-written) to accommodate the changing of one tool in the environment.  I would also insist on easy integrations via REST based technologies.  That allows you to create even simple integrations between tools using hyperlinks between RESTful objects.  Some people insist on more full featured integrations, but I typically find that the more “full featured” an integration is, the more “brittle” it is.  It makes it more difficult to upgrade tools, or introduce new tools into the environment.

I would also insist that the tools selected be able to easily scale to fit the size of my organization.  If you only have 100 developers, then your tools should scale to support those users and more, without large amounts of support staff needed.  Larger organizations will have more significant support requirements.  Keep in mind that security plays a role here.  Some cloud offerings will scale “forever”, but may not support the security requirements that your business requires.  I would look for tools that provide easy an intuitive user interfaces.  Nothing is worse than sitting in front of some tool or application, knowing that it can do everything you could ever want, and having no clue about how to do it.  Finally, I would look at the relative costs of the tools.  This includes licensing, support, hardware needed, and the administrative burden.

Your organization and industry may have some other things that need to be part of your philosophy.  Some organizations would put support of software development process on their list.  Others would consider support for particular regulatory standards.  The key here is to have a consistent philosophy for what you consider important in your tools.  You should then look at your current tools, and any potential tools, through this philosophical lens.

Summary

So you’ve read all of this, so now what should you take away?  You should have a philosophy regarding the tools that support your software development infrastructure.  Have 4 or 5 key capabilities that you value in your environment, and be consistent.  For me, I would consider the following:

• Don’t put all of your investment into a single vendor, or solely into open source.  Diversify to reduce your risk.
• Insist on adherence to open standards
• Look for integrations that allow you to link DATA, not link different tool applications
• Look for scalability to fit the needs of your organization, while keeping security in mind
• Look at overall cost – not just the cost of the licenses, but the cost of the hardware needed, the cost of administration, and the cost of support

and finally……

• Don’t just follow what I say, decide on the key things that are important to your organization.  Choose 6 or fewer things that are critical, make that part of your philosophy on software development tools, and focus on having all of your tools support that philosophy

Other Things To Read

I provide some links below to some of the better arguments for/against both open source and vendor supplied software.  I do NOT agree with the arguments in these articles/blogs, I only supply them to you as a way for you to be informed.  I “get” both arguments, and I agree with some of the points raised on both sides of this.  You should understand both sides of the argument, and decide what works best for your individual situation.  Life isn’t black and white; you ignore the shades of grey at your own peril.  Some of these posts border on fanaticism, but they represent well written arguments and observations on some of these points.

Why Open Source Misses the Point of Free Software – a convincing argument from Richard Stallman, but one that begins to attach too much ethical importance to the development of software.

The Problems of Open Source – a convincing argument from Dr. Tarver that rips Mr. Stallman, but one that tends to discount the quality of some open source products.

Open Source Sucks – one person’s view on the problems of open source….

Open Source Rocks – ….the same person’s view on why open source is great.

Socket.io and the Business of Open Source – a real-world look at how open source is used in producing vendor products, and the impacts of open source on those products.