Tag Archives: API

Does your LMS play nicely with others?

With the recent release of Learn with Mobile’s latest RESTful and webhook APIs, I found myself feeling very grateful that open interoperability and integration have started to become standard practice for leading cloud service providers.

Yes there is still software in every industry where its vendor continues to refuse to play nicely with others.  If you work in a large company or SME its probable that your LMS still fits this category, even if it’s only a few years old. But leading newer services all seem to understand the value of sharing and being open with each other.

Sharing and APIs have become a “must have” feature for the L&D thought leaders that will guide learning software choices in the next few years.  But with so many large providers in the learning industry still lagging behind, it’s worth sharing some history that helps us understand the risk of accepting this behaviour.

It’s not so many years ago software industry giants almost succeeded in taking this openness and opportunities away from not only learners, but all businesses as IT started to grow in its impact.  We all need to learn from past mistake to empower those who will be creating the future.

An explosion of choice

It’s fair to say world is full of apps.  Just between Google’s Play Store and Apple’s AppStore there are now more than 5 million apps available for you to choose from that can do everything from keeping you connected with friends, to ordering your pizza.

Likewise, over the last decade we have also seen an explosion in online cloud services as a replacement for traditional software for both business and individuals, many of which provide free options for personal or small business use.

The combined effect of these two trends have had on software in the last ten years cannot be overstated.  Between them they have helped to return our use of technology to its roots where everything is designed to do one job, but sets out to do it really well.

In theory now, we can always choose the best tools to solve each problem or goal we have.  But this only works if everybody plays nicely together.

Like children we all started by playing nicely

There has never been a time in technology where it’s been more important for everyone in the technology playground to play nicely together.  What may surprise you though is that this is not a new problem at all.  In fact it’s a problem Ken Thompson and Dennis Ritchie had already solved in 1969 as they were working on one of the most influence technologies of all time: Unix.

The theory used in the design of each app in Unix was remarkably similar to the design of today’s apps and cloud services: do one job really well, and then share with others so they can pick up where we left off.

In Unix the sharing was done through pipes and streams, now days apps share through RESTful APIs, and web hooks, but the core principles of well behaved software is the same.  Unfortunately we had to go through decades of badly behaved companies who didn’t want to share, before we could get back here again.

If you first got involved with PCs in the ’90s or 2000s it may be hard to believe that all software started off in such a friendly environment.  But software has its origins in an environment where everybody listened to one another, built on each other’s ideas, and everybody would always play with anybody else.  Much like young children in a playground, the only things that really mattered was if we were going to have fun while working together.

Playground bullies and the teenage years

As the IT industry matured and PCs started appearing offices, houses, and homes, software hit what is probably best described as its teenage years.

Large technology companies started to create huge pieces of software that did too much.  Software that didn’t focus on doing a single task well, but rather on doing lots of things, just about OK.  The idea here was simple, if we do lots of things OK, people will start to rely on us for everything.  Once they start using us for stock, they will have no choice but to use us for their order processing, and their accounts, and… Or once they start using us for HR they’ll also have to use us for e-learning, and facility management, and…

Why would anybody use an “OK” service if a great service was available as an alternative?  Well most of this software started off by being really good at just one thing.  But vendors started to bolt on additions that were at best “OK” and at worst unusable.   Before long like a teenager, applications from major vendors stopped talking to everyone that wasn’t in their friendship group.  The once open sharing became replaced with grunts, groans, and gestures and simple refusal to communicate or share anything with anyone.  The goal for companies selling software was that once you had committed to one solution from a vendor you were locked in to them.  No longer could you choose the best tool for the job and have everybody get along.  Now you simply had two choices: use your existing vendor for everything, or pay somebody to input and manage all your data (at least) twice so you had the right information in each piece of software you used.  Commercially for those producing the software, this plan worked well in the short term and a lot of money was made.  For everyone else in business though it meant IT become nothing but a growing headache with its unsatisfied demand for more money.

It’s was these teenage years that created giants such as SAP, Microsoft, Sage, and similar large companies many of which went on to face anti-trust cases throughout the world.  They controlled our HR, our Office Suites, our ERPs and CRMs, our accounts, our learning, and they wouldn’t even work with each other, never mind with anyone else

As the internet gained more attention, other companies started to pop up with friendly pubic images, such as Google, Facebook, and a re-vamped Apple, that spoke nicely about “playing with others” but only if you played the games they wanted, how they wanted you to, and you let them control your data and every aspect of the game.

Whether an old-style bully that kept demanding more and more of your lunch money, or a new style one that was friendly as long as you did what they asked, and then shared your secrets behind your back; these IT companies managed to hurt all industries at a time where technology should have been enabling people, not restricting them.

By actively choosing to refuse to work with anyone else, the giants of the IT industry took away consumer choice and created a set of playground rules that still cause problems and impact productivity in many businesses today.

The situation got so bad that the open source movement organised itself as a world-wide activism to fight against these trends and create software built on openness and sharing.  I, like many others, poured countless hours year after year into open source.  We shared a genuine concern that consumer choice and freedom was it risk, and if we did nothing would be entirely removed from technology within only a few years.  We started driven by fear, but became empowered by sharing.  We thrived on each other’s ideas.  We knew we could make a difference by working together.  We wanted to ensure the idea of technology that was open and shared with others never died out.

Reaching adulthood and Restoring Choice

As software reached its adult years, empowered by the surge in internet usage, and the renewed focus on sharing from open source as its activists entered businesses, people started demanding technology that worked together again.  An environment where small companies with big ideas to could start succeed was created.  People started to talk to each other and provide services designed to do one thing, really, really, well again.

In this more friendly environment ideas started to snowball.  Changes took hold that led to todays 5 million plus apps and numberless cloud services.  More importantly these changes once again restored genuine choice to both businesses and individuals on how to use technology.

Championed by services such as Xero, Salesforce.com, and Learn with Mobile.  Great software that does one thing really well again, has for a few years now even started to disrupt industries previously dominated by bullies.  With modern services using open RESTful APIs, easy to use web hooks, to talk to each other.  A new attitude has emerged that sees sharing and interacting with each other as a feature, not a risk.

There’s even websites such as “If this then that” (https://ifttt.com/) that exist simply to connect all these friendly services together in a way that works exactly how you want it.  Personalised experiences are becoming the norm.

Some of the giants have started to react by being more open and playing nicely again.  Microsoft is a great example of a bully company that has turned around not only its image but also its behaviour and services.  Once an example of the worst behaviour in IT, it is now an example for good to others who were once its peers in the playground.  But not everyone wants to change, and it’s going to take a long time for all of the giants to do make the transition into the new reality.

What does it mean for me?

Whether your looking at software to solve your learner’s next problem, improve your business performance, create a coaching culture, replace your LMS, or really looking into any software or service, you should ask potential vendors one key question “how well does this play with others?”  People will be used to this question by now, so it shouldn’t surprise anyone.  The historical giants are falling way behind others in this support.  But what should you do if the service you are looking at doesn’t provide an open API for others to integrate with? Simply move on and find something that does!

Even if the unfriendly software has a killer feature or two, if it won’t talk to others you will end up severely limiting your vender choice and wasting a lot of money.  What’s worse, by getting locked in you’ll find yourself falling ever further behind your competition in the years to come.  Far better to find a service that does almost everything you want well and invest in working with them to create a better alternative to that killer feature than to get locked in all over again.

That’s what the thought leaders in software recommend.  It’s what the thought leaders in L&D recommend.  And the same advice is finally being heard from thought leaders in all industries.

Technology, like good learning, should empower, not restrict.

Don’t let MAJOR version number worries stop you using Semantic Versioning (Semver)

Why Should I Use Semantic Versioning?

Semantic Versioning (semver) is specification for version numbers of software libraries and similar dependencies.

Its rules are not new, and are similar to how most library version numbers have been managed for years. However the idea of semver is that if libraries use exactly the same rules around version numbers then developers using libraries can know about library changes and compatibility direction from the version number.

The basic details are:

Given a version number MAJOR.MINOR.PATCH, increment the:

  1. 1.MAJOR version when you make incompatible API changes,
  2. 2.MINOR version when you add functionality in a backwards-compatible manner, and
  3. 3.PATCH version when you make backwards-compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

You can read the full specification at semver.org

In recent years semver has become very widely used, and in some spaces, expected. If you and the users of your library want to benefit from using a widely recognised version numbering system, I would recommend using semver above other system available due primarily to its significantly wide adoption at present.

Although semver will be very similar to the system you are using at present, as you apply its rules strictly you will probably find you made more backwards incompatible changes than you realised. It’s also likely that in the past you used the MAJOR version number as a marketing version number rather than strictly as an API incompatibility indicator. You’ll probably then find that your historical version number and methods has found its way outside of the pure library space to – with packages and even completed software applications matching version numbers to libraries.

So What are the Major Issues to Expect when Adopting Semver

Most the major issue people experience switching from their old versioning system to semver involve the major version number and API incompatablity.

Here are a few of the most common along with my recommendations on how you can avoid the issue or changes your processes to minimise its impact.

Our major version number is used for marketing. We want people to know that version 3.x.x is a very big improvement over 2.x.x

This is an important one as in many cases if semver is accepted and no other policies are changed, you could end up hitting version 42.x.x before you’ve had time to stop it. At that point you’ll have a very awkward choice to make to get your major version under control, and probably a lot of commercial pressure to “roll back” the number and start using the major version number for marketing again.

As you reflect on your own internal policies you may be tempted to come up with a “semver like” versioning system along the lines of MARKETING.MAJOR.MINOR.PATCH. Don’t do it. While I agree that style of numbering achieves what you want, you lose all the benefits of adopting semver if you don’t adopt it exactly and strictly. Remember your library was already using something like semver before you even heard of semver.

Our library is used in a few applications but we really like the freedom of breaking our APIs as we need to right now, so we won’t call out library 1.0.0 yet, well stay at 0.x.x.

It can be tempting to allow yourself freedom for as long as possible with a 0.x.x version. Maybe your library “is mostly used internally” or maybe “we’ll make it version 1.0.0 when we know the timeline for 2.x.x”.

The truth is the benefits of semver only really kick in after you reach the magic 1.0.0 version number. We can all recall open source projects that still sport 0.9.x version numbers after more than 10 years of development, because people haven’t yet got to a point where everything is perfect.

Remember that the version 1.0.0 is really for users of your library, not for you as the creator. Once you hit version 1.0.0 you are telling people that they should use your library. It functions as it should. They can benefit from it. They can use it in production.

If your library isn’t at that stage yet, then yes stick to a 0.x.x scheme, or a 1.0.0-alphax scheme, but in my view “release early, release often” isn’t about releasing incomplete software before it’s useful, it’s about release software as soon as its useful, and allowing its users to be involved in its evolution. This is nowhere more true than in library development.

We just released version 3.0.0 but version 2.32.0 actually had all the new developments in it. All we did for 3.0.0 was remove deprecated classes.

Unless you use completely separate branches for 3.x.x. and 2.x.x and a long term internal or public 3.0.0-alphax scheme for your next major release, then yes the x.0.0 releases can feel a lot less exciting than they used to.

I said before that the 1.0.0 release is for your libraries users, not for you. Thankfully 2.0.0 and later x.0.0 release are actually better for you than your users. For your users the version changes clearly warns them of incompatibilities that they may now need to work with. For you it represents the liberation from having to maintain and continually consider your depreciated APIs in everything you do. You took the worries of “how will it actually work in the wild” with you previous 2.32.0 release. So sit back and enjoy the x.0.0 releases.

I want to follow semver strictly, but how do I define an incompatible API change?

There are two general ways to define an incompatible API change (sometimes called a breaking change):

  1. Binary Breakage
  2. Source Breakage

Which one of these you consider appropriate for the management of your API depends on how you make your library available to your user. For example, if your library is managed through installation into a system wide library area (e.g. the Global Assembly Cache (GAC) for .NET or /usr/lib for *NIX) then you should use binary breakage because your distribution method allows post-compile substitution of the library into compiled applications.

More commonly now libraries are designed to be shipped as part of a bundle for a specific application (ClickOnce installs on Windows, ipk files on iOS, .apk files on Android, etc.) this means that an individual application does not have the ability (or at least the design intent) for its libraries to be post-compile substituted. So in this case you should use source breakage to define your API computability. On .NET many libraries now use Nuget to distribute the libraries, this is very much in line with the bundling of libraries with software, and so if your library is designed to be managed through Nuget you should consider Source breakage as the best measure of incompatibility.

A good discussion on some of the ways your API changes can cause source/binary breakage in .NET can be stackoverflow here: http://stackoverflow.com/questions/1456785/a-definitive-guide-to-api-breaking-changes-in-net

At a minimum you must consider source breakage, and therefore increase your major version number, if any application using your library will require any source code changes before it can recompile after updating from the previous version of the library. Changes in behaviour of a call from one version to the next cannot all be covered (otherwise in theory every bug fix changes behaviour so may be undesirable to someone which would mean increasing the major version number for just about every patch change) so use your common sense. If any current appropriate use of your API will behave worse rather than better after an update, I would consider that to be a breaking change.

I can manage the deprecation process for my concrete and base classes but everything I do to an Interface results in an incompatible API change.

If you are using Interfaces as part of the abstraction of your library (and generally you should be see my post on Dependency Inversion) then you will run into a really frustrating reality – everything you do to an interface will basically end up being a breaking change in your API.

Yes you can also provide concrete base classes that most people subclass to fulfil the interface, and yes you may have updated the base class with sensible default behaviour, and yes this does mean that most of your users (if not all in practice) have no code to change to be able to recompile after a library update. But some might and this makes it breaking. The fact you exposed an interface to allow for any implementation of those abstracted needs by classes outside your library does mean your users have got a lot of flexibility in the abstraction, but with semver it also means every change you make to an interface really is a breaking change.

You could use techniques like starting to build version numbers into your interface or namespace names to avoid breaking changes if you have to – but that will create far more work for you and your libraries users than you save in the long run. Either accept the change and bump the major version number, or postpone the change until the next planned major version number change. Either way you may very well find you’ll have to plan changes to your interfaces a lot stricter after adopting semver than they were before.

A final tip you may want to consider if you have a lot of “standard” interfaces as part of your library may be to place these into their own library separate to the rest of your code. You can then use your package manager (e.g. Nuget) to make the main library dependent on the interface library/package. You would then be able to bump your “interface library’s” major version number correctly, but keep tight control over the major version number of the main library people use. Doesn’t solve everything, but for many it is a good balance to work around a very specific concern when using semver.

I use nuget (or a similar packaging tool) but how do I handle changes to version numbers that only affect the package not the library itself?

Semver versioning does not make specific provisions for packing tools so its down to the packing tools themselves to recommend an appropriate approach. Like most packaging tools Nuget’s recommendation is to use the same version number as the library being packaged. This makes sense and avoids a lot of confusion for end users, however it does leave the question: what do I do if I have to put out a new version of the package (e.g. because package scripts or metadata has changed) but the underlying library hasn’t changed version number.

Some packaging systems, such as NetBDS’s pkgsrc, handle this by having a separate package version numbers that are appended to the official version number to make the full version number for the package (see http://www.netbsd.org/docs/pkgsrc/pkgsrc.html#bumping-pkgrevision). Where this is available you should make use of these package version numbers as a sequential number and consider it separate to the libraries own semver version number. It is good practice to reset the package number to 0 after the libraries own version number changes however.

In the case of Nuget there is not yet a separate package version number, so if you want to stick to the guidelines of using the same version number as the library you’re going to have to make a decision if you find a package only change that needs to be released. If you have control over both the library and the package, then consider the change the same as any other “patch” and bump the patch number. If you don’t have control over the libraries version number, you are going to have to decide if you can risk updating the package “in place” without a version number bump – not recommended as existing users will not get the changes and user support can be confusing with multiple package versions with the same name – or if you may need to temporarily move the package away from semver versioning for the package (not the library) to give a number x.y.z.p where p is the package revision number, until the next relese of the library comes out and lets you move back to matching the libraries semver version.

My version number used to represent sideways compatibility with closely related libraries, how can I keep this while using semver?

Semver version strings are used to represent a single aspect of version control – API compatibility. Other features that version strings can be used for – such as marketing numbers (covered in detail above) and sideways computability to specific versions of other libraries is not included in the version number design. This means you’ll need to handle it a different way.

You may choose to follow semver rules in how your version string is constructed, but match your release cycle to the original library. Although you normally would only reflect internal API changes in your library’s version number, not those of its dependencies, if you feel your library is tightly coupled and needs to match the same version number, then you could reflect this in your version number and mirroring it in your version numbers. This can be a significant help to your libraries user’s some time especially if there may be a slight delay between the dependent package being updated, and your own becoming available that’s compatible with it.

In cases where the coupling is looser, you probably don’t want to be tied into the release model and version numbers of the other library anyway, so now would be a good time to break it. Include compatibilities in the package metadata description, release notes, or project website. If your using a package manager for distribution that manages inter-package dependencies anyway, the your end user is unlikely to notice the difference anyway and will soon adjust if they did use your old version numbering style anyway.



Above I’ve tried to tackle some of the major worries around semver and controlling the major version number. If you have anything to add please feel free to do so in comments or direct feedback.

As someone involved in both creating libraries and package management systems for many years, I can see only good in a standardised version numbering scheme such as semver becoming as widely used as possible. Yes they have their problems, but every choice in versioning is a compromise at some point. It would be great a few years from now if we’re looking back at times where everyone managed their version control strings in different ways as a way of the past. But this only happens if everyone plays their part and sticks to the standards strictly and completely.

Should you adopt semver? Yes you should– but make sure you plan for its adoption properly and change your internal processes to match. This way you’ll be able to look back on the move to semver as an empowering choice to join developers manage dependencies collectively, and not as the point your project lost control of your version numbers.