Automated Deployments #2: Configuration Management

It’s amazing how much engineering time is spent on arguing about the difference in abstraction strategies, followed by someone saying “just copy the files up, but make sure not to touch the config”.  This is a recipe for disaster.  There are three common failure scenarios:

  • Someone takes a copy of the live system, runs some tests and accidentally enters the test data into the live system.  I once saw that happen with a stress test.  It wasn’t funny.  (In fairness, it’s pretty funny in retrospect.)
  • Someone uploads a debug environment, rendering the live system unstable.  (This is mostly a web-related scenario.)
  • A new version is correctly released, but it required a config change which never made it into the production config.

Now, most people run with this policy because “don’t touch the config” produces fewer failures than “touch the config”.  You could argue that most of these scenarios are associated with not carrying out the instructions to the letter.  However, this is to miss the point.

Successful processes minimize the chance of human error.

If someone forgot a step, and that guy is not a muppet, your process has too many steps.  Our release process has one.  Exactly one.  I loathe processes that seem to have as their principal benefit that you know who to blame when it goes wrong.  I would much rather things didn’t go wrong in the first place.  So, we’re looking for a process that guarantees that the environmental differences are respected, but that changes required by the code are propagated.

Types of Environmental Factor

Configuration management is a big and scary subject, and is the proper preserve of your IT department, not your developers.  However, if you concentrate on just the bits that matter to developers, it need not be that big an undertaking.  Let’s go back to basics.  In general terms, there are three common sorts of .NET application:

  • A standard windows client application.  This includes console and GUI apps. 
  • A windows service.
  • A web site or web service.

For standard windows applications, your environmental delta will usually be in the app.config file.  Unless you have multiple installs of the one service on a machine, it is unlikely you’ll have any environmental changes in windows services.  Web sites themselves are typically identical on all deployment environments.  The fact remains that nearly all installs of corporate applications can be summarized as follows:

  • Copy some files
  • Fix the config file
  • Set up the windows service
  • Set up the service in IIS.

Now, to produce a perfect install, you end up messing around with InstallShield or WiX or some such tool.  However, to cover 95% or more of environmental issues, all you really need is a way of fixing the config file.  I’ll remark at this point that since you have control over the entire ecosystem, you can ensure that your system doesn’t require weird settings you can’t handle.  Equally, I’d go out of your way to eliminate stuff like custom config sections.  They’re more trouble than they’re worth.

In practice, in our environment, we have a phenomenal number of programs are the only config entries we ever change upon deployment are:

  • AppSettings
  • ConnectionStrings (We encrypt the connection strings when we apply the delta)
  • Setting compilation debug equal to false (I can’t stress how important this one is.)

I may one of these days publish the code we use for this (it’s a powershell cmdlet) but the fact remains, it’s easy enough to implement on your own.  Incidentally, I can highly recommend you don’t use the Enterprise Library solution.  It’s quite complex and has weird bugs (e.g. it won’t work on a network file).

Storing Environmental Deltas

When we were designing this system, we consider the following models:

  • We use a model in which all deltas are stored in the same file with the master.  The program then determines which environment it needs to use.
  • The development config is the master, deltas are separate files applied to it upon deployment
  • Deltas are applied to produce all configs, including the development config.  The deltas are applied to a master.config.
  • We use the user settings features built into AppSettings.
  • We just have different configs for each environment.  The deployment process just copies up the right one.

There are die-hard fans of all approaches, but I’ll outline why I believe the second to be superior. 

The monolithic file approach is attractive at first because everything’s together but suffers from catastrophic unmanageability as you get a lot of settings (which is a problem you shouldn’t have, but may have).  Furthermore, there is the inelegance of having to deploy information for one environment to another (unless you write a post-processor, in which case you might as well have opted for alternatives 2 or 3).  The self-discovery aspect is attractive, and the monolithic file is easy enough to put into source control.  Just putting all of the configs into source control has its attractions as well, but suffers from the fact that 90% of the XML will be the same in each file, making it hard to track down the differences.  I prefer a model with explicit deltas.

The built-in features sound attractive, because it feels like Microsoft has already done the heavy lifting for you.  However, you’re pretty much guaranteed to be still modifying the web.config anyway, and you’ve split the config into multiple parts not only for management, but for the deployment environment too.  A lot of people practice this method by having the deltas only present

Finally, we’re left with the choice between having an abstract master file and having the master file be the local development config.  Here, I’d argue that the local development file will be edited by developers directly whether you like it or not.  Best to embrace that than have it as a failure point each time it happens.

Final Thoughts

The level of configuration management you need for .NET apps is pretty easy to implement, which makes it a pity very few people both.  All you really need is a couple of xml pokes and you’re done.  One of the great benefits is that all of your environmental information is in source control (you can even make the program that applies the deltas encrypt the data if you regard that as desirable) which makes it much easier to check things in a large heterogeneous environment.  (Again, not a problem you should have, but a problem you may have.)

And yes, the first failure scenario mentioned at the top is also the reason you should have a firewall between development and production.  Next time I’ll talk about configuration that doesn’t appear in the .NET config file.

Published by

Julian Birch

Full time dad, does a bit of coding on the side.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s