Month: April 2009

I was recently reading a fairly detailed comparison by Andrey Shchekin of the various injection containers out there in the .NET space.  I found it interesting that the last feature he regarded as important was the ability to tell the container to mock a particular object. 

Now, when I wrote an article about evaluating IoC containers, it didn’t even occur to me to include this.  Now, Andrey includes many points that I haven’t covered, like the fact that Castle hasn’t had a stable release in well over a year, but on this point I think it’s best left out.

Why?  Because I think that using the feature is a code smell.  Certainly, I’ve injected mocks into containers before today (StructureMap 1.0 with old school Rhino Mocks, not a thing of beauty) but frankly, it was because I didn’t understand how to use containers as well as I do now.

So, how should you use them?  You should inject your dependencies into the constructor, period.  There’s times you need setter injection (configuring Retlang, sadly, falls into this category, so does most of Microsoft’s stuff), but I wouldn’t recommend designing that way, it vastly complicates your analysis.

You might want to dynamically reference components, but that’s why I wrote AutoGen.  (As a way to tell if someone feels strongly about something, see how many weekends he’s spent on it.)  Even if you don’t have something like this, you still ought to be using your own builder interface and passing that in.  At the very least, you’re explicitly documenting your actual interaction with the container.

So why should you be injecting mocks into your container?  Well, the only reason I can think for doing this is if you’re referring to your container statically.  You shouldn’t be doing this, for any number of reasons.  It doesn’t help that most of the StructureMap examples still use static paradigm, but it’s not actually a recommended method of doing it.

In short, if you are looking for the ability to inject mocks into a container, I would recommend you take a look at how you interact with the container in the first place.  Chances are you’ve actually managed to use the container in a style that containers are meant to discourage.

So, my patch got rejected for a variety of good reasons.  One is the standard problem of not having checked everything: I spent so much time making sure that the nant build worked I missed that I hadn’t included the relevant csproj files.  Of course, I have any number of problems getting castle to build at the best of times, but I’ve started to regard this as a constant of the universe.

Another is the Castle coding standards.  Now, trivial coding standards are important, by which I mean that whilst it doesn’t matter if tabs are two or four spaces, it definitely matters that the entire code base applies the same convention.  Jeff wrote more eloquently than I can manage on the subject, so go read what he has to say rather than let me labour the point.  What i will say is that I am heartily tired of all of the arguments I’ve witnessed (and to my shame, sometimes participated in) on the subject of code formatting.  Castle uses a couple of conventions I don’t usually, mostly because it’s more convenient for me to use different ones, but these things are easily fixable with some hack coding.  More seriously was the lack of any documentation other than the tests, so I went back and fixed that up.

Then there’s the question of the Common Service Locator code.  I mean, it’s all very well that AutoGen passes Ayende’s tests, but the CSL is an MS-PL codebase, and Castle is Apache.  It’s not clear that you can even include the assemblies.  This is a conceptual minefield at the least.  So, I’ve removed the CSL dependency and rewritten the tests.  This may not be the end of the matter.

Anyway, here’s the code to hack source files into Castle compliance.  Not elegant, but it gets the job done.

namespace CastleStandard {
    using System;
    using System.Collections.Generic;
    using System.IO;

    class Standardizer {

        private string licenseHeader =
@"// Copyright 2004-2009 Castle Project - http://www.castleproject.org/
// 
// Licensed under the Apache License, Version 2.0 (the ""License"");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// 
//     http://www.apache.org/licenses/LICENSE-2.0
// 
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an ""AS IS"" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.";
        private int tabSize = 4;

        public string Standardize(string code) {
            var result = InternalStandardize(code);
            if (result != InternalStandardize(result)) {
                throw new Exception("Standardization code was not idempotent.");
            }
            return result;
        }
        string InternalStandardize(string code) {
            var reader = new StringReader(code);
            var licenseReader = new StringReader(licenseHeader);
            var usingStatements = new List<string>();
            var writer = new StringWriter();
            string codeLine = null;
            bool isLicensePresent = true;
            foreach (var licenseLine in Lines(licenseReader)) {
                if (isLicensePresent) {
                    codeLine = reader.ReadLine();
                }
                writer.WriteLine(licenseLine);
                isLicensePresent = codeLine == licenseLine;
            }
            // If the license wasn't present, we have a line we need to push back onto the reader
            var lines = isLicensePresent
                ? Lines(reader)
                : Prepend(codeLine, Lines(reader)); 
            bool hasEncounteredNamespace = false;
            foreach (var sourceLine in lines) {
                if (usingStatements != null) {
                    usingStatements = ProcessUsingStatements(writer, usingStatements, ref hasEncounteredNamespace, sourceLine);
                } else {
                    writer.WriteLine(Tabify(sourceLine));
                }
            }
            if (usingStatements != null) {
                throw new Exception("Using statements were never dealt with.");
            }
            return writer.ToString();
        }

        private List<string> ProcessUsingStatements(TextWriter writer, List<string> usingStatements, ref bool hasEncounteredNamespace, string sourceLine) {
            if (!hasEncounteredNamespace && sourceLine.StartsWith("using")) {
                usingStatements.Add(sourceLine);
            } else if (sourceLine.StartsWith("namespace")) {
                hasEncounteredNamespace = true;
                if (!sourceLine.Contains("{")) {
                    writer.WriteLine(Tabify(sourceLine));
                }
            }
            if (hasEncounteredNamespace && sourceLine.Contains("{")) {
                writer.WriteLine(sourceLine);
                if (usingStatements.Count > 0) {
                    foreach (var usingLine in usingStatements) {
                        writer.WriteLine("t" + usingLine);
                    }
                    writer.WriteLine();
                }
                usingStatements = null;
            }
            return usingStatements;
        }

        IEnumerable<T> Prepend<T>(T value, IEnumerable<T> list) {
            yield return value;
            foreach (var t in list) {
                yield return t;
            }
        }

        string Tabify(string line) {
            int spaces = 0;
            int chars = 0;
            foreach (char c in line) {
                if (c == 't') {
                    chars++;
                    spaces += tabSize;
                } else if (c == ' ') {
                    chars++;
                    spaces++;
                } else {
                    break;
                }
            }
            return string.Concat(
                new string('t', spaces / tabSize),
                new string(' ', spaces % tabSize),
                line.Substring(chars)
                );
        }

        IEnumerable<string> Lines(TextReader reader) {
            string line;
            while (null != (line = reader.ReadLine())) {
                yield return line;
            }
        }

    }
}

 

They’ve gone to a lot of effort to make nServiceBus easy to get working, but it still takes about half an hour with the readme.  A couple of notes that should save you some time:

• Relatively obviously, you’ve got to put MSBuild on the path.  This will usually fix that:  set path=C:WINDOWSMicrosoft.NETFrameworkv3.5;%path%
• Build_Src.bat doesn’t put anything into the build/output directory, which is unfortunate since build_samples.bat expects it to.  build_with_strong_name.bat does, however.
• For some reason, my box didn’t trust nServiceBus.  This gives you the (in)famous “The Project Location is Not Trusted” dialog box.  The following command fixed it:  caspol -machine -addgroup All_Code -url file://c:/nsb/* FullTrust -name nServiceBus  (note that you need to have set the path for this to work)
• After building nServiceBus, you can finally use the solution file.

All of this assumes that you’ve built NServiceBus in line with what I said on the previous post.  This is just enough to modify the FullDuplex sample to load balance.  Most of this has previously appeared on the yahoo groups, but I’ve just organized it for comprehension.

Step 1: Modify the configs

Amazingly, all you need to do is to modify the configs of the samples.  Here’s what you need to do.

Client

c:NSBSamplesFullDuplexClientbindebugClient.exe.config

<UnicastBusConfig DistributorControlAddress="" DistributorDataAddress="">
  <MessageEndpointMappings>
    <add Messages="Messages" Endpoint="distributordatabus" />
  </MessageEndpointMappings>
</UnicastBusConfig>

Distributor

c:NSBsrcdistributorNServiceBus.Unicast.Distributor.RunnerbinDebugNServiceBus.Unicast.Distributor.Runner.exe.config

Requires no changes.  It just works.  (!)

Server 1

C:NSBSamplesFullDuplexServerbinDebugServer.exe.config

<MsmqTransportConfig
  InputQueue="server1messagebus"
  ErrorQueue="error"
  NumberOfWorkerThreads="1"
  MaxRetries="5"
/>

<UnicastBusConfig DistributorControlAddress="distributorcontrolbus" DistributorDataAddress="distributordatabus">
  <MessageEndpointMappings>
    <add Messages="Messages" Endpoint="IGNORE" />
  </MessageEndpointMappings>
</UnicastBusConfig>

Obviously, Endpoint “IGNORE” is never used.  I’m just setting this to emphasize that messages from the server don’t use this setting in this scenario.

Server 2

Go to C:NSBSamplesFullDuplexServerbinDebug

Copy the entire folder to C:NSBSamplesFullDuplexServerbinDebug2

Change C:NSBSamplesFullDuplexServerbinDebug2Server.exe.config

<MsmqTransportConfig
  InputQueue="server2messagebus"
  ErrorQueue="error"
  NumberOfWorkerThreads="1"
  MaxRetries="5"
/>

<UnicastBusConfig DistributorControlAddress="distributorcontrolbus" DistributorDataAddress="distributordatabus">
  <MessageEndpointMappings>
    <add Messages="Messages" Endpoint="IGNORE" />
  </MessageEndpointMappings>
</UnicastBusConfig>

You need to create a full copy because of the way the SpringBuilder operates.  Two servers in the same folder will fail, with an extremely unhelpful error message (Object Variable not Set, to be precise.).

Step 2: Run the system

To run it:

• Open up four command windows
• C:NSBsrcdistributorNServiceBus.Unicast.Distributor.RunnerbinDebugNServiceBus.Unicast.Distributor.Runner.exe
• c:NSBSamplesFullDuplexServerbinDebugServer.exe
• c:NSBSamplesFullDuplexServerbinDebug2Server.exe
• c:NSBSamplesFullDuplexClientbinDebugClient.exe

You can now start servers, stop servers, watch messages queue up and recover, watch it load balance between the two. 

Step 3: See what it’s doing

If you start running parts of this system through the debugger, you’ll want to switch off some exceptions fairly quickly, since they happen all the time.

• System.Messaging.MessageQueueException:  every time it reads from an empty queue
• System.BadImageFormatException:  when it tries to read a config file as an assembly.  (really)

Can anyone think of a good reason to make an exception class internal?  I mean, surely the whole point of raising typed exceptions is so that people can process the exact exception that they’re looking at in the debugger.  It’s not like the debugger prints up “nearest public type”.  You can always make the constructor internal if you’re really afraid of marauding bands of developers intent on breaking your code.

And the offender that’s annoyed me so much?  System.Management.Automation.ParameterBindingValidationException.  Thanks, powershell.

I recently pointed out to a (rather drunk) friend that successful processes are easier to follow than not follow.  He remarked that I could usefully spend a career on the direct consequences of that statement.  It occurred to me that, rather than give up code forever, I might usefully note down some of my thoughts on the subject.  A note: I’ve been both a chief and an indian in various organizations, and have moved between the two roles more than once.  This has given me a certain perspective on processes, both why they are important, and why they are a waste of time.

Why bother with a process at all?

Now, In our standard approval process, any developer can submit any release for approval.  There’s no tracking of who looks after which projects, it’s just not necessary.  There’s one exception to that, and it’s the one that proves the rule.  DBAs were getting informed of proposed database changes too late in the day.  By making DBAs the only ones who could submit database releases for approval, we ensured that they got an opportunity for review at an early stage.

In general terms, there are only three reasons to have a process:

• You need to pass the audit
• You need to capture some information
• Something goes wrong without it.

If your process isn’t satisfying one or more of these conditions, I’d reconsider why you’re bothering.

Audit Points

Dealing with audit points can be a pain.  All to often, it feels like what you’re being asked to do is ridiculous.  However, it’s well worth indulging in a bit of self-examination when going through these things.  One firm I worked at actually had a pretty good track record on testing its systems.  However, the auditor pointed out that we weren’t saving the test results anywhere.  The more we thought about it, we came to the conclusion that it was important that not only did we test our systems, but that we could prove we had.  So we addressed the audit point.  Of course, the next year the same auditor came back and read our test documents.  She said they weren’t standardized.  We told her we didn’t care.

Again, this wasn’t a knee jerk reaction, we considered what standardized tests would buy us.  The development team was ten-strong, and the systems we developed were quite heterogeneous.  So the work was varied, the test requirements were varied, and the team was small enough that everyone could understand everyone else’s testing.  There just wasn’t a benefit to changing our policy of “use your best judgement”, and the cost of writing a response that said we weren’t going to do anything about it was pretty small.

Finally, bear in mind that there is a massive amount audits don’t pick up.  An auditor has a mental checkbox in her head of how the process should work.  You know your business better than that.  Now, no-one’s likely to tell the auditors about a serious problem the auditor didn’t directly ask about, but that doesn’t mean you don’t need to fix it.  On good days, you can kill two birds with one stone.  Look for creative ways to address audit points that address your own concerns.  For instance, auditors are very keen on separation of duties.  Most small departments can’t, for instance, employ a separate release manager.  Most managers see this as a choice between either ignoring the audit point or seriously damaging their productivity.  An automated release system can have the button pressed by anybody (you could get compliance to do it if your sense of humour ran that way…).  Not only that, but the investment actually pays off: your project velocity goes way up.

Data Capture

In my experience, you need to be really careful with this one.  There’s quite a few reasons this will go wrong, but one of the really basic ones is that people don’t like doing data capture.  I’ll give an example, I used to work for a dotcom.  Now, one of the most important figures that a commercial website has is conversion: the percentage of people that go from one step in the process to the next.  It’s a number you monitor extremely carefully.  The marketing department wanted to ask a question “How did you hear about us?”.  Over the objections of other teams, they insisted that the question be made mandatory.  You simply couldn’t use the site without answering this.

Conversion went down by 1%.  1% was £50,000 a month.  The great thing about working for a professional dotcom is that you’ve got these numbers at your fingertips.  I’m sure the marketing team found this information useful, but they weren’t about to convince anyone it was £50,000 worth of useful.  Especially when you consider that you have no idea how many people who did answer the question actually gave the right answer.  I’ll write more about data capture another time.

Something’s going wrong

This is the single best reason for putting a process in place.  If there’s a developer who keeps checking in dud code, you’re going to want to institute code reviews pretty quickly.  Your urgency is going to be lower if the build isn’t getting broken.  Equally, if all the work in the team is getting done in reasonable time, you’re unlikely to be running prioritization meetings.  Get overloaded, or under deliver, and that’s the shovel that will dig you out the hole.  Agile processes are equally problem orientated: stand-up meetings are designed to improve communication, increase commitment and promote team cohesion. 

Developers are, to a great extent, pretty hostile to processes (with the exception of those they originated).  This is because, frankly, they’ve often cut their teeth in organizations with bad processes.  But the more you subject processes to the same cost/benefit analysis that your CTO applies to everything, the more you’ll see what’s worth doing and what isn’t.  This cuts both ways.  Processes always introduce friction.  A daily meeting can lose you an hour a day.  This means that the benefit has be of the order of 16% additional productivity the rest of the time.  (This is one of the reasons you’ve got to keep stand-up meetings short.  At 15 minutes, the break-even point is 3.5%.  I’m assuming a useful working day of 7.5 hours here.)

In other words, it’s not just enough that a process prevents something going wrong.  It’s got to cost the company less to implement the process than to just live with the damage.  A broken build once a year isn’t worth preventing, once a week that blocks a team of fifteen is.  If you really want to change your organization, putting in decent controls on what’s important is one of the most productive things you can do.  And the processes that don’t make the cut?  I suggest a bonfire.

If you’re using JSON.NET (and if you’re using Microsoft’s libraries, you really need to start) you may have run into the way it serializes dates.  Basically, for good reasons, it returns Dates in the format “/Date(1198908717056+1200)/”.  This, of course, requires you to actually parse the date yourself, which is a bit painful, especially since you need to worry about time zones.

function parseJsonDate(jsonDate) {
    var safeDate = jsonDate.match(/([0-9]+)([+-])([0-9][0-9])([0-9][0-9])/);
    var minutes = parseInt(safeDate[3]) * 60 + parseInt(safeDate[4]);
    var offset = minutes * 60 * 1000 * parseInt(safeDate[2] + "1");
    var result = new Date(parseInt(safeDate[1]) + offset);
    var offset = result.getTimezoneOffset();
    return result;
}

This is obviously ugly, but it’s getting me the right results.  However, if anyone find any bugs, please let me know.

So, I’ve decided to try turning AutoGen into a patch for the Castle project.  The first problem I’ve encountered is that TypedFactoryFacility is considered part of the MicroKernel.  Since AutoGen references Castle.DynamicProxy directly, it’s not really a goer to put it there.  This basically means that I need to add it to the extremely large facilities folder.  On the other hand, it gives me an opportunity to work directly with nant, which isn’t a bad thing.  It certainly makes you tidy up your references (and about time too… I had a reference to System.Data.  Needless to say, I didn’t have a depedency.)

The Castle project are pretty aggressive about warnings as errors, so I had to remove an unused field.  It was there for a future feature but, of course, YAGNI applies.  It was a speed-bump in any event.  I’m way too used to working with legacy code which seems to generate more warnings than executable code.  It seems that you need to disable signing when you’re nanting a sub-build.

None of this is that bad.  The tests, on the other hand, were a right royal pain.  Castle uses NUnit 2.2.8 (it’s really finicky about NUnit versions), the tests were written 2.4 style.  Cue a significant amount of hacking around.  Also awkward was the test requirement for the Common Service Locator.  There’s certainly an element of cargo-cult programming in what I’m doing right now: the build files are a maze.  I’ve got to admit, I’ve always had problems getting Castle to build (it appears that people on the mailing list appreciate this, but it’s not like it’s fixed) and so changing the build is a bit of a scary operation.

Anyway, the patch is now in donjon, so we’ll see what happens.

I couldn’t disagree more with what Max is saying here.  Now, I’ve worked at some pretty conservative companies, and pushed through process improvements everywhere I’ve been.  Believe it or not, one time the first thing I did was implement source control.  I’d say that reworking my department was part of my job description, but my job title is Architect.  Remaking a three thousand man company?  Not something I can do on my own.  For that, you need senior level management’s involvement.  If you can’t get it, that’s unfortunate, but if the change is worth making, it’s the job of those that can make it happen.

There do exist progressive companies that have extremely senior people with a hawk’s eye on process.  Google is the obvious case, where a set of coding standards and acceptance standards is implemented across the whole,huge, development community.  If you’re at a similarly sized organization, the chances are that you’ve got different processes in different areas: basically, the areas in which co-ordination is both possible and productive.  Again, I’m not saying you shouldn’t try to co-ordinate with your peers, but if it’s either unworkable or positively detrimental (“We want to standardize on SSADM.”) you can just walk away and don’t blame yourself.

It’s not the only one, though.  How about Starbucks?  It’s not always perfect, but the fact remains that every time you walk into one, you’re seeing an organization where process optimization is taken seriously by the CEO of a massive multi-national.

Of course, coffee shops is most of what Starbucks does, code is most of what google does.  There are plenty of organizations in which development isn’t the primary focus.  It’s still surprising when you discover that this is true of a dotcom, but it does happen.  Now, I do see way too many people who think that process re-engineering is somebody else’s problem, either someone more senior or someone in another department, but it’s always management’s problem.  Seriously, what manager can you think of who shouldn’t be responsible for what his reports are doing?

I’ve recently been building a couple of very small fluent interfaces.  These have been focused on component testing, where some of their disadvantages aren’t as important.  Let me give an example:

public void PartialRecoveryFromTwoSessions() {
    When
        .SessionStartsAt("1 Jan 1990 21:25:00", s => s
            .MessagesProcessed(3, 5, 1))
        .And.SessionStartsAt("1 Jan 1990 21:35:00", s => s
            .MessagesProcessed(2, 6))
    .Then
        .RecoveryStartsFrom(4)
        .RecoveryWillSkip(5, 6);
}

Now, the great advantage of expressing your test in this fashion is that it’s quite easy for someone to look at the test and see what you mean.  However, there are a couple of disadvantages worth mentioning:

• Although it makes your intent obvious, it doesn’t make it obvious that your intent has, in fact been satisfied.
• Since you’ve just written a short testing framework, you really ought to write some tests for the framework itself.  That would probably involve mocking the object under test.
• Writing this style in C# typically involves creating builder objects.  Personally, I try to avoid writing mutable objects wherever  possible, and builder objects are about as mutable as they come.
• The overhead of creating an actual framework for testing rather discourages the practice of test-first development.

Despite this, the improved ability of the test to communicate intent makes it well worth doing.

How it works

Martin Fowler’s article gives you the abstract details of how this sort of thing works in C# and Java, but I thought it would be worth explaining the example I’ve built here.  You’ll note that there are a large number of violations of standard practice in this explanation.  That’s one of the major reasons I’m using this principally in testing code: I really don’t like the stylistic continuity from standard coding practices.

• When is a property that creates a new CaseBuilder object.  This object also acts as the stub repository (this implementation doesn’t use a mocking framework, so all test doubles are actual physical objects).  Obviously, you’d never violate separation of concerns like that in production code.
• SessionStartsAt is a method that creates a session and adds it to the repository.  It then returns the CaseBuilder to allow the method chaining.
• To configure the session, you implement an action on the session.  This is a pattern I learnt from the Fluent NHibernate code base.  The advanatage of this is that you don’t need to worry about how to get back to the CaseBuilder object after you’ve finished configuring the session.
• “And” does nothing, it just returns the builder object.  It’s only there for (natural language) readability.
• “Then” actually performs the test and returns a CheckBuilder.  By returning a new type, you restrict what appears in auto-complete and prevent people from expressing logical nonsense in the tests.

If you need to customize how the test runs, you tend to need to insert another delegate into the system.  In general, though, I’m quite pleased with the technique.  There’s less than 300 lines for the framework outlined above, most of which is relatively trivial code of the form “set property, then return “this” “.