Month: September 2012

So, the winner of the first Clojure Macro Challenge (magic partial) is Rich Hickey.  Check out the macro defcurried on line 132.  I don’t imagine for a moment he read this blog, it’s just that expressing the reducers library without it would have been an exercise in frustration and obfuscating syntax.  However, his implementation is a bit special-case; it only produces the curried version and only curries on the last parameter.  So I decided to rip off his code and see whether or not I could come up with a more general solution:

(defn- curry 
       [[params1 params2] body]
    (cons (vec params1)
          (if (empty? params2)
              body
              (list (apply list 'fn (vec params2) body)))))

(defn do-curried [symbol to-fn params]
      (let [result (split-with (complement vector?) params) 
            [[name doc meta] [args & body]] result
            [doc meta] (if (string? doc) [doc meta] [nil doc])
            body (if meta (cons meta body) body)
            arity-for-n #(-> % inc (split-at args) (to-fn body))
            arities (->>
                 (range 0 (count args))
                 (map arity-for-n)
                 reverse)
            before (keep identity [symbol name doc])]
       (concat before arities)))

(defmacro defn-curried
  "Builds a multiple arity function similar that returns closures
          for the missing parameters, similar to ML's behaviour."
  [& params]
  (do-curried 'defn curry params))

(defmacro fn-curried
  "Builds a multiple arity function similar that returns closures
          for the missing parameters, similar to ML's behaviour."
  [& params]
  (do-curried 'fn curry params))

Incidentally, this was written in the LightTable InstaRepl, which is excellent for developing this kind of abstract hack.  The one feature it’s missing is proper destructuring support.  If you take a look above, you’ll see an identifier “result” that only really exists to allow the InstaRepl to process it more easily.

Anyway, I hope someone finds this useful.  I know I will.  After using ML/F#, writing functions that return other functions feels like an anti-pattern.

I was recently witness to an interesting twitter exchange between Stu Herbert and Dan North on the subject of development practices.  It occurred to me that the disagreement was fundamentally that they were addressing different groups.  Stu felt that the term “best practices” implied “optional” to too many people who should be using automated testing.  Dan, on the other hand, dislikes the term best practices because it’s devoid of context.

As Dan explained in an old Oredev talk (watch it if you haven’t), the problem with best practices is that their purpose is to drag people at the start of the Dreyfus model up to competency.  However, there’s a downside: enforcement of best practices exerts a drag on people beyond the competency stage.  From an expert’s point of view, everything is optional and is simply a matter of cost benefit analysis.  The problem is, good advice for experts is lousy advice for novices and vice versa.

I Don’t Understand It, So It Must Be Easy

The Dunning-Kruger effect is an observation that, the less competence you have at something, the easier you think it is.  I suspect it’s about to be renamed the “Apple Maps” effect.  Here’s where we really hit Stu’s problem: pretty much everyone thinks they’re an expert, except possibly for the experts.  Everyone with a stand up meeting is an authority on lean and agile.  I’ve seen team after team in my career who thought they had the experience to determine whether or not automated testing was appropriate to their project when not one of them had even tried it.  (In fairness to these teams, most uncoached first attempts at TDD are disasters.)

An Idiot Is Someone Who Doesn’t Know What You Learned Last Week

So, how do you push teams like this along the path of skills acquisition?  Well, usually, you proclaim that something is The One True Way and convince them that, by doing these things, they will be superior to other teams that haven’t embraced The Truth.  Despite having an element of the cargo cult about it, this will actually be true.  You’ve managed to circumvent the DK effect for a while, but it’s not gone.  As with all cognitive biases, they’re never, ever, gone.

If you haven’t ever used an IoC container, TDD or CI, I pretty much guarantee that trying them will make you a better developer.  Usually in ways that you’ll find hard to explain to people who haven’t.  If you’re stuck in the tar-pit, Scrum and other prescriptive agile methodologies will get you further along the path of understanding how to deliver value.  But any rules-based understanding is only half the journey.  Once we’ve climbed the hill, we’re still only at the competent stage.  All too often, we’ve set up camp.  If you walk in any direction, we’re going down again.  But anyone who cares to look can see beautiful mountains in the distance.  Let’s go visit them.

The story of how this blog went down is short. (I rushed something and made a mistake).  The story of how I got it back up is epic. (Anyone who thinks Microsoft tools are easy to set up may be interested in some timeshare opportunities I’ve got available for a limited time only.)  However, a small but crucial part was played by this page.  It’s extremely boring: it just checks that Lucene has the permissions it needs to work.  The page above could have been achieved by a log file, but making it visible front and centre had a great advantage: I could use it to show my service provider what the problem was and give them a way to check that they’d fixed it. (Before this point, it was trial and error.)  It’s amazing how many systems fail when they’re misconfigured without telling you what the configuration problem is.  Diagnostics aren’t about data, they’re about communication.

Anyone who looks out from .NET from time to time will know that our libraries are missing quite a lot that other platforms take for granted.  A decent embeddable web server is one.  Thankfully, the Nancy team have addressed this issue with the Nancy self-hosting engine.  It’s far from the fastest thing on the planet, it’s no substitute for IIS, and it’s not going to give the authors of Jetty any sleepless nights, but it does hit one sweet spot that is close to my heart: live diagnostics.

You see, even if you do your logging and your alerting well, these diagnostics are after the fact and rarely contain detailed enough information to solve certain kinds of problems.  These are the problems where you need to know what’s going on right now and why?  There’s two reasons for this:

• Typically you don’t have live access to the in-memory structure of the application
• There’s just too much information. You need some way to drill down.

The too much information problem is serious.  Quite simply, you can’t sensibly log the level of detail for every last transaction you sometimes wish you had for just one or two.  The in-memory problem is so large many don’t even notice it’s there.  Rather than take a look at what their program is doing, they accept that they’re just going to have to theorize about what state it is in and how it got there.  An embedded web server solves both problems.  You might be thinking that this kind of quality control can be achieved by unit testing, and you’d be right, except that you wouldn’t have captured the amount of time it takes to develop a test when you’re not sure what’s wrong.  I recently (successfully) put an automated trading system live, and I would estimate that 40% of the test suite was developed with the aid of the live diagnostics the system provided.

Next time you’re writing a back-end service, try embedding a small web server that just prints up a single page of information about what’s happening.  Once you’ve had a taste, I’m sure you’ll find it addictive.

I’ve used VS2012 for a couple of days now and have been pleasantly surprised.  It’s a lot faster than VS2010, which was an appalling piece of engineering.  The dark theme, my personal preference, is basic but pleasant.  It doesn’t quite make its way everywhere: property pages, for instance are not only dark theme unaware but 100% as ugly as they were in previous version.  The new event log is gorgeous and I’m hoping it’s going to prove useful.

On the other hand, it doesn’t exactly feel finished.  Autocomplete on symbols pops under the function documentation tooltip, a bug that seems to have sailed straight past beta testing.  They’ve also added a well-meaning feature to look up help on exceptions that just doesn’t reflect the way that exceptions are actually implemented in the .NET Framework.  For instance, if you’re having a problem connecting to SQL Server, it’ll send you straight to the general SqlException page.  To fix this, they either need to make the feature smarter, or make exceptions in .NET a lot more specific. 

Oh yes, and unless you read English in a different way from the rest of the human race, you’ll want to read the contents of this page pretty much straight after booting it for the first time.