Exceptional APIs

Axioms for public API design:

Axiom 1
NullReferenceExceptions that come out of your code ARE YOUR FAULT!

Axiom 2
The fewer exceptions your code can possibly throw, the better!

Applications of these axioms:

Let's say we're writing a public API, you and I. In it, let's say we have the following awesome method:

public void Foo(string s) {
    Bar(s.Length);
}

You will notice that we dereference s to get its Length property. If some misguided user of ours were to pass null to Foo, an NRE would pop out of our code like an overweight stripper out of a wedding cake. Awkward! And then the stack trace would get passed all over school and all the kids would laugh at us.

Lemma 1
From axiom 1 it follows: Null-check EVERYTHING YOU DEREFERENCE unless you know where it came from.

We could do a conditional dereference:

public void Foo(string s) {
    if (s != null) Bar(s.Length);
}

Or we could verify the argument:

public void Foo(string s) {
    if (s == null) throw new ArgumentNullException("s");
    Bar(s.Length);
}

If our method absolutely needs to dereference the object in order to do its job, then argument verification is the way to go. This may seem like trading one exception type for another, but it's really not. Argument exception types are perfectly acceptable - they inform our misguided user what went wrong and how to make it right. On the other hand, NREs mean that we didn't verify the argument or null-check before dereferencing. And they mean that all the kids will laugh at us.

Let's say that we decide on argument verification. Now let's say that we want to add a convenience overload:

public void Foo() {
    Foo("");
}

Looks good, right? WRONG!

Lemma 2
From axiom 2 it follows: Use 'null' as a sentinel between overloads for the default value when null is not an otherwise permissible value.

My favorite non-fiction book of all time, Framework Design Guidelines, specifically says not to do this. It is wrong. Well, sort of. FDG advises against using null as a "magic" sentinel, period. I argue that sentinel null is correct for parameters which are omitted in convenience overloads. Here is why:

To recap, our API currently looks like this:

public void Foo() {
   Foo("");
}

public void Foo(string s) {
    if (s == null) throw new ArgumentNullException("s");
    Bar(s.Length);
}

Let's consider the possible ways our misguided user can use this API. If he or she calls the convenience overload, or passes a string literal, everything's honkey dorey:
Foo(); // This is fine
Foo("How the hell do I use this API?"); // This too is just fine

But if misguided user passes null to Foo(string), or passes a variable which may be null, things aren't so rosey:
Foo(null); // Exception city!
Foo(someString); // It depends...

How is our misguided user to know what is and is not a safe call? They could look at our documentation. Assuming we wrote any. And assuming we correctly documented all of the parameters which need to be non-null. Or they could test passing null and see if it throws.

(As a side note, I wonder all the time about whether I can pass null to an API. Documentation is usually no help and if I see a parameterless overload, I generally assume that I can)

More likely, they will do none of the above and just write something resembling the last example call, passing a variable. A variable which is usually not null. A variable which is never null during testing. But a variable which, when released into the wild may, under some unforeseen circumstance, be null. Then all the kids would cry.

The solution:

public void Foo() {
    Foo(null);
}

public void Foo(string s) {
    if (s == null) s = "";
    Bar(s.Length);
}

All possible inputs to this API are valid and there is zero change of an exception. This is better!

Some may complain about semantics purity or somesuch. They are wrong.

So, if you have some public API which takes a parameter that a) is not allowed to be null, and b) has a default value for the purpose of convenience overloads, use the sentinel null. Just do it.

Now Is The Winter of Our Optional and Named Parameters

Optional and named method parameters are coming to C# 4. This means you can provide default values for method parameters. When you call the method, you can name only the parameters you want to specify - default values will be used for all other parameters.

Here's how you use it:

public void Foo (
    int doodad = 1,
    string humdinger = "",
    string wuchacallit = "STELLAAAA!")
{
    // Do stuff here
}

void Test ()
{
    Foo ();
    Foo (humdinger = "Beard Lust");
    Foo (doodad = 5,
         wuchacallit = "SHAMU!");
    Foo (wuchacallit = "Kia",
         humdinger = "Ora",
         doodad = 42);
}

Here's how it really works:

public void Foo (
    [Optional, DefaultParameterValue(1)]
    int doodad,
    [Optional, DefaultParameterValue("")]
    string humdinger,
    [Optional, DefaultParameterValue("STELLAAAA!")]
    string wuchacallit)
{
    // Do stuff here
}

void Test ()
{
    Foo (1, "", "STELLAAAA!");
    Foo (1, "Beard Lust", "STELLAAAA!");
    Foo (5, "", "SHAMU!");
    Foo (42, "Ora", "Kia");
}

The compiler just sprinkles the default values into every callsite. There are a few problems with this approach. Let's suppose I release version 1 of my awesome library with the above Foo method. You compile. All is well. Now let's say I release version 2 of my library, in which the default value "STELLAAAA!" is changed to "KHAAAAN!". But your code still has the old default value baked in. You need to re-compile your code to get the new default value. There is also the problem that injecting the full argument list into every callsite bloats the size of the code. Bigger code means more to JIT and fewer cache hits

How it should work:

struct FooSettings {
    int doodad_value = 1;
    string humdinger_value = "";
    string wuchacallit_value = "STELLAAAA!";

    public int doodad {
        get { return doodad_value; }
        set { doodad_value = value; }
    }
    public string humdinger {
        get { return humdinger_value; }
        set { humdinger_value = value; }
    }
    public string wuchacallit {
        get { return wuchacallit_value; }
        set { wuchacallit_value = value; }
    }
}

public void Foo (FooSettings settings)
{
    // Do stuff
}

void Test ()
{
    Foo (new FooSettings ());
    Foo (new FooSettings { humdinger = "Beard Lust" });
    Foo (new FooSettings {
        doodad = 5,
        wuchacallit = "SHAMU!" });
    Foo (new FooSettings {
        wuchacallit = "Kia",
        humdinger = "Ora",
        doodad = 42 });
}

Thanks to C# 3's object initialization, you can squint at the call and almost see the named parameter syntax (just ignore "new FooSettings"). This pattern of using a special "settings" type for passing arguments to methods already exists in the framework (see XmlReader.Create and XmlWriter.Create for an example). I am proposing that the compiler auto-generate these types and provide full optional/named parameter sugar. The compiler-generated types would be publicly nested within the type containing the method and named "[MemberName]Settings" by default.

This is better than callsite default value injection because:

• It versions well


• It adds a fixed amount of additional code (the type), whereas injection adds more code every time you use it


• It is CLS compliant

This is how C# 4 should do optional and named parameters.

Generic Type Parameters AS Method Parameters

I have long had an interest in method contracts (pre- and post-conditions). I followed Spec# and I continue to follow the Pex project. I am also a big fan of doing argument verification at the highest possible level of a public API. I was working on a public API today which takes a Type object. My method looked like this:

public string GetClassName (Type type) {
    if (type == null) {
        throw new ArugmentNullException ("type");
    }

    if (!type.IsSubclassOf (typeof (UpnpObject)) &&
        type != typeof (UpnpObject)) {
        throw new ArgumentException (
            "The type does not derive from UpnpObject.",
            "type");
    }

    // do stuff with 'type'
}

It then occurred to me that I can do the same thing with a generic type parameter, but get all the checks for free!

public string GetClassName<T> () where T : UpnpObject {
    //do stuff with 'typeof (T)'
}

Tada! Using generic type parameters as method parameters is nothing new (Aaron has something like this in the Banshee service stack), but the really neat thing is that you can use the generic constraints as a kind of argument pre-condition. If you have a method which takes a Type, consider using a generic type parameter rather than a method parameter. It guarantees that 'null' cannot be passed and it allows you to specify ancestry, interfaces, ref/value types, and the presence of a default constructor.

C# 4 is NOW!

WHAT?
Generic type variance support just landed in mcs. This is a C# 4 language feature.

WHERE?
You can give it a go by checking out SVN trunk and compiling your variant code with gmcs -langversion:future.

REALLY?
Well, this adds compiler support for variance but the Mono VM isn't up to speed on its variance handling. This means that you can compile the code but it won't actually run on Mono (until we fix that, which I am also doing). You can run it on the .NET 2.0 VM.

WHAT THE HELL ARE YOU TALKING ABOUT?
Generic type variance is like this:

Let's say I have some IEnumerable<string>, like so:

IEnumerable<string> myStrings = GetSomeStrings ();

Now let's say I have some other method which takes an IEnumerable<object>, like so:

void DoStuff (IEnumerable<object> someObjects)
{
    foreach (object o in someObjects) {
        // do some stuff with each object
    }
}

POP QUIZ: Can I pass myStrings to DoStuff in C# 3? Strings are objects, right? And IEnumerable<T> is just a way of getting Ts. So if strings are objects, and IEnumerable<string> just returns strings, then we can also say that it returns objects. Just like IEnumerable<object>. So it should work, right?

ANSWER: Negatorz!

This is a problem of generic type variance. There are two kinds of variance: covariance and contravariance. The above example is covarant, meaning that you want to broaden the type of an output. Contravariance is the opposite: narrowing the type of an input. Let's consider a delegate:

delegate void Handler<T> (T input);

And let's say that we have some Handler<object>:

Handler<object> myHandler = delegate (object o) {
    // do something with the object
}

Now let's say that we have a method with takes a Handler<string>

void HandleStrings (Handler<string> handler, IEnumerable<string> strings)
{
    foreach (string s in strings) {
        handler (s);
    }
}

We want to pass myHandler to HandleStrings. A Handler<object> takes objects, and strings are objects, so anything which is a Handler<object> should also be a legal Handler<string>. This is an example of contravariance.

It may surprise you to know, but the CLI has supported generic type variance since version 2. The rules are:

• Variant type parameters are only allowed in interfaces and delegate types.


• Contravariant type parameters can only be used as by-value method parameter types.


• Covariant type parameters can only be used as method return types and generic arguments to inherited interfaces.


• Only reference types are variant (this isn't explicitly stated in the spec, but it is the case).


• Languages may choose to ignore variance and treat all generic parameters as invariant.

For whatever reason, the C# language team has so far chosen not to support generic type variance. Well, that will be changing in 2010. The preview given by Anders Hejlsberg at PDC '08 revealed that C# 4 will finally support variance. But who wants to wait? Especially considering that this has been a .NET VM feature since 2006. So you can now use this in gmcs if you pass -langversion:future.

Covariance (which, as you will remember, can only be used as a method return type or as a generic argument to an inherited interface) is denoted with the "out" keyword before the type parameter identifier:

interface IFoo<out T> : IBar<T>
{
    T Bat { get; }
}

Contravariance (which is legal only as the type of a by-value method parameter) is denoted with the "in" keyword:

interface IFoo<in T>
{
    void Bar (T bat);
}

So there you go! Now we just need to get Mono's VM variance support polished off. I'm sure I'll have good news for you about that shortly. Thanks goes to Marek Safar for reviewing patches. If you have questions about how or why variance works (it's kind of tricky to get your head around), leave a comment. I might do a post delving into all of the little rules behind variance.

Equality Now!

There are three primary ways to handle equality in .NET: overriding Object.Equals, overloading the == and != operators, and implementing IEquatable<T>. Framework Design Guidelines offers pretty good advice on when to use what. A quick summary:

• If you want custom equality logic, whatever else you might do, override Object.Equals. This method is used by the various data structures in System.Collections (and elsewhere in the BCL) to determine equality. It's the first best way to do equality.
• IEquatable<T> should be implemented by structs with custom equality. Because calling Object.Equals on a struct involves boxing, and the implementation for value types uses reflection (!!!!!), IEquatable<T> gets around both of those problems. When implementing IEquatable<T>, always override Object.Equals as well.
• Operator overloading is a little bit tricky because it's really just a compiler feature. The compiler has a lookup rule for the operator implementation which takes the most derived types along the operands' type ancestry. Whereas Object.Equals is a virtual method whose overridden implementation will always be used, overloaded operators will only be used if both operands are of static (compile-time) types that are or derive from the types specified in the overload. Overloading operators is a matter of discretion. It's more commonly done with value than reference types. If you overload the equality operators, also override Equals (an implement IEquatable<T> if the type is a struct).

When you are just overriding Equals, here is the pattern I find works the best:

public override bool Equals (object obj)

{

  MyRefType mine = obj as MyRefType;

  return mine != null && /* custom equality logic here */;

}

If you want to overload operators and it's a reference type, here's the thing to do:

public override bool Equals (object obj)

{

  MyRefType mine = obj as MyRefType;

  return mine == this;

}

public static bool operator == (MyRefType mine1, MyRefType mine2)

{

  if (Object.ReferenceEquals (mine1, null)) {

    return Object.ReferenceEquals (mine2, null);

  }

  return !Object.ReferenceEquals (mine2, null) &&

  /* custom equality logic here /*;

}

public static bool operator != (MyRefType  mine1, MyRefType  mine2)

{

  return !(mine1 == mine2);

}

If you have a value type, here's the scenario without overriding the operators.

public override bool Equals (object obj)

{

  return obj is MyValueType && Equals ((MyValueType)obj);

}

public bool Equals (MyValueType mine)

{

  return /* custom equality logic here */

}

And here's the value type with operator overloads

public override bool Equals (object obj)

{

  return obj is MyValueType && Equals ((MyValueType)obj);

}

public bool Equals (MyValueType mine)

{

  return mine == this;

}

public static bool operator == (MyType mine1, MyType mine2)

{

  return /* custom equality logic here */

}

public static bool operator != (MyType  mine1, MyType  mine2)

{

  return !(mine1 == mine2);

}

The purpose of the above patterns is to centralize the equality logic in one place. You'll notice in the example which uses all three approaches, the == operator is the only place with actual equality logic. I find this just makes thing easier. If you want, you can have custom logic in the != overload (the logical inverse of ==), but that means you have to make changes in two places if you alter the equality logic, and it's really easy to make a mistake with logic operators.

Want to share your tips for equality? Have a better pattern? Leave a comment!

Advanced Topics in Inefficiency: Anonymous Methods

This is the first in what may be a series of posts on various theoretical (and not so theoretical) corner cases in common code. Despite being obtuse, these issues are useful to explore both to avoid inefficiencies and to better understand what's happening behind the code. Today's topic: anonymous methods.

class Foo()
{
    void Bar()
    {
        var thing1 = new Thing();
        var thing2 = new Thing();

        DoSomeStuff (() => thing1.Shimmy());
        DoOtherStuff(() => thing2.Shake());
    }
}

There is a potential memory problem with this method. It's not obvious from looking at the code, but both things must be garbage collected together. As long as there is an active reference to one, the other will live on as well. If 'Thing' is a heavy type, this could keep significant memory from being reclaimed on the heap. To better understand, let us look at how the C# compiler handles anonymous methods.

The above example demonstrates "local variable capture." This means local variables from the enclosing method body can be used inside closures (such as the two lambdas above). To accomplish this, the C# compiler shunts the values of the local variables to an object. The type of the object is generated by the compiler. In essence, the compiler turns the above code into this:

class Foo()
{
    class GeneratedTypeForMethodBar
    {
        public Thing thing1;
        public Thing thing2;

        public void AnonymousMethod1()
        {
            thing1.Shimmy();
        }

        public void AnonymousMethod2()
        {
            thing2.Shake();
        }
    }

    void Bar()
    {
        var closure_object =
            new GeneratedTypeForMethodBar();
        closure_object.thing1 = new Thing();
        closure_object.thing2 = new Thing();

        DoSomeStuff(closure_object.AnonymousMethod1);
        DoOtherStuff(closure_object.AnonymousMethod2);
    }
}

This is actually a very clever way of achieving local variable capture since it makes use of the CLI's pre-existing garbage collector to clean up the captured variables. The problem is, the compiler shunts all local variables to a single object. In our above example, the two anonymous methods do not reference any of the same local variables, but both local variables are stored in the same object. This can lead some captured variables to become prisoner variables: they are no longer needed, but they cannot be garbage collected. Suppose that our 'DoSomeStuff' method just invokes the delegate and returns. No problem. But now suppose that our 'DoOtherStuff' method holds on to the delegate, perhaps planing to invoke it later. Or suppose we were to return the second lambda, allowing the caller to hold the delegate as long as they please. That delegate holds a reference to the 'closure_object' which holds a reference to both Things, even though that delegate just needs 'thing2'. There is no way for any code to reach 'thing1' but it won't be garbage collected until we're done with 'thing2'.

Solution?

Well, we could modify the compiler to generate a type for each set of local variables that appear in only one anonymous method, like so:

class Foo()
{
    class GeneratedTypeForMethodBar1
    {
        public Thing thing1;

        public void AnonymousMethod()
        {
            thing1.Shimmy();
        }
    }

    class GeneratedTypeForMethodBar2
    {
        public Thing thing2;

        public void AnonymousMethod()
        {
            thing2.Shake();
        }
    }

    void Bar()
    {
        var closure_object1 =
            new GeneratedTypeForMethodBar1();
        closure_object1.thing1 = new Thing();

        var closure_object2 =
            new GeneratedTypeForMethodBar2();
        closure_object2.thing2 = new Thing();

        DoSomeStuff(closure_object1.AnonymousMethod);
        DoOtherStuff(closure_object2.AnonymousMethod);
    }
}

This poses problems as well. First of all, we are instantiating two (or more) generated-type objects rather than one. Object instantiation is not cheap and that could potentially slow down certain code. Also, this approach cannot be used to optimize more complex scenarios. Suppose we have five anonymous delegates, each referencing some of seven local variables like so: a {1 2} b {2 3} c {3 4 5} d {1 5 6} e {6 7}. In these situations we must default to the one-compiler-generated-type-for-everything approach.

Ultimately, the lesson here is just to be aware of these potential issues. If you find via profiling that objects are not being garbage collected and you make heavy use of anonymous methods, you might want to examine your closures to make sure this isn't causing the problem.

And what do people think about modifying the compiler as proposed above? Also, anyone who comes up with a better mechanism for local variable capture gets cool points. Double points if your solution doesn't require VM changes.

P.S. Thanks to Michael for help with this post.

Mono.Upnp: A Bun In The Oven

Aaron's post has piqued some interest, so I thought I'd give up the goods.

WTF

Mono.Upnp is a fully managed UPnP implementation. The client and server stacks are in separate libraries to give embedded devices a compact solution for either. The client library is a solid implementation of the UPnP standard designed to robustly cope with other, less compliant UPnP devices (seriously, no one actually implements the UPnP standard correctly. Especially not router manufacturers, may they die a thousand deaths). The client API is also highly extensible, allowing easy consumption of services which offer functionality not established in the spec. The server library offers developers an easy way to expose their services over UPnP: attributes indicate UPnP-visible members and the library handles all of the network plumbing. In addition, all of the official UPnP device and service APIs (the specs which sit on top of the UPnP architecture - for example, the MediaServer API) are wrapped both for client and server use.

Holy Shit!

Indeed. Well, almost. All of the above sounds great, but it's not done yet. In fact, it's still quite experimental. While I've made great progress in impressive screenshots, actually listening to your Banshee collection on a PS3 is still a few hack-o-thons off. But things are moving along nicely and I will certainly keep you up to date.

OK

In the mean time, curious monkeys can find the source in the mono-upnp module of the Mono SVN. Like I said, you shouldn't be using this code, but you're welcome to check it out.