Important: some of these are out of date and superceded by later notes I took after talking with Microsofties. I’ve tried to mark them as such where I remember with a [*].

Visual Studio 11

No release date of course, same as Win8. Feels like “definitely not this year”. Did you all download the preview yet? It’s available for everybody. Win8 and the dev tools (VS11, Blend 5, etc.) are online.

Anyway, what’s new…

• After 10 years they finally updated the image editor. Now it supports transparency and DDS’s.
• VS has an FBX viewer w/ property editor. Right in the IDE you can open a 3D model, select meshes, set properties like color and transform.
• They put a lot of work into a new thing called DGSL that is your typical shader graph.
• Preview rendering at different stages of the pipeline, things like pins and wires connecting them to output a final render.
• Good for the XNA crowd I guess? Seems like a distraction for the VS team.
• Pix is integrated into VS.
• You can debug a 3D app, take a frame grab, then in VS walk the drawprim calls backwards from a pixel. Get call stack per graphic event and all that.
• Being integrated into VS means that this this will have a nice, consistent UI, yay.
• There is this great DOM Explorer (JS/HTML only)
• Reminds me a lot of the tools built into Firebug or Chrome’s dev mode, except it runs through VS.
• Even on a server web app you can hover over a rendered web page and it will point you at the aspx line that generated it.
• Productivity Power Tools are part of VS now. You probably already know this. Well, perhaps they’ll fix some of its many bugs if it’s an in-box addon.
• Nice random improvements to the VS UI like categories for Intellisense (“commonly used by you” vs. “all”) and minor new features in snippets.
• Maybe now R# will pick up this much-needed feature that VisualAssist has had for a while..
• Await/async is deeply supported in the debugger.
• Step through async code (like WinRT’s ShowDialogAsync) procedurally without needing to get confused by all the underlying thread context switches. I like.
• Tons of new unit testing support. Now is the time when I remind myself to look into unit testing again..
• The Kernel Debugger can now run from inside of VS. Sure to come in handy next time you’re developing a device driver (har).
• Static code analysis is now a plug-in architecture.
• He talked about, for example, the kernel mode driver validation static analysis tools that just show up as part of the compile.
• Works on C++ or C# or whatever.
• I expect we’ll see the same type of integration for FxCop/StyleCop
• Javascript in VS now has a REPL command pane, just like the browsers have had.
• Full support for ECMAScript 5. I’m only familiar with 4, which added (I think) static typing. Well, fake static typing. Anyway, now we have 5! That’s 1.0 better than 4!
• No word of improving E&C like supporting altering a function containing a lambda..I’m going to ask them about this. [* answer: no]
• Ooh a nice new color picker for CSS. Probably for Xaml as well.
• VS11 projects are going to be the same format as VS2010.
• No upgrade required, forward/backward compatible. That will make testing the beta on our stuff easier.

WinRT (Windows Runtime – the Metro API)

• “Language Projection” is what they’re calling the layer between the language and the kernel services. It maps a language onto the underlying API’s, reflection and type system (the metadata).
• They are dropping the least-common-denomenator “CLI” system and instead are mapping things as close as possible to the language you’re using.
• Example: on an array returned by the core, C# uses “.Add()” to add an item, whereas Javascript uses “.append()”. I like this. Least common denominator probably wasn’t working out for them anyway because nobody really cared about making things VB-compatible when writing libraries in C#.
• WinRT types are automatically adapted to the language. For example getting an array back from WinRT will come back to C# as an IList<T>, done through adaptors but not by making a new list (no copying). In C++ they have adaptors to get it into STL iterator form, etc.
• Async is embedded very deeply.
• Martyn said that in .NET when they provided a sync and an async version of the same function, people would just use the sync version and then get crap behavior and experience.
• New stuff heavily encourages use of async, though not sure if that means the sync functions are unavailable [* they are].
• New Windows Metadata format that is used sort of like the Reference Assemblies, sounds like, except this metadata IS the OS. Not libraries on top, but the actual Windows API’s.
• Format is based on .NET type system but extended to meet the new needs.
• Metadata catalog is extensible (again, Metro apps only, for custom WinRT objects) but only privately. No publishing of shared assemblies. Not that they are using the word “assembly” any more…
• Sharing across apps is done exclusively through “contracts”, which are part of the OS. If you want to communicate across apps it must be done through a preexisting contract, looks like.
• YAY! A new string type!!! This one is called HSTRING and is probably the 15th string type I’ll have used.
• It gets mapped onto whatever the local language thinks of as a string but sounds just like a .NET string – immutable, and so does not copied.
• WinRT is heavily interface driven, as is now common in the rest of the programming world
• IUnknown is now IInspectable. They said IUnknown confused people or something. Anyway, IInspectable -> ISpecificInterface -> ConcreteClass (runtime, private).
• C++ binds statically as usual.
• C#/VB binding is mostly static but also uses some runtime metadata (very cheap though)
• JS/HTML binding is dynamic dispatch at runtime, works completely through metadata as you expect a dynamic language to operate
• Some MC++ changes, like “gcnew” is now “ref new”. [* turns out this is not true. See later notes.]

Other

• Lots and lots of stuff about Azure, which is pretty neat but I’ve heard is very expensive for large-scale use.
• Pithy statement I just made up: “swipe is the new right click”.
• TFS marches forward.
• The new stuff looks really cool, and I’d consider using their hosted TFS if I had a small studio. They even support scrum directly.

Expo Floor Queries

WPF

• What’s New in WPF Version 4.5 Developer Preview: http://msdn.microsoft.com/library/bb613588(v=VS.110).aspx
• WinRT is definitely Metro only.
• We cannot embed Metro in WPF, and vice versa (well we can, but they said it would be awful, dirty, and please run away).
• Metro really is a completely independent sandbox that has little to do with Windows.
• [* Some of the above is a lie, as I found out on Friday. WinRT is definitely useable outside of Metro. See later notes I’ll post.]
• Very little work has been done in WPF. It still exists! Visual Studio and the Expression suite are all WPF apps and will remain that way. But for 4.5 they focused exclusively on perf and stability. I suppose that’s ok.
• I lost track of how many times I’d ask someone about desktop apps (WPF) and would get asked why we didn’t just “rewrite it all” for Metro. Ah, Microsoft, never change!

Blend/VS

• The reason Silverlight has data-binding breakpoints is because the Silverlight team put the hooks in there independently and VS was able to use them.
• Until someone does that for non-SL Xaml (whether Metro or WPF) we will not have this functionality in VS.
• The reason JS/HTML has such freaking awesome debugging support (like interactive mode) is completely provided by the hooks already developed independently in IE over many years.
• Again, work that Blend/VS did not have to do.
• So they’re waiting for debug hooks in Xaml frameworks, which may never appear. Complicated problem to solve and limited resources/initiative to do it.
• We should use DynamicResource and not StaticResource as a general rule.
• They know that debugging resources sucks.
• They want to work on the problem. They aren’t working on the problem. They have no plans to do so.
• They recommended using DynamicResource (again) and binary-search style commenting out stuff to narrow down problems. Yeah, so, nothing new here.

I sent out notes about the conference to my team and figured I’d post them here as well in case it would be useful to anyone. Stripping out any Bungie-specific commentary that wouldn’t be generally applicable of course.

Important: some of these are out of date and superceded by later notes I took after talking with Microsofties. I’ve tried to mark them as such where I remember with a [*].

Notes From Tuesday

I’m writing these up while it’s still fresh in my head..

Two big takeaways for me:

1. Windows 8 will actually be really awesome. Seriously.
2. It won’t matter to production tools developers. Business as usual.

As a consumer, I am really looking forward to Windows 8. They did what needed to be done in order to live in the modern world: new API, breaking of backwards compatibility, heavy integration of touch and gestures, complete guidance provided for new design, large library and protocols for connecting apps together, support for web development on the desktop to bring in all those crazy kids, plus some really forward looking design…

But only for Metro – “the new stuff”. Windows 8 will run all the apps that 7 did, but there are now two classes: desktop apps and Metro apps. Desktop apps are 3DS Max, Photoshop, Word, Notepad, etc. Metro apps are 100% new. It is a brand new platform in just about every way. Really don’t even need to call it Windows.

Metro has clearly been the focus of Build and therefore Microsoft. So far today, the only thing I heard about ordinary desktop apps is that “they will still work great”. They all run within a context in Metro called “Desktop”, which is already seeming very antiquated, almost like Windows XP Mode was with Vista. Everything MS is talking about here is geared towards Metro. And it’s seriously cool. I look forward to switching all my home machines over to it, especially my girl’s Dell Duo. Win8 is fast, lean, and cool as hell.

So I don’t have much to report on the future of our platform: Win32/64 + WPF/.NET. I’ll be finding out what I can over the next three days.

I did take some notes, though. Here they are in no particular order:

Non-Metro:

• No mention of WPF all day. Not even whispered. Is WPF rolled into Silverlight? Is WPF on life support? What about frogs? No word yet.
• New C++ standards compliant compiler is shipping with Visual Studio vNext. I saw the “auto” keyword being used for duck typing.
• They kept making a big deal about performance as a top priority, over and over again. This is a pillar of Win8.
  • Perf is even more important than in Windows 7 due to touch support, because people are unwilling to put up with any latency whatsoever when using a touch interface. So look for a lot of async support integrated at all levels.
• Cider is gone. Blend’s designer is now integrated into Visual Studio. Big yay. Always annoyed me that something would work in one place and not the other, or vice versa. (Now we can have things not work in both places, but at least it will be consistent.)
• There’s a new debug view that lets you view parallel stacks of async/TPL tasks as well as the program flow through the tasks. I’m going to ask them if this can be applied to our own scheduling system (or if we should switch to Task in order to take advantage).
• Looks like they upgraded the remote debugging, way easier to set up. Even have remote Javascript breakpoints. Definitely won’t need Firebug in this world.
  • There is even a DOM inspector for JS that works over remote debugging. Not only are the UI elements hot as you hover/touch them (like in Snoop or Chrome dev) but you can modify properties on the fly while debugging. Remotely! OMG!
• Blend has a crazyass new interactive mode where you can run the app inside of Blend, fully interactive, then pause it on the screen you want to modify and switch to design mode to update the Xaml. Sort of like play-in-editor. (Think Expanders or other state-dependent stuff..a bit of a pain to do right now.)

Metro:

• XAML is coming to C++ which compiles down to native x86/ARM. Appears to be Managed C++ due to use of ^ ref modifier [* nope]. Makes sense. Don’t know how they’d make any of this work with pure C++.
• Thankfully they’re not just copying Apple but fixing a lot of stupid UI problems on the iphonepodpad. My favorite is how they said that the modal rearranging of the home screen is crap and how folders bury things too far. Couldn’t agree more.
  • They are also expanding the “touch language” beyond existing standards of swipe-to-scroll and pinch-to-zoom/rotate. Selection, context bars, app switching..
  • Totally eliminating the “mouse-ish” stuff like holding down a touch to bring up the context right-click. All of those old Windows 4 Pen-isms are gone. This is why touch sucked on Win7/Vista and is finally usable on 8.
  • BTW if you have any time at all, be sure to watch Jensen Harris’s “big picture” talk from today (should be posted online sometime on Wednesday). I thought it was superb.
• Metro apps do not have access to Win32 api’s (such as MessageBox or anything in GDI)! – no overlapping windows etc
  • All graphics are done through DirectX only.
  • Metro apps are totally different. Don’t even have a message loop. [* not true]
• I saw examples of the “await” and “async” keywords a lot in the demos. This is a key part of delivering high perf UI. Their expectation is that nothing is synchronous.
• WinRT (windows runtime, the new API) appears to be a true new API and not just a forms/.net style wrapper.
  • This one appears designed well from the start! Takes advantage of modern design patterns and not held back by obsession with backwards compat.
  • They said several times that this is definitely not a “layer on top” (like how winforms wrapped up user32 etc.) but talks straight to the kernel. [* mostly true]
  • Fully object-oriented, has reflection, even in C++.
• Async support features are implemented in all languages.
  • For example in Javascript, known as “promises”: mediaCapture.initializeAsync().then(function() {…}) type stuff.
• The javascript integration is pretty damn cool. Has data binding even. Hope this brings web devs in to make things for Win8 (obviously MS does too).
• Can write native C++ to reference-in from javascript. They gave a demo of HLSL compute shader-based image processing lib written in C++ and used in Metro javascript app. Neato.
• Single file .appx package for distro. Yay finally. No more installers! Turns out this is just a zip file too, like Java did with jars. Yay, no more proprietary package formats!
• They have a very cool Metro simulator that runs your app through a fake local RDP session that makes it easy to test different rez / dpi / orientation without needing a VM or separate computer with the correct display.

Trying to create a new notebook on SkyDrive through OneNote and can’t choose “Web” from File | New? Error is “This service is disabled by policy. Contact your system administrator to enable it.”?

Solution:

HKEY_CURRENT_USER\Software\Microsoft\Office\Common\WebIntegration\WebIntegrationEnabled=1


if (obj == null) { obj = defaultValue; }
Console.WriteLine(obj.Name != null ? obj.Name : "<null>");

…into this:

obj = obj ?? defaultValue;
Console.WriteLine(obj.Name ?? "<null>");

Awesome. That covers a lot of simple use cases. Now, there’s another super common case: accessing members of objects that may or may not be null. Like this:

var name = obj != null ? obj.Name : "<null>";
var lname = obj != null && obj.Name != null ? obj.Name.ToLower() : "<null>";

Yuck.

It would be great if C# had a ?. operator that lets you tersely express “return the member if the object is not null, otherwise return null”. Then we could do this instead:

var name = obj?.Name ?? "<null>";
var lname = obj?.Name?.ToLower() ?? "<null>";

Well, we can do something like this using one of my favorite tools, the extension method.

public static class ObjectOperations
{
    [DebuggerStepThrough]
    public static TReturn NullOr<TIn, TReturn>(
        this TIn obj, Func<TIn, TReturn> func,
        TReturn elseValue = default(TReturn))
        where TIn : class
    {
        return obj != null ? func(obj) : elseValue;
    }
}

NullOr means “this object can be null or we’ll run the function”. Access to the member can be expressed via a lambda, so the above code can now be expressed like this:

var name = obj.NullOr(_=>_.Name) ?? "<null>";
var lname = obj.NullOr(_=>_.Name).NullOr(_=>_.ToLower()) ?? "<null>";

Not bad. It isn’t as terse as I’d like, but it is fluent and more readable, at least to me. Also, being a lambda, you really can put whatever you like in there. It’s not limited to just member access.

Extra

For fun I threw in an “elseValue” clause that lets you provide a default other than null. I haven’t had cause to use this yet, because I find the ?? more readable. Of these, which do you prefer?

var name = obj.NullOr(_=>_.Name) ?? "<null>";
var name = obj.NullOr(_=>_.Name, "<null>");

I think I prefer the ?? version because it expresses “no matter what, if the result of this entire chain is null, use this instead”, whereas the default gets a little lost when it’s part of the final function call. It also is more work to rearrange the clauses, should the order of lookups be changed for whatever reason.

(I also wrote this up at StackOverflow and earned myself a nice Necromancer badge.)

Aside from a better configuration language, what I really like about coding fluently is that it’s a steady stream of left-to-right operators, like a pipeline. You think of the next operation and just chain it along, never having to back up. Domain specific sublanguages like LINQ (which I adore, incidentally) can make this even more compact, but I’m talking about the function-based fluent operators.

For enumerables, it’s easy:

var names = list.Where(i => i.Foo > 5).Cast<IBar>().Select(j => j.Name);

Take the list, filter it, cast to the type we want, and pull out an element. Operation by operation, left to right. But what about for a single object? That fluency hits a brick wall when you run into C#’s old C-style casting on an individual object.

var name = ((IBar)list.First()).Name;

Yuck. Not fluent, and hard to read. As you’re typing, you have to back up, stick the casted type on the front in parens, and, oh yeah, have to also go wrap the whole thing in parens to be able to continue the method chain. I’ve always hated having to do this and in the past would usually pull out the casting into a separate local.

It’s also easy to fix. We just need a little extension method:

public static partial class Extensions
{
    [DebuggerStepThrough]
    public static T To<T>(this object o)
        { return (T)o; }
}

Now it’s nice and fluent:

var name = list.First().To<IBar>().Name;

So we now have the system Cast<T> operator for casting elements within an enumerable, and To<T> for casting the overall type. Solid.

Edit absolute_privacy.php. Easiest way is through the admin interface under Edit Plugins. Change this line:

if ( !$user || ($user->user_login != $username) ) {

To this:

if ( !$user || (strtolower($user->user_login) != strtolower($username)) ) {

Now you can log in with whatever case you like.

Make sure you do the following:

• You must have the 64 bit compilers installed or you won’t be able to add 64-bit configurations to p4dn. Most likely you didn’t check the box for this when originally installing Visual Studio. Just go to add/remove programs, “change”, and check 64-bit under the C++ area.
• The p4dn library requires Perforce’s p4api native library, which comes in many flavors. It is included in the distribution for P4.Net, but it is an old version. I recommend getting the latest API from their FTP site.
• I used the p4api_vs2008_dyn and p4api_vs2008_dyn_vsdebug configurations from the ntx64 and ntx86 builds.
• Why dynamic instead of static? .NET is all about DLL’s – using static just causes problems with the linker with conflicting runtimes.
• Update the include and lib paths in the p4dn C++ project settings so that the library can find your p4api. If you don’t do this you’ll get #include errors about missing paths.

[Update: Stefan posted in a comment below a much simpler method. I recommend going with this instead of my considerably more complex method.]

Here’s a problem that was bugging me at my last job that I finally got around to solving last week on the bus: letting your top-level project run as AnyCPU and automatically choose a 32-bit or 64-bit low level  native/managed DLL based on environment bit width.

Say you’re using Shawn Hladky’s great P4.Net project (source) so your C# can speak Perforce – two great tastes that taste great together. P4.Net talks to the server using a native Perforce API. Now, unlike a C# EXE, which is usually AnyCPU (i.e. “let the jitter decide”), the native code that talks to the Perforce server must be compiled as either x86 or x64. This causes two big problems.

The first problem is that your app will crash with a confusing error if run on the wrong version of the .Net Framework! Say you’re using the x86 P4.Net and you run your app on Win7-x64. The jitter will compile the app in 64 bit, but on the first reference to P4.Net, it tries and fails to load the x86 DLL, and puts up an unhelpful error about a bad image format.

The easy workaround to this, of course, is to mark your EXE as x86 instead of AnyCPU. Solved. Unfortunately, you have to remember to do this with every single app that references P4.Net. Or references a DLL that indirectly references P4.Net. It’s like a virus in that way, but we can handle it.

Well, no. That leads to the second problem. What happens if you reference P4.Net (directly or indirectly) into an app that really does need 64 bit? Like, say, some memory-hoovering game build related tool that runs on the server farm? Well now you need a 64-bit version, not only of P4.Net, and not only of your EXE, but of every single DLL that is referenced on the path down to your new P4.Net_x64.DLL. Now we have a real problem. The pain in the ass to maintain kind of problem. Do we really want to have our tool chain output 32 and 64 bit versions of everything, just in case?

Had Microsoft supported fat binaries like NeXTSTEP did back in the early 90’s, we’d just have P4.Net with 32 and 64 bit in the same DLL, and go on with our lives. But no, we have to jump through hoops. This article is the story of how to jump through those hoops to make your bits go.

I’m actually a bit shocked that Microsoft hasn’t extended PE and their OS loaders to support fat binaries. There would be zero perf cost, and it’s been a long time since we had to worry about the size of binaries on disk (content overwhelms executable code size in nearly every app today).

About P4.Net

If I’m going to continue to use P4.Net for my example, I need to give a little more background. P4.Net is built from three components:

1. p4api.lib: A native C++ API (headers and libs) provided by Perforce to talk to their server directly through sockets, without running p4.exe or using the COM object.
2. p4dn.dll: A bridge assembly that statically links in p4api, and uses Managed C++ to export p4api as a low level set of .Net types.
3. p4api.dll: A managed C# API that wraps up the low level p4dn and adds functionality to make it easier to work with. This is what everybody does an “add reference” on to talk to P4 from C#.

Note that p4api.dll is not strictly necessary as a separate assembly. The low level types exported by p4dn.dll could instead be kept internal, and all of that C# code from p4api.dll be written in Managed C++ and moved into p4dn.dll, entirely eliminating the need for p4api.dll.

Personally I was a fan of Managed C++, up until C# 3.0 where we started getting all kinds of nice language syntax to write better code more compactly. Today, I suppose I’d keep the extra DLL just for easier maintenance.

The Solution

In a nutshell, the solution is to trick the loader! Reference a p4dn.dll that does not exist, and use the AssemblyResolve event to intercept the load and reroute it to the correct bit size assembly.

It’s simple in concept but has a lot of details that took me a whole bus ride to figure out all the way (happily, there was a lot of traffic). Here is what I ended up doing to make it work how I wanted:

1. Rename the x86 output of p4dn.dll to p4dn.proxy.dll.
2. Update the x86 linker input settings to add __DllMainCRTStartup@12 to Force Symbol References.
3. Build a new x64 configuration for p4dn, using the x86 configuration (well, ‘Win32’) as a template. Have it output to p4dn.x64.dll.
4. Update the x64 linker input settings to add _DllMainCRTStartup to Force Symbol References.
5. Add a post-build event to p4dn’s x86 configuration that deletes p4dn.x86.* and copies the p4dn.proxy.* to p4dn.x86.*.
6. Update p4api to reference p4dn.proxy.dll. Not the csproj, but the actual DLL.
7. Update the SLN settings to make p4api dependent on p4dn.
8. Add a post-build step to p4api to delete p4dn.proxy.dll.
9. Set all p4api and p4dn configurations to output to the same bin folder.
10. Add a static constructor to P4API.P4Connection that registers an event handler on AppDomain.CurrentDomain.AssemblyResolve to pick the right DLL when the proxy is requested. I’ve pasted my code at the bottom of this post.

Once this is done, you’ll be able to have p4api as well as any assemblies that reference it set to AnyCPU. It will, upon first usage of the P4Connection class, fail to resolve the proxy and reroute to the correct bit width DLL.

A few notes on the above:

• The _DllMainCRTStartup is required because, without it, I got a crash from uninitialized memory systems in the CRT DLL’s that p4dn was linked to. This happened regardless of static vs. dynamic linking. I didn’t bother to find out the real reason for it. The different symbol names for 32 bit vs. 64 bit are because the convention changed when Microsoft went to 64 bit.
• The name of the DLL being referenced must match the original name of the DLL being built. That is, if you were to have p4dn outputting to p4dn.x86.dll then renaming it to proxy, and then referencing that, then it will actually look for the referenced DLL’s “true” name of p4dn.x86.dll and never call your hook.
• In projects that reference p4api it’s best to set the references to non-private (clear the “Copy Local” flag) and have a post-build step that just copies whatever is in the p4api bin folder. That makes sure you get the exact files that you need. None of this will work if you accidentally end up with the proxy file existing.

Here’s the code for my hook function. Note that it attempts to catch problems with the post-build scripts.

static P4Connection()
{
	string assemblyDir = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
	if (	File.Exists(Path.Combine(assemblyDir, "p4dn.proxy.dll"))
		|| !File.Exists(Path.Combine(assemblyDir, "p4dn.x86.dll"))
		|| !File.Exists(Path.Combine(assemblyDir, "p4dn.x64.dll")))
	{
		throw new InvalidOperationException("Found p4dn.proxy.dll which cannot exist. "
			+ "Must instead have p4dn.x86.dll and p4dn.x64.dll. Check your build settings.");
	}

	AppDomain.CurrentDomain.AssemblyResolve += (_, e) =>
	{
		if (e.Name.StartsWith("p4dn.proxy,", StringComparison.OrdinalIgnoreCase))
		{
			string fileName = Path.Combine(assemblyDir,
				string.Format("p4dn.{0}.dll", (IntPtr.Size == 4) ? "x86" : "x64"));
			return Assembly.LoadFile(fileName);
		}
		return null;
	};
}

Microsoft, if you’re listening: FAT BINARIES.

I have two things in mind when I say “rich base class”. First is for mixing in object functionality. I’ve spoken about this before. There really is no way to do this nicely in C#. However, today’s post is about the other thing C++ gives you with multiple inheritance: utility methods.

Consider this interface and implementing class, paying attention to the overloads:

interface ILogger
{
	void Log(string message);
	void LogLine(string message);
	void Log(string format, params object[] args);
	void LogLine(string format, params object[] args);
}

class DebugLogger : ILogger
{
	void ILogger.Log(string message)
		{ Debug.Write(message); }
	void ILogger.LogLine(string message)
		{ Debug.WriteLine(message); }
	void ILogger.Log(string format, params object[] args)
		{ Debug.Write(string.Format(format, args)); }
	void ILogger.LogLine(string format, params object[] args)
		{ Debug.WriteLine(string.Format(format, args)); }
}

The more versions of the log function you want, the more work you have to do in any class implementing the interfaces. You could create a LoggerBase that does all of this, but that is severely limiting.

In C++ it’s easy of course. Make a mixin class.

class Logger
{
public:
	void Log(const char* message)
		{ OnLog(message); }
	void LogLine(const char* message)
		{ OnLog(message); OnLog("\n"); }

	void LogF(const char* format, ...)
	{
		char buffer[2000];
		va_list args;
		va_start(args, format);
		vsprintf_s(buffer, format, args);
		va_end(args);
		Log(buffer);
	}

	void LogLineF(const char* format, ...)
	{
		char buffer[2000];
		va_list args;
		va_start(args, format);
		vsprintf_s(buffer, format, args);
		va_end(args);
		LogLine(buffer);
	}

protected:
	virtual void OnLog(const char* message) = 0;
};

class DebugLogger : public Logger
{
	virtual void OnLog(const char* message)
	{
		OutputDebugString(message);
	}
};

Exactly one virtual method is required in the derived class. To add new overloads, you just add them to the base, and have them call the virtual. All end up with dynamic behavior. With interfaces in C#, you’re forced to implement each overload. This ends up with a lot of duplication everywhere you implement this same interface. Worse, if you want to add more utility functions to the interface, you break every implementing class, which must now implement that function as well.

The other day, it hit me that there’s an easy and perhaps obvious solution to this in C#: extension methods.

interface ILogger
{
	void Log(string message);
}

static partial class Extensions
{
	public static void LogLine(this ILogger logger, string message)
	{
		logger.Log(message);
		logger.Log("\n");
	}

	public static void LogFormat(this ILogger logger, string format, params object[] args)
	{
		logger.Log(string.Format(format, args));
	}

	public static void LogLineFormat(this ILogger logger, string format, params object[] args)
	{
		logger.LogFormat(format, args);
		logger.Log("\n");
	}
}

class DebugLogger : ILogger
{
	void ILogger.Log(string message)
	{
		Debug.Write(message);
	}
}

This will do what I want. I can implement a minimal interface, and easily add new utility functions. It’s not quite as good as C++, because I can’t store any data in my extension methods, and I must work solely through the published interface, but it’s good enough for 80%.

In one very small way it’s actually better than C++. In C++, that base class isn’t always something that can be changed. Perhaps it was provided by a standard library or a third party. Yet in C#, anybody can create an extension class to add functionality to any other class. So you can add all the overloads you like.

In my previous post, I talked about why I have stopped using finalizers for unmanaged resource collection. I want this to be done through the disposable pattern instead, forcing the programmer to manage resources manually.

Ironically, finalizers are a great way to verify this.

Sander van Rossen quickly figured out where I was going with this and proposed in a comment that we can just assert in a finalizer. We just need a couple more things:

1. We need to track the source of the object to figure out where the leak is.
2. We need to ensure that finalizers run on shutdown, or our assert will never get hit.

This is familiar territory to C++ programmers. Most of us use memory management libraries that provide leak detection and reporting. Let’s do something similar in C#.

DisposableBase

If you look around online, you’ll find some full-featured IDisposable base classes intended to deal with the full finalization model. We can eliminate most of that as either too complicated or unnecessary. We just need a few things:

• A stack trace grabbed at construction time
  This will be used for the error report. Because this is expensive to gather, we need to control it with an #if that is off by default, only turned on if needed to help an investigation. Most of the time a leaked disposable will be easy to find by inspection, but in the 5% case we will need a lot more context.
• A finalizer that reports the problem
  It could throw an exception, fire an assert, or route to an error reporter. Depends on the application. In my example I just have it output to the debug window for a demo.
• Disposal helper methods
  These wrap up disposal a bit, so inheritors only need to implement OnDispose. The most important feature, though, is that Dispose will call GC.SuppressFinalize when our object is disposed. This eliminates the performance cost of having a finalizer in the normal case when clients are disposing this class properly. This is why the finalizer has no “if” in it – if it ever gets called, then we have a bug.

This is what I am currently using as my base class for handling unmanaged resources:

// comment out unless diagnosing a leak
#define DEBUG_DISPOSE

public abstract class DisposableBase : IDisposable
{
    // store stack at point of construction for possible later use
#	if DEBUG_DISPOSE
    StackTrace _trace = new StackTrace(true);
#	endif

    // finalizer will not be called if object was properly disposed
    ~DisposableBase()
    {
        string message = "!! Forgot to dispose a " + GetType().FullName;
#		if DEBUG_DISPOSE
        message += "\n\nStack at construction:\n\n" + _trace + "!!";
#		endif
        Debug.WriteLine(message);
    }

    public bool IsDisposed { get; private set; }

    public void Dispose()
    {
        ThrowIfDisposed();
        IsDisposed = true;

        try { OnDispose(); }
        finally { GC.SuppressFinalize(this); }
    }

    protected abstract void OnDispose();

    protected void ThrowIfDisposed()
    {
        if (IsDisposed)
            throw new ObjectDisposedException(GetType().FullName);
    }
}

Demonstration

Here is a simple test app that shows what happens if we forget to dispose an instance.

public class DatabaseConnection : DisposableBase
{
    protected override void OnDispose()
        { Debug.WriteLine("Disposing"); }
}

public class Program
{
    static void Main(string[] args)
    {
        using (var remembered0 = new DatabaseConnection())
        using (var remembered1 = new DatabaseConnection())
        {
        }

        var forgotten = new DatabaseConnection();

        GC.Collect();
        GC.WaitForPendingFinalizers();
    }
}

The first two instances dispose fine, but the third is leaked and so we get a log to the output window:

Disposing
Disposing
!! Forgot to dispose a DatabaseConnection

Stack at construction:

   at DisposableBase..ctor() in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 18
   at DatabaseConnection..ctor()
   at Program.Main(String[] args) in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 89
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
!!

We can easily zero in on the exact spot where the leaked resource was allocated and wrap it in a ‘using’ to resolve.

Note the use of GC.Collect and GC.WaitForPendingFinalizers right before the test application exits. This is necessary in order to force all finalizing objects to be collected and reported before process shutdown, otherwise they are simply dropped by the system when the process’s memory is released.

These two can even be called during normal app run as well for leak testing without needing to wait for shutdown. This would be useful in a live service with a periodic leak for which we want an in-session nonfatal log of leaks.

What About External Classes?

This takes care of our own classes, where we have full control. But what about system or third party classes that are using finalizers? With those it’s back to square one.

Well, I suppose would write a helper class..

public class SafeDisposer<T> : DisposableBase where T : IDisposable
{
    public SafeDisposer(T disposable) { Obj = disposable; }

    public T Obj { get; private set; }

    protected override void OnDispose()
    {
        Obj.Dispose();
        Obj = default(T);
    }
}

public static class SafeDisposer
{
    public static SafeDisposer<T> Wrap<T>(T disposable) where T : IDisposable
        { return new SafeDisposer<T>(disposable); }
}

And equivalent updates in the demo code:

public class Program
{
    static void Main(string[] args)
    {
        using (var remembered2 = SafeDisposer.Wrap(new StringReader("foo")))
        using (var remembered3 = SafeDisposer.Wrap(new StringReader("poo")))
        {
            remembered2.Obj.Peek();
        }

        var forgotten2 = SafeDisposer.Wrap(new StringReader("boo"));

        GC.Collect();
        GC.WaitForPendingFinalizers();
    }
}

And output:

!! Forgot to dispose a SafeDisposer`1[[System.IO.StringReader, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]

Stack at construction:

   at DisposableBase..ctor() in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 18
   at SafeDisposer`1..ctor(T disposable) in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 53
   at SafeDisposer.Wrap[T](T disposable) in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 67
   at Program.Main(String[] args) in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 98
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
!!

So to use, we just wrap a disposable at the point of creation in SafeDisposer.Wrap() and access it through “.Obj”. Not bad, but not great either. I’m ok with the wrapper function, but the required access via the Obj member is a pain, and also means that converting an unsafe disposable into a safe one requires updating a lot of code.

Another option is (maybe) to use a dynamic proxy system to inject the functionality we need, letting client code remain unchanged except at the point of creation. Or perhaps a run-time system that patches system assemblies to do the injection.

I’ll leave this as an exercise for the reader because I think we’re probably getting seriously diminishing returns at this point. Most disposable objects in a large system will be classes that we have full control over. The few objects not under our control will likely be low level primitives that will be wrapped up by our own foundation classes anyway.