Tag Archives: source code

Robust 3rd Party Integrations with NServiceBus

A common question about NServiceBus is how to use it to integrate with an external partner. The requirements usually go something like this:

  • The third party will contact us via a web service, passing us a transaction identifier and a collection of fields.
  • If we successfully receive the message in the web service, we respond with a HTTP 200 OK status code.  If they do not receive the acknowledgement, they will assume a failure and attempt to retry the web service later.
  • Once we receive the message from the third party, we need to distribute (think publish) the contents of the message to more than one internal process, each of which are completely independent of each other.
  • We need to logically receive each message once and only once. In other words, it would be a “Very Bad Thing” for one of the internal subscribing processes to receive the same notification more than once.

This was most recently asked in this StackOverflow question, where it became difficult to explain more within the 600 character comment limit. The best explanation is example code, so here it is.

Read more »

Flexible Reporting with LINQ and C# 4.0 dynamic keyword

It’s commonly very difficult to question business people about reporting requirements.  It’s not really their fault either – they just can’t know exactly what they want until they’re trying to answer a question and can’t easily do it with the reports you’ve given them.

This is why it’s good to make reports as flexible and updateable as possible, but with as little developer required to update the reports as possible.

If you’re operating in an environment where all database access must be via stored procedures, this is a really big problem.  It’s really unlikely that the changes requested by business can be implemented with the same stored procedure you naïvely created for your first attempt.  I’ve seen scenarios where a database has stored procedures with the suffixes GetReport, GetReport2, GetReport3, GetReport4, etc.  Yuck.

Even if you’re using Entity Framework, LINQ to SQL, or some other data layer framework that enables more free-form access to the database, you can’t always ensure that all report queries will result in good execution costs and actually be performant.

This is why it can sometimes be a good idea to perform a very basic database query (via stored procedure if necessary) to get a base set of data, and then perform more conditional operations on it in memory with LINQ.  It’s a pain to do a “Name Contains” filter in a stored procedure (especially if there are a dozen other options) but with LINQ it’s no big deal.

IEnumerable<DataItem> data = GetBaseData();

if (!String.IsNullOrEmpty(nameFilter))
	data = data.Where(d => d.Name.IndexOf(nameFilter, StringComparison.OrdinalIgnoreCase) >= 0);

This is really great for simple filters, but gets difficult when we want to do more complex grouping and aggregating functions, such as grouping by Hourly/Daily/Weekly/Monthly and/or by other data points.

The remainder of this article will show how this can be done with static code, and then how we can drastically increase the maintainability of this same code by employing the dynamic keyword introduced in C# 4.0.

Read more »

Rolling Counter

Sometimes I need a rolling counter, especially in diagnostics-related scenarios. How many requests have occurred in the last hour? It’s not okay for a counter object to drop out of cache every hour, because the value will be meaningless if I happen to observe it at (Cache Drop + 3 minutes).

A real rolling counter is needed in these situations. The counter must increment, and then at some point those hits must drop off.

But especially in these situations, low impact is the key. A Queue where each item contains a timestamp is too unruly. Too many objects are created and too much cleanup is required. Less is more.

Read more »

Injecting NServiceBus into ASP.NET MVC 3

In the past, integrating NServiceBus into a web application typically meant adding the fluent configuration block to the Global.asax file and then keeping a static reference to the IBus property, because ASP.NET has not been very friendly to dependency injection.

In ASP.NET MVC 3, Microsoft has changed that by adding some nice abstractions around dependency resolution.  With a little effort, this allows us to inject NServiceBus (and its dependency injection container) into the dependency resolution pipeline.

In this example, I will show how to add NServiceBus to an MVC 3 web application so that common NServiceBus types (IBus chief among them) are injected into your controller classes.

Read more »

Apple iOS Push Notifications with NServiceBus

I’m no big fan of Apple’s Push Notifications API, but the fact is that iOS applications are no passing fad.  As developers providing server-side resources to mobile applications, our job is to do so as efficiently as possible.

For .NET developers able to commit financial resources, third party solutions such as Urban Airship can be the answer.  For developers wishing to control their own destiny, the open-source apns-sharp provides Apple push notifications and feedback services in a C# library.

Considering the ridiculous complexity of the Apple Push Notification format, it would behoove any developer to use apns-sharp instead of trying to re-invent the wheel.  NServiceBus together with apns-sharp would offer the reliability and scalability needed to successfully send push notifications for a high-capacity enterprise system.

Unfortunately, until recently apns-sharp and NServiceBus didn’t work and play well together.  I contributed to the apns-sharp to address these shortcomings.  In this article, I will describe how to use these modifications to apns-sharp to send push notifications with NServiceBus.

Read more »

HgRemote: Extending Mercurial to Non-Developers

I hear lots of stories about how designers and developers can’t get along. Personally, I don’t get it. I love our web designers. They do something I can’t. I don’t do pretty. I can design a user interface and make it functional, but that’s not enough. Our designers can turn that around and make it gorgeous. We complement each other, and I try to take care of them whenever I can.

So when we made the leap from self-hosted Subversion to Kiln-hosted Mercurial, I wanted to better integrate our designers in the development process. I wanted them to have the same advantages of version control that I, as a developer, was accustomed to. And I really wanted to put an end to opening directories and finding “default.aspx”, “default – Copy.aspx”, “default – Copy (2).aspx”, etc.

The problem with Mercurial is that it doesn’t work too well for web designers that don’t have a locally hosted webserver on their workstation. When testing a bunch of CSS changes, an “edit, save, commit, push, refresh, check, repeat as necessary” workflow does not work. Designers need to be able to modify files on a shared design webserver where they can preview their changes immediately and then commit when done with a task.

Additionally, the designers only need access to the website directory, and attempting to do a 3-way merge makes them run for the hills. And why wouldn’t it – I don’t even enjoy that.

So like most developers, I figured there must be a way I can fix this problem with software.

Read more »

Implementing an NServiceBus Saga Persister

By default, NServiceBus uses Fluent NHibernate to persist saga data to either a SQL or SQLite database. With its plugin architecture, it’s relatively easy to swap out different parts with your own implementations, but it’s not always completely obvious how to go about it. In this article I will attempt to illustrate an attempt to do just that. It may not be perfectly generalized code that could be included in the NServiceBus codebase, but I believe it gets the job done.

Read more »

Batch or Partition a collection with LINQ

Since Generics were first introduced with .NET 2.0 I have had a utility method I used to batch a list into smaller lists of a fixed size, usually to batch up a bunch of database inserts into manageable groups of 10 or 25 or so.

Today I needed to do the same thing except I will be dealing with a potentially VERY large data set with some fairly complex computations built in. Forcing it all into a list and batching the list means I will have to hold all of that garbage in memory.

So I started looking for a LINQ implementation that would deal with only one batch at a time and keep the memory footprint low. Read more »

Resizing Images from a WordPress.com XML Export

Find out how to preprocess a WordPress export file before import to avoid multi-megapixel images in the post body slowing down page loads.

Read more »

Easier Complex IComparable Implementations

I know a lot of us are using the LINQ OrderBy() method to get our data shuffled in the right order, but on occasion I still do like implementing IComparable, especially when defining the default, intrinsic sort scheme for a particular class.

What I don’t like is implementing IComparable when I want to compare on more than one thing.

I don’t like this kind of code because the engineer in me balks at essentially doing each comparison more than once, first for equality, and then for direction:

public int CompareTo(SomeObject other)
{
	if (this.Prop1 != other.Prop1)
		return this.Prop1.CompareTo(other.Prop1);
	else if (this.Prop2 != other.Prop2)
		return this.Prop2.CompareTo(other.Prop2);
	// .... etc.
	return 0;
}

However, expanding it to get rid of this double evaluation yields this nastiness:

public int CompareTo(SomeObject other)
{
	int cmp = this.Prop1.CompareTo(other.Prop1);
	if (cmp != 0)
	{
		cmp = this.Prop2.CompareTo(other.Prop2);
		if (cmp != 0)
			return cmp;
	}
	return 0;
}

Yuck. I decided I needed a way to have some of the elegance of LINQ in an IComparable shell, and here it is:

public class ComplexCompare
{
	private int value;

	private ComplexCompare()
	{
	}

	public static ComplexCompare By<T>(T a, T b)
	{
		return By(a, b, true);
	}

	public static ComplexCompare By<T>(T a, T b, bool ascending)
	{
		ComplexCompare cc = new ComplexCompare();
		if (ascending)
			cc.value = Comparer<T>.Default.Compare(a, b);
		else
			cc.value = Comparer<T>.Default.Compare(b, a);
		return cc;
	}

	public static ComplexCompare By<T>(T a, T b, IComparer<T> comparer)
	{
		ComplexCompare cc = new ComplexCompare();
		cc.value = comparer.Compare(a, b);
		return cc;
	}

	public ComplexCompare ThenBy<T>(T a, T b)
	{
		return ThenBy(a, b, true);
	}

	public ComplexCompare ThenBy<T>(T a, T b, bool ascending)
	{
		// Only compare more specific items if the preceding items have been equal
		if (value == 0)
		{
			if (ascending)
				this.value = Comparer<T>.Default.Compare(a, b);
			else
				this.value = Comparer<T>.Default.Compare(b, a);
		}
		return this;
	}

	public int End()
	{
		return value;
	}
}

ComplexCompare can be used like this:

public int CompareTo(SomeObject other)
{
	return ComplexCompare.By(this.Prop1, other.Prop1) // ascending by default
		.ThenBy(this.Prop2, other.Prop2, false) // but easy to change to descending
		.ThenBy(this.Prop3, other.Prop3) // each call can compare a completely different type
		.End(); // stops the fun and returns the int value
}

Now a hardcore computer science person would tell me that all this extra abstraction adds overhead, and in a really tight loop with millions of rows, using this scheme to compare items would surely be catastrophic! However, I’m not usually one to give in to Micro-Optimization Theater; I’m usually comparing 30-40 items, not millions. My primary concern is that code I write can be instantly understood when viewed by one of my peer developers, and I think ComplexCompare (although I’m not in love with the name) will help me to do that.