Monthly Archives: July 2011

System.Security.Cryptography and Thread Safety

Posted by David Boike on July 26, 2011 1 comments

Are you experiencing either of these exceptions?

System.Security.Cryptography.CryptographicException
Padding is invalid and cannot be removed.

System.IndexOutOfRangeException
Index was outside the bounds of the array.

System.IndexOutOfRangeException
Probable I/O race condition detected while copying memory. The I/O package is not thread safe by default. In multithreaded applications, a stream must be accessed in a thread-safe way, such as a thread-safe wrapper returned by TextReader’s or TextWriter’s Synchronized methods. This also applies to classes like StreamWriter and StreamReader.

Well at least the last one is descriptive, but that is the LEAST likely to occur.

If you’re seeing any of these exceptions, there’s a good chance you’ve run afoul of a secret of the System.Cryptography namespace.  Almost nothing is thread safe.

It’s an easy error to make. We know encryption is processor intensive, and it seems like it would be smart to incur the costs of setting up ICryptoTransform objects for encryptors and decryptors once and then store them in a static variable. Any state they might share would seem to be reference data like keys and salt and init vectors, so as long as we use a new CryptoStream for each operation, what could go wrong?

Well, lots.

Internally, the implementations of ICryptoTransform (and I assume other objects) use objects from the System.IO namespace like buffers and streams that we would never think of sharing between threads, but it’s hard to know that from a simple call to ICryptoTransform.TransformBlock().

So, if you run into any of the exceptions above, try either creating your System.Cryptography objects each time you need them, or mark them with the ThreadStaticAttribute.  Remember that with [ThreadStatic], a static initializer will not execute for each thread, so check it for null before you use it, then initialize if null.

Stuck between Netflix and a hard place

Posted by David Boike on July 13, 2011 3 comments

There are two types of people in the world right now: those who are angry at Netflix and those who don’t have Netflix.

Like everyone else, I received the email yesterday notifying me that as of September 1, 2011, my $10 Netflix plan that includes 1 DVD at a time and online unlimited streaming will be discontinued. Instead, they offer separate plans for DVDs and for streaming.  1 DVD at a time will now cost $8, and unlimited streaming will now also cost $8.  There is no discount for bundling, so if I want to continue the same level of service, it will now cost me $16 per month.

It’s not the money that bothers me. Prices were bound to go up.  Maybe this is a pretty severe jump all at once, but it’s not completely unexpected.

What bothers me is the false choice it represents. If money does indeed talk (and I believe it does) then Netflix is asking me to choose from these options:

  1. I like getting DVDs from you, but I don’t care for your streaming service. Please take my money and keep the DVDs coming.
  2. I love your streaming service, but DVDs in the mail is so 2003. Please take my money and let me stream to my heart’s content, but don’t make me walk out to the mailbox.
  3. I like DVDs and I also like streaming, and I’m willing to pay more money for both.
  4. Netflix, you suck. Cancel my subscription.

I don’t believe that any of these four options correctly captures my real intent:

I would be willing to pay $16 per month, maybe even more, just for the streaming service, provided that the streaming selection didn’t suck.

Read more »

Flexible Reporting with LINQ and C# 4.0 dynamic keyword

Posted by David Boike on July 10, 2011 None comments

It’s commonly very difficult to question business people about reporting requirements.  It’s not really their fault either – they just can’t know exactly what they want until they’re trying to answer a question and can’t easily do it with the reports you’ve given them.

This is why it’s good to make reports as flexible and updateable as possible, but with as little developer required to update the reports as possible.

If you’re operating in an environment where all database access must be via stored procedures, this is a really big problem.  It’s really unlikely that the changes requested by business can be implemented with the same stored procedure you naïvely created for your first attempt.  I’ve seen scenarios where a database has stored procedures with the suffixes GetReport, GetReport2, GetReport3, GetReport4, etc.  Yuck.

Even if you’re using Entity Framework, LINQ to SQL, or some other data layer framework that enables more free-form access to the database, you can’t always ensure that all report queries will result in good execution costs and actually be performant.

This is why it can sometimes be a good idea to perform a very basic database query (via stored procedure if necessary) to get a base set of data, and then perform more conditional operations on it in memory with LINQ.  It’s a pain to do a “Name Contains” filter in a stored procedure (especially if there are a dozen other options) but with LINQ it’s no big deal.

IEnumerable<DataItem> data = GetBaseData();

if (!String.IsNullOrEmpty(nameFilter))
	data = data.Where(d => d.Name.IndexOf(nameFilter, StringComparison.OrdinalIgnoreCase) >= 0);

This is really great for simple filters, but gets difficult when we want to do more complex grouping and aggregating functions, such as grouping by Hourly/Daily/Weekly/Monthly and/or by other data points.

The remainder of this article will show how this can be done with static code, and then how we can drastically increase the maintainability of this same code by employing the dynamic keyword introduced in C# 4.0.

Read more »

Backing up Hudson, with Hudson

Posted by David Boike on July 9, 2011 None comments

At work we use Hudson Continuous Integration for our build servers because, among other reasons:

  • It’s FREE!
  • It runs on Windows (for our C# builds) and on Mac/Linux (for our iOS/Android builds).
  • It has a web-based GUI that is MUCH easier to use than the XML-driven config used by CruiseControl.NET, which we used before switching to Hudson.
  • It has a rich system of plugins for adding functionality.
  • Did I mention it’s FREE?

The one nice thing about CruiseControl.NET was that because it had one complex XML configuration file, I would only edit that file in source control so that I could back out my changes if I screwed it up. Now I need a way to back up the Hudson configuration files so that if one of my build servers goes up in flames, I can get my team back in business quickly.

A good backup solution needs to be automatic and offsite, and due to the magic of distributed version control and the inherent job execution nature of Hudson, we can back up Hudson with Hudson. If this isn’t the ultimate in universe folding in on itself awesome, I don’t know what is.

Read more »

Adventures in Screen Scraping with YQL

Posted by David Boike on July 5, 2011 3 comments

When coding for work, everything of course has to be done the Right Way®. This isn’t always super exciting, so it is sometimes liberating to cut loose and work on a side project that mashes together a whole bunch of technologies without worrying too much about stability, reliability, scalability, or even if it will continue to run tomorrow. These R&D projects will never have even a single line of code directly pushed into even a development repository, but more often than not I find that I take concepts learned and tested during these coding sessions and apply them in some later project. Even if the entire project is thrown away in relatively short order, some concept of value survives for the long haul.

Plus, it’s just fun.

Recently my wife and I got the very exciting (and scary!) news that we were pregnant with our first child. The little guy or girl’s arrival is still over 5 months away, but already we’re wrestling with tons of difficult questions, and one particularly overwhelming one is “How are we going to decide where to send our child for day care?”

We live in the great state of Minnesota where the Department of Human Services maintains a searchable Licensing Info Lookup website for all sorts of things, including (but not limited to) family child care. Anyone with a child care license can be found here, along with address, phone number, if they can accept newborn infants and how many, etc.

Just one problem. We live on the border of two big suburbs, so you do a search for both cities and together you get over 150 results, and no map.

This is where my inner geek starts to get excited. I’ve got a copy of Visual Studio. I can fix this problem. Let’s do it.

Read more »