Tag Archives: LINQ

Flexible Reporting with LINQ and C# 4.0 dynamic keyword

It’s commonly very difficult to question business people about reporting requirements.  It’s not really their fault either – they just can’t know exactly what they want until they’re trying to answer a question and can’t easily do it with the reports you’ve given them.

This is why it’s good to make reports as flexible and updateable as possible, but with as little developer required to update the reports as possible.

If you’re operating in an environment where all database access must be via stored procedures, this is a really big problem.  It’s really unlikely that the changes requested by business can be implemented with the same stored procedure you naïvely created for your first attempt.  I’ve seen scenarios where a database has stored procedures with the suffixes GetReport, GetReport2, GetReport3, GetReport4, etc.  Yuck.

Even if you’re using Entity Framework, LINQ to SQL, or some other data layer framework that enables more free-form access to the database, you can’t always ensure that all report queries will result in good execution costs and actually be performant.

This is why it can sometimes be a good idea to perform a very basic database query (via stored procedure if necessary) to get a base set of data, and then perform more conditional operations on it in memory with LINQ.  It’s a pain to do a “Name Contains” filter in a stored procedure (especially if there are a dozen other options) but with LINQ it’s no big deal.

IEnumerable<DataItem> data = GetBaseData();

if (!String.IsNullOrEmpty(nameFilter))
	data = data.Where(d => d.Name.IndexOf(nameFilter, StringComparison.OrdinalIgnoreCase) >= 0);

This is really great for simple filters, but gets difficult when we want to do more complex grouping and aggregating functions, such as grouping by Hourly/Daily/Weekly/Monthly and/or by other data points.

The remainder of this article will show how this can be done with static code, and then how we can drastically increase the maintainability of this same code by employing the dynamic keyword introduced in C# 4.0.

Read more »

Rolling Counter

Sometimes I need a rolling counter, especially in diagnostics-related scenarios. How many requests have occurred in the last hour? It’s not okay for a counter object to drop out of cache every hour, because the value will be meaningless if I happen to observe it at (Cache Drop + 3 minutes).

A real rolling counter is needed in these situations. The counter must increment, and then at some point those hits must drop off.

But especially in these situations, low impact is the key. A Queue where each item contains a timestamp is too unruly. Too many objects are created and too much cleanup is required. Less is more.

Read more »

Batch or Partition a collection with LINQ

Since Generics were first introduced with .NET 2.0 I have had a utility method I used to batch a list into smaller lists of a fixed size, usually to batch up a bunch of database inserts into manageable groups of 10 or 25 or so.

Today I needed to do the same thing except I will be dealing with a potentially VERY large data set with some fairly complex computations built in. Forcing it all into a list and batching the list means I will have to hold all of that garbage in memory.

So I started looking for a LINQ implementation that would deal with only one batch at a time and keep the memory footprint low. Read more »