Amazon S3 Bucket Management with C#: Part 3 – Listing files in a Bucket

Before getting started

Skill Level: Beginner

Assumptions:

  1. You already gone through Part 1 – Creating a Bucket and Part 2 – Deleting a Bucket

Additional information: I sometimes cover small sub-topics in a post. Along with AWS, you will also be exposed to:

  • async, await, Task
  • Rhyous.SimpleArgs

Step 1 – Add a ListFiles method to BucketManager.cs

  1. Edit file called BucketManager.cs.
  2. Enter this new method:
            public static async Task ListFiles(AmazonS3Client client, string bucketName)
            {
                var listResponse = await client.ListObjectsV2Async(new ListObjectsV2Request { BucketName = bucketName });
                if (listResponse.S3Objects.Count > 0)
                {
                    Console.WriteLine($"Listing items in S3 bucket: {bucketName}");
                    listResponse.S3Objects.ForEach(o => Console.WriteLine(o.Key));
                }
            }
    

    Note: I noticed there was a ListObjectsAsync and a ListObjectsV2Async. I assumed the one with V2 is newer and should be used for new code. The documentation for ListObjectsV2Async confirmed this.

Step 2 – Update the Action Argument

We now need to make this method a valid action for the Action Argument.

  1. Edit the ArgsHandler.cs file to define an Action argument.
                        ...
                        AllowedValues = new ObservableCollection<string>
                        {
                            "CreateBucket",
                            "DeleteBucket",
                            "ListFiles"
                        },
                        ...
    

Notice: We didn’t have a step 3. We wrote some S.O.L.I.D. code in Part 1 and Part 2, which made it really easy for us to implement this method.

Homework: I also read in the documentation that only 1000 files will be listed when a call to ListObjectsV2Async is made. What if you have more than 1000 files, how would you list them all?

Go to: Part 4 – Uploading a file to a Bucket

Return to: Managing Amazon AWS with C#


Amazon S3 Bucket Management with C#: Part 2 – Deleting a Bucket

Before getting started

Skill Level: Beginner

Assumptions:

  1. You already gone through Part 1 – Creating a Bucket where you have already:
    1. Created a new Console Application Project
    2. Added NuGet Package
    3. Created a BucketManager.cs
    4. Coded Program.cs.
    5. Added command line arguments

Additional information: I sometimes cover small sub-topics in a post. Along with AWS, you will also be exposed to:

  • async, await, Task
  • Reflection
  • Rhyous.SimpleArgs
  • Single Responsibility Principal (S of S.O.L.I.D.) or Don’t Repeat Yourself (DRY)
  • Dependency Injection – Method Injection (D of S.O.L.I.D.)

Step 1 – Add a DeleteBucket method to BucketManager.cs

  1. Edit file called BucketManager.cs.
  2. Enter this new method:
            public static async Task DeleteBucket(string bucketName)
            {
                var region = RegionEndpoint.GetBySystemName(ConfigurationManager.AppSettings["AWSRegion"]);
                var client = new AmazonS3Client(region);
                await AmazonS3Util.DeleteS3BucketWithObjectsAsync(client, bucketName);
                Console.WriteLine($"Deleted S3 bucket: {bucketName}");
            }
    

    Notice that there is more involved with deleting a bucket than creating a bucket. A bucket may not be empty. It could have files in it already. Because of this we call a helper method, DeleteS3BucketWithObjectsAsync, that deletes a bucket even if it has objects, i.e. files, in it.

Step 2 – Configure an Argument for the Action to call

Since BucketManager can now can both Create and Delete a bucket, we need an argument to specify what action we would like to call.

  1. Edit the ArgsHandler.cs file to define an Action argument.
    SimpleArgs allows for arguments to be declarative and provides most the features you would want in command line arguments without having to write those features for every new application.

                    new Argument
                    {
                        Name = "Action",
                        ShortName = "a",
                        Description = "The action to run.",
                        Example = "{name}=default",
                        DefaultValue = "Default",
                        AllowedValues = new ObservableCollection&lt;string&gt;
                        {
                            "CreateBucket",
                            "DeleteBucket"
                        },
                        IsRequired = true,
                        Action = (value) =&gt;
                        {
                            Console.WriteLine(value);
                        }
                    }
    

    Notice: We don’t have to validate that the correct method was passed in as a variable because SimpleArgs will do this for us by simply declaring them as AllowedValues. When AllowedValues is declared, only those values are allowed. Any other value will result in the application stopping and outputting the list of valid arguments.

Step 3 – Edit the Program.cs

We are now going to use reflection to call the appropriate function based on our action parameter.

    1. Edit the OnArgumentsHandled method of Program.cs.
              internal static void OnArgumentsHandled()
              {
                  var action = Args.Value("Action");
                  var bucketName = Args.Value("Bucket");
                  var flags = BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Static | BindingFlags.FlattenHierarchy;
                  MethodInfo mi = typeof(BucketManager).GetMethod(action, flags);
      
                  var task = mi.Invoke(null, new[] { bucketName }) as Task;
                  task.Wait();
              }
      

Step 4 – Make the code S.O.L.I.D.

We have broken the Single Responsibility Principal (S in S.O.L.I.D.) or Don’t Repeat Yourself (DRY) rule. Let’s notice it and fix it.

    1. Notice in BucketManager.cs that both methods are breaking two rules:
      • Don’t Repeat Yourself or DRY: We have two methods repeating the same two lines of code.
                    var region = RegionEndpoint.GetBySystemName(ConfigurationManager.AppSettings["AWSRegion"]);
                    var client = new AmazonS3Client(region);											
        
      • Single Responsibility Principal: Each method has one repsponsibility. CreateBucket should only create a bucket on the Amazon S3 server. DeletBucket should only delete a bucket. Currently, both methods are having to instantiate a client and figure out the region. That isn’t the responsibility of these methods.
    2. Let’s solve this with Method Injection. Method Injection is a form of Dependency Injection (D in S.O.L.I.D.). We will pass the client into the method.
      Note: I am using method injection because all the methods are static, so Constructor Injection is not an option. Property Injection is an option. A Lazy-injectable Property would also be a very good option here.
    3. Alter the methods to take in an AmazonS3Client object.
      using Amazon.S3;
      using Amazon.S3.Util;
      using System;
      using System.Threading.Tasks;
      
      namespace Rhyous.AmazonS3BucketManager
      {
          public class BucketManager
          {
              public static async Task CreateBucket(AmazonS3Client client, string bucketName)
              {
                  await client.PutBucketAsync(bucketName);
                  Console.WriteLine($"Created S3 bucket: {bucketName}");
              }
      
              public static async Task DeleteBucket(AmazonS3Client client, string bucketName)
              {
                  await AmazonS3Util.DeleteS3BucketWithObjectsAsync(client, bucketName);
                  Console.WriteLine($"Deleted S3 bucket: {bucketName}");
              }
          }
      }
      
    4. Update the Program.cs to instantiate the client and pass it into the methods.
              internal static void OnArgumentsHandled()
              {
                  var action = Args.Value("Action");
                  var bucketName = Args.Value("Bucket");
      
                  var flags = BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Static | BindingFlags.FlattenHierarchy;
                  MethodInfo mi = typeof(BucketManager).GetMethod(action, flags);
      
                  var region = RegionEndpoint.GetBySystemName(ConfigurationManager.AppSettings["AWSRegion"]);
                  var client = new AmazonS3Client(region);
      
                  var task = mi.Invoke(null, new object[] { client, bucketName }) as Task;
                  task.Wait();
              }
      

      Notice we now only create a client one time.

      Note: The OnArgumentsHandled in our Program.cs is now doing four things. As you can see above, it has each thing it does in a pair of lines, with each pair of lines separated by a double space. This is where we want to be careful to not overdo it when following design patterns. We only have eight lines of code. Program.cs is supposed to be a program. It is fine for now. Let’s not change it. Notice it. Consider changing it. This time we decided to leave it, but we can keep it in mind. If more lines are added, then that method probably should be changed.

Go to: Part 3 – Listing files in a Bucket

Return to: Managing Amazon AWS with C#


Amazon S3 Bucket Management with C#: Part 1 – Creating a Bucket

Before getting started

Skill Level: Beginner

Assumptions:

  1. You already have Visual Studio installed.
  2. You are familiar with creating projects in Visual Studio.
  3. We assume you have already gone to AWS and registered with them. If you haven’t done that already, stop and go there now. Amazon has a free tier and you can create an account here: https://aws.amazon.com/free

Additional Information: I sometimes cover small sub-topics in a post. Along with AWS, you will also be exposed to:

  • .NET Core 2.0 – If you use .NET Framework, the steps will be slightly different, but as this is a beginner level tutorial, it should be simple.
  • async, await, Task
  • Rhyous.SimpleArgs

Step 1 – Create the project

  1. Open Visual Studio.
  2. Go to File | New Project.
  3. Choose Console Application.
    Give it any name you want.
    I am going to call my project Rhyous.AmazonS3BucketManager.

Step 2 – Add NuGet Packages

  1. Right-click on your project and choose Management NuGet Packages.
  2. Search for AWSSDK.S3.
  3. Install the NuGet package and all the dependencies.
  4. Search for System.Configuration.ConfigurationManager.
  5. Install it.

Step 3 – Create a BucketManager.cs file

  1. Create a new file called BucketManager.cs.
  2. Enter this code:
    using Amazon;
    using Amazon.S3;
    using System;
    using System.Configuration;
    using System.Threading.Tasks;
    
    namespace Rhyous.AmazonS3BucketManager
    {
        public class BucketManager
        {
            public static async Task CreateBucket(string bucketName)
            {
                var region = RegionEndpoint.GetBySystemName(ConfigurationManager.AppSettings["AWSRegion"]);
                var client = new AmazonS3Client(region);
                await client.PutBucketAsync(bucketName);
                Console.WriteLine($"Created S3 bucket: {bucketName}");
            }
        }
    }
    

Step 4 – Edit the Program.cs

  1. Add the following to Program.cs.
            static void Main()
            {
                var task = BucketManager.CreateBucket("my.new.bucket");
                task.Wait();
            }
    

Step 5 – Create/Edit the App.config

  1. If there isn’t an app.config in your project, create one.
  2. Right-click on your project and choose Add | New Item.
  3. Search for Application Configuration File.
    Make sure it is name app.config.
  4. Add an appSetting for your AWS profile name.
    Add an additional appSetting for your chosen AWS region.

    <?xml version="1.0" encoding="utf-8" ?>
    <configuration>
      <appSettings>
        <add key="AWSProfileName" value="yourprofilename"/>
        <add key="AWSRegion" value="us-west-2" />
      </appSettings>
    </configuration>
    

Step 6 – Configure an Argument for the bucket name.

We are going to be adding to this program in subsequent posts. For this reason, we are going to use Rhyous.SimpleArgs library for our command line arguments as it provides ready-made command line argument features.

  1. Install another NuGet Package.
  2. Right-click on your project and choose Management NuGet Packages.
  3. Search for Rhyous.SimpleArgs
  4. Install it.
  5. Create an ArgsHandler.cs file to define the arguments:
    Note: If you used a .NET core project you have to create this file. If you created a .NET Framework file, this file should have been created for you and you have but to edit it.

    using Rhyous.SimpleArgs;
    using System;
    using System.Collections.Generic;
    using System.Text.RegularExpressions;
    
    namespace Rhyous.AmazonS3BucketManager
    {
        public class ArgsHandler : ArgsHandlerBase
        {
            public override void InitializeArguments(IArgsManager argsManager)
            {
                Arguments.AddRange(new List<Argument>
                {
                    new Argument
                    {
                        Name = "Bucket",
                        ShortName = "b",
                        Description = "The bucket name to create. No uppercase or underscores allowed.",
                        Example = "{name}=my.first.bucket",
                        DefaultValue = "my.first.bucket",
                        IsRequired = true,
                        CustomValidation = (value) => 
                        {
                            return Regex.IsMatch(value, "^[a-z0-9.]+$");
                        },
                        Action = (value) =>
                        {
                            Console.WriteLine(value);
                        }
                    }
                });
            }
    
            public override void HandleArgs(IReadArgs inArgsHandler)
            {
                base.HandleArgs(inArgsHandler);
                Program.OnArgumentsHandled();
            }
        }
    }
    
  6. Update Program.cs as follows:
    using Rhyous.SimpleArgs;
    using System;
    
    namespace Rhyous.AmazonS3BucketManager
    {
        class Program
        {
            static void Main(string[] args)
            {
                new ArgsManager<ArgsHandler>().Start(args);
            }
    
            internal static void OnArgumentsHandled()
            {
                var bucketName = Args.Value("Bucket");
                var task = BucketManager.CreateBucket(bucketName);
                task.Wait();
            }
        }
    }
    

Now for fun, you can delete the app.config and change them to parameters.

Next:

  • Deleting a Bucket
  • Return to: Managing Amazon AWS with C#


    I just hit 4000 points on StackOverflow!

    I just hit 4000 points on StackOverflow!


    profile for Rhyous at Stack Overflow, Q&A for professional and enthusiast programmers


    Understanding async and await, Task.WaitAll, Task.Run, and parallelism

    Ok. You are probably new to async and await or maybe you aren’t new but you’ve never deep dived into it. You may not understand some simple truths:

    1. aync/await does NOT give you parallelism for free.
    2. Tasks are not necessary parallel. They can be if you code them to be.
    3. The recommendation “You should always use await” is not really true when you want parallelism, but is still sort of true.
    4. Task.WhenAll is both parallel and async.
    5. Task.WaitAll only parallel.

    Here is a sample project that will help you learn.

    There is more to learn in the comments.
    There is more to learn by running this.

    Note: I used Visual Studio 2017 and compiled with .Net 7.1, which required that I go to the project properties | Build | Advanced | Language Version and set the language to C# 7.1 or C# latest minor version.

    using System;
    using System.Diagnostics;
    using System.Threading.Tasks;
    
    namespace MultipleAwaitsExample
    {
        class Program
        {
            static async Task Main(string[] args)
            {
                Console.WriteLine("Running with await");
                await RunTasksAwait();
                Console.WriteLine("Running with Task.WaitAll()");
                await RunTasksWaitAll();
                Console.WriteLine("Running with Task.WhenAll()");
                await RunTasksWhenAll();
                Console.WriteLine("Running with Task.Run()");
                await RunTasksWithTaskRun();
                Console.WriteLine("Running with Parallel");
                RunTasksWithParallel();
            }
    
            /// <summary>
            /// Pros: It works
            /// Cons: The tasks are NOT run in parallel.
            ///       Code after the await is not run while the await is awaited
            ///       **If you want parallelism, this isn't even an option.**
            ///       Slowest. Because of no parallelism.
            /// </summary>
            public static async Task RunTasksAwait()
            {
                var group = "await";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                await MyTaskAsync(1, 500, group);
                await MyTaskAsync(2, 300, group);
                await MyTaskAsync(3, 100, group);
                Console.WriteLine("Code immediately after tasks.");
                watcher.Stop();
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            /// <summary>
            /// WaitAll behaves quite differently from WhenAll
            /// Pros: It works
            ///       The tasks run in parallel
            /// Cons: It isn't clear whether the code is parallel here, but it is.
            ///       It isn't clear whether the code  is async here, but it is NOT.
            ///       There is a Visual Studio usage warning. You can remove async to get rid of it because it isn't an Async method.
            ///       The return value is wrapped the Result property of the task
            ///       Breaks Aync end-to-end
            ///       Note: I can't foresee usecase where WaitAll would be preferred over WhenAll.
            /// </summary>
            public static async Task RunTasksWaitAll()
            {
                var group = "WaitAll";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                var task1 = MyTaskAsync(1, 500, group);
                var task2 = MyTaskAsync(2, 300, group);
                var task3 = MyTaskAsync(3, 100, group);
                Console.WriteLine("Code immediately after tasks.");
                Task.WaitAll(task1, task2, task3);
                watcher.Stop();
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            /// <summary>
            /// WhenAll gives you the best of all worlds. The code is both parallel and async.
            /// Pros: It works
            ///       The tasks run in parallel
            ///       Code after the tasks run while the task is running
            ///       Doesn't break end-to-end async
            /// Cons: It isn't clear you are doing parallelism here, but you are.
            ///       There is a Visual Studio usage warning
            ///       The return value is wrapped the Result property of the task
            /// </summary>
            public static async Task RunTasksWhenAll()
            {
                var group = "WaitAll";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                var task1 = MyTaskAsync(1, 500, group);  // You can't use await if you want parallelism
                var task2 = MyTaskAsync(2, 300, group);
                var task3 = MyTaskAsync(3, 100, group);
                Console.WriteLine("Code immediately after tasks.");
                await Task.WhenAll(task1, task2, task3); // But now you are calling await, so you are sort of still awaiting
                watcher.Stop();
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            /// <summary>
            /// Pros: It works
            ///       The tasks run in parrallel
            ///       Code can run immediately after the tasks but before the tasks complete
            ///       Allows for running non-async code asynchonously
            /// Cons: It isn't clear whether the code is doing parallelism here. It isn't.
            ///       The lambda syntax affects readability
            ///       Breaks Aync end-to-end
            /// </summary>
            public static async Task RunTasksWithTaskRun()
            {
                var group = "Task.Run()";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                await Task.Run(() => MyTask(1, 500, group));
                await Task.Run(() => MyTask(2, 300, group));
                await Task.Run(() => MyTask(3, 100, group));
                Console.WriteLine("Code immediately after tasks.");
                watcher.Stop();
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            /// <summary>
            /// Pros: It works
            ///       It is clear in the code you want to run these tasks in parallel.
            ///       Code can run immediately after the tasks but before the tasks complete
            ///       Fastest
            /// Cons: There is no async or await.
            ///       Breaks Aync end-to-end. You can workaround this by wrapping Parallel.Invoke in a Task.Run method. See commented code.
            /// </summary>
            public /* async */ static void RunTasksWithParallel()
            {
                var group = "Parallel";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                //await Task.Run(() => 
                Parallel.Invoke(
                    () => MyTask(1, 500, group),
                    () => MyTask(2, 300, group),
                    () => MyTask(3, 100, group),
                    () => Console.WriteLine("Code immediately after tasks.")
                );
                //);
                
                watcher.Stop();
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            public static async Task MyTaskAsync(int i, int milliseconds, string group)
            {
                await Task.Delay(milliseconds);
                Console.WriteLine($"{group}: {i}");
            }
    
            public static void MyTask(int i, int milliseconds, string group)
            {
                var task = Task.Delay(milliseconds);
                task.Wait();
                Console.WriteLine($"{group}: {i}");
            }
        }
    }
    

    And here is the same example but this time with some return values.

    using System;
    using System.Diagnostics;
    using System.Threading.Tasks;
    
    namespace MultipleAwaitsExample
    {
        class Program1
        {
            static async Task Main(string[] args)
            {
                Console.WriteLine("Running with await");
                await RunTasksAwait();
                Console.WriteLine("Running with Task.WaitAll()");
                await RunTasksWaitAll();
                Console.WriteLine("Running with Task.WhenAll()");
                await RunTasksWhenAll();
                Console.WriteLine("Running with Task.Run()");
                await RunTasksWithTaskRun();
                Console.WriteLine("Running with Parallel");
                RunTasksWithParallel();
            }
    
            /// <summary>
            /// Pros: It works
            /// Cons: The tasks are NOT run in parallel.
            ///       Code after the await is not run while the await is awaited
            ///       **If you want parallelism, this isn't even an option.**
            ///       Slowest. Because of no parallelism.
            /// </summary>
            public static async Task RunTasksAwait()
            {
                var group = "await";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                // You just asign the return variables as normal.
                int result1 = await MyTaskAsync(1, 500, group);
                int result2 = await MyTaskAsync(2, 300, group);
                int result3 = await MyTaskAsync(3, 100, group);
                Console.WriteLine("Code immediately after tasks.");
                watcher.Stop();
                // You now have access to the return objects directly.
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            /// <summary>
            /// WaitAll behaves quite differently from WhenAll
            /// Pros: It works
            ///       The tasks run in parallel
            /// Cons: It isn't clear whether the code is parallel here, but it is.
            ///       It isn't clear whether the code  is async here, but it is NOT.
            ///       There is a Visual Studio usage warning. You can remove async to get rid of it because it isn't an Async method.
            ///       The return value is wrapped the Result property of the task
            ///       Breaks Aync end-to-end
            ///       Note: I can't foresee usecase where WaitAll would be preferred over WhenAll.
            /// </summary>
            public static async Task RunTasksWaitAll()
            {
                var group = "WaitAll";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                var task1 = MyTaskAsync(1, 500, group);
                var task2 = MyTaskAsync(2, 300, group);
                var task3 = MyTaskAsync(3, 100, group);
                Console.WriteLine("Code immediately after tasks.");
                Task.WaitAll(task1, task2, task3);
                watcher.Stop();
                // You now have access to the return object using the Result property.
                int result1 = task1.Result;
                int result2 = task2.Result;
                int result3 = task3.Result;
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            /// <summary>
            /// WhenAll gives you the best of all worlds. The code is both parallel and async.
            /// Pros: It works
            ///       The tasks run in parallel
            ///       Code after the tasks run while the task is running
            ///       Doesn't break end-to-end async
            /// Cons: It isn't clear you are doing parallelism here, but you are.
            ///       There is a Visual Studio usage warning
            ///       The return value is wrapped the Result property of the task
            /// </summary>
            public static async Task RunTasksWhenAll()
            {
                var group = "WaitAll";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                var task1 = MyTaskAsync(1, 500, group);  // You can't use await if you want parallelism
                var task2 = MyTaskAsync(2, 300, group);
                var task3 = MyTaskAsync(3, 100, group);
                Console.WriteLine("Code immediately after tasks.");
                await Task.WhenAll(task1, task2, task3); // But now you are calling await, so you are sort of still awaiting
                watcher.Stop();
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            /// <summary>
            /// Pros: It works
            ///       The tasks run in parrallel
            ///       Code can run immediately after the tasks but before the tasks complete
            ///       Allows for running non-async code asynchonously
            /// Cons: It isn't clear whether the code is doing parallelism here. It isn't.
            ///       The lambda syntax affects readability
            ///       Breaks Aync end-to-end
            /// </summary>
            public static async Task RunTasksWithTaskRun()
            {
                var group = "Task.Run()";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                int result1 = await Task.Run(() => MyTask(1, 500, group));
                int result2 = await Task.Run(() => MyTask(2, 300, group));
                int result3 = await Task.Run(() => MyTask(3, 100, group));
                Console.WriteLine("Code immediately after tasks.");
                watcher.Stop();
                // You now have access to the return objects directly.
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            /// <summary>
            /// Pros: It works
            ///       It is clear in the code you want to run these tasks in parallel.
            ///       Code can run immediately after the tasks but before the tasks complete
            ///       Fastest
            /// Cons: There is no async or await.
            ///       Breaks Aync end-to-end. You can workaround this by wrapping Parallel.Invoke in a Task.Run method. See commented code.
            /// </summary>
            public /* async */ static void RunTasksWithParallel()
            {
                var group = "Parallel";
                Stopwatch watcher = new Stopwatch();
                watcher.Start();
                // You have to declare your return objects before hand.
                //await Task.Run(() => 
                int result1, result2, result3;
                Parallel.Invoke(
                    () => result1 = MyTask(1, 500, group),
                    () => result2 = MyTask(2, 300, group),
                    () => result3 = MyTask(3, 100, group),
                    () => Console.WriteLine("Code immediately after tasks.")
                );
                //);
                // You now have access to the return objects directly.
                watcher.Stop();
                Console.WriteLine($"{group} runtime: {watcher.ElapsedMilliseconds}");
            }
    
            public static async Task<int> MyTaskAsync(int i, int milliseconds, string group)
            {
                await Task.Delay(milliseconds);
                Console.WriteLine($"{group}: {i}");
                return i;
            }
    
            public static int MyTask(int i, int milliseconds, string group)
            {
                var task = Task.Delay(milliseconds);
                task.Wait();
                Console.WriteLine($"{group}: {i}");
                return i;
            }
        }
    }
    

    Excluding Integration and other tests when running Unit Tests locally in Visual Studio

    Often, a solution may have both Unit Tests and Integration Tests. Unit Tests should be highly specific, and should be testing one class object. Integration Test could be vastly more complex and could use Selenium, or require a real database server, etc. Even if a solution doesn’t have Integration Tests, it may have slow Unit Tests. For example, I have a test that takes about 2 minutes to run because it is testing that when creating 10 million random strings, the character distribution is pretty equal.

    I don’t want to either slow Unit Tests or Integration Tests to run every single time I build on my local dev box. I do have an automated build system and builds kick off on check-in. At that time, these slow tests will run every time. But locally, it is just an unnecessary delay.

    Usually I am writing in a specific project and that project has specific tests and I can easily choose to only run tests in my current project. But what if I am updating a library that many projects use. I want to know quickly if anything is broke, so I need to run most of the tests in the entire solution.

    Visual Studio allows for tests to be tagged with a TestCategoryAttribute.

    You can mark different tests with different names: [TestCategory(“Slow”)], [TestCategory(“Integration”)], [TestCategory(“Performance”)], [TestCategory(“Selenium”)]

    Example:

    [TestMethod]
    [TestCategory("Slow")]
    public void TestDistributionInTenMillionCharacters()
    {
        // some code here
    }
    

    All other tests are left without a test category. Now, if you want to run all tests that aren’t slow, you can do this in Visual Studio Test Explorer by grouping tests using the “Traits” selection option.

    Once you have marked all tests with an appropriate TestCategoryAttribute, you can sort by Trait. Now it is simple to click-to-highlight the No Traits group and click Run | Selected Tests.


    Avoiding Dependency Injection’s Constructor Injection Hell by Using the Custom or Default Pattern

    Dependency Injection makes the claim that a concrete implementation of an interface is going to change. The problem is that theory rarely meets reality. In reality, there usually exists one single concrete implementation of most interfaces. Usually there exists one Test implementation. Other than those two concrete interfaces, a second production implementation of an interface almost never exists. When there are multiple concrete implementations of an interface, it is usually in a plugin situation and is handled differently, using plugin loading technology. So we can exclude plugin scenarios, whether UI or other, and focus on day-to-day dependency injection.

    In some projects, 100% of the objects have a single production implementation and a single test implementation. Other projects, you may see 90%/10%, but again, rarely are their multiple.

    In other projects, you will find a second production concrete implementation, but it is replaces the previous implementation, not running alongside it.

    So why are we not coding to the common case?

    Construction Injection

    Constructor Injection assumes a few things:

    1. Your class should not even instantiate without something injected into the constructor.
    2. Your class should not work without something injected in.
    3. Your class should have zero dependencies, except of standard types and simple POCO model classes.

    It sounds great in theory, but in practice, it usually leads to constructor injection hell. If you are wondering what Constructor Injection hell is, have a look at this previous post: Constructor Injection Hell.

    Property Injection Issues

    Property Injection or Method injection are also assuming a few statements are true.

    1. Your class can be instantiated without something injected into the constructor.
    2. Your methods should throw exceptions if a required property has not yet been injected.
    3. Your class should have zero dependencies, except of standard types and simple POCO model classes.

    Method Injection Issues

    Method Injection or Method injection are also assuming a few statements are true.

    1. Your class can be instantiated without something injected into the constructor.
    2. Your methods should throw exceptions if a required property is not injected when the method is called.
    3. Your class should have zero dependencies, except of standard types and simple POCO model classes.

    Dependency Injection Problems

    There are problems caused with Dependency Injection.

    1. Dependency Injection Hell
    2. Your code doesn’t work without an injected dependency
    3. You cannot test your code without an injected dependency.

    Should You Stop Using Dependency Injection

    So now that you’ve been educated on the truth that theory and reality never mix, and most of Dependency Injection is a waste of time, let me tell you: Keep doing it because you still will benefit from Dependency Injection.

    Don’t stop doing it, but instead add to it a simple pattern. The Custom or Default pattern.

    Custom or Default Pattern

    This pattern is so simple that I cannot even find it mentioned in any list of software development patterns. It is so simple and obvious, that nobody talks about it. But let’s talk about it anyway.

    Make everything have a default value/implementation. If a custom implementation is supplied use it, otherwise use the default.

    It is a simple pattern where all the Dependency Injection rules are simplified into two:

    1. Your class should instantiate without something injected into the constructor.
    2. Your class and method should work without something injected in.
    3. Your class should have zero dependencies, except of standard types and simple POCO model classes or default implementations.

    Notice the rules change. And these changes make your life so much easier.

    What adding this pattern says is that it is OK to depend on a default implementation. Yes, please, support Dependency Injection, but don’t forget to provide a default implementation.

    Constructor Injection with the Custom or Default Pattern

    This pattern could be implemented with Constructor Injection.

    public class SomeObjectWithDependencies
    {
        internal IDoSomething _Doer;
        public SomeObjectWithDependencies(IDoSomething doer)
        {
            _Doer = doer ?? new DefaultDoer();
        }
    
        public void DoSomething()
        {
            _Doer.DoSomething();
        }
    }
    

    However, you didn’t solve Constructor Injection hell. You could solve that with a Service Locator. Here is the same pattern with a Service Locator. I’ll only do one Service Locator example, though it could be done a few ways.

    public class SomeObjectWithDependencies
    {
        internal IDoSomething _Doer;
        public SomeObjectWithDependencies(IDoSomething <span class="hiddenGrammarError" pre="">doer)
        {
            doer</span> = doer ?? ServiceLocator.Instance.DefaultDoer;
        }
    
        public void DoSomething()
        {
            _Doer.DoSomething();
        }
    }
    

    Many developers will immediately react and say that both example result in coupled code. SomeObjectWithDependencies becomes coupled to DefaultDoer or ServiceLocator.

    Well, in theory, this is coupling. Your code will never compile, the earth will stop moving, you will rue the day you made this coupling, and your children will be cursed for generations, blah, blah, blah. Again, theory and reality aren’t friends. The code always compiles. The coupling is very light. Remember, it is only coupled to a default. You have now implemented the Custom or Default pattern. With one line of code, you solved a serious problem: With regular Dependency Injection, your code doesn’t work without an injected dependency.

    Guess what, you had coupling anyway. Usually you had to spin up some third library, a controller or starter of some sort, that hosted your dependency injection container, and then you had to couple the two objects of code together and now you have an entire third library just to keep two classes from being friends.

    The above is still very loose coupling. It is only coupling a default. It still allows for a non-default concrete implementation to be injected.

    Property Injection with the Custom or Default Pattern (Preferred)

    This pattern could be implemented with Property Injection. However, this pattern suddenly becomes extremely awesome. It is no longer just a standard property. It becomes a Lazy Injectable Property.

    A Lazy Injectable Property becomes key to solving most of your Constructor Injection hell.
    Since it is very rare that a implementation other than the default is ever used, it is extremely rare that a Dependency Injection container is ever really needed to inject a dependency. You will find yourself doing Dependency Injection with a container.

    This item

    public class SomeObjectWithDependencies
    {
        public IDoSomething Doer
        {
            get { return _Doer ?? (_Doer = new IDoSomething()); }
            set { _Doer = value; }
        } private IDoSomething _Doer;
    
        public void DoSomething()
        {
            _Doer.DoSomething();
        }
    }
    

    Or if you have a lot of such dependencies, you can move them into a DefaultServiceLocator .

    public class DefaultServiceLocator() : 
    {
            public IDoSomething Doer
            {
                get { return _Doer ?? (_Doer = new ConcreteDoSomething()); }
                set { _Doer = value; }
            } private IDoSomething _Doer;
    
            // more lazy injectable properties here
    }
    
    public class SomeObjectWithDependencies(IDoSomething doer)
    {
        public IDefaultServiceLocator ServiceLocator
        {
            get { return _ServiceLocator ?? (_ServiceLocator = new DefaultServiceLocator()); }
            set { _ServiceLocator = value; }
        } private IDoSomething _ServiceLocator;
    
        public void DoSomething()
        {
            ServiceLocator.DefaultDoer.DoSomething();
        }
    }
    

    Eliminating Cylclomatic Complexity by replacing switch/case with a method or a Dictionary<TKey, func<>>

    Cyclomatic Complexity is a measurement of how many paths your code could traverse. Switch/case statements are often immediate Cyclomatic Complexity concerns.

    Cyclomatic Complexity Example

    Imagine the following code:

    public void Foo(int val)
    {
        switch (val)
        {
            case 0:
                // ... code here
                break;
            case 1:
                // ... code here
                break;
            case 2:
                // ... code here
                break;
            case 3:
                // ... code here
                break;
            case 4:
                // ... code here
                break;
            case 5:
                // ... code here
                break;
            case 6:
                // ... code here
                break;
            case 7:
                // ... code here
                break;
        }
    }
    

    In the above code, there are 8 paths. The Cyclomatic Complexity is not small. This makes unit tests difficult. It is complexity that is unnecessary. Unnecessary complexity leads to bugs.

    Replacing a switch/case statement

    Almost invariably, the switch/case statement can be replaced in a way that removes the cyclomatic complexity. There are three replacements methods I am going to dicuss here.

    1. Replace with Methods
    2. Replace with Methods and a dictionary provided parameter.
    3. Replace with Dictionary<TKey, Func<>>

    To know which one to choose, you have analyze the code. In the above example, I left out the code. I just put a place holder for it.

    // ... code here

    After analyzing the code, you should be able to pick one of the following:

    Method

    So you should use a method if you can.

    Example 1
    Imagine the following snippet. This is an easy one. You should pick out the replacement without having to be told what it is.

    public void Foo(int val)
    {
        switch (val)
        {
            case 0:
                Bar.Do(0);
                break;
            case 1:
                Bar.Do(1);
                break;
            case 2:
                Bar.Do(2);
                break;
            case 3:
                Bar.Do(3);
                break;
            // , ... , 
            case 7:
                Bar.Do(7);
                break;
        }
    }
    

    As you can see here, each method is easily following a pattern. We can replace the switch statement with a single method call.

    public void Foo(int val)
    {
        Bar.Do(val);
    }
    

    Look, that one was obvious and it was intended to be obvious. It isn’t always going to be obvious.

    Example 2
    Imagine the following snippet. This is also an easy one, but not quite as easy as above. Hopefully, you pick out the replacement without having to be told what it is.

    public void Foo(int val)
    {
        switch (val)
        {
            case 0:
                Bar.Do0();
                break;
            case 1:
                Bar.Do1();
                break;
            case 2:
                Bar.Do2();
                break;
            case 3:
                Bar.Do3();
                break;
            // , ... , 
            case 7:
                Bar.Do7();
                break;
        }
    }
    

    Notice there is a pattern. We know the method name to call on Bar because we can see the pattern: “Do” + val

    We could easily use reflection to eliminate cyclomatic complexity here.

    Note: While reflection is often deemed slow and a cause of performance issues, in practice, unless looping through large data sets, any performance loss from reflection is not measurable.

    public void Foo(int val)
    {
        typeof(Bar).GetMethod("Do" + val).Invoke();
    }
    

    We traded Cyclomatic Complexity for Reflection and a possible, but unlikely performance issue. If this code is used in a loop for millions of instances in a data set, you might not want to do this.

    Method and a dictionary provided parameter

    Example 1
    Imagine the code is more like this, in which different case statements call different overloaded values.

    public void Foo(int val, ObjA a)
    {
        switch (val)
        {
            case 0:
                Bar.Do(a, 3);
                break;
            case 1:
                Bar.Do(a, 7);
                break;
            case 2:
                Bar.Do(a, 5);
                break;
            case 3:
                Bar.Do(a, 100);
                break;
            case 4:
                Bar.Do(a, 9);
                break;
            case 5:
                Bar.Do(a, 12);
                break;
            case 6:
                Bar.Do(a, -1);
                break;
            case 7:
                Bar.Do(a, int.MaxValue);
                break;
        }
    }
    

    So every case statement is doing something different. However, notice that what it does different is a static int. We can create a static parameter dictionary of type Dictionaryint, int>.

    internal Dictionary<int, int> ParamMap = new Dictionary<int, int> { {0,3}, {1,7}, {2,5}, {3,100}, {4,9}, {5,12}, {6,-1}, {7, int.MaxValue } };
    
    public void Foo(int val, ObjA a)
    {
        Bar.Do(a, ParamMap[val]);
    }
    
    This uses a static, prebuilt dictionary that completely eliminates Cyclomatic Complexity.
    
    Notice all the Cyclomatic Complexity is gone. This code never branches. There is very little left to test.
    
    <b>Example 2</b>
    Imagine the code is more like this, in which different case statements call different overloaded values.
    
    
    public void Foo(int val, ObjA a, ObjB b, ObjC c)
    {
        switch (val)
        {
            case 0:
                Bar.Do(a);
                break;
            case 1:
                Bar.Do(b);
                break;
            case 2:
                Bar.Do(c);
                break;
            case 3:
                Bar.Do(a, c);
                break;
            case 4:
                Bar.Do(b, c);
                break;
            case 5:
                Bar.Do(b, c, a);
                break;
            case 6:
                Bar.Do(b, c, a * .01);
                break;
            case 7:
                Bar.Do(a, b, c);
                break;
        }
    }

    This looks harder doesn’t it. The Cyclomatic Complexity can still be simplified. How are we going to do it? We can’t use a

    Well, one option is to use a Dictionary<int, object[]>.

    public void Foo(int val, ObjA a, ObjB b, ObjC c)
    {
        var Dictionary<int, object[]> paramMap = new Dictionary<int, object[]>();
        paramMap.Add(0, new []{ a });
        paramMap.Add(1, new []{ b });
        paramMap.Add(2, new []{ c });
        paramMap.Add(3, new []{ a, c });
        paramMap.Add(4, new []{ b, c });
        paramMap.Add(5, new []{ b, c, a });
        paramMap.Add(6, new []{ b, c, a * .01 });
        paramMap.Add(7, new []{ a, b, c });
        typeof(Bar).GetMethod("Do").Invoke(paramMap[val]); // Reflection allows for passing in a dynamically sized list of parameters.
    }
    

    The solution is almost exactly the same as above. The differences are:

    1. The dictionary is dynamic, based on the passed in parameters, so we have to build it dynamically.
    2. The parameters are dynamic so we call the method with reflection to allow for dynamic parameters.

    The dictionary still completely eliminates Cyclomatic Complexity. Notice all the Cyclomatic Complexity is gone. This code never branches. There is very little to test.

    There is the overhead of creating a Dictionary and the overhead of reflection, but again, unless you plan to use this for looping through large data sets, the performance difference is negligible.

    Dictionary<TKey, Func<>>

    Sometimes there isn’t much common at all. Sometimes, the complexities very greatly.

    // ... code here

    Imagine the code that goes there is vastly different. Imagine you just can’t find much common ground. In these situations, you can use Dictionary<TKey, Func<>>. The pattern is to put the dictionary in its own class file. Then the object that uses it can have an injectable IDictionary<TKey, Func<>>. Injection options are: Constructor injection, Method injection, property injection. I lean toward a property injection variation called a Lazy Injectable Property.

    Question: What generic paramaters should be used for the Dictionary?
    Answer: The TKey is clearly the type of the val property, which in the example is an int.

    Question: What generic parameters should be used for the Func<>?
    Answer: Well, you need to think through to get this answer. First, you need to find the Lowest Common Parameter Set. Second you need to check the return type.

    Finding the Lowest Common Parameter Set

    If you look at one of the above methods, you can easily get the lowest common parameter set by writing down each and every parameter pass in. Remember this one from above?

    public void Foo(int val, ObjA a, ObjB b, ObjC c)
    {
        // Switch/case statement here . . .
    }
    

    The lowest common parameter set is: a, b, c. If you look at the full implementation further up, you will notice that none of the methods take in val, so val is not included in the parameter set.

    So now we can create our Dictionary. We will have three input parameters.

    Action<> vs Func<>

    This is easy. The only notable difference is that Action<> takes in parameters and returns void. Func<> takes in parameters and returns the type specified in the last generic type.

    So as there is no return value in the above example, we can use this code:

    public Class FuncDictionary : Dictionary<int, Action<ObjA, ObjB, ObjC>>
    {
        public FuncDictionary()
        {
            this.Add(0, (a, b, c) => { Bar.Do(a); } ); // Parameters b, c are ignored. That is ok.
            this.Add(1, (a, b, c) => { Bar.Do(b); } );
            this.Add(2, (a, b, c) => { Bar.Do(c); } );
            this.Add(3, (a, b, c) => { Bar.Do(a, c); } );
            this.Add(4, (a, b, c) => { Bar.Do(b, c); } );
            this.Add(5, (a, b, c) => { Bar.Do(b, c, a); } );
            this.Add(6, (a, b, c) => { Bar.Do(b, c, a * .01); } );
            this.Add(7, (a, b, c) => { Bar.Do(a, b, c); } );
        }
    }
    

    Now look at the foo code.

    // Lazy injectable property
    internal IDictionary<int, Action<ObjA, ObjB, Objc> ActionDictionary
    {
        get { return _ActionDictionary ?? (_ActionDictionary = new FuncDictionary()); }
        set { _ActionDictionary = value; }
    } private IDictionary<int, Action<ObjA, ObjB, Objc> _ActionDictionary;
    
    public void Foo(int val, ObjA a, ObjB b, ObjC c)
    {
        ActionDictionary[val].Invoke(a, b, c);
    }
    

    In all the previous methods, we resolved Cyclomatic Complexity by taking a method with 8 branches, and reducing that 1 method to 0 branches. We can also get 100% code coverage with 1 unit test.

    1, Methods
    0, Cyclomatic Complexity

    In this final Dictionary<TKey, Func<>> example, we end up with 8 methods that need testing.

    8, Methods
    0, Cyclomatic Complexity

    The reason is that we still have to test all 8 methods in the FuncDictionary. However, when that work was in the switch/case statement, those were harder to isolate for unit tests. Now, all eight methods are isolated and unit tests are simplified.


    Three most basic rules every software development should follow

    1. Model classes have properties and nothing more.
      Note: Properties are basically getter and setter methods. In languages without properties, getters and setters are the equivalent. Do not use getter or setter methods as methods for anything more complex than default instantiation of the type. Example: You can make sure a List is not null in a getter and lazy load it, but don’t do much more. Or a calculated property might have minor logic but it is all internal to the model.
    2. Any class with methods can reference primitives, collections and lists, interfaces, and model classes. Do not referencing a class with methods directly from any other class.
    3. 10/100 Rule (Slightly bendable rule). No method should have more than 10 lines of code, including curly braces and comments. No Class should be more than 100 lines of code, including curly braces and comments.

    Look, there is S.O.L.I.D., there is D.R.Y, there is S.R.P., and many other rules. There are design patterns, including the gang of four patterns, and many others. There are architectures, MVC, MVVM, etc. But if you, as a software developer, follow the above three rules, you will automatically end up following most of the other rules. Design patterns will naturally be used in your code, even if you don’t know them, though I recommend you still learn and know about them.


    Back up and restore a single table with foreign keys using SQL Server

    Today I needed to backup a single database table, then test a change to the data, then if the change failed, restore the original data.

    Below is how I did this.

    What I used

    I used SQL Server Management Studio to do all of the following steps.
    I performed the steps on a database running on a SQL 2012 database server.

    Part 1 – Backup the table

    SELECT *
    INTO MyTable_Bak
    FROM MyTable;
    

    Note: This will work usually, however, it won’t work if you have a calculated column. If you have a calculated column, create the table first, then specify the columns you are inserting. I didn’t have a calculated column, so I didn’t take time to figure this out.

    Part 2 – Restoring the table

    Step 1 – Finding the Foreign Key Constraints

    SELECT Name, Object_Name(parent_object_id) as [Table]
    FROM sys.foreign_keys
    WHERE referenced_object_id = object_id('MyTable')
    

    The results were like this:

    Name                    Table
    FKDDED6AECAD1D93C0      MyOtherTable1
    FK166B6670AD1D93C0      MyOtherTable2
    

    Step 2 – Get the Drop and Create for each Foreign Key

    In SQL Management Studio Express, I went to each table in the above list, and did the following:

    1. Locate the foreign key under Database | MyDb | Tables | dbo.MyTable | Keys.
    2. Right-click on the Foreign Key and choose Script Key as | Drop and Create to | Clipboard.
    3. Paste this into the query window.
    4. Delete the USING MyDb statement and separate the DROP statement from the two ALTER TABLE statements.
    5. Repeat for the next foreign key constraint, grouping the DROP statements and the ALTER TABLE statements together.

    Step 3 – Run the DROP statements

    Run the two DROP statements created above.

    ALTER TABLE [dbo].[MyOtherTable1] DROP CONSTRAINT [FKDDED6AECAD1D93C0]
    GO
    ALTER TABLE [dbo].[MyOtherTable2] DROP CONSTRAINT [FK166B6670AD1D93C0]
    GO
    

    Step 4 – Restore the table

    I used this query to restore the table from the backup.

    SELECT * FROM MyTable
    SET IDENTITY_INSERT dbo.MyTable ON; 
    TRUNCATE TABLE MyTable ;
    INSERT INTO MyTable (Id, Col1, Col2, Col3) -- Specify all columns here
    SELECT (Id, Col1, Col2, Col3)              -- Specify all columns again here
    FROM MyTable_Bak
    

    Step 5 – Restore the foriegn key constraints

    Run the ALTER TABLE scripts you grouped together from Step 2.

    ALTER TABLE [dbo].[MyOtherTable2]  WITH CHECK ADD  CONSTRAINT [FKDDED6AECAD1D93C0] FOREIGN KEY([MyTableId])
    REFERENCES [dbo].[MyTable] ([Id])
    GO
    
    ALTER TABLE [dbo].[MyOtherTable2] CHECK CONSTRAINT [FKDDED6AECAD1D93C0]
    GO
    
    ALTER TABLE [dbo].[MyOtherTable2]  WITH CHECK ADD  CONSTRAINT [FK166B6670AD1D93C0] FOREIGN KEY([MyTableId])
    REFERENCES [dbo].[MyTable] ([Id])
    GO
    
    ALTER TABLE [dbo].[MyOtherTable2] CHECK CONSTRAINT [FK166B6670AD1D93C0]
    GO
    

    Conclusion

    Your table is restored.

    Observations

    1. This is a simple process for a table with only a few foriegn key constraints but could be difficult if you have dozens of foreign key constraints.
    2. Also, this process might not work perfectly with calculated columns without changes to the above process.
    3. It should be easier to do this.

    If you know of a better way that doesn’t require $oftware, let me know.


    Cleaning a customer folder in Visual Studio project

    I have a Plugins directory in a Visual Studio project. I would like to delete all files from it when I clean.

    Here is how I did this:

    1. Edit the .csproj file.
    2. Add the following to the very bottom of the .csproj file, just above the terminating </Project> tag.
        <Target Name="afterClean">
            <ItemGroup>
              <FilesToDelete Include="Plugins\**\*"/>
          </ItemGroup>
          <Delete Files="@(FilesToDelete)" />
        </Target>
      
    3. Save the .csproj file.

    That should do it.

    Update 12/14/2016:
    You should be aware that when doing a custom clean, that in Visual Studio choosing Clean and then Build in two steps is not the same as doing Rebuild.

    Clean <-- Cleans all projects Build <-- Builds all projects Rebuild <-- For each project, clean the project, then rebuild the project.So Rebuild is a huge problem and here is why: Since my code is pretty decoupled, my plugins don't reference the project hosting the plugins. So there is not dependency to guarantee the plugin-hosting project cleans and builds first. So when running reubild, a plugin might clean and build and then copy its files to the plugin directory. This could happen before the plugin-hosting project cleans and builds. So you can imagine that once the plugin-hosting project cleans and builds, the newly copied plugin files are cleaned. To fix this, I had to manually add a dependency or just not use Rebuild.


    Constructor Injection Hell

    So I am a fan of dependency injection (DI), inversion of control (IoC), and the way DI and IoC allow for simplistic methods and Unit Tests. With DI, you can do method injection, property injection, or constructor injection. I don’t care which one a project uses, as long as they keep it simple.

    Constructor Injection

    This article is focussing on constructor injection. Constructor injection seems to be very popular, if not the most popular method of DI. Constructor Injection is considered to have a benefit because it requires the instantiator to provide all the dependencies an object needs in order to create an instance of it.

    An Example of Constructor Injection Hell

    Recently, I started working with NopCommerce, which uses DI heavily. They use Autofac and register objects with Autofac so it can provide concrete instances of any interfaces.

    I am going to use NopCommerce as an example of what not to do. Now before I do this, I want to explain that NopCommerce overall has a very good architecture. Better than most. Finding something that I consider a “what not to do” in a project should not steer you away from NopCommerce. In fact, their plugin model and architecture works quite well.

    Below is an example of constructor injection gone wrong from the OrderProcessingService.cs file in NopCommerce.

    #region Ctor
    
            /// <summary>
            /// Ctor
            /// </summary>
            /// <param name="orderService">Order service</param>
            /// <param name="webHelper">Web helper</param>
            /// <param name="localizationService">Localization service</param>
            /// <param name="languageService">Language service</param>
            /// <param name="productService">Product service</param>
            /// <param name="paymentService">Payment service</param>
            /// <param name="logger">Logger</param>
            /// <param name="orderTotalCalculationService">Order total calculationservice</param>
            /// <param name="priceCalculationService">Price calculation service</param>
            /// <param name="priceFormatter">Price formatter</param>
            /// <param name="productAttributeParser">Product attribute parser</param>
            /// <param name="productAttributeFormatter">Product attribute formatter</param>
            /// <param name="giftCardService">Gift card service</param>
            /// <param name="shoppingCartService">Shopping cart service</param>
            /// <param name="checkoutAttributeFormatter">Checkout attribute service</param>
            /// <param name="shippingService">Shipping service</param>
            /// <param name="shipmentService">Shipment service</param>
            /// <param name="taxService">Tax service</param>
            /// <param name="customerService">Customer service</param>
            /// <param name="discountService">Discount service</param>
            /// <param name="encryptionService">Encryption service</param>
            /// <param name="workContext">Work context</param>
            /// <param name="workflowMessageService">Workflow message service</param>
            /// <param name="vendorService">Vendor service</param>
            /// <param name="customerActivityService">Customer activity service</param>
            /// <param name="currencyService">Currency service</param>
            /// <param name="affiliateService">Affiliate service</param>
            /// <param name="eventPublisher">Event published</param>
            /// <param name="pdfService">PDF service</param>
            /// <param name="rewardPointService">Reward point service</param>
            /// <param name="genericAttributeService">Generic attribute service</param>
            /// <param name="paymentSettings">Payment settings</param>
            /// <param name="shippingSettings">Shipping settings</param>
            /// <param name="rewardPointsSettings">Reward points settings</param>
            /// <param name="orderSettings">Order settings</param>
            /// <param name="taxSettings">Tax settings</param>
            /// <param name="localizationSettings">Localization settings</param>
            /// <param name="currencySettings">Currency settings</param>
            public OrderProcessingService(IOrderService orderService,
                IWebHelper webHelper,
                ILocalizationService localizationService,
                ILanguageService languageService,
                IProductService productService,
                IPaymentService paymentService,
                ILogger logger,
                IOrderTotalCalculationService orderTotalCalculationService,
                IPriceCalculationService priceCalculationService,
                IPriceFormatter priceFormatter,
                IProductAttributeParser productAttributeParser,
                IProductAttributeFormatter productAttributeFormatter,
                IGiftCardService giftCardService,
                IShoppingCartService shoppingCartService,
                ICheckoutAttributeFormatter checkoutAttributeFormatter,
                IShippingService shippingService,
                IShipmentService shipmentService,
                ITaxService taxService,
                ICustomerService customerService,
                IDiscountService discountService,
                IEncryptionService encryptionService,
                IWorkContext workContext,
                IWorkflowMessageService workflowMessageService,
                IVendorService vendorService,
                ICustomerActivityService customerActivityService,
                ICurrencyService currencyService,
                IAffiliateService affiliateService,
                IEventPublisher eventPublisher,
                IPdfService pdfService,
                IRewardPointService rewardPointService,
                IGenericAttributeService genericAttributeService,
                ICompanyService companyService,
                ShippingSettings shippingSettings,
                PaymentSettings paymentSettings,
                RewardPointsSettings rewardPointsSettings,
                OrderSettings orderSettings,
                TaxSettings taxSettings,
                LocalizationSettings localizationSettings,
                CurrencySettings currencySettings)
            {
                this._orderService = orderService;
                this._webHelper = webHelper;
                this._localizationService = localizationService;
                this._languageService = languageService;
                this._productService = productService;
                this._paymentService = paymentService;
                this._logger = logger;
                this._orderTotalCalculationService = orderTotalCalculationService;
                this._priceCalculationService = priceCalculationService;
                this._priceFormatter = priceFormatter;
                this._productAttributeParser = productAttributeParser;
                this._productAttributeFormatter = productAttributeFormatter;
                this._giftCardService = giftCardService;
                this._shoppingCartService = shoppingCartService;
                this._checkoutAttributeFormatter = checkoutAttributeFormatter;
                this._workContext = workContext;
                this._workflowMessageService = workflowMessageService;
                this._vendorService = vendorService;
                this._shippingService = shippingService;
                this._shipmentService = shipmentService;
                this._taxService = taxService;
                this._customerService = customerService;
                this._discountService = discountService;
                this._encryptionService = encryptionService;
                this._customerActivityService = customerActivityService;
                this._currencyService = currencyService;
                this._affiliateService = affiliateService;
                this._eventPublisher = eventPublisher;
                this._pdfService = pdfService;
                this._rewardPointService = rewardPointService;
                this._genericAttributeService = genericAttributeService;
                this._companyService = companyService;
    
                this._paymentSettings = paymentSettings;
                this._shippingSettings = shippingSettings;
                this._rewardPointsSettings = rewardPointsSettings;
                this._orderSettings = orderSettings;
                this._taxSettings = taxSettings;
                this._localizationSettings = localizationSettings;
                this._currencySettings = currencySettings;
            }
    
            #endregion
    

    Problems in the Constructor Injection Implementation

    So what is wrong with the above constructor? Well, a lot. Look, this is just bad code. While constructor injection is a good idea, taking it to this extreme is not a good idea. In fact, it is a terrible idea.

    1. The Constructor has too many parameters. While there is no limit, there is a best practice. See this stack overflow post: How many parameters are too many?
    2. The Constructor breaks the 10/100 rule. The constructor, with comments, method parameters, and method body is 126 lines of code. The method itself is far more than 10 lines of code, it is 39 lines of parameters and 39 more lines of member assignments, and is 80 lines of code.
    3. The Constructor breaks the keep it super simple (KISS) principle. Having to new up 39 concrete instances of the parameters in order to create an object is not simple. Imagine mocking 39 interface parameters in a Unit Test. Ugh!
    4. This constructor is a hint that the entire class is doing too much. The class is 3099 lines and clearly breaks the single responsibility principle. It is not the OrderProcessingService’s responsibility to store 39 dependent services.
    5. The constructor breaks the Don’t Repeat Yourself (DRY) principle. Almost all other classes in NopCommerce use constructor injection to access services.

    Options for Refactoring

    Option 1 – Container object

    You could create a container that has all of these dependecies, a dependency model object for the OrderProcessingService. This object would house the 39 dependent services and settings. But Option 2 would be better.

    Option 2 – Accessor objects

    Looking at this from the Single Responsibility Principle, shouldn’t there be one class and interface, a ServiceAccessor : IServiceAccessor that allows one to access any dependent service? Instead of passing in 30 services, wouldn’t it make more sense to pass in a single object called a ServiceAccessor that implements IServiceAccessor? Should there be a ServiceAccessor of some sort? Turns out there is a static: EngineContext.Current.Resolve(). Since it is a static, maybe you could wrap it in a ServiceAccessor : IServiceAccessor object.

    There are also a lot of “settings” objects passed into the constructor? Shouldn’t there be a SettingsService? Well, there is. One can pass in the ISettingsService and then call _settingService.LoadSetting().

    Instead of passing in 39 parameters, methods with a single responsibility to fetch a service should be used.

    Option 3 – Refactor the class

    Since the class is 3099 lines. If the class were broken into logical pieces, naturally, the constructor for each smaller piece would have less parameters.


    How to convert a string to an enum in C#?

    Use this extension method:

    using System;
    
    namespace Rhyous.Extensions
    {
        public static class StringExtensions
        {
            public static T AsEnum<T>(this string str, T defaultValue)
            {
                try { return (T)Enum.Parse(typeof(T), str, true); }
                catch { return defaultValue; }
            }
        }
    }
    

    So imagine you have this enum:

    public enum LogLevel
    {
       Debug,
       Information,
       Warning,
       Error,
       Fatal
    }
    

    Call it like this:

    var levelStr = "Error";
    LogLevel level = levelStr.AsEnum(LogLevel.Info);
    

    Unit testing calls to complex extension methods

    This article isn’t about unit testing an extension method. That is pretty straight forward. This article is about unit testing and object that calls an extension method where the extension method is difficult to test. Likely the method is difficult to test because it touches an external system, such as a database or a remote web service.

    If you have an extension method that is simple and doesn’t touch and external system, it is easy to unit test. Look at the example below. There is nothing blocking you from Unit Testing code that calls this method.

    public static int Add(this int left, int right) 
    {
        return left + right;
    }
    

    Now image the extension method is more complex, say for a shopping cart.

    public static void PlaceOrder(this Order order) 
    {
        SaveToDb(Order);
        ChargeCreditCard(Order.CreditCardDetails);
    }
    

    How are you going to unit test code that calls an extension method that place an order and charges a customer’s Credit Card. Yikes. That is little harder to Unit Test, right?

    How to Unit Test a call to a complex extension method

    Imagine you have the following code:

    1. An object you are test called ObjectUnderTest
      public class ObjectUnderTest
      {
          private void MyObject = new MyObject();
      
          public object SomeFunction() 
          {
              return myObj.DoWork(val);
          }
      }
      
    2. An dependent object MyObject : IMyObject
      public class MyObject : IMyObject
      {
       // ... some code
      }
      
    3. An extension method on IMyObject: DoWork(this IMyObject obj, string value).
      public static object DoWork(this IMyObject obj, string value)
      {
          // really complex stuff and touches external systems
      }
      

    You need Unit Tests for SomeFunction(). Imagine that all other code is 100% unit tested. But you are struggling with how to Unit Test SomeFunction because it has two dependencies:

    1. MyObject
    2. DoWork

    The Unit Tests should not call the real DoWork because it does really complex stuff and touches external systems. However, you need the parent method to provide a valid return value.

    Well, you could just drop the ExcludeFromCodeCoverageAttribute on the method and move on. But what if there are a half-dozen other objects that call the parent method that also need to be tested and they need a return value from SomeFunction()? It would be best to solve this in this object as so you only change one class file, not a half-dozen.

    One option to resolve this is to use dependency injection. Dependency Injection (DI) simply means that any dependencies can be injected. When some people hear DI, they think they immediately need the huge overhead of an IoC Container. IoC containers are nice and have their uses. But using an IoC container only to allow unit tests substitute a dependency is a huge overkill. If your project already has an IoC container, feel free to use it. Otherwise, I recommend you use a simpler option. I prefer an internal lazy injectable property.

    Creating a Lazy Injectable Property

    An internal lazy injectable property is a property that is instantiated on first use if it is null, but always for code with internal access to swap out the property value. Here is the syntax:

    Note: This assumes your unit tests references your project already, has InternalsVisibleTo configured, and has Moq from NuGet applied to the test project.

        internal IMyObject MyObjectProperty
        {
            get { return _MyObject ?? (_MyObject = new MyObject()); }
            set { _MyObject= value; }
        } private List<object> _MyObject;
    

    Look how simple the above code is. If _MyObject is null, the first time MyObjectProperty is called, it is instantiated to a new MyObject().It is internal because only the unit test will every replace it. I don’t really want this property exposed elsewhere. We can use InternalsVisibleTo to allow the Unit Tests access.Now my ObjectUnderTest will look like this:

    public class ObjectUnderTest
    {
        internal IMyObject MyObjectProperty
        {
            get { return _MyObject ?? (_MyObject = new MyObject()); }
            set { _MyObject= value; }
        } private IMyObject _MyObject;
    
        public object SomeFunction()
        {
            var val = "doesn't matter for this example";
            return MyObjectProperty.DoWork(val);
        }
    }
    

    Now, in the unit test, the MyObjectProperty can be replaced with a mock IMyObject.

    [TestMethod]
    public void SomeFunctionTest()
    {    // Arrange
        var mockMyObject = new Mock<IMyObject>();
        var objUnderTest = new ObjectUnderTest();
        objectUnderTest.MyObjectProperty = mockMyObject.Object;
    
        // More to come . . .
    }

    However, it is questionable whether this is even necessary. Does MyObject do anything that requires this level of abstraction? Not in this example. It isn’t the object itself that is complex, it is the extension method that really needs to be injectable.

    Creating a Lazy Injectable Property for a method

    You might be asking yourself, “What about the extension method? It is a method not an object. How can I inject that?” Well, you can. Remember, even methods can be treated as objects. The answer doesn’t change much. The only difference is understanding how to treat a method as an object.You can objectify methods using multiple objects such as Action, Func, Predicate, delegate, etc. I am not going to go into how to do that here beyond the minimal needed to accomplish this task.

    Quick tip: Use Action for void methods, Predicate for methods return bool, Func for methods with any return value, delegate if you have ref or out paramters.

    Here are the steps:

    1. Create the following Lazy Injectable Property inside ObjectUnderTest:

      Note: I am using Func because it has a return value of object. (See the Quick Tip a few lines up.) Since I have two paramters and a return type, I will specifically use the generic Func.

          internal Func<IMyObject, string, object> DoWorkMethod
          {
              [ExcludeFromCodeCoverage]
              get { return _DoWorkMethod ?? (_DoWorkMethod = (obj, val) => { return obj.DoWork(val); }); }
              set { _DoWorkMethod = value; }
          } private Func<IMyObject, string, object> _DoWorkMethod;
      
    2. Change SomeFunction() to run the method via the Action object instead of running the method directly.
          public object SomeFunction()
          {
              var val = "doesn't matter for this example";
              return DoWorkMethod.Invoke(MyObjectProperty, val);
          }
      
    3. In your Unit Test, you can create your ObjectUnderTest. Then you can swap out the DoWork method object.
      [TestMethod]
      public void SomeFunctionTest()
      {
          // Arrange
          var mockMyObject = new Mock<IMyObject>();
          var objUnderTest = new ObjectUnderTest();
          objUnderTest.MyObjectProperty = mockMyObject.Object;
          bool methodWasCalled = false;
          objUnderTest.DoWorkMethod = (obj, val) => {
              methodWasCalled = true;
              return new object();
          };
              
          // Act
          var result = objUnderTest.SomeFunction();
      
          // Assert
          Assert.IsTrue(methodWasCalled);
      }
      

    You are now 100% covered. The only code we can’t cover is the lambda call to obj.DoWork because we can’t Unit Test that as it touches an external system. Which is why we marked it with the ExcludeFromCodeCoverageAttribute.


    A SerializableDictionary in C#

    If you create a static Dictionary in code, every time you need to change the dictionary, you have change code, recompile, and redeploy. Wouldn’t it be nice if you didn’t have to change code. What if you could create your dictionary in an Xml file and deserialize it. You can now make the change outside of code.

    using System.Collections.Generic;
    using System.Xml.Serialization;
    
    namespace Rhyous.EasyXml
    {
        [XmlRoot("Dictionary")]
        public class SerializableDictionary<TKey, TValue>
            : Dictionary<TKey, TValue>, IXmlSerializable
        {
            public string KeyName = "key";
            public string ValueName = "value";
    
            #region constructors
            public SerializableDictionary()
            {
            }
    
            public SerializableDictionary(IEqualityComparer<TKey> comparer)
                : base(comparer)
            {
            }
            #endregion
    
    
    
            #region IXmlSerializable Members
            public System.Xml.Schema.XmlSchema GetSchema()
            {
                return null;
            }
    
            public void ReadXml(System.Xml.XmlReader reader)
            {
                var keySerializer = new XmlSerializer(typeof(TKey), null, null, new XmlRootAttribute(KeyName), null);
                var valueSerializer = new XmlSerializer(typeof(TValue), null, null, new XmlRootAttribute(ValueName), null);
    
                var wasEmpty = reader.IsEmptyElement;
                reader.Read();
    
                if (wasEmpty)
                    return;
    
                while (reader.NodeType != System.Xml.XmlNodeType.EndElement)
                {
                    var key = (TKey)keySerializer.Deserialize(reader);
                    var value = (TValue)valueSerializer.Deserialize(reader);
                    Add(key, value);
                    reader.MoveToContent();
                }
                reader.ReadEndElement();
            }
    
            public void WriteXml(System.Xml.XmlWriter writer)
            {
                var keySerializer = new XmlSerializer(typeof(TKey));
                var valueSerializer = new XmlSerializer(typeof(TValue));
    
                foreach (TKey key in Keys)
                {
                    keySerializer.Serialize(writer, key);
                    valueSerializer.Serialize(writer, this[key]);
                }
            }
            #endregion
        }
    }