Amazon Ec2 Instance Management with C#: Part 3 – Uploading and Importing a Key Pair

Before getting started

Skill Level: Beginner

Assumptions:

  1. You have completed Part 1 and 2 of Managing Amazon AWS with C# – EC2

Additional Information: I sometimes cover small sub-topics in a post. Along with AWS, you will also be exposed to:

  • .NET Core 2.0 – If you use .NET Framework, the steps will be slightly different, but as this is a beginner level tutorial, it should be simple.
  • Rhyous.SimpleArgs

Details

We may already have a key pair that we want to use, so we don’t want to create a new one. If that is the case, it can be uploaded.

Step 1 – Get key in the correct format

I used OpenSSL to do this.

  1. Download OpenSSL.
  2. Run this command:
    [sh]
    .\openssl.exe rsa -in c:\users\jbarneck\desktop\new.pem -pubou
    t -out c:\users\jbarneck\desktop\new.pub
    [sh]

Step 2 – Edit InstanceManager.cs file

We’ve created InstanceManager.cs in Part 1. Let’s edit it.

  1. Add a method to read the key file from disk and upload and import the key pair.
  2.         public static async Task ImportKeyPair(AmazonEC2Client ec2Client, string keyName, string keyFile)
            {
                var publicKey = File.ReadAllText(keyFile).Trim().RemoveFirstLine().RemoveLastLine();
                string publicKeyAsBase64 = Convert.ToBase64String(Encoding.UTF8.GetBytes(publicKey));
                await ec2Client.ImportKeyPairAsync(new ImportKeyPairRequest(keyName, publicKeyAsBase64));
            }
    

Notice: We are calling RemoveFirstLine() and RemoveLastLine(); This is because key files have a header and footer that must be removed before sending up to AWS. We’ll do this in the next section.

Step 3 – Add methods RemoveFirstLine and RemoveLastLine

  1. By the time this publishes, you should only need to install Rhyous.String.Library. Otherwise, add this class file:
    namespace Rhyous.AmazonEc2InstanceManager
    {
        public static class StringExtensions
        {
            public static string RemoveFirstLine(this string text, char newLineChar = '\n')
            {
                if (string.IsNullOrEmpty(text))
                    return text;
                var i = text.IndexOf(newLineChar);            
                return i > 0 ? text.Substring(i + 1) : "";
            }
    
            public static string RemoveLastLine(this string text, char newLineChar = '\n')
            {
                var i = text.LastIndexOf(newLineChar);
                return (i > 0) ? text.Substring(0, i) : "";
            }
        }
    }
    

Step 4 – Configure command line Arguments.

We already have an Actions arguments to edit.

  1. Add DeleteKeyPair as a valid action to the Action argument.
  2. Add an additional argument for the key file.
                . . .
                new Argument
                {
                    Name = "Action",
                    ShortName = "a",
                    Description = "The action to run.",
                    Example = "{name}=default",
                    DefaultValue = "Default",
                    AllowedValues = new ObservableCollection<string>
                    {
                        "CreateKeyPair",
                        "DeleteKeyPair",
                        "ImportKeyPair"
                    },
                    IsRequired = true,
                    Action = (value) =>
                    {
                        Console.WriteLine(value);
                    }
                },
                . . .
                new Argument
                {
                    Name = "KeyFile",
                    ShortName = "pem",
                    Description = "The full path to a public key already created on your file system in PEM format. The full Private key won't work.",
                    Example = "{name}=c:\\My\\Path\\mykeyfile.pem",
                    CustomValidation = (value) => File.Exists(value),
                    Action = (value) =>
                    {
                        Console.WriteLine(value);
                    }
                }

You can now upload a public key file for use on the Amazon Cloud.

Next: Part 4

Return to: Managing Amazon AWS with C#

Interviewing: A developer should have a portfolio

I recently started interviewing for some contract positions, one a Software Developer in Test position and one a Senior Software Developer position. I am deeply surprised by the candidates complete lack of having an online presence. As I thought more about this, I realized that we have reached a point of maturity in the Software Developer roles that portfolios are now expected. I expected every candidate to have an active account on some open source source control repository, i.e. GitHub, and have a portfolio of code there.

Portfolio

When it is time to interview for a position as a developer, you should have a portfolio. The days of coding on a whiteboard should be over. Instead, an interviewer should be able to easily see your code and what you have or haven’t done.

There shouldn’t be a question about whether you can write code. Instead, the question should be: Based on the code we can see this individual has written, can they be a good fit for our team?

Proprietary code

Your portfolio cannot include proprietary code. End of discussion. If you are a developer and you can’t find code that isn’t proprietary to put into your portfolio, then what are you doing?

Open Source/Non-proprietary code

Even when working with proprietary code, there are many pieces of code that are so ubiquitous that they probably should be part of the .NET framework. You may use this code in every project you work on. Such as common string extensions in C#, or a more complete string check in javascript that checks if a string is undefined, null, empty, or whitespace.

Even better is if your code is not just stored, but it is available to be used, such as with NuGet, npm, Maven, or other code or library packaging tool. This shows that you not only have a portfolio, but you aren’t going to waste your hours rewriting code you have already written.

Where to keep your portfolio

I used to have mine on SourceForge but have since switched to GitHub. Visual Studio online is another option. Where you store your portfolio of your work does not matter as much as the fact that you do store it.

GitHub is where I chose. But you can easily Google for GitHub competitors if you want it to be elsewhere.

Brand your portfolio

My internet handle is Rhyous. Every piece of code I write that is part of my portfolio (Non-proprietary or not for someone else’s open source project) is now branded with Rhyous. Some of my older code may not be, but my new code is. For example, all my namespaces in C# now start with Rhyous. That makes it very easy to differentiate projects I have forked vs projects that I have developed.

What your portfolio must show

It must show:

  • You have skills as a developer.
  • SOLID principals.
  • An understanding of the importance of Unit Tests.
  • You refuse to waste time writing the same code twice.**
  • You can work on projects with other developers.
  • You bring more than just your skill set, you bring your ready-made building blocks.

** I find this to be so very important!

My Portfolio

My portfolio shows my skills as a developer. My code uses SOLID principals. Much of my code is Unit Tested.

I don’t like to write the same code twice. I for one, will never have to write a CSV parser in C# again as I have a good quality one: Rhyous.EasyCsv. Parsing arguments? I’ll never write an argument parser again because I have Rhyous.SimplArgs. I will never have to write many of my string extensions again as I can easily grab them for any C# project from my Rhyous.StringLibrary NuGet package. Tired of using TryGetValue to get values from your dictionary? Try Rhyous.Collections and use the NullSafe dictionary, which still used the TryGetValue but moves it inside the indexer so you don’t have to worry about it.

What a lack of portfolio shows

It has never clicked for you. What I mean by “It” is the idea of code reuse. The idea of object oriented programming. The idea of standing on the shoulders of giants. The simple idea of using building blocks as a kid and make things from building block.

Go out and make your portfolio and fill it with building blocks so every time you build something new, you can build on foundation building blocks that are SOLID and just get better.

Amazon Ec2 Instance Management with C#: Part 2 – Deleting a Key Pair

Before getting started

Skill Level: Beginner

Assumptions:

  1. You have completed Part 1 of Managing Amazon AWS with C# – EC2

Additional Information: I sometimes cover small sub-topics in a post. Along with AWS, you will also be exposed to:

  • .NET Core 2.0 – If you use .NET Framework, the steps will be slightly different, but as this is a beginner level tutorial, it should be simple.
  • Rhyous.SimpleArgs

Step 1 – Edit InstanceManager.cs file

We’ve created InstanceManager.cs in Part 1. Let’s edit it.

  1. Add a method to delete the key pair.
  2.         public static async Task DeleteKeyPair(AmazonEC2Client ec2Client, string keyName)
            {
                await ec2Client.DeleteKeyPairAsync(new DeleteKeyPairRequest { KeyName = keyName });
            }
    

Step 5 – Configure command line Arguments.

We already have an Actions arguments to edit.

  1. Add DeleteKeyPair as a valid action to the Action argument.
  2.                 . . .
                    new Argument
                    {
                        Name = "Action",
                        ShortName = "a",
                        Description = "The action to run.",
                        Example = "{name}=default",
                        DefaultValue = "Default",
                        AllowedValues = new ObservableCollection<string>
                        {
                            "CreateKeyPair",
                            "DeleteKeyPair"
                        },
                        IsRequired = true,
                        Action = (value) =>
                        {
                            Console.WriteLine(value);
                        }
                    },
                    . . .
    

Next:

  • Part 3 – Uploading and Importing a Key Pair
  • Return to: Managing Amazon AWS with C#

    Amazon Ec2 Instance Management with C#: Part 1 – Creating a Key Pair

    Before getting started

    Skill Level: Beginner

    Assumptions:

    1. You already have Visual Studio installed.
    2. You are familiar with creating projects in Visual Studio.
    3. We assume you have already gone to AWS and registered with them. If you haven’t done that already, stop and go there now. Amazon has a free tier and you can create an account here: https://aws.amazon.com/free
    4. Some of the points that were discussed in the Amazon S3 article series will not be repeated here.

    Additional Information: I sometimes cover small sub-topics in a post. Along with AWS, you will also be exposed to:

    • .NET Core 2.0 – If you use .NET Framework, the steps will be slightly different, but as this is a beginner level tutorial, it should be simple.
    • async, await, Task
    • Rhyous.SimpleArgs
    • Reflection

    Note: As this is the first post of a series, there is going to be some setup. If you just want the quick and code, look at the code in Step 3.

    EC2 Instance Prerequisites

    Before creating an instance, there are some prerequisites. This Part 1 discusses the first of these prerequisites: creating a Key pair.

    Step 1 – Create the project

    1. Open Visual Studio.
    2. Go to File | New Project.
    3. Choose Console Application.
      Give it any name you want.
      I am going to call my project Rhyous.AmazonEc2InstanceManager.

    Step 2 – Add NuGet Packages

    1. Right-click on your project and choose Management NuGet Packages.
    2. Search for AWSSDK.EC2.
    3. Install the NuGet package and all the dependencies.
    4. Search for System.Configuration.ConfigurationManager.
    5. Install it.
    6. Search for Rhyous.SimpleArgs.
    7. Install it.

    Step 3 – Create an InstanceManager.cs file

    1. Create a new class file called InstanceManager.cs.
    2. Add a method to generate the key pair.
    3. Add a method to save the key pair to disc as a .pem file.
    4. using Amazon.EC2;
      using Amazon.EC2.Model;
      using System;
      using System.Collections.Generic;
      using System.IO;
      using System.Threading.Tasks;
      
      namespace Rhyous.AmazonEc2InstanceManager
      {
          public class InstanceManager
          {
              public static async Task&amp;lt;KeyPair&amp;gt; CreateKeyPair(AmazonEC2Client ec2Client, string keyName, bool writePem = false, string keyOutputDirectory = null)
              {
                  var keyPair = (await ec2Client.CreateKeyPairAsync(new CreateKeyPairRequest { KeyName = keyName }))?.KeyPair;
                  if (writePem)
                      await SaveKeyPairToDisc(keyName, keyOutputDirectory, keyPair);
                  return keyPair;
              }
      
              public static async Task SaveKeyPairToDisc(string keyName, string keyOutputDirectory, KeyPair keyPair)
              {
                  var path = Path.Combine(keyOutputDirectory, $&amp;quot;{keyName}.pem&amp;quot;);
                  await File.WriteAllTextAsync(path, keyPair.KeyMaterial);
                  Console.WriteLine($&amp;quot;They key pair was saved to: {path}&amp;quot;);
              }
          }
      }
      

      Notice: To create the Key pair, we use an instance of an AmazonEC2Client with a simple CreateKeyPairRequest model.
      Notice: Saving the Key pair has really nothing to do with Amazon EC2 Instance management. This is just a simple File.WriteAllTextAsync call.

    Step 4 – Create/Edit the App.config

    Of course, it is important to know what Amazon Web Services account we are working with, so we will store this in the app.config.

    1. If there isn’t an app.config in your project, create one.
    2. Right-click on your project and choose Add | New Item.
    3. Search for Application Configuration File.
      Rename it to app.config.
    4. Add an appSetting for your AWS profile name.
    5. Add an additional appSetting for your chosen AWS region.
      &amp;lt;?xml version=&amp;quot;1.0&amp;quot; encoding=&amp;quot;utf-8&amp;quot; ?&amp;gt;
      &amp;lt;configuration&amp;gt;
        &amp;lt;appSettings&amp;gt;
          &amp;lt;add key=&amp;quot;AWSProfileName&amp;quot; value=&amp;quot;yourprofilename&amp;quot;/&amp;gt;
          &amp;lt;add key=&amp;quot;AWSRegion&amp;quot; value=&amp;quot;us-west-2&amp;quot; /&amp;gt;
        &amp;lt;/appSettings&amp;gt;
      &amp;lt;/configuration&amp;gt;
      

    Step 5 – Configure command line Arguments.

    We are going to be adding to this program in subsequent posts. For this reason, we are going to use Rhyous.SimpleArgs library for our command line arguments as it provides ready-made command line argument features. If you have been through the S3 series, you will already be familiar with Arguments and the convention we chose to match command line arguments to the method parameter names.
    Note: You should have already installed the Rhyous.SimpleArgs NuGet package.

    1. Create an ArgsHandler.cs file to define the arguments:
      Note: If you used a .NET core project you have to create this file. If you created a .NET Framework file, this file should have been created for you and you have but to edit it.
    2. Add an Action argument.
      Note: We are going to keep the exact same convention of having an Action argument that allows us to choose which method to run and for each method and have an Argument to match each method parameter name.

    3. Add an Argument for each dynamic parameter in the CreateKeyPair method.
      • string keyName
      • string keyOutputDirectory

      Note: The AmazonEC2Client parameter is not a dynamic parameter. It is fixed and doesn’t change.

    4. Also, make sure to call Program.OnArgsHandled() in the HandleArgs method.
    5. using Amazon.EC2.Util;
      using Rhyous.SimpleArgs;
      using System;
      using System.Collections.Generic;
      using System.Collections.ObjectModel;
      
      namespace Rhyous.AmazonEc2InstanceManager
      {
          public class ArgsHandler : ArgsHandlerBase
          {
              public override void InitializeArguments(IArgsManager argsManager)
              {
                  Arguments.AddRange(new List&amp;lt;Argument&amp;gt;
                  {
                      new Argument
                      {
                          Name = &amp;quot;Action&amp;quot;,
                          ShortName = &amp;quot;a&amp;quot;,
                          Description = &amp;quot;The action to run.&amp;quot;,
                          Example = &amp;quot;{name}=default&amp;quot;,
                          DefaultValue = &amp;quot;Default&amp;quot;,
                          AllowedValues = new ObservableCollection&amp;lt;string&amp;gt;
                          {
                              &amp;quot;CreateKeyPair&amp;quot;
                          },
                          IsRequired = true,
                          Action = (value) =&amp;gt;
                          {
                              Console.WriteLine(value);
                          }
                      },
                      new Argument
                      {
                          Name = &amp;quot;KeyName&amp;quot;,
                          ShortName = &amp;quot;k&amp;quot;,
                          Description = &amp;quot;A key pair name.&amp;quot;,
                          Example = &amp;quot;{name}=MyKeyPair&amp;quot;,
                          Action = (value) =&amp;gt;
                          {
                              Console.WriteLine(value);
                          }
                      },
                      new Argument
                      {
                          Name = &amp;quot;keyOutputDirectory&amp;quot;,
                          ShortName = &amp;quot;k&amp;quot;,
                          Description = &amp;quot;A key pair name.&amp;quot;,
                          Example = &amp;quot;{name}=MyKeyPair&amp;quot;,
                          Action = (value) =&amp;gt;
                          {
                              Console.WriteLine(value);
                          }
                      }
                  });
              }
      
              public override void HandleArgs(IReadArgs inArgsHandler)
              {
                  base.HandleArgs(inArgsHandler);
                  Program.OnArgumentsHandled();
              }
          }
      }
      

    Step 6 – Add the MethodInfoExtension

    This has nothing to do with AWS, but we built it in the S3 series, so were are going to use it for this series as well. It is a class file that uses reflection to dynamically detect the needed parameters of our action methods. I am not going to go into it any more that that in this post. I’ve modified this file to work for this project. Just add it to your project. If you want to understand it more, go through the S3 posts.

    1. Create a MethodInfoExtensions.cs.
      using Amazon;
      using Amazon.EC2;
      using Rhyous.SimpleArgs;
      using System;
      using System.Collections.Generic;
      using System.Configuration;
      using System.Reflection;
      
      namespace Rhyous.AmazonEc2InstanceManager
      {
          public static class MethodInfoExtensions
          {
              public static List&amp;lt;object&amp;gt; DynamicallyGenerateParameters(this MethodInfo mi)
              {
                  var parameterInfoArray = mi.GetParameters();
                  var parameters = new List&amp;lt;object&amp;gt;();
                  var region = RegionEndpoint.GetBySystemName(ConfigurationManager.AppSettings[&amp;quot;AWSRegion&amp;quot;]);
                  foreach (var paramInfo in parameterInfoArray)
                  {
                      if (paramInfo.ParameterType == typeof(AmazonEC2Client))
                          parameters.Add(Activator.CreateInstance(paramInfo.ParameterType, region));
                      if (paramInfo.ParameterType == typeof(string))
                          parameters.Add(Args.Value(paramInfo.Name));
                  }
      
                  return parameters;
              }
          }
      }
      

    Step 7 – Edit the Program.cs

    1. Update Program.cs as follows:
      using Rhyous.SimpleArgs;
      using System;
      
      namespace Rhyous.AmazonS3BucketManager
      {
          class Program
          {
              static void Main(string[] args)
              {
                  new ArgsManager&amp;lt;ArgsHandler&amp;gt;().Start(args);
              }
      
              internal static void OnArgumentsHandled()
              {
                  var bucketName = Args.Value(&amp;quot;Bucket&amp;quot;);
                  var task = BucketManager.CreateBucket(bucketName);
                  task.Wait();
              }
          }
      }
      

    Now for fun, you can delete the app.config and change them to parameters.

    Next: Deleting a key pair
    Return to: Managing Amazon AWS with C#

    Understanding async and await, Task.WaitAll, Task.Run, and parallelism: part 2

    So another issue with async, await came up. I was using Task.WaitAll to make it parallel, but it wasn’t parallel.

    var task1 = Method1();
    var task2 = Method2();
    Task.WhenAll(task1, task2)
    

    Now Method1() and Method2() had some code that took some time. Now my code was making some large web service calls, but we can demonstrate this more easily just by sleeping for 10 seconds.

    public async Task Method1()
    {
        Thread.Sleep(10000); // Ten seconds
    }
    
    public async Task Method2()
    {
        await Task.Delay(10000); // 10 second task
    }
    
    

    I expected these to run in parallel, but they didn’t. Let me explain why.

    When calling a method that returns task, it doesn’t actually return until the first await. Notice Method1() doesn’t have an await. So Method1() will run synchronously until the first await.

    Method2(), however, is pretty much the same, but does have an await. So it returns immediately.

    Getting parallelism is easy. Wrap the method in Task.Run().

    public async Task Method1()
    {
    Task.Run(Thread.Sleep, 10000)); // Ten seconds
    }