How to Replace WCF Serialization with Json.Net without Wrapping and with UriTemplate Support

First, yes, I am still using WCF. Let’s move passed that concern to the real concern.

There are a dozen blog posts out there that explain how to replace the WCF serializer with Json.Net, however, every last one of them says that you must use wrapping and using parameters in the UriTemplate is not supported. https://blogs.msdn.microsoft.com/carlosfigueira/2011/05/02/wcf-extensibility-message-formatters

Just search the internet for WCF IDispatchMessageFormatter Json.Net. You will find all the articles that only work without UriTemplate support.

Well, I needed it to work with UriTemplate support without wrapping.

Turns out that this solution is far easier than I expected. I came accross this solution only after spending hours browsing Microsoft’s code.

So, to start, using parameters in the UriTemplate means that your Url or Url parameters will be specified in the UriTemplate and will have parameters.

For example, the Odata spec says that you should access an entity by Id with this a Url similar to this one:

https://somesite.tld/some/service/Users(1)

Then the method for the WCF service is like this:

[OperationContract]
[WebInvoke(Method = "GET", UriTemplate = "Users({{id}})", ResponseFormat = WebMessageFormat.Json)]
OdataObject Get(string id);
public virtual OdataObject Get(string id) 
{
    // code here
}

That is fine for a GET call as it doesn’t have a body. But what about a POST, Patch, or PUT call that does have a body? And what about now that the world is realizing that a GET sometimes needs a body?

Also, the examples provided a lot of code to figure out if it is a GET call and not even use the custom Json.Net IDispatchMessageFormatter. None of that code is necessary with this solution.

Let’s look at a PUT call that updates a single property of an entity as this has two parameters in the UriTemplate as well as a message body.

[OperationContract]
[WebInvoke(Method = "PUT", UriTemplate = "Users({{id}})/{{Property}} ResponseFormat = WebMessageFormat.Json)]
string UpdateProperty(string id, string property, string value);

public virtual OdataObject Put(string id, string property, string value)
{
// code here to update user
}

So there are two parameters in the UriTemplate, id and property, and the last parameter, value, is in the message body. Not a single solution for replacing the WCF serializer with Json.Net supports this scenario. Until now.

The goal is to deserialize the request with Json.Net. But the solutions provided break UriTemplate parameters in trying to reach the goal. The goal is not to replace the default WCF UriTemplate parameter work.

So now we can define a new problem: How do we deserialize the body with Json.Net but still have the UriTemplate parameters handled by WCF? The code to deserialize is the same code for both the parameters and the message body. We need to get the parameters without having WCF use the default deserializer for the message body.

Turns out, this problem is easy to solve.

Microsoft published their WCF code. Look at this code, lines 50-54: https://github.com/Microsoft/referencesource/blob/master/System.ServiceModel.Web/System/ServiceModel/Dispatcher/UriTemplateDispatchFormatter.cs

If you notice in line 50, WCF has the number of parameters from the Url and Url parameters and it subtracts that from the total list of parameters. If the message has not body, the subtraction result is always 0. If the message has a body, the subtraction always results in 1, telling WCF to deserialize the body. Well, I want WCF to do what it normally does with UriTempalte parameters, so if there is no body, use the WCF default stuff (which all the blogs say to do, but they do it the hard way).

Solution:

  1. In the custom EndPointBehavior, on the override, store the default IDispatchMessageFormater and pass it into the CustomDispatchMessageFormatter.
protected override IDispatchMessageFormatter GetReplyDispatchFormatter(OperationDescription operationDescription, ServiceEndpoint endpoint)
{
    var parentFormatter = base.GetReplyDispatchFormatter(operationDescription, endpoint);
    return new CustomDispatchMessageFormatter(this, operationDescription, parentFormatter);
}
  1. If there is no body, use the WCF default DeserializeRequest method. This vastly simplifies the code on the blogs out there. The other examples had masses of code upstream that just wasn’t needed when message.IsEmpty could be used.
  2. If there is a body but no parameters, just use Json.Net.
  3. If there is a body and there are UriTemplate parameters, create a temparary parameter array 1 size smaller and pass that into the default serializer.
  4. Copy the temp array to the orignal array.
  5. Then just deserialize with Json.Net.
public void DeserializeRequest(Message message, object[] parameters)
{
     if (message.IsEmpty || parameters.Length == 0)
         ParentFormatter.DeserializeRequest(message, parameters);
     else
         DeserializeMessageWithBody(message, parameters);
} 

private void DeserializeMessageWithBody(Message message, object[] parameters)
{
     if (parameters.Length > 1)
     {
         object[] tmpParams = new object[parameters.Length - 1];
         ParentFormatter.DeserializeRequest(message, tmpParams);
         tmpParams.CopyTo(parameters, 0);
     }
     if (message.GetWebContentFormat() != WebContentFormat.Raw)
         throw new InvalidOperationException("Incoming messages must have a body format of Raw.");
     byte[] rawBody = message.GetRawBody();
         var type = OperationDescription.Messages[0].Body.Parts.Last().Type;
         parameters[parameters.Length - 1] = RawBodyDeserializer.Deserialize(rawBody, type);
}

The deserializer becomes vastly simplified now that it isn’t trying to also handling wrapped parameters.

public class RawBodyDeserializer : IRawBodyDeserializer
{
    public object Deserialize(byte[] rawBody, Type type)
    { 
        using (MemoryStream ms = new MemoryStream(rawBody))
        using (StreamReader sr = new StreamReader(ms))
        {
            JsonSerializer serializer = new JsonSerializer();
            return serializer.Deserialize(sr, type);
        }
    }
}

Debugging Open Source dependencies included as NuGet packages

You may encounter the need to debug into a dependency that is NuGet package. If this NuGet package is proprietary, you need to contact the vendor. However, if the NuGet package is open source, perhaps on GitHub, then you have all the tools you need to debug into it. Debugging into an open source NuGet package is what this article is about.

We are going to use Rhyous.StringLibrary for this example. It is a simple open source project that provides some common extensions to strings. These are extensions that are often found duplicated in many different projects and sometimes multiple times in the same project.

Step 1 – Check out the Source

Check out the repo from GitHub. You need a Git client. If you don’t have one, you can use GitHub Desktop or the one that is included in the Windows install of Git.

  1. Check out the repository: 
    git fetch https://github.com/rhyous/StringLibrary.git 

Step 2 – Compare Assembly Versions

Some NuGet packages have different assembly versions than the code. I know, they shouldn’t be it happens. Make sure that the assembly version of the dll reference via the nuget package is the same as the assembly version in the downloaded source.

  1. In your project that references the NuGet package, expand the references, highlight the dll that came from the NuGet package, and note the assembly version.

  2. In the download NuGet package source project, check the Assembly version. This is different in .NET Framework and .Net Standard, but it should be easy to figure out in both.

Step 3 – Build the Solution

  1. Open the StringLibrary.sln in Visual Studio.
  2. Click Build.
  3. Go to the output directory and copy the dll and pdb files.

Step 4 – Copy the dll and pdb to your solution

If you go to your project that references the dll, find and highlight the reference and go to properties, you can see the full path to the referenced dll.

  1. Go to the solution folder of the project you are working on.
  2. Go to your project that references the dll.
  3. Under References, locate the dll.
  4. Go to Properties of the dll reference by pressing F4.
  5. Note the path to the dll.
  6. Go into the Packages directory.
  7. Find the folder for Rhyous.StringLibrary.
  8. Locate the dll folder. 
  9. Rename the existing rhyous.stringlibrary.dll to rhyous.stringlibrary.dll.orgininal.
  10. Copy the compiled dll and pdb from Step 2 to this folder.
  11. Clean and build your solution.

Step 5 – Add a breakpoint

You should now be able to step into the Rhyous.StringLibrary source from your project.

Note: If you have two instances of Visual Studio open, one for your project and one for Rhyous.StringLibrary project, you may think you put the break point in on the one with the SimplePluginLoader project. You don’t.  You don’t even need the Rhyous.StringLibrary project open, unless you need to make a change and recompile and recopy the dll and pdb to the packages directory. You simply need to step into the code in order to set a break point.

Note: One trick is to go to Tools | Options | Debugging | General and turn off Step over Property operators (Managed Only).

  1. Debug your poject.
  2. Put a break point on the call to Rhyous.StringLibrary you would like to step into.
  3. Step into the call to Rhyous.StringLibrary.
    Once you have stepped into the call, you should see it’s source.
    Continue stepping into or over or whatever you would like.
    Once you are in the source, you can add breakpoints.
    Note: If you know how to add a break point without first stepping into the project, let me know.

You should now be easily debugging your NuGet package.

Why long emails should be avoided as a Dev Lead

I keep failing to a avoid a common mistake as a leader. Sending long emails. It seems so easy. For whatever reason, as the dev lead, I cannot talk to a person face-to-face so I write a long email.

I could spend time talking about why email is bad, or I could show you how emails make people feel by showing you an email dialogue.

Why long emails should be avoided:

Dev Lead: I’m being a good mentor. Write a nice long email that will help the team grow on a subject A, that includes tons of info on Subject A, including its 5 benefits. I send this email to Dev1 and CC the other two members of my team.
Feels good about his leadership.

Dev 1: What the dev thinks: Uh, oh. The dev lead is having a hissy fit again. Looks like he is pissed at something I did. What a jerk.
Feels angry.

Dev 2: Oh no. I have no idea what the dev lead is talking about. Do I know my stuff? Googles and tries to learn what the dev lead is talking about.
Feels shamed.

Dev 3: Ugh! Why is he trying to teach me crap I already know.
Feels patronized.

Manager: Hey, the team didn’t appreciate that email.

Dev Lead: Feels like a poor leader.

Manager: Feels like he is losing his team.

Why it would have happened better face-to-face:

Dev Lead: Hey devs. I want to discuss subject A. What do you know about it already?

Dev 1: I’ve used it before

Dev 2: Stays silent.

Dev 3: I know all about Subject A.

Dev Lead: OK, Dev 3, tell us about subject A.

Dev 3: Gives four excellent points about subject A. One of them the dev lead didn’t know.

Dev Lead: Adds two points about subject A that Dev 3 didn’t know. Changes his list from 5 to 6 adding the one item Dev 3 did knew.
Feels impressed by Dev 3.

Dev 1: Feels growth.

Dev 2: Feels good to be introduced to a new subject.

Dev 3: Impressed that the dev lead let him educate the team.
Feels more respect for dev lead. Also notes that the Dev Lead knew things he didn’t and thinks he should listen more.

Manager: Feels good about the team.

It is all about the feelings, and there is something about face-to-face team interaction that leads to good feelings and something about long emails that always leads to bad feelings.

So, if you look at the face-to-face interaction, you can see that it all started with a short question. You could simulate this in a short email:

Dev Lead: Who can give me all the benefits of Subject A using only the knowledge in your head. No browser search allowed until after you respond.

Dev 1: Responds with the single most common benefit if subject A.

Dev 2: Doesn’t respond.

Dev 3: Responds with four items, one that the dev lead didn’t now about.

Dev Lead: Interesting. Here are the items that the team responded with. I added two more benefits for a total of 6. Should we use subject A to get those 6 benefits in our project?

Now imaging the response was crickets.

Dev Lead: Who can give me all the benefits of Subject A.

Dev 1: Doesn’t respond.

Dev 2: Doesn’t respond.

Dev 3: Responds with one item.

Dev Lead: Subject A is interesting and important to our project. I am going to create a quick training on it.

Dev Lead: Writes a doc on it and sends it to the team.

Team: Feels good to learn something new.

Manager: Feels like the team is running itself.

Tips

  1. Keep emails short.
  2. Use many short emails.
  3. Ask questions, preferable one-liners:
    1. Start by asking your team what they already know first.
    2. Ask follow-up questions second
  4. Compile responses into a bulleted list
    1. Add to the list if you can
    2. Ask questions about the list
  5. Thank the team

I am going to put these tips into practice next time I feel like sending a long email.

Code Review – Quick Reference

This is a simple check-list to make code reviews more valuable. Simply check these rules.

Download a single page word document: Code Review Cheat Sheet

Does the code follow the 10/100 Rule?

This is a quick check rule that isn’t extremely rigid. See the 10/100 rule of code

Method has less than 10 lines

Is the method that was added or changed 10 lines or less? (There are always exceptions such as Algorithms)

100

Is the class 100 lines or less?
Note: Model classes should have zero functions closer to 20 lines. Logic classes should be sub-100 lines.

Is the code S.O.L.I.D.

S.O.L.I.D. is an acronym. See this link: https://en.wikipedia.org/wiki/SOLID

Single Responsibility Principal

Does each class have a single responsibility? Does each method have a single responsibility?
Is this the only class that has this responsibility? (No duplicate code or D.R.Y. (Don’t Repeat Yourself)

Open/Closed Principle

Can you extend the functionality without modifying this code? Config, Plugins, event registration, etc.
Is there configuration is this code? If so, extract it. Configuration does not belong in code.

Liskov substitution principal

Is inheritance used? If so, does the child type cause issues the parent type wouldn’t cause?

Interface segregation principle

Does the code use interface-based design?
Are the interfaces small?
Are all parts of the interface implementations without throwing a NotImplementedException?

Dependency inversion principle

Does the code reference only interfaces and abstractions?
Note: If new code references concrete classes with complex methods, it is coded wrong.

Is the code Unit Tested

99% coverage

Is the Code 99% covered? Is code not covered marked with the ExcludeFromCodeCoverageAttribute?

Parameter Value Tests for methods with parameters

Are all parameter values that could cause different behavior covered?
See these links:
Unit testing with Parameter Value Coverage (PVC)
Parameter Value Coverage by type

Naming things

Typos

Are your names typo free?

Naming convention

Do your file names, class names, method names, variable names match existing naming conventions?

Big O

Do you have any glaringly obvious Big O problems? n or n2 vs when it could be constant or log n.
See: https://en.wikipedia.org/wiki/Big_O_notation

Parameter Value Coverage by Type

This article is a reference to Unit Testing with Parameter Value Coverage (PVC).

Primitive or Value Types

See this reference.

Short Name.NET ClassTypeWidthRange (bits)
byteByteUnsigned integer80 to 255
sbyteSByteSigned integer8-128 to 127
intInt32Signed integer32-2,147,483,648 to 2,147,483,647
uintUInt32Unsigned integer320 to 4294967295
shortInt16Signed integer16-32,768 to 32,767
ushortUInt16Unsigned integer160 to 65535
longInt64Signed integer64-9223372036854775808 to 9223372036854775807
ulongUInt64Unsigned integer640 to 18446744073709551615
floatSingleSingle-precision floating point type32-3.402823e38 to 3.402823e38
doubleDoubleDouble-precision floating point type64-1.79769313486232e308 to 1.79769313486232e308
charCharA single Unicode character16Unicode symbols used in text
boolBooleanLogical Boolean type8True or false
objectObjectBase type of all other types
stringStringA sequence of characters
decimalDecimalPrecise fractional or integral type that can represent decimal numbers with 29 significant digits128±1.0 × 10e−28 to ±7.9 × 10e28

byte

  1. Zero, 0, which is also byte.MinValue.
  2. A positive byte between 0 and 255.
  3. byte.MaxValue or 255

sbyte

  1. Zero, 0, which is also sbyte.MinValue.
  2. A positive sbyte between 0 and 127.
  3. A negative sbyte between -128 and 0.
  4. sbyte.MaxValue or 127
  5. sbyte.MinValue or -128

int

  1. A positive int between 0 and 2,147,483,647
  2. A negative int between -2,147,483,648 and 0
  3. Zero, 0
  4. int.MaxValue or 2,147,483,647
  5. int.MinValue or -2,147,483,648

uint

  1. Zero, 0, which is also uint .MinValue.
  2. A positive uint between 0 and 4,294,967,295.
  3. uint .MaxValue or 4,294,967,295

short

  1. A positive short between 0 and 32,767
  2. A negative short between -32,768 and 0
  3. Zero, 0
  4. short.MaxValue or 32,767
  5. short.MinValue or -32,768

ushort

  1. Zero, 0, which is also ushort .MinValue.
  2. A positive ushort, such as 1 through 65,535.
  3. ushort.MaxValue or 65,535

long

  1. A positive long between 0 and 9,223,372,036,854,775,807
  2. A negative long between -9,223,372,036,854,775,808 and 0
  3. Zero, 0
  4. long.MaxValue or 9,223,372,036,854,775,807
  5. long.MinValue or -9,223,372,036,854,775,808

ulong

  1. Zero, 0, which is also ulong.MinValue.
  2. A positive ulong between 0 and 18,446,744,073,709,551,615.
  3. ulong.MaxValue or 18,446,744,073,709,551,615

float

  1. A positive float between 0 and 3.402823E+38
    1. Note: This includes the float.Epsilon, but you could test double.Epsilon separately
  2. A negative float between -3.402823E+38 and 0
  3. Zero, 0.0
  4. float.MaxValue or 3.402823E+38
  5. float.MinValue or -3.402823E+38
  6. float.NaN
  7. float.PositiveInfinity
  8. float.NegativeInfinity

double

  1. A positive double between 0 and 1.79769313486232E+308
    1. Note: This includes the double.Epsilon, but you could test double.Epsilon separately
  2. A negative double between -1.79769313486232E+308 and 0
  3. Zero, 0.0
  4. double.MaxValue or 1.79769313486232E+308
  5. double.MinValue or -1.79769313486232E+308
  6. double.NaN
  7. double.PositiveInfinity
  8. double.NegativeInfinity

decimal

  1. A positive double between 0 and 79,228,162,514,264,337,593,543,950,335
  2. A negative double between -79,228,162,514,264,337,593,543,950,335 and 0
  3. Zero, 0
  4. double.MaxValue or 79,228,162,514,264,337,593,543,950,335
  5. double.MinValue or -79,228,162,514,264,337,593,543,950,335

string

  1. A null string
  2. An empty string, String.Empty, or “”
  3. One or more spaces ” “
  4. One or more tabs ” “
  5. A new line or Environment.NewLine
  6. A valid string.
  7. An invalid or junk string
  8. A string with many special characters: `~!@#$%^&*()_-+=,.<>/\?[]{}|
  9. Unicode characters such as Chinese
  10. An long string, over 256 characters, or even 1 million characters.
  11. (Occasionally) Case sensitivity. For example, for string comparisons, case sensitivity of a string is a required Parameter Value Coverage test.

Struct

  1. It is impossible to know. You need to define this per struct you create. For example, if your struct is a point with int values X and Y, then it is simply the int list above twice, once for X and once for Y.

Enum

  1. Any of the enums.
  2. You may need to do each of the enums, depending on how your enum is used.

Class or Reference Types

Class Object

Objects that are defined with the class keyword need the following tested:

  1. Null (This might go away or become optional in .NET 4.8)
  2. Instantiated
  3. Class properties can be primitive or value types, reference types, etc., and may need to be tested according to the type of the property.

Array, List, Dictionary, and other collections

Array, List, Collection

  1. Null
  2. Empty (instantiated with no items)
  3. Not empty but values of array are tested according to the value type. For example, an int[] would need to have the values tested in the ways listed above for int.
    1. Pay attention to how the code you are testing uses teh items in an array or list. If the items are objects, do you need to check if the list has a null item in the list?

Dictionary

  1. Null
  2. Empty (instantiated with no items)
  3. Key exists
  4. Key doesn’t exist
  5. Value at key is tested according to its value type. For example, a Dictionary<string, int> would need to have the values tested in the ways listed above for int.