Archive for the ‘Open Source’ Category.

We are already in the age of Microlibraries

Early developers spoke of the ability to create small building blocks of code, and to reuse code over and over. This has happened. The proof is seen in many languages. In C# (dotnet), NuGet packages are used over and over again. In Javascript, npm loads thousands of libraries for any given web project.

However, there has been another move in this area that is beneficial that many people haven’t really taken the time to define. This is the move to microlibraries.

What is a Microlibrary?

A microlibrary is different from past libraries because they are:

  1. Smaller libraries that encompass less
  2. Larger libraries have been broken up and multiple microlibraries are now replacing a prior large library.

What you are seeing is S.O.L.I.D. principles applied to libraries. Many libraries broke the S in solid by having multiple responsibilities. However, the owners of many open source libraries have noticed this and have split their libraries into responsibilities.

Examples of Microlibraries

There are plenty of examples in Javascript and other languages, but the idea of microlibries can be best described by looking at dotnet core. Microsoft has adopted the idea of microlibraries, though I doubt they use the term yet. No longer does dotnet include everything, as it did with .Net Framework. Instead, the many pieces of dotnet are now microlibries that can be consumed from NuGet.

  • See https://github.com/dotnet and you will find well over two hundred separate repositories. Never before has dotnet been so decoupled. This is a glowing example of microlibraries.

For a personal example, I have created the Rhyous libraries that you can find on NuGet. I don’t have one giant Rhyous NuGet package. I have many Microlibaries.

  • Rhyous.Collections
  • Rhyous.EasyCsv
  • Rhyous.EasyXml
  • Rhyous.StringLibrary
  • Rhyous.SimpleArgs
  • Rhyous.SimplePluginLoader
  • Rhyous.SimplePluginLoader.Autofac (notice that this is an add-on to Rhyous.SimplePluginLoader, but is still separate as Autofac integration is a separate concern.)
  • etc . . .

We’ve always had libraries, what changed?

The move to microlibraries has been occurring for well over a decade. It just appears that nobody has put a name to it. The greatest enabler of microlibraries has been:

  1. The tooling around package management.
  2. The tooling around continuous delivery. Automated check-in, code-reviews, build, test, deploy, publish.

Package Management

C# has NuGet, javascript has npm, Java has maven. The ability to easily create, publish, find, and consume a package has made microlibraries possible.

Continuous Delivery

Microlibraries need to be published to package management systems. As the tooling has improved, the ability to automate this has simplified. Microsoft has Azure DevOps, GitHub (also now Microsoft) has it’s actions, and Appveyor also makes it easy. Not to mention many of the these tools provide the tooling free to open source projects.

Continuous Delivery as Code has become the norm. Even the most simple of open source projects (most of mine) can have an automated process for check-in, code review, build, test, and publishing a package with very minimal work. And that work is checked in to a build file (AppVeyor and Azure DevOps both use yaml) which can easily be copied and pasted to other small projects with only the strings (names and paths) of the build file changing.

The benefits of Microlibraries

The smaller the libraries, the easier they are to work with. Also, the easier they are to be complete and rarely touched. Smaller means every part becomes easier. Many smaller pieces means we more easily see the commonalities. This ability to see commonalities is what lead to the package management systems which lead to further shrinking libraries as management their inclusion in new projects became easier.

Code

Less code is easier to maintain. It is easier to refactor and change. Since the project is far smaller, the idea of refactoring to be solid and testable code is less daunting and soon the code is refactored, more testable. An example of this is with AutoMapper. They used to have everything as untestable static classes, but recently in a major release, they replaced their statics and now support dependency injection, making the library more solid and testable.

Adding new features becomes much easier with small projects.

Build

Build is smaller and easier. The entire build and test process is very small, allowing for feedback within minutes on code changes (i.e. pull requests).

A one-step build (one of the top items of the Joel test) is so much easier with a microlibrary.

Build decoupling. Have you ever heard of those builds that take an hour. Some are worse, and take four hours or even a day. Microlibraries solved this. Any given build pulls a lot of already built code, and builds only the minimal code needed. Your final application may not be a microlibary, but it may encompass many microlibraries. Your build should take minutes because instead of building everything every time, it uses already built microlibraries and only builds the code for you final application. If you still have four hour builds, you might want to take a look at how you can split your build into microlibraries.

Tests

Tests are less and easier. The more tests, the less bugs. The more likely a new change doesn’t cause regression.

Again, your final application doesn’t need to run all the tests for each microlibrary. The microlibrary’s own build and test process does that. That means you can focus your tests on your application, which is faster.

Learning / Onboarding

It is far easier to learn a small amount of code. It is easier to understand even without good comments and excellent test coverage. If those are missing, adding comments and unit test code coverage is not as overwhelming.

New developers can usually pick up a microlibrary and get the code, get it building, and tests running within an hour. Onboarded developers can be productive in the first week.

Conclusion

Microlibraries are not just the future, the are already the present. The benefits are so great, that any other method of releasing code seems antiquated.

 

 

 

Splitting sentences in C# using Stanford.NLP

So I need to break some sentences up. I have a pretty cool regex that does this, however, I want to try out Stanford.NLP for this. Let’s check it out.

  1. Create a Visual Studio C# project.
    I chose a New Console Project and named it SentenceSplitter.
  2. Right-click on the project and choose “Manage NuGet Packages.
  3. Add the Stanford.NLP.CoreNLP nuget package.
  4. Add the following code to Program.cs (This is a variation of the code provide here: http://sergey-tihon.github.io/Stanford.NLP.NET/StanfordCoreNLP.html
    using edu.stanford.nlp.ling;
    using edu.stanford.nlp.pipeline;
    using java.util;
    using System;
    using System.IO;
    using Console = System.Console;
    
    namespace SentenceSplitter
    {
        class Program
        {
            static void Main(string[] args)
            {
                // Path to the folder with models extracted from `stanford-corenlp-3.4-models.jar`
                var jarRoot = @"stanford-corenlp-3.4-models\";
    
                const string text = "I went or a run. Then I went to work. I had a good lunch meeting with a friend name John Jr. The commute home was pretty good.";
    
                // Annotation pipeline configuration
                var props = new Properties();
                props.setProperty("annotators", "tokenize, ssplit, pos, lemma, ner, parse, dcoref");
                props.setProperty("sutime.binders", "0");
    
                // We should change current directory, so StanfordCoreNLP could find all the model files automatically 
                var curDir = Environment.CurrentDirectory;
                Directory.SetCurrentDirectory(jarRoot);
                var pipeline = new StanfordCoreNLP(props);
                Directory.SetCurrentDirectory(curDir);
    
                // Annotation
                var annotation = new Annotation(text);
                pipeline.annotate(annotation);
    
                // these are all the sentences in this document
                // a CoreMap is essentially a Map that uses class objects as keys and has values with custom types
                var sentences = annotation.get(typeof(CoreAnnotations.SentencesAnnotation));
                if (sentences == null)
                {
                    return;
                }
                foreach (Annotation sentence in sentences as ArrayList)
                {
                    Console.WriteLine(sentence);
                }
            }
        }
    }
    

    Warning! If you try to run here, you will get the following exception: Unrecoverable error while loading a tagger model

    java.lang.RuntimeException was unhandled
      HResult=-2146233088
      Message=edu.stanford.nlp.io.RuntimeIOException: Unrecoverable error while loading a tagger model
      Source=stanford-corenlp-3.4
      StackTrace:
           at edu.stanford.nlp.pipeline.StanfordCoreNLP.4.create()
           at edu.stanford.nlp.pipeline.AnnotatorPool.get(String name)
           at edu.stanford.nlp.pipeline.StanfordCoreNLP.construct(Properties A_1, Boolean A_2)
           at edu.stanford.nlp.pipeline.StanfordCoreNLP..ctor(Properties props, Boolean enforceRequirements)
           at edu.stanford.nlp.pipeline.StanfordCoreNLP..ctor(Properties props)
           at SentenceSplitter.Program.Main(String[] args) in c:\Users\jbarneck\Documents\Projects\NLP\SentenceSplitter\SentenceSplitter\Program.cs:line 20
           at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
           at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
           at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
           at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
           at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
           at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
           at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
           at System.Threading.ThreadHelper.ThreadStart()
      InnerException: edu.stanford.nlp.io.RuntimeIOException
           HResult=-2146233088
           Message=Unrecoverable error while loading a tagger model
           Source=stanford-corenlp-3.4
           StackTrace:
                at edu.stanford.nlp.tagger.maxent.MaxentTagger.readModelAndInit(Properties config, String modelFileOrUrl, Boolean printLoading)
                at edu.stanford.nlp.tagger.maxent.MaxentTagger..ctor(String modelFile, Properties config, Boolean printLoading)
                at edu.stanford.nlp.tagger.maxent.MaxentTagger..ctor(String modelFile)
                at edu.stanford.nlp.pipeline.POSTaggerAnnotator.loadModel(String A_0, Boolean A_1)
                at edu.stanford.nlp.pipeline.POSTaggerAnnotator..ctor(String annotatorName, Properties props)
                at edu.stanford.nlp.pipeline.StanfordCoreNLP.4.create()
           InnerException: java.io.IOException
                HResult=-2146233088
                Message=Unable to resolve "edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger" as either class path, filename or URL
                Source=stanford-corenlp-3.4
                StackTrace:
                     at edu.stanford.nlp.io.IOUtils.getInputStreamFromURLOrClasspathOrFileSystem(String textFileOrUrl)
                     at edu.stanford.nlp.tagger.maxent.MaxentTagger.readModelAndInit(Properties config, String modelFileOrUrl, Boolean printLoading)
                InnerException: 
    
    
  5. Download the stanford-corenlp-full-3.4.x.zip file from here: http://nlp.stanford.edu/software/corenlp.shtml#Download
  6. Extract the stanford-corenlp-full-2014-6-16.x.zip.
    Note: Over time, as new versions come out, make sure the version you download matches the version of your NuGet package.
  7. Extract the stanford-corenlp-3.4-models.jar file to stanford-corenlp-3.4-models.
    I used 7zip to extract the jar file.
  8. Copy the stanford-corenlp-3.4-models folder to your Visual Studio project files.
    Note: This is one way to include the jar file in your project. Other ways might be a copy action or another good way would be to use an app.config appSetting. I chose this way because it makes all my files part of the project for this demo. I would probably use the app.config method in production.
  9. In Visual Studio, use ctrl + left click to  highlight the stanford-corenlp-3.4-models folder and all subfolders.
  10. Open Properties (Press F4), and change the namespace provider setting to false.
  11. In Visual Studio, use ctrl + left click to  highlight the files under the stanford-corenlp-3.4-models folder and all files in all subfolders.
  12. Open Properties (Press F4), and change the Build Action to Content and the Copy to Output Directory setting to Copy if newer.
  13. Run the code.

 

Note: At first I tried to just load the model file. That doesn’t work. I got an exception. I had to set the @jarpath as shown above. I needed to copy all the contents of the jar file.

Results

Notice that I through it curve ball by ending a sentence with Jr. It still figured it out.

I went or a run. Then I went to work. I had a good lunch meeting with a friend name John Jr. The commute home was pretty good.

However, I just tried this paragraph and it did NOT detect the break after the first sentence.

Exit Room A. Turn right. Go down the hall to the first door. Enter Room B.

I am pretty sure this second failure is due to the similarity in string with a legitimate first name, middle initial, last name.

Jared A. Barneck
Room A. Turn

Now the question is, how do I train it to not make such mistakes?

The two-clause BSD License

Here is the two-clause BSD License, sometimes called the FreeBSD License or the Simplified BSD License.

Copyright (c) <Year>, <Owner>
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
   list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
   this list of conditions and the following disclaimer in the documentation
   and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Utah Open Source Conference 2012

How to compile WinNFSd with Visual Studio?

Recently I needed an NFS server on Windows, preferably written in C++. A long time ago I had used WinNFSd and sure enough the project still exists on Sourceforge, though unfortunately it hasn’t been updated since 2005.

However, I found that someone had updated it here: https://github.com/noodle1983/winnfsd-nd

So the big question, how do you compile this on windows with Visual Studio?

Step 1 – Download and extract the WinNFSd source

  1. Go to https://sourceforge.net/projects/winnfsd and download the source.
    Note: You can alternately download the git hub source as it has an update you might like:
    https://github.com/noodle1983/winnfsd-nd
  2. Click the Zip button at the top of the page to download the source as a zip.
    Note: Alternately if you have git working already you can clone this repo.
  3. Extract the zip file to a directory.  Remember where you extracted it as we will copy the source files later.

Step 2 – Create a new Visual Studio solution

  1. In Visual Studio, go to File | New | Project.
  2. Select Other Languages | Visual C++ | Empty Project.
    Note: Depending on your Visual Studio configuration you may Visual C++ in a different place.
  3. Name the solution WinNFSd.
  4. Click Ok.

Step 3 – Add the WinNFSd files to your solution

  1. In Visual Studio, right-click on your Project and click Open Folder in Windows Explorer.
  2. Create a new folder to hold your source code.
    Note: I simply named my folder src.
  3. Copy the source you extracted in Step 1 into the src directory.
  4. Highlight all the files in the src directory.
  5. Drag the files into Visual Studio and drop them on your project.
Note: If you try to build now, you will get 22 errors in debug mode and maybe 17 in release mode.

Step 4 – Configure the project properties

  1. In Visual Studio, right-click on your project and choose Properties.
    Note: The Configuration should say Active(Debug) currently.
  2. Go to Configuration Properties | Linker | Input.
  3. Add ws2_32.lib to the Additional Dependencies.
  4. Change the Configuration to Release and add ws2_32.lib for release as well.

Step 5 – Handle the Visual Studio C++ Runtime

If you were to compile now, and try to run your project on a different machine (not the one running Visual Studio) you would likely get an error due to a missing dll.  Here is the error you will likely receive.

The program can’t start because MSVCR100.dll is missing from your computer. Try reinstalling the program to fix this problem.

I am not going to explain the solution again here, because it is all documented here:
Avoiding the MSVCR100.dll or MSVCR100D.dll is missing error

Choose the best of the three solutions for you from the link above.

Note: For this single file exe, I prefer the statically linked option.

Step 6 – Build WinNFSd

  1. You should now be able to click Build | Build Solution and it should build.

You should be able to test both debug and release.

Note: I received 37 warnings, which would be nice to resolve, but I wouldn’t worry too much about them.

 

Using Open Source in Proprietary Software

I was consulted by a Lawyer about open source licenses and since it is not that difficult, I thought I would share some of the simplicity with you.

Open Source software might seem like a nightmare only lawyers can understand. But this is not actually true. Determining if an Open Source library is usable in a commercial product is can become a simple task.

I am going to make this easy for you.

Think of Open source license as you would a traffic light. Green is good to go, yellow is proceed with caution but be ready to stop, Red is stop but with time may become green.

  • Green licenses
    BSD/MIT/Apache licenses (and a few others)
  • Yellow Licenses
    LGPL, Sun/Oracle
  • Red Licenses
    GPL, 3rd Party Commercial

Green licenses

These are close to 100% free but not quite…usually the only restrictions are simple things…like things you should not do but wouldn’t do anyway:

• “don’t remove license from the code files”
• “don’t use the name of the author or their place of business to promote your product”.

Which since we don’t do anyway, the code is essentially 100% free.

These licenses are usually short and don’t require a lawyer as anyone can pretty much read and understand them.

You don’t even have to include these licenses in any “pop-up” license agreements. As long as internally where we store these files the license remains in the source, we are fine.

You usually don’t even have to provide a mirror for this code and you can ignore requests for the source.

Yellow Licenses

LGPL or Oracle isn’t as bad, but they can be a pain. Usually you are free to use these. You cannot link using the source or static link to the library, or you are essentially GPL (a Red license) but if you dynamically link to an LGPL library, then you can be free to go. Any changes to the library can be made, but you have to treat them as GPL.

So basically if you ship their software using their installer or a loose file but you don’t change the file, you are likely good to go with minimal effort.

You have to include these licenses often times in a popup or when your own license is used.

Red Licenses

GPL
GPL means that anything you write that touches the GPL code must be licensed with GPL as well. Also, if you distribute GPL code, you must provide a “mirror” or be able to provide the GPL code when asked for it. You must also provide your own code (which is now GPL) when asked for it.

People often call GPL a virus as anything that touches it is infected by the GPL as well. However, from another point of view, someone has spent a lot of time giving their code/knowledge to the world and they deserve the right to keep this knowledge free and it is just a way to prevent someone from stealing their knowledge.

Your company can use GPL just fine if two things are true: 1) The feature is isolated and 2) the feature is not really a “differentiator” in our market. For example, a TFPT service. It really wouldn’t matter if a commercial company use GPL because TFTP is isolated to its own service and TFTP is ubiquitous and there are hundreds of different TFPT servers out there.

It may also be possible to that GPL won’t hurt you if you are first to market. If you release code and become a dominate player or market share leader, it might not matter that others can see your code.
3rd Party Commercial
3rd Party Commercial licenses absolutely cannot be used unless the company is willing to sell us the license and we are willing to pay the license fee. There may also be other agreements made in the license negotiation.

You have to include these licenses often times in a popup.

Track your licenses

It is a simple task to track the software you use and the license you use.  Create three lists, Green, Yellow, and Red lists and life will be easy for you. You may only need to review a license once. Once you have places a license in the green list, any software that uses the license is in the green list.

It is also important to keep a copy of the software you use and the license you used to obtain this software. This can be important to do. Imagine if your software uses a BSD licensed tool that is re-licensed using GPL in the next version. You can keep the old BSD licensed version and continue to use it.

Conclusion

Any license but the 3rd party Commercial licenses are possibilities all the time. Only 3rd Party Commercial licenses may be denied you but usually for enough money they are available.

Be thorough and be careful.

Just be aware of the license and know when to use them an when not to.

See a previous post of mine:

Differences between the BSD/FreeBSD Copyrights and the GNU Public License (GPL)

DISCLAIMER

I am not a lawyer. I am not responsible in any way for the misuse of a license based on this post, even if the post is has some piece of data that is blatantly wrong. It is the responsibility of the user of licensed or copyrighted software to make sure the license agreement or copyright is adhered to properly.

Cool Open Source Games you should contribute to (from Shinnok’s Rants)

Open Source games

Cool Open Source Games you should contribute to

I found this site about open source games. I didn’t know about most of these.

Time to download and play one.

If you have already played one, please comment and let me know how you liked the game.

Too bad the only one I have played is Extreme Tux Racer.

The 8 Types of Technical Documentation and Why Each Is Important

Technical documentation is critical to the success of any software. However, most creators of software struggle to provide adequate documentation for their product. Rare is the software that is praised for its documentation. When documentation is praised, it is often only praised for having some documentation, which is more than most, but in reality documentation is usually still inadequate.

So what constitutes adequate documentation? Well, if a user wants to do something with your software and the documentation helps them succeed in a timely manner, then the documentation is adequate. However, accomplishing this is not as easy as it sounds.

Why most companies fail to document properly

Most companies do not document their product thoroughly for a few reasons.

  • Lack of a defined list of all types of documentation
  • Lack of understanding of each type of documentation
  • Documentation is not made a priority and lacks of funding

Lack of a defined list of all types of documentation

Many cannot name more than one or two forms of documentation. To be successfully with documentation, a software company must first enumerate the types of documentation. Then it must learn about each type of documentation and understand the role that each type of documentation plays. It is also critical to understand the different target audiences each type has. Also, what are the common mistakes made when trying to create each type of documentation so these mistakes can be avoided.

Attempts are made to document software in different ways. However, because a complete documentation set is not defined, success is nearly impossible. To make matters worse, there is little to no reporting or visibility into the level of documentation a given piece of software has. I have never encountered software that has reached a 100% documentation level.

In order to succeed there must be an understanding of the types of documentation.

  1. Step-by-Step Walk-thrus – Also called Guides, How to’s, or Examples, Quick Start Guides
  2. Product feature documentation – This is lists all the features and settings without really any real world examples. Often the help button inside the software points to sections of this document.
  3. Troubleshooting Documentation – What to do when a failure occurs. Where are the logs and how to read them. How to turn on or increase logging and debugging.
  4. Knowledge-base (Problem, Cause, Resolution), Frequently Asked Questions (FAQ), and Forums
  5. Code, API, or SDK Documentation
  6. Internal Development Documentation – Such as code and development documentation, internal only features use by developers and/or testers, architecture documentation (Note: For open source projects this information is usually public)
  7. Real life customer implementations – Examples of how a company has a product implemented in real life
  8. Marketing documentation – Basic over views of the value the software has for the company, ROI claims, general feature lists, costs, etc…

The worst documentation of all is of course the absence of documentation. However, most software companies are unaware that there are entire areas of documentation that are lacking. To have complete documentation you must provide it in all of these areas.

Lack of understanding of each type of documentation

Since most software companies are unaware of the list above, it makes sense that they don’t understand the items on the list. This is why they have no direction and their documentation is a sporadic combination of the different documentation types, never fully succeeding to accomplish the primary goal of documentation, which is to enable the reader to succeed.

In order to create excellent documentation, a full understanding of each type of documentation is requisite. Without this understanding, documentation your documentation will continue to be lacking.

The lack of understanding also leads to assumptions that are not true. Some think that if they try to document every setting their software has they will have complete documentation. Usually when this is done, there is so much effort put into this that providing a simple example is forgotten. Often I hear this question:

Why would an example be needed, every feature is documented?

I would answer this question as follows:

Information overload. Now there is so much documentation in one white paper that someone who wants to do something simple is unsure that it is simple. They don’t know which features they must setup and which are unnecessary or should remain as defaults.

I often find this with Open Source documentation and unfortunately when a user asks for an example they are often rudely told to “Read the Manual” or RTM. However, the manual is usually hundreds of pages and they probably need to read one page of the manual but just don’t know where to start.

If have seen documentation using only examples as well. However, when an attempt is made to deviate from the examples, there is nothing left in the documentation to provide the guidance necessary to succeed.

Some documentation is better defined, such as that created from the results of support calls, forums, or mailing lists. Because this type of documentation is completely reactionary, this is one area of documentation that is better defined. The documentation is created after a problem is experienced and has to be dealt with. However, once created, it exists to benefit others. As this documentation type is better defined you might not be surprised to know that it has its own acronym: KCS or Knowledge Centered Support.

The goal of this article is to raise awareness of all types of technical documentation and make them all as well-defined as support documentation.

Documentation is not made a priority and lacks of funding

Investing in documentation is expensive. But it is usually and expense that pays off. If an analyst has to choose between two competing software applications and one is well documented and one is not, the well documented software application is likely to be chosen. Many organizations fail to see the ROI in documentation and therefore choose not to invest.

It is obviously that lack of funding for documentation is an industry wide phenomenon. While technical writing has been around since even before software, a standard for documentation whether it be creating documentation, updating documentation, managing documentation, and reporting on documentation has yet to formally exist. However, I did find this link, which shows I am not the only one who has identified this problem: http://www.hci.com.au/iso/

So lets get back to our list. Below I will go through each type of documentation and provide some information on it.

Type 1 – Step-by-step Walk-thrus – Also called Guides, How to’s, or Examples, Quick Start Guides

Description

This type of documentation is nothing more than actions that the reader will take to accomplish something with your software. This documentation, when done right, could be followed by the most computer illiterate. If they read and follow each step, even if they have no idea what they are doing, they should succeed.

You may also want to read: Your project’s ‘Getting Started’ tutorial sucks – Why time to success matters

Role

To provide the most common, most tested, most successful, and best overall example of how to accomplish some particular task from start to finish with your software.

Audience

Most commonly, trainees and new or evaluation users. However, anyone who wants to achieve the results the step-by-step guide leads to is included. This is most often, but is not limited to, users of your software. It includes deployment engineers, configuration specialists, support engineers, and demo or sales engineers.

Common Article Names

  • Quick Start Guide
  • Step-by-step Guide for setting up “Software X”
  • How to configure “Some Feature” of “Software X”

Common mistakes

There are many common mistakes

  • Not clearly defining the starting point of the walk-thru. Think of the starting point of a software that installs on Windows. What version of Windows, what other software must be installed, etc…
  • Defining the starting point clearly, but using a starting point most people don’t know how to get to. For example, you starting point should probably not say “have SQL Server installed and a database created with credentials” without providing steps.
  • Assuming the reader knows how to accomplish a task, so the documentation simply states to “do task x” instead of walking the reader through doing the task.
  • Skipping steps or forgetting steps.
  • The development department changes the steps just before release but the documentation is not updated to match.
  • Trying to simultaneously provide Product Feature Documentation in the middle of your steps. A link or note is acceptable for steps or settings that customers commonly customize.
  • Trying to provide comprehensive troubleshooting documentation after each step. It is great to have a link or a reference to troubleshooting documentation but it shouldn’t interfere with the walk-thru.
  • Only creating step-by-step guides for a couple common features of your software.
  • Failing to add documentation after use. For example, when a consultant, support engineer, or other employee struggles to set up a not-well-documented feature and once successful, they still don’t document it.

Type 2 – Product feature documentation

Description

This type of documentation is a description of every feature and setting. What it is used for, when and why one would use the feature or setting.

Role

This is for users who need to stray from the common walk-thrus and need to know what alternate and uncommon settings are used for so they can determine which they need in their particular environment.

Audience

Any customer/user who needs more than the most common features. Your own support representatives and architect or professional services teams. Consultants who recommend your product or are trusted to determine if your product meets a feature set for potential customers/users.

Common Article Names

  • The Software X Handbook
  • Software X: The Complete Reference
  • Understanding Feature Y of Software X

Common mistakes

  • Burying the features in other documentation, such as walk-thrus.
  • Not including at least a comment about when the feature would be used.
  • Not being aware of the features your customers/users are aware of and using. There are lots of “unintended features” and you should capture them in documentation.
  • Not letting customers contribute to this documentation in some way, even if it is just comments (this is the best way to solve the above issue, too).

Type 3 – Troubleshooting documentation

Description

This documentation describes steps to diagnose problems. It includes information on logs files. It includes information on the behind the scenes business your software is doing, such as process/thread work, file or data interaction, etc…

If the users tries to do some task with your software and it fails, to them, a single task failed. However, to fix it, one might need to know that behind the scenes ten different processes occurred. It is important to be able to diagnose which background processes worked and pinpoint which one failed, so you don’t troubleshoot all ten background processes when only maybe the seventh is the problem.

Role

To help customers/users get pasts unexpected issues and to help support engineers diagnose issues. These don’t have to always be public, but should be in the hands of your support engineers.

Audience

This is for support engineers more than customers, though the more experienced and “get your hands dirty” customers/users will use it. Engineers who do on site installation or on site configuration may need this information for when they run into bumps.

Common Article Names

  • Feature X: The complete troubleshoot guide
  • Troubleshooting Feature X

Common mistakes

  • Confusing “Problem, Cause, Resolution” documentation (also called Knowledge Base articles) with Troubleshooting documentation.
  • Not creating this documentation because you assume product feature documentation covers this. It doesn’t.
  • Providing this documentation but not providing complete troubleshooting steps for whatever reason. Especially if troubleshooting is done with 3rd party software and outside your own product it is assumed outside the scope when it is not. For example, a product that requires a DNS server, should provide steps to make sure that a DNS server is configured as the product expects. You may not have to write such documentation if the 3rd party vendor has some, but you should link to/reference it in your own documentation.
  • Writing documents that have lists of “fixes to try”. This documentation should almost never include “fixes”, but instead should diagnose the issue or pinpoint the problem so precisely that the fix becomes obvious whether the fix currently exists or not.

Type 4 – Knowledge-base (Problem, Cause, Resolution)

Description

This documentation is most commonly the result of customer support tickets/cases. It lists a specific problem, a specific cause of the problem, and a single resolution to that problem. As mentioned early this is one of the more well-defined areas of documentation. Read more here about KCS or Knowledge Centered Support.

Role

To make it so an issue only has to be troubleshot and fixed once. Once an issue is fixed, the Problem, Cause, Resolution can be documented and the fix can be applied without troubleshooting when the same Problem and Cause occurs.

To keep knowledge in-house. Tech Support is a high turnover position so keeping knowledge in-house is not always the easiest task.

Audience

Customers who experience a problem. Support engineers or other employees to whom the problem is reported.

Common Article Names

Frequently Asked Questions or FAQ.

Common Issues.

Common mistakes

  • Providing multiple Problems, Causes, or fixes in the same article.
  • Providing a problem and a list of fixes with no way to determine which fix is the correct fix.
  • Having an article that recommends a fix even when a customer is not really having that problem.
  • Failing to provide a good search for the knowledge base articles.

Type 5 – Code, API, or SDK Documentation

Description

This documentation describes how others use your code or libraries to write add-ons, plugins, integration, or otherwise customize your application through code. Do not confuse this with Internal Development Documentation. This type is for external users or resellers or middle-ware companies.

Role

This documentation helps others code with your code and libraries. Software that a customer/user takes the time and expense to modify to fit their environment becomes “sticky”, meaning the customer/user is likely to be loyal.

Audience

Systems Analyst / Developers / Integration Engineers / Middle-ware companies / Resellers. Customers who need to extend your product to meet a business need. Or in an open source environment, how others can use your code to extend their own project.

Common Article Names

  • Software X SDK Documentation
  • Class or Function Reference for Software X API

Common mistakes

  • Providing zero documentation on this
  • Providing incorrect documentation about a function
  • Updating code but not updating the documentation
  • Deprecating code but not informing the consumer
  • Not providing the first type of documentation: Samples, walk-thrus, etc…

Type 6 – Internal Development Documentation

Description

This is used for internal developers continue future enhancements and otherwise maintain a piece of software.

Role

To help developers work with a piece of code. To overcome turnover so new developers can pick up code another developer created. To provide architecture and design of each piece of code. To give UML (usually the classes and their methods),

Audience

Internal developers. Sometimes support.

Common Article Names

There are really no common names, but usually these types of documentation are internal only.

Common mistakes

  • Not writing such documentation at all.
  • Not documentation all parts of the code: Classes, Functions, design and architecture, supported features, etc…

Type 7 – Real life customer implementations

Description

This is documentation about customers success stories. About how they implemented your software in their environment (which is usually as messed up as everyone else’s environments).

Role

To demonstrate that the software can be successful and has proven itself in real life customer environments.

Audience

Other customers / System Analysts / Internal Employees in charge of future enhancements and road maps

Common Article Names

  • Product X Success Stores
  • How Company Y succeeded with Product X

Common mistakes

  • Not providing any customer success stories.
  • Providing success stories from unhappy customers who when contacted, speak poorly of your product

Type 8 – Marketing documentation

Description

This is documentation that doesn’t really say much more than is needed to let a customer know about a software solution.

Role

To acquire more customers. To help potential customers determine features sets quickly.

Audience

Systems Analysts / Consultants / Sales Engineers / Evaluation customers.

Common Article Names

  • Product X
  • The Product X Feature Set
  • What Product X can do for you business

Common mistakes

I don’t know a lot of the mistakes made in this documentation type, as my exposure to marketing is limited. I almost forgot this documentation type.

  • Too complex, including information or overly complex images or diagrams that are hard to understand

Conclusion

Hopefully after reading this article, you have a greater understanding of documentation.

Now that you know all the types of documentation, there are other problems to address. How to write the documentation. How to choose the priority for writing these types of documentation. How to balance the cost of documentation against the opportunity cost of not having documentation.

Some day, I will also have to write a post on how to deal with “versioning” documentation including updating documentation when Software versions change. I think there is a market for a piece of software that does nothing but track documentation. Hopefully it is well documented. 🙂

Reading about Apache Axis2 Web Services

I have recently started reading an e-book from PACKT Publishing.

With the internet and internet-based offering being renamed to “The Cloud” which is just a fancy buzz word marketing came up with that they don’t even themselves know what it means, understanding web services, which is mostly what “The Cloud” will be based on is going to be key in a developer’s job security.

Being able to provide a cloud offering without licensing costs because you are using free software such as Apache, is nice too.

Which brings me back to this book on Apache Axis2 Web Services.

I’ll let you know if it is well worth the read.

Getting Java to work in Firefox 3.6 in FreeBSD or PC-BSD 8.2

Firefox 3.6 no longer works with the Java versions in PBIs. Instead the OpenJDK6 must be installed. However, as of writing this document there is not a PBI for OpenJDK6, so we must install it from the ports jail.

  1. Install ports on your system as follows: How to install ports on FreeBSD?
  2. Open the command prompt.
  3. su to root.
    $ su
    Password:
    #
  4. As root, get the latest ports.# portsnap fetch extract
  5. Install openjdk6.Note: There is an openjdk7 but it didn’t appear to have a web option yet, so hold off until it has one, use openjdk6.
    #
    #
    cd /usr/ports/java/openjdk6
    make WITH_WEB=yes BATCH=yes install
  6. Run this link command to link the library for the java plugin to your plugins directory.
    # ln -s /usr/local/openjdk6/jre/lib/IcedTeaPlugin.so ~/.mozilla/plugins/IcedTeaPlugin.so

Testing the Java plugin

You may want to verify that java is working in your Firefox install. You want to know two things:

  1. Is the plugin detected by Firefox?
  2. Does a java application work?
      1. Open Firefox. If Firefox is open already, close it, and re-open it.
      2. In the URL enter: about:pluginsYou should now see the IcedTea-Web Plugin (using IcedTea-Web 1.0.1).
      3. In Firefox, go to the following URL:
        http://www.java.com/en/download/testjava.jsp 
    1. Here are steps to make those two verifications. 

      Note; This is pretty much exactly like the handbook states.
      http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/desktop-browsers.html

Opening konsole and a Ports Jail konsole can be confusing on PC-BSD

If I open Ports Jail, then a regular konsole, the regular konsole gets the Ports Jail icon.  I can’t really tell the konsole sessions apart.

The same things happens if I do it the other way around.  If I open a regular konsole, then Ports Jail, the Ports Jail gets the konsole icon. Again, I can’t really tell the konsole sessions apart.

This causes me some confusion.  Which konsole am I running?  I have to take a moment to verify or close them both.

Solution: Opening the Ports Jail with a different konsole profile

One solution is to have the Ports Jail konsole session use its own konsole profile that uses a custom konsole profile.

Here is how I configured that:

  1. I created a LavenderOnBlack color scheme, which is a copy of the GreenOnBlack with Lavender instead of Green. Normally I use the GreenOnBlack color scheme.~/kde4/share/apps/konsole/LavenderOnBlack.colorscheme
    [Background]
    Color=0,0,0
    Transparency=false
    
    [BackgroundIntense]
    Color=0,0,0
    Transparency=false
    
    [Color0]
    Color=0,0,0
    Transparency=false
    
    [Color0Intense]
    Color=104,104,104
    Transparency=false
    
    [Color1]
    Color=250,75,75
    Transparency=false
    
    [Color1Intense]
    Color=255,84,84
    Transparency=false
    
    [Color2]
    Color=24,178,24
    Transparency=false
    
    [Color2Intense]
    Color=84,255,84
    Transparency=false
    
    [Color3]
    Color=178,104,24
    Transparency=false
    
    [Color3Intense]
    Color=255,255,84
    Transparency=false
    
    [Color4]
    Color=92,167,251
    Transparency=false
    
    [Color4Intense]
    Color=84,84,255
    Transparency=false
    
    [Color5]
    Color=225,30,225
    Transparency=false
    
    [Color5Intense]
    Color=255,84,255
    Transparency=false
    
    [Color6]
    Color=24,178,178
    Transparency=false
    
    [Color6Intense]
    Color=84,255,255
    Transparency=false
    
    [Color7]
    Color=178,178,178
    Transparency=false
    
    [Color7Intense]
    Color=255,255,255
    Transparency=false
    
    [Foreground]
    Color=192,0,192
    Transparency=false
    
    [ForegroundIntense]
    Bold=true
    Color=210,0,210
    Transparency=false
    
    [General]
    Description=LavenderOnBlack
    Opacity=0.92
    
  2. Next I created a copy of the shell.profile named PortsJail.profile.
    ~/kde4/share/apps/konsole/LavenderOnBlack.colorscheme

    [Appearance]
    ColorScheme=LavenderOnBlack
    
    [Cursor Options]
    CustomCursorColor=192,0,0
    UseCustomCursorColor=true
    
    [General]
    MenuIndex=6
    Name=PortsJail
    Parent=FALLBACK/
    RemoteTabTitleFormat=%h : %u
    TabBarMode=1
    TabBarPosition=1
    
    [Scrolling]
    HistoryMode=2
    
    [Terminal Features]
    BlinkingCursorEnabled=true
    
  3. I then edited the Ports Jail shortcut on the desktop to pass it the approprate TerminalOptions to use the PortsJail.profile. Here is the one line I changed.
    TerminalOptions=--profile /usr/home/jared/.kde4/share/apps/konsole/PortsJail.profile
    

Now whenever I open the ports Jail, I can easily tell it is the Ports Jail and not the regular konsole because it is using a LavenderOnBlack profile.

TortoiseSVN Addin for Visual Studio – Disappointing

I wanted TortoiseSVN for Visual Studio 2010, so I searched and I was so excited to find one here: http://tsvnaddin.codeplex.com/SourceControl/list/changesets

I wanted this to work so bad, but it didn’t.  It doesn’t support Visual Studio 2010.

There is a bug you can vote for here: http://tsvnaddin.codeplex.com/workitem/16368

However, I checked an there hasn’t been an update to this project in in almost a year.

I sent an email to the person who did all the commits and asked if the project was dead.  If he doesn’t respond, I have to assume the answer is yes.

Utah Open Source Conference 2010

Hey all,

The Utah Open Source Conference 2010 was pretty fun.  It was really my first open source conference.  Yes, I have been into open source for 10 years, specifically FreeBSD, but somehow I haven’t really attended the conferences.

I will probably attend conferences more often.

What was there about FreeBSD?

PC-BSD and the folks as iXSystems sent me with some swag.  Howard Logsdon helped me man the GUBUG booth.

There was a great presentation on FreebSD Jails given by Chris Edwards. (http://www.utosc.com/presentation/157/) Supposedly they are going to post a recording of the presentation. Until then, check it here: (http://wiki.zelut.org/doku.php/presentations:freebsd-jails)

Who else was there?

So Novell SUSE brought some nice laptop bags, and they were pretty good.  I have a nice Ogio laptop back pack, so I am giving this laptop bag to my wife.  It is just big enough to fit her 17″ HP laptop.

There was Fedora, Ubuntu, KDE, and GNOME.  There was a boot on XDMC and MythTV.

There was a very cool company there called Fusion-IO.  They have an awesome hard drive, though they are not cheap. 7k for the cheapest drive. But for some companies, it would be worth it.
When is the next Conference?

You can go here to see upcoming events: http://www.freebsd.org/events/events.html

There is the Meet BSD California 2010 conference November 6 and 7th.

To see when the next Utah Open Source Conference is you should go to the site and register so that when the date is announced, you will be informed.

Blogilo: Writing a post using Blogio, a KDE app on PC-BSD

Hey all,

I decided to try a post using Blogio. Supposedly it is a nice tool where I can write my blogs offline, save them locally, and post them when I can.

So lets see how some of the features work.

This is a heading 1

However, after creating heading 1 and then hitting enter, it did not make the next line’s type to Paragraph. I changed it to paragraph and it was still wrong. I had to click Html editor and then click back to the visual editor to make this work.

This is a heading 2

Same problem with heading 2.

Here is a quote

Ending quote was easy. Just hit the quote button again with the mouse.

  1. This is a numbered
  2. list of items
  3. And we will see if it works.

The font size works, except not for headings.

Well, I am going to be using SilverStripe in a few months. I wonder if it will integrate with that.

Update: There was an issue where when I published this, it didn’t show up right away and said it “missed schedule” whatever that means.  I updated it and now shows up.

An interesting Artic: Unix's Revenge

The article is titled Unix’s Revenge and it was quite a good read.

Here is a quote:

Now we’ve entered a new decade of devices where Unix(-like) operating systems will, on a CPU basis, probably out-install Windows. Not only is iOS based on Unix, but Android and MeeGo and even Bada are based on Linux as are QNX and WebOS. Google, Apple, HP, RIM, Samsung and Nokia are all now betting heavily on Unix or Unix-like implementations. The success is so overwhelming that there are really only two hold-outs: Microsoft and the rapidly depreciating Symbian.

Read the full article here:
http://www.asymco.com/2010/09/29/unixs-revenge/