Posts tagged ASP .NET

This blog now running MVC3 and RavenDB !

From today, this blog will be running a home-grown blogging system built on MVC3, Razor and RavenDB.

It had been running Sitecore Express until now, but i decided to ditch it. A Sitecore installation is simply too much of a hassle for a simple site like this. Also, the rich text editor in Sitecore was not really fit for posting code snippets. I will be refining this new blogging solution over the next few weeks, and hopefully it will give me a renewed interest in actually posting content on this blog :-)

I built this blogging system myself (no CMS or other framework base) to learn more about MVC3 and RavenDB. Conlusion: MVC3 is nice, Razor view syntax is extremely cool.

RavenDB is also easy to get started with and to learn. It is what I would call a no-frills NoSQL document database. So if one has data storage needs that fit into the NoSQL camp, and is building on .NET, i think the choice is a no-brainer. You do have to be aware that it is still a young product, which is changing rapidly.


A Relative Path Facility For Castle Windsor

At work, we use Castle Windsor for Dependency Injection. In Castle Windsor, as with any dependency injection framework, you can configure components identified by an interface, that can be resolved at runtime using the dependency injection framework. Components can have dependencies, which can be yet other components and so on. In this way, you can have your dependency injection framework create a whole graph of objects for you.

One limitation we run into now and then, is with components, that depend on a file path to work. Typically, we need to know the full path of the file to load it. But hardcoding the full path in the configuration file is generally a bad idea, it will create problems when you move your web application between environments. Also, we cannot just pass the path as a virtual path to the component and then have the component call Server.MapPath to map the path - since that would mean changing the interface of the component just to accomodate the injection framework, which is not a good idea. And, what is worse, you would create a dependency on System.Web in a place where it probably isn't needed.

Now, one way to get around this would be to create a wrapper interface, IFilePath, which only should exist in order to be passed into the component and being able to convert the path. This also involves changing the component and generally feels like a bad idea.

Luckily, the Windsor IoC container offers a large variety of extension points - one being facilities. So I wrote a facility, that allows paths configured in Castle Windsor to be relative. The way this works is by registering an ISubDependencyResolver in the IKernel instance. When resolving a dependency, Windsor will ask the ISubDependencyResolver whether it can resolve the dependency using the CanResolve method. By examining the passed ComponentModel and in particular it's configuration node, I look for a custom attribute on the dependency, pathType. If found (and the dependency is of type string), then we can easily resolve the dependency by taking the relative path in the configuration tag and making it absolute.

This will allow you to have your Windsor configuration look like this (notice the one-line facility registration - this is what registers the custom facility in Windsor, and makes us able to register the path dependency as a virtual path):

         1:   <castle>
         2:     <facilities>
         3:       <facility id="pathResolver" type="dr.Castle.WebPathFacility.RelativePathSupportFacility, dr.Castle.WebPathFacility" />
         4:     </facilities>
         5:     <components>
         6:       <component id="dummy"
         7:                  service="dr.Castle.WebPathFacility.Test.IDummy, dr.Castle.WebPathFacility.Test"
         8:                  type="dr.Castle.WebPathFacility.Test.Dummy, dr.Castle.WebPathFacility.Test" >
         9:         <parameters>
         10:           <path pathType="Relative">App_Data/test.xml</path>
         11:         </parameters>
         12:       </component>
         13:     </components>
         14:   </castle>
 

The valid values for pathType are:

         1:         private enum PathType
         2:         {
         3:             /// <summary>
         4:             /// The path is absolute (we will do nothing to it).
         5:             /// </summary>
         6:             Absolute = 0,
         7:             /// <summary>
         8:             /// The path is a virtual path to a web application resource.
         9:             /// </summary>
         10:             Virtual,
         11:             /// <summary>
         12:             /// The path is relative to the current directory.
         13:             /// </summary>
         14:             Relative
         15:         }

The code for the facility it self is really simple, since it simply registers our dependency resolver to the Kernel. The advantage of using a facility, is that it can be declared in the config, and Windsor will automatically initialize for all containers you create:

         1: 
        using Castle.MicroKernel.Facilities;
         2:  
         3: 
        namespace dr.Castle.WebPathFacility
         4: {
         5:     public class RelativePathSupportFacility : AbstractFacility
         6:     {
         7:         protected override void Init()
         8:         {
         9:             Kernel.Resolver.AddSubResolver(new PathParameterDependencyResolver());            
         10:         }
         11:     }
         12: }

Finally, the implementation of ISubDependencyResolver, that makes this possible:

         1: 
        using System;
         2: 
        using System.Collections.Generic;
         3: 
        using System.IO;
         4: 
        using System.Linq;
         5: 
        using System.Web;
         6: 
        using Castle.Core;
         7: 
        using Castle.MicroKernel;
         8:  
         9: 
        namespace dr.Castle.WebPathFacility
         10: {
         11:     /// <summary>
         12:     /// Custom dependency resolver, that will inspect the parameters collection for the pathType attribute, and, if found, convert the dependency to 
         13:     /// a absolute path based on the path type.
         14:     /// </summary>
         15:     class PathParameterDependencyResolver : ISubDependencyResolver
         16:     {
         17:         /// <summary>
         18:         /// Holds the supported conversion operations.
         19:         /// </summary>
         20:         private static readonly Dictionary<PathType,Func<string, string>> conversions = new Dictionary<PathType, Func<string, string>>
         21:                                                                                            {
         22:                                                                                                {PathType.Absolute, path => path},
         23:                                                                                                {PathType.Relative, path => Path.Combine(Environment.CurrentDirectory,path) },
         24:                                                                                                {PathType.Virtual,  path => HttpContext.Current.Server.MapPath(path)}
         25:                                                                                            };
         26:  
         27:         /// <summary>
         28:         /// Cache of the type path parameters.
         29:         /// </summary>
         30:         private readonly Dictionary<string,PathParameter> typePathParameters = new Dictionary<string, PathParameter>();
         31:  
         32:         /// <summary>
         33:         /// Resolves the specified dependency.
         34:         /// </summary>
         35:         /// <param name="context">Creation context</param>
         36:         /// <param name="contextHandlerResolver">Parent resolver</param>
         37:         /// <param name="model">Model of the component that is requesting the dependency</param>
         38:         /// <param name="dependency">The dependcy to satisfy</param>
         39:         /// <returns><c>true</c> if the dependency can be satsfied by this resolver, else <c>false</c>.</returns>
         40:         /// <returns>The resolved dependency</returns>
         41:         public object Resolve(CreationContext context, ISubDependencyResolver contextHandlerResolver, ComponentModel model, DependencyModel dependency)
         42:         {
         43:             PathParameter parameter = GetPathParameter(model, dependency);
         44:             if (parameter == null) 
         45:                 throw new ApplicationException(String.Format("Cannot resolve dependency {0}", dependency));
         46:             if (!conversions.ContainsKey(parameter.Type))
         47:                 return parameter.Value;     // Unknown conversion
         48:  
         49:             return conversions[parameter.Type](parameter.Value);
         50:         }
         51:         /// <summary>
         52:         /// Determines whether this sub dependency resolver can resolve the specified dependency.
         53:         /// </summary>
         54:         /// <param name="context">Creation context</param>
         55:         /// <param name="contextHandlerResolver">Parent resolver</param>
         56:         /// <param name="model">Model of the component that is requesting the dependency</param>
         57:         /// <param name="dependency">The dependcy to satisfy</param>
         58:         /// <returns><c>true</c> if the dependency can be satsfied by this resolver, else <c>false</c>.</returns>
         59:         public bool CanResolve(CreationContext context, ISubDependencyResolver contextHandlerResolver, ComponentModel model, DependencyModel dependency)
         60:         {            
         61:             if ( dependency.DependencyType == DependencyType.Parameter && dependency.TargetType.Equals(typeof(string)) )
         62:             {
         63:                 PathParameter parameter = GetPathParameter(model, dependency);
         64:                 return parameter != null;
         65:             }
         66:             return false;
         67:         }
         68:  
         69:         /// <summary>
         70:         /// Finds the parameter by looking at the cache, then in the model configuration.
         71:         /// </summary>
         72:         /// <param name="model"></param>
         73:         /// <param name="dependency"></param>
         74:         /// <returns></returns>
         75:         private PathParameter GetPathParameter(ComponentModel model, DependencyModel dependency)
         76:         {
         77:             if (!typePathParameters.ContainsKey(model.Name))
         78:                 typePathParameters.Add(model.Name, GetPathParameterInternal(model, dependency));
         79:  
         80:             return typePathParameters[model.Name];
         81:         }
         82:  
         83:         /// <summary>
         84:         /// Finds the parameter by looking at the model configuration.
         85:         /// </summary>
         86:         /// <param name="model"></param>
         87:         /// <param name="dependency"></param>
         88:         /// <returns></returns>
         89:         private PathParameter GetPathParameterInternal(ComponentModel model, DependencyModel dependency)
         90:         {
         91:             var parametersContainer = model.Configuration.Children.SingleOrDefault(n => n.Name == "parameters");
         92:             if ( parametersContainer != null )
         93:             {
         94:                 var parameterNode = parametersContainer.Children.SingleOrDefault(n => n.Name == dependency.DependencyKey);
         95:                 string pathType = parameterNode.Attributes["pathType"];
         96:                 if (pathType != null)
         97:                 {
         98:                     PathType type;
         99:                     if (!Enum.TryParse(pathType, true, out type))
         100:                         throw new ApplicationException(
         101:                             String.Format("Configuration error: Invalid pathType value '{0}'", pathType));
         102:  
         103:                     return new PathParameter {Type = type, Value = parameterNode.Value};
         104:                 }
         105:             }
         106:             return null;
         107:         }
         108:  
         109:         /// <summary>
         110:         /// Holds a path parameter
         111:         /// </summary>
         112:         private class PathParameter
         113:         {
         114:             /// <summary>
         115:             /// Value as entered in config
         116:             /// </summary>
         117:             public string Value { get; set; }
         118:             /// <summary>
         119:             /// Type of path.
         120:             /// </summary>
         121:             public PathType Type { get; set;}
         122:         }
         123:  
         124:         /// <summary>
         125:         /// Defines the types of paths supported by <see cref="PathParameterDependencyResolver" />
         126:         /// </summary>
         127:         private enum PathType
         128:         {
         129:             /// <summary>
         130:             /// The path is absolute (we will do nothing to it).
         131:             /// </summary>
         132:             Absolute = 0,
         133:             /// <summary>
         134:             /// The path is a virtual path to a web application resource.
         135:             /// </summary>
         136:             Virtual,
         137:             /// <summary>
         138:             /// The path is relative to the current directory.
         139:             /// </summary>
         140:             Relative
         141:         }
         142:     }
         143: }

Now, I am finally able to use virtual paths in my configuration files, with a minimum of noise. Great. Please notice, that the "Relative" path type might not make sense for a real application (since it uses Environment.CurrentDirectory as base), but it can be really helpful in test configurations. The primary reason for creating this is pathType="virtual", which maps to Server.MapPath.


Sitecore Dictionary gotcha when using the master database

During some projects at work, we were having a real weird problem with the Sitecore Dictionary feature. This is the built in feature in Sitecore that lets you localize short texts, such as what to put in the "Read more" or "Next page" links on a web page. Of course, there are many ways to do this, but since Dictionary items is a supported Sitecore feature, and they can be edited from within the Sitecore environment, it seems reasonable to use this for Sitecore sites.

This is how it works: You put dictionary items under /sitecore/system/Dictionary. Each dictionary item consists of a shared key and a localized phrase field. That is, the key is the same for all languages, the phrase varies by language. To get a phrase at runtime, you use the static Translate.Text method and pass the key as a parameter, and it will return the correct phrase in the current context language (or you can explicitly pass a Language). This is also described on SDN here (you will need an SDN account to access the article). There even is an XSLT extension function for the Translate methods. Very nice.

The problem we were facing started to occur when we moved some of our front-end developers (who does most of the localization texts) away from working on a test-server, and instead having their own development version on their own workstation, working there (the way it really should be done - also gives you a better chance of running CI). The problem was, that on one developer's workstation you would see one set of Dictionary items on the website, on another workstation there would be a different set. Sometimes all dictionary phrases was completely missing. Inside the Sitecore Content Editor, the data seemed to be correct.

So I started digging around to find an explanation. I started using Reflector to determine what was really going on in the Sitecore.Globalization.Translate class. I quickly found out, that the class internally keeps a Hashtable containing the languages, and in that a Hashtable for each language's texts, in memory. Great for quick lookups. So when and how is the Hashtable filled with data ? This is where it gets tricky. When a phrase is first requested (after a application restart), Sitecore will look for a file called dictionary.dat in the temp directory and try to load it. This is where the Translate class keeps it's persistent cache: Each time a key is added, it will get saved to the file (in addition to being kept in memory); and it will try to load it from there when a phrase is requested, and there is no dictionary data in memory. If reading this file fails, it will rebuild it from the data in the database. Here lies the first problem: You might know that Sitecore operates with a core database (for Sitecore itself), a master database for content and a web database that content is served from. As it turns out, the dictionary will only ever be re-populated with data from the core database. This is hard-coded in the class. And we were adding all our Dictionary items to the master database (since this is really where they should be, it is content). This was my first "that's funny" moment - If the dictionary cache only rebuilds from core; how could our setup ever have worked ??

Time to research some more. After some more tinkering around, I found the Sitecore.Globalization.ItemSaveEventHandler. This is an event handler, that is hooked up to the ItemSaving event in web.config. What this does, is that whenever an item is saved, if it is of type "dictionary item", it will add the key and phrase to the internal language hashtable, which will also trigger a save of the dictionary.dat file. This event handler however, does not care which database is being used. Both master and core (and web for that matter) database saves, will trigger a update of the cache.

This explained everything. If Sitecore is being used on the same machine only, dictionary items in the master database, will work. If it is deployed to production, it will work, because we typically copy the entire website folder, including the temp files, and thus the "correct" dictionary.dat. However, if Sitecore is used in parallel on different servers, you will start seeing the errors we were seing, inconsistent and/or missing dictionary entries. This could be an issue during development if you setup the environment on different developer workstations, but also in staging environments or in load-balanced environments (depending on your setup). If dictionary items (again, in the master database), is saved in parallel on different environments, it will generate different dictionary.dat files with different phrases, and it is impossible to merge. And if you lose dictionary.dat, you can't restore the master dictionary entries without saving each item again.

Revisiting the SDN documentation, though it is quite thin on the subject, it does state that you should add your own dictionary items to the core database. It feels wrong to do this, since our own dictionary items would then be mixed with Sitecore's own internal ones; and because I don't think the core database is a place to store customer data. So I guess it is not a bug that storing dictionary items in the master is not supported, but it would be a nice and reasonable feature. But I do think that there is a bug here; in that it is at all possible to use dictionary items stored in master, when it is not supported. It should definitely work consistently; and not be some half-baked feature that works "occasionally, if you use the right setup". It should also be noted that the Dictionary node is already in the master database in a clean installation, so there are no alarm clocks going off when the novice Sitecore programmer starts using it.

Now, we already have solutions in production using this approach, and time committed on ongoing projects for using it this way. So we needed a way to ensure that dictionary items in the master database would behave consistently and just work.

The solution is below. It is basically a class that can rebuild the Sitecore dictionary from both databases. It reuses the Sitecore logic by invoking their Load and Save methods using reflection, and overwriting the static _languages hashtable field in the Translate class. This is really not pretty, it is a ugly hack, and I would definitely prefer not to use reflection to call into methods that were never intended to be used from outside the class. That being said however, it seems to work - but of course there are no guarantees, and if it blows up or kills your kitten; I'm not responsible.

To use it, simply call the Rebuild method. I used a custom IHttpHandler for the purpose, so I can call the URL whenever needed (don't deploy the handler it into production however ;-) ). After the cache has been rebuilt, you can share the dictionary.dat with other development machines just by copying it, or you can just rebuild when needed at each developer's discretion.

         1: using System;
         2: using System.Reflection;
         3: using Sitecore.Data;
         4: using Sitecore.Configuration;
         5: using System.Collections;
         6: using Sitecore.Data.Items;
         7: using Sitecore.SecurityModel;
         8: 
         9: namespace Webdanmark.SitecoreCMS.Common.Utility
         10: {
         11:     /// <summary>
         12:     /// This class supports rebuilding the Sitecore dictionary from both Core and Master databases.
         13:     /// Default implementation from Sitecore can only rebuild from Core, which leads to various issues if
         14:     /// the temp dictionary.dat file is lost, or editing happens on multiple servers.
         15:     /// </summary>
         16:     /// <remarks>
         17:     /// This class tinkers with Sitecore private methods and internal workings. Not pretty.
         18:     /// This is a hack to workaround a limitation in Sitecore without re-implementing the whole thing.
         19:     /// </remarks>
         20:     public class DictionaryRebuilder
         21:     {
         22:         /// <summary>
         23:         /// Event fired when progress in the task occurs.
         24:         /// </summary>
         25:         public event EventHandler<DictionaryRebuilderEventArgs> Progress;
         26:         /// <summary>
         27:         /// Databases.
         28:         /// </summary>
         29:         private readonly Database[] databases;
         30:         /// <summary>
         31:         /// The Translate type.
         32:         /// </summary>
         33:         private readonly Type translateType;
         34:         /// <summary>
         35:         /// Load method.
         36:         /// </summary>
         37:         private readonly Action<Hashtable, Item> loadMethod;
         38:         /// <summary>
         39:         /// Binding flags for a private static member.
         40:         /// </summary>
         41:         private static readonly BindingFlags privateStatic = BindingFlags.Static | BindingFlags.NonPublic;
         42:         /// <summary>
         43:         /// Save method
         44:         /// </summary>
         45:         private readonly Action saveMethod;
         46:  
         47:         /// <summary>
         48:         /// Initializes a new instance of the <see cref="DictionaryRebuilder"/> class.
         49:         /// </summary>
         50:         public DictionaryRebuilder()
         51:         {
         52:             databases = new[] { Factory.GetDatabase("core"), Factory.GetDatabase("master")};
         53:             translateType = typeof(Sitecore.Globalization.Translate);
         54:             loadMethod = (Action<Hashtable, Item>) FindMethod<Action<Hashtable,Item>>("Load", privateStatic, typeof (Hashtable), typeof (Item));
         55:             saveMethod = (Action) FindMethod<Action>("Save", privateStatic);
         56:         }
         57:  
         58:  
         59:         /// <summary>
         60:         /// Rebuilds the dictionary cache.
         61:         /// </summary>
         62:         public void Rebuild()
         63:         {
         64:             Hashtable rootTable = new Hashtable(10);
         65:             foreach (var db in databases)
         66:             {
         67:                 var langs = db.Languages;
         68:                 SendMessage("\nProcessing {0} database, {1} languages.", db.Name, langs.Length);
         69:                 foreach (var language in langs)
         70:                 {
         71:                     string languageKey = language.ToString();
         72:                     Hashtable languageTable;
         73:                     if (rootTable.ContainsKey(languageKey))
         74:                         languageTable = (Hashtable)rootTable[languageKey];
         75:                     else
         76:                         rootTable[languageKey] = languageTable = new Hashtable();
         77:  
         78:                     RebuildLanguage(db, language, languageTable);
         79:                 }
         80:             }
         81:             SendMessage("\nLanguages loaded.");
         82:             ReplaceSitecoreTable(rootTable);
         83:             SendMessage("Writing data cache to file.");
         84:             saveMethod();
         85:             SendMessage("\nDone.");
         86:         }
         87:  
         88:         /// <summary>
         89:         /// Finds the method.
         90:         /// </summary>
         91:         /// <typeparam name="TDelegate">The type of the delegate.</typeparam>
         92:         /// <param name="name">The name.</param>
         93:         /// <param name="bindingFlags">The binding flags.</param>
         94:         /// <param name="parameterTypes">The parameter types.</param>
         95:         /// <returns></returns>
         96:         private Delegate FindMethod<TDelegate>(string name, BindingFlags bindingFlags, params Type[] parameterTypes)            
         97:         {
         98:             MethodInfo method = translateType.GetMethod(name, bindingFlags, Type.DefaultBinder, parameterTypes, null);
         99:             return Delegate.CreateDelegate(typeof (TDelegate), method);
         100:         }
         101:  
         102:         /// <summary>
         103:         /// Replaces the sitecore table.
         104:         /// </summary>
         105:         /// <param name="hashtable">The hashtable.</param>
         106:         private void ReplaceSitecoreTable(Hashtable hashtable)
         107:         {
         108:             FieldInfo fi = translateType.GetField("_languages", privateStatic);
         109:             fi.SetValue(null,hashtable);
         110:         }
         111:  
         112:         /// <summary>
         113:         /// Rebuilds the language.
         114:         /// </summary>
         115:         /// <param name="db">The db.</param>
         116:         /// <param name="language">The language.</param>
         117:         /// <param name="languageTable">The language table.</param>
         118:         private void RebuildLanguage(Database db, Sitecore.Globalization.Language language, Hashtable languageTable)
         119:         {
         120:             using (new SecurityDisabler())
         121:             {
         122:                 var dictionaryRoot = db.GetItem("/sitecore/system/dictionary", language);
         123:                 if (dictionaryRoot == null)
         124:                 {
         125:                     SendMessage("\tNo dictionary found in {0} for {1}", db.Name, language.Name);
         126:                     return;
         127:                 }
         128:  
         129:                 SendMessage("\tProcessing {0}", language.Name);
         130:                 loadMethod(languageTable, dictionaryRoot);
         131:             }
         132:         }
         133:  
         134:         /// <summary>
         135:         /// Sends the message.
         136:         /// </summary>
         137:         /// <param name="msg">The MSG.</param>
         138:         /// <param name="inserts">The inserts.</param>
         139:         private void SendMessage(string msg, params object [] inserts)
         140:         {
         141:             if (Progress != null)
         142:             {
         143:                 var args = new DictionaryRebuilderEventArgs {Message = String.Format(msg, inserts)};
         144:                 Progress(this, args);
         145:             }
         146:         }
         147:  
         148:         /// <summary>
         149:         /// Event arguments
         150:         /// </summary>
         151:         public class DictionaryRebuilderEventArgs : EventArgs
         152:         {
         153:             /// <summary>
         154:             /// Gets or sets the message.
         155:             /// </summary>
         156:             /// <value>The message.</value>
         157:             public string Message { get; set; }
         158:         }        
         159:     }
         160: }

 And you could use it like this:

        1: DictionaryRebuilder builder = new DictionaryRebuilder();
        2: builder.Progress += (s, e) => Response.WriteLine(e.Message);
        3: builder.Rebuild();

If you want to display progress while rebuilding, hookup the Progress method to some event handler. The one in my example won't compile for you, since Response.WriteLine is an extension method in one of our common libraries.

Disclaimer: The contents of this Blog post is my own opinions only. I am not affiliated with Sitecore in any way. Some of the technical details was extracted using Reflector, and some are educated guesswork. I might be wrong, and Sitecore might very well change the implementation in a later version, so that the information above does no longer apply. This was done on Sitecore 6.0.1. rev 090212, but I suspect that the general idea is the same in previous Sitecore versions. 


Getting your public IP from a PowerShell script

I often work on computers at different locations, and often, I need to know what public IP I am using to connect to the internet. Of course, this is easy to find out - I can just go to a website that tells me my IP, such as http://myip.dk/.

But I find this to be suboptimal. If I am configuring something, finding the IP involves firing up a browser, going to the site, and copying the IP displayed. It is a speed bump when I am trying to be productive - and the display-my-ip sites are often covered in commercials, which I dislike.

So today, I decided to write a PowerShell script that can tell me my current public IP address. First, I needed a reliable way of finding it. Of course, I could just screen-scrape off a site such as http://myip.dk/, but it has some disadvantages. I can't know if the html structure will change - and it would mean that I would have to download all of the HTML just to get a few bytes of IP address. Furthermore, I don't know whether it would be legal at all.

Therefore, I started by writing a small ASP .NET HTTP handler, that could tell me my IP. I put the following simple code in the ProcessRequest method:

 1:     public void ProcessRequest (HttpContext context) {
 2:         context.Response.ContentType = "text/plain";
 3:         context.Response.AddHeader("X-RemoteIP", HttpContext.Current.Request.UserHostAddress);
 4:         context.Response.Write(HttpContext.Current.Request.UserHostAddress);        
 5:     }

This simply writes the IP address the handler is accessed with, to the response as well as to a custom http header. I then deployed this handler to my website.

Next, writing the PowerShell script, was equally simple; we can simply use the handy System.Net.WebClient class:

 1: $ipFinderHost = "http://www.somedomain.org/GetIP.ashx"
 2: $c = new-object System.Net.WebClient
 3: $c.DownloadString($ipFinderHost)

And voila, I have a PowerShell script that displays my public IP address. And, since I have the PowerShell Community Extensions installed, I can use the set-clipboard cmdlet to copy it to the clipboard.

 1: get-myip | set-clipboard

Much nicer than manually copying from the text in a browser :-) If you decide to use this script yourself, obviously you will need to change the URL in the script to where you have deployed the GetIP.ashx handler.


How to create an ASP .NET Captcha Control (part 2)

This is the second in a 3 part series on how to create an ASP .NET Captcha control. The previous post can be found here. This time we will look at how the CAPTCHA image can be generated using the built-in .NET framework classes.

Image generation approach
As described in the first post, the idea is to create an image showing a word, and let the user repeat it by typing it into a textbox. The image should be hard to read for OCR software, so that the CAPTCHA is hard to beat for automated bots. The way we will be doing this, is by stretching and warping the text, and adding noise. Luckily, this is easy to accomplish by using the GraphicsPath class to draw the string, and then use the Warp method on the GraphicsPath object.

Generating the image: Step by step
The first step in generating the image is to create a Bitmap object with the appropiate dimensions. We will also need a Font for the text, and a Brush for painting the text. I also declare a rectangle that is slightly smaller than the actual image, which will be used as the drawing bounds later. This will help to ensure, that the text fits on the image after the transformations:

        /// 
        /// Generates the CAPTCHA image.
        /// 
        /// 
        public static byte[] GenerateImage()
        {
            // Create image.
            var image = new Bitmap(size.Width, size.Height, PixelFormat.Format24bppRgb);
            var imgRectangle = new Rectangle(10, 10, image.Width - 10, image.Height - 10);
            // Get font and brush.
            var brush = new HatchBrush(HatchStyle.SolidDiamond, Color.Black, Color.FromArgb(rand.Next(160),rand.Next(160),rand.Next(160)));
            var font = new Font(fonts[rand.Next(fonts.Length - 1)], imgRectangle.Height, FontStyle.Italic, GraphicsUnit.Pixel);
 
Notice that we use a HatchBrush so that the word will be drawn using a hatch pattern. Ensuring that the text is not solid color, will help defeat OCR attacks. The actual font used is also chosen at random from a predefined list.
				
The next step is to get a Graphics object from the image, and use it to draw on the image. We'll wrap the code using the Graphics object in a using region to ensure that the instance is disposed as soon as we don't need it more. We then fill the background with white color and create a GraphicsPath instance, to which the selected Captcha word is added using the AddString method. The path object can now be warped, by stretching the corners a random amount. We also rotate the text a bit (between --10 and 10 degrees):

 
            // draw on the image.
            using(Graphics g = Graphics.FromImage(image))
            {
                g.FillRectangle(Brushes.WhiteSmoke, 0, 0, image.Width, image.Height);
                var path = new GraphicsPath();
                // Make sure text fits
                while (g.MeasureString(CaptchaWord, font).Width > imgRectangle.Width)
                    font = new Font(font.FontFamily, font.Size - 1, font.Style);
                
                path.AddString(CaptchaWord, font.FontFamily, (int)font.Style, font.Size, imgRectangle, StringFormat.GenericDefault);
                float v = 4;
                var warpPoints = new PointF[]
                                           {
                                                new PointF(rand.Next(imgRectangle.Width) / v,  rand.Next(imgRectangle.Height) / v),
                                                new PointF(imgRectangle.Width - rand.Next(imgRectangle.Width) / v,  rand.Next(imgRectangle.Height) / v),
                                                new PointF(rand.Next(imgRectangle.Width)/v, imgRectangle.Height - rand.Next(imgRectangle.Height) / v), 
                                                new PointF(imgRectangle.Width - rand.Next(imgRectangle.Width) / v, imgRectangle.Height - rand.Next(imgRectangle.Height)/ v) 
                                            };
                var warpMatrix = new Matrix();
                warpMatrix.Rotate(rand.Next(20) - 10);
                path.Warp(warpPoints, imgRectangle, warpMatrix, WarpMode.Perspective);                
                g.FillPath(brush, path);
 
The next step is to add a bit of noise to the image. This is done by drawing some small elipses (dots) randomly in the image, with a random color. This is implemented with a LINQ query selecting the details for each random dot:
 
                // Add some noise.
                var noise = from e in Enumerable.Range(0, NoiseAmount)
                         select new
                                    {
                                        X = rand.Next(image.Width),
                                        Y = rand.Next(image.Height),
                                        R = 1f + (float)rand.NextDouble() * 3f,
                                        Brush = new SolidBrush(Color.FromArgb(rand.Next(255), rand.Next(255), rand.Next(255)))
                                    };
 
                foreach (var p in noise)
                    g.FillEllipse(p.Brush, p.X, p.Y, p.R, p.R);
            }
 
Finally, the resulting image is saved to PNG format in-memory and returned from the method. 
 
   // Save to buffer and return raw png image bytes.
            using(var buffer = new MemoryStream())
            {
                image.Save(buffer, ImageFormat.Png);
                return buffer.GetBuffer();
            }
 
We will use a custom http handler by implementing IHttpHandler to send the image to the client. This will be the subject for the next post in the series. 

How to: Create an ASP.NET CAPTCHA Control (part 1)

As I explained in my previous post, I developed a CAPTCHA ASP.NET control for this blog. In the next few posts, I will explain the steps involved in doing this, and how you can develop your own CAPTCHA control.

Preparations
There are some variations on CAPTCHA tests, the most common one requiring the user to input the characters displayed on an image. The idea is that only a human will be able to read these characters; so if the challenge response is correct, it is most likely a "real human" submitting the data. Since modern OCR software can be quite efficient, it is neccessary to make the charaters hard-to-read by altering shape, adding noise or lines. Of course these measures also make the CAPTCHA harder to read for a human. For my CAPTCHA control, I decided to create a control, that emphasizes on ease-of-use for the end user. Therefore, the images generated should be easy to read.

When deciding which characters to display on the image, there are generally two approaches: Generate some randomly, or choose between a pre-defined set of words. I choose the latter approach, since it would be easiest for a human to recognize an actual word. Therefore, I am storing a list of English words, from which I select one randomly whenever I need to generate a CAPTCHA.

Step one: Creating the basic control
I have chosen to implement the CAPTCHA as a UserControl, so that the look and/or different parts of the control can be changed at a later time, if I need to do so. So I created a UserControl and placed an image tag and a textbox on it. These are the essential parts of the CAPTCHA control.

The basic control implementation does the following: Whenever the control is shown, a word is selected randomly for the challenge. A unique, random URL for the CAPTCHA image is also generated. The purpose of using a unique URL is to ensure that the browser does not display an old CAPTCHA image because it caches it locally.

The selected word is stored in Session state. Alongside the URL, it is exposed as a public static property, that populates on-demand. This makes sure that the image-rendering code will be able to get the correct word, and the encapsulation ensures that I can change the storage if necessary. This is the implementation of these two properties:

1:         /// 
2:         /// Gets the captcha URL.
3:         /// 
4:         /// The captcha URL.
5:         public static string CaptchaUrl 
6:         { 
7:             get
8:             {
9:                 if (MyContext.Session[CaptchaUrlKey] == null)
10:                     MyContext.Session[CaptchaUrlKey] = String.Format("/captcha/{0}.ashx", rand.Next());
11:                 return (string)MyContext.Session[CaptchaUrlKey];
12:             }
13:         }
14:  
15:         /// 
16:         /// Gets the captcha word.
17:         /// 
18:         /// The captcha word.
19:         public static string CaptchaWord
20:         {
21:             get
22:             {
23:                 if ( MyContext.Session[CaptchaWordKey] == null)
24:                 {
25:                     string listWords = Settings.User["CaptchaWords"];
26:                     var words = listWords.Split(',');
27:                     MyContext.Session[CaptchaWordKey] = words[rand.Next(words.Length - 1)].Trim();
28:                 }
29:                 return (string)MyContext.Session[CaptchaWordKey];
30:             }

When the control is displayed, the image on the control is databound to the CaptchaUrl property; so it will display the image containing the correct word. The request the browser sends for the image will get handled by a separate http handler (which we will discuss in a later post); which will output the generated image.

On postback, the control will check the text the user has entered, and if it matches the generated word, a public property called "IsValid" will be set to true. This indicates to the control on which our CAPTCHA resides, that the user has passed the CAPTCHA test. After the check, the word and URL is reset, so a new CAPTCHA will be generated if the control is shown again.

A slightly better approach would be to implement the control as a .NET Validator control, so that it could take part in the page validation along with other validator controls. This would eliminate the need of the other controls on the page being aware of the CAPTCHA. Doing this would not be much more work; one would simply need to inherit from the abstract BaseValidator class and implement the neccessary methods.


Hacking ASP.NET: Trace information

All ASP .NET developers propably know about the trace feature in ASP .NET. Provided you have enabled tracing in web.config, (using <trace enabled="true" /> in the system.web element; requesting the url /trace.axd will provide you with a nice list of trace information for the previous requests.

I have often thought about putting the wealth of information to better use; perhaps making more detailed reports based on the trace information. This could be useful during testing. Unfortunately, as far as I can tell, there is no other way to get the information, than requesting Trace.axd. There seems to be no supported programmatic way of doing this.

So I set about finding out, how this could be done. At first I thought about creating a screen-scraper for requesting trace.axd and collecting the information. But this would be impractical; especially when large amounts of data should be collected.

A better approach seemed to be to find out how ASP .NET actually stores this information. Since trace.axd is actually an IHttpHandler (System.Web.TraceHandlers.TraceHttpHandler), the natural starting point was using Reflector to view the internals of this class. It did not take long to figure out, that the HttpRuntime class has a static internal property named Profile of the type System.Web.Util.Profiler, which is internal. This is the class responsible for collecting the Trace information, and has a GetData method. This method returns the current trace information as an IList containing DataSets.

Armed with this information, I wrote a small class that uses reflection to obtain the profiling data. The class looks like this:

   1:  using System;
   2:  using System.Collections;
   3:  using System.Collections.Generic;
   4:  using System.Data;
   5:  using System.Linq;
   6:  using System.Reflection;
   7:  using System.Web;
   8:   
   9:  namespace dr.TraceAnalyzer
  10:  {
  11:      /// 
  12:      /// Proof-of-concept class for accessing trace data using reflection.
  13:      /// 
  14:      public class TraceData
  15:      {
  16:          /// 
  17:          /// Data
  18:          /// 
  19:          private IList data = null;
  20:          /// 
  21:          /// Gets the trace data in its raw list-of-datasets representation.
  22:          /// 
  23:          public IList Data
  24:          {
  25:              get
  26:              {
  27:                  if (data == null)
  28:                      GetCurrentData();
  29:                  return data;
  30:              }
  31:          }
  32:   
  33:          /// 
  34:          /// Returns the response time for each request stored in the trace data.
  35:          /// 
  36:          public IEnumerabledouble>double> RequestResponseTimes
  37:          {
  38:              get
  39:              {
  40:                  GetCurrentData();
  41:                  var sets = from d in Data.Cast()
  42:                             select d;
  43:                  return from set in sets
  44:                               let traceTable = set.Tables["Trace_Trace_Information"]
  45:                               where traceTable != null && traceTable.Rows.Count > 0
  46:                               select (double) traceTable.Rows[traceTable.Rows.Count - 1]["Trace_From_First"];
  47:              }
  48:          }
  49:   
  50:          /// 
  51:          /// Gets the current data from the Profiler instance's GetData method.
  52:          /// 
  53:          /// 
  54:          public IList GetCurrentData()
  55:          {
  56:              var profiler = GetProfiler();
  57:              Type profilerType = profiler.GetType();
  58:              MethodInfo method = profilerType.GetMethod("GetData", BindingFlags.Instance | BindingFlags.NonPublic);
  59:              return data = (IList) method.Invoke(profiler, null);
  60:          }
  61:   
  62:          /// 
  63:          /// Use reflection to get the Profiler instance.
  64:          /// 
  65:          /// 
  66:          private object GetProfiler()
  67:          {
  68:              Type runtimeType = typeof (HttpRuntime);
  69:              PropertyInfo profileProperty = runtimeType.GetProperty("Profile",
  70:                                                                     BindingFlags.NonPublic | BindingFlags.Static);
  71:              if (profileProperty != null)
  72:              {
  73:                  return profileProperty.GetValue(null, null);
  74:              }
  75:   
  76:              throw new ApplicationException("Reflection to get profiler instance failed.");
  77:          }
  78:      }
  79:  }

I have yet to decide what I am going to use the trace data for. But an obvious way to use it would be to represent some of the performance data that is collected, as a graph. For now, I have added a property, RequestResponseTimes, that returns a list of the total time taken for each request stored in the trace data.

 

And, please remember to disable tracing when putting your site into production ;-)


Debugging ASP .NET memory leak - watch for static references

A while ago, I needed to debug an ASP .NET website, which was eating more and more memory the longer it was running, and never seemed to release the memory again. Even though the number of concurrent users was more or less constant, the memory usage seemed to be proportional to the total number of sessions since last application restart.

What was happening here ? Could it be session-state elements that never got garbage collected ? To find out, I simulated load on a test version of the website, let the sessions time out and used WinDbg and the Son of Strike dll extension command "gcroot" to find out which objects was stll alive and why they could not be collected by the garbage collector. First I forced a few garbage collections by calling GC.Collect() to make sure that the garbage collector had actually run.

I found out that some objects, that we keep in Session was rooted to an object array, which, it turned out, represented static references.
After some investigation, I found the problem in the code: A developer had created a static event, and had the objects in session subscribe to this event. This meant that the session objects could not be collected, since there were still reachable references to the objects.

It turned out that the event did not need to be static, so the fix was rather easy to implement. The debugging of the issue, however, took some time.

The morale of this is, that even though we have managed code with memory management, you should still keep an eye on memory usage during development. And with ASP .NET, you should be very careful with static references, since they could root your objects so that they can't be garbage collected.

You should also be aware, that static objects are shared for all users. This should not come as a surprise, if you know just a bit about the ASP .NET process model, but it is my experience that it is easy to forget, even for experienced developers. Because the static objects are shared, you should treat them as such, and remember to use proper synchronization methods when accessing them..

I might blog more in the future about this kind of issues.


Built on Sitecore Xpress

As from today, this blog runs on Sitecore Xpress. I have spent the day setting it up.

So, what is Sitecore Xpress ? Sitecore Xpress is a personal edition of the professional Sitecore CMS product, which just recently was released and made free of charge for developers. This is really cool, since Sitecore is a very good, mature and professional CMS, that has loads of features and is built on .NET. It also has a lot of extension points, which makes it a realy nice development platform for large enterprise websites. The free Xpress edition has almost all the features of the full product, though it is restricred to one user and can only be used for personal, non-commercial sites (details here).

Since we use Sitecore a lot at work, and it is so developer-friendly, the decition to use Sitecore Xpress for my personal website, was really easy to make. It will make a good platform for the various features I would like to add to my web site in the feature.

A nice example on how easy use and setup Sitecore is, that although there are no built-in blogging application in Sitecore, it took me just a day to setup the entire website, including the blog and migration of content from the old platform. The blog is actually a tree of so-called Sitecore content Items. Items is the cornerstone of Sitecore. Using a few XSLT renderings, the blog page, archives and individual posts and comments can be displayed. No additional code was written to achieve this. The only code I have written for this website so far, is a small User Control ( < 30 lines of code ) that allows users to add comments to the articles.

I am probably going to be blogging a bit more about Sitecore and ASP .NET related subjects in the future. I also hope that this change of platform will increase my motivation for writing more posts, so the frequency of new posts should increase.