Posts tagged .NET

What is TypeScript and why should I care ?

This tuesday, I watched Anders Hejlsberg present Microsoft's new language bet, TypeScript. This is basically JavaScript with types. It compiles to JavaScript, so, at runtime, it is JavaScript, nothing else. And therefore, it will run whereever JavaScript can run: In any browser, on node.js and so on.

TypeScript adds types as a first class citizen to JavaScript. This means you can use classes and interfaces in your code, and have the compiler do type checking. TypeScript is compiled to the idiomatic JavaScript, we would have to write ourselves, if we want to do object oriented programming in JavaScript. It also adds modules, and a bit nicer anonymous function syntax. Already available is some really nice tooling for Visual Studio 2012.

This is an example of how a snippet of TypeScript looks:

class Greeter {
    greeting: string;
    constructor (message: string) {
        this.greeting = message;
    }
    greet() {
        return "Hello, " + this.greeting;
    }
}   

var greeter = new Greeter("world");
var greeting = greeter.greet();

Why should I care ?

We are building larger and larger applications in JavaScript. This is true both to support great user experiences on the web, but JavaScript is also used more and more for many other purposes: You can create standalone applications or servers in node.js, you can use HTML+JavaScript for Windows 8 programs and other platforms. The lack of types in JavaScript means that development and debugging time can be slowed down, and bugs can be hard to find.

You might not agree with this, but consider:

  • Have you ever misspelled a method in JavaScript, and not found out until that exact method ? This does not happen with TypeScript.
  • Ever had to look some method signature up in the jQuery documentation, then spending time on Google to find the right one - with the TypeScript tools, you get auto completion right in the editor (Visual Studio for now).

Basically you get all the safety nets of a statically typed language, while still having all the benefits from JavaScript. The static typing also allows for features like auto-completion and refactoring support. One of the core benefits from using TypeScript is added tool support, and in turn, developer productivity.

Is this not just a rip-off of CoffeeScript ?

No, I don't think so. CoffeeScript is about fixing the syntax of JavaScript, but it does not touch the type system. This is the core difference betweeen the two. You might like or dislike the CoffeeScript syntax, but the CoffeeScript compiler is not aware of types.

I think TypeScript does a good job of keeping a familiar syntax, while reducing the amount of typing needed to create, for instance, a class. TypeScript also does automatic capture of the this variable for anonymous functions, which is very neat.

Go try it out

You can download the bits for VS or node.js here, and there is also an online playground, where you can run TypeScript directly in your browser. It's a preview, but is seems pretty stable to me.

Maybe it is not for you, maybe you are the dynamic language type of guy - But, I am excited about this and for fans of static typing, this is great news.

I might even get excited about the prospect of building a node.js app now.


A new home for the Chrome Password Recovery Tool

In 2008, I created the Chrome Password Recovery Tool. However, the download links on that page have been lost in a server migration.

Since I have been getting some email about the absence of a download of the tool, I decided to release it as open source on CodePlex. So go get it from here, if you need it.

This latest version supports the latest Chrome version (10), and enables reading the passwords while Chrome is running.


This blog now running MVC3 and RavenDB !

From today, this blog will be running a home-grown blogging system built on MVC3, Razor and RavenDB.

It had been running Sitecore Express until now, but i decided to ditch it. A Sitecore installation is simply too much of a hassle for a simple site like this. Also, the rich text editor in Sitecore was not really fit for posting code snippets. I will be refining this new blogging solution over the next few weeks, and hopefully it will give me a renewed interest in actually posting content on this blog :-)

I built this blogging system myself (no CMS or other framework base) to learn more about MVC3 and RavenDB. Conlusion: MVC3 is nice, Razor view syntax is extremely cool.

RavenDB is also easy to get started with and to learn. It is what I would call a no-frills NoSQL document database. So if one has data storage needs that fit into the NoSQL camp, and is building on .NET, i think the choice is a no-brainer. You do have to be aware that it is still a young product, which is changing rapidly.


A Relative Path Facility For Castle Windsor

At work, we use Castle Windsor for Dependency Injection. In Castle Windsor, as with any dependency injection framework, you can configure components identified by an interface, that can be resolved at runtime using the dependency injection framework. Components can have dependencies, which can be yet other components and so on. In this way, you can have your dependency injection framework create a whole graph of objects for you.

One limitation we run into now and then, is with components, that depend on a file path to work. Typically, we need to know the full path of the file to load it. But hardcoding the full path in the configuration file is generally a bad idea, it will create problems when you move your web application between environments. Also, we cannot just pass the path as a virtual path to the component and then have the component call Server.MapPath to map the path - since that would mean changing the interface of the component just to accomodate the injection framework, which is not a good idea. And, what is worse, you would create a dependency on System.Web in a place where it probably isn't needed.

Now, one way to get around this would be to create a wrapper interface, IFilePath, which only should exist in order to be passed into the component and being able to convert the path. This also involves changing the component and generally feels like a bad idea.

Luckily, the Windsor IoC container offers a large variety of extension points - one being facilities. So I wrote a facility, that allows paths configured in Castle Windsor to be relative. The way this works is by registering an ISubDependencyResolver in the IKernel instance. When resolving a dependency, Windsor will ask the ISubDependencyResolver whether it can resolve the dependency using the CanResolve method. By examining the passed ComponentModel and in particular it's configuration node, I look for a custom attribute on the dependency, pathType. If found (and the dependency is of type string), then we can easily resolve the dependency by taking the relative path in the configuration tag and making it absolute.

This will allow you to have your Windsor configuration look like this (notice the one-line facility registration - this is what registers the custom facility in Windsor, and makes us able to register the path dependency as a virtual path):

         1:   <castle>
         2:     <facilities>
         3:       <facility id="pathResolver" type="dr.Castle.WebPathFacility.RelativePathSupportFacility, dr.Castle.WebPathFacility" />
         4:     </facilities>
         5:     <components>
         6:       <component id="dummy"
         7:                  service="dr.Castle.WebPathFacility.Test.IDummy, dr.Castle.WebPathFacility.Test"
         8:                  type="dr.Castle.WebPathFacility.Test.Dummy, dr.Castle.WebPathFacility.Test" >
         9:         <parameters>
         10:           <path pathType="Relative">App_Data/test.xml</path>
         11:         </parameters>
         12:       </component>
         13:     </components>
         14:   </castle>
 

The valid values for pathType are:

         1:         private enum PathType
         2:         {
         3:             /// <summary>
         4:             /// The path is absolute (we will do nothing to it).
         5:             /// </summary>
         6:             Absolute = 0,
         7:             /// <summary>
         8:             /// The path is a virtual path to a web application resource.
         9:             /// </summary>
         10:             Virtual,
         11:             /// <summary>
         12:             /// The path is relative to the current directory.
         13:             /// </summary>
         14:             Relative
         15:         }

The code for the facility it self is really simple, since it simply registers our dependency resolver to the Kernel. The advantage of using a facility, is that it can be declared in the config, and Windsor will automatically initialize for all containers you create:

         1: 
        using Castle.MicroKernel.Facilities;
         2:  
         3: 
        namespace dr.Castle.WebPathFacility
         4: {
         5:     public class RelativePathSupportFacility : AbstractFacility
         6:     {
         7:         protected override void Init()
         8:         {
         9:             Kernel.Resolver.AddSubResolver(new PathParameterDependencyResolver());            
         10:         }
         11:     }
         12: }

Finally, the implementation of ISubDependencyResolver, that makes this possible:

         1: 
        using System;
         2: 
        using System.Collections.Generic;
         3: 
        using System.IO;
         4: 
        using System.Linq;
         5: 
        using System.Web;
         6: 
        using Castle.Core;
         7: 
        using Castle.MicroKernel;
         8:  
         9: 
        namespace dr.Castle.WebPathFacility
         10: {
         11:     /// <summary>
         12:     /// Custom dependency resolver, that will inspect the parameters collection for the pathType attribute, and, if found, convert the dependency to 
         13:     /// a absolute path based on the path type.
         14:     /// </summary>
         15:     class PathParameterDependencyResolver : ISubDependencyResolver
         16:     {
         17:         /// <summary>
         18:         /// Holds the supported conversion operations.
         19:         /// </summary>
         20:         private static readonly Dictionary<PathType,Func<string, string>> conversions = new Dictionary<PathType, Func<string, string>>
         21:                                                                                            {
         22:                                                                                                {PathType.Absolute, path => path},
         23:                                                                                                {PathType.Relative, path => Path.Combine(Environment.CurrentDirectory,path) },
         24:                                                                                                {PathType.Virtual,  path => HttpContext.Current.Server.MapPath(path)}
         25:                                                                                            };
         26:  
         27:         /// <summary>
         28:         /// Cache of the type path parameters.
         29:         /// </summary>
         30:         private readonly Dictionary<string,PathParameter> typePathParameters = new Dictionary<string, PathParameter>();
         31:  
         32:         /// <summary>
         33:         /// Resolves the specified dependency.
         34:         /// </summary>
         35:         /// <param name="context">Creation context</param>
         36:         /// <param name="contextHandlerResolver">Parent resolver</param>
         37:         /// <param name="model">Model of the component that is requesting the dependency</param>
         38:         /// <param name="dependency">The dependcy to satisfy</param>
         39:         /// <returns><c>true</c> if the dependency can be satsfied by this resolver, else <c>false</c>.</returns>
         40:         /// <returns>The resolved dependency</returns>
         41:         public object Resolve(CreationContext context, ISubDependencyResolver contextHandlerResolver, ComponentModel model, DependencyModel dependency)
         42:         {
         43:             PathParameter parameter = GetPathParameter(model, dependency);
         44:             if (parameter == null) 
         45:                 throw new ApplicationException(String.Format("Cannot resolve dependency {0}", dependency));
         46:             if (!conversions.ContainsKey(parameter.Type))
         47:                 return parameter.Value;     // Unknown conversion
         48:  
         49:             return conversions[parameter.Type](parameter.Value);
         50:         }
         51:         /// <summary>
         52:         /// Determines whether this sub dependency resolver can resolve the specified dependency.
         53:         /// </summary>
         54:         /// <param name="context">Creation context</param>
         55:         /// <param name="contextHandlerResolver">Parent resolver</param>
         56:         /// <param name="model">Model of the component that is requesting the dependency</param>
         57:         /// <param name="dependency">The dependcy to satisfy</param>
         58:         /// <returns><c>true</c> if the dependency can be satsfied by this resolver, else <c>false</c>.</returns>
         59:         public bool CanResolve(CreationContext context, ISubDependencyResolver contextHandlerResolver, ComponentModel model, DependencyModel dependency)
         60:         {            
         61:             if ( dependency.DependencyType == DependencyType.Parameter && dependency.TargetType.Equals(typeof(string)) )
         62:             {
         63:                 PathParameter parameter = GetPathParameter(model, dependency);
         64:                 return parameter != null;
         65:             }
         66:             return false;
         67:         }
         68:  
         69:         /// <summary>
         70:         /// Finds the parameter by looking at the cache, then in the model configuration.
         71:         /// </summary>
         72:         /// <param name="model"></param>
         73:         /// <param name="dependency"></param>
         74:         /// <returns></returns>
         75:         private PathParameter GetPathParameter(ComponentModel model, DependencyModel dependency)
         76:         {
         77:             if (!typePathParameters.ContainsKey(model.Name))
         78:                 typePathParameters.Add(model.Name, GetPathParameterInternal(model, dependency));
         79:  
         80:             return typePathParameters[model.Name];
         81:         }
         82:  
         83:         /// <summary>
         84:         /// Finds the parameter by looking at the model configuration.
         85:         /// </summary>
         86:         /// <param name="model"></param>
         87:         /// <param name="dependency"></param>
         88:         /// <returns></returns>
         89:         private PathParameter GetPathParameterInternal(ComponentModel model, DependencyModel dependency)
         90:         {
         91:             var parametersContainer = model.Configuration.Children.SingleOrDefault(n => n.Name == "parameters");
         92:             if ( parametersContainer != null )
         93:             {
         94:                 var parameterNode = parametersContainer.Children.SingleOrDefault(n => n.Name == dependency.DependencyKey);
         95:                 string pathType = parameterNode.Attributes["pathType"];
         96:                 if (pathType != null)
         97:                 {
         98:                     PathType type;
         99:                     if (!Enum.TryParse(pathType, true, out type))
         100:                         throw new ApplicationException(
         101:                             String.Format("Configuration error: Invalid pathType value '{0}'", pathType));
         102:  
         103:                     return new PathParameter {Type = type, Value = parameterNode.Value};
         104:                 }
         105:             }
         106:             return null;
         107:         }
         108:  
         109:         /// <summary>
         110:         /// Holds a path parameter
         111:         /// </summary>
         112:         private class PathParameter
         113:         {
         114:             /// <summary>
         115:             /// Value as entered in config
         116:             /// </summary>
         117:             public string Value { get; set; }
         118:             /// <summary>
         119:             /// Type of path.
         120:             /// </summary>
         121:             public PathType Type { get; set;}
         122:         }
         123:  
         124:         /// <summary>
         125:         /// Defines the types of paths supported by <see cref="PathParameterDependencyResolver" />
         126:         /// </summary>
         127:         private enum PathType
         128:         {
         129:             /// <summary>
         130:             /// The path is absolute (we will do nothing to it).
         131:             /// </summary>
         132:             Absolute = 0,
         133:             /// <summary>
         134:             /// The path is a virtual path to a web application resource.
         135:             /// </summary>
         136:             Virtual,
         137:             /// <summary>
         138:             /// The path is relative to the current directory.
         139:             /// </summary>
         140:             Relative
         141:         }
         142:     }
         143: }

Now, I am finally able to use virtual paths in my configuration files, with a minimum of noise. Great. Please notice, that the "Relative" path type might not make sense for a real application (since it uses Environment.CurrentDirectory as base), but it can be really helpful in test configurations. The primary reason for creating this is pathType="virtual", which maps to Server.MapPath.


Using Expression Trees To Break The Law Of Demeter

I am sure most programmers have heard about the Law Of Demeter, which is the principle that a classshould only have limited knowledge about other classes, and only talk to objects closely related to the current object. This is sometimes presented as "you should not have more than one dot in each expression". In other words, this would be breaking the law:


string
name = order.Customer.Name;

 

While I do appreciate the idea behind the Law Of Demeter, specifically that individual classes should not know too much about each other; I think the above code would often be perfectly acceptable. Phil Haack has a blogpost going into further details about this: The Law of Demeter Is Not A Dot Counting Excercise, and others agree. I think Martin Fowler explains it best: "I'd prefer to call it the Occasional Useful Suggestion of Demeter".

So, most of us will probably (hopefully) agree, that it is OK to use more than one dot in a statement, when appropiate. One such place might be when doing UI in a ASP .NET application, and one needs to display information about an order and it's details. But here arises a problem, we will need to check each of the expression parts for null to ensure that we do not accidentally cause a NullReferenceException. This leads to ugly code, especially in a data-binding scenario, such as:

<%# order == null ? null : order.Customer == null ? null : order.Customer.Name %>

 

This question on StackOverflow asks about exactly that, how do we get rid of such explicit and repeated null checking ? It got me thinking, it must be possible to solve this using expression trees. It turns out, it is in fact possible, as I state in my answer on StackOverflow. We can in fact build an extension methods, which looks at an expression tree, evaluates each part of it seperately, checks for null each time, and ultimately returns the correct value; or null if one of the expression parts where null. This is my implementation of such a method:

 1: using System;
 2: using System.Collections.Generic;
 3: using System.Linq.Expressions;
 4:  
 5: namespace dr.IfNotNullOperator.PoC
 6: {
 7:     public static class ObjectExtensions
 8:     {
 9:         public static TResult IfNotNull<TArg,TResult>(this TArg arg, Expression<Func<TArg,TResult>> expression)
 10:         {
 11:             if (expression == null)
 12:                 throw new ArgumentNullException("expression");
 13:  
 14:             if (ReferenceEquals(arg, null))
 15:                 return default(TResult);
 16:  
 17:             var stack = new Stack<MemberExpression>();
 18:             var expr = expression.Body as MemberExpression;
 19:             while(expr != null)
 20:             {
 21:                 stack.Push(expr);
 22:                 expr = expr.Expression as MemberExpression;
 23:             } 
 24:  
 25:             if (stack.Count == 0 || !(stack.Peek().Expression is ParameterExpression))
 26:                 throw new ApplicationException(String.Format("The expression '{0}' contains unsupported constructs.",
 27:                                                              expression));
 28:             
 29:             object a = arg;
 30:             while(stack.Count > 0)
 31:             {
 32:                 expr = stack.Pop();
 33:                 var p = expr.Expression as ParameterExpression;
 34:                 if (p == null)
 35:                 {
 36:                     p = Expression.Parameter(a.GetType(), "x");
 37:                     expr = expr.Update(p);
 38:                 }
 39:                 var lambda = Expression.Lambda(expr, p);
 40:                 Delegate t = lambda.Compile();                
 41:                 a = t.DynamicInvoke(a);
 42:                 if (ReferenceEquals(a, null))
 43:                     return default(TResult);
 44:             }
 45:  
 46:             return (TResult)a;            
 47:         }
 48:     }
 49: }

There are some caveats though, in the current version it will only work with simple member access, and it only works on .NET Framework 4, because it uses the MemberExpression.Update method, which is new in v4.

It works by examining the expression tree representing your expression, and evaluating the parts one after the other; each time checking that the result is not null.

I am sure this could be extended so that other expressions than MemberExpression is supported, and I might update it at a later point to support more complicated expressions. Consider this as proof-of-concept code, and please keep in mind that there will be a performance penalty by using it (which will probably not matter in many cases, but don't use it in a tight loop :-) ). I have not done any measurements on the performance yet, and I am also sure that one could make some optimizations to it.

Here is a zip containing the code as well as a few unit tests: IfNotNullExtension.zip.

What do you think about this approach to null checking ? Would you consider this extension method useful (provided that it performs adequately for the scenario) ?


Last day at TechEd

It's friday, and TechEd is over for this time.

My first session on friday was about little-known secrets in Microsoft Silverlight 3. This was a really good session for advanced Silverlight development, and I took away many tricks - including the ability to download and use assemblies dynamically and asynchronously; and to use the OS client stack instead of the browser stack for network access.

Second session was about extending Visual Studio 2010's architecture modelling tools. This was a code-rich session, where we were walked through creating 3 extensions for the modelling tools. With VSIX packages, deployment of Visual Studio extensions are now much easier. The coding experience when creating extenions has also been made much nicer in the new verison of Visual Studio 2010. It is a no-frills experience, where you only need to work in the problem domain, and not jump through hoops to make Visual Studio do what you want.

The last session of this year's TechEd is about Pex and Code Contracts. I am writing this while waiting for the session to be begin - it's a very interesting topic, and I might do a full length blog post about Pex and Code Contracts at a later time.

This has been a very educational and interesting week. I have learned about architecture and design, new tools and techniques. In general, the quality of the talks has been very high (there were a few misses, but it's been an overall good experience). The only problem has been to select the right session, when there were multiple interesting selections in the same time slot, which happened to me a lot. For instance, I never got to see a talk about the Concurrency Runtime (CCR), because there were always something more interesting on the menu. Now, I need to get home and get into the gym - it's been a week with good foods, eggs and bacon each morning at the hotel, so I need it :-) I Might be coming back next year !


Day Four at TechEd over

TechEd is coming to an end, day four is now over. There are three sessions on friday, then it's over.

I started the day with a session on C# 4.0 Dynamic: The why’s and hows. It was being done by Alex Turner, who is Program Manager for the C# compiler. This was a very interesting walk through why C# should have dynamic features, and why it has been designed as it is. There has gone a lot of design thought into the dynamic design, and I certainly think that the final design they’ve chosen is the right one. He demoed creating your own dynamic types from C# which can respond to any method call – very cool. I can certainly see some good use cases for the C# dynamic keyword.

Next, I went to see a talk about Windows Communication Foundation: Developer’s Guide to Windows Communication Foundation, SOA and success. Interesting, and with some very good thoughts on interoperability. My most important take-away from that session, is that if you need to be interoperable, try to do REST.

In the afternoon, I went to see Tess Ferrandez present on ASP .NET post-mortem debugging (well, the techniques apply to any .NET process, I think, but it was presented towards ASP .NET). This is the kind of debugging you get to do when your process consumes too much memory, hangs, or explodes; in the production environment, without you being able to reproduce the issue locally. When this kind of debugging is needed, something is on fire, and you will get stress fixing it. But apart from that, I do find this kind of debugging challenging and kind-of-fun ;-) Tess demonstrated using WinDbg, SOS.dll (Son-of-Strike, someone please explain the name to me), Debug Diag and other tools. She demonstrated detecting a memory issue, a poor performance issue, and a crash issue using these tools. She also demonstrated doing these using Visual Studio 2010, with its new ability to open memory dumps, and do debugging on them. With this cool new feature, you can do almost everything you can do in a normal debug session, but in a memory dump, that you might have obtained from some production server. You can see the stack trace, the locals, and examine the value of objects. The only thing you cannot do, is run/step back/forward, of course, the dump is an image of the process at a specific time. Very neat is the Parallel Stacks feature, where Visual Studio will visualize the stack of each thread for you, which makes it easy to identify contention in your locking, as well as other thread sync issues.

Last session of the day was by Magnus Mårtensson. This was an architecture talk about design with dependency injection and ensuring extensibility. Very interesting.


Third day at TechEd

Once again, I attended some very interesting talks at TechEd. This mornings sessions was entitled “The daily Scrum”, about doing Scrum and agile development. This was mostly a Q&A session with answers to many of the practical problems one might encounter when trying to be agile.

Next, I had a real hard time deciding between staying on the agile track and attending the “Tools and Agile Teams” talk versus hearing Don Syme speak about F#. I chose the F# session, which I think was a good choice. Don is one of the primary architects behind F#, so it would have been a shame not to hear him speak about it. This talk really drove home some points about F#, and why it helps you do parallel programming, with immutability, the async language construct and Agents. Another good point is that F# should not be used for anything, and in a large application, Don suggested that only a small DLL might be written in F# - it should be used as a tool, where you needed. Don also showed some really impressing demos, using Direct3D from F#.

After lunch, I attended Roy Osherove’s talk about unit testing. His main points where to write maintainable, consistent and readable unit tests, and proceeded to show this can be done. He suggested using test reviews in order to get started writing good unit tests, which I think is a very good idea. Very insightful talk.

The last session of today was about cloud computing: “ Deep Dive into Developing Line-of-Business Applications Running in the Cloud ”. I don’t think this was a good session. There was too much demoing of an app in the cloud, and too little talk about the actual architecture behind it. Also, the presenters neglected to do any introduction to the Azure tools, I guess they expected everyone attending to know about those in advance.


Day Two At TechEd Europe

Today started off fresh with 2 Sharepoint sessions The first one was an introduction to Sharepoint 2010 for developers, and while I haven’t done any development on Sharepoint before, based on the feedback, it will be tons easier to do Sharepoint development with 2010. The second session on Sharepoint was somewhat relevant and somewhat a  miss. While it did provide some good information, there was not really anything new, if you had attended the first session.

During lunch, I had been invited to a lunch session by Microsoft Denmark on IIS 7.5. The speaker was  a real expert on the subject, Bernhard Frank. Very interesting and good  food, but had to cut the session short in order to make it to the next session.

Next was a presentation by Brian Harry about TFS 2010 and its new version control features. There are some real goodies coming ,, in 2010, and Brian demonstrated better branching and branch visualization, support for rollback and improved labeling. Very nice, and something I can really see the need for in my own organization.

I also attended a session on software architecture by Ralf Westphal. He discussed architecture at a high level, and you should not view the architecture as UML class diagram, layered architechture diagram or something like that. Instead he advocated functional  building blocks, or functional units as he called it; which recursively consists of yet another set of functional units. This way, you get a hierarchy of functional units from the one application, through synchronous components till methods in a class. While surely one of the most abstract talks today, I took some very good points with me from the talk.

Lastly today, there was an ASP .NET MVC2: “What’s new” session that I attended. It really competed with the “Pumping Iron” session (about IronRuby/Python), but as it turns out, that session was overbooked, so I made the right choice. There is some really great improvements in MVC2, which boils down to improving productivity on the framework. This means support for partial renderings based on invoking of controllers, and templated views. A cool demo was demonstrating the validation features, where you can define your validation rules in the model (as annotations out-of-the-box, but it’s extensible, so you can store your rules wherever you like). I think MVC2 might just be the release that is mature enough to be tried out on a real project – I am sure our frontend developers will love it.


Got a Twitter account

Oh, BTW, I got a twitter account. Follow me on http://twitter.com/dennisriis, if interested.

TechEd Europe 2009 Day One

Today was my first day at TechEd Europe 2009 in Berlin, after having arrived to the hotel on sunday evening. I chose to drive to Berlin myself from Denmark, which went just fine, until I scratched a rim on the way down into a very narrow passage to the parking garage. Damn. The TechEd experience so far has been just great, although there was some queue for registration this morning. I guess it is hard to avoid when you have 7000+ people attending an event, and they all, more or less, arrive at the same time.

My first session at TechEd was titled "ADO.NET Entity Framework in Microsoft Visual Studio 2010 and Microsoft .NET Framework 4" and was about Entity Framework 4 with speaker Eric Nelson. Wait-a-minute, you might say, because last version of the EF was the initial release, 1.0. It seems that the EF has skipped a couple of versions so that it gets the same version number as the .NET Framework it ships with. That being said, the EF ships some of it's features as a separate download, but the core should be in .NET FX 4.

So what's to say about EF4 ? Well, it seems that Microsoft has fixed it. By that I mean that many of the problems that existed with the initial version has been eliminated or the experience has been improved. This includes better tooling and designer support. There is support for model-first development, where you drag-and-drop your model in a designer, and lets the framework generate the database for you (technically speaking, this also existed in v1, but it required you to do a lot of manual stuff, or as Eric Nelson put it, you would be in for a whole world of pain taking that route.

Something I am more excited about is the ability to control the generated code using T4 templates. This enables different scenarious such as POCO objects, which was missing from v1. There are some built-in templates so you don't need to write them from scratch. If you adhere to the "develop against an interface" to "do TDD" group of people (count me in), it would be quite easy to change the built in templates, so that you would get some nice interfaces to work against. Very nice.

Also, Eric demoed a Code-Only, or "persistance ignorance" support in EF4. With this, you can take some objects, and persist them to a database, and the framework will itself create a database, the schema, and do CRUD operations. While nice for demos, I really can't see the application of this for real-world projects larger than toy-size.

It seems that EF4 is now a serious contender in the ORM world, and I think I will try it out on a real project when I get the chance (of course, it probably needs to go out of beta first).

 

Next session was about ASP .NET 4 and Visual Studio 2010 improvements. This was a very interesting lap around a lot of small improvements and features, that will life better and easier for the web developer. This includes an inheritable viewstate setting, better controls for ClientID generation and better standards-compliant markup from the built in controls; as well as better control over the markup that is emitted. Also, a Code nugget syntax for emitting HTML encoded strings has been added, which will prove handy. One of the things that looks really good is the improved Publish dialog in 2010 and the support for Web.config merges, so that you can have one .config to rule them all, but keep transformations that you can apply automagically when deploying. Together this means, that you can click a button and get your website deployed. I didn't have the chance to ask if this is supported as MSBuild tasks as well, but I suspect it is. I'll have to track down someone who can answer this during the conference. Shouldn't be too hard :-)

 

The rest of the day was keynote sessions. First was the "Developer General" session, by Jason Zander (project manager for the Visual Studio team). Jason talked about the development ecosystem and the effort Microsoft has put into VS 2010 to make a better development experience. This includes push-of-a-button deployment of Sharepoint parts, instead of 22 manual steps. He might have been exaggerating about the 22 steps, but is sounds nice. Oh, he also announced that Microsoft has acquired the Teamprise client suite, which makes it possible for non-Windows, non-Micrsoft, non-.NET devs to work with Team Foundation Server.

Lastly there was the "real" keynote by Stephen Elop, President of Microsoft Business Division. This was a typically "fluffy" talk which did not have much real content for developers. It was interesting though, and he demoed some new features in Windows Server 2008 R2 and Exchange Server 2010. He got the biggest applause when demonstrating, that Outlook Web Access 2010 now runs seamlessly in Firefox & friends :-)

Tomorrow, I have another busy day lined up. Just have to figure out how to be 3 places at once ...

 


Sitecore Dictionary gotcha when using the master database

During some projects at work, we were having a real weird problem with the Sitecore Dictionary feature. This is the built in feature in Sitecore that lets you localize short texts, such as what to put in the "Read more" or "Next page" links on a web page. Of course, there are many ways to do this, but since Dictionary items is a supported Sitecore feature, and they can be edited from within the Sitecore environment, it seems reasonable to use this for Sitecore sites.

This is how it works: You put dictionary items under /sitecore/system/Dictionary. Each dictionary item consists of a shared key and a localized phrase field. That is, the key is the same for all languages, the phrase varies by language. To get a phrase at runtime, you use the static Translate.Text method and pass the key as a parameter, and it will return the correct phrase in the current context language (or you can explicitly pass a Language). This is also described on SDN here (you will need an SDN account to access the article). There even is an XSLT extension function for the Translate methods. Very nice.

The problem we were facing started to occur when we moved some of our front-end developers (who does most of the localization texts) away from working on a test-server, and instead having their own development version on their own workstation, working there (the way it really should be done - also gives you a better chance of running CI). The problem was, that on one developer's workstation you would see one set of Dictionary items on the website, on another workstation there would be a different set. Sometimes all dictionary phrases was completely missing. Inside the Sitecore Content Editor, the data seemed to be correct.

So I started digging around to find an explanation. I started using Reflector to determine what was really going on in the Sitecore.Globalization.Translate class. I quickly found out, that the class internally keeps a Hashtable containing the languages, and in that a Hashtable for each language's texts, in memory. Great for quick lookups. So when and how is the Hashtable filled with data ? This is where it gets tricky. When a phrase is first requested (after a application restart), Sitecore will look for a file called dictionary.dat in the temp directory and try to load it. This is where the Translate class keeps it's persistent cache: Each time a key is added, it will get saved to the file (in addition to being kept in memory); and it will try to load it from there when a phrase is requested, and there is no dictionary data in memory. If reading this file fails, it will rebuild it from the data in the database. Here lies the first problem: You might know that Sitecore operates with a core database (for Sitecore itself), a master database for content and a web database that content is served from. As it turns out, the dictionary will only ever be re-populated with data from the core database. This is hard-coded in the class. And we were adding all our Dictionary items to the master database (since this is really where they should be, it is content). This was my first "that's funny" moment - If the dictionary cache only rebuilds from core; how could our setup ever have worked ??

Time to research some more. After some more tinkering around, I found the Sitecore.Globalization.ItemSaveEventHandler. This is an event handler, that is hooked up to the ItemSaving event in web.config. What this does, is that whenever an item is saved, if it is of type "dictionary item", it will add the key and phrase to the internal language hashtable, which will also trigger a save of the dictionary.dat file. This event handler however, does not care which database is being used. Both master and core (and web for that matter) database saves, will trigger a update of the cache.

This explained everything. If Sitecore is being used on the same machine only, dictionary items in the master database, will work. If it is deployed to production, it will work, because we typically copy the entire website folder, including the temp files, and thus the "correct" dictionary.dat. However, if Sitecore is used in parallel on different servers, you will start seeing the errors we were seing, inconsistent and/or missing dictionary entries. This could be an issue during development if you setup the environment on different developer workstations, but also in staging environments or in load-balanced environments (depending on your setup). If dictionary items (again, in the master database), is saved in parallel on different environments, it will generate different dictionary.dat files with different phrases, and it is impossible to merge. And if you lose dictionary.dat, you can't restore the master dictionary entries without saving each item again.

Revisiting the SDN documentation, though it is quite thin on the subject, it does state that you should add your own dictionary items to the core database. It feels wrong to do this, since our own dictionary items would then be mixed with Sitecore's own internal ones; and because I don't think the core database is a place to store customer data. So I guess it is not a bug that storing dictionary items in the master is not supported, but it would be a nice and reasonable feature. But I do think that there is a bug here; in that it is at all possible to use dictionary items stored in master, when it is not supported. It should definitely work consistently; and not be some half-baked feature that works "occasionally, if you use the right setup". It should also be noted that the Dictionary node is already in the master database in a clean installation, so there are no alarm clocks going off when the novice Sitecore programmer starts using it.

Now, we already have solutions in production using this approach, and time committed on ongoing projects for using it this way. So we needed a way to ensure that dictionary items in the master database would behave consistently and just work.

The solution is below. It is basically a class that can rebuild the Sitecore dictionary from both databases. It reuses the Sitecore logic by invoking their Load and Save methods using reflection, and overwriting the static _languages hashtable field in the Translate class. This is really not pretty, it is a ugly hack, and I would definitely prefer not to use reflection to call into methods that were never intended to be used from outside the class. That being said however, it seems to work - but of course there are no guarantees, and if it blows up or kills your kitten; I'm not responsible.

To use it, simply call the Rebuild method. I used a custom IHttpHandler for the purpose, so I can call the URL whenever needed (don't deploy the handler it into production however ;-) ). After the cache has been rebuilt, you can share the dictionary.dat with other development machines just by copying it, or you can just rebuild when needed at each developer's discretion.

         1: using System;
         2: using System.Reflection;
         3: using Sitecore.Data;
         4: using Sitecore.Configuration;
         5: using System.Collections;
         6: using Sitecore.Data.Items;
         7: using Sitecore.SecurityModel;
         8: 
         9: namespace Webdanmark.SitecoreCMS.Common.Utility
         10: {
         11:     /// <summary>
         12:     /// This class supports rebuilding the Sitecore dictionary from both Core and Master databases.
         13:     /// Default implementation from Sitecore can only rebuild from Core, which leads to various issues if
         14:     /// the temp dictionary.dat file is lost, or editing happens on multiple servers.
         15:     /// </summary>
         16:     /// <remarks>
         17:     /// This class tinkers with Sitecore private methods and internal workings. Not pretty.
         18:     /// This is a hack to workaround a limitation in Sitecore without re-implementing the whole thing.
         19:     /// </remarks>
         20:     public class DictionaryRebuilder
         21:     {
         22:         /// <summary>
         23:         /// Event fired when progress in the task occurs.
         24:         /// </summary>
         25:         public event EventHandler<DictionaryRebuilderEventArgs> Progress;
         26:         /// <summary>
         27:         /// Databases.
         28:         /// </summary>
         29:         private readonly Database[] databases;
         30:         /// <summary>
         31:         /// The Translate type.
         32:         /// </summary>
         33:         private readonly Type translateType;
         34:         /// <summary>
         35:         /// Load method.
         36:         /// </summary>
         37:         private readonly Action<Hashtable, Item> loadMethod;
         38:         /// <summary>
         39:         /// Binding flags for a private static member.
         40:         /// </summary>
         41:         private static readonly BindingFlags privateStatic = BindingFlags.Static | BindingFlags.NonPublic;
         42:         /// <summary>
         43:         /// Save method
         44:         /// </summary>
         45:         private readonly Action saveMethod;
         46:  
         47:         /// <summary>
         48:         /// Initializes a new instance of the <see cref="DictionaryRebuilder"/> class.
         49:         /// </summary>
         50:         public DictionaryRebuilder()
         51:         {
         52:             databases = new[] { Factory.GetDatabase("core"), Factory.GetDatabase("master")};
         53:             translateType = typeof(Sitecore.Globalization.Translate);
         54:             loadMethod = (Action<Hashtable, Item>) FindMethod<Action<Hashtable,Item>>("Load", privateStatic, typeof (Hashtable), typeof (Item));
         55:             saveMethod = (Action) FindMethod<Action>("Save", privateStatic);
         56:         }
         57:  
         58:  
         59:         /// <summary>
         60:         /// Rebuilds the dictionary cache.
         61:         /// </summary>
         62:         public void Rebuild()
         63:         {
         64:             Hashtable rootTable = new Hashtable(10);
         65:             foreach (var db in databases)
         66:             {
         67:                 var langs = db.Languages;
         68:                 SendMessage("\nProcessing {0} database, {1} languages.", db.Name, langs.Length);
         69:                 foreach (var language in langs)
         70:                 {
         71:                     string languageKey = language.ToString();
         72:                     Hashtable languageTable;
         73:                     if (rootTable.ContainsKey(languageKey))
         74:                         languageTable = (Hashtable)rootTable[languageKey];
         75:                     else
         76:                         rootTable[languageKey] = languageTable = new Hashtable();
         77:  
         78:                     RebuildLanguage(db, language, languageTable);
         79:                 }
         80:             }
         81:             SendMessage("\nLanguages loaded.");
         82:             ReplaceSitecoreTable(rootTable);
         83:             SendMessage("Writing data cache to file.");
         84:             saveMethod();
         85:             SendMessage("\nDone.");
         86:         }
         87:  
         88:         /// <summary>
         89:         /// Finds the method.
         90:         /// </summary>
         91:         /// <typeparam name="TDelegate">The type of the delegate.</typeparam>
         92:         /// <param name="name">The name.</param>
         93:         /// <param name="bindingFlags">The binding flags.</param>
         94:         /// <param name="parameterTypes">The parameter types.</param>
         95:         /// <returns></returns>
         96:         private Delegate FindMethod<TDelegate>(string name, BindingFlags bindingFlags, params Type[] parameterTypes)            
         97:         {
         98:             MethodInfo method = translateType.GetMethod(name, bindingFlags, Type.DefaultBinder, parameterTypes, null);
         99:             return Delegate.CreateDelegate(typeof (TDelegate), method);
         100:         }
         101:  
         102:         /// <summary>
         103:         /// Replaces the sitecore table.
         104:         /// </summary>
         105:         /// <param name="hashtable">The hashtable.</param>
         106:         private void ReplaceSitecoreTable(Hashtable hashtable)
         107:         {
         108:             FieldInfo fi = translateType.GetField("_languages", privateStatic);
         109:             fi.SetValue(null,hashtable);
         110:         }
         111:  
         112:         /// <summary>
         113:         /// Rebuilds the language.
         114:         /// </summary>
         115:         /// <param name="db">The db.</param>
         116:         /// <param name="language">The language.</param>
         117:         /// <param name="languageTable">The language table.</param>
         118:         private void RebuildLanguage(Database db, Sitecore.Globalization.Language language, Hashtable languageTable)
         119:         {
         120:             using (new SecurityDisabler())
         121:             {
         122:                 var dictionaryRoot = db.GetItem("/sitecore/system/dictionary", language);
         123:                 if (dictionaryRoot == null)
         124:                 {
         125:                     SendMessage("\tNo dictionary found in {0} for {1}", db.Name, language.Name);
         126:                     return;
         127:                 }
         128:  
         129:                 SendMessage("\tProcessing {0}", language.Name);
         130:                 loadMethod(languageTable, dictionaryRoot);
         131:             }
         132:         }
         133:  
         134:         /// <summary>
         135:         /// Sends the message.
         136:         /// </summary>
         137:         /// <param name="msg">The MSG.</param>
         138:         /// <param name="inserts">The inserts.</param>
         139:         private void SendMessage(string msg, params object [] inserts)
         140:         {
         141:             if (Progress != null)
         142:             {
         143:                 var args = new DictionaryRebuilderEventArgs {Message = String.Format(msg, inserts)};
         144:                 Progress(this, args);
         145:             }
         146:         }
         147:  
         148:         /// <summary>
         149:         /// Event arguments
         150:         /// </summary>
         151:         public class DictionaryRebuilderEventArgs : EventArgs
         152:         {
         153:             /// <summary>
         154:             /// Gets or sets the message.
         155:             /// </summary>
         156:             /// <value>The message.</value>
         157:             public string Message { get; set; }
         158:         }        
         159:     }
         160: }

 And you could use it like this:

        1: DictionaryRebuilder builder = new DictionaryRebuilder();
        2: builder.Progress += (s, e) => Response.WriteLine(e.Message);
        3: builder.Rebuild();

If you want to display progress while rebuilding, hookup the Progress method to some event handler. The one in my example won't compile for you, since Response.WriteLine is an extension method in one of our common libraries.

Disclaimer: The contents of this Blog post is my own opinions only. I am not affiliated with Sitecore in any way. Some of the technical details was extracted using Reflector, and some are educated guesswork. I might be wrong, and Sitecore might very well change the implementation in a later version, so that the information above does no longer apply. This was done on Sitecore 6.0.1. rev 090212, but I suspect that the general idea is the same in previous Sitecore versions. 


Getting your public IP from a PowerShell script

I often work on computers at different locations, and often, I need to know what public IP I am using to connect to the internet. Of course, this is easy to find out - I can just go to a website that tells me my IP, such as http://myip.dk/.

But I find this to be suboptimal. If I am configuring something, finding the IP involves firing up a browser, going to the site, and copying the IP displayed. It is a speed bump when I am trying to be productive - and the display-my-ip sites are often covered in commercials, which I dislike.

So today, I decided to write a PowerShell script that can tell me my current public IP address. First, I needed a reliable way of finding it. Of course, I could just screen-scrape off a site such as http://myip.dk/, but it has some disadvantages. I can't know if the html structure will change - and it would mean that I would have to download all of the HTML just to get a few bytes of IP address. Furthermore, I don't know whether it would be legal at all.

Therefore, I started by writing a small ASP .NET HTTP handler, that could tell me my IP. I put the following simple code in the ProcessRequest method:

 1:     public void ProcessRequest (HttpContext context) {
 2:         context.Response.ContentType = "text/plain";
 3:         context.Response.AddHeader("X-RemoteIP", HttpContext.Current.Request.UserHostAddress);
 4:         context.Response.Write(HttpContext.Current.Request.UserHostAddress);        
 5:     }

This simply writes the IP address the handler is accessed with, to the response as well as to a custom http header. I then deployed this handler to my website.

Next, writing the PowerShell script, was equally simple; we can simply use the handy System.Net.WebClient class:

 1: $ipFinderHost = "http://www.somedomain.org/GetIP.ashx"
 2: $c = new-object System.Net.WebClient
 3: $c.DownloadString($ipFinderHost)

And voila, I have a PowerShell script that displays my public IP address. And, since I have the PowerShell Community Extensions installed, I can use the set-clipboard cmdlet to copy it to the clipboard.

 1: get-myip | set-clipboard

Much nicer than manually copying from the text in a browser :-) If you decide to use this script yourself, obviously you will need to change the URL in the script to where you have deployed the GetIP.ashx handler.


Introducing: The Google Chrome Password Recovery Tool

Today, I wanted to backup all my passwords stored in Google Chrome. I thought that would be an easy task, but it turns out, that this is not supported, at least as far as I can tell. There is an option to view the shared passwords one-by-one, but that was not really an option for me.

So, I decided to write a small program to extract the passwords from Chrome. Since Chrome (or Chromium, to be exact), is open source, I pulled the source from http://dev.chromium.org/getting-involved, compiled it, and starting looking around trying to figure out how passwords are stored. The setup and build experience was much nicer than what I have tried with other open source projects I have looked at; there are detailed build instructions with only a few steps available, and after setting up, it just works, in a Visual Studio 2008 solution. A full recompile does take some time however (45 minutes on my machine).

I quickly found out that Chrome stores most of its user and configuration data in small SQLite databases stored on disk in the AppData/Local/Google/Chrome/User Data directory. So, reading the data was no problem after grabbing a copy of the ADO .NET Provider for SQLite, as well as the sqlite3.dll binary from http://www.sqlite.org/. The data I was after (user names and passwords) is stored in the file named Web Data. This contains a table named logins, which contains the URL for which the login is valid, some details about the html form where the password has been used (in order to allow Chrome to auto-fill password boxes for you), and the username and password. It also contains a "preferred" and "blacklisted_by_user" column.

Decrypting the passwords

The passwords is, obviously for security reasons, not stored in plain text. Rather, they are encrypted, so I needed to figure out how they are encrypted and how to decrypt them. The answer lies in the Chromium source, where the Encryptor class contains the following method:

bool Encryptor::EncryptString(const std::string& plaintext,
                              std::string* ciphertext) {
  DATA_BLOB input;
  input.pbData = const_cast<BYTE*>(
    reinterpret_cast<const BYTE*>(plaintext.data()));
  input.cbData = static_cast<DWORD>(plaintext.length());

  DATA_BLOB output;
  BOOL result = CryptProtectData(&input, L"", NULL, NULL, NULL,
                                 0, &output);
  if (!result)
    return false;

  // this does a copy
  ciphertext->assign(reinterpret_cast<std::string::value_type*>(output.pbData),
                     output.cbData);

  LocalFree(output.pbData);
  return true;
}

As it turns out, the Windows Data Protection (DPAPI) is used to encrypt the data, namely the CryptProtectData function as shown above. Therefore, I can relatively easy decrypt the data, using the CryptUnprotectData function, as long as I do not try to decrypt the password of other users - the DPAPI encrypts with a key based on the current user's login credentials. I first tried to do the decrypting in C# using P/Invoke the the CryptUnprotect function, but for some reason, I could not get that to work. I kept getting a credentials dialog from Windows when trying it, which is not what i want. Luckily, after googling a bit, I found out that there already exist a managed wrapper for doing this, namely the ProtectedData class. After switching to using this, there were no problems decrypting the passwords.

The password recovery tool

I wrote a tiny C# console program to dump the passwords. They can be dumped to the console (default) or to an XML file. Running the program without parameters will try to dump all the passwords to the console. You might need to have Chrome closed while doing this. Run the program with the -help switch to display a short usage information message.

I am including the program for download here - both in a precompiled form and the C# source. It requires .NET Framework 3.5. The program as well as it's source is free for use non-commercially and is provided without any warranty or obligations for me, neither explicit or implied. It probably won't kill your cat, but don't come to me crying about it if it does ;-). If you wish to use the source or derivate thereof in a commercial product, contact me for permission first.

Download:

What is missing ?
It would be nice to have an option to import the exported passwords into a new Chrome installation on another computer. I am considering adding it, but don't really need it at this time. It should be relatively easy - if you happen to develop the feature based on my work, please email me the source.


Making The HTC Touch Diamond Vibrate

One of the minor problems I had when making the Stopwatch for my HTC Touch Diamond, was to make the phone vibrate automatically. It seems there are no managed way of doing this. However, after a bit of googling around, I found out that the vibrator typically can be addressed as a LED object using the Open NET CF Framework. So I decided to throw together a tiny wrapper class around this functionality, so I can use it generally in the future. The most useful thing here, I think, is the ability to have the phone vibrate using a given on-off pattern in a fire-and-forget pattern that works well when programming Compact Framework forms.

This is the simple Vibrator class:

         1: using System;
         2: using System.Threading;
         3: using OpenNETCF.WindowsCE.Notification;
         4:  
         5:  namespace dr.WM.Common
         6: {
         7:     /// <summary>
         8:     /// Vibrator class. Works on HTC Touch Diamond, not tested anywhere else.
         9:     /// (Mostly, The LED index could be different on other devices.)
         10:     /// </summary>
         11:     public class Vibrator
         12:     {
         13:         /// <summary>
         14:         /// Index of the Vibrator LED.
         15:         /// </summary>
         16:         private const int VibratorLedIndex = 1;
         17:         /// <summary>
         18:         /// LED instance.
         19:         /// </summary>
         20:         private readonly Led led = new Led();
         21:         /// <summary>
         22:         /// Whether the Run thread is allowed to run.
         23:         /// </summary>
         24:         private bool allowRun = false;
         25:         /// <summary>
         26:         /// Starts this instance.
         27:         /// </summary>
         28:         public void Start()
         29:         {
         30:             allowRun = true;
         31:             led.SetLedStatus(VibratorLedIndex,Led.LedState.Blink);
         32:         }
         33:  
         34:         /// <summary>
         35:         /// Stops this instance.
         36:         /// </summary>
         37:         public void Stop()
         38:         {
         39:             allowRun = false;
         40:             led.SetLedStatus(VibratorLedIndex, Led.LedState.Off);            
         41:         }
         42:  
         43:         /// <summary>
         44:         /// Starts a vibrating sequence by specifying the vibrate and pause times.
         45:         /// Vibration will run until the Stop method is called.
         46:         /// </summary>
         47:         /// <param name="msVibrate">The vibrate time in milliseconds.</param>
         48:         /// <param name="msPause">The pause time in milliseconds.</param>
         49:         public void StartSequence(int msVibrate, int msPause)
         50:         {
         51:             StartSequence(msVibrate,msPause,0);
         52:         }
         53:         /// <summary>
         54:         /// Starts a vibrating sequence by specifying the vibrate and pause times.
         55:         /// Vibration will run for the specified total time, or until the Stop method is called.
         56:         /// </summary>
         57:         /// <param name="msVibrate">The vibrate time in milliseconds.</param>
         58:         /// <param name="msPause">The pause time in milliseconds.</param>
         59:         /// <param name="totalLength">The total time to vibrate.</param>
         60:         public void StartSequence(int msVibrate, int msPause, int totalLength)
         61:         {
         62:             allowRun = true;
         63:             ThreadPool.QueueUserWorkItem(Run,
         64:                                          new RunState
         65:                                              {VibrateTime = msVibrate, PauseTime = msPause, TotalTime = totalLength});
         66:         }
         67:  
         68:         /// <summary>
         69:         /// Thread worker for a vibrating sequence.
         70:         /// </summary>
         71:         /// <param name="state">The state.</param>
         72:         private void Run(object state)
         73:         {
         74:             long begin = Environment.TickCount;
         75:             RunState runState = (RunState)state;
         76:             while(allowRun && (runState.TotalTime <= 0 || Environment.TickCount - begin < runState.TotalTime))
         77:             {
         78:                 led.SetLedStatus(VibratorLedIndex, Led.LedState.Blink);
         79:                 Thread.Sleep(runState.VibrateTime);
         80:                 led.SetLedStatus(VibratorLedIndex, Led.LedState.Off);
         81:                 Thread.Sleep(runState.PauseTime);
         82:             }
         83:         }
         84:  
         85:         /// <summary>
         86:         /// Helper for passing vibration state to the worker thread.
         87:         /// </summary>
         88:         private struct RunState
         89:         {
         90:             public int VibrateTime { get; set; }
         91:             public int PauseTime { get; set; }
         92:             public int TotalTime { get; set; }
         93:         }
         94:     }
         95: }

Please note that this might (propably) will not work on other devices, since the vibrator might not be on the same LED index. One could refactor the class and make a couple of vibrator on/off virtual protected methods, and call these from the Start / Stop methods. That way, it could be easy to make the class general enough for use on other devices, you would just need to implement the start and stop operations. However, there might be an easier way of doing this using an unmanaged API (actually I hope there is, since collecting info about all types of devices in order to figure out how to fire the vibrator, seems as an unfeasible task).

It seems that the Klaxon Open-Source alarm clock for Windows Mobile has just been made Open Source. I think I will have a look at the source to see whether my way of using the vibrator is feasible, or the Klaxon author uses a better approach ;-)


A Stopwatch for Windows Mobile

I have got a new mobile phone, a HTC Touch Diamond. Besides the fact that it has a sleek design and is much easier to work with when reading email and browsing the web than my old phone.

However, that it is not the only reason for buying the Diamond. Another, very important reason, is that it runs Windows Mobile 6.1 - and therefore I can write my own programs for it using pretty much the same toolset as I use for any other .NET program. Granted, there are stuff missing in the Compact Framework compared to the full-blown framework (Expression trees anyone ?), but it is normally quite easy to find alternatives, and the Compact Framework does make it quite easy to program the device.

My first application for the Touch is a simple Stopwatch program. I wrote it, because there was no stopwatch and/or timer program on the Touch when I got it, so why not write my own ;-) The application it is quite simple, but I learned quite a deal about the device and the Compact Framework while developing it. It essentially relies on the Environment.TickCount counter to measure time, so it might not be 100% accurate - but for my needs (such as heating pizza's), it is quite sufficient.

If anyone's interested, you may download the source from here. If you want to compile it, you will need a copy of the OpenNET CF Framework, because I needed to use some parts of it for making the phone vibrate when the alarm goes off. (It could be replaced with some P/Invoke calls, but i got lazy ;-)

The application itself has the following features:

  • Simple stopwatch
  • Timer with alert (vibration and sound)
  • Configurable alarm sound (only .wav files, sorry).
  • Settings are remembered (stored in Application Data)

Last Day at Jaoo

Wednesday was the last official day of the JAOO conference, and once again it featured a bunch of interesting talks. I attended these:

50 in 50
This was todays keynote by Richard P. Gabriel and Guy L. Steele Jr. Before the talk, there had been some speculations about the title; was it 50 programming languages in 50 minutes ? Or what did it mean exactly ? It turned out to be 50 comments about programming and programming languages, in 50 minutes. These focused on the history of programming and did so in an entertaining and enlightening manner. This was a certainly a great talk - and for a "young" programmer like my self what was not even born when Ada and Algol 60 appeared; it provided also some historical insight. Only downside to this talk, is that the schedule was affected by the fact that it was more like 50 in 75 - the talk took about 75 minutes; but with this quality of technical and on the same time entertaining talk, that does not really matter for me.

Five Considerations For Software Developers
This was also a dual talk with two presenters - Frank Bushmann and Kevlin Henney. They talked about architecure and specifically five considerations that drives design quality. Those were:
  • Economy - the idea that software must be built for a reason and should not have an complicated or elaborate design just because it *might* be needed in the future.
  • Visibility - in the sense that the design must be easily discoverable.
  • Spacing - basically the idea to separate concerns and make sure not to bake the design into deep inheritance hierarchies that xould better be expressed with composition.
  • Symmetry - in that API's should be symmetric with the example that if you can create something with a Factory, said Factory should also be able to destroy it
  • Emergence


LINQ + New Microsoft Things
This talks title is actually wrong, since Erik Meijer primarily talked about LINQ, and very little about "New Microsoft Things". To be fair, he did not have much time to cover it all since the talk got started late because of the schedule slip at the keynote earlier on the day. LINQ was covered well, however, and from a slightly different angle than Anders Hejlsberg talked about earlier in the week. Erik talked about Expression trees and how they represent code as data. This makes it possible to hand an expression tree to an interpreter for a given query language, that can then execute it in the given domain. This is why we (in theory) could forget all other query languages such as XQuery or SQL, and only use Linq-to-Xxx - given that someone writes a Xxx extension to LINQ, of course.

Real World Refactoring
This talk about Refactoring by Neal Ford addressed the challenges that goes into actually performing refactorings in code. It was very hands-on and offered some good advice on how to structure refactorings. One of the best pieces of advice, I think, was to time-restrain major (multi-day) refactoring efforts to an estimated period of time before-hand. If you cannot complete the planned refactoring in the planned time, take the time to rethink the problem, and find out if you are doing it right. If not, you can throw the refactored code away and try again, instead of keeping on a track, that might resolve to more complicated code than before; because new knowledge has beeng gained during the process or because the refactorings was not well enough planned and thought out in advance.

JavaScript As An Assembly Language
This second presentation by Erik Meijer was primarily about Volta, an exciting new technology from Microsoft's Live Labs. The project basically promises to make it easier to make multi-tier applications that can run on the server and work with any client, with parts being executed on the client. This is done by decorating methods with custom attributes, that marks them for running on the client. The Volta compiler will then "compile" those to javascript, that can run on any client (or, if Silverlight is available on the client, the code will run in Silverlight as .NET IL). Erik explained the technology behind, and how they generate javascript code and the various problems involved in that. I do not think that this technology is quite ready to be used in the wild yet, but it should definitely be interesting to see how it evolves in the future. The documentation site on Live Labs seems to be down for the moment, however, this blogpost also explains the technology in more detail.

Concurrent Programming with Concurrent Extensions to .NET
In this talk, Joe Duffy, gave an introduction to the parallel extensions to .NET, a new API for writing concurrent applications with .NET. These extension is in CTP right now (so it's preview technology, not recommandable for production use). Joe promised though, that these APIs will be part of the .NET Framework version 4 release. These new APIs promise to make it easier to write concurrent applications with .NET with little overhad, both mentally for the programmer, but also performance-wise for the machine. The presentation featured running demos and code, and I believe that the new APIs are quite well-designed and that there is definitely a need for this kind of API in todays world of multi-core hardware. However, as Joe pointed out, there is no such thing as a free lunch; and even when using this API, of course you need to think hard over concurrency issues and side-effects before you can put it to use. The system makes it easier for you to program concurrently; but you can still fail badly if you do not understand what it does under the covers.

JAOO Day Two

Today on JAOO has also been packed with interesting talks. I attended these:

V8: The Javascript engine inside Google Chrome
Keynote by Lars Bak about the all-new javascript engine that was implemented by his team in Århus for Google Chrome. This javascript engine is about 10 times faster than other javascript implementations. Lars explained how this is possible in a very dynamic language like javascript, by creating classes that can be reused; if another object with the same properties is created, which tends to happen often. Optmizations can then be applied to those classes. Furthermore, Chrome compiles the Javascript code to native code so that it can run really fast on any platform. Lars mentioned a few other major improvements, and this was a very inspiring and interesting talk - even though he had some problems with the projector in the beginning.

Failure Comes In Flavours
This talk by Mikael Nygard (who is not danish, by the way, though his name resembles a danish name pretty much) was divided into two sessions. In the first, Mikael talk about anti-patterns that lead to failure - such as depending too hard on third parties, or waiting forever for some external call. This talk featured some "war stories" about failures Mikael had helped to resolve in the past. In the second session, Mikael offered his advice on how to avoid failure and some patterns that can help in this. Though focus was very much on big enterprise SOA systems; the principles can be applied anywhere. I learned a thing or two in this session, that can be applied to my every-day work on web applications in a smaller scale.

Not your Grandfather's Architecture: Taking Architecture into the Agile World
In this talk, James Coplien talked about Agile architecture. He presented his ideas about adding roles as a concept to the object-oriented world of classes and objects. He argued that programs could be made simpler and get rid of polymorphism using his approach; I am not sure that I agree. There was no working code demo, so his ideas is still kind of abstract in my mind.

Successfully applying REST - Integration, Web-style
Stefan Tilkov talked about REST integration; a subject that I find very interesting, since we already use it in projects at work, and are planning to use it even more extensively. Stefan had some very interesting points about REST, and how it can be applied to a SOA world. I really like the clean interface you can make of a REST service, without much of the overhead and scaffolding that is neccessary in SOAP, for instance.

Top Ten Software Architecture Mistakes
This was a talk focusing on what not to do in architecture; so that we can avoid mistakes or bad decisions in our architecture. The talker, Eion Woods, had his list of 10 mistakes and how to avoid them, which was presented with a little bit of humour. At least some of the items is going to be on my list of things to check before beginning development of new projects.


JAOO Day One

Today, I've been attending the first day at the JAOO Conference. It has been an interesting day, and I am amazed by all the very talented people that speaks here at the conference, as well as by the quality of the talks. I have been attending these sessions:

Opening Keynote: Where Are Programming Languages Going, by Anders Hejlsberg
In this talk, Anders presented his take on how the programming languages will be evolving in the future. One point made, was that programming languages has not evolved much over the last 25 years - which was exemplified by a "Hello world" program in Pascal versus one in C#. The C# one was neither the shortest nor the most concise one. Anders believes that we will se more implementation of what he calls "internal domain-specific-languages", such as LINQ. Furthermore, he thinks that in the future the gap between functional and imperative programming languages will diminish, since they are already today starting to borrow the best elements from each other. Finally, he talked about concurrency and in-language concurrency constructs such as isolation, which Anders predicts also will be part of the main-stream languages in the near future.

Continuous Integration
The Continuous Integration talk was a great introduction to me into Continuous integration, since it is something, that I have little to zero experience with. We do have a build server that does nightly builds at work, but as Chris Read, the speaker, pointed out, that is not CI, though it is a step on the way toward succesful CI.

Google Chrome: The Invisible Browser
This was a talk by Ben Goodger, the Google Chrome UI tech lead, about the minimalist Chrome UI and the architecture and decisions behind it. It was interesting, though there was not much new to learn here. (Or perhaps I should know better than to attend UI talks, when UI does not really interest me. I am trying to learn ;-) )

LING and C# 3.0
This was the second talk by Anders Hejlsberg this day, and featured the new features in C# 3.0. This was info that I knew (mostly) in advance, but Anders explained both the how and the why behind the features - which was really interesting.

PowerShell
The talk about PowerShell was really good; even for me who know and use PowerShell in advance. It got beyond the covers on why the architecture and implementation works as it does, which was interesting and enlightening, and I left with a better understanding of PowerShell.

The Scala Programming Language
Scala is a language for the JVM, that I did not know much about in advance. The talk was interesting, but in "real life", I am probably never going to use it.

Why Functional Programming (still) Matters
This talk by Erik Meijer was propably the most interesting and entertaining one on Day One of JAOO. With enthusiasm, Erik explained about side-effects and why they are bad and what one should do about them (make them explicit if they cannot be avoided). He also demonstrated a few side effects, that can hit you in C# or other main stream languages with closures and lambdas, which was a pleasant reminder for me.


Attending JAOO

I will be attending the JAOO conference the next couple of days. It is my first time, but I have high expectations. I am not sure which sessions I will attend, but of course the opening keynote by Anders Hejlsberg will be a must.

I find it hard to choose - there are so many interesting subjects and speakers; and so little time ;-) If you are going to JAOO, feel free to drop a comment with suggestions on which sessions is a must for you, and why.

Parsing XML with PowerShell

I'm addicted to PowerShell. This cool scripting environment is simple to use, and with very few lines of script; it is possible to accomplish tasks that otherwise often would be a lot of tedious work. (If we didn't have PowerShell, I would propably wip up a C# program to do the same, but PowerShell is really lightweight, is interactive and is generally very forgiving for small tasks where you just "want the job done".

As an example, today I needed to look at a log files generated by Visual Studio to figure out why the environment wouldn't start on my home PC. As it turns out, these log files are actually XML files. Of course I could have just started reading through the XML, but all the angle brackets confuses my brain; when I'm actually mostly interested in the text content of the log file.

So, five minutes later, this 3-line script; parse-vslog.ps1 was born:

1: param( [string]$file = $(throw "required parameter" ) )
2: $log = [xml](get-content $file)
3: $log.activity.entry | select record,type,description | format-table -wrap -auto

This is what happens in the script:

On line 1, we declare that we need a $file parameter (variables and parameters is prefixed with $ in PowerShell), that should be required.

On line 2 we use the get-content cmdlet to get the contents of a file. PowerShell has a lot of XML helping features, one of which is the ability to "cast" the content to XML using the [xml] construct. What really happens behind the scenes, is that PowerShell instantiates an XmlDocument and loads the text content of the file in that.

Last, on line 3, we take advantage of the fact that PowerShell let's us select XML nodes by using simple dotted notation. Here we are interested in all the the /activity/entry nodes. We pass the result along the pipeline and selects the 3 most important values using the select cmdlet. And, lastly, we format the output nicely with format-table, specifying that we would like the cmdlet to auto-select the column widths (-auto) and that text output should be wrapped on multiple lines (-wrap).

So insted of having to look at XML that goes on like this:

1: xml-stylesheet type="text/xsl" href="ActivityLog.xsl"?>
2: activity>
3:   entry>
4:     record>1record>
5:     time>2008/06/15 15:44:18.220time>
6:     type>Informationtype>
7:     source>Microsoft Visual Studiosource>
8:     description>Visual Studio Version: 9.0.21022.8description>
9:   entry>
10:   entry>
11:     record>2record>
12:     time>2008/06/15 15:44:18.221time>
13:     type>Informationtype>
14:     source>Microsoft Visual Studiosource>
15:     description>Running in User Groups: Administrators Usersdescription>
16:   entry>
17:   entry>
18:     record>3record>
19:     time>2008/06/15 15:44:18.221time>
20:     type>Informationtype>
21:     source>Microsoft Visual Studiosource>
22:     description>ProductID: 91904-270-0003722-60402description>
23:   entry>
24:   entry>
25:     record>19record>
26:     time>2008/06/15 15:44:19.094time>
27:     type>type>
28:     source>Microsoft Visual Studiosource>
29:     description>Destroying Main Windowdescription>
30:   entry>
31: activity>
32:  

Now, I can get this much nicer output in the console (note that the XML above has been shortened for the blog. It was actually around 150 lines):

record type        description
------ ----        -----------
1      Information Visual Studio Version: 9.0.21022.8
2      Information Running in User Groups: Administrators Users
3      Information ProductID: 91904-270-0003722-60402
4      Information Available Drive Space: C:\ drive has 42128211968 bytes; D:\ drive has 38531145728 bytes; E:\ drive h
                   as 127050969088 bytes; F:\ drive has 117087354880 bytes
5      Information Internet Explorer Version: 7.0.6001.18063
6      Information Microsoft Data Access Version: 6.0.6001.18000
7      Information .NET Framework Version: 2.0.50727.1434
8      Information MSXML Version: 6.20.1076.0
9      Information Loading UI library
10     Information Entering function CVsPackageInfo::HrInstantiatePackage
11     Information Begin package load [Visual Studio Source Control Integration Package]
12     Information Entering function CVsPackageInfo::HrInstantiatePackage
13     Information Begin package load [team foundation server provider stub package]
14     Information End package load [team foundation server provider stub package]
15     Information End package load [Visual Studio Source Control Integration Package]
16     Information Entering function VBDispatch::GetTypeLib
17     Information Entering function LoadDTETypeLib
18     Error       Leaving function LoadDTETypeLib
19                 Destroying Main Window
 

I think this is a good representative of the strength of PowerShell. Using only a few lines of script and a minimum of time, I created a reusable script, that will probaply save a lot of time in the future.


ReSharper 4 Available

The good folks over at Jetbrains has finally released version 4 of their ReSharper productivity enhancing tool with support for C#3.0.

Highly recommended.


Clever use of C# 3.0 LINQ Expressions

Jafar Husain shows us a quite clever way to use a C# 3.0 LINQ expression to get a symbol name. I think this is a really good idea to use in cases when you need the name of a symbol as a string, since it avoids using hard-coded strings and gives us the option to use automatic refactorings without breaking stuff. And since it is implemented as an extension method; it does not pollute the interface of your classes. There might be a slight performance penalty when using this method; but I think it will be neglible for all but some extreme cases. But if you need to use it in a tight loop, you should propably make some performance measurements in advance ;-)

Now, if only C# 3.0 had been available a couple of years ago, when I wrote a lot of statically typed datasets (automatically generated from the database schema using CodeSmith, of course). The bulk of the code in those datasets where properties that would generally retrieve a value from a DataRow in a column named the same as the property. If I could have used the approach mentioned above, I am sure it would have saved me some sweat during later refactorings of the code in question.

It is a Good Thing our tools and languages continually evolves.


How to create an ASP .NET Captcha Control (part 2)

This is the second in a 3 part series on how to create an ASP .NET Captcha control. The previous post can be found here. This time we will look at how the CAPTCHA image can be generated using the built-in .NET framework classes.

Image generation approach
As described in the first post, the idea is to create an image showing a word, and let the user repeat it by typing it into a textbox. The image should be hard to read for OCR software, so that the CAPTCHA is hard to beat for automated bots. The way we will be doing this, is by stretching and warping the text, and adding noise. Luckily, this is easy to accomplish by using the GraphicsPath class to draw the string, and then use the Warp method on the GraphicsPath object.

Generating the image: Step by step
The first step in generating the image is to create a Bitmap object with the appropiate dimensions. We will also need a Font for the text, and a Brush for painting the text. I also declare a rectangle that is slightly smaller than the actual image, which will be used as the drawing bounds later. This will help to ensure, that the text fits on the image after the transformations:

        /// 
        /// Generates the CAPTCHA image.
        /// 
        /// 
        public static byte[] GenerateImage()
        {
            // Create image.
            var image = new Bitmap(size.Width, size.Height, PixelFormat.Format24bppRgb);
            var imgRectangle = new Rectangle(10, 10, image.Width - 10, image.Height - 10);
            // Get font and brush.
            var brush = new HatchBrush(HatchStyle.SolidDiamond, Color.Black, Color.FromArgb(rand.Next(160),rand.Next(160),rand.Next(160)));
            var font = new Font(fonts[rand.Next(fonts.Length - 1)], imgRectangle.Height, FontStyle.Italic, GraphicsUnit.Pixel);
 
Notice that we use a HatchBrush so that the word will be drawn using a hatch pattern. Ensuring that the text is not solid color, will help defeat OCR attacks. The actual font used is also chosen at random from a predefined list.
				
The next step is to get a Graphics object from the image, and use it to draw on the image. We'll wrap the code using the Graphics object in a using region to ensure that the instance is disposed as soon as we don't need it more. We then fill the background with white color and create a GraphicsPath instance, to which the selected Captcha word is added using the AddString method. The path object can now be warped, by stretching the corners a random amount. We also rotate the text a bit (between --10 and 10 degrees):

 
            // draw on the image.
            using(Graphics g = Graphics.FromImage(image))
            {
                g.FillRectangle(Brushes.WhiteSmoke, 0, 0, image.Width, image.Height);
                var path = new GraphicsPath();
                // Make sure text fits
                while (g.MeasureString(CaptchaWord, font).Width > imgRectangle.Width)
                    font = new Font(font.FontFamily, font.Size - 1, font.Style);
                
                path.AddString(CaptchaWord, font.FontFamily, (int)font.Style, font.Size, imgRectangle, StringFormat.GenericDefault);
                float v = 4;
                var warpPoints = new PointF[]
                                           {
                                                new PointF(rand.Next(imgRectangle.Width) / v,  rand.Next(imgRectangle.Height) / v),
                                                new PointF(imgRectangle.Width - rand.Next(imgRectangle.Width) / v,  rand.Next(imgRectangle.Height) / v),
                                                new PointF(rand.Next(imgRectangle.Width)/v, imgRectangle.Height - rand.Next(imgRectangle.Height) / v), 
                                                new PointF(imgRectangle.Width - rand.Next(imgRectangle.Width) / v, imgRectangle.Height - rand.Next(imgRectangle.Height)/ v) 
                                            };
                var warpMatrix = new Matrix();
                warpMatrix.Rotate(rand.Next(20) - 10);
                path.Warp(warpPoints, imgRectangle, warpMatrix, WarpMode.Perspective);                
                g.FillPath(brush, path);
 
The next step is to add a bit of noise to the image. This is done by drawing some small elipses (dots) randomly in the image, with a random color. This is implemented with a LINQ query selecting the details for each random dot:
 
                // Add some noise.
                var noise = from e in Enumerable.Range(0, NoiseAmount)
                         select new
                                    {
                                        X = rand.Next(image.Width),
                                        Y = rand.Next(image.Height),
                                        R = 1f + (float)rand.NextDouble() * 3f,
                                        Brush = new SolidBrush(Color.FromArgb(rand.Next(255), rand.Next(255), rand.Next(255)))
                                    };
 
                foreach (var p in noise)
                    g.FillEllipse(p.Brush, p.X, p.Y, p.R, p.R);
            }
 
Finally, the resulting image is saved to PNG format in-memory and returned from the method. 
 
   // Save to buffer and return raw png image bytes.
            using(var buffer = new MemoryStream())
            {
                image.Save(buffer, ImageFormat.Png);
                return buffer.GetBuffer();
            }
 
We will use a custom http handler by implementing IHttpHandler to send the image to the client. This will be the subject for the next post in the series. 

How to: Create an ASP.NET CAPTCHA Control (part 1)

As I explained in my previous post, I developed a CAPTCHA ASP.NET control for this blog. In the next few posts, I will explain the steps involved in doing this, and how you can develop your own CAPTCHA control.

Preparations
There are some variations on CAPTCHA tests, the most common one requiring the user to input the characters displayed on an image. The idea is that only a human will be able to read these characters; so if the challenge response is correct, it is most likely a "real human" submitting the data. Since modern OCR software can be quite efficient, it is neccessary to make the charaters hard-to-read by altering shape, adding noise or lines. Of course these measures also make the CAPTCHA harder to read for a human. For my CAPTCHA control, I decided to create a control, that emphasizes on ease-of-use for the end user. Therefore, the images generated should be easy to read.

When deciding which characters to display on the image, there are generally two approaches: Generate some randomly, or choose between a pre-defined set of words. I choose the latter approach, since it would be easiest for a human to recognize an actual word. Therefore, I am storing a list of English words, from which I select one randomly whenever I need to generate a CAPTCHA.

Step one: Creating the basic control
I have chosen to implement the CAPTCHA as a UserControl, so that the look and/or different parts of the control can be changed at a later time, if I need to do so. So I created a UserControl and placed an image tag and a textbox on it. These are the essential parts of the CAPTCHA control.

The basic control implementation does the following: Whenever the control is shown, a word is selected randomly for the challenge. A unique, random URL for the CAPTCHA image is also generated. The purpose of using a unique URL is to ensure that the browser does not display an old CAPTCHA image because it caches it locally.

The selected word is stored in Session state. Alongside the URL, it is exposed as a public static property, that populates on-demand. This makes sure that the image-rendering code will be able to get the correct word, and the encapsulation ensures that I can change the storage if necessary. This is the implementation of these two properties:

1:         /// 
2:         /// Gets the captcha URL.
3:         /// 
4:         /// The captcha URL.
5:         public static string CaptchaUrl 
6:         { 
7:             get
8:             {
9:                 if (MyContext.Session[CaptchaUrlKey] == null)
10:                     MyContext.Session[CaptchaUrlKey] = String.Format("/captcha/{0}.ashx", rand.Next());
11:                 return (string)MyContext.Session[CaptchaUrlKey];
12:             }
13:         }
14:  
15:         /// 
16:         /// Gets the captcha word.
17:         /// 
18:         /// The captcha word.
19:         public static string CaptchaWord
20:         {
21:             get
22:             {
23:                 if ( MyContext.Session[CaptchaWordKey] == null)
24:                 {
25:                     string listWords = Settings.User["CaptchaWords"];
26:                     var words = listWords.Split(',');
27:                     MyContext.Session[CaptchaWordKey] = words[rand.Next(words.Length - 1)].Trim();
28:                 }
29:                 return (string)MyContext.Session[CaptchaWordKey];
30:             }

When the control is displayed, the image on the control is databound to the CaptchaUrl property; so it will display the image containing the correct word. The request the browser sends for the image will get handled by a separate http handler (which we will discuss in a later post); which will output the generated image.

On postback, the control will check the text the user has entered, and if it matches the generated word, a public property called "IsValid" will be set to true. This indicates to the control on which our CAPTCHA resides, that the user has passed the CAPTCHA test. After the check, the word and URL is reset, so a new CAPTCHA will be generated if the control is shown again.

A slightly better approach would be to implement the control as a .NET Validator control, so that it could take part in the page validation along with other validator controls. This would eliminate the need of the other controls on the page being aware of the CAPTCHA. Doing this would not be much more work; one would simply need to inherit from the abstract BaseValidator class and implement the neccessary methods.


Hacking ASP.NET: Trace information

All ASP .NET developers propably know about the trace feature in ASP .NET. Provided you have enabled tracing in web.config, (using <trace enabled="true" /> in the system.web element; requesting the url /trace.axd will provide you with a nice list of trace information for the previous requests.

I have often thought about putting the wealth of information to better use; perhaps making more detailed reports based on the trace information. This could be useful during testing. Unfortunately, as far as I can tell, there is no other way to get the information, than requesting Trace.axd. There seems to be no supported programmatic way of doing this.

So I set about finding out, how this could be done. At first I thought about creating a screen-scraper for requesting trace.axd and collecting the information. But this would be impractical; especially when large amounts of data should be collected.

A better approach seemed to be to find out how ASP .NET actually stores this information. Since trace.axd is actually an IHttpHandler (System.Web.TraceHandlers.TraceHttpHandler), the natural starting point was using Reflector to view the internals of this class. It did not take long to figure out, that the HttpRuntime class has a static internal property named Profile of the type System.Web.Util.Profiler, which is internal. This is the class responsible for collecting the Trace information, and has a GetData method. This method returns the current trace information as an IList containing DataSets.

Armed with this information, I wrote a small class that uses reflection to obtain the profiling data. The class looks like this:

   1:  using System;
   2:  using System.Collections;
   3:  using System.Collections.Generic;
   4:  using System.Data;
   5:  using System.Linq;
   6:  using System.Reflection;
   7:  using System.Web;
   8:   
   9:  namespace dr.TraceAnalyzer
  10:  {
  11:      /// 
  12:      /// Proof-of-concept class for accessing trace data using reflection.
  13:      /// 
  14:      public class TraceData
  15:      {
  16:          /// 
  17:          /// Data
  18:          /// 
  19:          private IList data = null;
  20:          /// 
  21:          /// Gets the trace data in its raw list-of-datasets representation.
  22:          /// 
  23:          public IList Data
  24:          {
  25:              get
  26:              {
  27:                  if (data == null)
  28:                      GetCurrentData();
  29:                  return data;
  30:              }
  31:          }
  32:   
  33:          /// 
  34:          /// Returns the response time for each request stored in the trace data.
  35:          /// 
  36:          public IEnumerabledouble>double> RequestResponseTimes
  37:          {
  38:              get
  39:              {
  40:                  GetCurrentData();
  41:                  var sets = from d in Data.Cast()
  42:                             select d;
  43:                  return from set in sets
  44:                               let traceTable = set.Tables["Trace_Trace_Information"]
  45:                               where traceTable != null && traceTable.Rows.Count > 0
  46:                               select (double) traceTable.Rows[traceTable.Rows.Count - 1]["Trace_From_First"];
  47:              }
  48:          }
  49:   
  50:          /// 
  51:          /// Gets the current data from the Profiler instance's GetData method.
  52:          /// 
  53:          /// 
  54:          public IList GetCurrentData()
  55:          {
  56:              var profiler = GetProfiler();
  57:              Type profilerType = profiler.GetType();
  58:              MethodInfo method = profilerType.GetMethod("GetData", BindingFlags.Instance | BindingFlags.NonPublic);
  59:              return data = (IList) method.Invoke(profiler, null);
  60:          }
  61:   
  62:          /// 
  63:          /// Use reflection to get the Profiler instance.
  64:          /// 
  65:          /// 
  66:          private object GetProfiler()
  67:          {
  68:              Type runtimeType = typeof (HttpRuntime);
  69:              PropertyInfo profileProperty = runtimeType.GetProperty("Profile",
  70:                                                                     BindingFlags.NonPublic | BindingFlags.Static);
  71:              if (profileProperty != null)
  72:              {
  73:                  return profileProperty.GetValue(null, null);
  74:              }
  75:   
  76:              throw new ApplicationException("Reflection to get profiler instance failed.");
  77:          }
  78:      }
  79:  }

I have yet to decide what I am going to use the trace data for. But an obvious way to use it would be to represent some of the performance data that is collected, as a graph. For now, I have added a property, RequestResponseTimes, that returns a list of the total time taken for each request stored in the trace data.

 

And, please remember to disable tracing when putting your site into production ;-)


Fighting comment spam

I've been hit by comment spam. Suddenly, one of the posts on this site had _a lot_ of comments, all with advertisements for some suspect sites. Needless to say, I've removed those comments.

So, what to do about that ? I decided to implement a CAPTCHA on the site. It is a pretty standard one, requiring you to repeat a word, that is shown as an image and garbled, so that image recognition software has a hard time interpreting it. I could have found a complete control for it in a few minutes just by searching Google; but I implemented it myself as an ASP .NET control, just for fun. Also, I believe that the security in a unique CAPTCHA algorithm is much better. If spammers develop software to defeat CAPTCHA's, naturally they are only going to target big sites to maximize their profits, and not bother trying to break a CAPTCHA, that is only used on my little site.

A nice example on this is the fact that Jeff Atwood's blog is using one of the simplest CAPTCHAs conceivable, a static image containing the same word each time, that the user must repeat in a textbox. Apparently, that is enough to stop most of the spam on his blog. Another example of a really simple CAPTCHA is to simply include a hidden field on the page. If the field gets filled out on postback, it is most likely a spam-bot posting it, since the average user would never notice, let alone filling out, the hidden field. I like that idea particularly, because it does not require the user to think or do anything. (So why did I go for the image approach in implementing my own CAPTCHA ? Probably because I wanted to try out implementing one ;-))

I am probably going to blog about the techniques going into developing a CAPTCHA in ASP .NET in the near future. In the meanwhile, dear reader, please try out the comments feature, and let me know if you find the CAPTCHA image easy enough, or too easy, to read.


Got one of those RSS thingys

Someone pointed out that there was no RSS feed on my blog - i forgot to add it.

It has been fixed now, and you should be able to read my blog in your favourite RSS aggregator.


Debugging ASP .NET memory leak - watch for static references

A while ago, I needed to debug an ASP .NET website, which was eating more and more memory the longer it was running, and never seemed to release the memory again. Even though the number of concurrent users was more or less constant, the memory usage seemed to be proportional to the total number of sessions since last application restart.

What was happening here ? Could it be session-state elements that never got garbage collected ? To find out, I simulated load on a test version of the website, let the sessions time out and used WinDbg and the Son of Strike dll extension command "gcroot" to find out which objects was stll alive and why they could not be collected by the garbage collector. First I forced a few garbage collections by calling GC.Collect() to make sure that the garbage collector had actually run.

I found out that some objects, that we keep in Session was rooted to an object array, which, it turned out, represented static references.
After some investigation, I found the problem in the code: A developer had created a static event, and had the objects in session subscribe to this event. This meant that the session objects could not be collected, since there were still reachable references to the objects.

It turned out that the event did not need to be static, so the fix was rather easy to implement. The debugging of the issue, however, took some time.

The morale of this is, that even though we have managed code with memory management, you should still keep an eye on memory usage during development. And with ASP .NET, you should be very careful with static references, since they could root your objects so that they can't be garbage collected.

You should also be aware, that static objects are shared for all users. This should not come as a surprise, if you know just a bit about the ASP .NET process model, but it is my experience that it is easy to forget, even for experienced developers. Because the static objects are shared, you should treat them as such, and remember to use proper synchronization methods when accessing them..

I might blog more in the future about this kind of issues.


Built on Sitecore Xpress

As from today, this blog runs on Sitecore Xpress. I have spent the day setting it up.

So, what is Sitecore Xpress ? Sitecore Xpress is a personal edition of the professional Sitecore CMS product, which just recently was released and made free of charge for developers. This is really cool, since Sitecore is a very good, mature and professional CMS, that has loads of features and is built on .NET. It also has a lot of extension points, which makes it a realy nice development platform for large enterprise websites. The free Xpress edition has almost all the features of the full product, though it is restricred to one user and can only be used for personal, non-commercial sites (details here).

Since we use Sitecore a lot at work, and it is so developer-friendly, the decition to use Sitecore Xpress for my personal website, was really easy to make. It will make a good platform for the various features I would like to add to my web site in the feature.

A nice example on how easy use and setup Sitecore is, that although there are no built-in blogging application in Sitecore, it took me just a day to setup the entire website, including the blog and migration of content from the old platform. The blog is actually a tree of so-called Sitecore content Items. Items is the cornerstone of Sitecore. Using a few XSLT renderings, the blog page, archives and individual posts and comments can be displayed. No additional code was written to achieve this. The only code I have written for this website so far, is a small User Control ( < 30 lines of code ) that allows users to add comments to the articles.

I am probably going to be blogging a bit more about Sitecore and ASP .NET related subjects in the future. I also hope that this change of platform will increase my motivation for writing more posts, so the frequency of new posts should increase.


NEF File reader is on hold ...

I have not had very much time to blog nor work on my NEF file reader project for a while. It has been turbulent and busy times at work, lately.

This post is just to let you know, that the project is curently on hold. Perhaps I will resume it later, when I have more free time at hand. If anyone is interested in a peek at the source code as it looks now (incomplete and raw), please let me know; and I can mail it to you. There is not too much functionality, however. The largest obstacle in getting further with the project, is the need to decode the Huffman-encoded sample bit lengths embedded in the file - I have not resolved this yet.


NEF file reader update

Progress on the NEF / RAW file reader is slow, since i only work on it occasionally when I feel like coding in my spare time. However, I made some progress tonight, that I would like to update you about.

First, NEF files are really TIFF files. So I need to parse them as a TIFF file to get some useful information out of them. TIFF files consists of so-called IFD tags, which in parts contains metadata about the image as well as the actual image data. Currently I have developed a basic IFD tag parser, that parses all of the IFD tags in a NEF file.

Furthermore I have found, by looking at the IFD tags, that there are at least 3 embedded images in each NEF files. These are a small, low-quality 160x120 thumbnail represented as RGB data, a nearly full size JPG in low quality, as well as the actual NEF image data. While implementing the IFD parser, I had a lot of help from this TIFF FAQ, as well as from the official Adobe TIFF specification version 6.

Of course, the NEF image data is the interesting part. I have studied the file format, and made these conclusions; mostly based on what I can tell from the dcraw source code:

  • The NEF file consists of Width * Height samples.
  • The data is represented as a Color Filter Array, that is each sample represents the intensity of a single color (since this is what the camera shoots, as many other digital cameras);
  • therefore, I must interpolate 2 of the colors for each sample, to get a full-color, full-scale image.
  • Each sample is 12 bits.

But this is not all; it is not so simple as to just read 12 bits per sample. The data is compressed. As far as I can tell currently; the following is the compression scheme.

  • A µ-law type compression curve is used to limit the number of possible sample values to 634 (and consequently companding the 12 bit values to log2(634) = 9.4 bits of actual accuray; while preserving the 12 bits dynamic range).
  • The curve values are embedded in the file.
  • Samples are indices to values in the curve.

Samples are encoded (compressed) like this:

  • A simple predictor is used, so each value read is actually the difference between a previous sample value and the current. This keeps values stored in the file low and keeps differences between each value needed to be stored, low.
  • Each value is  stored as a run-length encoded length (in bits) of the value, and immediately thereafter, the actual value.
  • Run length encoding is accomplished by using a classic Huffman table.

So, now I need to implement the reading of the actual sample values and interpolate the values to come up with a full color image. After that, I probably need to implement the reading or approximation of white balance of the image, so that the program will be able to produce actually usable images.


Hobby project - Nikon NEF file reader

So, my new hobby is digital photography. I have bought a slightly used Nikon D50 camera, and has already taken a lot of pictures.

If you're into digital photography, you will know that most cameras can produce both JPG and RAW output, where the JPG images are compressed by the camera firmware before being written to the flash card. The RAW files are uncompressed data as it was sampled by the camera CMOS (typically).  Remembe, JPG is a lossy compression format, so detail will be lost if you just save the JPG's. Therefore, if you are serious about photography, you will want to shoot in RAW.

However, as RAW files are essentially untouched image data, a RAW file needs to be processed by some image processing software before it can be used. This can be a cumbersome process, loading up the camera manufacturers software and maybe Photoshop or another image processing utility, processing each image, perhaps adjusting colors and saving the file into a more common format.

Therefore I have decided to try and create a .NET program that does the conversion for me. For "version 1", it will probably only support the NEF file format that my Nikon produces (Nikon's proprietary RAW format), but perhaps I will add support for others along the way. This program will not be a replacement for professional photo processing software, but is meant to be an easy to use tool, that can be used when one simply needs to convert a NEF file to a JPEG or bitmap representation quickly.

The NEF file format is not documented anywhere that I have been able to find on the web. But, looking at the files, they seem to be based on the TIFF image format, and they carry the TIFF file header.

Also, I have found that Dave Coffin has created an excellent utility for doing just what I want, dcraw. I could just as well just use that, but I think that it will be an interesting challenge to try to implement it myself. But I am quite sure that digging in to the dcraw source code will provide me with most of the details that I need to decode the format. 


Using a HTTP module to enforce a single domain

When I first setup this blog, you could access it on both "driis.dk" and "www.driis.dk". For various reasons, it is best practice to have a single url for the same content (it makes analyzing the site traffic easier, and avoids ambiguous URLs), so I wanted to setup the site to just use the www version of the domain name.

Now, there is a couple of ways you could do this. The easiest way would be to just setup IIS to redirect incoming requests to http://www.driis.dk. However, this site is hosted by a hosting company, where I do not have access to the IIS manager. 

Instead, I decided to write a custom HTTP module to do the redirecting. HTTP modules are great, as they allow you to hook into the ASP .NET application pipeline using the evens of the HttpApplication object. This makes it is a very strong tool to build advanced functionality. In this case, I decided to handle the BeginRequest event to check if the domain name is correct. If not, I issue a HTTP Redirect and ends the request. This way, all of the work associated with rendering the page is avoided if we just want to redirect anyway. The code is really simple, and basically goes like this:

         /// <summary>
        /// Handles the BeginRequest event.
        /// </summary>
        /// <param name="sender"></param>
        /// <param name="e"></param>
        private void HandleBeginRequest(object sender, EventArgs e)
        {
            HttpApplication app = (HttpApplication) sender;
            string currentDomain = app.Context.Request.Url.DnsSafeHost.ToLower();
            if ( currentDomain != TargetDomain )
            {
                string newUrl = String.Format("http://{0}/{1}", TargetDomain, app.Request.Url.PathAndQuery);
                app.Response.Redirect(newUrl);
                app.CompleteRequest();
            }
        }

 Another advantage of this method, is that once it is configured in web.config, it always works. If you had used the IIS method, you would have to remember to configure IIS correctly again if the site switches server.

You can download the code as well as the compiled assembly here: dr.BlogEngine.DomainEnforcement.zip (19,18 kb)

This is a domain enforcement module in its simplest form. It simply looks at all incoming requests, and if they don't match the specified domain, the request is redirected to the corresponding url on the correct domain. Feel free to use the module, expand it or use it as inspiration. Just don't blame me if it kills a kitten.

To use the module, simply add it to the <httpModules> section in web.config:

<add name="DomainEnforcement" type="dr.BlogEngine.DomainEnforcement.DomainEnforcementModule,dr.BlogEngine.DomainEnforcement"/>

Then add an appSetting specifying the domain you would like to use as the primary domain:

<appSettings>
  <add key="EnforceDomain" value="www.driis.dk" />
</appSettings>

That's all there is to it.


Setting up BlogEngine.NET

After deciding to get this blog, I needed to find some software to run it on. I briefly considered writing it myself, but decided that it would be too much effort, and that I probably never would be able to finish it, considering work and other projects.

So I started hunting around for some open source blogging software, and after an hour of googling, I decided on BlogEngine.NET. My main considerations where that it needed to be open source .NET (so I can fix stuff if it breaks), highly customizable and easy to extend. It was also important that it would work without a database, since my current web host only provides access to a sluggish and unreliable MySQL server (but hey, it's cheap ;-)

I also considered dasBlog, but BlogEngine.NET seemed simpler and more suitable for my simple needs. 

Setting stuff up was simpler than expected.  BlogEngine.NET target audience is developers, according to their web page, so I was prepared to be hand-editing a myriad of obscure configuration files - But after downloading the source and creating a virtual directory on my local machine for the application, it just ran. I actually started editing the settings.xml file before I found out that a UI was provided for it. So i customized some settings, and did a bit of changes to one of the default templates, before I was ready to go online. I actually think that setting it up for debugging in Visual Studio took longer than the actual customizations (My home machine runs Vista, and I had some problems with ASP .NET and IIS7, which I was able to resolve by using the "Classic ASP.NET pipelining" mode).

I downloaded the source, so that I could try to compile it for myself and poke a bit around, before ultimately deciding to base my blog on this application. I haven't spent too much time looking at the code, but it seems to be well designed and thought-through.

Anyways, uploading the site via FTP to my web host was no problem. I used Visual Studio to make a precompiled copy of the web site first.

Setting up the blog, customizing it and publishing it took less than 2 hours. I did not get stuck at all, and everything behaved as expected; overall it was a nice experience. So thumbs up to the BlogEngine.NET developers !