Saturday, January 31, 2009

Customizing the code of CCNet : part 3 : Creating a publisher via a plugin

The proces of creating a plugin is already described in the docs, but I'll make a more detailed one here. I'll use the same filePublisher as in my previous post, but this time it will be in the form of a plugin, not as part of the code of CCNet.

Benefits of a plugin :
  • you can easily link to your companies software libs
  • when a new version of CCNet comes out, you do not have to change the code again
Downside of a plugin :
  • Nobody else can extend/improve it
  • if there is a breaking change in CCNet, you will have to make the change to make it work again
Back to the code :
Create a class library, and name the project FilePublisher. You will have the following :


Rename Class1 to FilePublisher, and past in the following code (it's the same as in the previous post:

using System.Collections;
using System.IO;
using System.Xml.Serialization;
using Exortech.NetReflector;

namespace ThoughtWorks.CruiseControl.Core.Publishers
{
[ReflectorType("filePublisher")]
public class FilePublisher : ITask
{
private string resultFile = "Result.txt";

[ReflectorProperty("resultFile")]
public string ResultFile
{
get { return resultFile; }
set { resultFile = value; }
}

public void Run(IIntegrationResult result)
{
PublishIt(ResultFile, result);
}


private void PublishIt(string targetFile, IIntegrationResult result)
{
StreamWriter Result = new StreamWriter(targetFile, false);
System.Text.StringBuilder Info = new System.Text.StringBuilder();

Info.AppendFormat("Project {0} has status {1}", result.ProjectName, result.Status);
Info.AppendLine();
Info.AppendFormat("Modifications :");
Info.AppendLine();
foreach (Modification mod in result.Modifications)
{
Info.AppendFormat(mod.ToString());
Info.AppendLine();
}
Result.WriteLine(Info.ToString());
}
}
}

Next include references to the following dll's, each can be found in the server folder of the installation folder of CCNet.
° NetReflector
° ThoughtWorks.CruiseControl.Core
° ThoughtWorks.CruiseControl.Remote

In order to let CCNet see this assembly, it must have follow a specific naming : 'ccnet.*.plugin.dll' (where the star represents the name you choose). So our assembly name will be ccnet.FilePublisher.plugin.



Compile and copy the assembly into the folder containing the CruiseControl.NET assemblies. Now you can use this publisher in the same way as in the previous post, by modifying ccnet.config like so :

<publishers>
<filePublisher resultFile="c:\logsresult.txt" />
</publishers>


That was easy ;-)

Thursday, January 29, 2009

Raising some warnings as errors

Today a program just threw an unexpected exception : dll not found.
First I was amazed, how could this happen? How can a program in .Net complain at run-time about a missing dll. I mean how could the program be compiled without this dll?
Turned out that the programmer used some lazy loading, and the compiler (msbuild) DID warn about this, but it is a warning, not an error (warning MSB3245).

I really wanted to fix this situation, I wanted this specific warning to be threated as an error. It turned out to require some creative work.
At work I'm programming with VB.Net, and the compiler options are a bit different than those of C#. In VB.Net you have an option to threat ALL warnings as errors, but this is not an option (yet). There are some warnings, for which we do not have found a way to remove them, and they do not break the program or so.
Another option was to manually place in the project file (using notepad) each warning-number that should be handled as an error.
Now this is not a real solution : this meant changing 200+ projects, and when there is another warning that should be threated as an error, we must again change all project files.

So I was thinking of passing an extra argument to MSBuild, but there is no such argument. With csc, you have the /warnaserror option, but not in msbuild :-(.

The solution I came up with :
° pass another extra argument to msbuild :
/fl /flp:warningsonly;logfile=warnings.txt
this creates a new file warnings.txt that contains only the warnings
° in my current build script, add an extra task that parses this warnings.txt file and checks for certain warning numbers. If it contains a line that matches a list of predefined warnings, fail the build.

Should the list of warnings ever be changed, this is a very small update. Just change this list in every build script. And since i have a program that creates the build scripts, this is just 1 minute of work ;-)

Tuesday, January 27, 2009

Customizing the code of CCNet : part 2 : Creating a publisher

Some stuff you need to know :
Tasks and publishers are the same in ccnet, the only difference is the handling of errors. If a class defined in the tasks section throws an error, the execution of the entire tasks section in the ccnet.config file is stopped. If an error occurs in a class defined in the publisher section, the current publisher will stop, but the next publisher of the publishers section will still be called.

When you open the solution c:\source\ccnet\project\ccnet.sln
you see that there are (for the moment) 10 projects :


CCTray : The CCTray application
CCTrayLib : Library for CCTray, here resides all its logic
Console : The CCtray Console server application
Core : Here resides the majority of the functionality
Objection : Code responsible for creating objects (heavy code)
Remote : Communication layer server
(communication between cctray/dashboard and a CCNet
Service : The CCNet service counterpart of the CCNet console application
UnitTests : All the unit tests of CCNet
Validator : A winform application that validates a ccnet.config file
WebDashboard : The Dashboard application

Now, the publishers are part of the Core, so if you expand the core project,
you will see a publishers folder. This holds all the publishers.



For example, we'll be creating a very simple publisher : FilePublisher.
This will create a txt file with the results of a build.
For configuration : it will take the path of the file.

Now, creating this publisher :
° Create a class FilePublisher in the publishers folder
° change .publishers into .Publishers in the namespace
° add using Exortech.NetReflector;
° make the class public

You have now the following :

using System.Collections;
using System.IO;
using System.Xml.Serialization;
using Exortech.NetReflector;

namespace ThoughtWorks.CruiseControl.Core.Publishers
{
public class FilePublisher : ITask
{
public void Run(IIntegrationResult result)
{
throw new System.NotImplementedException();
}

}
}

For letting the CCNet-system know that this is a publisher/task, the class must
implement the interface ITask. This interface foresees a Run method with an arguement of IIntegrationResult; The argument holds all the information of the current build.
So in fact, writing this publisher is nothing more than :
° opening the defined target file
° write the wanted properties of IIntegrationResult
° close the file
you see, not that hard.

Ok, back to the code. First add the code for the file argument. This is done by
adding a public property, lets say ResultFile. If you want this property to be definable in ccnet.config, you must decorate it with a Reflector attribute, coming from the Exortech.NetReflector namespace.
Also, the class must have such an attribute, to property is passed to the correct class.

using System.Collections;
using System.IO;
using System.Xml.Serialization;
using Exortech.NetReflector;

namespace ThoughtWorks.CruiseControl.Core.Publishers
{
[ReflectorType("filePublisher")]
public class FilePublisher : ITask
{
private string resultFile = "Result.txt";

[ReflectorProperty("resultFile")]
public string ResultFile
{
get { return resultFile; }
set { resultFile = value; }
}

public void Run(IIntegrationResult result)
{
throw new System.NotImplementedException();
}

}
}

All that is left, is the code that actually writes the wanted results to the file.
This results for example in :

using System.Collections;
using System.IO;
using System.Xml.Serialization;
using Exortech.NetReflector;

namespace ThoughtWorks.CruiseControl.Core.Publishers
{
[ReflectorType("filePublisher")]
public class FilePublisher : ITask
{
private string resultFile = "Result.txt";

[ReflectorProperty("resultFile")]
public string ResultFile
{
get { return resultFile; }
set { resultFile = value; }
}

public void Run(IIntegrationResult result)
{
PublishIt(ResultFile, result);
}


private void PublishIt(string targetFile, IIntegrationResult result)
{
StreamWriter Result = new StreamWriter(targetFile, false);
System.Text.StringBuilder Info = new System.Text.StringBuilder();

Info.AppendFormat("Project {0} has status {1}", result.ProjectName, result.Status);
Info.AppendLine();
Info.AppendFormat("Modifications :");
Info.AppendLine();
foreach (Modification mod in result.Modifications)
{
Info.AppendFormat(mod.ToString());
Info.AppendLine();
}
Result.WriteLine(Info.ToString());
}
}
}


In ccnet.config, you define it as follows in the publishers section :

<publishers>
<filePublisher resultFile="c:\logsresult.txt" />
</publishers>

Customizing the code of CCNet : part 1 : Getting the source

In order to make alterations to the code, one must first have it ;-) This article will show you how to do this, step by step.

First be sure that you have a copy of VS2005 VS2008, not another version. You can use a copy of VSExpress 2008 if you do not have VS2005 VS2008 (choose the C# edition of the express editions). I do all my work in a virtual machine, this makes it possible to work with different versions of VS on the same physical machine. (We changed the format to VS2008)

OK, now we got our machine up, and VS2008 is installed, the next step is to download the source of source-forge. At the time of writing, CCNet is hosted in svn, so we must have a svn-client to get it. You can always download the source of a specific nightly build of ccnet, but making patches is far more difficult in that case. My favorite svn client is tortoise svn, which you can download here.
Installing tortoise svn is a breeze : install, next, next, finish ;-)
If you need help with svn / tortoise, there is great help at the site of tortoise-svn.

Next step is to actually get the source. First make a folder where the source will be pulled to, let's say c:\source\CCNet. Create these folders first.
Next right click on the folder CCNet, you get a dialog as follows :


Click 'SVN Checkout ...'
In the next dialog, enter the value below for the url and press ok :
https://ccnet.svn.sourceforge.net/svnroot/ccnet/trunk


Now the source is in c:\source\ccnet


Now we got the source, we can start adding functionality in CCNet.
Next post will be on creating a simple publisher. Stay tuned ...

How to contribute to CCNet

I think there are many people who use CCNet, but think : this feature should work like this, or it would be cool if CCNet had XYZ.

Many of these people find their way to the forums : CCNet-user, CCNet-devel
and most of these items also get on the todo list. But from a developer point of view, it would be nice to have patches ;-)
An issue with a patch on it, is likely to get more rapidly committed to the trunc, because (most) of the work is already done. In the next posts, I'm planning on explaining on how the more easy parts of the code work, so one can extend CCNet to suit their needs. This does NOT mean that you must supply your changes back to CCNet, but it would be nice ;-) The advantage of supplying the change : it is in the code, so when you upgrade your CCNet server, you do not have to re-patch your self.

Stuff I was planning on explaining in detail :
° writing a publisher (changing CCNet, and via a plugin)
° writing a task (changing CCNet, and via a plugin)

I think these are the parts that most users want to customize,
if there are others, feel free to inform me.

Thursday, January 22, 2009

CCNet 1.4.3 Release imminent

Release 1.4.3 of CCNet is near, Dave Cameron wants to release it near Feb 1.
A list of changes can be found here : Solved issues

Highlights :
° Lots of priorities in a queue can cause some of the lowers to never be checked
° New configuration element for task blocks: Description
° errors during getmodifications cause the build to fail
° Subversion Source Control should implement clean copy
° emailpublisher must have an LDAP converter for retrieving the email addresses
° Show the breakers of the build in the dashboard and CCTray
° email publisher customization

plus the usual bug fixes and so.

Dave also asked for a feature freeze now, and to test the current trunc as much as possible, so if there are people who want to help, please do and report any problems.
These can be reported via the issue list or via the user groups at google : ccnet-user or ccnet-devel.

So that means I won't be posting patches for a few days ;-)

Tuesday, January 20, 2009

background info on CI

For those who like a bit of background info on CI, there is a lot to find.

Be sure to read these :
THE reference by Martin Fowler
WikiPedia Info

When I started to do CI, some years ago, it was a getting used to at first, but you really learn to appreciate it. Even if one does not do everyting 'by the book' it pays off. When you're in a team, a dedicated buildserver is a must! The bare minimum is doing a compile, so at least know that the source in source control is complete and it is not broken. How many times haven't you heard : 'But it compiles on my pc' ?

At my current work, there was no buildserver, so I set one up. Just to make sure that the code compiles. And now it mostly is. (still using vss for the moment). Next stuff I added was to place some compiled dll's back into source control, so others could get them and uses as references. Before, somebody did a build from his PC, and placed them in source control. This works, but there is no guarantee that the person had the latest sources, or the source he compiled with was in sourcecontrol!

Now, we're also having deployment projects, so placing programs to a Test environment, and from there to QA. Currently we're working on going from QA to Prod, but this is somewhat different. We have about 100 customers, so pushing the programs to every one of them is not an option. For the moment we're thinking of placing them at an external FTP site, and let the customers get them from there, so we only have to upload once.

It is best to slowly add functionality to the build server. When you have a team of people who are not used to working with a source control system, let alone writing tests, you're committing suicide if the buildserver does the following from day 1 :
° compile
° run tests
° run fxcop
° run simian
° run NDepend
and let the build break if one of them throws a warning or error.

bottom line : slowly does it.
It is a change in thinking, and this takes time.

Monday, January 19, 2009

Small items to do

Here are some small items I plan on working on the next days :

This makes a more friendly build progress display
New configuration element for task blocks

Very handy if you've got lots of ccnet projects.
Tabbed view of projects

A bug in the queues, which is not nice
A queue's lockqueues attribute should allow a space to follow the comma between multiple queue names

As a side note : BVC has been forked at codeplex
BigVisibleCruise II

Contributing to CCNet

When I first started contributing to CCNet 1.5 years ago, CI was still a buzz. Now it's getting more and more main stream. Back in those days, there where not many CI-systems around. There was CruiseControl, Draco, and maybe some others. I do not know if Hudson already existed back than. Team System from MS surely wasn't.
The reason I started contributing was that although CCNet did his job, it was still very basic, (I started at version 1.0 RC1, which was .Net framework 1.1).
After all this time CCNet matured a lot, thanks to the many people sending patches and pointing out problems and so.

When I tell friends what I do, they often ask how much time I spent on it, and frankly I do not know exact. What I do know it's quite a lot. Some bugs / features are easy to do, others take several days or weeks to get done. Many say that I am crazy to spent so much time without getting paid, but I also learn a lot. And it takes only a few people with a bit of time to get this great tool further. It's either this, or spending $$$ on Team System.

These are the contributions I found the most valuable I made for CCNet :
° enhancing the e-mail publisher (notifications, e-mail adress lookup, custom subjects, ...)
° implement a change history overview
° improve some error messages, so you know where the error is and why
° show a build progress in the dashboard and cctray
° artifact clean up
° expose messages in the dashboard, so integration with other programs is better (BVC)
° a graphical build history overview
° publish source control exceptions

Expect a todo list in the next post ...

Some Intresting links :
CCNet Documentation
CCNet Issue page
CCNet in Action