CCNet 1.7 out soon

CCNet 1.7 is due out soon. A lot of work has gone into updating the wiki and incorporating work from contributors.

Anyone wanting to give the 1.7 a spin, please do. All comments are welcome.

An excerpt from the 1.7 release notes :

Backwards compatibility issues

There were a few items that broke when upgrading to 1.5 or 1.6, sorry for that. Here's a list of those that are known to be fixed

• Nant : newline in causes "Target ' ' does not exist in this project."
• Git : Merge Commits in GIT are being ignored by CCnet, causing "No modifications detected."
• CCTray : Prevent Interval Trigger from modifying cctray detail column..
• CCTray : app balloon shows always report builds even if it is set only to show warnings or errors.
• BuildPublisher : KeepLastXBuilds broken in 1.7

What's different, needs some attention:

• Configuration_Preprocessor, it's best that you start every included file must with :
• if you used git, the first build with CCNet 1.7 will list ALL changes done. This is due to the fix of the Git repository. Maybe it's best to delete the first build artifact, and clean up the history.xml file for each project with Git. Or take a backup of the history.xml files, upgrade to CCNet 1.7, and place the backup's back.

Main new features

• xml highlighter for the BuildLog and ProjectConfiguration
• Plastic SCM 4.0 plugin
• Updating the dashboard activity status automatically (refreshinterval needs to be set in dashboard.config)
• add xsl for ms-test and mstest coverage for vs2010
• new modification filter : Multi filter
• show description of the project in dashboard and CCtray
• new xslt Task : allows to do XSL transformation during the build
• Dashboard admin page has a better look, with tooltips and the packages divided in sections
• added a couple of packages for existing xsl files, making it easier to use them.

Statistics publisher

There was an issue that the statistics publisher does not work correctly. Now I know the statistics publisher is an old one, and one where the configuration is rarely altered, besides the 10 standard foreseen statistics.
Gathering extra statistics is not that difficult, add a name for the element, and a XPath expression. For example:

<statistic name="AmountOfFailures" xpath="sum(//test-results/@failures)" />

Now this is good when the build log is simple xml, but when you have sections with namespaces, things become nasty.
If you're working with VS2010, and are merging the test results, you have encountered this problem before. These test results are stored in the "http://microsoft.com/schemas/VisualStudio/TeamTest/2010" namespace.
This problem is now fixed, I've added support for namespaces.

Below you'll find a xmlFile containing test data coming from a demo project, and I'll use this file as an example :
Suppose you want to know the total amount of tests and the amount of failed ones. This information can be retrieved via the following XPath queries : /TestRun/ResultSummary/Counters/@total and /TestRun/ResultSummary/Counters/@failed
Because the test-result file is merged into the master buildlog file, the TestRun node is not the root node anymore.
Normally you could/would fix this by changing it into //TestRun/ResultSummary/Counters/@total, giving the following config for the statistic :

<statistic name="AmountOfTests" xpath="//TestRun/ResultSummary/Counters/@total" />

But this gives MS.Internal.Xml.XPath.XPathSelectionIterator as result. Not very intuitive if you do not work with XPath every day.The problem is the // operator, XPath asumes there can be many TestResult nodes in the xmlfile(even if we know there will only be one), and returns an iterator. To bypass this kind of thing, a FirstMatch class was (and still is) foreseen to handle this, resulting in following setting :

<firstMatch name="AmountOfTests" xpath="//TestRun/ResultSummary/Counters/@total" />

But now the result is Null (empty string) in the file. With the support for namespaces, the config is as follows :

<firstMatch name="AmountOfTests" xpath="//mstest:TestRun/mstest:ResultSummary/mstest:Counters/@total" >
  <namespaces>
     <namespaceMapping prefix="mstest" url="http://microsoft.com/schemas/VisualStudio/TeamTest/2010" />
  </namespaces>
</firstMatch>

Resulting in the wanted result of 2 for the file below. This is a huge step forward, but there is still a problem :
suppose you want some statistic with a calculation of a setting with a namespace, and one without a namespace.
This poses a problem with XSLT 1.0, because we can not set a prefix for the default namespace. To overcome that problem we'll have to support XSLT 2.0. where there is functionality foreseen.
This request is added on the todo list.
The test file :

  <TestRun id="32083ab3-68fe-40ab-a13d-c289aef12a28"
           name="cover_me"
           runUser="ruben"
           xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
    <TestSettings name="Cover"
                  id="9839cf84-0738-4c12-885d-99158bdda72f">
      <Description>These are default test settings for a local test run.</Description>
      <Deployment userDeploymentRoot="C:\Users\ruben\Documents\Visual Studio 2010\Projects\CCNetStatistics"
                  useDefaultDeploymentRoot="false"
                  runDeploymentRoot="cover_me" />
      <NamingScheme baseName="cover_me"
                    appendTimeStamp="false"
                    useDefault="false" />
      <Execution>
        <TestTypeSpecific>
          <UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
            <AssemblyResolution>
              <TestDirectory useLoadContext="true" />
            </AssemblyResolution>
          </UnitTestRunConfig>
        </TestTypeSpecific>
        <AgentRule name="LocalMachineDefaultRole">
          <DataCollectors>
            <DataCollector uri="datacollector://microsoft/CodeCoverage/1.0"
                           assemblyQualifiedName="Microsoft.VisualStudio.TestTools.CodeCoverage.CoveragePlugIn, Microsoft.VisualStudio.QualityTools.Plugins.CodeCoverage, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
                           friendlyName="Code Coverage">
              <Configuration>
                <CodeCoverage xmlns="">
                  <Regular>
                    <CodeCoverageItem binaryFile="C:\Users\ruben\Documents\Visual Studio 2010\Projects\CCNetStatistics\CCNetStatistics\bin\Debug\Agents.dll"
                                      pdbFile="C:\Users\ruben\Documents\Visual Studio 2010\Projects\CCNetStatistics\CCNetStatistics\bin\Debug\Agents.instr.pdb"
                                      instrumentInPlace="true" />
                  </Regular>
                </CodeCoverage>
              </Configuration>
            </DataCollector>
          </DataCollectors>
        </AgentRule>
      </Execution>
    </TestSettings>
    <Times creation="2011-09-04T20:40:22.8989434+02:00"
           queuing="2011-09-04T20:40:24.1937456+02:00"
           start="2011-09-04T20:40:24.3341459+02:00"
           finish="2011-09-04T20:40:28.0469524+02:00" />
    <ResultSummary outcome="Failed">
      <Counters total="2"
                executed="2"
                passed="1"
                error="0"
                failed="1"
                timeout="0"
                aborted="0"
                inconclusive="0"
                passedButRunAborted="0"
                notRunnable="0"
                notExecuted="0"
                disconnected="0"
                warning="0"
                completed="0"
                inProgress="0"
                pending="0" />
      <ResultFiles>
        <ResultFile path="LTREMRUBEN\data.coverage" />
      </ResultFiles>
    </ResultSummary>
    <TestDefinitions>
      <UnitTest name="TestInfiltration"
                storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
                id="93fe6bec-7340-59e7-ba45-0a396dc614ff">
        <Execution id="792491ae-0c70-45e3-a474-1f1cb130226b" />
        <TestMethod codeBase="c:/users/ruben/documents/visual studio 2010/projects/ccnetstatistics/testagents/bin/debug/TestAgents.DLL"
                    adapterTypeName="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
                    className="TestAgents.UnitTest1, TestAgents, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
                    name="TestInfiltration" />
      </UnitTest>
      <UnitTest name="TestReport"
                storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
                id="09504e3d-ea5f-fda7-5ee0-32f8cc895784">
        <Execution id="c98799b7-8675-43b9-ae03-7eb7fd2acb70" />
        <TestMethod codeBase="c:/users/ruben/documents/visual studio 2010/projects/ccnetstatistics/testagents/bin/debug/TestAgents.DLL"
                    adapterTypeName="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
                    className="TestAgents.UnitTest1, TestAgents, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
                    name="TestReport" />
      </UnitTest>
    </TestDefinitions>
    <TestLists>
      <TestList name="Smoke"
                id="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff"
                parentListId="8c43106b-9dc1-4907-a29f-aa66a61bf5b6">
        <TestLinks>
          <TestLink id="93fe6bec-7340-59e7-ba45-0a396dc614ff"
                    name="TestInfiltration"
                    storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
                    type="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
          <TestLink id="09504e3d-ea5f-fda7-5ee0-32f8cc895784"
                    name="TestReport"
                    storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
                    type="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
        </TestLinks>
      </TestList>
      <TestList name="Lists of Results"
                id="8c43106b-9dc1-4907-a29f-aa66a61bf5b6" />
      <TestList name="All Loaded Results"
                id="19431567-8539-422a-85d7-44ee4e166bda" />
    </TestLists>
    <TestEntries>
      <TestEntry testId="93fe6bec-7340-59e7-ba45-0a396dc614ff"
                 executionId="792491ae-0c70-45e3-a474-1f1cb130226b"
                 testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff" />
      <TestEntry testId="09504e3d-ea5f-fda7-5ee0-32f8cc895784"
                 executionId="c98799b7-8675-43b9-ae03-7eb7fd2acb70"
                 testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff" />
    </TestEntries>
    <Results>
      <UnitTestResult executionId="792491ae-0c70-45e3-a474-1f1cb130226b"
                      testId="93fe6bec-7340-59e7-ba45-0a396dc614ff"
                      testName="TestInfiltration"
                      computerName="LTREMRUBEN"
                      duration="00:00:00.0342893"
                      startTime="2011-09-04T20:40:25.9721488+02:00"
                      endTime="2011-09-04T20:40:26.3777495+02:00"
                      testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b"
                      outcome="Passed"
                      testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff"
                      relativeResultsDirectory="792491ae-0c70-45e3-a474-1f1cb130226b">
      </UnitTestResult>
      <UnitTestResult executionId="c98799b7-8675-43b9-ae03-7eb7fd2acb70"
                      testId="09504e3d-ea5f-fda7-5ee0-32f8cc895784"
                      testName="TestReport"
                      computerName="LTREMRUBEN"
                      duration="00:00:00.0538581"
                      startTime="2011-09-04T20:40:26.4089495+02:00"
                      endTime="2011-09-04T20:40:26.4713496+02:00"
                      testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b"
                      outcome="Failed"
                      testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff"
                      relativeResultsDirectory="c98799b7-8675-43b9-ae03-7eb7fd2acb70">
        <Output>
          <ErrorInfo>
            <Message>Assert.AreEqual failed. Expected:&lt;&gt;. Actual:&lt;all went ok&gt;. </Message>
            <StackTrace>
              at TestAgents.UnitTest1.TestReport() in c:\users\ruben\documents\visual studio 2010\Projects\CCNetStatistics\TestAgents\UnitTest1.vb:line 17
            </StackTrace>
          </ErrorInfo>
        </Output>
      </UnitTestResult>
    </Results>
  </TestRun>

Cradiator : WPF frontend for CCNet

I've been working on improving my WPF skills, and what's better than making a WPF front-end for CCNet? Good thing is that this already exists : Cradiator.

I've made a fork on github, so that PandaWood (the maintainer of Cradiator) can pull in my changes as he wants. Since this is my first encounter with Git, I must admit that it takes some getting used to, but it works like a charm.

You can find my fork here.

I've made the following updates :
° added a server-regex, to filter on (build)server name
° added a name on the view
° shows the amount of projects fetched from the URL
° option 'Show only broken'
° option 'Show server name'
° option 'Show out of date' projects


A bit of explanation on the new options :

Show Only Broken

This shows only the broken projects according to selection (project name, category name, server name). If no broken projects are found a smiley and the view name are shown, so you know which view is ok.

Show Server Name

This just shows the server name below the projectname. Comes in handy when you have projects with the same name on different build servers. (Compile on Windows and on Linux for example)

Show Out Of Date

This shows the projects that are outside of an allowed time difference in minutes. The highest LatestBuildTime of all projects is taken. This comes in handy when you have projects(mostly on different build servers) that should do an action at a certain time. This option allows to show the projects which did not build, regardless if they are broken or not. For example the windows build machine was down from 20:00 to 23:00 hours, but we had a FullBuild defined at 22:00.
Now Cradiator shows that the FullBuild on the windows server did not take place.

I use this for some time at work, and it is great. This makes it a lot easier to keep an eye on all the installations I do with CCNet. Remember 80+ servers with each about 30 projects, this makes 2400 projects to watch, so some filtering is needed.

If you get the code and build it, you can see all the new options in action, I update the debugProjects file to show them without needing to point your buildserver.

Suggestions or remarks are welcome

Continous Installation : going beyond the I of installation

The installation of software towards our customers goes very smooth :-) I've set this system up about 2 years ago, and it is a very appreciated part in our company. Hundreds of installations done.
From the start on I extended the functionality beyond the basic installation part, from day 1 actually, but I never wrote about it yet. I also collected some data of those servers : firstly just the application event log. The idea was to send us the error logs, so we were informed if a customer called. And we can do statistics on these errors. What application throws the most errors, are these technical, validation, communication, ... related and so on. Very usefull information.
Later on, I extended it even more to watch the server itself, CPU usage, hard disk usage and even SQL Server information : backup ok, size of database, size of tables, ...
All this info is send to our central SQL server for analysis in the same way as the rest : via the FTP site. This also proved very usefull since we discovered hard disk failures and bad memory modules before they could do serious damage !

But now a new challenge appeared : Hyper-V

Our next servers are more powerfull and we want to reduce the amount of time we need when we have to replace one. The answer is Hyper-V.
These servers are loaded with Hyper-V Core, and above that we will host 1 or more virtual servers. If the hardware needs updating, we can just put in a bigger server with Hyper-V Core on it, move the virtual servers on to it, and we're done. All settings are kept as the were before the move : Big Advantage !!

The catch : we also need to watch the Hyper-V machine itself as we do the current servers. System team looked around for Hyper-V remote stuff, but could not find anything (payable). When I heard of this, I said :
Just use CCNet :-D We use it on all of our other servers and it works, so why not on a Hyper-V one?

And it worked indeed, Hyper-V Core comes with .Net framework 2.O loaded, and that's all I need. To be on the safe side I use the same (very old) version of CCNet as we do on the other servers : 1.4.2 (yikes indeed very old).
But we only use the exec task and the basic triggers, so it is all we need.
just copy the CCNet setup exe on it via a share, run it through a command prompt, and CCNet is installed. Could not be easier !
Now we can monitor and maintain the Hyper-V servers the same as the other ones.

Continous Installation : How it works

It's been a while since my last post ;-) but here is a new one. This post will give more detailed information of the setup I have to install software on 75 servers, and keep an eye on them. As you know, I use CCNet for distributing our software to our customers (about 70 for the moment). These installations involve click-once applications, database upgrades, server checks, ....

I made a presentation in openoffice and ms office so you can have a look, download the format of your choice and start the presentation (F5). Here's a short overview of the presentation :

• Company is connected to an external Ftp Server via the Internet
• Customers have an application server with CCNet pre-installed, which will connect at pre-defined times (schedule trigger) with the Ftp server
• When there is a need for a change (new installation/ program),we upload a modified ccnet.config and the needed files.
• When the schedule trigger fires, CCNet.Config is downloaded, and copied to the path where CCNet watches it. Meaning the new configuration will be read
• If it needs, other files are also downloaded and executed / installed

Now, the 10 point question :
how do we know if an installation was ok, the server was updated, ... ?
At preset times, about 7 times a day, each appserver uploads the state of its CCNet service, this is the data you can see in the dashboard. This data is downloaded to the company and processed.
So we see if the update was ok ('build' ok or failed), and if the update was done or not, via the last executed time.

CCNet 1.4.3 is released

The 1.4.3 release is finally done, here are the release notes . My favorites are

• Source Control errors during GetModifications will now fail the build
• Breakers of a build are listed on the Dashboard and in CCTray
• Dashboard supports theming

And ofcourse the usual bugfixes and smaller improvements.
You can download it from here

Continous Installation observations

I set up a dashboard on a separate PC, so I can keep an eye on the situation on all the servers of the customers. This works, but I found a small issue : speed.
For the moment there are 20 servers installed, and they are all remote. And off course some of them do not have a fast connection :-(
I'll look in to some items to speed things back up. Things I encountered so far :

• Changing the sort order seems to refetch all data, which is not needed
• Navigating back from a buildserver in the farm grid, causes this grid to refetch the data slowing down navigation
• when a second person also opens a browser on his pc, the data is again fetched

A fix for all this would be to cache the grid data at the server(dashboard) level, according to a setting controlling how long this data must be cached. This will improve the speed tremendously and lesson the burden on the network. The refresh data button will re-query all the buildsevers, and so update the information. A small remark here, when you use listeners,( extending the default detail data of the grid) the refresh button is sometimes placed off screen. Maybe a fix for this is to locate the refresh button at the left side of the screen.

When I have ironed out all the small stuff with the Continuous Installation setup, I'll make a post on that. I've encountered some things that could be usefull. Stay tuned for it .

CI : Continous Installation

In my first post I said that my company also wanted to use CCNet to place our programs at the servers of our customers (110+). Now this is possible because we also supply the servers (the customers lease them). Today we did our first test, and it looks very promising. For the moment we have about 15 servers installed (each customer has 1), so the sooner we get this procedure sorted out, the better. These 15 were manually updated, and it took about 20 minutes to install / upgrade 1 program per server. This is including the download of the software.
Now with CCNet installed at these servers, it costs us no time anymore. Well only a few minutes actually. This is how we set things up :

• Whenever a program passes QA, we zip it and upload to an external FTP server
• each server at the customers site monitors this ftp server, and downloads the new software : CCNet Project 1
• the installation itself (unzipping and the like) is CCNet project 2, which responds only to a force build
• and there is also a CCNet Update project (project 3) which contains a ccnet config file, so updating the CCNet servers at the customers is also automated

The only time we now have to 'spent' on installing new software at our customers site, is the time needed to upload it to the FTP server. From that point on, all the servers update themselves. Now this is a massive speed gain. For the moment we have 5 programs (Click-Once apps + WCF services + SQL Databases). So suppose we have to update these manually at once for all customers, it would take us : 5 * 20 * 110 minutes : 11.000 minutes, this is 183 hours corresponding to 7.6 days work (24 hour day). So even trying to do this by hand is ludicrous.
The positive (and also downside) is that this setup forces us to test every step extremely. If there is something wrong with the upgrade script : 110 customers on the phone, not good ;-) We do test the setup at a local spare server, before we press the force build button, just to make sure we do not commit suicide.
Another benefit of having CCNet at our customers server, is that scheduling is now very easy, suppose they want to have a certain report printed/mailed at 07:00, we just update the ccnet config file, upload to the ftp server, and presto : done.
This setup will also be a nice opportunity to check the dashboard, CCTray and BVC2 with a very large amount of CCNet projects. If these apps can handle this load, CCNet is a very stable program. If all goes well, I'll try to place the FTP stuff in the trunc, this will benefit other people as well.

Current Work load ...

Merging the security branch into the trunc :
The project admin said that there should be at least 2 people who know this part of the code, before it can be merged, so I've been busy the last week with the security mergure. This is a very big change to the code see the blog of Craig.

I hope to get this part done in the following days/week(s), so this great addition can be used by everybody, and the work is not lost. Later the other parts of the security branch will also be merged. The biggest problems I encountered :

• very big changes in a long time frame(both in trunc and branch)
• I never had any real intrest into security so the terminology used (roles, permissions, assertions, ...)was new and sometimes confusing
• the security branch also has other functionality (translation, messaging), which is not fully finished yet
• I never did a branch merge, so this was a challenge ;-)

Customizing the code of CCNet : part 3 : Creating a publisher via a plugin

The proces of creating a plugin is already described in the docs, but I'll make a more detailed one here. I'll use the same filePublisher as in my previous post, but this time it will be in the form of a plugin, not as part of the code of CCNet.

Benefits of a plugin :

• you can easily link to your companies software libs
• when a new version of CCNet comes out, you do not have to change the code again

Downside of a plugin :

• Nobody else can extend/improve it
• if there is a breaking change in CCNet, you will have to make the change to make it work again

Back to the code :
Create a class library, and name the project FilePublisher. You will have the following :


Rename Class1 to FilePublisher, and past in the following code (it's the same as in the previous post:

using System.Collections; using System.IO; using System.Xml.Serialization; using Exortech.NetReflector; namespace ThoughtWorks.CruiseControl.Core.Publishers { [ReflectorType("filePublisher")] public class FilePublisher : ITask { private string resultFile = "Result.txt"; [ReflectorProperty("resultFile")] public string ResultFile { get { return resultFile; } set { resultFile = value; } } public void Run(IIntegrationResult result) { PublishIt(ResultFile, result); } private void PublishIt(string targetFile, IIntegrationResult result) { StreamWriter Result = new StreamWriter(targetFile, false); System.Text.StringBuilder Info = new System.Text.StringBuilder(); Info.AppendFormat("Project {0} has status {1}", result.ProjectName, result.Status); Info.AppendLine(); Info.AppendFormat("Modifications :"); Info.AppendLine(); foreach (Modification mod in result.Modifications) { Info.AppendFormat(mod.ToString()); Info.AppendLine(); } Result.WriteLine(Info.ToString()); } } }

Next include references to the following dll's, each can be found in the server folder of the installation folder of CCNet.
° NetReflector
° ThoughtWorks.CruiseControl.Core
° ThoughtWorks.CruiseControl.Remote

In order to let CCNet see this assembly, it must have follow a specific naming : 'ccnet.*.plugin.dll' (where the star represents the name you choose). So our assembly name will be ccnet.FilePublisher.plugin.



Compile and copy the assembly into the folder containing the CruiseControl.NET assemblies. Now you can use this publisher in the same way as in the previous post, by modifying ccnet.config like so :

<publishers> <filePublisher resultFile="c:\logsresult.txt" /> </publishers>

That was easy ;-)

Customizing the code of CCNet : part 2 : Creating a publisher

Some stuff you need to know :
Tasks and publishers are the same in ccnet, the only difference is the handling of errors. If a class defined in the tasks section throws an error, the execution of the entire tasks section in the ccnet.config file is stopped. If an error occurs in a class defined in the publisher section, the current publisher will stop, but the next publisher of the publishers section will still be called.

When you open the solution c:\source\ccnet\project\ccnet.sln
you see that there are (for the moment) 10 projects :

CCTray       : The CCTray application
CCTrayLib    : Library for CCTray, here resides all its logic
Console      : The CCtray Console server application
Core         : Here resides the majority of the functionality
Objection    : Code responsible for creating objects (heavy code)
Remote       : Communication layer server
                (communication between cctray/dashboard and a CCNet 
Service      : The CCNet service counterpart of the CCNet console application
UnitTests    : All the unit tests of CCNet
Validator    : A winform application that validates a ccnet.config file
WebDashboard : The Dashboard application

Now, the publishers are part of the Core, so if you expand the core project,
you will see a publishers folder. This holds all the publishers.



For example, we'll be creating a very simple publisher : FilePublisher.
This will create a txt file with the results of a build.
For configuration : it will take the path of the file.

Now, creating this publisher :
° Create a class FilePublisher in the publishers folder
° change .publishers into .Publishers in the namespace
° add using Exortech.NetReflector;
° make the class public

You have now the following :

using System.Collections; using System.IO; using System.Xml.Serialization; using Exortech.NetReflector; namespace ThoughtWorks.CruiseControl.Core.Publishers { public class FilePublisher : ITask { public void Run(IIntegrationResult result) { throw new System.NotImplementedException(); } } }

For letting the CCNet-system know that this is a publisher/task, the class must
implement the interface ITask. This interface foresees a Run method with an arguement of IIntegrationResult; The argument holds all the information of the current build.
So in fact, writing this publisher is nothing more than :
° opening the defined target file
° write the wanted properties of IIntegrationResult
° close the file
you see, not that hard.

Ok, back to the code. First add the code for the file argument. This is done by
adding a public property, lets say ResultFile. If you want this property to be definable in ccnet.config, you must decorate it with a Reflector attribute, coming from the Exortech.NetReflector namespace.
Also, the class must have such an attribute, to property is passed to the correct class.

using System.Collections; using System.IO; using System.Xml.Serialization; using Exortech.NetReflector; namespace ThoughtWorks.CruiseControl.Core.Publishers { [ReflectorType("filePublisher")] public class FilePublisher : ITask { private string resultFile = "Result.txt"; [ReflectorProperty("resultFile")] public string ResultFile { get { return resultFile; } set { resultFile = value; } } public void Run(IIntegrationResult result) { throw new System.NotImplementedException(); } } }

All that is left, is the code that actually writes the wanted results to the file.
This results for example in :

using System.Collections; using System.IO; using System.Xml.Serialization; using Exortech.NetReflector; namespace ThoughtWorks.CruiseControl.Core.Publishers { [ReflectorType("filePublisher")] public class FilePublisher : ITask { private string resultFile = "Result.txt"; [ReflectorProperty("resultFile")] public string ResultFile { get { return resultFile; } set { resultFile = value; } } public void Run(IIntegrationResult result) { PublishIt(ResultFile, result); } private void PublishIt(string targetFile, IIntegrationResult result) { StreamWriter Result = new StreamWriter(targetFile, false); System.Text.StringBuilder Info = new System.Text.StringBuilder(); Info.AppendFormat("Project {0} has status {1}", result.ProjectName, result.Status); Info.AppendLine(); Info.AppendFormat("Modifications :"); Info.AppendLine(); foreach (Modification mod in result.Modifications) { Info.AppendFormat(mod.ToString()); Info.AppendLine(); } Result.WriteLine(Info.ToString()); } } }

In ccnet.config, you define it as follows in the publishers section :

<publishers> <filePublisher resultFile="c:\logsresult.txt" /> </publishers>

Customizing the code of CCNet : part 1 : Getting the source

In order to make alterations to the code, one must first have it ;-) This article will show you how to do this, step by step.

First be sure that you have a copy of VS2005 VS2008, not another version. You can use a copy of VSExpress 2008 if you do not have VS2005 VS2008 (choose the C# edition of the express editions). I do all my work in a virtual machine, this makes it possible to work with different versions of VS on the same physical machine. (We changed the format to VS2008)

OK, now we got our machine up, and VS2008 is installed, the next step is to download the source of source-forge. At the time of writing, CCNet is hosted in svn, so we must have a svn-client to get it. You can always download the source of a specific nightly build of ccnet, but making patches is far more difficult in that case. My favorite svn client is tortoise svn, which you can download here.
Installing tortoise svn is a breeze : install, next, next, finish ;-)
If you need help with svn / tortoise, there is great help at the site of tortoise-svn.

Next step is to actually get the source. First make a folder where the source will be pulled to, let's say c:\source\CCNet. Create these folders first.
Next right click on the folder CCNet, you get a dialog as follows :


Click 'SVN Checkout ...'
In the next dialog, enter the value below for the url and press ok :
https://ccnet.svn.sourceforge.net/svnroot/ccnet/trunk


Now the source is in c:\source\ccnet


Now we got the source, we can start adding functionality in CCNet.
Next post will be on creating a simple publisher. Stay tuned ...

How to contribute to CCNet

I think there are many people who use CCNet, but think : this feature should work like this, or it would be cool if CCNet had XYZ.

Many of these people find their way to the forums : CCNet-user, CCNet-devel
and most of these items also get on the todo list. But from a developer point of view, it would be nice to have patches ;-)
An issue with a patch on it, is likely to get more rapidly committed to the trunc, because (most) of the work is already done. In the next posts, I'm planning on explaining on how the more easy parts of the code work, so one can extend CCNet to suit their needs. This does NOT mean that you must supply your changes back to CCNet, but it would be nice ;-) The advantage of supplying the change : it is in the code, so when you upgrade your CCNet server, you do not have to re-patch your self.

Stuff I was planning on explaining in detail :
° writing a publisher (changing CCNet, and via a plugin)
° writing a task (changing CCNet, and via a plugin)

I think these are the parts that most users want to customize,
if there are others, feel free to inform me.

CCNet 1.4.3 Release imminent

Release 1.4.3 of CCNet is near, Dave Cameron wants to release it near Feb 1.
A list of changes can be found here : Solved issues

Highlights :
° Lots of priorities in a queue can cause some of the lowers to never be checked
° New configuration element for task blocks: Description
° errors during getmodifications cause the build to fail
° Subversion Source Control should implement clean copy
° emailpublisher must have an LDAP converter for retrieving the email addresses
° Show the breakers of the build in the dashboard and CCTray
° email publisher customization

plus the usual bug fixes and so.

Dave also asked for a feature freeze now, and to test the current trunc as much as possible, so if there are people who want to help, please do and report any problems.
These can be reported via the issue list or via the user groups at google : ccnet-user or ccnet-devel.

So that means I won't be posting patches for a few days ;-)

background info on CI

For those who like a bit of background info on CI, there is a lot to find.

Be sure to read these :
THE reference by Martin Fowler
WikiPedia Info

When I started to do CI, some years ago, it was a getting used to at first, but you really learn to appreciate it. Even if one does not do everyting 'by the book' it pays off. When you're in a team, a dedicated buildserver is a must! The bare minimum is doing a compile, so at least know that the source in source control is complete and it is not broken. How many times haven't you heard : 'But it compiles on my pc' ?

At my current work, there was no buildserver, so I set one up. Just to make sure that the code compiles. And now it mostly is. (still using vss for the moment). Next stuff I added was to place some compiled dll's back into source control, so others could get them and uses as references. Before, somebody did a build from his PC, and placed them in source control. This works, but there is no guarantee that the person had the latest sources, or the source he compiled with was in sourcecontrol!

Now, we're also having deployment projects, so placing programs to a Test environment, and from there to QA. Currently we're working on going from QA to Prod, but this is somewhat different. We have about 100 customers, so pushing the programs to every one of them is not an option. For the moment we're thinking of placing them at an external FTP site, and let the customers get them from there, so we only have to upload once.

It is best to slowly add functionality to the build server. When you have a team of people who are not used to working with a source control system, let alone writing tests, you're committing suicide if the buildserver does the following from day 1 :
° compile
° run tests
° run fxcop
° run simian
° run NDepend
and let the build break if one of them throws a warning or error.

bottom line : slowly does it.
It is a change in thinking, and this takes time.

Small items to do

Here are some small items I plan on working on the next days :

This makes a more friendly build progress display
New configuration element for task blocks

Very handy if you've got lots of ccnet projects.
Tabbed view of projects

A bug in the queues, which is not nice
A queue's lockqueues attribute should allow a space to follow the comma between multiple queue names

As a side note : BVC has been forked at codeplex
BigVisibleCruise II

Contributing to CCNet

When I first started contributing to CCNet 1.5 years ago, CI was still a buzz. Now it's getting more and more main stream. Back in those days, there where not many CI-systems around. There was CruiseControl, Draco, and maybe some others. I do not know if Hudson already existed back than. Team System from MS surely wasn't.
The reason I started contributing was that although CCNet did his job, it was still very basic, (I started at version 1.0 RC1, which was .Net framework 1.1).
After all this time CCNet matured a lot, thanks to the many people sending patches and pointing out problems and so.

When I tell friends what I do, they often ask how much time I spent on it, and frankly I do not know exact. What I do know it's quite a lot. Some bugs / features are easy to do, others take several days or weeks to get done. Many say that I am crazy to spent so much time without getting paid, but I also learn a lot. And it takes only a few people with a bit of time to get this great tool further. It's either this, or spending $$$ on Team System.

These are the contributions I found the most valuable I made for CCNet :
° enhancing the e-mail publisher (notifications, e-mail adress lookup, custom subjects, ...)
° implement a change history overview
° improve some error messages, so you know where the error is and why
° show a build progress in the dashboard and cctray
° artifact clean up
° expose messages in the dashboard, so integration with other programs is better (BVC)
° a graphical build history overview
° publish source control exceptions

Expect a todo list in the next post ...

Some Intresting links :
CCNet Documentation
CCNet Issue page
CCNet in Action