Statistics publisher

There was an issue that the statistics publisher does not work correctly. Now I know the statistics publisher is an old one, and one where the configuration is rarely altered, besides the 10 standard foreseen statistics.
Gathering extra statistics is not that difficult, add a name for the element, and a XPath expression. For example:

<statistic name="AmountOfFailures" xpath="sum(//test-results/@failures)" />

Now this is good when the build log is simple xml, but when you have sections with namespaces, things become nasty.
If you're working with VS2010, and are merging the test results, you have encountered this problem before. These test results are stored in the "http://microsoft.com/schemas/VisualStudio/TeamTest/2010" namespace.
This problem is now fixed, I've added support for namespaces.

Below you'll find a xmlFile containing test data coming from a demo project, and I'll use this file as an example :
Suppose you want to know the total amount of tests and the amount of failed ones. This information can be retrieved via the following XPath queries : /TestRun/ResultSummary/Counters/@total and /TestRun/ResultSummary/Counters/@failed
Because the test-result file is merged into the master buildlog file, the TestRun node is not the root node anymore.
Normally you could/would fix this by changing it into //TestRun/ResultSummary/Counters/@total, giving the following config for the statistic :

<statistic name="AmountOfTests" xpath="//TestRun/ResultSummary/Counters/@total" />

But this gives MS.Internal.Xml.XPath.XPathSelectionIterator as result. Not very intuitive if you do not work with XPath every day.The problem is the // operator, XPath asumes there can be many TestResult nodes in the xmlfile(even if we know there will only be one), and returns an iterator. To bypass this kind of thing, a FirstMatch class was (and still is) foreseen to handle this, resulting in following setting :

<firstMatch name="AmountOfTests" xpath="//TestRun/ResultSummary/Counters/@total" />

But now the result is Null (empty string) in the file. With the support for namespaces, the config is as follows :

<firstMatch name="AmountOfTests" xpath="//mstest:TestRun/mstest:ResultSummary/mstest:Counters/@total" >
  <namespaces>
     <namespaceMapping prefix="mstest" url="http://microsoft.com/schemas/VisualStudio/TeamTest/2010" />
  </namespaces>
</firstMatch>

Resulting in the wanted result of 2 for the file below. This is a huge step forward, but there is still a problem :
suppose you want some statistic with a calculation of a setting with a namespace, and one without a namespace.
This poses a problem with XSLT 1.0, because we can not set a prefix for the default namespace. To overcome that problem we'll have to support XSLT 2.0. where there is functionality foreseen.
This request is added on the todo list.
The test file :

  <TestRun id="32083ab3-68fe-40ab-a13d-c289aef12a28"
           name="cover_me"
           runUser="ruben"
           xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
    <TestSettings name="Cover"
                  id="9839cf84-0738-4c12-885d-99158bdda72f">
      <Description>These are default test settings for a local test run.</Description>
      <Deployment userDeploymentRoot="C:\Users\ruben\Documents\Visual Studio 2010\Projects\CCNetStatistics"
                  useDefaultDeploymentRoot="false"
                  runDeploymentRoot="cover_me" />
      <NamingScheme baseName="cover_me"
                    appendTimeStamp="false"
                    useDefault="false" />
      <Execution>
        <TestTypeSpecific>
          <UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
            <AssemblyResolution>
              <TestDirectory useLoadContext="true" />
            </AssemblyResolution>
          </UnitTestRunConfig>
        </TestTypeSpecific>
        <AgentRule name="LocalMachineDefaultRole">
          <DataCollectors>
            <DataCollector uri="datacollector://microsoft/CodeCoverage/1.0"
                           assemblyQualifiedName="Microsoft.VisualStudio.TestTools.CodeCoverage.CoveragePlugIn, Microsoft.VisualStudio.QualityTools.Plugins.CodeCoverage, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
                           friendlyName="Code Coverage">
              <Configuration>
                <CodeCoverage xmlns="">
                  <Regular>
                    <CodeCoverageItem binaryFile="C:\Users\ruben\Documents\Visual Studio 2010\Projects\CCNetStatistics\CCNetStatistics\bin\Debug\Agents.dll"
                                      pdbFile="C:\Users\ruben\Documents\Visual Studio 2010\Projects\CCNetStatistics\CCNetStatistics\bin\Debug\Agents.instr.pdb"
                                      instrumentInPlace="true" />
                  </Regular>
                </CodeCoverage>
              </Configuration>
            </DataCollector>
          </DataCollectors>
        </AgentRule>
      </Execution>
    </TestSettings>
    <Times creation="2011-09-04T20:40:22.8989434+02:00"
           queuing="2011-09-04T20:40:24.1937456+02:00"
           start="2011-09-04T20:40:24.3341459+02:00"
           finish="2011-09-04T20:40:28.0469524+02:00" />
    <ResultSummary outcome="Failed">
      <Counters total="2"
                executed="2"
                passed="1"
                error="0"
                failed="1"
                timeout="0"
                aborted="0"
                inconclusive="0"
                passedButRunAborted="0"
                notRunnable="0"
                notExecuted="0"
                disconnected="0"
                warning="0"
                completed="0"
                inProgress="0"
                pending="0" />
      <ResultFiles>
        <ResultFile path="LTREMRUBEN\data.coverage" />
      </ResultFiles>
    </ResultSummary>
    <TestDefinitions>
      <UnitTest name="TestInfiltration"
                storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
                id="93fe6bec-7340-59e7-ba45-0a396dc614ff">
        <Execution id="792491ae-0c70-45e3-a474-1f1cb130226b" />
        <TestMethod codeBase="c:/users/ruben/documents/visual studio 2010/projects/ccnetstatistics/testagents/bin/debug/TestAgents.DLL"
                    adapterTypeName="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
                    className="TestAgents.UnitTest1, TestAgents, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
                    name="TestInfiltration" />
      </UnitTest>
      <UnitTest name="TestReport"
                storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
                id="09504e3d-ea5f-fda7-5ee0-32f8cc895784">
        <Execution id="c98799b7-8675-43b9-ae03-7eb7fd2acb70" />
        <TestMethod codeBase="c:/users/ruben/documents/visual studio 2010/projects/ccnetstatistics/testagents/bin/debug/TestAgents.DLL"
                    adapterTypeName="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
                    className="TestAgents.UnitTest1, TestAgents, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
                    name="TestReport" />
      </UnitTest>
    </TestDefinitions>
    <TestLists>
      <TestList name="Smoke"
                id="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff"
                parentListId="8c43106b-9dc1-4907-a29f-aa66a61bf5b6">
        <TestLinks>
          <TestLink id="93fe6bec-7340-59e7-ba45-0a396dc614ff"
                    name="TestInfiltration"
                    storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
                    type="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
          <TestLink id="09504e3d-ea5f-fda7-5ee0-32f8cc895784"
                    name="TestReport"
                    storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
                    type="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
        </TestLinks>
      </TestList>
      <TestList name="Lists of Results"
                id="8c43106b-9dc1-4907-a29f-aa66a61bf5b6" />
      <TestList name="All Loaded Results"
                id="19431567-8539-422a-85d7-44ee4e166bda" />
    </TestLists>
    <TestEntries>
      <TestEntry testId="93fe6bec-7340-59e7-ba45-0a396dc614ff"
                 executionId="792491ae-0c70-45e3-a474-1f1cb130226b"
                 testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff" />
      <TestEntry testId="09504e3d-ea5f-fda7-5ee0-32f8cc895784"
                 executionId="c98799b7-8675-43b9-ae03-7eb7fd2acb70"
                 testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff" />
    </TestEntries>
    <Results>
      <UnitTestResult executionId="792491ae-0c70-45e3-a474-1f1cb130226b"
                      testId="93fe6bec-7340-59e7-ba45-0a396dc614ff"
                      testName="TestInfiltration"
                      computerName="LTREMRUBEN"
                      duration="00:00:00.0342893"
                      startTime="2011-09-04T20:40:25.9721488+02:00"
                      endTime="2011-09-04T20:40:26.3777495+02:00"
                      testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b"
                      outcome="Passed"
                      testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff"
                      relativeResultsDirectory="792491ae-0c70-45e3-a474-1f1cb130226b">
      </UnitTestResult>
      <UnitTestResult executionId="c98799b7-8675-43b9-ae03-7eb7fd2acb70"
                      testId="09504e3d-ea5f-fda7-5ee0-32f8cc895784"
                      testName="TestReport"
                      computerName="LTREMRUBEN"
                      duration="00:00:00.0538581"
                      startTime="2011-09-04T20:40:26.4089495+02:00"
                      endTime="2011-09-04T20:40:26.4713496+02:00"
                      testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b"
                      outcome="Failed"
                      testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff"
                      relativeResultsDirectory="c98799b7-8675-43b9-ae03-7eb7fd2acb70">
        <Output>
          <ErrorInfo>
            <Message>Assert.AreEqual failed. Expected:&lt;&gt;. Actual:&lt;all went ok&gt;. </Message>
            <StackTrace>
              at TestAgents.UnitTest1.TestReport() in c:\users\ruben\documents\visual studio 2010\Projects\CCNetStatistics\TestAgents\UnitTest1.vb:line 17
            </StackTrace>
          </ErrorInfo>
        </Output>
      </UnitTestResult>
    </Results>
  </TestRun>

New site for CCNet

As you may have read, we moved away from Thoughtworks hosting to another server :

After years of good collaboration with Thoughtworks, it appeared that Thoughtworks had less resources to spent for CCNet.
This problem came painfully visible when their site was hacked with a virus. It took a long time for the help page to come back up, and the build server is still not up.

Some months ago we (the active members) decided to move the hosting to another server, so we would be less dependent on Thoughtworks. No hard feelings, it were great years of cooperation but time moved on.

The reason it took a few months to set everything up was due to lack of time of us. Communication could have been better, but we're all doing this (maintaining CCNet) in our own free time, without getting paid. Hopefully this downtime will not happen again.

The license of CCNet is still the same as it was before the move, we just moved the build server and the documentation site, everything else stays the same till further notice :-)
The move to the new site also has some plus points :

we can update / alter the site at any time.
there is 1 site for the wiki, bug reports and forum (before there was Jira and Confluence, no forum) meaning 1 user and password
a Linux build server is also up and running, meaning more Mono compatibility.
moved from svn to Git, which should make it a lot easier for people to contribute.
Chili has some options that Jira did not have : Calendar, Gantt chart, ....

Now lets get CCNet back in the picture, stay tuned for more news.


This server is provided by Daniel Nauck, so I owe him a lot for this !
Now we have a build server back :-)

It seems that the build is broken, so we need to fix that asap, and continue to update the new documentation wiki. When this is done, we can start to update CCNet like before.

Keeping you posted ...

Yet another trigger : installTrigger

At work I use CCNet for 2 main different things :
° as Continuous Integration : compile, test, package, code statistics, ...
° as Continuous Installation : install our software at the customers site ( see articles)

For the installation part, I previously changed the ccnet.config to reflect the needed changes. For example change the requested time of a schedule trigger, add the names of the server(s) where to install or not to install the software.
Now this works great, but it is error prone!
Remember ccnet.config is the core configuration of CCNet, a typo there could stop the service. And reviving it is not fun, remember I have a small 100 servers to maintain!!

So the idea came for a new trigger, to remove the parts that change a lot in ccnet.config to another file. Just moving to another file is not enough though, if there is an error in that file, ccnet may not crash. Meaning that using the pre-processor or XML-Entities is out of the question.

Now I already have an xml file for each customer, that holds what software the customer has, what sql server settings are needed, where the software needs to come, ....

Meaning this trigger should just read this xml file, and return the needed things.
I just needed to add the wanted integration time and wanted version to this file, to get it to work.


The new trigger is the InstallTrigger (cool name actually) and is a copy of the schedule trigger with the following extra parts :
° CheckInstallationNeededForProgramToInstall (the name of the program as known in the xml file)
° DeploySettingFilePath : where the xml file is located
° CruiseInstallProject : the corresponding CCNet project name that does the installation (more on this later)
° UpdateDeploySettingsProject : Name of CCNet project that downloads updated xml files from the ftp site
° removed the Time property, as this will come from the xml file


The most important things of the code :
° at start, get the wanted datetime of the xml file, if the date is before today, copy the time part to todays date
° when an integration is done, and the installation failed, get the wanted datetime from the xmlfile, but add 1 day (re-schedule automatically).
° when an integration is done, and the installation was ok, return datetime.maxvalue
° when the UpdateDeploySettingsProject ran, re-read the xml file.
° use datetime.utcNow iso dateTime.Now, this is a LOT faster (look here

Now why I added the CruiseInstallProject property?
One could easily say that this is not needed, because the trigger is inside the project that will install the requested software.
Wrong bet :-)
Remember a previous post : forcing multiple builds at once

So when I want to install, let's say Bookkeeping, also our security module (for login and so) needs to be installed.
So I have the following CCNet projects :

CCNetProjectName	InstallTrigger property cruiseInstallProject	Tasks
InstallSecurity	InstallSecurity	install security program
InstallBookKeeping		install bookkeeping program
FullInstallBookkeeping	InstallBookKeeping	ForceBuild InstallSecurity ForceBuild InstallBookKeeping

This makes it a lot safer to do the installations, and even gives us an easy way to schedule the same program at different dates/times for different customers.
It is the same CCNet project, just the xml file for the involved customer needs to change.

Below is the code of the InstallTrigger.

Imports Exortech.NetReflector Imports ThoughtWorks.CruiseControl.Remote Imports ThoughtWorks.CruiseControl.Core.Config Imports System.Globalization Imports ThoughtWorks.CruiseControl.Core.Util <ReflectorType("installTrigger")> _ Public Class InstallTrigger Implements ITrigger Private m_name As String Private dtProvider As DateTimeProvider Private m_nextBuild As DateTime = DateTime.MinValue Private m_previousBuild As DateTime Private triggered As Boolean Private _includeServerNames As String() Private _excludeServerNames As String() Private _serverConfig As Data.DeploySetting = Nothing Private _requestedInstallDate As DateTime = DateTime.MaxValue Private _cruiseInstallProject As String = Nothing Private _cruiseInstallProjectStateFile As String Private _updateDeploySettingsStateFile As String Private _UpdateDeploySettingsVersion As String Private _UpdateDeploySettingsProject As String = "UpdateDeploySettings" Private _UpdateDeploySettingsProjectLastTimeChecked As DateTime Private _integrationDone As Boolean Private _ServerNameIsOk As Boolean Public Sub New() Me.New(New DateTimeProvider()) End Sub Public Sub New(ByVal dtProvider As DateTimeProvider) Me.dtProvider = dtProvider End Sub <ReflectorArray("includeServerNames", required:=False)> _ Public Property IncludeServerNames() As String() Get Return _includeServerNames End Get Set(ByVal value As String()) _includeServerNames = value End Set End Property <ReflectorArray("excludeServerNames", required:=False)> _ Public Property ExcludeServerNames() As String() Get Return _excludeServerNames End Get Set(ByVal value As String()) _excludeServerNames = value End Set End Property <ReflectorProperty("name", Required:=False)> _ Public Property Name() As String Get If m_name Is Nothing Then m_name = [GetType]().Name End If Return m_name End Get Set(ByVal value As String) m_name = value End Set End Property <ReflectorProperty("buildCondition", Required:=False)> _ Public BuildCondition As BuildCondition = BuildCondition.IfModificationExists <ReflectorArray("weekDays", Required:=False)> _ Public WeekDays As DayOfWeek() = DirectCast(DayOfWeek.GetValues(GetType(DayOfWeek)), DayOfWeek()) Private _checkInstallationNeededForProgramToInstall As String Private _deploySettingFilePath As String = "D:\InstallPath" <ReflectorProperty("checkInstallationNeededForProgramToInstall", required:=True)> _ Public Property CheckInstallationNeededForProgramToInstall() As String Get Return Me._checkInstallationNeededForProgramToInstall End Get Set(ByVal value As String) Me._checkInstallationNeededForProgramToInstall = value End Set End Property <ReflectorProperty("deploySettingFilePath", required:=False)> _ Public Property DeploySettingFilePath() As String Get Return Me._deploySettingFilePath End Get Set(ByVal value As String) Me._deploySettingFilePath = value End Set End Property <ReflectorProperty("updateDeploySettingsProject", required:=True)> _ Public Property UpdateDeploySettingsProject() As String Get Return Me._UpdateDeploySettingsProject End Get Set(ByVal value As String) Me._UpdateDeploySettingsProject = value End Set End Property <ReflectorProperty("cruiseInstallProject", required:=True)> _ Public Property CruiseInstallProject As String Get Return _cruiseInstallProject End Get Set(value As String) _cruiseInstallProject = value End Set End Property Private Sub SetNextIntegrationDateTime() Dim now As DateTime = dtProvider.Now m_nextBuild = RequestedInstallDateTime() If m_nextBuild = DateTime.MaxValue Then Return If now >= m_nextBuild OrElse now.Date = m_previousBuild.Date Then m_nextBuild = m_nextBuild.AddDays(1) End If m_nextBuild = CalculateNextIntegrationTime(m_nextBuild) End Sub Private Function CalculateNextIntegrationTime(ByVal nextIntegration As DateTime) As DateTime While True If IsValidWeekDay(nextIntegration.DayOfWeek) Then Exit While End If nextIntegration = nextIntegration.AddDays(1) End While Return nextIntegration End Function Private Function IsValidWeekDay(ByVal nextIntegrationDay As DayOfWeek) As Boolean Return Array.IndexOf(WeekDays, nextIntegrationDay) >= 0 End Function Public Overridable Sub IntegrationCompleted() Implements ITrigger.IntegrationCompleted Dim now As DateTime = dtProvider.Now If triggered Then m_previousBuild = now 'to force a re-read of the deploymentsettings file after X seconds. 'State file does not seem to be updated yet _UpdateDeploySettingsProjectLastTimeChecked = DateTime.UtcNow _integrationDone = True End If triggered = False End Sub Public ReadOnly Property NextBuild() As DateTime Implements ITrigger.NextBuild Get If m_nextBuild = DateTime.MinValue Then 'first time initialise FirstTimeInitialize() If _ServerNameIsOk Then ThoughtWorks.CruiseControl.Core.Util.Log.Debug("Loading deploysettings file ") _serverConfig = New Data.DeploySetting(DeploySettingFilePath) SetNextIntegrationDateTime() End If End If If Not _ServerNameIsOk Then Return DateTime.MaxValue 'to keep watching the deploy settings file after a second 'Fire' If isDeploySettingsFileIsUpdated() Then ThoughtWorks.CruiseControl.Core.Util.Log.Debug("Re-loading deploysettings file") _serverConfig = New Data.DeploySetting(DeploySettingFilePath) SetNextIntegrationDateTime() End If Return m_nextBuild End Get End Property Public Function Fire() As IntegrationRequest Implements ITrigger.Fire Dim now As DateTime = dtProvider.Now If now > NextBuild AndAlso IsValidWeekDay(now.DayOfWeek) Then triggered = True Return New IntegrationRequest(BuildCondition, Name) End If Return Nothing End Function Private Function ServerNameIsOk() As Boolean Dim includeServersSpecified As Boolean = IncludeServerNames IsNot Nothing AndAlso IncludeServerNames.Length > 0 Dim excludeServersSpecified As Boolean = ExcludeServerNames IsNot Nothing AndAlso ExcludeServerNames.Length > 0 If includeServersSpecified Then For Each srv In IncludeServerNames If srv.Equals(System.Net.Dns.GetHostName, StringComparison.CurrentCultureIgnoreCase) Then Return True If srv.Equals(System.Environment.MachineName, StringComparison.CurrentCultureIgnoreCase) Then Return True Next Return False End If If excludeServersSpecified Then For Each srv In ExcludeServerNames If srv.Equals(System.Net.Dns.GetHostName, StringComparison.CurrentCultureIgnoreCase) Then Return False If srv.Equals(System.Environment.MachineName, StringComparison.CurrentCultureIgnoreCase) Then Return False Next Return True End If Return True End Function Private Function RequestedInstallDateTime() As DateTime Try Dim CCNetInfo = GetInfoFromCCNetStateFile(_cruiseInstallProjectStateFile) Dim LastSucessFullLabel = CCNetInfo.LastSucessFullLabel Dim projectVersion As Version If String.IsNullOrEmpty(LastSucessFullLabel) OrElse String.Equals("unknown", LastSucessFullLabel, StringComparison.CurrentCultureIgnoreCase) Then projectVersion = New Version(0, 0) Else projectVersion = New Version(LastSucessFullLabel) End If 'specific company code to get the data from the config, can not post this :-( 'dummy code folows Dim progInfo = _serverConfig.GetInfo(CheckInstallationNeededForProgramToInstall) 'end dummy code If projectVersion < progInfo.WantedVersion Then If progInfo.WantedFrom.Date < Now.Date Then 'should the server be down for some time or so _requestedInstallDate = Now.Date.AddHours(availableApplication.WantedFrom.Hour).AddMinutes(availableApplication.WantedFrom.Minute) Else _requestedInstallDate = progInfo.WantedFrom End If If _integrationDone Then 'if installation failed, re-schedule it for the next day at the same time, giving us a day to investigate, 'otherwise try again for temporary errors If Not CCNetInfo.IsSuccesfull Then _requestedInstallDate = _requestedInstallDate.AddDays(1) _integrationDone = False End If Else _requestedInstallDate = DateTime.MaxValue End If Catch ex As Exception 'on error, just set it to not scheduled _requestedInstallDate = DateTime.MaxValue Dim ei = String.Format("Error in RequestedInstallDateTime : {0} ", ex.ToString) ThoughtWorks.CruiseControl.Core.Util.Log.Error(ei) EventLog.WriteEntry("CCNet-InstallTrigger", ei, EventLogEntryType.Error) End Try Return _requestedInstallDate End Function Private Function GetInfoFromCCNetStateFile(cruiseProjectStateFile As String) As StateFileInfo Dim result As New StateFileInfo Try If IO.File.Exists(cruiseProjectStateFile) Then Dim stateFileContents As String = IO.File.ReadAllText(cruiseProjectStateFile) Dim xdoc As New Xml.XmlDocument xdoc.LoadXml(stateFileContents) result.LastSucessFullLabel = xdoc.SelectSingleNode("IntegrationResult/LastSuccessfulIntegrationLabel").InnerText Dim LastIntegrationStatus = xdoc.SelectSingleNode("IntegrationResult/Status").InnerText result.IsSuccesfull = LastIntegrationStatus = "Success" End If Catch ex As Exception EventLog.WriteEntry("CCNet-InstallTrigger", "Error reading state file of project : " + cruiseProjectStateFile + ex.ToString, EventLogEntryType.Error) End Try Return result End Function Private Function isDeploySettingsFileIsUpdated() As Boolean If DateTime.UtcNow.Subtract(_UpdateDeploySettingsProjectLastTimeChecked).TotalSeconds > 5 Then 'check the state file every X seconds Dim news = GetInfoFromCCNetStateFile(_updateDeploySettingsStateFile) _UpdateDeploySettingsProjectLastTimeChecked = DateTime.UtcNow If _UpdateDeploySettingsVersion <> news.LastSucessFullLabel Then _UpdateDeploySettingsVersion = news.LastSucessFullLabel Return True End If End If Return False End Function Private Sub FirstTimeInitialize() _cruiseInstallProjectStateFile = IO.Path.Combine(New IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly.Location).DirectoryName, CruiseInstallProject + ".state") _updateDeploySettingsStateFile = IO.Path.Combine(New IO.FileInfo(System.Reflection.Assembly.GetExecutingAssembly.Location).DirectoryName, _UpdateDeploySettingsProject + ".state") _UpdateDeploySettingsVersion = GetInfoFromCCNetStateFile(_updateDeploySettingsStateFile).LastSucessFullLabel _UpdateDeploySettingsProjectLastTimeChecked = DateTime.UtcNow _ServerNameIsOk = ServerNameIsOk() End Sub Private Class StateFileInfo Public Property LastSucessFullLabel As String Public IsSuccesfull As Boolean End Class End Class

Tuning CCNet to your whishes : a new trigger

There was a request on the user list to run a nightly "publish" build, which takes the output of the other "child" projects and copies them to a new output directory. Since the output is quite large (> 1 GB), this should only be done each night if something changed in any of the other projects, but only if all of them have run successfully.
You can read the full details also on Stack overflow. Now one should be able to do this using the existing functionality, but that is not the case :-(. The reason is mainly that CCNet has grown from a simple CI server to a more feature rich one over the years, but here and there are still leftovers from the beginning, read the simple CI server. Triggers are just events that occur, and do not know a state, there is the project trigger, but that is more of a fire and forget one. So restarting a server poses a problem, combining multiple project triggers into one trigger poses a problem with timings over a day and so on.

To handle this situation, the only option is to write some code that does it. Thankfully CCNet has a plugin architecture that works quite well, see my previous creating a plugin about it for more details.

As an example of this request, I will create a new trigger, because this will be the fastest way to get a result with just a few lines of code adjustments needed. For starters I just copied the code of the existing schedule trigger into a new class, named ScheduleTriggerExtended.

Changes done :
° added property SubProjectsToWatch
The project names of the child projects to watch. For simplicity only the project name, meaning that it has to be on the same build server and has browse rights (or no security setup)
° added property ServerUri
the uri of the ccnet server, example : tcp://localhost:21234/CruiseServerClient.rem
° added property SubProjectsSavePath
A path reachable by the CCNet server (preferably local). This is used to store the timestamps of the subprojects. So we can check if the timestamp changed between the integration of a subproject and the build of the involved subproject.
° updated public IntegrationRequest Fire
Whenever the scheduled time is reached, also check if the subProjects are changed, by comparing the current last build times with the saved ones.
° updated public void IntegrationCompleted
Save the new buildtimes of the subprojects to the SubProjectsSavePath for later reference.

When reading this all over, the best approach would be to create a 'CCNet source control', that can scan other CCNet projects for changes, and filter them according to some filters (state, build time, ...)

Anyway, below is the trigger class, all changes are highlighted with comment : added code. Copy entire source into an assembly with name something like : CCNet.whatever.plugin
Example : CCNet.RuWi.plugin
And set references to NetReflector.dll, ThoughtWorks.CruiseControl.Core.dll, ThoughtWorks.CruiseControl.Remote.dll


A config is also provided below the code.

using System; using System.Globalization; using Exortech.NetReflector; using ThoughtWorks.CruiseControl.Core.Config; using ThoughtWorks.CruiseControl.Core.Util; using ThoughtWorks.CruiseControl.Remote; namespace ThoughtWorks.CruiseControl.Core.Triggers { [ReflectorType("scheduleTriggerExtended")] public class ScheduleTriggerExtended : ITrigger, IConfigurationValidation { private string name; private DateTimeProvider dtProvider; private TimeSpan integrationTime; private DateTime nextBuild; private bool triggered; private Int32 randomOffSetInMinutesFromTime/* = 0*/; Random randomizer = new Random(); /// <summary> /// Initializes a new instance of the <see cref="ScheduleTrigger"/> class. /// </summary> public ScheduleTriggerExtended() : this(new DateTimeProvider()) { } /// <summary> /// Initializes a new instance of the <see cref="ScheduleTrigger"/> class. /// </summary> /// <param name="dtProvider">The dt provider.</param> public ScheduleTriggerExtended(DateTimeProvider dtProvider) { this.dtProvider = dtProvider; this.BuildCondition = BuildCondition.IfModificationExists; WeekDays = (DayOfWeek[])DayOfWeek.GetValues(typeof(DayOfWeek)); } /// <summary> /// The time of day that the build should run at. The time should be specified in a locale-specific format (ie. H:mm am/pm is acceptable for US locales.) /// </summary> /// <version>1.0</version> /// <default>n/a</default> [ReflectorProperty("time")] public virtual string Time { get { return integrationTime.ToString(); } set { try { integrationTime = TimeSpan.Parse(value); } catch (Exception ex) { string msg = "Unable to parse daily schedule integration time: {0}. The integration time should be specified in the format: {1}."; throw new ConfigurationException(string.Format(CultureInfo.CurrentCulture, msg, value, CultureInfo.CurrentCulture.DateTimeFormat.ShortTimePattern), ex); } } } /// <summary> /// Adds a random amount of minutes between 0 and set value to the time. This is mainly meant for spreading the load of actions to a central server. /// Value must be between 0 and 59. /// </summary> /// <version>1.4</version> /// <default>0</default> [ReflectorProperty("randomOffSetInMinutesFromTime", Required = false)] public int RandomOffSetInMinutesFromTime { get { return randomOffSetInMinutesFromTime; } set { randomOffSetInMinutesFromTime = value; if (randomOffSetInMinutesFromTime < 0 || randomOffSetInMinutesFromTime >= 60) throw new ConfigurationException("randomOffSetInMinutesFromTime must be in the range 0 - 59"); } } /// <summary> /// The name of the trigger. This name is passed to external tools as a means to identify the trigger that requested the build. /// </summary> /// <version>1.1</version> /// <default>ScheduleTrigger</default> [ReflectorProperty("name", Required = false)] public string Name { get { if (name == null) name = GetType().Name; return name; } set { name = value; } } /// <summary> /// The condition that should be used to launch the integration. By default, this value is <b>IfModificationExists</b>, meaning that an integration will /// only be triggered if modifications have been detected. Set this attribute to <b>ForceBuild</b> in order to ensure that a build should be launched /// regardless of whether new modifications are detected. /// </summary> /// <version>1.0</version> /// <default>IfModificationExists</default> [ReflectorProperty("buildCondition", Required = false)] public BuildCondition BuildCondition { get; set; } /// <summary> /// The week days on which the build should be run (eg. Monday, Tuesday). By default, all days of the week are set. /// </summary> /// <version>1.0</version> /// <default>Monday-Sunday</default> [ReflectorProperty("weekDays", Required = false)] public DayOfWeek[] WeekDays { get; set; } //added code /// <summary> /// The project names of the child projects to watch. For cimplicity only the project name, /// meaning that it has to be on the same build server and has browse rights (or no security setup) /// </summary> [ReflectorArray("subProjectsToWatch", Required = true)] public string[] SubProjectsToWatch { get; set; } /// <summary> /// the uri of the ccnet server, example : tcp://localhost:21234/CruiseServerClient.rem /// </summary> [ReflectorProperty("serverUri", Required = true)] public string ServerUri { get; set; } /// <summary> /// A path reachable by the CCNet server (preferably local). This is used to store the timestamps of the subprojects. /// So we can check if the timestamp changed between the integration of a subproject and the build of the involved subproject. /// </summary> [ReflectorProperty("subProjectsSavePath", Required = true)] public string SubProjectsSavePath { get; set; } /// <summary> /// Looks at the timestamp saved in the reference file of the subprojects. If that timestamp differs from the last /// integration timestamp of the subproject, a re-build is needed. /// </summary> /// <returns></returns> private bool SubProjectsAreChanged() { // nothing to watch if (SubProjectsToWatch == null || SubProjectsToWatch.Length == 0) return false; //the save path does not exist, so we must do the first build if (!System.IO.Directory.Exists(SubProjectsSavePath)) return true; //get the last build info from the build server var cm = new RemoteCruiseManagerFactory().GetCruiseManager(ServerUri); var buildInfos = cm.GetCruiseServerSnapshot().ProjectStatuses; foreach (string subprojectName in SubProjectsToWatch) { string locationOnDisk = System.IO.Path.Combine(SubProjectsSavePath, subprojectName); if (! System.IO.File.Exists(locationOnDisk)) return true; DateTime LastTimeBuildInIntegration; string data = System.IO.File.ReadAllText(locationOnDisk); if (DateTime.TryParse(data, out LastTimeBuildInIntegration)) { var lastbuildinfo = GetSubProjectLastBuildInfo(subprojectName, buildInfos); if (lastbuildinfo.BuildStatus != IntegrationStatus.Success) return false; if (LastTimeBuildInIntegration < lastbuildinfo.LastBuildDate ) return true; } else { // file is empty, the contents is not a date, is corrupted, .... throw new Exception(string.Format("Contents of {0} --{1}-- is not convertable to a datetime", locationOnDisk,data)); } } return false; } private Remote.ProjectStatus GetSubProjectLastBuildInfo(string projectName, Remote.ProjectStatus[] serverInfo) { foreach (ProjectStatus ps in serverInfo) { if (ps.Name == projectName) return ps; } throw new Exception(string.Format("Project {0} not found on server with uri {1}", projectName, ServerUri)); } private void SaveSubProjectsLastBuildTimes() { // nothing to watch if (SubProjectsToWatch == null || SubProjectsToWatch.Length == 0) return ; if (!System.IO.Directory.Exists(SubProjectsSavePath)) System.IO.Directory.CreateDirectory(SubProjectsSavePath); DateTime now = dtProvider.Now; foreach (string subprojectName in SubProjectsToWatch) { string locationOnDisk = System.IO.Path.Combine(SubProjectsSavePath, subprojectName); System.IO.File.WriteAllText(locationOnDisk, now.ToLongDateString()); } } //end added code private void SetNextIntegrationDateTime() { if (integrationTime.Minutes + RandomOffSetInMinutesFromTime >= 60) throw new ConfigurationException(string.Format(System.Globalization.CultureInfo.CurrentCulture, "Scheduled time {0}:{1} + randomOffSetInMinutesFromTime {2} would exceed the hour, this is not allowed", integrationTime.Hours, integrationTime.Minutes, RandomOffSetInMinutesFromTime)); DateTime now = dtProvider.Now; nextBuild = new DateTime(now.Year, now.Month, now.Day, integrationTime.Hours, integrationTime.Minutes, 0, 0); if (randomOffSetInMinutesFromTime > 0) { Int32 randomNumber = randomizer.Next(randomOffSetInMinutesFromTime); nextBuild = nextBuild.AddMinutes(randomNumber); } if (now >= nextBuild) { nextBuild = nextBuild.AddDays(1); } nextBuild = CalculateNextIntegrationTime(nextBuild); } private DateTime CalculateNextIntegrationTime(DateTime nextIntegration) { while (true) { if (IsValidWeekDay(nextIntegration.DayOfWeek)) break; nextIntegration = nextIntegration.AddDays(1); } return nextIntegration; } private bool IsValidWeekDay(DayOfWeek nextIntegrationDay) { return Array.IndexOf(WeekDays, nextIntegrationDay) >= 0; } /// <summary> /// Integrations the completed. /// </summary> /// <remarks></remarks> public virtual void IntegrationCompleted() { if (triggered) { // added code SaveSubProjectsLastBuildTimes(); // added code SetNextIntegrationDateTime(); } triggered = false; } /// <summary> /// Gets the next build. /// </summary> /// <value></value> /// <remarks></remarks> public DateTime NextBuild { get { if (nextBuild == DateTime.MinValue) { SetNextIntegrationDateTime(); } return nextBuild; } } /// <summary> /// Fires this instance. /// </summary> /// <returns></returns> /// <remarks></remarks> public IntegrationRequest Fire() { DateTime now = dtProvider.Now; //added code //if (now > NextBuild && IsValidWeekDay(now.DayOfWeek)) if (now > NextBuild && IsValidWeekDay(now.DayOfWeek) && SubProjectsAreChanged()) // end added code { triggered = true; return new IntegrationRequest(BuildCondition, Name, null); } return null; } void IConfigurationValidation.Validate(IConfiguration configuration, ConfigurationTrace parent, IConfigurationErrorProcesser errorProcesser) { string projectName = "(Unknown)"; var project = parent.GetAncestorValue<Project>(); if (project != null) { projectName = project.Name; } if (integrationTime.Minutes + RandomOffSetInMinutesFromTime >= 60) { errorProcesser.ProcessError("Scheduled time {0}:{1} + randomOffSetInMinutesFromTime {2} would exceed the hour, this is not allowed. Conflicting project {3}", integrationTime.Hours, integrationTime.Minutes, RandomOffSetInMinutesFromTime, projectName); } } } }

Example Config

<cruisecontrol xmlns:cb="urn:ccnet.config.builder"> <project name="Child1" > <artifactDirectory>D:\temp\Integration\Child1</artifactDirectory> <workingDirectory>D:\temp\Integration\Child1</workingDirectory> <triggers /> <tasks > <nullTask/> </tasks> <publishers> <xmllogger /> </publishers> </project> <project name="Child2" > <artifactDirectory>D:\temp\Integration\Child2</artifactDirectory> <workingDirectory>D:\temp\Integration\Child2</workingDirectory> <triggers /> <tasks > <nullTask/> </tasks> <publishers> <xmllogger /> </publishers> </project> <project name="Child3" > <artifactDirectory>D:\temp\Integration\Child3</artifactDirectory> <workingDirectory>D:\temp\Integration\Child3</workingDirectory> <triggers /> <tasks > <nullTask/> </tasks> <publishers> <xmllogger /> </publishers> </project> <project name="Integrator" > <artifactDirectory>D:\temp\Integration\Intgrator</artifactDirectory> <workingDirectory>D:\temp\Integration\Integrator</workingDirectory> <triggers> <scheduleTriggerExtended time="14:08" buildCondition="IfModificationExists" name="Scheduled" serverUri="tcp://localhost:21234/CruiseServerClient.rem" subProjectsSavePath="d:\temp"> <subProjectsToWatch> <name>Child1</name> <name>Child3</name> </subProjectsToWatch> <weekDays> <weekDay>Saturday</weekDay> </weekDays> </scheduleTriggerExtended> </triggers> <tasks > <nullTask/> </tasks> <publishers> <xmllogger /> </publishers> </project> </cruisecontrol>