Gathering extra statistics is not that difficult, add a name for the element, and a XPath expression. For example:
<statistic name="AmountOfFailures" xpath="sum(//test-results/@failures)" />Now this is good when the build log is simple xml, but when you have sections with namespaces, things become nasty.
If you're working with VS2010, and are merging the test results, you have encountered this problem before. These test results are stored in the "http://microsoft.com/schemas/VisualStudio/TeamTest/2010" namespace.
This problem is now fixed, I've added support for namespaces.
Below you'll find a xmlFile containing test data coming from a demo project, and I'll use this file as an example :
Suppose you want to know the total amount of tests and the amount of failed ones. This information can be retrieved via the following XPath queries : /TestRun/ResultSummary/Counters/@total and /TestRun/ResultSummary/Counters/@failed
Because the test-result file is merged into the master buildlog file, the TestRun node is not the root node anymore.
Normally you could/would fix this by changing it into //TestRun/ResultSummary/Counters/@total, giving the following config for the statistic :
<statistic name="AmountOfTests" xpath="//TestRun/ResultSummary/Counters/@total" />But this gives MS.Internal.Xml.XPath.XPathSelectionIterator as result. Not very intuitive if you do not work with XPath every day.The problem is the // operator, XPath asumes there can be many TestResult nodes in the xmlfile(even if we know there will only be one), and returns an iterator. To bypass this kind of thing, a FirstMatch class was (and still is) foreseen to handle this, resulting in following setting :
<firstMatch name="AmountOfTests" xpath="//TestRun/ResultSummary/Counters/@total" />But now the result is Null (empty string) in the file. With the support for namespaces, the config is as follows :
<firstMatch name="AmountOfTests" xpath="//mstest:TestRun/mstest:ResultSummary/mstest:Counters/@total" >
<namespaces>
<namespaceMapping prefix="mstest" url="http://microsoft.com/schemas/VisualStudio/TeamTest/2010" />
</namespaces>
</firstMatch>
Resulting in the wanted result of 2 for the file below. This is a huge step forward, but there is still a problem :
suppose you want some statistic with a calculation of a setting with a namespace, and one without a namespace.
This poses a problem with XSLT 1.0, because we can not set a prefix for the default namespace. To overcome that problem we'll have to support XSLT 2.0. where there is functionality foreseen.
This request is added on the todo list.
The test file :
<TestRun id="32083ab3-68fe-40ab-a13d-c289aef12a28"
name="cover_me"
runUser="ruben"
xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<TestSettings name="Cover"
id="9839cf84-0738-4c12-885d-99158bdda72f">
<Description>These are default test settings for a local test run.</Description>
<Deployment userDeploymentRoot="C:\Users\ruben\Documents\Visual Studio 2010\Projects\CCNetStatistics"
useDefaultDeploymentRoot="false"
runDeploymentRoot="cover_me" />
<NamingScheme baseName="cover_me"
appendTimeStamp="false"
useDefault="false" />
<Execution>
<TestTypeSpecific>
<UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
<AssemblyResolution>
<TestDirectory useLoadContext="true" />
</AssemblyResolution>
</UnitTestRunConfig>
</TestTypeSpecific>
<AgentRule name="LocalMachineDefaultRole">
<DataCollectors>
<DataCollector uri="datacollector://microsoft/CodeCoverage/1.0"
assemblyQualifiedName="Microsoft.VisualStudio.TestTools.CodeCoverage.CoveragePlugIn, Microsoft.VisualStudio.QualityTools.Plugins.CodeCoverage, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
friendlyName="Code Coverage">
<Configuration>
<CodeCoverage xmlns="">
<Regular>
<CodeCoverageItem binaryFile="C:\Users\ruben\Documents\Visual Studio 2010\Projects\CCNetStatistics\CCNetStatistics\bin\Debug\Agents.dll"
pdbFile="C:\Users\ruben\Documents\Visual Studio 2010\Projects\CCNetStatistics\CCNetStatistics\bin\Debug\Agents.instr.pdb"
instrumentInPlace="true" />
</Regular>
</CodeCoverage>
</Configuration>
</DataCollector>
</DataCollectors>
</AgentRule>
</Execution>
</TestSettings>
<Times creation="2011-09-04T20:40:22.8989434+02:00"
queuing="2011-09-04T20:40:24.1937456+02:00"
start="2011-09-04T20:40:24.3341459+02:00"
finish="2011-09-04T20:40:28.0469524+02:00" />
<ResultSummary outcome="Failed">
<Counters total="2"
executed="2"
passed="1"
error="0"
failed="1"
timeout="0"
aborted="0"
inconclusive="0"
passedButRunAborted="0"
notRunnable="0"
notExecuted="0"
disconnected="0"
warning="0"
completed="0"
inProgress="0"
pending="0" />
<ResultFiles>
<ResultFile path="LTREMRUBEN\data.coverage" />
</ResultFiles>
</ResultSummary>
<TestDefinitions>
<UnitTest name="TestInfiltration"
storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
id="93fe6bec-7340-59e7-ba45-0a396dc614ff">
<Execution id="792491ae-0c70-45e3-a474-1f1cb130226b" />
<TestMethod codeBase="c:/users/ruben/documents/visual studio 2010/projects/ccnetstatistics/testagents/bin/debug/TestAgents.DLL"
adapterTypeName="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
className="TestAgents.UnitTest1, TestAgents, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
name="TestInfiltration" />
</UnitTest>
<UnitTest name="TestReport"
storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
id="09504e3d-ea5f-fda7-5ee0-32f8cc895784">
<Execution id="c98799b7-8675-43b9-ae03-7eb7fd2acb70" />
<TestMethod codeBase="c:/users/ruben/documents/visual studio 2010/projects/ccnetstatistics/testagents/bin/debug/TestAgents.DLL"
adapterTypeName="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
className="TestAgents.UnitTest1, TestAgents, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
name="TestReport" />
</UnitTest>
</TestDefinitions>
<TestLists>
<TestList name="Smoke"
id="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff"
parentListId="8c43106b-9dc1-4907-a29f-aa66a61bf5b6">
<TestLinks>
<TestLink id="93fe6bec-7340-59e7-ba45-0a396dc614ff"
name="TestInfiltration"
storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
type="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
<TestLink id="09504e3d-ea5f-fda7-5ee0-32f8cc895784"
name="TestReport"
storage="c:\users\ruben\documents\visual studio 2010\projects\ccnetstatistics\testagents\bin\debug\testagents.dll"
type="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.ObjectModel, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</TestLinks>
</TestList>
<TestList name="Lists of Results"
id="8c43106b-9dc1-4907-a29f-aa66a61bf5b6" />
<TestList name="All Loaded Results"
id="19431567-8539-422a-85d7-44ee4e166bda" />
</TestLists>
<TestEntries>
<TestEntry testId="93fe6bec-7340-59e7-ba45-0a396dc614ff"
executionId="792491ae-0c70-45e3-a474-1f1cb130226b"
testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff" />
<TestEntry testId="09504e3d-ea5f-fda7-5ee0-32f8cc895784"
executionId="c98799b7-8675-43b9-ae03-7eb7fd2acb70"
testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff" />
</TestEntries>
<Results>
<UnitTestResult executionId="792491ae-0c70-45e3-a474-1f1cb130226b"
testId="93fe6bec-7340-59e7-ba45-0a396dc614ff"
testName="TestInfiltration"
computerName="LTREMRUBEN"
duration="00:00:00.0342893"
startTime="2011-09-04T20:40:25.9721488+02:00"
endTime="2011-09-04T20:40:26.3777495+02:00"
testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b"
outcome="Passed"
testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff"
relativeResultsDirectory="792491ae-0c70-45e3-a474-1f1cb130226b">
</UnitTestResult>
<UnitTestResult executionId="c98799b7-8675-43b9-ae03-7eb7fd2acb70"
testId="09504e3d-ea5f-fda7-5ee0-32f8cc895784"
testName="TestReport"
computerName="LTREMRUBEN"
duration="00:00:00.0538581"
startTime="2011-09-04T20:40:26.4089495+02:00"
endTime="2011-09-04T20:40:26.4713496+02:00"
testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b"
outcome="Failed"
testListId="39ed6b71-dbd6-41f1-a30d-eebe96ec74ff"
relativeResultsDirectory="c98799b7-8675-43b9-ae03-7eb7fd2acb70">
<Output>
<ErrorInfo>
<Message>Assert.AreEqual failed. Expected:<>. Actual:<all went ok>. </Message>
<StackTrace>
at TestAgents.UnitTest1.TestReport() in c:\users\ruben\documents\visual studio 2010\Projects\CCNetStatistics\TestAgents\UnitTest1.vb:line 17
</StackTrace>
</ErrorInfo>
</Output>
</UnitTestResult>
</Results>
</TestRun>
Posts
No comments:
Post a Comment