Integrating SLIM+Fitnesse with CruiseControl

On the current project we use SLIM framework within FitNesse for Acceptance testing. We rely quite heavily on the Fitnesse tests results to certify our builds. Hence it was important for us to automate running our SLIM tests and to properly publish the result in the cruise page.
I use the ant scripts to execute Fitnesse test runner from a command-line, parse the resulting XML and transform it using style sheet and finally write the results to the xml files in the cruise log directory. Сonsequently the CruiseControl dashboard with the results looks like here

To have such view each suite should have its own xml log in an appropriate format. I do this just running each suite separately. For this purpose I use the names fitnesse command to get a list of the suites and then run them using the suite Fitnesse command, so it is pretty easy to save the results in the different files. If I don't need to run the whole bunch of the suites (for example: only suites of a particular virtual machine) I use the suiteFilter parameter of the suite command.

	<target name="run all fitnesse tests">
	<loadresource property="FrontPage_names">
	<url url="${fit.Server}/${fit.WorkingPage}?names"/>
	</loadresource>
	<for list="${FrontPage_names}" delimiter="${line.separator}" param="suite">
	<sequential>
	<antcall target="exec fitnesse suite" >
	<param name="suite" value="@{suite}"/>
	<param name="log.file" value="${working.dir}\@{suite}Result.log"/>
	</antcall>
	<xmlproperty file="${working.dir}\@{suite}Result.log"/>
	<antcall target="save log as xml" >
	<param name="suite" value="@{suite}"/>
	<param name="log.file" value="${working.dir}\@{suite}Result.log"/>
	</antcall>
	</sequential>
	</for>
	</target>

view raw run-all-fitnesse-tests.xml hosted with ❤ by GitHub

	<target name="exec fitnesse suite">
	<java jar="${fit.Dir}\fitnesse.jar" fork="true" outputproperty="suite_result">
	<arg value="-c"/>
	<arg value="${fit.WorkingPage}.${suite}?suite&amp;format=xml&amp;suiteFilter=${test.lab.name}"/>
	<arg value="-d"/>
	<arg value="${fit.Dir}"/>
	<arg value="-p"/>
	<arg value="${fit.port}"/>
	</java>
	<echo file="${log.file}">&lt;?xml version="1.0"?&gt;${line.separator}</echo>
	<for list="${suite_result}" delimiter="${line.separator}" param="line">
	<sequential>
	<if>
	<equals arg1="@{line}" arg2="&lt;testResults&gt;" />
	<then>
	<property name="xml.started" value="true" />
	</then>
	</if>
	<if>
	<and>
	<isset property="xml.started"/>
	<not>
	<isset property="xml.finished"/>
	</not>
	</and>
	<then>
	<echo message="@{line}${line.separator}" file="${log.file}" append="true" />
	</then>
	</if>
	<if>
	<equals arg1="@{line}" arg2="&lt;/testResults&gt;" />
	<then>
	<property name="xml.finished" value="true" />
	</then>
	</if>
	</sequential>
	</for>
	</target>

view raw exec-fitnesse-suite.xml hosted with ❤ by GitHub

As you see the 'exec fit suite' target is a procedure that runs fitnesse against the given suite and store it's output into the 'suite_result' property. After that there is a small trick. Unfortunately, the output contains not only test's result in the xml format, but also the fitnesse command output

FitNesse (v20100103) Started...
 port:              9125
 root page:         fitnesse.wiki.FileSystemPage at C:\FitNesse/FitNesseRoot
 logger:            none
 authenticator:     fitnesse.authentication.PromiscuousAuthenticator
...
Exit-Code: 0

So, it is necessary to filter it and here only strings between <testResults> and </testResults> are written to the logs.
As I already have said, the suiteFilter parameter of the suite command is used to run only particular suites. In  this case fitnesse tries to execute each suite but if it does not fit for the given criterion it's tests do not start but we have the xml output as well. Such output does not have relativePageName tag, so in the 'save log as xml ' I just check it and write xml logs only for suites that were really ran.

	<target name="save log as xml">
	<xmlproperty file="${log.file}" collapseAttributes="true"/>
	<condition property="suite.failed">
	<or>
	<not>
	<equals arg1="${testResults.finalCounts.wrong}" arg2="0"/>
	</not>
	<not>
	<equals arg1="${testResults.finalCounts.ignores}" arg2="0"/>
	</not>
	<not>
	<equals arg1="${testResults.finalCounts.exceptions}" arg2="0"/>
	</not>
	</or>
	</condition>
	<if>
	<isset property="testResults.result.relativePageName"/>
	<then>
	<xslt in="${log.file}" out="${working.dir}\${suite}Result.xml" style="rfittests.xsl" />
	<fail if="suite.failed" message="Suite ${suite} on ${test.lab.name} is failed"/>
	<echo message="Suite ${suite} on ${test.lab.name} is succeded"/>
	</then>
	<else>
	<echo message="Suite ${suite} for ${test.lab.name} is skipped"/>
	</else>
	</if>
	</target>

view raw save-fitnesse-log-as-xml.xml hosted with ❤ by GitHub

Pay attention to the 'xslt' ant's task in the 'save log as xml'target. It is used here to write fitnesse test's results in the particular format by processing they via the XSLT.

	<?xml version="1.0" encoding="UTF-8"?>
	<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
	<xsl:output method="xml" indent="yes"/>
	<xsl:variable name="fitresults.list" select="//testResults"/>
	<xsl:template match="/" mode="fittests">
	<testsuite name="{$fitresults.list/rootPath}">
	<xsl:for-each select="$fitresults.list/result">
	<testcase name="{relativePageName}">
	<xsl:variable name="wrong" select="format-number(counts/wrong,'0')"/>
	<xsl:variable name="ignores" select="format-number(counts/ignores,'0')"/>
	<xsl:variable name="exceptions" select="format-number(counts/exceptions,'0')"/>
	<xsl:variable name="total.errors" select="$wrong + $ignores + $exceptions"/>
	<xsl:if test="$total.errors > 0">
	<error>http://my-builder:8000/<xsl:value-of select='pageHistoryLink'/></error>
	</xsl:if>
	</testcase>
	</xsl:for-each>
	</testsuite>
	</xsl:template>
	<xsl:template match="/">
	<xsl:apply-templates select="." mode="fittests"/>
	</xsl:template>
	</xsl:stylesheet>

view raw fitnesse-cc.xslt hosted with ❤ by GitHub

That's it for today. Feel free to ask me questions.

Problems use runProgramInGuest in Windows 2008

I meet a problem when use runProgramInGuest method of vmrun to run Acceptance Tests for my product in Windows 2008 os on the ESXi 4.0.0 server. The problem is that I can't see any active window in the windows 2008 console and as a result it is hard to develop and support user interface acceptance tests. It happens because of a program invoked by RunProgramInGuest is running under session 0 that is non-interactive. It is made in order to better protect the system against security risks. But what if a session 0 service needs to interact with the user? For example : a backup program wants to inform the user that it cannot start an automated backup because the external USB hard drive is disconnected. For this purpose there is the Interactive Services Detection Services (UI0Detect). It can be invoked by the service that needs to interact with the user. UI0Detect in turn presents to the user a window telling him that a particular service needs some interaction. The user gets two choices : "Show me the message", or "Remind me in a few minutes".



Clicking "Show me the message" switches the display to the service which wanted to interact with the user. By default, the startup mode of this service is Manual. Just use command below in cmd to change this mode.

sc config UI0Detect start= auto

So this very gracefully solves my problem and I'm happy - my product will be tested in windows 2008.