Using sandboxing in your tests

As you know mocks aren't stubs: stubs are used to return canned data to your system under test, so that you can make some assertions on how your code reacts to that data. Mocks, on the other hand, are used to specify certain expectations about how the methods of the mocked object are called by your system under test. But you should still use a mock library or framework even when you want to use stubs, because these libraries make it very easy to instantiate and work with stubs.

	my_mock = mock.Mock()
	my_mock.some_method.return_value = "calculated value"
	my_mock.some_attribute = "value"
	assertEqual("calculated value", my_mock.some_method())
	assertEqual("value", my_mock.some_attribute)

view raw simple-fake-object.py hosted with ❤ by GitHub

Recenlty I've discovered another interesting approach. It's provided by the tl.testing package that allows to sandboxe of directories and files. Here is my example. This code implements a search for a given string in files. I use doctest to check the functionality.

	def listfiles(folder):
	for root, dirs, files in os.walk(folder):
	for fname in files:
	if not contrib.ishidden(fname):
	yield os.path.join(root, fname)
	def search(folder, path, search_pattern):
	"""
	>>> from tl.testing.fs import new_sandbox
	>>> new_sandbox('''\\
	... f test-1.js asdfg
	... f test-2.js asdft
	... f .test-2.js asdfg
	... f test-3.js hjkl
	... ''')
	>>> result = search(os.getcwd(), 'cases', 'asdf')
	>>> 'cases/test-1.js' in result
	True
	>>> 'cases/test-2.js' in result
	True
	>>> 'cases/.test-2.js' in result
	False
	>>> 'cases/test-3.js' in result
	False
	"""
	searches = {}
	for filepath in listfiles(folder):
	result = match( filepath, search_pattern)
	if result:
	filepath = filepath.replace(folder, path).replace('\\', '/').replace('//', '/')
	searches[filepath] = result
	return searches
	def match(filepath, search_pattern):
	"""
	>>> from tl.testing.fs import new_sandbox
	>>> new_sandbox('''\\
	... f test-1.js asdfg
	... ''')
	>>> find(os.getcwd() + '/test-1.js', 'asdf')
	[[(0, 'asdfg', 'asdf')]]
	"""
	regexp = re.compile(r'('+ search_pattern +r')')
	f = open(filepath, 'r')
	filecontent = f.read()
	f.close()
	matches = regexp.finditer(filecontent)
	all_results = []
	for match in matches:
	start = match.start()
	lineno = filecontent.count('\n', 0, start)
	if lineno > 0: lineno -= 1
	lines = filecontent.replace('\r','').split('\n')[lineno: lineno+3]
	match_result = []
	for line in lines:
	highlight = re.search( search_pattern, line )
	if highlight: highlight = highlight.group(0)
	match_result += [ (lineno, line, highlight,) ]
	lineno += 1
	all_results += [ match_result ]
	return all_results

view raw tl.testing.fs.demo.py hosted with ❤ by GitHub

InfoQ: Brian Marick on Test Maintenance

Very interesting: Brian Marick discusses the difficulties met trying to maintain tests that are vital to a project’s success, and how mocking frameworks can help, providing advice on writing unit and integration tests InfoQ: Brian Marick on Test Maintenance

"The Expression Problem is a new name for an old problem. The goal is to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts)."

~ Philip Wadler

Each new user of a new system uncovers a new class of bugs.

~ Brian W. Kernighan

CSS property Overflow & scroll bar: how to test

The CSS property Overflow can be used when displaying content. It regulates the scrollbars. You can use 4 variables in the element overflow, namely auto, hidden, scroll and visible. Auto will automatically add a scrollbar when needed. Hidden will not show a scrollbar, but ‘hide’ the content that usually expand the box. Scroll will always add a scrollbar. The value visible will not expand the div, but will just display all the content without changing the div’s height.

Very convenient technic if you want to let a scroll bar appear when content of your DIV gets to a specific height. For more details you can read this article for example, but in this tutorial I'm going to cover testing aspects of this technic.

Actually here there is a problem. How is it possible to check that your scroll bar appears or not when dynamic content of your DIV changes? You can't just get a visibility of the scroll bar and verify it. But we can use the scrollTop jQuery method, that gets the current vertical position of the scroll bar for the given element: the vertical scroll position is the same as the number of pixels that are hidden from view above the scrollable area. If the scroll bar is at the very top, or if the element is not scrollable, this number will be 0.

	$('#show-dialog').scrollTop(position);
	equal( $('#show-dialog').scrollTop(), position, "it's scrollable");

view raw css-property-overflow-test.js hosted with ❤ by GitHub

The first scrollTop call with the position parameter sets the current vertical position of the scroll bar for the given element. If this element is scrollable, the second empty scrollTop call will return value that is equal the given position value, otherwise it will return zero.

Using FITNesse and Selenium for acceptance testing, example 2

This example is about using FITNesse QueryTable fixture with Selenium. Let's consider next scenario. There is a list of SharePoint sites and an administrator can grant permissions to users for any site in this list. A result of such action is a report that contains list of the users with sites they have permissions for.

The best way to check that the action is competed succefully is to check that given user is presented in this report and has appropriate permissions set. It is very easy to do using the QueryTable fixture.

	!|script|
	|start|Browser|
	|$port=|get port|
	|go|http://${IP_HOST}:$port/make/search?filter=wappl&search_limit=20|
	|wait for load complete;|
	|set selected;|id=${wa7}|
	|set selected;|id=${wa8}|
	|click|id=btnGrant-button|
	!|script|
	|is element displayed;|id=dialogGrant|
	|check|get_value|id=id_scope_grant|${wa8};${wa7};|
	|type;|id=id_users_grant|${domain}\${U1};${domain}\${U2}|
	|click|xpath=//input[@name="Full Control"]|
	|submit|xpath=//form[@id="formGrantPermissions"]|
	!|script|
	|wait for element is not displayed;|id=perpetum|${actionTimeout}|
	|Subset Query:permissions actions report|${domain}\${U1}|
	|site|level|
	|${SITE_COLLECTION1}|${PM}|
	|${SITE_COLLECTION2}|${PM}|
	|${wa7}|${PM}|
	|${wa8}|${PM}|

view raw fitnesse-subset-query hosted with ❤ by GitHub

Just define a class with the query method and a constructor that takes a user name. Pay attention that this QueryTable class should aggregate the Browser class instance to access data on the html page.

	class PermissionsActionsReport():
	def __init__(self,u):
	self.ip = Browser()
	self.user = u
	def query(self):
	ret = []
	i=0
	while True:
	i=i+1
	try:
	curuser = self.ip.get_text('xpath=//*[@id="result"]/ul/li[%s]/b'%str(i))
	if self.user.lower() == curuser.lower():
	break
	except:
	return []
	j=0
	while True:
	j=j+1
	try:
	url = self.ip.get_text('xpath=//*[@id="result"]/ul/li[%s]/table/tbody/tr[%s]/td/div/span/a'%(str(i),str(j)))
	permissions = self.ip.get_text('xpath=//*[@id="result"]/ul/li[%s]/table/tbody/tr[%s]/td[2]'%(str(i),str(j)))
	print "%i. %s %s" % (j, url, permissions)
	if type(url)!=type(u""):
	return ret
	ret.append([['site',url],['level',permissions]])
	except:
	return ret

view raw QueryTable.py hosted with ❤ by GitHub

But actually here is a problem. An architecture of FITNesse does not allow the PermissionsActionsReport QueryTable class instance to access existed instance of the Browser class that was used to make the action and that contains the result of this action. Fortunately this obstacle can be solved easy using the singleton pattern.

	class Browser(object):
	__active = False
	def __new__(type, browser):
	if not '_instance' in type.__dict__:
	type._instance = object.__new__(type)
	return type._instance
	def __init__(self, browser):
	if browser == 'chrome':
	from selenium.chrome.webdriver import WebDriver
	elif browser == 'firefox':
	from selenium.firefox.webdriver import WebDriver
	else:
	from selenium.ie.webdriver import WebDriver
	if not self.__active:
	self.__wd = WebDriver()
	self.__active = True

view raw webdrive-2.py hosted with ❤ by GitHub

Using FITNesse and Selenium for acceptance testing, example 1

I have alredy written that Fitnesse+SLIM+Selenium is a pretty powerfull tandem to be used for testing web applications. Also I have a blog post about Selenium 2/WebDriver that is much better for testing web applications than first Selenium. In this and upcoming posts I gonna to share my experience of testing complex use cases in my web application using Fitnesse and WebDriver.

For example, there is a list of the SharePoint Sites. You can use checkboxes and a toolbar to select and execute a set of actions for them. Let's do Copy List: select a site and click the Copy List button. It will araise the Select Lists dialog box.

Here you have to select lists what you want to copy and press the Next button. In the next dialog box it's necessary to select target site. Actually a list of these site can be very long. But there is a search here that allows you to narrow a scope: type a name of site that you want to target and press the Search button. Now find the site in the search result and press copy.

Now wait for operation to be completed.

Actually the scenario forks here. It can be completed successfully or not.

If operation is failed, you will see an error description and can click to the SharePoint log files link to see more details.

Here the fitnesse test that checks this scenario.

And here it's code

	!|script|
	|start|Browser|
	|$port=|get port|
	|go|${searchCopyUrl}|
	|wait for load complete;|
	|toggle;|id=${SourceSite}|
	|click|id=btnCopyList-button|
	!|script|
	|wait for element present;|xpath=//input[@id="${list}"]|10|
	|toggle;|id="${list}"|
	|click|id=dlgMoveListsOKButton-button|
	|wait for element present;|id=copy_username_box|10|
	!|script|
	|type;|id=copy_username_box|${TargetSite}|
	|click|id=copy_search_button-button|
	|wait for element present;|xpath=//label[@for="${TargetSite}"]|10|
	|click|xpath=//label[@for="${TargetSite}"]|
	|click;|id=dlgMoveOKButton-button|
	|wait for load complete;|
	!|script|
	|wait for element is displayed;|id=perpetum|
	|wait for element is not displayed;|id=perpetum|${copyTimeout}|
	|ensure|is element present;|id=trendErrors|
	|ensure|is element present;|xpath=//a[text()="SharePoint log files"]|
	|click|xpath=//a[text()="SharePoint log files"]|
	|wait for load complete;|
	|wait for element present|id=result|
	|check|eval|return document.getElementById("result").innerText|=~/with [^0] errors./|
	!|script|
	|close|

view raw CheckLogFilesLinkOnError hosted with ❤ by GitHub

As you can see the working element here is the Browser python class

	class NullElement:
	def __getattr__(self, name):
	raise Exception(elementNotFound)
	def is_displayed(self):
	return 0
	class Browser(object):
	def __init__(self, browser = 'iexplore'):
	if browser == 'chrome':
	from selenium.chrome.webdriver import WebDriver
	elif browser == 'firefox':
	from selenium.firefox.webdriver import WebDriver
	else:
	from selenium.ie.webdriver import WebDriver
	self.wd = WebDriver()
	def __getattr__(self, name):
	return getattr(self.wd, name)
	def get_port(self):
	return variables.get_port()
	def maximize(self):
	script = "window.moveTo(0, 0);window.resizeTo(window.screen.availWidth, window.screen.availHeight);"
	self.execute_script(script)
	def go(self, url):
	self.get(url)
	self.maximize()
	def get_by_id(self, locator):
	type, separator, value = locator.partition('=')
	if type.lower() == 'id':
	return self.find_element_by_id(value)
	def get_by_xpath(self, locator):
	type, separator, value = locator.partition('=')
	if type.lower() == 'xpath':
	return self.find_element_by_xpath(value)
	def find_element(self, locator):
	element = NullElement()
	try:
	element = self.get_by_id(locator) or self.get_by_xpath(locator) or log.Except(wrongLocator)
	except Exception, e:
	log.Except(e)
	return element
	def toggle(self, locator):
	'''
	Just toggle a checkbox by given locator
	Returns: true or false, that depends on the given checkbox state
	Actually it should(or might) contain only two string:
	checkbox = self.find_element(locator)
	return checkbox.toggle()
	but unfortunately I can't get such approach worked in a tests suite.
	I mean that a single test works fine, I never got issues, but if a suite is running
	I constantly have problems that checkbox is not toggled (have no idea why, very hard to debug in suite)
	So I use javascript to toggle a checkbox
	'''
	script = '''
	var elem = $('input[%(locator)s]')
	if (elem.length === 0) {
	throw('%(exception)s')
	}
	var state = elem.attr('checked');
	elem.attr('checked', !state);
	return elem.attr('checked');
	''' % { 'locator': locator, 'exception': elementNotFound }
	return self.execute_script(script)
	def click(self, locator):
	elem = self.find_element(locator)
	elem.click()
	def wait_for(self, condition, time_frame):
	wait_time = int(time_frame)
	def timeout(wait_time):
	return not wait_time
	while not condition():
	time.sleep(1)
	wait_time = wait_time - 1
	if timeout(wait_time):
	raise Exception('Timed out after %s sec' % time_frame)
	def wait_for_element_present(self, locator, timeout=defaultTimeout):
	self.wait_for(lambda: self.is_element_present(locator), timeout)
	def type(self, locator, text):
	elem = self.find_element(locator)
	elem.send_keys(text)
	def wait_for_element_is_displayed(self, locator, timeout=defaultTimeout):
	self.wait_for(lambda: self.is_element_displayed(locator), timeout)
	def wait_for_element_is_not_displayed(self, locator, timeout=defaultTimeout):
	self.wait_for(lambda: not self.is_element_displayed(locator), timeout)
	def is_element_present(self, locator):
	element = self.find_element(locator)
	return not isinstance(element, NullElement)
	def eval(self, script, *args):
	return self.execute_script(script, *args)

view raw webdrive-1.py hosted with ❤ by GitHub