It’s possible to integrate TestPartner tests with the Quality Center test lab and execute the tests locally on a machine. In order to do this, the following issues need to be addressed:
- A “VAPI-XP-TEST” needs to be setup in Quality Center. This acts as a placeholder for the TestPartner Test.
- The VAPI test needs to be executed from the QC test lab and actually launch the test in TestPartner on the local machine.
- After the test has run we need to be able to analyse the results and determine the overall run status so that we can update the Quality Center test lab with a “Passed” or “Failed” status.
- After analysing the results, we need to attach them to the Quality Center test lab, so that any testers investigating the reasons for failure can look into potential problems.
Before you start, download the “Test Partner Results Extractor.xls” file to the machine where you need to execute the tests on. This component is required to perform the results analysis after executing the test, and is explained later on. For the purpose of this example I recommend you create a directory and download the file to "C:\TestPartner\".
So here's how you do it.....
1) Setting up the VAPI-XP-TEST
- Go to the Quality Center Test Plan
- Select Tests > New Test and choose these settings (note: you must give the VAPI test the same name as your TestPartner TestScript / Visual Test).
- Press Ok.
- If there are any mandatory QC fields, fill them in.
- In the VAPI wizard, choose these settings:
- Click on Finish.
- Now select the test from the test lab, and click on the “Test Script” tab.
- Paste the following code into the Test Script:
' ====================================================
' VAPITest01 [VBScript]
' Created by David Hartley
' 01/06/2011 16:34:25
' ====================================================
' ----------------------------------------------------
' Main Test Function
' Debug - Boolean. Equals to false if running in [Test Mode] : reporting to Quality Center
' CurrentTestSet - [OTA COM Library].TestSet.
' CurrentTSTest - [OTA COM Library].TSTest.
' CurrentRun - [OTA COM Library].Run.
' ----------------------------------------------------
Sub Test_Main(Debug, CurrentTestSet, CurrentTSTest, CurrentRun)
' clear output window
TDOutput.Clear
'***************** VARIABLES TO BE MODIFIED **************************
Dim strTPDatabase : strTPDatabase = "Test Partner"
Dim strTPUsername : strTPUsername = "David Hartley"
Dim strTPPassword : strTPPassword = "pass"
Dim strTPProject : strTPProject = "Common"
'Test Name - this code can launch a Visual Test or a Test Script. Only fill in the test name
'for the type of test you are launching, e.g if it's a visual test then leave strTPTestScriptName blank
Dim strTPTestScriptName : strTPTestScriptName = ""
Dim strTPVisualTestName : strTPVisualTestName = CurrentTSTest.TestName
Dim strTestPartnerResultsExtractorName : strTestPartnerResultsExtractorName = "Test Partner Results Extractor.xls"
Dim strResultsDirectory : strResultsDirectory = "C:\TestPartner\"
'***********************************************************************************************
Dim WshShell, objExecObject, strOutput
Dim strCommand, strTestTypeCommand, strRes
Dim strTestName
'Now setup some variables to handle to test results
Dim strMacroName : strMacroName = "ExtractResultsFromTP"
Dim strExpectedExcelResultsPath
Dim strTimeStamp
Dim strTemp
Dim objExcel, objExcelWorkbook
'setup the timestamp in YYYYMMDD_HHMM format
'get the year
strTimeStamp = year(now())
'get the month
strTemp = month(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp
'get the day
strTemp = day(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp & "_"
'get the hour
strTemp = hour(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp
'get the minute
strTemp = minute(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp
'get the seconds
strTemp = second(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp
'setup testname variable
if strTPTestScriptName <> "" then
strTestName = strTPTestScriptName
else
strTestName = strTPVisualTestName
end if
'Now we can setup the expected excel results path
strExpectedExcelResultsPath = strResultsDirectory & strTestName & " Results " & strTimeStamp & ".xls"
'Now start setting up variables used to send the command
if strTPTestScriptName <> "" then
strTestTypeCommand = """ -s """ & strTPTestScriptName & """"
else
strTestTypeCommand = """ -t """ & strTPVisualTestName & """"
end if
'create a windows shell object
Set WshShell = CreateObject("WScript.Shell")
'construct the command we want to send to it
strCommand = "TP -d """ & strTPDatabase & """ -u """ & strTPUsername & """ -p """ & strTPPassword & """ -r """ & strTPProject & strTestTypeCommand
'this is useful for debug purposes
TDOutput.Print "Launching TP with this command:"
TDOutput.Print strCommand
'Now run the command which will launch TestPartner - if nothing happens, put the command into a
'command prompt window (start > run > "cmd") to see if there are any errors from it
Set objExecObject = WshShell.Exec(strCommand)
strOutput = objExecObject.StdOut.readall()
if strOutput <> "" then
TDOutput.Print strOutput
'if this was a visual test, then this outputs a playback error but we can still go on to retrieve the results
if instr(strOutput, "Playback error") <= 0 then
If Not Debug Then
TDOutput.Print "Test Failed to launch"
CurrentRun.Status = "Failed"
CurrentTSTest.Status = "Failed"
end if
exit sub
end if
end if
'if we reach here then the test ran correctly, we can now go and retrieve the results using the excel workbook
Set objExcel = CreateObject("Excel.Application")
'objExcel.visible = true 'show excel *** Uncomment this if you want to see what is happening ***
'open the extractor workbook - this will do the donkey work of retrieving the results
TDOutput.Print "Opening " & strResultsDirectory & strTestPartnerResultsExtractorName
set objExcelWorkbook = objExcel.Workbooks.Open(strResultsDirectory & strTestPartnerResultsExtractorName,false,true)'opens it readonly
'now setup the variables on the config sheet
objExcelWorkbook.sheets("config").select
'setup the variables
objExcelWorkbook.sheets("config").cells(1,2) = strTPDatabase
objExcelWorkbook.sheets("config").cells(2,2) = strTPUsername
objExcelWorkbook.sheets("config").cells(3,2) = strTPPassword
objExcelWorkbook.sheets("config").cells(4,2) = strTPProject
objExcelWorkbook.sheets("config").cells(5,2) = strTestName
objExcelWorkbook.sheets("config").cells(6,2) = strTimeStamp
objExcelWorkbook.sheets("config").cells(7,2) = strResultsDirectory
'now run the macro to generate the results (this causes VAPI to crash so wrap error handling around it)
TDOutput.Print "Retrieving results"
on error resume next
objExcel.run strMacroName
on error goto 0
'if we're not in debug mode then open the results and determine the pass/fail status
If Not Debug Then
TDOutput.Print "Opening results worksheet: " & strExpectedExcelResultsPath
'now open the results worksheet to determine the pass/fail status
set objExcelWorkbook = objExcel.Workbooks.Open(strExpectedExcelResultsPath,false,true)'opens it readonly
'what is the overall status of the results (this is calculated by the Excel Workbook)?
if objExcelWorkbook.sheets("Results").cells(1,5) = "Pass" then
TDOutput.Print "Test passed"
CurrentRun.Status = "Passed"
CurrentTSTest.Status = "Passed"
else
TDOutput.Print "Test Failed"
CurrentRun.Status = "Failed"
CurrentTSTest.Status = "Failed"
end if
objExcelWorkbook.saved = true
objExcelWorkbook.close
'now upload the workbook results to the curentRun - this means
'that when you double click the results in QC you will see this as an
'attachment
TDOutput.Print "Uploading results from: " & strExpectedExcelResultsPath
set attachF = CurrentRun.Attachments
Set theAttachment = attachF.AddItem(null)
theAttachment.FileName = strExpectedExcelResultsPath
theAttachment.Type = 1
theAttachment.Post
end if
objExcel.quit
set objExcelWorkbook = nothing
set objExcel = nothing
TDOutput.Print "Finished"
End Sub
' Created by David Hartley
' 01/06/2011 16:34:25
' ====================================================
' ----------------------------------------------------
' Main Test Function
' Debug - Boolean. Equals to false if running in [Test Mode] : reporting to Quality Center
' CurrentTestSet - [OTA COM Library].TestSet.
' CurrentTSTest - [OTA COM Library].TSTest.
' CurrentRun - [OTA COM Library].Run.
' ----------------------------------------------------
Sub Test_Main(Debug, CurrentTestSet, CurrentTSTest, CurrentRun)
' clear output window
TDOutput.Clear
'***************** VARIABLES TO BE MODIFIED **************************
Dim strTPDatabase : strTPDatabase = "Test Partner"
Dim strTPUsername : strTPUsername = "David Hartley"
Dim strTPPassword : strTPPassword = "pass"
Dim strTPProject : strTPProject = "Common"
'Test Name - this code can launch a Visual Test or a Test Script. Only fill in the test name
'for the type of test you are launching, e.g if it's a visual test then leave strTPTestScriptName blank
Dim strTPTestScriptName : strTPTestScriptName = ""
Dim strTPVisualTestName : strTPVisualTestName = CurrentTSTest.TestName
Dim strTestPartnerResultsExtractorName : strTestPartnerResultsExtractorName = "Test Partner Results Extractor.xls"
Dim strResultsDirectory : strResultsDirectory = "C:\TestPartner\"
'***********************************************************************************************
Dim WshShell, objExecObject, strOutput
Dim strCommand, strTestTypeCommand, strRes
Dim strTestName
'Now setup some variables to handle to test results
Dim strMacroName : strMacroName = "ExtractResultsFromTP"
Dim strExpectedExcelResultsPath
Dim strTimeStamp
Dim strTemp
Dim objExcel, objExcelWorkbook
'setup the timestamp in YYYYMMDD_HHMM format
'get the year
strTimeStamp = year(now())
'get the month
strTemp = month(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp
'get the day
strTemp = day(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp & "_"
'get the hour
strTemp = hour(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp
'get the minute
strTemp = minute(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp
'get the seconds
strTemp = second(now())
if len(strTemp) = 1 then strTemp = "0" & strTemp
strTimeStamp = strTimeStamp & strTemp
'setup testname variable
if strTPTestScriptName <> "" then
strTestName = strTPTestScriptName
else
strTestName = strTPVisualTestName
end if
'Now we can setup the expected excel results path
strExpectedExcelResultsPath = strResultsDirectory & strTestName & " Results " & strTimeStamp & ".xls"
'Now start setting up variables used to send the command
if strTPTestScriptName <> "" then
strTestTypeCommand = """ -s """ & strTPTestScriptName & """"
else
strTestTypeCommand = """ -t """ & strTPVisualTestName & """"
end if
'create a windows shell object
Set WshShell = CreateObject("WScript.Shell")
'construct the command we want to send to it
strCommand = "TP -d """ & strTPDatabase & """ -u """ & strTPUsername & """ -p """ & strTPPassword & """ -r """ & strTPProject & strTestTypeCommand
'this is useful for debug purposes
TDOutput.Print "Launching TP with this command:"
TDOutput.Print strCommand
'Now run the command which will launch TestPartner - if nothing happens, put the command into a
'command prompt window (start > run > "cmd") to see if there are any errors from it
Set objExecObject = WshShell.Exec(strCommand)
strOutput = objExecObject.StdOut.readall()
if strOutput <> "" then
TDOutput.Print strOutput
'if this was a visual test, then this outputs a playback error but we can still go on to retrieve the results
if instr(strOutput, "Playback error") <= 0 then
If Not Debug Then
TDOutput.Print "Test Failed to launch"
CurrentRun.Status = "Failed"
CurrentTSTest.Status = "Failed"
end if
exit sub
end if
end if
'if we reach here then the test ran correctly, we can now go and retrieve the results using the excel workbook
Set objExcel = CreateObject("Excel.Application")
'objExcel.visible = true 'show excel *** Uncomment this if you want to see what is happening ***
'open the extractor workbook - this will do the donkey work of retrieving the results
TDOutput.Print "Opening " & strResultsDirectory & strTestPartnerResultsExtractorName
set objExcelWorkbook = objExcel.Workbooks.Open(strResultsDirectory & strTestPartnerResultsExtractorName,false,true)'opens it readonly
'now setup the variables on the config sheet
objExcelWorkbook.sheets("config").select
'setup the variables
objExcelWorkbook.sheets("config").cells(1,2) = strTPDatabase
objExcelWorkbook.sheets("config").cells(2,2) = strTPUsername
objExcelWorkbook.sheets("config").cells(3,2) = strTPPassword
objExcelWorkbook.sheets("config").cells(4,2) = strTPProject
objExcelWorkbook.sheets("config").cells(5,2) = strTestName
objExcelWorkbook.sheets("config").cells(6,2) = strTimeStamp
objExcelWorkbook.sheets("config").cells(7,2) = strResultsDirectory
'now run the macro to generate the results (this causes VAPI to crash so wrap error handling around it)
TDOutput.Print "Retrieving results"
on error resume next
objExcel.run strMacroName
on error goto 0
'if we're not in debug mode then open the results and determine the pass/fail status
If Not Debug Then
TDOutput.Print "Opening results worksheet: " & strExpectedExcelResultsPath
'now open the results worksheet to determine the pass/fail status
set objExcelWorkbook = objExcel.Workbooks.Open(strExpectedExcelResultsPath,false,true)'opens it readonly
'what is the overall status of the results (this is calculated by the Excel Workbook)?
if objExcelWorkbook.sheets("Results").cells(1,5) = "Pass" then
TDOutput.Print "Test passed"
CurrentRun.Status = "Passed"
CurrentTSTest.Status = "Passed"
else
TDOutput.Print "Test Failed"
CurrentRun.Status = "Failed"
CurrentTSTest.Status = "Failed"
end if
objExcelWorkbook.saved = true
objExcelWorkbook.close
'now upload the workbook results to the curentRun - this means
'that when you double click the results in QC you will see this as an
'attachment
TDOutput.Print "Uploading results from: " & strExpectedExcelResultsPath
set attachF = CurrentRun.Attachments
Set theAttachment = attachF.AddItem(null)
theAttachment.FileName = strExpectedExcelResultsPath
theAttachment.Type = 1
theAttachment.Post
end if
objExcel.quit
set objExcelWorkbook = nothing
set objExcel = nothing
TDOutput.Print "Finished"
End Sub
' ====================================================
- Now you need to configure the variables specific to the test. In the header, change these variables:
strTPDatabase – The name of the database to connect TP to.
strTPUsername – A username that can execute the test.
strTPPassword – password for that username.
strTPProject – The name of the project where the test is saved in.
strTPTestScriptName – If the test is a TestScript, set this value to “CurrentTSTest.TestName”, otherwise set the value to blank “”.
strTPVisualTestName– If the test is a Visual Test, set this value to “CurrentTSTest.TestName”, otherwise set the value to blank “”.
strTestPartnerResultsExtractorName - The name of the excel tool that analyses the results, the default value is "Test Partner Results Extractor.xls".
strResultsDirectory – A location on the machine where the results will be saved to. The excel tool (strTestPartnerResultsExtractorName) must also be placed in this directory. You must include a “\” character at the end of the path name.
2) Executing from Quality Center
Now the VAPI test has been setup, you can execute the test from Quality Center. Create a test lab and select the VAPI test. To run it, use the run button – you can only run it on the local machine (so check the “Run All Tests Locally” box).
Whilst it’s running, the Output dialog should display some useful messages. The script works by sending a shell command to invoke TestPartner: “TP –d [database] –u [username] –p [password] –r [project] –t (or –s) [testname]”
3) Analysing Results and Attaching them to Quality Center
Once the test has been run, the VAPI test then invokes Excel and loads the “Test Partner Results Extractor.xls” spreadsheet. This spreadsheet contains a macro which will go and retrieve the test results, export them to an XML file and then import the XML into an Excel workbook, which is saved to the local drive. This happens silently in the background.
Once the results have been opened in Excel, the Excel code parses the results columns and determines if the run has passed or failed, which the VAPI code then uses to set the pass or failed status in Quality Center.
The VAPI code then takes the saved results workbook and attaches it to the Quality Center Test Set results so that you can see the run results and reasons for the pass / fail status. In the screenshot below you can see a green paperclip in the attachments – this is where you can find the results.
Clicking the paperclip takes you to the excel file containing the results.
So that explains how to launch Test Partner tests from Quality Center, obtain a passed/failed status and have the results uploaded to Quality Center. There are a few things to consider with this approach:
- The tests can only be executed locally on the machine.
- The results are in Excel format – this means that screenshots are not included so you may still need to go into TestPartner to investigate the results.
- Sometimes QC doesn’t save the changes you’ve made to your VAPI script – to check if it has, load another VAPI script and then load your script to see if the changes have been applied.
- If TestPartner fails to launch the test (check your parameters have been setup, especially the test type), then there will not be any attachments in Quality Center – simply a Failed status.
