There is no problem with your responses - keep in mind this is a journey of thinking and no thoughts are judged good or bad.
Also this is a dialogue (hence the need to Blog it)
I have not digested your comments, but would like you to give thought to the problem at hand - There is a delay in the CIG testing cycle due to environmental issues. This is not a once off situation, so how do you solve the problem. And then how do you solve the problem with mechanisation.
From: Van Der Merwe, Ben B [mailto:Ben.VanDerMerwe2@standardbank.co.za]
Sent: 03 October 2008 07:27 AM
To: Derrick Beling; Phale, Mafatshe M
Cc: Mckenzie, Craig C; du Plessis, Johann J; Johnson, Judy J; Perold, Louise L; salome
Subject: RE: CIG defect test cycle update
Derrick, I agree whole heartedly - and I am sorry if my response seemed to be a bit negative - it was intended to be on the realistic side (i.e. how do you do the required testing and mechanise at the same time under the current project constraints).
The other aspect is whether it makes sense to automate / mechanise while you are still in the exploratory / ad hoc testing phase. During this phase the application (i.e. CIG) still changes frequently and it is often not possible to complete a test case - either because there are serious bugs or there is a deployment / configuration issue. A formal process is not good under these circumstances (from there the 'ad hoc' approach). It is normally best to go to the developers so that the basic issues / bugs / configuration can be sorted out promptly.
One other aspect that worries me is the automation / mechanisation of the business process versus the automation / mechanisation of test execution. The two must not be confused.
The automater / mechanic (:) is in a good position to automate the business process (i.e TPP payments) and do some default checks (for example: have we been able to navigate successfully to the next screen, check for payment status, etc), but the automation of the test execution requires the involvement of the testing SME - in this case the test analyst.
It is even possible to regard the mechanisation of the business process and the mechanisation of test execution as two separate processes. Various combinations are possible. It is possible to automate the business processes and then do the actual testing of the results manually (using any tools that may speed up this process, i.e. comparison tools). It is also possible to do the business processes manually and just automate the test execution - I think this is where the challenge lies.
It may even be possible to automate both - but the automation of the test processes is the more challenging one.
Ben
From: Derrick Beling [mailto:dhb@mtom.co.za]
Sent: 02 October 2008 17:01 PM
To: Van Der Merwe, Ben B; Phale, Mafatshe M
Cc: Mckenzie, Craig C; du Plessis, Johann J; Johnson, Judy J; Perold, Louise L; salome
Subject: RE: CIG defect test cycle update
We actually need to Blog this
Your point on mindshift is important. However the mindshift to mechanisation / automation must happen at all levels. And hence the email. We are starting with the level at which we have control, that is with us - Testers, Test Analysts, Snr Test Analysts, Test Leads need to be thinking in terms of how mechanisation can solve problems. Which will lead us to what problems we want to solve with mechanisation.
Put it another way - if you are able to say Do 1,2,3 and you solve Problem X with a cost of Y and a benefit of Z - then you have a chance of getting the time and money. If you just say put mechanisation in the project plan, then it is not going to happen. More specifically if we say "Automate 10 scripts, run this check list and compare to these logs" and you will eliminate the problem that Judy has raised then it will happen.
The most important part is that this is a thinking exercise, not a doing exercise. Which means what - your email outlines challenges and has a problem statement "between a rock and a hard place". Thus the need for a solution cannot be disputed.
So, having expanded on the problem, we must :-
1. determine whether the problem is clearly stated and is complete
2. find a solution to the problem (here we can apply the principles of testing for bugs, for example - how else can we recreate the problem)
3. determine what of the solution can be mechanised / automated (in the true definition of mechanisation, the mechanisation itself could determine the solution - Load Testing is an example, you cannot load test if you are not mechanised).
4. Be future gazing - what we could do if....;not past gazing - what we can't do because...
From: Van Der Merwe, Ben B [mailto:Ben.VanDerMerwe2@standardbank.co.za]
Sent: 02 October 2008 01:33 PM
To: Derrick Beling; Phale, Mafatshe M
Cc: Mckenzie, Craig C; Johnson, Judy J; Perold, Louise L; salome; du Plessis, Johann J
Subject: RE: CIG defect test cycle update
Hi Derrick,
CIG phase 2 is new code and has resulted in a multitude of defects being logged. We are almost ready to move on from the ad hoc/ exploratory testing phase, which opens up the opportunity for mechanisation / automation.
From an end to end testing perspective we have a minimal amount of time left - barely enough to finish one test cycle. An automated solution will require the equivalent of at least two test cycles (duration) to accomplish. We are between a rock and a hard place. The new BVA screens are java based and can be automated (using the current workarounds that are in place). The defect test cycle mainly applies to defects on the these screens, but the objective is to progress to the end to end test phase which will reveal defects in the core CIG application.
The main outstanding issue is that we have not been able to execute one single end to end test case - for various reasons including an emergency freeze (firewall rules are not in place), late deployments / deployment issues, etc. An end to end test consists of a customer payment file, submitted from C:D Windows to CIG PPD which is routed to Business Online test environment. The expected result is that this payment file is delivered successfully to BOL (according to the CIG scenario that has been executed and a wild card based file naming convention) and various response messages which must be routed back successfully the the customer partner (C:D Windows).
The same set of tests must be repeated for a C:D Unix 'customer' installation - the current state is that we have acquired the Unix server!
The main issue for me is that automation planning is not reflected in the project plans. It is almost done as an after thought. We have to invest at least a month in the proper automation of this project, but we do not have enough time left to complete the actual testing that is required..
I think a mind shift is required in the project planning approach so that automation / mechanisation is an objective and not a side show
Ben.
From: Derrick Beling [mailto:dhb@mtom.co.za]
Sent: 02 October 2008 12:46 PM
To: Van Der Merwe, Ben B; Phale, Mafatshe M
Cc: Mckenzie, Craig C; Johnson, Judy J; Perold, Louise L; salome
Subject: FW: CIG defect test cycle update
Some more thinking to be done. Look at Judy's problem as set out here.
My question is -
How can automation / mechanisation solve, or assist in solving, the problem ?
(Include anyone else who has an interest in automation on this - I'm thinking of Simon etc)
From: "Johnson, Judy J" <Judy.Johnson@standardbank.co.za>
Sent: Fri, 26/9/2008 10:18
To: Derrick Beling <dhb@mtom.co.za>
Subject: FW: CIG defect test cycle update
Hello Derrick - as you can see from this mail we have more delays to our CIG testing cycle - this time it's environmental issues, but it's pretty much more of the same kind of impact to testing. ..... Judy
______________________________________________
From: Johnson, Judy J
Sent: 26 September 2008 09:55 AM
To: Fuchs, Karl K; Frazer, Shirley; Van Stade, Carike; Loubser, Juan J (CIB); de Vleeschauwer, Camiel C; Damjanovic, Jovanka J; Bright, Shannon S
Cc: Abdullah, Zainab Z; du Plessis, Johann J; Frazer, Shirley; Kasiram, Roshni R; Kgaphola, Doctor; Kruger, Hanna; Kwesaba, Lusanda L; Mase, Luvuyo L; Padayachee, Kiroshnee K; Phale, Mafatshe M; Phukubje, Ivy I; Seraseng, Simon S; Tshitimbi, Ntungufhadzeni N; Tshitimbi, Ntungufhadzeni N; Vadachia, Mahomed M; Van Der Merwe, Ben B; Zwane, Siphesihle S
Subject: CIG defect test cycle update
Importance: High
Greetings everyone
We were supposed to get started with our CIG defect cycle testing yesterday however, this did not happen as it was discovered that both the INT and UAT1 environments did not have the upgraded version of GIS. This resulted in Shannon having to spend at least six hours yesterday on sorting this out for both INT and UAT1 environments.
This morning after we were given the goahead to test we still were unable to continue due to the following issues:-
1. Bebstation link to thin BVA is pointing to UAT2 instead of UAT1 - this has been resolved
2. UAT1 Websphere configuration for MQ is incorrect - resulting in inability to Create a Customer Partner - we are waiting on Evert to resolve this problem
The plan to have all the critical and cignificant defects retested and closed by EOD today is looking less and less likely as we still do not have access to the application and also have a clean database and therefore will have to capture all relevant data before we are able to start any retesting of defects.
Regards, Judy
Judy Johnson
Micro to Mainframe (Pty) Ltd
for
Standard Bank Group Ltd.
Infrastructure and Testing
Tel: +27 11 631 1465
Fax: +27 11 636 4633
Cell: +27 (0)83 5156037
Email: judy.johnson@standardbank.co.za
___________________________________________________________________________________________________________________
