среда, 16 июня 2010 г.

TRT. 1st Meeting summary

We’ve finally had our first ‘Testers of the Round Table’ (TRT) meeting. Despite the fact that it got a little too formal at the beginning, we’ve managed to create a warm friendly atmosphere and had a pretty good meeting.

The following topics were raised:

  1. Introduction and memo
  2. The problem of testers’ education
  3. Bad management and inadequate customer: ‘survival’ strategies
  4. Microclimate within test team: its influence on testing efficiency
  5. Tester’s efficiency evaluation
  6. Job changing: how to make the right choice

1. Introduction and memo

Testers of the Round Table – is a community, created by testers and for testers. The goal of TRT is for testers to share ideas, discuss problems they face in everyday work, speak out about vital issues of testing, work out solutions for complex accidents.

Memo of TRT:

  • Everybody is equal in learning
  • Everybody has the right to have his own opinion
  • Everybody can choose the way of education he likes best
  • Everybody must be respected regardless whether he is right or wrong
  • Nobody can criticize and blame other viewpoints based upon assumptions
  • There is no ‘right’ and ‘wrong’ viewpoints – there are only different ones

2. The problem of testers’ education

This issue was discussed in the context of the countries of former USSR where there are no higher educational facilities that provide educational courses for testers.

I’ve emphasized the following ways the testers of the mentioned countries can acquire knowledge (each of those five points was discussed in pretty much detail):

  • Certifications as a way for testers to: a) reorganize and restructure the knowledge they have; b) gain more knowledge as they proceed with studying for certification exams
  • Getting knowledge from testing-related areas for better understanding of business domain they work with, using behavioral sciences to improve communication, applying various science principles and laws to improve testing practices
  • Forums and Communities as places where testers can find answers for some complex issues, and to share opinions on tools, practices, metrics, etc.
  • Blogging as a means of learning how to shape the thoughts into words and as a source for deeper understanding of the subject the tester writes about
  • Mentoring as a way for testers to a) share their knowledge with others who seek it at the moment; b) to learn about a certain subject from a more experienced person in live communication

After that part of conversation was over another speaker took the lead. Oxana continued on ‘Job changing: how to make the right choice’ and ‘Tester’s efficiency evaluation’.

3. Job changing: how to make the right choice

We’ve covered the first several steps (where to start from, CVs, job interviews) of the process rather quickly and focused on the key aspect of the topic: how to choose the ‘right’ company from the give alternatives. Here the opinions vary:

Oxana proposed a way to compare alternatives based on the formula

where:

Gi – parameters value (according to selected scale)

Wi – parameters weight (total of ‘1’)

I opposed that you’ll probably not consider those factors at all if you like the way your interview went, or just ‘have the right feeling’ about a company. Also there is a ‘super-factor’ (e.g. your best friend works in one of the companies that you consider as an alternative) which also can outweigh all others tied together.

4. Tester’s efficiency evaluation

The keynote of this topic was actually ‘how not to evaluate tester’s efficiency’ supported by the examples from participants’ previous jobs. E.g. one of the TRT participants has mentioned a report he had to send out at the end of each week to his manager (the QA Manager)

- Test Case coverage, % (I’m not saying that this metric isn’t any good, but in order for it to be useful tester must conduct coverage analysis on some basis, while in the case it’s almost a random number)

- Created new tests, #

- Updated existing tests, #

- Tests ran (Manual, Automated, Performance), #; #; #

- Resolved issues (in current iteration), #

- Issues created, #

- Issues processed (verified, reopened, etc.), #

As you can see, this report will better describe activities of a Quantity Assurance dept. than a Quality Assurance one.

When Oxana was done with her topics she passed the lead to Victor who raised the following two questions: ‘Bad management and inadequate customer: ‘survival’ strategies’ and ‘Microclimate within test team: its influence on testing efficiency’

5. Bad management and inadequate customer: ‘survival’ strategies

This is a situation that can happen to any specialist. So, what to do when the customer is ‘inadequate’ and your PM happens to do absolutely nothing for the team.

My advice is to turn to your managers – in the case of a tester those are the QA Manager (the person who runs the QA Dept.) and some sort of PM Manager (the person who curate project managers). In most cases these people are very adequate and intelligent, so you’ll be able to work out a decent ‘survival’ strategy together. If ‘not’, well… then you probably should start updating your CV at the moment you realize that it’s a ‘not’.

6. Microclimate within test team: its influence on testing efficiency

The keynote of the topic was being initiative, creative and pro-active. No matter what the microclimate is, a great tester is always in the mood to make a difference. You do not need to wait for someone to take actions toward achieving the goal – do it yourself. If your lead cannot handle the team – do it yourself. The tester must not be afraid of the ‘witch-hunt’ that might be initiated against him, but do his job the right way no matter what.

To summarize all written above, I must say that the meeting went well. We’ve met all the criteria that have been set up for the meeting. We’ve learned a lot and had a great time communicating.

P.S. I want to say thank you to the 1st meeting participant, so here I go: “Thank you guys, a lot!”

понедельник, 14 июня 2010 г.

QA Team Leader. Tips for team building

The effectiveness, efficiency and success of the test team very much depend on its leader. I am not saying that it’s the only key success factor, but you should not underestimate how much a good test team leader can contribute to the overall success. I’ve put together some points that a team leader should use in his work.


1. The importance of team spirit
Leader is a part of the team. Every team member must see that. Otherwise they will lose the desire to work efficiently and contribute to the project. Many good testers forget that despite the ‘lead’ addition to their job title they still remain testers.


2. Test team lead – is a tester first of all

Leader must conduct testing himself. It doesn’t mean that he has to forget about other ‘lead’ activities and grant all his time to testing. The amount of time that a team leader can give to testing will very much depend on the size of the team in inverse ratio.

3. Writing test cases, checklists, etc.
It’s not a good idea to leave writing test cases, test scenarios, checklists, etc. entirely to subordinates. In this case team members might interpret those activities unimportant and unworthy to spend the time on it, and that will reduce both the quantity and quality of test coverage. In addition to that writing tests is a good way to stay on track.

4. Learn together
All team members along with their lead must learn together. This concerns all aspects of learning. Besides that leader must avoid the tendency to dive into all new functionality first.


5. Everybody makes mistakes
Every person is fallible. It’s what we humans are good at. Leader must admit his mistakes and do not hide them. This will require a certain level of courage to do, but it will be rewarded.


6. No monopoly on ideas
This means that the leader shouldn’t be the only one whose ideas have a high value. A good team leader must encourage generation of ideas and out-of-the-box thinking of his team. Also it’s very important to engage team members in the decision-making (of course if the competencies of the latter allow to do so).

7. Build a trusted team
Leader must create a team he can rely on. He should be careful in the case, but not too careful. He should reward the desire and ability to solve complex tasks. The relationship within the team must give the feeling of importance to all team members, they must see that they contribute to the project’s success.

8. Input appreciation
Every team member is making some input into the project on a everyday basis. This small bits of input are no less important than the big ones and should be rewarded as well. I’m not saying that the leader must give a medal on each such occasion, but a simple mentioning during e.g. the daily scrum meeting (well, if you are doing scrum of course) will be enough. This makes the testers feel important and motivate them to work better. The leader must set up goal for his team and give rewards for achieving those goals.

9. Proactivity is welcomed
Leader should compliment his subordinates’ proactivity. IMO, practivity should lead to a wide range of authorities.


10. No knowledge ownership
Project knowledge must be available to all its members. Leader must avoid holding some chunks of knowledge for himself. Every team member must know the major part of the project (ideally if its 100%). Also, every tester on the team must know the TC base (the principle is the same).

суббота, 12 июня 2010 г.

Automating Flex with QTP

Not very long ago I faced the problem of recognizing Flex objects using HP QuickTest Professional. Even with Flex 3.0.0 Add-in being installed the objects couldn't be recognized.
But it we've managed to find the reasonable solution.
So, here are the steps:
1. Launch Flex Builder
2. Create a new Flex
3. Choose it in navigator
4. Select Project > Properties > Flex Compiler
5. Type the following into the "Additional compiler arguments" field:

-include-libraries "flex_builder_dir\sdks\3.0.0\frameworks\libs\automation.swc"
"flex_builder_dir\sdks\3.0.0\frameworks\libs\automation_agent.swc"
"flex_builder_dir\sdks\3.0.0\frameworks\libs\qtp.swc"
"flex_builder_dir\sdks\3.0.0\frameworks\libs\automation_dmv.swc"

( -include-libraries works directly with Flex Builder's directory (in Windows by default it is "C:\Program Files\Adobe\Flex Builder 3\")
6. Click OK to save changes and then once more click OK to close the project's properties
7. Compile the Flex application

Speaking of negative moments, I have to mention the major two:
1. You will have to build two versions of your application in parallel (with and without flex libs) - this is not an option for some projects
2. There is a "probe effect", although it is not too severe

P.S. Works only with IE7, Flex 3.0.0 Add-in is also required.

Also available in russian.

Mind Maps. Upgrading checklists

I think that every tester (even the rookie) knows such testing artifact as checklist. In bare outlines this artifact is a list of items that need to be checked in scope of certain functionality. In my current company it is generally used for ‘testing’ functionality that is not covered with tests and requires checking of a large amount of objects.


Some testers consider checklist to be good as is – in most cases it’s a rich text file with or without hierarchy that lists all items that are mandatory to be checked in scope of the functionality.

Others are always looking for an opportunity to improve some activities in their work processes. Using Mind Maps as an upgrade for checklist can be considered as such improvement. When using a mind map you represent the items to check in the form of diagram built around the main idea (which is your functionality to be tested) that shows the connections between objects (your check items).


By using a mind map you can make the checking process more transparent, vivid and descriptive. You can focus on a specific part of functionality by collapsing the map branches you are not working on at the moment. The diagram makes the checking progress/status obvious (e.g., using accent colors for: pass, fail, not ran, blocked, etc).


When it comes to choosing tools for mind mapping I advise the following two: 1. Mind Manager (imo, the best one) and 2. Free Mind (well, that one is free)


Also available in russian.

I've got the power... shell

While testing my projects I often have to do some amount of repetitive actions. Mostly those actions are simple, BUT they do require some time to get them done. Well, I got really annoyed by that and decided to automate those actions.

This is the moment where powershell has entered the house. The essence of my automation is to write a function, add it to powershell profiles on my local and remote machines and run it whenever I need to do the mentioned earlier repetitive actions.

Here is how I did it:

Launch powershell and create its profile by running the following command (if you do not have it already): new-item -type file -path $profile -force
Open the newly created profile in notepad using the following: notepad $profile
Write a function:
function DoSomething([string]$Param1 = "") {
if ($Param1 -eq "") {
Write-Host "You are an IDIOT!!!! You need to enter Param1 value!" -ForegroundColor Red
return
}
$A_OUTPUT = "D:\Destination\"
$B_INPUT = "D:\Files\Images\"
$C_FILE = "Info.txt"

$currentDate = Get-Date -Format yyyyMMdd
$destinationFolder = $A_OUTPUT + $currentDate

if ((Test-Path -path $destinationFolder) -ne $True)
{
New-Item $destinationFolder -type directory | Out-Null
}

$Destination = "$destinationFolder\$Param1"

if ((Test-Path -path $Destination) -ne $True)
{
New-Item $Destination -type directory | Out-Null
}

$files = ls $B_INPUT

$files | %{copy -path $_.FullName -Destination $Destination}
Write-Host "$($files.Count) files" -ForegroundColor Yellow -noNewLine
Write-Host " copied from $B_INPUT to $Destination"

New-Item ($Destination + "\" + $C_FILE) -itemType File -Force | Out-Null
Write-Host "$C_FILE" -ForegroundColor Yellow -noNewLine
Write-Host " is created"
Write-Host "Files $Param1 are reorganizes!" -ForegroundColor Green
}
Save the profile. Here you can bump into a security issue – your operating system can tell you: “you are not allowed to do that, young man”. In this case you should ask it politely:
1. Launch PowerShell as Administrator
2. Run the following command: Set-ExecutionPolicy unrestricted
3. Agree to whatever it’ll ask you :)
After this you are good to go.

Relaunch PowerShell.
Run your function with parameter Test_B13: DoSomething Test_B13
And your function copies files from D:\Files\Images to D:\Destination\\Test_B13 and creates an empty text file Info.txt for description.

You’ll get the following PowerShell response (take into account that the source location has 3 files in it):
3 files copied from D:\Files\Images to D:\Destination\\Test_B13
Info.txt is created
Files Test_B13 are reorganized!

By implementing this into my work process I saved about 50% of time on those routine tasks (or even more if working on remote servers). This time I use more effectively now.

P.S. You can take a quick PowerShell tour here

Also available in russian.

Make life brighter – color your Powershell

Many of those who prefer to use powershell in their job do not like its original colors. I am among them. Well, that’s not a problem at all – because you can personalize your powershell interface – for convenience, usability and just aesthetics.

I prefer to enable the coloring by writing a simple function within the powershell profile:

# Prompt

function prompt
{
$_locationStackDepthString = New-Object string ([char] '+'), (Get-Location -Stack).Count
$color = 'Cyan'
Write-Host '>> ' -nonewline -ForegroundColor $color
Write-Host $(Get-Date -Format T) -ForegroundColor 'DarkYellow' -NoNewLine
Write-Host " " $PWD.Path -ForegroundColor 'Green'
Write-Host ($_locationStackDepthString + '>') -nonewline -ForegroundColor $color
$host.UI.RawUI.WindowTitle = "POWERSHELL"
return " "
}

This is how I prefer to see my powershell window.

You can use any other combination of colors. If you decide to do so – you might want to start your way from here.

Also available in russian.

пятница, 11 июня 2010 г.

Great Testers. A viewpoint

The question of “What makes good tester a great tester?” is one of those that concern every decent tester. There is no single “correct” answer to this question, but I guess many testers do have an opinion regarding the subject. I do.

If someone asks me the question, I will mention the following points (maybe some others as well, but the following ones for sure):

- The ability and desire to learn (the great tester must be proactive and have a thirst for knowledge in order to succeed; he must learn new things every day and never stop in his quest for apprehending the art of testing)

- Strong communication skills (every tester must realize that he works with people; communication is a key to enjoying your time at work; communication within the team, communication with customer, mentoring - e.g. workshops, seminars – it’s all very important for becoming a true guru of testing)

- Understanding of QA and Testing theory (the deeper the understanding the better - good tester must be able to study and analyze the process and to introduce improvements)

- Knowledge of testing-related areas (development, management, marketing, sales, psychology - it helps to better understand the business ideas of projects to effectively direct the testing effort, to improve communication and understand the decision-making bias of other team members,…)

- Responsibility (the decent tester must be responsible for what he's doing and be ready to accept responsibility that exceeds his jurisdiction, the more complex the tasks are the more responsibility they require - the more business value the tester adds to the project)

- Attention to details (it this case I mean not only the attention to detail used in testing but the one that is used for the whole project – e.g. noticing the mood of a teammate to determine whether it’s the right moment to talk and resolve some issue)

- Organization skills (self-organization first of all, the great tester must be almost pedantic when it comes to organizing the aspects of his work)

- Stress-resistance (it’s no secret that the first one to blame in case of any kind of failure is the testers; in order to save nerves for some other occasions :) the tester must be resistant to stress)

- An open mind (out-of-the-box thinking, an excellent tester is not just using someone’s methods and theories, he tries to think of something more, he's reinventing testing in some of its aspects)
- The desire to share knowledge (no knowledge ownership regarding the project, the tester should be willing to share his knowledge and ideas with others - to make a difference)
- An enthusiasm for breaking things (that one by J. Whittaker, can listen to it here; a great tester always enjoys what he’s doing, he is passionate and inspired when it comes to testing; a great tester loves testing)

Screenshoting. Weapons of choice

Most testers on everyday basis create defect reports, or bug reports or just submit bugs – you can name it as you wish. I’ve already shared my opinion regarding the importance of writing a good bug report. There I mentioned attachments as one of the three most important parts of a report. So, this is the part where screen content capture tools come to the stage. Let’s talk about them.

The niche of screen content capture tools is really large nowadays (you can check for yourself e.g. here or here or anywhere on the Internet) – well, it’s not only testers who use this software.
I’ve tried some tools and made my decision. So here they are – my “weapons of choice”: 1. SnagIt and 2. Win7 Snipping Tool

1. SnagIt

SnagIt – is a rather powerful tool that allows not only to make a screenshot fast and easy but to also add tons of different stuff to it, like borders, text with shadows and other effects, arrows, stamps and such. This makes it really convenient to emphasize the target screenshot spots and create kind of explanation schemas. SnagIt has lots of preferences and settings, e.g. 10+ profiles for capturing different content types (region, full screen, scrolloing window, menu with time delay, video…) thus it is a great aid when you need to make a very precise and detailed screenshot. As I’ve already mentioned there is a possibility to record video, which “weights” almost nothing after you archive it, which is really nice.
Besides all that great stuff that I’ve named, SnagIt has pretty good exporting capabilities – e.g. exporting to Word, Excel, PowerPoint, IM and sending via email or FTP

To cut a long story short, I can say that it’s an ideal tool for creating precise and beautiful screenshots for your needs.

2. Snipping Tool

Snipping tool – is a built-in screen capture tool in Windows 7. It is a good example of minimal art in designing software to solve a particular task (in our case - screenshoting). Within Snipping Tool you will not find dozens of settings and tweaks, but you’ll see a set of options capable to create a good screenshot fast. There are 4 basic capturing profiles (free-form, rectangle, window and screen) which actually covers most of cases. Here we have only two tools – pen and highlighter and this is not enough in half of cases. There is no export, but you can still send the created file using email.
To sum it up, Snipping Tool – is a pretty nice tool for simple screenshoting.

To put it in a nutshell: for creation of “heavy” and complex screenshots – use SnagIt, and when you need to attach just a simple screenshot - Snipping Tool will do.

Also available in russian.

Writing defect reports

Most projects use some kind of issue tracking system. It might be something custom (e.g. a bug-tracker based on sharepoint), some free tool like Bugzilla, or a fully loaded commercial tool – e.g. JIRA. But no matter what tool you choose, the template used for bug reporting is pretty much the same. Of course it is very important to accompany your issue with a valid priority, severity, build # or version, assignee and such, but I would like to emphasize the following three fields: Summary, Description, Attachments. The information provided in these fields will directly impact the perception and understanding of the defect and its future. Providing decent info to these fields requires from tester some efforts and a certain level of knowledge of AUT.
So, let’s get to the subject.

1. Summary
The defect’s curriculum vitae, aka Summary – this is the first thing anyone will see in you defect report, thus very important. Lots of stuff depends on how good the summary is defined. It is a success if the developer got by no less than 80% the essence of the defect after he’s read the summary.
I advise you keep in mind the following things while writing your next defect summary:

1) denote the platform (e.g., test, branch, prod, etc.) if there is more than one

2) specify the component (e.g., аdmin, reports, upload, navigation, etc.) if the following part of summary may fit several components and can be interpreted ambiguously

3) point to browser, OS or any other specific part (e.g., fails in FF 3.6 on Mac) if the defect is reproduced under custom configuration
4) mention the reproduction frequency (e.g., occasionally or 4 of 10 times) if you cannot reproduce it each time
5) use the definitions and names equal to what’s used in your AUT (e.g., not “photos”, but “images” or not “scroll bar”, but “slider”). Do this even if you know that it’s not the correct name for the subject

2. Description
Steps presentment or the Description – is the most important part of the whole report. The future of the defect depends on these simple step essays. On one hand, if the ‘Steps to reproduce’ section is too short and laconic, there is a big possibility that the bug will be returned to tester with resolution ‘incomplete’, ‘rejected’ or something similar which is a waste of time. On the other hand, too many details will “frighten” the developer and he’ll leave that particular issue for later – when he has time for reading novels like that.

By saying that I can conclude that it is vital to choose an optimum-scale Description detail level. My personal choice is a rather detailed defect descriptions (a bit more detailed than what you’ve probably consider medium detail level) with grouping easy unambiguous steps into one
Remember, a good problem description makes a half of its solution.

3. Attachment
I consider it a testing decency to attach some sort of visual ‘proof’ to defects (of course it’s not all defects that require a visual aid, but most do). The most widespread kind of such ‘proof’ is a screenshot (my ‘weapons of choice’ are described here). Actually, having so many screen-content capture tools makes it no problem to attach a video as well. Some developers will really appreciate video attachments, because it can save their time spent on understanding the root cause.

Speaking about defects I must say that a good tester not only finds defects but also makes sure that the found defect gets fixed. And remember, if your bug has returned to you with resolution any other than ‘Resolved: Fixed’ – it’s your fault.

P.S. If you are not sure about how good you are coping with the three mentioned points – do not hesitate and ask those who works with your ‘essays’ on everyday basis – your dev team.

Also available in russian.