Hear feedback from Real Testers about how to maintain a high rating and take great tests.
Top Tips from a Highly Rated Test Participant Wfree
Tips from Top Tester Lisbeth!
Meet Tester Kevin!
Top Tips from a Highly Rated Test Participant Wfree
Test participant Wfree consistently gets good ratings so we decided to ask her what advice she might have for others seeking to do well on their tests.
Tell us a little bit about you. What is your occupation? What are your hobbies?
I live in Seattle Washington. I’m currently a freelance product manager. I used to be a product manager in multiple IT companies. My hobbies? I like reading, traveling, hiking, things like that.
What brought your attention to UserTesting?
I was browsing online last year and somehow it caught my eye. I’m quite familiar with user testing (usability testing). I think this is a great thing to do in my spare time.
What is the most important part of testing?
10 years ago, because I used to work in an enterprise software company, people didn’t even care what users think. The engineers would kind of dominate in deciding how to build the features and the products and now it’s great to see that so many companies think user testing is very important and they try to validate the design and their assumption of user requests beforehand.
Lastly, what are your top 3 testing tips as a top-rated tester?
1) The first one is to be honest. Because I think what the product team wants to hear from the testers is their honest feedback about their products. Especially if there is something wrong. And it’s great to just tell them what you don’t feel is good, or something you really feel is very good. I think both sides are very valuable for them. Honest feedback.
2) Try to explain your opinions as detailed as possible. Because I think most of the tests on UserTesting.com are unmoderated tests. If you are doing a moderated test, both sides can have communications and interactions. For example, if I say something the moderator doesn’t quite get, then they can ask me to rephrase it or to confirm it to get the right feedback. A lot of unmoderated testing is just based on the tester’s speech or writing; if they (the customer) have more questions they cannot follow up. It’ll be great if the tester can explain themselves really clearly and that will help the other side to understand you better.
3) The third one is, try to be outspoken. When I open a webpage I will try to speak as much as I can about what I observe on the website. If you speak too little you won’t be able to provide much valuable feedback. It will also help them to get a better sense of you as a tester or user.
Tips from Top Tester Lisbeth!
Our testers often write in to express concern over their Recent Average Ratings.
Hope you guys are all doing well there in Support! My UserTesting.com username is BenjiRu. I’m writing because I have question about my ratings. I’m at 3.5 stars. I thought I was doing okay, but now I’m wondering how can I boost up my rating??
If only there were a formula for the five-star performance! Alas, we have not found one. In lieu of such a perfect recipe, we can offer two things: (1) general advice that works well in nearly all situations, and (2) specific inside tips that some of our five-star testers have shared with us. This series will focus on the second option, showcasing some of our best testers as they discuss the keys to their craft.
This sort of inquiry is all too common here in Support. After all, it’s no wonder why BenjiRu is concerned about his Recent Average Rating: a tester’s star rating is what determines the amount of tests that are distributed to her or his Available Tests column. The higher your Recent Average Rating, the higher your likelihood of snatching up more testing opportunities.
We start with Lisbeth.
Lisbeth works as a Business Analyst and Consultant in Illinois. She also freelances as an editor and writer for various publications (both online and in print). I started by asking Lisbeth how long she’s been a tester with UserTesting.com.
1. How long have you been testing with us, and what brought you to us?
Lisbeth: [Off and on for] over four years. A colleague was looking into different testing platforms for a site we were working on and came across User Testing. Intrigued, I decided to sign up as a tester to get a better feel from the other side, and liked it so much I stayed. :)
2. What do you like best about being a tester with UserTesting.com?
Lisbeth: It benefits everyone. I’ve been on both sides of development, and when you’re in the thick of creating a site it’s easy to lose perspective. Things that seem blatantly obvious to the user are invisible to the project team simply because you’re too close to really gauge usability. From a user perspective, it’s a way to make a little extra money that also ultimately helps people to build better sites.
3. How important to you is your Recent Average Rating?
Lisbeth: It’s the variable by which I define my own excellence, and the marker which assures me whether I’m on track o need to step up my game. What I especially like is that it’s now based on the last 12 tests--something that keeps all testers striving towards perfection.
4. Do you accept any and all tests for which you qualify, or are you choosy when it comes to selecting tests to take?
Lisbeth: I’m moderately choosy. If I don’t feel I am well-equipped for whatever reason, I’ll pass. I’ve also quit tests which have over 20 questions--I personally feel this requires too much rushing through the test, which would deliver a less than optimal performance.
5. What’s the first thing you do when you launch the screen recorder to start recording?
Lisbeth: Introduce myself and read the scenario [by clicking "Previous" on the task bar] to ensure I’m on the same page as the client. I then read the task--and each task--out loud.
6. How do you work through the test tasks?
Lisbeth: I read through the task in its entirety, and keep each question in mind as I'm performing the test. If a question is ambiguous, I'll explain that and why, and then answer it to the best of my ability. If there are multiple questions in a task, I'll review the questions aloud to ensure I've answered them all before I move to the next task, explaining what I'm doing as I go.
I do tend to double-check everything as it's easy to miss something that may be integral to the client's insight..
7. How do you handle tasks that don’t make sense to you? What strategies do you employ when you’re not sure what the client is getting at?
Lisbeth: I'll explain first why the question is confusing or ambiguous - this just to ensure that the client knows what the communication gap is, so that they can amend it potentially for future tests - and answer it to the best of my ability.
If a question really doesn't make sense, I'll either pause the test and write support for clarification, or abandon it and report it under "other problems" so that User Testing support is at least aware of the problem, again in hopes of notifying the client for future tests.
8. Sometimes it’s easy to get quiet during a test. How do you remind yourself to keep speaking up? What are some good habits for remembering to always express your thoughts?
Lisbeth: Fortunately, I'm fairly verbose in real life, and love to explain things so thinking "out loud" is a natural inclination. I use that methodology in the same way - showing exactly what I'm doing, demonstrating what I'm looking for and why, and then mapping the intuitive process - where I feel like I "should" go, or why something doesn't work.
I never worry about appearing stupid or that I "should" know how to do something, and approach each test as I would in real life. I'm also not afraid to admit when something confuses me or appears counterintuitive - many times, programmers work with a different mindset than the customer, so subjective perspective can be invaluable in the testing process.
10. Have you noticed any changes to a website you’ve tested based on your feedback?
Lisbeth. Tons! This is the most rewarding aspect of being a tester - many clients write me to let me know that what I found produced results and how they changed their website because of it. I've often been invited to review the changes and provide additional feedback as well (which I'm happy to do).
11. Lastly, please feel free to tell us anything else that you think is pertinent to your success as a tester with us!
Lisbeth: Approaching each test as if it were the first, and genuinely wanting to help someone make things better are pivotal to excellence in testing.
III. Lisbeth’s A-B-C’s
Lisbeth shares some great advice in her answers to the questions above. Here are the Lisbeth’s main take-aways--the most important A-B-C’s that every tester should keep in mind before a test.
A. Do not necessarily accept every test. If you do not feel that you fall within the intended scope of the test, it is better for your star-rating to walk away and only give feedback when you are certain that it will be most valuable for the client.
B. Start each test by re-reading the scenario to ensure you and the client are on the same page.
C. Show, not tell! Demonstrate every aspect of what you are trying to express to the client.
Stay tuned for more profiles from our highest-rated testers!
We think that these three points can benefit every tester, no matter their star-rating.
Back to Top
Meet Tester Kevin!
As always, in our Support Center we hope to provide a wealth of insight for you so you can increase your ratings (and make clients happy!). To help in this goal, we've recruited one of our top rated testers, Kevin. We asked Kevin questions about his success as a tester, and why he thinks clients give him high ratings. There's no perfect method for being a tester, but this is about as close as it gets!
1. Please give your name, location, and a little bit about
you. What is your occupation? What are your hobbies?
I grew up and have spent most of my life based in the Midwest of the United States and one of the benefits of UserTesting for me is working with client sites from all over the world. My primary occupation is doing consulting work for companies of all sizes across multiple industries (financial - banking/insurance, government, retail, non-profit, etc). In my spare time, I enjoy spending time with family and just about any outdoor activity such as running, biking, hiking, etc.
2. What brought your attention to UserTesting.com?
I was seeking interesting, challenging work to provide some extra income and needed the flexibility that freelance work provides. During my research for this type of work, I came across the UserTesting.com site which seemed to meet my needs. In addition, unlike some freelance work, UserTesting also seemed to be a trustworthy, credible, reliable company with excellent references and excellent client credentials.
3. What do you like most about being a tester with
For the most part, UserTesting has delivered on my needs - over the years I have had interesting and challenging work along with the good feeling that comes with having been able to help some of the largest best known brands like Google, Amazon, and HP as well as small companies looking to startup.
4. What do you do to ensure your rating stays high? What
tips do you have for increasing your rating?
a) Ensure you meet the requirements for the test and fully understand the scenario. While this seems simple, it's easy to read too quickly and miss a key point once you begin recording. I have made it a habit to read the requirements and scenario at least twice - once before accepting the test, and once after starting the recording.
b) Take a few minutes if needed to finish up any tasks you were working before accepting the test. Since usertests usually interrupt other work of mine, before completing a test, I pause to finish what I was doing so I can sit down with a clear mind, without distractions, and give the client/test 100% of my time during the recording.
c) Try to imagine yourself as an employee of the client company or a member of the client organization during the 15-20 minutes of the test. In addition to my role as a user of the site, I believe this is important because as an "employee" of that company I want to determine how I can help them be even more successful.
d) Talk about what you are thinking as much as possible and focus on clear enunciation. By doing this, rather than just seeing the actions that I performed, clients can understand why I performed them and can hopefully understand me with a minimal amount of mumbling, um, ah, etc.
e) Ensure you read/understand each task and have completed it before moving on to the next task. Similar to the requirements/scenario, it's easy to read too quickly and miss a key point which could result in a lower rating for your test. Also ensure you stick to the task without working too far ahead or doing clicks outside the scope of the test.
f) Ensure you are constructive and polite. Clients already know there may be issues and that is why they asked for our help. Rather than, for example, saying a web site or web page is terrible, instead explain in specific detail where it might need improvement - for example, "I'm unsure of the purpose of this site because the home page is lacking an about link and is lacking any sort of statement about mission/vision/purpose."
5. Do you accept any and all tests that you qualify for, or
are you choosy when it comes to selecting tests to take? If you’re choosy, what
makes you decide to take that particular test?
Assuming I will be available, I accept most tests where I am qualified. The primary area where I tend to be choosy is with tests requiring a download/executable in order to minimize the risk (albeit, a small risk) of having a client file corrupt my machine. I have also avoided a couple tests where the client seem to have missed or ignored the UserTesting time limit/# tasks guidelines without offering any bonus (for example, one test had 48 tasks).
6. What’s something you feel like you improved upon since
starting out as a tester?
As with most work, I think I have become more efficient over time. My initial tests probably took a bit longer as I was learning the session recorder and the routine provided by the test tasks but now I know what to expect. Also, most of the items noted above under tips in question 4 were things I had to make a habit of doing.
7. Please describe as best you can the way you work through
the tasks. (For example: Do you take each sub-task individually? Do you
double-check to see that you’ve completed everything in each Task?)
I read the whole task before starting it because sometimes, clients will have the sentences out of order and/or improperly worded, for example one task might be, "Please complete the checkout process. Use fictional information for the fields. Apply the discount code ABC123. Before you click submit, tell us what you expect to see on the next page”. If I failed to first read the whole task, I would have done the first sentence and missed key information.
8. How do you handle tasks that don’t make sense to you? What strategies do you employ when you’re not sure what the client is getting at?
This does happen on a small percentage of my tests - usually it's because the client forgot to proof read before posting the test - they are human and make mistakes. Sometimes it's a large issue like an incorrect user id/password where I will just cancel the test. Other times it's a small issue where just one task fails to make sense - usually I will read the task out loud once or twice
so that the client can hear the issue and if possible will try to interpret what the client may have wanted as best as I can and then move on to the next task. For example, I had a client where the first three tasks worked in Firefox but the 4th task failed so I switched to Internet Explorer and was able to proceed. The client was appreciative that I had found the issue and also appreciated that I did not just give up.
9. Sometimes it’s easy to get quiet during a test. How do you remind yourself to keep speaking up? What are some good habits for remembering always to express your thoughts?
For me, this was initially an issue I had to overcome but I could really see the difference in my ratings between tests where I spoke less versus tests where I spoke more so the ratings were a good motivation for me.
10. What’s the most important thing needed to be a great tester?
Remember that the testers are the "face" and/or "front door" to UserTesting. In other words, the client impression of UserTesting will be primarily driven by the work delivered by the testers. At the end of my tests, I thank the clients for the opportunity to help. I think I first noticed this with airline advertising (i.e. - we know you have choice when you fly, thanks for picking us). Similar to airlines, there are other choices for clients to test their sites and instead they chose us, the testers at UserTesting. The work we perform is often the most visible impression that clients will get from UserTesting. If they have a good experience, they may return with more work, and they may recommend UserTesting to others which means more work/jobs for all testers.
Back to Top