No one would launch a website without testing it first. You want to make sure it’s easy to navigate and free of technical errors. When you run a call for abstracts, papers, or speakers, you want that process to be as easy as possible for your submitters. Which means it’s equally important to leave time for adequate user testing of your abstract management system.
The benefits of user-testing your abstract management system
Taking time to thoroughly test your system may add time to the front-end of your project timeline. But it offers significant benefits:
- Reduces confusion and frustration for your submitters
- Results in fewer questions and technical support requests for you and your staff
- Ensures you receive correct and complete submissions
- Reduces the amount of data cleanup needed on the back end of your process
Remember: at least some of your users will navigate your system in a way you never anticipated!
How many people do we need for testing?
You should test your submission site each year, even if you’re using the same software and process. In this case, you or members of your immediate team are the only ones who need to perform testing.
If you are using a new abstract management platform or made changes to your process, you’ll want to recruit at least 3 more testers. But even if this isn’t possible, any testing you can perform is better than none!
Who should perform the user testing?
It’s important to remember that your submitters are a diverse group of individuals. Their tools, internet bandwidth, language preferences, and technical savvy will vary. Your group of testers should reflect the diversity of your submitters and reviewers.
If possible, create a group of testers that includes:
- A member of your organization who is not part of your team
- An industry member from outside your organization who is familiar with your conference
- An individual who has no connection to your industry (friends, neighbors, relatives)
Keep in mind, if someone who has never submitted an abstract can make their way through the process, you can feel confident everything will run smoothly when you open your call for papers site.
How much time should I dedicate to testing?
1-3 weeks is an ideal timeline for testing. This includes the initial testing, making changes, and final testing.
The length of time will depend on whether you are opening a new site or re-opening an existing one. With a new site, plan on closer to three weeks. With an existing site, you may only need a few days.
How do we test our abstract management system?
Set up a framework
Before starting the testing process, set up a structured framework that each tester can follow. This framework should reflect your desired workflow and your established rules and requirements for submitters and reviewers. It should list each specific action you want the tester to take, and what the desired outcome is supposed to be.
Follow your workflow
The most efficient way to test your call for papers site is to follow the order of your workflow, from initial submission to final acceptance.
Address both technical issues and usability.
When testing your abstract management site, there are two categories of issues to watch for: technical bugs and errors, and usability for submitters, reviewers, and staff.
Technical bugs and errors
This is where you and your planning team should focus your time. You are essentially trying to “break” the system. Does it give error messages in unexpected places? Do the rules you put into place hold up, or are you able to get around them? Some examples include:
- What happens when you skip required fields?
- What happens when you upload a different file type than indicated?
- Can you submit more submissions than allowed? Or start a new submission before the first one is completed?
- Can you use different character types than allowed?
- Can you put in data that doesn’t belong (for instance, text in a number-based field)?
- Can a reviewer still submit a review if they disclose a conflict?
- Can you build and run the reports you need?
- Are you able to double-book speakers or rooms in the scheduling tool?
Usability for submitters, reviewers, and staff
You and your team will be too close to the process to gauge the clarity of your instructions or forms. But your volunteer testers will help the most because they can provide a fresh perspective. Their input will tell you:
- If basic functions are intuitive
- Whether it’s easy to understand how to complete each field, and in what order
- That the information displayed is relevant to that submitter
- Whether visual or text cues are needed, and if so, are they helpful
- Whether you are going to get clean, consistent data from each submitter and reviewer
Download the Abstract Management User Testing Checklist
To help you organize and structure your testing process, we’ve created this Abstract Management User Testing Checklist that you and your testers can follow. Use the fillable PDF as-is to log the results of your testing, or use it as a starting point to build your own customized testing framework.
Share The Love!