In our previous tutorial, we stressed the importance of knowing exactly who it is that is giving you all of this valuable feedback. If you are using testers who have already completed some prior research, you already have this information and don't need to burden your beta testers by asking them to provide it again. When you send out the beta test e-mail invitation, simply merge a unique identifier into the URL (hint: use a query string) which will allow you to pass the ID into the dataset, and later, merge data from your earlier research with this latest wave.

If you are starting from scratch, the first thing you'll want to do is profile, and perhaps screen, these candidates to determine their appropriateness to the project. Give them a short survey which gathers some key metrics that may affect their relationship with your product. Such metrics might include:

• demographics (age, sex, income, etc.)
• webographics (connection speed, Web experience level, etc.)
• product relationship (usage levels of similar products, spending levels in the category, etc.)

The goal here is to get enough background info on participants to assess how important a particular type of feedback is to your business objectives. With information on respondent background handy, the task of sifting through feedback is greatly simplified when you know which comments come from core customers versus those users with minimal involvement in your category.

Let 'Em at It!

Once you've selected the people who will be hacking on your site, raise the curtains and let them explore your newest pride and joy. Things at this stage will be fairly routine, and you'll be doing the usual backend tweaking.

But here's the twist: While most beta tests provide simple mailto: links to give users the opportunity to provide feedback as they surf, we recommend employing forms and a .cgi. Using a mini survey will allow you to quickly categorize the feedback being provided, route it for follow-up and, most importantly, give you an easy way to understand the type of user it came from.

Do this by augmenting your feedback form with some key metrics (typically gleaned from your profiling survey), which describe a user's relationship to your product. For an investing site, for example, you might ask:

• How often do you trade stocks?
• How often do you visit investment-related websites?

These two questions allow you to quickly categorize the feedback as coming from a die-hard, a middle-weight or a dabbler.

An easy way of keeping the feedback mechanism in plain sight (and encourage usage) is to throw the whole thing into a frame set, with the feedback form in one constantly viewable frame, and the rest of the site in the other. If you're worried about throwing off the look or usability of your site because of this temporary design change, you can always put a link to the feedback form in the header or footer of your page. Clicking on the feedback link can launch a new JavaScript window which contains the form.

If you aren't quite ready to dive in and generate your own survey-conducting code, there are a lot of resources out there that can help you get started. Products like InsightExpress, Market Tool's Zoomerang, and Catapult Systems' Inquisite allow users to conduct online surveys with no technical knowledge whatsoever.

Reality Check

Even though collecting feedback through a structured Web form instead of a free-form e-mail greatly increases the quality of user responses, the data gathered should still be considered qualitative and subject to bias. Ever heard the term "silent majority?" The most outspoken or involved members of the beta testing group are likely to dominate the information flow, while less-involved or subdued visitors quietly go about their business using the site. So keep in mind that the views of loudmouths are not necessarily representative of the entire beta testing population.

In addition to the bug fixes, the best use for information at this stage is to identify issues to be quantified in the next phase of the beta testing program: the follow-up survey.