It is often difficult to keep the study focused on a few important variables and hypotheses. New and interesting possibilities frequently appear. Some researchers may use a shotgun approach, collecting everything that might be useful. However, the more variables that you have, the more difficult the research process becomes. It is also more likely that you will confuse major and minor variables. More is not better unless you are planning on a series of research initiatives based on one larger pool of data with different variables being used at different times. Being selective in what we collect and what we ask is one of the most difficult of all research tasks. Be vigilant and remind yourself regularly just what your study is about and what is the heart of the matter.
Your population and a sampling frame must be clearly identified and both must make sense. Each variable should be clearly identified and their values should be operationally defined. A code book should be available to guide data collection.
You must be able to identify and characterize your research methods. Once you know the proper names, you should do a reasonable literature review on the methods that you will use to learn more about their assets and liabilities. Resources may include research methods textbooks, articles on the method, and articles on various research topics that have used the method.
You are responsible for being familiar with the major strengths and weaknesses of your methods. You may have to defend them with an editor or those who review your work.
The data collection process must be full revealed so you will need to think through the process on a step-by-step basis. It's a good idea to list each separate step in the data collection process as if you were going to flow chart it. Review your list to insure that it is as logical and complete as possible. Test your list by applying it to several representative cases.
Identify and list snags or problems encountered or likely to be encountered. For each obstacle identified, find one or more ways to compensate or overcome the problem. There are many sources of error in data collection. Here are a few:
Here is an example of a data collection problem that was overcome. A questionnaire with a low response rate on a pilot test was redesigned to include an endorsement cover letter, reduced length, and a better benefit/rationale statement.
If problems cannot be overcome, it may be necessary to change the research design or delete a troublesome variable.
You are responsible for convincing the reader that your data collection method was appropriate and free from error.
It is important to record problems encountered and steps taken in your research diary so that you can discover what you did and why later. This is especially important if you change or amend an operational definition and wish to be consistent in the future. Reliability is of particular concern with data collection, especially if others are involved. Check your coding sheets of inappropriate values such as the fourteen year old widows found in an old census document. Reliability testing with another is an important and needed step.
