The proper APA Style reference for this manuscript is:
REICHARD, R.J., (1999). The Effects of Computers in Surveys. National Undergraduate Research Clearinghouse, 2. Available online at http://www.webclearinghouse.net/volume/. Retrieved September 26, 2023
. |
The Effects of Computers in Surveys
REBECCA J. REICHARD
Missouri Western State University PSYCHOLOGY
Sponsored by: Brian Cronk (cronk@missouriwestern.edu)
|
| The purpose of this study is to record the effects of the use of computers in surveys. Computers are being used more and more in today`s technological society. One example of increased computer usage is the transformation of the Graduate Records Exam (GRE) from a paper test to a computerized test. The study consists of a survey containing a variety of questions which will be administered to 73 Missouri Western State College (MWSC) general psychology students. The questions on the survey include basic demographic questions, sample GRE questions, the Rosenburg self-esteem survey, and personal questions coming from the MWSC Unity Office. The survey will be administered via three different media. One group of subjects will complete the survey through each of the following media: Internet survey, paper-and-pencil survey, and oral group survey. The participants in the three different conditions did not differ significantly in the number of GRE answers reported correctly. Analysis revealed that participants in the oral condition reported significantly higher self-esteem than the participants in the computer condition. Participants` self-esteem in the written condition were not significantly different from either of the other two groups. Two of the personal questions differed between the groups. This research is important because of the expansive trend toward the use of computers in recent years. |
INTRODUCTIONHave you used a computer today? For many people using a computer is a major part of their daily lives. Computers are continuing to become more prevalent everyday. Many college professors require students to get class assignments or notes from the Internet. Ertmer (1994) notes computer experience has gained wide recognition as a critical component of the education process. Not only does increased computer usage exist in education, but it is also indispensable in business. Companies use electronic mail, better known as e-mail, for immediate and direct communication all over the world. Kraemer (1981) contends e-mail provides the instantaneous information exchange allowing people to work in spite of geographic dispersion, schedules, time zones, access to secretaries, and energy costs. One example of the effects of the computer age is the transformation of the Graduate Records Exam (GRE) from a paper test to a computerized test. Interviewing and training in business are also moving to a computer-based system. The computer age has reached our social lives as well. People spend hours surfing the Internet and talking in chat rooms. There are even on-line dating services. Kling (1980) predicted computers will possibly have social repercussions as far-reaching as the automobile and telephone. Wherever one looks, whether in education, business, or pleasure; computers are a dominate part of life. Computer technology brings many benefits to our lives. Messages can be sent instantaneously at the click of a button to a countless number of employees (Kraemer, 1981). For a person to write and mail the same amount of letters, the time and money expended would far exceed the amount required for computer communication. The time it takes to drive to the post office alone is greater than that expended using e-mail. Not only does e-mail save time for the initial sender, but the respondent can reply within minutes of receiving a message. Whereas, using the postal system it takes days just to receive a letter. Another advantage of computer technology is the enormous degree of information on the Internet. To do the same amount of research without a computer, a person would not only need library access; but he or she would also have to manually search for the desired material. If the library does not have a particular reference, the person must send off for inter-library loans and wait a week to even receive the needed information. Whereas on the Internet, information is available with the click of the mouse. Kiesler (1984) sums it up by explaining that electronic communication differs from any other communication in speed, time, space, ease of use, fun, audience, and opportunity for feedback (Kiesler, 1984). The undeniable advantages of computers over other communication passageways are far-reaching.Although the speed and breadth of computer communication is evident, some pitfalls exist. Computer communication relies exclusively on text. In traditional forms of communication; head nods, smiles, eye contact, distance, tone of voice, and other nonverbal behavior give the communicators information they can use to regulate, modify, and control exchanges (Kiesler, 1984). Certain inferments must be made when using computer-mediated text-based communication. The computer is often considered to be one of the most socially distancing and impersonal modes of communication (Matheson & Zanna, 1988). This causes a practice that Kiesler (1984) refers to as flaming which is when a person expresses oneself more strongly on the computer than one would in other communication settings. Another pitfall exists in the extensiveness of the Internet. Although the Internet is a great communication and reference tool, it may be too vast. With the abundance of available sites, it becomes hard to find exactly what is needed. These disadvantages must be taken into account when examining the use of computers. Regardless of these pitfalls, computers are a vital communication source in today`s society. How does this new communication source effect psychology? Kiesler (1984) suggests the possibility, through experimental research, of using electronic communication for surveys, questionnaires, and interactive polling. The focus of this study is the use of computer surveys. Computer networks add value to the electronic survey because these systems locate participants automatically, deliver survey instruments to remote locations, and permit participants to answer questions at their own convenience (Kiesler, 1986). Even if the only contributions of electronic surveys are to reduce research costs as compared to the costs of interviewing, telephoning, and sending questionnaires through the mail; the electronic survey may become widespread (Kiesler, 1986). Another advantage of the electronic survey is the elimination of the need for a person to stand between participants and the computer that stores and analyzes the data (Kiesler, 1986). This can eliminate the human error that may occur during data entry. Another problem with mail surveys is the low response rate. Kiesler and Sproull`s (1986) results suggested that the electronic survey can elicit good response rates with a faster turnaround time and fewer item incompletions than a regular mail survey. Boothe-Kewley, Edwards and Rosenfeld (1992) concluded that participants who completed a computer questionnaire found it to be more interesting, more important, and felt more aware of their thoughts and feelings than participants using a paper-and-pencil questionnaire. Elimination of human error, a higher and faster response rate, and a more enjoyable experience are just a few of the advantages of computer surveys. Psychology can gain from computer technology through using electronic surveys. One factor affecting the ability to establish the use of computer surveys is the participant`s knowledge of computers. Ertmer (1994) contends in order to embrace computer technologies, such as word processing tools, e-mail systems, and information data bases, individuals must feel confident and comfortable using them. On the other hand, Martin and Nagao (1989) concluded that the degree of hands-on computer experience did not have a significant effect on participants` socially desirable responses, expressed comfort levels, or expressed resentment in the computerized interview condition. Nevertheless, participant`s computer knowledge is definitely a factor that must be taken in consideration in the use of electronic surveys. Another possible benefit of electronic surveys other than the previously mentioned points is a reduction in socially desirable responses, or SDR. SDR occurs when the participant responds in a way that makes him or her look better. Computer surveys may solve this data collection problem. An individual’s response to very personal and embarrassing material may be less truthful to another person but more free to an impartial machine (Lautenschlager & Flaherty, 1990). By taking the focus of communication off the self and the social contest, and moving it toward the communication task itself, computers may help eliminate socially desirable responses (Matheson and Zanna, 1988). Several studies have shown the effect of computers lowering SDR. For example, Kiesler and Sproull (1986) found that participants who answered electronically gave less socially desirable responses than those answering on paper. Another study by Martin and Nagao (1989) found that participants in both computer and paper-and-pencil conditions scored lower on a measure of socially desirable responding and reported their grade point averages and scholastic aptitude scores more accurately than those in the face-to-face condition. They suggested this was due to the pressure inherent in the face-to-face condition influencing the participant to stretch the truth in an effort to make a good impression (Martin & Nagao, 1989). This pressure and other nonverbal cues would be nonexistent in both the computer and the paper-and-pencil conditions. Though both paper-and-pencil and computer conditions served to reduce applicants socially desirable responding, the computer condition generally did so to a greater degree (Martin & Nagao, 1989). It was suggested that the increased accurate responses were due to the “big brother effect.” The participants believed their answers were subject to instant checking and verification through other computer databases (Martin & Nagao, 1989). Tourangeau and Smith`s (1996) study provided further evidence that computer-assisted self-administration increases participants` willingness to make potentially embarrassing admissions in surveys. They suggested that the participant interaction directly with the computer helped convince participants of the legitimacy and scientific value of the study (Tourangeau & Smith, 1996). If the idea of decreasing socially desirable responses on computer surveys can be verified through future research it could be very beneficial for psychologists seeking truthful responses to use computer surveys. The "Big Brother Syndrome" is the growing and pervasive fear of computers monitoring and controlling people`s lives. Individuals are becoming more aware that computer communications can be monitored. Government agencies and private organizations now routinely monitor employee communication on both local and global e-mail systems. Thus, while computers previously may have created an impersonal, anonymous social situation, the emerging Big Brother Syndrome of the 1990s reflects an unpleasant, stressful atmosphere that may raise impression management concerns (Rosenfeld, Booth-Kewely, Edwards & Thomas, 1996). As mentioned, some studies have shown paper-and-pencil surveys produce the same results as computer surveys. Boothe-Kewley, Edwards and Rosenfeld (1992) found that computer and paper-and-pencil modes of administration yield similar results. The available data suggests a computerized condition may be functionally equivalent to a paper-and-pencil condition with respect to the way that participants respond to the questions (Martin & Nagao, 1989). When computer-administered questionnaires yield similar scale means, standard deviations, and reliabilities, an argument for the use of computers is that they are typically regarded more favorably by respondents (Boothe-Kewley, Edwards & Rosenfeld, 1992). If this equality is proven, computer surveys can legitimize wide-spread use in place of paper-and-pencil surveys. The utility of the electronic survey will depend on its comparability to other methods of survey administration (Kiesler, 1986). Given that organizations will probably increase their reliance on computers for administering self-report instruments, there is a clear need to determine whether responses on computerized attitude and personality instruments are systematically different from those obtained with paper-and-pencil (Boothe-Kewley, Edwards & Rosenfeld, 1992). Researchers should attempt to identify the boundary and contextual conditions that produce differences in computer versus paper-and-pencil responses (Boothe-Kewley, Edwards & Rosenfeld, 1992). The purpose of this study is to compare the effects of using computers to collect data in surveys with the more traditional survey methods such as oral and paper-and-pencil methods and to examine the differences among four different types of information: demographic, self-esteem, scholastic, and personal. It is proposed the computer condition will produce less social desirable responding than both oral and paper-and-pencil conditions
METHODParticipantsThe participants were 73 general psychology students from Missouri Western State College in St. Joseph, Missouri. The gender composition was 16 male and 57 female. The racial composition was one Asian American, three African American, 66 Caucasian, and three Other. All participants received extra credit for their participation. The specific demographic information for the participants was acquired by the first three questions on the survey. ApparatusThe study consisted of a survey containing a variety of multiple-choice questions. The fifty questions on the survey include basic demographic questions, sample GRE questions, the Rosenburg (1989) self-esteem survey , and personal questions coming from the MWSC Unity Office. The sample GRE questions consisted of math, synonyms, etc. with five multiple-choice answers available. An example of the Rosenburg (1989) self-esteem survey follows: I feel that I have a number of good qualities. The participants replied to a possibility of four choices from strongly agree, agree, disagree, or strongly disagree. An example of a personal question used to determine social desirable responses follows: Have you or someone in your family ever battled with substance abuse? The participants responded yes or no to these questions. Three different types of media were used in this study. Each participant was administered the survey either in a computer, web-based condition, paper-and-pencil condition, or oral condition. ProcedureThe participants volunteered from general psychology classes without knowing which condition they would be assigned. Upon arrival, the participants signed up on a separate sheet of paper to receive credit and maintain anonymity. The researcher gave the instructions orally in all three conditions. The order of the questions remained the same for all three conditions. In the computer condition, the students turned on the computer, began Netscape Navigator 3, and typed in the address of the survey given by the researcher. The participants were instructed to point and click using the mouse to select the appropriate answer and to scroll down the page for further questions. After completing the fifty questions, the participants clicked on the submit icon at the bottom of the page and quietly left the room. In the paper-and-pencil condition, the researcher passed out the written survey, an answer sheet and a pencil to each participant. The participants were instructed to neither write on the survey nor write their names on the answer sheet. Answers were recorded by the participant on the answer sheet. After completing the survey, the participants returned the survey, answer sheet, and pencil to the researcher and left the room. In the oral condition, each participant was told to raise his or her hand corresponding to the accurate response. The participants were told the researcher would be recording the number of participants replying to each response. A camera was hidden in the room to record individual responses. The researcher read each question and response on the survey two to three times. On the final reading, each participant raised his or her hand according to the appropriate response, and the researcher recorded the number of hands per response. The participants were finished with the study at this time.
RESULTSA one-way ANOVA was conducted comparing the GRE scores of participants in the computer condition, written condition, and the oral condition. No significant difference was found (F(2,70)=1.306, p=.277). The participants in the three different conditions did not differ significantly in the number of GRE answers reported correctly (computer: M=4.5143, sd=1.5024; written: M=4.8947, sd=1.7918; oral: M=5.2632, sd=1.7589).A one-way ANOVA was conducted comparing the self-esteem scores of participants in the computer condition, written condition, and the oral condition. A significant difference was found between the conditions (F(2,69)=4.393, p=.016). Tukey`s HSD was used to determine the nature of the differences between the conditions. This analysis revealed that participants in the oral condition reported significantly higher self-esteem (M=35.8947, sd=2.2335) than the participants in the computer condition (M=32.0882, sd=4.7378). Participants in the written condition (M=34.3684, sd=6.0019) were not significantly different from either of the other two groups. A one-way ANOVA was conducted comparing personal question scores of participants in the computer condition, written condition, and the oral condition. The personal question that read "Have you ever discriminated against anyone?" had possible answers being yes and no. A significant difference was found between the conditions (F(2,72)=11.946, p=.000). Tukey`s HSD was used to determine the nature of the differences between the conditions. This analysis revealed the participants in the oral condition answered yes significantly more (100%) than the participants in the computer condition (40%) and significantly more than the participants in the written condition (58%).A one-way ANOVA was conducted comparing personal question scores of participants in the computer condition, written condition, and the oral condition. The personal question that read "Do you have a close friend of a sexual orientation other than your own?" had possible answers being yes and no. A significant difference was found between the conditions (F(2,51)=6.800, p=.002). Tukey`s HSD was used to determine the nature of the differences between the conditions. This analysis revealed the participants in the computer condition answered yes significantly more (100%) than the participants in the oral condition (53%) and significantly more than the participants in the written condition (53%).A one-way ANOVA was conducted comparing personal question scores of participants in the computer condition, written condition, and the oral condition. No other personal questions demonstrated a significant difference.
DISCUSSIONThe purpose of this study is to compare the effects of using computers to collect data in surveys with the more traditional survey methods such as oral and paper-and pencil methods. Results are expected to show whether the difference in media produces a difference in responses with a special focus on the questions of personal nature. Lautenschlager and Flaherty (1990) hypothesized that computer administration of non-cognitive instruments containing sensitive and personal items may reduce socially desirable responding by offering greater anonymity and by appearing impersonal and nonjudgmental.Perhaps computer surveys will decrease the "big brother effect" and result in truthful and accurate responses by subjects to personal questions. If so, the computer survey will be an important tool in collecting personal data. This research has important implications for the expansive trend toward the use of computers in recent years.No difference was found between the three sources of media on the GRE responses. It was expected to find the best scores in the written condition followed by the computer condition and then the oral condition. In fact, the oral condition oddly had the best scores on the GRE sample questions. Peer pressure appeared to be a large factor in all of the oral questions. Being true for the GRE questions as well, perhaps, once one participant figured out the correct answer and confidently raised his or her hand, then the unsure participants followed suit. If this explanation is in fact correct, then it makes sense that the oral condition would have the best answers because they were in a sense putting all their heads together to answer each question. Whereas, each participant in the written and computer conditions only had himself to rely on for the correct answer. Although there was no significant difference on the number of correct GRE questions, this explanation may account for the trend which occurred. The GRE sample questions appeared too difficult for most of the participants. The GRE was designed for graduating students on the way to graduate school. The participants in this study were not yet at that level. Perhaps, the questions were too hard which is why there was no significant difference found. All the participants guessed at most of the GRE questions. On a total of 12 questions, the average number correct for all participants was 4.8. In future research, it would be beneficial to use questions which are not as difficult for the participants in order to determine a more accurate record of the effects of computer versus written versus oral media on academic questions. As expected, the Rosenburg (1989) self-esteem survey revealed a significant difference with the oral participants reporting higher self-esteem than the computer participants. This result is most likely due to the effects of peer pressure when answering the oral questions. In the United States, it is the socially accepted response to indicate high self-esteem. Perhaps, those participants in the oral condition who reported not as high self-esteem were really the more secure individuals because, facing the pressure to reply in a socially acceptable manner, they may have replied more honestly. In future research, it may be preferred to test self-esteem on paper separately from all other questions in order to find the real self-esteem score. Then the real self-esteem score can be compared to who replied in a more socially accepted manner on the personal questions via differing media. On three of the 37 personal questions a significant difference was found between the groups. Several of the other questions also approached significance. These results show that the type of media does effect the responses of the participants on certain personal questions. Strangely, participants in the oral condition reported "yes" in response to the following question: Have you ever discriminated against anyone? Both the written and computer participants replied "no" significantly more than those in the oral condition. This may have been because in one of the oral sessions, there was a couple strongly voiced non-traditional students. These older participants were around when discrimination was rampant. Because of this history, these participants led the group in being the first to raise their hands in response to this question. As previously mentioned, peer pressure appeared to be a large factor in responses in the oral condition. After the older participants boldly raised their hands, many other participants followed. Not to imply the younger participants were lying, but perhaps they felt more comfortable admitting discriminatory activity after others had already admitted it. It should be noted that there was not nearly any difference between the computer and written conditions. Therefore, significance on this question is attributed to the group dynamics of the oral condition. Another personal question which found a significant difference between conditions was the following: Do you have a close friend of a sexual orientation other than your own? Participants in the computer condition answered yes significantly more than in both the written and the oral conditions. It is suggested that because the computer offers more anonymity, the participants in this condition were more likely to answer honestly. One may wonder why was this particular question significantly different when others were not. In western Missouri, it is not socially acceptable to have a different sexual orientation other than the norm. Perhaps, because this was one of the more threatening questions, the participants only felt comfortable revealing this information to the computer.Computers are an everyday part of life. Many surveys are already conducted over the Internet. It is important to understand the effects differing media has on participant`s responses. Future research should look more closely at the demographic of participants who voluntarily choose to answer computer surveys. The evidence of the effects of computers eliciting truthful responses to personal questions began showing in this small study. Perhaps, future research can examine these effects further. By computers being able to elicit honest responses, it can be easier to learn the truth about our society enabling us to better deal with ourselves.
REFERENCESBooth-Kewley, S., Edwards, J. E., & Rosenfeld, P. (1992). Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference? Journal of Applied Psychology, 77, (4) 562-566.Ertmer, P., Evenbeck, E., Cennamo, K. S., Lehman, J. D. (1994). Enhancing self-efficacy for computer technologies through the use of positive classroom experiences. ETR&D, 42, (3) 45-62.Kiesler, S., Siegal, J., McGuire, T. W. (1984). Social psychological aspects of computer-mediated communication. American Psychologist, 39, (10) 1123-1134. Kiesler, S. & Sproull, L. S. (1986). Response effects in the electronic survey. Public Opinion Quarterly, 50, 402-413. Kling, R. (1980). Social analyses of computing: Theoretical perspective in recent empirical research. Computing Surveys, 12, (1) 61-110.Lautenschlager, G. J., Flaherty, V. L. (1990). Computer administration of questions: More desirable or more social desirability? Journal of Applied Psychology, 75, (3) 310-314. Matheson, K. & Zanna, M. P. (1988). The impact of computer-mediated communication of self-awareness. Computers in Human Behavior, 4, 221-233.Martin, C. L. & Nagao, D. H. (1989). Some effects of computerized interviewing on job applicant responses. Journal of Applied Psychology, 74, (1) 72-80. Rosenberg, M. (1989). Society and the Adolescent Self-Image. Hanover, NH: University Press of New England 325-327.Rosenfeld, P., Booth-Kewely, S., Edwards, J. E. & Thomas, M. D. (1996). Responses on computer surveys: Impression management, social desirability, and the big brother syndrome. Computers in Human Behavior, 12, (2) 263-274.Rosenfeld, P., Doherty, L. M., Carroll, L. K. (1987). Microcomputer-based organizational survey assessment: Applications to training. Journal of Business and Psychology, 2, (2) 182-193.Tourangeau, R. & Smith, T. W. (1996). Asking sensitive questions the impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275-304. |
Submitted 5/12/99 1:26:12 PM
Last Edited 11/23/99 7:05:25 PM
Converted to New Site 03/09/2009
|
