Students   Counselors   Staff

 

• Home
• Scanning Services
• Course Test Out
  • CIT 106
  • CIT 112
  • K201
• National Testing
• Placement Testing
  • Retest Request Form
• Reports
• About Us

 

Home : Report Link : 96 Testing Center Report

Measurement and Evaluation Services for Students, Faculty, Administrators, and Researchers

Organization, Mission, Vision, Values, and Goals

The Testing Center is a component of the Office of the Vice Chancellor for Planning and Institutional Improvement (PAII). The mission of PAII is to integrate the functions of institutional planning, implementation, and evaluation in ways that will continuously improve IUPUI. Figure 1 below shows the organizational structure and mission areas of the Testing Center.

Figure 1. Organization and Mission Areas of the Testing Center

The Testing Center's mission is to provide assessment and evaluation support through the collection and processing of test data, creation of assessment instruments, and the lending of measurement expertise to constituencies throughout the campus community. Our vision is to provide integrated assessment and evaluation information in ways that will continuously improve IUPUI.

All Testing Center activities incorporate the following values:
• Work meets the needs of the sponsoring academic unit or individual.

• Results are thoroughly analyzed and explained.

• Work is timely, accurate, and reliable.

• Information is readily available to those who need it, secure from those who do not.

  Our staff is committed to the following work ideals:

• Professionalism

• Responsiveness

• Thoroughness

• Accessibility

• Friendliness

• Sensitivity to data confidentiality issues

The IUPUI Testing Center's goals are manifested through its seven programs which are described below. We continually strive to make each program more complete, up-to-date, and responsive to the diverse needs of the university community.

Testing Center Advisory Committee. The Testing Center Advisory Committee consists of representatives from the departments of English (Dr. Susanmarie Harrington), mathematics (Dr. Jeffrey Watt), the School of Education (Ms. Mary Wolting), the Admissions Office (Dr. Alan Crist), and the Undergraduate Education Center (Dr. Barbara Metzner). Its purpose is to help guide Testing Center policies and procedures, and to act as a vehicle for disseminating information throughout the IUPUI campus. The committee met twice during 1996.

Overview

Two major events shaped the character of this year. First, the Testing Center assumed responsibility for campus-wide scanning and course evaluation work beginning with the new fiscal year, July 1. As reported previously, we had been negotiating with Integrated Technologies since 1993 for transfer of this function. However, negotiations were slowed as we discussed resource allocations that would allow the Testing Center to continue the quality work Integrated Technologies had provided in the past. In the spring, we formulated a compromise which would allow the Testing Center to receive partial resource support through the transfer of responsibilities, and to begin charging for services.

A number of TC characteristics underlie and support the recommendation for change, including:
• The TC has performed scanning and course evaluation work on a cost-recovery basis for a number of units on campus (and across the IU System) for several years.

• The TC staff have expertise concerning the improvement of testing and assessment. They regularly present workshops on this topic and have consulted with faculty about assessment, testing, and grading in other venues.

• The TC supports technology enhancements (such as LXR Test® and Micrograde®) which are geared to the improvement of assessment and grading.

• The TC provides supplemental services such as survey forms design that can be used in research or other grant activities.

• Compared to the commercial sector or units setting up their own scanning operations, the TC is the most cost-effective scanning supplier available at IUPUI.

• Finally, the Integrated Technologies transition committee saw that optical scanning was intimately linked to the mission statement of the TC.

The Scanning Office is located in the Union Building, G015 and is open from 8:00 a.m. to 8:00 p.m. Monday through Friday and 8:00 a.m. through 5:00 p.m. on Saturday (except University holidays). Parking spaces for drop-off and pick-up are located in the south parking lot of the Union Building just adjacent to Ball Residence. A secure night drop box is available for after-hours drop-off. The Scanning Office has established, through liaison with the Offices of Printing Services, a twice-daily courier service designed to pick up and deliver scanned documents throughout campus. The work of this new unit is discussed in the Scanning and External Testing section of the report.

The second event which shaped our year revolved around improvements to the computerized adaptive placement test in mathematics that began in late 1995. We were able to refine the math test and complete the first stage of bringing the reading test into an adaptive format. This work is more completely addressed in the Placement Testing Section of the report.

Though there were no large-scale campus-wide TQM efforts this past year, the Testing Center continued its support of quality-in-daily-work improvements. These are documented in the Development section of the report, and enhance the quality of our products and processes in the Testing Center.

Placement Testing

Changes Made in Placement Testing Procedures

The present section describes the changes made in placement testing procedures for the IUPUI-sponsored placement tests, including administration, scoring, and reporting processes. These tests were developed for the purpose of matching students with instruction appropriate to their academic preparation in English, mathematics, and reading. As the numbers of students taking the respective IUPUI placement tests continues to increase, so do the incremental improvements made at the Testing Center. Apart from the new staff hires, several significant changes were implemented to facilitate the efficiency, quality and economy of placement testing procedures at the Testing Center. With regard to placement testing, the major changes included the cutover to a computerized reading placement test and the latest versions of the Nelson-Denny reading tests (Forms G and H), improvement in the computerized adaptive testing (CAT) in mathematics, revision of directions for the English placement exams, and hardware and software upgrades in the Microcomputer Testing Facility. A brief description of each change is presented in turn.

Computerized Reading Placement Test. Although there was no widespread dissatisfaction with the Nelson-Denny Reading Test, Form E among counselors, the near zero predictive validity coefficients for reading, coupled with the cumbersome test administration, suggested that an assessment more closely linked to the curriculum was appropriate. The vision for the reading test was that it would perform as a placement test for all students, but would also act as a diagnostic procedure for those who were recommended for developmental coursework. Consequently, the linearly computerized reading placement test was implemented in late June 1996 (Shermis, Wolting, & Lombard, 1996).

The IUPUI computerized reading test consists of five parts, namely, comprehension, reading rate, and three types of vocabulary tests (Word Attack, Words in Context, and Words in Isolation). Development of the computerized adaptive version of the reading test is currently underway and should be completed later this year.

Computerized Adaptive Testing (CAT) in Mathematics. As reported in the Testing Center Annual Report for 1995, the choice to convert the conventional computerized mathematics placement test into an adaptive procedure was made primarily to achieve increased efficiency in the use of both students' time and test items. The implementation of CAT procedures at the Microcomputer Testing Facility has offered a valuable demonstration of how incoming students can be better served through use of computer technology and advanced measurement techniques such as Item Response Theory (IRT). Furthermore, the computerized adaptive mathematics placement test seems to have facilitated the needed improvements in the observed correlation coefficients between placement test scores and specified mathematics outcome measures.

Since the switch-over to CAT, the Testing Center has been collecting information to evaluate the effectiveness of computerized adaptive testing in mathematics. As reported in the Annual Placement Validity Report for 1995/1996, preliminary results from the computerized adaptive mathematics placement test are quite promising. So far, it seems the current adaptive mathematics test is working properly in that the new testing procedure is relatively efficient and yields higher correlation coefficients with the mathematics outcome measures. A complete report of the placement validity study is available at the Testing Center web site (http://assessment.iupui.edu/testing).

Transfer of Chemistry Placement Testing Responsibilities to the Chemistry Department.Effective April 1, 1996, all chemistry placement testing operations were transferred from the Testing Center to the Chemistry Department. Note that on October 21, 1995, the chemistry placement exam was converted into an individually-administered paper-and-pencil test (with the aid of computerized test instructions) at the Microcomputer Testing Facility. Between January and April 1996, the Testing Center had administered the chemistry placement exam to only six students. Currently, however, all chemistry testing responsibilities including test registration, arrangements for test sites, test administration/proctoring, scoring and reporting/uploading of students' chemistry placement test scores on IUTS are being handled by the Chemistry Department.

Validity Study of IUPUI Placement Test Scores

The annual validity study of the IUPUI placement test scores in English, reading, and mathematics was conducted and the final draft report was completed in November 1996. Although the report found that the current validity coefficients for the IUPUI placement tests were generally weak, several positive developments or recommendations were initiated on the basis of the investigation. A summary of the main recommendations is presented in turn for each content domain.

Mathematics. To improve the validity coefficients for mathematics, the new mathematics placement exam utilizing computerized adaptive testing procedures was implemented to replace the conventional mathematics test. This was actually accomplished in October 1995. The choice to convert a computerized test into a CAT was made primarily to achieve increased efficiency in the use of both students' time and test items. The other recommendations made for mathematics include: (a) a possible revision of the current mathematics placement cutoff scores, and (b) a general reminder that decisions regarding course placement should include a review of students' previous academic records and not be made on the basis of placement test scores alone.

Reading. Partly because of the low correlation coefficients between the Nelson-Denny (N-D) Reading Test Forms E and F scores and final course grades, a new reading placement test utilizing CAT procedures is currently being developed to replace the linearly computerized reading placement test. This is important because previous pilot information on the new reading test designed to function in a CAT environment has produced promising results. Meanwhile, a non-adaptive version of the reading test was implemented at the Testing Center in late June 1996.

English. Because of the near zero validity coefficients for English placement test scores, a major recommendation was made for the English Department to consider developing alternative (portfolio) assessments to replace/supplement the existing English placement exams.

An overall recommendation from the placement test validity report was that the Registrar create a "barring mechanism" to enforce compliance with placement recommendations. The bar would prevent a student from enrolling in a class that was inappropriate for the student based on placement test scores unless there was a special request by the concerned department.

Training of Work Study Employees

On-the-job training and formal training sessions continue to be provided to the work study students in order to facilitate professionalism, thoroughness, and efficiency among the Microcomputer Testing Center (MTF) staff. A variety of topics (e.g., new placement testing procedures, test registration or scheduling of students for testing, customer service, generation of placement test reports, test security, confidentiality of information, Testing Center emergency procedures, teamwork, work ethics, etc.) is usually covered during formal training sessions conducted approximately twice per month. The topics are selected on the basis of the needs of the Testing Center, proctors, receptionists, and/or on the basis of a placement testing exit survey conducted at the conclusion of placement test administration. The MTF survey was designed to solicit information such as expertise and disposition of the test proctors, examinees' perceptions of the testing situation, examinees' computing background, and so on. In addition to providing some suggestions for training topics, information from the MTF exit survey is used internally to monitor proctor performance and specify potential technical innovations that might improve the examinees' test experience.

Furthermore, bi-weekly group meetings of work-study employees have continued to offer the work-study staff an opportunity to present and discuss their concerns and/or suggestions for the improvement of placement testing operations at the Testing Center.

MTF Satisfaction Survey. The placement testing exit survey was designed to solicit information about students' computing background, their perceptions of the testing experience, expertise and disposition of the proctors, and some demographic characteristics. The information obtained from the survey is used internally to monitor proctor behavior, suggest training topics, and specify possible technical innovations and recommendations that might improve the efficiency and effectiveness of the placement testing operations. The survey also provides valuable information for external communication purposes with other service units by demonstrating how the placement testing operations fit in with other IUPUI enrollment activities.

A summary of the survey results for the data collected in 1996 is as follows. The results are based on a total of 3515 respondents, which represents approximately a 50% increase in response rate over last year's sample size. Overall, the results of the present survey suggest that most students have a very favorable disposition towards the Testing Center's computerized placement testing program. For instance, 3254 out of 3515 (or approximately 93%) respondents found the behavior of the MTF staff to be courteous or very courteous. In contrast, only one percent (or 37 out of 3515) of students reported that the MTF staff were somewhat impatient or rude. Also, a majority of the respondents (95%) reported the perception that the MTF staff had sufficient or quite extensive knowledge of computers. In contrast, approximately five percent of the respondents found the MTF staff to have inadequate computer knowledge. Perhaps this is due to the presence of new work-study students training on the job in the Microcomputer Testing Facility. The unfavorable situations are normally addressed and corrected through formal work-study training sessions and/or bi-weekly meetings.

With respect to the clarity of test directions, 97% and 98% of the respondents reported that the directions for the English and reading exams, respectively, were quite understandable or overly simple. However, only 92% of the students reported similarly for mathematics exams. Perhaps the periodic addition of ongoing research directions to the existing mathematics instructions may have contributed to the lower percentage rate observed for mathematics. Otherwise, a majority of the students reported that the three placement exams had very clear test directions.

A rather remarkable finding was that approximately 50% of the students found the reading exam to be too demanding, whereas about 49% felt that the reading test was an accurate measure of knowledge. In contrast, only 28% of the respondents felt that the mathematics exam was too demanding; and 67% reported that the mathematics test was an accurate measure of knowledge. For English placement exams, 19% of the students perceived the tests to be too demanding; whereas 78% of the respondents said the exam was a valuable learning experience. Figure 2 shows a summary of the student perception of the accuracy of the IUPUI placement tests.

Figure 2. Student Perception of the Accuracy of Placement Tests

Regarding students' experience in using computers, the present survey results indicate that only about two percent of the respondents had no computer experience at all. In contrast, approximately 29% of the 1996 exit survey respondents reported having a great deal of computer-related experience. Despite the difference in response rates, these particular findings have remained unchanged from the 1995 survey results. With respect to facility or ease in using computers, nearly 56% and 51% of the 1996 and 1995 survey respondents, respectively, reported that using computers was very easy. In contrast, only approximately three percent of the respondents reported that use of computers was very confusing. Perhaps this might help explain why there is currently an observed increase in the proportion of students who reported feeling quite at ease in taking the computerized tests. A majority of students (56%) reported having adequate knowledge of computers, and only a small proportion of students (3%) reported feeling very confused with computerized testing. The development and implementation of improved sets of computerized instructions (e.g., the adaptive mathematics instructions currently implemented at MTF) seem to make the students' testing experience more pleasant, less confusing, and more "user friendly". In addition, thorough training of our test proctors in test administration procedures as well as proctors' vigilance or alertness during test administration, can help to minimize or eliminate unnecessary examinee confusion.

The Testing Center is in a continuous process of improving the efficiency and effectiveness of the placement testing operations, partly through the implementation of computerized adaptive testing procedures and the placement testing exit survey.

Student Phone Survey. In addition to the placement testing exit survey, the Testing Center administers a phone survey to students who call to register for placement tests. Placement testing reservations can be made Monday through Friday from 8:00 a.m. to 5:00 p.m. The phone survey was designed to obtain specific information pertaining to (a) how easy it was for students to call the Testing Center for test registration purposes, (b) what (if any) problems were encountered by students in calling the Testing Center, (c) how long it took the receptionist to register the student, (d), the reason for a lengthy phone conversation (if applicable), (e) number of students requesting rescheduling, and (f) students' reasons for rescheduling the placement test(s) (if applicable).

A total of 3016 respondents participated in the phone survey conducted between January and December 1996. The results of the phone survey are presented in Table A.1 (see Appendix A). While most individuals were able to get through on the first call, it was distressing to note that about 12% of those who responded reported calling four or more times. Figure 3 shows the breakdown for this quality indicator. Most repeat calls came form individuals trying to contact the TC outside of business hours or during peak registration times. Part of the problem should be addressed when phone reservations are directed to the Enrollment Center.

Figure 3. Number of Times that Respondents Reported Calling the Testing Center Prior to Making a Reservation.

Scanning

As was mentioned previously, a campus-wide scanning operation was established on July 1, 1996. This office is staffed 72 hours per week with one manager and two full-time staff. One individual is supported through permanent funding while the other staff member is supported from income generated by the Scanning Office. The office also receives part-time support from 3-4 work-study students. Responsibilities are divided into three general areas: forms design, course evaluation production, and scanning.

Scan Forms Design. The Scanning Office of the Testing Center offers scoresheet design on a cost-recovery basis. This service was implemented to permit faculty and staff to inexpensively design their own survey and measurement instruments in scannable form. In 1996, this service was used by both academic and administrative units and generated approximately $8,200 of income. We have recently negotiated a contract with Kinkos to perform high speed printing from scan forms designed through the Scanning Office.

Course Evaluation. The Student Evaluation of Teaching (SET) program was developed in 1994 with the School of Public and Environmental Affairs, using SPEA's 5 IU campuses. In Fall 1995, we also added the School of Medicine to the program. Course evaluation activities for the School of Medicine cover 6 campuses throughout the state. This past year we added IUPUI units to the system that will ultimately allow customized reporting over multiple semesters. We produced approximately 47,426 course evaluation sheets this past year, with an income of $7,114. We scanned and processed approximately 32,000 scoresheets with an income of $4,800.

Scanning. From July to December, the Scanning Office scanned and scored 27,037 scoresheets. This operation received approximately $4,500 in income last year. Figure 1 shows the distribution of scanning volume by month. Since its inception at the beginning of the fiscal year, most of the volume generated from scanning has been in the processing of paper-and-pencil tests.

Figure 4. Scanning volume by month (Jul-Dec 1996)

External Testing

External testing refers to the part of the IUPUI Testing Center that administers tests which are not part of the placement testing program. Such tests include Independent Studies tests and exams that are done on behalf of profit and non-profit testing companies. This responsibility area also includes scanning, course evaluation activities, and forms design. This area is linked to IUPUI's goals of learning and collaboration.

Strong Interest Inventory Testing. In 1995, the Consulting Psychology Press computerized the Strong Interest Inventory, a vocational interest measure used by the School of Business, and the paper-and-pencil version of the Myers-Briggs Type Inventory (MBTI). This step was a great benefit to students since they could take the interest inventory at times that were convenient to them. Moreover, instructors benefited from the transition to computerized testing since they no longer had to devote instructional time to that assessment. The students were able to take the test and receive immediate feedback. Prior to 1995, students took paper-and-pencil versions of the Strong, which then had to be sent out for scoring. This process sometimes took up to a month to complete.

In 1996, the IUPUI Testing Center was chosen as a beta test site for the computerized Strong Interest Inventory Test, Version 4.0. We tested 780 students from the School of Business and the Division of Continuing Studies. In 1997, we plan to administer the MBTI by computer as well.

Independent Studies Testing. In 1994, we computerized the administration of the Independent Studies tests and began administering the tests in the MTF lab. We have continued to give the Independent Studies exams from IU as well as from other institutions in the MTF lab. We gave approximately 1,070 of these exams over the past year. This operation brought in approximately $14,000 in soft money last year. As a consequence of this extra-mission activity, we were able to continue providing the students with more flexible hours and better service by using the MTF lab.

In the spring of 1994, we moved the Independent Studies testing to computerized administration. In late 1994, we implemented an exit survey similar to the one that is given to students taking placement tests. This quick survey queries students about the manner in which they were greeted, the checking of identification, and how quickly they were tested after arriving at the MTF. This information is used both to provide feedback to our work-study proctors and to our clients in the Division of Continuing Studies. For 1996, out of a sample of 435 responses, 393 students (90.3%) felt that they had been greeted in a friendly manner (See Figure 5). Three hundred sixty-one students reported they were able to start their test within five minutes of arriving at the Microcomputer Testing Facility. Three hundred eighty-seven students felt that they were given adequate instructions on how to use the computer prior to starting the test. Ninety-seven percent of the students were comfortable or neutral about using the computer for testing activities (13% higher than last year). Forty-four percent of the students were in the 20-24 age group, followed by 17% reporting ages between 25 and 29. Seventy-three percent of these students were female and 90.3% were Caucasian.

Figure 5. Responses to the Query of Whether or Not the Student Had Been Greeted in a Friendly Manner.

National and State Testing. In addition to our internal testing operation, we also administered state and national tests for the benefit of students and staff. Over 6,000 students took exams under this program, which serves the needs of the students and professional individuals. Tests administered under this program include the following:

Development

Development refers to two components of Testing Center activity. One component focuses on test development and is staffed by two graduate student research assistants. The second component centers around the development of computer-based tests, supplemental multi-media, and data collection mechanisms. This section is staffed by a .75 FTE programmer and three work-study students. Both components are linked to the IUPUI goals of learning and collaboration.

Test Development. Efforts to improve upon test and data collection procedures were supported through a number of studies conducted in the Testing Center this year. The first experiment (Shermis & Lombard, 1996a) examined the differences between data collected via e-mail and that collected by self-completed mail questionnaires. This research was an outgrowth of a survey sponsored by the National Council on Measurement in Education during the fall of 1994 on technology use of its membership. The study concluded that data collected by e-mail could be accomplished more efficiently and with good integrity, but that overall response rates were not as high as for data collected by self-completed mail questionnaires. The authors speculated that over time this trend would change.

The second study (Shermis & Lombard, 1996b) looked at whether or not student performance on placement tests is negatively affected by computer and/or test anxiety. This has been a concern ever since we switched to the computerized testing format in the fall of 1993. The results of the study suggest that at least for math, student reports of anxiety are related to test-taking anxiety and not their interactions with technology.

During the spring of 1996, we completed a study which was designed as a validation effort for the Merkler Preference Style Inventory (Shermis, Mzumara, & Webb, 1996). The MPSI is being contemplated as a possible quick vocational interest inventory for our incoming students. The study revealed that the MPSI has good concurrent validity properties, but that a predictive validity study would have to be completed before it warranted general adoption.

In the summer we began work on a study that would permit us to evaluate the feasibility of asking screening questions as a means of determining students' initial ability level for math. The hope is that by starting students off with questions closer to their ability levels, the adaptive tests can more quickly determine student placements. The "screener study" has undergone several iterations and is currently in the data collection phase.

Our final study explored the feasibility of allowing students to select the difficulty level of questions that they might encounter on the math placement test. This demonstration study will allow us to determine whether we should incorporate "self-adaptive testing" as part of our offerings to incoming students. Data collection is completed and the study will be presented at the annual meeting of the National Council on Measurement in Education this coming spring in Chicago.

General Support. The computer development team is tasked with responding immediately to problems that might arise from any one of the operational areas of the Testing Center. In addition to providing general software support, the team also addresses hardware and communications problems as well.

Computer Hardware and Software Upgrades in the Microcomputer Testing Facility. The computers in the Microcomputer Testing Facility were upgraded from 8 megabytes (MB) to 16 MB of memory, which facilitated the improved presentation of test directions and items. In addition, the Compaq computers were upgraded to Windows 95. This will permit the MTF to run the latest multi-media and Internet software. The Macintosh machines were upgraded to System 7.5.3.

IUTS Development. Based on a proposal submitted to the Software Development Policy Committee (SDPC), system-wide funding was provided to convert the Indiana University Testing System report program (IUTS) to a client-server environment. A number of problems had been identified with the current (mainframe) system, including the lack of storage vectors for outcome information, restrictions on modifications to the existing system, and restrictions on the time that the mainframe was available for the uploading of scores. In addition to addressing these problems, the new system will merge the test score database with a scheduling function. It will also permit a more sophisticated query and reporting procedure.

Programming is being provided by University Computing Services (UCS) using Sybase software and a server located in Bloomington. A meeting was hosted in Indianapolis last spring with representatives from most of the IU campuses to articulate the vision of the new system. Most of the campuses have agreed to participate in piloting the software. Completion of the conversion project is scheduled for spring 1997.

Automatic Uploading of Placement Test Scores and Placement Test Reports. One of the problems with MTF procedures identified last year had to do with rare, but significant errors committed by proctors when they uploaded scores to IUTS. In place of the manual entry of scores, a computer routine was developed to automate the process. Thus, the individual lab machines now automatically upload the mathematics and reading placement test scores from the local area network directly to IUTS three times per day. The automatic upload of mathematics and reading test scores has significantly reduced human or clerical errors, and thereby greatly improved the accuracy and efficiency of test score reporting to IUTS users campus-wide. Similarly, the FoxPro application for test report generation was improved and the procedure for producing hard copies of the students' test reports has now been automated. Consequently, automation of test score reporting procedures has enabled test proctors to be more vigilant and responsive to students' needs during test administration. The work on this project was performed by Integrated Technologies through support from the Vice Chancellor for Planning and Institutional Improvement.

SET Development. Most of the work that has been done on the Student Evaluation of Teaching (SET) database has been accomplished through an external programmer. The SET program allows academic units to create, scan, and score course evaluations in a longitudinal format. Based in FoxPro, this database has the capability of producing customized reports employing easy-to-use macros. Programming costs are supported internally though scanning income.

Contracted Work. The Development Office contracted with the Department of Psychology to convert the item bank from their current test development environment (Trilogy) to LXR Test 5.0. The Department administers all of its objective tests in the Introductory Psychology course via computer, which saves a significant amount of money in supplies, equipment, and personnel.

The TC Development Office also continued support for web pages for the National Council on Measurement in Education (NCME) server. Development for the NCME server is a reimbursed activity conducted on behalf of that non-profit educational organization.

HTML Programming. In addition to contracted support with NCME, the Testing Center also maintains a web page for the dissemination of test times and locations, Scanning Office hours and charges, requests for IUTS access, and requests for re-testing. In addition, Testing Center reports, bulletins, and general information memoranda are located on the web. The Testing Center Web Server can be reached at http://www.assessment.iupui.edu/testing/.

Teaching, Research, and Service

In this final section, we list some of the external activities that speak to our contributions beyond the student service mission of the Testing Center.

Teaching

Mark Shermis designed and taught two courses for the Department of Psychology: Clinical Rehabilitation Psychology Assessment I (Psych I664) and Clinical Rehabilitation Psychology Assessment II (Psych I669). Howard Mzumara taught one course for the School of Education: Evaluation Models & Techniques (Educ Y535).

Research

(Articles)

Shermis, M. D., & Chang, S. H. (in press). The use of IRT to investigate the hierarchical nature of a college mathematics curriculum. Educational and Psyhological Measurement.

Shermis, M.D., Falkenberg, B., Appel, V.M., & Cole, R.W. (1996). Construction of a faking detector scale for a biodata survey instrument. Military Psychology, 8(2), 83-94.

Shermis, M. D., Fulkerson, J., & Banta, T. W. (1996). Computerized adaptive math tests for elementary talent development selection. Roeper Review, 19(2), 91-95.

Shermis, M. D., Stemmer, P. M., & Webb, P. M. (1996). Computerized adaptive skill assessment in a statewide testing program. Journal of Educational Computing Research, 29(1), 49-67.

Shermis, M. D., Wolting, M., & Lombard, D. (1996). Development of a Computerized Test for College Reading Placement. Journal of Developmental Education, 20(2), 18-24.

(Articles Under Review)

Printz, B., Shermis, M. D., & Webb, P. M. (1996). Stress Buffering Factors Related to Adolescent Coping: A Path Analysis. Manuscript submitted to the Journal of Adolescent Health.

Shermis, M. D. (1995). Using computerized adaptive testing for college mathematics. Manuscript submitted to Educational and Psychological Measurement.

Shermis, M. D., & Lombard, D. (1996a). A comparison of survey data collected by self-completed mail questionnaire and electronic mail. Manuscript submitted to the Journal of Business and Psychology.

Shermis, M. D., & Lombard, D. (1996b). Effects of Computer-Based Test Administrations on Test Anxiety and Performance. Manuscript submitted to Computers in Human Behavior.

Shermis, M. D., Webb, P. M., & Mzumara, H. R. (1995, December). An assessment of the concurrent validity and reliability of the Merkler Style Preference Inventory (MSPI). Manuscript submitted to Journal of Career Development.

(Reports)

Mzumara, H. R., Shermis, M. D., & Wimer, D. (1996, November). Validity of the IUPUI placement test scores for course placement: 1995-1996. Indianapolis, IN: IUPUI Testing Center.

Shermis, M. D., Mzumara, H. R., Kiger, B., & McBee, V. (1996). The Testing Center Annual Report 1995. Indianapolis, IN: IUPUI Testing Center.

(Presentations)

Laidig, J. M., Shepard, M. J., Shermis, M. D., & Webb, P. M. (1996, April). Assessment of mathematical calculation skills via computerized adaptive testing. Presentation made at the North American Conference for Learning Resources in Nursing, Indianapolis, IN.

Mzumara, H. R. (1996, February). Workshop on grading and marking. Workshop sponsored by the IUPUI Faculty Development Office, Indianapolis, IN.

Shermis, M. D. (1996, March). Assessment technology support. Workshop sponsored by the IUPUI Faculty Development Office, Indianapolis, IN.

Shermis, M. D. (1996, April). Computerized adaptive testing in the basic skill areas. Presentation given to the faculty at Northeastern Illinois University, Chicago, IL.

Shermis, M. D. (1996, May). Interpreting test scores for parents. In-service given at Indian Creek Elementary School, Indianapolis, IN.

Shermis, M. D. (1996, August). Interpreting test scores for parents. In-service given at the Lawrence Metropolitan School District, Indianapolis, IN.

Shermis, M. D., & Mzumara, H. R. (1996, November). Assessment technology support. Workshop given at the fifth annual Assessment Conference, Indianapolis, IN.

Shermis, M. D., & Mzumara, H. R. (1996, November). Interpreting placement test scores. Presentation given to the faculty of Ben Davis High School, Indianapolis, IN.

Shermis, M. D., Stemmer, P. M., Berger, C. F., & Anderson, G. E., (1996, April). Web magic: Navigating the Internet for measurement research. Workshop presented at the annual meetings of the National Council on Measurement in Education, New York, New York.

Shermis, M. D., Watt, J., Harrington, S., Wolting, M., Galyen, L., & McGregor, J. (1996, November). College placement testing via the web: Early warning systems for high schools. Presentation made at the fifth annual Assessment Conference, Indianapolis, IN.

Howard Mzumara was a participant in the Instrument Fair conducted at the 1996 Assessment Conference in Indianapolis, IN.

(Grant Applications)

Mzumara, H. R., & Shermis, M. D. (1996, February). Inside Indiana University testing and grading practices: Implications for quality improvement and training in classroom assessment. Grant submitted under the auspices of the Strategic Directions Charter of Indiana University ($12,969--unfunded).

Shermis, M. D., & Mzumara, H. R. (1996, February). Equating placement tests across the campuses of Indiana University. Grant submitted under the auspices of the Strategic Directions Charter of Indiana University ($27,460--unfunded).

Shermis, M. D., & Mzumara, H. R. (1996, October). College Placement Testing Through the World Wide Web: Preparing Students for Post-Secondary Education. Grant submitted under the auspices of the Strategic Directions Charter of Indiana University ($63,333--under review).

Consultant Presentations

Mark Shermis gave lectures last year in test construction, statistics, research design, and I/O psychology for the Association for Advanced Training in the Behavioral Sciences (Ventura, CA) in various cities throughout the country.

Journal Reviewer

Mark Shermis was a journal reviewer for the following scholarly publications:

Archives of Physical Medicine and Rehabilitation (1 manuscript)

Conference Reviewer

Shermis was also a reviewer for conference papers for the following organizations:

American Educational Research Association (AERA)

National Council on Measurement in Education (NCME)

Mark Shermis Committee Work

(National Committees)

Chair, NCME Committee on Electronic and Alternate Communications

Member, American Psychological Association Continuing Education Committee

(IU System-wide Committee)

Joint Sub-committee on Learning

(University-wide Committees)

Academic Affairs Committee

Academic Policy and Planning Committee

Administrative Council

Professional Communications Committee

Program Review and Assessment Committee

Testing Center Advisory Committee

(Department of Psychology Committees)

Methodology Group

(TQM)

Executive Steering Group for Student Enrollment Support Systems

Co-chair, Research Team for the Enrollment Management Group

IMIR/Systems Management Steering Group

Howard Mzumara Committee Work

TC Representative to the Enrollment Center Steering Group/Entry Process Action Team

TC Representative to the Orientation and UEC Staff Group

Member of the Strategic Directions Planning on Quality -- Study Group 4 (Developing assessment measures and infrastructure for data collection that promote improvement and demonstrate accountability to external stakeholders)

APPENDIX A

Table A.1

Results of the Phone Survey for the 1996 Sample.

1. NUMBCALL Number of Times Calling TC
(How many times did you call before reaching us?)

2. PROBFACE Student Problems Faced
(What problems did you face in reaching the Testing Center?)

3. REGTIME Student Registration Time
[How long did it take you to register the student completely (with FoxPro)?]

4. RESCHEDL Student Rescheduling?
(Is the student rescheduling?)

5. WHYDELAY Reason for Delayed Registration
(If the call lasted longer than 7 minutes, the reason was:)

6. WHYRESCH Reason for Rescheduling
(The reason for rescheduling was:)