SECTION III THE USER Sandra Browning In order to develop a user population for SUPARS, many aVenues for advertising the service were explored. These included classified and display ads in canpus newspapers, as well as large posters placed on canpus. Letters to faculty and graduate students were mailed, introducing the service and asking them to announce the service to their classes. Once the population was developed as much information as possible was gathered on the user; his problems, his characteristics and his attitudes. As it is more effective to use several methods and make oonparisons, many methods for achieving this goal were pursued. During the hours of SUPARS operation a telephone aid service was maintained by the staff. At the sane tine an unobtrusive data collection program, STAIPAC, was being run. During the last week SUPARS was in operation, a telephone interview was conducted of a sanple (100) of registrants, both users and non-users of the system, in order to learn their use of and reaction to SUPARS. Another sanple was drawn and administered the semantic differential, aimed at identifying and measuring attitudes toward SUPARS. During every phase of the SUPARS operation, the staff attexrpted to assist the users. First through the publicity canpaign, which reintroduced the service to the canpus oonftunity, and next by sending a User's Manual to each person oonpleting a registration form, During the actual operation of SUPARS, the telephone aid service attempted to supply answers to the user's problems, while STATPAC gathered data unobtrusively. The telephone interview and the semantic differential were used as a neans of gathering inforrtation concerning attitudes toward SUPARS and suggestions for inprovenent were also accepted. 1. PIOFILE OF REGISTRANTS To be eligible to use SUPARS during its six week operating period, all potential users were required to ooitplete a registration form (Appendix V.) Questions on the registration form were aimed at collecting demographic-type information, information regarding registrants' usage of Psychological Abstracts, and information about registrants' previous experience with computer terminals and retrieval systems. Data from all oarrpleted questionnaires were entered into the SUPARS Registration Retrieval System (SRRS), a small-scale retrieval system using APL which was especially designed for this purpose. Usage of the Inquiry or retrieval portion of SRRS allowed quick compilation of the descriptive information presented below. Descriptions presented in this section deal with the population of 57 registrants in terms of a user/non-user dichotomy * Users were identified as such from STATPAC output, which lists the social security numbers of all those registrants who signed onto SUPARS and at least once entered a delta (A) to begin a search. Several problems with this definition of users should be noted. First of ail, registrants who tried to sign onto SUPARS, but for sane reason failed to reach the point of entering a delta, are not here considered as users* Responses to the telephone interview (see part 4 of this section) indicate that nearly one-third of those classified at the time of the interview as non-users by the STATPAC definition had tried to use SUPARS at least once. Secondly, twenty social security numbers listed by STATPAC as user numbers oould not be matched with registration forms. And finally, there is evidence that some users, rather than signing off SUPARS after corpleting their searches, left the system operating for others to use. These subsequent lasers then entered searches which were recorded by STATPAC under the social security numbers of the original users. Thus, the total number of users (143) referred to belcw represents a somewhat conservative estimate of the total number of people who used the SUPARS system in some way. The graphs and tables below are arranged into four sections: (1) cumulative growth of the registrant population, (2) demographic data, (3) use of Psychological Abstracts by registrants, and (4) computer experience of registrants. Questions on the registration form from which the relevant raw data were originally drawn are listed preoeeding the appropriate section of data description. a. Cumulative Growth of the Registrant Population Figure 9 depicts the growth curve of SUPARS daily registration. (Date registered was considered to be the date on which a registration form reached the SUPARS office). During the 11 weeks of the registration period, which extended from October 4, 1971 to December 17, 1971, a total of 596 people registered to use SUPARS. The third and fourth weeks of the period were the weeks of the heaviest registration. By day 16, one-third of the population had been registered. On day 20, the half-way point of registration was reached. And by day 23, b«^o-thirds of SUPARS" potential users had registered. Last year during a 14-week registration period, SUPARS registered 349 potential users. The half-way point of registration occurred on day 30; the two-thirds mark was reached on day 37. (3) A oonparison of data for the two periods thus indicates that during the current year more potential users were registered more rapidly and over a shorter period of time than were registered during the comparable period last year. This is most likely accounted for by this year's more extensive publicity canpaign, coaled with the certain degree of familiarity with SUPARS already present in the university population due to last year's operation. 58 CF: 10/4 10/11 10/18 10/25 11/1 11/8 11/L5 11/22*11/29 Days of Registration Period 12/6 1 2 A 3 •Thanksgiving vacation - no registration forms processed. Figure No. 9. Cumulative Growth of SUPARS H Registrant Population 59 b. Demographic Data (1) University Status Question 6: Are you primarily: Faculty, Undergraduate student, Graduate student, Staff, Administrator, Visitor, Other? Table V lists the number and percentage of users and non-users in each of the seven categories of university status. Graduate students accounted for the largest proportion of both users and non-users, with 62.4% of the users and 45.6% of the non-users being graduate students. The undergraduate and faculty groups were the second and third largest groups in both the user and non-user populations. Of the three university status subgroups mentioned, only the graduate student group contained more users than non-users. In last year's registrant population, the three major academic subgroups were also graduate students, undergraduates, and faculty. Hcwever, the different methods used in accounting for multiple responses to the academic status question (See Table V) make exact between-year comparisons somewhat tenuous (2) Departmental Status Question 7: What is your academic major or department? As indicated by Table V I , the three individual departments accounting for the largest nuntoer of both users and non-users are, as would be expected from the nature of the data base and of SUPARS' affiliation, Education, Psychology, and Library Science. In the user group, Library Science accounted for 24.7% of the total, Education for 21.3%, and Psychology for 20.7%. Among the non-users, nearly one-third of the total was accounted for by Education registrants and one-quarter by Psychology registrants. Approximately one-third of both the user and non-user population falls into the category, "Other." Table VII lists the full range of departments represented in the total SUPARS registrant population. For all major departments, there were more non-users than users. A comparison of this year's data with last year's indicates that for both operating periods, the same major departments reported the heaviest rate of registration. However, for last year's group of registrants, the proportion of registrants from the Psychology Department was higher (39.5%), accounting for nearly t^o-fifths of all registrants. This difference is probably due, not to any marked decrease this year, in interest on the part of those affiliated with the Psychology Department, but rather to the broader range of departments contributing registrants this year and to increased registration from such departments as Business, Journalism, Hone Economics, and Social Work, (3) Time Spent on Teaching, Research and Other Activities v 60 TABLE V NUMBER OF SUPARS REGISTRANTS BY UNIVERSITY STATUS* NUMBER OF REGISTRANTS STATUS i ! /klroinistrator 1 Faculty i Users 2 1.3% 11 7.4% Graduate Student 93 62.4% ' Nonusers 4 .8% 82 17.3% 216 45.6% 30 Total 6 1.0% 93 14.9% 309 49.6% 32 6.3% 5.1% 161 27.2% 25.8% 7 .8% 1.1% 15 1.9% 623 100% 100% 2.4% ! i La , a . *' j Staff t. • 2 1.3% 32 1 Undergraduate i 129 21.5% i Visitor 3 2.0% 4 Other 6 4.0% 9 Total 149 100% 474 *Note: registrants were permitted to answer in more than one category. When a respondent answered in more than one category, each answer was tabulated as a separate response, reflected by the inflated oolurm totals* For last year's data analysis, all multiple responses were totaled into the category "other". 61 TABLE VI NUMBER OF SUPARS REGISTRANTS BY MAJOR DEPARTMENTAL CATEGORIES* NUMBER OF REGISTRANTS DEPART EOT Users Education 32 21.3% Nbnusers 149 31.0% 149 24.6% ,459.4% 156 31.3% 32.5% 12 2.0% 2.5% 15 2.4% ^i Total 181 28.7% Psychology 31 20.7% 118 23.7% 82 13.0% 203 32.2% Library Science 37 -24.7% 47 Other** Jo Answer 3 150 Total 100% 480 100% 630 100% *Registrants were permitted to answer with more than one response. In cases of multiple responses, each response was tabulated separately, giving the inflated coluran totals. **See Table VII for a detailed listing of all departments. 62 TABLE V I I NUMBER CF SUFARS REGISTRANTS BY DEPARTMENTAL CATEGORIES Department Accounting Anthropology Architecture Users Non-Users 2 Total Art Biology Business Chemistry Economics Education Engineering Sfliglish Forestry Geology History Home Bconaaics International Relations Journaliam Journalism Library i i i 2 3 1 1 32 2 1 3 1 — 5 4 3 5 13 1 1 149 2 5 6 — 1 — 6 — — — — Law Liberal Arts Library Processing Library Science Math Music Philosophy Physical Education Political Science Pre~Law Pre-Madical Psychology Psychological Research Center Public Administration Rcmance languages Russian Studies Systems Information Science Social Sciences Program Sociology Social Work Speech Student Affairs Office Student Personnel Administration Syracuse University Research Cbrp Upstate Medical Center Urban Studies No Answer/&iestion does Not Apply 37 — — — 2 14 2 15 1 1 6 1 45 2 1 2 — 1 2 — 1 31 — 1 — — 3 1 1 8 4 — 4 1 2 118 1 4 1 1 3 1 8 25 7 1 — • 1 — — — 3 150 63 1 1 1 12 480 2 6 5 4 7 16 2 2 181 4 6 9 1 2 15 2 21 1 1 6 1 82 2 1 2 1 6 1 3 149 1 5 1 1 6 2 9 33 11 1 1 1 1 1 15 630 Question 8: What percent of your tine is engaged in: Teaching and/or learning, Research, Other? Tables VIII through X present data on the percentages of time spent by registrants in teaching/learning, research, and other activities. As shewn by Table VIII, approximately one-third of the users reported spending between 76 and 100 per cent of their time on teaching and/or learning activities. Another quarter of tlie users indicated spending 26-50% of their time on these activities. The same categories also accounted for the largest proportions of respondents in the non-user group. According to Table IX, a little over one-half of both users and non-users reported spending between 0-25% of their time on research activities. Similarly, as shown in Table X, over one-half of both users and nonusers reported spending 0-25% of their time on activities other than teaching, learning, or research. It is to be noted that, in answering this question, respondents were not required to total their answers to 100%. Conparable data was not presented in last year's final report. (4) Previous SUPARS Registration and Use Question 9: Did you register to use SUPARS last year? Question 10: If yes, on the average, how often did you use SUPARS — more than once a day, once a day, several times a week, several times a month, once a month, less frequently, no time? As indicated in Table XI, the greatest majority (76.9% and 85.9% respectively) of both users and non-users did not register to use SUPARS last year, although there is a slight tendency for a larger proportion of the user group than of the non-user group to report registration. Data from Table XII indicates that the majority of users who had registered last year reported using SUPARS several times a week or several times a month. Together these two categories account for two-thirds of the responses. Non-users who had registered with SUPARS last year most often reported having used SUPARS several times a month, with a somewhat higher percentage of (this year's) users reporting such usage. More non-users than users (19.4% as oenpared with 9.1%) reported using SUPARS no time. c. Use of Psychological Abstracts by Registrants (1) Reoent Use of Psychological Abstracts Question 11: On the average, over the last 2-3 months, how frequently have you used Psychological Abstracts — more than once a 64 TABLE VIII PERCENTAGE OF TIME ENGAGED IN TEACHING AND/OR LEARNING NUMBER OF REGISTRANTS I Peraent of Time Users Nonusers Total 1 0-25% 'i 19 13.3% 36 25.2% 54 11.9% 129 28.5% 80 17.7% 123 34.3% 27.1% 67 14.0% 14.8% 73 12.2% 165 27.7% 99 16.6% 172 28.9% 87 14.6% | j 1 26-50% i i 51-75% 19 ^13.3% 49 j 76-100% No Answer 20 TOTAL i i i • • • • • • , I 143 100% 453 100% 596 100% 65 TABLE DC PERCENTAGE OF TIME ENGAGED IN RESEARCH ^ — i Percent of Time Number of Registrants Users Nonusers Total i 0-25% 79 55.2% 228 50.3% 76 18.9% 16.8% 13 0.0% 2.9% 13 3.5% 2.9% 123 22.4% 27.1% 453 100% 100% 307 51.5% 103 17.3% 13 2.2% 18 3.0% 155 26.0% i 26-50% [ 27 51-75% 76-100% * . - 5' 32 j No Answer 1 • ! TOTAL 143 596 100% J 66 TABE£ X PERCENTAGE OF TIME ENGAGED IN OTHER ACTIVITIES • " • • " - • • • ' • " ! Nunfoer of Registrants Users 0-25% 82 57.3% 13 9.1% 5 3.5% 76-100% • " • ' . " i Percent of Time ', Nonusers 245 54.1% 32 7.1% 13 2.9% 8 Total 327 54.9% 45 7.5% 18 3.0% 10 ! i . i | 26-50% l j 51-75% 1 2 1.4% 41 28.7% 155 1.8% 196 34.2% 1.7% | 32.9% i No Answer Total 143 100% 453 100% 596 100% 67 TABLE XI REGISTRATION FOR SUPARS LAST YEAR f Indication of \ Registration Nurrber of Registrants Users Ifonusers Total ! Yes 33 23.1% 62 13.7% 389 76.9% 85.9% 95 15.9% 1 1 Nb 110 499 83.7% 2 0.4% 0.3% No Answer 0.0% 2 Total 143 100% 453 100% 596 100% '••••-' •• i day, once a day, several tines a week, several tines a month, once a month, less frequently, no time? Table XIII shews that both the user and non-user groups most frequently reported no usage of Psychological Abstracts in the 2-3 month period preoeeding registration, ihis response accounted for 37.1% of the users and 33.1% of the nonusers. Approximtely another one-fifth of both groups reported their previous usage of Psychological Abstracts to have been several tirres a month. These data are ocnparable to last year's data. (2) Future Use of Psychological Abstracts Question 12:Estimate how frequently you see yourself using Psychological Abstracts in the next 2-3 months — 1 more than onoe a day, once a day, several times a week, severed, times a month, onoe a month, less frequently, no tine? 68 TABLE XII USAGE OF SUPARS BY LAST YEAR'S SUPARS REGISTRANTS 1 1 . . . 1 1 ! ! • Ml •• 1. 1 • — "II" ' 1 l » ... • 1 !•• lJurttoer of Registrants Frequency of Usage* Users A B 0.0% 0.0% 10 36.4% 10 30.3% 1 6.1% 4 12.1% 3 9.1% No Answer 1 3.0% Total 33 100% 62 100% 0.0% 95 100% 10 16.1% 12 19.4% 1 1.0% 15 15.8% 17.7% 14 14.7% 19 30.7% 13 13.7% ] 16.1% 29 30.5% 22 23.2% 0.0% 1 3.0% 0.0% Nonusers Total 1 1.1% j ': c D 12 1 J E F 1 G *A B C D E P G More than onoe a day Onoe a day Several tines a week Several tines a month Once a month Less frequently No tine 69 TABLE XIII AVERAGE USAGE OF PSYgPLOGICAt, ABSTRACTS IN THE 2-3 MOMEH PERIOD PRECEDING REGISTRATION Number of Registrants J Frequency | of Usage* | u II • » — • Users 1 0.7% Nonusers 3 0.7% 1 Total 4 A 'B 0.7% 1 0.7% 2 0.2% 0.3% 33 * 5.5% 126 21.1% 76 12.8% 79 13.3% 203 34.1% 73 12.2% 4.6% 1 C D V 12 8.4% 28 19.6% 21 98 21.6% ; 62 13.7% 61 13.5% 150 33.1% 57 12.6% 453 E , ,. -. 14 :'" * 9.8% 18 12.6% F G No Answer 53 37.1% 16 11.2% Total 143 100% 596 100% 100% 70 Table XIV indicates registrants' anticipated use of Psychological Abstracts in the 2-3 month period following SUPARS registration. The data shew that, although previous usage of Psychological Abstracts was low (Table XIII), both users (38.5%) and nonusers (44.4%) most frequently responded that they anticipated using P^chological Abstracts several tiroes a month for the next 2-3 months. A slightly higher percentage of users than nonusers anticipated using Psychological Abstracts more often than several times a month. Again, these oata are conparable to last year's. (3) Recent and Future Use of Psychological jfostracts for Preparation of a Term Paper, Thesis, or Speech Question 14: Have you used Psychological Abstracts in the last 2-3 months to help you prepare a term paper f thesis, etc. Question 15: Do you need Psychological Abstracts in the next 2-3 months to help you find references or abstracts for a term paper, thesis, speech, etc., you are currently preparing or planning to prepare? Consistent with the data reported in Table XIII, data in Table XV indicate that over two-thirds of both the user and nonuser groups had not used Psychological Abstracts in the 2-3 months preceding SUPARS registration in preparing a term paper, etc. Similarly, consistent with Table XIV, Table XVI shews that over threequarters of both groups of registrants intended to use Psychological Abstracts in the months following registration to help prepare a paper, thesis, or speech. (4) Specific Need for Psychological Abstracts Question 16: What is your specific need for Psychological Abstracts in the near future — to find a specific reference or abstract, to keep up to date in one or two content areas, to survey the literature in general, to exhaustively review a specific topic or area, to find several current references in an area, other? As shown in Table XVII, approximately one-third of all SUPARS users responded that their immediate need for Psychological Abstracts was for exhaustively reviewing a specific topic or area. Approximately one-quarter of all nonusers also indicated this choice. Among the users, another 38% indicated that they would use Psychological Abstracts to find several current references in an area or to keep up to date in one or two content areas. Forty percent of the nonusers indicated that they needed Psychological Abstracts to find several current references in an area or to survey the literature in general. Data from last year indicated that the mjority (61%) of registrants 71 TABLE XIV ANTICIPATED USAGE OF PSYCHOLOGICAL ABSTRACTS IN THE 2-3 t MO!*TH PERIOD FOLLOWING REGISTRATION ! Number of Registrants Frequency of Usage* Users Nonusers 3 0.7% 4 2.8% 2 0.4% 77 22.4% 17.0% 201 38.5% 44.4% 73 9.1% 16.1% 50 12.6% 11.0% 31 7.7% 6.8% 16 6.3% Total 143 100% 3.5% 596 100% 100% 25 4.2% 42 7.0% | 68 11.4% 86 14.4% 256 43.0% 109 18.3% 0.7% 6 1.0% Total 4 0.7% A i ! 1 c 32 J D E 55 13 1 F 18 G ] No Answer 11 9 453 i *A B C D E F G More than onoe a day Onoe a day Severed times a week Several times a month Onoe a month Less frequently No time 72 TABLE XV PAST USE CF PSYCHOLOGICAL ABSTRACTS FOR PAPER, TBESTS, AND SPEECH PREPARATION 1 Indication of Usage i 5 t • 1 • • . • .1 •• ••-. . I I 1 Number of Registrants Users 39 27.3% Nonusers 105 23.2% 318 65.0% 11 7.7% 143 100% 453 100% 30 6.6% 70.2% 41 6.9% 596 100% 411 Total 144 24.2% YES j NO | 93 68.9% 1 i No Answer TOTAL [ • 73 TAELE XVI ANTICIPATED FUTURE USE OF PSYCHOLOGICAL ABSTRACTS FOR PAPER, THESIS, AND SPEECH PREPARATION Indication of Usage Users Number of Registrants Nonusers 1 Total V ' ""•"" YES NO ! _, 108 75.5% 29 20.3% 6 4.2% 351 77.5% 70 15.5% 32 7.1%, 459 77.0% 99 16.6% 38 6.4% •No Answer TOTAL 143 453 < : 100% 596 100% , loot . 74 TABLE XVII ANTICIPATED TOPES OF USE OF PSYCHOLOGICAL ABSTRACTS* r •' Number of Registrants Type of Usage** Users Nonusers 118 12.3% 14.0% 136 18.4% c Total 150 13.6% 184 | 16.7% 201 A B 32 48 16.2% 165 36 13.8% 81 31.0% 19.6% 224 26.6% 173 226 20.6% 16 23 1.9% 9 13 1.1% 841 1102 100% 305 18.2% j 27.7% D E F No Answer 53 20.3% 7 2.7% 4 1.5% 20.5% 2.1% [ 1.2% Total 261 L_ i 100% 100% *»espandents were permitted to indicate multiple responses to this question. Column totals reflect these multiple responses. **A To find a specific reference or abstract B 3b keep up to date in one or two content areas C To survey the literature in general D To exhaustively review a specific topic or area E To find several current references in an area F Other 75 TABLE XVIII PREVIOUS EXPERIENCE WITH COMPUTER TERMINALS J Amovint of Experience Number of Registrants Users A lot 16 11.2% Some 67 46.8% None 60 42.0% No Answer 0.0% Total 0.0% 0.0% 271 59.8% 159 35.1% 331 55.5% Nonusers 23 5.1% 226 Total 39 6.5% 37.9% 1 143 100% 453 100% 596 100% anticipated using Psychological Abstracts to exhaustively review a specific topic or area. Although this category was the most frequently reported this year also, responses tended to be less concentrated in this category, instead being more evenly distributed among categories A-E. d. Computer Experience of Registrants (1) Previous Experience with Computer Terminals Question 17: What previous experience do you have with computer terminals — a lot, some, none? As indicated by Table XVTII, users as a group have had more experience with ocnputer terminals than have nonusers. Over half (58%) of the users reported a lot or some experience with computer terminals, while only 40.2% of the nonusers reported such experience. A greater proportion of this year's registrants reported some experience with computer terminals; only 25.8% of last year's registrants reported some experience with terminals. 76 TABUF! XIX PREVIOUS EXPERIENCE WITH COMPUTER-BASED RETRIEVAL SYSTEMS Amount of Experience User's"' A lot 9 Number of Registrants Nonusers 8 Total 17 6.3% Some 38 26.6% None 96 67.1% No Answer 0.0% 6 342 97 1.8% 135 21.4% 438 75.5% 6 1.3% 2.8% 22.7% 73.5% 1.0% 143 100%- ;' 453 100% 596 100% I (2) Previous Experience with Conputer-Based Retrieval Systems Question 18: What previous experience do you have with ocnputerbased retrieval systems — a lot, sate, none? The majority of both the user and nonuser groups reported no experience with cxatputer-based retrieval systems, A greater percentage of lasers (32.9%) than nonusers (23.2%) have had a lot or some experience with retrieval systems. These data are similar to the comparable data from last year's registrant population. e. Surcmary In summary, the total nunber of registrants this year was 596, with 143 users and 453 nonusers. Approximately 16% of this year's registrants had registered with SUPARS last year. Registrants from Education, Psychology, and library Science accounted for the largest proportion of the registrant population. In general, this year's data revealed few major differences between users and nonusers, with the exception that users tended to have had more previous experience with cxuputer terminals than nonusers. 77 Finally, coirparison of this year's registrant data with last year's, shows that, except for size and rate of registration, data configurations for both groups are highly similar, A larger proportion of current registrants, however, did report some previous experience with ooqputer terminals. This last trend may be accounted for by the re-registration of 95 users from last 3ar. 2. PUBLICITY - Margaret Mucia Publicity this year was aimed at the canpus community only, whereas last year the Syracuse netropolitan area was covered. A total of ten display ads (6 column inches) were ran in canpus newspapers (The Daily Orange and the Syracuse University Record.) Five classified ads were also run but additional ones were discontinuea on October 29, 1971 as they were deemed non-effective. M l these ads were similar in scope and composition to those of last year. Large posters (22" x 28") were placed in six key locations on canpus with an attached box of registration forms. Approximately 1,000 forms were made available in this manner. Upon request, a registration form was mailed to anyone who could not conveniently dbtain one from any of the canpus locations. Upon receipt of the form by the SUPARS office a Users Manual was sent to each registrant. Another avenue of advertising pursued was the distribution of letters to faculty >nd graduate students in those departments most likely to be interested in SUPARS. TVto seta were distributed, one being an introductory letter briefly explaining SUPARS, and the second requesting faculty members to announce to their classes the availability of SUPARS. Attached to all letters were registration forms. Nine slide-tape demonstrations were given by the SUPARS staff to various classes such as Introduction to Information Technology, Social Psychology and Foundations of Instructional Technology. 3. 4220 I0GS - June Brewer During the six weeks that SUPARS was operational, a telephone number was known to the users for them to call for aid with problems. This number (extension 4220) was iranned from 8 a.m. - 10 a.m. and 7 p.m. - 9 p.gu (Monday through Friday) by various staff members who were trained in the use of the system. Each incoming call was logged on a special log sheet. The problems were numbered in sequence as they were received; and a description of the problem and the action taken to solve it by the staff member were included. Data from the original log sheets were categorized and summarized by day in Log Summary A (Table XX) Log Surrmary B (Table XXI) categorizes and summarizes the data from Summary A by week and gives totals for all categories. Log Summary B will oe described in this section, column by column, as to the patterns of calls and the information exchanges carried on via the telephone aid service. 78 a. Description of Log Summary B. Coiuim 1: Vfeek In this coluron, each of the six weeks of SUPARS university-wide operation is designated by the dates of that week. Column 2: Total Nurtfeer of Calls 'Ihe total number of calls received per week on extension 4220 is logged in this column. It can be seen that telephone calls ranged from 32 (a three day week) to 30. During the six weeks of operational service a total of 402 calls from users were recorded. The mean nuntoer of calls per week was 67. Tfae totals across the week are often greater than the nurtber of calls because seme users had more than one problem. Column 3; General Information All calls for operation hours and requests for registration forms are included in this column. The total nurtber of calls reaeived in this category was 86, the second largest total for any category. Coluron 4: User Problems "*• .•• A user problem was defined as a problem originating with the user rather than one originating with the SUPARS program or with the Syracuse University Confuting Center and its equipment. Examples of user problems are: a user needs help in obstructing his search properly; a user cannot sicyi onto the system because he has omitted a line of input. Column 4 indicates no particular trend over the six week time period in the frequencies of this type of call. Cadis ranged from a lew of 5 (during the first week) to a peak of 13 per week. The total nuriber of calls in this category was 67. Column 5: PA UNDEFINED When the staff at the carrputing oenter failed to load the SUPARS disk packs at the time the system was scheduled to begin its daily operation a "PA UNDEFINED" message was printed out to users attenpting to sign onto the system. Also, after an ABEND (defined later) the "PA UNDEFINED" message would go out until the ABEND was corrected. Column 5 logs the nunber of calls per week pertaining to this problem. The frequency of calls in this category was faiily low and it is assumed that the oonmand was self-explanatory and thus users did not need to call and find out what it meant. This category accounted for 37 calls* Cojuran 6: ABEND An ABEND is a system error which causes the SUPARS oonputer program to stop processing. Cadis in this category were inquiries requesting information of system malfunction resulting from ABENDS. The second week of operation resulted in the greatest number of cadis, 19, but there were continuous 79 queries throughout the six.week period and resulted in 38 total calls. Column 7: TOO MANY USERS The "TOO MANY USERS" command was printed when all slots in the queues . * r filled; the user had to keep trying to sign on. Calls of this nature %ee resulted in 19, the srrallest total for all categories. It can only be assumed that the command was understood by the users that did not call# because this was one of the mo6t frequently printed commands to carte from the system. Column 8: System Status When a user inquired if SUPARS was or would be in operation that particular day, a call was logged into this category. The first two weeks of the university-wide servioe resulted in 68 calls in this category because of tha problems SUPARS had in debugging the system. If you eliminate the first two waeks, the x was 7.5 calls, from 30 total calls. However, by including the first two weeks for a total you reach 98 calls, which was the largest number of calls in any category. Column 9: SUOC Problems Figures in this category refer to user calls pertaining to system nonfunctioning which was attributable to cxxiputing center problems, i.e. equipment trouble, social security numbers not accepted, or general system failures. Frequency of calls per week ranged from 0-22; the total number of calls in this category was 63, the fourth highest number of calls in any category. Column 10: SUOC Messages Calls made to the SUPARS office by the oonputing center staff members were logged in this column. Very infrequently made, these calls usually informed SUPARS of impending delays in the loading of the SUPARS disk packs or of operating system trouble. Only four such calls were received during the six weeks of SUPARS university-^tide operation. b. Summary By allowing a user to know that there is "scneone out there who will help" is of primary importance in a retrieval system which can seem to be too mechanized for the uninitiated. During the last three weeks of SUPARS operation the largest group of calls were for information on why things weren't working properly. The problem was with the Syracuse University Computer Center, which was beginning the change-over from the IBM 360 to the IBM 370 conputer. Nothing could be done about the situation but users were able to relieve their frustrations by calling extension 4220 and hearing what had caused the bed conditions. It is felt that without the telephone aid servioe the premise of SUPARS — to make available a service to individuals — would not have been as effective. 80 "T C O r w 3 P w CM ro in v> x 00 CM r* B ro CO in VD £fe CM CJ N ro in co ro CM in CM CM 00 I $ CTk O CN CM CO oo CM in CM a CM CO I h 11 81 1 I in 5 H I 13 Is £3 £3 05 co m CM V> £ 00 ft) 00 CM m C M m CM in LO CM if) CM en 55 CM ro CO CM CO <\ in t-1 £^5 vD CM KD CO cr\ CM CM I CO CM I CM I in <7* CM O CO & M CM I CM I £2 3? £2 CO i CM gs CM CM 82 CM i *y CO e n ^» r* CO co in co CM CM H rH rH ^* rH rH H rH .?• rH ff i- " -. • - VP CM CM CM ro co CO rH co CM rH CM • co CO rH CO rH rH CN J - oo • rH ^ rH rH CM •H CM rH a\ Wednesday | 12-15 1 Thursday i 12-16 Thursday 12-9 Friday 12-10 Monday 12-13 9 | 12-14 83 Friday 12-17 8 CN CO CN CN ex en 5,3 O CO CM CN CO Csi ro <> ^ cr\ to c^ CN ^O CO LO CO kO Lf) VD CO O rH J in CN m rd V> £ in in CO CN CN CO id CO o CO (N CO CN o in o> l 1 rH rH rH rH rH rH ! I CN ^ CN CN as CN CO O VD r H CN CN co r^ rH rH I I CN CN rH rH &3 I t r H CN 84 4. STATPAC - Margaret M u d a and Michael McMullen In cxaijunction with the telephone aid service, the telephone interviews and the semantic differential, STATPAC was used to gather information on the user. STATPAC is a specially developed computer program used to collect, store, and retrieve all elements of user interaction. A flow chart of the data flew and accompanying documentation is given in Appendix IV. STATPAC was distinguished from the other three methods because it was an unobtrusive general izable data gathering retrieval program. Because STATPAC was an experimental program under continual revision, data recorded was not always 100% complete. As validation checks of output data were made with actual user interaction, program improvements were made to correct the system. STATPAC was designed to show growth patterns as well as usage patterns. Growth was most evident in the nurrber of lasers who entered their social security number to sign onto the system and the nunber and types of search inquiries entered. As shewn in Table XXII, the data indicate a definite increase in the user population and the nunber of searches. (This will be discussed in greater depth below.) It should be noted that because of the experimental nature of SUPARS, fluctuations in usage due to prograrmting changes, 06/360 computer difficulties, and variations in actual use patterns were reflected in ah uneven growrth rate. • * . STATPAC consisted of two summaries, Summary 1 and Summary 2. The summaries were developed by the project staff to monitor data about the most important parameters of system operation. These summaries are only two of the many configurations available. The distinguishing feature between the two was the different aarnbinations of fields included. For exanple, Summary 1 included the following fields: Social Security Nunber, Log Nurrber, Type of Search (A, AS, A10, Date, Clock Tine, Terminal Number, Input Lines and Output Lines. A A is the character on the terminal used to initiate the different types of searches. A A gives access to the document data base; a LV to the vocabulary data base; and a AS to the search data base. The primary sort was by social security number, with the secondary sort by log number, each in ascending order. a. Summary 1 Exanple of a line from Sunmary 1 Sgj 36XXXXXX0* LOGN DELTA 7444 DEL CALENDAR 11/18/71 OflCK 0.00 TER 52 INPUT LINES Ll L2 L3 L4 PSYODLOGY COMPUTER (?) ATTITUDE Ll and L2 and L3; OJEFOT LINES 13399 44 *The social security number is not given in its entirety in order to insure the user's security and to protect against invasion of privacy. 85 In Table XXII weekly totals are given for the five weeks SUPARS was available for canpus use. The contents of each column are discussed in detail below. The letter preceding each section refers to the corresponding letter for each column in Table XXII. The data was collected on the IBM 360/ 50 OS. (a) Social Security Nurrber (SSN) Each tine a user signed onto SUPARS his social security nuntoer was recorded along with his interaction with the system. An increase in the number of people who used the system is evident from this column. Although sate people may have used the system in more than one week, the overall user rate increased except for the last week. The decrease evident in Week 5 can be explained by difficulties arising from the Syracuse University Confuting Center, and not from the SUPARS program itself. The mean over the total period was 50 users per week. The total number of times social security nunbers were entered durings its 5 weeks of operation was 250. (b) Nuntoer of Log Numbers (LOGN) Each timfe a user typed a delta (A) a log nuntoer was generated. As there were wore users on the system during the second, third, and fourth weeks, there were an increased nuntoer of log nunbers generated. The mean was 502 per week. (c) Number of Terminals (TER) Syracuse University has 100 IBM 2741 terminals located in buildings on canpus, but the highest use during any one week occurred during Week 4. There were a total of 42 in use at that tine. This can be explained in part by the fact that some users found it difficult to sign on due to the high use of the system, and consequently as one user conpleted his searching, another would begin at the sane terminal, eliminating the sign on procedure. The mean was 37. (d), (e), (f) Nuntoer of Deltas (DELTA) (DDB-Document Data Base, SDBr-Search Data Base, VDB-Vocabulary Data Base) As a query was begun, STATPAC logged the type of search A, AS, LV. The nuntoer of A searches far outnumber either of the two other options, AS or bV. During the first four waeks the LV data base was rot in operation and therefore could not be searched. The mean of A searches was 461 and AS searches was 32 over the five week period. Additional data in S T M P A C includes (a) the nuntoer of search inquiries initiated by a user, (b) the ratio of the nuntoer of inquiries made by one user to the total nuntoer of inquiries. For exanple, one user initiated 120 inquiries and accounted for .05 of the total searches. The total nuntoer of searches initiated was 2399, and the total nuntoer of different users who initiated search inquiries was 143. There were 17 users who initiated one search inquiry. Of the 143 users of SUPARS the mean number of inquiries initiated 86 * > 9 $ CM CD in o ON ON ON CM 3 rd 13 <§ m o> in ON in in in o CM 8 i •8. O CO fti 0 m CO o CM 00 en 00 i i 3I c^ o in O CM ON ON ON VD ON en in CM I «s in in an CM i 6 i V> £ \D O in CM o rd oc 5 o CM CM a oo 5 < <3 I I CM CM I CO < N *r in in i 87 ! S § was 17, although there were wide ranges of use, from those who used the system only onoe to twice to one user who initiated 120 search inquiries. b. Sumtary 2 Sumtary 2, which sorted all other fields by date, consisted of the following fields; date, log number, type of search, social security nuntoer, terminal nutter, clock time, CPU time, cost, maximum items possible, nutter of items printed, and input and output lines. The data presented in while SUPARS/DPS was under The fact that the data was search experts and not the XXII. Table XXIII is the most current and was gathered the operational control of the IBM 370/155 OS. gathered from search inquiries performed by staff canpus users distinguishes this table from Table 2. The following explain the column headings that are shown in Example Those which are the same as the one in Example 1 are duly noted. Exanple of a line from Summary 2 DATE 11/30/71 Date LOGN DELTA 7454 A SSN 31XXXXX00 TER 52 CLOCK CPU POST MIP 2101 1 4.45 43 NPRINT 20 ihe date for each day of operation was recorded in ascending order, in the first oolunii. Social Security Nuntoer (SSN) Ihis column is the same as in Summary 1. Terminal (TER) This ooliitn i s the same as in Surtmary 1. Clock Time ( L C ) COK i ihe clock time was measured in the same manner as CPU time, from the time a user struck a A to the time he hit another A or typed )CFF to disconnect from the SUPARS program. It is the total time a user was connected. CPU Time (CPU) The Central Processing Unit time of the ocnputing system was recorded. Cost (COST) Cost is the total cost of each particular log nuntoer. 88 TABLE XXIII MEAN CPU TIME AND COST OVER THE THREE DATA BASES 1 (Delta [^Number of initiated search inquiries Mean CPU Tine Mean EXCP Count* Mean Cost Document Data Base AS Search Data Base LV 824 5.45 407.92 1.16 216 0.24 67.45 0.13 Vocabulary Data Base 318 0.11 41.37 0.08 * EXCP count is a measure of I/O activity plus any internal channel 89 Maxiinum Items Possible (ME?) The term Maximum was used in the Maximum Items Possible message because DPS limitations cannot handle searches with complex operator's. Clhose other than AND, OR or NOT), without processing the search a second tine. Therefore this number is the initial count and is not always the exact nuiriber a search may retrieve. If very broad inquiries are input, this column will reflect a large number of Maximum Items Possible, whereas if a very specific topic were being searched, the Maximum Items Possible would be very small. Therefore the number of searches during any one week may not always correspond with the number of naximum items possible. For exanple, if in one week the topics searched were of a very specific nature, the total surmary of the number of items possible may be low. c. Summary By means of STATPAC many variables could be studied regarding the user and his interface with the SUPARS system in an unobtrusive manner without having to observe the individual directly. These variables were printed in fctfo different summaries, Summary 1 and Suninary 2. Summary 1 shows growth by increased number of users signing on, and the increased nunber and variety of searches. Summary 2 indicates growth by nunber of users in relation to time of searches and the costs of those searches. STATPAC is a good example of the wide range of variables and combinations of those variables which can be studied and analyzed without confounding user data by the reactions an individual generally makes or withholds when his behavior is being observed. This study and analysis will hopefully lead to a better understanding of the user and his needs. 90 5. TELEPHONE INTERVIEWS OF SUPARS REGISTRANTS - Jeffrey Katzer One of the techniques we used to assess reactions to SUPARS is the structured interview (other techniques are discussed elsewhere in this report.) The major objective of this study was to obtain reliable estimates of the reactions of users and registered nonusers toward SUPARS. The survey was planned to shed light on several specific questions. 1. Can we identify the variable (s) which differentiate the population of registrants into users and nonusers? 2. What mechanisms are effective in the education of SUPARS users? 3. How often and to what extent is the system wanted? 4. What are the major problems encountered by users? 5. How do users judge the quality of their researdi output? 6. How do users and-nonusers characterize their reactions toward the -system? /•. •* In an atterrpt to obtain user's reactions during the first year's operation, Frierson aonducted lengthy personal interviews with a random saitple of 20 registrants (SUPARS 1970 Final Report). A major goal of that study was to develop and test the interview guide, and for that reason we were willing to tolerate the small sarrple size. This year, we wanted to increase the sanple size to increase the reliability of the information. For practical reasons it was necessary to change the format from a personal interview to a phone interview. The interview gii.de, pretested last year, was also changed in order to incorporate the differences between the 1970 and 1971 SUPARS system as well as the new phone format. Several drafts of the interview guide were prepared and pretested. All of the interviewers were trained in the use of this instrument. The phone interview was conducted of SUPARS0 registrants during the week of Deaentoer 13, 1971. By delaying the survey until the last week of service to the university,1 we hoped to maximize the nuraber af registrants contacted who would be users. ttofortunately this delay affected our completion peraentagb because several of our potential interviewees oould not be contacted because they left town early far the upaorreLng holidays. X A later decision extended SUPARS service after the University's Christmas recess. 91 TABLE XXIV FREQUENCY OF CCMPLEIED INTERVIEWS Conpleted Nonoonpleted Total Users Nonusers Total 35 28 63 18 19 37 53 47 100 Our first oonaern was to try to determine if a response oonpletion bias was present, and if so, the nature of the bias. Within each category (user and nonuser) the oonpletion response rate was at least 60%. While not as high as desired, this percentage is a minimal rate to give us same assuranae that those who conpleted the interview represented the total sanple. It should also be noted that the percentage of users and nonusers with conpleted interviews (55% and 45% respectively) corresponds closely with the overall percentage of users and nonusers in the sairple (53% and 47%). Lastly, it should be noted that none of the 37 nonoonpleted interviews were due to an active refusal to be intervievted. Every person in the sanple contacted, conpleted the interview. As a further check on possible response oonpletion bias, we conpared the information reported on the registration forms of those oonpleting the interview with those whom we could not contact. Among the users we found no disoernable differences in terms of academic status, departmental affiliatioa, or anticipated use of PsycholoyLcal Abstracts. Similar results were found among the nonusers with one major exception,""academic status. All of the seven faculty nonusers conpleted their interviews. This probably reflects the fact that the faculty could not leave town for the holidays before the official last day of classes (which was the last day for internviewing) . Students usually leave for the holidays earlier; in fact 37% of ail non-respondents were students known to have left town. A response bias is evident: faculty are over represented in the group of nonusers who conpleted their interviews. This should be kept in mind when interpreting our findings. However, it will not seriously affect the interpretation of any differences between users and nonusers — our major area of interest. The remainder of this report of the telephone survey will be based only upon the aonpleted interviews (N=63). 92 a. Ihe Sanple A random sample of 100 registrants was drawn from two separate populations. Using STATPAC (a log of the system which collects, stores and retrieves all user interaction with SUPARS) we obtained a list of users2 of the system. At that time 97 registrants used the system at least once.3 From the list of users a random sample of 50 were selected for the phone survey. Another random sanple of 50 were selected from the remaining 470 registrants (nonusers.) During each interview several questions were asked to determine if the original classification of user or nonuser was correct. Because several days had elapsed between collecting the list of users from STATPAC and the obnpletion of the interviews, and because of the limitations of STATPAC3, we decided to define a user as a registrant who signed onto SUPARS, as ascertained by the interview. That is, a user is defined as one who responded positively to Question 3 (in Section I) of the interview:H They were able to sign onto SUPARS at least once. Ctoss-echecking the interview with STAEPAC we found three registrants originally designated as nonusers, who had used the system. Ihe final survey, therefore, was ocnposed of 53 users and 47 nonusers. b. Completed Versus tfanoonpieted Interviews Of the 100 possible interviews, 63 were cotpleted. The breakdown of completed and nonccnpleted interviews between the users and nonusers is presented in Table XXIV. 2 STAH>AC defines a user to be anyone who signed onto SUPARS successfully. We did not check STATPAC to see if the user actually made any SUPARS searches. However, a check was made during the interview to see if the STKEPAC-designated user actually made searches — see text. This matter is a conservative lower limit. As discussed elsewhere in this report, we have evidenae to suggest that once seme users signed on they left the system operating upon completing their personal searches. Other users then entered searches which were collected by STATPAC under the social security number of the original user. 3 *A copy of the phone interview schedule and a registration form are included in Appendix V. 93 TABLE XXV ACADEMIC STATUS OF USERS Faculty Undergraduate Student Graduate Student Staff AdminisOther trator Total Frequency of Users Percentage of Users 1 3% 8 23% 26% 23 65% 50% 1 3% 5% 1 3% 1% 1 3% 3% 35 100% Percentage of Registrants1 ! 15% 100% 1 Since registrants were permitted to indicate more than one status position, the frequency of registrants in these categories totals to mace than the 567 registered when the phone survey began. c. Users; Description Thirty-five users ooirpleted the phone interviews. Academic status of the interviewed SUPARS users is presented above in Table XXV. While it is not surprising that the great majority (88%) of users were students, we anticipated a greater percentage of faculty users. If SUPARS presents an information service for those doing research, it seemed reasonable to expect more usage by faculty. As shown in Table XXV, approximately 15% of SUPARS registrants were faculty but they made up only 3% of lasers. We have aollected seme hearsay evidence which suggests that for at least some faculty, graduate students were used to test the system's availability and usefulness. Moreover, it is highly likely that graduate students performed SUPARS searches for seme faculty members. This pattern of research assistants (in this case graduate students) making searches for researchers is understandable, and occurs frequently in at least one other interactive bibliographic reference retrieval system (SUNY Biomedical System) Users were affiliated with 10 different academic departments or area specialities. The largest group of users came from Library Science (10). Ihis probably reflects the fact that the SUPARS project was administered by the School of Library Science. Severed instructors in the school used SUPARS 94 in their classes as a pedagogical tool — rather than a research tool. Both the Psychology Department and the Education Department had seven users each. Social Work, the next more frequent category, had three users. All other areas had one user each. Users found out about SUPARS in a variety of ways, Forty-one percent of the users mentioned learning about the system directly frcm our publicity efforts (newspaper notices, posters on caitpus, slide presentations.) Six users remenbered SUPARS from the service offered during the Pall of 1970 (S3Ql).b All of the users received a copy of the 1971 User's Manual upon registering.6 In general, they found the Manual well written and quite useful. Ninety-seven percent of users took their Manual with them to the terminal. The absence of an explanation of common error messages was the major oonplaint about the Manual (S3Q4 - 13). Thirty-one users remenbered receiving a Pocket Reminder Card (a folded 3 x 5 sized svranary of the main sign on and search procedures.) All registrants should have been sent one. The Pocket Reminder Card seemed to be less useful than the Manual; 39% of the users reported they did not find it helpful. Except for twb users statlhg that the print was too small on the card, no other ooftplaints were made. It is difficult to determine why the card was not used more frequently (S3Q15-19). As a backip for the Manual and the Pocket Reminder Card, the SUPARS staff manned a user-service phone (extension 4220) during the hours of SUPARS operation and for most of the remainder of the working day. Ttoenty-three users balled 4220 at least once; most users (56%) called less than four times during the entire SUPARS operation (S3Q22-23). Users judged the information supplied by SUPARS staff via extension 4220 to be helpful (78%), (S3Q24)7 d. Users: General Usage Patterns A breakdown of the frequency and duration of attenpted and actual terminal use is presented in Tables XKVI and XXVII. 5 Items in parentheses identify tiie interview question applicable to the results discussed. In this case it was Section 3, Question 1. ^Ihqugh we offered instruction of the use of the terminal and construction of searches, very few users requested such add. Our instructional plan was to rely on the User's Manual as the primary means for user self-education. 7 A nore conplete description of users, based upon infonration contained on the registration forms, and a log of the calls reaeived on extension 4220 have been presented in this section. 95 TABLE xxvi Once a Week Less Than Once a Week FREQUENCY OF ATTEMPTED USE OP SUPARS (S1Q1) Several Times a Week Daily Total Frequency Percentage 2 6% 17 49% 7 20% 9 26% 35 101% TABLE XXVII AVERAGE TIME SPENT USING SUPARS IN ONE SESSION AT THE TERMINAL • — « I I lllml 1 I I (S1Q18)l *.„ Less lhan 15 Minutes 15-30 Min. 30-60 Min. 60-120 Min. Total Frequency Percentage 0 0% 4 12% 8 24% 21 63% 332 99% lf Ihough SUPARS was available for four hours each weekday, these hours were distributed into two, two-hour sessions. 2 This question was only asked of the 33 users who were able to obtain output from at least one of their searches. The major lindtation in interpreting these data is the lack of control over the length of tiiae each respondent used SUPARS before the phone interview. Seme respondents might have been using SUPARS from the first day it was available. Others might have begun the day before the interview. It is possible that respondents in either of these groups oonpleted the sane number of searches.8 [ 0nae the output from STATPAC is available and tested, we will be able to determine the actual use (number of days# number of searches, and amount of time at the terminal) for each user. 96 Taking the information in Tables XXVI and XXVII at face value, it appears that three-quarters of SUPARS users tried to use the system at least onpe a week, and once they managed to sign onto the system, they stayed on it far at least 15 minutes — most of the time longer* A more detailed look at the responses of these twenty-one users who reported spending about one to two hours at the terminal each tine they used SUPARS raises sone questions* Ten of the 21 users in this category also claim to make only 2 to 5 searches on the average in one terminal session* Under good conditions it is not unusual for a user to oonplete several times that number of searches in a two hour period. It is possible that these users were doing a great amount of thinking about search strategies between searches. Or they might have had to wait for a large amount of output to be printed. Their likely explanations would be that either they ware teaching themselves how to use SUPARS and prooeded cautiously with errors, or they might have experienced oonputer problems with SUPARS per se or with the total operating system. Table XXVIII presents the number of searches users estimated that they made in each terminal session and across all terminal sessions. TABLE XXVIII FREQUENCE AND PERCENTAGE OF ESTIMATED NUMBER OF SEARCHES MADE 1 In one Frequency terminal session. (S1Q17) Percentage Totaled for all Frequency cerminal sessions. Percentage (S1Q11) 2 6% 2-5 17 51% 5-10 9 27% 10-20 4 12% 20-50 1 3% over 50 0 0% total 33 99% 1 3% 8 23% 7 20% 3 9% 10 29% 6 17% 35 101% In general, the median number of searches made at each terminal session is in the 2-5 range.. Over all of his terminal sessions the user makes an average (median) of 10-20 searches. Both of these figures suggest a less frequent use of SUPARS than had been anticipated, (though sane information needs could be met with a few nuttoer of searches.) Certainly the delays in starting the system each day and the frequent software failures did not help increase the usage of the system. While all of the users vaere able to sign onto the system, 31 of them (89%) experienced some difficulty in signing an at least once. The different types 97 of problems experienced by users in signing on are listed in Table XXIX* TABLE XXIX PROBLEMS EXPERIENCED B¥ USERS WHICH HAMPERED ATTEMPTS TO SIGN ON (S1Q8) Problem Frequency No terminal available Keyboard frozen Unexplained Messages (Other than those below) SUPARS not available when it should have been (PA UNDEFINED message) Complete IBM/360 Operating system failure Area storing user's search filled1 (TOO MANY USERS message) 1 1 4 6 9 21 42 2 As each SUPARS user signs on, a 1200 byte storage area is set aside to hold each search as it was entered. Given a limited amount of oore available it was impossible to have an unlimited number of user storage areas. Eight of these areas were unavailable. Users who signed an after the last storage area was allocated were sent the "TOO MANY USERS" message. Forty of the forty-two problems mentioned by the users are software problems. T\*enty-five of the forty-two problems were SUPARS problems. It is amazing that users had the persistence to continue trying to use the system after so many problems. The frequency of occurrence of the "TOO MMflf USERS" message helps explain the "corrective" action taken by some of the users (described in footnote 3.) After successfully signing on to the SUPARS system other types of problems became possible. TWenty-one users (60%) experienced one or more problems while searching. The twenty-six problems reported by these users are classified into the four categories presented in Table XXX. 98 TABLE XXX PROBLEMS EXPERIENCED BY SUPARS USERS AFTIR SIGNING ON (S1Q10) Problems Frequency LSbn-operation; such as SUPARS or operating system failures Free text; such as user supplied keyword not in dictionary1 SUPARS interaction; such as uninterpretable output or operators not used or not working oorrectly Other problems; such as typing errors, keyboard jamming 6 4 12 4 26 1 Pursuing this type of problem further, we asked 33 users if they preferred a free-text system or one in which the vocabulary was more controlled. 42% preferred the free-text; 36% preferred the controlled vocabulary; 12% wanted both possibilities and the others had no preference (S2Q11.) Of these problems, 69% of them are software related and 58% specifically due to difficulties with SUPARS. e. Users: Reactions One of the purposes of the phone interview was to get users1 iitpressions of the quality of output retrieved by SUPARS. Thirty of the 33 users asked, felt that SUPARS generally retrieved the information they wanted. (S2Q1). We also asked those 29 interviewees who have used the printed version of Psychological Abstracts to ccnpare it with SUPARS in terms of speed, ease and quality of search output. Sixty-nine percent judged SUPARS to be faster, and 3 users who did not agree with this majority said that SUPARS would be faster if it worked all the time, or if he could always get on the system — see footnote to Table XXIX (S2Q5.) Most of the users (76%) considered SUPARS to be easier to use then Psychological Abstracts (S2Q6) • Responses to two questions asking lasers to judge the relevance of SUPARS retrieved output are presented in Table XXXI. 99 TABLE XXXI FREQUENCE AND PERCENTAGE OF RESPONSES: "Have you located more relevant information with SUPARS" Yes than you wofcLd have found using Psychological Abstracts (S2Q7) No Don't Know fjrpi^l Total Frequency Percent 15 54% 8 28% 3 11% 2 7% 28 100% which you probably would not have found without SUPARS (S2Q8) Frequency Percent 18 64% 4 14% 6 21% 0 0% 28 101% One user felt that "quality" of abstracts from Psychological Abstracts is better than frcm SUPARS, but for "quantity" the situation is reversed. Another user, giving an affirmative response to the second question, felt that he would not have expended the extra amount of effort to make manual searches in Psychological Abstracts. While responses such as shown in Table XXXI are encouraging, it most be remembered that these data are based upon user estimates, not behaviors such as actual use of retrieved abstracts in term papers. Parenthetically, it is interesting to note that cognitive consistency theories would predict a favorable user response, such as shown in Table XXXI, if the retrieved abstracts were nonrelevant and user-SUPARS interaction was a negative experience. As described below many users found SUPARS frustrating. (Also see Table XXXIII) Users were asked to state those aspects of SUPARS they liked irost and liked least. Of the 46 positive responses, about one-third mentioaned speed or quickness of using SUPARS. Other representative positive responses were "fun," "thorough," "don't have to copy printout," and "dig ocnputers." (S2Q9). Of the 46 least-liked items, 22 of them (48%) were related to the relative unavailability of SUPARS, (poor service hours, too few days, too many users, too many system failures, etc.) Other typical negative responses were "irrelevant output," "weird commands," "more expensive," and "don't knew what words to use in a search." (S2Q10) 100 In order to get an overall assessment of user's reactions to SUPARS, much later in the interview, wa asked each user to state two or three adjectives \vhich best described his experiences with the system (S3Q27). Each user, of his cwn aoaord, gave at least one negative and one positive response to the question. Of the 78 total responses, 21 were the sane adjective, "frustrating." The remaining responses varied greatly, e.g. "cantankerous," "challenging," "helpful," "tine consuming," "notivatinj," and "useful." The picture that seems to be painted by user responses is a conflicting one. In general, they find it frustrating to use, but persist because they are intrigued with its speed, novelty, and potential. There is no discounting the negative experiences users had with the system — especially the difficulty of getting signed on when they wished to. However, most of the users (94%) said they would register for SUPARS if it became available again *S3Q26.} And, at the end of the interview, vfoen the user was asked if he had any questions, eleven of the 23 who had, wanted to know if the SUPARS service oould be extended into January, 1972 (S3Q28). Several users volunteered to help SUPARS keep running past its deadline by asking their chairmen to contribute deparbnental computer funds to the project. f. Nonusers; Description Ttoenty-. a 2. ghl 5 >.. 1 H 8 3 2 Mi Ms H"BS| $ mB o to 8 O I CO 10 ON 2 ON 4 "^ 3 S 103 u In contrast to the "tryers", the nonusers who never attempted to sign on to the system (N=19) had uniformly positive adjectives about SUPARS: "accessible/' "innovative/1 "personalized/1 "good/" "great service/1 "interesting/1 and "exciting idea." It seems that all pre-use contact with SUPARS via publicity, Users Manual, etc. helped produce the attitude we wished — and thought necessary for a openly-accessible/ computerized system without intermediaries. This positive feeling, however, was changed by actual initial contact with the system. Inability to sign on successfully when the potential user wants to and the inability to remain on the system once having signed on, seemed to be the major contributors of the negative attitudes toward the system. These data are not of the kind necessary to shew a causal relationship between attitudes toward the system and contact with it. Nonetheless, Table XXXIII is interesting to consider. g. Summary Listed belcw are the conclusions drawn from this work in response to the six questions of interest identified at the beginning of this subsection. To help put these conclusions in some perspective, each will be presented with the corresponding findings from,last year's study conducted by Frierson. As mentioned earlier, only 20 registrants were interviewed last year. With such a small sanple size it is difficult to identify reliable findings. For this reason we will only be concerned with broad "trends" apparent over several variables or highly discrepent findings. 1. This year, the only observable difference between users and registered nonusers was academic status: faculty members are more likely to be nonusers. An analysis of several variables from last year's study could not differentiate users from nonusers. 2. In both years of SUPARS service, the reactions of registrants to the publicity-education-aid package (e.g. Users Manual, phone extension 4220, etc.) was, for the most part, highly favorable. 3. Last year, SUPARS was available to the public for one four hour time block each day (this year SUPARS was offered for 2 hours in the morning and 2 in the evening.) Even with longer terminal sessions none of the 12 users interviewed last year made more than 4 SUPARS searches per session. Over all terminal sessions, 58% of last year's users made 10 or less searches. The corresponding figures for this year are, 57% of users made 5 or less searches per terminal session, and 46% made less than 10 searches over all sessions (See Table XXVIII). Both figures are similar over the two years, alley are puzzling because they are much lower than we anticipated and causal obervation of user behavior indicates. Several possible explanations a personal oopy of the Users Manual, we deposited several copies in various high-usage locations around canpus. Registrants without manuals were told the location of these referenae copies. 104 for this low USG v**re mentioned earlier. It will take a thorough analysis of STATPAC data and seme systematic observations of users at terminctLs before v a are confident in these data, and possibly have seme * e-rfplanation for their size, 4. Problems encountered by users were, for the most pari:, the same in last year's operation and this year's. While this seems to indicate that the system was not inproved between operations, that is not the case. /Approximately 32% of all problems experienced by users this year were not due directly to SUFARS (e.g. operating system failure, or SUPARS not loaded when user attempted to sign on) • The major problem encountered both years was the inability of a user to sign on because of high use (the TOO MANY USER message.) While this is counted above as a SUPARS problem it is a function of the size of core portion available to SUPARS which in turn is a function of demand on the aonputer by other interactive and batch systems. Inprovements were made in this year's system which decreased the number of SUPARS generated error messages. Offsetting this improvement was the fact that this year's operation could not get completely debugged before the system was offered to the public. Prograntrdng problems and drastically decreased turn-around time prevented us from meeting our planned time table for dewlqament. 5. A difference is apparent between quality of SUPARS* output as judged by last year's and this year's users. Last year only 50% of the users felt the output contained information they wanted. This year the corresponding figure was 91%, a sizeable increase (if reliable). It is possible that iirprowaments in the system accounted for this difference. Another possibility suggests that users rated output quality high in order to resolve an inconsistency between their registering for the system and the frustration using it. We have no evidence bearing on either of these possibilities, one way or the other. 6. General reactions of this year's users toward SUPARS are sumBrized in Table XXXIII. Attitude may be thought of as a function of the degree of contact with the system. Our public relations-educaticn-aid package creates a favorable, inviting view of SUPARS. Initial use of the system is often frustrating, and unless a user persists, he frequently acmes away with strong negative attitudes. Users who do persist recognize both the positive and negative aspects of the system. Comparable data from last year's interviews are not directly available. While users voioed praise and conplaints, there did not seem to be the intensity of feeling then as now. If we aonsider user's attitudes as measured by an attitude scale, last year's users, in general, liked the system. (Frlerson & Atherton.) 105 6. SEMANTIC DIFFERENTIAL - Patricia Moell One objective of the SUPARS research project has been to measure user reaction to an on-line interactive bibliographic retrieval system. Frierson and Atherton (SUPARS Final Report, 1970) reported on a survey of attitudes toward SUPARSvvrtiich indicated that registrants exhibited "a definite positive reaction toward SUPARS and catputerized information retrieval systems in general." The present report describes a replication of that study to determine if user reaction to SUPARS remains positive. A Semantic Differential was employed to measure user reaction both to SUPARS as a system and to the services provided by the SUPARS staff. This instrument is considered to be "a reliable means of measuring user reaction to the SUPARs system and perhaps to ccmputer-based information retrieval systems in general/1 (Frierson and Atherton) The Semantic Differential (SD) was used because it could identify independent dirnensions of users' attitudes toward the system. There were 20 SD's (one for each concept — see Table XXXIV); each SD consisted of 10 bipolar adjective scales.1 Seven equal intervals separated the ends of each adjective pair. Each respondent was instructed to readt to the concept by marking an interval between each set of adjective pairs. If his reaction to the concept was strongly reflected by an adjective, he was to mark an interval close to that adjective. Each adjective pair, therefore, became an attitude scale. The ten adjective scales and an example of a SD are presented in Table XXXV. a. Description of Procedure A SD package plus a preliminary information page and a direction sheet (see Appendix VI) was constructed for each respondent. In each packet the concepts were randomly ordered. To avoid response set, the bipolar adjectives were reflected (e.g.; good-bad became bad-good) in four different combinationsf so that 5 concepts had the same page layout of bipolar adjectives. For ease of scoring, however, the adjective pairs were presented in the same order on the page, perhaps creating a sequence effect. A SD packet was given to 29 students in the School of Library Science during regular class sessions. In addition, 73 SD packets were nailed to 21 Library Sctiool students and 52 Syracuse University students in other disciplines. It was planned that only students who ware users of the system would receive the packets but 10 recipients previously identified as users (through class 1 Though each respondent was asked to react to each concept with 19 adjective scales. The. additional 9 scales are being used in a replication of the original factor analysis (see 1970 Final Report, Section IV-B-5). These 9 scales were not analyzed in this report. 106 TABLE X X X W CONCEPTS USED IN THE SUPARS SEMANTIC DIFFERENTIAL PPCKET 1. Psychological Abstracts 2. 4. PA UNDEFINED TOO MANY USERS - TRY AGAIN SOON 3. Computer 5. 6. SUPARS MAXIMUM DOCUMENTS FOUND 7. 8. 9. 10. 12. 13. 14. JL5; 16-. 17. 18. 19. 20. Extension 4220 How I fourri out about SUPARS LI Myself and computers Usefulness of SUPARS to roe Constructing my search logically Computer Search Free Retrieval Service A Ttetfninal )EXEC PA Keyword LIST RECORD 11. OUTPUT Concepts 2 and 4 are default messages which tell the user he is unable to use SUPARS. Concepts 3, 9, 10, 13, 14, 15, 17 and 19 are related to ocxrputer-based search systems in general. Concepts 5, 7, 8, and 12 relate to general SUPARS features. Cbnoepts 6, U , 16, 18, and 20 are oonnnn oonponenets of the SUPARS usersystem acnrounications language. 107 TABLE XXXV EXAMPLE OF A SEMANTIC DIFFERENTIAL USED IN THE PRESENT STUDY TOO MANY USERS slow active good successful hot colocful beautiful heavy serious big ; ; : ; : ; ; : : : : : : : : : : _: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : __: : ; fast passive bad unsuccessful cold colorless ugly light humorous small :• : ' : : : : 108 vo m o in o i is 8& £ H « i«-*~« 8 V£> ro ro ft 1B < D en 0) (1) It is 3°« " 8 $& if I it •H 1 Q 6 *8 0) 4^ O D H Cfl i s 0) 109 I 1 CD 00 u > m CM CN to CO ro ro oo co CO CO CO N H ssax j Q T°3lI a) - H O VO VO r> vo 00 VO r H CO (*S -J£i£°I£dg co Ch CO in in CM CO CO CO • • • vo CO O CM VO ^ *<# in TJ* «3* co in CO CO • • • CO CO CO oo 9 M o m EH, 4M TnjssaDonsunl -Tn^s^aoong pgg-poogf 9ATSSg3>9AT^ I . 4J w G o CN (dm 3-H ^ o x s - ^ s r e j l •H U C 0) O . H > O4 P W CO o co & m o o CM m co ro C M C M 00 ^r co co CO CO co H oo.m in 00 ro rn ^0 co CO CM CO CO CM l-P rd U •H DlcO OJ-P O rd •H U (d tj> 0* -P o 2 (A CM O 04 CM 1 o C O < c o a) a) Q) 3 §•6 C M 3 3 rH E f + J CO o C p$ O H co -w c p -p u « CM w a D 4 ( U O U) 0) a d ) Q) 6 ^ m 8o E-PwOGwgcuwSi* 3 CO G Q) -P H a* 5 co g >i o o on am 4J M _y m rH • 2 a 0 M H vp CO • • rH • rH o H rH TT^^S-Bxg snojcouinH -snoxjcas ^A^aH-m^TT ision II dty TJ* •H C eo >i rH • o oo r f oo o o\ vo O ^r ^ oo r>* O O 00 00 r > en o o ro VO VO o LO r > rH o o VO 00 rH O rH o * 1.06 . 0.90 : 1.18 1.06 1.28 0.99 1.24 0.88 0.98 1.13 1.11 VO ( N LO OJ •H rH rH 1.27 a* w > H U ^Tfefl -TnjT^nHag ssaxJcoxoo -XnjjeoxoQ PTO0-1OH imension I esirabilit H 'AIRS VO * rH ,Q a 1 ~Oi T S S ^ c J - . a A T ^ O V \ Q) rH O -H > 04 Q W CO 1.36 rnjssaoonsun < 1 -Xnjssaoons ppg-pooo Q ion tive w h3 H O •H to rd M-i o oo r o ro r o *a* rH rH rH OJ ^ rH in o *a* ro rH rH rH rH 00 «3* rH MOXS-^SPJ * > H5 rs ing ray logica search ieval nd out rH UPARS CU C O rH O O o C d LP ical C < -P D H 0) H 3 +3 O Sl W -P O rH Q) 3 0 U ( D < D * H < d r a - P - P ^ O ^ H - p H > C H 3 P H 6 4J «J D H -H 0 6+J C 0 O 0 2 >1 O 0 £ . IS! C C S < O D C H a), O a) > O O H - p ^ O OIH y. a) 6 £ s G PC C O O Q) C O 0 0 - H «W 4J C tf C O 3 H < C H 0 3 Oi M-l D < (D C O 04 X 0 C • » • O r4 O CO 00 G\ CN * O • • en o O H C 0 -H c 8 Q M ro o CN iH VO 00 0 0 O KO O O H O H • • • • <<* in rH i n 00 O co C o CM oo r^ CN ^ rH ^ CM CN I o in g 4J < D •H rH P C O £ C O C 0 •H T> < D M 04 0) g -H XI «8 C O •H M-l S3 >, -p •H 0 x a rH 03 U tq 4J ~ r H 41S g § 04 •5! T3 T3 M CdrH 'OrH -p <*H CO 0 W -H X < O H 0 § BDYQDNY1 aDYnDNVT JiTflYJaa kajindwoD •* snvans *s 117 e. Conclusions The results of this study indicate a slightly positive reaction toward SUPARS and ccmputerized information retrieval systems in general. They also indicate a more negative reaction to SUPARS and other systems by this year's registrants in comparison with last year's. The most positive reaction was recorded in the evaluative-specific dimension, lending credence to the assumption that the dimensions of desirability and enormity are less specif ically applicable to the concepts used. The standard deviations of all the means are relatively lew. This may indicate similarity of opinion concerning SUPARS and other corputer retrieval systems. Since most of the means tend to be near neutral (or 4), however, the low standard deviation probably reflects a high concentration of neutral reactions. An examination of individual scores would lend support to this conclusion. The results are generally in accord with those of last year's study, indicating reliability of the testing instrument. The overall less positive reactions recorded may be a result of actual differences in attitude toward SUPARS and computerized retrieval systems this year as compared with last. It would be helpful to compare the results of user reactions via the interview method for the two years to see if this conclusion holds true. (This is included elsewhere in this report.) f. Discussion Several factors nust be considered in relation to the results of the study. First, for purposes of scoring, bipolar adjective pairs were not randomly distributed on a page. Since an examination of individual concept pages would show that scores tend toward 4 in the middle and lower ranges of the page, it would seem that the dimensions of desirability and enormity might suffer fron the location of adjective pairs. Attitude pairs could be distributed randomly on the page with no loss of scoring ease by assigning numbers to each attitude pair in coribination with a 1 or 2 to indicate which way the score should be read. These numbers could be keypunched along with the score and then read by computer. Second, the difference in user reactions to concepts last year and this year might be a function of differences in service. Of interest is the fact that for both years computer is a more attractive concept than SUPARS, despite the fact that free retrieval service is the most positive concept of all. Perhaps the more negative image of the concept SUPARS has some causal factors. Third, the many neutral reactions might be an indication of a concept rather than a lack of interest. Extension 4220, which scored a 4.02 in enormity brought forth statements such it." In other words, neutrality may be a result of a lack of with the concept rather than neutral feelings about it. of lack of use for example, as "never use familiarity 118 Fourth, the fact that this year's division of concepts into dimension "as a result of last year's rotated factor matrix might render the results doubtful if a factor analysis of this year's results alter the ocnpanents of these dimensions. Last, the similarity in structure of responses over the two years indicates that the Semantic Differential tends to measure these concepts in the same ways. The less positive results of the study might well indicate, therefore, a greater dissatisfaction with SUPARS this year than last. 7. IMPLICATIONS AND PROJECTIONS - Kenneth Cook a. User Orientation The necessity for the integration of the user as a major part of designing interactive retrieval systems cannot be overenphasized. For the system that does not use intermediaries, user integration becomes of paramount concern. Consideration for the user goes well beyond just the design stage. Squally important is a need to inform, instruct, and educate people vdho are users and potential users. The final, and perhaps the most critical aspect of user orientation, is the necessity of determining user reaction in terms of attitudes, opinions, and behavior. The various technicjies used to evaluate user reaction to the SUPARS/BPS system indicated that problems related to the user and the user interface with the system were almost as prevalent as problems related directly to hardware and software component malfunctions or interruptions. A study dealing with free-text retrieval (4) found that over 85% of the problems and retrieval failures resulted fron either the search process or the user/ system interaction. What these findings seem to imply is that an important overlooked function in the development of a user-oriented retrieval system is the inclusion of a strong publicity/information and educatiorv^instruction effort. Tto make the assumption that people will be eager to approach v t a rit they might consider to be a mysterious system controlled by an unseen computer simply because the designers were enamored with the system is, at best, a guess, and at worst, the basis for potentially maintaining a little used or understood system. b. Publicity and Instruction Functions Because the boundary between educatiorv/iiistru^ and publicity/information is difficult to draw, and seme activities encompass both areas, a distinction between the two is itade here simply for descriptive purposes. The techniques mentioned below were found to have a positive effect on increasing the number of SUPAPS/DPS users and instructing users to be selfActivated and independent of an intermediary. 119 In the area of publicity/information, some of the most useful techniques related to making the existence of the system known and popular were (1) personally addressed letters announcing the upocming system to potential opinion leaders, (2) short, descriptive brochures explaining vdiat the system can do, when it will operate, how to get more information, who is in charge, and why the system is being operated, (3) posters, notices, or advertisements in-house organ, (4) small-group demonstrations and talks, preferably with hands-on terminal experience, augmented either with slides or movies and (5) locally postered examples of computer generated input and output to prepare users with examples of what to expect when they use the system. In the general area of education/instruction, demonstrations and handson experience with a terminal can be helpful when slanted to emphasize how to actually use the system. In addition, the education/information functions most useful included: (1) a special telephone number that users could call whenever they needed help with use of the system, and provided a search expert to talk with, (2) an easily available user's manual emphasizing self-instruction, with extensive use of input and output interaction, and a step-by-step procedure for developing search inquiries. The Users Manual was found to be one of the most important and helpful devices in developing a user population that could independently learn hew to use an interactive retrieval system, (3) a small, "Pocket Reminder Card" containing a summary of the essential elements of using the system that could be used in place of the manual when a user had acquired the basic skills of searching. c. User Control of Interaction In addition to the areas of publicity/information and instruction/ education, a user approach to. system design has to emphasize a maximum of user control in the interactive dialogue. User control implies that the interactive dialogue should provide an individual with a chance to constantly switch tactics at any point in the dialogue. For example, this might mean the ability to use a single keystroke to cancel an inquiry that was being developed and begin another, or the chance to use a single keystroke and junp from one data base to another. This ability to switch tactics would have as its goal the opportunity for an individual to make a decision and implement it at a terminal as quickly as he mentally makes the decision. • • The SUPARS/DPS system was implemented to provide as many decision/implementation points as possible in the user interaction. In effect, the user remained in control as much as possible and did not have to wait for lengthy oenputer replies or type more than one or two keystrokes to implement a change that he has just decided on. Another aspect of the user orientation to free-text searching without intermediaries means that an individual should have additional supply of potential search keywords available at any point in the interactive dialogue. The ability to easily obtain new words through a free-text vocabulary, or through previously stored search inquiries was a key user-related improvement in the current research. 120 d. Obtaining User Response One of the final requirements for developing a user orientation in designing and redesigning is the use of techniques to obtain user response that can be used as feedback to modify aspects of the total user/software system. An in£artant facet of obtaining user response is the need for using more than one method of data collection to insure that a single method, in itself, wauld not be a biasing factor in obtaining a certain type of response. The other related facet is the need for gathering more than one type of response or behavior in order to obtain more than one dimension of a user's response to the system, ftiis multi-method, multi-trait technique was employed in the current research and provided the advantage of avoiding a myopic viewpoint when analyzing the complexity of the user's response to an interactive system. Several generally accepted techniques of the social sciences have been employed to obtain abjective and verifiable data, such as a Semantic Differential Attitude Scale, personal, in-depth interviews, records of telephone inquiries, and unobtrusive measure of user interaction. e. Conclusion The difference between the ability to design a highly adaptive retrieval system and a highly adaptive system that is well known and maximally used might lie in the designer's ability to shift his focus from defining the "system11 as the hardware and software to an enlarged concept of the system that puts the user on an equal footing. Projected interactive systems of the future not able to support highly trained intermediaries will have to follow the axiom of "beginning with the user and designing outward." The beginiiing-with-the-user of SUPARS/DPS and its concurrent high use has been in great part a function of the techniques discussed in the previous sections. As Paisley (1968) points out "...Computer based storage and retrieval systems that sure not integrated into these social, political, and economic systems will be expensive, unused novelties." (5) Paisley's statement could also be viewed from the other side of the coin to emphasize that "social systems of users that are not integrated into computer retrieval systems can result in uninterested, little informed nonusers. 121 APPENDIX IV PROGRAM DESCRIPTION OF SUPARS STATPAC Michael McMullen The SUPARS STATPAC was written to fill the need for a package of programs to analyze and sunmarixe the activity that occurred through the Search Monitor, The major considerations in the writing of the package were: 1. The package must be usable by relative non-prograimers * 2. The diverse data types and forms demand a flexible package Which could handle rather variable records. The approach which was chosen was to make use of the catpile-time preprocessor available in IEM Pl/U What was written was a large body of pre-processor statements which would conditionally create the source code for any program which was desired for analyzing the Monitor statistics, •Dais program is henceforth referred to as MACRO. This approach was taken because of the facility with which it fulfilled the t * major criteria stated above. The package is relatively easy to use *> since the input to MACRO consists of simple keyword assignment statements. The user nead have very few concerns about the data types being handled since once the basic data description is inserted in the program it will take care of all data conversions. Data Flow of SUPARS Log The activity log as produced by the Monitor consists, for each search number generated, of a possible 14 different types of records. Each of these different types of records are produced by a different type of activity within the monitor. One record is produced v f e the Monitor tin is brought up and records the date and time thus producing a date for all records that follow. For each search a separate record is produced for each of the following things: the entry of a delta (time and type), the input of a search line or the production of a line of output (the text of that line), the entry of a GO or STAT or. CFF or the pushing of attention to halt output, etc. Doe to factors such as the greatly increased handling and preparation needed for each use of the log data it was elected to combine the many snail records for each search into one large record. To accomplish this a PL/1 program was written which merged the records for each search. This program is henceforth referred to as CONVERT. Since the log records as produced by the ffonitor were not strictly in order by search, due to the fact that several searches could be in progress 122 simultaneously, it was first necessary to sort the records as to search nimber. This was done through the use of the IBM SORT/MERGE package. These sorted records were then used as input to CONVERT. Before beginning the actual processing of the records several parameters are first set in the program. The first of these are conversion factors to be used in the ocnputation of cost figures for each search. Also set were two parameters which determined the first and last search to be processed. The records produced by CONVERT consist of a constant header followed by a variable portion. The constant header contained such information that had a oonsistant length and form, such as search number, user number, terminal number, etc. The variable portion of the record contained those items which were conditionally produced by the monitor and which could occur in variable numbers and lengths, these were such things as the input and output lines and search error codes. ! At the same time that the records were being statistical analyses another set of records were These records were the ones that served as input the data base for later delta S searches through reformatted for use in being produced by CONVERT. for the establishment of the Monitor. CONVERT also did a certain amount of data cleaning and supplied default (usually zero) values fbr toy record types which were not received for a particular search. The flow of data through STATPAC programs, MONITOR, SORT, and CONVERT, is graphically shown in Figure No. 1. A detailed description of the STRTPAC program follows. The programs referred to are all explained in the flowcharts labeled Figures 2 through 5. Figure No. 2 describes the overall logic flow for producing STAT output. Figure No. 3 is a more detailed description of the conversion program vdhich produces the records STATPAC operates on. The actual STATPAC processing programs, MACRO and PROCESS are explained in Figures 4 and 5, respectively. 123 START MQUTTAR PERFORMS SEARCHES AND PRODUCES LOG RAW MONITOR LOG QRZ. SORT LOG INTO INPUT ORDER i CONVERT * SORTED I MONITOR LOG CONVERT 'COIl VERT ED LOG FOR AS SEARCil DATA BASE CONVERTED LOG FOR STAT. ANALYSIS STOP Figure No. 1 data Flow of SUPAPS Log 124 rmatu ORIENTED INPUT /USER ORIENTED INPUT IEFORMAT 1 CONTROL STATEMENT REFORMAT y U'.ACRO INPUT DATA ,SET U CRO PL/1 PREPROCESSOR PROGRAM PROGRAM FROM LIBRARY - > PRINTED REPORT STOP Figure No. 2 Program Flow for Producing SD\T Output 125 COW VERT START READ SORTED LOG RECORD ORTED LOG FILE SET PARAMETERS J [ SET END-OF-FILE\ FLAG /READ UP TO FIRST RECORD WANTED 1EV / LOG ^JH UMBER ^ ' CONVERTED LOG STAT FILE /WRITE j / A S SEARCH -^-/CONVERTED N { DATA 1 RECORDS I \ BASE \SOURCE INITIALIZE RECORD BUFFERS > STOP Figure No. 3 Program f o r Conversion o f Log t o STAT Usable Form 126 SFT CLOCK COST,CPU, EXCp IN OUT-PUT SET "AXTWn\ PRINTED ,AND\ REPEA TED © CATENATE INPUT LINE TO OUTPUT RECORD CATENATE VO-VTPUT ITEM DELTA VARIABLES Figure 3 (Continued) 127 ' GO H7A TEMF.ViT Figure 3 (Continued) 128 \SECTIONS OF CODE j MARKED BY BRACKETS J ARE CONDITIONALLY i INCLUDED IN ! PROGRAM I CREATED MACRO START SET PARAMETERS ALL = 1 CALL SORT - < • [CONVERTED I LOG I FILE > 'READ LOG RECORD 'END \ ALL = 0 CREATE NEl' N OF FILE J? i > M VARIABLES Figure NO. 7 PRINTED REPORT PROCESS ^ * £fifi£££2 SECTION OF CODE MAY DF REPEATED ANY NUMPER OF TIMES TWTNH~ FINAL COUNTS ( STOP J Figure No. 4 MACRO - General O u t l i n e o f STAT Programs 129 PROCESS ENTER ) Were you aware t h a t demonstrations were, available? Yes Ifo I lave you ever called extension 4220 for helo with questions or problems about SUPARS? Yes Io f (GO TO QUESTION 21) How often liave you called extension 4220 for help? Has the person ansvering been able to help you? Yes Io f Do you intend to use SUPARS again this week? Yes Io f Don't know Vtould you sign up for SUPARS if it became available again? Yes . Io f Don't know 144 23. Give me UJD or three adjectives or other vorcls which might l^est describe or characterize your experience *dth SUPA^S. Do you have any auestions now about SUPfiRS **tich y°u v.ould like to as];? 24. Thank you for your time and cooperation. 145 1. Name 2. Date 3. Social Security^ 4. Campus/Home Address 5. Cantpus/ISome Phone 6. Are you primarily: Faculty Undergraduate student Graduate Student^ Staff Mministrator 7. What is your academic major or department? Question does not apnly 3. What percent of your time is engaged in: Teaching and/or learning Research Other 9. Did you register for SUPAPS last year? YES NO 10. If YES, on the average, how often did you use SUPARS? a. more than once a day c. several times a weak e. once a month b. once a day d. several tines a month f. less frequently 11. On the average, over the last 2-3 months, how freouentlv have you usedPsychological Abstracts? 12. Estimate how frequently you see yourself using Psychological Abstracts in the next 2-3 months (assuming computer service were !HT available)? Select ONE choice from 'a-g' above. 13. Vftiat recent interest areas are important to your use of Psychological Abstracts? (Most) (2nd) " (3rd) — - \ 14. I lave you used Rsycliological 7\bstracts in the last 2-3 months to heln you prepare a terra paper, thesis, soeech, etc. (If YES, nlease x give title or topic) , : • . _ _ 15. Do you need Psycliological Abstracts in the next 2-3 months to help you find references or abstracts for a term paper, thesis, sneedi, etc., you are currently preparing or planning to prepare? (If YES, please give title or topic) 16. Wliat is your specific need for Psycliological Abstracts in the near future? To find a specific reference or abstract? To keep up to date in one or tv.o content areas? To survey the literature in general? To exhaustively review a specific topic or area? Tb find several current references in an area (but not necessarily exliaustively review that area?)_ Other (Specify) 17. What previous experience do you have with conputer terminals? A lot? Same? None If you you liad with catputcr-l^ased retrieval 18. What previous experience doliave haveoj