Analyzing Ethical Scenarios

Blaise W. Liffick, Ph.D.
Department of Computer Science
Millersville University
Millersville, PA 17057

Proceedings of ETHICOMP95 Conference.
De Montfort University.
March 28-30, 1995. Leicester, UK.

Abstract

One of the approaches to teaching about the ethical issues related to computer technology is the use of the ethical scenario (or case study). Unfortunately, few authors of texts for this area give any concrete methodology for analyzing such scenarios. Students, therefore, tend to flounder when asked to, for instance, write an essay about some event depicted in a scenario. This paper provides a methodology for analyzing such scenarios.

 

Introduction

In its 1991 Computing Curriculum report, the Association for Computing Machinery (in conjunction with the Institute of Electrical and Electronic Engineers) identified for the first time the importance of including a "social and professional context" as part of the core curriculum for all computer science majors [Tucker, et al, 1991]. According to this report, "undergraduates also need to understand the basic cultural, social, legal, and ethical issues inherent in the discipline of computing" [page 11].

There are several techniques that have been employed in order to present these topics to students. The most popular technique appears to be what is variously called the case study or ethical scenario, a short narrative of one or more events that involve one or more ethical issues. The intent is that a student reading such a scenario is expected to analyze the participants' actions and arrive at a judgment concerning their ethical nature.

Such scenarios, as related to the field of computing, were first illustrated by John Parker, who conducted a workshop attended by a diverse group of professionals, including computer scientists, psychologists, sociologists, and lawyers [Parker, 1977]. The group's task was to analyze numerous scenarios related to computing, specifically attempting to identify the ethical issues involved. Parker's goal was to "develop the concepts of unethical practices that are unique or prevalent in the computer science and technology fields," with the final results "to be used in the development of more explicit codes of ethics among AFIPS constituent societies..." [pp. 4-5].

While this seminal work provides a general framework for analyzing such scenarios, unfortunately it is not detailed enough for use in an undergraduate computer science setting. In addition, the framework itself assumes that participants are, in fact, experts in their fields. This can hardly be assumed when such scenarios are analyzed by undergraduate students, some of whom may be as yet marginally experienced even with computing.

Many recent authors have followed Parker's lead, however, and include this type of scenario in their texts. For instance, Forester and Morrison [1994] include a section of lengthy scenarios at the end of their book, to be used for "further discussion." Each scenario is meant to illustrate the particular aspect of ethics addressed in each chapter. Dejoie et al [1991] includes lists of short scenarios, some of which have been adapted from Parker. Rosenberg [1992] provides several case studies, along with his own analysis of them. He notes that these studies may not be useful in teaching students how to behave ethically, but at least would identify some of the ethical dilemmas facing computer professionals. Course syllabi circulated at the 1991 National Conference on Computing and Values indicate that scenario analysis is a widely used technique in ethics education for computer science students.

However, none of these references suggest a methodology for analyzing such scenarios that is appropriate to an undergraduate (or even graduate) level. Suggesting the use of Parker's technique, for instance, would require that students be able to extract underlying ethical principles that address the issues of the scenario, without suggesting how this can be done. This is clearly expecting too much of undergraduate students, many of whom are weak in their understanding of ethical theories, their ability to perform critical analysis, and their general writing skills. While the very best students might be somewhat successful with such an indefinite task, the majority of students tend to flounder, easily becoming disenchanted with the entire exercise.

It is necessary, therefore, to provide students with a methodology for approaching such a task. This would provide a means of organizing the information of the scenario, as well as their own thoughts about it, preliminarily to attempting to write about it. This technique of organizing is common within the realm of critical writing. The revered (and much-maligned by students) technique of outlining is one form of this. Guides to clear writing also provide clues to a more detailed methodology. Trimble [1975], for instance, in a section on critical analysis, suggests that students should "[range] back and forth through the plot in pursuit of textual evidence" to support the viewpoint the critic is taking, "using those details to demonstrate a point" [p. 26].

Following is one method of approaching such an analysis, which is currently being used by students enrolled in a senior-level computer science course "Social Consequences of Computing" at Millersville University. The intent of this methodology is to assist students in breaking a scenario down into its component parts, in order to better isolate and organize the key elements.

The Methodology

The following set of guidelines allows students to dissect a scenario, place the issues identified into a social context, and take advantage of available tools (e.g., various codes of ethics) for analyzing the ethical issues presented. This approach is analogous to a classical computer programming methodology known as top-down analysis or step-wise refinement. The idea of this programming technique is to break a large problem into successively smaller (and, presumably, more easily solved) problems, until solutions to these sub-problems become obvious. The total solution to the problem is then constructed by carefully recombining these smaller solutions.

Every computer science major should be entirely familiar and comfortable using this technique by the time they are juniors (in general, those students who have not mastered this approach do not survive to this level!). However, while such students might be completely at home using this technique for creating computer programs, they are also often highly intimidated by the task of critically analyzing prose. While students have undoubtedly been taught an outlining method for developing an essay in various writing courses, this method is one of composition, i.e., of putting pieces together in their proper order.

The task of critical analysis, however, begins with decomposition, the breaking down of an idea expressed in prose into its component parts. The essential ideas and issues must be identified. The purpose of the proposed methodology is to assist students in that decomposition. Once this has been accomplished, the student can then use traditional writing techniques to develop an essay in which the student presents his or her opinions about the scenario, along with arguments that can be supported with material from the text of the scenario itself.

To begin, consider Scenario 1, adapted from Parker [1977]. While it might be possible for a practicing professional with several years experience to form a valid opinion about this scenario, it is much more difficult for students to do so.

 

Interactive Human Judgment in a Life-Critical System

The spokesperson for a union of airline maintenance workers charges that the airline has introduced a computer program to perform functions which should require interactive human judgment if safety is to be ensured. The program is one which schedules maintenance, and which reassigns aircraft when emergencies arise because airplanes unexpectedly become unusable. The systems analyst, under whose direction the program was written, is aware that not all operational factors have been taken into consideration in the program, but he had been assured by management that the decision rules used in the program conform to all the requirements of the IATA (International Air Transport Association). In his opinion the program should have been an interactive one, where a person is involved in some of the final decision making, but the company was not prepared to go to the additional expense of an interactive system. When testing his program, he could not devise an example where the existing program produced an action which failed to meet a safety condition. Because he could not document reasons for his doubts, and also in part because he was inclined to be defensive about his own work, when he was asked to testify in an inquiry dealing with the union's complaint, he did not volunteer his opinion on how the system should have been designed.

Scenario 1.

The following steps assist the student in analyzing such a scenario.

(1) List Participants and Their Actions

Make a list of all of the participants involved, and their respective actions. This list should include the following groups: primary participants, those who have taken specific, obvious actions; secondary participants, those who have been acted upon or otherwise been affected by the actions of the primary participants, or who are mentioned in the scenario but take no direct action themselves; and implied participants, those who are not specifically identified by name, but who may have a stake in the outcome of the events described by the scenario - implied participants may be primary or secondary in nature.

In Scenario 1, we might list the following participants and their actions:

Primary Participants:

Systems analyst

designed program knowing that not all operational factors had been taken into account

informed management of his concerns

failed to volunteer his opinion when testifying

Airline management

disregarded analyst's concerns

indicated decision rules used in program conform to IATA requirements

opposed making system interactive

Union spokesperson

charges airline with ignoring safety factors

Union

files complaint against airline

Inquiry board

holds hearing

Secondary Participants:

Maintenance workers

must use the new maintenance software

IATA

set requirements for airplane maintenance

Implied Participants:

Programming staff (primary)

implemented systems analyst's design

Airplane crews and passengers (secondary)

must travel in planes maintained by developed system

(2) Reduce List through Simplifying Assumptions

It should be somewhat obvious from the above that many of those listed as participants are of minimal interest in the analysis of this scenario. Their actions are either trivial, or are not of direct concern ethically. It happens that the list of participants and their actions often winds up in a "kitchen sink" situation, with every possible person even remotely associated with the event listed. It is important to weed out those who have limited impact on the real issues addressed in the scenario. Clearly there is a need to cut this list down to a manageable length, so that time is not spent analyzing people or issues that are not truly of interest.

For instance, it can perhaps be assumed that the union acted in good faith when filing charges against the airline, that they were not engaged in grandstanding for the sake of publicity, nor were they attempting to harass management during contract negotiations. If the union's complaint is legitimate, we can ignore their action in this matter, and need not comment on it in the final written analysis of the scenario.

Other simplifications include:

There is no distinction between the union spokesperson and the union.

The inquiry board is legitimate, and will do its job properly.

The IATA has established sound maintenance requirements for all aircraft.

The union represents the maintenance workers, and while there is a distinction between these groups to some extent, the distinction is unimportant in this scenario.

While the main issue appears to be the overall safety of the crews and passengers, this is stating the obvious and can probably be eliminated from the list.

While a defense of "I just followed orders" is hardly legitimate in many cases, the programming team who did the actual implementation can probably be eliminated from consideration, on the assumption that they followed standard programming practices in coding the programs for the system. Another consideration is that it was the systems analyst who was in charge of the outcome of the project, and who, therefore, developed the specifications that the programmers implemented.

As a result of these assumptions, all the participants except the systems analyst and the airline's management can be eliminated. While this reduction was perhaps obvious in a certain respect, it was necessary to go through the steps of including all of the other participants, and then eliminate them, for the sake of completeness. It is all too easy to assume that those who are mentioned most prominently in a scenario are the only ones who have an impact on the final analysis.

(3) Legal Considerations

Certainly, if an action is against the law, it is probably unethical. List what laws cover the actions discussed in the scenario (clearly, students must be introduced to the pertinent laws regarding privacy, hacking, etc.). Obviously, students are not expected to be as knowledgeable as lawyers about all aspects of applicable law. However, they should at least be aware of the major laws directly related to their field.

A subset of legal considerations is company policy. There may be no current laws that govern a particular action, but there may be a company policy that requires or prohibits certain actions. Such policies must be explicitly stated in the given scenario - they should never be assumed.

Although the text of pertinent laws or samples of company policies may not be readily available to undergraduate students, the concept should be introduced so that it is not overlooked when they are eventually in real-world situations. Also, just because there is not a law or policy prohibiting an action, this does not mean that the action is ethical.

In the example scenario, there are likely to be many laws pertaining to the regulation of the airline industry. The IATA must have some authority in the matter, since it has developed a set of maintenance requirements. It is impossible, however, to analyze this (probably lengthy and numerous) set of laws that might have bearing in this case. One must assume that any violation of law will be determined by the board of inquiry.

A legal issue that would be pertinent to this case, however, would be liability law. What liability does the airline management assume in this case, should an airplane crash due to faulty maintenance? the systems analyst? the programming staff? the maintenance workers? While, again, it would take a legal specialist a great deal of time to present the entire set of laws and legal decisions that affect the case, it is clearly important to provide some guidance to students on this issue. Clearly life-critical systems present special problems of liability. Students can at least be made aware that liability is an issue that they should discuss with their employers and, perhaps, a good attorney.

(4) List Possible Options of the Participants

Make a list of what options the participants may have had before they chose the path of action described in the scenario. While this list could potentially be infinite (e.g. "the participant could have committed suicide..."), clearly only pertinent and viable options need be listed.

The scenario may not list the options faced by the participant. In that case, the options are implied. Obviously, if the participant chose Action A, Action B (or C, etc.) was not chosen, which must be known if the reader is going to make a judgment of ethical behavior. This list helps ascertain whether or not there existed another alternative for the participant that was purely ethical, or whether the participant perhaps had chosen the least non-ethical option.

In Scenario 1, the analyst could have

refused to participate in the design unless all operational factors were incorporated

suggested involving the maintenance workers themselves in the design effort

contacted the IATA directly about his concerns

offered his opinion to the inquiry board

contacted the media about his concerns

The airline management could have

involved the maintenance workers themselves in the design effort

ensured that all operational factors were incorporated in the design

taken the systems analyst's advice and agreed to allow the system to include an interactive feature

(5) List Possible Justifications for the Participants' Actions

In order to evaluate why the participants may have chosen their respective response(s) to the situation, compile a list with as many reasons as might be suggested by the scenario. List justifications explicitly offered in the scenario, as well as any other justifications that make sense, being careful not to make assumptions. For instance, when considering possible justifications for the main participant in a scenario, it is easy to suggest a defense if you, yourself, have been in a similar situation and had to make the same kind of decision.

In this scenario, the analyst may justify his actions (or, in this case, non-action) by stating that he

simply followed a "management decision."

couldn't prove there was a problem, even with additional testing.

had only his own professional opinion (which might be taken simply as a "hunch") that the system was deficient.

The airline management might state that

the cost of redesigning the system for interactive use is prohibitively expensive, and could put the company in a financially precarious position.

they do in fact meet the IATA requirements, and that there is no reason to be any more stringent in their maintenance processes.

it is unreasonable to expect the company to base an expensive redesign on a single employee's "hunch," however informed or expert the person is.

It is important that this list be limited to legitimate justifications, and not be simply a compilation of rationalizations (unless they are somehow explicitly stated in the scenario).

(6) List Key Statements

List quotations from the text that are important to the analysis. These will probably have been used to create parts of the previous lists, and so should be mostly obvious. Jot down any phrase that is used as the basis for previously listed items.

These phrases might follow an action and begin with an explicit or implied 'even though,' 'however,' or 'instead of.' Other important phrases include those that indicate secrecy, such as 'without getting approval' or 'without telling anyone.' Unmistakable indicators are those expressions that suggest the participants' refusal to accept responsibility for their actions, statements where the participants make excuses for their behavior, or statements that demonstrate personal gain for the main participant.

For this scenario, the following phrases are important

"...not all operational factors have been taken into consideration..."

"...assured by management that the decision rules used in the program conform to all the requirements of the IATA."

"...the company was not prepared to go to the additional expense..."

"...he could not devise an example where the existing program produced an action which failed to meet a safety condition."

"...he did not volunteer his opinion on how the system should have been designed."

(7) List Questions Raised

According to Parker (1979), the reader should not make assumptions regarding what else might have happened in the scenario. He warns against assuming that just because one thing has happened, we can infer the circumstances leading up to that event. He instructs the reader to rely strictly on what was included in the scenario because of excess baggage that might be carried into the analysis by the reader. Otherwise what may follow is a lengthy deliberation exploring extensions of the given scenario, which may result in the discussion of a situation that only vaguely resembles the original scenario.

However, natural language is ambiguous by its very nature. No matter how carefully a particular scenario might have been crafted, and regardless of exhortations to rely solely on what is directly supportable by the text, there are few instances where something isn't ambiguous about a given scenario. In addition, there are often simply unknowable pieces of information that have a direct bearing on the outcome of a scenario.

In Scenario 1, for instance, were standard design and programming practices followed in this project? There is no information available within the scenario to assist in answering this question. One could argue that a great deal hinges on the answer to this one question, however.

It must be considered that such ambiguities point to a fundamental flaw in the scenario, i.e. that the author did not craft the case carefully enough. One would think that an important goal in developing a scenario would be to make it entirely unambiguous. The nature of natural language, however, makes this a nearly impossible task.

Other questions that might be raised with the example scenario include

Was the issue of including representatives of the maintenance crew on the design team ever brought up? Was there a conscious decision to exclude them?

Could a cost/benefit analysis have been done to demonstrate the efficacy of an interactive system? What about a prototype?

Should a computer ever be given sole authority in matters where human lives are clearly at risk?

This last question, of course, is widely open-ended, and is ultimately the focal point of the discussion.

Real life is full of ambiguities. The vagaries of natural language make ambiguity in nearly any set of statements inevitable. Although human beings are generally adept at dealing with ambiguity on a regular basis, clearly there would not be a need for the extensive legal system of our modern society if we all agreed on interpretations of events, statements, etc. It is necessary to at least acknowledge that ambiguities exist even in the most carefully crafted statements. Thus it is necessary to perform some amount of exploration into the various possible interpretations that a scenario might evoke from multiple readers.

(8) Other Models, Related Issues

Scenarios in computer science may appear to be unique, but when broken into pieces or distilled into actions irrespective of the hardware or environments involved, may be similar to other real world systems for which ethical issues have largely been resolved. These models can serve as analogies for a given situation. Once the relationship of two seemingly dissimilar scenarios is made obvious, a conclusion may be easier to draw.

Using the example of electronic mail, we can look at the models presented by usage of telephones, the postal system, or inter-office mail. When we ask, 'does a company have the right to monitor electronic mail,' we might ask the similar question, 'does an employer have the right to monitor employee phone calls, open employee letters or read someone's inter-office memo?' Questions like these may have been answered quite emphatically for the existing models, and could be applied to the seemingly new, yet age-old question raised by computer-related application. This is similar to the notion of homomorphism in discrete mathematics.

Nearly any life-critical system might serve as an analogy for the example scenario. The Therac-25 disaster certainly comes to mind. Another is the O-ring incident of the Challenger space shuttle. One other analogy that might come to mind would be the development of the Mercury space capsule. After input from the original seven astronauts (who were intended initially to be just passengers, not pilots), the capsule was heavily modified to include on-board flight controls. Clearly in this instance, it became crucial to involve those most immediately affected by an automated system, the direct users.

(9) Comparison to Codes of Ethics

The revised Code of Ethics for ACM members appeared in the Communications of the ACM in February, 1993. Other professions, such as the Data Processing Management Association and the Institute of Electrical and Electronics Engineers, have published similar Codes of Ethics used for governing the actions of their members. The actions of the scenario participants should be compared directly to these established codes, looking for instances that apply.

Many of these codes are listed in appropriate texts for use in a course on the social consequences of computing (e.g. Huff and Finholt, 1994; Dejoie et al, 1991). A comprehensive list of such codes is now also available via Internet ("Codes of Conduct or Codes of Ethics from Around the World", W3 address http://info.cs.vt.edu/~janlee/WorldCodes.html). It is important that students be exposed to these codes, and that they be given opportunities to apply them.

For the example, at least the following ACM principles apply:

1.1 - Contribute to society and human well-being.

1.2 - Avoid harm to others.

1.3 - Be honest and trustworthy.

2.1 - Strive to achieve the highest quality...[in your] professional work.

2.5 - Give comprehensive and thorough evaluations of computer systems and their impacts, including analysis of possible risks.

3.2 - Manage personnel and resources to design and build information systems that enhance the quality of working life.

3.4 - Ensure that users...have their needs clearly articulated during the assessment and design of requirements...

In addition, the following Data Processing Management Association (DPMA ) rules apply ("[As a member] I shall..."):

Accept full responsibility for the work that I perform.

Not misrepresent or withhold information concerning the capabilities of equipment, software or systems.

Never misrepresent or withhold information that is germane to a problem or situation of public concern nor will allow any such known information to remain unchallenged.

It should be pointed out that these rules can clearly be applied to the systems analyst in evaluating his actions, but it is less clear what their meanings are in terms of evaluating the airline management's actions. It is unfair to hold non-members to ethical standards set by a professional organization. However, since these two participants are essentially antagonists in this scenario, what supports one participant undermines the position of the other. What can be derived from this list are statements that the analyst might use to support his case before a board of inquiry, or directly to management.

 

The Place of Ethical Theory

The reader will undoubtedly notice that there is not any discussion of ethical theories in this methodology. This is intentional, for several reasons. First, the course in which this methodology is applied is a computer science course, not a philosophy course. Whether ethical theories should be taught in such a course is, certainly, a timeless issue of debate. It is not our intention to revisit this debate. The realities of the situation are that there is little class time available for a detailed presentation of ethical theories, to a depth where students might be reasonably expected to apply them. In addition, students appear to have little affinity for such theories, and have great difficulty internalizing them.

Second, it is not at all clear that learning such theories make students any more adept at solving ethical conundrums. Furthermore, when faced with a dilemma once they become professionals, will having the ability to cite how specific ethical theories might view the situation either help them make a decision, or defend it to management?

A more significant reason for not including ethical theories in this methodology is, however, that it would be redundant to do so. The codes of ethics and conduct of professional organizations already incorporate a particular point of view about the nature of ethics as applied to that profession. While these codes might not satisfy a philosopher in either form or content, they are nonetheless the expression of that profession's position on ethics.

Historically such codes have often been viewed as somewhat superfluous, as little more than flowery sentiment, with no practical aspect to them. This methodology points out, however, that these codes can indeed be put to practical use.

Composition

Once the student follows this methodology to completion, the task of decomposition is over and the student must now reassemble the parts into a coherent essay, i.e., must construct a composition. The student has numerous small parts on which to draw for this composition process. It is now up to the student to select the most important items from the various lists to discuss in his or her essay.

One caveat, however, is that students tend to view the decomposition methodology as an outline for their final essay. Without additional directions on how to develop the essay itself, they frequently will simply start at step 1 of the process, write about the information in their list for a few paragraphs, then move to the step 2 list, write a short time, etc. until they reach the end of the steps. While this may produce a coherent presentation of the information, it also creates rather uninteresting essays, all sounding essentially the same (whether this is a positive thing is a moot point). Students must typically be encouraged to use their own creativity to liven up the final review and its presentation of the facts.

Conclusions

Experience with teaching this methodology to date has been somewhat mixed. Some students appear to like it, others do not. This attitude may be due to a number of factors, not the least of which is a student's approach to the writing process itself. For some students, any written assignment is to be despised, and adding a process on top of the task might only serve to make the task even more onerous.

Another reason for student dislike is the open-ended nature of the task itself. Most students seem to prefer dealing with questions that have a clearly definable "right" answer. The methodology presented in this paper is in no way an attempt to come up with a process that produces a single answer to an ethical dilemma that can be considered "right." However, students have been almost universally trained to expect such an outcome, not only from their science classes (where experiments clearly have only one correct solution), but also their social science and humanities classes, where the type of assignments and tests that are given rely heavily on questions that the student is expected to answer in a particular way.

In addition, it is more difficult for instructors to provide appropriate feedback to students for such open-ended questions and answers. It is often difficult to explain to students why they got the grade they did (although, clearly, if an instructor cannot justify a particular grade in a way that the student can understand, there is good cause to be suspicious of the grade). There is a certain subjectivity to the evaluation of a student's performance of a task such as scenario analysis. While, as instructors, we prefer not to think that such is the case, because it is much more difficult to defend such evaluations, this is true when evaluating any significant writing assignment (consider how we all have our own favorite literature authors).

While these problems are not likely to disappear, the major positive result of this methodology overrides any of the negatives. Experience with senior-level computer science majors enrolled in the course "Social Consequences of Computing" (a required course for CS majors) at Millersville University shows clearly that this methodology improves the performance of students in scenario analysis. The quality of papers has risen significantly since this methodology has been introduced, especially in the case of marginal students. In addition, this methodology actually does provide a way for the professor to defend what would normally appear to be a purely subjective grade. If a student has omitted certain aspects of the analysis, the professor can point to any area that is lacking and suggest that the student overlooked one aspect or another, hence the resulting grade. In addition, the student's analysis itself could be graded for completeness, as well as used as a checklist when grading the final essay.

One additional benefit to this methodology is that it parallels a process with which the students are already familiar, namely top-down structured programming. Not only does this make this new process more understandable to the student, but it provides a concomitant strengthening of their understanding of the top-down methodology they use in programming tasks as well.

To date, the evaluation of this methodology has been anecdotal. A rigorous study of the method awaits future research, which could take either a qualitative or an attitudinal approach. However, overall reactions from students have been favorable in general.

 

References

Anderson, Ronald E., Deborah G. Johnson, Donald Gotterbarn, Judith Perrolle (1993). Using the New ACM Code of Ethics in Decision Making. Communications of the ACM, 36,2:98-107 (Feb. 1993).

Dejoie, Roy, George Fowler, David Paradice (1991). Ethical Issues in Information Systems. Boston, MA: Boyd & Fraser Publishing Company.

Forester, Tom, and Perry Morrison (1994). Computer Ethics, 2nd edition. Cambridge, MA: The MIT Press.

Huff, C. and T. Finholt (1994). Social Issues in Computing. New York: McGraw-Hill, Inc.

Parker, Donn B. (1979). Ethical Conflicts in Computer Science and Technology. Arlington, Virginia: AFIPS Press.

Rosenberg, R. (1992). The Social Impact of Computers. Boston, MA: Academic Press.

Trimble, J. (1975). Writing with Style, Englewood Cliffs, NJ: Prentice-Hall.

Tucker, A. (ed.). Computing Curricula 1991: Report of the ACM/IEEE-CS Joint Curriculum Task Force. New York: ACM Press. 1991.