An Applied Social Systems Procedure for Generating Purposive Sound Futures
Bar-Ilan University, School of Education, Ramat-Gan, Israel
This paper presents the social systems theoretical framework and the procedural design of a group interaction procedure using a technique of iteration with control feedback named Imen-Delphi (ID). The ID procedure suggests an applied social systems technique that can improve the efficiency of a group of panellists to invent change through inter-group discussions and attain some level of agreement regarding their preferred future change.
Keywords: social systems; Imen-Delphi (ID); change; emergence; group interaction; agreement.
The Delphi technique was originally designed as a procedure to solicit and synthesize the forecasts of groups of experts, and was to be primarily used to obtain accurate estimates of projected dates of future occurrences. It began in the early 1950s when the RAND Corporation adapted the name Delphi for a procedure in an early experiment called ‘Project Delphi’. The experiment attempted to select (from the viewpoint of a Soviet strategic planner) an optimal US industrial target system and to estimate the number of bombs required to reduce munitions output by a prescribed amount. The procedure aimed at the elicitation of the most reliable consensus of group opinion through a series of intensive questionnaires (five rounds) interspersed with controlled opinion feedback, while avoiding direct confrontation of the members of the group with one another.
The experiment was sponsored by the US Air Force which, for security reasons, kept the study unpublished for at least another decade. The procedure suggested that the judgement of experts is permissible as scientific evidence in fields which have not yet developed to the point of establishing formal scientific laws. The procedure intended to replace direct debate by a carefully designed programme of sequential individuals' interrogations (best conducted by questionnaires), interspersed with information and opinion feedback derived by computed consensus from the earlier parts of the programme. Dalkey (1967) defined the procedure as a set of procedures for eliciting and refining group opinion, usually experts. Turoff (1970) also offered a similar definition saying that Delphi is a procedure for the systematic solicitation and collation of informed judgements in a particular topic, using a set of carefully designed sequential questionnaires interspersed with summarized information and opinion feedback derived from earlier responses.
Later, Linstone and Turoff (1975) presented what seems to be today an acceptable definition of the Delphi that may be characterized as a procedure for structuring the communication process of a group, so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem.
Since its introduction in the early 1950s the Delphi procedure has experienced ever-increasing utilization, praise and criticism. The Delphi procedure received its metaphoric name from the mythic Creek oracle -- Delphi -- where Apollo claimed the ability to foresee the future (Judd, 1971; Mcbride, 1974). This article reports on a design of a procedure, a variant of the Delphi named Imen-Delphi (ID). It aims to facilitate a group discussion which enables the participants to generate new future images, and emerge by establishing a common future working mission.
As Delphi has spread, many variants of the process have emerged out of the conventional procedure. Each variant aimed at improving the procedure in order to respond to needs and critiques. The attempts to refine the Delphi procedure were undertaken by Olaf Helmer (1966) himself, the originator of Delphi. Since then many other versions have been suggested to reflect and solve different problems. Variants that developed a practical approach to Delphi as a means to facilitate group judgement formulation and future decision making were suggested from the introduction of the conventional Delphi. The procedure that is reported here - the Imen Delphi - draws from and is built on the theoretical framework and strength of the following major variants.
Soon after the conventional Delphi was introduced, Donald Pyke (1970) developed a variant which he called the Sequence of Opportunities and Negatives (SOON). The Delphi was expanded to include an index of desirability as considered by those who will be affected by the occurrence of the event, and an index of feasibility reflecting major difficulties likely to be encountered by those engaged in the reification of the event.
The Policy Delphi (PD) is a unique feature in the development of the practical approach. A PD procedure, while it maintains anonymity and uses the function of the feedback, does not aim at obtaining consensus but rather to enhance the opportunity for considering all the different views of the participants. It seeks to generate the greatest possible number of opposing views on the potential resolutions of a major policy issue. The PD is, therefore, a tool for generating options and suggests several alternative courses of action to be considered by the policy makers. It helps to eliminate the bottle-neck problem of the committee approach and of the classical Delphi by providing clear delineation of all differing views, thereby providing the opportunity for all the participants to adequately prepare their perspectives on the respective issues (Turoff, 1975).
The Decision Delphi (DD) adds new features to PD. It does not deal with experts nor with lobbyists or advocates but with actual decision makers in order to recruit the panellists with regard to their actual position in the decision-making hierarchy (Ranch, 1979). In this procedure, anonymity is not fully implemented. The panellists’ names are known from the beginning, but the responses are kept unidentified with any one participant (quasi-anonymity), in that the prestige of the others would provide a challenge and an incentive.
Another interesting procedure that provides the opportunity to conduct multiple information gathering rounds, in face-to-face settings such as workshops and seminars, while maintaining participant anonymity, is OSCAR (On-Site Conferencing and Researching). The OSCAR conference has been presented by Harkins and several associates of Anticipatory Sciences Inc. (1983). The goals of an OSCAR conference are to assist in planning and strategy formation, to clarify what the future has to offer, and to suggest ways to deal with change. It provides new ways to think and act. It is a technology of time and event which can be put together in many ways to help achieve alternative understandings of reality.
Although many procedures exist for use in group judgement formulation and decision making, only the Qualitative Controlled Feedback (QCF) procedure incorporates a set of elements that could overcome the weaknesses identified in other group judgement, brainstorming or decision-making methods (Press, 1978, 1983). It does not require the members involved in the group interaction to reach consensus or ‘group decision’. It does not allow group members to have interactions when they are making judgements, and it does not permit group members to know the identities of fellow group members, or their individual judgements on the issues in question.
The QCF operates differently from the conventional Delphi in several ways. First, QCF feedback is qualitative rather than quantitative, which means that statements explaining individual judgements rather than group response means and standard deviations are provided as feedback. These qualitative responses may contain information, insights, perspectives and nuances not provided in quantitative responses, which may be of value to individual respondents and to those using the research results. Finally, the QCF does not attempt to achieve consensus. Although the majority or all of the participants may agree on one or many items, consensus is never suggested as the process goal.
Finally, the Ethnographic Delphi Futures Research (EDFR)(Poolpatarachewin, 1980) represents a creative synthesis of Ethnographic Futures Research (EFR) (Textor, 1979) and the Delphi procedure. It was designed to combine the strengths of both procedures while minimizing their methodological weaknesses. Its advantage is the assurance that the participants will be intensely involved in generating the issues to be considered for group response so that the scope and focus of the issues under consideration cannot be significantly narrowed or distorted by the biases of the researcher.
This article reports on a design of a procedure that combines the strengths of most of the above Delphi variants, while minimizing their weaknesses, to reflect the need to conduct discussions around issues concerning non-expert groups. We named the procedure an Imen-Delphi (ID) exercise (the ability to emerge is in me).
The ID is best described as a procedure for eliciting and refining non-expert group opinions about their future. The ID is an exercise for structuring the communication process of a group, which allows the group to effectively learn experts’ forecasts, deal with some of their personal future complex problems, generate new future images, establish common ground, and determine a communal future working mission. It is best described as responsibility, self-awareness and concepts enhancement procedure. The ID draws its philosophical foundations from social systems thinking since it does not aim to generate any forecasts but rather to guide a group to solicit a consensus and grow. Following is a definition of the TD exercise, its theoretical framework, procedural design and characteristics.
The ID procedure is built on the strength of the conventional Delphi and the OSCAR procedure. The ID procedure is iterative - self repeating and self adjusting. Like OSCAR, it is a continuous and focused conversation among concerned individuals, which is carried out in rounds (see Fig. 1 for research design). Here is how it could work (Passig, 1992, 1993, 1996, 1997):
The researcher then can develop a questionnaire based on the questions of the panellists. This questionnaire constitutes the first round of an ID procedure.
Repeated attempts should be made to explain to the panellists the reason of having them learn their future. They should be told that they are accustomed to thinking that the causes of the present lie in the past. However, in a very real sense the causes of the present lie in the future, which means that the image of the future people have in their minds can have a dramatic effect on what they do in the present. Repeated attempts need to be made to convince the panellists that the reasons for this kind of study are (1) to inform them about the possibilities of the future being discussed, and (2) to help them assess their present actions accordingly. It should be emphasized that individuals who actively imagine themselves in a variety of clear future roles tend to generate long-term goals. Most importantly, they tend to work enthusiastically and self-confidently toward their attainment. The idea of having them learn their future, thus, is to help them generate positive personal goals, and promote their confidence in attaining those goals. This idea needs to be enlarged in various opportunities explaining that it is important to learn and create complex images of the future since on their basis it is possible to develop skills through which one may adapt to change and create change.
The panellists can then be asked:
The underlying purpose of the second round is to facilitate a thorough interaction that would generate specific ideas listed as statements, which the group could carry out as a redefined mission around the issue under discussion. Therefore, the second-round questionnaires should be designed around proposed mission statements. The statements that comprise the second round are narrated in the course of the reporting process. While the panellists are completing the first-round questionnaire, the researcher is engaged in organizing the answers that were received. The researcher summarizes the responses in reports coinciding with the issues under consideration. These reports should be presented to the panellists. They will be asked to review the first-round reports and to submit mission statements around which the second round questionnaire will be formed.
The panellists will be asked the following three questions on each one of the second-round mission statements:
(1) Do you prefer this statement to be fulfilled in your future?
(2) What is the likelihood that this statement will apply to your future?
(3) How important is this statement to the future of your group?
The purpose of this round is to help organize thoughts and focus the discussion around more specific solutions for (1) preferred futures, (2) expected futures and (3) important futures.
The ID procedure is aimed at producing some type of agreement on an alternative future mission: complete disagreement, plurality, bipolarity, majority or complete consensus. The second round is designed to achieve that purpose. The first question (preference) is designed to solicit a Cscale answer: (a) definitely yes, (b) possibly yes, (c) probably not and (d) absolutely not. The second question (expectation) is also designed to solicit a Cscale answer: (a) certain, (b) uncertain, (c) probable and (d) improbable. The third question (importance) is again designed to solicit a 4-scale answer: (a) very important, (b) important, (c) average and (d) not important.
This procedure is similar to the QCF procedure developed by Press (1978) to be used to facilitate group judgement formulation and decision making. The second-round design of an ID reflects some QCF features. Like the QCF it does not require members involved in the group interaction to reach consensus or 'group decision', and it does not allow the group members to have interactions when they are making judgements. However, the panellists agree to base their agreement on the majority of the votes.
At the beginning of a final third-round session, the panellists should be given feedback concerning their panel’s responses to the missionstatements of the second round. This list of mission statements that comprise the third round should be adapted from the mission statements of the second round that receive the majority vote as being very important goals for the future of the panel. The panellists are then asked the following two questions on each one of the third-round mission statements:
(1) Are you satisfied with what the group did to achieve these goals?
(2) What would you suggest/recommend doing in order to promote/achieve these goals?
(They should be asked to be very specific.)
The purpose of a third round is to have the panellists take responsibility, formulate a final proposed list of future mission statements and to generate new ideas and recommendations.
This procedure reflects a scientific paradigm advanced today by the systems sciences. It calls for a new meaning and purpose for science which believes that its institutions should primarily serve humanity by generating purposive sound actions and not just theoretical propositions, no matter how well they are confirmed by research.
It has been suggested in the literature that the study of personal images of the future can empower our thinking and help us develop clearer concepts and purposive sound actions. Humanity is confronted by a great challenge that of imagining the ideal, and then acting to transform the imperfect present into a perfect future.
According to the Dutch futurist Fred L. Polak (1972), as humans we respond to this challenge by creating images of the future. Through the creation of future imagery we redraw the boundaries of the unknown, thus raising our species to new levels of insight and purposefulness. We move from the concrete to the abstract, thus freeing ourselves from the bonds of time and space. We develop our awareness of ideal values, taking the first step toward conscious creation of culture. Images of the future provide the propelling power for culture creation, states Polak.
‘Images of the future’ represents a fundamental and powerful concept advanced by future research. Ruthanne Kurth-Schai (1984) identified images of the future as four types of creations: complex, organized, dynamic and purposive creations.
(1) Images of the future are complex creations. They may be held individually or collectively. They may contain both cognitive and affective components. They may be structured and communicated through language, pictures and emotions. They may be derived from past experience, present circumstance and future expectations.
(2) Images of the future are organized creations. They have been described as crystallized expectations developed into systematic projections. Rather than existing as a series of discrete and separate impressions, collections of images of the future are most often organized through webs of interrelationships into fairly coherent wholes. Therefore, although individuals or social groups may hold numerous and diverse images of the future simultaneously, it is still possible for them to approach the future from broad systemic perspectives.
(3) Images of the future are dynamic creations. They are responsive to new experience and to changing circumstances. Over time, previously held images are altered or discarded as new images are synthesized and incorporated.
(4) Images of the future are purposive creations. General systems theorists define purposive behaviour as that which exhibits choice in order to accomplish goals. In order to exhibit purpose, a system must demonstrate both the ability to generate goals and the ability to choose responsibly between alternative courses of action directed at goal attainment. The generation of goals and the process of making responsible choices are both linked to the process of making value judgements. All of these processes are employed in the creation of future imagery.
The Delphi variants, as well as other procedures which have been developed to date, aimed to explore the first three types of future imagery. These three creations are best described as explanatory images .f the future derived from past experience, present circumstances and future expectations. However, to the best of our knowledge, no procedures were designed to explore and develop the purposive images of the future - demonstrating the ability to generate new goals and suggesting alternative courses of action.
We therefore aimed at making an attempt to develop a procedure - the ID procedure - that can empower a group of people to evolve within their system and generate new diverse ideas and solutions about their future. The ID procedure is best described as an exploratory investigation of group's purposive images of the future, and as a responsibility, self- awareness and concept enhancement technique. The procedure suggests an exploratory investigation into two aspects of a group's future-imagery: the complexity of their future imagery, and their capacity to draw upon their future imagery and suggest a future mission.
Sociology of the Actor
In order to strengthen the theoretical framework for the ID procedure, one might claim that culture is one of the myths social scientists use to work with. The general idea advanced by social sciences about culture is basically describing the way humankind conceptualizes its framework. During one and a half centuries, social sciences as well as applied social sciences employed several anthropological theories to social movements, such as the Theories of Progress, Diffusionism and Functionalism, to understand how a society evolves and competes in an industrial era. Sociology, in this context, is defined as the study of order and movement (Barber, 1971), of modernization and social organization (Leon, 1982). However, social studies hardly understand social life’s capacity of transformation and rationalization. Social studies, as Touraine (1988) says, scarcely discusses how an aggregate system in motion and progress can form a stable and integrated system at the same time capable of maintaining its broad equilibrium and its internal organization, while activating its improving mechanisms. Classical sociology, he adds, provides no answer to this question, and the idea of collective consciousness (Parsons, 1977, 1978) serves more to obscure this issue than to enlighten it.
Few researchers, including Alain Touraine and Edgar Morin (1988), acknowledged that the representation of society as a pure system of order and domination is nothing but an ideological reaction incapable of analysing a social reality dominated by rapid and complex changes. They defined another type of analysis, at the centre of which is the idea of social action. This analysis defines ‘Culture’ as set of resources and models that social actors seek to manage, to control, appropriate or transform into social organizations they negotiate among themselves (Touraine, 1988). Indeed, Touraine spoke about the return of the actor as his main idea of social theory in a post-industrial society. Hence, Morin (1974, 1988) expresses the ambiguity of a recourse to models borrowed from the cultural sciences by trying to go against the grain, and to find in physics and especially in biology modes of thought that are in agreement with a renewed sociology of the actor.
Acknowledging Touraine’s questions on classical sociology, we adopted the sociology of the actor as the conceptual framework for developing the ID procedure.
In order to identify the methodological contributions of this new variant of Delphi, we suggest a framework from which this procedure may derive. A framework for approaching the topic of methodology has been provided by Churchman (1971). He identifies five distinct types of inquiring systems from which all research methods are derived. The five types may be summarized as follows:
(1) The Lockean System. In this system truth is perceived as experiential, concepts are empirically and inductively derived, and fact is established on the basis of the existence of sufficient widespread agreement by a group of experts. The Lockean system.represents the epitome of experimental consensual systems, and is best suited for well-structured problems for which there exists a strong empirical position.
(2) The Leibnizian System. In this system truth is perceived as analytic, empirical data is perceived as irrelevant, and all problems are reduced to mathematical or symbolic representations. The Leibnizian system represents the epitome of formal symbolic systems, and is best suited for problems which are well understood and/or simple enough to be comprehensively modelled.
(3) The Kantian System. In this system, truth is perceived as a synthetic combination of theory and empirical data. For every research problem, at least two alternative complementary models of the solution are created. The Kantian system represents the epitome of multi-model synthetic systems. It is best suited for inherently ill-structured problems which are difficult to formulate in Leibnizian or Lockean terms because the nature of such problems does not allow a clear consensus or simple analytic attack.
(4) The Hegelian System. In this system truth is perceived as conflictual, information results from the interpretation of data, dialectical debate can be formulated with respect to any issue, and for each research problem at least two completely antithetical representations must be created. It is further assumed that dialectical confrontation between experts or models results in creative synthesis. The Hegelian system represents the epitome of conflictual synthetic systems.
(5) The Singerian-Churchmanian System. In this system truth is perceived as pragmatic in the sense that it is relative to the overall goals and objectives of the inquiry. Truth is further perceived as teleological or explicitly goal oriented. The Singerian-Churchmanian system represents a holistic orientation toward inquiry in that the psychological, sociological and ethical components of a research problem are viewed as inseparable from its physical representation or theoretical presuppositions. It is the epitome of synthetic multimodel, interdisciplinary systems.
In its basic form the ID procedure is a regular Delphi procedure, which is a Lockean system (Turoff, 1975). It is interesting to note that a large proportion of the criticism directed at the Delphi model is formulated on the basis of Lockean/ Leibnizian perspectives. Perhaps this is partially due to the fact that both perspectives are best suited to dealing with well-structured, relatively simple research problems for which there exists either a large body of empirical evidence or well established scientific laws. The issues chosen for consideration in Delphi studies are typically ill structured, complex and associated with high levels of uncertainty. Such issues are very difficult to approach from Lockean/Leibnizian frameworks.
Nonetheless, Turoff (1975) pointed out that the Delphi is actually moving toward Kantian, Hegelian and perhaps even Singerian-Churchmanian patterns of inquiry, with greater emphasis increasingly focused upon its ability to draw forth divergent opinions and possible alternatives, synthesize new approaches to the issues under consideration, function as an educational tool, and initiate the process of holistic thinking and self-reflection.
However, we claim that the ID procedure clearly moves the general Delphi exercise toward Singerian-Churchmanian patterns of inquiry. ID suggests greater emphasis on a specific group of panellists and increases its ability to draw forth divergent opinions and possible alternatives for specific trends. It also synthesizes new approaches to issues under consideration, functions as an educational tool and initiates both the processes of holistic thinking and self-reflections (ID agreement).
We assume that the ID exercise, as a variant of the conventional Delphi technique, can accomplish a significant transition - a more coherent relationship between its practical application and its behavioural foundation.
In order to achieve this transition toward a Singerian-Churchmanian type of inquiry, the ID aims to focus on confronting structured trends, combining them with personal abstractions, and creating a ‘synergy’. That synergy is assumed to help a group of panellists to generate sound action to the issues under consideration demonstrating that it is a subjective inquiry generating opportunity.
Albertson and Cutler (1976), in the summary of their criticism of the Delphi procedures, contended that the forecast generated by a Delphi panel of experts represents a highly abstract and simplified image of the future, which is explicable in terms of the current world view held by the forecasters. However, in an ID procedure this critique can be turned upside down because ID has the advantage of aiming to achieve a self-reflective agreement.
Furthermore, the ID procedure does not limit areas of interest, as other Delphi procedures severely do, and favours responses within a wide scope of participant self-awareness. The ID procedure also encourages panellists not to concede any opinion on the basis of personality factors.
The ID may serve as a procedural response to some aspects of Sackman's (1975) critique. The ID does not capitalize on the fallacy of the expert halo effect, nor does it encourage conformity and penalize the dissident.
Some may contend that EFR aims exactly at eliciting the panellists’ or informants’ images of possible, probable and preferable futures. For that matter the EFR is also designed to encourage spontaneity and creativity on the part of the informants. Therefore, why do we need a new challenging ID tool for probing subjective abstractions?
Moreover, the EDFR procedure, which was subsequently developed to assure that the participants would be intensely involved in generating the issues to be considered for group response, basically looks for the same subjective future imagery.
In effect, EDFR intends to probe and construct personal future imagery that is extremely important in the process of assessing present concepts about some future alternatives. The ID, by contrast, is focused on confronting structured forecasts, combining them with personal abstractions, and creating a synergy - an Imen learning experience--a convergent emergence of experts’ studies and personal reflections.
Stated in its simplest form, the Imen procedure tends to invent and extrapolate personal conjectures and/or future imagery from panellists’ backgrounds. As far as we are concerned this procedure is the first to attempt to probe such a synergy.
Actually, concern was expressed regarding whether or not the panellists could accomplish any educated ideas throughout the procedure, after a relatively short exposure to future trends. We initially shared this concern and were delighted to discover, upon completion of such procedures (Passig, 1993, 1996, 1997), that their answers and recommendations far exceeded mere ‘thoughts’ and entered the realm of ‘scholarly suggestions’. We were pleased with the quantity and quality of the data generated by the panellists. On the basis of both the panellists’ evaluations and the results of several exercises we have conducted (Passig, 1993, 1996, 1997) and others we are still conducting, it appears that the ID procedure successfully meets the exploratory and purposive criteria and provides an emergence experience for the majority of the panellists who participate. On the basis of both the number of the future mission statements proposed by the panellists, and the quality of the recommendations, it appears that the ID procedure can transfer the panellists’ conceptual approach to the future to active, highly organized, clearly focused and value-oriented solutions.
Bohm and Feat (1987) suggest that the increasing fragmentation and specialization of science have led to ‘the point where the whole activity is losing its meaning’. Therefore, they say: ‘we need to change what we mean by science’, and bring about basic changes that ‘would represent a significant move toward liberating the surge of creativity that is needed if science is to help in confronting the deeper problems of humanity’.
We find Bohm and Feat's theme of calling for a new science powerfully articulated in Mitroff and Churchman's recent ‘Manifesto’ (1992). They provide definitions and concepts of science that are very different from the prevailing tradition. They believe that the institution of science should ‘exist primarily to serve humanity, and not the narrow specialized interests of disciplines; any science or discipline that loses sight of this fundamental principle not only forfeits its legitimacy, but its basic right to existence’. Here are a few main points of their manifesto:
Standing on the shoulders of the founding fathers of the systems movement, the ID procedure is a response to this ‘manifesto’ of systems service. It is our belief that the methodological contributions of this procedure demonstrate that scientists can realize the challenge of this vision. Systems science can be successfully placed at the service of humanity. The results of ID’s procedures that have been conducted (Passig, 1993, 1996, 1997) and others that we are still conducting demonstrate that science can leap out from the confines of the ‘abstraction’ and open our horizon toward social action. The suggested future mission formulated by the panellists, and their recommendations on how to promote it, placed the scholarship of this procedure into the service of human betterment and the improvement of the conditions of groups of panellists.
Albertson, L., and Cutler, T. (1976). Delphi and the image of the future. Futures 8(5), 397-404.
Barber, B. (ed.) (1971). Stability and Social Change, Little Brown, Boston, MA.
Bohm, D. and Feat, D. (1987). Science Order and Creativity, Bantam Books, New York.
Churchman, C. W. (1971). The Design of Inquiring Systems, Basic Books, New York.
Dalkey, N. C. (1967). Delphi. Rand Corporation, USA.
Harkins, A. and Kurth-Schai, R. (1983). OSCAR: an applied social technology variant of the Delphi method. Futurics 7(3), 1-7.
Helmer, O. (1966). Social Technology, Basic Books, New York.
Judd, Robert C. (1971). Use of Delphi method in higher education. Technological Forecasting and Social Change 4(2), 173-186.
Kurth-Schai, R. (1984). Reflections from the Hearts and Minds of Children: A Delphi Study of Children's Personal, Global, and Spiritual Images of the Future. Ph.D. Dissertation, University of Minnesota.
Leon, H. M. (ed.) (1982). Talcott Parsons on Institutions and Social Evolution, University of Chicago Press, Chicago.
Linstone, H. A. and Turoff, M. (eds) (1975). The Delphi method: techniques and applications. University of Chicago Press, Chicago.
McBride, A. (1974). Delphi technique and futures planning in Catholic education. Momentum May, pp.9-17.
Mitroff, I. I. and Churchman, W. C. (1992). A manifesto for the systems sciences: outrage over the state of science. General Systems Bulletin XXII(I), pp 23-41.
Morin, E. (1974). L'Unité de 1'Home: Invariants Biologiques et Universaux Culturels; Essais et Discussion. Presentés et Commenté par Edgar Morin et Massimo Piatteli-Palmarini, Edition du Seuil, Paris.
Morin, E. (1988). Simplicité et Complexité. Sous la direction de Mauro Ceruti êt Edgar Morin, Institute Italiano di Cultura, Paris.
Parsons, T. (1977). Social Systems and the Evolution of Action Theory, Free Press, New York.
Parsons, T. (1978). Action Theory and the Human Condition, Free Press, New York.
Passig, D. (1992). Back to my Future: A Handbook for Self Directed Awareness of Future Jewish Life, TT of Minneapolis, Minnesota.
Passig, D. (1993). Reactions to Experts’ Forecasts by a Group of Jewish Teenagers: An Imen-Delphi Exercise - An Applied Social Methodology - A Variant of the Delphi Forecasting Technique. PhD. thesis, University of Minnesota.
Passig, D. (1996). Developing communal future Jewish imagery with a group of teenagers. Journal of Jewish Communal Service 72(3), 210-216.
Passig, D. (1997). Imen-Delphi: a Delphi variant procedure for emergence: human organization. Journal of the Society for Applied Anthropology 56(1), 53-63.
Polak, F. L. (1972). In Crossing the Frontier of the Unknown. Toffler, A. (ed.), Random House, New York.
Poolpatarachewin, C. (1980). Ethnographic Delphi futures research: Thai University pilot project. Journal of Cultural and Educational Futures 2(4), 11-19.
Press, S. J. (1978). Qualitative controlled feedback for forming group judgments and making decisions. Journal of the American Statistical Association 73(363), 526-535.
Press, S. J. (1983). Multivariate Group Assessment of Probabilities of Nuclear War. (Technical Report #121), Department of Statistics, University of California, Riverside.
Pyke, D. (1970). Distribution of Characteristics of the Sulphide Ores of the Timmins Area, Department of Mines, Toronto.
Ranch, W. (1979). The decision Delphi. Technological Forecasting and Social Change 15(3), 159-169.
Sackman, H. (1975). Delphi critique: Expert Opinion, Forecasting and Group Process. Columbia University Press, New York.
Textor, R. B. (1979). The natural partnership between ethnographic futures research and futures education. Journal of Cultural and Educational Futures 1(1), 13-19.
Touraine, A. (1988). Return of the Actor: Social Theory in Post Industrial Society, University of Minnesota Press, Minneapolis.
Turoff, M. (1970). The design of a policy Delphi. Technological Forecasting and Social Change 2(2), 51-62.
Turoff, M. (1975). The Policy Delphi. In Delphi method - Techniques and Applications. Linstone, H. A. and Turoff, M. (eds). pp.6-15.
*Correspondence to: David Passig, Bar-Ilan University, School of Education, 52900 Ramat-Gan, Israel.