As a result of a Congressionally
Directed Activity, the Central Intelligence Agency (CIA)
conducted an evaluation of a 24-year, government-sponsored program
to investigate ESP and its potential use within the Intelligence
The American Institutes for Research (AIR)
was contracted to conduct the review of both research and
The American Institutes for
Research Review of the Department of Defense's STAR GATE Program
STAR GATE Program - A Commentary
by Edwin C. May, Ph.D.
(Journal of Parapsychology. 60. 3-23. March 1996)
An Assessment of the Evidence for
Psychic Functioning Professor Jessica Utts,
Division of Statistics, University of California, Davis
Evaluation of Program on Anomalous Mental Phenomena
by Ray Hyman,
University of Oregon, Eugene, Oregon
Response to Ray Hyman's Report
of September 11, 1995
"Evaluation of Program on Anomalous Mental Phenomena."
Professor Jessica Utts Division of Statistics University of
The American Institutes for Research
Review of the Department of Defense's
Program - A Commentary
by Edwin C. May, Ph.D.
Cognitive Sciences Laboratory
Palo Alto, California
(The Journal of Parapsychology. 60. 3-23. March 1996)
As a result of a Congressionally Directed Activity, the Central
Intelligence Agency (CIA) conducted an evaluation of a 24-year,
government-sponsored program to investigate ESP and its potential
use within the Intelligence Community.
The American Institutes for
Research was contracted to conduct the review of both research and
operations. Their 29 September 1995 final report was released to the
public 28 November 1995.
As a result of AIR's assessment, the CIA
concluded that a statistically significant effect had been
demonstrated in the laboratory, but that there was no case in which
ESP had provided data that had ever been used to guide intelligence
operations. This paper is a critical review of AIR's methodology and
conclusions. It will be shown that there is compelling evidence that
the CIA set the outcome with regard to intelligence usage before the
evaluation had begun.
This was accomplished by limiting the
research and operations data sets to exclude positive findings, by
purposefully not interviewing historically significant participants,
by ignoring previous DOD extensive program reviews, and by using the
questionable National Research Council's investigation of
parapsychology as the starting point for their review. While there
may have been political and administrative justification for the CIA
not to accept the government's in-house program for the operational
use of anomalous cognition, this appeared to drive the outcome of
As a result, they have come to the wrong
conclusion with regard to the use of anomalous cognition in
intelligence operations and significantly underestimated the
robustness of the basic phenomenon.
As part of the fiscal year 1995 defense appropriations bill,
responsibility for the government-sponsored investigation and use of
ESP* was transferred to the Central Intelligence Agency.
Congressionally Directed Action, the CIA was instructed to conduct a
retrospective review of the 24-year program, now known as STAR GATE,
that resided primarily within the Intelligence Community.
The analysis was to include the research
that was conducted since 1972 at
SRI International and later at
Science Applications International Corporation. In addition, the CIA
was to include an assessment of the intelligence-gathering utility
of anomalous cognition (AC), and the program history was to be
declassified (CIA Public Affairs Office, 1995). Initiated in June
1995, the evaluation was to be completed by 30 September 1995.
The CIA contracted with the American Institutes for Research (AIR)
to manage the review. They, in turn, formed a "blue-ribbon" panel
that included psychologist Professor Ray Hyman from the University
of Oregon and statistician Professor Jessica Utts from the
University of California at Davis. AIR contributed Michael Mumford,
Ph.D. and Andrew Rose, Ph.D. to the panel to provide unbiased
assessment on methodological issues. The President of AIR, David Goslin, Ph.D., served as coordinator of the evaluation effort.
I was asked by CIA to provide administrative support, technical
documents, and briefings on an as-needed basis for the review. This
work was supported by a small contract to Science Applications
The CIA-sponsored AIR investigation concluded that a statistically
significant laboratory effect has been demonstrated but more
replications were needed. In no case had the anomalous cognition
information ever been used to guide intelligence operations (Mumford,
Rose, and Goslin, 1995).
I question the validity of their and the CIA's conclusions because
Limited the data sets in the
As a way of officially ignoring anomalous cognition's
positive contributions to intelligence, only a small fraction of
the operational remote viewing database was examined. That was
the final data collected just before the unit closed, a time
widely known as problematic. In their laboratory evaluations,
they restricted the investigation to only the
government-sponsored research and then insisted on the need for
more outside replications. In doing so, they ignored the
conclusions of one of their own investigators who showed that
the government-sponsored research had been already been
Failed to contact significant
Because of the complexity of the 24-year
program, it is impossible to conduct an in-depth and accurate
evaluation without significant contact with the program's many
major participants. The review focused on the project's reports,
but they were written to satisfy specific contract requirements
and were not designed individually or in total to serve as a
program justification; thus, these documents provide a
substantially incomplete picture of the program.
In addition to questioning the validity
of CIA/AIR's conclusions, I find such serious problems with their
evaluation methodology that I have become reluctantly convinced that
their conclusions were set before their investigation began.
The investigators failed to:
Apply consistent criteria for
acceptance or rejection of anomalous cognition. The
investigators were troubled by possible non-AC alternative
explanations for the statistically significant laboratory
results, yet ignored similar alternatives for the failed
operations. For example, well-known psychological effects such
as bad morale, failed expectations, and a lack of a supportive
environment, were not discussed as potential alternatives for
the failed operations. In their positive forms, all of these
psychological effects are critical for excellence in any human
Avail themselves of the previous
exhaustive reviews conducted by various organizations within the
DOD, all but one of which was positive. Since the CIA was
allowed only four months to complete the evaluation, it is
surprising that they chose not to use this resource.
Reject a discredited evaluation of
parapsychology conducted by the National Research Council (NRC).
They knew that the NRC investigators were not cleared for access
to the vast majority of SRI's research, yet the AIR
investigation relied heavily on the NRC's review to question the
SRI research results prior to 1988.
Use neutral government scientific
evaluation resources such as the Military Services' or the CIA's
Scientific Advisory Boards. Instead they commissioned external
investigators with previously published conclusions about
parapsychology. The CIA could then justify whatever conclusion
they wished, because it would be consistent, by definition, with
at least one of their external reviewers.
To recognize a potential significant
conflict of interest for Dr. David Goslin, president of AIR and
a report co-author. He had administrative responsibility for the
discredited NRC investigation of parapsychology.
Finally, since the political situation
and the status of the program had significantly deteriorated
technically and administratively, I speculate that this contributed
to the underlying reason why the CIA did not want the program even
before the evaluation began.
In this paper, I will expand upon these topics to demonstrate
clearly that the outcome and conclusions drawn by AIR and
subsequently the CIA were set before the investigation began, and
that methodological and administrative choices were made to assure
that the results of the investigation would support the CIA's
In addition, I will document that they
have come to the wrong conclusion with regard to the use of
anomalous cognition in intelligence operations and greatly
underestimated the robustness of the phenomenon.
Critique of the CIA/AIR Conclusions
Limited Database for the Evaluation of Research and
The program evaluation was set from the beginning to only include
government-sponsored research. If the evaluation was confined to the
assessment of the scientific quality of the research, then perhaps
this is not a bad idea, given that the Congress was trying to
determine whether there was merit to continue. Upon closer
inspection, however, even in this case, limiting the scope of the
evaluation to exclude replications is scientifically invalid. The
evidence for or against a statistically-based phenomenon cannot rest
on the evidence provided by a few investigators in two laboratories
(i.e., SRI and SAIC). Rather, science demands that the evidence rest
in replications. Yet, the reviewers were requested not to look
outside the STAR GATE project.
In the CIA's briefing to Congress, they list three points as
attributed to the AIR investigation (May, 1995g) and I quote:
"the data do not establish that
a paranormal phenomenon is involved, nature of source not
"the data have not been
"the boundary constraints
critical to obtaining statistically significant experimental
results are not practical in real world of intelligence
No statistically based phenomena can be
established without replication, yet the investigators were
instructed not to look for any. (Utts, ignored this instruction and
clearly showed that a conceptual replication has been well
established in the literature and that significant statistical
consistencies existed between the SRI and SAIC data sets.) Since the
investigators were restricted at the outset, the top two bullets
above are true by construction-not by analysis.
A casual scan of my collection of technical journals found four
independent replications of remote viewing (Dunne and Bisaha, 1979;
Schlitz and Gruber, 1980; Schlitz and Haight, 1984; and Targ et al.,
1995). Rather than more replications as called for by AIR and Hyman,
what is needed is a meta-analysis of all the AC studies to date and
more attention on potential mechanisms.
Perhaps I should rest my case here. The CIA/AIR conclusions appeared
to be designed into the investigation. Their final bullet above is
questionable on its face value, because it is true by the nature of
intelligence, not because of a valid criticism of the program's
operational AC. The only valid measure of intelligence utility for
anomalous cognition is a top-level out-come measure, not a
statistical analysis. In short, do end-users come back for more? Do
any end-users have cases they can point to that helped solve an
intelligence problem? The CIA and AIR say no, but as I will show
below, that conclusion was also arrived ate by construction rather
than by analysis.
I first learned of the CIA/AIR's plan for the evaluation of the
intelligence value of anomalous cognition from Mumford during the
July meeting of the "blue-ribbon" panel at which I was invited to
present material and answer questions. At that date, Mumford claimed
that they were only going to look back three years from the end of
the 24-year program.
I told him that I was convinced that this would
not provide an honest picture of the utility of AC. I informed the
panel that I could easily predict the outcome based on my knowledge
of the morale of the government's viewers, the substandard
management by Defense Intelligence Agency (DIA) officials, the
tasking (i.e., what data they were after) and the inappropriate
Mumford attempted to justify his decision by saying he did not want
to rely on memory and hearsay. He would not have to, because there
was an extensive written history including testimonials to official
organizations within the Intelligence Community. Mumford reiterated
that he was sticking to his plan, regardless.
I objected to this decision to ignore existing data. I called the
individual at CIA who had been assigned to manage the review,
hereafter called the Point of Contact or POC, and insisted that some
of the officials I had previously named had to be contacted. I
learned later that the names and phone numbers of at least six
individuals had been given to the POC. These end-users were both on
active duty and retired who have already been on written record as
attesting to the value of AC-derived intelligence data in solving
After the AIR report had been given to Congress, but before it was
released to the public and before I had seen it, I called many of
the individuals on the list. Most were not contacted and those that
were, told the CIA representative the case specifics and value of
their individual circumstances. Some of the positive findings
occurred before the final year but within the last three years of
Finally, even a cursory investigation of the written record of
intelligence operations would have revealed substantial evidence of
the operational utility of anomalous cognition. Minimally, there
exists enough data to claim prima facie utility with regard to the
method, and selected cases are beyond doubt as to AC's specific
Joseph McMoneagle, one of the original government viewers beginning
in 1978 and a consultant to the SRI/SAIC and Cognitive Sciences
Laboratory, in 1984 was granted a Legion of Merit award for
excellence in intelligence service. The Legion of Merit is rarely
awarded for other than 20 or 30 years service, yet McMoneagle
received his on the following basis.
I quote, with permission, from McMoneagle's citation:
"...He [McMoneagle] served most
recently as a Special Project Intelligence Officer for SSPD, SSD,
and 902d MI Group, as one of the original planners and movers of
a unique intelligence project that is revolutionizing the
intelligence community. While with SSPD, he used his talents and
expertise in the execution of more than 200 missions, addressing
over 150 essential elements of information [EEI].
contained critical intelligence reported at the highest echelons
of our military and government, including such national level
agencies as the Joint Chiefs of Staff, DIA, NSA, CIA, DEA, and
the Secret Service, producing crucial and vital intelligence
unavailable from any other source..."
How is it that the CIA and AIR could not
find compelling evidence for the operational utility of anomalous
cognition? They clearly chose not to look.
Important Program Participants: Never
From 1985 through 1990, the research project at SRI International
enjoyed substantial, on-going, and written scientific oversight of
the major portion of the AC database at SRI.
Twelve individuals, who
are world-renowned in their individual disciplines, were chosen by
the client and other government officials to serve on our Scientific
Oversight Committee. In addition, they were selected on the basis of
the scientific reputations and on the basis of their skepticism.
"Believers" were not allowed on the committee.
The SOC's responsibilities were
Evaluate our written protocols prior
to conducting any experiments. The protocol that was actually
used for each investigation was the convergence of round-robin
exchange with the SOC.
Exercise un-announced drop-in
privileges to observe experiments in progress. Approximately one
half of the SOC availed themselves of this opportunity.
Review the then classified final
research reports as if they were technical journal submissions
in their individual disciplines. The disciplines included
physics, philosophy, psychology, electrical engineering,
statistics, and astronomy. Their reviews were in writing and
appended, un-edited, to our each final report.
Suggest approaches for research in
the next year of the 5-year contract.
During the SAIC time, the SOC was
limited to only five members but they had the same charter. Three of
the five came from the SOC at SRI. At SAIC we established two
additional oversight committees. An Institutional Review Board
(i.e., human use committee) was established with nine members who
were health and medical professionals and are renowned in their
disciplines as well. The list included one Nobel laureate as did
SAIC's Scientific Oversight Committee. Besides assuring the
protection of our human subjects, they also served as a less formal
scientific oversight committee.
The third oversight committee at SAIC was for policy. The three
members of this committee came from formerly very senior positions
in the DOD and their job was to assure that we were meeting our
obligations to the DOD and supporting its mission.
Of these 17 individuals who had intimate knowledge of the inner
workings of this project, scientifically, methodologically, and
administratively only one was contacted by CIA. It was that single
individual who provided the names of satisfied end-users I discussed
The SOC's comments were available to the AIR reviewers in written
form, and many of the committee members lived on the east coast and
even a few lived in Washington. The CIA/AIR investigators could have
easily contacted them. They didn't.
The failure to contact significant program participants does not end
with these committees. I provided the POC with the names and phone
numbers of numerous other pertinent individuals. The list included
the previous project director for STAR GATE who had retired less
than a year before the review and the Commander for a
still-classified client who initiated a single contract that
accounted for a significant fraction of all the funding for the
project over the 24 years.
In addition, I gave the POC the names of
a number of the original government viewers. In short, with
interviews of mostly local people the CIA could have gained
significant insight to the scientific, operational, managerial, and
political aspects of the STAR GATE project.
They chose to ignore these resources.
One of AIR's significant methodological flaws is important with
regard to the assessment of operations.
In the Section on the
Evaluation Plan in the report, Mumford et al. (Page 2-1, 1995)
correctly required of the laboratory investigations,
"...unambiguous [emphasis added]
evidence for the existence of the phenomenon... ."
Following this lead, Hyman hypothesized
a number of alternative explanations for the observed statistical
significance other than the anomalous cognitive one, although he
admits he couldn't find any obvious flaws in the methodology (Mumford
et al., 1995, Page 3-75).
For example, he is troubled that during
the SAIC research, a single judge was used to conduct all the
laboratory evaluations. Although Hyman does not propose how this
might effect the result, he is correct in proposing the hypothesis
that it might somehow affect the outcome. (Hyman lists other
alternatives as well, but this one illustrates the point.)
As it turns out, Utts finds statistical
homogeneity (i.e., meaningful consistency) among the results from
SRI, SAIC, and replications elsewhere when that single judge was not
involved. Thus, this hypothesis must be rejected. This same
consistency also rejects the other alternatives Hyman proposes, as
Yet, AIR fails to apply the same "unambiguous" criteria to their
evaluation of the efficacy of AC in intelligence operations. In this
case, why operations may have failed. In particular, in their
discussion in the Section on Evaluating the Utility of Remote
Viewing in Intelligence Operations they list a number of "boundary
conditions" that might affect anomalous cognition in operations.
These include a number of physical and methodological issues such as
feedback and whether a sender or distance to the target might be
They did not discuss or propose any psychological issues that may
have been the deciding factors as to why the operations failed in
their limited sample. For example, it is well-known that human
performance of any kind and most certainly AC-performance is
profoundly affected by the morale, the expectations of the
participants, and the emotional environment in which the performance
is expected (e.g., home-team effect in sports).
But none of these
potentially critical factors was discussed in the context of
reaching the unambiguous conclusion that AC was useless in
I had discussed these points in my meeting with the blue-ribbon
panel in July, 1995. In particular, having spent considerable time
with the government remote viewing unit, I was knowledgeable about
what psychologists call "set and setting."
That is, I saw first hand
and reported to the panel that during the last two years (i.e., the
time of the operational evaluation) the emotional environment had
deteriorated to the point that the viewers wanted to leave the unit,
and some of the staff had already left in disgust (May, 1995i). The
morale was so low that doing excellent remote viewing, or
practically anything else, would be out of the question. The AIR
investigators interviewed the government remote viewers (Mumford et
al., 1995, Page 4-9) and learned of these problems, first hand (May,
These critically important factors were completely left out of the
discussion in the report and no alternate hypotheses were suggested
to question their "unambiguously negative conclusion about the use
of AC in intelligence operations.
Previous Program Reviews
Even before I was officially under contract with CIA, I gave the POC
either copies of, or pointers to, a number of classified program
reviews that had been conducted in the past.
One important aspect of the program was its on-going and rigorous
review and technical oversight. Everyone involved (i.e., the
government sponsors, SRI, and SAIC) were correctly concerned that
the research should be as rigorous as possible and that the program
could be justified within the Intelligence Community and DOD. These
reviews were extensive and were conducted by General military
officers, senior members of the Intelligence Community, respected
scientists from many disciplines, and end-users of the AC
These remain classified, and with one exception, were positive with
regard to the existence of AC and its successful contributions to
intelligence. Even the negative one only wanted to stop the research
but continue the operations! The final such review was conducted in
In addition to the written reviews, from 1985 through 1990 the
program enjoyed the continued oversight of a high-ranking military
officer from the still-classified sponsor and a GS-15 geneticist
from DIA as permanent on-site observers at SRI.
The POC is a Ph.D. scientist and at the time seemed dedicated to the
best job possible. He informed me, however, that the CIA intended to
ignore the previous reviews and start fresh. Given that the review
had to be in Congress in four months, I could not conceive how it
could be effective and accurate and ignore the substantial amount of
After all, a complete analysis could,
and should have, included a review of the previous classified DOD
A Thread of Bias, Potential Conflict of
Interest, and Suppression of Data
In the early days of the project, Targ and Puthoff (1974a) reported
on a series of experiments they conducted at SRI with Mr. Uri
Geller, an Israeli magician/psychic.
George Lawrence from the
Advanced Research Projects Agency (ARPA) accompanied by two
consultants, Ray Hyman and Robert Van de Castle, came to SRI
requesting an opportunity to see an experiment in progress with
Puthoff and Targ correctly denied access
to the ARPA representatives because of technical and administrative
protocol issues. After all, with such controversy swirling about
Geller, it is easy to become quite paranoid about who is trying to
trick whom. The safest and the most scientifically sound course is
not to allow anyone except the direct research team to witness
formal experiments regardless of credentials (Targ and Puthoff, 1977
and May, 1996).
Yet, as part of their cover story, Time magazine (Jaroff, 1974)
quoted Ray Hyman's claim that the SRI tests were carried out with
"incredible sloppiness." The irony is that the tests that Hyman and
Lawrence witnessed at SRI were indeed conducted with "incredible
sloppiness," but the experiments they witnessed were of their own
making and had nothing at all to do with protocols of those
experiments to which they had been denied access (Targ and Puthoff,
1974b and May, 1996).
It is clear that Lawrence and Hyman had
strongly held positions and were willing to report their experiences
at SRI inaccurately. Thus we see the first evidence of a negative
bias on the part of Lawrence and Hyman.
In 1984, their biases were again demonstrated. The Army Research
Institute (ARI) commissioned the American Academy of Sciences to
investigate the potential of certain techniques that propose to
enhance human performance (Druckman and Swets, 1988). Although
performance enhancement has never been the claim of research
parapsychology, the National Research Council included
parapsychology as one of the topics to be studied.
The same George Lawrence formerly from ARPA was ARI's project monitor, and he asked that
Ray Hyman be
commissioned to head the investigation into parapsychological
phenomena. David Goslin, Executive Director of the Commission on
Behavioral and Social Sciences and Education for the National
Research Council, served as overall project director and agreed to
On parapsychology, the NRC study concluded (Druckman and Swets,
"The committee finds no scientific
justification from research conducted over a period of 130 years
for the existence of parapsychological phenomena. It therefore
concluded that there is no reason for direct involvement by the
Army at this time. We do recommend, however, that research in
certain areas be monitored, including work by the Soviets and
the best work in the United States.
The latter include that
being done at Princeton University by Robert Jahn; at Maimonides
Medical Center in Brooklyn by Charles Honorton, now in
Princeton; at San Antonio by Helmut Schmidt; and at the Stanford
Research Institute by Edward (sic) May. Monitoring could be
enhanced by site visits and by expert advice from both
proponents and skeptics. The research areas included would be
psychokinesis with random even generators and Ganzfeld effects."
By the time the NRC began their
investigation, I was the project director at SRI International.
program was highly classified at that time and special access was
required before any aspect of the project could be discussed even
with individuals with appropriate security clearences.* Thus,
neither the in-house DIA classified program nor the NRC
investigators, and particular Ray Hyman, had access to over 80% of
all the remote viewings conducted during the SRI years. None of the
research reports from this contract were kept with the DIA remote
viewing group. So even though Hyman had access to the this group, he
was denied access to and probably even unaware of the SRI data of
that time period.
I was not even allowed to meet with Hyman in our laboratory or
office space; he and I met in a separate building at SRI that was
not associated with project. Our discussions were confined to our
published account of a careful random number generator experiment
that we had conducted in 1979.†
In the overall summary shown above, remote viewing was not even
mentioned although an analysis of the early studies at SRI and later
studies at Princeton are contained in the body of the NRC report.
With regard to their conclusion on remote viewing:
"...the literature on remote viewing
has managed to produce only one possibly successful experiment
that is not seriously flawed in its methodology-and that one
experiment provides only marginal evidence for the existence of
The parapsychology section of the NRC
study was a mockery of good science and serves as an excellent model
for a pseudo-scientific investigation. The methodology for the NRC
investigation and their conclusions were soundly criticized and
shown to be without scientific merit (Palmer, Honorton, and Utts,
The four major points drawn by Palmer et al. are summarized:
"The NRC claimed they could find no
evidence for parapsychological phenomena during the last 130
years, yet they examined only 10% of the systematic scientific
effort in parapsychology."
"The two principal evaluators of parapsychological research, Ray
Hyman and James Alcock, were publicly committed to a negative
position on parapsychology at the time the NRC research
Committee was formed. [Note added by May: In addition, the
phrase "..the total accumulation of 130 year's worth of
psychical investigations has not produced any consistent
evidence for paranormality..." can be found in Hyman (1986) and
the NRC conclusion (1988), and thus demonstrates his stated bias
before the NRC investigation was complete.]"
"The Committee's method of assessing parapsychology violates its
own stated guidelines for research evaluation, which specify the
identification and assessment of plausible alternatives. With
regard to the better parapsychological experiments, the
Committee admits, "We do not have a smoking gun, nor have we
demonstrated a plausible alternative" (Druckman and Swets, 1988,
"The report selectively omits important findings favorable to
parapsychology contained in one of the background papers
commissioned for the Committee, while liberally citing from
other papers supportive of the Committee's [negative] position.
The principal author of the favorable paper, an eminent Harvard
psychologist, was actually asked by the Chair of the NRC
Committee to withdraw his favorable conclusions."
This last point is particularly
troublesome and reveals the political nature of what should have
been a carefully conducted scholarly investigation that usually
characterizes the National Research Council.
Violating one of the
basic tenets of science to report all findings, the NRC Committee
asked Professor Robert Rosenthal to:
"...omit the section of our paper
evaluating the Ganzfeld research domains. I refused to do so but
was so shocked and disappointed by this request that I discussed
this request with a number of colleagues in the Harvard
departments of Psychology and of Statistics. Without exception
they were as shocked as I was.
In the end, censorship did not occur, and Monica Harris' and my
paper is available in its entirety in a kind of preprint format
from the National Academy Press.*"
Rosenthal's and Harris' commissioned
paper listed the Ganzfeld methodological quality to be superior to
the typical quality of the other four areas they considered
In addition to the significant methodological flaws and the attempt
to suppress positive findings, the NRC study was essentially
contradicted in it's major conclusion by a one-day workshop hosted
by the Office of Technology Assessment, the research arm of the US
Congress (Office of Technology Assessment, 1989). The OTA did not
completely exonerate the field of research parapsychology; there is
no scientific endeavor that cannot be improved. The OTA did,
however, clearly demonstrate that the research cannot simply be
dismissed-a view directly opposite to the NRC's conclusion.
In continuing the development of a potential conflict of interest, I
point out once again that David Goslin had administrative
responsibility for this seriously flawed NRC investigation.
When the CIA was searching for someone to conduct their technical
review of the STAR GATE program, they were turned down by the
National Research Council in part because of the time constraint and
in part because of the substantial negative publicity that resulted
from their previous report on parapsychology (May, 1995e). Instead,
AIR was commissioned to conduct the review. AIR's president is David
Let me now summarize the thread of bias and potential conflict of
interest. Ray Hyman and George Lawrence were denied access to SRI
experiments with Uri Geller in 1974. Ray Hyman has a long history of
a negative bias with regard to parapsychology. In 1985, George
Lawrence commissioned the National Academy of Sciences to
investigate parapsychology and picked Hyman to direct the effort. In
1986, David Goslin presided over a methodologically flawed review.
In 1995, David Goslin assumed responsibility for the CIA-sponsored
investigation of the STAR GATE program.
It is not a surprising that the NRC study is liberally quoted in the
AIR report because it supports the possibly predisposed views of
CIA/AIR, albeit from a flawed investigation.
Since Professor Jessica Utts was one of the co-authors of the formal response to the NRC
study, I questioned her (May, 1995f):
"Since you were a contributing
author to the reply [to the NRC investigation] and since the
reply soundly criticized the NRC's review methodology, I was
surprised to see that you did not mention the NRC study or the
PA's [Parapsychological Association] reply in your section of
the AIR's report. Considering the weight that the AIR
investigators placed on the NRC study, I feel it was a
substantial oversight for you not have added your first-hand
criticism of the NRC report as part of your remarks."
So that I make no errors in
interpretation, I print, with permission, her complete reply (19
"This is in response to your
question about why I did not mention the National Research
Council's 1988 evaluation of parapsychology in my report to AIR.
The answer is that I was explicitly asked by AIR staff NOT to
mention the NRC report in my review! This is very troubling to
me for a number of reasons.
First, you are correct in stating that I was aware that the NRC
committee was not shown much of the relevant remote viewing data
when they did their review, and that they did not in fact even
know the data existed. As you also noted, I co-authored a
critical review of the NRC report shortly after it was
published, illustrating a number of weaknesses with it.
What you may not know is that in addition to those problems, the
statistical method the NRC committee relied on for its findings
(called "vote-counting") has been completely discredited, and is
known to produce misleading results. I raised this point at the
July meeting Ray Hyman and I attended with the AIR staff at
their Palo Alto office, and it was substantiated by Stanford
Statistics Professor Lincoln Moses, who had been asked by the
AIR staff to attend the meeting to comment on that and related
statistical issues. (Had the NRC committee included a
statistician, that serious flaw, and the subsequent misleading
results, may have been avoided. I am sorry to say that even at
our meeting in Palo Alto, Ray did not seem to understand the
problem, and he was the principal "statistician" for the NRC
When I was explicitly asked by AIR staff NOT to mention the NRC
report in my review, I assumed they had realized the problems
with it, and, especially given the involvement of the AIR
President with the NRC Committee, were happy to let it fade into
Given that background, I was quite disappointed to see that AIR
made liberal use of the NRC report in their conclusions. Had I
known they were going to do that, I certainly would have
discussed the multiple problems with it in my report. By not
mentioning it, an uninformed reader may assume that I support
it, which I certainly do not.
I would also like to explain another omission in my report that
occurred for much the same reason. Despite the claims Ray Hyman
is making in the media, we were shown very little of the
"operational" remote viewing work. One of the few documents we
were shown was a list of "[the former DIA project officer's]
best" remote viewing successes. Since the list provided almost
no detail, you may recall that I asked you for names and numbers
of individuals I could contact to get more information about
those purported operational successes. In a memo dated August 1,
1995, you provided me with phone numbers for [ a former DIA
project officer, a former senior DIA official, a military
General who had program responsibility], and Joseph McMoneagle.
You sent a copy of the memo to the AIR staff.
Shortly after you sent me that memo, I was contacted by the AIR
staff and told that I was NOT to contact any of those
individuals. Thus, I was not able to gain any details about the
operational remote viewing work. I thought you should know that,
in case you were wondering why I requested that information and
then did not use it. Again, I am clueless as to why Ray Hyman is
making claims in the media that we had access to the operational
work for our review. I do not think he was given access to any
information not shown to me. I don't know how he can
substantiate the claims he's making about remote viewing being
useless for intelligence.
He may be correct, but he has very
little data on which to base that conclusion."
While a case can be made that Professor
Utts should not be contacting people with regard to operations
because she did not possess a clearance at the time, the individuals
I named are professionals and would not disclose classified
information to an uncleared person.
Regardless, the AIR
investigators cannot be excused from the attempt to suppress
intellectual findings by, or to limit the research of, a noted
academic that may be germane to the stated goals of the
The NRC study was discredited in print and I had discussed that
issues in detail with AIR's blue ribbon panel.
Biased Investigators on the AIR's
Since our research program had been reviewed by various Science
Advisory Boards including DIA's, it seemed prudent and natural that
the CIA should ask their own Board or one of many that reside in the
Washington area to conduct the program's technical evaluation. I
even provided names and phone numbers of individuals who I know on
various boards to expedite the contact.
Instead, Utts and Hyman were chosen to act as the expert reviewers.
At first glance, this seems like a reasonable approach given that no
learning curve would be required. I told the POC that I thought this
was not a good plan and that I could easily predict their
conclusions based on their previous writing. See Hyman (1986) and
Utts (1991) as samples. I reiterated that an in-place Science
Advisory Board would better serve that evaluation.
What better way to conclude whatever you wish than to build into the
evaluation protocol a priori stated scholarly views that are known
to span the opinion space. This guarantees that the concluding
remarks by CIA will, by definition, be consistent with some
evaluator on the team. That is exactly what happened.
In the CIA's presentation to Congress,
eight separate bulleted points are allotted to Hyman's conclusion
while only four are allotted to Utts' and none are given to Utts'
important rebuttal to Hyman (May, 1995g).
Good Advice Ignored
Since most of the work under review occurred under my watch as the
contractor program director, I could obviously not be involved in
the analysis directly, but as part of my contract responsibility, I
was asked to advise the review process.
In a 4-page document (May,
1995a), I indicated in words and figures how a review might proceed.
The major point was that acceptance criteria for operations and
research should be set prior to the review so that they could be
used to judge the validity of the program in an unbiased way.
(Arguably, one could say that I had a vested interest in the outcome
and my views should be ignored; however, I only provided suggestions
from a top-down perspective and did not suggest any details that
could be considered self-serving. It was beneficial to the program
and to me personally to have the most honest and rigorous review
possible, and I was completely confident that such a review could
only be positive.)
The criteria for the research could easily be adopted from the
established and accepted scientific rules for evidence.
my memorandum (May, 1995a):
"The existence of anomalous mental
phenomena cannot be statistically determined from the results of
a single laboratory. The requirements for replication of a
statistical phenomenon and the methods for the analysis of
cross-laboratory results are well developed."
Not only was this advice ignored, it was
ignored by fiat. The reviewers were instructed to only look at
research results from SRI and SAIC. Fortunately for scientific
credibility, Professor Utts ignored this statistically invalid
directive. Such action by CIA with regard to their review can only
add to the evidence that they were either only interested in a
negative outcome or statistically naive.
Determining the efficacy of operations was much more difficult.
Would one successful operation be sufficient justification to
continue the program, or would all the operations have to be useful?
What constitutes a successful operation? A one percent hit rate
might be considered miraculous by one customer, but a 50% hit rate
might be useless to another. I made no attempt to suggest what that
judgment criteria should be; I only urged that it be set in advance.
It was not done as a matter of official policy or even informally as
a guideline. As it turned out, the POC later informed me that only a
single case would be sufficient as far as he was concerned, but he
was careful to say that the decision was being made at "a much
higher pay grade then his." I learned later that they were only
going to examine the last set of AC operations from the 24-year
program. I and they knew that these cases were not representative of
the program at large. This point will be expanded below.
Early in the review, I was request to provide a list of my 10-best
examples of research that supported the existence of anomalous
cognition. In a memorandum (May, 1995b), I complained about that
In part, I quote:
"Since the complete document set
will be available to AIR, I recommend the following approach:
For the period at SRI from 1973
to 1989 (this also covers the pre NRC report date) use the
[in-house] meta-analysis as a guideline for the assessment
with spot checks to the primary documents to validate the
Use all the work conducted under
the SAIC program from 1991 through 1994 as the simplified
test set of documents. I think that includes 4 final reports
and perhaps 10 major projects within that set.
Conduct the final evaluation
from both sources of data. (One thing that could be done is
to use the results of the meta-analysis of the SRI data to
predict what might happen during the SAIC research. The
meta-analysis could be predictive only if there were a
genuine phenomenon. In my view, this would add to the
This approach avoids the file draw
problem [i.e., not publishing studies that fail to meet
statistical significance] altogether and includes most of the
documents I would count as my 10 anyway. I can only think of a
few other studies that I might want to include and all of them
have been accepted for publication in peer-reviewed journals."
I responded in part again to the same
request (May, 1995c):
Although the request seems straight
forward at the outset, to establish the existence of Remote
Viewing on the basis of a subset of the total data set does not
conform to the accepted practice for meta-analysis as set forth
in Rosenthal (1991) and Hedges and Olkin (1985).
I went on to comply to the request in
such a way that the complete record would be examined to avoid any
accusation of a so-called "file-drawer" problem by including in my
list a detailed in-house meta-analysis covering the period from 1973
to 1989 (May, Utts, Trask, Luke, Frivold, and Humphrey, 1989). This
analysis was conducted as part of contractual requirement to a
AIR ignored the CIA directive by including the National Research
Council's review of parapsychology as a support for their
conclusions about research. Knowing full well that the NRC
investigators did not have access to any SRI reports from 1985
onward (May, 1995d), they featured it prominently in their final
Little Contact with the Program's Principal
I would like to emphasize my role, or lack of it, in the CIA/AIR
evaluation of the STAR GATE program.
As I said before, it was
inappropriate for me to be involved in the actual assessment;
however, it is especially important to learn from the critical
details that never make it into official reports.† To illustrate my
point, of all the "blue-ribbon" panelists, Professor Utts was the
most familiar with the project; she had served as a visiting
scientist for a year during the SRI era.
Even with her intimate knowledge she
called me at least 12 times to seek clarification on specific points
in the documents she was reading. Professor Hyman never called and
the AIR team not only did not call but refused to return my
multi-faceted communication attempts. As a result of AIR negligence,
their report contains numerous errors of fact and errors of
I was the director of the government-sponsored investigation of
anomalous mental phenomena for 10 of the 24-year history. I presided
over 70% of the total contractor budget, 85% of the program's data
collection, and had intimate knowledge of and responsibility for the
project. For AIR to not use this resource is scientifically
As the review process was coming to an end, I formally sought the
opportunity to provide a written commentary to the AIR report to be
included with the blue-ribbon panel's reports (May, 1995h). Given
that Utts and Hyman were given space to comment on each others
work,* and since most of the science that was being reviewed was
work conducted under my direction, it seemed only natural to include
That request and a similar one to AIR was ignored.
Political Reason Why CIA may not have
Wanted the Program
Under the reluctant auspices of the DIA, the program transitioned
from SRI to Science Applications International Corporation in 1991.
We recognized shortly thereafter that DIA did not welcome the
responsibility as the contracting agency. The reason DIA management
was not anxious to have the program was complex and not associated
with the technical aspects.
Some of the DIA management had previous
negative experiences with senior military officers who had become
"believers," oversold the program's capability, and were known as
"loose cannons" in the community.
This reluctance manifested in two important ways. First of all, the
initial financial support for the program in 1991 came directly as
part of the supplemental Defense Appropriations bill and was
considered by Congress as "seed" money. DIA was expected to request
follow-on support as part of the overall DIA annual budget requests.
Those requests never happened; all program support through 1995 came
from the Appropriations bills.
One consequence was, that a member of
the staff of the Senate Appropriations Committee became increasingly
disappointed with the DIA and began to micro-manage the program with
disastrous results. A second consequence was that an attempt was
made in 1993 to transfer the program to CIA. No willing recipient
could be found there to accept the program. Even then the CIA did
not want program responsibility.
Secondly, the negative attitude from senior DIA management filtered
down the chain of command. For example, the final project officer
who had direct responsibility for the program before it closed had
little knowledge of the program; no knowledge of its substantial
history; no technical background to manage such a project; ignored
the research results; and created a crushing atmosphere with his
management style. The morale was so bad that viewers and officials
within the government's remote viewing unit repeatedly asked me to
This placed me in a very difficult position as a
contractor. I informed middle management at DIA of the problems with
In short, the program was in shambles. The operations that were
conducted during the last few years of the project, for the most
part, were destined to and did fail. It was this program, including
personnel, that was to be transferred to CIA by 1 July 1995. In my
professional opinion, which I shared with the POC, the program, as
it was configured, would not produce successful AC intelligence
So, CIA had strong and valid reasons not to want the program. The
Agency was soundly criticized in the press for mishandling the Ames
case and other excesses, so they did not need another controversy.
In my opinion, the last thing they would want would be to inherit a
Congressionally micro-managed program in severe internal distress no
matter what its content or potential might be. Yet, by law they had
to comply with the Congressional Directed Action and conduct the
No wonder that it was probably done in
such a way to assure a negative outcome with regard to operations.
It is impossible for me to prove whether or not the CIA determined
the outcome of the investigation before it began. What is obvious,
however, is that the evaluation domain of the research and
particularly the operations were restricted to preclude positive
The CIA did not contact or ignored people who possessed
critical knowledge of the program, including some end-users of the
intelligence data. Investigators were chosen who either had
previously published conclusions or who possessed a serious
potential for a conflict of interest.
With the exception of the significantly
flawed National Research Council's review, all the DOD previous
evaluations of the research and intelligence applications were
ignored. I am forced to conclude that either the AIR investigators
were not competent to conduct a proper review of such a complex
program-a view to which I do not subscribe-or they knew exactly what
they were doing; they wanted to demonstrate a lack of intelligence
utility for anomalous cognition. They did so by construction rather
than by careful analysis.
Let us grant for the moment that my supposition is true; the CIA
wanted to kill the program. Why was such a detailed rebuttal
necessary? After all, an agency should be able to express their
wishes with regard to the acceptance of any program that the
Congress might assign. In fact, I see it as part of the various
agency's responsibility to inform Congress of what might, or might
not, be possible.
Rejecting the STAR GATE program on the basis of an
incomplete and incorrect analysis not only creates a false legacy,
it does not easily allow for other organizations in the public or
private sector to assume responsibility for a new version of the
Aside from setting the record straight,
I felt obligated to show that as the result of their flawed
methodology, the CIA/AIR greatly underestimated the statistical
robustness of the research results and significantly undervalued the
potential for anomalous cognition in intelligence operations.
Druckman, D. and Swets, J A. Ed.
(1988). Enhancing Human Performance. National Academy Press,
Washington, D.C. 167-208.
Dunne, B. J. and Bisaha, J. P.
(1979). Precognitive Remote Viewing in the Chicago Area: A
replication of the Stanford Experiment. Journal of
Parapsychology. 43, No. 1. 1-16.
Hedges, L. V. and Olkin, I. (1985).
Statistical Methods for Meta-Analysis. Academic Press, Inc.
Harcourt Brace Javanovich. New York, NY.
Hyman, R. (1986). Parapsychological
Research: A Tutorial Review and Critical Appraisal. Invited
Paper. Proceedings of the IEEE. 74, No. 6. p. 825.
Jaroff, L. (1974). Boom Times on the
Psychic Frontier. Time Magazine. 4 March. 56-72.
May, E. C., Utts, J. M., Trask, V.
V, Luke, W. L. W., Frivold, T. J, and Humphrey, B. S. (1989).
Review of the Psychoenergetic Research Conducted at SRI
International (1973-1988). Final Report-Task 6.0.1, Project
1291. SRI International, Menlo Park, CA.
May, E. C. (1995a). Memorandum to
POC. 6 June 1995.
May, E. C. (1995b). Memorandum to
POC. 14 June 1995.
May, E. C. (1995c). Memorandum to
POC. 19 June 1995.
May, E. C. (1995d). In May's only
meeting with the AIR "blue-ribbon" panel during which the lack
of NRC access was discussed in detail. July 1995.
May, E. C. (1995e) Personal
communication. The CIA point of contact.
May, E. C. (1995f). E-mail letter to
Professor Utts, 17 December.
May, E. C. (1995g). Personal
communication. US Senate Appropriations Committee Staff Member.
May, E. C. (1995h). Memorandum to
POC. 14 August 1995.
May, E. C. (1995i). Personal
communication. Foreign analyst for the unit, June, 1995.
May, E. C. (1995,j). Personal
communication. A government remote viewer, August 1995.
May, E. C. (1996). Personal
communication. Hal Puthoff.
Mumford, M. D., Rose, A. M., and
Goslin, D. A. (1995). An Evaluation of Remote Viewing: Research
and Applications. The American Institutes for Research report.
Office of Technology Assessment
(1989). Report of a Workshop on Experimental Parapsychology.
Journal of the American Society for Psychical Research. 83, No.
Palmer, J. A., Honorton, C. and Utts,
J. (1989). Reply to the National Research Council Study on
Parapsychology. Journal of the American Society for Psychical
Research.. 83, No 1. 31-50.
Public Affairs Office, Central
Intelligence Agency, 6 September 1995.
Rosenthal, R. (1990). Letter to the
Editor. Psychological Science.. 1, No. 5. p.329.
Rosenthal, R. (1991). Meta-Analysis
Procedures for Social Research.. Sage Publications, London, UK.
Schlitz, M. J. and Gruber, E.
(1980). Transcontinental Remote Viewing. Journal of
Parapsychology. 44, No. 4. 305-318.
Schlitz, M. J. and Haight. J.
(1984). Remote Viewing Revisited: An Intrasubject Replication.
Journal of Parapsychology. 48, No. 1. 39-50.
Targ, R. and Puthoff. H. E. (1974a).
Information Transmission Under Conditions of Sensory Shielding.
Nature. 252. 602-607.
Targ, R. and Puthoff, H. E. (1974b).
Geller: experimenters reply. New Scientist. Letters. 7 November.
Targ, R. and Puthoff, H. E. (1977).
Mind-Reach. Delacorte Press.
Targ, R, Katra, J, Brown, D., and
Wiegand, W. (1995). Viewing the Future: A Pilot Study with an
Error-Detecting Protocol. Journal of Scientific Exploration. 9,
No. 3. 367-380.
Utts, J. (1991). Replication and
Meta-Analysis in Parapsychology. Statistical Science. 6, No. 4.