EWG Comments to BPA Advisory Panel

Assessment Fails Basic Scientific Standards

August 2007

BPA Experts Find Hundreds of Errors in Government Assessment

Download a PDF of EWG's comments below, or view the press release.

August 6, 2007

Dr. Michael D. Shelby
Director
Center for the Evaluation of Risks to Human Reproduction
National Institute of Environmental Health Sciences
Department of Health and Human Services
P.O. Box 12233
MD EC-32
Research Triangle Park, NC 27709

Re: Failure of CERHR Assessment of BPA to Meet Basic Scientific Standards. Supplemental Comments on the Interim Draft NTP-CERHR Report on the Reproductive and Developmental Toxicity of Bisphenol A.

Dear Dr. Shelby:

You must be aware of the publication last week of a consensus statement on bisphenol A (BPA) signed by 38 independent specialists in BPA toxicity from around the world. These scientists concluded that BPA presents a clear risk to human health (CHCS 2007). The statement and the comprehensive review papers that accompany it underscore, by way of contrast, the hopeless corruption of the ongoing review of BPA being conducted by your Center, the Center for the Evaluation of Risks to Human Reproduction, or CERHR.

The Environmental Working Group (EWG) has conducted a detailed analysis of the comments by 9 scientists conducting BPA research at 5 laboratories in the U.S. and E.U., submitted to you as public comments in response to CERHR’s interim draft BPA assessment (Vandenberg et al. 2007; Schonfelder 2007; Prins 2007; vom Saal 2007; Welshons 2007; Zoeller 2007). Our analysis shows that the CERHR panel’s assessment of BPA utterly fails to meet basic, universally understood standards for scientific reviews and data quality, including those laid out in NIH policy and federal law.

These standards require that assessments be accurate, unbiased, consistent, complete, and conducted by those with the necessary expertise to ensure objectivity. Instead, our review of scientists’ comments reveals that the CERHR assessment might contain nearly 300 errors of fact and interpretation; is biased, inconsistent, incomplete; and clearly fails to meet the most basic scientific standards. Among our findings, which are detailed in the attached table, are that the CERHR assessment is:

• Inaccurate – 297 errors: Reviewers identified 297 potential errors in documentation and analysis of study results, and in interpretation of the study findings and their significance, that are in conflict with the peer reviewed literature.
• Incomplete – 195 instances of incomplete study reviews: Reviewers documented 195 instances where the panel assessment is incomplete, including incomplete documentation of relevant test results or missing justifications for panel assertions.
• Inconsistent – 48 basic inconsistencies: Reviewers documented 48 instances in which the panel inconsistently applied criteria for study evaluation.
• Biased. The assessment heavily favors industry studies over government and independent studies. In its most recent assessment, the Panel rejected government and independent studies at nearly 3 times the rate of industry studies (Vandenberg et al. 2007).

Consider also the following, striking differences between the CERHR panel and the BPA panel which released the consensus statement last week (this panel convened in Chapel Hill, NC, and is referred to as the “Chapel Hill panel” for purposes of this document).

Both panels are funded by NIH, but are different in almost every other aspect:

• The objectivity of the CERHR assessment is compromised by CERHR contractor’s potential conflicts of interest. CERHR panel members were selected by a contractor subsequently fired over potential conflicts of interest. The panel lacks members with expertise in BPA, and has just 12 members to assess over 500 BPA-related papers. The initial draft was prepared by the contractor who was fired over potential conflicts of interest, calling into question the validity of the contractor’s work. In contrast, the Chapel Hill panel includes 38 of the world’s most published BPA experts from top universities and government institutions.
• The accuracy and consistency of the CERHR assessment is compromised by the panel’s lack of organization. Within the CERHR panel, study reviews were conducted independently by each scientist, prompting one panel member to state in a recent article in Risk Policy Report that “one thing that has plagued this review is that each reviewer was assigned a bunch of papers, and they reviewed them without any other input.” In contrast, the Chapel Hill review was conducted in a highly structured, organized manner: 4 breakout groups were each asked to address 4 critical issues related to BPA, and only if there was consensus among all 4 groups were responses incorporated into the final consensus statement.
• The objectivity, accuracy, and consistency of the CERHR assessment is compromised because it has not been subjected to a standard peer review. The assessment of the CERHR panel has not been subjected to standard peer review. Hundreds of factual errors and errors of interpretation, inconsistencies, and completeness were found in current draft upon external review by BPA experts. In contrast, the work of the Chapel Hill panel was subjected to standard and comprehensive internal and independent external peer review.

We question the Center’s ability to produce a scientifically sound document from this process when the comments you have received from BPA experts include statements calling into question the ability of the panel to meet the most basic scientific standards:

• “Is the panel purposefully misrepresenting data or grossly misunderstanding it?" (Vandenberg et al. 2007)
• “There are two general aspects [of the assessment] which to me represent the antithesis of valid science.” (Welshons 2007)
• “The criteria established by the panel are arbitrary.” (Vandenberg et al. 2007)
• “highly curious” [comment on an uninformed critique] (Zoeller 2007)
• “If one were seeking to establish a mechanism that would be virtually certain to underestimate the potential for harm to be caused by a chemical, the CERHR mechanism is exactly the process that they would want to establish to achieve that objective.” (vom Saal 2007).

The public has now paid for two assessments of BPA toxicity, the one conducted by your panel, which has failed to meet the most basic standards for the conduct of scientific reviews; and a peer reviewed assessment by a panel of BPA specialists (the Chapel Hill panel), which issued its final assessments last week. If you proceed with the CERHR panel process the public will have to pay for this assessment four times all told, because your assessment will require both a thorough peer review, and a complete revision from top to bottom of the current, corrupted document.

This is a high-stakes public health issue. Given the need to restore public confidence in your process after the conflict of interest concerns that have plagued it, we urge that, instead of trying to salvage the hopelessly broken work of CERHR on BPA, you instead dissolve your current panel and adopt the recommendations that the Chapel Hill panel issued last week. Their work is peer reviewed and exceeds established scientific standards. We would also urge you to invest your Center’s resources in conducting studies and forwarding policies that will help reduce the public’s exposures to this chemical that so clearly poses risks to human reproduction.

We are certain you will agree that health standards for toxic chemicals like BPA be set to protect vulnerable populations such as young children. EWG has recently requested data from infant formula manufacturers on BPA levels in their products, to help fill data gaps left by FDA’s meager, 14-sample study. We have also completed a new analysis of infant formula showing, based on available data, that babies who drink ready-to-feed formula can easily be exposed to BPA in amounts that exceed those found harmful in laboratory studies, and are exposed to BPA at greater levels than any other segment of the population. Our correspondence to formula manufacturers and our new infant formula analysis are available on our website at www.ewg.org.

BPA poses risks to human reproduction and is an urgent public health issue. We urge you to lead on this issue instead of spending energy to rectify a corrupt product from a corrupt process.

Sincerely,

[signed]
Anila Jacob, M.D., M.P.H
Senior Scientist

[signed]
Jane Houlihan
Vice President for Research

References

All references listed below except CHCS (2007) are currently available in pdf form at http://cerhr.niehs.nih.gov/chemicals/bisphenol/pubcomm-bisphenol.html.

CHCS (Chapel Hill Consensus Statement). 2007. vom Saal FS, Akingbemi BT, Belcher SM, Birnbaum LS, Crain DA, Eriksen M, Farabollini F, Guillette LJ, Hauser R, Heindel JJ, Ho SM, Hunt PA, Iguchi T, Jobling S, Kanno J, Keri RA, Knudsen KE, Laufer H, LeBlac GA, Marcus M, McLachln JA, Myers JP, Nadal A, Newbold RR, Olea N, Prin GS, Richter CA, Rubin BS, Sonnenshein C, Soto AM, Talsness CE, Vandenbergh JG, Vendenberg LN, Walser-Kuntz DR, Watson CS, Welsons WV, Wetherill Y, Zoeller RT. Integration of Mechanisms, Effects in Animals and Potential to Impact Human Health at Current Levels of Exposure. Reproductive Toxicology 24(2): 2007, in press.

Prins GS. 2007. Letter from Dr. Gail S. Prins, College of Medicine, Department of Urology, University of Illinois at Chicago, to Dr. Michael Shelby, National Institute of Environmental Health Sciences. June 19 2007.

Schonfelder G. 2007. Letter from Dr. Gilbert Shonfelder, Institute of Clinical Pharmacology and Toxicology, Campus Benjamin Franklin, Charity University Medicine Berlin, to Dr. Michael Shelby, National Institute of Environmental Health Sciences. Received June 20 2007.

Vandenberg LN, Maffini MV, Rubin BS, Soto AM. 2007. Response to the Interim Draft of the NTP-CERHR Report on the Reproductive and Developmental Toxicity of Bisphenol A. Tufts University School of Medicine. Department of Anatomy and Cellular Biology. Boston, MA. June 20, 2007.

Vom Saal 2007. Comments on the Interim CERHR Report on Bisphenol A (April 2007). Comments from Dr. Freferick vom Saal, Division of Biological Sciences, University of Missouri, Columbia, MO, to the National Institute of Environmental Health Sciences. Received June 20, 2007.

Welshons WV. 2007. Comments from Dr. Wade V. Welshons, Department of Biological Sciences, Verterinary Medicine, University of Missouri, Columbia MO, to Dr. Michael Shelby, National Institute of Environmental Health Sciences. June 20 2007.

Zoeller RT. 2007. Letter from Dr. R. Thomas Zoeller, Biology Department, Morrell Science Center, University of Massachusetts Amherst, to Dr. Michael Shelby, National Institute of Environmental Health Sciences. May 19 2007.

Attachment

Table. CERHR Assessment of BPA fails to meet basic scientific standards for data quality and objectivity.
A summary of comments to CERHR from independent BPA scientists.

Scientific standards for objectivity and data quality	CERHR assessment fails to meet the standard in this area…	Failure of CERHR assessment according to BPA expert review comments	Reviewer (reference)
Accurate	Errors in interpreting significance of study findings	Panel mistakenly concluded that study effect is of unknown relevance (Ho et al, Cancer Research 2006), when a substantial literature demonstrates its relevance, as described in Prins (2007).	Prins 2007
Accurate, Complete	Errors in conclusions on statistical analysis	Panel deemed a study "inadequate based on inappropriate statistics" with no justification for this conclusion, even though the study relied on accepted, rigorous statistical analysis (one-way analysis of variance) unlikely to merit criticism if it had been assessed by the panel (Zoeller et al. 2005).	Zoeller 2007
Accurate	Errors in understanding study implications	Panel criticized a study for failing to include a positive control (Zoeller et al. 2005), but failed to recognize that the study provided first-ever data on the effect measured, so no positive control compound would be available. The reviewer calls the panel's uninformed critique "highly curious."	Zoeller 2007
Accurate	Errors in understanding study implications	Panel failed to understand a major finding of a critical thyroid study, which showed that BPA produced a profile of effects that were consistent with the interpretation that BPA acts as a selective indirect antagonist on the beta thyroid hormone receptor (Zoeller et al. 2005).	Zoeller 2007
Accurate	Failure to correct errors of fact noted by commenters	Factual errors in prior draft of document, noted in review comments, were not corrected in this draft.	vom Saal 2007
Complete, Accurate	Lack of scientifically sound reasoning in discounting important study observations	Reviewer notes that observations in Zoeller et al. (2005) are discounted by the panel for reasons that are unclear or do not appear to be valid.	Zoeller
Accurate	Misrepresentation of study significance	Reviewer states that panel systematically misrepresented significance of metabolic studies.	Vandenberg et al. 2007, page 8
Accurate	Errors in understanding of need for consistency in control studies	Panel mistakenly called an industry-funded study a "replication" of an independent study which found effects, even though the industry study used a feed known to be contaminated with estrogenic compounds that mask the effects of BPA. It is standard scientific practice in replication to use identical feed to the study being replicated. The industry study found no BPA-related effects.	vom Saal 2007
Accurate	Errors in interpreting scientific literature on animal feed contamination and influence on BPA study findings	The panel failed to consider that two industry-funded studies of BPA which failed to find effects used animal feed known to be contaminated with estrogenic substances that can completely mask the effects of BPA (Thigpen et al. 2003). The panel also raised concerns about feed contamination in a study relying on feed proven to be free of the contaminants at any levels that would affect study findings (see 5 peer reviewed studies referenced in vom Saal 2007).	vom Saal 2007
Accurate	Errors in interpreting the role of positive controls in study validation	The panel failed to note concerns that an industry-funded studies showing no effects from BPA also found no effects in its positive control animals. Positive control animals are exposed to a substance known to produce the same effect scientists are seeking to explore with the study's test substance. It is standard scientific practice to consider a study as having failed if the positive control animals fail to show a response, because this failure means that the study design or test conditions would not allow the study to reveal effects from the test substance, either.	vom Saal 2007; Schonfelder 2007
Reliable, Accurate	Errors in interpretation stemming from lack of panel members who specialize in BPA research	Reviewer notes that failure to integrate the range of available data on animal feed contamination, and other related failings of the panel, likely stems from the fact that panel composition largely excludes members who specialize in BPA research.	vom Saal 2007
Accurate	Errors in interpreting study results as "false positives."	The panel categorized as potential "false positives" findings replicated in many experiments.	vom Saal 2007
Accurate	Errors in interpreting relevance of exposure route.	The panel categorized studies with continuous instead of episodic exposures useless in their evaluation of BPA toxicity in the complete absence of data to support that conclusion, and without recognizing studies which support the continuous sources of exposure for humans, including BPA contamination in dust and air. The panel justified exclusion of these studies in part by expressing concerns that injection results in excess unmetabolized BPA in the bloodstream relative to oral exposures, but failed to recognize that this form of exposure mimics human fetal exposures, and also failed to note the substantial body of literature showing unmetabolized BPA in human tissues and fluids that further support the relevance of continuous exposure studies.	vom Saal 2007
Accurate	Errors in interpreting relevance of exposure route and potential for "false positives."	The panel categorized as potential "false positives" findings replicated in many experiments. The panel categorized studies with continuous instead of episodic exposures useless in their evaluation of BPA toxicity in the complete absence of data to support that conclusion, and without recognizing studies which support the continuous sources of exposure for humans, including BPA contamination in dust and air. The panel justified exclusion of these studies in part by expressing concerns that injection results in excess unmetabolized BPA in the bloodstream relative to oral exposures, but failed to recognize that this form of exposure mimics human fetal exposures, and also failed to note the substantial body of literature showing unmetabolized BPA in human tissues and fluids that further support the relevance of continuous exposure studies.	vom Saal 2007
Accurate	Errors in analysis of study findings.	The panel noted a "lack of clarity" in mouse strain in a study deemed of limited usefulness even though the mouse strain is clearly stated in the study ("CF-1 mice were purchased from Charles River Laboratories…", vom Saal et al 1998).	vom Saal 2007
Accurate	Errors in analysis of study findings.	The panel noted a need for consideration of testis weight in a study deemed of limited usefulness, even though the study clearly documents the effect of testis weight and uses it as the basis for analysis (vom Saal et al 1998).	vom Saal 2007
Accurate	Errors in analysis of study findings.	The panel inaccurately characterized study findings, stating that studies did not find statistically significant effects on the prostate at 0.020 mg/kg bw/day, when the studies clearly report statistically significant effects at that dose.	vom Saal 2007
Accurate	Errors in analysis of study scope	Panel mistakenly concluded that study implications are exclusive of estrogenic endpoints when study authors clearly state otherwise.	vom Saal 2007
Accurate	Errors in analysis of the nature of the study	Panel failed to understand the study analysis (Howdeshell et al. 1999), which relied on a litter-based analysis consistent with a significant literature on the effects of intrauterine position on pup response, but which was confused by the panel reviewer as potentially a pup-based analysis and deemed of marginal utility.	vom Saal 2007
Accurate	Errors in analysis of study design	The panel noted confusion on whether the pup or litter was used as the statistical unit for analysis for a study (Gupta 2000 PSEBM) when the author clearly states that 15 individual pups from 15 separate litters were used in the analysis.	vom Saal 2007
Complete	Exclusion of important review findings	The panel included findings from a published critique of Gupta (2000, PSEBM) but failed to note that the model used as the basis for the critique (Elswick et al. 2000) was deemed "misleading", "illogical" and "flawed" by the NIH Low Dose Review Panel.	vom Saal 2007
Accurate	Lack of understanding of statistical significance in scientific studies	Panel rejected studies for evaluation based solely on judgment that the studies provided "an insufficient number of animals for rigorous statistical analysis," reducing "confidence in the results." The number of study animals required is based on power analysis and is small when the expected magnitude of the effect is great, where small numbers of animals can yield statistically significant results. NIH guidance requires the use of the fewest animals possible to achieve statistical significance. With their conclusion the panel ignores basic, widely accepted, NIH-endorsed statistical principles behind the design of toxicological studies.	vom Saal 2007; Vandenberg et al. 2007
Accurate	Errors in interpretation of toxicological implications and validity of study design.	The panel inappropriately excluded some studies because of their use of DMSO as a vehicle to administer BPA to test animals, failing to note that DMSO exposures are far below those that are known to be biologically active, that DMSO did not produce effects in control animals, that DMSO is not associated with effects related to those under investigation for BPA, and that DMSO has been the vehicle of choice for a wide range of related studies.	Vandenberg et al. 2007, pg 2
Consistent	Inconsistent evaluation of study design	The panel erroneously dismissed studies using DMSO as an administration vehicle, but did not similarly discuss or assess a wide range of other potentially problematic administration vehicles in other studies, including corn oil that can be contaminated with estrogenic compounds that might mask the effects of BPA.	Vandenberg et al. 2007, pg 3
Consistent	Inconsistent application of evaluation criteria	The panel proposed a hypothesis that findings in a BPA study they reviewed could be due to interactions between BPA and the administration vehicle (olive oil), but failed to note the same concern with respect to the other 13 studies evaluated that use olive oil as the administration vehicle.	Vandenberg et al. 2007, pg 3
Consistent, Complete	Inconsistent application of evaluation criteria	The panel found fault with injection as dosing vehicle but failed to note concerns with other dosing vehicles, including stress induced by oral gavage that has been shown to mask the effects of low doses of hormones, and inaccuracies stemming from non-uniform feed and water exposures.	Vandenberg et al. 2007, pg 5
Accurate	Arbitrary and capricious choice of study evaluation criterion	The panel arbitrarily chose 7 as an appropriate sample size for animal experiments without justification. The reviewer notes that this "capricious choice... is contrary to the understanding of statistical power and sample size analysis, which should be done by the experimenter a priori, i.e. before conducting the experiment."	Vandenberg et al. 2007, pg 5
Accurate	Errors in interpreting non-monotonic dose responses	The panel criticized 12 studies as having "non-dose related" results, demonstrating a lack of understanding of the widely observed non-monotonic responses characteristic of many endocrine studies and widely reported in the scientific literature, having been observed in 40% of the studies with a design that would allow for its detection.	Vandenberg et al. 2007, pg 5
Consistent, Complete	Inconsistent application and inadequate documentation of evaluation criteria	The panel dramatically changed study evaluation findings between drafts of the assessment but failed to document changes in evaluation criteria that resulted in these alterations, and failed to note if evaluation criteria were established at all in advance of the initial review.	Vandenberg et al. 2007, pg 3
Consistent	Inconsistent application of evaluation criteria	In Section 3 of the assessment the panel criticized 43% of all studies reviewed for what the panel perceived as an inadequate sample size (n<7), but then inexplicably failed to note this same concern in 11% of other studies reviewed that also had an n<7.	Vandenberg et al. 2007, pg 7; Schonfelder 2007
Consistent	Inconsistent application of evaluation criteria	The panel rejected a number of studies from consideration because of their use of DMSO as an administration vehicle, but inexplicably accepted two other studies that also used DMSO, and for one, supplied by the plastics industry, failed to even note it as a concern.	Vandenberg et al. 2007, page 8
Accurate, Complete	Inaccurate interpretation of significance of pharmacokinetic studies	The panel failed to note concentrations of BPA in critical tissues in pharmacokinetic study as the determining factor for toxicity, or significance of study for fetal exposures.	Vandenberg et al. 2007, page 8
Accurate	Inaccurate representation of study findings	In review of a study of anogenital distance, the panel noted that "study authors concluded that the endpoint was not affected by prenatal, lactional and/or post-wearning exposures to bisphenol A," when, in fact, the study authors did not draw this conclusion (Rubin et al. 2001).	Vandenberg et al. 2007, page 9
Accurate	Lack of understanding of effect of dose and timing on endocrine studies	The panel noted a lack of reproducibility associated with a studies of LH serum levels, but failed to recognize that BPA scientists would not attempt to reproduce endocrine effects in female rats from fetal development until weaning by studying the same endocrine effects in male rats during puberty (Rubin 2001, Akingbemi 2004).	Vandenberg et al. 2007, page 9
Accurate	Lack of understanding of dose response	The panel cited concerns about a "lack of dose response releationships" in a study that found effects only at the highest dose (Rubin et al. 2001), a situation in which consideration of dose response is irrelevant.	Vandenberg et al. 2007, page 9
Accurate	Misrepresentation of study doses	The panel erroneously listed doses included in a study of BPA effects in mammary glands (Murray 2007).	Vandenberg et al. 2007, page 9
Accurate, Complete	Misrepresentation of studies and failure to document	In a discussion of a study for which the panel failed to provide a reference, the panel noted a mammary gland finding that has never been observed in a study (mammary gland alterations in pubertal and adult mice). They appear to be referencing a study conducted on gestational day 18 mice, but if so, mistakenly note that the study used subcutaneous silastic implants for administration (Vandenberg et al. 2007).	Vandenberg et al. 2007, page 9
Accurate	Failure to recognize failed experiments	Four studies were considered adequate by the panel even though their positive controls failed, a clear, widely accepted sign of a failed experiment.	Vandenberg et al. 2007, page 10
Accurate	Failure to recognize key components of study design	The panel failed to adhere to standard scientific practice in considering the need for and results from negative controls in endocrine studies, and in multiple instances ignored common potential sources of contamination such as phytoestrogens in feed, estrogenic activity of oil vehicles, plastics in the animal environment, and contamination in tissue culture experiments.	Vandenberg et al. 2007, page 10
Consistent	Inconsistent application of evaluation criteria	The panel inconsistently treated the lack of information about feed in study documentation, listing it as a weakness in only 14% of the studies reviewed that failed to document it.	Vandenberg et al. 2007, page 10
Accurate, Complete	Failure to recognize key components of study design	In studies with oil vehicles the panel failed to note as a weakness failure to include negative controls, and the panel failed to discuss findings in negative controls when they were included. (Note that for none of the oil vehicle studies including negative controls were the results in those animals discussed by the panel.)	Vandenberg et al. 2007, page 10
Accurate, Complete	Failure to recognize key components of study design	The panel failed to recognize the documented potential for contamination from polycarbonate cages, bottles, and other plastics used in BPA experiments.	Vandenberg et al. 2007, page 10
Unbiased	Bias in study evaluations	In this most recent draft of the assessment, the panel changed many of their evaluations of study adequacy, and found many more studies inadequate than had been proposed in the previous draft, but in making these changes they rejected independently funded studies at three times the rate of industry funded studies.	Vandenberg et al. 2007, page 11
Unbiased, Complete	Bias in study inclusion, Incomplete list of studies for assessment	In the most recent assessment draft the panel stripped from the report, without explanation, the results description for 38 studies that were included in the previous draft. Only one of these was funded by industry, and 32 of these were originally considered adequate.	Vandenberg et al. 2007, page 8
Unbiased	Potential bias in study interpretation based on perception of quality of the research group	Panel review noted strengths of studies to include "the expertise of the group" and "well conducted by a respected lab," indicating reviewers' preferences for labs and calling into question the objectivity of the review	Vandenberg et al. 2007, page 36, 56
Reliable	Reliance on upublished industry studies and translations not subjected to peer review	Reliance on unpublished industry studies not subjected to peer review process, and reliance on translations of select parts of studies published in Japanese, provided to the panel by the American Plastics Council	Vandenberg et al. 2007, page 32, 36, 41, 53, 57
Accurate	165 instances of errors in documentation, analysis, interpretation, and evaluation of significance	Reviewer identified 165 potential errors in documentation of study results, and in interpretation of the study findings and their significance, in conflict with the peer reviewed literature. These are described in the detailed section of this reviewer's comments. Data presented in the comment's summary section are, in contrast, documented individually in this table.	Vandenberg et al. 2007, pg 16-60
Complete	95 instances of incomplete documentation and consideration of study findings, interpretations, and conclusions	Reviewer documented 95 instances where panel assessment appears to be incomplete, including incomplete documentation of relevant test results or justification for panel assertions. These are described in the detailed section of this reviewer's comments. Data presented in the comment's summary section are, in contrast, documented individually in this table.	Vandenberg et al. 2007, pg 16-60
Consistent	39 instances of inconsistent application of evaluation criteria	Reviewer documented 39 instances in which the panel inconsistently applied criteria for study evaluation. These are described in the detailed section of this reviewer's comments. Data presented in the comment's summary section are, in contrast, documented individually in this table.	Vandenberg et al. 2007, pg 16-60
Accurate, Complete	88 instances of panel failing to consider absence of testing for background contamination	Reviewer documented 88 instances of the panel failing to consider the potential effect of contamination in studies which did not test for estrogenicity of administration vehicle. These are described in the detailed section of this reviewer's comments. Data presented in the comment's summary section are, in contrast, documented individually in this table.	Vandenberg et al. 2007, pg 16-60
Unbiased	Apparent bias of errors to favor industry-funded assessments	Reviewer notes apparent bias in distribution of errors in panel assessment that would favor industry-funded assessments and would lead to an assessment underestimating potential for harm from BPA exposures.	vom Saal 2007
Expertise, Accurate, Complete, Reliable	Exclusion of BPA specialists from the panel	Reviewer notes that errors and lack of attention to critical issues are likely related to the panel's relative lack of expertise in BPA research and lack of familiarity with BPA-related scientific literature.	vom Saal 2007
Consistent, Complete	Failure to establish and document defensible criteria for assessing the BPA literature	Reviewer notes that criteria established by the panel for assessment of study utility are arbitrary, many without explanation or merit.	Vandenberg et al. 2007, pg 2
Accurate, Expertise, Reliable	Failure to follow accepted scientific standards for review and study evaluation	Reviewer notes that two fundmental aspects of the assessment are the "antithesis of valid science:" Failure to institute standard peer review of panel findings that would ensure accuracy; failure to apply established criteria for inclusion of and interpreting results from control animals that are critical to understanding BPA study findings.	Welshons
Consistent, Accurate	Inaccurate and incomplete assessments of study findings and implications	Reviewer notes that developmental toxicity data section of the panel's assessment is "filled with inconsistencies and inaccurate statements."	Vandenberg et al. 2007, page 44
Accurate	Inaccurate representation of study findings	Reviewer notes that the panel is either "purposefully misrepresenting data or grossly misunderstanding it," and that summaries provided for some studies are "completely inaccurate and do not represent the experiments conducted and/or the results obtained."	Vandenberg et al. 2007, page 8
Accurate, Reliable	Inaccurate interpretations resulting from review of studies by panel without the requisite specialized knowledge.	Reviewer notes that the panel makes "fundamental errors due to ignorance" in its assessments of BPA studies, and comments that the panel reviews do not meet journal standards for peer review.	Welshons
Accurate	Lack of understanding of endocrine research	Panel has failed to understand the basic requirement in endocrine research for study controls and the implications of results in control animals. The reviewer calls this failing "beyond my comprehension" given the widely understood need for negative controls in positive experiments, and positive controls in negative experiments in endocrine research. The reviewer notes that in modern endocrine research, many of the studies without controls that the panel considers acceptable would, in fact, be unpublishable.	Welshons
Accurate	Misunderstanding of reliable measure for reproduction status in test animal	Panel criticized study for not documenting an indicator of mating that is not considered reliable for the test animals (the researcher subsequently provided the panel with the appropriate measure).	Schonfelder 2007
Accurate	Failure to find basic information in study documentation	Panel criticized study for not documenting animal numbers when the study documentation clearly stated this information.	Schonfelder 2007
Accurate	Failure to find basic information in study documentation	Panel noted that they estimated study findings from a graph, when the study findings were provided in full in the study documentation.	Schonfelder 2007
Accurate	Error in understanding standard statistics used in study design	Panel criticized a study as having "too few animals to reach a conclusion with certainty" when the study identified statistically significant effects and used the appropriate number of animals needed given the expected magnitude of the effect.	Schonfelder 2007

Table References

Akingbemi BT, Sottas CM, Koulova AI, Klinefelter GR, Hardy MP. 2004. Inhibition of testicular steroidogenesis by the xenoestrogen bisphenol A is associated with reduced pituitary luteinizing hormone secretion and decreased steroidogenic enzyme gene expression in rat Leydig cells. Endocrinology. 2004 Feb;145(2):592-603.

Elswick BA, Welsch F, Janszen DB. 2000. Effect of different sampling designs on outcome of endocrine disruptor studies. Reprod Toxicol. 2000 Jul-Aug;14(4):359-67.

Gupta C. 2000. Reproductive malformation of the male offspring following maternal exposure to estrogenic chemicals. Proc Soc Exp Biol Med. 2000 Jun;224(2):61-8.

Ho SM, Tang WY, Belmonte de Frausto J, Prins GS. Developmental Exposure to Estradiol and Bisphenol A Increases Susceptibility to Prostate Carcinogenesis and Epigenetically Regulates Phosphodiesterase Type 4 Variant 4. Cancer Res 2006 66: 5624-5632.

Howdeshell KL, Hotchkiss AK, Thayer KA, Vandenbergh JG, vom Saal FS. 1999. Exposure to bisphenol A advances puberty. Nature. 1999 Oct 21;401(6755):763-4.

Murray TJ, Maffini MV, Ucci AA, Sonnenschein C, Soto AM. 2007. Induction of mammary gland ductal hyperplasias and carcinoma in situ following fetal bisphenol A exposure. Reprod Toxicol. 2007 Apr-May;23(3):383-90.

Prins GS. 2007. Letter from Dr. Gail S. Prins, College of Medicine, Department of Urology, University of Illinois at Chicago, to Dr. Michael Shelby, National Institute of Environmental Health Sciences. June 19 2007.

Rubin BS, Murray MK, Damassa DA, King JC, Soto AM. 2001. Perinatal exposure to low doses of bisphenol A affects body weight, patterns of estrous cyclicity, and plasma LH levels. Environ Health Perspect. 2001 Jul;109(7):675-80.

Schonfelder G. 2007. Letter from Dr. Gilbert Shonfelder, Institute of Clinical Pharmacology and Toxicology, Campus Benjamin Franklin, Charity University Medicine Berlin, to Dr. Michael Shelby, National Institute of Environmental Health Sciences. Received June 20 2007.

Thigpen JE, Haseman JK, Saunders HE, Setchell KD, Grant MG, Forsythe DB. 2003. Dietary phytoestrogens accelerate the time of vaginal opening in immature CD-1 mice. Comp Med. 2003 Dec;53(6):607-15.

Vandenberg LN, Maffini MV, Rubin BS, Soto AM. 2007. Response to the Interim Draft of the NTP-CERHR Report on the Reproductive and Developmental Toxicity of Bisphenol A. Tufts University School of Medicine. Department of Anatomy and Cellular Biology. Boston, MA. June 20, 2007.

Vom Saal 2007. Comments on the Interim CERHR Report on Bisphenol A (April 2007). Comments from Dr. Freferick vom Saal, Division of Biological Sciences, University of Missouri, Columbia, MO, to the National Institute of Environmental Health Sciences. Received June 20, 2007.

Welshons WV. 2007. Comments from Dr. Wade V. Welshons, Department of Biological Sciences, Verterinary Medicine, University of Missouri, Columbia MO, to Dr. Michael Shelby, National Institute of Environmental Health Sciences. June 20 2007.

Zoeller RT. 2007. Letter from Dr. R. Thomas Zoeller, Biology Department, Morrell Science Center, University of Massachusetts Amherst, to Dr. Michael Shelby, National Institute of Environmental Health Sciences. May 19 2007.

Zoeller RT, Bansal R, Parris C. 2005. Bisphenol-A, an environmental contaminant that acts as a thyroid hormone receptor antagonist in vitro, increases serum thyroxine, and alters RC3/neurogranin expression in the developing rat brain. Endocrinology. 2005 Feb;146(2):607-12.