On the go? Download our Mobile App

TITLE
share
food categories menu
site menu
 
Learn more
 

Ingredient Concern Methodology

EWG's Food Scores evaluates and scores food ingredients and contaminants based on the effects they could have on human health and the environment.

We refer to it collectively as the ingredient concern score. The score is determined by the following factors:

  1. Concerns about food additives listed in the Food and Drug Administration's Priority-based Assessment of Food Additives (PAFA) database.
  2. Concerns about food contaminants. The contaminants EWG scores in food are:
    • Mercury in seafood
    • Bisphenol A in canned foods
    • Arsenic in rice
    • Pesticide residue levels on fresh and frozen fruits and vegetables
    • Persistent organic pollutants (POPs) in fatty meats, dairy, and seafood.
  3. Concerns about how food is produced. The factors EWG scores are:
    • Antibiotics fed to conventionally raised meat and dairy animals
    • Hormone and non-hormone growth promoters administered to conventionally raised meat animals and recombinant bovine growth hormone (rBGH) injections in dairy cows

We also consider whether a processed food is certified organic or conventional, but this adjustment is made at the end of the scoring process and does not affect the ingredient concern score specifically.

To come up with ingredient concern scores, EWG generates a baseline score for every ingredient and the six food contaminants that we consider in our database. This baseline score is determined with a weight-of-the-evidence approach based on existing data on potential human health and environmental effects. The method for determining a numeric score for ingredients and food contaminants is explained in detail in the section below titled Algorithms for Scoring an Ingredient or Contaminant. Ingredient concern scores range from 1 (best) to 10 (worst).

The basic algorithm to determine the ingredient concern for a food ingredient or contaminant is:

Substance score x Food weighting factor = Ingredient concern score

For example:

Arsenic (10) X weighting factor (0.25 or 0.5) = arsenic concern in rice (2.5 or 5.0)

The substance score for ingredients and contaminants reflects the concern for these compounds on their own - not necessarily for concern about their presence in food. For example, arsenic scores 10 because it is a known human carcinogen and causes other health effects such as cardiovascular damage. However, its score as a contaminant in food ranges from 2.5 to 5, depending on the amount of arsenic in the food. The food-weighting factor for arsenic in rice is modified to account for the fact that arsenic concentrations range from 1-to-200 parts per billion in rice-based foods.

EWG's food weighting factors adjust the substance score to reflect the level of concern for the substance in food. Food weighting factors consider the concentrations measured in food and the evidence linking ingestion of the substance in food to human health effects or environmental concerns. If the presence of an ingredient or contaminant in food is of lesser concern, the substance score is reduced more (multiplied by a smaller food weighting factor) while more significant concerns are reduced less (multiplied by a larger food weighting factor).

EWG has also identified ingredients that may cause allergic reactions in sensitive individuals, including ingredients that may be one of the eight major food allergens, as defined by the FDA. Since allergic reactions are only relevant to a small group of sensitive persons and there is no concern for most people, food allergy concerns are not included in the ingredient concern scores.

Scoring Food Additives

Food additives are thousands of natural and synthetic compounds added for a variety of reasons - preservatives, emulsifiers, dough conditioners, coloring and sweeteners. EWG evaluated safety data for additives listed in FDA's Priority-based Assessment of Food Additives (PAFA) database.

EWG used a number of resources to identify toxicity concerns relevant to PAFA ingredients. Among them are safety evaluations in the Joint Food and Agriculture Organization/World Health Organization Expert Committee on Food Additives (JECFA 1972-2014) monographs, publications of the European Food Safety Authority (EFSA 2003-2014), FDA guidance documents and data in its PAFA database (FDA 2013d), the U.S. Code of Federal Regulations (2014) and peer-reviewed literature. EWG's substance scoring system assigns a level of concern for each identified health effect based on the authority of the data source and the severity of the health effect. For example, a health effect reported by the World Health Organization is given more consideration than an effect reported in a single peer-reviewed study, and a major illness such as cancer is given more weight than a minor or transient symptom such as gastrointestinal irritation. For a complete description of how endpoints are scored, please see the section on Algorithms for Scoring an Ingredient or Contaminant below.

The substance score for each additive is then adjusted by its Acceptable Daily Intake (ADI) value. ADIs are a government agency's estimates of the amount of a substance in food that can be ingested daily over a lifetime without appreciable risk. ADIs are expressed on a body-weight basis. An ADI is typically calculated by dividing the No Observable Adverse Effect Level (NOAEL, the highest dose of a substance that produces no harmful effect) by 100. ADIs for additives range over orders of magnitude and integrating them into the system incorporates a risk-based element. The increments for adjusting the food-weighting factor, based on the ADI, are one order of magnitude apart. The following rules apply to how the ADI weights the substance score for food additives:

  • If the ADI is less than 1 milligram per kilogram of body weight per day, the score is fully weighted (food weighting factor is 1.0).
  • If the ADI is between 1 and 10 mg/kg of body weight per day, the food weight factor is 0.7.
  • If the ADI is more than 10 but less than 100 mg/kg of body weight per day, the food weight factor is 0.5.
  • If the ADI is 100 mg/kg of body weight per day or more, the food weight factor is 0.2.
  • If an ADI is unspecified due to non-toxicity, no score is assigned unless EWG determines that there are relevant health concerns.

In cases where toxicological information is available but no ADI has been set, EWG calculates a surrogate ADI by taking the NOAEL from the available data and dividing that number by 300. EWG selected 300 to account for inter- and intra-species uncertainty, which typically carries an uncertainty factor of 100, multiplied by an additional factor of 3 to account for children's exposure. Since JECFA and FDA often use a safety factor of 100, EWG's approach is more conservative. FDA guidance for the safety assessment of food additives says that additional factors to account for sensitive populations, such as children, are acceptable in assessing safety (FDA 2000).

Challenges in Food Additive Scoring

Toxicological studies of food additives sometimes involve giving animals unusually large doses of a compound in order to produce a health effect. This is because substances that are intentionally added to food are generally not highly toxic, and in some cases very large quantities of the test substance must be administered to produce an adverse effect in animals. These doses can be many times higher than what humans would encounter in their diets. Therefore, the health effect observed in the animal becomes less relevant to the anticipated effects in people. Sometimes such high-dose animal studies are the best or only information available to evaluate the toxicity of a food additive. This type of data is difficult to incorporate into a database that evaluates concerns in the American diet, where the concentration of the substance in question is much lower than that given to laboratory animals.

In cases where a health effect associated with an additive is reported in a high-dose animal study, EWG reduces how much that study affects the additive's substance score. The score for the reported health effect is adjusted by multiplying it by 0.5. This is done to reflect that the reported health effect is less relevant to people and should not contribute heavily to the substance score. To be categorized as a high-dose animal study, the NOAEL must be 250 mg/kg of body weight per day or higher.

EWG's assessment of registered food additives is based both on the scientific evidence of adverse effects of ingesting the additive, and modified by the ADI as determined by an authoritative government agency, where available. Ingredient concerns for food additives range from 0 to 9.

Scoring Methyl Mercury Contamination

The substance score for methyl mercury is 10, based on clear evidence of effects on neurodevelopment and other adverse health outcomes in laboratory animals and in people (ATSDR 1999; EPA 2001). Methyl mercury is a common contaminant in fish and shellfish and is neurotoxic to the developing brain. The FDA advises women of childbearing age and children to avoid consuming fish species that are the most highly contaminated (FDA 2014a).

Methyl mercury concentrations vary widely in commercial fish and shellfish species. For our assessment of mercury levels, EWG relied on the most comprehensive database available, a meta-analysis of approximately 300 studies measuring mercury concentrations in seafood (Karimi 2012). For this meta-analysis, researchers documented that mercury concentrations in seafood are in many cases higher than those found by the FDA's seafood monitoring program due to the limited scope of FDA's monitoring and the relatively small number of samples collected.

The abundance of human studies linking mercury intake from food to lasting neurobehavioral impacts in offspring makes methyl mercury a major contaminant concern in the food supply. The food weighting factors for methyl mercury in seafood are as follows:

  • For seafood species with the highest mean mercury concentrations (>0.25 ppm) the mercury food weight factor is 1. A pregnant woman could exceed EPA's Reference Dose for methyl mercury if she ate this species frequently during pregnancy. The ingredient concern dial indicates very high concern (red).
  • For seafood species with moderate mean mercury concentrations (>0.1 - ≤ 0.25 ppm) the mercury food weight factor is 0.6. A woman of average weight (166 pounds) could eat a single 4-ounce serving of this seafood per week without exceeding EPA's Reference Dose for methyl mercury, but more frequent consumption could pose a health risk. The ingredient concern dial indicates moderate concern (orange).
  • For seafood species with the lowest mean mercury concentrations (<= 0.1 ppm) the mercury food-weighting factor is 0. An average weight woman could eat up to 12 ounces of this species per week without exceeding 75 percent of EPA's Reference Dose for methyl mercury, making consumption of this species a low concern. The dial indicates low concern (green).
ingredient dials for mercury levels in fish

Approximate ingredient concern scores for seafood based on the scientific evidence at the time of release and the weighting factors for high, moderate and low mercury contamination in fish.

Scoring Bisphenol A Contamination

Most canned food manufacturers in the United States use metal cans with a BPA-based resin lining, and traces of BPA leach into the food (Noonan 2011). The substance score for BPA is 8.8, based on clear evidence of endocrine disruption in animals at low doses, and the potential for serious adverse effects (NTP 2008; Rochester 2013; Rubin 2011; Richter 2007).

BPA concentrations are typically in the parts per billion range in canned food, and BPA concentrations vary widely among food samples (Liao 2013; EWG 2007). A small number of manufacturers have begun to package their food in metal cans with BPA-free linings. EWG assigns a single food weighting factor of 0.75 for BPA to all canned goods unless we were able to confirm that the company uses BPA-free cans, in which case the ingredient concern score is 0. We set a food-weighting factor of 0.75 because of the uncertain but potential harms to human health and the opportunity for food companies to switch to BPA-free cans. This results in a moderate ingredient concern score of 6.6 for BPA in canned foods.

ingredient dial for BPA in canned products

Approximate ingredient concern score for BPA contamination in canned foods, based on the scientific evidence at the time of release and the weighting factor that considers low-dose toxicity concerns.

Scoring Arsenic Contamination

The substance score for arsenic is 10, based on a robust body of evidence of adverse health effects in people and animals, including cancer, developmental toxicity, neurotoxicity, cardiovascular disease, skin lesions, abnormal glucose metabolism and diabetes (EFSA 2009; ATSDR 2007).

EWG evaluates arsenic as a contaminant in rice and rice-based ingredients such as rice syrup, rice milk and rice flour which have higher concentrations of arsenic than other fruits, vegetables and grains. A single serving of rice, rice cereal, rice crackers or rice cakes can contain more arsenic than the EPA allows in a glass of drinking water. Arsenic contamination is a concern for most rice-related products, however test data shows that rice vinegar. EWG has identified rice-based foods that pose a concern for arsenic contamination based on testing performed by the FDA and Consumer Reports (FDA 2013b; Consumer Reports 2012).

Arsenic contamination is of most concern in flavored and unflavored rice, rice-based breakfast cereals, crackers, noodles and rice milk. In these types of foods the substance score for arsenic is multiplied by a food-weighting factor of 0.5. The ingredient concern dial will be 5, reflecting moderate concern for arsenic contamination in rice. In foods where arsenic concentrations are lower, the arsenic score will be weighted by 0.25, reflecting a lesser concern.

ingredient dials for rice-based products

Approximate ingredient concern scores based on the scientific evidence at the time of release and the food-weighting factor for rice-based products and rice-containing products.

Scoring Pesticide Residues on Food

The substance score for pesticides considers both human health and environmental considerations. The score for pesticides is 10 to account for human health risks and environmental impacts, including water contamination and effects on wildlife (EPA 2014).

The food-weighting factor for pesticide residues on fresh and frozen fruits and vegetables draws on data produced annually by the U. S. Department of Agriculture Pesticide Data Program, which publishes annual monitoring data of pesticide concentrations on fruits and vegetables. Using the methodology derived from EWG's Shopper's Guide to Pesticides in Produce, we consider the number of pesticide residues detected on produce and their concentrations (EWG 2014). EWG's food-weighting factor for pesticide residues on fruits and vegetables identifies three levels of concern:

  • Foods identified on EWG's annual Dirty Dozen list receive a pesticide food-weighting factor of 0.3. The Dirty Dozen lists the twelve types of produce that have the highest levels and numbers of pesticide residues.
  • Foods listed on EWG's annual Clean Fifteen list have a pesticide food-weighting factor of 0.2. These are fruits and vegetables with the lowest number and levels of pesticide residues. However, since they are conventionally grown, pesticides were likely used during cultivation. The pesticide hazard concern score therefore reflects primarily environmental and workplace concerns.
  • Foods on neither the Clean Fifteen nor the Dirty Dozen lists receive a pesticide food weighting factor of 0.25.
  • Foods that are certified organic receive the lowest score and a "No Concern Identified" label appears above the ingredient concern dial.
ingredient dials for pesticide residues

Approximate ingredient concern scores based on the scientific evidence at the time of release and the weighting factors for Dirty Dozen, conventional produce and Clean Fifteen produce.

Scoring Persistent Organic Pollutant (POPs) Contamination in Seafood, Meat and Dairy

Monitoring data from the FDA Total Diet Study and European Food Safety Authority identifies dioxins, furans and other persistent organic pollutants (POPs) as common contaminants in some meat, seafood and dairy products (FDA 2007b; EFSA 2010; EFSA 2012). The substance concern score for POPs is 10, due to a number of serious health effects associated with dioxins and furans, including cancer, immunotoxicity, reproductive toxicity and other illnesses (WHO 2010).

Fish species with high concentrations of POPs include farmed salmon, trout, sardines, catfish, herring and bluefish. Risk-based consumption advice in 2005 suggested that consumers limit farmed salmon to fewer than 10 meals per month (Foran 2005), and very few people are expected to exceed this guideline. Furthermore, there is evidence that POP levels have subsequently declined in some farmed fish due to changes in the feedstock (Shaw 2006).

The POP score for these seafood species is weighted by a food weight factor of 0.2, because human studies suggest that POPs generally pose a lesser concern than mercury in seafood, and because there are no data to confirm that the risks of POP concentrations in commercial seafood outweigh the nutritional benefits. EWG lacks information to distinguish between farmed and wild fish in our database, and salmon is not included in the high POPs group. All other seafood has relatively low concentrations of POPs and has an ingredient concern score of 0.

For dairy products and meats, the concentration of POPs is closely related to fat content, and scores for POP contamination in these foods are based on fat percentage. For the meat and dairy products highest in fat - fat content greater than or equal to 50 percent - the food weighting factor for POPs is 0.2. For less fatty foods of 15-to-49 percent fat content, the food weighting factor is 0.1. Foods with a fat content of less than 15 percent do not receive a POPs concern score. POPs are a concern in both conventionally raised and organic meat and dairy products. The meat and dairy foods scored for POP contamination are:

  • Beef
  • Goat
  • Sheep/lamb
  • Bison
  • Cow milk
  • Sheep milk
  • Goat milk

Pork, chicken and eggs have low concentrations of POPs and an ingredient concern score of 0.

ingredient dials for persistent organic pollutants

Approximate ingredient concern scores based on the scientific evidence at the time of release and the weighting factors for POPs contamination in certain meat, dairy, and seafood products.

Production Concerns For Conventionally Raised Meat, Eggs and Dairy Products

Animals used for conventional production of foods such as meat, dairy and eggs are often treated with antibiotics and growth promoters such as hormones to prevent illness and promote faster growth. Conventionally produced animal-based products have different production concerns. For example, hormones are not a concern in chicken and pork because USDA regulations do not allow their use in these animals. Antibiotics are overused in meat and dairy production, and meat animals are fed the greatest amounts. Growth promoters such as Ractopamine are not fed to poultry. Table 1 identifies which concerns apply to various types of animal-based ingredients, according to available data. Data references are provided in the sections below that describe specific concerns.

Table 1. Production concerns for conventional meat and dairy products

 AntibioticsHormones and Growth promoters
ChickenX 
PorkXX
BeefXX
TurkeyXX
GoatX 
Sheep & lambXX
BisonX 
Fish & shellfish  
Cow milk/dairyXX
Sheep milk/dairyX 
Goat milk/dairyX 
EggsX 

Antibiotic use in meat, eggs and dairy products

The vast majority of antibiotic use in the United States is not for fighting human illness but for meat and dairy production. Many conventionally raised animals are continually given antibiotics throughout their lives to promote faster growth and prevent sickness, rather than as treatment for an illness.

Repeated administration of low doses of antibiotics causes bacteria to develop resistance to them, posing serious risks to human health. Antibiotic-resistant bacteria are frequently detected in raw chicken, beef, turkey and pork samples (EWG 2013; FDA 2011). The Centers for Disease Control and Prevention estimates that each year 2 million Americans are sickened and 23,000 die of infections by drug-resistant bacteria (CDC 2014). Mexico, Taiwan, the Netherlands, Germany, Denmark and Sweden have initiated national bans on antibiotics for growth promotion (Maron 2013). Denmark has taken the most dramatic steps to decrease the use of antibiotics in meat production with little disruption to food supplies or harm to animal welfare (Cogliani 2011).

The FDA recently acknowledged the problem and called for a reduction in sub-therapeutic antibiotic use in commercial animal feed in the U. S., but veterinarians can still authorize daily low-dose antibiotics for disease prevention. The agency has also requested that companies producing veterinary feed with antibiotics stop selling them for growth promotion. Instead of changing its regulations, however, FDA has asked companies to voluntarily remove the term "growth promotion" from the labels of medicated feed (FDA 2012; FDA 2013c). FDA has not taken steps to track the use of medically important antibiotics in meat and milk production to gauge the effectiveness of these voluntary measures. Many advocates fear that the new measures will have little impact on the overall use of antibiotics in agriculture (Pew Charitable Trusts 2014).

EWG's score for antibiotics in meat production is 10, based on the threat of developing antibiotic resistance and the indirect risks to human health. The food-weighting factor for meat animals that are likely to receive sub-therapeutic treatment is 0.7 to 0.8, based on the prevalence of antibiotic use. Species that are commonly fed antibiotics include chicken, pork, beef, turkey and veal. There is poorer data about the use of sub-therapeutic antibiotics in goats, rabbits and game animals, but antibiotic use is presumed to be less common in these animals. The food-weighting factor for antibiotics is 0.6 in these meats. In addition to antibiotics, turkeys are commonly fed an arsenic-based medication, Nitarsone, for prevention of blackhead disease during their first six weeks of life.

Sub-therapeutic antibiotics are not routinely administered to dairy cows and egg-laying chickens. Milk and eggs gathered from animals being treated with antibiotics cannot be sold if antibiotic residues are detected. However, overcrowded and unsanitary conditions cause diseases that require antibiotic treatment. The food weighting factor for antibiotic use in the production of eggs, non-chicken poultry meat and dairy ingredients from cows, sheep and goats is 0.4 to reflect concerns for antibiotic resistance.

Hormones and other growth promoters in meat and dairy production

In the United States both hormone treatments and a class of adrenal hormones known as beta-agonists are administered to animals to boost meat and milk production.

Six different steroid hormones can be administered to beef cattle to promote growth. These hormone treatments are common in the U.S. beef industry (USDA 2013) but not allowed in Australia, Japan, New Zealand and most European countries. The European Union Food Safety Authority (EFSA) concluded that the lack of data about trace hormone residues in meat makes it difficult to determine if there are risks to human health (EFSA 2007). Estradiol is a known carcinogen and affects the endocrine, developmental, immune and nervous systems. EFSA notes that even minor differences in estradiol residues in the meat could affect human health (EFSA 2007).

Some dairy cows receive routine injections of recombinant bovine growth hormone, or rBGH, to boost milk output. There is little information about the prevalence of rBGH treatment in conventional dairy cows, but available data indicate that an estimated 20 percent of animals are treated (FDA 2014b). rBGH is not allowed in Europe due to concerns that treatments may change the nutritional qualities of the milk and because of concerns about animal welfare. European officials note that rBGH treatment may increase the concentration of insulin-like growth factor, or IGF-1, in milk. Increases in IGF-1 are linked to several chronic diseases in people, notably breast and prostate cancers (EC 1999). The European Commission expressed concern that increases in udder infections caused by rGBH treatments may result in more use of antibiotics (EC 1999).

Non-hormone growth promoters fed to meat animals include Ractopamine and Zilpaterol, synthetic adrenal hormones known as "beta-agonists" that are commonly fed to beef, swine and sometimes to turkeys in the United States prior to slaughter. Both treatments promote the development of lean muscle instead of fat. However, they also affect the animal's bronchial smooth muscle, liver and kidneys. These treatments are not allowed in Europe (Serratosa 2006). Adverse effects are common in treated animals and include increased heart rate and hoof defects. Tyson Foods stopped using Zilpaterol in feed due to concerns about animal welfare after noticing a record number of hoof defects in treated cattle (Penn State Extension 2013).

EWG adjusted the antibiotic ingredient concern score to reflect additional issues posed by hormones and artificial growth agents. For beef, pork and turkey, we boost the food-weighting factor by 0.2 to account for the widespread use of hormone and beta-agonist growth promoters. We increased the food-weighting factor for sheep meat and cow milk by 0.1 to account for potential hormone use. Growth promoters and hormones are not used in chicken, veal and other animals raised for meat, dairy and egg production.

Some meat, eggs and dairy products are certified as being raised without the use of antibiotics and hormones. The requirements vary based on the certifying authority.

The antibiotic food-weighting factor is reduced to 0.1 for food certified by the Animal Welfare Approved and Certified Humane systems, which set high standards for documentation and verification to ensure no misuse of antibiotics and that no growth promoters are used during production. The antibiotic food-weighting factor is reduced to 0.2 when food manufacturers claim that meat and dairy were produced without hormones and antibiotics or are certified by standards that have less rigorous requirements. (example: American Humane Certified). USDA Process Verified seals and Global Animal Partnerships set rigorous production standards and ratings, but EWG lacks information to identify these certified products in our database.

The food-weighting factor for cow milk and dairy products claiming to be rBGH-free is reduced from 0.5 to 0.4 because antibiotics may still be administered to dairy cows. UDSA-certified organic meat, eggs and dairy have no ingredient concern score because USDA does not permit the use of antibiotics and hormones, the animals eat organic feed and producers must adhere to protections for animal welfare.

The ingredient concern score for antibiotics and growth-promoting agents is applied in full for meat, dairy and egg products. These scores are reduced by 50 percent in foods where meat, egg or dairy ingredients are expected to make up about half of the product. Twenty-five percent of the score is applied when meat, dairy or egg ingredients are expected to make up a minor component of the food.

Table 2. Ingredient scoring for antibiotics, hormones and growth promoters in meat, dairy and eggs

Animal products in databaseAntibiotic useHormones and other growth promotersIngredient Concern Score
Beefsub-therapeutic commonhormone implants common, also beta-agonists10
Pork sub-therapeutic commonbeta-agonists common10
Turkey sub-therapeutic commonbeta-agonists possible, arsenic-based treatment common10
Chicken and vealsub-therapeutic commonnot permitted8
Lamb or Sheep meatsub-therapeutic possiblehormone implants possible7
Rabbit, goat, and game animalsunknownnot permitted6
Cow milkprimarily illness~20% cows receive rBGH injections5
Poultry eggs, duck and goose meatprimarily illnessnot permitted4
Sheep and goat milkprimarily illnessnot permitted4
ingredient dials for persistent organic pollutants

Approximate ingredient concern scores for conventionally raised meat and dairy products based on the usage of antibiotics, hormones and growth-promoting agents with the specific food weighting factors listed above.

Scoring Organically Produced Foods

The pesticide ingredient concern score is employed for fresh and frozen fruits and vegetables only. Pesticide use and residue concerns for other foods are considered in the overall score for the food product and described in more detail in the Overall Product Score methodology. For foods other than fresh and frozen produce that are not certified organic, the overall product score is increased by 0.5 points. For products made with more than 70 percent organic ingredients, the overall product score is increased by only 0.2 points when manufacturers can show that this product has been certified to contain at least 70 percent organic ingredients. These score adjustments reflect the environmental and health benefits of organic food production compared to conventional production.

Allergies and Other Health Effects Applicable Only to People with Special Conditions

Health effects that are restricted to people with special conditions, such as allergies, have been identified but are not included in the ingredient scoring since these effects are not applicable to most people. This includes sensitivity to MSG.

EWG identified ingredients considered major food allergens by the FDA. These are ingredients or additives that contain peanuts, tree nuts, crustacean shellfish, fish, milk, eggs, wheat and soybeans (FDA 2006). However, we do not intend this database to serve as a reference for foods that people with allergies can and cannot eat. People with food allergies, particularly those that are severe or life-threatening, should always read the ingredient label and check packaging for warnings such as "processed in a facility that also processes peanuts."

Algorithms for Scoring an Ingredient or Contaminant

The scoring of food additives and contaminants follows a comprehensive weight-of-evidence approach. This provides a framework for evaluating the range of published information on substance safety for each possible health effect. All this information is then combined into a total substance score. This final ingredient concern score is weighted as described above when the ingredient is found in a food.

Where there is clear and convincing scientific evidence from authoritative sources that certain substances harm human health, EWG determined that a score of 10, the highest level of concern, was appropriate. This applies to arsenic, methyl mercury and pesticides as a group. EWG also applies an overall score of 10 to food ingredients derived from animals treated with antibiotics and hormones, because there is clear evidence that these practices pose serious public and environmental health threats.

Definitions:

Finding: A conclusion associated with a peer-reviewed study, agency-determined categorization or data point in a data set.

Example 1. Data set: Proposition 65 list of carcinogens. Finding: Proposition 65 listing for BHA: known carcinogen.
Example 2. Peer-reviewed study. Finding: Oishi et al. found that propyl paraben is an endocrine-disrupting compound.

Endpoint: An effect or specific parameter associated with a finding. Endpoints include carcinogenicity, developmental toxicity and gastrointestinal effects. EWG assigns an endpoint to each finding. EWG calculates separate hazard and safety scores for each endpoint.

Endpoint group: EWG assigns each endpoint to one or more endpoint groups organized by biological system. An example of an endpoint group would be cancer, which is a collection of the endpoints for specific types of cancer (e.g. colon cancer, breast cancer, lung cancer). Likewise, the endpoint group digestive system toxicity/disease would include endpoints such as gastrointestinal irritation and liver effects. After EWG calculates separate scores for each endpoint we then calculate a single, consolidated score for the endpoint group.

Scoring subcategories: EWG assigns each endpoint group to one scoring subcategory. An example is the hazard, health subcategory, which includes the endpoint groups of various health hazards, such cancer or endocrine disruption. Another subcategory is restrictions, which allows scoring to account for bans or restrictions that may be placed on a particular substance. An example of an endpoint group in the restrictions subcategory is government enforceable restrictions, which includes endpoints such as banned in the European Union.

Scoring category: EWG assigns each endpoint group, separated by scoring subcategory, to a broad scoring category of health. EWG calculates an overall substance score as a function of this scoring category.

Example of a tiered structure for scoring categories, subcategories, endpoint groups and endpoints:

  • Health (scoring category)
    • Hazard, health (scoring subcategory)
      • Developmental, endocrine and reproductive toxicity (endpoint group)
        • Damage to fertility (endpoint)

Scoring a substance that may be considered an ingredient or contaminant

Step 1: Information collection

Scientific data sets, peer-reviewed literature and other information sources are collected and reviewed to identify findings relevant to health and the environment that are applicable to substances in the database. Where possible, findings from authoritative bodies, such as the World Health Organization's IARC cancer classifications, are used for scoring. If such authoritative determinations are not available, EWG may refer to peer-reviewed literature. Findings include determinations that a substance causes or may cause specific diseases such as breast cancer or liver damage, which are considered endpoints. A finding may also indicate that a substance does not pose a risk to human health.

Step 2: Assessing the data and assigning numeric values

Each finding is assigned to an endpoint that dictates what question is being evaluated - for example, does this substance cause cancer? EWG then evaluates what each finding says about the safety or hazard of a substance and assigns it appropriate scoring values. The values depend on:

  1. Strength of association: A numeric value assigned to each finding that reflects how strongly it is associated with a particular endpoint, such as cancer. The strength of association values and their definitions are listed below.
  2. Strength of associationDefinition
    -10No association, confirmed
    -5Limited evidence of no association
    0Unknown (no data or conflicting data)
    5Limited evidence of an association
    8Likely/probable association
    10Confirmed association

    Example: The World Health Organization finds that ingested nitrites are probably human carcinogens (2A). EWG assigns this finding a strength of association of 8 = Likely/probable association.

  3. Scope of assessment: A categorization determined by the credibility and comprehensiveness of the data. A finding or determination by a comprehensive agency assessment, such as a National Toxicology Program report on carcinogenicity, is given a higher value than a finding generated by a single peer-reviewed study. EWG assigns a Scope Tier to each finding, as defined in the table below:
  4. Scope of dataset or studyNature of data considered or generated in the dataset or studyWho evaluated or generated the data (authors)?Form of documentationExamples
    Scope tier 6Absolute, comprehensive weight-of-the-evidence assessment based on data sufficient for complete understanding and certainty of scientific fact.Finding by a completely authoritative scientific body or known as fact.Full, publicly available documentation of data collection, methods, and conclusions.Oxygen is a gas at standard temperature and pressure.
    Scope tier 5Comprehensive, weight-of-the-evidence evaluation of available scientific information. Data is relevant to a broad issue.Authoritative government agency or permanent non-governmental body with accepted, comparable authoritative standing.Full, publicly available documentation of data collection, methods and conclusions. Findings may be based on specific criteria.IARC1 Monographs on the Evaluation of Carcinogenic Risks to Humans, the UNECE2 Globally Harmonized System of Classification and Labeling of Chemicals.
    Scope tier 4Weight-of-the-evidence evaluation of available scientific information. May be limited to evaluation of data relevant to one aspect of a broader issue.Authoritative body of experts, including standing and ad hoc government and industry advisory panelsFull, publicly available documentation of data, methods, and conclusions. Findings may be based on specific criteria.An EPA3 committee report on endocrine-disrupting chemicals. A JECFA4 toxicological evaluation of a food additive.
    Scope tier 3Weight-of-the-evidence evaluation of available scientific information on a limited subject.Qualified expert(s).Independently evaluated and published in a peer-reviewed journal. Findings may be based on specific criteria.A review article on neurotoxic compounds that evaluates a significant body of information from a variety of data sources. A CIR5 panel final report on a class of fragrance chemicals.
    Scope tier 2Evaluation of credible scientific data on a limited or single subject.Qualified expert(s).Independently evaluated and published in a peer-reviewed journal, or authored by a credible source.A published scientific study on the reproductive toxicity of a particular substance.
    Scope tier 1A finding generated from screening-level testing or a method less rigorous than alternate tests that more closely approximate real-world conditions.Qualified expert(s).Findings reported by a credible source, such as a government agency.Screening-level studies including in-vitro testing and reports of a single data point.

    1 International Agency for Research on Cancer
    2 United Nations Economic Commission for Europe
    3 US Environmental Protection Agency
    4 Joint FAO/WHO Expert Committee on Food Additives
    5 Cosmetic Ingredient Review

    Example: The Environmental Protection Agency finding that benzene is a known human carcinogen was determined by a rigorous, comprehensive review of all available data on all routes of exposure, and it makes its review process publicly available. EWG assigns this finding to Scope Tier 5.

  5. Severity: EWG assigns a numeric weight called severity to each endpoint, which is intended to reflect the relative impact of each endpoint on health, the environment and other factors considered in EWG's ratings. Severity values are scaled in high, moderate and low categories according to their relative effects. Low values typically indicate safer, more desirable attributes, which are scored positively.
    • High: Includes endpoints associated with:
      • systemic diseases that can be significantly debilitating or fatal, such as cancer.
      • conditions that can cause multi-generational impacts, including endocrine disruption, developmental effects and reproductive problems.
    • Moderate: Includes endpoints associated with:
      • significant but reversible physical discomfort, such as gastrointestinal irritation.
    • Low: Includes endpoints associated with:
      • little to no potential for adverse effects to human health, such as an EPA Group E designation (evidence of non-carcinogenicity for humans).
  6. Uncertainty (u): A value that EWG assigns to each finding, from among the categories below, that reflects the extent of uncertainty of a particular finding. EWG uses this parameter to reflect cases in which uncertainty is greater than what is typical for data within a given tier. Uncertainty may reflect data collection gaps and method limitations.
  7. The assigned uncertainty values (u) are defined below:

    • Default: u = 1
    • Limited uncertainty - exceeding typical: u = 0.5
    • No data: u = 0

    Example: Uncertainty. A review paper reports that a substance is not associated with reproductive toxicity and provides limited data to support that claim. Several independent, peer-reviewed studies that show the substance is associated with reproductive toxicity are not mentioned in the review. The finding that the substance is not a reproductive toxicant may be therefore assigned a limited uncertainty value of 0.5.

Calculating a score for a single endpoint. EWG combines the findings from multiple data sets and makes a determination about the overall strength of association, positive or negative, between a substance and a particular health or environmental endpoint. This determination is based on:

  1. Weight of the evidence. The overall strength of association between a hazard and a substance will be driven by the amount of evidence that supports that association.
  2. Resolution of conflicting data. If there are conflicting findings - for example, if one agency determines that a substance is a known endocrine disruptor and another finds it does not have enough data to make a determination - EWG takes a precautionary position and assigns a value that reflects some hazard association.

Algorithms for consolidating findings for a single endpoint

The numerical score for a single endpoint for a single substance in the database must account for many findings that vary in their strength of association, scope of assessment and uncertainty. The goal is to determine an overall strength of association for a given endpoint, considering all findings for that endpoint. The result will be an aggregate strength of association value ranging between -10 and +10. A +10 score indicates that a substance is definitively associated with an endpoint (such as a substance known to be a carcinogen), while a -10 score indicates that a substance is definitively not associated with an endpoint (such as a substance that is known to be non-carcinogenic).

Step 3: Consolidating findings at each scope of assessment tier

EWG calculates a separate score for each scope of assessment tier. Each tier score incorporates all findings that fall within that tier. The tier score represents a weighted average of the strength of association for the findings at a given tier.

Equation 1

equation

where

equation

where SA = strength of association, u = uncertainty for each finding, severity = severity score, and w is a weighting factor that resolves cases of conflicting findings, with some showing an association between a substance and an endpoint and others not. In these cases, the weighting factor serves to emphasize the findings that show an association, reflecting the precautionary principle. The weighting function increases linearly with increased strength of association.

Step 4: Consolidating scope of assessment tier scores into an endpoint score

EWG consolidates tier scores into a single endpoint score using a tier weight and based on the number of studies in each tier. It ranges from -10 to +10.

Equation 2

equation

where TS is the tier score, TW is the tier weight, and Max(TW) is the maximum calculated tier weight across all tiers, as described in Equation 4 below. The sums are performed over all scope of assessment tiers (0 to 6).

Calculating the tier weight is accomplished using the following equation: Equation 3

equation

where (see table below for values associated with these variables):

TWmax is the maximum possible weight for a tier
TWmin is the minimum possible weight for a tier
ΔTW = TWmax - TWmin
n90 is the number of studies required to reach TWmin + 90%(ΔTW)
nfindingsmax is the maximum number of findings on a tier that fall at the same strength of association

Scope of assessment tierTWminTWmaxn90
610101
599.92
469.53
347.55
224.010
10.51.015

When multiple findings on a tier come to similar conclusions as represented by the strength of association, those findings taken together have an increased tier weight (TWmax). This reflects that there is a higher level of consensus on the question of how strongly a substance is associated with an endpoint.

Step 5: Calculating the consolidated scores for endpoint groups

We use the weight-of-evidence method as shown in Equation 4 and Equation 5 below to combine the scores for different health endpoints to calculate an overall unweighted substance score.

Weight-of-evidence score

The consolidated score is a combination of all severity scores with a weighting factor that emphasizes the findings that show an association, reflecting the precautionary practice when data conflict.

Equation 4

equation

where S is the score from each endpoint, group or subcategory being consolidated; Sneg are scores < 0; Smax is the maximum score; the sum of logs are limited to scores > 1; and wea is a weight of evidence adjustment given by:

Equation 5

equation

where rank is an ordered list of scores ranged 0-10 ranked from highest (10) to lowest (0).

References

ATSDR. 1999. Toxicological Profile for Mercury. Atlanta, GA: U.S. Department of Health and Human Services, Public Health Service

ATSDR. 2007. Toxicological Profile for Arsenic. Atlanta, GA: U.S. Department of Health and Human Services, Public Health Service

CDC. 2013. Antimicrobial Resistance: Threat Report 2013. Centers for Disease Control and Prevention. http://www.cdc.gov/drugresistance/threat-report-2013 CFR. 2014. US Government Printing Office. Available: http://www.gpo.gov/fdsys/browse/collectionCfr.action?collectionCode=CFR

Cogliani C, Goossens H, Greko C. 2011. Restricting antimicrobial use in food animals: lessons from Europe. Microbe 6, 6: 274-279

Consumer Reports. 2012. Arsenic in your food. Consumer Reports magazine, November 2012. http://www.consumerreports.org/cro/magazine/2012/11/arsenic-in-your-food/index.htm

EC. 1999. Report on Public Health Aspects of the Use of Bovine Somatotrophin - 15-16 March 1999. European Commission, Scientific Committee on Veterinary Measures relating to Public Health. http://ec.europa.eu/food/fs/sc/scv/out19_en.html

EFSA. 2003-2014. European Food Safety Journal. Available: http://www.efsa.europa.eu/en/publications/efsajournal.htm

EFSA. 2007. EFSA Opinion of the Scientific Panel on contaminants in the food chain (CONTAM) related to hormone residues in bovine meat and meat products. European Food Safety Authority. July 18, 2007. http://www.efsa.europa.eu/en/efsajournal/pub/510.htm

EFSA. 2009. Scientific Opinion on Arsenic in Food. EFSA Journal 2009; 7(10):1351

EFSA. 2010. Results of monitoring of non dioxin-like PCBs in food and feed. European Food Safety Authority. EFSA Journal 8(7):1701 http://www.efsa.europa.eu/en/efsajournal/pub/1701.htm

EFSA. 2012. Update of monitoring of dioxins and PCBs in food and feed. European Food Safety Authority. EFSA Journal 10(7):2832. http://www.efsa.europa.eu/en/efsajournal/pub/2832.htm

EPA, 2014. Office of Pesticide Programs. Pesticides: Health and Safety; Pesticides: Environmental Effects Available: http://www.epa.gov/pesticides/ecosystem/ [Accessed Aug. 27, 2014]

EWG. 2007. Bisphenol A: Toxic Plastics Chemical in Canned Food. Environmental Working Group. http://www.ewg.org/research/bisphenol

EWG. 2013. Superbugs Invade American Supermarkets. Environmental Working Group. http://www.ewg.org/meateatersguide/superbugs/

EWG. 2014. EWG Shopper's Guide to Pesticides in Produce. Environmental Working Group. http://www.ewg.org/foodnews/

EPA. 2001. Methylmercury (MeHg) (CASRN 229-92-6). U.S. Environmental Protection Agency, Integrated Risk Information System. Last revised July 2001.

FDA. 2006. Food Allergen Labeling And Consumer Protection Act of 2004 Questions and Answers. http://www.fda.gov/Food/GuidanceRegulation/GuidanceDocumentsRegulatoryInformation/Allergens/ucm106890.htm

FDA. 2007a. Redbook 2000. Guidance for Industry and Other Stakeholders Toxicological Principles for the Safety Assessment of Food Ingredients (Revised July 2007). Food and Drug Administration. http://www.fda.gov/downloads/Food/GuidanceRegulation/UCM222779.pdf

FDA. 2007b. Dioxins Analysis Results/Exposure Estimates. Total Diet Study. Food and Drug Administration. Updated November 2007. http://www.fda.gov/Food/FoodborneIllnessContaminants/ChemicalContaminants/ucm077444.htm

FDA. 2011. National Antimicrobial Resistance Monitoring System (NARMS) Retail Meat Annual Report, 2011. http://www.fda.gov/AnimalVeterinary/SafetyHealth/AntimicrobialResistance/NationalAntimicrobialResistanceMonitoringSystem/ucm334828.htm

FDA. 2012. Guidance for Industry: The Judicious Use of Medically Important Antimicrobial Drugs in Food-Producing Animals. #209. April 13, 2012

FDA. 2013a. Priority-based Assessment of Food Additives. Food and Drug Administration

FDA. 2013b. Arsenic in Rice and Rice Products. Food and Drug Administration. http://www.fda.gov/food/foodborneillnesscontaminants/metals/ucm319870.htm

FDA. 2013c. Phasing Out Certain Antibiotic Use in Farm Animals. Food and Drug Administration, Consumer Updates. http://www.fda.gov/forconsumers/consumerupdates/ucm378100.htm

FDA. 2013d. FDA Center for Veterinary Medicate submission to Joint FAO/WHO Expert Committee (JECFA) on Food Additives. November 5-14, 2013. http://www.fda.gov/downloads/AnimalVeterinary/SafetyHealth/ProductSafetyInformation/UCM383073.pdf

FDA. 2014a. Fish: What Pregnant Women and Parents Should Know. Draft Updated Advice by FDA and EPA. Food and Drug Administration and Environmental Protection Agency. June 2014. http://www.fda.gov/Food/FoodborneIllnessContaminants/Metals/ucm393070.htm

FDA. 2014b. Bovine Somatotropin (BST). Food and Drug Administration. http://www.fda.gov/downloads/AnimalVeterinary/SafetyHealth/ProductSafetyInformation/UCM383073.pdf

Foran JA, Carpenter DO, Hamilton MC, et al. 2005. Risk-based consumption advice for farmed Atlantic and wild Pacific salmon contaminated with dioxins and dioxin-like compounds. Environmental Health Perspectives. 113(5):552-6

Karimi R, Fitzgerald TP, Fisher NS. 2012. A quantitative synthesis of mercury in commercial seafood and implications for exposure in the United States. Environ Health Perspectives 120(11):1512-9

Liao C, Kannan K. 2013. Concentrations and profiles of bisphenol A and other bisphenol analogues in foodstuffs from the United States and their implications for human exposure. J Agric Food Chem. 15;61(19):4655-62

Maron DF, Smith TJS, Nachman KE. 2013. Restrictions on antimicrobial use in food animal production: an international regulatory and economic survey. Globalization and Health 9:48. http://www.globalizationandhealth.com/content/9/1/48

Noonan GO, Ackerman LK, Begley TH. 2011. Concentration of bisphenol A in highly consumed canned foods on the U.S. market. J Agric Food Chem. 59(13):7178-85

NTP. 2008. NTP-CERHR Monograph on the Potential Human Reproductive and Developmental Effects of Bisphenol A. NIH Publication No. 08-5994

Penn State Extension. 2013. Use of Beta-Agonists in Cattle Feed. Penn State College of Agricultural Sciences. http://extension.psu.edu/animals/beef/nutrition/articles/use-of-beta-agonists-in-cattle-feed

Pew Charitable Trusts. 2014. New FDA Policies on Antibiotic Use in Food Animal Production. February 2014.

Richter CA, Birnbaum LS, Farabollini F, Newbold RR, Rubin BS, Talsness CE, Vandenbergh JG, Walser-Kuntz DR, vom Saal FS. 2007. In vivo effects of bisphenol A in laboratory rodent studies. Reprod Toxicol. 24(2): 199-224

Rochester JR. 2013. Bisphenol A and human health: a review of the literature. Reprod Toxicol. 42:132-55

Rubin BS. 2011. Bisphenol A: an endocrine disruptor with widespread exposure and multiple effects. J Steroid Biochem Mol Biol. 127(1-2): 27-34

Serratosa J, Blass A, Rigau B, et al. 2006. Residues from veterinary medicinal products, growth promoters and performance enhancers in food-producing animals: a European Union perspective. Rev Sci Tech. 25(2): 637-53

Shaw SD, Brenner D, Berger ML. 2006. PCBs, PCDD/Fs, and organochlorine pesticides in farmed Atlantic salmon from Maine, eastern Canada, and Norway, and wild salmon from Alaska. Environmental Science and Technology. 40(17):5347-54

USDA. 2013. The Use of Growth-Promoting Implants in the U.S. Feedlots. U.S. Department of Agriculture, Animal and Plant Health Inspection Service. http://www.aphis.usda.gov/animal_health/nahms/feedlot/downloads/feedlot2011/Feed11_is_Implant.pdf

WHO. 1972-2014. Joint FAO/WHO Expert Committee on Food Additives (JECFA) monographs. Food and Agriculture Organization/World Health Organization. WHO Food Additives Series. http://www.who.int/foodsafety/chem/jecfa/publications/monographs/en/

WHO. 2010. Exposure To Dioxins And Dioxin-Like Substances: A Major Public Health Concern. Available: http://www.who.int/ipcs/features/dioxins.pdf?ua=1 [Accessed Aug. 27, 2014]

 

eat well guide

find sustainable food!

The Eat Well Guide helps consumers find locally grown and sustainably produced food. Listings include farms, restaurants, stores, farmers' markets, and CSAs throughout the United States.

Visit the Eat Well Guide

Information

Food Categories:

Search:

Share:

Want more food tips?

Sign up to receive email updates, action alerts, healthy eating tips, promotions to support our work and more from EWG! You can opt out at any time.

Select one:

Consumer Contact Form:

EWG reserves the right to make public any information provided to it by companies. I understand that EWG does not accept any confidential or trade secret information that EWG does not have permission to share publicly. If I am contacting EWG on behalf of a company, by clicking "submit," I affirm that I have the authority to contact EWG on my company's behalf and to provide EWG permission to publicly disseminate all information provided.

Company Contact Form:

I would like to update my product's:







Other inquiries:








EWG reserves the right to make public any information provided to it by companies. I understand that EWG does not accept any confidential or trade secret information that EWG does not have permission to share publicly. If I am contacting EWG on behalf of a company, by clicking "submit," I affirm that I have the authority to contact EWG on my company's behalf and to provide EWG permission to publicly disseminate all information provided.

Get the Guide