Study guide

1.3 Evaluating Experimental Design, Sample Size, Validity & Sources of Error

HiSET Science Study Guide Exam

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Scientific Inquiry, Data Interpretation, Experimental Design & Scientific Evaluation

1.1 The Scientific Method, Hypotheses & Experimental Variables 1.2 Interpreting Data Visualizations: Graphs, Data Tables, Charts & Trends 1.3 Evaluating Experimental Design, Sample Size, Validity & Sources of Error 1.4 Scientific Models, Evidence-Based Explanations & Correlation vs. Causation 1.5 Identifying the Research Question & Selecting the Best Procedure 1.6 Hypotheses, Assumptions, Data & Conclusions: Relevance and Source Reliability

2 Chapter 2: Cell Structure, Function & Transport

2.1 Cell Theory, Prokaryotes vs. Eukaryotes, and Cellular Organization 2.2 Eukaryotic Organelles: Nucleus, Mitochondria, Ribosomes, ER, Golgi, and Lysosomes 2.3 Plant vs. Animal Cell Specializations and Cellular Transport (Diffusion, Osmosis, and Active Transport)

3 Chapter 3: Cellular Energy & Cell Division

3.1 Photosynthesis: Light Reactions, Calvin Cycle, Chloroplast Structure, and Energy Storage in Glucose 3.2 Cellular Respiration, Glycolysis, Krebs Cycle, Electron Transport, and Aerobic vs. Anaerobic Fermentation 3.3 The Cell Cycle, Mitosis (Asexual Reproduction/Somatic Growth) vs. Meiosis (Gamete Formation, Crossing Over, Genetic Variation)

4 Chapter 4: Molecular Genetics, Inheritance & Punnett Squares

4.1 DNA Structure, Double Helix, Nucleotides, and Semi-Conservative Replication 4.2 RNA Types, Transcription, Translation, and the Universal Genetic Code 4.3 Mendelian Inheritance, Dominant vs. Recessive Alleles, Genotype vs. Phenotype, and Monohybrid Punnett Squares

5 Chapter 5: Complex Genetics, Mutations & Biotechnology

5.1 Complex Inheritance: Incomplete Dominance, Codominance, Sex-Linked Traits, Polygenic Traits, and Pedigrees 5.2 Genetic Mutations: Point Mutations, Frameshift Mutations, Chromosomal Aberrations, and Environmental Mutagens 5.3 Modern Biotechnology: Recombinant DNA, PCR, Gel Electrophoresis, GMOs, and Gene Therapy Ethics

6 Chapter 6: Evolution, Natural Selection & Classification

6.1 Darwinian Natural Selection, Adaptation & Evolutionary Fitness 6.2 Evidence for Evolution: Fossils, Anatomy & Molecular Homology 6.3 Speciation Mechanisms, Cladograms & Biological Classification

7 Chapter 7: Ecology, Food Webs & Biogeochemical Cycles

7.1 Levels of Ecological Organization, Biotic & Abiotic Factors, and Global Biomes 7.2 Trophic Levels, Food Webs & Ecological Energy Pyramids 7.3 Biogeochemical Cycles: Hydrologic, Carbon-Oxygen & Nitrogen Cycles

8 Chapter 8: Population Dynamics, Ecological Interactions & Succession

8.1 Community Interactions: Competition, Predation & Symbiosis 8.2 Population Dynamics: Growth Curves & Limiting Factors 8.3 Ecological Succession, Biodiversity & Ecosystem Resilience

9 Chapter 9: Human Body Systems & Homeostasis

9.1 Circulatory and Respiratory Systems: Heart Chambers, Systemic vs. Pulmonary Circuits, Blood Vessels, Blood Components, Alveolar Gas Exchange, and Oxygen/CO2 Transport 9.2 Digestive and Excretory Systems: Mechanical vs. Chemical Digestion, Peristalsis, Enzyme Action, Small Intestine Absorption, and Nephron Function 9.3 Nervous and Endocrine Systems: Neuron Anatomy, Action Potentials, Synapses, Central vs. Peripheral Nervous System, Reflex Arcs, Endocrine Glands and Hormones, and Skeletal-Muscular Lever Action 9.4 Immune Defense, Pathogens, Vaccines, and Systemic Homeostasis via Negative Feedback Loops

10 Chapter 10: Chemistry: Atoms, Periodic Table & Chemical Bonding

10.1 Atomic Structure, Subatomic Particles & Isotopes 10.2 The Periodic Table, Groups, Periods & Periodic Trends 10.3 Chemical Bonding: Ionic, Covalent & Metallic Bonds

11 Chapter 11: Chemical Reactions, States of Matter, Acids & Bases

11.1 Physical vs. Chemical Changes, Reaction Types & Conservation of Mass 11.2 States of Matter, Kinetic Molecular Theory & Phase Transitions 11.3 Solutions, Solubility, Acids, Bases & the pH Scale

12 Chapter 12: Physics: Motion, Newton's Laws & Energy Conservation

12.1 Kinematics: Distance, Displacement, Velocity & Motion Graphs 12.2 Newton's Laws of Motion, Friction & Universal Gravitation 12.3 Work, Power & Conservation of Mechanical Energy

13 Chapter 13: Physics: Waves, the Electromagnetic Spectrum, Electricity & Magnetism

13.1 Wave Mechanics: Properties, Behaviors & Sound Waves 13.2 The Electromagnetic Spectrum: Waves, Photons & Applications 13.3 Electricity, Circuits & Electromagnetism

14 Chapter 14: Earth's Structure, Plate Tectonics & the Rock Cycle

14.1 Earth's Interior Structure: Compositional & Mechanical Layers 14.2 Plate Tectonics: Seafloor Spreading, Convection & Plate Boundaries 14.3 Earthquakes, Volcanoes & the Rock Cycle

15 Chapter 15: Geologic Time, Fossils, Hydrosphere & the Water Cycle

15.1 Weathering, Erosion, Deposition & Soil Horizons 15.2 Geologic Time, Stratigraphy & the Fossil Record 15.3 The Global Hydrosphere, Groundwater Dynamics & Oceans

16 Chapter 16: Atmosphere, Weather Systems & Climate Change

16.1 Earth's Atmosphere: Composition, Pressure Dynamics & Thermal Layers 16.2 Weather Systems, Air Masses, Frontal Boundaries & Severe Storms 16.3 Climate Dynamics, the Greenhouse Effect & Earth's Resources

17 Chapter 17: Astronomy: Solar System, Earth-Moon System & the Universe

17.1 The Solar System & Gravitational Motion 17.2 The Earth-Moon-Sun System: Cycles, Seasons, Phases & Tides 17.3 Stars, Stellar Evolution & Modern Cosmology

1.3 Evaluating Experimental Design, Sample Size, Validity & Sources of Error

Key Takeaways Explain This Topic AI

Internal validity requires isolating the independent variable so that observed changes in the dependent variable can be attributed solely to the experimental manipulation.

External validity assesses whether experimental findings can be generalized beyond the laboratory setting to real-world populations or ecosystems.

Large sample sizes and multiple experimental trials reduce the influence of random error and individual anomalies, yielding representative mean values.

Systematic errors consistently skew data in one direction due to equipment or procedural defects and cannot be corrected by averaging repeated trials, whereas random errors cause unpredictable fluctuations around the true mean that cancel out with repetition.

Last updated: September 2026

Evaluating Experimental Design, Sample Size, Validity & Sources of Error

Quick Answer: A valid experiment tests only one independent variable at a time, uses an adequate sample size, includes control groups, and holds all other factors constant. Internal validity means the experiment truly measured what it intended to measure without confounding variables. Systematic errors (such as an uncalibrated scale) skew all data in one direction and cannot be fixed by averaging trials, while random errors (unpredictable fluctuations) cancel out through large sample sizes and repeated trials.

On the HiSET Science test, several questions present experimental setups that contain subtle or obvious flaws. Your task is to evaluate whether the methodology supports the stated conclusions, identify uncontrolled variables, and recommend procedural improvements.

Methodological Rigor: Validity and Reliability

High-quality scientific research rests on three core standards:

1. Internal Validity

Internal validity measures how conclusively an experiment proves that changes in the dependent variable were caused solely by the independent variable. If an unmonitored factor (such as temperature or soil type) varies between groups, it becomes a confounding variable , destroying internal validity and preventing causal claims.

2. External Validity (Generalizability)

External validity reflects how well laboratory findings apply to real-world populations or natural ecosystems. A fertilizer that doubles yield in a sterile greenhouse might fail in open fields subject to drought and pests. While the greenhouse study has high internal validity, external validity requires field testing.

3. Reliability and Reproducibility

Reliability (Precision): The degree to which a procedure yields consistent measurements under identical conditions.

Reproducibility: The ability of independent researchers to duplicate the experiment following published methods and obtain matching results. Findings that cannot be replicated are rejected.

Confounding Variables and Methodological Flaws

A confounding variable is an uncontrolled extraneous factor that correlates with both independent and dependent variables, obscuring the true relationship. Common design flaws tested on the HiSET include:

Failure to Standardize Conditions: Exposing test groups to differing parameters (e.g., sunlight vs. shade).

Selection Bias: Non-random assignment of subjects. Placing healthier subjects in the treatment group biases results.

Lack of Placebo / Blinding Controls: In clinical studies, participant expectations can trigger changes (the placebo effect ). To eliminate bias:

Single-Blind Study: Participants do not know whether they are receiving the active treatment or a placebo.

Double-Blind Study: Neither participants nor researchers administering treatments know group assignments until data collection ends.

Sample Size, Replication, and Statistical Power

A frequent flaw in flawed experiments is an insufficient sample size (n). Testing only one or two subjects yields unreliable data:

The Danger of Small Sample Sizes

Living organisms exhibit natural genetic and environmental variation. If a hormone is tested on two animals, one may grow rapidly due to genetics while the other suffers an unobserved illness. With n = 2, individual anomalies dominate data, producing false conclusions.

The Law of Large Numbers

As sample size increases (n ≥ 30, 100), individual anomalies and measurement noise average out. The sample mean approaches the true population mean, and standard error decreases, giving the statistical power needed to detect genuine treatment effects.

Repeated Trials (Replication)

Every experiment must incorporate multiple independent trials (k ≥ 3). Testing only once leaves results vulnerable to transient equipment fluctuations or contamination. Replication confirms consistency, allowing researchers to calculate standard deviation and identify outliers.

Categorizing Scientific Error: Systematic vs. Random Error

Scientists distinguish between two fundamentally different categories of experimental error:

Feature Systematic Error (Bias) Random Error (Noise)

Definition Predictable bias that skews all measurements consistently in one direction (+ or -). Unpredictable fluctuations that cause measurements to scatter above and below the true value.

Primary Causes Miscalibrated instruments (e.g., scale reading +0.5 g too high), zero-offset errors, contaminated reagents. Minor drafts, electrical noise, human eye parallax when reading a meniscus, slight temperature shifts.

Impact on Data Destroys accuracy (closeness to true value) while maintaining apparent precision. Decreases precision (repeatability) while the average value may still center on the true value.

Reduced by Averaging? NO. Repeating the measurement 1,000 times reproduces the identical skewed bias every time. YES. Because random fluctuations occur equally in positive and negative directions, averaging cancels them out.

Corrective Action Recalibrate instruments, redesign protocols, re-zero balances. Increase sample size (n) and collect multiple repeated measurements.

HiSET Scenario Walkthrough: Auditing a Flawed Experiment

Flawed Scenario: A student tests whether Pesticide X protects tomato plants from aphids better than water alone. She uses 6 plants:

Group 1 (Pesticide X): 3 large, flowering plants in loam soil in a greenhouse are sprayed with Pesticide X.

Group 2 (Control): 3 small seedlings in sandy soil in an outdoor garden are sprayed with water.

After 4 weeks, Group 1 has fewer aphids and higher fruit yield. The student concludes Pesticide X is highly effective.

Methodological Audit & Critique

This experiment contains severe flaws that invalidate the conclusion:

Confounding Variables: Plant maturity, soil type, moisture, sunlight, and temperature all differed between groups.

Inadequate Sample Size: Testing only 3 plants per group provides virtually no statistical power; a single resistant plant could skew results.

Selection Bias: Healthier plants were placed in the treatment group.

Redesign Protocol: Use at least 50 uniform seedlings of identical age; pot them in identical soil; keep all plants in the same greenhouse; randomly assign 25 to Pesticide X and 25 to water; record aphid counts at regular intervals.

Loading diagram... Methodological Audit: Evaluating Scientific Validity and Reliability