Study guide
1.3 Evaluating Experimental Design, Sample Size, Validity & Sources of Error
HiSET Science Study Guide Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Scientific Inquiry, Data Interpretation, Experimental Design & Scientific Evaluation
1.1 The Scientific Method, Hypotheses & Experimental Variables 1.2 Interpreting Data Visualizations: Graphs, Data Tables, Charts & Trends 1.3 Evaluating Experimental Design, Sample Size, Validity & Sources of Error 1.4 Scientific Models, Evidence-Based Explanations & Correlation vs. Causation 1.5 Identifying the Research Question & Selecting the Best Procedure 1.6 Hypotheses, Assumptions, Data & Conclusions: Relevance and Source Reliability
2 Chapter 2: Cell Structure, Function & Transport
2.1 Cell Theory, Prokaryotes vs. Eukaryotes, and Cellular Organization 2.2 Eukaryotic Organelles: Nucleus, Mitochondria, Ribosomes, ER, Golgi, and Lysosomes 2.3 Plant vs. Animal Cell Specializations and Cellular Transport (Diffusion, Osmosis, and Active Transport)
3 Chapter 3: Cellular Energy & Cell Division
3.1 Photosynthesis: Light Reactions, Calvin Cycle, Chloroplast Structure, and Energy Storage in Glucose 3.2 Cellular Respiration, Glycolysis, Krebs Cycle, Electron Transport, and Aerobic vs. Anaerobic Fermentation 3.3 The Cell Cycle, Mitosis (Asexual Reproduction/Somatic Growth) vs. Meiosis (Gamete Formation, Crossing Over, Genetic Variation)
4 Chapter 4: Molecular Genetics, Inheritance & Punnett Squares
4.1 DNA Structure, Double Helix, Nucleotides, and Semi-Conservative Replication 4.2 RNA Types, Transcription, Translation, and the Universal Genetic Code 4.3 Mendelian Inheritance, Dominant vs. Recessive Alleles, Genotype vs. Phenotype, and Monohybrid Punnett Squares
5 Chapter 5: Complex Genetics, Mutations & Biotechnology
5.1 Complex Inheritance: Incomplete Dominance, Codominance, Sex-Linked Traits, Polygenic Traits, and Pedigrees 5.2 Genetic Mutations: Point Mutations, Frameshift Mutations, Chromosomal Aberrations, and Environmental Mutagens 5.3 Modern Biotechnology: Recombinant DNA, PCR, Gel Electrophoresis, GMOs, and Gene Therapy Ethics
6 Chapter 6: Evolution, Natural Selection & Classification
6.1 Darwinian Natural Selection, Adaptation & Evolutionary Fitness 6.2 Evidence for Evolution: Fossils, Anatomy & Molecular Homology 6.3 Speciation Mechanisms, Cladograms & Biological Classification
7 Chapter 7: Ecology, Food Webs & Biogeochemical Cycles
7.1 Levels of Ecological Organization, Biotic & Abiotic Factors, and Global Biomes 7.2 Trophic Levels, Food Webs & Ecological Energy Pyramids 7.3 Biogeochemical Cycles: Hydrologic, Carbon-Oxygen & Nitrogen Cycles
8 Chapter 8: Population Dynamics, Ecological Interactions & Succession
8.1 Community Interactions: Competition, Predation & Symbiosis 8.2 Population Dynamics: Growth Curves & Limiting Factors 8.3 Ecological Succession, Biodiversity & Ecosystem Resilience
9 Chapter 9: Human Body Systems & Homeostasis
9.1 Circulatory and Respiratory Systems: Heart Chambers, Systemic vs. Pulmonary Circuits, Blood Vessels, Blood Components, Alveolar Gas Exchange, and Oxygen/CO2 Transport 9.2 Digestive and Excretory Systems: Mechanical vs. Chemical Digestion, Peristalsis, Enzyme Action, Small Intestine Absorption, and Nephron Function 9.3 Nervous and Endocrine Systems: Neuron Anatomy, Action Potentials, Synapses, Central vs. Peripheral Nervous System, Reflex Arcs, Endocrine Glands and Hormones, and Skeletal-Muscular Lever Action 9.4 Immune Defense, Pathogens, Vaccines, and Systemic Homeostasis via Negative Feedback Loops
10 Chapter 10: Chemistry: Atoms, Periodic Table & Chemical Bonding
10.1 Atomic Structure, Subatomic Particles & Isotopes 10.2 The Periodic Table, Groups, Periods & Periodic Trends 10.3 Chemical Bonding: Ionic, Covalent & Metallic Bonds
11 Chapter 11: Chemical Reactions, States of Matter, Acids & Bases
11.1 Physical vs. Chemical Changes, Reaction Types & Conservation of Mass 11.2 States of Matter, Kinetic Molecular Theory & Phase Transitions 11.3 Solutions, Solubility, Acids, Bases & the pH Scale
12 Chapter 12: Physics: Motion, Newton's Laws & Energy Conservation
12.1 Kinematics: Distance, Displacement, Velocity & Motion Graphs 12.2 Newton's Laws of Motion, Friction & Universal Gravitation 12.3 Work, Power & Conservation of Mechanical Energy
13 Chapter 13: Physics: Waves, the Electromagnetic Spectrum, Electricity & Magnetism
13.1 Wave Mechanics: Properties, Behaviors & Sound Waves 13.2 The Electromagnetic Spectrum: Waves, Photons & Applications 13.3 Electricity, Circuits & Electromagnetism
14 Chapter 14: Earth's Structure, Plate Tectonics & the Rock Cycle
14.1 Earth's Interior Structure: Compositional & Mechanical Layers 14.2 Plate Tectonics: Seafloor Spreading, Convection & Plate Boundaries 14.3 Earthquakes, Volcanoes & the Rock Cycle
15 Chapter 15: Geologic Time, Fossils, Hydrosphere & the Water Cycle
15.1 Weathering, Erosion, Deposition & Soil Horizons 15.2 Geologic Time, Stratigraphy & the Fossil Record 15.3 The Global Hydrosphere, Groundwater Dynamics & Oceans
16 Chapter 16: Atmosphere, Weather Systems & Climate Change
16.1 Earth's Atmosphere: Composition, Pressure Dynamics & Thermal Layers 16.2 Weather Systems, Air Masses, Frontal Boundaries & Severe Storms 16.3 Climate Dynamics, the Greenhouse Effect & Earth's Resources
17 Chapter 17: Astronomy: Solar System, Earth-Moon System & the Universe
17.1 The Solar System & Gravitational Motion 17.2 The Earth-Moon-Sun System: Cycles, Seasons, Phases & Tides 17.3 Stars, Stellar Evolution & Modern Cosmology
1.3 Evaluating Experimental Design, Sample Size, Validity & Sources of Error
Key Takeaways Explain This Topic AI
Internal validity requires isolating the independent variable so that observed changes in the dependent variable can be attributed solely to the experimental manipulation.
External validity assesses whether experimental findings can be generalized beyond the laboratory setting to real-world populations or ecosystems.
Large sample sizes and multiple experimental trials reduce the influence of random error and individual anomalies, yielding representative mean values.
Systematic errors consistently skew data in one direction due to equipment or procedural defects and cannot be corrected by averaging repeated trials, whereas random errors cause unpredictable fluctuations around the true mean that cancel out with repetition.
Last updated: September 2026
Evaluating Experimental Design, Sample Size, Validity & Sources of Error
Quick Answer: A valid experiment tests only one independent variable at a time, uses an adequate sample size, includes control groups, and holds all other factors constant. Internal validity means the experiment truly measured what it intended to measure without confounding variables. Systematic errors (such as an uncalibrated scale) skew all data in one direction and cannot be fixed by averaging trials, while random errors (unpredictable fluctuations) cancel out through large sample sizes and repeated trials.
On the HiSET Science test, several questions present experimental setups that contain subtle or obvious flaws. Your task is to evaluate whether the methodology supports the stated conclusions, identify uncontrolled variables, and recommend procedural improvements.
Methodological Rigor: Validity and Reliability
High-quality scientific research rests on three core standards:
1. Internal Validity
Internal validity measures how conclusively an experiment proves that changes in the dependent variable were caused solely by the independent variable. If an unmonitored factor (such as temperature or soil type) varies between groups, it becomes a confounding variable , destroying internal validity and preventing causal claims.
2. External Validity (Generalizability)
External validity reflects how well laboratory findings apply to real-world populations or natural ecosystems. A fertilizer that doubles yield in a sterile greenhouse might fail in open fields subject to drought and pests. While the greenhouse study has high internal validity, external validity requires field testing.
3. Reliability and Reproducibility
Reliability (Precision): The degree to which a procedure yields consistent measurements under identical conditions.
Reproducibility: The ability of independent researchers to duplicate the experiment following published methods and obtain matching results. Findings that cannot be replicated are rejected.
Confounding Variables and Methodological Flaws
A confounding variable is an uncontrolled extraneous factor that correlates with both independent and dependent variables, obscuring the true relationship. Common design flaws tested on the HiSET include:
Failure to Standardize Conditions: Exposing test groups to differing parameters (e.g., sunlight vs. shade).
Selection Bias: Non-random assignment of subjects. Placing healthier subjects in the treatment group biases results.
Lack of Placebo / Blinding Controls: In clinical studies, participant expectations can trigger changes (the placebo effect ). To eliminate bias:
Single-Blind Study: Participants do not know whether they are receiving the active treatment or a placebo.
Double-Blind Study: Neither participants nor researchers administering treatments know group assignments until data collection ends.
Sample Size, Replication, and Statistical Power
A frequent flaw in flawed experiments is an insufficient sample size (n). Testing only one or two subjects yields unreliable data:
The Danger of Small Sample Sizes
Living organisms exhibit natural genetic and environmental variation. If a hormone is tested on two animals, one may grow rapidly due to genetics while the other suffers an unobserved illness. With n = 2, individual anomalies dominate data, producing false conclusions.
The Law of Large Numbers
As sample size increases (n ≥ 30, 100), individual anomalies and measurement noise average out. The sample mean approaches the true population mean, and standard error decreases, giving the statistical power needed to detect genuine treatment effects.
Repeated Trials (Replication)
Every experiment must incorporate multiple independent trials (k ≥ 3). Testing only once leaves results vulnerable to transient equipment fluctuations or contamination. Replication confirms consistency, allowing researchers to calculate standard deviation and identify outliers.
Categorizing Scientific Error: Systematic vs. Random Error
Scientists distinguish between two fundamentally different categories of experimental error:
Feature Systematic Error (Bias) Random Error (Noise)
Definition Predictable bias that skews all measurements consistently in one direction (+ or -). Unpredictable fluctuations that cause measurements to scatter above and below the true value.
Primary Causes Miscalibrated instruments (e.g., scale reading +0.5 g too high), zero-offset errors, contaminated reagents. Minor drafts, electrical noise, human eye parallax when reading a meniscus, slight temperature shifts.
Impact on Data Destroys accuracy (closeness to true value) while maintaining apparent precision. Decreases precision (repeatability) while the average value may still center on the true value.
Reduced by Averaging? NO. Repeating the measurement 1,000 times reproduces the identical skewed bias every time. YES. Because random fluctuations occur equally in positive and negative directions, averaging cancels them out.
Corrective Action Recalibrate instruments, redesign protocols, re-zero balances. Increase sample size (n) and collect multiple repeated measurements.
HiSET Scenario Walkthrough: Auditing a Flawed Experiment
Flawed Scenario: A student tests whether Pesticide X protects tomato plants from aphids better than water alone. She uses 6 plants:
Group 1 (Pesticide X): 3 large, flowering plants in loam soil in a greenhouse are sprayed with Pesticide X.
Group 2 (Control): 3 small seedlings in sandy soil in an outdoor garden are sprayed with water.
After 4 weeks, Group 1 has fewer aphids and higher fruit yield. The student concludes Pesticide X is highly effective.
Methodological Audit & Critique
This experiment contains severe flaws that invalidate the conclusion:
Confounding Variables: Plant maturity, soil type, moisture, sunlight, and temperature all differed between groups.
Inadequate Sample Size: Testing only 3 plants per group provides virtually no statistical power; a single resistant plant could skew results.
Selection Bias: Healthier plants were placed in the treatment group.
Redesign Protocol: Use at least 50 uniform seedlings of identical age; pot them in identical soil; keep all plants in the same greenhouse; randomly assign 25 to Pesticide X and 25 to water; record aphid counts at regular intervals.