Study guide

Notation, Unit Conversions, Ratios, and Decibels

ASQ Certified Calibration Technician CCT

Track your progress as you read

Finished sections get a check mark.

1 SI Definitions and Realizations

The Seven SI Base Units and the 2019 Revision Realizing SI Units in the Laboratory

2 Derived Units, Prefixes, and Constants

Derived Units and Physical Calculations SI Prefixes and Compound-Unit Conversions Physical Constants and Their Applications

3 Measurement Standards and Selection

Types and Roles of Measurement Standards Selecting Standards for an Operational Hierarchy

4 Traceability and Standard Substitution

Traceability Evidence and Impact Review Evaluating and Authorizing Standard Substitution

5 Dimensional and Mechanical Metrology

Linear Displacement and Dimensional Instruments Geometry, Thermal Expansion, and Mass Force, Torque, Pressure, Density, and Viscosity

6 Temperature, Humidity, Volume, and Flow

Temperature Sensors and Thermometer Corrections Humidity, Volume, and Flow

7 Electrical, Optical, and Time Measurements

Voltage, Current, Resistance, and AC Measurement Time, Frequency, Light, Power, and Energy

8 Measurement Methods

Direct, Indirect, and Ratio Methods Transfer, Differential, and UUT Substitution Methods

9 Instrument Characteristics and Measurement Data

Sensitivity, Variability, and Precision Bias, Linearity, Stability, and Hysteresis Data Formats, Readability, and Suitability Resolution, Confidentiality, and Data Integrity

10 Specifications and Error Sources

Interpreting IM&TE Specification Terms Specification Calculations and Operating Qualifiers Instrument, Operator, and Circuit Errors Environmental, Geometric, and Interference Errors

11 Measurement Assurance

Check Standards and Statistical Process Control Proficiency Comparisons and Gage R&R

12 Procedures and Governing Requirements

Procedure Structure and Published Practice Sources Method Development, Deviations, and Handling Regulations, Contracts, and Requirement Precedence

13 Instrument Adjustment and Standardization

Calibration, Zero, and Span Adjustment Linearization, Nulling, and Digital Adjustment

14 Calibration Environment

Temperature and Humidity Control ESD, Dust, Vibration, and Pressure

15 Operations, Roles, and Scheduling

Receiving, As-Found Testing, and Calibration Flow Asset Identification, Tracking, and Release Laboratory Roles and Initial Scheduling Interval Review, Overdue Control, and Workloads

16 Method and Software Validation

Method Verification and Validation Studies Software, Firmware, and Spreadsheet Validation

17 Records and Formal Reporting

Controlled Records and Calibration Certificates Status Labels and Out-of-Tolerance Reporting

18 Uncertainty Concepts and Decision Risk

Error, Uncertainty, and Measurand Definition TAR, TUR, Tolerance Consumption, and Capability Conformity Decisions and False-Accept Risk Guard Bands and Conformity Reporting

19 Uncertainty Models and Budgets

Measurement Models and Uncertainty Contributions Distribution Selection and Budget Assembly

20 Type A and Type B Evaluation

Type A Statistics and Repeatability Outlier Review and Worked Type A Analysis Type B Distribution Models

21 Uncertainty Reporting and Applied Mathematics

Propagation, Covariance, and Effective Degrees of Freedom Coverage Factors and Expanded-Uncertainty Reporting Notation, Unit Conversions, Ratios, and Decibels Rounding, Algebra, Graphs, and Angular Conversions

22 Quality Systems and Tools

Quality-System Components, Planning, and Improvement The Seven Basic Quality Tools

23 Audits and Corrective Action

Audit Types, Planning, and Nonconformance Control Impact Assessment, Root Cause, and Effectiveness

24 Safety, Standards, and Professional Conduct

Electrical, Pressure, Chemical, and Laboratory Hazards PPE, Safety Data Sheets, 6S, and Safe Restoration Quality Standards, GUM, and VIM Accreditation Scope and Professional Ethics

Notation, Unit Conversions, Ratios, and Decibels

Key Takeaways Explain This Topic AI

Engineering notation uses exponents that are multiples of three.

Area and volume conversions require squared and cubed linear conversion factors.

Power-ratio decibels use 10 log₁₀; voltage-ratio 20 log₁₀ requires the appropriate impedance relationship.

Last updated: October 2026

Daily operations in calibration laboratories require rapid, error-free execution of applied technical mathematics. From converting non-SI process transmitter readings into base SI units to resolving dynamic decibel attenuation on RF spectrum analyzers, the calibration technician must possess a rigorous command of mathematical principles.

Scientific vs. Engineering Notation and SI Metric Prefixes

In metrology, physical quantities span dozens of orders of magnitude—from sub-atomic displacements in laser interferometers ( 10 − 12 m 10^{-12}\text{ m} 1 0 − 12 m ) to electrical insulation resistances ( 10 12 Ω 10^{12}\ \Omega 1 0 12 Ω ).

Scientific Notation

A number expressed in scientific notation has the form: m × 10 n , where 1 ≤ ∣ m ∣ < 10 and n ∈ Z m \times 10^n, \quad \text{where } 1 \le |m| < 10 \text{ and } n \in \mathbb{Z} m × 1 0 n , where 1 ≤ ∣ m ∣ < 10 and n ∈ Z

Example: 1 , 475 , 000 Ω = 1.475 × 10 6 Ω 1,475,000\ \Omega = 1.475 \times 10^6\ \Omega 1 , 475 , 000 Ω = 1.475 × 1 0 6 Ω

Example: 0.000000035 s = 3.5 × 10 − 8 s 0.000000035\text{ s} = 3.5 \times 10^{-8}\text{ s} 0.000000035 s = 3.5 × 1 0 − 8 s

Engineering Notation

Engineering notation imposes an additional operational constraint: the exponent n n n must be an integer multiple of 3 ( n ∈ { … , − 9 , − 6 , − 3 , 0 , 3 , 6 , 9 , 12 , … } n \in \{\dots, -9, -6, -3, 0, 3, 6, 9, 12, \dots\} n ∈ { … , − 9 , − 6 , − 3 , 0 , 3 , 6 , 9 , 12 , … } ), and the mantissa m m m satisfies 1 ≤ ∣ m ∣ < 1000 1 \le |m| < 1000 1 ≤ ∣ m ∣ < 1000 .

Engineering notation is preferred in calibration because every exponent of 3 corresponds directly to an established SI prefix , facilitating direct instrument range selection:

SI Prefix Symbol Multiplication Factor Engineering Exponent Calibration Laboratory Example

Tera T \text{T} T 1 , 000 , 000 , 000 , 000 1,000,000,000,000 1 , 000 , 000 , 000 , 000 10 12 10^{12} 1 0 12 Insulation resistance: 2.4 × 10 12 Ω = 2.4 T Ω 2.4 \times 10^{12}\ \Omega = 2.4\text{ T}\Omega 2.4 × 1 0 12 Ω = 2.4 T Ω

Giga G \text{G} G 1 , 000 , 000 , 000 1,000,000,000 1 , 000 , 000 , 000 10 9 10^9 1 0 9 RF frequency standard: 10.0 × 10 9 Hz = 10.0 GHz 10.0 \times 10^9\text{ Hz} = 10.0\text{ GHz} 10.0 × 1 0 9 Hz = 10.0 GHz

Mega M \text{M} M 1 , 000 , 000 1,000,000 1 , 000 , 000 10 6 10^6 1 0 6 Megohmmeter test: 1.5 × 10 6 Ω = 1.5 M Ω 1.5 \times 10^6\ \Omega = 1.5\text{ M}\Omega 1.5 × 1 0 6 Ω = 1.5 M Ω

Kilo k \text{k} k 1 , 000 1,000 1 , 000 10 3 10^3 1 0 3 High-voltage divider: 25.0 × 10 3 V = 25.0 kV 25.0 \times 10^3\text{ V} = 25.0\text{ kV} 25.0 × 1 0 3 V = 25.0 kV

(none) (none) 1 1 1 10 0 10^0 1 0 0 Base SI unit: 1.00000 V 1.00000\text{ V} 1.00000 V

Milli m \text{m} m 0.001 0.001 0.001 10 − 3 10^{-3} 1 0 − 3 Shunt current: 4.5 × 10 − 3 A = 4.5 mA 4.5 \times 10^{-3}\text{ A} = 4.5\text{ mA} 4.5 × 1 0 − 3 A = 4.5 mA

Micro μ \mu μ 0.000001 0.000001 0.000001 10 − 6 10^{-6} 1 0 − 6 Strain gage output: 125.0 × 10 − 6 V = 125.0 μ V 125.0 \times 10^{-6}\text{ V} = 125.0\ \mu\text{V} 125.0 × 1 0 − 6 V = 125.0 μ V

Nano n \text{n} n 0.000000001 0.000000001 0.000000001 10 − 9 10^{-9} 1 0 − 9 Time interval counter: 25.0 × 10 − 9 s = 25.0 ns 25.0 \times 10^{-9}\text{ s} = 25.0\text{ ns} 25.0 × 1 0 − 9 s = 25.0 ns

Pico p \text{p} p 0.000000000001 0.000000000001 0.000000000001 10 − 12 10^{-12} 1 0 − 12 Capacitance standard: 100.0 × 10 − 12 F = 100.0 pF 100.0 \times 10^{-12}\text{ F} = 100.0\text{ pF} 100.0 × 1 0 − 12 F = 100.0 pF

Femto f \text{f} f 0.000000000000001 0.000000000000001 0.000000000000001 10 − 15 10^{-15} 1 0 − 15 Ultra-low electrometer: 10.0 × 10 − 15 A = 10.0 fA 10.0 \times 10^{-15}\text{ A} = 10.0\text{ fA} 10.0 × 1 0 − 15 A = 10.0 fA

English to Metric Unit Conversions

Under international treaty agreements dating back to the 1959 International Yard and Pound Agreement, English (US Customary) units are legally defined in terms of exact SI metric equivalents.

Length, Area, and Volume Conversions

Length: 1 inch (in) = 25.4 mm = 0.0254 m (EXACT by definition) 1\text{ inch (in)} = 25.4\text{ mm} = 0.0254\text{ m} \quad \text{(EXACT by definition)} 1 inch (in) = 25.4 mm = 0.0254 m (EXACT by definition) 1 foot (ft) = 12 in = 304.8 mm = 0.3048 m 1\text{ foot (ft)} = 12\text{ in} = 304.8\text{ mm} = 0.3048\text{ m} 1 foot (ft) = 12 in = 304.8 mm = 0.3048 m 1 microinch ( μ in) = 10 − 6 in = 0.0254 μ m = 25.4 nm 1\text{ microinch (}\mu\text{in)} = 10^{-6}\text{ in} = 0.0254\ \mu\text{m} = 25.4\text{ nm} 1 microinch ( μ in) = 1 0 − 6 in = 0.0254 μ m = 25.4 nm

Area: 1 in 2 = ( 25.4 mm ) 2 = 645.16 mm 2 = 6.4516 × 10 − 4 m 2 1\text{ in}^2 = (25.4\text{ mm})^2 = 645.16\text{ mm}^2 = 6.4516 \times 10^{-4}\text{ m}^2 1 in 2 = ( 25.4 mm ) 2 = 645.16 mm 2 = 6.4516 × 1 0 − 4 m 2

Volume: 1 in 3 = ( 25.4 mm ) 3 = 16 , 387.064 mm 3 = 16.387064 cm 3 = 16.387064 mL 1\text{ in}^3 = (25.4\text{ mm})^3 = 16,387.064\text{ mm}^3 = 16.387064\text{ cm}^3 = 16.387064\text{ mL} 1 in 3 = ( 25.4 mm ) 3 = 16 , 387.064 mm 3 = 16.387064 cm 3 = 16.387064 mL 1 US Gallon = 231 in 3 = 3.785411784 Liters (L) 1\text{ US Gallon} = 231\text{ in}^3 = 3.785411784\text{ Liters (L)} 1 US Gallon = 231 in 3 = 3.785411784 Liters (L)

Mass and Force Conversions

Mass: 1 pound-mass (lb) = 0.45359237 kg (EXACT by definition) 1\text{ pound-mass (lb)} = 0.45359237\text{ kg} \quad \text{(EXACT by definition)} 1 pound-mass (lb) = 0.45359237 kg (EXACT by definition) 1 ounce (oz) = 1 16 lb = 28.349523125 g 1\text{ ounce (oz)} = \frac{1}{16}\text{ lb} = 28.349523125\text{ g} 1 ounce (oz) = 16 1 ​ lb = 28.349523125 g

Force: 1 pound-force (lbf) = 1 lb × 9.80665 m/s 2 = 4.4482216152605 N ≈ 4.44822 N 1\text{ pound-force (lbf)} = 1\text{ lb} \times 9.80665\text{ m/s}^2 = 4.4482216152605\text{ N} \approx 4.44822\text{ N} 1 pound-force (lbf) = 1 lb × 9.80665 m/s 2 = 4.4482216152605 N ≈ 4.44822 N

Pressure Conversions

Pressure is force per unit area ( P = F / A P = F/A P = F / A ). Converting between English and metric pressure units is a core CCT skill: 1 psi = 1 lbf 1 in 2 = 4.4482216 N 6.4516 × 10 − 4 m 2 = 6 , 894.757 Pa = 6.894757 kPa 1\text{ psi} = \frac{1\text{ lbf}}{1\text{ in}^2} = \frac{4.4482216\text{ N}}{6.4516 \times 10^{-4}\text{ m}^2} = 6,894.757\text{ Pa} = 6.894757\text{ kPa} 1 psi = 1 in 2 1 lbf ​ = 6.4516 × 1 0 − 4 m 2 4.4482216 N ​ = 6 , 894.757 Pa = 6.894757 kPa

From Unit To Unit Exact or Standard Multiplier Direct Conversion Formula

psi \text{psi} psi kPa \text{kPa} kPa 6.894757 6.894757 6.894757 P kPa = P psi × 6.894757 P_{\text{kPa}} = P_{\text{psi}} \times 6.894757 P kPa ​ = P psi ​ × 6.894757

psi \text{psi} psi bar \text{bar} bar 0.06894757 0.06894757 0.06894757 P bar = P psi × 0.06894757 P_{\text{bar}} = P_{\text{psi}} \times 0.06894757 P bar ​ = P psi ​ × 0.06894757

bar \text{bar} bar kPa \text{kPa} kPa 100.000 100.000 100.000 P kPa = P bar × 100 P_{\text{kPa}} = P_{\text{bar}} \times 100 P kPa ​ = P bar ​ × 100

bar \text{bar} bar psi \text{psi} psi 14.50377 14.50377 14.50377 P psi = P bar × 14.50377 P_{\text{psi}} = P_{\text{bar}} \times 14.50377 P psi ​ = P bar ​ × 14.50377

atm \text{atm} atm Pa \text{Pa} Pa 101 , 325 101,325 101 , 325 P Pa = P atm × 101 , 325 P_{\text{Pa}} = P_{\text{atm}} \times 101,325 P Pa ​ = P atm ​ × 101 , 325

atm \text{atm} atm psi \text{psi} psi 14.69595 14.69595 14.69595 P psi = P atm × 14.69595 P_{\text{psi}} = P_{\text{atm}} \times 14.69595 P psi ​ = P atm ​ × 14.69595

Torr \text{Torr} Torr Pa \text{Pa} Pa 101325 / 760 ≈ 133.322368 101325/760\approx133.322368 101325/760 ≈ 133.322368 Multiply Torr by 101325 / 760 101325/760 101325/760 ; distinguish physical mercury-column conventions.

inHg, conventional \text{inHg, conventional} inHg, conventional psi \text{psi} psi 0.491154 0.491154 0.491154 P psi = P inHg × 0.491154 P_{\text{psi}} = P_{\text{inHg}} \times 0.491154 P psi ​ = P inHg ​ × 0.491154

inHg, conventional \text{inHg, conventional} inHg, conventional kPa \text{kPa} kPa 3.386389 3.386389 3.386389 P kPa = P inHg × 3.386389 P_{\text{kPa}} = P_{\text{inHg}} \times 3.386389 P kPa ​ = P inHg ​ × 3.386389

inH 2 O (at 60 ∘ F) \text{inH}_2\text{O (at } 60^\circ\text{F)} inH 2 ​ O (at 6 0 ∘ F) Pa \text{Pa} Pa 248.84 248.84 248.84 P Pa = P inH2O × 248.84 P_{\text{Pa}} = P_{\text{inH2O}} \times 248.84 P Pa ​ = P inH2O ​ × 248.84

inH 2 O (at 60 ∘ F) \text{inH}_2\text{O (at } 60^\circ\text{F)} inH 2 ​ O (at 6 0 ∘ F) psi \text{psi} psi 0.03606 0.03606 0.03606 P psi = P inH2O × 0.03606 P_{\text{psi}} = P_{\text{inH2O}} \times 0.03606 P psi ​ = P inH2O ​ × 0.03606

  • Ratios, Percentages, Parts Per Million (ppm), and Decibels (dB)
  • Percentages vs. Parts Per Million (ppm)

In high-accuracy electrical and dimensional metrology, fractional tolerances are too minute to express comfortably as percentages. The metrology community uses parts per million (ppm) : Fraction = Error Nominal \text{Fraction} = \frac{\text{Error}}{\text{Nominal}} Fraction = Nominal Error ​ Percentage (%) = Fraction × 10 2 = Error Nominal × 100 \text{Percentage (\%)} = \text{Fraction} \times 10^2 = \frac{\text{Error}}{\text{Nominal}} \times 100 Percentage (%) = Fraction × 1 0 2 = Nominal Error ​ × 100 Parts Per Million (ppm) = Fraction × 10 6 = Error Nominal × 1 , 000 , 000 \text{Parts Per Million (ppm)} = \text{Fraction} \times 10^6 = \frac{\text{Error}}{\text{Nominal}} \times 1,000,000 Parts Per Million (ppm) = Fraction × 1 0 6 = Nominal Error ​ × 1 , 000 , 000

  • Direct Conversion Rules
  • 1 % = 10 , 000 ppm 1\% = 10,000\text{ ppm} 1% = 10 , 000 ppm
  • 0.1 % = 1 , 000 ppm 0.1\% = 1,000\text{ ppm} 0.1% = 1 , 000 ppm
  • 0.01 % = 100 ppm 0.01\% = 100\text{ ppm} 0.01% = 100 ppm
  • 0.001 % = 10 ppm 0.001\% = 10\text{ ppm} 0.001% = 10 ppm
  • 0.0001 % = 1 ppm 0.0001\% = 1\text{ ppm} 0.0001% = 1 ppm
  • Worked Example: Converting ppm Tolerance to Absolute Voltage

A multifunction calibrator is specified on its 10.00000 V 10.00000\text{ V} 10.00000 V range with a 1-year specification of: Tolerance = ± ( 15 ppm of Reading + 3 ppm of Range ) \text{Tolerance} = \pm (15\text{ ppm of Reading} + 3\text{ ppm of Range}) Tolerance = ± ( 15 ppm of Reading + 3 ppm of Range )

If the technician sets the calibrator output to 5.00000 V 5.00000\text{ V} 5.00000 V : Reading error = 5.00000 V × ( 15 × 10 − 6 ) = 0.000075 V = 75 μ V \text{Reading error} = 5.00000\text{ V} \times (15 \times 10^{-6}) = 0.000075\text{ V} = 75\ \mu\text{V} Reading error = 5.00000 V × ( 15 × 1 0 − 6 ) = 0.000075 V = 75 μ V Range error = 10.00000 V × ( 3 × 10 − 6 ) = 0.000030 V = 30 μ V \text{Range error} = 10.00000\text{ V} \times (3 \times 10^{-6}) = 0.000030\text{ V} = 30\ \mu\text{V} Range error = 10.00000 V × ( 3 × 1 0 − 6 ) = 0.000030 V = 30 μ V Total Absolute Tolerance = 75 μ V + 30 μ V = 105 μ V = 0.000105 V \text{Total Absolute Tolerance} = 75\ \mu\text{V} + 30\ \mu\text{V} = 105\ \mu\text{V} = 0.000105\text{ V} Total Absolute Tolerance = 75 μ V + 30 μ V = 105 μ V = 0.000105 V

Decibels (dB) in Power and Voltage Metrology

The decibel (dB) is a logarithmic ratio used in RF, acoustics, and signal processing.

Power Ratios ( 10 log ⁡ 10 10 \log_{10} 10 lo g 10 ​ )

Because power is directly proportional to energy, the decibel power ratio is defined as: dB = 10 log ⁡ 10 ( P 2 P 1 ) \text{dB} = 10 \log_{10}\left(\frac{P_2}{P_1}\right) dB = 10 lo g 10 ​ ( P 1 ​ P 2 ​ ​ )

+ 3 dB ≈ 2 × power +3\text{ dB} \approx 2\times\text{ power} + 3 dB ≈ 2 × power (e.g., 10 mW → 20 mW 10\text{ mW} \to 20\text{ mW} 10 mW → 20 mW )

+ 10 dB = 10 × power +10\text{ dB} = 10\times\text{ power} + 10 dB = 10 × power (e.g., 1 mW → 10 mW 1\text{ mW} \to 10\text{ mW} 1 mW → 10 mW )

+ 20 dB = 100 × power +20\text{ dB} = 100\times\text{ power} + 20 dB = 100 × power (e.g., 1 mW → 100 mW 1\text{ mW} \to 100\text{ mW} 1 mW → 100 mW )

− 3 dB ≈ 0.5 × power -3\text{ dB} \approx 0.5\times\text{ power} − 3 dB ≈ 0.5 × power (half-power cut-off frequency)

− 10 dB = 0.1 × power -10\text{ dB} = 0.1\times\text{ power} − 10 dB = 0.1 × power ( 90 % 90\% 90% attenuation)

Voltage and Field Ratios ( 20 log ⁡ 10 20 \log_{10} 20 lo g 10 ​ )

By Joule's law, power is proportional to voltage squared ( P = V 2 / R P = V^2 / R P = V 2 / R ). When impedances are equal ( R 1 = R 2 R_1 = R_2 R 1 ​ = R 2 ​ ): dB = 10 log ⁡ 10 ( V 2 2 / R V 1 2 / R ) = 10 log ⁡ 10 [ ( V 2 V 1 ) 2 ] = 20 log ⁡ 10 ( V 2 V 1 ) \text{dB} = 10 \log_{10}\left(\frac{V_2^2 / R}{V_1^2 / R}\right) = 10 \log_{10}\left[\left(\frac{V_2}{V_1}\right)^2\right] = 20 \log_{10}\left(\frac{V_2}{V_1}\right) dB = 10 lo g 10 ​ ( V 1 2 ​ / R V 2 2 ​ / R ​ ) = 10 lo g 10 ​ [ ( V 1 ​ V 2 ​ ​ ) 2 ] = 20 lo g 10 ​ ( V 1 ​ V 2 ​ ​ )

+ 6 dB ≈ 2 × voltage +6\text{ dB} \approx 2\times\text{ voltage} + 6 dB ≈ 2 × voltage (doubling voltage)

+ 20 dB = 10 × voltage +20\text{ dB} = 10\times\text{ voltage} + 20 dB = 10 × voltage (decade voltage increase)

  • + 40 dB = 100 × voltage +40\text{ dB} = 100\times\text{ voltage} + 40 dB = 100 × voltage
  • − 6 dB ≈ 0.5 × voltage -6\text{ dB} \approx 0.5\times\text{ voltage} − 6 dB ≈ 0.5 × voltage (half-voltage)

− 20 dB = 0.1 × voltage -20\text{ dB} = 0.1\times\text{ voltage} − 20 dB = 0.1 × voltage ( 90 % 90\% 90% voltage drop)

Warning

Classic Exam Trap: Using 10 log ⁡ 10 10 \log_{10} 10 lo g 10 ​ for voltage ratios or 20 log ⁡ 10 20 \log_{10} 20 lo g 10 ​ for power ratios. Always verify whether the physical quantity is power ( 10 log ⁡ 10 10 \log_{10} 10 lo g 10 ​ ) or field/voltage/current ( 20 log ⁡ 10 20 \log_{10} 20 lo g 10 ​ ).

Worked conversion checks with units

Consider a flow indication of 2 US gallons per minute. The volume factor is approximately 3.785412 litres per US gallon, giving about 7.570824 litres per minute. Dividing by sixty gives about 0.1261804 litre per second. The gallon definition matters: the imperial gallon has a different volume. A number labeled only “gallons” is incomplete information for a defensible conversion. Record the specified convention before calculating.

Area and volume provide another check. A rectangular opening measuring 2 inches by 3 inches has area 6 square inches. Multiplying by 645.16 square millimetres per square inch gives 3870.96 square millimetres. Multiplying only by the linear factor 25.4 would leave incorrect dimensions. A cubed-length conversion likewise needs the cubed linear factor, not the area factor. Cancel units on paper before entering numbers into a calculator.

Now suppose an amplifier’s output power is ten times its input power. The gain is 10 dB. If input and output resistances are equal, the associated voltage ratio is the square root of ten, approximately 3.162. A tenfold voltage ratio would instead correspond to a hundredfold power ratio and 20 dB under those equal-resistance conditions. With different resistances, calculate the actual power ratio from voltage squared divided by resistance. Do not use twenty times the voltage logarithm as a substitute for that impedance information.

To reverse the calculation, a power ratio equals ten raised to the gain in decibels divided by ten. Thus a negative 10 dB attenuation leaves one tenth of the initial power. Check direction: output divided by input gives negative gain for attenuation, while input divided by output gives a positive loss. Both conventions can appear in instrument documentation, so identify the stated ratio and sign before comparing results.

A conversion factor can be exact while a measurement remains uncertain. The defined inch-to-metre factor does not improve the uncertainty of the measured length. Convert the result and its uncertainty consistently, retaining intermediate precision until reporting.

Pressure-column conventions matter. The 248.84 Pa water-column factor above is for 60°F, not 20°C. Conventional inch-of-water uses a different factor, approximately 249.0889 Pa. Check the instrument convention before conversion. See NIST SP 811 conversion factors .