How to Calculate Precision
Determine the highest measured value., Find the lowest measured value., Subtract the lowest value from the highest., Report the range as the precision.
Step-by-Step Guide
-
Step 1: Determine the highest measured value.
It helps to begin by sorting your data in numerical order, from lowest to highest.
This will ensure that you do not miss any values.
Then select the value at the end of the list.
For example, suppose you are testing the precision of a scale, and you observe five measurements: 11, 13, 12, 14,
12.
After sorting, these values are listed as 11, 12, 12, 13,
14.
The highest measurement is
14. -
Step 2: Find the lowest measured value.
Once your data has been sorted, finding the lowest value is as simple as looking at the beginning of the list.
For the scale measurement data, the lowest value is
11. , The range of a set of data is the difference between the highest and lowest measurements.
Just subtract one from the other.
Algebraically, the range can be expressed as:
Range=x(max)−x(min){\displaystyle {\text{Range}}=x(max)-x(min)} For the sample data, the range is:
Range=x(max)−x(min)=14−11=3{\displaystyle {\text{Range}}=x(max)-x(min)=14-11=3} , When reporting data, it is important to let the readers know what you have measured.
Because there are different measures of precision, you should specify what you are reporting.
For this data, you would report Mean=12.4, Range=3, or simply that the Mean=12.4±3.The mean is not actually part of calculating the range or precision, but it is generally the primary calculation for reporting the measured value.
The mean is found by adding up the sum of the measured values and then dividing by the number of items in the group.
For this set of data, the mean is (11+13+12+14+12)/5=12.4. -
Step 3: Subtract the lowest value from the highest.
-
Step 4: Report the range as the precision.
Detailed Guide
It helps to begin by sorting your data in numerical order, from lowest to highest.
This will ensure that you do not miss any values.
Then select the value at the end of the list.
For example, suppose you are testing the precision of a scale, and you observe five measurements: 11, 13, 12, 14,
12.
After sorting, these values are listed as 11, 12, 12, 13,
14.
The highest measurement is
14.
Once your data has been sorted, finding the lowest value is as simple as looking at the beginning of the list.
For the scale measurement data, the lowest value is
11. , The range of a set of data is the difference between the highest and lowest measurements.
Just subtract one from the other.
Algebraically, the range can be expressed as:
Range=x(max)−x(min){\displaystyle {\text{Range}}=x(max)-x(min)} For the sample data, the range is:
Range=x(max)−x(min)=14−11=3{\displaystyle {\text{Range}}=x(max)-x(min)=14-11=3} , When reporting data, it is important to let the readers know what you have measured.
Because there are different measures of precision, you should specify what you are reporting.
For this data, you would report Mean=12.4, Range=3, or simply that the Mean=12.4±3.The mean is not actually part of calculating the range or precision, but it is generally the primary calculation for reporting the measured value.
The mean is found by adding up the sum of the measured values and then dividing by the number of items in the group.
For this set of data, the mean is (11+13+12+14+12)/5=12.4.
About the Author
Donna Wood
Creates helpful guides on hobbies to inspire and educate readers.
Rate This Guide
How helpful was this guide? Click to rate: