When I solve for [1/n-1] at beginning of the variance formula my solution is [.1111] when my calculator is set to 4 decimal places. If I were to set the calculator to 8 decimal places my solution would be [.11111111]. With each additional .1 my Sxy X2 Y2 answers slightly change.

What decimal place would you recommend I program my calculator to compute the statistics at? The solutions rounding to four decimal places are seen below:

When I compute the covariance following the exact same process I end up with Sxy= 23.0421
Sx2=7.0660
Sy2=92.4463
Thanks for your help.

Your course directors typically ask for final answers to be reported to four decimal places. Itâ€™s a good idea to keep track of an additional decimal place for your rough work, so set your calculator to five decimal places.

Please note: Although they are expecting your answers reported to four decimal places, you are not expected to match the expected answer exactly. They use answer keys with more than enough room for the kinds of errors that will occur due to rounding at either four or five decimal places.

Hi Jason

When I solve for [1/n-1] at beginning of the variance formula my solution is [.1111] when my calculator is set to 4 decimal places. If I were to set the calculator to 8 decimal places my solution would be [.11111111]. With each additional .1 my Sxy X2 Y2 answers slightly change.

What decimal place would you recommend I program my calculator to compute the statistics at? The solutions rounding to four decimal places are seen below:

When I compute the covariance following the exact same process I end up with Sxy= 23.0421

Sx2=7.0660

Sy2=92.4463

Thanks for your help.

Your course directors typically ask for final answers to be reported to four decimal places. Itâ€™s a good idea to keep track of an additional decimal place for your rough work, so set your calculator to five decimal places.

Please note: Although they are expecting your answers reported to four decimal places, you are not expected to match the expected answer exactly. They use answer keys with more than enough room for the kinds of errors that will occur due to rounding at either four or five decimal places.