As you reduce from six digits to five, the answer is rounded from six digits
to five. This continues all the way down to one digit of accuracy. This
behavior is characteristic of a perfectly stable computation: the length
of the floating point numbers affects the number of digits that are produced,
but doesn't affect what those digits are. Contrast this behavior with that of
our original formula.
Return to lesson.
Joseph L. Zachary
Department of Computer Science
University of Utah