That is to be expected. Floating point computation, unlike
mathematically
precise arithmetic, is sensitive to the order in which values are
summed
up. However, the precision can only improve when you introduce more
partial sum variables. That's because the magnitude of each individual
accumulator is smaller, hence there is more overlap of mantissa bits
between accumulator and each summand. (Assuming you add only positive
values.)
That isn't true all the time. To get the best precision you'll have
to sort them according to magnitude and sum from least to greatest
magnitude. That sort of operation order could have its precision
harmed by breaking into partial sums, depending on how that was done.
In this case, operation order is vitally important to getting precise
answers. In typical code, which doesn't bother with sorting,
reordering into multiple accumulators can harm or improve your
precision. It is difficult to say which will happen without looking
at the actual list of numbers.
vecLib doesn't bother with sorting. Libm, however, frequently takes
advantage of operation order to deliver results correct to half an
ulp or so. In most cases, we arrange the algorithm so that we know at
compile time which numbers are larger than the others, eliminating
the sorting problem. Libm typically only has to deal with a sum of
three or four numbers, so this is much easier to do.
You can also keep an extended precision head-to-tail representation
around in certain cases where you know which of two addends has a
greater magnitude:
float a = big_number;
float b = small_number;
float sumHead = a + b;
float sumTail = b - ( sumHead - a );
Likewise on machines with MAF units, you can calculate infinitely
precise products using a head to tail representation:
float prodHead = a * b;
float prodTail = a * b - prodHead;
This can be handy in cases where small segments of your algorithm
need more precision than the hardware supports.
Sometimes the relative error is rather high, like 11% or so.
That does sound like way too much.
This can happen when accumulating mixtures of positive and negative
numbers. Certain subtractions of very similar numbers may cause a
loss of nearly all your precision. (The precision was technically
lost in earlier roundings.) All and all, I'd say it's probably more
efficient to use double precision accumulators than it is to spend
time sorting inputs, however. The precision (and range!) of the
product is completely covered by a double and more of the additions
will be infinitely precise.
Note that double precision can still suffer from the same sorts of
catastrophic cancellation problems. However, it tends to happen much
less often. There do remain cases, particularly with complex
arithmetic, where this sort of problem can happen frequently.
