Re: Strange issue when converting from hex to float
Re: Strange issue when converting from hex to float
- Subject: Re: Strange issue when converting from hex to float
- From: Ian was here <email@hidden>
- Date: Fri, 22 Jul 2005 08:36:35 -0700 (PDT)
Here's a copy of the offending method...thanks for
offering to take a look.
Ian
--- Greg <email@hidden> wrote:
> Can you post the whole method so I can quickly run a
> test program.
>
> On 22/07/2005, at 1:42 PM, Ian was here wrote:
>
> > I've been using the debugger to step through my
> code
> > each time. I have verified that I am getting an
> > accurate decimal number from the hex conversion.
> It's
> > just the division that's turning it to zero. I've
> done
> > the division using a calculator, and get the
> desired
> > results.
> >
> >
> >
> > --- John Stiles <email@hidden> wrote:
> >
> >
> >> Any of these combinations should be sufficient.
> >> What happens when you step through it in the
> >> debugger?
> >>
> >>
> >> Ian was here wrote:
> >>
> >>
> >>> I've tried several combos and they all yield a
> >>>
> >> value
> >>
> >>> of zero.
> >>>
> >>> float fAlpha = alphaComp / 255.0f;
> >>>
> >>> float fAlpha = (float)alphaComp
> /
> >>> 255.0f;
> >>>
> >>> float fAlpha = alphaComp /
> 255.0;
> >>>
> >>>
> >>> Doesn't seem to want to do the division??
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --- Dan Treiman <email@hidden> wrote:
> >>>
> >>>
> >>>
> >>>
> >>>> Dividing an long by long integer returns a
> long,
> >>>> thats the way the C
> >>>> compiler does it. And integer division
> truncates
> >>>> the result to an
> >>>> integer.
> >>>> Try casting one of your arguments as a float,
> that
> >>>> will force the
> >>>> compiler to generate floating point division
> >>>>
> >>>> i.e.
> >>>> float fAlpha = alphaComp / 255.0f;
> >>>>
> >>>> -DT.
> >>>>
> >>>> On Jul 21, 2005, at 11:54 AM, Ian was here
> wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>> I have written a method that converts hex
> values
> >>>>>
> >>>>>
> >>>>>
> >>>> to
> >>>>
> >>>>
> >>>>
> >>>>> floating point numbers. I am having a really
> >>>>>
> >>>>>
> >>>>>
> >>>> strange
> >>>>
> >>>>
> >>>>
> >>>>> problem occur in the conversion process.
> >>>>>
> >>>>>
> >>>>>
> >>>>> // Convert each hex segment to decimal. This
> code
> >>>>> works fine!
> >>>>>
> >>>>> long alphaComp = strtol(
> alphaValue, NULL, 16
> >>>>>
> >> );
> >>
> >>>>> long redComp = strtol( redValue,
> NULL, 16 );
> >>>>> long greenComp = strtol(
> greenValue, NULL, 16
> >>>>>
> >> );
> >>
> >>>>> long blueComp = strtol( blueValue,
> NULL, 16 );
> >>>>>
> >>>>>
> >>>>>
> >>>>> The following code segment always returns
> >>>>>
> >> zero!!??
> >>
> >>>>>
> >>>>>
> >>>>>
> >>>>> // Convert each decimal number to float.
> >>>>>
> >>>>> float fAlpha = alphaComp / 255L;
> >>>>> float fRed = alphaComp / 255L;
> >>>>> float fGreen = alphaComp / 255L;
> >>>>> float fBlue = alphaComp / 255L;
> >>>>>
> >>>>>
> >>>>> Any ideas would be greatly appreciated. Thank
> >>>>>
> >> you.
> >>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>
> >>
> >>>
> ____________________________________________________
> >>>
> >>>>
> >>>>
> >>>>
> >>>>> Start your day with Yahoo! - make it your home
> >>>>>
> >>>>>
> >>>>>
> >>>> page
> >>>>
> >>>>
> >>>>
> >>>>> http://www.yahoo.com/r/hs
> >>>>>
> >>>>>
> _______________________________________________
> >>>>> Do not post admin requests to the list. They
> will
> >>>>>
> >>>>>
> >>>>>
> >>>> be ignored.
> >>>>
> >>>>
> >>>>
> >>>>> Cocoa-dev mailing list
> >>>>>
> >>>>>
> >>>>>
> >>>> (email@hidden)
> >>>>
> >>>>
> >>>>
> >>>>> Help/Unsubscribe/Update your Subscription:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>
> >>
>
> >>
> >>>
> >>>
> >>>
> >>>>> This email sent to email@hidden
> >>>>>
> >>>>>
> >>>>>
>
=== message truncated ===
____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs
Attachment:
convertHextToNSColor.sitx
Description: 1065542667-convertHextToNSColor.sitx
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden