Issue when converting Hex to Decimal

[Simon Knight]

I have code that reads portions of binary code from image files. The reading of data is done using the command

read from file

which requires a start position and a length. These two numbers have to be passed as decimal values. This is a problem as the start position and length are stored in hexadecimal which means that the hex values read from the files have to be converted to decimal before the data can be located and read.

To conduct the conversion I have the following function:

Code: Select all

Private function HexToDecimal pBytes,pIsLittleEndian
   /* pBytes is a comma seperated list of bytes read from file
   */
   
   ## Each loop starts with the units, followed by squared, followed by pwr 4, pwr 6 ...
   put the number of items in pBytes into tByteCount
   put 0 into tPwr
   Switch pIsLittleEndian
      Case true
         Repeat for each item tByte in pBytes
            put tByte into tInt
            add tInt*(16^tPwr) to tValue
            add 2 to tPwr
         end Repeat
         break
      Case false
         Repeat with N = tByteCount down to 1
            put (item N of pBytes) into tInt
            add tInt*(16^tPwr) to tValue
            add 2 to tPwr
         end Repeat
         break
   end Switch
   
   return tValue
end HexToDecimal

This works when the values are small i.e. when the stored pointer points to somewhere near the beginning of the file. Unfortunately when the decimal decode gets large, for example

0x1332F47 = decimal 20131655

livecode pulls a trick and saves the number as

2.01317e+07

which is not accurate enough.

I wonder if there is a way of stopping livecode from storing the number in scientific notation.

[richmond62]

-
That's happening because you are being too clever . . .

. . . borrow my spade and start getting down and dirty:

[Simon Knight]

That's happening because you are being too clever

Ha ha thats funny - too stupid more like.

Excellent stack thanks. I have no idea why I missed baseconvert but I'm off digging with your spade. TVM

[Simon Knight]

Here is my solution :

Code: Select all

Private function HexToDecimal pBytes,pIsLittleEndian
   /* pBytes is a comma separated list of bytes read from file
   Note these bytes are decimal values
   e.g. 41,46,51,1
   
   pIsLittleEndian is boolean
   */
   
   ## Convert the bytes back to hex, noting byte order.
   put the number of items of pBytes into tByteCount
   If pIsLittleEndian then
      Repeat with n= tByteCount to 1 step -1
         put item n of pBytes into tByte
         put baseconvert(tByte,10,16)  into tHex
         if length(tHex) is 1 then put 0 before tHex
         put tHex after tHexBytes
      end Repeat
   else
      Repeat with n=1 to  tByteCount
         put item n of pBytes into tByte
         put baseconvert(tByte,10,16)  after tHex
         if length(tHex) is 1 then put 0 before tHex
         put tHex after tHexBytes
      end Repeat
   end if
   
   put baseconvert(tHexBytes,16,10)  after tDecDecode
   return tDecDecode
   
end HexToDecimal

Many thanks for the hint.
S

[Simon Knight]

Refactored:

Code: Select all

Private function HexToDecimal pBytes,pIsLittleEndian
   /* pBytes is a comma seperated list of bytes read from file
   Note these bytes are decimal values
   e.g. 41,46,51,1
   pIsLittleEndian is boolean
   */
   
   ## Convert the bytes back to hex, noting byte order.
   put the number of items of pBytes into tByteCount
   
   Repeat with n=1 to  tByteCount
      put item n of pBytes into tByte
      put baseconvert(tByte,10,16)  into tHex
      if length(tHex) is 1 then put 0 before tHex
      If pIsLittleEndian then
         put tHex before tHexBytes
      else
         put tHex after tHexBytes
      end if
   end Repeat
   
   put baseconvert(tHexBytes,16,10)  after tDecDecode
   return tDecDecode
   
end HexToDecimal