Greetings!

Can someone explain this conversion routine in a language an 8-yr old
can understand?

Public Function MReal2Single(M() As Byte) As Single
Static IEEE_Exp As Byte, IEEE(0 To 3) As Byte
  If M(3) = 0 Then Exit Function

  IEEE_Exp = M(3) - 2
  IEEE(3) = (M(2) And &H80) Or (IEEE_Exp \ 2)
  IEEE(2) = ((CLng(IEEE_Exp) * 128) And &HFF) Or (M(2) And &H7F)
  IEEE(1) = M(1)
  IEEE(0) = M(0)
  RtlMoveMemory MReal2Single, IEEE(0), 4
End Function

Just trying to get my head around this.

From what I have been able to gleen from the net, MBF is an old format
for dealing with floating point values?

So if I want to use a value that is in MBF (such as a Date), it has to
first be converted to IEEE format using something like the function
above, right?

What exactly is this function doing and why?

1. If M(3) = 0 then Exit Function

Why? What does it mean if it contains a zero? That it does not have an
"Exponential" and should (required)? Is this a value that dictates
what is to be done with the radix (ie: 5e3 where 3 is the value to be
found in M(3)).

2. IEEE(3) = (M(2) And &H80) Or (IEEE_Exp \ 2)?

What is this doing?


Perhaps there is an easy to follow write-up already avaiable somewhere
that someone here can direct me to. While I do not need to know how
this works but only that it does, I'm really interested in the inner
workings.

Appreciate any help.

Thanks in advance!

Webbiz

Webbiz wrote:
It's shifting the bits around in a single-precision float, to convert
it from MBF to IEEE format. That's easy enough to do because the two
types have the same basic precision and range. Doubles are a different
story -- not quite so easy.

You can pick up a tiny free DLL from our web site to do conversions in
both directions on floats and doubles. Written in ASM, and much
faster -- which matters only if you're doing a lot of them.

http://www.microdexterity.com/demos/mbfiee32.zip

Show quoteHide quote

Thanks for the link Jim.

I actually have a copy of this from years ago.

I'm still not clear on this.

The information is contained in an array holding 4 bytes. I believe
this to be in MBF format.

Currently, I have this code that the array is passed to and converts
this 4 byte array into a SINGLE type value.

Public Function MReal2Single(M() As Byte) As Single
Static IEEE_Exp As Byte, IEEE(0 To 3) As Byte
  If M(3) = 0 Then Exit Function

  IEEE_Exp = M(3) - 2
  IEEE(3) = (M(2) And &H80) Or (IEEE_Exp \ 2)
  IEEE(2) = ((CLng(IEEE_Exp) * 128) And &HFF) Or (M(2) And &H7F)
  IEEE(1) = M(1)
  IEEE(0) = M(0)
  RtlMoveMemory MReal2Single, IEEE(0), 4
End Function

So my new question is this: Is the function DxToIEEEs suppose to do
the same job as the above routine? DxToIEEEs routine appears to
require a SINGLE passed to it, not an array of 4 bytes.

Where am I going wrong in my thinking?

Thanks.

Webbiz






Show quoteHide quote

Webbiz wrote:
Show quoteHide quoteHow do you know anything is going wrong? Is the above function not
working for you?

View this in a fixed-pitch font for clarity:

The layout for MBF Singles is:

  m(3)      m(2)    m(1)     m(0)
eeeeeeee smmmmmmm mmmmmmmm mmmmmmmm
   `--- 8 bit biased exponent.
        If this byte is zero, then the number is zero.

The exponent is biased, that is, the real exponent is these eeeeee's
minus 0x83

The bit layout for IEEE Singles is:

seeeeeee emmmmmmm mmmmmmmm mmmmmmmm
`- -- sign bit, 1 = negative

The IEEE exponent has a bias of 0x81 (going from memory -- actual
values not important)


If the exponent is 0, the value is 0. Unlike IEEE, MBF does not
support denormals -- an IEEE value is zero only if all its bits are
zero.

Re-range the exponent bias from 131 to 129  '** or similar

Shift the exponent right one bit, and move the sign bit from the
second byte

This completes the one-bit shift right of the exponent into the second
byte, and merges the MSB of the mantissa.

Transfer the remainder of the mantissa to the new value, in place.

The DxToIEEEs function can take a byte array instead of a single if
you use a Declare to avoid the typelib. The included CVS() example
uses a string, but the same idea works for a byte array.

Hey Jim.

Oh, the function works great. Didn't mean to imply there was a
problem. I'm only trying to wrap my mind around what is going on with
this code.

I have code pieces found on the internet or kindly provided by folks
like you here on the newsgroup that I'll use to deal with some issue I
may have with a programming project. Yet, often I don't have a lot of
time to dig deep and see 'why' they solved the problem or 'what' they
actually do.

So what ends up happening is that my progress from novice is slow and
long. I simply want to start understanding more 'deeper' things so
that I can take all the ideas I have in my head and turn them into
great code without so much bloat.

But yes, the code I'm studying right now was written by Olaf and it is
working wonderfully.  :-)

I don't understand the above. What do you mean by "if you use a
Declare to avoid the typelib."? What would be an example of how the
DxToIEEEs can accept an array(0 to 3) of byte and return an IEEE
single?

Thanks for the bit level explanation!

Cheers!
Webbiz

Webbiz wrote:
It won't be as fast as the in-place version, but it's straightforward.

If there's any way you can get your MBF floats into Singles to start
with, it will be faster to convert them directly using DxToIEEEs.

The Declare overrides the Typelib, which would ordinarily enforce
typing. The following is air code, but worth a try. (-:


  '--------------------

Private Declare Sub BaToIEEEs Alias "DxToIEEEs" _
    Lib "mbfiee32" (ByRef X As Byte)

' If needed ...
'
' Private Declare Sub CopyMemory Lib "kernel32" _
'  Alias "RtlMoveMemory" _
'  (ToVar As Any, FromVar As Any, ByVal cbLen As Long)


Function BaMBFs(Mbf() As Byte) As Single

   BaToIEEEs Mbf(0)        ' changes the input array !
   CopyMemory BaMBFs, Mbf(0), 4

End Function 'BaMBFs

  '--------------------

Let me know how it goes.

">Let me know how it goes."

I would if I could, but the info you provided is a bit above my pay
grade. :-|  Think 8th grade level of understanding. Sorry.


"in-place" - What does that mean?

The code I had posted earlier. Doesn't that put MBF floats into
singles (already converted to IEEE I believe)?

Public Function MReal2Single(M() As Byte) As Single
Static IEEE_Exp As Byte, IEEE(0 To 3) As Byte
  If M(3) = 0 Then Exit Function

  IEEE_Exp = M(3) - 2
  IEEE(3) = (M(2) And &H80) Or (IEEE_Exp \ 2)
  IEEE(2) = ((CLng(IEEE_Exp) * 128) And &HFF) Or (M(2) And &H7F)
  IEEE(1) = M(1)
  IEEE(0) = M(0)
  RtlMoveMemory MReal2Single, IEEE(0), 4
End Function

The only MBF info I've personally come across is found in arrays of 4
bytes. This is dealing with Metastock formatted data as they use MBF.

I have to assume then that IF the above code does convert from MBF
array to IEEE single that there would be no advantage to DxToIEEEs in
this scenario. I have to assume that there would be no speed gain from
using DxToIEEEs over the above since code would have to be written to
convert the MBF float to single anyway to feed into DxToIEEEs. That
seems like more coding than the above.

And if the above assumptions are correct, then I have to think that
DxToIEEEs was written for MBF that is already in a variable as single.
So I guess MBF singles exist elsewhere but simply not in the data
source I happen to be working on.

What you are saying then is that the Declare statement above is
letting you change the type that can be passed to the subroutine,
right? So by delaring a Sub BaToIEEEs with a parameter of X as Byte,
you allow DxToIEEEs to work on an array of Byte as opposed to a
single?  How is it that the original SUB DxToIEEEs would know how to
work on this array of bytes when it was originally compiled to work on
singles? Of it this because you are dealing with the same number of
bytes in a single as you are with the array?

As you can see, I'm not experienced in these 'override' scenarios.

Show quoteHide quoteThanks Jim.  :-)

Webbiz



Show quoteHide quote

You can even work in the faster typelib-mode of
Jims Dll IMO (without any Declare-based
redefinition), if you change all these Type-Defs
(which currently contain these 0-3 ByteArrays)
into Singles (which also take 4Bytes and should
therefore not cause the slightest problem in the
parts which read these re-typed arrays from files).

Then of course also adjust the code a bit, since
Jims fast routine is a Sub as I understand it - I would
not wrap it in just another function, to not lose speed -
you will only have to make sure, that you call the
Typelib-defined Sub against each Single in all your
Typed Arrays (exactly once).
This should convert the "wrong" Single-Content
(4ByteMBF) into the right one (4ByteIEEE).

Olaf

Webbiz wrote:
The function BaToIEEEs would be a direct drop-in replacement for the
function MReal2Single. It requires only 7th-grade understanding. (-:


The Sub DxToIEEEs works directly on MBF singles in place, no copying.


I'm saying, if you can get the input variables directly as MBF singles
instead of as byte arrays, you would have the fastest method, which is
to apply DxToIEEEs. No copying, no shifting, just direct conversion. I
don't know how the MetaStock data got into byte arrays, but if it
could get into singles instead, you'd get best performance.


Yes.


I think that's up to you. Somehow, a byte array is filled with data
off disk. Fill a Single instead.

Are these byte arrays in a UDT, for example? If so, change them to As
Single and that should do it.


Show quoteHide quoteThe ASM code doesn't know anything about what it's operating on, it
just sees memory. The Typelib tells the _VB compiler_ that it should
enforce type safety and pass only Singles. The Declare just tells it
something else, namely, to pass a byte array.


Yes, exactly. That's how your original code worked as well.

FWIW, DxToIEEEs is part of Stamina that we carved out for free use.
Stamina also has DxArrayMBF, which will operate directly on one or
more fields of an array of a UDT that has MBF content. If you have a
flat set of records with MBF fields, this would convert them all in
one very fast call. It was suggested by a user who manipulates a lot
of stock data, as I recall. (-:

Jim Mack wrote:
Sorry, it's the function BaMBFs that would drop in.

Ah. Got it. Much like ByRef I assume?

The Metastock file MASTER apparently contains the stock header
information in the following format:

Public Type TMetaStock
  ID As Byte
  Reserved As Integer
  Rec_Len As Byte
  Num_Fields As Byte
  Reserved1 As Integer
  File_Name As String * 16
  Reserved2 As Byte
  Flag As Byte
  Init_Date(0 To 3) As Byte
  Fin_Date(0 To 3) As Byte
  Time As Byte
  Intr_Time As Byte
  Reserved3 As Byte
  Symbol As String * 14
  Reserved4 As Byte
  Autorun_Flag As Byte
  Reserved5 As Byte
End Type

The dates here...

  Init_Date(0 To 3) As Byte
  Fin_Date(0 To 3) As Byte

....is the information that is being retrieved as arrays of 4 bytes in
MBF format.

So what you are saying then is to load this data directly into Single
variables like this...

  Init_Date As Single
  Fin_Date As Single

We'd bypass the need to convert the array of 4 bytes from MBF to IEEE
and then into a Single and just go directly to a converted IEEE single
using the DxToIEEEs?

DxToIEEEs Init_Date

I'm not sure if the array of bytes is a UDT as you are asking, but a
search on the web I found the following format for the MASTER file, as
follows:

/*
* MASTER file description
*  floats are in Microsoft Basic format
*  strings are padded with spaces, not null terminated
*/
struct rec_1 {
    u_short num_files;        /* number of files master contains */
    u_short file_num;        /* next file number to use (highest F#
used) */
    char zeroes[49];
};
struct rec_2to255 {        /* description of data files */
    u_char file_num;        /* file #, i.e., F# */
    char file_type[2];        /* CT file type = 0'e' (5 or 7 flds) */
    u_char rec_len;        /* record length in bytes (4 x num_fields)
*/
    u_char num_fields;        /* number of 4-byte fields in each record
*/
    char reserved1[2];        /*  in the data file */
    char issue_name[16];    /* stock name */
    char reserved2;
    char CT_v2_8_flag;        /* if CT ver. 2.8, 'Y'; o.w., anything
else */
    float first_date;        /* yymmdd */ 
    float last_date;                                  
    char time_frame;        /* data format:
'I'(IDA)/'W'/'Q'/'D'/'M'/'Y' */
    u_short ida_time;        /* <b>intraday</b> (IDA) time base */
    char symbol[14];        /* stock symbol */
    char reserved3;        /* <b>MetaStock</b> reserved2: must be a
space */
    char flag;            /* ' ' or '*' for autorun */
    char reserved4;
};


Looking at the above code, it would seem to me that the decision to
load the data as an array of 4 bytes or as a 'Single' (same as Float,
right?) is up to the programmer. Correct?

    float first_date;        /* yymmdd */ 
    float last_date;                                  


Sorry for the VB 101 questions, but remember 7th grader here. :-b

Show quoteHide quoteGot it!

I'm going to assume that, if the issue here is of simply changing my
VB structure to hold singles rather than array of 4 bytes as it
currently does, that I really don't need to concern myself with
DxArrayMBF. And other than the routine that I'm currently using to
convert an array of 4 bytes of MBF into IEEE, I have no other use for
having an array of 4 bytes that I can see.


So assuming you agree with the above, it really comes down to personal
choice which way to go. And if that is the case, which would provide,
in your opinion, an advantage or edge or better way to go?

Thanks!

Webbiz

Webbiz wrote:
Show quoteHide quoteYes, replacing the byte arrays with Singles is the idea.

However, Init_Date doesn't exist independently of the containing Type,
so you'd be doing something like:

   DxToIEEEs MyVariable.Init_Date


   ... snip ...


Yes, exactly. How you view a set of bits is up to you (within reason,
and some limits imposed by the language).


Not exactly. The "array" in the name refers to an array of the Master
Type, not an array of bytes.

If you have one single variable of the MetaStock Master Type, then
DxArrayMBF doesn't offer any advantage. If you have an array of such
types, for example, read directly from a file, DxArrayMBF can convert
ALL the singles in ALL the members of that Type array in one call. For
a largish file it could be tens of times faster than doing each entry
individually.

But in no case do you "need" DxArrayMBF -- you can do what you need to
using the free version.

Show quote Hide quote Let me see if I understand this.

What you are saying is that an array of the above Metastock structure
passed to DxArrayMBF would result in EVERY Single type variable being
coverted from MBF to IEEE?

The Master file can hold no more than 254 records, each representing
one stock. Would you consider 254 (assuming it was maxed out) to be a
significant number of array(items) or small stuff for something like
DxArrayMBF?

You mention 'free version'. I assume this means DxArrayMBF is a
royalty product?

Thanks.

Webbiz

Webbiz wrote:
Yes. You pass in a key descriptor telling the function where the
floats are located in the UDT, and the function converts them in
place.


It's not just the number you do in one gulp, but how many gulps over
how much time. 254 isn't a huge number, but I don't know what else
your software is doing, or how often. If the conversion is within a
real-time tracking process, then it's probably worth it. Otherwise,
probably not. But it's your program...


It's part of our Stamina library, which is a commercial product having
some 500 functions written in ASM for use by VB and VBA. DxToIEEEs is
also in the library, but we long ago decided that the community could
use a good 32-bit version of the widely used 16-bit MBFIEE.DLL, so we
put those few functions into our free MBFIEE32.DLL

We should probably let the commercial talk drop here -- I really just
wanted to mention it for completeness.

Show quote Hide quote Read this article:

PRB: Converting MBF to IEEE in Visual Basic for Windows
http://support.microsoft.com/kb/140520/EN-US/

http://en.wikipedia.org/wiki/Single_precision

Show quote Hide quote The above was just a port of some C-Code I've
googled (don't find that site anymore, but it was
in the context of MetaStock-conversion-routines,
so I don't questioned myself all that much about
especially that special line which returns with zero
in case M(3) is zero too.
If M(3) = 0 Then Exit Function 'returns with a Null-Single then

Yeah, that looks somewhat weird (along the lines ... are the
other 24Bits really of no value, just in case the exponent-bits
are at zero?).

But as said, since that was found in the context of MetaStock-
data-conversion I then thought - Ok, maybe in that context
the possible data-range for stocks-data is somehow "reduced"
or  "well-defined" - no problem. The tests were Ok too.

Now whilst trying to find that C-Source again, I stumbled
over that nice write-up below, and it seems this weird line
above makes sense not only in MetaStock-conversion-
contexts of MBF-encoded data.
http://www.boyet.com/Articles/MBFSinglePrecision.html

So, (also from Jims last reply) I think now that my
ported routine is "safe to use" also generally
(with some minor cons, which are mentioned in
the link above).
Though - as Jim already said in his first post - somewhat
slower working (especially in PCode) than alternative
binaries which were build with compilers who support
Bit-Shifting-Ops natively.

Olaf

Glad to see that I'm not the only one that 'borrows' code snippets.
Exactly. I thought that was odd as well. Jim did a good job in
explaining this about MBF. While IEEE would consider all bits off to
equal zero, it would appear that MBF need only the exponential to be
zero to be a non value.

I actually stumbled on this page as well as many others the other day.
Just so you know, I spend a lot of time doing internet searches on
questions I have BEFORE I'll give up and ask here. What I have found,
however, is that having someone 'explain' it differently often helps.

When Jim mentioned "faster", my ears perked up and that's why I was
wondering if plugging in the DxtoIEEEs in place of some of my existing
code would be a benefit. As you know, I'm dealing with
hundreds/thousands of records having to be read and stored into DB. If
I can shave off a micro-second with less code, why not? Also, I just
want to know 'why'. 

Thanks Olaf!

Webbiz


Show quoteHide quote

write a bitmap into an array of picture boxes
VSFlexGrid 8 for VB 6
Contemplating the Migration
Newsgroup Access
A quote for Karl
Free or low cost PDF viewer ocx?
Toolbar Dilema
Sending email without client
Where are the VB6 Reference entries kept?
Comdlg32 Not Properly Registerred on Vista Box