I've been experimenting with the dhRichClient3 thread classes, and so
far I have been impressed, but I have having a couple of problems.
First, let me explain my class:

1) There is a single Sub that takes a ByVal string parameter that is a
path to a PDF file.
2) This sub loops from1 to the page count of the PDF and shells out to
an open source EXE (mupdftool.exe) that converts rach page into a PPM
image file.
3) The sub then converts the PPM image file into a JPEG held in a byte
array (using freeimage.dll).
4) The sub raises an event with the Page number, Total Pages and JPEG
byte array as parameters.

The class works great when used directly in a test app, but I have
some issues when run as a thread using dhRichClient3.

1) If the JPG byte array that is returned in the PageCompleted event
has a ubound > 65066 bytes, the thread dies with no error message/
MethodFinished event being fired. Ubound <=65066 works as expected.
I've solved this problem by splitting the byte array into bands and
raising the event for each band instead of for the entire image. The
bands then get reassembled client-side. This solution is okay if
required (even though it adds some complexity and margin for error),
but I was wondering if it is the best way to go about it? Is there a
technical reason for the limit, or is it just for efficiency (or am I
just doing something wrong)?

2) When I call CancelExecution or close my app, there is a 99%+ chance
that I will get a Dr.Watson error message "<EXENAME> has generated
errors and will be closed by Windows (once or twice I have not gotten
this error). You will need to restart the program". My class has the
following event and function defined (which I understand is required
for cancelling out of long running subs):

Public Event CancelCheck(p_Cancel As Boolean)

Private Function CancelExecution() As Boolean
    RaiseEvent CancelCheck(CancelExecution)
End Function

I also cancel my long running loops when the cancel flag is raised. Is
there anything else that I need to have in my class to prevent this
problem? I can also post more code if required.

Thanks in advance for any help.

JPB,

I've been working with this wrapper myself for some weeks now with the
help of Olaf. When I started, it was suggested that I discuss this VB
wrapper in the microsoft.public.vb.databases newsgroup rather than
here. By doing so, Olaf has been very helpful in responding to
questions on the dhRichClient3 wrapper.

Since it is DB based, you may want to join us on that newsgroup for
dhRichClient3 discussions.

Just a suggestion of course.  :-)

Cheers!

Webbiz



On Sun, 7 Jun 2009 06:54:21 -0700 (PDT), JPB
<jasonpeterbr***@gmail.com> wrote:

Show quoteHide quote

Hi Webbiz,

Thanks a lot for the reply, I will take the discussion over to the
other group.



On Jun 8, 11:19 am, Webbiz <nos***@forme.thanks.com> wrote:
Show quoteHide quote

Oh, your welcome JPB.

My apologies for a typo.

The group is microsoft.public.vb.database and not 'databases' as I
typed earlier.  Ooops.

Cheers!

Webbiz


On Mon, 8 Jun 2009 08:58:31 -0700 (PDT), JPB
<jasonpeterbr***@gmail.com> wrote:

Show quoteHide quote

That's another "documentation-issue" as with all of my RichClient-
stuff currently, I assume <g>.

Yes, the maximum-size of all serialized parameters within an
Event, raised from the thread may not exceed 64kByte.
This resitriction is not there in case of sent Thread-Method-
*results* (which you can look at, as something like a
"final Event" from within a ThreadMethod). So the results
of "serverside" Thread-functions can be as large as your
system (your App) can handle it in case of return-values.

But in case of the raised Events from within such a method,
a crash (whilst exceeding that size) is of course not acceptable.
Just forgot about that limitation in the internal Pipe-Channel-
handling (for Event-Params) and have now builtin a check for that -
resulting then in a nice error-message at the clientside, in case
the things you stuff into the Event-Params are too large.

But before I put out a new BugFix-release of the RichClient,
let's first make sure, that your other problems also vanish.

Yep - that would have been my recommendation too.
64kByte-Bufs are large enough chunks, so that the Event-
protocol-overhead is negligible (timing- or performance-wise).

Yep, that's an efficiency-thing - I could enhance the Pipe-Buffer-
Size internally to e.g. 2MB or something like that, but that
affects the performance on that pipe as my tests have shown
(especially the requests-per second against the "server-thread"
when only smaller requests with not that voluminous parameter-
sets are done - it's a latency-thing).


The CancelExecution-command should work without any
crash, as long as it isn't called *within* a teardown that
is already "in-progress" (the terminating main-form for
example).
CancelExecution should be used, to just cancel the
actually running Job-(or JobQueue) - or to "prepare"
the teardown-process.

VB is a bit picky regarding the teardown of an App -
it does not expect any running threads, when it enters
that mode - so you are good advised, when you make
sure, that all threads are playing well with an App that
is about to enter its teardown.

The initiation for a thread-termination is a simple
Set ThreadHandlerObject = Nothing - either
explicitely - or implicit (when the hosting clientside
ThreadHandlerObject goes out of scope).

In the Class_Terminate of such a clientside ThreadHandler-
Instance is a builtin TimeOut, which delays the hard
termination of the "known remote-threadobject" per
TerminateThread-API for a while (with the goal to avoid it
completely).
Before such an hard TerminateThread-call (which could
be the cause for the crashes, depending on the internal
state of the remote-thread and its internal ThreadObjects
and "actions"), the clientside handler tries to close the
remote-thread gracefully and it waits as long as you've
specified in the .TimeOutSecondsToHardTerminate-
property.

A thread can only receive (and react to) that graceful-close-
request from the clientside, when it is in its Idle-state, or enters
its Idle-state within these TimeoutSeconds I just described above.
Idle-state means, that the RemoteThread-Class does *not*
perform any internal actions anymore (simply spoken,
it is not "within any Sub- or Function".

And regarding what I've stated above - that is the way
to ensure more "granularity" within your long-running
jobs.

At least once within a TerminateTimeoutSeconds-Interval
that already is "entered at the clientside" - you will have to
check within a long running method at the serverside:

Public Sub LongRunner()
do
    ProcessChunk() 'should not take longer than TimeOut
    If CancelExecution Then Exit Sub '<---that's important
loop
End Sub

Or in case the above (private) SubRoutine "ProcessChunk"
is also a longrunner in itself, then change to:

Public Sub LongRunner()
do
    If Not ProcessChunk Then Exit Sub 'exit immediately
    If CancelExecution Then Exit Sub '<---that's important
loop
End Sub

Private Function ProcessChunk() as Boolean
    do
        'process only smaller parts of your internal job and then...
        'from time to time...
        If CancelExecution Then Exit Function 'return False
    sub_loop
    ProcessChunk = True
End Function

Also ensure, that you cleanup your interal Helper-Objects
within the Class_Terminate of such a ThreadClass
appropriately.


If the problem persists on your side, it would be good,
if you could send me a smaller test-set which is able to
force these teardown-crashes here on my side too.

The E-Mail-address in my posting-header is capable to
receive replies.

Will see what I find out in that case - but try to "spread" that
If CancelExecution Then Exit ...
over your long-running code first - and also check for
DoEvents in your ThreadClass - that can sometimes
also be the cause for reentering stuff you think you have
cancelled.

Generally - my threadhelpers work best, when you use them
in a more granular way - they can handle up to 10000
thread-requests per second - and since the threadclass
keeps its state (the processed data) and is *not* restarted
over and over again (it behaves like a real server) - you
can also try to change the way you work currently against
it - much more granular - page-based - just split your
jobs more and play a bit more ping-pong with the main-
thread - it is not that "costly" as you may think.

If you know for sure (after splitting up your approach in
the way I just described), that the (remaining) functions
of your ThreadClasse, which are triggered from the clientside,
always are able to return within the TimeOut-Interval
(before a hard-termination can take place) - then you will
not even need to clutter your ServerClass-methods with
all these additional CancelExecution-checks.

Olaf

Not a problem, I'd rather have new and exciting code than
documentation any day :) Mostly because I do have some time to
experiment and learn hands-on (which is my preferred method in most
cases).

Noted, thank you. I figured it was the case (as I was bumping up
suspiciously close to 65k), but it is good to know for sure.
..
The error message will prove useful too, especially for anyone that
might experience the problem in the future.


I've found that by putting my CancelExecution checks *before* any
RaiseEvent calls in my thread sub, I can now call CancelExecution
client-side without a crash (so far it seems to be working anyway).
I'm not sure if it is because RaiseEvent is causing the crash after
CancelExecution has been called, or if it's just because I've put more
CancelExecution checks in that the crashing has stopped, but either
way it seems to work.

A question about CancelExecution - Do I have to use Exit Sub/Function,
or can I use Exit For in my loop (because I have some cleanup to do
before the function terminates)?

I'm pretty sure my thread is in and idle state when my form is closing
because I have put a Leaving event as the last line of the sub and it
does get passed to the clientside. I'm still experimenting here, so I
will get back to you when I have tried some more things to see if I
can get rid of the problem.

In my thread class, I call a few external modules that are created
with your regfree method the first time they are referenced, and held
in a static variable for subsequent references. I do not explicitly
set these references to Nothing. Something like:

Public Function libSomething() as CSomething
   Static so_Lib As CSomething

   If so_Lib Is Nothing Then
      set so_Lib = regfree.GetInstance("<Path to DLL", "CSomething")
   End If

   Set libSomething = so_Lib
End Function

Could this be part of the tear-down problem? Should I be creating all
references in Class_Initialize and clearing them all in
Class_Terminate?

I will also try splitting methods out into smaller chunks instead of
having one long running loop.

Show quote Hide quote Yep - it shouldn't just crash in that case... - that's a too
weird kind of "user-notification". ;-)

Yep, placing it before will ensure, that you don't put anything
anymore on to the pipe, which is "soon to be closed" at
the clientside in either case.

Good to know - will nonethless take another look, on what I
can do, that a clientside-intitiated "close-pipe" (due to a clientside
teardown) behaves more stable, even if it yet contains Event-
Param-stuff which was able to "slip-through" from the server-
side.

Will post here, when the new release is ready.

If that cleanup-stuff does not need "a few seconds", and
does not retrigger a new cascade of potential "long-runners"
then yes, sure.

Some notes, to shade some more light on all that...:

At the serverside there is just a hidden "Proxy-Class",
running on the same thread as your server-class -
this Proxy-Class already "knows" that the client has
initiated a teardown and is now in its wait-loop.

Your serverside CancelExecution-checks (within your
own-serverclass, that is hosted within the hidden proxy-
class) simply has to ensure, that your class-instance will
be able "to be set to nothing" by the proxy-class.
And that is only possible, if your server-class reaches
its "I'm not within any Sub/Function anymore"-state
as soon as possible (within the TimeOut-Interval).


Show quoteHide quoteNormally that should work - after all a static-var within
a function is treated the same as a private-var at class-
level - it automatically goes out of scope (and terminates)
when the hosting class goes out of scope too.

It can be a problem, when you need to ensure a certain
"terminate-order" over your helper-objects - then it
is better to handle the Set Nothing explicitely in your
Class_Terminate.
Another catch could be, when your helper-instances
behave somehow asynchronously themselfes (internally).
Always better then, to close - or cleanup them beforehand
(explicitely).

But for normal helperclass-instances, which have no DoEvents
or Timers inside, all that should not be necessary (no special
handling in class_terminate - not even a class_terminate itself
would be needed in your server-class).

Another "good-idea" (even generally) is, to make sure you
don't use DoEvents anywhere in your serverside classes or
private helper-classes.
This way you can be more sure about the real *state* of
your server-class at each "line-number".

HTH,

Olaf

Ok, the new release is online now, which contains the above fix.
Version 3.0.11 - download as usual:
www.thecommon.net/3.html

Also tried, to harden the teardown more - at the clientside
and at the serverside as well - with more blocker-flags - let's
see, if that proves helpful on your side with your "longrunners",
combined with the cancel-check.

Olaf

Hi Olaf, thank you for your prompt reply and work! I have tested the
new version with my modified project (more ping pong, less looping in
the thread class) and so far, everything seems to be working 100%
except for when I close my form - I still get the Dr. Watson crash.
I'm going through all of my code (including code in libraries that are
referenced in the thread class) to make sure I haven't left any
dangling references.

I'll also try to create a pared down test project ASAP to see if I can
demonstrate a minimal problem case.

A couple of questions:

1) Is it okay for any code in my thread class (or a referenced class
of my thread class) to use the New keyword to instantiate classes, or
should I always be using the regfree method of dhRichClient3?
2) How about for classes that are part of the same project as the
class that is instantiating them? For example, if a DLL has 2 classes,
can I just use New to create a reference to one class from the other,
or should I use regfree to get a reference to the class through the
compiled DLL?
3) If my thread class references a compiled library, and that compiled
library doesn't clean its references up properly, could that also be a
situation that causes a crash when the form unloads? For example, my
thread class references another DLL, which in turn references a third
DLL. Will I need to dig to the bottom of all of those DLLs to look for
a reference problem?

Thanks again for all of your help.

May I see the Unload-Events - and what you basically
are doing within these Events?

That would be nice - maybe whilst doing so, you will find
that maybe another lib, used within your thread-class(es)
is the culprit, maybe not suited for hosting within an STA.
In case everything you use in that thread-class is written
by yourself, please check that your AX-Dll-projects
always have the "ApartmentThreaded" switch set
(that is normally the default) - and maybe (but normally
not needed) just test with the unattended-execution-switch
to "on" - and also the "retain in memory"-switch to "on".
That is MS-recommendation for threadpooled hostage
within COM+ - maybe that helps on your side too, regarding
the teardown.


The regfree instancing is always optional - if you deliver your
solution with a setup (also registering the Thread-Dlls), then
you can safely use also the normal New-instancing (also
against dhRichClient3-classes, in case you register that lib).

In that case New will always work without looking at registry-
entries - as soon as you are "within a lib" (no matter how you
instantiated your class in question), then all classes are
known "lib-wide" and safe to use per 'New' (the private ones
of course, but also the public-multiuse-ones too).

Yes - in case your library in question is able, to somehow create
cycle-refs internally - that should be avoided.
Please make sure, that everything is able to terminate properly
(even when no explicite "Set nothings" are placed in the
Class_Terminate of a hosting thread-class).

Maybe look at the small Ping-Example I've posted in the
other thread - I've done nothing special regarding "Set
Nothing" there - not in the Thread-Class cThrd (which
internally makes use of a private Helper-Class cPing)
and also not in the Form_Unload at the clientside (no
special Set ThreadHelperClass = Nothing or something
like that).

I can close the whole App - either in the IDE or also
running as an Exe even while the Ping-Jobs are currently
running (on 32 threads).

Have done that of course repeatedly - not possible to bring
the whole thing down with a crash.

So, maybe just place a WriteLog in all your Class-Terminates,
to check, if all classes are running "over it" in your teardown.

A reference-problem should not be the cause - because
you would then get error-messages already whilst the
thread is running (in case "Sub-" functionality of Sub-Classes
is not reachable/callable).

But the deeper the "Lib-stack" the more difficult to find errors.
Especially cycle-ref-bugs (and sometimes also DoEvents-
loops, or other async-delays), which could be a good cause
for your crashes, because at the point the main-thread of a
VB-application exits, it does not expect other threads to
run anymore. If a certain thread is not able to be closed
before that point, then these crashes can happen (and in
case of a cycle-ref, created within hosted Sub-Libs of
a threadclass - this would prevent that worker-thread from
a clean termination, before the apps main-thread is also
being closed finally).

Olaf

Eureka! I finally found the problem. I ultimately resorted to
commenting out large swaths of code, and then uncommenting small
logical chunks and re-compiling until I got to the culprit. 4 DLLs and
a handful of classes down the chain from my client app,  I made a
single call to an InstrRev replacement class that I found at VBSpeed:

http://www.xbeat.net/vbspeed/c_InStrRev.htm#InStrRev08

I've switched that back to the standard VB6 InstrRev and now
everything is working 100% as expected. I guess the TLB and/or API
calls were causing problems with the thread classes in some way, but I
don't know why or how at this point. I'm just glad it's all working
now!

Thanks again Olaf, for all of your help and detailed responses to my
questions. It has all been invaluable. I hope the dhRichClient3 wiki
gets started soon so that I can contribute back in some way.

Whoah - seems you were busy for a while with that isolation-
efforts... ;-)

Regarding all these fast string-ops, whose principals I also
use in my dictionary-classes, I've had to remove the
direct usage of  the API-Call: SysAllocStringByteLen
for example (if used directly against function-result-
identifiers).
The only safe way I ended up as an (relative fast working)
replacement was a combination of:
StringIdentifier = Space$(LenOfResultString)
RtlMoveMemory(byval StrPtr(StringIdentifier ), ..., ...)

It all worked before of course with tha ole-API-call, but
it was causing instabilities then since around XP/SP2 IIRC.

Maybe they have changed something in the oleaut32.dll
where this API-Call is hosted, regarding "non-compatible"
heap-alloc-functions (compared to the heap-allocator
which is used within the VB-runtime - but that's only
speculation from my side). And this call was also causing
crashes in normal, singlethreaded Apps - though not that
often as I've seen them, when running for example the
RPC-calls in threaded mode.

So, the SysAllocStringByteLen-Call is now (at least for my
code) "considered evil".

Anyway, glad everything is working now on your side.

I will take your word...<g>

Olaf

VB6 on Vista Home Premium problem
Excel Execution from VB Fails on 2nd Attempt
Use an Addin to automatically add date/time stamp to each edited line of VB6 code?
Sub .... or Private Sub....
Moving .exe somtimes works
In High Density Mode - Looking for previous control counting post
How to create the project referencing library, which user may not have on his computer?
MS Access Query in VB6
listing all audio devices
fso.Drives replacement