Showing posts with label processing. Show all posts
Showing posts with label processing. Show all posts

Thursday, March 22, 2012

Thread Not Processing?

Hi all.
Hopefully someone will have run into this problem and will be able to
shine soem light on it for me.
Right. I have a class that is being called from an ASPX page which gets
all of its properties set and then calls a function which in effect
means it has to process quite a lot of files [essentially doing finds
on some xml documents and then spitting other files out].
As a stand alone method being called from ASP.NET there is no problem
at all, it runs perfectly, processing all the files and delivering the
right results so there are no problems with my class functionally for
what I'm trying to do.
The problem comes when we have thousands of files to process and the
whole thing slows down - aha says I, I'll run the method in a different
thread in the background, give the user a natty processing screen and
they will know what is going on.
for some reason I can't get the whole thing to process though.
I call the thread and kick off the process and that works fine, but in
the foreach loop I use to process all the files it just stalls after
the first process, seemingly waiting for something or the thread has
been suspended...
_filesProcessed++;
lock (System.Web.HttpContext.Current.Session) {
System.Web.HttpContext.Current.Session["ProcessCompleted"] = false;
System.Web.HttpContext.Current.Session["CurrentlyProcessing"] =
filename;
System.Web.HttpContext.Current.Session["PercentageComplete"] =
_filesProcessed.ToString();
}
Weirdly this is the bit it seems to get stuck on as absolutely
everything up to this point is executing. Could the lock be waiting for
the whole session object to become free? I had it executing without the
lock before and it did the same thing.
Any ideas why this won't work in a thread would be very much welcome.
Cheers
AndrewFHi AndrewF:

>_filesProcessed++;
>lock (System.Web.HttpContext.Current.Session) {
> System.Web.HttpContext.Current.Session["ProcessCompleted"] = false;
> System.Web.HttpContext.Current.Session["CurrentlyProcessing"] =
>filename;
> System.Web.HttpContext.Current.Session["PercentageComplete"] =
>_filesProcessed.ToString();
>}
>Weirdly this is the bit it seems to get stuck on as absolutely
>everything up to this point is executing. Could the lock be waiting for
>the whole session object to become free? I had it executing without the
>lock before and it did the same thing.
>
In a worker thread the HttpContext.Current property is going to be
null. The thread is not associated with a request. I'm 99% sure your
worker thread is now throwing a NullReferenceException and is silently
dying.
I have a description of the problem in the following article:
http://www.odetocode.com/Articles/112.aspx
You'll have to package up the data required by the worker thread and
pass it along so that the worker thread won't need to access a
HttpContext.
Scott
http://www.OdeToCode.com/blogs/scott/
Hi Scott,
Thanks for that... In the end this is what I had to do and pass data
back and forth that way. To be honest this will work better anyway as
it means the classes aren't specific to ASP.NET and I can use them in a
windows forms project as well.
Cheers for the explanation as well - really useful.
Cheers
AndrewF

Thread object vs Threadpool?

Hi all!
In my webb app I'm about to introduce a way of processing long running tasks
in the background.
As the long running task gathers data from a database process them and
compile them into a file appx 80 mbytes large it is a resource demanding
task both of the CPU and the memory.
So I think I should restrict how many of these tasks that should be able to
start and make them run in the background.
First I read about the Thread class and thougth: Fine I'll use a thread and
then I'll write a utility class that will queue the tasks and make them run
either one after one or not too many at a time.
I'd set threads' priority to belowNormal or lowest.
Then I discovered the threadpool class and thought: Hey here's the queuing
abillity I was looking for. But then I found no way of setting the priority
of the thread that would start the long running task and I don't want to
make the web pages serve slowly.
Does anyone have any advice on whether I should use the threadpool or create
thread objects "on my own" in the code?
Thanks in advance!
mortbThreadPool class is mostly used for fire and forget tasks.
Use Thread class if extended control is needed.
-
Milosz Skalecki
MCP, MCAD
"mortb" wrote:

> Hi all!
> In my webb app I'm about to introduce a way of processing long running tas
ks
> in the background.
> As the long running task gathers data from a database process them and
> compile them into a file appx 80 mbytes large it is a resource demanding
> task both of the CPU and the memory.
> So I think I should restrict how many of these tasks that should be able t
o
> start and make them run in the background.
> First I read about the Thread class and thougth: Fine I'll use a thread an
d
> then I'll write a utility class that will queue the tasks and make them ru
n
> either one after one or not too many at a time.
> I'd set threads' priority to belowNormal or lowest.
> Then I discovered the threadpool class and thought: Hey here's the queuing
> abillity I was looking for. But then I found no way of setting the priorit
y
> of the thread that would start the long running task and I don't want to
> make the web pages serve slowly.
> Does anyone have any advice on whether I should use the threadpool or crea
te
> thread objects "on my own" in the code?
> Thanks in advance!
> mortb
>
>

Tuesday, March 13, 2012

Thread object vs Threadpool?

Hi all!

In my webb app I'm about to introduce a way of processing long running tasks
in the background.
As the long running task gathers data from a database process them and
compile them into a file appx 80 mbytes large it is a resource demanding
task both of the CPU and the memory.
So I think I should restrict how many of these tasks that should be able to
start and make them run in the background.

First I read about the Thread class and thougth: Fine I'll use a thread and
then I'll write a utility class that will queue the tasks and make them run
either one after one or not too many at a time.
I'd set threads' priority to belowNormal or lowest.

Then I discovered the threadpool class and thought: Hey here's the queuing
abillity I was looking for. But then I found no way of setting the priority
of the thread that would start the long running task and I don't want to
make the web pages serve slowly.

Does anyone have any advice on whether I should use the threadpool or create
thread objects "on my own" in the code?

Thanks in advance!
mortbThreadPool class is mostly used for fire and forget tasks.
Use Thread class if extended control is needed.
-
Milosz Skalecki
MCP, MCAD

"mortb" wrote:

> Hi all!
> In my webb app I'm about to introduce a way of processing long running tasks
> in the background.
> As the long running task gathers data from a database process them and
> compile them into a file appx 80 mbytes large it is a resource demanding
> task both of the CPU and the memory.
> So I think I should restrict how many of these tasks that should be able to
> start and make them run in the background.
> First I read about the Thread class and thougth: Fine I'll use a thread and
> then I'll write a utility class that will queue the tasks and make them run
> either one after one or not too many at a time.
> I'd set threads' priority to belowNormal or lowest.
> Then I discovered the threadpool class and thought: Hey here's the queuing
> abillity I was looking for. But then I found no way of setting the priority
> of the thread that would start the long running task and I don't want to
> make the web pages serve slowly.
> Does anyone have any advice on whether I should use the threadpool or create
> thread objects "on my own" in the code?
> Thanks in advance!
> mortb
>

thread restarting mid http request processing

I'm experiencing a really strange error at the moment, and was wondering if
anyone else has run into this.
This is a seemingly straightforward asp.net application -- i'm allowing the
user to make choices on a web form, collect the information, populate my
domain model, and then persist the domain model into session state which is
using the out of proc asp.net session state service.
The user behavior is that on a seemingly random interval, changes are simply
not persisted. Looking at the debugger, I can see that when this happens, m
y
thread responsible for processing the request is shut down and the request i
s
picked up by a new thread -- but apparently without the context of the
previous thread -- hence the data not being persisted.
Does anyone have any thoughts on why this is happening and how to go about
correcting?
tia,
_howardasp.net is thread agile, so switching threads is common. are you using any
STA components (vb6)? they will have this behavior in the agile model.
-- bruce (sqlwork.com)
"howard dierking" <howard dierking@.discussions.microsoft.com> wrote in
message news:0BF0E8CC-8FDB-4FEC-8360-13F665C19CBC@.microsoft.com...
> I'm experiencing a really strange error at the moment, and was wondering
if
> anyone else has run into this.
> This is a seemingly straightforward asp.net application -- i'm allowing
the
> user to make choices on a web form, collect the information, populate my
> domain model, and then persist the domain model into session state which
is
> using the out of proc asp.net session state service.
> The user behavior is that on a seemingly random interval, changes are
simply
> not persisted. Looking at the debugger, I can see that when this happens,
my
> thread responsible for processing the request is shut down and the request
is
> picked up by a new thread -- but apparently without the context of the
> previous thread -- hence the data not being persisted.
> Does anyone have any thoughts on why this is happening and how to go about
> correcting?
> tia,
> _howard
Hi Bruce,
no STA components - all asp.net
_howard
"bruce barker" wrote:

> asp.net is thread agile, so switching threads is common. are you using any
> STA components (vb6)? they will have this behavior in the agile model.
> -- bruce (sqlwork.com)
>
> "howard dierking" <howard dierking@.discussions.microsoft.com> wrote in
> message news:0BF0E8CC-8FDB-4FEC-8360-13F665C19CBC@.microsoft.com...
> if
> the
> is
> simply
> my
> is
>
>
are you using thread local storage? it will have the same problem. how are
you storing the model data? also when do you update the session?
-- bruce (sqlwork.com)
"howard dierking" <howarddierking@.discussions.microsoft.com> wrote in
message news:9D21988F-C431-4D66-B1BC-0959F9DB033A@.microsoft.com...
> Hi Bruce,
> no STA components - all asp.net
> _howard
>
> "bruce barker" wrote:
>
any
wondering
allowing
my
which
happens,
request
about
no explicit tls model data is stored as an actual session variable, so the
timing of it's persistance will be set by the normal control life cycle.
_howard
"bruce barker" wrote:

> are you using thread local storage? it will have the same problem. how are
> you storing the model data? also when do you update the session?
> -- bruce (sqlwork.com)
>
> "howard dierking" <howarddierking@.discussions.microsoft.com> wrote in
> message news:9D21988F-C431-4D66-B1BC-0959F9DB033A@.microsoft.com...
> any
> wondering
> allowing
> my
> which
> happens,
> request
> about
>
>

thread restarting mid http request processing

asp.net is thread agile, so switching threads is common. are you using any
STA components (vb6)? they will have this behavior in the agile model.

-- bruce (sqlwork.com)

"howard dierking" <howard dierking@dotnet.itags.org.discussions.microsoft.com> wrote in
message news:0BF0E8CC-8FDB-4FEC-8360-13F665C19CBC@dotnet.itags.org.microsoft.com...
> I'm experiencing a really strange error at the moment, and was wondering
if
> anyone else has run into this.
> This is a seemingly straightforward asp.net application -- i'm allowing
the
> user to make choices on a web form, collect the information, populate my
> domain model, and then persist the domain model into session state which
is
> using the out of proc asp.net session state service.
> The user behavior is that on a seemingly random interval, changes are
simply
> not persisted. Looking at the debugger, I can see that when this happens,
my
> thread responsible for processing the request is shut down and the request
is
> picked up by a new thread -- but apparently without the context of the
> previous thread -- hence the data not being persisted.
> Does anyone have any thoughts on why this is happening and how to go about
> correcting?
> tia,
> _howardHi Bruce,

no STA components - all asp.net

_howard

"bruce barker" wrote:

> asp.net is thread agile, so switching threads is common. are you using any
> STA components (vb6)? they will have this behavior in the agile model.
> -- bruce (sqlwork.com)
>
> "howard dierking" <howard dierking@.discussions.microsoft.com> wrote in
> message news:0BF0E8CC-8FDB-4FEC-8360-13F665C19CBC@.microsoft.com...
> > I'm experiencing a really strange error at the moment, and was wondering
> if
> > anyone else has run into this.
> > This is a seemingly straightforward asp.net application -- i'm allowing
> the
> > user to make choices on a web form, collect the information, populate my
> > domain model, and then persist the domain model into session state which
> is
> > using the out of proc asp.net session state service.
> > The user behavior is that on a seemingly random interval, changes are
> simply
> > not persisted. Looking at the debugger, I can see that when this happens,
> my
> > thread responsible for processing the request is shut down and the request
> is
> > picked up by a new thread -- but apparently without the context of the
> > previous thread -- hence the data not being persisted.
> > Does anyone have any thoughts on why this is happening and how to go about
> > correcting?
> > tia,
> > _howard
>
are you using thread local storage? it will have the same problem. how are
you storing the model data? also when do you update the session?

-- bruce (sqlwork.com)

"howard dierking" <howarddierking@.discussions.microsoft.com> wrote in
message news:9D21988F-C431-4D66-B1BC-0959F9DB033A@.microsoft.com...
> Hi Bruce,
> no STA components - all asp.net
> _howard
>
> "bruce barker" wrote:
> > asp.net is thread agile, so switching threads is common. are you using
any
> > STA components (vb6)? they will have this behavior in the agile model.
> > -- bruce (sqlwork.com)
> > "howard dierking" <howard dierking@.discussions.microsoft.com> wrote in
> > message news:0BF0E8CC-8FDB-4FEC-8360-13F665C19CBC@.microsoft.com...
> > > I'm experiencing a really strange error at the moment, and was
wondering
> > if
> > > anyone else has run into this.
> > > > This is a seemingly straightforward asp.net application -- i'm
allowing
> > the
> > > user to make choices on a web form, collect the information, populate
my
> > > domain model, and then persist the domain model into session state
which
> > is
> > > using the out of proc asp.net session state service.
> > > > The user behavior is that on a seemingly random interval, changes are
> > simply
> > > not persisted. Looking at the debugger, I can see that when this
happens,
> > my
> > > thread responsible for processing the request is shut down and the
request
> > is
> > > picked up by a new thread -- but apparently without the context of the
> > > previous thread -- hence the data not being persisted.
> > > > Does anyone have any thoughts on why this is happening and how to go
about
> > > correcting?
> > > > tia,
> > > > _howard

Thread was being aborted error message

I have this web application that runs for about 5 minutes doing to
database processing, about 50% of the time I get the error message
Thread was being aborted.

I am looking for hint at where to look, I've been banging my head on
this issue for a couple of months (when I have time to bang my head on
this).

Here's some information on my app:

..Net 1.1
I am not using either response.redirect or server.transfer
I'm not threading anything (single synchronous thread)
My ASP.Net UI calles a .Net class for the processing
Tested on 3 different dedicated IIS servers with same results

Works fine if I install it on a desktop (any desktop) running IIS
Works irratically when I install it on a server running either Windows
Server 2000 or Windows Server 2003

What should I be looking at?Alex,
Are you saying that a particular ASP.NET web page kicks off some process
that runs for 5 minutes? The default ASP.NET Script timeout is 90 seconds.
You can change this, and you can also change the httpRuntime executionTimeout
value to a larger number by bringing this element into your web.config to
override the setting in machine.config.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com
"Alex A." wrote:

Quote:

Originally Posted by

I have this web application that runs for about 5 minutes doing to
database processing, about 50% of the time I get the error message
Thread was being aborted.
>
I am looking for hint at where to look, I've been banging my head on
this issue for a couple of months (when I have time to bang my head on
this).
>
Here's some information on my app:
>
..Net 1.1
I am not using either response.redirect or server.transfer
I'm not threading anything (single synchronous thread)
My ASP.Net UI calles a .Net class for the processing
Tested on 3 different dedicated IIS servers with same results
>
Works fine if I install it on a desktop (any desktop) running IIS
Works irratically when I install it on a server running either Windows
Server 2000 or Windows Server 2003
>
What should I be looking at?
>
>


Peter,

The script timeout was set to 900 seconds (15 minutes), I had
encountered this issue on another project.

The issue is sporadic, it'll either work or not and when it doesn't it
a thread abort message.

Nothing fancy is being done, it's mainly reading about 300 invoices
from a database converting the data into a format and saving it to a
file on a server.

Peter wrote:

Quote:

Originally Posted by

Alex,
Are you saying that a particular ASP.NET web page kicks off some process
that runs for 5 minutes? The default ASP.NET Script timeout is 90 seconds.
You can change this, and you can also change the httpRuntime executionTimeout
value to a larger number by bringing this element into your web.config to
override the setting in machine.config.
Peter
>
--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com
>
>
>
>
"Alex A." wrote:
>

Quote:

Originally Posted by

I have this web application that runs for about 5 minutes doing to
database processing, about 50% of the time I get the error message
Thread was being aborted.

I am looking for hint at where to look, I've been banging my head on
this issue for a couple of months (when I have time to bang my head on
this).

Here's some information on my app:

..Net 1.1
I am not using either response.redirect or server.transfer
I'm not threading anything (single synchronous thread)
My ASP.Net UI calles a .Net class for the processing
Tested on 3 different dedicated IIS servers with same results

Works fine if I install it on a desktop (any desktop) running IIS
Works irratically when I install it on a server running either Windows
Server 2000 or Windows Server 2003

What should I be looking at?


Besides the script timeout, etc. You also have both a connection timeout and
a command timeout (that's with SQLClient). Very often developers forget about
the commandTimeout property.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com
"Alex A." wrote:

Quote:

Originally Posted by

Peter,
>
The script timeout was set to 900 seconds (15 minutes), I had
encountered this issue on another project.
>
The issue is sporadic, it'll either work or not and when it doesn't it
a thread abort message.
>
Nothing fancy is being done, it's mainly reading about 300 invoices
from a database converting the data into a format and saving it to a
file on a server.
>
Peter wrote:

Quote:

Originally Posted by

Alex,
Are you saying that a particular ASP.NET web page kicks off some process
that runs for 5 minutes? The default ASP.NET Script timeout is 90 seconds.
You can change this, and you can also change the httpRuntime executionTimeout
value to a larger number by bringing this element into your web.config to
override the setting in machine.config.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com
"Alex A." wrote:

Quote:

Originally Posted by

I have this web application that runs for about 5 minutes doing to
database processing, about 50% of the time I get the error message
Thread was being aborted.
>
I am looking for hint at where to look, I've been banging my head on
this issue for a couple of months (when I have time to bang my head on
this).
>
Here's some information on my app:
>
..Net 1.1
I am not using either response.redirect or server.transfer
I'm not threading anything (single synchronous thread)
My ASP.Net UI calles a .Net class for the processing
Tested on 3 different dedicated IIS servers with same results
>
Works fine if I install it on a desktop (any desktop) running IIS
Works irratically when I install it on a server running either Windows
Server 2000 or Windows Server 2003
>
What should I be looking at?
>
>


>
>


The SQL timeout is set to 5 minutes, but reading the actual database
only takes a second or two.

I think I've made sure every possible timeout is set to a high number,
not that I'm aware of every possible place a timeout might occur but I
think I've covered the basic ones that you've mentionned.

Also what I've tried to do is try and pin point where the error is
occuring.
I've set up a big try catch using the ThreadException but it never hits
my
ThreadException, I've put the ThreadException where my call to the
external Class Library is made so if the ThreadException occurs in
there it should catch it but it never does.

I am completly baffled.

Peter wrote:

Quote:

Originally Posted by

Besides the script timeout, etc. You also have both a connection timeout and
a command timeout (that's with SQLClient). Very often developers forget about
the commandTimeout property.
Peter
>
--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com
>
>
>
>
"Alex A." wrote:
>

Quote:

Originally Posted by

Peter,

The script timeout was set to 900 seconds (15 minutes), I had
encountered this issue on another project.

The issue is sporadic, it'll either work or not and when it doesn't it
a thread abort message.

Nothing fancy is being done, it's mainly reading about 300 invoices
from a database converting the data into a format and saving it to a
file on a server.

Peter wrote:

Quote:

Originally Posted by

Alex,
Are you saying that a particular ASP.NET web page kicks off some process
that runs for 5 minutes? The default ASP.NET Script timeout is 90 seconds.
You can change this, and you can also change the httpRuntime executionTimeout
value to a larger number by bringing this element into your web.config to
override the setting in machine.config.
Peter
>
--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com
>
>
>
>
"Alex A." wrote:
>
I have this web application that runs for about 5 minutes doing to
database processing, about 50% of the time I get the error message
Thread was being aborted.

I am looking for hint at where to look, I've been banging my head on
this issue for a couple of months (when I have time to bang my head on
this).

Here's some information on my app:

..Net 1.1
I am not using either response.redirect or server.transfer
I'm not threading anything (single synchronous thread)
My ASP.Net UI calles a .Net class for the processing
Tested on 3 different dedicated IIS servers with same results

Works fine if I install it on a desktop (any desktop) running IIS
Works irratically when I install it on a server running either Windows
Server 2000 or Windows Server 2003

What should I be looking at?



Alex,
Hook the Application_Error event in Global.asax and use:

Exception ex=Server.GetLastError.GetBaseException()
// log or report the exception here

Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com
"Alex A." wrote:

Quote:

Originally Posted by

The SQL timeout is set to 5 minutes, but reading the actual database
only takes a second or two.
>
I think I've made sure every possible timeout is set to a high number,
not that I'm aware of every possible place a timeout might occur but I
think I've covered the basic ones that you've mentionned.
>
Also what I've tried to do is try and pin point where the error is
occuring.
I've set up a big try catch using the ThreadException but it never hits
my
ThreadException, I've put the ThreadException where my call to the
external Class Library is made so if the ThreadException occurs in
there it should catch it but it never does.
>
I am completly baffled.
>
Peter wrote:

Quote:

Originally Posted by

Besides the script timeout, etc. You also have both a connection timeout and
a command timeout (that's with SQLClient). Very often developers forget about
the commandTimeout property.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com
"Alex A." wrote:

Quote:

Originally Posted by

Peter,
>
The script timeout was set to 900 seconds (15 minutes), I had
encountered this issue on another project.
>
The issue is sporadic, it'll either work or not and when it doesn't it
a thread abort message.
>
Nothing fancy is being done, it's mainly reading about 300 invoices
from a database converting the data into a format and saving it to a
file on a server.
>
Peter wrote:
Alex,
Are you saying that a particular ASP.NET web page kicks off some process
that runs for 5 minutes? The default ASP.NET Script timeout is 90 seconds.
You can change this, and you can also change the httpRuntime executionTimeout
value to a larger number by bringing this element into your web.config to
override the setting in machine.config.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com




"Alex A." wrote:

I have this web application that runs for about 5 minutes doing to
database processing, about 50% of the time I get the error message
Thread was being aborted.
>
I am looking for hint at where to look, I've been banging my head on
this issue for a couple of months (when I have time to bang my head on
this).
>
Here's some information on my app:
>
..Net 1.1
I am not using either response.redirect or server.transfer
I'm not threading anything (single synchronous thread)
My ASP.Net UI calles a .Net class for the processing
Tested on 3 different dedicated IIS servers with same results
>
Works fine if I install it on a desktop (any desktop) running IIS
Works irratically when I install it on a server running either Windows
Server 2000 or Windows Server 2003
>
What should I be looking at?
>
>
>
>


>
>