[java] AsyncMessagingCore throws InterruptedException on more complex dataflow

classic Classic list List threaded Threaded
3 messages Options
Ela
Reply | Threaded
Open this post in threaded view
|

[java] AsyncMessagingCore throws InterruptedException on more complex dataflow

Ela
Hi,

I want to use GPars with Java to define a data flow (I use Dataflow from groovyx.gpars.dataflow for it).
My dataflow is a bit complex, but to simplify if, lets imagine that we have one report, that consist of some number of sub-reports. The report is ready only when all sub-reports are done, and then the process of generating each sub-report requires executing couple of tasks, that have some dependencies on each other (I massively simplified that part in the sample code). Each sub-report requires exactly the same tasks to be executed.
So what I ended up with was a top-level dataflow for the report, and multiple dataflows for sub-reports within it.  
Now, it all works great, but.. when I execute it multiple times, after let's say 40'000-60'000th execution, I can see this exception:

---------------------
The actor processing thread has been interrupted Actor Thread 479

java.lang.InterruptedException
        at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:133)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
---------------------

and my dataflow freezes, because I try to retrieve the value from DataflowQueue, that never arrives there.

Here's my sample code:
https://gist.github.com/elaPa/8bfa648806b47d236b99

My question is: am I doing something wrong? Do I have this issue because I have a dataflow inside the dataflow ? (FYI: I wrote a similar experimental code for a single dataflow, and I couldn't make it fail...) Is this just not a recommended way of using gpars? Why is AsyncMessagingCore being interrupted?

Any help / advice would be much appreciated!

Thank you,
Ela

---------
my dependencies:
org.codehaus.gpars:gpars:1.2.1
org.codehaus.groovy:groovy-all:2.4.1
java 1.7
Reply | Threaded
Open this post in threaded view
|

Re: [java] AsyncMessagingCore throws InterruptedException on more complex dataflow

Paolo Di Tommaso
I'm experiencing a different exception but I'm almost sure that is related to the same problem. 

In my case, in some circumstances that I'm unable to reproduce, I'm getting a java.nio.channels.ClosedByInterruptException error in my application based on Gpars dataflow.

As stated by the Javadoc this "exception is received by a thread when another thread interrupts it while it is blocked in an I/O operation upon a channel". 

Thus, as in your case it looks like there's a component invoking a Thread.interrupt method that raise that exception. 


A possible cause of this problem, could be the shutdown of the thread pool but it doesn't look to be source of the problem in your case nor in mine. 

Investigating in the Gpars code, I've noticed that the interrupt method is invoked in different points. For example DataflowProcessor.terminate() invokes AbstractLoopActor.terminate() that at line 211 call Thread.terminate() 


Since in your code you are terminating all processors, this *may* the point that raise the exeception




However I've still missing the logic behind interrupting the thread. It would be greatly appreciated if a Gpars core committer would comment on this and how prevent this issue. 


Thanks a lot. 

Cheers,
Paolo



On Wed, Mar 11, 2015 at 3:47 PM, Ela <[hidden email]> wrote:
Hi,

I want to use GPars with Java to define a data flow (I use Dataflow from
groovyx.gpars.dataflow for it).
My dataflow is a bit complex, but to simplify if, lets imagine that we have
one report, that consist of some number of sub-reports. The report is ready
only when all sub-reports are done, and then the process of generating each
sub-report requires executing couple of tasks, that have some dependencies
on each other (I massively simplified that part in the sample code). Each
sub-report requires exactly the same tasks to be executed.
So what I ended up with was a top-level dataflow for the report, and
multiple dataflows for sub-reports within it.
Now, it all works great, but.. when I execute it multiple times, after let's
say 40'000-60'000th execution, I can see this exception:

---------------------
The actor processing thread has been interrupted Actor Thread 479

java.lang.InterruptedException
        at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:133)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
---------------------

and my dataflow freezes, because I try to retrieve the value from
DataflowQueue, that never arrives there.

Here's my sample code:
https://gist.github.com/elaPa/8bfa648806b47d236b99

My question is: am I doing something wrong? Do I have this issue because I
have a dataflow inside the dataflow ? (FYI: I wrote a similar experimental
code for a single dataflow, and I couldn't make it fail...) Is this just not
a recommended way of using gpars? Why is AsyncMessagingCore being
interrupted?

Any help / advice would be much appreciated!

Thank you,
Ela

---------
my dependencies:
org.codehaus.gpars:gpars:1.2.1
org.codehaus.groovy:groovy-all:2.4.1
java 1.7



--
View this message in context: http://gpars-user-mailing-list.19372.n3.nabble.com/java-AsyncMessagingCore-throws-InterruptedException-on-more-complex-dataflow-tp4025088.html
Sent from the GPars - user mailing list mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: [java] AsyncMessagingCore throws InterruptedException on more complex dataflow

Russel Winder-3
Paolo,

Just a quick note to say I spotted this email, but cannot investigate
immediately. Unless someone grabs hold of this before then, I can take a
look at this middle of next week.

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email