Grails, GPars and data peristance

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Grails, GPars and data peristance

allisoneer
Something is not getting flushed. A simplified example of what's happening:

def person = new Person(...)
person.save(flush: true)

println "Number of people after save: " + Person.all.size()

def dummyList = [1, 2, 3, 4, 5]

GParsPool.withPool {
    println "Number of people after withPool: " + Person.all.size()
    dummyList.eachParallel {
        println "Number of people after eachParallel " + Person.all.size()
        Person.withTransaction {
        ...

This outputs:

Number of people after save: 1
Number of people after withPool: 1
Number of people after eachParallel: 0

I don't understand if I have to do something with Session and Transaction to make the data persist or if there is something else to know about GPars. What is going on here at the underlying hibernate level?

I want the recently created Person to be visible within the parallel closure.
Reply | Threaded
Open this post in threaded view
|

Re: Grails, GPars and data peristance

Russel Winder-3
On Fri, 2012-12-28 at 22:28 -0800, allisoneer wrote:

> Something is not getting flushed. A simplified example of what's happening:
>
> def person = new Person(...)
> person.save(flush: true)
>
> println "Number of people after save: " + Person.all.size()
>
> def dummyList = [1, 2, 3, 4, 5]
>
> GParsPool.withPool {
>     println "Number of people after withPool: " + Person.all.size()
>     dummyList.eachParallel {
>         println "Number of people after eachParallel " + Person.all.size()
>         Person.withTransaction {
>         ...
>
> This outputs:
>
> Number of people after save: 1
> Number of people after withPool: 1
> Number of people after eachParallel: 0
>
> I don't understand if I have to do something with Session and Transaction to
> make the data persist or if there is something else to know about GPars.
> What is going on here at the underlying hibernate level?
>
> I want the recently created Person to be visible within the parallel
> closure.
I suspect this is a "Groovy scoping within scripts" issue and nothing to
do with GPars per se.  As an experiment remove the def in the variable
declarations so that they become entries in the script binding. If this
solves my problem then my hypothesis is correct and it is a Groovy scope
issue.

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Grails, GPars and data peristance

allisoneer
Sorry that my original post didn't show it, but this is inside a method.

When I try this in a script from the grails console, I don't get a problem, the size is always 1.  It works both with and without def.
Reply | Threaded
Open this post in threaded view
|

Re: Grails, GPars and data peristance

Russel Winder-3
On Sat, 2012-12-29 at 14:48 -0800, allisoneer wrote:
> Sorry that my original post didn't show it, but this is inside a method.

In which case my idea doesn't apply.

> When I try this in a script from the grails console, I don't get a problem,
> the size is always 1.  It works both with and without def.

Now that is strange, if the code is inside a method inside a class, the
scoping should have made a difference.

Is there any chance of you creating a small project exhibiting the
problem that you can share with us so I can experiment locally?

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Grails, GPars and data peristance

allisoneer
UDPATE:
I did a bad job of simplifying my original problem.  It is crucial to understand that in my actual application,
def person = new Person(...)
person.save(flush: true)
occurs in an integration test which I now realize is transactional.  When I set transactional to false for the integration test class, the problem "migrates."  Now, the threads can see the newly created person class but on return to the integration test which called the multi-threaded service method, the changes made that method are not visible.

All of this only happens during an integration test. It works fine under a functional test.  So, I think it has something to do with my misunderstanding of sessions and transactions and how they work with integration tests.

GPars factors into the behavior because you have to declare a new transaction for each thread.  I don't know what other effects GPars has on session or transaction.  

Here is a full example:

class Person {
    String name
}
-------------------------------------------------
import org.junit.Test
class MyServiceIntegrationTests extends GroovyTestCase {
    def myService
//    static transactional = false

    @Test
    void testDemo() {
        def person = new Person(name: "Person 1")
        person.save(flush: true)
        println "Number of people after save: " + Person.all.size()

        myService.demo()
        println "Name after demo: " + person.name
    }
}
---------------------------------------------------
import groovyx.gpars.GParsPool

class MyService {
    def demo() {
        println "Number of people in MyService.demo: " + Person.all.size()
 
        GParsPool.withPool {
            println "Number of people after withPool: " + Person.all.size()
            Person.all.eachParallel { person ->
                println "Number of people after eachParallel " + Person.all.size()
                Person.withTransaction {
                    println "Number of people in eachParallel withTransaction: " + Person.all.size()
                    person = Person.lock(person.id)
                    person.name = "new name"
                    person.save(flush: true)
                }
            }
        }
    }
}

Note: I don't know why the person = Person.lock(person.id) is required here, but without it I get a NonUniqueObjectException

Without static transactional = false, "Number of people after eachParallel" is 0 and person.name throughs a null pointer exception because there is not Person with id person.id (and so Person.lock(person.id) returns null).

With transactional = false, the number of people is always 1, but "Name after demo" remains as "Person 1"
Reply | Threaded
Open this post in threaded view
|

Re: Grails, GPars and data peristance

allisoneer
After going over all that, I just figured out that I can access the new data if I wrap the call to demo AND the declaration of person in Person.withNewSession.  However, wrapping just the call to demo doesn't do it.

It works, but I don't understand why and it seems like a Grails question.

code:
    void testDemo() {
        Person.withNewSession {
            def person = new Person(name: "Person 1")
            person.save(flush: true)
            println "Number of people after save: " + Person.all.size()

            myService.demo()
            println "Name after demo: " + person.name
        }
        println "Name outside of session: " + person.name
    }

"Name outside of session" will be "new name"
Reply | Threaded
Open this post in threaded view
|

Re: Grails, GPars and data peristance

Vaclav
Administrator
Hi,

there seems to have been some discussions about this topic earlier:

http://fbflex.wordpress.com/2010/06/11/writing-batch-import-scripts-with-grails-gsql-and-gpars/

I hope this helps.

Václav



On Mon, Dec 31, 2012 at 1:16 AM, allisoneer <[hidden email]> wrote:
After going over all that, I just figured out that I can access the new data
if I wrap the call to demo AND the declaration of person in
Person.withNewSession.  However, wrapping just the call to demo doesn't do
it.

It works, but I don't understand why and it seems like a Grails question.

code:
    void testDemo() {
        Person.withNewSession {
            def person = new Person(name: "Person 1")
            person.save(flush: true)
            println "Number of people after save: " + Person.all.size()

            myService.demo()
            println "Name after demo: " + person.name
        }
        println "Name outside of session: " + person.name
    }

"Name outside of session" will be "new name"



--
View this message in context: http://gpars-user-mailing-list.19372.n3.nabble.com/Grails-GPars-and-data-peristance-tp4024774p4024779.html
Sent from the GPars - user mailing list mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
E-mail: [hidden email]
Blog: http://www.jroller.com/vaclav
Linkedin page: http://www.linkedin.com/in/vaclavpech