GPars progress

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

GPars progress

Russel Winder-3
Happy New Year.

Hopefully everyone knows and loves GPars. Which is good. The "issue" is
that the world moves on, but currently GPars is not. The most obvious
issue (to me anyway) is the arrival of JDK 8 and the streams mechanism
for data parallelism: the GPars approach to this needs to be changed to
integrate with this. Of course this is about Java 8 and not everyone is
using it yet. What else is there about GPars that needs to evolve? Good
question.

There is a "sort of" road map for GPars at
http://gpars.codehaus.org/Roadmap. It would be good to get views,
opinions, changes. Also it would be good to get contributions: new
tests, updated documentation, tested evolutions of the code base.

Whilst the base version of Java for Groovy is JDK6, GPars has three
flavours, JDK6, JDK7, JDK8. Looking at the situation when Groovy has a
base version of JDK7 will allow dropping of all the JDK6 stuff, but is
there other stuff that needs doing. Is there other stuff for JDK8?

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: [gpars-dev] GPars progress

Johannes Link
One more thing which should be fixed is STM support. Multiverse seems
to be no longer supported and the current version (0.7 GA) does not
support anything but the simplest scenarios.

Johannes

2014/1/2 Russel Winder <[hidden email]>:

> Happy New Year.
>
> Hopefully everyone knows and loves GPars. Which is good. The "issue" is
> that the world moves on, but currently GPars is not. The most obvious
> issue (to me anyway) is the arrival of JDK 8 and the streams mechanism
> for data parallelism: the GPars approach to this needs to be changed to
> integrate with this. Of course this is about Java 8 and not everyone is
> using it yet. What else is there about GPars that needs to evolve? Good
> question.
>
> There is a "sort of" road map for GPars at
> http://gpars.codehaus.org/Roadmap. It would be good to get views,
> opinions, changes. Also it would be good to get contributions: new
> tests, updated documentation, tested evolutions of the code base.
>
> Whilst the base version of Java for Groovy is JDK6, GPars has three
> flavours, JDK6, JDK7, JDK8. Looking at the situation when Groovy has a
> base version of JDK7 will allow dropping of all the JDK6 stuff, but is
> there other stuff that needs doing. Is there other stuff for JDK8?
>
> --
> Russel.
> =============================================================================
> Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
> 41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
> London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: [gpars-dev] GPars progress

Eric MacAdie
I have a general question: Could agents be used for the same scenarios that STM is used for?

= Eric MacAdie



On Thu, Jan 2, 2014 at 2:25 PM, Johannes Link <[hidden email]> wrote:
One more thing which should be fixed is STM support. Multiverse seems
to be no longer supported and the current version (0.7 GA) does not
support anything but the simplest scenarios.

Johannes

2014/1/2 Russel Winder <[hidden email]>:
> Happy New Year.
>
> Hopefully everyone knows and loves GPars. Which is good. The "issue" is
> that the world moves on, but currently GPars is not. The most obvious
> issue (to me anyway) is the arrival of JDK 8 and the streams mechanism
> for data parallelism: the GPars approach to this needs to be changed to
> integrate with this. Of course this is about Java 8 and not everyone is
> using it yet. What else is there about GPars that needs to evolve? Good
> question.
>
> There is a "sort of" road map for GPars at
> http://gpars.codehaus.org/Roadmap. It would be good to get views,
> opinions, changes. Also it would be good to get contributions: new
> tests, updated documentation, tested evolutions of the code base.
>
> Whilst the base version of Java for Groovy is JDK6, GPars has three
> flavours, JDK6, JDK7, JDK8. Looking at the situation when Groovy has a
> base version of JDK7 will allow dropping of all the JDK6 stuff, but is
> there other stuff that needs doing. Is there other stuff for JDK8?
>
> --
> Russel.
> =============================================================================
> Dr Russel Winder      t: <a href="tel:%2B44%2020%207585%202200" value="+442075852200">+44 20 7585 2200   voip: [hidden email]
> 41 Buckmaster Road    m: <a href="tel:%2B44%207770%20465%20077" value="+447770465077">+44 7770 465 077   xmpp: [hidden email]
> London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: [gpars-dev] GPars progress

Vaclav
Administrator
In reply to this post by Johannes Link
Yes, indeed. Multiverse seem to have stopped evolving and we should look around for options.

Vaclav



On Thu, Jan 2, 2014 at 9:25 PM, Johannes Link <[hidden email]> wrote:
One more thing which should be fixed is STM support. Multiverse seems
to be no longer supported and the current version (0.7 GA) does not
support anything but the simplest scenarios.

Johannes

2014/1/2 Russel Winder <[hidden email]>:
> Happy New Year.
>
> Hopefully everyone knows and loves GPars. Which is good. The "issue" is
> that the world moves on, but currently GPars is not. The most obvious
> issue (to me anyway) is the arrival of JDK 8 and the streams mechanism
> for data parallelism: the GPars approach to this needs to be changed to
> integrate with this. Of course this is about Java 8 and not everyone is
> using it yet. What else is there about GPars that needs to evolve? Good
> question.
>
> There is a "sort of" road map for GPars at
> http://gpars.codehaus.org/Roadmap. It would be good to get views,
> opinions, changes. Also it would be good to get contributions: new
> tests, updated documentation, tested evolutions of the code base.
>
> Whilst the base version of Java for Groovy is JDK6, GPars has three
> flavours, JDK6, JDK7, JDK8. Looking at the situation when Groovy has a
> base version of JDK7 will allow dropping of all the JDK6 stuff, but is
> there other stuff that needs doing. Is there other stuff for JDK8?
>
> --
> Russel.
> =============================================================================
> Dr Russel Winder      t: <a href="tel:%2B44%2020%207585%202200" value="+442075852200">+44 20 7585 2200   voip: [hidden email]
> 41 Buckmaster Road    m: <a href="tel:%2B44%207770%20465%20077" value="+447770465077">+44 7770 465 077   xmpp: [hidden email]
> London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
E-mail: [hidden email]
Blog: http://www.jroller.com/vaclav
Linkedin page: http://www.linkedin.com/in/vaclavpech
Reply | Threaded
Open this post in threaded view
|

Re: [gpars-dev] GPars progress

Vaclav
Administrator
In reply to this post by Eric MacAdie
In brief, agents typically protect access to a single data entity, while Stm provides transactional ACI(D) guarantees across multiple data elements.
Also, agents act in a very pessimistic manner, while Stm can apply various optimistic strategies.

Vaclav



On Fri, Jan 3, 2014 at 2:45 AM, Eric MacAdie <[hidden email]> wrote:
I have a general question: Could agents be used for the same scenarios that STM is used for?

= Eric MacAdie



On Thu, Jan 2, 2014 at 2:25 PM, Johannes Link <[hidden email]> wrote:
One more thing which should be fixed is STM support. Multiverse seems
to be no longer supported and the current version (0.7 GA) does not
support anything but the simplest scenarios.

Johannes

2014/1/2 Russel Winder <[hidden email]>:
> Happy New Year.
>
> Hopefully everyone knows and loves GPars. Which is good. The "issue" is
> that the world moves on, but currently GPars is not. The most obvious
> issue (to me anyway) is the arrival of JDK 8 and the streams mechanism
> for data parallelism: the GPars approach to this needs to be changed to
> integrate with this. Of course this is about Java 8 and not everyone is
> using it yet. What else is there about GPars that needs to evolve? Good
> question.
>
> There is a "sort of" road map for GPars at
> http://gpars.codehaus.org/Roadmap. It would be good to get views,
> opinions, changes. Also it would be good to get contributions: new
> tests, updated documentation, tested evolutions of the code base.
>
> Whilst the base version of Java for Groovy is JDK6, GPars has three
> flavours, JDK6, JDK7, JDK8. Looking at the situation when Groovy has a
> base version of JDK7 will allow dropping of all the JDK6 stuff, but is
> there other stuff that needs doing. Is there other stuff for JDK8?
>
> --
> Russel.
> =============================================================================
> Dr Russel Winder      t: <a href="tel:%2B44%2020%207585%202200" value="+442075852200" target="_blank">+44 20 7585 2200   voip: [hidden email]
> 41 Buckmaster Road    m: <a href="tel:%2B44%207770%20465%20077" value="+447770465077" target="_blank">+44 7770 465 077   xmpp: [hidden email]
> London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email






--
E-mail: [hidden email]
Blog: http://www.jroller.com/vaclav
Linkedin page: http://www.linkedin.com/in/vaclavpech
Reply | Threaded
Open this post in threaded view
|

Re: [gpars-dev] GPars progress

Johannes Link
In reply to this post by Eric MacAdie
In some cases yes, but STM is typically used when you need transactional behaviour across entities. This cannot be done w/ actors unless you introduce some kind of distributed transaction protocol. And we know how "good" distributed tx work in databases.

Johannes

Am 03.01.2014 um 02:45 schrieb Eric MacAdie <[hidden email]>:

I have a general question: Could agents be used for the same scenarios that STM is used for?

= Eric MacAdie



On Thu, Jan 2, 2014 at 2:25 PM, Johannes Link <[hidden email]> wrote:
One more thing which should be fixed is STM support. Multiverse seems
to be no longer supported and the current version (0.7 GA) does not
support anything but the simplest scenarios.

Johannes

2014/1/2 Russel Winder <[hidden email]>:
> Happy New Year.
>
> Hopefully everyone knows and loves GPars. Which is good. The "issue" is
> that the world moves on, but currently GPars is not. The most obvious
> issue (to me anyway) is the arrival of JDK 8 and the streams mechanism
> for data parallelism: the GPars approach to this needs to be changed to
> integrate with this. Of course this is about Java 8 and not everyone is
> using it yet. What else is there about GPars that needs to evolve? Good
> question.
>
> There is a "sort of" road map for GPars at
> http://gpars.codehaus.org/Roadmap. It would be good to get views,
> opinions, changes. Also it would be good to get contributions: new
> tests, updated documentation, tested evolutions of the code base.
>
> Whilst the base version of Java for Groovy is JDK6, GPars has three
> flavours, JDK6, JDK7, JDK8. Looking at the situation when Groovy has a
> base version of JDK7 will allow dropping of all the JDK6 stuff, but is
> there other stuff that needs doing. Is there other stuff for JDK8?
>
> --
> Russel.
> =============================================================================
> Dr Russel Winder      t: <a href="tel:%2B44%2020%207585%202200" value="+442075852200">+44 20 7585 2200   voip: [hidden email]
> 41 Buckmaster Road    m: <a href="tel:%2B44%207770%20465%20077" value="+447770465077">+44 7770 465 077   xmpp: [hidden email]
> London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: [gpars-dev] GPars progress

Russel Winder-3
In reply to this post by Eric MacAdie
On Thu, 2014-01-02 at 19:45 -0600, Eric MacAdie wrote:
> I have a general question: Could agents be used for the same scenarios that
> STM is used for?

The core problem here is that TM is a thing that should be, and now is,
hardware supported, cf. latest generation of Intel memory. Thus any
software construct such as agents used to implement STM is missing the
point, it is a software construction for what should be part of the
underlying memory model.

So native code languages will very soon have HSTM and not have to use
STM, cf. Intel C and Intel C++, soon to be followed by GCC and thus I
hope D, Go, and Haskell. This leaves the question of whether the JVM
will have this in reasonable time – or whether the JVM misses the boat
completely.

As a strategy I think we need the most lightweight STM possible, if we
have it at all as a user level technique. TM always was a hardware
technique, and quite rightly. STM was a Haskell "hack" to get TM into
Haskell because it had no other possible technique for handling
asynchronous parallelism sensibly. Clojure then picked it up for very
much similar reasons. It is though an infrastructure tool, not a
sensible application level model: STM is an attempt to make shared
memory parallelism work, it is a hack, not a sensible application
architecture!

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: [gpars-dev] GPars progress

Russel Winder-3
In reply to this post by Vaclav
On Fri, 2014-01-03 at 08:31 +0100, Václav Pech wrote:
> Yes, indeed. Multiverse seem to have stopped evolving and we should look
> around for options.

The most lightweight option for us is to fork the project and take on
the maintenance ourselves. We did this reasonably successfully with
extra166y. The difference here though is that we should probably leave a
forked Multiverse as a separate project and not bring it into the GPars
codebase.

Perhaps we can "reverse into" the extant Multiverse project. If the
current owner really is letting it go fallow then perhaps some of us
could join his team and take up the developments. I'd be happy to be
involved with this.

Another approach:

What was the reason Multiverse was chosen originally? Is there another
close contender that remains maintained that it would be reasonable to
witch to (given the effort that will involve.

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: [gpars-dev] GPars progress

Johannes Link
In reply to this post by Russel Winder-3
2014/1/3 Russel Winder <[hidden email]>:
> The core problem here is that TM is a thing that should be, and now is,
> hardware supported, cf. latest generation of Intel memory. Thus any
> software construct such as agents used to implement STM is missing the
> point, it is a software construction for what should be part of the
> underlying memory model.

The hardware-supported TM scenarios are very limited since the amount
of memory which can be managed by HTM is limited by (some
non-deterministic fraction of) the available hardware  cache. That's
why any TM library will always have a fall back to (slow) STM. My
guess is that the typical usage scenarios in languages like Clojure
would have to use the software fallback most of the time. Why? Because
a  persistent object of non-trivial size will be spread across many
cache lines, and each additional cache line to consider for a
transaction raises the odds for HTM not working. Since the hardware
fallback in case of a failing cache is (slow and concurrency
preventing) locking, a TM library would probably try to implement its
own fallback mechanism when in doubt. My guess: HTM will not live up
to expectations except for programs with a very close focus on (more
or less) manual memory management.

> So native code languages will very soon have HSTM and not have to use
> STM, cf. Intel C and Intel C++, soon to be followed by GCC and thus I
> hope D, Go, and Haskell. This leaves the question of whether the JVM
> will have this in reasonable time – or whether the JVM misses the boat
> completely.

IMO, HTM requires a whole new set of JVM byte code commands. Java 11 anyone?

>
> As a strategy I think we need the most lightweight STM possible, if we
> have it at all as a user level technique. TM always was a hardware
> technique, and quite rightly. STM was a Haskell "hack" to get TM into
> Haskell because it had no other possible technique for handling
> asynchronous parallelism sensibly. Clojure then picked it up for very
> much similar reasons. It is though an infrastructure tool, not a
> sensible application level model: STM is an attempt to make shared
> memory parallelism work, it is a hack, not a sensible application
> architecture!

I don't consider the approach that Clojure takes toward STM a hack. It
does not cover all imaginable STM scenarios but _iff_ it fits your
problem, it provides a manageable solution with reasonable performance
in case of low contention; much simpler to get right than a lock-based
approach. YMMV.

As for GPars: STM (of any implementation) does not really make a lot
of sense without persistent data types. That's why I recommend to
tackle both problems in a single go.

Johannes

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email