Partly, but "fine-grained parallelism" refers to how small you can make the sub-parts of the parallel execution and then adds "Another definition of granularity takes into account the communication overhead between multiple processors or processing elements." which is confusing! t compute / t communicate (where communicate probably does not differentiate between memory copy and waiting for memory due to a lock or cache-miss)?
According to that definition If you have a really tiny task that has fast memory (embarrassingly parallel + little memory) and a huge task that has really slow memory they are the same (1/1 == 100000/100000) so really what does that definition say!
Also it seems only instruction levels of parallelism are fine-grained. This means everything I will ever do and talk about is coarse-grained. So how can I tell people my software is 10x faster than Erlang for a MMO type of server problem space?
I'm going to access contiguous memory with 2 separate threads simultaneously without locks in C when I finalize my 3D MMO engine client this fall and then I'll understand more on how these things really work, right now I'm a little confused.
I think "Joint Parallelism " is more telling, it puts focus on the bottle neck of our civilization which is / and will always be: memory speed!
F.ex. Erlang is completely meaningless for "joint parallelism", because it uses memory copying instead of monitors/locks and even there Java can be even more performant according to the creator of the Java concurrency package:
"While I'm on the topic of concurrency I should mention my far too brief chat with Doug Lea. He commented that multi-threaded Java these days far outperforms C, due to the memory management and a garbage collector. If I recall correctly he said "only 12 times faster than C means you haven't started optimizing"." - Martin Fowler (https://martinfowler.com/bliki/OOPSLA2005.html)
I have mailed Doug to get an explanation, but the only thing I can tell you is that my implementation of what he talks about in that quote is proof that he is right. How he is right, I still don't totally understand! He hasn't and probably wont reply though!
More worrying is the memory copying that the kernel is doing, I think the last step for computing advances is to go back and simplify the OS.
My prediction is that we will get kernel bypass for network IO pretty soon and disk IO will follow after that, at least for servers.
According to that definition If you have a really tiny task that has fast memory (embarrassingly parallel + little memory) and a huge task that has really slow memory they are the same (1/1 == 100000/100000) so really what does that definition say!
Also it seems only instruction levels of parallelism are fine-grained. This means everything I will ever do and talk about is coarse-grained. So how can I tell people my software is 10x faster than Erlang for a MMO type of server problem space?
I'm going to access contiguous memory with 2 separate threads simultaneously without locks in C when I finalize my 3D MMO engine client this fall and then I'll understand more on how these things really work, right now I'm a little confused.
I think "Joint Parallelism " is more telling, it puts focus on the bottle neck of our civilization which is / and will always be: memory speed!
F.ex. Erlang is completely meaningless for "joint parallelism", because it uses memory copying instead of monitors/locks and even there Java can be even more performant according to the creator of the Java concurrency package:
"While I'm on the topic of concurrency I should mention my far too brief chat with Doug Lea. He commented that multi-threaded Java these days far outperforms C, due to the memory management and a garbage collector. If I recall correctly he said "only 12 times faster than C means you haven't started optimizing"." - Martin Fowler (https://martinfowler.com/bliki/OOPSLA2005.html)
I have mailed Doug to get an explanation, but the only thing I can tell you is that my implementation of what he talks about in that quote is proof that he is right. How he is right, I still don't totally understand! He hasn't and probably wont reply though!
More worrying is the memory copying that the kernel is doing, I think the last step for computing advances is to go back and simplify the OS.
My prediction is that we will get kernel bypass for network IO pretty soon and disk IO will follow after that, at least for servers.