'Why stream created with Spliterators is not being processed in parallel?
This is probably very basic, but I am not a Java person. Here is my processing code which simply prints and sleeps:
private static void myProcessings(int value)
{
System.out.println("Processing " + value);
try
{
Thread.sleep(2000);
}
catch (InterruptedException e)
{
e.printStackTrace();
}
System.out.println("Finished processing " + value);
}
Now, this parallel stream seems to work in parallel:
IntStream iit = IntStream.rangeClosed(1,3);
iit.parallel().forEach(Main::myProcessings);
// output:
// Processing 2
// Processing 1
// Processing 3
// Finished processing 3
// Finished processing 2
// Finished processing 1
But this one (made from an Iterator) does not:
static class MyIter implements Iterator<Integer>
{
private int max;
private int current;
public MyIter(int maxVal)
{
max = maxVal;
current = 1;
}
@Override
public boolean hasNext()
{
return current <= max;
}
@Override
public Integer next()
{
return current++;
}
}
MyIter it = new MyIter(3);
StreamSupport.stream(Spliterators.spliteratorUnknownSize(it, 0), true)
.forEach(Main::myProcessings);
// output:
// Processing 1
// Finished processing 1
// Processing 2
// Finished processing 2
// Processing 3
// Finished processing 3
What am I doing wrong in the custom Iterator version? (I am using Java 8)
Solution 1:[1]
One way is to give an estimate of the size of the stream:
Spliterators.spliterator(it, 3, 0);
The number (3 here) doesn't have to be precise, but if you give say 10000, only one thread will be used for an actual size of 3. If you give say 10, multiple threads will be used, even with a size of 3.
The estimate (3 in my example) is used to determine the size of the batches (number of tasks sent to one thread before moving on to the next thread). If you provide a large estimated number and only submit a few tasks, they will probably all be grouped and run on the first thread and nothing will be sent to the second thread.
Solution 2:[2]
There was a flaw in Spliterators.spliteratorUnknownSize()
implementation. I fixed it in Java 19, see JDK-8280915. Since 19-ea+19-1283 early access build the problem is not reproduced anymore, your code is parallelized correctly without explicit size specification:
Processing 2
Processing 3
Processing 1
Finished processing 3
Finished processing 1
Finished processing 2
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | |
Solution 2 | Tagir Valeev |