'How to check if exists any duplicate in Java 8 Streams?

In java 8, what's the best way to check if a List contains any duplicate?

My idea was something like:

list.size() != list.stream().distinct().count()

Is it the best way?

javajava-8duplicatesjava-stream


Solution 1:[1]

Your code would need to iterate over all elements. If you want to make sure that there are no duplicates simple method like

public static <T> boolean areAllUnique(List<T> list){
    Set<T> set = new HashSet<>();

    for (T t: list){
        if (!set.add(t))
            return false;
    }

    return true;
}

would be more efficient since it can give you false immediately when first non-unique element would be found.

This method could also be rewritten as (assuming non-parallel streams and thread-safe environment) using Stream#allMatch which also is short-circuit (returns false immediately for first element which doesn't fulfill provided condition)

public static <T> boolean areAllUnique(List<T> list){
    Set<T> set = new HashSet<>();
    return list.stream().allMatch(t -> set.add(t));
}

or as @Holger mentioned in comment 

public static <T> boolean areAllUnique(List<T> list){
    return list.stream().allMatch(new HashSet<>()::add);
}

Solution 2:[2]

I used the following:
1. return list.size() == new HashSet<>(list).size();.

I'm not sure how it compares to:
2. return list.size() == list.stream().distinct().count();
and
3. return list.stream().sequential().allMatch(new HashSet<>()::add);
in terms of performance.

The last one (#3) has possibility to handle not only collections (e.g. lists), but also streams (without explicitly collecting them).

Upd.: The last one (#3) seems to be the best not only because it can handle pure streams, but also because it stops on the first duplicate (while #1 and #2 always iterate till the end) — as @Pshemo said in comment.

Solution 3:[3]

You can use the counting collector.

Stream.of(1, 3, 4, 6, 7, 5, 6)
            .collect(Collectors.groupingBy(
                    Function.identity(), Collectors.counting()))
            .entrySet().stream().anyMatch(e -> e.getValue() > 1)

Solution 4:[4]

Started this class as a StreamTool, but I think there must be an even better way with reduce or similar:

public class StreamTool {

    /**
     * Whether stream records are unique in that stream.
     * @param <T> Type of records
     * @param records
     * @return true if there are no duplicates, false otherwise
     */
    public static <T> boolean isUnique(Stream<T> records) {
        return records.allMatch(new HashSet<>()::add);
    }
}

Solution 5:[5]

Given array arr,

arr.length != Arrays.stream(arr).distinct().count()

will help check for duplicates

Solution 6:[6]

Use set.add() it is faster.

Set<T> items = new HashSet<>();
list.stream().filter(n -> !items.add(n)) 
            .collect(Collectors.toSet());

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1
Solution 2
Solution 3 Will Humphreys
Solution 4 geekdenz
Solution 5 Pshemo
Solution 6 itro

Related Questions