Stored procedure takes more than 30 seconds to execute

Question

I am optimizing the stored procedure in which I get list of more than 10K user ids in one table variable as a parameter. I am setting flag @ContainsUserIds if there is any data into the table variable. In main query selecting the all the users that are present in table variable or all the users if no data found in table variable (There are more condition in where clause).

The problem is here that the statement in where clause takes more than 30 second which has OR condition. If I removed the first condition to check @ContainsUserIds = 0 then it will execute within a second. Can someone please help me to optimize this query.

This Stored procedure is called from different position so the User ids may not pass.

CREATE PROCEDURE [core].[spGetUsersByFilter]      
(      
  @ClientId                nvarchar(38)  = NULL,        
  @UserIds                 [UserIdList] readonly      
)      
AS      
BEGIN   
    SET NOCOUNT ON         
      
    DECLARE @ContainsUserIds BIT = 0,
    SET @ContainsUserIds = CASE 
                                WHEN EXISTS(SELECT TOP 1 1 FROM @UserIds) 
                                    THEN 1 
                                ELSE 0 
                            END;   

    SELECT DISTINCT ES.Id
    FROM [db].[Users] E
    JOIN [db].[UserDetails] ES ON ES.UserId = E.Id 
    WHERE E.[Active] = 1  
         AND (@ContainsUserIds = 0 OR E.UserId IN(SELECT Item FROM @UserIds)) 
         AND ES.ClientId = @ClientId
END