"Array dimensions exceeded supported range" when creating a dataframe

Question

I'm getting this exception when trying to create a DataFrame with only 50 millions rows. Any ideas how to avoid this problem?

[Exception] [JvmBridge] Array dimensions exceeded supported range.
   at System.IO.MemoryStream.set_Capacity(Int32 value)
   at System.IO.MemoryStream.EnsureCapacity(Int32 value)
   at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at Microsoft.Spark.Interop.Ipc.SerDe.Write(Stream s, String value)
   at Microsoft.Spark.Interop.Ipc.PayloadHelper.ConvertArgsToBytes(MemoryStream destination, Object[] args, Boolean addTypeIdPrefix)
   at Microsoft.Spark.Interop.Ipc.PayloadHelper.ConvertArgsToBytes(MemoryStream destination, Object[] args, Boolean addTypeIdPrefix)
   at Microsoft.Spark.Interop.Ipc.PayloadHelper.BuildPayload(MemoryStream destination, Boolean isStaticMethod, Int32 processId, Int32 threadId, Object classNameOrJvmObjectReference, String methodName, Object[] args)
   at Microsoft.Spark.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] args)
System.Threading.SemaphoreFullException: Adding the specified count to the semaphore would cause it to exceed its maximum count.
   at System.Threading.SemaphoreSlim.Release(Int32 releaseCount)
   at Microsoft.Spark.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] args)
   at Microsoft.Spark.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object arg0, Object arg1)
   at Microsoft.Spark.Interop.Ipc.JvmBridge.CallNonStaticJavaMethod(JvmObjectReference jvmObject, String methodName, Object arg0, Object arg1)
   at Microsoft.Spark.Interop.Ipc.JvmObjectReference.Invoke(String methodName, Object arg0, Object arg1)
   at Microsoft.Spark.Sql.SparkSession.CreateDataFrame(IEnumerable`1 data, StructType schema)
   at Submission#29.<<Initialize>>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.CodeAnalysis.Scripting.ScriptExecutionState.RunSubmissionsAsync[TResult](ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, StrongBox`1 exceptionHolderOpt, Func`2 catchExceptionOpt, CancellationToken cancellationToken)