Powershell large hex range

Question

I am trying to make a range of hex from a000000000-afffffffff, but the values are too large for Int32. Getting error Cannot convert value "687194767360" to type "System.Int32". Error: "Value was either too large or too small for an Int32."

Here is the code (0xa000000000..0xafffffffff|% ToString X10).ToLower()

Answer 1

The endpoints of .., the range operator, must fit into [int] values, which your numbers exceed.

• Additionally, since you're trying to collect the output of the streaming 0xa000000000..0xafffffffff| % ToString X10 pipeline command in memory in full, by enclosing it in (...) and calling method .ToString() on it, the collected outputs would have to fit into an array, and the count of elements in your range exceed the max. capacity of a .NET array too.

Similarly, the [Linq.Enumerable]::Range() method unfortunately also only accepts [int] endpoints.

However, you can implement a (custom) streaming solution, where PowerShell creates and emits each formatted number (string) to the pipeline one by one:

$num = 0xa000000000
while ($num -le 0xafffffffff) { ($num++).ToString('x10') }

Note:

• While this works, it will be slow.

• You won't be able to capture the result as a whole in a collection in memory.

  • If you were to stream the output to a file unfiltered, you'd end up with a file 704 GB(!) / 768 GB(!) in size on Unix-like platforms / Windows.
  • In-memory, partitioning the output into chunk(s) small enough to fit into array(s) is the only option.

Answer 2

Not possible to hold the collection as a whole using System.Array:

The array size is limited to a total of 4 billion elements, and to a maximum index of 0X7FEFFFFF in any given dimension (0X7FFFFFC7 for byte arrays and arrays of single-byte structures).

Nor using List<T>:

.NET Framework only: For very large List objects, you can increase the maximum capacity to 2 billion elements on a 64-bit system by setting the enabled attribute of the configuration element to true in the run-time environment.

Assuming you have enough memory to hold 68,719,476,735 elements which I don't think you do :), you could do a list of lists:

$hex = [System.Collections.Generic.List[object]]::new()
$tmp = [System.Collections.Generic.List[string]]::new()

for($i = 0xa000000000; $i -le 0xafffffffff; $i++) {
    if($tmp.Count -eq [int]::MaxValue) {
        $hex.Add($tmp)
        $tmp.Clear()
    }
    $tmp.Add($i.ToString('x10'))
}

You can use $i.ToString('x10') or '{0:x10}' -f $i to produce lower-case hex. See Hexadecimal format specifier (X) for details.