How to Ignore lines with StreamWriter WriteLine

Question

trying to figure out how to ignore or stop specific lines from being written to file with StreamWriter. Here is the code I'm working with from How to pass arguments to program when using variable as path :

$LogDir     = "c:\users\user" # Log file output directory
$PlinkDir   = "C:"            # plink.exe directory
$SerialIP   = "1.1.1.1"       # serial device IP address
$SerialPort = 10000           # port to log

function CaptureWeight {
    Start-Job -Name WeightLog -ScriptBlock {
        filter timestamp {
            $sw.WriteLine("$(Get-Date -Format MM/dd/yyyy_HH:mm:ss) $_")
        }

        try {
            $sw = [System.IO.StreamWriter]::new("$using:LogDir\WeightLog_$(Get-Date -f MM-dd-yyyy).txt")
            & "$using:PlinkDir\plink.exe" -telnet $using:SerialIP -P $using:SerialPort | TimeStamp
        }
        finally {
            $sw.ForEach('Flush')
            $sw.ForEach('Dispose')
        }
    }
}

$job = CaptureWeight     # For testing, save the job
Start-Sleep -Seconds 60  # wait 1 minute
$job | Stop-Job          # kill the job
Get-Content "$LogDir\WeightLog_$(Get-Date -f MM-dd-yyyy).txt" # Did it work?

And the output is this:

05/09/2022_14:34:19   G+027800 lb
05/09/2022_14:34:20 
05/09/2022_14:34:20   G+027820 lb
05/09/2022_14:34:21 
05/09/2022_14:34:21   G+027820 lb
05/09/2022_14:34:22 
05/09/2022_14:34:22   G+027820 lb

Without the TimeStamp, every other line is blank. I have a couple lines to cleanup the logs, one removes every other line one removes lines with zero weights:

   Set-Content -Path "$LogDir\WeightLog_$(get-date -f MM-dd-yyyy).txt" -Value (get-content -Path "$LogDir\WeightLog_$(get-date -f MM-dd-yyyy).txt" | Where-Object { $i % 2 -eq 0; $i++ })

   Set-Content -Path "$LogDir\WeightLog_$(get-date -f MM-dd-yyyy).txt" -Value (get-content -Path "$LogDir\WeightLog_$(get-date -f MM-dd-yyyy).txt" | Select-String -Pattern '00000' -NotMatch)

If files get to be too large these can take a while to run, would be nice to not have them written to start with.

Thanks!

Edit, This is what I ended up with:

#****************Serial Scale Weight Logger********************

$LogDir     = "c:\ScaleWeightLogger\Logs"  # Log File Output Directory
$PlinkDir   = "c:\ScaleWeightLogger"       # plink.exe Directory
$SerialIP   = "1.1.1.1"     # Serial Device IP Address
$SerialPort = "10000"             # Serial Device Port to Log
$MakeWeight = "000\d\d\d"         # Minimum weight to log
[datetime]$JobEndTime = '23:58'   # "WeightLog" Job End Time
[datetime]$JobStartTime = '00:02' #Use '8/24/2024 03:00' for a date in the future
# https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_regular_expressions

function StartWeightCapture {
    Start-Job -Name WeightLog -ScriptBlock {
        filter timestamp {
            # Set Output Filter, Do Not Write Blank Lines or Weight Matching...
            if([string]::IsNullOrWhiteSpace($_) -or $_ -match $using:MakeWeight) {
                # skip it
                return 
        }
        # Set TimeStamp Format Filter
        $sw.WriteLine("$(Get-Date -Format MM/dd/yyyy_HH:mm:ss) $_")
    }
        try {
            # Set File Path, Set $true to Append
            $sw = [System.IO.StreamWriter]::new("$using:LogDir\WeightLog_$(Get-Date -f MM-dd-yyyy).txt", $true)
            # Keep Memory Buffer Clear After Writting
            $sw.AutoFlush = $true
            # Start plink, Filter Output, Append TimeStamp 
            & "$using:PlinkDir\plink.exe" -telnet $using:SerialIP -P $using:SerialPort | TimeStamp
        }
        finally {
            # Discard Data After Writing
            $sw.ForEach('Flush')
            $sw.ForEach('Dispose')
        }
    }
}

function WeightCaptureEndTime {
   [datetime]$CurrentTime = Get-Date
   [int]$WaitSeconds = ( $JobEndTime - $CurrentTime ).TotalSeconds
   Start-Sleep -Seconds $WaitSeconds
 }

 function StopWeightCapture {
     Stop-Job WeightLog
     $AddDaysWhenInPast = 1
     [datetime]$CurrentTime = Get-Date
     If ($JobStartTime -lt $CurrentTime) { $JobStartTime = $JobStartTime.AddDays($AddDaysWhenInPast) }
     [int]$WaitSeconds = ( $JobStartTime - $CurrentTime ).TotalSeconds
     Start-Sleep -Seconds $WaitSeconds
     
 }   

while ($true) {
    StartWeightCapture
    WeightCaptureEndTime 
    StopWeightCapture
}

I'm launching it at boot with:

powershell -windowstyle hidden -ExecutionPolicy bypass "& "C:\ScaleWeightLogger\ScaleWeightLogger.ps1"" & exit

And got this to end it manually since it's in the background. It only grabs the PID of the main powershell process and not the job:

@echo off
for /F "tokens=2" %%K in ('
   tasklist /FI "ImageName eq powershell.exe" /FI "Status eq Running" /FO LIST ^| findstr /B "PID:"
') do (
   echo "PID is %%K, Ending process..."
   taskkill /F /PID %%K
)

pause
exit

Answer 1

If I understand correctly, adding this condition should avoid you the trouble of having to read the logs and skip the unwanted lines.

See String.IsNullOrWhiteSpace(String) Method and -match matching operator for details.

filter timestamp {
    # if this output is purely whitespace or it matches `00000` 
    if([string]::IsNullOrWhiteSpace($_) -or $_ -match '00000') {
        # skip it
        return 
    }
    $sw.WriteLine("$(Get-Date -Format MM/dd/yyyy_HH:mm:ss) $_")
}

Regarding the observation noted in previous question:

...when trying view the file while it's running, it seems like it updates (for viewing) about every 2 minutes, you get one 2 minute chunk of data that is about 2 minutes behind, the 2 minutes of data is there...

For this, you can enable the AutoFlush property from your StreamWriter.

Remarks has an excellent explanation of when it's worth enabling this property as well as the performance implications:

When AutoFlush is set to false, StreamWriter will do a limited amount of buffering, both internally and potentially in the encoder from the encoding you passed in. You can get better performance by setting AutoFlush to false, assuming that you always call Close (or at least Flush) when you're done writing with a StreamWriter.

For example, set AutoFlush to true when you are writing to a device where the user expects immediate feedback. Console.Out is one of these cases: The StreamWriter used internally for writing to Console flushes all its internal state except the encoder state after every call to StreamWriter.Write.

$sw = [System.IO.StreamWriter]::new("$using:LogDir\WeightLog_$(Get-Date -f MM-dd-yyyy).txt")
$sw.AutoFlush = $true