Rails - a way to check the last line while using smarter_csv gem

Question

I am using smarter_csv gem to parse a large CSV file on Heroku.

SmarterCSV.process(file_name, { chunk_size: 10, headers_in_file: false, user_provided_headers: user_provided_headers }) do |chunk|
  chunk.each do |row|
    # parse row
  end
end

Is there any way to check if the row is the last line?

Answer 1

SmarterCSV does not seem to have any utility for this, so I think the only way is to count the number of lines before and while processing:

line_count = `wc -l < file_name`.to_i
lines_processed = 0
SmarterCSV.process(file_name, { chunk_size: 10, headers_in_file: false, user_provided_headers: user_provided_headers }) do |chunk|
  chunk.each do |row|
    lines_processed += 1
    return if lines_processed == line_count
    # parse row
  end
end

Answer 2

smarter_csv does not do a read-ahead, so there is no way to know if the current row is the last row in the file.

The only way to do this would be to count the number of lines in advance, and then recognize the last line.