FluentD: record_transformer conditional statements

Question

How to put conditional if else statements in fluentd record_transformer and add output to column.

eg:

<filter nifi.*>
  @type record_transformer
  enable_ruby true
  auto_typecast true
  <record>
    NormalizedFileInByte ${a=1024;if record[DataMetricIn]=="GB";record["FilesInByte"].to_f*a.to_f;else;record["FilesInByte"].to_f;end;} 
  </record>
</filter>

I have put an if else statement and added a new column NormalizedFileInByte to existing log in the above example. But it errors.

Answer 1

this worked for me:

NormalizedFileInByte ${if record["message"].split(" ")[2].split(",")[0] == "PM"; record["message"].split(" ")[2].split(",")[0] ;end;

Answer 2

One can use record_transformer in fluentd conf, with conditional statements. It's just that one have to follow the Ruby syntax. See the below snippet for more clarity.

<label ABC>
 <filter ABX.logs>
  @type record_transformer
  enable_ruby
 <record>
  prefetch_count ${if record["logs"].include? "prefetch_count"; record["logs"].split('prefetch_count')[-1] ; else; 'NA' end;}
  # for complex conditional statements (if elsif else)
  task_name ${if record["logs"].include? "Task " and not record["logs"].include? "Task accepted:"; record["logs"].split('Task ')[-1].split('[')[0] ; elsif record["logs"].include? "Task accepted:"; record["logs"].split('Task accepted: ')[-1].split('[')[0] ; elsif record["logs"].include? "Received task:"; record["logs"].split('Received task: ')[-1].split('[')[0] ; else; 'NA' end;}
 </record>
</label>

In the above snippet, conditional statements are written in '{}' brackets. Please make sure to split or parse the line properly to get the exact value you need, otherwise, your result won't be as expected.