Count 3 types of delimiters (comma, semicolon, pipe) from different files

Question

I am trying to develop the similar thing. But my requirement is little bit different. I would like to count the number of delimiters in the 1st row and the 2nd row. If delimiters from both the rows are matching then fine otherwise the file need to be moved to the reject folder. Below is the script. But here I have a question, how do I count for 3 different delimiters in different files. For example I have comma, semicolon and pipe delimiters. In the below script in sed command how to check for 3 types of delimiters at the same time?

pathname=/opt/interfaces/sample_check/mvfiles/inbox

findresult=$(find $pathname -type f ( -name "messagemulti.csv" -or -name "messagesemi.txt" -or -name "comma2.txt" -or -name "messagepipe.txt" -or -name "tokkalodi.txt" -or -name "ADMC_POSITION-LT3213.csv" -or -name "DMC_CASHFLOW248.csv" -or -name "ADMC_EQBASKET-WEIGHTS_52387.csv" -or -name "ADMC_POSITION-DDD7.csv" -or -name "ADMC_POSITION-DDD7.csv" ))

Count=sed -n 1p $findresult | tr ',' '\n' | wc -l
Count2=sed -n 2p $findresult | tr ',' '\n' | wc -l

echo $Count echo $Count2

if [ $Count != $Count2 ]
then echo "Mis Match"
  mv $findresult /opt/interfaces/sample_check/mvfiles/reject
else echo "Match"
  exit
fi

Answer 1

If none of the delimiters are otherwise parts of the two first lines, you could use this generic GNU awk script:

parse.awk

BEGIN  { FS="[,;|]" }            # Set Field Separators
FNR==1 { count = NF }            # Remember Number of Fields from the first line
FNR==2 { 
  if(count != NF)                # If second line has the same number of fields
    print "Mismatch: " FILENAME  # Report the mismatched file
  nextfile                       # Skip to next input file
}

Run it like this:

awk -f parse.awk infile1 infile2 ...

This will output any files with a count mismatch.

Answer 2

I've just created a file, containing just commas, semi-colons and pipe characters:

echo ",,,;;;;|||||" >test.txt

Then I ran following command:

echo $(($(grep -o "," test.txt | wc -l) + $(grep -o ";" test.txt | wc -l) + $(grep -o "|" test.txt | wc -l)))

The result was 12, as expected.

Answer 3

I found something below to include all the delimiters at the same time. Please check if that is correct.

pathname=/opt/interfaces/sample_check/mvfiles/inbox

findresult=$(find $pathname -type f ( -name "messagemulti.csv" -or -name "messagesemi.txt" -or -name "comma2.txt" -or -name "messagepipe.txt" -or -name "tokkalodi.txt" -or -name "ADMC_POSITION-LT3213.csv" -or -name "DMC_CASHFLOW248.csv" -or -name "ADMC_EQBASKET-WEIGHTS_52387.csv" -or -name "ADMC_POSITION-DDD7.csv" -or -name "ADMC_POSITION-DDD7.csv" ))

Count=sed -n 1p $findresult | tr '[,;|]' '\n' | wc -l
Count2=sed -n 2p $findresult | tr '[,;|]' '\n' | wc -l

echo $Count echo $Count2

if [ $Count != $Count2 ]
then echo "Mis Match"
mv $findresult /opt/interfaces/sample_check/mvfiles/reject
else echo "Match"
exit
fi

Answer 4

You can count the delimiters by moviing them to a new line, replace by ';', and count them:

sed '2 s/[|,;]/\n;/g' ${findresult} | grep -c ';'

When you only want to know which have the same number, you do not need to count them.
Delete all other characters and replace any remaining character with a dot.

if [[ $(sed '1s/[^|,;]//g;s/././g' ${findresult}) != 
      $(sed '2s/[^|,;]//g;s/././g' ${findresult}) ]]; then

Answer 5

on bash shell type

i=
pathname=/opt/interfaces/sample_check/mvfiles/inbox
for file in messagemulti.csv messagesemi.txt comma2.txt messagepipe.txt tokkalodi.txt ADMC_POSITION-LT3213.csv DMC_CASHFLOW248.csv ADMC_EQBASKET-WEIGHTS_52387.csv ADMC_POSITION-DDD7.csv ADMC_POSITION-DDD7.csv
{
while read -r l ;do s=${l//[!,;\|]}; c=${#s}; ((++i==1))&&d=$c; if((i==2));then ((d==c)) ||mv -v $file $pathname/$file ;fi ;done<$file
}