how to clear map in awk?

Question

I have a file a.txt. There are many numbers in each line in a.txt:

1 2 3 1 2
5 6 7 7
19 20 20

I have to print each line without duplicate numbers, just like following results:

1 2 3
5 6 7
19 20

I use map in awk, but must clear map each time. I don't know how to clear map in awk when reading each line.

awk '{ split($0, arr, " "); \
for(i=1;i<=length(arr);i++){dup_map[arr[i]]=1;} \
for(num in dup_map){printf("%s ", num);} printf("\n"); clear dup_map; } \
}' a.txt

Could someone tell me how to clear map in awk ?

Answer 1

gawk has a function delete for this:

delete array

In awk this can be achieved by

split("", array)

The split function (see section Built-in Functions for String Manipulation) clears out the target array first. This call asks it to split apart the null string. Since there is no data to split out, the function simply clears the array and then returns.

Both options mentioned in the gawk online manual.

Answer 2

This is THE right way to do what you want:

$ awk '{
    delete(seen)
    for ( i=1; i<=NF; i++ ) {
        if ( !seen[$i]++ ) {
            printf "%s%s", (i>1 ? OFS : ""), $i
        }
    }
    print ""
}' file
1 2 3
5 6 7
19 20

Note the idiomatic use of an array named seen to keep track of which $i values have been seen before and that the fields will be printed in the order they occurred (you have other answers that will randomize their order by use of the in operator) and that there will be no trailing blank char printed at the end of each line (you have answers that do that too).

Answer 3

awk '{split("",M);for(i=1;i<=NF;i++)if($i in M)$i="";else M[$i]++;$0=$0}7' YourFile

Notes:

awk '# for non empty lines
   /./ {
      # reset array Map
      split( "", Map)
      # for each field (separtor is space)
      for( i=1; i<=NF ;i++) {
         # if the field content is in map, set it to empty string
         # if not, add it to the map
         if( $i in Map) $i=""
          else Map[ $i]++
         }
      # rewrite the line for single separator (not mandatory)
      $0=$0
      }
   # print the resulting line (default action)
   7 { print }
   ' YourFile

Answer 4

note that you don't need the initial splitting, since awk already splits fields for you,

$ awk '{delete a; 
        for(i=1;i<=NF;i++) a[$i]; 
        for(k in a) printf "%s ", k; 
        print ""}' file

1 2 3
5 6 7
19 20

or, with deleting entries instead of the array

$ awk '{for(i=1;i<=NF;i++) a[$i]; 
        for(k in a) 
          {printf "%s ", k; delete a[k]} 
        print ""}' file