'jq: Select using 2x seperate-level keys

Using jq on JSON data ...

I'm trying to select records where FromPort is 80 and CidrIp is 0.0.0.0/0 within the same element of the record.

Here is the query:

cat data.json |jq -r '.SecurityGroups | .[] | select((.IpPermissions[] | .IpRanges[] | .CidrIp == "0.0.0.0/0") and (.IpPermissions[] | .FromPort == 80))'

The data is below.

The results should only select Group3 but instead selects Group1 and Group3. I've tried various google searches but am unable to find the answer yet.

Very grateful for any help.

Data (data.json):

{
    "SecurityGroups": [
        {
            "GroupName": "Group1",
            "IpPermissions": [
                {
                    "FromPort": 80,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "1.2.3.4/24"
                        }
                    ]
                },
                {
                    "FromPort": 6789,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "1.2.3.4/24"
                        },
                        {
                            "CidrIp": "0.0.0.0/0"
                        }
                    ]
                }
            ]
        },
        {
            "GroupName": "Group2",
            "IpPermissions": [
                {
                    "FromPort": 443,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "1.2.3.4/24"
                        },
                        {
                            "CidrIp": "0.0.0.0/0"
                        }
                    ]
                },
                {
                    "FromPort": 6788,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "1.2.3.4/24"
                        }
                    ]
                }
            ]
        },
        {
            "GroupName": "Group3",
            "IpPermissions": [
                {
                    "FromPort": 80,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "1.2.3.4/24"
                        },
                        {
                            "CidrIp": "0.0.0.0/0"
                        }
                    ]
                },
                {
                    "FromPort": 6789,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "1.2.3.4/24"
                        }
                    ]
                }
            ]
        }
    ]
}

Solution 1:^[1]

The iteration inside your selection .IpPermissions[] | .IpRanges[] produces as many results as there are items in those arrays. For each true outcome, your select passes on the input in question (maybe several times). You need an aggregator like any or all to combine all the outcomes to one boolean result, so that select can pass on just either 1 or 0 copies of the corresponding input.

Furthermore, you iterate over .IpPermissions[] on both sides of the and in your condition which will produce a cartesian product of the outcomes.

Here is a solution that selects a group if there is at least one item in the .IpPermissions array that has a match for .FromPort and at least one match for .CidrIp in the .IpRanges array:

jq '
  .SecurityGroups[] | select(
    any(.IpPermissions[]; .FromPort == 80 and
      any(.IpRanges[]; .CidrIp == "0.0.0.0/0")
    )
  )
' data.json

{
  "GroupName": "Group3",
  "IpPermissions": [
    {
      "FromPort": 80,
      "IpProtocol": "tcp",
      "IpRanges": [
        {
          "CidrIp": "1.2.3.4/24"
        },
        {
          "CidrIp": "0.0.0.0/0"
        }
      ]
    },
    {
      "FromPort": 6789,
      "IpProtocol": "tcp",
      "IpRanges": [
        {
          "CidrIp": "1.2.3.4/24"
        }
      ]
    }
  ]
}

Demo

Solution 2:^[2]

You can use parenthesis () and and appropriately to filter like this: (.FromPort==80 and (.IpRanges[] | .CidrIp=="0.0.0.0/0" )). By doing this, we make sure that both conditions are matched by an entry in IpPermissions.

Filter

.SecurityGroups | map(select(.IpPermissions[] 
| (.FromPort==80 and (.IpRanges[] | .CidrIp=="0.0.0.0/0" ))))

Input

{
  "SecurityGroups": [
    {
      "GroupName": "Group1",
      "IpPermissions": [
        {
          "FromPort": 80,
          "IpProtocol": "tcp",
          "IpRanges": [
            {
              "CidrIp": "1.2.3.4/24"
            }
          ]
        },
        {
          "FromPort": 6789,
          "IpProtocol": "tcp",
          "IpRanges": [
            {
              "CidrIp": "1.2.3.4/24"
            },
            {
              "CidrIp": "0.0.0.0/0"
            }
          ]
        }
      ]
    },
    {
      "GroupName": "Group2",
      "IpPermissions": [
        {
          "FromPort": 443,
          "IpProtocol": "tcp",
          "IpRanges": [
            {
              "CidrIp": "1.2.3.4/24"
            },
            {
              "CidrIp": "0.0.0.0/0"
            }
          ]
        },
        {
          "FromPort": 6788,
          "IpProtocol": "tcp",
          "IpRanges": [
            {
              "CidrIp": "1.2.3.4/24"
            }
          ]
        }
      ]
    },
    {
      "GroupName": "Group3",
      "IpPermissions": [
        {
          "FromPort": 80,
          "IpProtocol": "tcp",
          "IpRanges": [
            {
              "CidrIp": "1.2.3.4/24"
            },
            {
              "CidrIp": "0.0.0.0/0"
            }
          ]
        },
        {
          "FromPort": 6789,
          "IpProtocol": "tcp",
          "IpRanges": [
            {
              "CidrIp": "1.2.3.4/24"
            }
          ]
        }
      ]
    }
  ]
}

Output

[
  {
    "GroupName": "Group3",
    "IpPermissions": [
      {
        "FromPort": 80,
        "IpProtocol": "tcp",
        "IpRanges": [
          {
            "CidrIp": "1.2.3.4/24"
          },
          {
            "CidrIp": "0.0.0.0/0"
          }
        ]
      },
      {
        "FromPort": 6789,
        "IpProtocol": "tcp",
        "IpRanges": [
          {
            "CidrIp": "1.2.3.4/24"
          }
        ]
      }
    ]
  }
]

Now you can see only Group3 is outputted.

Demo

https://jqplay.org/s/qZvARp_5LE

Or you could use any/2 as pmf suggested.

Filter

.SecurityGroups[] 
| select(any(.IpPermissions[]; .FromPort==80 
and any(.IpRanges[]; .CidrIp=="0.0.0.0/0")))

Demo

https://jqplay.org/s/Rw2_dtzwUA

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	pmf
Solution 2

'jq: Select using 2x seperate-level keys

Solution 1:[1]

Solution 2:[2]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]