'IoT Core OpenSearch Action Rule "mapper_parsing_exception"

I'm using the OpenSearch 1.2 version deployed on AWS.

I was following this AWS tutorial but with my own sensor data. I've created the following IoT Core Rule OpenSearch Action:

OpenSearchTopicRule:
    Type: AWS::IoT::TopicRule
    Properties:
      TopicRulePayload:
        Actions:
          - OpenSearch:
              Endpoint: !Join ['', ['https://', !GetAtt OpenSearchServiceDomain.DomainEndpoint]]
              Id: '${newuuid()}'
              Index: sensors
              RoleArn: !GetAtt IoTOSActionRole.Arn
              Type: sensor_data
        Sql: SELECT *, timestamp() as ts FROM 'Greenhouse/+/Sensor/Status'

The IoTOSActionRole has propper es:ESHttpPut permission. But when I try to create an index with following command send from Postman that would match the Type: sensor_data attribute:

curl --location --request PUT 'https://search-iot***-avt***i.eu-west-1.es.amazonaws.com/sensors' \
--header 'Content-Type: application/json' \
--data-raw '{
  "mappings": {
    "sensor_data": {
      "properties": {
        "ts": { "type": "long",
          "copy_to": "datetime"},
        "datetime": {"type": "date",
          "store": true},
        "deviceID": {"type": "text",
          "store": true},
        "humidity": {"type": "integer",
          "store": true},
        "temperature": {"type": "integer",
          "store": true},
        "lux": {"type": "integer",
          "store": true},
        "soil": {"type": "integer",
          "store": true}
}}}'

I receive an error:

{
    "error": {
        "root_cause": [
            {
                "type": "mapper_parsing_exception",
                "reason": "Root mapping definition has unsupported parameters:  [sensor_data : {properties={datetime={store=true, type=date}, temperature={store=true, type=integer}, humidity={store=true, type=integer}, soil={store=true, type=integer}, deviceID={store=true, type=text}, lux={store=true, type=integer}, ts={copy_to=datetime, type=long}}}]"
            }
        ],
        "type": "mapper_parsing_exception",
        "reason": "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [...]",
        "caused_by": {
            "type": "mapper_parsing_exception",
            "reason": "Root mapping definition has unsupported parameters:  [...}]"
        }
    },
    "status": 400
}

I've tried removing the 'type' "sensor_data" attribute as mentioned here (but that's ElasticSearch solution) and that allowed me to create an index with that mapping,

{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "sensors"
}

and then index pattern in OpenSearch Dashboard, but what happens then is that the IoT Core Rule even though it gets triggered does not result in any data ingestion to the OpenSearch domain. So I guess IoT Core Action tries to send that data with type sensor_databut there's no corresponding type in OS. Additionally, when I open the Discovery tab in OS dashboard I get this notice:

"undefined" is not a configured index pattern ID

Showing the default index pattern: "sensors*" (d97775d0-***a2fd725)

Sample data:

{
  "deviceID": "Tomatoes",
  "Greenhouse": 1,
  "date": "05-05",
  "time": "09:35:39",
  "timestamp": 1651743339,
  "humidity": 60,
  "temperature": 33.3,
  "lux": 9133.333,
  "soil": 78
}

What PUT call I have to make to create a sensor_data type mapping in OS that would match the type specified in IoT Core OpenSearch Action Rule?

UPDATE I've tried the same API call with Elasticsearch_7.10 and received the same mapper_parsing_exception response. I've tried removing the "store": true attribute. The only call accepted is the one omitting the sensor_data type attribute to the .../sensors/_mappings API.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source