Hi @Pablo1816 , as a workaround I would recommend you to use excelastic to import to a temporary index, and then use reindex API using a pipeline with the desired transformation to a proper index.

POST _reindex
{
  "source": {
    "index": "temp_index"
  },
  "dest": {
    "index": "real_index",
    "pipeline": "my_pipeline"
  }
}

@codingchili Bulk API allows to include a pipeline as well, similarly to giving the index name to bulk.

Here is a sample creating a pipeline that uppercase my field "country" and transforming client_ip from text to geoip:

PUT /_ingest/pipeline/my-country-uppercase
{
  "processors": [
    {
      "uppercase": {
        "field": "country"
      }
    },
    {
      "geoip": {
        "field": "client_ip"
      }
    }
  ]
}

Inserting recording with bulk saying to use my-country-uppercase pipeline:

POST _bulk
{"index" : { "_index" : "my_index", "_id" : "1" , "pipeline": "my-country-uppercase"}}
{"country" : "Brazil", "client_ip": "177.92.205.77" }

Searching index:

GET my_index/_search

Result:

{
  "took" : 340,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "my_index",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "country" : "BRAZIL",
          "geoip" : {
            "continent_name" : "South America",
            "region_iso_code" : "BR-MG",
            "city_name" : "Poços de Caldas",
            "country_iso_code" : "BR",
            "region_name" : "Minas Gerais",
            "location" : {
              "lon" : -46.5646,
              "lat" : -21.7869
            }
          },
          "client_ip" : "177.92.205.77"
        }
      }
    ]
  }
}

I will try to work on this enhancement request.

Pipeline #83

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions