Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Unknown option _type in ElasticsearchLogstashHandler #46667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
musahaidari opened this issue Jun 14, 2022 · 6 comments
Closed

Unknown option _type in ElasticsearchLogstashHandler #46667

musahaidari opened this issue Jun 14, 2022 · 6 comments
Labels
Bug MonologBridge Status: Needs Review

Comments

@musahaidari
Copy link

Symfony version(s) affected

6.0.*

Description

While using ElasticsearchLogstashHandler I get 400 error in my browser/window terminal depending on the order of the handlers in monolog.yml.

Expected behaviour:
The handler should send the log with no error to elastic search.

How to reproduce

While using this log handler with elasticsearch:8.2.2 I get HTTP/1.1 400 Bad Request returned for "http://127.0.0.1:4307/_bulk". in the browser. The following is the full stack trace:

Symfony\Component\HttpClient\Exception\ClientException:
HTTP/1.1 400 Bad Request returned for "http://127.0.0.1:4307/_bulk".

  at /home/musa/codes/modeer/modeer-web-app/vendor/symfony/http-client/Response/CommonResponseTrait.php:174
  at Symfony\Component\HttpClient\Response\CurlResponse->checkStatusCode()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/http-client/Response/TransportResponseTrait.php:69)
  at Symfony\Component\HttpClient\Response\CurlResponse->getHeaders()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/http-client/Response/TransportResponseTrait.php:270)
  at Symfony\Component\HttpClient\Response\CurlResponse::stream()
  at Generator->next()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/http-client/Response/ResponseStream.php:42)
  at Symfony\Component\HttpClient\Response\ResponseStream->next()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/monolog-bridge/Handler/ElasticsearchLogstashHandler.php:151)
  at Symfony\Bridge\Monolog\Handler\ElasticsearchLogstashHandler->wait()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/monolog-bridge/Handler/ElasticsearchLogstashHandler.php:131)
  at Symfony\Bridge\Monolog\Handler\ElasticsearchLogstashHandler->sendToElasticsearch()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/monolog-bridge/Handler/ElasticsearchLogstashHandler.php:76)
  at Symfony\Bridge\Monolog\Handler\ElasticsearchLogstashHandler->handle()
     (/home/musa/codes/modeer/modeer-web-app/vendor/monolog/monolog/src/Monolog/Logger.php:328)
  at Monolog\Logger->addRecord()
     (/home/musa/codes/modeer/modeer-web-app/vendor/monolog/monolog/src/Monolog/Logger.php:514)
  at Monolog\Logger->debug()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/event-dispatcher/Debug/TraceableEventDispatcher.php:312)
  at Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->postProcess()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/event-dispatcher/Debug/TraceableEventDispatcher.php:163)
  at Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->dispatch()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/http-kernel/HttpKernel.php:213)
  at Symfony\Component\HttpKernel\HttpKernel->handleThrowable()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/http-kernel/HttpKernel.php:106)
  at Symfony\Component\HttpKernel\HttpKernel->terminateWithException()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/http-kernel/EventListener/DebugHandlersListener.php:125)
  at Symfony\Component\HttpKernel\EventListener\DebugHandlersListener::Symfony\Component\HttpKernel\EventListener\{closure}()
     (/home/musa/codes/modeer/modeer-web-app/vendor/symfony/error-handler/ErrorHandler.php:541)
  at Symfony\Component\ErrorHandler\ErrorHandler->handleException()

Possible Solution

When sending the same data via CURL I get a better error message:

$ curl -XPOST http://127.0.0.1:4307/_bulk -H "Content-Type: application/json" -d '{"index":{"_index":"monolog", "_type":"_doc"}}
>                                                                                {"@timestamp":"2022-06-13T10:50:31.726357+04:30","@version":1,"host":"musa-XPS-13-9380","message":"Matched route \"{route}\".","type":"application","channel":"request","level":"INFO","monolog_level":200,"context":{"route":"home","route_parameters":{"_route":"home","_controller":"App\\Controller\\HomeController::index"},"request_uri":"http://127.0.0.1:4300/","method":"GET"}}
> '
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400}

This says we are sending an argument which is (no longer) supported, that being _type, if I remove that from the CURL payload everything works fine. Within the code, that is the line vendor/symfony/monolog-bridge/Handler/ElasticsearchLogstashHandler.php:114

Additional Context

No response
@musahaidari musahaidari changed the title Unknown option _doc in ElasticsearchLogstashHandler Unknown option _type in ElasticsearchLogstashHandler Jun 14, 2022
@alamirault
Copy link
Contributor

Indeed sending _type is deprecated since elasticsearch 7.0 and dropped in 8.0 https://www.elastic.co/guide/en/elasticsearch/reference/7.17/removal-of-types.html

I see 2 possibilities:

  • Add elasticsearch version (majorVersion ?) as argument to ElasticsearchLogstashHandler
  • Not sending _type and support only >7.0 elasticsearch. Last 6.X version is EOL since 2022-02-10 https://www.elastic.co/fr/support/eol

WDYT ?

I can make PR for best solution

@GromNaN
Copy link
Member

GromNaN commented Jun 14, 2022

This should be solved by Seldaek/monolog#1662.

What versions of monolog/monolog, symfony/monolog-bundle and elasticsearch/elasticsearch are installed?

@musahaidari
Copy link
Author

Hi,
I have the followings:

"monolog/monolog" =>  "2.4.0"
"symfony/monolog-bundle" => "v3.7.1"
"elasticsearch/elasticsearch" => "^7"

@GromNaN
Copy link
Member

GromNaN commented Jun 15, 2022 •
edited
Loading

You must upgrade "monolog/monolog": "^2.6.0"
https://github.com/Seldaek/monolog/blob/2.x/CHANGELOG.md#260-2022-05-10

"elasticsearch/elasticsearch: "^8" and "symfony/monolog-bundle": "^3.8.0"
symfony/monolog-bundle#438

@lyrixx
Copy link
Member

lyrixx commented Jun 15, 2022 •
edited
Loading

@GromNaN acutally, the handler is in Symfony itself, see https://github.com/symfony/symfony/blob/6.2/src/Symfony/Bridge/Monolog/Handler/ElasticsearchLogstashHandler.php#L120

When I wrote it, The type was still a thing.

I'm on it 👍🏼

@lyrixx lyrixx mentioned this issue Jun 15, 2022
Merged
@GromNaN
Copy link
Member

GromNaN commented Jun 15, 2022

Oh. I got distracted by the other Elasticsearch handler. Thanks for your attention.

@fabpot fabpot closed this as completed Jun 17, 2022
fabpot added a commit that referenced this issue Jun 17, 2022
@fabpot
bug #46684 [MonologBridge] Fixed support of elasticsearch 7.+ in Elas…
aa49d1c 
…ticsearchLogstashHandler (lyrixx)

This PR was merged into the 4.4 branch.

Discussion
----------

[MonologBridge] Fixed support of elasticsearch 7.+ in ElasticsearchLogstashHandler

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #46667
| License       | MIT
| Doc PR        |

Commits
-------

e14cea9 [MonologBridge] Fixed support of elasticsearch 7.+ in ElasticsearchLogstashHandler
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug MonologBridge Status: Needs Review
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@fabpot @GromNaN @lyrixx @xabbuh @musahaidari @alamirault @carsonbot