I think it's definitely possible, just that it hasn't been done yet. It's currently not on my roadmap, I want to implement other features first, but the hardware definitely allows it. I think if you want to implement it, FoA (another repo in this same org; fully written in Rust) is probably the place to do so.

If you have knowledge about the Wi-Fi Direct protocols and do not want to wait, you can also implement alot by combining esp_wifi_80211_tx() and the monitoring mode esp_wifi_set_promiscuous_rx_cb etc, see https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/network/esp_wifi.html. Encryption would be anoying to implement, but hopefully not needed for Wi-Fi direct?