This setup is based on the fact that Caddy will handle the TLS on your behalf. In case you're experiencing ssl related issues, you might want to patch your ingress-nginx-controller to activate the enable-ssl-passthrough flag.
In case you are running it locally with a non reachable FQDN, you might want to consider adding tls internal on the Caddy config for self-signed certificate.