On my web server, I’m running the Spark UI and have exported the following SPARK_HISTORY_OPTS:
export SPARK_HISTORY_OPTS="-Dspark.history.fs.logDirectory=${LOCAL_PATH_LOGS}
-Dspark.history.ui.port=18080
-Dspark.eventLog.enabled=true
-Dspark.ui.proxyRedirectUri=${SERVER_URL}"
./start-history-server.sh
The user interface is operational at https://$SERVER_URL/proxy/18080/.
The Executors tab for one of the programmes, however, is empty when I try to access it.
I can see that the API request /allexecutors gets a 404 because the URL is incorrectly formed by debugging from the browser. I see that the URL called is https://$SERVER_URL/proxy/18080/api/v1/applications/18080/allexecutors rather than https://$SERVER_URL/proxy/$APP_ID/allexecutors.
I’m not sure if this has anything to do with the issue. Although I checked the material, I am unable to resolve this issue. Would you kindly assist me? I’m grateful.