Flink: Failed to validate any of the configured directories to monitor
Issue:
Flink Dashboard unable to startup with following error messages:
+ echo 'Starting Flink History Server'
+ exec /opt/cloudera/parcels/FLINK-1.16.2-csa1.11.0.1-cdh7.1.9.0-387-45580652/bin/flink-historyserver start-foreground
Sep 27, 2024 2:21:45 PM com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
org.apache.flink.util.FlinkException: Failed to validate any of the configured directories to monitor.
at org.apache.flink.runtime.webmonitor.history.HistoryServer.<init>(HistoryServer.java:270)
at org.apache.flink.runtime.webmonitor.history.HistoryServer.<init>(HistoryServer.java:190)
at org.apache.flink.runtime.webmonitor.history.HistoryServer$1.call(HistoryServer.java:174)
at org.apache.flink.runtime.webmonitor.history.HistoryServer$1.call(HistoryServer.java:171)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1910)
at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.runtime.webmonitor.history.HistoryServer.main(HistoryServer.java:170)
Caused
Flink unable to find hadoop-conf
and yarn-conf
in this node.
Resolution
Add Role
YARN Gateway
to Flink Dashboard nodes.Make sure
alternatives --list | grep hadoop-conf
pointed to/etc/hadoop/conf.cloudera.yarn