flink1.7.2如何进行hdfs的kerberos认证

[ruochen]

   public static void main(String[] args) throws Exception {
        final ParameterTool parameterTool = ParameterTool
                .fromPropertiesFile(BizlogStreamWithEventTimeCleaner.class.getResourceAsStream(PROPERTIES_FILE_NAME))
                .mergeWith(ParameterTool.fromArgs(args));
        String resultTable = parameterTool.get(BIZLOG_RESULT_TABLENAME);
        //设置窗口大小
        Time windowSize = Time.milliseconds(parameterTool.getLong(BIZLOG_WINDOW_SIZE));
        //设置数据最大乱序时间
        Time maxOutOfOrder = Time.milliseconds(parameterTool.getLong(BIZLOG_WINDOW_MAXOUTOFORDE));
        hdfsAuthenticate(parameterTool);
        ...
    }
    private static void hdfsAuthenticate(ParameterTool parameterTool) throws IOException {
        String kerberosConfFile = BizlogStreamWithEventTimeCleaner.class.getClassLoader().getResource("krb5.conf").getPath();
        System.setProperty("java.security.krb5.conf", kerberosConfFile);
        Configuration conf = new Configuration();
        conf.set(HADOOP_SECURITY_AUTHENTICATION, "kerberos");
        //加载hadoop配置文件
        String principal = parameterTool.get("kerberos.princpal");
        String keytabName = parameterTool.get("keytab.name");
        String keytabPath = BizlogStreamWithEventTimeCleaner.class.getClassLoader().getResource(keytabName).getPath();
        UserGroupInformation.setConfiguration(conf);
        UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
    }


代码如上，在idea中直接运行可以认证通过，但是打成jar包提交到集群后报错如下：
Caused by: java.io.IOException: Login failure for biuri/[hidden email] from keytab file:/data/realtime-flink.jar!/kerberos.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user
这个是什么原因？或者应该如何进行正确的集群认证？

[Jeff Zhang]

See
https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/config.html#kerberos-based-security


杨文生-java开发 <[hidden email]> 于2019年8月27日周二 下午3:30写道：

>
>
>    public static void main(String[] args) throws Exception {
>         final ParameterTool parameterTool = ParameterTool
>
> .fromPropertiesFile(BizlogStreamWithEventTimeCleaner.class.getResourceAsStream(PROPERTIES_FILE_NAME))
>
>                 .mergeWith(ParameterTool.fromArgs(args));
>         String resultTable = parameterTool.get(BIZLOG_RESULT_TABLENAME);
>         //设置窗口大小
>         Time windowSize =
> Time.milliseconds(parameterTool.getLong(BIZLOG_WINDOW_SIZE));
>         //设置数据最大乱序时间
>         Time maxOutOfOrder =
> Time.milliseconds(parameterTool.getLong(BIZLOG_WINDOW_MAXOUTOFORDE));
>         hdfsAuthenticate(parameterTool);
>         ...
>     }
>     private static void hdfsAuthenticate(ParameterTool parameterTool)
> throws IOException {
>         String kerberosConfFile =
> BizlogStreamWithEventTimeCleaner.class.getClassLoader().getResource("krb5.conf").getPath();
>
>         System.setProperty("java.security.krb5.conf", kerberosConfFile);
>         Configuration conf = new Configuration();
>         conf.set(HADOOP_SECURITY_AUTHENTICATION, "kerberos");
>         //加载hadoop配置文件
>         String principal = parameterTool.get("kerberos.princpal");
>         String keytabName = parameterTool.get("keytab.name");
>         String keytabPath =
> BizlogStreamWithEventTimeCleaner.class.getClassLoader().getResource(keytabName).getPath();
>
>         UserGroupInformation.setConfiguration(conf);
>         UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
>     }
>
>
> 代码如上，在idea中直接运行可以认证通过，但是打成jar包提交到集群后报错如下：
> Caused by: java.io.IOException: Login failure for biuri/
> [hidden email] from keytab
> file:/data/realtime-flink.jar!/kerberos.keytab:
> javax.security.auth.login.LoginException: Unable to obtain password from
> user
> 这个是什么原因？或者应该如何进行正确的集群认证？
>
>
>
>

--
Best Regards

Jeff Zhang