Проблема с аутентификацией пользователя В Kerberos с keytab

Question

Я пытаюсь интегрировать Kerberos Hadoop с Pinot. И использую указанные ниже конфигурации.

Исполняемые файлы:

      export HADOOP_HOME=/usr/hdp/2.6.3.0-235/hadoop
export HADOOP_VERSION=2.7.3.2.6.3.0-235
export HADOOP_GUAVA_VERSION=11.0.2
export HADOOP_GSON_VERSION=2.2.4
export GC_LOG_LOCATION=/home/hdfs/Pinot/pinotGcLog
export PINOT_VERSION=0.7.1
export PINOT_DISTRIBUTION_DIR=/home/hdfs/Pinot_IMP_FOLDER/apache-pinot-incubating-0.7.1-bin
export HADOOP_CLIENT_OPTS="-Dplugins.dir=${PINOT_DISTRIBUTION_DIR}/plugins -Dlog4j2.configurationFile=${PINOT_DISTRIBUTION_DIR}/conf/pinot-ingestion-job-log4j2.xml"
export SERVER_CONF_DIR=/home/hdfs/Pinot_IMP_FOLDER/apache-pinot-incubating-0.7.1-bin/bin
export ZOOKEEPER_ADDRESS=<ZOOKEEPER_ADDRESS>
export CLASSPATH_PREFIX="${HADOOP_HOME}/hadoop-hdfs/hadoop-hdfs-${HADOOP_VERSION}.jar:${HADOOP_HOME}/hadoop-annotations-${HADOOP_VERSION}.jar:${HADOOP_HOME}/hadoop-auth-${HADOOP_VERSION}.jar:${HADOOP_HOME}/hadoop-common-${HADOOP_VERSION}.jar:${HADOOP_HOME}/lib/guava-${HADOOP_GUAVA_VERSION}.jar:${HADOOP_HOME}/lib/gson-${HADOOP_GSON_VERSION}.jar"
export JAVA_OPTS="-Xms4G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -Xloggc:${GC_LOG_LOCATION}/gc-pinot-server.log"

controller.conf

      controller.data.dir=<fs.defaultFS>/user/hdfs/controller_segment
controller.local.temp.dir=/home/hdfs/Pinot/pinot_tmp/
controller.zk.str=<ZOOKEEPER_ADDRESS>
controller.enable.split.commit=true
controller.access.protocols.http.port=9000
controller.helix.cluster.name=PinotCluster
pinot.controller.storage.factory.class.hdfs=org.apache.pinot.plugin.filesystem.HadoopPinotFS
pinot.controller.storage.factory.hdfs.hadoop.conf.path=/usr/hdp/2.6.3.0-235/hadoop/conf
pinot.controller.segment.fetcher.protocols=file,http,hdfs
pinot.controller.segment.fetcher.hdfs.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.controller.segment.fetcher.hdfs.hadoop.kerberos.principle='[e-mail]'
pinot.controller.segment.fetcher.hdfs.hadoop.kerberos.keytab='/home/hdfs/hdfs.keytab'


    pinot.controller.storage.factory.hdfs.hadoop.kerberos.principle='[e-mail]'
 
   pinot.controller.storage.factory.hdfs.hadoop.kerberos.keytab='/home/hdfs/hdfs.keytab'
controller.vip.port=9000
controller.port=9000
pinot.set.instance.id.to.hostname=true
pinot.server.grpc.enable=true

Ядерная информация:

      kinit -V -k -t /home/hdfs/hdfs.keytab [e-mail]
Using default cache: /tmp/krb5cc_57372
Using principal: [e-mail]
Using keytab: /home/hdfs/hdfs.keytab
Authenticated to Kerberos v5

СООБЩЕНИЕ ОБ ОШИБКЕ:

      END: Invoking TASK controller pipeline for event ResourceConfigChange::15fc3764_TASK for cluster PinotCluster, took 278 ms
START AsyncProcess: TASK::TaskGarbageCollectionStage
END AsyncProcess: TASK::TaskGarbageCollectionStage, took 0 ms
Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Trying to authenticate user '[e-mail]' with keytab '/home/hdfs/hdfs.keytab'..
Could not instantiate file system for class org.apache.pinot.plugin.filesystem.HadoopPinotFS with scheme hdfs
java.lang.RuntimeException: Failed to authenticate user principal ['[e-mail]'] with keytab ['/home/hdfs/hdfs.keytab']
    at org.apache.pinot.plugin.filesystem.HadoopPinotFS.authenticate(HadoopPinotFS.java:258) ~[pinot-hdfs-0.7.1-shaded.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
Caused by: java.io.IOException: Login failure for '[e-mail]' from keytab '/home/hdfs/hdfs.keytab': javax.security.auth.login.LoginException: Unable to obtain password from user

    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:962) ~[pinot-orc-0.7.1-shaded.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
    at org.apache.pinot.plugin.filesystem.HadoopPinotFS.authenticate(HadoopPinotFS.java:254) ~[pinot-hdfs-0.7.1-shaded.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
    ... 15 more
Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user

    at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:901) ~[?:1.8.0_241]
    at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:764) ~[?:1.8.0_241]
    at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617) ~[?:1.8.0_241]
    at org.apache.pinot.plugin.filesystem.HadoopPinotFS.authenticate(HadoopPinotFS.java:254) ~[pinot-hdfs-0.7.1-shaded.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
    ... 15 more
Failed to start a Pinot [CONTROLLER] at 21.954 since launch
java.lang.RuntimeException: java.lang.RuntimeException: Failed to authenticate user principal ['[e-mail]'] with keytab ['/home/hdfs/hdfs.keytab']
    at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:58) ~[pinot-all-0.7.1-jar-with-dependencies.jar:0.7.1-e22be7c3a39e840321d3658e7505f21768b228d6]
   

PS Я выполняю этого пользователя hdfs, а для файла keytab также пользователь hdfs. Я также дал 777 доступ к файлу hdfs.keytab.

Кто-то Пожалуйста, подскажите, в чем проблема. Я прочитал несколько блоков и везде обнаружил, что это из-за неправильной комбинации файлов prinicpal / keytab / у пользователя нет доступа / дайте 777 доступ к файлу / попробуйте с другим пользователем. попробовал все варианты, но пока ничего не работало.

Answer 1

Теперь это сработало. Я только что удалил 'из keytab и названия принципа. не удалось прочитать keytab с '.

Старая конфигурация:

pinot.controller.segment.fetcher.hdfs.hadoop.kerberos.principle = ' [e-mail]'pinot.controller.segment.fetcher.hdfs.hadoop.kerberos.keytab='/home / hdfs / hdfs.keytab' пино .controller.storage.factory.hdfs.hadoop.kerberos.principle ='[e-mail]' pinot.controller.storage.factory.hdfs.hadoop.kerberos.keytab = '/ home / hdfs / hdfs.keytab'

Новая конфигурация:

pinot .controller.segment.fetcher.hdfs.hadoop.kerberos.principle = hdfs @HDFSSITHDP.COMpinot.controller.segment.fetcher.hdfs.hadoop.kerberos.keytab= / home / hdfs / hdfs.keytab pinot .controller.storage .factory.hdfs.hadoop.kerberos.principle = hdfs @HDFSSITHDP.COM pinot.controller.storage.factory.hdfs.hadoop.kerberos.keytab = / home / hdfs / hdfs.keytab