Подключение к кластеру Kerberized hadoop с помощью модуля Python Impyla

Question

Я использую модуль Impyla для подключения к Kerberized Hadoop кластера. Я хочу получить доступ
hiveserver2/hive, но я получаю следующую ошибку:

test_conn.py

from impala.dbapi import connect
import os
connection_string = 'hdp296m1.XXX.XXX.com'
conn = connect(host=connection_string, port=21050,auth_mechanism="GSSAPI",kerberos_service_name='[e-mail]',password='testuser')
cursor = conn.cursor()
cursor.execute('select count(*) form t_all_types_simple_t')
print cursor.description
results = cursor.fetchall()

Трассировки стека:

[vagrant@localhost vagrant]$ python test_conn.py
Traceback (most recent call last):
  File "test_conn.py", line 4, in <module>
    conn = connect(host=connection_string, port=21050, auth_mechanism="GSSAPI",kerberos_service_name='[e-mail]',password='testuser')
  File "/usr/lib/python2.7/site-packages/impala/dbapi.py", line 147, in connect
    auth_mechanism=auth_mechanism)
  File "/usr/lib/python2.7/site-packages/impala/hiveserver2.py", line 758, in connect
    transport.open()
  File "/usr/lib/python2.7/site-packages/thrift_sasl/__init__.py", line 61, in open
    self._trans.open()
  File "/usr/lib64/python2.7/site-packages/thrift/transport/TSocket.py", line 101, in open
    message=message)
thrift.transport.TTransport.TTransportException: Could not connect to hdp296m1.XXX.XXX.com:21050

testuser - мой принципал Kerberos, который я буду использовать для создания kinit.

Answer 1

Похоже, ваше соединение неверно. Попробуйте,

from impala.dbapi import *
import sys, os
# set your parms
host=os.environ.get("CDH_HIVE",'x.x.x.x')
port=os.environ.get("CDH_HIVE_port",'10000')
auth_mechanism=os.environ.get("CDH_auth",'GSSAPI')
user='hive' 
db='mydb' 
# No password use kinit 
password=''
# hive is principal with krb
kbservice='hive'  

class Hive:

    def __init__(self,db):
        self.database=db
        self.__conn = connect(host=host,
                            port=port,
                            auth_mechanism=auth_mechanism,
                            user=user,
                            password=password,
                            database=db,
                            kerberos_service_name=kbservice
                            )


        self.__cursor = self.__conn.cursor()


h = Hive(db)