Using Connect-standalone in Kafka with Kerberos cluster

Kafka Connect is a tool for scalably and reliably streaming data between Apache Kafka and other systems. It can also be used in secured Kafka environment. In Kerberixed Kafka installation also you can use the Kafka Connect utilities. 
You just have to provide the Java security config and Kerberos config as parameters to connect utils. 
Below command shows how can you specify the security properties to the connect-standalone.sh.
The command assumes that you are in the Kafka installation directory.


bin/connect-standalone.sh connect_standalone.properties source.properties -Djava.security.auth.login.config=kafka-jaas.config -Djava.security.krb5.conf=krb5.conf

Here the kafka-jass.config should specify the file path which contains the KafkaClient properties like below.


  KafkaClient {
 com.sun.security.auth.module.Krb5LoginModule required
 useKeyTab=true
 keyTab=""
 storeKey=true
 useTicketCache=false
 serviceName="kafka"
 principal="";
};

And the krb5.conf should detail about the KDC server property. Sample shown below.
  
        [libdefaults]
 renew_lifetime = 7d
 forwardable = true
 default_realm = example.com
 ticket_lifetime = 24h
 dns_lookup_realm = false
 dns_lookup_kdc = false
 default_ccache_name = /tmp/krb5cc_%{uid}
 #default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
 #default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5

 [realms]
 example.com = {
 master_kdc = my.master.kdc.com
 admin_server = my.admin.com
 kdc = my.kdc.com
 }
If above command does not work of if Kafka connect is not able to use the kerberos config, then update the Kafka-run-class.sh file to pass these security parameters to exec command at the end "-Djava.security.auth.login.config=kafka-jaas.config -Djava.security.krb5.conf=krb5.conf"

No comments:

Post a Comment

Golang: Http POST Request with JSON Body example

Go standard library comes with "net/http" package which has excellent support for HTTP Client and Server.   In order to post JSON ...