tag:blogger.com,1999:blog-58933839819895241592024-03-23T21:54:04.287+05:30Hints LibraryI write to discover what I know. (Flannery O'Connor)Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.comBlogger178125tag:blogger.com,1999:blog-5893383981989524159.post-31233661931493292082021-03-16T09:31:00.004+05:302021-03-16T09:31:58.506+05:30Golang: Http POST Request with JSON Body exampleGo standard library comes with "net/http" package which has excellent support for HTTP Client and Server. In order to post JSON body during post request, we need to convert the data to byte array format and send it along with the request.You can convert the JSON to a byte array using "encoding/json" package. Then use the NewBuffer method to pass this byte array to the post method.
Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-63893046267365773542021-03-15T07:22:00.000+05:302021-03-15T07:22:12.187+05:30Golang: Http Get Request exampleGo standard library comes with "net/http" package which has excellent support for HTTP Client and Server. In this example we will be using "http.Get" method to execute Http Get method. We will convert the response to String and print it.
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func main(){
//Use get method to hit the rest API
resp, err := http.Get("https://Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-5547317022123231402021-03-14T13:25:00.006+05:302021-03-14T13:25:59.341+05:30Golang: Reading environment variablesUsually, environment variables are used to provide the configuration information for the program. You can set the environment outside of the program and you can assess these during execution. Go provides the simplest mechanism to access the environment variables. You can use 'OS' package to get or set the environment variables.
package main
import (
"fmt"
"os"
"strings"
)
func Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-47754651365210420132021-03-02T11:06:00.001+05:302021-03-02T11:06:15.471+05:30Golang: Write data to fileWriting data to file is the most common use case while writing a program. Here we are going to see multiple ways of writing data to file.
Write using ioutil.WriteFile
You can directly write data to file using ioutil.WriteFile" method. It will take care of creating file if the file does not exists.
package main
import (
"fmt"
"io/ioutil"
)
func main() {
writeStringToFile("Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-51700499571777980322021-02-28T20:55:00.002+05:302021-03-01T07:37:15.734+05:30Golang: Read data from File ( part II)In the last post, we have seen how we can read data from the file line by line. Sometimes if the file size small then you can read the whole file in one go. In this example, we will be reading the whole file in one go. We will be using 'ReadFile' from 'ioutil' lib to read the file content.
package main
import (
"fmt"
"io/ioutil"
)
func main() {
readFile("Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-60065246647117096982021-02-28T20:20:00.004+05:302021-03-02T09:02:41.311+05:30Golang: Read data from file line by line Reading data from files is a common use case. Most of the time you need to read data line by line.There are multiple ways to read data from the file. The simplest way is to open a file and use a scanner to read data line by line. The sample code below does the same.
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
readLineByLine("file_path")
}
func readLineByLine(Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-9530955787466859212021-02-27T19:15:00.003+05:302021-03-03T18:25:43.997+05:30Golang: How to know IP Address and HostnameSometimes you want to know the hostname and IP address of the machine you are running the program on. We can get the hostname using the OS package.For IP address we can use the Net package. You can read all IP addresses using InterfaceAddrs, you need to filter out the loopback address to get real IP.
package main
import (
"fmt"
"net"
"os"
)
func main() {
//Reading the hostname Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-11328620684683302032021-02-25T12:25:00.002+05:302021-03-01T09:18:08.330+05:30Golang: Check if file exists in go langIt is easy to check if the file with the given path exists or not in Go lang. You can use 'Stat' method from 'os'. Stat returns a FileInfo describing the named file. So if there is no error then you can safely assume the file exists.
import (
"fmt"
"os"
)
func main() {
if _, err := os.Stat("file.path"); err == nil {
fmt.Println("File with given path exists.")
Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-45640275841433526802020-10-30T09:33:00.000+05:302020-10-30T11:22:52.105+05:30Fix: Buffer overflow when available data size...This is an unusual problem you might face while consuming data from Kafka broker over SSL.
java.lang.IllegalStateException: Buffer overflow when available data size (16384) >= application buffer size (16384)
at org.apache.kafka.common.network.SslTransportLayer.read(SslTransportLayer.java:592)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:95)
at Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-35273038389271441962020-10-30T09:24:00.004+05:302020-10-30T09:24:19.479+05:30Fix: Python requests.get(url) times out but works in browserIf Python requests.get fails with a timeout and the same URL works in the browser then you will need to simulate the browser request as is while invoking requests.get. Usually, this will get resolved if you put the "user-agent" header, which tells the server which browser is requesting.
session = requests.Session()
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-38869383148777450472020-10-28T15:14:00.007+05:302020-10-28T17:55:27.189+05:30Most Useful Linux/Unix Commands Here is the list of commonly used commands on Unix/Linux systems.1. Split File based on number of lines
If you want to split text file "abc.txt" containing 50 lines by each line use the below command
csplit abc.txt 1 {49}
2. Find out Top 10 Memory intensive processes
If you want to figure out which are the top 10 memory consuming processes use the command below
ps -eo Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-17201582835533312232020-10-28T11:05:00.003+05:302020-10-28T13:41:49.335+05:30Kafka Performance BenchmarkingApache Kafka comes with the kafka-*-perf-test tool, which can be used to benchmark the performance of the Kafka cluster. This can be used as the first level of benchmarking tool before you run your production load tests. Kafka Producer TestYou can use kafka-producer-perf-test to measure the Kafka write performance. The below command demonstrates running the Kafka producer test for 100KUmesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-65706276001169375132019-05-08T22:18:00.002+05:302019-05-08T22:20:26.790+05:30Error: Zip64 archives are not supported by springframeworkSpringboot application may fail to start with error "Zip64 archives are not supported" with giving you stack trace as below.
Caused by: java.lang.IllegalStateException: Zip64 archives are not supported
at org.springframework.boot.loader.jar.CentralDirectoryEndRecord.getNumberOfRecords(CentralDirectoryEndRecord.java:124)
at org.springframework.boot.loader.jar.JarFileEntries.visitStart(Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-11253972843225114702018-09-09T22:54:00.000+05:302018-09-09T22:54:13.984+05:30Quickest way to launch camera app on Samsung Galaxy Note 9Camera is the mostly used app on smart phones, and its frequently required app. Having quick shortcut to launch the camera is always helpful.
Samsung Galaxy Note 9 offers the always on display mode, where it allows to enable on screen home button. Always on display also allows to set camera as a app to launch when double tap on home button.
To enable this go to Setting -> Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-86580203619155927292018-09-03T09:47:00.000+05:302018-09-03T09:47:02.027+05:30Enable/Disable always on Display on Samsung Galaxy Note 9If you like to save your battery more, you can turn off the always on display on your Samsung Galaxy Note 9.
To do so, go to Settings -> Display. Now on display settings scroll down to the end, you will find the always on Display setting.
Other option is go to Settings. Now using search bar search for "Always on".
Here enable/disable always on display by using the Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-8823715085695935092018-09-02T14:51:00.000+05:302018-09-02T14:51:21.422+05:30Using Connect-standalone in Kafka with Kerberos cluster Kafka Connect is a tool for scalably and reliably streaming data between Apache Kafka and other systems. It can also be used in secured Kafka environment. In Kerberixed Kafka installation also you can use the Kafka Connect utilities.
You just have to provide the Java security config and Kerberos config as parameters to connect utils.
Below command shows how can you specify the Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-59862903827622227592018-01-04T20:07:00.000+05:302018-01-04T20:07:05.013+05:30How to avoid host key checking during ansible script executionDuring execution of ansible playbook or ansible script on new remote host (which you have not connected via ssh so far) the script might fail with below error
UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ----}
To avoid this you can set the environment variable (ANSIBLE_HOST_KEY_CHECKING) to disable host check before ssh during ansible script execution.
Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-9740131046140597582018-01-01T11:05:00.000+05:302018-01-01T11:05:40.321+05:30HMaster not starting up, error "java.io.IOException: Filesystem closed"HMaster fails to start with below error
util.FSHDFSUtils: attempt=0 on file=hdfs://krbhdfs/apps/hbase/data/MasterProcWALs/state-00000000000000000001.log after 1ms java.io.IOException: Filesystem closed.
There could be multiple causes of the error.
If you are willing to loose the data, remove manually the logs under "apps/hbase/MasterProcWALs" and "apps/hbase/WALs" from HDFS. AfterUmesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-68483857974071046412017-12-30T08:43:00.000+05:302017-12-30T08:43:54.396+05:30RegionServer crashing with error "Direct buffer memory"Region server might crash with below error
INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=ip-100-122-218-159.us-east-1.ec2.aws.symcpe.net:2181,ip-******.us-east-1.ec2.aws.net:2181,ip-*****.us-east-1.ec2.aws.net:2181 sessionTimeout=180000 watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@253c1256
ERROR [main] regionserver.HRegionServerCommandLine: Region Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-66416009205973761772017-12-29T20:00:00.004+05:302017-12-29T20:00:50.660+05:30HMaster error during startupHMaster throws error as below during startup
FATAL [ip-:16000.activeMasterManager] master.HMaster: The coprocessor org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor threw java.lang.RuntimeException: java.io.FileNotFoundException: /etc/hbase/2.5.0.55-1/0/xasecure-audit.xml (No such file or directory)
This is happening because the ranger co-processor is defined in xml config Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-37513577104350468082017-12-27T11:42:00.001+05:302017-12-27T11:42:07.667+05:30Unable to start namenode. Error "couldn't find resource file location"Sometimes starting the namenode might throw below error.
ERROR config.RangerConfiguration (RangerConfiguration.java:addResourceIfReadable(110)) - addResourceIfReadable(ranger-hdfs-security.xml): couldn't find resource file location
INFO provider.AuditProviderFactory (AuditProviderFactory.java:(77)) - AuditProviderFactory: creating..
FATAL conf.Configuration (Configuration.java:loadResource(Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com2tag:blogger.com,1999:blog-5893383981989524159.post-58739387615634501882017-10-17T19:04:00.002+05:302017-10-17T19:04:34.985+05:30Install PIP on CentOS 7Python is widely used programing language and pip is the tool need to install different python packages.
Before you go ahead check if you have pip installed by using command below.
$ pip --version
pip 9.0.1 from /Library/Python/2.7/site-packages (python 2.7)
You can install pip in multiple ways on CentOS. I will discuss here two ways to install pip.
Install PIP using yup
Update the Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-25711444368081050492017-10-04T20:52:00.001+05:302017-10-04T20:52:24.869+05:30Setup python 2.7 on CentOS 7Python is most commonly used development and scripting tool. You can setup Python on CentOS by multiple ways.
Setup Python using Yum
This is simplest method to install development tools on CentOS. Run below commands to setup the python using yum.
[vagrant@localhost ~]$ sudo yum -y update
[vagrant@localhost ~]$ sudo yum groupinstall -y development
Or
[vagrant@localhost ~]$ sudo yum Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-78166424519871457522017-10-02T14:09:00.003+05:302017-10-02T14:09:46.465+05:30How to Install Java 8 on CentOS 7Java is most common requirement for many of the Open Source softwares. You can follow below steps to install Java on CentOS vm.
You can first check the list of available packages for Java using below command.
$ yum search java | grep 'java-'
You will get the list like above. Now depending on what is your requirement you can choose the package. If you don't see any list as above, you will Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0tag:blogger.com,1999:blog-5893383981989524159.post-80489157113862412132017-09-30T00:15:00.000+05:302017-09-30T00:15:26.204+05:30Setup CentOS 7 VM on VirtualBoxThis is step by step guide which will let you run the CentOS 7 VM on your virtual box installation. In this tutorials are illustrated using Mac.
Installing VirtualBox
VirtualBox is a powerful x86 and AMD64/Intel64 virtualization product for enterprise as well as home use
Make sure your if your machine allows virtualization. Check your host OS is supported for Virtualbox
Download VirtualBox Umesh Kumbharhttp://www.blogger.com/profile/16293798571529056244noreply@blogger.com0