Skip Ribbon Commands
Skip to main content
Navigate Up
Sign In

Quick Launch

Average Rating:

facebook Twitter
Email
Print Bookmark Alert me when this article is updated

Feedback

ERROR: "java.net.SocketTimeoutException: connect timed out" while writing to HDFS using Informatica PowerCenter
Problem Description

While running mapping in Informatica PowerCenter to write to HDFS, by using 'PowerExchange for Hadoop' adapter, mapping execution fails. In the session log, generated for the execution, the following message could be found:

 

Session Log Trace

 

2018-12-20 17:31:45 :(8247 | WRITER_1_*_1) : (IS | pc_is_indthsbde_spyker_thsb) : n_indthsbde001_node01 : WRT_8004 : Writer initialization failed [Error opening session output file [/data/informatica/shared/test.txt] [error=java.net.SocketTimeoutException: connect timed out

        at java.net.PlainSocketImpl.socketConnect(Native Method)

        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)

        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

        at java.net.Socket.connect(Socket.java:589)

        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)

        at sun.net.NetworkClient.doConnect(NetworkClient.java:175)

        at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)

        at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)

        at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)

        at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)

        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)

        at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)

        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)

        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)

        at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)

        at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:188)

        at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:131)

        at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)

        at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)

        at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClientProvider.java:549)
        at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClie]]. Writer terminating.

        at org.apache.hadoop.crypto.key.kms.KMSClientProvider$1.run(KMSClie]]. Writer terminating.

  ​

Cause

The encountered issue occurs, when 'Key Management Service' (KMS) is SSL enabled in the Hadoop cluster and secure connectivity to KMS cannot be established successfully from the Informatica server machine, where the PowerCenter Integration service is running. 

Solution

Perform the following steps to resolve the issue:

 

  1. Ensure that the latest cluster configuration files ('core-site.xml', 'hdfs-site.xml', 'yarn-site.xml', 'kms-site.xml', 'hive-site.xml', and so on) are present in the 'CLASSPATH' location, configured with PowerCenter Integration Service. If required, get the latest set of configuration files from Hadoop admin team and copy them into the location, specified in 'CLASSPATH'  environment variable of PowerCenter Integration Service.

 

pc_integration_service_classpath.png 


 

Note: If the PowerCenter Integration Service is running in 'Grid', then validate and ensure that the same set of configuration files are present across all the nodes that are part of the grid.

 

     2. Verify if the 'openssl' connectivity could be established successfully, to each of the KMS hosts, that would be specified in 'dfs.encryption.key.provider.uri'  attribute of 'hdfs-site.xml' file, from the Integration server machine.

 

For instance, consider the following entry in 'hdfs-site.xml'  file for the 'dfs.encryption.key.provider.uri' attribute:

 

  <property>

    <name>dfs.encryption.key.provider.uri</name>

    <value>kms://https@invglxbdm07.informatica.com;invglxbdm08.informatica.com:16000/kms</value>

  </property>

 

For the each of the hosts, run the following openssl commands to verify the connectivity:

 

openssl s_client -connect invglxbdm07.informatica.com:16000

openssl s_client -connect invglxbdm08.informatica.com:16000

 

 

    3. Once connectivity is confirmed to be working fine, for at least one of the 'openssl' commands from outside, restart the Integration Service, if there are any changes made to 'cluster configuration files'.

   4. Re-run the failed mapping/workflow, it should get executed successfully.

More Information
Applies To
Product: PowerCenter; Data Engineering Integration(Big Data Management)
Problem Type: Configuration; Connectivity; Security
User Type: Administrator; Developer
Project Phase: Configure; Implement
Product Version: Informatica 10.1; Informatica 10.1.1; HotFix; Informatica 10.2; Informatica 10.4
Database:
Operating System:
Other Software:

Reference
Attachments
Last Modified Date:3/31/2020 3:54 AMID:564403
People who viewed this also viewed

Feedback

Did this KB document help you?



What can we do to improve this information (2000 or fewer characters)