Skip Ribbon Commands
Skip to main content
Navigate Up
Sign In

Quick Launch

Average Rating:

facebook Twitter
Email
Print Bookmark Alert me when this article is updated

Feedback

ERROR: "Class com.wandisco.fs.client.FusionHdfs not found" while running Hadoop pushdown mappings from Informatica 10.2.1
Problem Description

While running mappings in Hadoop pushdown mode using Informatica 10.2.1 or later versions, pushdown jobs fails in the WANDisco enabled Hadoop cluster with error in loading WANDisco drivers. However, the same mapping was working fine in Informatica versions before 10.2.1.

 

Log Trace

 

java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.wandisco.fs.client.FusionHdfs not found

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2214)

at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2746)

at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2759)

at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)

at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2795)

at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2777)

at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:386)

..

Caused by: java.lang.ClassNotFoundException: Class com.wandisco.fs.client.FusionHdfs not found

at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2120)

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2212) ​

Cause

This issue occurs when the WANDisco client jar path is not included in the 'Cluster Classpath' attribute of the Hadoop connection, used for running the mapping.

 

Starting from Informatica 10.2.1 version, 'hadoopEnv.properties' file would not be available in Informatica server machine. For ease of use, a configuration of all the Hadoop pushdown job settings has been moved to the 'Hadoop Pushdown connection'. As a result, all the settings can be configured directly, either from Informatica Developer client or from Informatica Administrator console. 

Solution

To resolve the issue, perform the following steps:

 

  1. Login to Informatica Administrator console or launch Informatica Developer client.
  2. Navigate to 'Connections' tab in case of Admin console and 'Windows > Preferences > Connections > [Domain]' , when Developer client is used.
  3. Select the 'Hadoop Pushdown' connection, being used for running the jobs.
  4. Navigate to 'Hadoop Cluster Properties' section.
  5. Edit the 'Cluster Classpath'  attribute in the section.
  6. In the 'Cluster Classpath'  attribute window, include the 'Wandisco client' location:

 

<absolute_path_to_wandisco_client_folder>/lib/*

 

Example: If the Wandisco client is installed at '/opt/wandisco/fusion/client/' location in the Hadoop data nodes, then update the 'Cluster Classpath' attribute to include '/opt/wandisco/fusion/client/lib/*' location.


bdm_1021_add_jar_to_classpath_hadoop_connection.png


bdm_1021_added_classpath_hadoop_connection.png

 

      7. Once done, save the changes made to the Hadoop connection.

      8. Re-run the mapping and share the results.

 

Note


If the mapping is being executed in 'Blaze Engine' and if the 'Blaze Grid Manager Application' is already running in the Hadoop cluster, it would be required to restart the Blaze Engine for the changes to take effect. For more information on restarting Blaze Engine, refer to KB 521162.

More Information
Applies To
Product: Data Engineering Integration(Big Data Management); Data Engineering Quality(Big Data Quality); Data Engineering Streaming(Big Data Streaming); Enterprise Data Preparation; Enterprise Data Catalog
Problem Type: Configuration; Connectivity; Product Feature
User Type: Administrator; Data Analyst; Developer
Project Phase: Configure; Implement
Product Version: Informatica 10.2.1; Informatica 10.2.1 Service Pack 1; Informatica 10.2.2; HotFix; Informatica 10.4
Database:
Operating System:
Other Software:

Reference
Attachments
Last Modified Date:3/31/2020 4:17 AMID:534650
People who viewed this also viewed

Feedback

Did this KB document help you?



What can we do to improve this information (2000 or fewer characters)