Skip Ribbon Commands
Skip to main content
Navigate Up
Sign In

Quick Launch

Average Rating:

facebook Twitter
Email
Print Bookmark Alert me when this article is updated

Feedback

FAQ: Where is 'hadoopEnv.properties' file from Informatica DEI 10.2.1?
Answer
Informatica 'Data Engineering Integration' (DEI), earlier known as 'Big Data Management' (BDM), supports execution of mappings in the Hadoop Environment. For running mappings in Hadoop Environments, one of the following execution engines can be chosen:

 

  1. Spark Engine
  2. Blaze Engine
  3. Hive Engine ('Map Reduce' or 'Tez' modes)  (Available in Pre-Informatica 10.2.2 versions and not available from Informatica 10.2.2 version onwards)

 

In Informatica versions till 10.2.0, settings that are to be used while running the mappings in Hadoop environments, could be configured through 'hadoopEnv.properties' file at '$INFA_HOME/services/shared/hadoop/<distribution>/infaConf' location in the Informatica server machine.

 

Hadoop pushdown jobs settings that can be configured to include the following, but not limited to:

 

  • Environment variable settings of Hadoop data nodes.
  • LD_LIBRARY_PATH settings of Hadoop data nodes.
  • Hadoop engine specific Java CLASSPATH and Heap Memory settings
  • Cluster resource (Memory and vCores) configurations for every Hadoop execution engines.

 

Settings that are configured in 'hadoopEnv.properties' file would be applicable for all the Hadoop pushdown jobs, unless overridden at connection or mapping level.

  

Starting from Informatica 10.2.1 version, 'hadoopEnv.properties' file would not be available in Informatica server machine. For ease of use, the configuration of all the Hadoop pushdown job settings has been moved to the 'Hadoop Pushdown connection'. As a result, all the settings can be configured directly, either from Informatica Developer client or from Informatica Administrator console.


Cluster Environment Variable Settings

infa_1021_hadoop_connection_environment_variables.jpg 
  

Spark Engine Settings

infa_bdm_1021_spark_advanced_properties.jpg 

Blaze Engine Settings

 

infa_bdm_1021_blaze_advanced_properties.jpg 

​​​​​


More Information

Applies To
Product: Data Engineering Integration(Big Data Management); Data Engineering Quality(Big Data Quality); Enterprise Data Preparation; Data Engineering Streaming(Big Data Streaming)
Problem Type: Configuration; Connectivity; Product Feature
User Type: Administrator; Developer
Project Phase: Configure; Onboard
Product Version: Informatica 10.2.1; Informatica 10.2.1 Service Pack 1; Informatica 10.2.2; HotFix; Informatica 10.4
Database:
Operating System:
Other Software:

Reference

Attachments

Last Modified Date:2/11/2020 1:58 AMID:532971
People who viewed this also viewed

Feedback

Did this KB document help you?



What can we do to improve this information (2000 or fewer characters)