Skip Ribbon Commands
Skip to main content
Navigate Up
Sign In

Quick Launch

Average Rating:

facebook Twitter
Email
Print Bookmark Alert me when this article is updated

Feedback

FAQ: Will the Overridden Hadoop service configuration values persist after CCO refresh operation in Informatica DEI?
Answer

Informatica 'Data Engineering Integration' (DEI), earlier known as 'Big Data Management' (BDM), enables the organization to process large, diverse, and fast-changing data sets so as to get insights into their data. Big Data Management is used to perform big data integration and transformation without writing or maintaining Apache Hadoop code. 

 

In case of versions prior to Informatica 10.2.0, it would be required to run 'BDMConfig.sh' script for downloading the Hadoop cluster configuration site-xml files (core-site.xml, hdfs-site.xml, yarn-site.xml etc..) into Informatica server machine and to create the Hadoop cluster connections. Utility can be found under '$INFA_HOME/tools/BDMUtil' and configuration site-xml files would be downloaded into '$INFA_HOME/services/shared/hadoop/<distribution>/conf' location.

 

Starting from Informatica 10.2.0 version, 'Cluster Configuration Object' (CCO) got introduced and it could be created from 'Manage > Connections' tab in Informatica Administrator console. CCO would encapsulate all the Hadoop configuration site-xml files in web User Interface (UI), which were earlier downloaded using 'BDMConfig.sh' script. For more information on CCO configuration, refer to the following link of Informatica Administrator guide:

 

https://docs.informatica.com/big-data-management/data-engineering-integration/10-4-0/administrator-guide/cluster-configuration/cluster-configuration-overview.html​

 ​

In CCO, once created, value for any of the configurations in the site-xml files can be overridden, by editing it and providing the required value. Overridden configuration value in CCO would be used by pushdown jobs submitted from Informatica.

 

When there are any service configuration changes in the Hadoop cluster, it would be required to refresh the CCO, so as to make Informatica Hadoop pushdown jobs use the latest configurations. Even after the 'Refresh' operation on CCO, configurations that are overridden by Informatica would still persist in CCO. Overridden configurations can be seen from 'Overridden Properties' section in the CCO.


bdm_cco_get_overridden_attributes.jpg


For more information on 'CCO Refresh', refer 'Refresh the Cluster Configuration' section in the Informatica DEI Administrator guide, accessible from the following location:

 

https://docs.informatica.com/big-data-management/data-engineering-integration/10-4-0/administrator-guide/cluster-configuration/refresh-the-cluster-configuration.html​​


More Information
Certain configurations of Hadoop cluster services cannot be overridden from client applications and service configurations as Hadoop distribution server will always take precedence over the client configurations. For more information on  whether a required Hadoop service configuration can be overridden from client applications, check with the Hadoop Administrator.


Applies To
Product: Data Engineering Integration(Big Data Management); Data Engineering Quality(Big Data Quality); Data Engineering Streaming(Big Data Streaming); Enterprise Data Preparation; Enterprise Data Catalog
Problem Type: Configuration; Product Feature; Connectivity
User Type: Administrator
Project Phase: Configure; Onboard; Implement
Product Version: Informatica 10.2; Informatica 10.2.1; Informatica 10.2.1 Service Pack 1; Informatica 10.2.2; HotFix; Informatica 10.4
Database:
Operating System:
Other Software:

Reference

Attachments

Last Modified Date:3/31/2020 3:08 AMID:527856
People who viewed this also viewed

Feedback

Did this KB document help you?



What can we do to improve this information (2000 or fewer characters)