Skip Ribbon Commands
Skip to main content
Navigate Up
Sign In

Quick Launch

Average Rating:

facebook Twitter
Print Bookmark Alert me when this article is updated


ERROR: " Expected scheme-specific part" while running mappings in Spark Engine mode using Informatica DEI
Problem Description

While running mappings in 'Spark' execution engine using Informatica 'Data Engineering Integration' (DEI), earlier known as 'Big Data Management' (BDM), mapping execution fails. In the mapping run log, the following error trace could be observed:


Log Trace


2018-02-19 02:37:02.220 <CmdExecInProcessTasks-pool-2-thread-2> SEVERE: [Cleanup] [HadoopFSRmRfTask]java.lang.RuntimeException: java.lang.IllegalArgumentException: Expected scheme-specific part at index 5: hdfs:
        at com.informatica.platform.dtm.executor.hadoop.fs.impl.AbstractFileSystemImpl.globStatus(
        at java.util.concurrent.Executors$


Encountered issue occurs when HDFS 'Staging' and 'Event' log directories for 'Spark Engine' are specified with HDFS name node details in the 'Hadoop Pushdown' connection used for execution. However, it would be required to specify the HDFS path without name node details. That is, without 'hdfs://' protocol details.


Perform the following steps for resolving the encountered issue:


  1. Login to Informatica Administrator console or Informatica Developer Client tool.
  2. Edit the 'Hadoop Pushdown Connection' being used for running Spark jobs.
  3. Navigate to 'Spark Engine' section in the connection.
  4. Update the values of 'Spark Staging Directory' and 'Spark Eventlog directory' as below:




Spark Staging Directory


Spark Event Log directory





Spark Staging Directory


Spark Event Log directory



        5. Once updated, save the changes made to connection.

        6. Ensure that the folders specified as 'Spark Staging and Event Log' directory exists in HDFS and the impersonation user, specified under 'Common Attributes' section of the connection has required permissions on the folder. 

        7. Once verified, re-run the mapping in Spark Execution mode.

More Information
Applies To
Product: Data Engineering Integration(Big Data Management); Data Engineering Quality(Big Data Quality); Data Engineering Streaming(Big Data Streaming); Enterprise Data Preparation
Problem Type: Configuration; Connectivity
User Type: Administrator; Developer
Project Phase: Onboard; Configure
Product Version: Informatica 10.1; Informatica 10.1.1; HotFix; Informatica 10.2; Informatica 10.2.1; Informatica 10.2.1 Service Pack 1; Informatica 10.2.2; Informatica 10.4
Operating System:
Other Software:

Last Modified Date:3/31/2020 4:36 AMID:526526
People who viewed this also viewed


Did this KB document help you?

What can we do to improve this information (2000 or fewer characters)