Skip Ribbon Commands
Skip to main content
Navigate Up
Sign In

Quick Launch

Average Rating:

facebook Twitter
Email
Print Bookmark Alert me when this article is updated

Feedback

Metadata Extraction from Amazon S3 Resource Fails with "Error Code: 403 Forbidden; Request ID: CF2D0928CB1D697A)" in Enterprise Data Catalog
Problem Description
Metadata for Amazon S3 Resource is not getting extracted. In the Scanner log, the following error is seen:

2019-08-09 14:28:32,613 [pool-19-thread-1] WARN  com.infa.products.ldm.scanners.amazons3scanner.AmazonS3FileSystem- shade.com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: CF2D0928CB1D697A), S3 Extended Request ID: yWAZiOdcmvov/BmpXvimH+o+ve1LqXx+rJgPVeNNqiFsEIxgxlMj+qw5rH05g4EBE9o33PfCnzA=
at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1586)
at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1254)
at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1035)
at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:747)
at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:721)
at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:704)
at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:672)
at shade.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:654)
at shade.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:518)
at shade.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4158)
at shade.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4105)
at shade.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1218)
at com.infa.products.ldm.scanners.amazons3scanner.AmazonS3FileSystem.fetchAdditionalFileAttribute(AmazonS3FileSystem.java:344)
at com.infa.products.ldm.scanners.amazons3scanner.AmazonS3FileSystem.getAmazonS3FileSummary(AmazonS3FileSystem.java:234)
at com.infa.products.ldm.scanners.amazons3scanner.storage.AmazonS3Storage.getFileProperties(AmazonS3Storage.java:42)
at com.infa.products.ldm.filescanner.file.group.impl.FileGroupCreator$CrawlTask.call(FileGroupCreator.java:156)
at com.infa.products.ldm.filescanner.file.group.impl.FileGroupCreator$CrawlTask.call(FileGroupCreator.java:1)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

The Test Connection is successful and we are able to select the First Level Directory from the Resource Configuration.
Cause
When aws:SecureTransport:"true" is set to Allow only HTTPS on the S3 bucket,  accessing the contents of the bucket over HTTP will produce a 403 error.

N
ote
When https in s3 bucket URI  is not specified in the resource configuration in ldmadmin, by default http request will be issued to access the metadata from the specified s3 bucket.
Solution
​To resolve this issue,  specify the s3 Bucket URI in the following format and run the metadata load scanner on the Amazon s3 resource.

https://<bucket_name>.s3.amazonaws.com
More Information
Applies To
Product: Enterprise Data Catalog
Problem Type: Configuration; Connectivity
User Type: Administrator; Architect; Business Analyst; Developer
Project Phase: Configure; Implement
Product Version:
Database:
Operating System:
Other Software:

Reference
Attachments
Last Modified Date:11/6/2019 8:51 AMID:588941
People who viewed this also viewed

Feedback

Did this KB document help you?



What can we do to improve this information (2000 or fewer characters)