an account is not used within the Snap)._filename (A key/value pair with 'filename' key should be defined as a pipeline parameter.).region names and their details, see AWS Regions and Endpoints.If a Pipeline is created in the shared project and you want to read the ' asset.json' file from the shared project, enter ' asset.json' or 'sldb:///asset.json'.If a Pipeline is created in a project other than the shared project and you want to read the ' asset.json' file from the shared project, enter ' shared/asset.json' or 'sldb:///shared/asset.json'.In the WASB and WASBS file URL, the top directory should be the name of the 'Azure Storage container'. In the SnapLogic 4.3.2 release, WASB (Windows Azure Storage Blob) or WASBS protocol (wasb:/// or wasbs:///) support has been added to the Binary Snaps. If this property is a regular file, the Wildcard and Include subfolders property are ignored. Folders are not supported for these protocols.

The File property should be an absolute path for all protocols except sldb. For sldb files, the Snap can access only files in the same project directory or the shared project directory, and cannot access files in other projects.įor sldb, http and https protocols, URL for a regular file should be entered. Please note '://' is a separator between the file protocol and the rest of the URL and the host name and the port number should be between '://' and '/'. If the port number is omitted, a default port for the protocol is used. The hostname and port number are omitted in the sldb and s3 protocols. The File property should have the syntax: Specify the URL for the data source, which can be a directory or a file. The Hadoop cluster must open to the Groundplex server instance without any authentication. However, the HDFS protocol works only in a Groundplex. For most file protocols, the Snap behaves the same way in both Snaplex and Groundplex.Please note this feature is supported in the EC2-type Groundplex only.

Jcc.jvm_options = -DIAM_CREDENTIAL_FOR_S3=TRUE To enable this feature, the following line should be added to global.properties and the jcc (node) restarted: The IAM credential stored in the EC2 metadata is used to gain access rights to the S3 buckets. The 'IAM_CREDENTIAL_FOR_S3' feature is used to access S3 files from EC2 Groundplex, without Access-key ID and Secret key in the AWS S3 account in the Snap.

For example, if your folder contains 1,000,000 objects, you would make 1,000 requests and the List operation would cost you $0.005.You can use this read type Snap to read binary data from various sources such as SLDB, HTTP, S3, SFTP, HDFS, and produces a binary data stream at the output. Unlike the File Reader Snap, this Snap can read more than one file in the given directory and its subdirectories recursively. S3 List operations cost about $0.005 per 1,000 requests, where each request returns a maximum of 1,000 objects ( us-east-1 region).

Aws s3 copy wildcard