Browse All Tools
Platform/OS
Linux / Cross-platform (Server)
Category
Databases

PostgreSQL with Foreign Data Wrappers

PostgreSQL with Foreign Data Wrappers is a popular free database system that can reach out and work with data stored in many other places without copying it first. It lets you run normal database queries that pull information from files sitting in cloud storage as if they were regular database tables. It uses S3-compatible object storage by treating S3 buckets like a giant filing cabinet that it can open on demand to read spreadsheets, logs, or other files directly. This means companies can analyze huge amounts of data kept cheaply in S3 without the expense and hassle of loading everything into the database. The real-world value is that you get powerful data insights while only paying for simple storage and avoiding slow data movement steps.

Connection / Setup

  1. Create an account and log in to DiskSpace
  2. Click I'm ready to set up my storage
  3. Enter Bucket Name following the instructions
  4. Click Create Bucket and wait for it to be generated
  5. Find your Endpoint URL, Access Key, and Secret Key in the S3 Client Configuration section for use below

Connecting DiskSpace.sh (S3) to PostgreSQL with Foreign Data Wrappers

This guide explains how to connect your DiskSpace.sh S3-compatible storage to PostgreSQL using Foreign Data Wrappers (FDW). DiskSpace.sh is fully S3 API compatible and requires path-style addressing.

Connection Steps
  1. Install the required PostgreSQL extensions: CREATE EXTENSION aws_s3 CASCADE; or use postgres_fdw with a custom S3 wrapper if needed.
  2. Gather your DiskSpace.sh credentials: Endpoint URL, Access Key, and Secret Key.
  3. Configure the S3 connection parameters in your FDW setup, ensuring path-style requests are used.
  4. Create the foreign server and user mapping with the provided credentials.
  5. Test the connection by importing sample data or listing objects from your bucket.
Credential Fields
Path Style Requirement: DiskSpace.sh requires path-style requests. Use the format https://s3.diskspace.sh/bucket-name/key instead of virtual-hosted style.
Example Configuration
SELECT aws_s3.table_import_from_s3(
  'my_table'::regclass,
  '',
  '(format csv, header true)',
  'my-bucket',
  'data.csv',
  'us-east-1',
  'https://s3.diskspace.sh',
  'YOUR_ACCESS_KEY',
  'YOUR_SECRET_KEY'
);
Troubleshooting
  • Ensure the region is set to us-east-1 or left empty if your bucket does not use a specific region.
  • Verify that your IAM/user policy allows s3:GetObject, s3:ListBucket, and s3:PutObject permissions.
  • Confirm you are using path-style URLs. Virtual-hosted style requests will fail.
  • Check PostgreSQL logs for detailed error messages if the connection fails.
  • Make sure your endpoint URL does not include a trailing slash.
DiskSpace.sh S3 Integration • Path-style requests enforced