Airbyte

Airbyte is an open-source data integration platform. It offers 350+ pre-built connectors for syncing data into warehouses and vector databases. The PyAirbyte SDK and AI Connector Builder facilitate rapid connector creation.

intermediate30 min8 steps

The play

Install Airbyte via Docker Compose
Download the `docker-compose.yaml` file. Then, run Docker Compose to start Airbyte.
Access the Airbyte UI
Open your web browser and navigate to `http://localhost:8000`. Use the default credentials (airbyte/password) to log in.
Create a Source Connector
In the Airbyte UI, click 'Sources' then 'Create your first source'. Select a connector (e.g., PostgreSQL).
Configure the Source Connector
Enter the necessary credentials for your selected source (e.g., database host, port, username, password, database name).
Create a Destination Connector
Click 'Destinations' then 'Create your first destination'. Select a connector (e.g., Snowflake).
Configure the Destination Connector
Enter the credentials for your selected destination (e.g., account identifier, username, password, database name, schema name, warehouse).
Create a Connection
Click 'Connections' then 'Create your first connection'. Select your configured source and destination connectors.
Configure the Connection Settings
Define the replication frequency, data transformation settings and the streams to sync. Start the connection to begin data replication.

Starter code

version: "3.9"
services:
  airbyte-server:
    image: airbyte/server:latest
    ports:
      - "8000:8000"
    volumes:
      - airbyte_data:/data
volumes:
  airbyte_data:

Source

Articleairbyte.com