Connectors

DataFlow currently exposes 25 connector roles: 13 sources and 12 sinks. Sources read records into a pipeline; sinks write the resulting records to a destination. Availability can depend on the enabled edition and features.

Supported sources and sinks

13 sources	12 sinks
Zendesk	PostgreSQL
Google Sheets	ClickHouse
Google Drive	MySQL
Microsoft Excel	MongoDB
Custom HTTP API	Amazon S3
PostgreSQL	Kafka / Redpanda
MySQL	SFTP
MongoDB	Snowflake
Amazon S3	Apache Iceberg
Kafka / Redpanda	Google Sheets
SFTP	Webhook
Snowflake	DataFlow managed store
Apache Iceberg

The application catalog is the live source of truth. It combines built-in connectors with any administrator-provided manifest connectors and filters entries by the workspace's enabled features.

Connect from the application

1. Sign in and open Connectors.
2. Choose the service you want to connect.
3. Enter the connection details or complete OAuth.
4. Use Test connection before saving.
5. Open a pipeline and drag the source or sink from the palette.
6. Select the saved connection, configure the table, topic, file, or resource, and run the pipeline.

Credentials are stored separately from pipeline definitions, so the same saved connection can be reused without putting secrets in pipeline JSON.

Connect with an HTTP client library

DataFlow's authenticated REST API works with fetch, curl, or any HTTP client library. Create the credential once, then reference the returned connection ID from pipeline node configuration.

const response = await fetch("https://YOUR_DATAFLOW_HOST/api/connectors", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.DATAFLOW_TOKEN}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    provider: "postgres",
    name: "orders-production",
    config: { host: "db.example.com", database: "orders", user: "dataflow" },
    secret: { password: process.env.POSTGRES_PASSWORD },
  }),
});

if (!response.ok) throw new Error(`DataFlow returned ${response.status}`);
const { id: connectionId } = await response.json();

Use GET /api/connectors/catalog to read the connector catalog and GET /api/connectors to list saved connections. Keep tokens and connection secrets in environment variables or a secret manager, never in source control.

Custom REST endpoints without rebuilding

Administrators can mount connector manifest files through the deployment configuration. This is an operational extension point: users continue to choose and configure the connector through the DataFlow UI or API, and no application source changes or image rebuilds are required.