Edit on GitHub
Why Am I Receiving Connection Limit Exceeded Errors?link
Questionlink
Why is my connector failing due to number of connections exceeding allocated limits? The sync results in the following error: Unable to establish connection with the database. Concurrent connections have exceeded the max allocated limit.
Environmentlink
Considerationslink
A PostgreSQL database server has the following connection characteristics:
- There is a limit for the total number of concurrent connections defined by the
max_connections
server parameter. The default value PostgreSQL uses is 100 - Connection limits can be set per user in
pg_roles.rolconnlimit
A Fivetran PostgreSQL connector has the following characteristics:
- Each Fivetran PostgreSQL connector uses a minimum of 5 concurrent connections to the underlying database up to a maximum of 20 concurrent connections
- Connectors using the same PostgreSQL database share the concurrent connection limit defined by the
max_connections
server parameter
Therefore, a PostgreSQL connector may face a situation where the total number of concurrent connections (up to 20 per connector) exceeds the limit of the database server (defined by the max_connections
parameter).
Fivetran recommendationslink
We recommend the following:
- Increase the
max_connections
limit to accommodate 10 database connections per connector - For the database user who created the connector, set
pg_roles.rolconnlimit
to-1
to disable the per-user connection limit and rely on themax_connections
server-level limit - If you have multiple connectors using the same source database, consider using a single connector which syncs everything in the schema instead
- If you set up 12 or more connectors for a single database server, we recommend that you set a lower sync frequency