Testing and Performance Optimization

This section covers how to test, profile, and optimize your custom connector across its full lifecycle, from local development to production. Begin by verifying correctness locally, then profile CPU and memory usage to uncover bottlenecks. Use these insights to diagnose and resolve performance issues observed in production syncs.

• Local testing: Run fivetran debug to test your connector against live source data, inspect the local warehouse.db output, and reset the local environment between test runs.
• Connector performance analysis: Profile CPU usage with py-spy and flamegraphs to identify bottlenecks in your connector code, both locally and in production.
• Connector memory management: Measure and reduce memory usage using tracemalloc, psutil, and py-spy to keep your connector within the Connector SDK memory limits.