{"id":2091,"date":"2025-04-20T13:37:23","date_gmt":"2025-04-20T10:07:23","guid":{"rendered":"https:\/\/mshaeri.com\/blog\/?p=2091"},"modified":"2025-09-09T21:47:24","modified_gmt":"2025-09-09T18:17:24","slug":"design-patterns-i-use-in-external-service-integration-in-python","status":"publish","type":"post","link":"https:\/\/mshaeri.com\/blog\/design-patterns-i-use-in-external-service-integration-in-python\/","title":{"rendered":"Design Patterns I use in External API Integration in Python"},"content":{"rendered":"\n<p>When we are building systems that interact with external services like payment gateways, cloud storages, or notification systems, I think it&#8217;s essential to support different environments like <code><strong>dev<\/strong><\/code>, <code><strong>test<\/strong><\/code>, and <code><strong>prod<\/strong><\/code> environments. I believe that Every external resource integration must have a dummy driver in the project<strong>.<\/strong> Because it gives us the flexibility to swap between a real implementation for production and a dummy<strong> <\/strong>one for development and test environment. This way we can make sure we don&#8217;t have to expose external service credentials to every developer on the team to work with it, which is not reasonable and secure. This not only protects us from unexpected costs (especially when using paid services), but also shields our development from rate limits or downtime issues. Also, considering that test environments should always be fully isolated and free from dependencies on live external systems, this approach helps enforce that principle. <\/p>\n\n\n\n<p>In this post, I&#8217;ll explain how I implement drivers for external services to create a clean, flexible, and testable integration layer. The goal is to isolate third-party dependencies from the core application logic, support multiple environments (like development, testing, and production), and make it easy to switch between real and mock implementations without changing any business code. Here are the three key design patterns I use to bring this all together: <\/p>\n\n\n\n<ul>\n<li><strong><strong>Strategy<\/strong><\/strong> <strong>Pattern<\/strong><\/li>\n\n\n\n<li><strong>Factory<\/strong> <strong>Pattern<\/strong><\/li>\n\n\n\n<li><strong>Singleton<\/strong> <strong>Pattern<\/strong><\/li>\n\n\n\n<li><strong>Retry with Backoff Pattern<\/strong><\/li>\n<\/ul>\n\n\n\n<p>Assume we need to talks to a third-party weather API in our project to get weather information. Here&#8217;s what we want:<\/p>\n\n\n\n<ul>\n<li>In <code><strong>prod<\/strong><\/code>, fetch real weather from an external API.<\/li>\n\n\n\n<li>In <code><strong>dev<\/strong><\/code> or <code>test<\/code>, return fake (but valid) weather data.<\/li>\n\n\n\n<li>Ensure the weather client is initialized once and reused across the system.<\/li>\n\n\n\n<li>Keep the code loosely coupled and easy to extend.<\/li>\n<\/ul>\n\n\n\n<p><strong>Diagram 1<\/strong> illustrates the final architecture of the weather API integration we want to develop. In this design, the <strong>Weather Client<\/strong> creates a driver during its initialization using the <strong>Weather driver factory<\/strong>, based on the provided environment variable <code><strong>WEATHER_PROVIDER<\/strong><\/code>. It then delegates the actual weather retrieval to the driver&#8217;s <code><strong>get_weather()<\/strong><\/code> method, which is called within client&#8217;s  <code><strong>get_weather()<\/strong><\/code> method.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/mshaeri.com\/blog\/wp-content\/uploads\/2025\/04\/image.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"441\" src=\"https:\/\/mshaeri.com\/blog\/wp-content\/uploads\/2025\/04\/image-1024x441.png\" alt=\"\" class=\"wp-image-2099\" srcset=\"https:\/\/mshaeri.com\/blog\/wp-content\/uploads\/2025\/04\/image-1024x441.png 1024w, https:\/\/mshaeri.com\/blog\/wp-content\/uploads\/2025\/04\/image-300x129.png 300w, https:\/\/mshaeri.com\/blog\/wp-content\/uploads\/2025\/04\/image-768x331.png 768w, https:\/\/mshaeri.com\/blog\/wp-content\/uploads\/2025\/04\/image.png 1442w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption class=\"wp-element-caption\">Diagram 1. architecture of the weather API integration<\/figcaption><\/figure>\n\n\n\n<p>Let&#8217;s dive into implementation.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">1. Strategy Pattern For Swappable Drivers<\/h3>\n\n\n\n<p>The <strong>Strategy Pattern<\/strong> is all about defining a family of interchangeable algorithms\/drivers\/components and making them swappable at runtime. However, in our case, the <strong>strategy<\/strong> (i.e., the weather driver) is determined by an environment variable and <strong>cannot be altered<\/strong> once the application is running. This gives us a environment-specific behavior.<\/p>\n\n\n\n<p>Let&#8217;s write down our strategies. In our case, each &#8220;strategy&#8221; is a different driver for the weather service:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python\">from abc import ABC, abstractmethod\n\nclass WeatherDriver(ABC):\n    @abstractmethod\n    def get_weather(self):\n        pass\n\nclass RealWeatherDriver(WeatherDriver):\n    def get_weather(self):\n        return \"Real Weather from API\"\n\nclass DummyWeatherDriver(WeatherDriver):\n    def get_weather(self):\n        return \"Fake Weather Data\"\n<\/code><\/pre>\n\n\n\n<p>With this, our client (which we will implement in section 3) doesn\u2019t care <em>how<\/em> the weather is fetched. It just uses a <code><strong>WeatherDriver<\/strong><\/code>. That\u2019s the power of strategy.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Factory Pattern For Centralized Creation Logic<\/h3>\n\n\n\n<p>To create a driver instance in the weather client we need a <strong>factory<\/strong>. The <strong>Factory Pattern<\/strong> helps us centralize the creation of objects, especially when that creation involves some logic, or we have more than one type of object.<\/p>\n\n\n\n<p>Here, we decide which driver to create based on the given <code><strong>provider<\/strong><\/code> :<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python\">class WeatherDriverFactory:\n    @staticmethod\n    def create_driver(provider: str) -&gt; WeatherDriver:\n        if provider == \"real\":\n            return RealWeatherDriver()\n        elif provider == \"dummy\":\n            return DummyWeatherDriver()\n        else:\n            raise ValueError(f\"Unknown provider: {provider}\")\n<\/code><\/pre>\n\n\n\n<p>This factory prevents redundant decision-making across rest of our app. You want a weather driver? Ask the factory.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">3. Singleton Pattern for a Single, Shared Client Instance<\/h3>\n\n\n\n<p>This is our Config class which reads the strategy from the <code>.env<\/code> file. If the value is not found in the <code>.env<\/code> file , it will use the default value:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python\">from decouple import config\n\nclass AppConfig:\n    # The config reads the value from the .env file.\n    # If the value is not found in the .env file, it will use the default value.\n    WEATHER_PROVIDER = config('WEATHER_PROVIDER', default='dummy')<\/code><\/pre>\n\n\n\n<p>With all the pieces ready, it\u2019s time to build the <strong>WeatherClient<\/strong> , a clean abstraction that handles retrieving weather data without worrying about which provider\/driver is being used behind the scenes. I think external service integration is among the use cases where we should only have one instance of the client class in whole system. I used a <strong>lazy singleton<\/strong> with a <strong>metaclass<\/strong>. It&#8217;s only created when needed, and will be reused in the system after that:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python\">from config import AppConfig\n\nclass SingletonMeta(type):\n    _instances = {}\n    \n    def __call__(cls, *args, **kwargs):\n        if cls not in cls._instances:\n            cls._instances[cls] = super().__call__(*args, **kwargs)\n        return cls._instances[cls]\n\nclass WeatherClient(metaclass=SingletonMeta):\n    def __init__(self):\n        provider = AppConfig.WEATHER_PROVIDER\n        self.driver = WeatherDriverFactory.create_driver(provider)\n\n    def get_weather(self):\n        print(self.driver.get_weather())\n<\/code><\/pre>\n\n\n\n<p>Usage:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python\">\nclient = WeatherClient()\nclient.get_weather()\n<\/code><\/pre>\n\n\n\n<p>The client is ready to use. No matter how many times you call <code><strong>WeatherClient()<\/strong><\/code>, it will always return the same instance. Note that the <code><strong>SingletonMeta<\/strong><\/code> metaclass can be reused to create any other singleton class across the project. This approach keeps our code base clean and DRY, eliminating the need to repeat boilerplate logic for every singleton implementation.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Retry with Backoff Pattern<\/h2>\n\n\n\n<p>One other important pattern that increases the <strong>resilience and reliability<\/strong> of the system is the retry pattern with <strong>backoff <\/strong>strategy, which helps manage retries after failures. Instead of retrying requests immediately and repeatedly, which could overwhelm the system or external services, the backoff pattern introduces a delay between retries. This delay can be <strong>fixed<\/strong>, <strong>incremental<\/strong>, or <strong>exponential<\/strong>, allowing the system to recover gracefully while avoiding unnecessary load. By applying this pattern, systems become more fault-tolerant, reduce the risk of cascading failures, and improve overall stability under stress.<\/p>\n\n\n\n<p>Here\u2019s a simple example showing how to use exponential backoff using backoff package:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python\">import backoff\n\nclass WeatherClient(metaclass=SingletonMeta):\n    def __init__(self):\n        provider = AppConfig.WEATHER_PROVIDER\n        self.driver = WeatherDriverFactory.create_driver(provider)\n\n    @backoff.on_exception(\n         backoff.expo,                               # exponential backoff\n         requests.exceptions.RequestException,       # retry on these exceptions\n         max_time=30,                                # give up after 30 seconds total\n         max_tries=5                                 # or stop after 5 attempts\n     )\n    def get_weather(self):\n        print(self.driver.get_weather())<\/code><\/pre>\n\n\n\n<p>\ud83d\udca1 <strong>You can find the complete source code on GitHub:<\/strong> <a href=\"https:\/\/github.com\/birddevelper\/python_design_patterns\"> python_design_patterns<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Final Word<\/h3>\n\n\n\n<p>Working with <strong>external services<\/strong> is part of almost every modern application. But if we don\u2019t design it with best practices, we end up locked into brittle systems that are hard to test and harder to scale.<\/p>\n\n\n\n<p>Using this combo of patterns <strong>Strategy for behavior<\/strong>, <strong>Factory for creation<\/strong>, and <strong>Singleton for reuse<\/strong> makes my projects more maintainable, developer-freindly and test-friendly. In addition, if in some point of time in the future we decide to use different provider, it won&#8217;t be difficult to switch.<\/p>\n\n\n\n<p>I\u2019d love to hear your thoughts on this approach. Feel free to share any feedback or alternative patterns you prefer to use in your projects.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>When we are building systems that interact with external services like payment gateways, cloud storages, or notification systems, I think it&#8217;s essential to support different &hellip; <\/p>\n","protected":false},"author":1,"featured_media":2102,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,291,41],"tags":[97,297,314,293,296,292,39,294,295],"_links":{"self":[{"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/posts\/2091"}],"collection":[{"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/comments?post=2091"}],"version-history":[{"count":4,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/posts\/2091\/revisions"}],"predecessor-version":[{"id":2194,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/posts\/2091\/revisions\/2194"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/media\/2102"}],"wp:attachment":[{"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/media?parent=2091"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/categories?post=2091"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mshaeri.com\/blog\/wp-json\/wp\/v2\/tags?post=2091"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}