Tag: python

Create diagram as code in Python

In the previous post, we explored my custom ClickHouse backup agent, built upon the clickhouse-backup tool, logrotate, Cron and Bash scripts. I have also shared all the necessary resources for testing the agent on your local machine using Docker as well as Docker Compose or deploying it in a production environment. Let’s update the agent’s repo with some Python code.

You may be familiar with a main GitOps principle: use Git as the single source of truth; store your applications and infrastructure configurations in a Git repository along with application code. Kubernetes (yaml), Terraform (tf), Docker, Compose files, Jenkinsfile and even diagrams can be good examples of files kept in such repositories. But how to represent diagrams? As png, vsd or jpeg? Let’s pretend we’re developers and can draw diagrams using code.

The diagrams project brings this approach to life. I opted for Diagrams (mingrammer) because it’s free and built on Python and Graphviz, widely used language and tool that enable you to create various diagrams, whether it’s a flowchart or a cloud architecture. Another advantage is that the project is actively maintained and continuously developed. You can also check out other tools such as pyflowchart, mermaid, plantuml or terrastruct.

Let’s get started and draw a flowchart for the clickhouse backup agent using Diagrams (mingrammer). First, install Python (>3.7; mine is 3.11) and Graphviz (9.0.0, Windows in my env), then install diagrams module (0.23.4).

Diagrams include the following objects: node (=shapes; programming, azure, custom and others), edge (=connection lines; linkage between nodes), cluster (=group of isolated nodes) and diagram (represents your entire chart). Each object has it’s own attributes. Description of all attributes can be found at Graphviz docs. Also, check out basic examples to understand what we gonna “build”. I won’t describe every attribute. DYOR.

The first line of your code might look like this:

# import required modules
from diagrams import Diagram, Edge, Cluster, Node

Then we define attributes for each object (excerpt):

# define attributes for graphviz components
graph_attributes = {
    "fontsize": "9",
    "orientation": "portrait",
    "splines":"spline"
}

Next, we need to describe diagram object and it’s attributes (excerpt):

with Diagram(show=False, outformat="png", graph_attr=graph_attributes, direction="TB"):
    # nodes and icons
    start_end_icon = "diagram/custom-images/start-end.png"
    start = Node (label="Start", image=start_end_icon, labelloc="c", height="0.4", weight="0.45", **node_attributes)

I use general Node class with custom images which were taken from programming nodes and then optimized to my flowchart (I’ve deleted canvas and resized images). You could safely use diagrams.programming.flowchart node class instead, but be ready to play with height/width node’s attributes. Another way to add your own images as nodes is Custom node class.

We have described icons and shared nodes. Now we need to add the first group of nodes to represent the main process of the agent and flowchart (creating and uploading FULL backups):

# cluster/full backup
    with Cluster("main", graph_attr=graph_attributes):
       diff_or_full = Node (label="TYPE?", image=decision_icon, height="0.7", weight="", labelloc="c", **node_attributes )

Subroutine processes (diff backups and etc.) are clusters (excerpt):

# cluster/diff backup
    with Cluster("diff", graph_attr=graph_attributes):
      create_diff_backup = Node (label="Create DIFF", labelloc="c", height="0.5", weight="4", image=action_icon, **node_attributes)

Edges or connections between nodes are defined at the bottom (excerpt):

# Log connections
    diff_or_full - Edge(label="\n\n wrong type", tailport="e", headport="n", **edge_attributes ) - write_error

As a result, I’ve updated the repo with diagram as code; slightly modified GitHub actions by adding a new step to “draw” diagram and check python code. When I push new commits to the repo, the diagram is created and published as an artifact with nodes (start, end, condition, action, catch, input/output), four clusters (main, diff, log, upload log) and edges between nodes.

Looks pretty good, doesn’t it?

Python Coding: FizzBuzz challenge

FizzBuzz is a very common task, asked in Dev/DevOps interviews. You are given a range of numbers and need to write algorithm using the following rules: if the number is divisible by 3, print “Fizz”; if the number is divisible by 5, output “Buzz”; if the number is divisible by both 3 and 5, the result should be “FizzBuzz”.

The main goal of the task is to check how you understand loops, conditionals and simple math using one of programming or scripting languages. I solved the task using PowerShell years ago: check this gist.

As I started to learn Python, I decided to share FizzBuzz implementation in this language to show how simple and “elegant” the solution can be.

I used matplotlib and colorama to make a pie chart and add color text output respectively. Defined a function fizz_buzz with 2 arguments, and then used try/catch/finally statements to catch exceptions errors. Inside of the try, the for loop and if conditionals are described to meet all task’s rules. As a result, the function outputs numbers and categories based on rules, and makes a pie chart to show how many fizz, buzz, fizzbuzz found in percentage. 

import matplotlib.pyplot as plt
import colorama
from colorama import Fore, Back, Style
colorama.init()

def fizz_buzz(x,y):
    """Python version of popular Fizz Buzz task"""
    fb = 0 ; b = 0; f = 0; rest = 0 # start values
    fb_type = ['fizzbuzz','fizz','buzz','rest'] # plot labels
    fb_colors = ['r','y','c','g'] # plot colors
    fb_explode = [0.2, 0.1, 0.1, 0.1] # plot fraction of the radius
    try:
        for n in range(x,y):
            if ((n % 3 == 0) and (n % 5 == 0)):
                fb += 1
                print(Fore.RED + f"Found FizzBuzz: {n}")
            elif n % 3 ==0:
                f += 1
                print(Fore.WHITE + f"Found Fizz: {n}")
            elif n % 5 ==0:
                b += 1
                print(Fore.GREEN + f"Found Buzz: {n}")
            else: 
                rest += 1
                print(Style.BRIGHT + f"The rest is {n}")
            print(Style.RESET_ALL)
        fb_array = [fb, f, b, rest]
        plt.pie(fb_array, colors = fb_colors, explode = fb_explode, shadow = True, radius = 1.1, autopct = '%1.1f%%') # form a pie
        plt.legend(fb_type,loc='upper right') # show legend
        plt.show() # show a pie
    except:
        print(Style.BRIGHT + Fore.RED + "You provided wrong x and y")
        print(Style.RESET_ALL)
    finally:
        print(Style.BRIGHT + Fore.GREEN + "Author: github.com/rlevchenko")
        print(Style.RESET_ALL)

Result

Available at Gist