Modern art of simplifying python projects by distributing complexity.

By Maxim Danilov

Elevator Pitch

An overcomplicated project increases maintenance time. If a complete redesign is not possible, we can try to distribute the complexity across the existing codebase. The AI tools cannot help us if we have more than 30 000 lines of code, and we should discuss manual methods to handle the complexity.

Description

The purpose of this talk is to define the necessary steps for matured project to support successful adaptation for futured tasks.

Let’s imagine a successful startup. Brilliant ideas require quick realization. Customers are happy, investors ask to add more functionality. With time, the project consume an initial wave of hype and stabilize own grow. The team scales up with the project. Some programmers leave, new ones come in. Parts of the project are rewritten due to lack of documentation or technology not meeting business needs. But suddenly the productivity of the developers collapses, this is the so-called “after wave of hype life” with the following symptoms:

  • Exponential growth of required tools and technologies
  • Low incomes from additional features vs. high development costs
  • Enormous growth in the number of people involved

Since it is impossible to avoid this step, it is necessary to prepare for it in advance in order to minimize negative consequences and enhance positive impacts.

Preventive actions: - Implementing a Code Style Guide to unify parts of the code base. - Gather opinions on keeping legacy code or a complete refactoring. These will become future milestones on your project roadmap. - Documentation in the form of “What do I do if I die?”.

Spreading complexity through separating parts of a project into isolated blocks: - GPT tools, how they can help, and why they can’t help. - Manual tools and techniques for identifying and avoiding interfacing patterns in the project. - Looking at the structure of individual applications if microservices are not a good solution. - Packaging portions of code into separate libraries. - Separating settings.

For each topic, I will give examples of working with large projects that are at this stage. Despite different business types, geographical and social contexts, these projects shows similar architectural mistakes, which will be discussed.

Notes

This talk uses the ideas from my presentation at PyCon DE 2022 (https://2022.pycon.de/program/BG37DV/), at PyConSK 2024 (https://youtu.be/lSZW0ZMVtk8) and PyConLT 2024 (not published).

After two years of rethinking theory into practice, I want to share my experience on how reducing complexity ideas can be implemented by working with real projects.

Without financial aid for travel this talk will be not possible.