Any startup comes to the point where the first investment has been secured and a scaling plan should be structured, scaling here covers every aspect of it, from team and resources to processes and releases. All tech companies start deep thinking of the best ways to do that. In this post, we are going to share our recipe for scaling in this phase, or what’s also known as the 3rd Chasm of a triple Chasm model, where building a scalable and sustainable business is more important than ever as you already have a proven idea and a sort of complete product.
With that comes a ton of problems, how can we decide on features from product stand point? How can we manage the technical teams? How can we do source control? How can we do releases? etc
This post will give opinionated setup of standard guidelines around possible processes to manage these complex structures end to end.
Why all those processes?
Common sense does not scale. Every person’s common sense is different, and with a complex structure, you need a process followed by everyone, from C-level to developers so everyone can do their job.
Scaling Scrum Tech Teams (Nexus)
The first direct challenge is team structure, with a single team on a single product, everything is beautiful and the life is all roses and rainbows. Once a new team or new product is introduced, the startup starts to suffer, or at least to slow down.
If you are using scrum, where a team takes a product or a part of the product to work on end to end in a cross-functional self-organized way. Then Nexus might be the best choice as a possible process to scale your team.
Nexus guide includes all the needed methods to scale the team as well as manage cross team dependencies. Without Nexus, you always get bad symptoms like a team releasing something that breaks another team functionality, this is a common symptom of the lack of cross-team planning.
In Nexus, you have multiple scrum teams, with one or multiple integration teams to manage the cross-team refinement. Each team has a product owner, and the Nexus has a product owner that can decide if a dependency is worth it or not (Managing the Nexus backlog and priorities), and based on that a broader discussion can be held to eliminate the dependency or manage it.
Scaling Product Teams (Catalog/Portfolio/Product Management)
Although Nexus is more than enough to manage product and tech processes, but most of the startups we have worked with usually need a separate process to manage the product from a business standpoint, where stakeholders and product owners can have a shared vision of what’s going on before POs take it to the Engineering teams.
A very common way to do it is what’s called catalog portfolio management, under the company catalog there are multiple portfolios, each portfolio has multiple products, in a hierarchical way.
If we took Amazon as a hypothetical example, Amazon would be the catalogue, within Amazon there is the Ecommerce portfolio, streaming portfolio and the AWS portfolio, within AWS portfolio there are multiple products like EC2, ECS and so on.
NOTE: the above example is not necessarily representing Amazon’s structure and is just given as an example.
Each portfolio products usually don’t interact with another portfolio, making each portfolio a perfect candidate for a Nexus under the catalog. This catalog management usually handles ideation, discovery and pre-grooming activities (Every Step until the product owners agree with stakeholders on the new features and before writing the user stories)
Scaling Source Code (Monorepos)
Traditionally, each team code is stored in a separate repository. Which was OK until a new idea was introduced and took the world by a storm. Currently, Google for example stores billions of lines of code in a single repository for all developers to use, same as Microsoft, in what’s called a monorepo (Single Git repository)
Monorepos make the source control much easier, converting all complexity around source code management to almost linear (New team, new product, CI/CD, branching, releasing, etc), if you are using typescript we recommend taking a look at Nx.
Monorepo allows you to share a lot of code between your products in common libraries, as well as between frontend and backend for handling requests, responses, Graphql mutations & queries, form common validations in a single typed source of truth. More on that in our previous post about why we started recomming typescript as backend language.
With a huge monorepo, you always start to have a problem with branching, merging and releasing, so you need a process around it, and that’s where Trunk-based development kicks in.
Scaling Source Control Process (Trunk-based development)
Trunk-based development is a great way to manage a healthy cross-team monorepo, this includes branches structure, complex feature merging, releasing and so on.
Multiple complete guides exist for trunk-based development, but briefly in trunk based development when you implement a feature, you create a new branch, make sure your feature is covered by a feature flag (Or multiple flags), then merge it to the trunk which is the only long-live branch in the repository, before merging, the branch must pass all automated tests, once merged it’s part of your trunk.
To release, either create a tag on the branch or create a new branch for releasing, for hotfixes, you fix on the trunk first and either tag again or cherry-pick to the release branch (Depends on how you do the releases)
Trunk-based development is currently followed by most high throughput startups, it ensures much faster CD.
Where should I start?
Start with Conway’s law “Any organization that designs a system will produce a design whose structure is a copy of the organization’s communication structure”. Whatever your current situation, it can eventually match the above, but you need to always make sure you align the team to the process, even if the process itself is flexible. This way you can decide to start with converting the team to Nexus, making sure your communication channels are aligned to product teams and integration teams. Then you can start converting the code into a monorepo while converting the product teams to adapt a portfolio management process, and finally you can adapt the new monorepo the trunk-based development.
Let us know in the comments below if you are already using one of those processes, which ones worked for you and which ones you had to replace?