How software engineering is changing
AI. Like it or not, it will be the centrepiece of this essay, as it clearly represents the biggest change in our industry since I joined it in the late 00s. I don’t make this claim lightly: we have seen a lot of change in that period — the rise (and fall) of The Cloud; The App Stores; the emergence of DevOps practices; Rust becoming mainstream, etc. None of these has had nearly as big an impact on the industry, let alone outside of IT, as AI has.
Read this as “LLMs are making the biggest change in IT for the past X decades”. I ignore work from past AI Winters. Not because that work was not interesting or didn’t help us get where we are; it’s just that the LLMs represent the tip of the iceberg that made everyone talk about AI: even my dad talks to his phone when he has a question instead of “Googling it”.
To give you some context, I spent a decade as a software developer before switching to the management track at my current employer just over 6 years ago. My current position makes me a jack-of-all-trades with responsibilities across architecture, DevOps, project management, QA, and release management. The varying responsibilities shaped my usage of AI, but more on that later.
For me, the change began with the launch of Midjourney and ChatGPT. I remember generating a children’s storybook to read to my kids using these two tools back in December 2022. I was in awe of the capabilities and the seemingly endless opportunities to put it to good use. And then… I stopped using AI. Perhaps it was my lack of imagination or poor entrepreneurship skills, but I don’t remember any of my other experiments from that time. Apparently, they were not worth remembering.
The early days of LLMs held a big promise, but the outcome was at best a “meh”. For a year or so, we didn’t use AI at work, other than for simple experiments here or there — and I don’t mean for code generation. We didn’t have a strategy; we didn’t discuss the topic much, as far as I remember.
Even when Karpathy posted his famous article on vibe coding, it came at a time when the SOTA models couldn’t deliver good quality for me to use them at work for anything other than a search engine and a sparring partner for ideas. This means that in early 2025, we were using AI at work to a limited degree. We still didn’t have a policy on how to use AI, and anyone could use it, as long as they didn’t share code directly with it. You had to copy/paste and sanitise it carefully, which was clunky and didn’t change our ways of working much.
At that time, some of our colleagues began pushing for wider AI adoption at the company. We began having a weekly AI meeting, where a volunteer would present what they learned about AI recently, and how they put it to work. The most advanced use at the time was by a colleague who had set up various tools to read the source code and help him navigate the codebase. From today’s perspective, this sounds rudimentary, but let me remind you that this was merely a year and a half ago!
I work for a small-to-medium-sized Danish company. Our powerhouse has been delivering bespoke solutions to our clients’ problems.
It’s part of the job to switch between clients and projects. We strive not to do it often, but every now and then a project’s scope expands, shrinks, goes into maintenance mode, terminates, etc. When this happens, you usually get transferred to a new project within the same client or to a new client altogether. When this happens, you need to get up to speed, as you rarely get a greenfield project.
At our company, we rely on documentation. And I don’t mean only technical documentation, but also documentation used by designers, business people, sales, etc. Our source of truth is Confluence. Let’s not get into the details of why it’s Confluence: just take it for granted that if you want to orient yourself on a given project, or at our company in general, you go to Confluence. It contains hundreds of (mostly) well-maintained instructions in written or video form. Still, depending on the complexity of the project and the number of different tenants (could be different clients) on it, it could take an experienced person anything between a couple of days and several months to become an efficient contributor and team member. It is expected that you will deliver production code in your first or second week on a new project, but at that point you’d know a very tiny corner of the project, and you will expand your knowledge on the job.
When, while attending the regular weekly AI meeting, I saw how this colleague of ours had handcrafted tools so that he could get far more quickly into new codebases, it was a game-changing moment for me. And what he had done was basically a series of scripts that read a given repository or repositories, could read from Confluence or PDF files, and gave him a summary, or simply helped him find this or that. Again, not much, when seen from today’s perspective, but I could see how spending several dozen dollars per month could save you hours, if not days, especially when you are newly introduced to a project. And it had an even bigger payoff for principal engineers, who were expected to jump between projects and clients and help around or bootstrap a new initiative.
Still, when it came to coding contributions, the general notion at the company was that the models were not good enough. We had colleagues who relied heavily on AI to complete their tasks, but their work was largely frowned upon and for a good reason. At best, the code did what it was supposed to, but it often didn’t cover edge cases, didn’t integrate with or reuse existing features and APIs, and there were security issues on more than a handful of occasions. In general, the quality of such contributions was at the level of a junior to early mid-level developer. My team had a policy for code review, and I believe this did save us from some bugs and low-quality code reaching production. However, this became a source of a bit of contention between developers with inferior qualities and work ethics and the more experienced and reliable developers, who had to review and clean up after the former.
Looking back, I see it as an organisational mistake that we didn’t have open discussions around the use of AI at work and about good practices. Our weekly AI meeting was optional, and it had a bigger focus on showing cool things, and not enough focus on setting ground rules and establishing good practices.
Fast forward several months to Autumn 2025. I think it was around the time when Sonnet 4.5 and later when Opus 4.5 were released. I suddenly noticed a tangible improvement in AI-generated code, and there was a huge increase in the number of pet projects people around me started working on.
It was after the launch of Sonnet 4.6 that AI-coding became widely adopted in my team. Almost suddenly, we could see more and more examples of an agent fixing a bug or proposing a reliable architecture for a new feature.
It’s important to note that we have a frugal approach at our company, and we have never, to this day, had unlimited tokens or even wide adoption of Max-level subscriptions. At first, we had a handful of people with subscriptions; then it was rolled out company-wide, yet we received the basic subscription. This influenced the way we use AI and the predominant usage of cheaper models. One would use the most capable models sparingly and when it made sense. You’d use Opus to generate a plan or do research, then switch to Sonnet for the implementation.
Another curiosity is that we are not committed to a single provider. Most of my colleagues use the company-issued OpenAI subscriptions. Some, like me, use a personal Anthropic subscription that the company reimburses us for. This has allowed us to compare how models from each of the main AI labs do at a given time for given tasks. It also allowed us to more efficiently review each other’s changes using models to review PRs on our machines.
Speaking of reviews, that was my first widely adopted AI-related contribution at our company. In early November 2025, I introduced an AI code review as part of the pipeline running tests following the opening and subsequent pushes to a PR. At the time, Bitbucket’s Rovo Dev was expensive for us. To use it, you had to subscribe every user whose contributions you wanted to be reviewed by an AI tool. That was $20 per month per user. I would have had a hard time justifying the added cost, considering that we were paying the same for every developer’s AI subscription. This left me with the option of adding a custom implementation.
Get Vasil Manolov’s stories in your inbox
Join Medium for free to get updates from this writer.
Bitbucket allows you to define pipelines in a `bitbucket-pipelines.yml` file. I specifically targeted branch names prefixed by “ai-” so as to run my review only on select branches at first. The review consists of the following steps:
- Diff PR. We skip the review if the PR is huge, containing 2000+ LoC.
- Include a hand-crafted prompt for a code review.
- Include up to 10 changed files.
- Review.
- Post or update the PR comment. We always have 1 comment that gets updated (replaced) to keep it short and concise. If you want to see earlier comments, you can do so by checking your email inbox (I’ve never had to do that).
The restrictive rules for skipping are inspired twofold:
We strive to make small PRs that can be grasped by human reviewers. We are frugal, and a review on a huge PR could turn out to be expensive.
Later, I added options to skip the AI review based on the PR description or the latest commit message — in case the change you made didn’t require a review.
The end result costs ~$30 per month for the repositories my team uses. The maintenance cost it required so far is close to zero (I think I had to spend an hour on a couple of occasions to patch issues and verify it still works). When we add ~2 days I spent developing, testing, and refining it based on colleagues’ feedback, it paid off in the first month, if we compare it to the Revo approach we had as an alternative.
Close to a year later, my custom AI code review pipeline is still going strong. It was adopted in several more repositories. Recently, we added a flavour with a focus on security to even more repositories, used far outside of my team. I intend to switch between OpenAI and Anthropic models for the code review based on information in the commit: if it is obvious the change came from Claude, the reviewing model will be from OpenAI and vice versa. When I get to it, I am likely to relax the restrictions around PR size and number of changed files. We have had a handful of big PRs that I now regret were not reviewed by AI, and since the number of big PRs is minuscule at our company, I don’t expect the average monthly cost to change noticeably.
You might have figured that I care about the code quality in our application and am being wary of potential bugs being introduced.
Hands down, the most boring part of my job is testing. And not the exploratory testing or testing of specific features that are, in fact, interesting to me. I mean the grunt work of performing a regression test prior to releases, ensuring no bad bugs will be released to production. Like many organisations, we have a massive unit test suite and an acceptable integration test suite. However, these (mostly) cover our backends. For years, we’ve relied on Cypress and later Playwright to automate the testing of UIs and flows. These kinds of test suites are inherently flaky and require constant updates to ensure they reflect the reality of the codebase and the business requirements. The ultimate validation in my team has been the manual regression test performed on top of everything else that allowed us to go to bed feeling relaxed before upcoming huge releases. Equally, it has been the biggest toll on the people doing it. For the past couple of years or so, we have employed AI to our aid on this with varying success. Mostly in writing more tests or working on the stabilisation of existing ones.
Fast forward to the Spring/Summer of 2026 and the increased competence of AI models and harnesses. For me, it took a nudge from a developer team member, but eventually I relented and asked Claude to write manual test cases for the applications in our newest and most AI-empowered monorepo. I started with the biggest app and expected mediocre work at best. That app is written in React Native Expo and supports the mobile version for the web browser, an iOS app, and an Android app. Since the flow between these three is virtually the same, except for some specific native-only features and quirks, we’d test using the web browser, followed by a smoke test on an actual device. It took some back and forth until Claude figured out how to take screenshots with an appropriate viewport size, but just a few short hours after the first prompt, I was looking at a manual test suite that covered virtually all features and screens we have. It took a few more prompts to populate missing features in several screens and, more specifically, missing business rules for this or that behaviour. Then, Claude figured out what my intention was and did the same for the rest. The next phase took two days. A colleague and I meticulously went through all manual test cases and prompted Claude to further refine them, cross-referencing the Monday tickets with the new manual test cases. It took that long due to the sheer amount of material we now had at hand. Then, we validated it with developers working on various features and challenged them to find issues or missing cases. Either my colleagues wanted to make me feel good about my initiative, or Claude (and the subsequent manual sifting through its work and iterations) had done a pretty good job, because the number of issues found was a big fat 0.
The manual test cases are part of the monorepo and live in the app’s directory. They are MD files, which are easy to read for both people and agents. It may have been the old habit of using Confluence, but I wasn’t happy with the test cases being available only in the codebase. I knew they had to be there, but wanted them in Confluence as well. This is why we decided that the codebase will be the single source of truth, and I had Claude write a script that reads the MD files and persists them to Confluence. Whenever someone updates the manual test cases, they also update them in Confluence. You don’t need to have a Confluence MCP for that; you generate an API token and persist it in a `.env` file so the agent doesn’t have direct access to it.
The added benefit of having this documentation in Confluence is that any team member, even non-technical ones, can access and review it. It comes in handy when you onboard team members and business people at our company. Additionally, it allows us to easily showcase to the client why a certain feature behaves in a particular way, as we have oftentimes seen new employees there expect our apps to behave differently.
I wouldn’t like to rely on developers having to remember to update the manual test cases — as with all other documentation-adjacent matters, we have instructions in our `CLAUDE.MD` and `AGENTS.MD` files that instruct the agents to do so whenever a new feature is introduced, or an existing one is changed materially, including the looks. Since the test cases live in the codebase, we ensure with high confidence that they will be kept up to date. Over the past couple of months, I’ve done random checks and haven’t found any false or missing information in these files, despite the fairly large number of changes, mostly AI-generated, made to the repository.
Having that source of truth about the flows in our applications became an excellent foundation for building automated test suites. A QA engineer and I worked on the guidelines for our main test suites: smoke, regression, and full. Then, we let the agents write the tests and only did validation on their work, which still took ~3 days, as we discovered several issues and had them fixed along the way.
Back to our frugal nature, instead of prompting an agent to run the tests and report back, we built Jenkins pipelines that would run the Playwright test suites. The smoke suite is being run whenever a PR is merged into master. This happens from 0 to around 20 times per day. Next, we have the regression test suite that runs overnight (it takes just over 20 minutes, but we haven’t seen the need to run it more often than that). Lastly, we are still ironing out the full test suite, as there is more flakiness than you’d want to have. We aim to also run it nightly, potentially replacing the regression suite altogether, which might replace the smoke test, especially if we shave off a few more minutes from it.
Now that writing all these tests is pretty cheap and quick, we still need to figure out what makes the most sense for us. On one hand, I feel so much better for having considerably better and more stable test coverage, specifically the end-to-end frontend tests. On the other hand, I realise how much more we can increase the coverage and include the backoffice and other apps. But is the effort worth it? Should we, instead, focus on delivering value elsewhere, and get back at this in half a year, when who knows how much the models and tools would have progressed? Naturally, we don’t have the answers to these questions, and we will choose a path that makes sense to our organisation with the knowledge and skills we have now. Then revisit the decisions in the near future.
I might have become too dreamy, so let me bring you back to the near past. For different reasons and many excuses over the years, we managed to get ourselves stuck with Vue2 and Vuetify 1.8 apps in 2025. Yeap, quite ancient and out of support. You don’t want to know how much time we have spent on patching security vulnerabilities. Not to mention the effort it took to upgrade the necessary dependencies and not break (many) features or design elements just to be compliant with the required minimum Android SDK version. While doing experiments for migration to Vue3 and Pinia, we got lucky, as our client for these apps wanted to do a redesign and wanted to have a more native feel to the apps, instead of the current web vibe we had using Capacitor to wrap around our web applications. After performing a fairly good research, we chose React Native Expo as it provides a good balance between developer quality of life and the clients’ wishes and needs. Having no native developers, or the budget to hire such people, it only made sense for us to retool for React Native Expo and rewrite the apps. I can’t speak for my colleagues, but I will mention briefly that, based on their feedback, it was due to the capabilities of agents in early 2026 that allowed them to make the switch from web development.
As for me, the improvements to AI models and harnesses have allowed me to up my game at the DevOps/mobile development intersection. Instead of waiting for an open slot for one of our DevOps engineers, I took the initiative and, with the help of agents for research and later implementation, was able to add the necessary infrastructure for our new React Native Expo apps. I have experience with Jenkins and am old enough to have handled “admin tasks” back in the day before DevOps became widespread, but I certainly had no experience with building (the actual build process, not developing them) React Native Expo applications. A few short months later, I still don’t wait for DevOps to find time for my team. Instead, I own all of our app builds, including Jenkins pipelines, the Firebase app delivery for non-production apps, and getting them to TestFlight and the Google Play Store. To put a cherry on top, I recently added an over-the-air app update capability. A custom-built OTA capability that uses Jenkins and AWS S3 buckets to fit the frugal nature of our organisation, instead of the paid Expo service. And the time I put into it will pay off next month (3 months after I began working on it) as compared to the cost of building and distributing OTA app updates via Expo.
I have had my ups and downs in my AI adoption: hyped at the start of the LLMs, a short and personal AI winter, followed by a hot spring in the past year. Being a developer, turned manager, a.k.a. jack-of-all-trades, I do enjoy the current state where I make a high number of individual contributions with the help of agents. What is even better is the wide range of areas I am able to contribute to: features, bugfixes, DevOps contributions, QA automation. Let’s see how things will change over the coming months and years, and whether software development will look anything like what we are used to.
