There is no free software / open source license that will satisfy your requirements. These licenses generally focus on the freedoms of end users, they are not trying to restrict anyone. The freedom to use software for any purpose (including to compete with your offerings) is considered essential.
Using a freedom-preserving license with the purpose of restricting freedoms can work to some degree, but it will generally end with disappointment. It's perfectly acceptable if your software won't be open source, as long as it's not misleadingly labelled.
The AGPL is a copyleft license that is useful for preserving end user freedom in the context of SaaS software. If the software operator modifies the software, they must make the source code available to all remote users. This license used to be popular in the “we're open source but don't want SaaS competition” space. But that doesn't work: this license doesn't prevent SaaS offerings at all. It merely requires that users get access to the source code, but only if the source code was modified.
The Cryptographic Autonomy License (CAL) is an extension of this approach. It also enables the user to get access to their data stored by an SaaS offering, so that they could actually migrate to a self-hosted solution. While this limits lock-in effects of SaaS, this too doesn't prevent SaaS competition.
The CC-BY-NC / CC-BY-SA-NC licenses are not considered free or open source. They are very good licenses for creative works, but have two problems: They are less suitable for computer programs because they do not differentiate between the source code and other forms of the program. And the exact scope of the “NonCommercial” clause is fairly unclear.
There are other licenses that provide for some openness without the freedoms implied by open source. Those are sometimes called source-available. One of those approaches is to publish without any license, all rights reserved (see: what is the default license?). The SSPL is a non-free license that strongly discourages SaaS use, but is otherwise like the GPL. The Business Source License allows only non-production use, but automatically converts into an open-source license at some point in the future.
Could you create a suitable license yourself? I would beg you not to. Open source licenses have to consider copyright and contract law. They generally have to work in international jurisdictions, not just the legal system at home. They have to be good for the duration of copyright, which is often over a century in total. Drafting a good license is difficult.
There are many licenses that were written by well-intentioned amateurs, but this might introduce important oversights or even fatal flaws in their license mechanisms. Sometimes, they may fail to properly impose some license condition. Or they are written so unclearly that they might not be granting the rights they intend to grant. Sometimes, these licenses are created by recombining phrases and sections from existing licenses in a cargo-cult fashion, without understanding their connections. Such DIY licenses are often called a crayon license.
If you would like to create open source code without jeopardizing commercial interests, it might be better to move away from the license question to the question which parts can be published freely. Some parts of your software might be actually valuable to you – the functionality that sets you apart from competitors. And some code is unique to your situation – it doesn't matter if it's open source or not because it's useless to anyone else. However, there may be some modules that are useful but not particularly valuable to you, for example a configuration library or some frontend widget. Open-sourcing those might be possible without significant problems. You are most likely already making use of many open source tools and libraries. Instead of creating a new project, you could also consider with the maintenance of those (since this will ultimately also benefit you), or you could upstream new features that would help you with your work.
