In this programme, we spoke to András Érczfalvi, a presenter at TrendFM, about shadow AI. The programme was broadcast on 4 May 2026.
Participants
András Érczfalvi – TrendFM
András Tresch - Quattrosoft
13 per cent – that’s how many see it. And what about the rest?
According to IBM’s 2025 study, only 13 per cent of companies have identified a security incident relating to their own AI. At first glance, this figure is reassuring. The reality, however, paints a very different picture: the same study also revealed that 97 per cent of companies lack adequate AI governance, and what happens without governance generally never comes to light.
I discussed this with András Érczfalvi on 4 May 2026 on TrendFM’s programme ‘Monitor Délután’. The recording has since been made available on the TrendFM website: What is Shadow AI? When is artificial intelligence safe?
The discussion covered several areas of risk that are already having a measurable impact on Hungarian companies in terms of financial loss, customer trust and operational stability. As a manager, it is worth being aware of each of these, as they are already relevant and none of them will resolve themselves.
The first risk: shadow AI
‘Shadow AI’ is the phenomenon where colleagues start using AI tools in their work on their own initiative. Nobody stops them, and nobody tells them how to do it. They simply give it a go, find it convenient, and carry on from there.
The problem begins with the fact that, by default, most free AI tools allow the content of conversations to be used for further training of the model. One colleague, with the best of intentions, uploads a 50,000-line customer list for analysis. Another colleague uploads a confidential financial report. A third copies in the source code for debugging.
This data leaves the company network, often ending up on servers outside the European Union, and from there there is no turning back. From the perspective of the GDPR and NIS2, this is not a grey area, but a clear risk.
A ban is not the solution. Anyone who blocks the use of AI will only succeed in driving colleagues to use it on their own phones, via a VPN or on an alternative platform. People are creative when it comes to getting round the rules.
The effective response is for the company to provide a controlled AI tool, accompanied by clear rules, and to specify exactly what data may be used. This is precisely what we do at Quattrosoft: we have our own in-house AI assistant, and there are also regulations governing the use of external models.
The second risk: AI itself can be hacked
This comes as a shock to many, given that AI providers claim everything is secure, that they do not use the data for training, and that the infrastructure is protected. All of this remains true until the source code is leaked or a configuration error is discovered.
There have been two incidents this year that every manager should be aware of.
In one case, the complete source code of a major AI company’s coding assistant was made public due to an incorrect permissions setting. They tried to take it down, but once something is on the internet, it stays there. Competitors began analysing it within hours.
The other case is even more exciting. A new-generation language model was released which is so effective at finding software vulnerabilities that it was not made publicly available. Microsoft, Apple and a few other major companies are having their own source code analysed by Mythos before anyone else can access it. The question now is no longer how severe the vulnerability is, but how many hours it will take for an attacker to exploit it.
Why is this of interest to a Hungarian executive? Because a significant proportion of firewalls, routers and security devices run on Linux-based systems, and Linux’s source code is public. New types of AI tools can uncover security vulnerabilities that were previously invisible. The security model based on the premise that ‘the internal network cannot be breached’ will only remain effective for a limited time.
As a manager, it is worth asking yourself a specific question here: if anyone were to gain access to your internal network tomorrow, through which systems could they access confidential data, and what damage could this cause?
The 30 June 2026 audit deadline under the NIS2 regulation requires precisely this kind of deliberate risk assessment. Anyone who has not yet carried out this work risks a fine.
The third risk: AI agents that know too much
2026 turned out to be the year of AI agents. An agent is not simply a chatbot. An agent is granted access to systems, databases and email accounts, and carries out tasks independently.
In July 2026, Replit’s (a major developer platform) AI agent deleted a partner company’s entire live database in nine minutes. Nothing worked for three hours, and three months’ worth of data was lost. The backups were stored on the same server, so the agent deleted those as well. Only an offline backup from three months earlier remained, which they were able to restore. Both customer and partner data were lost.
When the AI agent was asked why it had done this, its response was that it hadn’t read the documentation ‘out of laziness’. In reality, of course, this is not a matter of laziness, but of optimisation: the service provider is saving on resources, and the system decides for itself what to read and what not to read.
The lesson to be learnt from this can be summarised in a single sentence: never grant AI agents permissions that could allow them to cause irreparable damage. They must not be able to delete data, transfer money or finalise contracts. The AI agent should prepare the decision, gather the relevant information and formulate the response, but the final step must always be taken by a human.
Large corporations have, in fact, been practising this for decades: the accountant does not make the transfer, but merely prepares it. The finance director approves it. The same logic must be applied to AI agents. Only now we are no longer talking about 100 transactions a year, but 100 per minute.
Risk number four: AI-generated code
We also discussed a fourth topic, which relates to software development but affects any company that operates or develops its own software, either independently or with an external partner.
Code written by AI assistants contains one and a half to twice as many errors and security vulnerabilities as code written by humans. As a result, the role of junior developers is diminishing, whilst the value of senior developers and code review is increasing significantly.
If your company is working on development projects, it is worth finding out how code generated by AI is reviewed, and what security protocols are in place for uploading source code to the AI. At Quattrosoft, our colleagues use AI, but every piece of generated code is reviewed by a senior developer before it goes into testing.
Fifth risk: Deepfake fraud is already here
By 2026, it will be possible to create a video call simulation in a matter of minutes, in which a ‘finance director’ gives instructions for an urgent bank transfer. The voice, the image, the manner – everything is convincing. According to Gartner’s forecast, by 2028 neither humans nor machines will be able to distinguish generated content from the real thing.
If your company does not yet have multi-factor authorisation in place for larger financial transactions, now is the time to introduce it. Not tomorrow. Now.
What should you do as a manager?
In the radio programme, I highlighted four specific steps that a manager at a large company can take as early as the coming weeks.
- Take a look at what your colleagues are actually using. An internal survey, a few conversations and an analysis of network traffic will give you an accurate picture. Most managers are surprised by the number of different AI tools being used simultaneously.
- Choose one AI tool that the company provides to staff. Draw up a set of guidelines for it: what data can be uploaded, what cannot, how customer information should be handled, and who has access to what. A ban is not an option; controlled use is the solution.
- Where an AI agent is used, access rights must be strictly limited on a role-based basis. The agent should only prepare data; it should not make decisions. There must always be human oversight for financial transactions and changes to customer data.
- Think carefully about the architecture. There are two ways to work with AI. One approach is to open up our system to AI and grant it access to the data. This is where disasters like the Replit case can occur. The other way is for our own system to query the AI with specific questions and receive only the answer in return. In this case, the potential risk is an incorrect answer, not data loss. The difference lies in who retains control. As a leader, it is worth understanding this architectural decision, as it has serious business implications.
Closing remarks
As I often say: AI isn’t black and white. I consider myself an AI realist; AI offers incredible business opportunities if we know what to use it for, how to use it, and what data to use. Uncontrolled use, however, carries extraordinary risks.
The full conversation with András Érczfalvi lasts around 25 minutes, and specific business examples are given throughout. As a manager, if you take the time to listen to it now, you’re likely to identify at least two areas of risk that you may need to address as early as tomorrow.
Listen to the full programme here: What is Shadow AI? When is artificial intelligence safe? – TrendFM Monitor Afternoon
And if you’d like to assess where your own company stands in terms of AI security, process digitalisation and regulatory compliance, please don’t hesitate to get in touch. Quattrosoft has been helping large enterprises make their operations transparent, efficient and secure for 30 years.
