GenAI risks in IT projects: a practical management guide

In today’s landscape, GenAI risks in IT projects have become a critical concern for any technology leader. Integrating generative artificial intelligence into corporate workflows promises unprecedented efficiency. This transition brings technical and operational challenges that we must mitigate from the planning phase. It is essential to understand how to protect the integrity of our projects before threats materialise.

The difference between a project that integrates GenAI successfully and one that fails almost always lies in the quality of the prior risk analysis. It is not about slowing innovation, but about building on a solid base that lets you scale with confidence.

GenAI risks in IT projects: why they are different

GenAI risks in IT projects are not like the traditional risks of a conventional deployment. In a classic project, we control known variables: infrastructure availability, software compatibility, delivery deadlines. However, with generative AI the threat catalogue expands into territories many teams have not yet mapped.

Model opacity and data dependency

The first differentiating factor is model opacity. An LLM is not a database we can audit record by record. Its answers are probabilistic, which introduces inherent uncertainty into any automated process. The acceptance criteria for a GenAI-based deliverable must include validations that would not be necessary in a traditional project.

In addition, the second factor is dependency on training data. If the model has been trained on biased, incomplete or outdated data, its outputs will inherit those flaws. For this reason, this has direct implications in regulated sectors such as banking, insurance or public administration, where a wrong decision based on an AI output can have serious legal and reputational consequences.

Accelerated obsolescence

The speed at which models evolve creates a risk of accelerated obsolescence. What works with one model today may require a complete reconfiguration when migrating to another. Managing GenAI risks in IT projects must treat this volatility as a constant, not an exception.

Main GenAI risks in IT projects: critical threats

Any software deployment based on generative models must be audited meticulously. In this context, monitoring data quality and output accuracy is only the starting point. According to the ENISA Threat Landscape 2024 report, AI-related risks are among the emerging threats with the greatest impact. These are the most relevant threats among the GenAI risks in IT projects that every Project Manager should have on their radar:

Hallucinations and prompt injection

Model hallucinations. LLMs generate plausible answers that can be completely incorrect. For example, in environments where critical decisions are made —diagnoses, financial reports, legal documentation— an undetected hallucination can have a devastating impact. Establishing human-validation processes is not optional: it is a mandatory step.

Prompt injection. On the other hand, an attacker can manipulate the model’s instructions to make it reveal confidential information or execute unauthorised actions. This attack vector is particularly dangerous in user-facing applications, where the input is not controlled.

Data leakage and algorithmic bias

Confidential data leakage. When teams use GenAI tools without clear policies, there is a risk that sensitive information —proprietary code, customer data, business strategies— ends up as part of an external model’s context. Because of this, the phenomenon of shadow AI, where employees use unauthorised tools, aggravates this problem.

Algorithmic bias. Likewise, models reproduce and amplify the biases present in their training data. In recruitment, credit-granting or risk-assessment processes, an undetected bias can lead to discrimination and to serious regulatory breaches.

If you want to go deeper into the landscape of AI-powered cyberthreats, I recommend my analysis on IT security 2026: proactive defence with applied AI.

GenAI risks in IT projects: control framework and the EU AI Act

The regulatory dimension is inseparable from the effective management of GenAI risks in IT projects. The EU AI Act establishes a risk-level classification system that directly affects how we plan and execute our projects:

Unacceptable risk: prohibited systems (social scoring, subliminal manipulation).
High risk: applications in critical sectors (health, justice, employment, infrastructure). They require conformity assessment, exhaustive technical documentation and human oversight.
Limited risk: transparency obligations (informing the user that they are interacting with an AI).
Minimal risk: free use with good-practice recommendations.

Implications for the Project Manager

For a Project Manager, this means the GenAI risk-analysis phase in IT projects must include a regulatory classification of the system before writing a single line of code. If your project falls into the high-risk category, you need an activity log, an impact assessment and human-oversight mechanisms by design.

This reality is complemented by regulations such as NIS2 and DORA, which reinforce operational-resilience and incident-management requirements in regulated sectors. If you work in banking or insurance, see my guide on DORA: the digital operational resilience regulation to understand the specific obligations.

In other words, regulatory compliance is not an add-on at the end of the project: it is a cross-cutting axis that must be built into every sprint. In addition, for a general overview of the regulatory landscape, I invite you to read my article on IT regulations 2026.

Mitigating GenAI risks in IT projects: a phased strategy

Without doubt, theory is necessary, but what defines a good PM is execution. This is a practical framework for managing GenAI risks in IT projects at each phase:

Initiation and planning phase

Initiation phase. Define the project’s risk appetite together with the sponsor. Classify the system according to the EU AI Act. Identify the data that will feed the model and assess its quality, provenance and sensitivity level. Establish the red lines: which decisions you will never delegate to AI without human validation.

Planning phase. Design specific controls: hallucination tests, bias audits, model-access policies, rollback procedures. Include in the communication plan a transparency strategy with stakeholders about the use of AI in the project. As the Project Management Institute (PMI) recommends, change management is as important as technical risk management.

Execution and closing phase

Execution phase. Implement continuous monitoring of the model’s outputs. In addition, set confidence thresholds below which the result requires human review, keeping a log of AI-related incidents (detected hallucinations, prompt-injection attempts, data leaks).

Closing phase. Finally, document the lessons learned specific to GenAI risks in IT projects. Assess whether the controls worked or need adjustment and produce a report of materialised risks vs. anticipated risks that serves as a baseline for future projects.

leadership must be transparent to foster a culture of responsible use. If you are interested in the professional profile that leads these projects, see my article on the AI Project Manager.

Metrics to control GenAI risks in IT projects

A risk-management strategy without metrics is a blind strategy. In the field of GenAI risks in IT projects, we need specific KPIs that go beyond the classic project indicators:

Essential KPIs

Hallucination rate. Measures the percentage of model outputs that, after human review, turn out to be incorrect or invented. This KPI must be measured every sprint and its trend must be decreasing.

Mean time to detect anomalies (MTTD). Also measures how long the team takes to identify a problematic output. Integration with SOAR and SIEM tools makes it possible to automate much of this detection.

Human-validation coverage. In addition, measures the percentage of critical GenAI-based decisions that go through human review before being executed. In the early phases of the project it should be 100%; as the model stabilises, it can be gradually reduced.

Incidents and visibility

AI-related security incidents. Record prompt-injection attempts, data leaks to the model and unauthorised uses (shadow AI). This record feeds the risk analysis of the next cycle.

The key is to integrate these metrics into the project’s existing dashboards, not to create a parallel system. This way, unified visibility allows management to make informed decisions about the level of risk the organisation is taking on.

GenAI risks in IT projects: lessons from AI Forge

In AI Forge, my artificial-intelligence lab, I have had the chance to experiment first-hand with many of these GenAI risks in real IT projects. Every functional prototype we build includes a risk analysis by design, applying the principles of the EU AI Act not as a bureaucratic obligation but as a quality tool.

Without doubt, the most valuable lesson I have drawn is that GenAI risks in IT projects are not static. What is a low risk today can escalate quickly if conditions change: a new model, an updated dataset, a modified regulation. For this reason, risk management must be a living process, integrated into the project’s continuous-improvement cycle.

Anticipate risk, do not chase it

Mastering this discipline allows professionals to lead digital transformation with confidence. If we learn to anticipate problems, we can harness the full potential of generative AI without compromising the security or reputation of our organisations.

In conclusion, the balance between technical experimentation and strategic control will define the leaders of the future.

Frequently asked questions about GenAI risks in IT projects

Risk management in IT projects with GenAI makes sense when it is tied to measurable objectives. I cover this in OKRs in IT projects.

This evolution falls within the trends in IT project management.

What are the main GenAI risks in IT projects?

The key risks are: (1) model hallucinations that generate false but believable information, (2) leakage of confidential data to LLM providers, (3) prompt injection and jailbreaking, (4) biases in outputs that affect critical decisions, (5) technological dependence on a single provider, (6) uncontrolled token costs, and (7) EU AI Act non-compliance due to a lack of traceability.

How do you manage the risk of hallucinations in GenAI systems?

To mitigate hallucinations, it is recommended to: implement RAG (Retrieval-Augmented Generation) with verified sources, human validation on critical outputs, per-answer confidence scoring, domain-specific fine-tuning, continuous evaluation with testing datasets, and contractual clauses that limit use to non-critical tasks until reliability is validated.

What is prompt injection and how is it prevented?

Prompt injection is an attack where a user embeds malicious instructions in the input to manipulate the LLM’s behaviour. It is prevented with: strict input validation, a clear separation between system instructions and user data, the use of models with guardrails (Anthropic Claude, OpenAI moderation), execution sandboxing, and human review in sensitive flows.

Keep reading

If you are concerned about the security of AI-generated code, I have a full analysis of the secure development lifecycle (SDLC) with AI.

A programme to run with little margin for error? See how I have done it.

See the nine case studies

Managing GenAI risks in IT projects: a practical guide