Fine-Tuning LLMs for GDPR Compliance Detection in Complex Legal Texts

Introduction & Research Problem

The European Union's General Data Protection Regulation (GDPR) imposes strict mandates on how consent is obtained from data subjects. Specifically:

• Article 4(11) requires consent to be freely given, specific, informed, and an unambiguous indication of the data subject's wishes.
• Article 7 outlines conditions for consent, including clear affirmative action and easy withdrawal mechanisms.

Evaluating whether enterprise privacy policies and terms of service meet these complex legal criteria requires processing thousands of pages of unstructured text. Traditional rule-based regex parsers fall short due to linguistic ambiguity, while off-the-shelf LLMs often hallucinate or fail on nuanced legal terms.

My Master's thesis, titled "Scientific Evaluation and Optimisation of AI Models for GDPR Articles 4(11) and 7 Compliance Detection in Legal Texts" (Grade: 2.7 / Good), established a modular evaluation framework comparing prompt engineering against domain-adapted fine-tuned LLMs.

---

Technical Approach & Architecture

To achieve accurate legal text classification while maintaining computationally viable training costs, I utilized Parameter-Efficient Fine-Tuning (PEFT) via Low-Rank Adaptation (LoRA).

                      +-----------------------------+
                      | Raw Legal Privacy Policies  |
                      +-----------------------------+
                                     |
                                     v
                      +-----------------------------+
                      | Tokenization & Preprocessing|
                      |   (LegalBERT / LLaMA)       |
                      +-----------------------------+
                                     |
                                     v
                      +-----------------------------+
                      |  LoRA Adapter Injection     |
                      |  (Rank r=8, alpha=16)       |
                      +-----------------------------+
                                     |
                                     v
                      +-----------------------------+
                      | Cross-Entropy Loss &        |
                      | Optimization (AdamW)        |
                      +-----------------------------+
                                     |
                                     v
                      +-----------------------------+
                      | GDPR Compliance Classifier  |
                      | (Articles 4(11) & 7 Labels) |
                      +-----------------------------+

---

1. Model Selection & Base Architecture

We benchmarked three distinct model paradigms:

1. Zero-Shot & Few-Shot Prompting: GPT-4 / Claude baseline prompting with legal context injection.
2. Domain-Specific Transformer Encoder: LegalBERT pretrained on EU legislation and court decisions.
3. Generative Autoregressive Models: LLaMA-2 7B / 13B with quantized 4-bit precision (QLoRA).

---

2. LoRA Fine-Tuning Code Implementation

Below is a snippet demonstrating how low-rank adaptation matrix parameters were configured using Hugging Face peft and transformers:

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import LoraConfig, get_peft_model, TaskType

model_id = "nlpaueb/legal-bert-base-uncased"

# Load base model for binary compliance classification
model = AutoModelForSequenceClassification.from_pretrained(
    model_id, 
    num_labels=2
)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Define LoRA Configuration
peft_config = LoraConfig(
    task_type=TaskType.SEQ_CLS,
    r=8,
    lora_alpha=16,
    lora_dropout=0.1,
    target_modules=["query", "value"]
)

model = get_peft_model(model, peft_config)
model.print_trainable_parameters()
# Output: trainable params: 294,914 || all params: 109,777,154 || trainable%: 0.268%

---

3. Evaluation & Results

The fine-tuned models were evaluated on a custom dataset of 1,200 manually annotated privacy policy segments from European e-commerce and SaaS platforms.

| Model Strategy | F1-Score (Article 4(11)) | F1-Score (Article 7) | Latency (ms/clause) | |---|---|---|---| | Zero-Shot Prompting (Baseline) | 0.68 | 0.62 | 1,450 | | Standard LegalBERT | 0.81 | 0.79 | 45 | | Fine-Tuned LegalBERT (LoRA) | 0.91 | 0.88 | 42 | | QLoRA LLaMA-2 (7B) | 0.89 | 0.86 | 320 |

Key Findings:

• Efficiency: LoRA fine-tuning trained only 0.27% of total parameters while outperforming zero-shot LLM prompts by over 23% in F1-score.
• Explainability: Combining BERT attention heads with rule-based post-hoc explanations provided auditors with clear visual highlighting of non-compliant clauses.
• Privacy: Local deployment ensured sensitive legal drafts were processed entirely within on-premise infrastructure, eliminating data leak risks.