Designing and Developing a Scalable Small Language Model (SLM) on Microsoft Azure

By Dr. Magesh Kasthuri

Small Language Models, commonly called SLMs, are becoming a practical alternative to very large general-purpose models for enterprises that need sharper domain behavior, lower latency, better cost control, and stronger governance. In an investment banking environment, these advantages are particularly valuable because the model must interpret complex financial language, respect confidentiality boundaries, follow regulatory expectations, and provide traceable outputs for high-value workflows such as deal screening, research summarization, risk review, compliance checks, and client pitch preparation.

This article explains how to design and develop a scalable SLM on the Microsoft Azure platform using Azure AI Foundry, Azure Machine Learning, Azure OpenAI capabilities, and a multi-agent engineering pattern that combines Microsoft Agent Framework concepts evolved from AutoGen, CrewAI-style role collaboration, and LangGraph-style stateful orchestration. The focus is an investment banking use case (as an example to practically explain the framework development) where the SLM supports analysts, relationship managers, risk officers, and compliance teams through a controlled, secure, and auditable agentic workflow.

1. Why an SLM Makes Sense for Investment Banking

Investment banking work is document-heavy, time-sensitive, and highly specialized. Analysts review annual reports, earnings transcripts, pitch books, regulatory filings, covenant documents, term sheets, sector reports, rating notes, internal risk memos, and client communications. A generic large model can help with broad reasoning, but it may not consistently understand the firm’s terminology, approval rules, formatting conventions, product taxonomy, risk language, and compliance expectations. A carefully designed SLM can be tuned for those patterns without carrying the operational cost of a very large model for every request.

Figure: Focussed SLM for Investment Banking Agent development

The goal is not to replace bankers or risk professionals. Instead, the SLM acts as a controlled assistant that accelerates reading, classification, extraction, drafting, and decision support. It can summarize market updates, identify deal risks, extract financial covenants, compare comparable companies, generate first-draft investment committee notes, or flag missing compliance evidence. The most useful SLM is therefore narrow enough to be dependable, yet scalable enough to serve several banking functions through secure APIs and agentic workflows.

2. Target Use Case: AI-Assisted Deal Intelligence and Risk Review

Consider a corporate finance team evaluating a potential debt financing transaction for a large manufacturing client. The team needs to process client financials, public filings, industry reports, credit rating notes, sanctions screening summaries, exposure data, ESG disclosures, and internal risk policies. The process usually requires repeated manual reading, cross-checking, and drafting. The proposed SLM-based solution supports the workflow through a set of specialized agents that prepare a preliminary deal intelligence pack for human review.

• Document ingestion: Collect and normalize financial statements, investor presentations, annual reports, internal policies, and analyst notes.
• Financial extraction: Identify revenue, EBITDA, leverage ratios, liquidity metrics, debt maturity profile, covenant thresholds, and sector-specific risk indicators.
• Risk interpretation: Compare extracted signals with risk appetite rules, credit policy thresholds, concentration limits, and regulatory obligations.
• Compliance review: Flag missing KYC evidence, sanctions concerns, market abuse risks, data handling constraints, and suitability considerations.
• Output generation: Draft a deal briefing note, exception summary, questions for the client, and an investment committee-ready executive summary.

3. Core Design Principles for a Scalable Azure SLM

A scalable SLM for investment banking must be designed with a strong balance between model quality, latency, explainability, security, and operational resilience. The architecture should avoid the common trap of treating fine-tuning as the entire solution. In practice, the model is only one component. Data governance, retrieval quality, agent orchestration, evaluation, monitoring, and human approval gates are equally important.

1. Domain specialization before model expansion: Start with a compact model and improve it with high-quality banking data, retrieval augmentation, and task-specific fine-tuning before moving to larger models.
2. Secure data boundaries: Separate public market data, licensed research, confidential client data, and internal policies using access controls, private networking, and classification labels.
3. Retrieval-first reasoning: Use retrieval augmented generation for facts that change frequently, such as market updates, credit ratings, regulations, and internal policy versions.
4. Fine-tune only where behavior must be consistent: Fine-tune the SLM for repeatable tasks such as covenant extraction, risk memo formatting, tone control, classification, and structured output generation.
5. Agentic decomposition: Break the workflow into specialized agents rather than asking one model to perform every step in a single prompt.
6. Auditability by design: Capture prompts, retrieved sources, model versions, tool calls, approvals, exceptions, and generated outputs for later review.
7. Human oversight: Keep final business decisions with authorized professionals, especially for credit approval, suitability, regulatory interpretation, and client-facing recommendations.

4. Azure-Based Reference Architecture

The reference architecture uses Microsoft Azure as the secure foundation for model development, deployment, orchestration, monitoring, and governance. Azure AI Foundry and Azure Machine Learning support model experimentation, fine-tuning, evaluation, and deployment. Azure OpenAI or compatible model endpoints provide managed inference where appropriate. Azure AI Search enables enterprise retrieval, while Azure Data Lake Storage, Microsoft Purview, Key Vault, Managed Identity, Private Link, Azure Kubernetes Service, Container Apps, API Management, and Azure Monitor provide the surrounding enterprise controls. This reference architecture is depicted in below figure.

Figure: Azure AI Reference Architecture for Investment Banking Agent

 

Architecture Layer	Azure Capability	Role in the SLM Platform
Data ingestion	Azure Data Factory, Event Grid, Logic Apps	Ingest public filings, research documents, internal policies, deal documents, and operational events.
Data storage	Azure Data Lake Storage Gen2, Azure SQL Database, Cosmos DB	Store raw, curated, structured, and semi-structured banking data with lifecycle policies.
Data governance	Microsoft Purview, sensitivity labels, data catalog	Classify data, manage lineage, define retention, and enforce governance policies.
Model development	Azure AI Foundry, Azure Machine Learning	Experiment, fine-tune, evaluate, register, and manage SLM versions.
Vector retrieval	Azure AI Search with vector and hybrid search	Retrieve relevant policies, filings, research notes, and historical deal material.
Agent orchestration	Microsoft Agent Framework, CrewAI services, LangGraph runtime	Coordinate specialized agents, stateful workflows, tool calls, approvals, and exception handling.
Application runtime	Azure Kubernetes Service, Azure Container Apps, Azure Functions	Host APIs, workers, orchestration services, and event-driven processing components.
Security	Microsoft Entra ID, Managed Identity, Key Vault, Private Link, Defender for Cloud	Protect identities, secrets, network access, workloads, and runtime posture.
Observability	Azure Monitor, Application Insights, Log Analytics	Track latency, failures, token usage, retrieval quality, agent traces, and model behavior.
Serving and access	API Management, Front Door, Private endpoints	Expose controlled APIs to banking applications, analyst portals, and internal workflow tools.

 

5. Logical Architecture Flow

The platform begins with data acquisition from approved sources. Documents are scanned, classified, cleaned, chunked, enriched with metadata, and indexed into Azure AI Search. Sensitive fields are masked or tokenized where required. Training data is curated separately and reviewed by subject matter experts before it is used for fine-tuning. The SLM is then trained or adapted using supervised fine-tuning methods, evaluated through banking-specific benchmarks, and deployed behind controlled inference endpoints.

At runtime, a user request enters through an analyst portal or workflow application. API Management authenticates the request, applies throttling, and routes it to the orchestration layer. LangGraph manages the stateful workflow. CrewAI-style agents perform specialized roles such as document analyst, financial analyst, risk reviewer, compliance checker, and report writer. Microsoft Agent Framework provides Azure-native agent composition, tool integration, conversation control, and production hosting patterns. The SLM performs domain-specific reasoning, while retrieval tools ground the response in approved enterprise content. If confidence is low or a policy threshold is breached, the workflow pauses for human approval.

6. Combining Microsoft Agent Framework, CrewAI, and LangGraph

The three frameworks should not be used as overlapping orchestration layers without clear boundaries. A clean design assigns each framework a distinct architectural responsibility. Microsoft Agent Framework is the Azure-aligned foundation for production-grade agents, tool integration, enterprise hosting, and operational control. CrewAI is useful for modeling business roles and collaborative responsibilities in a way that business stakeholders can understand. LangGraph provides deterministic, stateful orchestration when the workflow must be auditable, resumable, and governed through explicit transitions. This is depicted in below Reference architecture diagram.

Figure: Azure AI Multi-agent architecture with SLM for Investment Banking Agent

 

Framework	Recommended Responsibility	Investment Banking Example
Microsoft Agent Framework / AutoGen concepts	Enterprise-ready agent composition, tool access, human interaction, and Azure-native hosting.	Connect agents to Azure AI Search, internal APIs, approval systems, and model endpoints.
CrewAI	Role-based task decomposition and collaboration among domain-specific agents.	Define agents such as Credit Analyst, Compliance Reviewer, Market Research Analyst, and Memo Writer.
LangGraph	Stateful orchestration with conditional routes, checkpoints, retries, and human-in-the-loop controls.	Move the deal review from intake to extraction, risk scoring, compliance validation, memo drafting, and approval.

 

7. Agent Design for the Banking Workflow

A scalable SLM solution should use specialized agents rather than a single generic assistant. Each agent must have a narrow mandate, limited tools, clear input and output contracts, and a defined escalation path. This keeps the workflow understandable and reduces the risk of uncontrolled reasoning.

Figure: Logical Architecture for Multi-agent Investment Banking usecase

The following agent model is suitable for the deal intelligence scenario.

• Intake Agent: Validates the deal request, checks mandatory metadata, verifies access rights, and creates the workflow case.
• Document Analyst Agent: Reads filings, term sheets, presentations, and internal documents; extracts relevant sections; and prepares source-grounded notes.
• Financial Metrics Agent: Extracts financial indicators, normalizes values, calculates ratios, and highlights unusual movements.
• Risk Policy Agent: Compares the transaction profile against internal credit policy, risk appetite, exposure thresholds, and product rules.
• Compliance Agent: Checks evidence for KYC, sanctions screening, conflicts, market conduct obligations, and restricted-list considerations.
• Market Intelligence Agent: Summarizes sector trends, peer performance, rating movements, and macro factors from approved sources.
• Report Writer Agent: Produces the draft deal intelligence memo, exception log, open questions, and executive summary.
• Reviewer Agent: Performs consistency checks, identifies unsupported statements, and routes the case for human approval when required.

8. Data Preparation and Fine-Tuning Strategy

The quality of the SLM depends heavily on the quality of the training and evaluation data. For investment banking, the dataset should include sanitized historical deal summaries, approved credit memos, policy interpretation examples, financial extraction samples, compliance decision records, analyst question-answer pairs, and structured report templates. Data must be anonymized or pseudonymized before training unless the model is trained in a tightly controlled private environment with explicit approval.

Fine-tuning should focus on repeatable behavior rather than volatile factual knowledge. For example, it is reasonable to fine-tune the model to produce a risk memo in the firm’s preferred format, extract covenant fields consistently, classify a transaction type, or generate a structured exception summary. It is usually better to retrieve current market data, rating updates, and changing policies through Azure AI Search rather than embedding them into model weights. This separation improves freshness, reduces retraining pressure, and supports governance.

1. Collect: Gather approved domain examples from historical workflows, templates, and subject matter experts.
2. Clean: Remove duplicates, outdated language, confidential identifiers, and inconsistent labels.
3. Label: Tag examples by task, document type, risk category, product, region, and expected output format.
4. Split: Maintain separate training, validation, test, and adversarial evaluation sets.
5. Fine-tune: Use Azure AI Foundry or Azure Machine Learning to adapt the selected SLM with supervised examples.
6. Evaluate: Measure extraction accuracy, groundedness, formatting compliance, refusal quality, latency, and cost per workflow.
7. Register: Store model lineage, datasets, parameters, evaluation results, approval records, and deployment metadata.

9. Retrieval Augmented Generation (RAG) Pattern

Retrieval augmented generation is essential for this architecture because investment banking decisions rely on current and verifiable information. Azure AI Search can combine keyword, semantic, and vector retrieval so that the SLM receives the most relevant passages from approved documents. Every retrieved passage should carry metadata such as document owner, source system, classification, effective date, region, product line, and retention rule. This metadata allows the orchestration layer to decide whether the content can be used for a specific user, client, jurisdiction, or transaction type.

The SLM should be prompted to answer only from retrieved and authorized content for factual claims. If the evidence is missing, stale, or contradictory, the agent should ask for clarification or route the item to a human reviewer. This behavior is especially important for risk policy interpretation, regulatory language, and client-specific commitments.

10. Security, Compliance, and Responsible AI Controls

The architecture must assume that banking data is highly sensitive. Microsoft Entra ID should control authentication and authorization, while Managed Identity should remove the need for hard-coded secrets. Azure Key Vault should store secrets, certificates, and encryption keys. Private Link and virtual network integration should keep model endpoints, data stores, and search indexes away from the public internet. Microsoft Purview should classify documents, manage lineage, and support data governance reviews.

Responsible AI controls should be implemented as engineering features, not as afterthoughts. The system should filter unsafe requests, block unauthorized data access, enforce grounded responses, detect prompt injection attempts, maintain audit logs, and measure hallucination risk. Human approval should be mandatory for final credit recommendations, regulatory interpretations, exception approvals, and client-facing material. A model risk management function should review model purpose, limitations, validation results, change records, and operational incidents.

11. Deployment and Scaling Approach

The SLM platform should be deployed through a staged release model. Development teams can begin in a sandbox environment using synthetic or anonymized data. The next stage should validate the model with restricted historical cases. Production deployment should use blue-green or canary release patterns so that new model versions can be compared against existing versions before broad adoption. All deployment artifacts should be versioned, including prompts, agent definitions, retrieval settings, model packages, evaluation scripts, and infrastructure templates.

For scale, the architecture can use Azure Kubernetes Service for long-running orchestration services and Azure Container Apps or Azure Functions for event-driven components. API Management should enforce request limits, subscription controls, and usage policies. Azure Monitor and Application Insights should track latency, throughput, tool-call failures, retrieval miss rates, model confidence, approval delays, and token consumption. When workload demand grows, autoscaling should be applied independently to ingestion workers, vector indexing jobs, orchestration services, and inference endpoints.

12. Evaluation Framework

A banking SLM should be evaluated beyond generic language quality. The evaluation framework must check whether the model extracts financial facts correctly, follows internal templates, refuses unsupported requests, cites or references approved evidence, respects access boundaries, and escalates ambiguous cases. The evaluation should include normal cases, edge cases, adversarial prompts, conflicting documents, stale policy examples, missing data, and jurisdiction-specific variations.

Evaluation Area	Example Metric	Why It Matters
Extraction accuracy	Precision and recall for financial fields	Prevents incorrect deal analysis caused by wrong financial values.
Groundedness	Percentage of claims supported by retrieved evidence	Reduces hallucination risk and improves audit confidence.
Policy compliance	Correct classification of risk and compliance conditions	Ensures the workflow follows internal rules and regulatory expectations.
Format adherence	Template completion score	Improves usability for investment committee and review teams.
Security behavior	Unauthorized data refusal rate	Protects confidential information and supports access control.
Operational efficiency	Latency, throughput, cost per case	Confirms that the SLM is practical at enterprise scale.

13. Key Architecture Benefits

The proposed architecture gives the bank a practical path to domain-specific intelligence without losing control over data, risk, and operations. The SLM reduces cost and latency for repeated banking tasks. Retrieval keeps factual responses current and traceable. Agentic decomposition makes the workflow easier to govern. LangGraph adds reliable state control, CrewAI-style role modeling improves business readability, and Microsoft Agent Framework aligns the agent layer with Azure-native enterprise deployment. Together, these choices create a platform that can begin with a narrow use case and gradually expand into a broader banking intelligence fabric.

Conclusion

A scalable SLM for investment banking should be treated as a secure enterprise platform rather than a standalone model experiment. The strongest design combines curated domain data, retrieval augmented generation, selective fine-tuning, role-based agents, explicit workflow state, and strong Azure governance controls. Microsoft Azure provides the foundation for this approach through AI development services, managed infrastructure, security capabilities, observability, and enterprise integration patterns.

For investment banking teams, the real value comes from disciplined design: a model that knows the domain, agents that understand their responsibilities, workflows that can be audited, and controls that protect the institution. When implemented carefully, the SLM can help bankers work faster, reduce repetitive analysis, improve consistency, and strengthen review quality while keeping final judgment where it belongs—with accountable human experts.