By Oliver Cronk and David Rees, Scott Logic
The ESG Data Challenge: When Rules-Based Systems Fall Short
The sustainable investment sector faces a critical and growing challenge: effectively monitoring and validating Environmental, Social, and Governance (ESG) data in a landscape where greenwashing has become increasingly sophisticated. Traditional methods rely on deterministic rules created by human analysts – an approach that, while valuable, struggles to adapt to the dynamic and expanding nature of ESG reporting.
Consider a practical scenario: when assessing a company’s materiality disclosures against frameworks like the Taskforce on Nature-related Financial Disclosures (TNFD) or Global Reporting Initiative (GRI), rule-based systems might identify the presence or absence of required metrics. However, these frameworks merely illustrate what could be material to a company within a given industry; determining what is genuinely material remains the analyst’s responsibility in a highly complex, specialised field.
This challenge becomes particularly acute when evaluating companies that operate across multiple countries and sectors, where ESG reporting requirements and materiality frameworks often don’t align across different regulatory environments.
The Architectural Dilemma: Agentic vs Deterministic Approaches
When Scott Logic developed InferESG to address these challenges, we faced a pivotal architectural decision. Our initial approach leveraged a conversational, agentic AI framework – when presented with a question, an Intent Agent would decompose it into smaller prompts, a Supervisor would determine which specialist agents to engage, and the system would collate the responses.
While this architecture worked well for general ESG queries, we discovered it wasn’t ideal for the systematic analysis of sustainability reports. The Intent Agent, which excelled at breaking down conversational queries, struggled with the complexity and scope of full report analysis, often interpreting tasks inconsistently and providing incomplete coverage of key ESG topics.
This led to a crucial realisation: for report analysis, we already knew exactly what needed evaluation and the expected format of the response – so why rely on a Large Language Model (LLM) to determine this?
A Hybrid Solution: Combining Architectural Strengths
Rather than choosing between competing approaches, we developed a hybrid architecture that leverages the strengths of both deterministic workflows and agentic AI:
For report analysis: We implemented a structured workflow that removes the Intent Agent and Supervisor from the process, instead providing our own intention through a report workflow. This orchestrates the process using the uploaded sustainability file, synchronously chaining prompts and agents to obtain the company name and relevant materiality topics, then asynchronously producing a comprehensive analysis of environmental, social, and governance aspects.
For interactive exploration: We maintained the conversational, agentic architecture as a core component of the solution. After reviewing the initial structured report, analysts can ask follow-up questions like, “How does this company’s emissions reduction claims compare to their industry peers?” Here, the Intent Agent excels, breaking complex queries into specific tasks while the Supervisor orchestrates these across specialist agents.
This hybrid approach offers several key advantages:
- Consistency: The deterministic workflow ensures thorough coverage of all essential aspects of ESG analysis
- Flexibility: The agentic component allows for dynamic exploration of specific areas of interest
- Transparency: Clear workflow stages make it easier to understand and validate the analysis process
- Adaptability: The system can easily incorporate new frameworks as ESG standards evolve
The Technology Implementation
At the heart of InferESG lies an architecture that constrains AI agents within a hierarchical structure. For the report analysis component, a Report Agent coordinates the analysis process to:
- Determine the company and industry type
- Generate an overview of the sustainability document
- Conduct a materiality assessment
- Analyse featured environmental, social and governance topics
The system maintains a “scratchpad” feature that creates a detailed record of reasoning from each agent, providing a transparent audit trail that analysts can follow to understand exactly how conclusions were reached.
For the conversational component, when an analyst asks a question like “What external evidence exists to support or contradict these carbon reduction claims?”, the Intent Agent determines that the Web Agent should search for current news, regulatory filings, and other trusted public sources to provide context and verification alongside use of other the agents where appropriate.
Beyond ESG: Broader Architectural Implications
While developed for ESG analysis, this hybrid architecture carries broader implications for enterprise architects working across various domains. The key insight is that sometimes the best solution isn’t choosing between different approaches, but rather finding ways to combine their strengths.
This becomes particularly relevant as organisations increasingly deploy AI systems to solve complex analytical challenges. For tasks with well-defined parameters and expected outputs, deterministic workflows provide consistency and reliability. For exploratory analysis and unforeseen queries, agentic AI offers flexibility and adaptability.
By marrying these approaches, enterprise architects can build systems that maintain human oversight while leveraging AI to handle data-intensive tasks – keeping human analysts firmly in the driver’s seat with AI serving as powerful analytical tools rather than autonomous decision-makers.
As we navigate the rapidly evolving landscape of AI implementation, this balanced approach offers a valuable pathway forward: leveraging the transformative potential of AI while maintaining the transparency, consistency, and human judgment essential for effective enterprise governance.
Conclusion
The development of InferESG demonstrates how thoughtful architectural decisions can transform AI from a potential risk into a powerful ally for complex analytical tasks. By combining deterministic workflows with agentic AI, we’ve created a system that enhances rather than replaces human judgment – a principle that extends well beyond ESG analysis to inform how organisations can responsibly harness AI across various enterprise functions.
As enterprise architects and governance practitioners consider their own AI implementations, this hybrid approach offers a valuable template: one that embraces AI’s analytical capabilities while maintaining the human oversight necessary for effective governance and decision-making.
For more information on InferESG you can check out the GitHub repository: https://github.com/ScottLogic/InferESG or the case study: https://www.scottlogic.com/our-work/inferesg-agentic-ai-due-diligence
Full disclosure – this article was based on previous articles and adapted making use of Claude.ai Sonnet 3.7 with a final review and edit by Oliver.