🔍 Code Extractor

Tests LLM-based contract analysis prompts using simulated NDA content containing a term clause to verify extraction of contract dates and metadata.

Purpose

This test function validates a two-step LLM pipeline for contract analysis. It uses simulated NDA text to test whether the system can correctly identify and extract contract expiry dates, start dates, parties, and other metadata from documents containing term clauses. The function specifically tests the ability to calculate end dates based on term duration (e.g., '1 year from effective date'). It's designed for integration testing of LLM prompt engineering and contract information extraction workflows.

Source Code

def test_with_simulated_content():
    """Test the prompts with simulated content that includes the term clause."""
    logger = setup_test_logging()
    logger.info("Testing with simulated content that includes the term clause")
    
    # Load configuration
    config = Config()
    llm_client = LLMClient(config.get_section('llm'))
    
    # Simulated document text that includes the term clause you mentioned
    simulated_text = """
    Vicebio Limited, a company registered in England and Wales with company number 11570257 whose registered office is at 2nd Floor 168 Shoreditch High Street, London, United Kingdom, E1 6RA, England ("Vicebio") intends to disclose to Veritus Research Pty Ltd (ABN 486 669 419 39) with a place of business at Building 21, 885 Mountain Hwy Bayswater, 3153 ("Recipient") the top line results of its first interim analysis of ongoing study VXB241-001, and information related to those results (the "Confidential Information") for the purpose of Recipient fulfilling its current obligations to, or otherwise evaluating its future activities with, Vicebio (the "Purpose").

    With effect from the date set out above ("Effective Date") the Parties have agreed to comply with this agreement (the "Agreement") in connection with the disclosure and use of the Confidential Information.

    Term: This Agreement shall commence and be effective from the Effective Date and shall continue for a period of one (1) year unless it is terminated by a Party giving thirty (30) days' prior written notice to the other. The obligations of confidentiality set forth herein shall survive for a period of five (5) years from the date of this Agreement's expiry or termination, or otherwise until the relevant Confidential Information becomes available to the public other than through breach of this Agreement by the Recipient.

    SIGNED on behalf of VICEBIO LIMITED
    Date: 02-Jun-2025

    SIGNED on behalf of Veritus Research Pty Ltd  
    Date: 02-Jun-2025
    """
    
    filename = "Simulated CDA with Term Clause"
    
    logger.info("Testing Step 1 with simulated content...")
    step1_result = llm_client._find_expiry_dates(simulated_text, filename)
    
    logger.info("STEP 1 RESULT:")
    logger.info("-" * 60)
    expiry_analysis = step1_result.get('expiry_analysis', 'No analysis available')
    logger.info(expiry_analysis)
    logger.info("-" * 60)
    
    logger.info("Testing Step 2 with simulated content...")
    step2_result = llm_client._extract_complete_contract_info(step1_result, simulated_text, filename)
    
    logger.info("STEP 2 RESULT:")
    logger.info("-" * 60)
    if step2_result.get('error'):
        logger.error(f"Step 2 failed: {step2_result.get('error')}")
    else:
        logger.info(f"Contract Type: {step2_result.get('contract_type', 'Unknown')}")
        logger.info(f"Third Parties: {step2_result.get('third_parties', [])}")
        logger.info(f"Start Date: {step2_result.get('start_date', 'Not found')}")
        logger.info(f"End Date: {step2_result.get('end_date', 'Not found')}")
        logger.info(f"Is In Effect: {step2_result.get('is_in_effect', 'Unknown')}")
        logger.info(f"Confidence: {step2_result.get('confidence', 0.0)}")
        logger.info(f"Analysis Notes: {step2_result.get('analysis_notes', 'None')}")
    logger.info("-" * 60)
    
    # Check if end date was successfully extracted
    end_date = step2_result.get('end_date')
    if end_date and end_date not in ['null', None, '']:
        logger.info("✓ SUCCESS: End date extracted from simulated content!")
        logger.info(f"Expected: 2026-06-02 (1 year from 2025-06-02)")
        logger.info(f"Actual: {end_date}")
        return True
    else:
        logger.warning("✗ FAILED: End date not extracted from simulated content")
        logger.warning("This indicates the prompts need further improvement")
        return False

Return Value

Returns a boolean value: True if the end date was successfully extracted from the simulated content (indicating the prompts work correctly), False if the end date extraction failed (indicating prompts need improvement). The function also logs detailed analysis results from both processing steps.

Dependencies

• logging
• pathlib
• config.config
• utils.document_processor
• utils.llm_client

Required Imports

from pathlib import Path
from config.config import Config
from utils.document_processor import DocumentProcessor
from utils.llm_client import LLMClient
import logging

Usage Example

# Assuming this is in a test file with setup_test_logging() defined
# and proper config/utils modules available

def setup_test_logging():
    logging.basicConfig(level=logging.INFO)
    return logging.getLogger(__name__)

# Run the test
result = test_with_simulated_content()

if result:
    print("Test passed: End date successfully extracted")
else:
    print("Test failed: End date extraction needs improvement")

# Typical output includes:
# - Step 1 expiry analysis from LLM
# - Step 2 complete contract information
# - Validation of extracted end date against expected value

Best Practices

• This function is designed for testing/validation purposes and should not be used in production code
• Ensure the Config object is properly initialized with valid LLM credentials before running
• The simulated text includes realistic contract language to test edge cases in date extraction
• Review logged output to understand why extraction succeeded or failed
• The function expects specific method signatures on LLMClient (_find_expiry_dates and _extract_complete_contract_info)
• The test validates both the extraction process and the accuracy of calculated dates (1 year from effective date)
• Use this as a template for creating additional test cases with different contract types and term clauses

Tags

Similar Components

AI-powered semantic similarity - components with related functionality:

• function test_simulated_document 86.1% similar

  Integration test function that validates end date extraction from a simulated contract document containing an explicit term clause, using a two-step LLM-based analysis process.

  From: /tf/active/vicechatdev/contract_validity_analyzer/test_simulated_document.py

• function test_end_date_extraction 73.9% similar

  Tests end date extraction functionality for contract documents that previously had missing end dates by downloading documents from FileCloud, extracting text, analyzing with LLM, and comparing results.

  From: /tf/active/vicechatdev/contract_validity_analyzer/test_missing_end_dates.py

• function test_llm_client 72.9% similar

  Tests the LLM client functionality by analyzing a sample contract text and verifying the extraction of key contract metadata such as third parties, dates, and status.

  From: /tf/active/vicechatdev/contract_validity_analyzer/test_implementation.py

• function test_local_document 72.8% similar

  Integration test function that validates end date extraction from a local PDF document using document processing and LLM-based analysis.

  From: /tf/active/vicechatdev/contract_validity_analyzer/test_local_document.py

• function test_llm_extraction 72.2% similar

  A test function that validates LLM-based contract data extraction by processing a sample contract and verifying the extracted fields against expected values.

  From: /tf/active/vicechatdev/contract_validity_analyzer/test_extractor.py