🔍 Code Extractor

function clean_for_json_v5

Maturity: 41

Recursively traverses nested data structures (dictionaries, lists) and sanitizes numeric values by converting NaN and Inf to None, and normalizing NumPy numeric types to native Python types for JSON serialization.

File:
/tf/active/vicechatdev/vice_ai/smartstat_scripts/e4e8cb00-c17d-4282-aa80-5af67f32952f/analysis_1.py
Lines:
367 - 383
Complexity:
simple

Purpose

This function prepares complex nested data structures for JSON serialization by handling problematic numeric values that are not valid in JSON format. It converts NaN (Not a Number) and Inf (Infinity) values to None, converts NumPy integer types to Python int, and NumPy floating types to Python float. This is essential when preparing data from scientific computing libraries (pandas, numpy) for JSON export or API responses.

Source Code

def clean_for_json(obj):
    """Recursively clean NaN and Inf values from nested dictionaries and lists"""
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, float):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return obj
    elif isinstance(obj, np.integer):
        return int(obj)
    elif isinstance(obj, np.floating):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    return obj

Parameters

Name Type Default Kind
obj - - positional_or_keyword

Parameter Details

obj: The input object to be cleaned. Can be any Python type including nested dictionaries, lists, floats, NumPy numeric types, or primitive types. The function recursively processes nested structures to clean all numeric values within.

Return Value

Returns a cleaned version of the input object with the same structure but with sanitized values. NaN and Inf values (both Python float and NumPy types) are replaced with None. NumPy integer types are converted to Python int, and NumPy floating types are converted to Python float. All other types are returned unchanged. The return type matches the input type structure (dict returns dict, list returns list, etc.).

Dependencies

  • math
  • numpy

Required Imports

import math
import numpy as np

Usage Example

import math
import numpy as np

def clean_for_json(obj):
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, float):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return obj
    elif isinstance(obj, np.integer):
        return int(obj)
    elif isinstance(obj, np.floating):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    return obj

# Example usage
data = {
    'values': [1.5, float('nan'), float('inf'), np.float64(3.14)],
    'counts': [np.int64(10), np.int32(20)],
    'nested': {
        'bad_value': float('nan'),
        'good_value': 42
    }
}

cleaned = clean_for_json(data)
print(cleaned)
# Output: {'values': [1.5, None, None, 3.14], 'counts': [10, 20], 'nested': {'bad_value': None, 'good_value': 42}}

import json
json_string = json.dumps(cleaned)
print(json_string)

Best Practices

  • Always call this function before using json.dumps() on data that may contain NumPy types or NaN/Inf values
  • Be aware that NaN and Inf values are converted to None, which may affect downstream data analysis if the distinction between missing data and invalid numeric values is important
  • This function creates a new object structure rather than modifying in-place, so it's safe to use on shared data structures
  • For large nested structures, consider the memory overhead of creating a complete copy of the data
  • The function does not handle pandas DataFrame or Series objects directly - convert them to dictionaries or lists first using .to_dict() or .to_list()

Tags

data-cleaning json-serialization numpy data-preprocessing nan-handling infinity-handling type-conversion recursive data-sanitization scientific-computing

Similar Components

AI-powered semantic similarity - components with related functionality:

  • function clean_for_json_v2 98.5% similar

    Recursively traverses nested data structures (dicts, lists) and sanitizes numeric values by converting NaN and Inf to None, and numpy types to native Python types for JSON serialization.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/e9b7c942-87b5-4a6f-865e-e7a0d62fb0a1/analysis_2.py

  • function clean_for_json_v6 97.5% similar

    Recursively traverses nested data structures (dicts, lists) and sanitizes floating-point values by replacing NaN and Inf with None, while also converting NumPy numeric types to native Python types.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/d1e252f5-950c-4ad7-b425-86b4b02c3c62/analysis_4.py

  • function clean_for_json_v1 96.8% similar

    Recursively sanitizes nested data structures (dictionaries, lists, tuples) by converting NaN and Inf values to None and normalizing NumPy types to native Python types for JSON serialization.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/e4e8cb00-c17d-4282-aa80-5af67f32952f/project_1/analysis.py

  • function clean_for_json_v4 96.1% similar

    Recursively traverses nested data structures (dicts, lists, arrays) and converts NaN and Inf float values to None for safe JSON serialization, while also converting NumPy types to native Python types.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/7372154d-807e-4723-a769-4668761944b5/analysis_2.py

  • function clean_for_json_v13 92.7% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting NumPy types to native Python types and handling NaN/Inf values.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/d48d7789-9627-4e96-9f48-f90b687cd07d/analysis_1.py
← Back to Browse