🔍 Code Extractor

function clean_for_json_v6

Maturity: 41

Recursively traverses nested data structures (dicts, lists) and sanitizes floating-point values by replacing NaN and Inf with None, while also converting NumPy numeric types to native Python types.

File:
/tf/active/vicechatdev/vice_ai/smartstat_scripts/d1e252f5-950c-4ad7-b425-86b4b02c3c62/analysis_4.py
Lines:
307 - 323
Complexity:
simple

Purpose

This function prepares complex nested data structures for JSON serialization by handling problematic values that are not JSON-compliant. It converts NaN (Not a Number) and Inf (Infinity) values to None, and converts NumPy integer and floating-point types to native Python int and float types. This is essential when preparing data from scientific computing libraries (like pandas/numpy) for JSON export or API responses.

Source Code

def clean_for_json(obj):
    """Recursively clean NaN and Inf values from nested structures"""
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, float):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return obj
    elif isinstance(obj, np.integer):
        return int(obj)
    elif isinstance(obj, np.floating):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    return obj

Parameters

Name Type Default Kind
obj - - positional_or_keyword

Parameter Details

obj: The input object to be cleaned. Can be any Python type including nested structures like dictionaries, lists, floats, NumPy numeric types, or any other primitive type. The function recursively processes nested dictionaries and lists.

Return Value

Returns a cleaned version of the input object with the same structure. Float values that were NaN or Inf are replaced with None. NumPy integer types (np.integer) are converted to Python int. NumPy floating types (np.floating) are converted to Python float (or None if NaN/Inf). All other types are returned unchanged. The return type matches the input structure (dict returns dict, list returns list, etc.).

Dependencies

  • math
  • numpy

Required Imports

import math
import numpy as np

Usage Example

import math
import numpy as np

def clean_for_json(obj):
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, float):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return obj
    elif isinstance(obj, np.integer):
        return int(obj)
    elif isinstance(obj, np.floating):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    return obj

# Example usage
data = {
    'values': [1.5, float('nan'), float('inf'), np.float64(3.14)],
    'counts': [np.int64(10), np.int32(20)],
    'nested': {
        'bad_value': float('nan'),
        'good_value': 42
    }
}

cleaned = clean_for_json(data)
print(cleaned)
# Output: {'values': [1.5, None, None, 3.14], 'counts': [10, 20], 'nested': {'bad_value': None, 'good_value': 42}}

import json
json_string = json.dumps(cleaned)
print(json_string)

Best Practices

  • Use this function before calling json.dumps() on data that may contain NumPy types or NaN/Inf values
  • Be aware that NaN and Inf values are converted to None (null in JSON), which may affect downstream data analysis
  • This function creates a new cleaned copy of the data structure rather than modifying in-place
  • The function handles arbitrary nesting depth, but very deep structures may cause recursion limits
  • Consider the semantic meaning of replacing NaN/Inf with None in your specific use case - sometimes preserving these as strings might be more appropriate

Tags

json serialization data-cleaning numpy nan-handling infinity-handling type-conversion recursive data-preprocessing sanitization

Similar Components

AI-powered semantic similarity - components with related functionality:

  • function clean_for_json_v2 97.5% similar

    Recursively traverses nested data structures (dicts, lists) and sanitizes numeric values by converting NaN and Inf to None, and numpy types to native Python types for JSON serialization.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/e9b7c942-87b5-4a6f-865e-e7a0d62fb0a1/analysis_2.py

  • function clean_for_json_v5 97.5% similar

    Recursively traverses nested data structures (dictionaries, lists) and sanitizes numeric values by converting NaN and Inf to None, and normalizing NumPy numeric types to native Python types for JSON serialization.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/e4e8cb00-c17d-4282-aa80-5af67f32952f/analysis_1.py

  • function clean_for_json_v4 96.7% similar

    Recursively traverses nested data structures (dicts, lists, arrays) and converts NaN and Inf float values to None for safe JSON serialization, while also converting NumPy types to native Python types.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/7372154d-807e-4723-a769-4668761944b5/analysis_2.py

  • function clean_for_json_v1 96.6% similar

    Recursively sanitizes nested data structures (dictionaries, lists, tuples) by converting NaN and Inf values to None and normalizing NumPy types to native Python types for JSON serialization.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/e4e8cb00-c17d-4282-aa80-5af67f32952f/project_1/analysis.py

  • function clean_for_json_v11 93.4% similar

    Recursively sanitizes Python objects (dicts, lists, floats) to make them JSON-serializable by replacing NaN and infinity float values with None.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/f5da873e-41e6-4f34-b3e4-f7443d4d213b/analysis_4.py
← Back to Browse