🔍 Code Extractor

function clean_for_json_v7

Maturity: 41

Recursively traverses and sanitizes data structures (dicts, lists, numpy types) to ensure JSON serialization compatibility by converting numpy types to native Python types and handling NaN/Inf values.

File:
/tf/active/vicechatdev/vice_ai/smartstat_scripts/d1e252f5-950c-4ad7-b425-86b4b02c3c62/analysis_1.py
Lines:
461 - 481
Complexity:
moderate

Purpose

This function prepares complex nested data structures containing numpy arrays, pandas objects, and various numeric types for JSON serialization. It handles edge cases like NaN, Inf values (converting them to None), converts numpy types to native Python types, and recursively processes nested dictionaries and lists. This is essential when working with data science libraries that produce non-JSON-serializable objects that need to be exported or transmitted as JSON.

Source Code

def clean_for_json(obj):
    """Recursively clean data structure for JSON serialization"""
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, (np.integer, np.int64, np.int32)):
        return int(obj)
    elif isinstance(obj, (np.floating, np.float64, np.float32)):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    elif isinstance(obj, np.ndarray):
        return clean_for_json(obj.tolist())
    elif isinstance(obj, float):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return obj
    elif pd.isna(obj):
        return None
    return obj

Parameters

Name Type Default Kind
obj - - positional_or_keyword

Parameter Details

obj: Any Python object to be cleaned for JSON serialization. Can be a primitive type (int, float, str), numpy type (np.integer, np.floating, np.ndarray), pandas NA value, or nested structure (dict, list). The function handles all these types recursively.

Return Value

Returns a JSON-serializable version of the input object. Numpy integers/floats are converted to Python int/float, numpy arrays are converted to lists, NaN/Inf values become None, pandas NA values become None, and nested structures are recursively cleaned. Primitive types that are already JSON-serializable are returned unchanged.

Dependencies

  • numpy
  • pandas
  • math

Required Imports

import numpy as np
import pandas as pd
import math

Usage Example

import numpy as np
import pandas as pd
import math
import json

def clean_for_json(obj):
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, (np.integer, np.int64, np.int32)):
        return int(obj)
    elif isinstance(obj, (np.floating, np.float64, np.float32)):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    elif isinstance(obj, np.ndarray):
        return clean_for_json(obj.tolist())
    elif isinstance(obj, float):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return obj
    elif pd.isna(obj):
        return None
    return obj

# Example usage
data = {
    'array': np.array([1, 2, 3]),
    'int': np.int64(42),
    'float': np.float64(3.14),
    'nan': float('nan'),
    'inf': float('inf'),
    'nested': {
        'list': [np.int32(1), pd.NA, np.float32(2.5)]
    }
}

cleaned = clean_for_json(data)
json_string = json.dumps(cleaned)
print(json_string)

Best Practices

  • Always use this function before calling json.dumps() on data structures that may contain numpy or pandas objects
  • Be aware that NaN and Inf values are converted to None (null in JSON), which may affect downstream processing
  • The function modifies the structure by converting numpy arrays to lists, which may impact memory usage for large arrays
  • For very deeply nested structures, be mindful of Python's recursion limit (default ~1000 levels)
  • This function does not handle custom objects or datetime types - extend it if needed for those cases

Tags

json serialization data-cleaning numpy pandas type-conversion recursive data-processing nan-handling infinity-handling

Similar Components

AI-powered semantic similarity - components with related functionality:

  • function clean_for_json 95.9% similar

    Recursively traverses and sanitizes Python data structures (dicts, lists, tuples, numpy arrays) to ensure all values are JSON-serializable by converting numpy types, handling NaN/Inf values, and normalizing data types.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/f0b81d95-24d9-418a-8d9f-1b241684e64c/project_1/analysis.py

  • function clean_for_json_v8 94.6% similar

    Recursively traverses and converts a nested data structure (dicts, lists, numpy types, pandas NaN) into JSON-serializable Python primitives.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/d1e252f5-950c-4ad7-b425-86b4b02c3c62/analysis_5.py

  • function clean_for_json_v12 94.0% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting non-serializable types (NumPy types, pandas objects, tuples, NaN/Inf values) into JSON-compatible formats.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/290a39ea-3ae0-4301-8e2f-9d5c3bf80e6e/project_3/analysis.py

  • function clean_for_json_v1 93.1% similar

    Recursively sanitizes nested data structures (dictionaries, lists, tuples) by converting NaN and Inf values to None and normalizing NumPy types to native Python types for JSON serialization.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/e4e8cb00-c17d-4282-aa80-5af67f32952f/project_1/analysis.py

  • function clean_for_json_v13 92.6% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting NumPy types to native Python types and handling NaN/Inf values.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/d48d7789-9627-4e96-9f48-f90b687cd07d/analysis_1.py
← Back to Browse