🔍 Code Extractor

Recursively traverses and converts a nested data structure (dicts, lists, numpy types, pandas NaN) into JSON-serializable Python primitives.

Purpose

This function prepares complex data structures containing numpy arrays, pandas objects, and nested collections for JSON serialization by converting numpy types to native Python types, handling NaN/Inf values, and recursively processing nested structures. It's essential when working with data science libraries that produce non-JSON-serializable types that need to be exported or transmitted as JSON.

Source Code

def clean_for_json(obj):
    """Recursively clean data structure for JSON serialization"""
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, (np.integer, np.int64, np.int32)):
        return int(obj)
    elif isinstance(obj, (np.floating, np.float64, np.float32)):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    elif isinstance(obj, np.ndarray):
        return clean_for_json(obj.tolist())
    elif pd.isna(obj):
        return None
    return obj

Parameters

Parameter Details

obj: Any Python object to be cleaned for JSON serialization. Can be a dict, list, numpy array, numpy scalar (integer/float), pandas NA value, or any JSON-serializable primitive. Nested structures are supported and will be recursively processed.

Return Value

Returns a JSON-serializable version of the input object. Dictionaries remain dictionaries with cleaned values, lists remain lists with cleaned elements, numpy integers/floats are converted to Python int/float, numpy arrays are converted to lists, NaN/Inf values become None, pandas NA values become None, and other primitives are returned unchanged. The structure of nested objects is preserved.

Dependencies

• numpy
• pandas
• math

Required Imports

import numpy as np
import pandas as pd
import math

Usage Example

import numpy as np
import pandas as pd
import math
import json

def clean_for_json(obj):
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, (np.integer, np.int64, np.int32)):
        return int(obj)
    elif isinstance(obj, (np.floating, np.float64, np.float32)):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    elif isinstance(obj, np.ndarray):
        return clean_for_json(obj.tolist())
    elif pd.isna(obj):
        return None
    return obj

# Example usage
data = {
    'array': np.array([1, 2, 3]),
    'int': np.int64(42),
    'float': np.float32(3.14),
    'nan': np.nan,
    'inf': np.inf,
    'pd_na': pd.NA,
    'nested': {
        'list': [np.int32(1), np.float64(2.5)]
    }
}

cleaned = clean_for_json(data)
print(json.dumps(cleaned, indent=2))
# Output: {"array": [1, 2, 3], "int": 42, "float": 3.14, "nan": null, "inf": null, "pd_na": null, "nested": {"list": [1, 2.5]}}

Best Practices

• Use this function before calling json.dumps() on data structures that may contain numpy or pandas types
• Be aware that NaN and Inf values are converted to None (null in JSON), which may affect downstream processing
• The function preserves the structure of nested dictionaries and lists, making it safe for complex data hierarchies
• For large numpy arrays, consider the memory implications of converting to lists before calling this function
• This function does not handle custom objects or classes - only built-in types, numpy types, and pandas NA values

Tags

Similar Components

AI-powered semantic similarity - components with related functionality:

• function clean_for_json_v7 94.6% similar

  Recursively traverses and sanitizes data structures (dicts, lists, numpy types) to ensure JSON serialization compatibility by converting numpy types to native Python types and handling NaN/Inf values.

  From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/d1e252f5-950c-4ad7-b425-86b4b02c3c62/analysis_1.py

• function clean_for_json 92.8% similar

  Recursively traverses and sanitizes Python data structures (dicts, lists, tuples, numpy arrays) to ensure all values are JSON-serializable by converting numpy types, handling NaN/Inf values, and normalizing data types.

  From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/f0b81d95-24d9-418a-8d9f-1b241684e64c/project_1/analysis.py

• function clean_for_json_v12 91.5% similar

  Recursively sanitizes Python objects to make them JSON-serializable by converting non-serializable types (NumPy types, pandas objects, tuples, NaN/Inf values) into JSON-compatible formats.

  From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/290a39ea-3ae0-4301-8e2f-9d5c3bf80e6e/project_3/analysis.py

• function clean_for_json_v10 90.2% similar

  Recursively converts Python objects containing NumPy and Pandas data types into JSON-serializable native Python types.

  From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/c385e1f5-fbf6-4832-8fd4-78ef8b72fc53/project_1/analysis.py

• function clean_for_json_v3 89.7% similar

  Recursively converts Python objects (including NumPy and Pandas types) into JSON-serializable formats by handling special numeric types, NaN/Inf values, and nested data structures.

  From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/f0a78968-1d2b-4fbe-a0c6-a372da2ce2a4/project_1/analysis.py