🔍 Code Extractor

function clean_for_json_v3

Maturity: 42

Recursively converts Python objects (including NumPy and Pandas types) into JSON-serializable formats by handling special numeric types, NaN/Inf values, and nested data structures.

File:
/tf/active/vicechatdev/vice_ai/smartstat_scripts/f0a78968-1d2b-4fbe-a0c6-a372da2ce2a4/project_1/analysis.py
Lines:
613 - 630
Complexity:
moderate

Purpose

This function prepares complex Python objects for JSON serialization by converting NumPy integer and float types to native Python types, handling NaN and Inf values by converting them to None, converting NumPy arrays to lists, and recursively processing nested dictionaries and lists. It's particularly useful when working with data science libraries (NumPy, Pandas) that produce types incompatible with standard JSON serialization.

Source Code

def clean_for_json(obj):
    """Recursively clean object for JSON serialization"""
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, (np.integer, np.int64, np.int32)):
        return int(obj)
    elif isinstance(obj, (np.floating, np.float64, np.float32)):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    elif isinstance(obj, np.ndarray):
        return clean_for_json(obj.tolist())
    elif pd.isna(obj):
        return None
    else:
        return obj

Parameters

Name Type Default Kind
obj - - positional_or_keyword

Parameter Details

obj: Any Python object to be cleaned for JSON serialization. Can be a primitive type (int, float, str), NumPy type (np.integer, np.floating, np.ndarray), Pandas NA value, or nested structure (dict, list). The function handles all these types recursively.

Return Value

Returns a JSON-serializable version of the input object. NumPy integers become Python ints, NumPy floats become Python floats (or None if NaN/Inf), NumPy arrays become lists, Pandas NA values become None, dictionaries and lists are recursively cleaned, and other types are returned unchanged. The return type matches the structure of the input but with all types converted to JSON-compatible equivalents.

Dependencies

  • numpy
  • pandas
  • math

Required Imports

import numpy as np
import pandas as pd
import math

Usage Example

import numpy as np
import pandas as pd
import math
import json

def clean_for_json(obj):
    if isinstance(obj, dict):
        return {k: clean_for_json(v) for k, v in obj.items()}
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, (np.integer, np.int64, np.int32)):
        return int(obj)
    elif isinstance(obj, (np.floating, np.float64, np.float32)):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    elif isinstance(obj, np.ndarray):
        return clean_for_json(obj.tolist())
    elif pd.isna(obj):
        return None
    else:
        return obj

# Example usage
data = {
    'numpy_int': np.int64(42),
    'numpy_float': np.float64(3.14),
    'nan_value': np.nan,
    'inf_value': np.inf,
    'array': np.array([1, 2, 3]),
    'pandas_na': pd.NA,
    'nested': {
        'list': [np.int32(1), np.float32(2.5)]
    }
}

cleaned = clean_for_json(data)
json_string = json.dumps(cleaned)
print(json_string)
# Output: {"numpy_int": 42, "numpy_float": 3.14, "nan_value": null, "inf_value": null, "array": [1, 2, 3], "pandas_na": null, "nested": {"list": [1, 2.5]}}

Best Practices

  • Always use this function before calling json.dumps() on data structures containing NumPy or Pandas types
  • Be aware that NaN and Inf values are converted to None (null in JSON), which may affect downstream processing
  • The function preserves the structure of nested dictionaries and lists while converting their contents
  • For large NumPy arrays, consider the memory implications of converting to lists before calling this function
  • This function does not handle custom objects or classes - they will be returned unchanged and may still cause JSON serialization errors
  • The function uses isinstance checks in a specific order; the order matters for proper type detection

Tags

json serialization data-conversion numpy pandas type-conversion recursive data-cleaning nan-handling infinity-handling

Similar Components

AI-powered semantic similarity - components with related functionality:

  • function clean_for_json_v14 94.6% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting NumPy types to native Python types and handling NaN/Inf values.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/42b81361-ba7e-4d79-9598-3090af68384b/analysis_2.py

  • function clean_for_json_v12 94.5% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting non-serializable types (NumPy types, pandas objects, tuples, NaN/Inf values) into JSON-compatible formats.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/290a39ea-3ae0-4301-8e2f-9d5c3bf80e6e/project_3/analysis.py

  • function clean_for_json_v10 93.8% similar

    Recursively converts Python objects containing NumPy and Pandas data types into JSON-serializable native Python types.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/c385e1f5-fbf6-4832-8fd4-78ef8b72fc53/project_1/analysis.py

  • function clean_for_json_v13 93.5% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting NumPy types to native Python types and handling NaN/Inf values.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/d48d7789-9627-4e96-9f48-f90b687cd07d/analysis_1.py

  • function clean_for_json_v15 93.1% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting NumPy types to native Python types and handling NaN/Inf float values.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/290a39ea-3ae0-4301-8e2f-9d5c3bf80e6e/analysis_3.py
← Back to Browse