🔍 Code Extractor

function clean_for_json_v10

Maturity: 32

Recursively converts Python objects containing NumPy and Pandas data types into JSON-serializable native Python types.

File:
/tf/active/vicechatdev/vice_ai/smartstat_scripts/c385e1f5-fbf6-4832-8fd4-78ef8b72fc53/project_1/analysis.py
Lines:
630 - 651
Complexity:
moderate

Purpose

This function sanitizes complex Python objects (dictionaries, lists, NumPy arrays, Pandas data types) to make them JSON-serializable. It handles edge cases like NaN/Inf values, tuple keys in dictionaries, and various NumPy integer/float types. This is essential when preparing data for JSON export, API responses, or storage in JSON-based databases.

Source Code

def clean_for_json(obj):
    if isinstance(obj, dict):
        cleaned = {}
        for k, v in obj.items():
            # Convert tuple keys to strings
            if isinstance(k, tuple):
                k = str(k)
            cleaned[k] = clean_for_json(v)
        return cleaned
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, (np.integer, np.int64, np.int32)):
        return int(obj)
    elif isinstance(obj, (np.floating, np.float64, np.float32)):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    elif isinstance(obj, np.ndarray):
        return obj.tolist()
    elif pd.isna(obj):
        return None
    return obj

Parameters

Name Type Default Kind
obj - - positional_or_keyword

Parameter Details

obj: Any Python object that needs to be cleaned for JSON serialization. Can be a dict, list, NumPy array, NumPy scalar (integer/float), Pandas NA value, or any primitive type. The function recursively processes nested structures.

Return Value

Returns a JSON-serializable version of the input object. Dictionaries have tuple keys converted to strings and values recursively cleaned. Lists have all items recursively cleaned. NumPy integers/floats are converted to Python int/float. NumPy arrays are converted to lists. NaN and Inf values become None. Pandas NA values become None. Primitive types are returned unchanged.

Dependencies

  • numpy
  • pandas
  • math

Required Imports

import numpy as np
import pandas as pd
import math

Usage Example

import numpy as np
import pandas as pd
import math
import json

def clean_for_json(obj):
    if isinstance(obj, dict):
        cleaned = {}
        for k, v in obj.items():
            if isinstance(k, tuple):
                k = str(k)
            cleaned[k] = clean_for_json(v)
        return cleaned
    elif isinstance(obj, list):
        return [clean_for_json(item) for item in obj]
    elif isinstance(obj, (np.integer, np.int64, np.int32)):
        return int(obj)
    elif isinstance(obj, (np.floating, np.float64, np.float32)):
        if math.isnan(obj) or math.isinf(obj):
            return None
        return float(obj)
    elif isinstance(obj, np.ndarray):
        return obj.tolist()
    elif pd.isna(obj):
        return None
    return obj

# Example usage
data = {
    'array': np.array([1, 2, 3]),
    'int': np.int64(42),
    'float': np.float32(3.14),
    'nan': np.nan,
    'inf': np.inf,
    'pd_na': pd.NA,
    (1, 2): 'tuple_key',
    'nested': {'inner': np.array([4, 5])}
}

cleaned = clean_for_json(data)
print(json.dumps(cleaned, indent=2))

Best Practices

  • Always use this function before calling json.dumps() on data containing NumPy or Pandas types to avoid serialization errors
  • Be aware that NaN and Inf values are converted to None (null in JSON), which may affect downstream data analysis
  • Tuple keys in dictionaries are converted to string representations, which cannot be automatically converted back to tuples when deserializing
  • The function modifies the structure by converting NumPy arrays to lists, which may impact memory usage for large arrays
  • For very large nested structures, consider the recursion depth limit (Python default is ~1000 levels)
  • This function does not handle custom objects or other non-standard types - they will be returned unchanged and may still cause JSON serialization errors

Tags

json serialization data-conversion numpy pandas type-conversion recursive data-cleaning nan-handling json-serializable

Similar Components

AI-powered semantic similarity - components with related functionality:

  • function clean_for_json_v3 93.8% similar

    Recursively converts Python objects (including NumPy and Pandas types) into JSON-serializable formats by handling special numeric types, NaN/Inf values, and nested data structures.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/f0a78968-1d2b-4fbe-a0c6-a372da2ce2a4/project_1/analysis.py

  • function clean_for_json_v12 93.7% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting non-serializable types (NumPy types, pandas objects, tuples, NaN/Inf values) into JSON-compatible formats.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/290a39ea-3ae0-4301-8e2f-9d5c3bf80e6e/project_3/analysis.py

  • function clean_for_json_v14 91.6% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting NumPy types to native Python types and handling NaN/Inf values.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/42b81361-ba7e-4d79-9598-3090af68384b/analysis_2.py

  • function clean_for_json_v7 91.4% similar

    Recursively traverses and sanitizes data structures (dicts, lists, numpy types) to ensure JSON serialization compatibility by converting numpy types to native Python types and handling NaN/Inf values.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/d1e252f5-950c-4ad7-b425-86b4b02c3c62/analysis_1.py

  • function clean_for_json_v13 91.0% similar

    Recursively sanitizes Python objects to make them JSON-serializable by converting NumPy types to native Python types and handling NaN/Inf values.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/d48d7789-9627-4e96-9f48-f90b687cd07d/analysis_1.py
← Back to Browse