🔍 Code Extractor

function flatten

Maturity: 46

Recursively flattens an arbitrarily nested sequence containing lists, tuples, and dictionaries into a single-level generator of non-sequence elements.

File:
/tf/active/vicechatdev/patches/util.py
Lines:
2284 - 2308
Complexity:
simple

Purpose

This function is designed to flatten deeply nested data structures by recursively traversing through lists, tuples, and dictionaries, yielding individual elements that are not themselves sequences. It's useful for processing hierarchical data structures, extracting all leaf values, or preparing nested data for linear processing. Inspired by pandas' internal flatten utility.

Source Code

def flatten(line):
    """
    Flatten an arbitrarily nested sequence.

    Inspired by: pd.core.common.flatten

    Parameters
    ----------
    line : sequence
        The sequence to flatten

    Notes
    -----
    This only flattens list, tuple, and dict sequences.

    Returns
    -------
    flattened : generator
    """

    for element in line:
        if any(isinstance(element, tp) for tp in (list, tuple, dict)):
            yield from flatten(element)
        else:
            yield element

Parameters

Name Type Default Kind
line - - positional_or_keyword

Parameter Details

line: A sequence (list, tuple, dict, or any iterable) that may contain nested sequences. The function will recursively traverse this structure to extract all non-sequence elements. Can be arbitrarily nested with any combination of lists, tuples, and dictionaries.

Return Value

Returns a generator object that yields individual elements from the flattened sequence. Each yielded element is a non-sequence item (not a list, tuple, or dict) found at any level of nesting in the input. The generator produces elements in depth-first traversal order. For dictionaries, only the keys are yielded during iteration.

Usage Example

# Basic usage with nested lists
nested_list = [1, [2, 3, [4, 5]], 6]
flattened = list(flatten(nested_list))
print(flattened)  # Output: [1, 2, 3, 4, 5, 6]

# Mixed nested structures
mixed = [1, (2, 3), [4, [5, (6, 7)]], 8]
result = list(flatten(mixed))
print(result)  # Output: [1, 2, 3, 4, 5, 6, 7, 8]

# With dictionaries (yields keys)
with_dict = [1, {'a': 1, 'b': 2}, [3, 4]]
result = list(flatten(with_dict))
print(result)  # Output: [1, 'a', 'b', 3, 4]

# Using as generator for memory efficiency
for item in flatten([[1, 2], [3, [4, 5]]]):
    print(item)  # Prints: 1, 2, 3, 4, 5 (one per line)

Best Practices

  • The function returns a generator, so use list() to convert to a list if you need all values at once, or iterate directly for memory efficiency
  • When flattening dictionaries, only the keys are yielded, not the values. If you need values, extract them before flattening
  • The function only flattens list, tuple, and dict types. Other iterable types (like strings, sets) are treated as leaf elements and not flattened
  • Be cautious with deeply nested structures as the recursive nature could lead to stack overflow with extreme nesting depths
  • The function uses 'yield from' for efficient recursive generation without building intermediate lists
  • Strings are not flattened, so 'abc' will be yielded as a single element, not as individual characters

Tags

flatten nested-sequences recursion generator data-processing list-manipulation tuple-processing dict-processing sequence-flattening depth-first-traversal utility-function

Similar Components

AI-powered semantic similarity - components with related functionality:

  • function group_select 40.5% similar

    Recursively groups a list of key tuples into a nested dictionary structure to optimize indexing operations by avoiding duplicate key lookups.

    From: /tf/active/vicechatdev/patches/util.py

  • function lzip 39.7% similar

    A convenience wrapper around Python's built-in zip function that returns a list instead of an iterator.

    From: /tf/active/vicechatdev/patches/util.py

  • function clean_for_json_v8 39.4% similar

    Recursively traverses and converts a nested data structure (dicts, lists, numpy types, pandas NaN) into JSON-serializable Python primitives.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/d1e252f5-950c-4ad7-b425-86b4b02c3c62/analysis_5.py

  • function unique_zip 39.0% similar

    Returns a unique list of tuples created by zipping multiple iterables together, removing any duplicate tuples while preserving order.

    From: /tf/active/vicechatdev/patches/util.py

  • function clean_for_json_v1 37.0% similar

    Recursively sanitizes nested data structures (dictionaries, lists, tuples) by converting NaN and Inf values to None and normalizing NumPy types to native Python types for JSON serialization.

    From: /tf/active/vicechatdev/vice_ai/smartstat_scripts/e4e8cb00-c17d-4282-aa80-5af67f32952f/project_1/analysis.py
← Back to Browse