🔍 Code Extractor

Converts various datetime types (pandas, numpy, cftime, Python datetime) to an integer timestamp with a specified time unit.

Purpose

This function provides a unified interface for converting different datetime representations into integer timestamps. It handles multiple datetime types including pandas Timestamp/Period, numpy datetime64, Python datetime/date objects, and cftime types. The function is particularly useful for normalizing datetime data from different sources into a consistent integer format for numerical operations, storage, or comparison. It supports various time units (nanoseconds, microseconds, milliseconds, seconds, etc.) and handles timezone-aware datetimes correctly.

Source Code

def dt_to_int(value, time_unit='us'):
    """
    Converts a datetime type to an integer with the supplied time unit.
    """
    if pd:
        if isinstance(value, pd.Period):
            value = value.to_timestamp()
        if isinstance(value, pd.Timestamp):
            try:
                value = value.to_datetime64()
            except Exception:
                value = np.datetime64(value.to_pydatetime())
    elif isinstance(value, cftime_types):
        return cftime_to_timestamp(value, time_unit)

    # date class is a parent for datetime class
    if isinstance(value, dt.date) and not isinstance(value, dt.datetime):
        value = dt.datetime(*value.timetuple()[:6])

    # Handle datetime64 separately
    if isinstance(value, np.datetime64):
        try:
            value = np.datetime64(value, 'ns')
            tscale = (np.timedelta64(1, time_unit)/np.timedelta64(1, 'ns'))
            return value.tolist()/tscale
        except Exception:
            # If it can't handle ns precision fall back to datetime
            value = value.tolist()

    if time_unit == 'ns':
        tscale = 1e9
    else:
        tscale = 1./np.timedelta64(1, time_unit).tolist().total_seconds()

    try:
        # Handle python3
        if value.tzinfo is None:
            _epoch = dt.datetime(1970, 1, 1)
        else:
            _epoch = dt.datetime(1970, 1, 1, tzinfo=dt.timezone.utc)
        return int((value - _epoch).total_seconds() * tscale)
    except Exception:
        # Handle python2
        return (time.mktime(value.timetuple()) + value.microsecond / 1e6) * tscale

Parameters

Parameter Details

value: A datetime-like object to convert. Can be: pandas.Timestamp, pandas.Period, numpy.datetime64, Python datetime.datetime, Python datetime.date, or cftime datetime types. The function will attempt to convert any of these types to an integer timestamp.

time_unit: String specifying the time unit for the output integer. Default is 'us' (microseconds). Common values include 'ns' (nanoseconds), 'us' (microseconds), 'ms' (milliseconds), 's' (seconds), 'm' (minutes), 'h' (hours), 'D' (days). Must be a valid numpy timedelta unit string.

Return Value

Returns an integer or float representing the timestamp in the specified time unit, measured from the Unix epoch (January 1, 1970, 00:00:00 UTC). For timezone-naive datetimes, the epoch is assumed to be in the local timezone. For timezone-aware datetimes, the epoch is UTC. The return type is typically int but may be float for certain time units or edge cases.

Dependencies

• numpy
• pandas
• cftime
• datetime
• time

Required Imports

import numpy as np
import pandas as pd
import datetime as dt
import time

Conditional/Optional Imports

These imports are only needed under specific conditions:

import cftime

Usage Example

import numpy as np
import pandas as pd
import datetime as dt

# Convert pandas Timestamp to microseconds (default)
timestamp = pd.Timestamp('2023-01-15 12:30:45')
result = dt_to_int(timestamp)
print(f"Microseconds: {result}")

# Convert numpy datetime64 to nanoseconds
dt64 = np.datetime64('2023-01-15T12:30:45')
result_ns = dt_to_int(dt64, time_unit='ns')
print(f"Nanoseconds: {result_ns}")

# Convert Python datetime to seconds
py_dt = dt.datetime(2023, 1, 15, 12, 30, 45)
result_s = dt_to_int(py_dt, time_unit='s')
print(f"Seconds: {result_s}")

# Convert date to milliseconds
py_date = dt.date(2023, 1, 15)
result_ms = dt_to_int(py_date, time_unit='ms')
print(f"Milliseconds: {result_ms}")

Best Practices

• Ensure the time_unit parameter matches your intended precision and use case (e.g., 'ns' for high-precision timestamps, 's' for general purposes)
• Be aware of timezone handling: timezone-naive datetimes are treated as local time, while timezone-aware datetimes use UTC epoch
• The function handles multiple datetime types, but performance may vary; consider normalizing input types if processing large datasets
• For numpy.datetime64 values, the function first attempts nanosecond precision conversion; if that fails, it falls back to Python datetime conversion
• When working with pandas Period objects, they are automatically converted to timestamps at the period's start
• The function requires cftime_types and cftime_to_timestamp to be defined in the same module for cftime support
• Error handling is built-in for edge cases, but extremely old or future dates may cause overflow issues depending on the time unit

Tags

Similar Components

AI-powered semantic similarity - components with related functionality:

• function dt64_to_dt 66.4% similar

  Converts a NumPy datetime64 object to a Python datetime.datetime object by calculating seconds since Unix epoch.

  From: /tf/active/vicechatdev/patches/util.py

• function parse_datetime_v1 65.8% similar

  Converts various date representations (string, integer, pandas Timestamp) into a numpy datetime64 object using pandas datetime parsing capabilities.

  From: /tf/active/vicechatdev/patches/util.py

• function cftime_to_timestamp 61.0% similar

  Converts cftime datetime objects (or arrays) to timestamps since Unix epoch (1970-01-01 00:00:00) in specified time units, defaulting to microseconds.

  From: /tf/active/vicechatdev/patches/util.py

• function format_datetime_v1 57.8% similar

  Converts an ISO format datetime string into a human-readable UTC datetime string formatted as 'YYYY-MM-DD HH:MM:SS UTC'.

  From: /tf/active/vicechatdev/SPFCsync/dry_run_test.py

• function cast_array_to_int64 55.0% similar

  Converts a numpy array to int64 dtype while suppressing FutureWarning about datetime64 to int64 casting that is deprecated in newer numpy/pandas versions.

  From: /tf/active/vicechatdev/patches/util.py