Pandas version checks
-
[x] I have checked that this issue has not already been reported.
-
[x] I have confirmed this bug exists on the latest version of pandas.
-
[x] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
import xarray as xr
import numpy as np
import time
# Minimal reproducible example: Pandas Series print performance with large xarray DataArrays
print("Pandas Series Print Performance Issue")
print("=" * 40)
# Create minimal test data
np.random.seed(42)
time_coords = pd.date_range('2023-01-01', periods=2, freq='D')
x_coords = np.linspace(0, 10, 256)
y_coords = np.linspace(0, 10, 256)
# Create one large xarray DataArray
data = np.random.randn(2, 256, 256)
large_dataarray = xr.DataArray(
data,
coords={'time': time_coords, 'y': y_coords, 'x': x_coords},
dims=['time', 'y', 'x']
)
# Create minimal pandas Series with large DataArray
series = pd.Series({
'id': 1,
'data': large_dataarray,
'name': 'test_series'
})
print(f"DataArray size: {large_dataarray.nbytes / 1024 / 1024:.1f} MB")
print(f"DataArray shape: {large_dataarray.shape}")
# Method 1: Print Series directly
print("\nMethod 1: Print Series")
start_time = time.time()
print(series)
method1_time = time.time() - start_time
# Method 2: Extract DataArray first, then print it
print("\nMethod 2: Extract DataArray first, then print")
start_time = time.time()
extracted_da = series['data']
print(extracted_da)
method2_time = time.time() - start_time
# Results
print(f"\nTiming Results:")
print(f"Method 1 (print Series): {method1_time:.4f} seconds")
print(f"Method 2 (extract + print DataArray): {method2_time:.4f} seconds")
print(f"Difference: {abs(method1_time - method2_time):.4f} seconds")
if method1_time > method2_time:
ratio = method1_time / method2_time
print(f"Method 1 is {ratio:.1f}x slower than Method 2")
else:
ratio = method2_time / method1_time
print(f"Method 2 is {ratio:.1f}x slower than Method 1")
# Environment info
print(f"\nEnvironment:")
print(f"Pandas: {pd.__version__}")
print(f"XArray: {xr.__version__}")
print(f"NumPy: {np.__version__}")
Issue Description
Hi Pandas team, so I was working with pandas series and was trying to put an xarray into a cell.
So when I was trying to print out the pandas series with the xarray, I found that it is extremely slow, directly printing out the pandas series is 1000X slower than getting the xarray and then print out the xarray.
The above script is an example with an xarray inside a pandas series, and the time comparison between printing the pandas series directly and get the xarray first and them print the values.
Possible issue: String formatting with xarray.
Expected Behavior
Similar time consumption for directly printing the pandas series and get the xarray and print the content.
Installed Versions
Comment From: arthurlw
From a surface-level investigation, it seems the difference is caused because when printing a Series, pandas forces repr to be called on each element, whereas I assume extracting the data first requires calling repr only once.
Whether this is something that should be changed needs more discussion.
Thanks for opening this issue!