Pandas version checks
-
[x] I have checked that this issue has not already been reported.
-
[x] I have confirmed this bug exists on the latest version of pandas.
-
[ ] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
from datetime import datetime, timezone
# Saving and loading without timezones (works fine)
df = pd.DataFrame({
'timestamp': [
datetime(2025, 8, 14, 12, 34, 56, 0), # No microseconds
datetime(2025, 8, 14, 12, 34, 56, 1) # With microseconds
]
})
df.to_csv('test.csv', index=False)
## == test.csv == ##
# timestamp
# 2025-08-14 12:34:56.000000
# 2025-08-14 12:34:56.000001
####################
df2 = pd.read_csv("test.csv", parse_dates=["timestamp"])
print(df2.dtypes["timestamp"]) # datetime64[ns]
# Saving and loading with timezones (broken)
df = pd.DataFrame({
'timestamp': [
datetime(2025, 8, 14, 12, 34, 56, 0, tzinfo=timezone.utc), # No microseconds
datetime(2025, 8, 14, 12, 34, 56, 1, tzinfo=timezone.utc) # With microseconds
]
})
## == test.csv == ##
# timestamp
# 2025-08-14 12:34:56+00:00
# 2025-08-14 12:34:56.000001+00:00
####################
df.to_csv('test.csv', index=False)
df2 = pd.read_csv("test.csv", parse_dates=["timestamp"])
print(df2.dtypes["timestamp"]) # object
Issue Description
When saving a datetime column with timezones to a csv, pandas uses an inconsistent format across entries depending on whether the microseconds is zero or nonzero. This means that pandas then fails to parse that column when reading in the same file.
Expected Behavior
The save behavior for dates with timezones should match the behavior for dates without timezones in this case: i.e. use a consistent datetime format that includes microseconds for all rows.
Installed Versions
Comment From: prazian
take
Comment From: prazian
PR https://github.com/pandas-dev/pandas/pull/62139 ready for review