Pandas version checks
-
[x] I have checked that this issue has not already been reported.
-
[x] I have confirmed this bug exists on the latest version of pandas.
-
[x] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
### Using pandas 2.2.3
import pandas as pd
pd.DataFrame([['a', 'b'], ['c', 'd']]).astype('string').to_pickle('G:/temp/test2.pkl')
### Using pandas 2.3.0
import pandas as pd
df = pd.read_pickle('G:/temp/test2.pkl') # looks ok
df.dtypes # raises AttributeError: 'StringDtype' object has no attribute '_na_value'
df[0] + df[1] # also raises AttributeError
Issue Description
The code in a StringDtype object in 2.3 refers to an internal _na_value representation that appears not to have existed prior to 2.3.0. Pickled objects containing StringDtype columns pickled in earlier versions, including 2.2.3, may initially appear to unpickle successfully. However, listing the dtypes or even implicitly checking the dtypes by doing an operation, raises an AttributeError.
Expected Behavior
The documentation at read_pickle indicates backward compatibility to version 0.20.3, so a pickle from 2.2.3 should be readable and usable in 2.3.0.
A current workaround is something like this, to wrap the object in a freshly created 2.3.0-compatible dtype:
def unpickle_wrap(fn):
df = pd.read_pickle(fn)
for col, dtype in df.dtypes.items():
if pd.api.types.is_string_dtype(dtype):
df[col] = df[col].astype(object).astype('string')
return df
Installed Versions
(Edit: fixed example to make copy-pastable, and confirmed on main)
Comment From: Liam3851
take