Pandas RLS: 2.3 - Aurora Blog|java/go/python

Release Date: ???

Features:

[ ] Non-breaking PDEP-14 implementation https://github.com/pandas-dev/pandas/pull/58551:
[ ] https://github.com/pandas-dev/pandas/issues/54792
[ ] groupby.apply DeprecationWarning -> FutureWarning
[x] ~Publish Python 3.13 wheels~ (done in 2.2.3)
[x] ~Numpy 2.1 compatibility (may also require dropping Python 3.9)~ (done in 2.2.3)

cc @pandas-dev/pandas-core

Comment From: mroeschke

I updated the OP with the items to be included in 2.3 based on the pandas dev meeting notes. I believe the release date will be dependent on when the PDEP-14 is implemented and the pandas test suite passes with future.infer_string

The following PRs may need to be backported to 2.3:

Dropping Python 3.9: https://github.com/pandas-dev/pandas/pull/58238
Numpy 2.1 compat: https://github.com/pandas-dev/pandas/pull/59441
Python 3.13 build wheel changes: (PRs linked from https://github.com/pandas-dev/pandas/issues/59057)

These backports probably should probably come before/after https://github.com/pandas-dev/pandas/pull/59513 cc @WillAyd

Comment From: lithomas1

List of TODOS for infer_string (including xfailed tests) can be found here I think.

https://github.com/search?q=repo%3Apandas-dev%2Fpandas+TODO%28infer_string%29+language%3APython&type=code

Comment From: rhshadrach

There is both a 2.3.0 and a 2.3.x branch, which one should we target for PRs?

Comment From: WillAyd

2.3.x I made the 2.3.0 branch by mistake but can't delete it

Comment From: rhshadrach

@jorisvandenbossche - would you have the ability to delete?

Comment From: mroeschke

I went ahead and deleted the 2.3.0 branch

Comment From: jorisvandenbossche

I do wonder if we should consider doing a 2.2.3 with specifically the Python 3.13 and numpy 2.1 changes. Downstream projects might appreciate that we have compatible wheels relatively soon, and when we will be exactly ready for 2.3 is still a bit unsure.

Comment From: lithomas1

I do wonder if we should consider doing a 2.2.3 with specifically the Python 3.13 and numpy 2.1 changes. Downstream projects might appreciate that we have compatible wheels relatively soon, and when we will be exactly ready for 2.3 is still a bit unsure.

That's a good idea. If we follow the normal release process for 2.3.0, we'd have to do an RC aswell. Assuming we are able to release an RC next week somehow, that would only leave 2 weeks for addressing any issues from the RC before Python 3.13 comes out, which is a pretty tight schedule.

Personally, I think it's really important that we get a Python 3.13 compatible release out before 3.13 final is due on October 1st. https://peps.python.org/pep-0719/

Thoughts @pandas-dev/pandas-core?

Comment From: jorisvandenbossche

I don't think next week is realistic for 2.3.0, based on the current speed of progress (I think end of the month would be my best (optimistic) estimate)

Comment From: WillAyd

I would be OK with another 2.2.3 release if someone had time / interest for it. However, same word of caution as with 2.3 - I think getting the backport branch updated and passing tests is a rather large effort, so don't want to underestimate the time that may take

Comment From: mike-jn

Has the performance regression with infer strings mentioned in https://github.com/pandas-dev/pandas/issues/59657 been fixed in 2.3.x? Because when I enable this option in 2.2 I also experience a perf regression.

Comment From: jorisvandenbossche

@mikev-junimo good question. I took a quick look, and it seems this is not fixed on 2.3.x, only on main (thanks to https://github.com/pandas-dev/pandas/pull/57205, but that is not backported). I have opened a smaller PR to also just fix it for 2.3.x: https://github.com/pandas-dev/pandas/pull/60461

Comment From: mike-jn

@jorisvandenbossche thanks! I've been wanting to enable it in our env but couldn't due to that bug.

Comment From: mroeschke

@lithomas1 in today's dev meeting we discussed releasing 2.3 at the end of the month. Do you have bandwidth for the release?

Comment From: auderson

@mroeschke Hi, any news on release 3.0?

Comment From: mroeschke

Hi, any news on release 3.0?

Ideally pandas 3.0 would come out a month or two after the 2.3 release

Comment From: lithomas1

@lithomas1 in today's dev meeting we discussed releasing 2.3 at the end of the month. Do you have bandwidth for the release?

Probably not. I'm planning on stepping back from pandas/open source in about a month or so.

Happy to answer any quesitons about the release process, though.

Comment From: mroeschke

Sounds good @lithomas1. I can take care of the release.

Is https://pandas.pydata.org/docs/development/maintaining.html#release-process still up to date? Would appreciate any updates to those steps if anything has changed.

Comment From: MarcoGorelli

If there's going to be a 2.3, any objections to backporting https://github.com/pandas-dev/pandas/pull/60739 ?

(https://github.com/pandas-dev/pandas/pull/60739#issuecomment-2888291202 😳 🙈 )

Comment From: lithomas1

Sounds good @lithomas1. I can take care of the release.

Is https://pandas.pydata.org/docs/development/maintaining.html#release-process still up to date? Would appreciate any updates to those steps if anything has changed.

Yes I think so. It's been a while since the last release, though, so it's possible things are out of date. Do you have access to PyPI and the pandas website?

@datapythonista should be able to add your ssh key to the pandas website. I'm not sure who has maintainer permissions on PyPI - I can only publish releases.

Comment From: mroeschke

I don't think there's any more blockers for 2.3 so I'll start working on the release tomorrow Wednesday June 4th.

I was able to get maintainer permissions on PyPI for pandas. I'll contact @datapythonista on a separate channel to add my ssh key to the pandas website

Comment From: a-reich

@mroeschke will https://github.com/pandas-dev/pandas/issues/60639 be implemented in 2.3?

Comment From: mroeschke

will https://github.com/pandas-dev/pandas/issues/60639 be implemented in 2.3?

Yes, it will.

Comment From: mroeschke

pandas 2.3 wheels are now available on PyPI: https://pypi.org/project/pandas/2.3.0

pandas 2.3 wheels are being generated for conda-forge: https://github.com/conda-forge/pandas-feedstock/pull/226

I will announce the release on our mailing list tomorrow; not sure if I have access to announce on our socials.

Comment From: a-reich

FWIW the example from the release whatsnew note still raises for me on 2.3:

import pandas as pd, numpy as np
arr1 = pd.array([],pd.StringDtype("pyarrow", na_value=pd.NA))
arr2 = pd.array([], pd.StringDtype("python", na_value=np.nan))
arr1 == arr2 # NotImplementedError: eq not implemented for <class 'pandas.core.arrays.string_.StringArrayNumpySemantics'>

EDIT: created https://github.com/pandas-dev/pandas/issues/61570 to track this, although it's really a repeat of the original issue.