Locked issue

  • [x] I locked this voting issue so that only voting members are able to cast their votes or comment on this issue.

PDEP number and title

PDEP-15: Reject PDEP-10

Pull request with discussion

https://github.com/pandas-dev/pandas/pull/58623

Rendered PDEP for easy reading

https://github.com/pandas-dev/pandas/blob/c159851cc0762625f9e51f9d9bd1d18011b79aa7/web/pandas/pdeps/0015-do-not-require-pyarrow.md

Discussion participants

5 voting members (active maintainers) participated in the discussion

Voting will close in 15 days.

2025-06-22

Vote

Cast your vote in a comment below. * +1: approve. * 0: abstain. * Reason: A one sentence reason is required. * -1: disapprove * Reason: A one sentence reason is required. A disapprove vote requires prior participation in the linked discussion PR.

@pandas-dev/pandas-core

Comment From: datapythonista

-1

While there are surely some challenges and costs of requiring PyArrow, I think the benefits of already requiring PyArrow in pandas 3.0 are enough to justify the requirement. If pandas becomes simpler, faster and better for most users, and for the development of the project, and the cost is few edge cases having to stay in pandas 2.0 or find an alternative lightweight technology, that's fine with me. Also, the sooner we require PyArrow, the sooner the issues with PyArrow will be addressed, and most of the original concerns have been addressed anyway (including unsupported architectures, reducing installation size, support for wasm...).

Comment From: simonjayhawkins

-1

rejecting PDEP-10 is like throwing the baby out with the bathwater. The issue appears to be just the timing of making PyArrow the required dependency. Of course if others want to reject PDEP-10 and propose keeping PyArrow an optional dependency indefinitely then that's a different issue and I would potentially vote differently on that.

Comment From: rhshadrach

A call for vote is in violation of PDEP-1.

  • After 30 days, with a note that there is at most 30 days remaining for discussion, and that a vote will be called for if no discussion occurs in the next 15 days.
  • After 45 days, with a note that there is at most 15 days remaining for discussion, and that a vote will be called for in 15 days. ... After 30 discussion days, in case 15 days passed without any new unaddressed comments, the authors may close the discussion period preemptively, by sending an early reminder of 15 days remaining until the voting period starts.

The timeline requires a 15 day announcement to commence with voting. This has not occurred.

Comment From: datapythonista

Thanks for pointing that out @rhshadrach.

This PDEP is stalled, Thomas is not an active maintainer anymore, and I don't think there are technical details to discuss here. In my opinion it's just a formal document to reject PDEP-10, with a summary of the reasons. While it's nice to have the summary, I don't think a PDEP was necessary in the first place, we could also have voted again on PDEP-10 instead, or just vote via email as we used to do. In that sense feels a special PDEP.

Also, I don't think it makes sense to only allow people who commented in the PDEP to vote. This PDEP didn't have a technical discussion in the sense of defining an API or something. People could be totally happy with what was said here, and vote against it.

In any case, you're technically right. If you think the 15 days announcement is useful, let's do it. But after one year I don't think there is much to discuss in the content of this PDEP. I think it'd be better to understand what the team wants as soon as possible, and have the discussions on the technical details, not in this formalism. In general we've been using common sense over bureacracy, at least with the previous governance, which was very disconnected from reality. But it'll be good to know other's opinion.

Do you want to have the 15 day announcement? Will you update this PDEP with the feedback from the new discussions, or should we just wait the 15 days to be strict with our own policies?

Comment From: rhshadrach

Do you want to have the 15 day announcement?

I do think the announcement should happen, as PDEP-1 calls for, on both GitHub and email, as this is not my decision to make.

Comment From: datapythonista

I was just reading, and the policy also says A PDEP discussion will remain open for up to 60 days.. Not sure what exactly it implies, as it's not mentioned, but technically the PDEP is not under discussion anymore. I surely don't want to do it, but feels like technically this should be closed as rejected or as invalid since it violated the PDEP process in the first place.

I'll send the announcement, but personally, I think the governance should be to make our lives easier. It's not working so well, since this and other PDEPs are being an impediment to decision making, not facilitating it as it was intended. And we should probably put our efforts on updating the process when it's not useful. Just blindly following what is written, just because it's written, doesn't seem the right approach to me. From your comment I don't think you consider the announcement and waiting useful, but you think we should respect the process. It's a valid point (I disagree with), as feels like we'll just waste 15 days for no reason, but no big deal.

Closing here (I'll reopen in 15 days), and will send the announcement.

Comment From: Dr-Irv

Do you want to have the 15 day announcement?

I do think the announcement should happen, as PDEP-1 calls for, on both GitHub and email, as this is not my decision to make.

I do want to point out that here: https://github.com/pandas-dev/pandas/pull/58623#pullrequestreview-2264483517 , which was on August 27, 2004, I did call for the remaining 15 days of discussion, but never followed up to start the vote.

Nevertheless, things have changed in the 9+ months since, so having another 15 days of discussion is probably a good thing to do.

Comment From: Dr-Irv

@datapythonista I think it would be better if we opened a NEW voting issue on this topic so that there isn't confusion with the earlier discussion above.

Also, would be good to clarify that a +1 vote means we are not requiring pyarrow in 3.0, and -1 vote means we are requiring pyarrow in 3.0.

Comment From: Dr-Irv

-1

PDEP-14 already passed and says that pyarrow will NOT be a hard dependency for version 3.0

Comment From: rhshadrach

-1; same reason as @Dr-Irv

Comment From: jbrockmendel

-1; same as @rhshadrach

Comment From: mroeschke

-1; same reasons as the others

Comment From: WillAyd

-1; ditto

Comment From: jbrockmendel

Shoot, I got confused. I vote +1, i.e. to confirm the vote in PDEP14 to not require pyarrow in 3.0.

Comment From: Dr-Irv

I believe we can close this vote. We have 7 negative votes. If all 15 core team members vote, that would require 11 positive votes, which is impossible to obtain. If we receive the minimum quorum required (11 votes), it would still be impossible for PDEP-15 to pass.

So the interpretation of this outcome is as follows:

  • PDEP-15 is rejected
  • However, because in PDEP-14 there was an agreement that pyarrow will NOT be a hard dependency for version 3.0, PDEP-10 remains accepted, except for the specific details regarding requiring pyarrow in version 3.0. In other words, the plan is that pyarrow will be required in the future, but the timeline for that remains uncertain.

Please let the group know if you disagree with the interpretation of this outcome.

There is open discussion about possible ways forward in #61618 ,

Comment From: attack68

0. Due to lack participation in underlying discussion my options are in {0, 1} I believe.

Comment From: simonjayhawkins

I believe we can close this vote.

IIUC voters are allowed to change their vote at any time during the voting period. I recall @jorisvandenbossche explicitly say that at some point but can't find the discussion now. IIRC I was somewhere saying that if I had voted on PDEP-8 and then seen @jreback comment I would have wanted to change my vote.

Now IIUC we don't have any procedure for rescinding or updating a vote once cast, so it is probably allowed? Indeed, @jbrockmendel has changed his vote here.

So I would just let the vote run it's course for now to avoid further claims of procedural irregularities.

It probably won't change the outcome. What would be the benefit of closing the vote early?

Comment From: jorisvandenbossche

(I am fine with keeping it open to follow procedure, as indeed people can change their vote, although not sure that changes much in this case .. , see below)

I would personally rather not vote because:

  • The PDEP text is outdated, so even if would support the main propostion (i.e. not require pyarrow for 3.0), I can't really approve the current PDEP
  • It is not clear what either a approval or rejection actually means in practice. See Irv's question above about this at the start of the vote, or Brock's confusion switching his vote (and the confusion other people have expressed on the feedback issue at https://github.com/pandas-dev/pandas/issues/54466#issuecomment-3009044920). We should ideally have clarified that before voting.

But if I have to vote, I'll abstain (0)

I will add some more context to the PDEP discussion in https://github.com/pandas-dev/pandas/pull/58623, to keep most of the discussion there. EDIT: see https://github.com/pandas-dev/pandas/pull/58623#issuecomment-3023587822

Comment From: phofl

I think I am locked into {0, 1}, so voting 0 here

I agree with the others who voted -1

Comment From: datapythonista

Please let the group know if you disagree with the interpretation of this outcome.

Your summary seems accurate @Dr-Irv. From my side, I'm happy to finish the vote early, regardless of the -1/0/1 result I think it's clear what the team wants, which was my reason to call the vote. I'd probably close this PDEP, as I don't think feel having it as a rejected PDEP is very useful, but ok with that too.

The main takeaway for me here, besides confirming and formalizing what many assumed about PDEP-14 is that the governance needs improvements, as it's not being helpgul. I'm probably biased as I was one of the very few people who voted against the the new governance. But I think what we need is a governance that focus on solving the very specific problems we have regarding governance, and not focus on corporation-like processes and bureacracy that quite often we won't want to follow. Like waiting for this vote to close, have a formal 90 days PDEP process to remove the "PyArrow will be required warning"... If I had to decide the governance myself I'd define the basics on how a PDEP is approved, on when a PR can be merged, I'd appoint a BDFL that can behave as a referee/leader when having a flat organization prevents from taking action. Not that the BDFL needs to make any technical decision, but check he can check with the relevant person, propose a solution that all the interested parties can agree on... Regardless of what the details of the voting, who the BDFL is, or what the final outcome of the discussions is, I think a governance of this kind would make things way more efficient, and enjoyable.