• [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of pandas.

  • [X] (optional) I have confirmed this bug exists on the master branch of pandas.

Code Sample, a copy-pastable example

import numpy as np 
import pandas as pd 
np.random.seed(1)
# Create one dataframe 
dr = pd.date_range("2021-01-01", "2021-08-30")
ind = np.random.randint(0,5,len(dr))
something = np.random.random(len(ind))
df = pd.DataFrame({"date":dr, "ind":ind, "something":something})

# Define what to group on 
GROUP_ON = ["ind", pd.Grouper(key="date", freq="2W")]
# Copy used if you uncomment lines below 
GROUP_ON_COPY = ["ind", pd.Grouper(key="date", freq="2W")]

# Comment either of the two lines below, and it works 
df.groupby(GROUP_ON).mean().reset_index().groupby(GROUP_ON).mean()
df.groupby(GROUP_ON).mean().reset_index()

# Use a different groupby in the second case, and it works 
#df.groupby(GROUP_ON).mean().reset_index().groupby(GROUP_ON).sum()
#df.groupby(GROUP_ON_COPY).mean().reset_index()

# Bonus points: do only one groupby first, and it works 
# df.groupby(GROUP_ON).mean().reset_index()
# df.groupby(GROUP_ON).mean().reset_index()

Problem description

In some cases it seems that pd.Grouper cannot be reused. I was not able to figure out exactly why, or if the output can sometimes corrupt. It leads to very painful debugging.

Tested with master branch, and with python 3.8 and 3.9

Expected Output

pd.Grouper should be able to be used any number of times, or should raise explicit error upon any reuse.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : 2cb96529396d93b46abab7bbc73a208e708c642e python : 3.9.4.final.0 python-bits : 64 OS : Linux OS-release : 4.4.0-19041-Microsoft Version : #488-Microsoft Mon Sep 01 13:43:00 PST 2020 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : C.UTF-8 LOCALE : en_US.UTF-8 pandas : 1.2.4 numpy : 1.20.3 pytz : 2021.1 dateutil : 2.8.1 pip : 21.1.2 setuptools : 57.0.0 Cython : 0.29.23 pytest : 6.2.4 hypothesis : None sphinx : 4.0.2 blosc : None feather : None xlsxwriter : 1.4.3 lxml.etree : 4.6.3 html5lib : None pymysql : None psycopg2 : None jinja2 : 3.0.1 IPython : 7.24.1 pandas_datareader: None bs4 : 4.9.3 bottleneck : None fsspec : 2021.06.0 fastparquet : None gcsfs : None matplotlib : 3.4.2 numexpr : 2.7.3 odfpy : None openpyxl : 3.0.7 pandas_gbq : None pyarrow : None pyxlsb : None s3fs : None scipy : 1.6.3 sqlalchemy : None tables : 3.6.1 tabulate : None xarray : None xlrd : 2.0.1 xlwt : 1.3.0 numba : None

Comment From: mzeitlin11

Thanks for reporting this @mimakaev, I see this on current master as well. Agree that either of your alternatives would be much better - would be good to know if the lack of reusability is a bug or just implicit by design. Worth mentioning there has been discussion about deprecating this (#41297) in favor of alternatives mentioned in that issue (which may serve as a workaround for your use case).

Comment From: NowanIlfideme

It turns out this bug was the root cause of a completely different issue, "different shapes don't align" within some internal codes when grouping by datetime. It seems to be caching these codes for performance reasons, not expecting to be reused in a later context. It's likely that this even silently caused improper data after grouping!

Comment From: jbrockmendel

Is this still an issue on main? We made pd.Grouper immutable a while ago