This does not occur on the main branch, only 2.3.x. I plan to run a git-bisect later today.

pd.set_option("infer_string", True)
ser = pd.Series(["a", np.nan, "a", "a"])
print(ser.replace({"a": "b"}))
# 0      b
# 1    NaN
# 2      a
# 3      b
# dtype: str

Comment From: rhshadrach

This was https://github.com/pandas-dev/pandas/pull/57865. Might want to backport this, but will discuss in that PR. Closing.