Pandas version checks
-
[x] I have checked that this issue has not already been reported.
-
[x] I have confirmed this bug exists on the latest version of pandas.
-
[x] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
ser = pd.Series([4,6,9,None,10,13,15],index=[6,1,5,0,3,2,4],dtype='Int64')
indices = pd.Series([6,1,5,0,3,2,4],index=[6,1,5,0,3,2,4],dtype='int64')
values = pd.Series([4,6,9,None,10,13,15],index=[4,1,2,6,0,5,3],dtype='Int64')
ser.iloc[indices] = values
print(ser)
Issue Description
Traceback (most recent call last):
File "/home/marcogorelli/scratch/t.py", line 8, in <module>
ser.iloc[indices] = values
~~~~~~~~^^^^^^^^^
File "/home/marcogorelli/scratch/.venv/lib/python3.12/site-packages/pandas/core/indexing.py", line 911, in __setitem__
iloc._setitem_with_indexer(indexer, value, self.name)
File "/home/marcogorelli/scratch/.venv/lib/python3.12/site-packages/pandas/core/indexing.py", line 1944, in _setitem_with_indexer
self._setitem_single_block(indexer, value, name)
File "/home/marcogorelli/scratch/.venv/lib/python3.12/site-packages/pandas/core/indexing.py", line 2218, in _setitem_single_block
self.obj._mgr = self.obj._mgr.setitem(indexer=indexer, value=value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/marcogorelli/scratch/.venv/lib/python3.12/site-packages/pandas/core/internals/managers.py", line 415, in setitem
return self.apply("setitem", indexer=indexer, value=value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/marcogorelli/scratch/.venv/lib/python3.12/site-packages/pandas/core/internals/managers.py", line 363, in apply
applied = getattr(b, f)(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/marcogorelli/scratch/.venv/lib/python3.12/site-packages/pandas/core/internals/blocks.py", line 2129, in setitem
values[indexer] = value
~~~~~~^^^^^^^^^
File "/home/marcogorelli/scratch/.venv/lib/python3.12/site-packages/pandas/core/arrays/masked.py", line 320, in __setitem__
value, mask = self._coerce_to_array(value, dtype=self.dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/marcogorelli/scratch/.venv/lib/python3.12/site-packages/pandas/core/arrays/numeric.py", line 272, in _coerce_to_array
values, mask, _, _ = _coerce_to_data_and_mask(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/marcogorelli/scratch/.venv/lib/python3.12/site-packages/pandas/core/arrays/numeric.py", line 209, in _coerce_to_data_and_mask
if int(values[idx]) != original[idx]:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pandas/_libs/missing.pyx", line 392, in pandas._libs.missing.NAType.__bool__
TypeError: boolean value of NA is ambiguous
Note that for Int64[pyarrow] or 'int64`, it'"s fine
Expected Behavior
6 <NA>
1 6
5 13
0 10
3 15
2 9
4 4
dtype: Int
Installed Versions
Comment From: athithiyarajk
The check: if int(values[idx]) != original[idx] is the problem.
What's Happening
When a value in the pandas Series is missing (like a None value), it's treated as an internal pandas._libs.missing.NAType.
When you compare anything to that missing value (value != NAType), the comparison itself returns NAType, not a standard Python True or False.
The Python if statement then tries to evaluate this NAType as a boolean, which it can't do, throwing the exception.
The Proposed Solution
My proposal is to check the result of the comparison right before the if statement runs.
If the result is an object of type pandas._libs.missing.NAType, we override it and assign np.False_ instead.
This way, the if condition always receives a valid boolean. Crucially, assigning False handles the missing value case correctly: we assume a missing value is not a difference, and the if block is skipped.
I'll proceed with implementing a focused check for NAType to patch this up if this sounds good, please assign this issue to me.