Code Sample, a copy-pastable example if possible
import numpy as np
import pandas as pd
with pd.option_context('mode.use_inf_as_null', True):
s = pd.Series([np.nan, np.inf, 0])
d = DataFrame(OrderedDict([
('s', s),
('repr(s)', s.map(repr)),
('s == s', s == s),
('s == np.inf', s == np.inf),
('s.values == np.inf', s.values == np.inf),
]))
d
=>
s repr(s) s == s s == np.inf s.values == np.inf
0 NaN nan False False False
1 NaN inf True False True
2 0.0 0.0 True False False
Problem description
I expected s == s
to return False for inf values with use_inf_as_null
mode on, or at least for s == np.inf
to agree with s.values == np.inf
.
Expected Output
s repr(s) s == s s == np.inf s.values == np.inf
0 NaN nan False False False
1 NaN inf False True True
2 0.0 0.0 True False False
Output of pd.show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.3.final.0
python-bits: 64
OS: Darwin
OS-release: 15.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
pandas: 0.20.2
pytest: 3.1.2
pip: 9.0.1
setuptools: 33.1.1.post20170320
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.19.0
xarray: None
IPython: 5.0.0
sphinx: None
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.4.2
numexpr: 2.6.2
feather: 0.4.0
matplotlib: 2.0.2
openpyxl: 2.4.8
xlrd: 1.0.0
xlwt: None
xlsxwriter: None
lxml: 3.8.0
bs4: 4.5.3
html5lib: 0.999999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.5
s3fs: None
pandas_gbq: None
pandas_datareader: None
Comment From: chris-b1
If we're fully copying NaN
semantics, then the output for s = np.inf
is correct.
In [19]: s = pd.Series([np.nan, np.inf, 0])
In [20]: s == np.nan
Out[20]:
0 False
1 False
2 False
dtype: bool
I do agree on the s == s
output - I'm not sure this mode is that widely tested, PRs welcome.
Comment From: wcbeard
This also seems a bit weird
s = pd.Series([np.nan, np.inf, 0])
s.isin([np.inf])
=>
0 True
1 True
2 False
dtype: bool
Comment From: jbrockmendel
use_inf_as_null has been gone for a while. closing.