Installation check
- [x] I have read the installation guide.
Platform
Ubuntu 24.04.2 LTS
Installation Method
Built from source
pandas Version
Build from main
Python Version
py310
Installation Logs
Doing
git clone https://github.com/pandas-dev/pandas
cd pandas
python setup.py build_ext -j 4
Results in
Cython.Compiler.Errors.InternalError: Internal compiler error: 'free_threading_config.pxi' not found
File "/home/runner/work/fastparquet/fastparquet/pandas/setup.py", line 684, in
ext_modules=maybe_cythonize(extensions, compiler_directives=directives),
File "/home/runner/work/fastparquet/fastparquet/pandas/setup.py", line 424, in maybe_cythonize
return cythonize(extensions, *args, **kwargs)
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 1010, in cythonize
module_list, module_metadata = create_extension_list(
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line [85](https://github.com/dask/fastparquet/actions/runs/13975910111/job/39129502656?pr=951#step:6:86)9, in create_extension_list
kwds = deps.distutils_info(file, aliases, base).values
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 707, in distutils_info
return (self.transitive_merge(filename, self.distutils_info0, DistutilsInfo.merge)
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 716, in transitive_merge
return self.transitive_merge_helper(
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 722, in transitive_merge_helper
deps = extract(node)
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 6[87](https://github.com/dask/fastparquet/actions/runs/13975910111/job/39129502656?pr=951#step:6:88), in distutils_info0
cimports, externs, incdirs = self.cimports_externs_incdirs(filename)
File "Cython/Utils.py", line 129, in Cython.Utils.cached_method.wrapper
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 574, in cimports_externs_incdirs
for include in self.included_files(filename):
File "Cython/Utils.py", line 129, in Cython.Utils.cached_method.wrapper
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 556, in included_files
include_path = self.context.find_include_file(include, source_file_path=filename)
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Compiler/Main.py", line 299, in find_include_file
error(pos, "'%s' not found" % filename)
File "/usr/share/miniconda/envs/test/lib/python3.10/site-packages/Cython/Compiler/Errors.py", line 178, in error
raise InternalError(message)
Cython.Compiler.Errors.InternalError: Internal compiler error: 'free_threading_config.pxi' not found
I am trying to solve this trouble which has raised in this fastparquet CI workflow while working on some other stuffs.
Environment file is populated this way:
name: test_env
channels:
- conda-forge
dependencies:
- python=3.10
- bson
- lz4
- lzo
- pytest
- pandas
- dask
- pytest-cov
- thrift
- numpy
- cramjam
- pyspark=3.3
- packaging
- orjson
- ujson
- python-rapidjson
- meson-python
- pyarrow
Workflow is as follows.
pandas:
name: pandas
runs-on: ubuntu-latest
steps:
- name: APT
run: sudo apt-get install liblzo2-dev
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Fetch upstream tags
run: |
git remote add upstream https://github.com/dask/fastparquet.git
git fetch upstream --tags
- name: Setup conda
uses: conda-incubator/setup-miniconda@v3
with:
environment-file: ci/environment-py310.yml
- name: pip-install
shell: bash -l {0}
run: |
pip install Cython
pip install hypothesis
pip install pytest-localserver pytest-xdist pytest-asyncio
pip install -e . --no-deps # Install fastparquet
pip install versioneer # Needed for pandas build
git clone https://github.com/pandas-dev/pandas
cd pandas
python setup.py build_ext -j 4
pip install -e . --no-build-isolation
- name: Run Tests
shell: bash -l {0}
run: |
pytest -v fastparquet/ # fastparquet test suite against dev pandas
pytest --verbose pandas/pandas/tests/io/test_parquet.py
# Test parquet with different filesystems
pytest --verbose pandas/pandas/tests/io/test_gcs.py pandas/pandas/tests/io/test_fsspec.py -k "parquet"
Thanks for any help!
Comment From: WuYff
I also got this error. How to solve it?
Comment From: Alvaro-Kothe
Thanks for the report.
I reproduced your error on python3.10. My recommendation is to stop using setup.py, use pip install and also use python>=3.11, since older python versions are no longer supported on upstream.
Here are some commands to install pandas from source on python3.11:
git clone --depth=1 git@github.com:pandas-dev/pandas
cd pandas/
pip install cython versioneer meson-python numpy
pip install -ve . --no-build-isolation -C compile-args="-j4"
python -c 'import pandas; pandas.show_versions()'
Output
INSTALLED VERSIONS
------------------
commit : e87248e1a5d6d78a138039f2856a3aec6b9fef54
python : 3.11.13
python-bits : 64
OS : Linux
OS-release : 6.16.7-200.fc42.x86_64
Version : #1 SMP PREEMPT_DYNAMIC Thu Sep 11 17:46:54 UTC 2025
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : pt_BR.UTF-8
LOCALE : pt_BR.UTF-8
pandas : 0+untagged.1.ge87248e
numpy : 2.3.3
dateutil : 2.9.0.post0
pip : 25.2
Cython : 3.1.4
sphinx : None
IPython : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : None
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
psycopg2 : None
pymysql : None
pyarrow : None
pyiceberg : None
pyreadstat : None
pytest : None
python-calamine : None
pytz : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
qtpy : None
pyqt5 : None