Fixing ❌ “Poetry configuration is invalid” Errors in pre-commit.ci GitHub Actions#
Last Sunday I decided to try and understand the Sphinx codebase better after playing around with Jupyter Book and experiencing issue#1414. After falling down the codebase review rabbit hole I felt like fixing the bug itself was worthwhile. Eventually, fixing the issue proved to be easier than understanding where and what to fix or how to get the GitHub actions maze of checks to properly pass.
Introduction#
Sphinx is a marvelous and useful piece of software, but it’s also rather complex. In my opinion it can be daunting to both for developers trying to navigate it and for users trying to use the tool or debug issues when they arise. Jupyter Book, therefore, is a welcome piece of syntactic sugar that improves upon Sphinx’s user experience.
I’ll document the process I went through to navigate Sphinx’s unfamiliar codebase, understand the issue and fix it in a future post. For now, however, it suffices to say that after creating my fork, my fix branch and pushing the suggested fix I was expecting the checks to pass without issue.
I quickly realized I hadn’t installed the project’s pre-commit.ci git hooks. I was skeptical right away as I prefer the GitLab approach to CI/CD and felt for awhile that the GitHub CI landscape is hyper-fragmented and that leads to exploding complexity. Since I always use virtual environments managed with PipEnv when working with Python projects, it was relatively easy to install the pre-commit and its hooks.
Virtual Environment Setup with Pipenv#
I installed sphinx-external-toc in editing mode(-e) with its development(-d) and optional testing and code_style dependencies as one would expect:
$ pipenv install -d -e .[testing,code_style]
Installing -e .[testing,code_style]...
Resolving -e ....
Installing...
Adding sphinx_external_toc to Pipfile's [dev-packages] ...
✔ Installation Succeeded
Pipfile.lock (654b48) out of date, updating to (eb4938)...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (f7e3c98039a6123cd674eec84c0f7ea1ff71b36b8ebcc1dc8b16111681eb4938)!
Installing dependencies from Pipfile.lock (eb4938)...
Installing dependencies from Pipfile.lock (eb4938)...
The Pipfile looked as expected:
$ cat Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
[dev-packages]
tox = "*"
sphinx-external-toc = {editable = true, extras = ["code_style", "testing"], path = "."}
Package Extras and pre-commit#
sphinx-external-toc’s testing optional dependencies already included pre-commit as seen in pyproject.toml:
[...]
[project.optional-dependencies]
code_style = ["pre-commit~=2.12"]
rtd = [
"myst-parser~=0.17.0",
"sphinx-book-theme>=0.0.36",
]
testing = [
"coverage",
"pytest~=7.1",
"pytest-cov",
"pytest-regressions",
]
[...]
pre-commit Git Hooks#
So, pre-commit’s Python package should be installed in my Pipenv environment and available for use. Installing the desired hooks was then a matter of pre-commit install:
$ pre-commit install
pre-commit installed at .git/hooks/pre-commit
pre-commit’s hooks were thus installed and the checks intended by the project maintainers to always run were supposed to happen. Unfortunately, the following occurred after trying to commit and push some changes:
$ git commit -m 'fix typo in TODO definition'
[INFO] Installing environment for https://github.com/pycqa/isort.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/[...]/.cache/pre-commit/repo2nhf5f8m/py_env-python3/bin/python', '-mpip', 'install', '.')
return code: 1
stdout:
Processing [...]/.cache/pre-commit/repo2nhf5f8m
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
stderr:
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [14 lines of output]
Traceable (most recent call last):
File "[...]/.cache/pre-commit/repo2nhf5f8m/py_env-python3/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "[...]/.cache/pre-commit/repo2nhf5f8m/py_env-python3/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "[...]/.cache/pre-commit/repo2nhf5f8m/py_env-python3/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/tmp/pip-build-env-qgl92m2t/overlay/lib/python3.10/site-packages/poetry/core/masonry/api.py", line 41, in prepare_metadata_for_build_wheel
poetry = Factory().create_poetry(Path(".").resolve(), with_groups=False)
File "/tmp/pip-build-env-qgl92m2t/overlay/lib/python3.10/site-packages/poetry/core/factory.py", line 58, in create_poetry
raise RuntimeError("The Poetry configuration is invalid:\n" + message)
RuntimeError: The Poetry configuration is invalid:
- [extras.pipfile_deprecated_finder.2] 'pip-shims<=0.3.4' does not match '^[a-zA-Z-_.0-9]+$'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
At this point I kept thinking about how I’ve always disliked GitHub actions and why I am doing this to myself; going through endless rabbit holes just to submit a fix to a software I care about. After some tea I continued debugging and in a last ditch attempt to not understand pre-commit but still fix my issues I looked at its Command Line Interface(CLI) help:
$ pre-commit -h
usage: pre-commit [-h] [-V]
{autoupdate,clean,gc,init-templatedir,install,install-hooks,migrate-config,run,sample-config,try-repo,uninstall,validate-config,validate-manifest,help,hook-impl}
...
positional arguments:
{autoupdate,clean,gc,init-templatedir,install,install-hooks,migrate-config,run,sample-config,try-repo,uninstall,validate-config,validate-manifest,help,hook-impl}
autoupdate 👀 Auto-update pre-commit config to the latest repos' versions.
clean Clean out pre-commit files.
gc Clean unused cached repos.
init-templatedir Install hook script in a directory intended for use with `git config init.templateDir`.
install Install the pre-commit script.
install-hooks Install hook environments for all environments in the config file. You may find `pre-commit install --install-hooks` more useful.
migrate-config Migrate list configuration to new map configuration.
run Run hooks.
sample-config Produce a sample .pre-commit-config.yaml file
try-repo Try the hooks in a repository, useful for developing new hooks.
uninstall Uninstall the pre-commit script.
validate-config Validate .pre-commit-config.yaml files
validate-manifest Validate .pre-commit-hooks.yaml files
help Show help for a specific command.
Have You Tried Turning it On and Off?#
autoupdate caught my attention, as sphinx-external-toc is simple, works well and has been stale regarding features and codebase movement for quite some time with its last commit 4e83bb6 on Nov 25, 2022. I decided to try my luck:
$ pre-commit autoupdate
Updating https://github.com/pre-commit/pre-commit-hooks ... [INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
updating v4.3.0 -> v4.4.0.
Updating https://github.com/pycqa/isort ... [INFO] Initializing environment for https://github.com/pycqa/isort.
updating 5.10.1 -> 5.12.0.
Updating https://github.com/psf/black ... [INFO] Initializing environment for https://github.com/psf/black.
updating 22.6.0 -> 23.3.0.
Updating https://github.com/PyCQA/flake8 ... [INFO] Initializing environment for https://github.com/PyCQA/flake8.
updating 5.0.2 -> 6.0.0.
Updating https://github.com/pre-commit/mirrors-mypy ... [INFO] Initializing environment for https://github.com/pre-commit/mirrors-mypy.
updating v0.971 -> v1.4.1.
After staging my changes to the pre-commit configuration (.pre-commit-config.yml) which happened as part of the update the checks happened and worked as expected 🎉 :
$ git add -u
$ git status -s
M .pre-commit-config.yaml
M sphinx_external_toc/events.py
?? Pipfile
?? Pipfile.lock
$ pre-commit
check json...........................................(no files to check)Skipped
check yaml...............................................................Passed
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
isort....................................................................Passed
black....................................................................Passed
flake8...................................................................Passed
mypy.....................................................................Passed
Of course there where other issues, it’s the GitHub actions maze after all, but those were unrelated to the pre-commit hooks:
Final Words#
I hope you enjoyed this write-up and as always, don’t hesitate to share your feedback with me via social media or email!