Include pylintrc and pytest.ini files for each exercise

rzuckerm · May 4, 2024, 11:57am

It would be nice if I could lint my code locally with the same pylint configuration file as the server uses. I was quite surprised the first time I submitted a solution, and the server reported several linting errors.

Also, please include the pytest.ini file. I don’t like seeing warning when I run the tests locally. I discovered what I needed to do to suppress that error, and now I have to copy that file over for every project I do. I know there is Issue 2745 .

I’m willing to do the work on this, but someone will need to point me in the right direction as to what I need to change.

BethanyG · May 4, 2024, 10:20pm

Hi @rzuckerm ,

Thanks for posting.

Both a pytest.ini file and a pylintrc file are available in our content repo.

As stated in the comments in Issue 2745, placing both files at the root of the project (i.e. the /python folder) should make them discoverable to PyTest and PyLint, which will then pick them up for any exercise you complete. I think there may also be similar instructions under the Python Track Docs on the website.

After the files are in place (and you have navigated to a specific exercise folder), the PyTest command would be pytest -v name_of_testfile_test.py

So running that on a Mac in the bash shell in a (PY_3.11.5) virtual env for the Ghost Gobble Arcade Game exercise looks like this (I am omitting all of the verbose PyTest failures):

(PY_3.11.5) pwd
/Users/__/exercism/python/exercises/concept/ghost-gobble-arcade-game

(PY_3.11.5) pytest -v arcade_game_test.py 
platform darwin -- Python 3.11.5, pytest-7.4.3, pluggy-1.4.0 -- /usr/local/anaconda3/envs/PY_3.11.5/bin/python3.11
cachedir: .pytest_cache
rootdir: /Users/__/exercism/python
configfile: pytest.ini
plugins: subtests-0.11.0
collected 13 items                                                                                                                                                      

arcade_game_test.py::GhostGobbleGameTest::test_dont_lose_if_not_touching_a_ghost FAILED                                                                           [  7%]
arcade_game_test.py::GhostGobbleGameTest::test_dont_lose_if_touching_a_ghost_with_a_power_pellet_active FAILED                                                    [ 15%]
arcade_game_test.py::GhostGobbleGameTest::test_dont_win_if_all_dots_eaten_but_touching_a_ghost FAILED                                                             [ 23%]
arcade_game_test.py::GhostGobbleGameTest::test_dont_win_if_not_all_dots_eaten FAILED                                                                              [ 30%]
arcade_game_test.py::GhostGobbleGameTest::test_ghost_does_not_get_eaten_because_no_power_pellet_active FAILED                                                     [ 38%]
arcade_game_test.py::GhostGobbleGameTest::test_ghost_does_not_get_eaten_because_not_touching_ghost FAILED                                                         [ 46%]
arcade_game_test.py::GhostGobbleGameTest::test_ghost_gets_eaten FAILED                                                                                            [ 53%]
arcade_game_test.py::GhostGobbleGameTest::test_lose_if_touching_a_ghost_without_a_power_pellet_active FAILED                                                      [ 61%]
arcade_game_test.py::GhostGobbleGameTest::test_no_score_when_nothing_eaten FAILED                                                                                 [ 69%]
arcade_game_test.py::GhostGobbleGameTest::test_score_when_eating_dot FAILED                                                                                       [ 76%]
arcade_game_test.py::GhostGobbleGameTest::test_score_when_eating_power_pellet FAILED                                                                              [ 84%]
arcade_game_test.py::GhostGobbleGameTest::test_win_if_all_dots_eaten FAILED                                                                                       [ 92%]
arcade_game_test.py::GhostGobbleGameTest::test_win_if_all_dots_eaten_and_touching_a_ghost_with_a_power_pellet_active FAILED                                       [100%]

The corresponding PyLint command would be pylint name_of_codefile_to_lint.py. Here it is for the same exercise (I ran it on the stub, so Pylint is very cranky):

(PY_3.11.5) pwd
/Users/__/exercism/python/exercises/concept/ghost-gobble-arcade-game


(PY_3.11.5)  pylint arcade_game.py
************* Module arcade_game
arcade_game.py:27:0: C0301: Line too long (102/100) (line-too-long)
arcade_game.py:12:4: W0107: Unnecessary pass statement (unnecessary-pass)
arcade_game.py:4:14: W0613: Unused argument 'power_pellet_active' (unused-argument)
arcade_game.py:4:35: W0613: Unused argument 'touching_ghost' (unused-argument)
arcade_game.py:23:4: W0107: Unnecessary pass statement (unnecessary-pass)
arcade_game.py:15:10: W0613: Unused argument 'touching_power_pellet' (unused-argument)
arcade_game.py:15:33: W0613: Unused argument 'touching_dot' (unused-argument)
arcade_game.py:34:4: W0107: Unnecessary pass statement (unnecessary-pass)
arcade_game.py:26:9: W0613: Unused argument 'power_pellet_active' (unused-argument)
arcade_game.py:26:30: W0613: Unused argument 'touching_ghost' (unused-argument)
arcade_game.py:46:4: W0107: Unnecessary pass statement (unnecessary-pass)
arcade_game.py:37:8: W0613: Unused argument 'has_eaten_all_dots' (unused-argument)
arcade_game.py:37:28: W0613: Unused argument 'power_pellet_active' (unused-argument)
arcade_game.py:37:49: W0613: Unused argument 'touching_ghost' (unused-argument)

------------------------------------------------------------------
Your code has been rated at 0.00/10 (previous run: 0.00/10, +0.00)

We have not copied these config files into individual exercise folders for several reasons, the chief being that we would then have to update 288 files every time we needed to change configurations for the tools.

Having per-exercise config files also means more files to download for students, and it doesn’t help them learn how to configure Pytest or PyLint, which are generally set up to discover configurations files at the project level, with local overrides as needed.

While it is not ideal that students working locally have to do a manual one-time copy into the /python folder, until we modify the Exercism CLI (see this discussion and this topic), its the cleanest (and least problematic for maintenance) we can do.

I’ll add reviewing the track docs to my list, and see if there is a way to clarify the setup a bit.

Unfortunately, unless you know Go (our CLI is written in Go-lang), the work that needs to be completed for this is the modification of the CLI to allow common files in the shared/ directory to be downloaded and copied into place for tracks.

BethanyG · May 4, 2024, 10:21pm

[Add-on, since I was limited to 7000 characters above]

I’d welcome suggestions for the track docs however. In particular, clarifications or additions to tests, installation, tools, and tracebacks.

rzuckerm · May 6, 2024, 9:46pm

@BethanyG Thanks for you reply. No, Go is not one of the languages I know.

I did got the pytest.ini and pylintrc file from your content repo. I’ve not tried putting them in the python directory. I’ve just been copying them into the current project I’m working on. I’ll try that on the next exercise I work on. However, with regard to pylintrc, I’ve not really seen a lot of consistency between the settings in common pylintrc and what is flagged when I submit. For instance, in iteration 1 of Two-Fer, I got a warning for no module docstring, but in iteration 1 of Gigasecond I did not. How would I determine what pylintrc file is actually used when I submit an exercise?

rzuckerm · May 6, 2024, 10:53pm

@BethanyG I tried putting pytest.ini and pylintrc in the python directory. The pytest.ini worked just fine, but pylintrc did not. I was getting errors for lints that were disabled in the pylintrc. After reviewing the pylint command line docs, I saw no mention of the parent directory.

IsaacG · May 7, 2024, 12:38am

Did you name it pylintrc or .pylintrc?

BethanyG · May 7, 2024, 12:53am

@rzuckerm - PyLint’s discovery rules can be found here within the “Command Line Options” section. I’ve pointed at the version our online Analyzer uses, although the docs are almost the same for the current 3.1.0 version.

So I have to apologize, because I forgot that in order to discover the pylintrc file in a common directory, that directory needs to be a Python Package directory. Doing that for the exercise tree (even tho __init__.py files can be empty) is fugly. So one of the methods below is better for this situation:

Specify which pylintrc file to use by passing a (full or relative) path to it on the command line each run. For our example exercise, that would be pylint --rcfile ../../../pylintrc arcade_game.py. So running that would look like (I need to clean up the warnings in the current pylintrc file):

(PY_3.11.5) pwd
/Users/__/exercism/python/exercises/concept/ghost-gobble-arcade-game

(PY_3.11.5) pylint --rcfile  ../../../pylintrc arcade_game.py #running the linting after passing the pylintrc location.
pylint: Command line or configuration file:1: UserWarning: 'StandardError' is not a proper value for the 'overgeneral-exceptions' option. Use fully qualified name (maybe 'builtins.StandardError' ?) instead. This will cease to be checked at runtime in 3.1.0.
pylint: Command line or configuration file:1: UserWarning: 'Exception' is not a proper value for the 'overgeneral-exceptions' option. Use fully qualified name (maybe 'builtins.Exception' ?) instead. This will cease to be checked at runtime in 3.1.0.
pylint: Command line or configuration file:1: UserWarning: 'BaseException' is not a proper value for the 'overgeneral-exceptions' option. Use fully qualified name (maybe 'builtins.BaseException' ?) instead. This will cease to be checked at runtime in 3.1.0.

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 0.00/10, +10.00)

A .pylintrc file could also be placed in your “home” directory, provided your “home” directory isn’t /root. This could either be under .config/pylintrc or .pylintrc. I tested out a .pylintrc (same content as the pylintrc file) in my home directory, and that was picked up fine from the exercise directory and the virtual env without issue.

The PYLINTRC environment variable can be set for your shell, and that will hold for the entire session. You can use a relative or full path. So using our exercise example in the bash shell:

(PY_3.11.5) pwd
/Users/__/exercism/python/exercises/concept/ghost-gobble-arcade-game

(PY_3.11.5) export PYLINTRC="../../../pylintrc"
(PY_3.11.5) echo $PYLINTRC  #verifying the variable has been set
../../../pylintrc

(PY_3.11.5) pylint arcade_game.py #running the linting
pylint: Command line or configuration file:1: UserWarning: 'StandardError' is not a proper value for the 'overgeneral-exceptions' option. Use fully qualified name (maybe 'builtins.StandardError' ?) instead. This will cease to be checked at runtime in 3.1.0.
pylint: Command line or configuration file:1: UserWarning: 'Exception' is not a proper value for the 'overgeneral-exceptions' option. Use fully qualified name (maybe 'builtins.Exception' ?) instead. This will cease to be checked at runtime in 3.1.0.
pylint: Command line or configuration file:1: UserWarning: 'BaseException' is not a proper value for the 'overgeneral-exceptions' option. Use fully qualified name (maybe 'builtins.BaseException' ?) instead. This will cease to be checked at runtime in 3.1.0.

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 0.00/10, +10.00)

Hope that helps. I’ll reply to the linting question in a second post.

BethanyG · May 7, 2024, 1:48am

Our online Analyzer was designed to have custom coaching and rules in place for different exercises which would not necessarily be available offline. The work on the feedback and rules is not yet complete, but some exercises (mostly the concept exercises + Two Fer) do have a few custom rules and config files in place. The remainder use the common pylintrc file for now.

As work progresses on the Analyzer, the online version and linting through PyLint will diverge, as the online version will be more focused on suggesting improvements based on things like mentor feedback and suggested approaches to the exercise rather than linting errors. Linting errors will still be included, but there will also be things like

“suggest you use a list-comprehension here instead of a loop that appends to a list”

So that is the divergence you are seeing (although its quite rudimentary). Since having analyzer feedback on the website doesn’t block your progress in any way, we felt that having online and offline not match wasn’t a huge deal. We may revisit that as we fill out Analyzer rules and comments.