Python snippet extractor drops part of `yield from it`

The Python representer snippet extractor drops the from it part of yield from it: example (search for MatthijsBlom).

Posting here because the relevant repo does not welcome tickets. (Is this intended? It has very few issues.)

You mean the snippet extractor and not the representer?

I am not the maintainer for python and I am thereby not the right person to talk to. But here is the code which causes it: snippet-extractor/python.txt at main · exercism/snippet-extractor (github.com)

And the repo exercism/snippet-extractor (github.com).

I guess so.

@MatthijsBlom - Can you paste the code as you wrote it into this thread? It’s very very difficult for me to troubleshoot a representation if I don’t have the original code. As @Meatball has said the code that displays in the Community Solutions UI uses the generic snippet extractor with a python.txt file and not the Python representer. And since the representer uses the same module Python itself uses for creating ASTs, it is less likely that it is a Python representer bug (I am not saying it isn’t – just that its much less likely).

And yes - the tooling repos for Python are under the same contribution freeze that the main content repo is. Ping me in Slack or in a DM if you’d like to talk about it.

(edited to add) On second glance at the snippet extractor, I see that from appears in the Python format file. I am not remembering the rules for the extractor well enough to say if that is the culprit in your situation – but that’s where I’d start poking if I were troubleshooting what looked to be an “off” extraction.

-Bethany

@BethanyG Sorry, here is the code and a screenshot of the rendering

Code
from typing import Iterable


def flatten(iterable):
    def go(it):
        for e in it:
            if isinstance(e, Iterable) and not isinstance(e, str):
                yield from go(e)
            else:
                yield e

    return [e for e in go(iterable) if e is not None]
Rendering

image

The import is removed, as it should be, but the yield from is also cut short.

(It’s not the browser – there’s nothing in the HTML there.)

Yes, this is a snippet-extractor problem rather than a representer problem. I did not know the former existed separately. I’m looking into fixing it now.

I failed to figure it out. I tried

- from
+ from-->>import

as suggested by Example 2, but the tests fail and the report contains text (the encoding stuff) that I have no idea of where it comes from:

Edited tests report
  1) Failure:
SnippetExtractor::Languages::PythonTest#test_full_example [/.../snippet-extractor/test/languages/python_test.rb:31]:
--- expected
+++ actual
@@ -1,5 +1,3 @@
-"WORDS = re.compile(\"[a-z0-9]+(['][a-z]+)?\")
-
-def count_words(text):
-    return Counter(word.group(0) for word in WORDS.finditer(text.lower()))
-"
+# encoding: US-ASCII
+#    valid: true
+""


  2) Failure:
SnippetExtractor::Languages::PythonTest#test_full_example_extended [/.../snippet-extractor/test/languages/python_test.rb:69]:
--- expected
+++ actual
@@ -1,5 +1,3 @@
-"WORDS = re.compile(\"[a-z0-9]+(['][a-z]+)?\")
-
-def count_words(text):
-    return Counter(word.group(0) for word in WORDS.finditer(text.lower()))
-"
+# encoding: US-ASCII
+#    valid: true
+""

I don’t know Ruby.

@MatthijsBlom -

Apologies - this dropped off my radar. I will play with it later today, and see if I can get it to co-operate. One quick suggestion: try import-->>from. I don’t think that will work, but that is how you would import a specific thing in Python, and might “tie” from to import so that it’s ignored in other contexts.

The other way around, no? Failed attempt at proof by syntax highlighting:

from module import names
import names from module

It might be cheaper to let someone more familiar with the snippet extractor look at this.

Yup. Total brain fart there on my part! :laughing: Too many other languages in my head will be my excuse, but really it is probably early-onset senility. :wink:

And agree: time to call in someone who knows Ruby. @iHiD / @SleeplessByte - any insight here?

Hello!

What do you need help with?

The Python snippet extractor contains a pattern from to filter out from module import names statements. However, it also mutilates yield from iterable expressions. I tried to narrow down that from pattern to only match import statements, but failed.

Can you link me the code? I may be able to apply some tricks we used for TS/JS.

Reminder that this is GitHub - exercism/snippet-extractor and this is the Python pattern: https://github.com/exercism/snippet-extractor/blob/main/lib/languages/python.txt

1 Like

Ty! I’ll see what I can do

And ping me when it has been merged so I can re-run on existing solutions.