Pending TODOs for Template Automation

[BethanyG]

Just parking these here (and add your own, plz).
A TODO list for templates, template scripts, and other test file & template-related things.

• Macro-tize or script filter the additional_tests.json portion. Having to manually read additional_cases into templates is a PITA.
• Macro-tize error checking syntax see the commented out Python macro here
• Find an auto-formatter for JinJa files, so that they are clean, and we all agree on how they look
• Find a linter (long shot here) for JinJa files, so we are always clear on how to do things.
• Consider adding other helpful info to the test header file. Do we care what version of R these were done against? What about the version of Tidyverse? Other things we could add or annotate to help with all of this?
• Consider adding a master template that would cover 80-90 percent of the cases.
• Consider adding documentation for new contributors to make this process easier. Python docs here. Most likely needs updating/massaging.
• Consider adding test regeneration to CI (but only if it doesn't take forever).
• Better debug actions when rendering goes bad
• Some additional (potential) filters or macros for processing errors, highly nested data, and other JSON --> R data structure conversions.

[colinleach]

I'm on a 30-day trial of the Jinja2 Enhanced Support plugin on PyCharm. Pretty underwhelming so far, so I'm not rushing to commit $30/year. VSCode plugins all seem more interested in syntax highlighting. I haven't yet searched for a stand-alone formatter.

We tend to be relaxed about R versions. The base language is obsessive about backward compatibility.

Tidyverse may be more of an issue. I was really narked when dplyr broke the switch concept 3 days after I wrote it.

From what I've seen so far, test regeneration seems really fast. We'll get a better idea when we (a) have more templates and (b) try running it on GH.

[BethanyG]

I'm on a 30-day trial of the Jinja2 Enhanced Support plugin on PyCharm.

Yeah....I don't put much stock in PyCharm extensions. especially paid ones. There are a few, but ... generally not the ones JetBrains puts out. But YMMV. There might also be something out there in the OSS-verse that can be used as an external tool. I have yet to go down the rabbit hole.

[BethanyG]

Tidyverse may be more of an issue. I was really narked when dplyr broke the switch concept 3 days after I wrote it.

OMG. That. is. BS. Dang. Sorry that happened to you.

[BethanyG]

Ok. This is not a good sign. Guess I'll need to dig more there. The do have an online playground and I don't hate what it does. But there are grave concerns since there is no maintainer for the project.

this(jinja plugin for prettier) and this (formatter for jinja) look more promising..... Simon Wilson has a blog post about using prettier on jinja/Django. I hate to add JS as yet another dependency, tho.

And another.

But they are all focused on the HTML part. Not the general use-case. sigh. Will update as I find more.

[colinleach]

While adding bottle-song as a new, templated exercise, I found the template quite hard to debug. When there were syntax errors in the generated R script, it just returned an error code with no detail. Quite tricky, when I couldn't see the code it was complaining about.

How easy would it be to add a debug mode to generate_tests.py, to dump the raw R from the renderer while bypassing Air?

Today was pretty much worst case, as I was getting lost in deeply-nested JSON while distracted by a persistent headache. But still...

[BethanyG]

When the render fails (and it will fail if Air or Black or whichever formatter we use chokes), the script will print out the error stack trace, including the file that it failed to render/format. If the trace is big, it can be easy to miss the file path. Here's a screengrab of mine, yours could be a different path due to linix or Windows temp directory differences:

So when this happens, I often use vi to open the file (because I am lazy and its fast) - but it can easily be opened in an editor of your choice.

However, we could modify the script to open the file in vi automatically. It would be much much noisier, but we could do that if it feels like it would help. I might also be able to configure it to open in your editor - but that might take a bit of (longer) work.

Unless this is not happening in your case? Could you give me the template/code in question, so I can troubleshoot? Many thanks!

[colinleach]

the script will print out the error stack trace, including the file that it failed to render/format.

Aaah, so I missed the link to a file I could examine. I was missing a lot that day.

I withdraw my feature request.

Could you give me the template/code in question

Long gone, and maybe best we draw a veil over it. We're talking about syntax errors at the level of missing commas: really basic stuff, but potentially hard to spot in the clutter of Jinja2.

[BethanyG]

Aaah, so I missed the link to a file I could examine. I was missing a lot that day.

If it makes you feel better, when I was first working on updates in the Python repo, I managed to miss this as well. Wow did templating go faster when I figured it out! So your feature request is worth saving: I should take a look at how the error is reported and see if we can make the file path more obvious.

...and a note for pending documentation to also point it out. 😄

[colinleach]

One small gotcha I hit in Flatten Array is in the list_to_tuple filter:

    if len(case_array) == 1:
        return f"('{case_array[0]}')"

    return tuple(case_array)

The single quotes in the length-1 return value aren't ideal for integer vectors like (in this case) c(3). I worked around it with:

expected <- c{{ case["expected"] | list_to_tuple | replace("'", "") }}

No disaster, but we could probably find a cleaner solution at some point.

[BethanyG]

No disaster, but we could probably find a cleaner solution at some point.

I am still stuck on the Python repo's single quote rule. It's an easy fix to switch to double quotes for the R script.

[colinleach]

But around an integer we don't need quotes at all? I'm getting confused (easily done!).

[colinleach]

switch to double quotes for the R script

R doesn't care about single/double around strings (like Python, very unlike Julia), though the styler will probably push for double these days.

[BethanyG]

But around an integer we don't need quotes at all? I'm getting confused (easily done!).

Now I am confused. You did a quote replacement in the template, so I assumed it was the quotes that were the problem. Are you saying I need to check to see if the item extracted is an int or float and pass it through if so? The whole thing gets passed to JinJa as a string, so ....

[colinleach]

We could just use a different filter, to keep it simple:

def list_to_int_tuple(case_array):

    if len(case_array) == 1:
        return f"({case_array[0]})"

    return tuple(case_array)

Plus whatever type hints you recommend.

We're aiming for c(3), not c('3'). Both valid R, but different. And the tests will complain that 3 != '3'.

[BethanyG]

I'd rather not clutter things up with another filter that's for a sort of edge case (and we have to account for floats as well). I can easily do an isinstance within the existing function. Do you have a template/exercise I can test this on?NE

Nevermind. I'll use Flatten-Array 😄