Update user guide notebook

[liamhuber]

• Add content for calling recipe objects
• Edits through section 3.3
• Keep editing from section 3.4 onwards
• Add content for parsing recipes as workflow steps
• Add content for live data formats
• Add content for the toy WfMS
• Add an aside on flow control parsers injecting a workflow
• Re-execute and debug any problems

With the readme quickstart providing a very brief, top-down, @workflow-onwards summary, this user guide is a bottom-up description of the principles, features, and intent. Ideas on how to further restructure the documentation are welcome, but out-of-scope for this particular PR. Here I wonder, is the information clear? Presented in a helpful order? Complete enough to inspire confidence in using the package?

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[github-actions]

👈 Launch a binder notebook on branch pyiron/flowrep/user-guide

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (65c8b8c) to head (df9ecba).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #215   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           30        30           
  Lines         2009      2009           
=========================================
  Hits          2009      2009           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[liamhuber]

Closes #209

[samwaseda]

I didn't realize until this notebook that the for loop had completely changed. Now it supports only for-each?

[liamhuber]

I didn't realize until this notebook that the for loop had completely changed. Now it supports only for-each?

This has always been the case in pyiron_workflow, and is consistent with all the examples in #23. I checked out v0.1.10 and looked at the workflow tests -- I'm only skimming, but I don't notice any examples of for that reference their own data from past steps.

AFAIK "For" has always been the parallel, mapping sort of "for"; we leave the serial depends on previous steps sort of loop for "while". If we explicitly rename things from For to ForEach per #217 we would have semantic space for that to be the parallel "for" and introduce a new serial "for". The tradeoff there is whether the convenience is worth introducing a whole new recipe type (since you currently can accomplish serial loops with the while syntax).

Is there a concrete example of something you wished would run that currently doesn't?

[samwaseda]

This has always been the case in pyiron_workflow, and is consistent with all the examples in #23. I checked out v0.1.10 and looked at the workflow tests -- I'm only skimming, but I don't notice any examples of for that reference their own data from past steps.

ok we are actually not talking about the same thing, because I just checked the following workflow and it works:

def workflow_with_for(a=10, b=20):
    x = add(a, b)
    results = []
    items = my_for_loop(x, b)
    for ii in items:
        y = add(a, ii)
        z = multiply(a, y)
        results.append(z)
    return results

For the rest, let me think about it a bit. The one thing that disturbs me a lot right now is the fact that it became very unclear what we promise, compared to the previous "only function assignments".

[liamhuber]

ok we are actually not talking about the same thing, because I just checked the following workflow and it works:

Super. Yes, that looks like a perfectly reasonable workflow to me.

For the rest, let me think about it a bit. The one thing that disturbs me a lot right now is the fact that it became very unclear what we promise, compared to the previous "only function assignments".

👍
"Only function assignments" will still work, but now there is more.

The for-loops are probably the most complicated thing because you need to explicitly declare an accumulator variable and append to it. This is a bit of leg work, but at least it is clear and unambiguous to see what's happening when you read the function definition. The previous parser had the problem that the way for-loops were parsed and intended to be run was misaligned with how the function would actually execute when you simply called it. Now the recipe you get from parsing a for-loop lines up with the actual result from running it (i.e. both return list(s) of data).

In the long run, I think there is tension

1. Parse very flexibly so people can write what they're used to (cf. Parse in-line if statements #138)
2. Have clear and easy to understand restrictions on the parseble syntax
3. Don't have too much code to maintain

Where (1) is in conflict with (2) and (3) each independently.