Decrease syntax cost of usage

[jrudolph]

Hey, thanks for sharing this example here (and, of course, the original paper). I somehow missed it back at the time but recently found it. It seems somewhat similar in spirit to my speed project from 2014 built on Scala 2 macros (with a custom enhanced reify implementation), but without staging.

The staging approach is nice because it very explicitly moves the stream creation to compile time. I wonder, though, if this would be implemented for real there are some issues with using it:

• user needs some understanding of staging
• need two defs per stream (inline def + actual definition)
• multiple file restriction for inline def usage
• need to pass expressions for integer literals
• lambdas require explicit Expr => Expr form ("manual beta reduction")

etc.

I wonder what would be needed to make this more convenient to use. The approach used in speed was to pattern match on a given stream expression to extract the structure of a stream. The staging approach seems nicer but introduces some ceremony. Is there a middle ground? Also how far could you get with just inlining and powerful constant propagation?

/cc @nicolasstucki

[moatom]

@jrudolph
Hi! Thanks for your interesting argument. (Sorry for the delayed reply 🙏)

Here’s my personal and rough response (I would especially like to hear your opinion on 4):

1

• The user needs some understanding of staging
• Needs to pass expressions for integer literals

We prepared an int function for lifting integer literals: https://github.com/strymonas/strymonas-scala/blob/master/src/main/scala/strymonas/CdeSpec.scala#L25

I believe this is no longer a staging issue but rather a simple matter of type handling. (You can even leave it to the compiler through the given conversion.)

2

• Lambdas require an explicit Expr => Expr form ("manual beta reduction")

If the library (EDSL) provides sufficient streaming combinators (including https://github.com/strymonas/strymonas-scala/blob/master/src/main/scala/strymonas/CdeSpec.scala), would users still need to handle this?

All combinators in the streaming EDSL must follow this form, but defining them would primarily be the responsibility of library developers.

In my understanding, this wouldn't be a major issue...

(That said, it is true that the current Cooked/Raw interface sometimes requires additional ceremony for complex pipelines in the Scala 3 implementation:

https://github.com/strymonas/strymonas-scala/blob/master/bench/src/main/scala/benchmarks/TestPipelines.scala#L118-L130)

3

• Need two definitions per stream (inline def + actual definition)

In the first place, users wouldn’t even want to call inline def manually; instead, they would prefer to directly write the stream pipeline in place.

Thus, for example, an expression like:

​ xs.map(...).reduce(...)

should automatically be replaced with:

​ macroCalc(...)

where:

• inline def macroCalc(...): S = ${ macroCalcImpl(...) }
• def macroCalcImpl(...)(using Quotes): Expr[S] = xs.map(...).reduce(...)
• S is a meta expression for types

It seems that we need to hook into reduce to automatically generate macroCalcImpl and macroCalc for this purpose.

Does Scala 3 provide any feature for this?

4

• Multiple file restriction for inline def usage

I’m afraid I’m not entirely sure what you mean by this, but if you are referring to:

• The callee of inline def needing to be in a different file from its caller:
  • What exactly is the issue? Is it just awkward, or does it cause other problems?
• The general difficulty of inlining across multiple files:
  • Perhaps -Yretain-trees could help?

Anyway, do you have a concrete example?

5

Also, how far could you get with just inlining and powerful constant propagation?

Are you referring to performance?

We have a paper with a macro benchmark: https://dl.acm.org/doi/pdf/10.1145/3635800.3636962 (The results are shown in the figures on p. 65 and p. 67.)

[moatom]

If the library (EDSL) provides sufficient streaming combinators (including https://github.com/strymonas/strymonas-scala/blob/master/src/main/scala/strymonas/CdeSpec.scala), would users still need to handle this?

The issue is that this handles only primitive types and not user-defined classes now...

[moatom]

I’m afraid I’m not entirely sure what you mean by this, but if you are referring to:
- The callee of inline def needing to be in a different file from its caller:

Here, I was referring to the following compiler error:

// In a single file
object Main extends App {
  println(A.computeSomething()) // caller
}

import scala.quoted.*

object A {
  def add(x: Expr[Int], y: Expr[Int])(using Quotes): Expr[Int] = {
    '{ $x + $y }
  }

  def computeSomethingImpl(using Quotes): Expr[Int] = {
    val x = Expr(1)
    val y = Expr(2)

    add(x, y)
  }

  inline def computeSomething(): Int = ${ computeSomethingImpl } // callee
}

[error] 6 |  println(A.computeSomething()) // caller
[error]   |          ^^^^^^^^^^^^^^^^^^^^
[error]   |Cannot call macro method computeSomethingImpl defined in the same source file
[error] one error found