Parametrise modules

[nrc]

I would like to be able to parametrise modules by types and lifetimes. Type parameters are useful where many items in a module should use the same concrete type. E.g., taking some implementation as a parameter we want to ensure that all functions and data types in a module use the same implementor without annotating every item with the same type parameters. Likewise, parameterising by lifetimes is useful if we are to assume that many objects in a module have the same lifetime. This is especially useful in conjunction with arena allocation.

Details

Module declarations may have formal type and lifetime parameters and where clauses, e.g, mod foo<X, 'a> where X: Bar { ... }.

Module uses, including in use expressions which include an alias, can have actual type and lifetime parameters. E.g, let x: foo<int>::Baz = ...; or use foo<int, 'static> as int_foo.

The usual rules around well-formedness wrt bounds, and inference would apply.

[emberian]

I want this all the time when writing lots of generic code.

[eternaleye]

Massive agreement here.

[burdges]

I missed this issue and posted another version here https://internals.rust-lang.org/t/implicit-module-arguments/5022

In effect, I'm wondering there if modules could perhaps not use the usual Rust type parameter syntax but instead refer to items like types in the scope using them. These are parameters would be viewed as implicit instead of explicit. You read the scope to determine the module's parameters, not the call site.

The reason for making module parameters implicit would be that they are usually library configuration parameters. We want them out of the way so that code does not read as overly generic.

It might resemble :

mod ParamaterizedModule {
    implicit type 'some_lifetime;
    implicit type TypeParam : ParamaterTrait;
    implicit const ConstantParam : Type;
    ...
}

The bad part about this implict formulation is that afaik 'some_lifetime cannot exist at the module level, so you cannot just use this module except from inside an fn or impl.

[eternaleye]

@burdges: I think your proposal conflates things, in a way I find rather problematic.

1. Declaration syntax
   • There do exist languages which declare module parameters in the body - Coq, for one.
   • However, it's as I worded it - a language-wide thing. I feel this would be intensely jarring in Rust.
   • As you noted, it would require constructs that cannot be expressed in normal Rust.
   • implicit is not a keyword, IIRC, which has back-compat impacts.
2. Invocation syntax
   • If you don't pass them in at the "call site" (use), then what values do they get?
     • If they get inferred somehow, you now have cross-function inference, which is deeply problematic for many reasons (was discussed in-depth in the impl Trait RFC thread)
     • If they get defaulted, this is a duplicate syntax for defaulted parameters
       • If they get defaulted, how do you override the defaults? How are they parameters at all?
     • This is a non-issue if module-level parameters are just a shorthand for putting a parameter on every item, but it does rule out similar shorthand at the use site, which I would consider nice to have,
   • If you don't pass them in at the call site, how does someone reading code where they are used learn what the types involved are?
   • Is it possible to import a module twice, with different parameters (using renaming)?

In short, I think that proposal would not work out well at all, and strongly prefer:

• mod foo<'a, T: Trait, const x: usize = 3> { ... } - a declaration syntax that permits reusing all of the nice machinery we already have
• use foo<Vec<u8>> as foo_vec, an invocation syntax that is explicit in its behavior, and has a clear meaning to anyone familiar with Rust.

If module-level parameters are merely a shorthand for additional parameters on the items of the module, the latter can be left to later, and eventually defined as a similar shorthand for setting those parameters.

EDIT: Ah, I'd missed your "use whatever's fitting in-scope" bit. That's the least workable part of this, IMO - it's akin to Coq's curly-braced parameter declarations, which really only work because Coq is a proof assistant, and can provide incredibly detailed bounds on them. Without that, I suspect they'd be the next thing to useless, because an unbounded type parameter could be satisfied so many ways it's absurd.

[burdges]

As I posted in the internals thread, I think I agree with @eternaleye that implicit passing should be skipped. There are no soo many use lines that this will make anything awkward.

In any case, I do like the idea of type and constant parameters for modules. I mostly just wondered if this would be an opportunity to do something implicit really well.

[strega-nil]

Perhaps this paper would be of use? It brings parametrized modules to Haskell:

http://plv.mpi-sws.org/backpack/

[OvermindDL1]

Perhaps this paper would be of use? It brings parametrized modules to Haskell:

Or look at how OCaml does it (especially soon-coming implicit module support), which does first-class modules in a way that is perfectly efficient at run-time and compiles significantly faster than HKT's while supplying the entire power there-of (and far more, OCaml's HPT modules can do many things that HKT's cannot). An OCaml-like first-class module system would pretty well fix most (all?) of the remaining higher typing problems in rust.

[eternaleye]

@OvermindDL1: ML-style Functor-based modules have problems of their own; a significant motivation for Backpack was providing a similar level of power while avoiding those downsides. It's based on an earlier work by the same authors, called MixML, which entirely subsumed the features of ML-style modules.

[OvermindDL1]

ML-style Functor-based modules have problems of their own; a significant motivation for Backpack was providing a similar level of power while avoiding those downsides. It's based on an earlier work by the same modules, called "MixML," which entirely subsumed the features of ML-style modules.

OCaml's modules are not as limited as SML modules and although I've done a cursory glance over the MixML documentation I was not able to see what it gained (other than the examples being slower to compile). Over standard ML modules MixML seems to add recursive module definitions (which can be done in OCaml already) and mixins (already supported in OCaml), however OCaml supports a great deal that ML/SML do not support, including going far beyond MixML, those being that I can name off the top of my head:

• Recursive Modules, MixML supports these but not as higher ordered it seems...
• First class value modules, these are amazingly useful, and though a little verbose right now a soon-coming OCaml version adds Implicit Modules that will fix that with ease (and is a great feature to copy).
• Module type and module instance copying.
• Nesting signatures (though MixML handles this it seems).
• Higher Polymorphic Functors, which allow them to be composed into fascinating type hierarchies the likes of which go far beyond HKT's as well.

Among others that I cannot recall off hand. But overall yes MixML is higher than ML, but OCaml is much higher still, and OCaml's module design is well worth copying for 2 very very large reasons (not the syntax of course), those being compile-times, they were designed for being extremely easy and efficient to handle especially during optimization (very large OCaml projects with large amounts of modules still only take seconds to compile hundreds of files, unlike rust...), and the second being that they grew up based around necessity, it is not a Haskell'ish style research project, it is a real-world language that had its features added to solve problems, regardless of what they are, and consequently it is a very well tested style of which the only down-side is syntax verboseness that Implicit Modules solves soon anyway.

Also, I've not touched SML/ML itself in a decade, only OCaml for ML-style languages, so they might have some more developments as well...

[burdges]

If you're module is a file, then I suppose the syntax might be mod<...> where ...; or mod self<...> where ...; once early in the file.

[glaebhoerl]

I just want to note that "ML-style modules" is a much bigger feature than the "parameterizable modules" that are actually being proposed here. In particular the crucial part of "ML modules" is functors, that is, modules parameterized over modules, which (together with the ability to specify module signatures, by analogy to type signatures) allows one to express various kinds of ad-hoc polymorphism*. Allowing parameterization over modules together with module signatures would be a huge addition to Rust, especially since Rust already has its own solution for ad-hoc polymorphism (traits, a.k.a. type classes).

What is being proposed is merely being able to parameterize modules over types and lifetimes, which is a considerably smaller addition, and really only goes toward "abstracting the same things, just with less typing" rather than "new and more powerful abstractions". (Although abstracting a module over a type with a trait bound, mod foo<T: Bar> { ... }, ends up being roughly similar to an ML functor.)

* ("Polymorphism" is maybe not exactly the right word here given that functoring is explicit, but it's used for the same things.)

[comex]

@glaebhoerl I'd put it a different way… Rust already supports parameterized modules and module signatures; they're called impls and traits (with singleton types). If more expressivity or sugar would be needed to make them as useful as in ML, perhaps it should be added, but it should be an extension of the existing system, not its own separate thing.

[burdges]

I'd think modules parameterized over constants, types with trait bounds, and lifetimes provide an ergonomic and productivity win. There are frequently parameters you do not want to focus on in a first pass, so modules parameters give a "canonical easiest" way to export to them to the caller.

You can achieve similar functionality with a associated types and constants in a trait for singletons or uninhabited types, but now you're invested in deciding what goes into this trait and how it should parameterize every item in your module. I suppose const parameters may help eliminate that trait, but you'd want inherent impls to contain traits, structs, enums, and type aliases. They cannot right now.

It'd be lovely if mod M<..> where .. { ... } were functionally equivalent to

enum M<..> where .. {}
impl M<..>  where .. { ... }

In fact, if file modules supported parameters then conceivably inherent impls could be placed into files with some syntax like impl Foo<..> where .. mod foo<..>;

As an aside, there are nice translations in ML Modules and Haskell Type Classes: A Constructive Comparison including some explosion in complexity in both directions. In my reading, there are new "interesting" ways to obtain features from ML-style modules, but Rust already has most like privacy, namespace management, etc. via pub, use, etc.