Proposal: switch to a real module system for better modular designs

[kaosat-dev]

Curent system

Hi everyone ! right now we are using a custom and rather limited include system to enable modular design & part reuse, with a few major flaws:

pros

• simple
• custom

cons

• reinventing the wheel: all the issues we face have already been solved elsewhere, better
• global pollution : all variables declared in ANY include() are pushed to the global scope
• unclear syntax for export & import : ie what is exported ?? what if we have mutlple functions with the same name in different modules etc (hint, not possible)

const someValue = 42
const someText = 'foo'
coolPart = function() { 
   return cube({size:someValue}).union(text(someText)
}

• non idiomatic : the syntax teaches bad practices
• no relative path resolution
• greedy evaluation : any code in an imported module is immediatly evaluated & run (careful what you import!)
• not integrated with the rest of the javascript ecosystem

Proposal

The node.js & js ecosystem have been very successfully using modules for years, let's use that tech and knowledge to our advantage :

Note: this applies to both common.js & es6 modules as well

pros

(in addition to all the cons of the current system, which are all resolved in commonjs/es6 modules)

• clear , explicit and proven syntax :
  export: (coolPart.jscad)

const someValue = 42
const someText = 'foo' // this one is not exported, yet still available at the module level

function coolPart(){
  return cube({size:someValue}).union(text(someText)
}
module.exports = { //these are exported and available from any consumer
  someValue,
  coolPart
}

import

const {coolPart} = require('./coolPart.jscad') //we can selectively import features, etc

• all the existing javascript packages & tools at our disposal
• simpler to distribute package and use them (distribution with current syntax is not problematic, but use is)
• OpenJSCAD is javascript, we should embrace it , instead of half heartedly hiding it :)

cons

• higher complexity in browser (but less in node.js), there are many open source tools to code with modules in the browsers, they can be self hosted , but is more complex.

note: the commonjs require function / module loading is very simple and could also be emulated for a limited browser use

• possible dependency issues (as in , too much)

Of course this could be rolled out progressively, tested & tried, and would not need to impact the current implementation of include.

As you can likely see, I am quite passionate about the subject .
Any thoughts ?

[z3dev]

Actually, I think that the concept of include() came from OPENSCAD so possibly this compatibility can be maintained via the JSCAD library.

[kaosat-dev]

@z3dev true, but not sure if it can /should behave the same way : unlike Openscad we have a full programming language at our disposal :)

[gfwilliams]

Just my 2p after submitting the metaball PR above...

This looks great (esp the common.js format for defining modules), but IMO simplicity (and it working easily on https://openjscad.org) is key.

The majority of your users won't be Node.js devs, so asking them to submit to NPM will probably cut down the amount of submissions you get a lot. The great bonus of openjscad.org is it works in the browser, so asking for a full Node.js install just to submit code seems like a step backwards.

While I mentioned require, chances are it wouldn't be that standards compliant with NPM and might just annoy people. include(url) might be preferable from that point of view.

Also, while I said about common.js above, having the ability to find someone's jscad object online and just include it into your design would be amazing. Perhaps the loader could check for the existence of the main function or exports fields and decide how to interpret file file based on that (or maybe just the file extension?).

[Spiritdude]

Sorry for the late reply, I introduced include() as I wanted to keep things simple. In general OpenJsCad (the original) was OO-based and I wanted to add functions and UI to make it easier to use, openscad.js provided OpenSCAD-like JS functions, where include() was added in the same spirit.

Now that things we modular, it surely makes sense to adapt common ways, but keep include() as well. OpenJSCAD.org I did to be a hybrid: simple OpenSCAD-like, aside of the more elaborate OO approach which JS experts will use.

[gfwilliams]

Sorry, not sure how I missed that it's already there: https://en.wikibooks.org/wiki/OpenJSCAD_User_Guide#Including_Files

Looking at the docs, include via URL isn't possible yet... Would it be possible to add it? It'd be amazingly handy for referencing a file on GitHub, Gist or similar. Obviously cross-origin could be a pain, but I think GitHub at least is ok from that point of view?

[kaosat-dev]

@gfwilliams
while I agree that simplicity is great and should be at the forefront , I don't thing it needs to be incompatible with a sane way to define & (re)use code:

• just 'publishing' a javascript file with a package.json to github seems within reason (no need for npm for 'basic' use)
• include() is reinventing the wheel and doing it really badly (I spent enough time trying to improve things in there for the past few months to have nightmares)
• the semantics of include and mostly what and how you export it are horrible (globals !!) and frankly should be discouraged: even if people are new to Javascript when they start using jscad, we should not encourage broken design
• security: loading remote scripts without a sandbox, and using our current eval-based approach is a big nono
• what about the users that want both ? ie use it in the browser and in node (i am one of them , and not the only one :)
• I think we can have a way to do both, without resorting to bad practices that will bite us down the line

Side notes:

• this example shows that even if it really is a bit more complex, you can have stuff use node modules and also run in the browser & be modular https://esnextb.in/?gist=0a2ac2c4e189e27692ea964956a3a2e5
• most of the interesting jscad designs I found where both on npm & github
• I would argue that if you use code to create designs, you are at not allergic to tech :)

[kaosat-dev]

@gfwilliams while adding loading from an URL is possible to add it relatively simply I really don't think it should be done : simplicity is good

• but at the very worst we could have a node.js like require() running in the browser safely without having to resort to the include() hacks
• it would work with the same semantics as common.js modules, so compatible with node.js for those who want it
• I actually implemented something like that a while back , so I could do it again

[kaosat-dev]

hi @Spiritdude ! :) I am ok with keeping include as it is, but building upon its very fragile foundations seem like a bad idea for all the reasons above.
side note , we only need functions, not OO haha (I have come to embrace functional like programming in JS, and man does it rock)

[kaosat-dev]

btw I am also willing to provide a proof of concept for 'require-lite' functionality for demo purposes :)

[meta-meta]

I'm still VERY new to OpenJSCAD so I don't know the historic reasons for include() being the way it is and I'm not sure what the ultimate goals are for this project but I'm very interested in seeing a thriving open library of parametric CAD objects. Definitely opt for modules if possible. Having functions defined globally sounds totally insane to me as a front-end webapp developer with a functional programming bent. Coming in fresh, the OO-like function() main() { ... } had me wondering what magic parser was picking it up and what else was going on behind the scenes. It starts to feel like OpenJSCAD is a language extending javascript.

The wiki says

An OpenJSCAD script must have at least one function defined...

In es6 for say cube.js: export default () => { ... } gives you just that.
Then in some other module, import cube from './cube'

Essentially, I'll echo the "don't reinvent the wheel" arguments in this thread.

[meta-meta]

Check out https://aframe.io/aframe-registry/ for inspiration regarding a repo of community modules.