docs(SPEC): add specification of modules

nathanielc · nathanielc · commit 840245ba24dc · 2022-11-02T17:01:10.000-06:00
The implementation of this work is still in-progress see #4296
diff --git a/docs/SPEC.md b/docs/SPEC.md
@@ -1051,9 +1051,10 @@ The _main_ package is special for a few reasons:
 
     ImportDeclaration = [ Attributes ] "import" [identifier] string_lit
 
+
 Associated with every package is a package name and an import path.
 The import statement takes a package's import path and brings all of the identifiers defined in that package into the current scope under a namespace.
-The import statment defines the namespace through which to access the imported identifiers.
+The import statement defines the namespace through which to access the imported identifiers.
 By default the identifier of this namespace is the package name unless otherwise specified.
 For example, given a variable `x` declared in package `foo`, importing `foo` and referencing `x` would look like this:
 
@@ -1426,6 +1427,176 @@ A query specification defines what data and operations to perform.
 The execution model reserves the right to perform those operations as efficiently as possible.
 The execution model may rewrite the query in anyway it sees fit while maintaining correctness.
 
+## Modules
+
+>NOTE: Modules are not fully implemented yet, follow https://github.com/influxdata/flux/issues/4296 for details.
+
+A module is a collection of packages that can be imported.
+A module has a module path, version and a collection of packages with their source code.
+
+### Module path
+
+The module path is the import path of the top level package within the module.
+Additionally major versions of a module of two or greater must add a final element to the path of the form `v#` where `#` is the major version number.
+For modules at version zero or one the path must not contain the major version as it is not necessary.
+A change from `v0` to `v1` may include a breaking change but once `v1` is published any future breaking changes will be a new major version.
+
+Example
+
+    foo/bar    // module path of foo/bar for version zero or one
+    foo/bar/v2 // module path of foo/bar for major version two
+
+### Module versions
+
+All modules are versioned using a [semantic version number](https://semver.org/) prefixed with a `v`, i.e. `vMAJOR.MINOR.PATCH`.
+Once a module version has been published it cannot be modified.
+A new version of the module must be published.
+
+### Module registry
+
+A modules is hosted and stored on a registry.
+A module path is unique to the the registry that hosts the module.
+Module paths need not be unique across different registries.
+
+#### Registry attribute
+
+The `registry` attribute defines the available registries.
+The registry attribute expects two arguments, the first is the name of the registry and the second is the `$base` URL of the registry API endpoint.
+See the [Registry API](#registry-api) for more details.
+
+The runtime may define default registries, a registry attribute will override any default registry.
+The standard library will never contain a top level package named `modules` or any name containing a `.`.
+This makes it possible for the runtime to use `modules` or any name containing a `.` (i.e. a DNS name) as a default registry name.
+
+Example:
+
+    @registry("modules", "http://localhost/modules")
+    @registry("example.com", "https://example.com/api/modules")
+    package main
+
+### Importing modules
+
+Flux modules are imported using an import declaration.
+The import path may contain specifiers about which registry and which versions of a module should be imported.
+
+An import path follows this grammar:
+
+    ImportPath       = ModulePath [ "/" PackagePath ] [ Version ] .
+    ModulePath       = PathElement [ "/" PathElement ] [ MajorVersion ] .
+    MajorVersion     = "/v" int_lit .
+    PackagePath      = PathElement { "/" PathElement } .
+    Version          = PreVersion | MinVersion
+    PreVersion       = "@pre"
+    MinVersion       = "@v" int_lit "." int_lit "." int_lit "." .
+    PathElement      = ascii_letter { ascii_letter } .
+    ascii_letter     = /* alpha numeric and underscore ASCII characters */
+
+Per the grammar a module path may have up to three path elements:
+
+* Registry name
+* Module name
+* Module major version
+
+A package path may have an arbitrary depth and must not begin with a major version.
+
+When resolving an import path the first path element of the module path is compared against the defined registry names.
+If a match is found the import path is understood to be relative to the registry.
+If no match is found the import path is understood to be an import from the standard library.
+It is an error to specify a version on imports from the standard library.
+The standard library version is implicit to the runtime.
+
+When no version information is provided the latest version of the module is used.
+A _minimum_ version may be specified.
+An import may also specify the version `pre` which is the most recent pre-release version of the module.
+
+
+Examples
+
+    @registry("modules", "http://localhost/modules")
+    package main
+
+    import "foo"                         // imports package `foo` from the standard library
+    import "modules/foo"                 // imports the latest version 0.x or 1.x version of the `foo` module from the `modules` registry
+    import "modules/foo/a/b/c"           // imports package a/b/c from the latest 0.x or 1.x version
+    import "modules/foo@v1.5.6"          // imports at least version 1.5.6
+    import "modules/foo/v2"              // imports the latest 2.x version
+    import "modules/foo/v2/a"            // imports package `a` from the latest 2.x version
+    import "modules/foo/v2/a@v2.3.0"     // imports package `a` from at least version 2.3.0 of the `foo` module
+    import "modules/foo/v2/a/b/c"        // imports package `a/b/c` from the latest 2.x version
+    import "modules/foo/v2/a/b/c@v2.3.0" // imports package `a/b/c` from at least version 2.3.0
+    import "modules/foo/a/b/c@pre"       // imports package `a/b/c` from the latest pre-release 0.x or 1.x version
+    import "modules/foo@pre"             // imports the latest pre-release 0.x or 1.x version
+    import "modules/foo/v2@pre"          // imports the latest pre-release 2.x version
+
+### Version resolution
+
+When multiple modules both depend on a specific version of another module the maximum version of the minimum versions is used.
+Major versions of a module are considered different modules (they have different module paths), therefore multiple major versions of a module may be imported into the same Flux script.
+
+When multiple import declarations exist for the same module at most one import declaration must specify version information.
+
+Example
+
+```
+// a.flux
+package a
+
+import "foo@v1.1.0"
+```
+
+```
+// b.flux
+package b
+
+import "foo@v1.2.0"
+```
+
+```
+// main.flux
+@registry("modules", "http://localhost/modules")
+package main
+
+import "modules/a"
+import "modules/b"
+```
+
+Package `main` depends on module `foo` via both of the modules `a` and `b`.
+However `a` and `b` specify different versions of `foo`.
+The possible versions of `foo` include `1.1.0` and `1.2.0`.
+Flux will pick the maximum version of these possible versions, so version `1.2.0` of `foo` is used.
+This is sound because module `a` has specified that it needs at least version `1.1.0` of `foo` and that constraint is satisfied.
+
+### Registry API
+
+Modules can be published and downloaded over an HTTP API from a registry.
+Modules are immutable, once a version is published it cannot be modified, a new version must be published instead.
+
+The HTTP API will have the routes listed in the following table where `$base` is the anchor point of the API, `$module` is a module path without the registry name, and `$version` is a semantic version of the form `vMAJOR.MINOR.PATCH`.
+
+| Method | Path                          | Description                                                                                                                                                |
+| ------ | ----                          | -----------                                                                                                                                                |
+| GET    | $base/$module/@v/list         | Returns a list of known versions of the given module in plain text, one per line.                                                                          |
+| GET    | $base/$module/@v/$version.zip | Returns a zip file of the contents of the module at a specific version.                                                                                    |
+| GET    | $base/$module/@latest         | Returns the highest released version, or if no released versions exist the highest pre-release version of the given module in plain text on a single line. |
+| GET    | $base/$module/@pre            | Returns the highest pre-released version of the given module in plain text on a single line.                                                               |
+| POST   | $base/$module/@v/$version     | Publish a new version of the module where the POST body contains multipart formdata for the contents of the module.                                        |
+
+As an example, to download the zip file for a module `foo` at version `v0.5.6`, for an API endpoint anchored at `https://example.com/flux/modules/` use this URL `https://example.com/flux/modules/foo/@v/v1.5.6.zip`.
+Or for the module `foo/v2` at version `v2.3.4` the URL is `https://example.com/flux/modules/foo/v2/@v/v2.3.4.zip`.
+
+Examples
+
+The following examples use a $base of `/flux/modules/`
+
+    GET /flux/modules/foo/@v/list          # Return a list of versions for the module foo
+    GET /flux/modules/bar/@v/v1.3.4.zip    # Return a zip file of the bar module at version 1.3.4
+    GET /flux/modules/bar/v2/@v/v2.3.4.zip # Return a zip file of the bar module at version 2.3.4
+    GET /flux/modules/bar/@latest          # Return the latest 0.x or 1.x version of bar
+    GET /flux/modules/bar/@pre             # Return the latest  0.x or 1.x pre-release version of bar
+    GET /flux/modules/bar/v2/@latest       # Return the latest 2.x release version of bar
+    GET /flux/modules/bar/v2/@pre          # Return the latest 2.x pre-release version of bar
+
+
 ## Versions and editions
 
 Flux follows a [semantic versioning scheme](https://semver.org/) such that breaking changes are clearly communicated as part of the version information.
