From ac9c18f8b8c7b9ee4688ad0b1889b1c2956173b5 Mon Sep 17 00:00:00 2001 From: Jan Nieuwenhuizen Date: Mon, 4 Nov 2019 19:59:13 +0100 Subject: [PATCH] doc: Release update. * BLURB: Release update. * HACKING: Remove roadmap. * INSTALL: Release update. * NEWS: Add 0.21 section. * README: Release update. * ROADMAP: New file. * configure.sh: Release update. * doc/announce/ANNOUNCE-0.21: New file. * doc/mes.texi: Release update. --- BLURB | 13 ++-- BOOTSTRAP | 22 +++---- HACKING | 56 ---------------- INSTALL | 51 +++++++++----- NEWS | 19 +++++- README | 45 ++++++------- ROADMAP | 91 +++++++++++++++++++++++++ build-aux/install.sh.in | 2 +- configure.sh | 6 +- doc/announce/ANNOUNCE-0.20 | 2 +- doc/announce/ANNOUNCE-0.21 | 132 +++++++++++++++++++++++++++++++++++++ doc/mes.texi | 132 ++++++++++++++++++++----------------- mes/module/mes/repl.mes | 1 + 13 files changed, 395 insertions(+), 177 deletions(-) create mode 100644 ROADMAP create mode 100644 doc/announce/ANNOUNCE-0.21 diff --git a/BLURB b/BLURB index edf350dc..7c394f20 100644 --- a/BLURB +++ b/BLURB @@ -1,4 +1,7 @@ +GNU Mes is a Scheme interpreter and C compiler for bootstrapping the GNU +system. + GNU Mes was created to address the security concerns that arise from bootstrapping an operating system using binary blobs, which is common practice for all software distributions. Mes is a Scheme interpreter @@ -8,13 +11,13 @@ halved the size of opaque, uninspectable binaries that were needed to bootstrap GNU Guix, a functional GNU/Linux distribution that focusses on user freedom, reproducibility and security. That reduction was achieved by replacing GNU Binutils, GNU GCC and the GNU C Library with Mes. The -final goal is to help create a full source bootstrap for any interested -UNIX-like operating system. After three years of volunteer work funding -will enable us to make another big step forward and reach an important -new milestone. +final goal is to help create a full source bootstrap for UNIX-like +operating systems. After three years of volunteer work funding will +enable us to make another big step forward and reach an important new +milestone. GNU Mes - https://gnu.org/software/mes GNU Guix - https://gnu.org/software/guix Bootstrappable Builds - https://bootstrappable.org Reproducible Builds - https://reproducible-builds.org -Reduced Binary Seed bootstrap - http://joyofsource.com/reduced-binary-seed-bootstrap.html +Reduced Binary Seed bootstrap - https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/ diff --git a/BOOTSTRAP b/BOOTSTRAP index 82e3c422..c8b83cdd 100644 --- a/BOOTSTRAP +++ b/BOOTSTRAP @@ -1,22 +1,22 @@ -*- org -*- #+TITLE: bootstrappable.org project -Copyright © 2016, 2017, 2018 Jan (janneke) Nieuwenhuizen +Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen Copying and distribution of this file, with or without modification, are permitted in any medium without royalty provided the copyright notice and this notice are preserved. * What? -** Full source bootstrapping for GuixSD +** Full source bootstrapping for GNU Guix -A package in GuixSD is can be uniquely identified by the hash of its -source code, its dependencies and its build recipe. +A package in Guix is uniquely identified by the hash of its source code, +its dependencies, and its build recipe. Every package can be built from source, except for the bootstrap binaries. -*** From the GuixSD manual +*** From the Guix manual The distribution is fully “bootstrapped” and “self-contained”: each package is built based solely on other packages in the distribution. @@ -25,7 +25,7 @@ The root of this dependency graph is a small set of “bootstrap binaries”, provided by the ‘(gnu packages bootstrap)’ module. For more information on bootstrapping, *note Bootstrapping::. -**** GuixSD bootstrap tarballs +**** Guix bootstrap tarballs $ du -schx $(readlink $(guix build bootstrap-tarballs)/*) 2.1M /gnu/store/9623n4bq6iq5c8cwwdq99qb7d0xj93ym-binutils-static-stripped-tarball-2.28.1/binutils-static-stripped-2.28.1-x86_64-linux.tar.xz @@ -66,7 +66,7 @@ about the bootstrap binaries and compilers? ** Software: MesCC-tools, M2-Planet, GNU Mes and MesCC ** MesCC-tools - https://github.com/oriansj/mescc-tools + https://savannah.gnu.org/projects/mescc-tools *** hex.0: ~500 byte well-documented, self-hosting hex assembler @@ -101,13 +101,13 @@ be source. * TODO ** remove or upstream patches from tcc-boot ** prepare src/mes.c for M2-Planet transpiler. -** arm, the Hurd +** ARM, the Hurd ** fix bootstrap-loops: (Nyacc?, mes.M2, psyntax.pp?) ** make GNU gcc (8.0?) bootstrappable again, remove [need for] tcc stage * DONE -** upstream mes-boot to GuixSD. -** replace GuixSD bootstrap for x86_64. -** replace GuixSD bootstrap for x86. +** upstream mes-boot to Guix. +** replace Guix bootstrap for x86_64. +** replace Guix bootstrap for x86. ** add full source gcc-4.7 package build. ** bootstrap gcc+glibc-built binutils-20.1, gcc-4.1.0. ** have tcc-boot compile gnutools triplet: binutils-2.14, gcc-2.95.3, glibc-2.2.5. diff --git a/HACKING b/HACKING index 81525878..c4096c6b 100644 --- a/HACKING +++ b/HACKING @@ -60,62 +60,6 @@ we have no guile (GUILE)...but we should have hex2, M1, and mes.M1. That's a bootstrap problem which is currently ignored by using the mes-seed package. mes.M1 will be produced by M2-Planet from mes.c. - -* ROADMAP -** TODO -*** release 0.x, unsorted - - mes: prepare src/mes.c for M2-Planet transpiler, - Jeremiah branched-out from mes; see https://github.com/oriansj/mes-m2. - - mes/mescc: proper docstrings, api reference documentation. - - replace bootstrap-binaries with Gash: bash, coreutils, grep, gzip, - sed, tar. - - mes: real module support, Guile compatible (define-module, - define-public, export). - - mescc: ARMv7/AArch64 support. -*** after release 1.0 - - replace initial gcc-2.95.3 with gcc-3.x or 4.x - - use 3rd party libc (ucLibc, dietlibc, ...) after Mes and reduce - need for bootstrappably-rich Mes C Library? - - mes/mescc: bootstrap a `bootstrap-Guile' before bootstrapping tcc? - - tcc: remove or upstream patches from tcc-boot. - - tcc: build 0.9.27 directly instead of via 0.9.26, see tinycc - wip-bootstrappable@0.9.27 branch - - mes/mescc: bootstrap a minimal-Guile? - + libguile/{eval,init,list,strings,values,..}. - + ice-9/eval.scm - - mescc: have mes-tcc pass all scaffold/tests, scaffold/tinycc tests. - - syntax-case bootstrap problem - + resolve portable syntax-case bootstrap, or - + get full source syntax-case up (Andre van Tonder?) - https://srfi.schemers.org/srfi-72/srfi-72.html, or - + ... drop it? - - mescc: the Hurd support. -** DONE -*** 0.19 GNU mes now compiles TinyCC in ~8min and supports building Bash and GNU Tar. -*** 0.18 GNU mes now supports GuixSD bootstrap (x86,x86_64) and has native x86_64 support. -*** 0.17.1 GNU Mes now allows removing glibc, binutils and gcc from the GuixSD bootstrap. -*** 0.17 GNU Mes is now an official GNU project and bootstraps gcc-4.7.4. -*** 0.16.1 Mes now has info docs and installs ootb on Debian buster/testing. -*** 0.16 Mes Lib C now bootstraps glibc-2.2.5, binutils-2.20.1, gcc-4.1.0. -*** 0.15: MesCC now has a libc+gnu that supports compiling binutils-2.14, gcc-2.95.3 and glibc-2.2.5. -*** 0.14: Mes+MesCC now compiles a slightly patched self-hosting tcc. -*** 0.13: Mes+MesCC now compiles a modified, functional tcc.c (~25,000LOC) in 1h30'. -*** 0.12: Mes+MesCC now compiles mes.c (~3000LOC) in ~4min. -*** 0.11: MesCC now compiles a mes-tcc that passes 26/69 of mescc's C tests. -*** 0.10: Mescc now compiles a mes-tcc that compiles a trivial C to a running a.out. -*** 0.9: Mescc now writes M1 macro assembly files and compiles tcc. -*** 0.8: Mescc now writes object files in stage0's labeled hex2 format. -*** 0.7: Mescc supports -E, -c, -o options, include more complete set of header files, -enough to work on compiling tinycc's tcc.c albeit a somewhat modified version. -*** 0.6: Work with unmodified, unbundled Nyacc; compile 33/55 tinycc's tests/test2 suite. -*** 0.5: Mutual self-hosting Scheme interpreter and C compiler: mes.c and mescc, - Support call-with-current-continuation, refactor catch/throw -*** 0.4: Support Nyacc, Gcc-compiled Mes compiles minimal main.c using nyacc -*** 0.3: Garbage collector -*** 0.2: Support psyntax -*** 0.1: Mes eval/apply feature complete; support syntax-rules, compile main.c using LALR, dump ELF - - * DEBUG MES_DEBUG= mes ** Levels diff --git a/INSTALL b/INSTALL index 43f91a1c..554226d0 100644 --- a/INSTALL +++ b/INSTALL @@ -1,7 +1,7 @@ -*- org -*- #+TITLE: Building and Installing GNU Mes -Copyright © 2016, 2017,2018 Jan (janneke) Nieuwenhuizen +Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen Copying and distribution of this file, with or without modification, are permitted in any medium without royalty provided the copyright @@ -19,12 +19,14 @@ similar way. * Regular build ** Prerequisites *** Guix - guix environment -l guix.scm #64 bit + 32bit - guix environment --system=i686-linux -l guix.scm #32 bit only + #+BEGIN_SRC bash + guix environment -l guix.scm #64 bit + 32bit + guix environment --system=i686-linux -l guix.scm #32 bit only + #+END_SRC *** Other GNU/Linux - - [[https://github.com/oriansj/mescc-tools][mescc-tools]], 0.5 is known to work. - - [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.86.0 is known to work. + - [[https://savannah.gnu.org/projects/mescc-tools/][mescc-tools]], 0.6.1 is known to work. + - [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.93.0 is known to work. - GNU Gcc, 4.9.3 is known to work. - GNU Guile, version 2.0.12 or is know to work. - SH, /bin/sh, GNU Bash 4.3 is known to work. @@ -34,37 +36,54 @@ similar way. - Perl, 5.22 is known to work. ** Build it + + #+BEGIN_SRC bash ./configure make + #+END_SRC ** Check it + + #+BEGIN_SRC bash make check + #+END_SRC ** Install it + + #+BEGIN_SRC bash make install + #+END_SRC * Guix it - guix package -f guix.scm + + #+BEGIN_SRC bash + guix build -f guix.scm + #+END_SRC * Bootstrap build -The bootstrap build is being developed in Guix, on wip-bootstrap, see -http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/mes.scm?h=wip-bootstrap +The bootstrap build is part of Guix, see +https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/commencement.scm ** Prerequisites - - [[https://github.com/oriansj/mescc-tools][mescc-tools]], 0.5 is known to work. - - [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.86.0 is known to work. - - [[https://gitlab.com/janneke/mes-seed][mes-seed]], 0.17.1 is known to work. + - [[https://savannah.gnu.org/projects/mescc-tools/][mescc-tools]], 0.6.1 is known to work. + - [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.93.0 is known to work. ** Build it - - ./configure.sh [--prefix=] - ./build.sh + + #+BEGIN_SRC bash + sh configure.sh [--prefix=] + sh bootstrap.sh + #+END_SRC ** Check it - ./check.sh + #+BEGIN_SRC bash + sh check.sh + #+END_SRC ** Install it - ./install.sh + #+BEGIN_SRC bash + sh install.sh + #+END_SRC diff --git a/NEWS b/NEWS index f0353f92..a44c8c54 100644 --- a/NEWS +++ b/NEWS @@ -2,7 +2,7 @@ #+TITLE: GNU Mes NEWS – history of user-visible changes #+STARTUP: content hidestars -Copyright © 2016,2017,2018 Jan (janneke) Nieuwenhuizen +Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen Copying and distribution of this file, with or without modification, are permitted in any medium without royalty provided the copyright @@ -10,6 +10,23 @@ Copyright © 2016,2017,2018 Jan (janneke) Nieuwenhuizen Please send Mes bug reports to bug-mes@gnu.org. +* Changes in 0.21 since 0.20 +** Core +*** Mes can now be bootstrapped with Gash and Gash Core Utils. +*** Mes now supports a Scheme-only bootstrap. +*** Mes now supports -c EXPR. +** MesCC +*** Mes C Library now supports bootstrapping GNU Awk, GNU Bash, GNU SED, and GNU Tar. +*** Mes C Library now has limited float support in vfprintf, vsnprintf, vsscanf. +**** 7 new functions +abtod, atof, creat, dtoab, execlp, isgraph, mknod, readlink, strtod, +symblink. +**** 5 new stubs +getgrgid, getgrnam, getpgid, getpgrp, mktime, setgrent. +** Noteworthy bug fixes +*** A bug with `mes -c EXPR' has been fixed. +*** The REPL now works again on x86_64. +*** --with-system-libc now works again. * Changes in 0.20 since 0.19 ** Core *** The build system has been simplified, again. diff --git a/README b/README index f53d8881..00280cc6 100644 --- a/README +++ b/README @@ -3,21 +3,21 @@ #+SUBTITLE: Maxwell Equations of Software -[[https://www.gnu.org/software/mes][GNU Mes]] brings a [[http://joyofsource.com/reduced-binary-seed-bootstrap.html][Reduced Binary Seed bootstrap]] to [[https://www.gnu.org/software/guix][GNU Guix]]. This -bootstrap has halved the size of opaque, uninspectable binaries that -were needed to bootstrap Guix. The final goal is to help create a full -source bootstrap as part of the [[http://bootstrappable.org][bootstrappable builds]] effort for any -interested UNIX-like operating system. +[[https://www.gnu.org/software/mes][GNU Mes]] is a Scheme interpreter and C compiler for bootstrapping the GNU +system. Since version 0.20 it brings a [[https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/][Reduced Binary Seed bootstrap]] to +[[https://www.gnu.org/software/guix][GNU Guix]]. This bootstrap has halved the size of opaque, uninspectable +binaries that were needed to bootstrap Guix 1.0. The final goal is to +help create a full source bootstrap as part of the [[http://bootstrappable.org][bootstrappable builds]] +effort for UNIX-like operating systems. Mes consists of a mutual self-hosting Scheme interpreter written in -~5,000 LOC of simple C and a Nyacc-based C compiler written in Scheme. -This mes.c is [[https://github.com/oriansj/mes-m2][being simplified]] to be transpiled by [[https://github.com/oriansj/m2-planet][M2-Planet]]. +~5,000 LOC of simple C, and a C compiler written in Scheme. This mes.c +is [[https://github.com/oriansj/mes-m2][being simplified]] to be transpiled by [[https://github.com/oriansj/m2-planet][M2-Planet]]. -The Scheme interpreter (mes.c) has a Garbage Collector, a library of -loadable Scheme modules-- notably Dominique Boucher's [[https://github.com/schemeway/lalr-scm][LALR]], Pre-R6RS -[[https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html][portable syntax-case]] with R7RS ellipsis, Matt Wette's [[https://www.nongnu.org/nyacc][Nyacc]] --and test -suite just barely enough to support a simple REPL and simple -C-compiler: MesCC. +The Scheme interpreter has a Garbage Collector, a library of loadable +Scheme modules-- notably Dominique Boucher's [[https://github.com/schemeway/lalr-scm][LALR]], Pre-R6RS [[https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html][portable +syntax-case]] with R7RS ellipsis, Matt Wette's [[https://www.nongnu.org/nyacc][Nyacc]] --and test suite just +enough to support a REPL and a C99 compiler: MesCC. Mes+MesCC can compile an only [[http://gitlab.com/janneke/tinycc][lightly patched TinyCC]] that is self-hosting. Using this tcc and the Mes C library we now have a @@ -64,16 +64,17 @@ General Public Licence version 3 or later. See the file [[file:COPYING][COPYING * Links [0] https://www.gnu.org/software/mes - [1] https://www.gnu.org/software/guix - [2] http://bootstrappable.org - [3] https://github.com/oriansj/mes-m2 - [4] https://github.com/oriansj/m2-planet - [5] https://github.com/schemeway/lalr-scm - [6] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html - [7] https://www.nongnu.org/nyacc - [8] http://gitlab.com/janneke/tinycc - [9] [[http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf]] - [10] https://github.com/oriansj/stage0 + [1] https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/ + [2] https://www.gnu.org/software/guix + [3] https://bootstrappable.org + [4] https://github.com/oriansj/mes-m2 + [5] https://github.com/oriansj/m2-planet + [6] https://github.com/schemeway/lalr-scm + [7] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html + [8] https://www.nongnu.org/nyacc + [9] https://gitlab.com/janneke/tinycc + [10] [[http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf]] + [11] https://github.com/oriansj/stage0 * Legalese Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen <[[mailto:janneke@gnu.org][janneke@gnu.org]]> diff --git a/ROADMAP b/ROADMAP new file mode 100644 index 00000000..c4dd4d8b --- /dev/null +++ b/ROADMAP @@ -0,0 +1,91 @@ + -*- org -*- +#+TITLE: Tentative GNU Mes Road Map + +Copyright © 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen + + Copying and distribution of this file, with or without modification, + are permitted in any medium without royalty provided the copyright + notice and this notice are preserved. + +The goals of the GNU Mes project are two-fold: + + - to bootstrap the GNU system purely from source, without using any binary + seeds. + + - to aid the Reproducible-Builds.org effort; demonstrate the impact and + importance of GNU and Software Freedom. + +Since its inception, the project has gone a long way towards that goal. Below +is a list of items we want for version "1.0" of GNU Mes. + +You're welcome to discuss this road map on bug-mes@gnu.org or #bootstrappable +on Freenode! + +* TODO +** release 0.x, unsorted + - Introduce the Reduced Binaries Seed bootstrap to NixOS (Debian, + Gentoo, ...?) + - Full Source Bootstrap: compile Mes.c using M2-Planet. Jeremiah + branched-out from mes; see https://github.com/oriansj/mes-m2, also see the + branches: wip-m2-merge and wip-m2. + - Scheme-only bootstrap bootstrap-binaries with Gash and Gash Core + Utilities: Awk, Bash, Core Utilities, Grep, Gzip, Make, Sed, Tar. + - Full Guile-compatible module support: define-module, define-public, + export, use-modules, ... + - ARMv7/AArch64 support: Mes C Library and MesCC. + - Simplify eval-apply and other core functionality (see mes-m2). +** after release 1.0 + - Replace the initial gcc-2.95.3 with gcc-4.6.4. + - tcc: remove or upstream patches from tcc-boot. + - tcc: build 0.9.27 directly instead of via 0.9.26, see tinycc + wip-bootstrappable@0.9.27 branch + - Bootstrap a `bootstrap-Guile' before bootstrapping tcc? + - Skip tcc: Build gcc using MesCC. + - better garbage collector. + - mes/mescc: proper docstrings, api reference documentation. + - mes/mescc: bootstrap a minimal Guile? + + libguile/{eval,init,list,strings,values,..}. + + ice-9/eval.scm + - syntax-case bootstrap problem + + resolve portable syntax-case bootstrap, or + + get full source syntax-case up (Andre van Tonder?) + https://srfi.schemers.org/srfi-72/srfi-72.html, or + + ... drop it? + - Support the Hurd. There is a wip-hurd branch; most needed now are fork + and exec. +* DONE + - 0.21 GNU Mes now supports a Scheme-only bootstrap and is packaged in + Debian GNU/Linux. + - 0.20 GNU Mes brings the Reduced Binary Source bootstrap to Guix. + - 0.19 GNU Mes now compiles TinyCC in ~8min and supports building Bash and + GNU Tar. + - 0.18 GNU Mes now supports GuixSD bootstrap (x86,x86_64) and has native + x86_64 support. + - 0.17.1 GNU Mes now allows removing glibc, binutils and gcc from the GuixSD + bootstrap. + - 0.17 GNU Mes is now an official GNU project and bootstraps gcc-4.7.4. + - 0.16.1 Mes now has info docs and installs ootb on Debian buster/testing. + - 0.16 Mes Lib C now bootstraps glibc-2.2.5, binutils-2.20.1, gcc-4.1.0. + - 0.15: MesCC now has a libc+gnu that supports compiling binutils-2.14, + gcc-2.95.3 and glibc-2.2.5. + - 0.14: Mes+MesCC now compiles a slightly patched self-hosting tcc. + - 0.13: Mes+MesCC now compiles a modified, functional tcc.c (~25,000LOC) in + 1h30'. + - 0.12: Mes+MesCC now compiles mes.c (~3000LOC) in ~4min. + - 0.11: MesCC now compiles a mes-tcc that passes 26/69 of mescc's C tests. + - 0.10: Mescc now compiles a mes-tcc that compiles a trivial C to a running + a.out. + - 0.9: Mescc now writes M1 macro assembly files and compiles tcc. + - 0.8: Mescc now writes object files in stage0's labeled hex2 format. + - 0.7: Mescc supports -E, -c, -o options, include more complete set of + header files, enough to work on compiling tinycc's tcc.c albeit a somewhat + modified version. + - 0.6: Work with unmodified, unbundled Nyacc; compile 33/55 tinycc's + tests/test2 suite. + - 0.5: Mutual self-hosting Scheme interpreter and C compiler: mes.c and + mescc, Support call-with-current-continuation, refactor catch/throw + - 0.4: Support Nyacc, Gcc-compiled Mes compiles minimal main.c using nyacc + - 0.3: Garbage collector + - 0.2: Support psyntax + - 0.1: Mes eval/apply feature complete; support syntax-rules, compile main.c + using LALR, dump ELF diff --git a/build-aux/install.sh.in b/build-aux/install.sh.in index a4316064..c8ecb67e 100644 --- a/build-aux/install.sh.in +++ b/build-aux/install.sh.in @@ -67,9 +67,9 @@ cp\ BOOTSTRAP\ COPYING\ HACKING\ - INSTALL\ NEWS\ README\ + ROADMAP\ ${DESTDIR}${docdir} if test -f ChangeLog+; then diff --git a/configure.sh b/configure.sh index 9479ef93..c6ae3881 100755 --- a/configure.sh +++ b/configure.sh @@ -254,7 +254,7 @@ GNU Mes is configured for system: $mes_system Run: - ./build.sh to build mes - ./check.sh to check mes - ./install.sh to install mes + sh build.sh to build mes + sh check.sh to check mes + sh install.sh to install mes EOF diff --git a/doc/announce/ANNOUNCE-0.20 b/doc/announce/ANNOUNCE-0.20 index 3a71ad81..1a427c1d 100644 --- a/doc/announce/ANNOUNCE-0.20 +++ b/doc/announce/ANNOUNCE-0.20 @@ -129,5 +129,5 @@ janneke and Danny. [8] https://www.nongnu.org/nyacc [9] https://gitlab.com/janneke/tinycc [10] http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf -[11] https://github.com/oriansj/stage0 +[11] https://savannah.nongnu.org/projects/stage0 [12] https://nlnet.nl/project/GNUMes diff --git a/doc/announce/ANNOUNCE-0.21 b/doc/announce/ANNOUNCE-0.21 new file mode 100644 index 00000000..9264f855 --- /dev/null +++ b/doc/announce/ANNOUNCE-0.21 @@ -0,0 +1,132 @@ + +Subject: GNU Mes 0.21 released + +<#secure method=pgpmime mode=sign> + +We are pleased to announce the release of GNU Mes 0.21, representing +54 commits over 10 weeks. + +Mes has now brought the Reduced Binary Seed bootstrap to Guix (bootstrap +a GNU/Linux system without binary GNU toolchain or equivalent). See +https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/ + +This release supports a Scheme-only bootstrap: Mes can now be built with +Gash and the experimental Gash Core Utils instead of using GNU Awk, GNU +Bash, the GNU Core Utilities, GNU Grep, GNU Gzip, GNU Make, GNU Sed, and +GNU Tar. Also, the Mes C Library now supports bootstrapping those. +Finally, this release brings Mes as a package to Debian GNU/Linux. + +We are excited that the Nlnet Foundation[12] is now sponsoring this +work! + +Next targets: + + - Introduce the Reduced Binaries Seed bootstrap to NixOS (Debian, + Gentoo, ...?) + - Scheme-only bootstrap: use Guile, Gash and Gash Core Utils to remove + awk, bash, core utilities, grep, gzip, make, sed, tar, etc. from the + Guix bootstrap binaries + - ARM support + - Full Source Bootstrap: compile Mes.c using M2-Planet + - Reduced Binary Seed bootstrap for ARM + - the Hurd + +Packages are available in Guix master. + +* About + + GNU Mes[0] brings a Reduced Binary Seed bootstrap[1] to GNU Guix[2]. + This bootstrap has halved the size of opaque, uninspectable binaries + that were needed to bootstrap Guix. The final goal is to help create + a full source bootstrap as part of the bootstrappable builds[3] effort + for any interested UNIX-like operating system. + + It consists of a mutual self-hosting Scheme interpreter written in + ~5,000 LOC of simple C and a Nyacc-based C compiler written in Scheme. + This mes.c is being simplified[4] to be transpiled by M2-Planet[5]. + + The Scheme interpreter has a Garbage Collector, a library of loadable + Scheme modules-- notably Dominique Boucher's LALR[6], Pre-R6RS + [portable syntax-case[7] with R7RS ellipsis, Matt Wette's Nyacc[8] + --and test suite just enough to support a REPL and a C99 compiler: + MesCC. + + Mes+MesCC can compile an only lightly patched TinyCC[9] that is + self-hosting. Using this tcc and the Mes C library we now have a + Reduced Binary Seed bootstrap for the gnutools triplet: glibc-2.2.5, + binutils-2.20.1, gcc-2.95.3. This is enough to bootstrap Guix for + i686-linux and x86_64-linux. + + Mes is inspired by The Maxwell Equations of Software: LISP-1.5[10] -- John + McCarthy page 13, GNU Guix's source/binary packaging transparency and + Jeremiah Orians's stage0[11] ~500 byte self-hosting hex assembler. + +* Download + + git clone git://git.savannah.gnu.org/mes.git + + Here are the compressed sources and a GPG detached signature[*]: + https://ftp.gnu.org/gnu/mes/mes-0.21.tar.gz + https://ftp.gnu.org/gnu/mes/mes-0.21.tar.gz.sig + + Use a mirror for higher download bandwidth: + https://ftpmirror.gnu.org/mes/mes-0.21.tar.gz + https://ftpmirror.gnu.org/mes/mes-0.21.tar.gz.sig + + Here are the MD5 and SHA1 checksums: + + xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx mes-0.21.tar.gz + xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx mes-0.21.tar.gz + + [*] Use a .sig file to verify that the corresponding file (without the + .sig suffix) is intact. First, be sure to download both the .sig file + and the corresponding tarball. Then, run a command like this: + + gpg --verify mes-0.21.tar.gz.sig + + If that command fails because you don't have the required public key, + then run this command to import it: + + gpg --keyserver keys.gnupg.net --recv-keys 1A858392E331EAFDB8C27FFBF3C1A0D9C1D65273 + + and rerun the 'gpg --verify' command. + +* Get informed, get involved + + See https://bootstrappable.org + Join #bootstrappable on irc.freenode.net. + +* Changes in 0.21 since 0.20 + ** Core + *** Mes can now be bootstrapped with Gash and Gash Core Utils. + *** Mes now supports a Scheme-only bootstrap. + *** Mes now supports -c EXPR. + ** MesCC + *** Mes C Library now supports bootstrapping GNU Awk, GNU Bash, GNU SED, and GNU Tar. + *** Mes C Library now has limited float support in vfprintf, vsnprintf, vsscanf. + **** 7 new functions + abtod, atof, creat, dtoab, execlp, isgraph, mknod, readlink, strtod, + symblink. + **** 5 new stubs + getgrgid, getgrnam, getpgid, getpgrp, mktime, setgrent. + ** Noteworthy bug fixes + *** A bug with `mes -c EXPR' has been fixed. + *** The REPL now works again on x86_64. + *** --with-system-libc now works again. + +Greetings, +janneke and Danny. + +[0] https://www.gnu.org/software/mes +[1] https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/ +[2] https://www.gnu.org/software/guix +[3] https://bootstrappable.org +[4] https://github.com/oriansj/mes-m2 +[5] https://github.com/oriansj/m2-planet +[6] https://github.com/schemeway/lalr-scm +[7] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html +[8] https://www.nongnu.org/nyacc +[9] https://gitlab.com/janneke/tinycc +[10] http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf +[11] https://savannah.nongnu.org/projects/stage0 +[12] https://nlnet.nl/project/GNUMes diff --git a/doc/mes.texi b/doc/mes.texi index 7fed87db..011494aa 100644 --- a/doc/mes.texi +++ b/doc/mes.texi @@ -13,7 +13,7 @@ @set OPENPGP-SIGNING-KEY-ID 1A858392E331EAFDB8C27FFBF3C1A0D9C1D65273 @copying -Copyright @copyright{} 2018 Jan (janneke) Nieuwenhuizen@* +Copyright @copyright{} 2018,2019 Jan (janneke) Nieuwenhuizen@* Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or @@ -34,7 +34,7 @@ Documentation License''. @titlepage @title GNU Mes Reference Manual -@subtitle Full Source Bootstrapping of the GNU Guix Operating System +@subtitle Full Source Bootstrapping for the GNU system @author Jan (janneke) Nieuwenhuizen @page @@ -124,8 +124,9 @@ that we can trust. Mes consists of a mutual self-hosting Scheme interpreter written in C and a Nyacc-based (see @pxref{NYACC User's Guide,,, nyacc-ug, NYACC User's Guide}) C compiler written in Scheme. The Scheme interpreter -@file{mes.c} is about 5000LOC of restricted C and intended to be -bootstrappable using a very simple C compiler. +@file{mes.c} is about 5,000LOC of restricted C, to be compiled with +M2-Planet@footnote{See @url{https://github.com/oriansj/m2-planet}}, a +very simple C compiler. If we want to trust our computers to do what we instructed them to do then we need to be able to inspect all instructions---all @@ -255,10 +256,10 @@ binary seed that all software distributions inject are the so called @emph{bootstrap binaries}. Bootstrap binaries are the initial binary seeds that are used to start building the distribution. -The GNU Guix operating system has a relatively small closure of -bootstrap binaries: GNU binutils, GNU gcc, GNU Libc, GNU Guile, and -``Static binaries'' (think: bash, bzip2, coreutils, gawk, grep, gzip, -patch, sed, tar, xz). +The GNU Guix operating system, version 1.0 had a relatively small +closure of bootstrap binaries: GNU binutils, GNU gcc, GNU Libc, GNU +Guile, and ``Static binaries'' (think: bash, bzip2, coreutils, gawk, +grep, gzip, patch, sed, tar, xz). @example $ du -schx $(readlink $(guix build bootstrap-tarballs)/*) @@ -270,8 +271,8 @@ $ du -schx $(readlink $(guix build bootstrap-tarballs)/*) 33M total @end example -only a 33MB download that unpacks to 252BM of opaque binaries, that we -most probably have the source of, shall we review these together? ;-) +only a 33MB download that unpacks to a 252MB @emph{seed} of opaque +binary code. @example $ for i in $(readlink $(guix build bootstrap-tarballs)/*);\ @@ -285,6 +286,13 @@ $ du -schx * 252M total @end example +During the Guix 1.1 development series we managed to create the first +reduction by 50% of the Guix @emph{bootstrap binaries}@footnote{See +@url{https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/}}. +This was a very important step because the ~250MB @emph{seed} of binary +code was practically non-auditable, which makes it hard to establish +what source code produced them. + @node Full Source Bootstrap @section Full Source Bootstrap @@ -433,7 +441,7 @@ enjoy software Freedom 1. @cindex installing Mes Mes is available for download from its website at -@url{http://www.gnu.org/pub/gnu/mes/}. This section describes the +@url{https://www.gnu.org/pub/gnu/mes/}. This section describes the software requirements of Mes, as well as how to install it and get ready to use it. @@ -458,21 +466,14 @@ GNU Mes depends on the following packages: @item @url{http://gnu.org/software/guile/, GNU Guile}, version 2.0.13 or later, including 2.2.x; @item @url{http://www.gnu.org/software/make/, GNU Make}. -@item @url{https://savannah.gnu.org/projects/nyacc/, NYACC}, 0.86.0 is known to work. +@item @url{https://savannah.gnu.org/projects/nyacc/, NYACC}, 0.93.0 or later, including 0.99.0. @item @url{http://gcc.gnu.org, GCC's gcc}, version 2.95.3 or later. -@end itemize - -The following dependencies are optional: - -@itemize -@item -Installing @url{https://github.com/oriansj/mescc-tools/, mescc-tools}, -version 0.5 or later, will allow you to have MesCC assemble and link. +@item @url{https://savannah.gnu.org/projects/mescc-tools/, mescc-tools}, version 0.6.1 or later, @end itemize @cindex Guile, compatibility Mes is compatible with GNU Guile, so it is possible to share the same -Scheme code between both. Of course, Mes only supports the minimal +Scheme code between both. Currently Mes only supports the minimal subset of R5RS and Guile extensions to run MesCC. @node Bootstrap Requirements @@ -483,23 +484,21 @@ package. The bootstrap build procedure for Mes is similar to building GNU software and goes like this @example -export prefix=/usr/local # for example -export MES_SEED=../mes # for example -# optionally set some other environment variables -sh build.sh +sh configure.sh --prefix=/your/prefix/here +sh bootstrap.sh sh check.sh sh install.sh @end example -See @file{build.sh} for inspiration on what environment variables to -set. +See @file{configure.sh} and @file{bootstrap.sh} for inspiration on what +environment variables to set. Bootstrapping Mes depends on the following packages: @itemize @item a POSIX-compatible shell -@item @url{https://github.com/oriansj/mescc-tools/, mescc-tools}, version 0.5 or later. -@item @url{http://gitlab.com/janneke/mes-seed/, mes-seed}, version 0.17 or later. +@item @url{https://github.com/oriansj/mescc-tools/, mescc-tools}, version 0.6.1 or later. +@item @url{https://savannah.gnu.org/projects/nyacc/, NYACC}, 0.93.0 or later, including 0.99.0. @end itemize @node Running the Test Suites @@ -593,34 +592,51 @@ responsibility. @node The Mes Bootstrap Process @section The Mes Bootstrap Process -The Mes full source bootstrap process@footnote{TBH, the current state of -affairs demands to label this a `Reduced Binary Seed bootstrap'} is -currently being developed in Guix@footnote{See -@file{gnu/packages/commencement.scm} in the @var{core-updates} branch in Guix +The Reduced Binary Seed bootstrap currently adopted by Guix@footnote{See +@file{gnu/packages/commencement.scm} in the @var{master} branch in Guix git -@url{http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/mes.scm?h=wip-bootstrap}}. -In it's intiial form it is only available for x86-linux. +@url{http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/commencement.scm}}. +In its intiial form it is only available for x86-linux. Currently, it goes like this: @verbatim -mescc-tools-source + mescc-tools-seed => mescc-tools - -mes-source + mescc-tools + mescc-seed => mes - -tcc-source + mes + tinycc-seed => tcc - -binutils-source + mes + tcc => binutils0 - -gcc-source + tcc + binutils0 => gcc-core - -glibc-source + kernel-headers-source + binutils0 + gcc => glibc0 - -binutils-source + binutils0 + gcc + glibc => binutils - -gcc-source + binutils + gcc-core + glibc => gcc - -glibc-source + binutils + gcc + glibc0 => glibc + gcc-mesboot (4.9.4) + ^ + | + glibc-mesboot (2.16.0) + ^ + | + gcc-mesboot1 (4.7.4) + ^ + | + binutils-mesboot (2.20.1a) + ^ + | + gcc-mesboot0 (2.95.3) + ^ + | + glibc-mesboot0 (2.2.5) + ^ + | + gcc-core-mesboot (2.95.3) + ^ + | + make-mesboot0, diffutils-mesboot, binutils-mesboot0 (2.20.1a) + ^ + | + tcc-boot + ^ + | + tcc-boot0 + ^ + | + mes-boot + ^ + | + * + bootstrap-mescc-tools, bootstrap-mes (~10MB) + bootstrap-bash, bootstrap-coreutils&co, bootstrap-guile (~120MB) @end verbatim @c This graph is generated from wip-bootstrap, doing: @@ -636,14 +652,14 @@ that builds the rest of Guix. Work is ongoing to remove these binary seeds that were intentionally injected by our own doing as temporary shortcut @example -mescc-tools-seed, mes-seed, tinycc-seed +bootstrap-mescc-tools (seed), bootstrap-mes (seed) @end example For now, these additional non-bootstrapped dependencies (i.e., binary seeds) are taken for granted @example -BOOTSTRAP-GUILE, bash, bzip2, coreutils, gawk, grep, gzip, patch, sed, +bootstrap-guile, bash, bzip2, coreutils, gawk, grep, gzip, patch, sed, tar, xz @end example @@ -659,7 +675,7 @@ The @command{mes} command is the Scheme interpreter whose prime directive is to run the @command{MesCC} program. For convenience and testing purposes, @command{mes} tries to mimic -Guile. +guile. @example mes @var{option}@dots{} @file{FILE}@dots{} @@ -704,9 +720,6 @@ variable. @item -C,--compiled-path=@var{directory} Accepted and ignored for Guile compatibility. -@item ---dump -dump binary program to stdout - @item -l @var{file} Load Scheme source code from @var{file}, and continue processing the command line. @@ -721,9 +734,6 @@ and the command-line arguments---the list provided by the @item -h@r{, }--help Display help on invoking Mes, and then exit. -@item ---load -load binary program [module/mes/boot-0.32-mo] - @item -v@r{, }--version Display the current version of Mes, and then exit. diff --git a/mes/module/mes/repl.mes b/mes/module/mes/repl.mes index f3244203..9f2c3547 100644 --- a/mes/module/mes/repl.mes +++ b/mes/module/mes/repl.mes @@ -30,6 +30,7 @@ (define welcome (string-append "GNU Mes " %version " Copyright (C) 2016,2017,2018,2019 Jan (janneke) Nieuwenhuizen +Copyright (C) 2019 Danny Milosavljevic GNU Mes comes with ABSOLUTELY NO WARRANTY; for details type `,show w'. This program is free software, and you are welcome to redistribute it