mes/BOOTSTRAP

bootstrappable.org project                                           -*- org -*-

* What?
** Full source bootstrapping for GuixSD

A package in GuixSD is can be uniquely identified by the hash of its
source code, its dependencies and its build recipe.

Every package can be built from source, except for the bootstrap
binaries.

*** From the GuixSD manual

   The distribution is fully “bootstrapped” and “self-contained”: each
package is built based solely on other packages in the distribution.

The root of this dependency graph is a small set of “bootstrap
binaries”, provided by the ‘(gnu packages bootstrap)’ module.  For more
information on bootstrapping, *note Bootstrapping::.

**** GuixSD bootstrap tarballs

$ du -schx $(readlink $(guix build bootstrap-tarballs)/*)
2.1M	/gnu/store/9623n4bq6iq5c8cwwdq99qb7d0xj93ym-binutils-static-stripped-tarball-2.28.1/binutils-static-stripped-2.28.1-x86_64-linux.tar.xz
18M	/gnu/store/437xwygmmwwpkddcyy1qvjcv4hak89pb-gcc-stripped-tarball-5.5.0/gcc-stripped-5.5.0-x86_64-linux.tar.xz
1.8M	/gnu/store/55ccx18a0d1x5y6a575jf1yr0ywizvdg-glibc-stripped-tarball-2.26.105-g0890d5379c/glibc-stripped-2.26.105-g0890d5379c-x86_64-linux.tar.xz
5.7M	/gnu/store/bqf0ajclbvnbm0a46819f30804y3ilx0-guile-static-stripped-tarball-2.2.3/guile-static-stripped-2.2.3-x86_64-linux.tar.xz
5.8M	/gnu/store/j8yzjmh9sy4gbdfwjrhw46zca43aah6x-static-binaries-tarball-0/static-binaries-0-x86_64-linux.tar.xz
33M	total
$ for i in $(readlink $(guix build bootstrap-tarballs)/*);\
  do sudo tar xf $i; done
$ du -schx *
130M	bin
13M	include
54M	lib
51M	libexec
5.2M	share
252M	total

* Why?
** Reproducibility is essential to Software Freedom

Reproducible builds are a set of software development practices that
create a verifiable path from human readable source code to the binary
code used by computers.

*** What about the bootstrap binaries and the compilers?

We have the sources: they always lead to bitwise-same binary, but what
about the bootstrap binaries and compilers?

*** The current way out: Ignore the problem

 ``recipe for yoghurt: add yoghurt to milk''

*** New solution: Full source bootstrapping path

* How?
** Software: MesCC-tools, M2-Planet, Mes and MesCC
** MesCC-tools

    https://github.com/oriansj/mescc-tools

*** hex.0: ~500 byte well-documented, self-hosting hex assembler

This 500 byte program is written in ASCII hex.  When converted
byte-for-byte from ASCCI to binary we have the only binary seed that
our full source bootstrap path needs.

We bless this simple and easily verifyable binary and consider it to
be source.

*** hex1: next level hex assembler written in hex.0
*** M0: a macro assembler written in hex.1
*** M1: a macro assembler written in M0
*** hex2: a hex2 linker written in M0
*** M2-Planet: a self-hosting M2 (C-with-structs) transpiler written in M1
*** Mes: A Scheme interpreter written in C, with cpp transformed into M2
*** MesCC: A C compiler written in Scheme
*** tcc-boot: a patched version of TinyCC

** Mes and MesCC

    https://gitlab.com/janneke/mes

*** mes.c: a Scheme interpreter prototyped in C ~3000 Lines
*** mescc: a C compiler written in Scheme (uses Nyacc C99 parser in Scheme)
*** mes.M2: this Scheme interpreter in preprocessed M2

** TinyCC

    https://gitlab.com/janneke/tinycc

* TODO
** upstream mes-boot to GuixSD.
** add full source gcc-4.7 package build.
** replace GuixSD bootstrap for x86.
** remove or upstream patches from tcc-boot
** prepare src/mes.c for M2-Planet transpiler.
** fix bootstrap-loops: (Nyacc?, mes.M2, psyntax.pp?)
** make GNU gcc (8.0?) bootstrappable again, remove [need for] tcc stage
** integrate with GuixSD
** x86_64, arm, the Hurd

* DONE
** have tcc-boot compile gnutools triplet: binutils-2.14, gcc-2.95.3, glibc-2.2.5.
** have tcc-boot's mes-tcc compile a fully functional tcc
** hex.0, hex.1, M0
** M1, hex2_linker prototyped in C
** M2-Planet is now self-hosting, written in M2
** mes+mescc are mutual self-hosting
** patched tcc compiled with mes+mescc
** during development we run mescc on Guile (mes is ~30 times slower)
** tcc compiled with gcc is known to compile gcc

* Contact
** #bootstrappable, #guix on freenode
** bootstrappable.org