diff --git a/BOOTSTRAP b/BOOTSTRAP new file mode 100644 index 00000000..42da81ee --- /dev/null +++ b/BOOTSTRAP @@ -0,0 +1,96 @@ +bootstrappable.org project -*- org -*- + +* What? +** Full source bootstrapping for GuixSD +* Why? +** Reproducibility is essential to Software Freedom +Reproducible builds are a set of software development practices that +create a verifiable path from human readable source code to the binary +code used by computers. +*** What about the compiler? +We have the sources: they always lead to bitwise-same binary, but what +about the compiler? +*** The current way out: Ignore the problem + + ``recipe for yoghurt: add yoghurt to milk'' + +*** From the GuixSD manual + + The distribution is fully “bootstrapped” and “self-contained”: each +package is built based solely on other packages in the distribution. + +The root of this dependency graph is a small set of “bootstrap +binaries”, provided by the ‘(gnu packages bootstrap)’ module. For more +information on bootstrapping, *note Bootstrapping::. + +*** New solution: Full source bootstrapping path, Stage0 and Mes +* How? +** Software: Stage0 and Mes +** Stage0 + +*** hex.0: amazing ~300 byte self-hosting hex assembler that we consider to be source +*** a M0 macro assembler written in .0 +*** a M1 macro assembler written in M0 +*** a hex2 linker written in M0 + +Look: + https://git.savannah.nongnu.org/cgit/stage0.git/tree/Linux%20Bootstrap/hex0.hex + +Do: + git clone https://git.savannah.nongnu.org/git/stage0.git + cd stage0 + make + bin/hex < Linux\ Bootstrap/hex0.hex > hex-1 + chmod +x hex-1 # later: bin/exec_enable hex-1 + ./hex-1 < Linux\ Bootstrap/hex0.hex > hex-2 + +** Mes + + https://gitlab.com/janneke/mes + +*** mes.c: a scheme interpreter prototyped in C ~1400 Lines +*** mescc.scm: a C compiler written in Scheme (uses Nyacc C99 parser in Scheme) +*** mes.M1: this scheme interpreter in annotated M1 assembly + +** Naive recipe + hex + hex.0 => hex-1 + hex-1 + M0.0 -> M0 + M0 + M1.M0 -> M1.0 + M0 + hex2_linker.M0 -> hex2_linker + M1 + mes.M1 -> mes.hex2 + hex2_linker + stage0/elf32-header.hex2 + mes.hex2 + elf32-footer.hex2 -> mes + mes + mescc.scm + tcc.c -> tcc.M1 -> tcc.hex2 -> mes-tcc + mes-tcc + gcc.c -> gcc + *done* +** Less naive recipe +*** Use hex2_linker, M1 prototyped in C from + https://github.com/oriansj/mescc-tools +*** Remember that mes.M1 is compiled by mescc from prototyped src/mes.c in C. +*** stage0/mescc-tools: TODO +*** Mes + M1 -f stage0/x86.M1 -f mlibc/crt1.M1 --LittleEndian --Architecture=1 > mlibc/crt1.hex2 + M1 -f stage0/x86.M1 -f mlibc/mini-libc-mes.M1 --LittleEndian --Architecture=1 > mlibc/libc-mes.hex2 + M1 -f stage0/x86.M1 -f src/mes.M1 --LittleEndian --Architecture=1 > src/mes.hex2 + hex2 --LittleEndian --Architecture=1 --BaseAddress=0x1000000 -f stage0/elf32-header.hex2 -f mlibc/crt1.hex2 -f mlibc/libc-mes.hex2 -f src/mes.hex2 -f stage0/elf32-footer-single-main.hex2 > src/mes-mes + exec_enable src/mes-mes +*** Tinycc + ./build.sh + ./link.sh +* DONE +** stage0: hex.0, M0 done; M1, hex2_linker prototyped in C +** tcc compiled with mescc correctly compiles: int main () {return 42;} +** mes+mescc.scm are mutual self hosting +** during development we run mescc.scm on Guile (mes is slooowww) +** tcc compiled with gcc is known to compile gcc + +* TODO +** fix mescc.scm so that tcc can correctly compile gcc +** fix bootstrap-loops: (Nyacc?, mes.M1?, psyntax.pp?) +** make GNU gcc (8.0?) bootstrappable again, remove [need for] tcc stage +** stage1/2 LISP, FORTH? +** rain1's LISP compiler? +** integrate with GuixSD +** x86_64, arm? +* Contact +** #bootstrappable, #guix on freenode +** bootstrappable.org diff --git a/HACKING b/HACKING index 06874478..5187cd7c 100644 --- a/HACKING +++ b/HACKING @@ -41,8 +41,8 @@ Running ./make.scm produces a `script' file. * ROADMAP ** TODO -*** release 0.10 - - mescc: compile minimal main.c with tcc.mes https://gitlab.com/janneke/tinycc wip-mescc +*** release 0.11 + - mescc: compile a mes-tcc that can compile gcc-4.7. *** release 0.x, unsorted - mescc: support unsigned comparison, arithmetic. - mescc: support bitfields. @@ -64,6 +64,7 @@ Running ./make.scm produces a `script' file. - work to replace GuixSD's bootstrap binaries for x86 - more architectures? ** DONE +** 0.10: Mescc now compiles a mes-tcc that compiles a trivial C to a running a.out. ** 0.9: Mescc now writes M1 macro assembly files and compiles tcc. ** 0.8: Mescc now writes object files in stage0's labeled hex2 format. ** 0.7: Mescc supports -E, -c, -o options, include more complete set of header files, diff --git a/INSTALL b/INSTALL index c8ebddaf..44fc2fbc 100644 --- a/INSTALL +++ b/INSTALL @@ -8,8 +8,8 @@ Building and Installing Mes * Prerequisites ** Bootstrapping For bootstrapping, the M1 macro assembler and hex2 linker and Nyacc -*** [[https://github.com/oriansj/MESCC_Tools][MESCC_Tools]], 0.2 is known to work. -*** [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.80.3 is known to work. +*** [[https://github.com/oriansj/mescc-tools][mescc-tools]], 0.2 is known to work. +*** [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.81.0 is known to work. For development ** Development For development, there are some more dependencies. @@ -19,8 +19,8 @@ For development, there are some more dependencies. guix environment --system=i686-linux -l guix.scm #32 bit only *** Other GNU/Linux -**** [[https://github.com/oriansj/MESCC_Tools][MESCC_Tools]], 0.2 is known to work. -**** [[https://savannah.gnu.org/projects/nyacc][Nyacc]], 0.80.3 is known to work. +**** [[https://github.com/oriansj/mescc-tools][mescc-tools]], 0.2 is known to work. +**** [[https://savannah.gnu.org/projects/nyacc][Nyacc]], [[https://gitlab.com/janneke/nyacc][0.80.41 {patched 0.80.4}]] is known to work. **** GNU Gcc, 4.9.3 is known to work. The idea is to drop this requirement, of course. @@ -28,9 +28,6 @@ The idea is to drop this requirement, of course. Guile is not strictly necessary but it is used by the configure script and in development. -**** GNU Make, version 4.0 is know to work. -Not strictly necessary. - **** /bin/sh, GNU Bash 4.3 is known to work. Let's just say for now that it gets awkward without a shell. @@ -44,16 +41,10 @@ Let's just say for now that it gets awkward without a shell. ./configure make -** For a quick build, skip running mescc on Mes - make MES_SKIP_MES=1 - * Check it make check -** For a quick check, skip running mescc on Mes - make MES_SKIP_MES=1 - * Install it make install diff --git a/NEWS b/NEWS index 24719c18..89ce0166 100644 --- a/NEWS +++ b/NEWS @@ -10,11 +10,49 @@ Copyright © 2016,2017 Jan Nieuwenhuizen Please send Mes bug reports to janneke@gnu.org. +* Changes in 0.10 since 0.9 +** Core +*** 2 new functions +logand, lognot. +** Mescc +*** Mes now includes M1 sources for full source bootstrapping up to tcc. +*** Mescc now depends on Nyacc 0.81.0. +*** Mescc now compiles a mes-tcc that compiles a trivial C to a running a.out. +*** Mescc now supports several (as used my tinycc) struct by value assignments. +*** Mescc now has _start in a separate crt1.c. +*** Mescc now supports initializer arithmetic. +*** Mescc now supports arrays of struct of arbitrary size. +*** Mescc now supports pointer arithmetic. +*** Mescc now defines __i386__, __linux__. +*** Mescc now supports --*p, ++*p. +*** Mescc now supports int foo[bar] = {baz,...}. +*** Mescc now supports \x00 in strings in M1. +*** Mescc now supports complex enum expressions. Thanks, rain1! +*** Mescc now supports short. +*** Mescc now supports foo.bar = foo.baz = bla. +*** Mescc now supports foo = bar < baz and comparison permutations. +*** Mescc now supports (anonymous) structs in unions. +*** Mescc now writes M1 strings if possible. +**** 2 improved mlibc functions +malloc, realloc. +**** 13 new mlibc functions +fclose, fgetc, fprintf, fwrite, memcpy, memmove, memset, snprintf, strcat, +strchr, strrchr, strtoull, vnsprintf. +**** 15 new tinycc-support tests +76-pointer-arithmetic.c, 77-pointer-assign.c, 78-union-struct.c, +79-int-array.c, 7a-struct-char-array.c, 7b-struct-int-array.c, +7c-dynarray.c, 7d-cast-char.c, 7e-struct-array-access.c, +7f-struct-pointer-arithmetic.c, 7g-struct-byte-word-field.c, +7h-struct-assign.c, 7i-struct-struct.c, 7j-strtoull.c, +7k-for-each-elem.c. +** Noteworthy bug fixes +*** Mescc has many fixes for foo.bar[baz], foo[bar].baz with ./-> permutations. +*** Many bugs and limitations in mescc have been fixed and removed. * Changes in 0.9 since 0.8 ** Core *** Mes now builds better in non-Guix[SD] legacy distros/environments. Thanks, rain1! ** Mescc -*** Mescc now compiles a modified tinycc into a running [mostly segfaulting] tcc.mes. +*** Mescc now compiles a modified tinycc into a running [mostly segfaulting] mes-tcc. https://gitlab.com/janneke/tinycc branch wip-mescc. *** Mescc now supports locals stack frame larger than 1 byte offset. *** Mescc now supports incomplete struct initializers {0}. @@ -31,7 +69,7 @@ https://gitlab.com/janneke/tinycc branch wip-mescc. *** Mescc now supports stdarg variable argument lists. *** Mescc now has __GNUC__ unset, #f (WAS: 0). *** Mescc now depends on Nyacc 0.80.3. -*** Mescc now depends on M1 from [[https://github.com/oriansj/MESCC_Tools][MESCC_Tools]] v0.2. +*** Mescc now depends on M1 from [[https://github.com/oriansj/mescc-tools][mescc-tools]] v0.2. *** Mescc now compiles to M1 object files. **** 7 new mlibc functions calloc, close, getcwd, lseek, sprintf, unlink, vprintf. @@ -48,7 +86,7 @@ stdnoreturn.h, sys/select.h, sys/wait.h. * Changes in 0.8 since 0.7 ** Mescc *** Mescc now has experimental annotation support for hex2. -*** Mescc now depends on the hex2 linker from [[https://github.com/oriansj/MESCC_Tools][MESCC_Tools]]. +*** Mescc now depends on the hex2 linker from [[https://github.com/oriansj/mescc-tools][mescc-tools]]. Direct ELF output support has been removed. ELF symbol and string table creation has been removed. *** Mescc has been simplified by leveraging use labels in hex2 output. @@ -139,7 +177,7 @@ open-input-string, read-string, string-delete, with-throw-handler. *** Evaluate arguments of OR only once. *** Fix assq-set! *** Fix ,expand in the REPL. - + * Changes in 0.4 since 0.3 ** Core *** Improved performance. @@ -225,7 +263,7 @@ algorithm has been implemented. *** Make+Bash voodoo has been replaced by build-aux/mes-snarf.scm. *** Apply now accepts multiple list arguments. *** Apply of character, inernal, number, string is an error. -*** Quasisyntax reading +*** Quasisyntax reading * Changes in 0.1 (since progress report #2) ** Core diff --git a/doc/ANNOUNCE-0.10 b/doc/ANNOUNCE-0.10 new file mode 100644 index 00000000..2dcc64d5 --- /dev/null +++ b/doc/ANNOUNCE-0.10 @@ -0,0 +1,96 @@ +Subject: Mes 0.10 released + +I am pleased to announce the release of Mes 0.10, representing 82 +commits over 6 weeks. Mescc now compiles a bootstrappable-modified +TinyCC into a mes-tcc that in turn can successfully compile a trivial +C program. + +* About + + Mes[0] aims to create full source bootstrapping for GuixSD[1] as + part of the bootstrappable builds[2] project. + + It currently consists of a mutual self-hosting [close to Guile-] + Scheme interpreter prototype in C and a Nyacc-based C compiler in + [Guile] Scheme. This C prototype will be rewritten in stage0[3] + M1 assembly (or possibly stage2 slow-LISP, or ...). + + The Scheme interpreter prototype (mes.c) has a Garbage Collector, + a library of loadable Scheme modules-- notably Dominique Boucher's + LALR[4], Pre-R6RS portable syntax-case[5] with R7RS ellipsis, Matt + Wette's Nyacc[6], Guile's PEG[7] --and test suite just barely + enough to support a simple REPL (repl.mes) and simple C-compiler + (mescc.mes). + + The simple C compiler can compile a modified TinyCC[8] that is + starting to work! A gcc-compiled tcc is known[9] to compile GCC. + + Mes is inspired by The Maxwell Equations of Software: LISP-1.5[10] + -- John McCarthy page 13, GNU Guix's source/binary packaging + transparency and Jeremiah Orians's stage0 ~300 byte self-hosting + hex assembler. + +* Download + + git clone https://gitlab.com/janneke/mes + + wget https://gitlab.com/janneke/mes/repository/archive.tar.gz?ref=v0.10 -O mes-0.10.tar.gz + +Mes runs from the source tree and can also be built, packaged and +installed in Guix[SD] by the usual + + guix package -f guix.scm + +* Changes in 0.10 since 0.9 + ** Core + *** 2 new functions + logand, lognot. + ** Mescc + *** Mes now includes M1 sources for full source bootstrapping up to tcc. + *** Mescc now depends on Nyacc 0.81.0. + *** Mescc now compiles a mes-tcc that compiles a trivial C to a running a.out. + *** Mescc now supports several (as used my tinycc) struct by value assignments. + *** Mescc now has _start in a separate crt1.c. + *** Mescc now supports initializer arithmetic. + *** Mescc now supports arrays of struct of arbitrary size. + *** Mescc now supports pointer arithmetic. + *** Mescc now defines __i386__, __linux__. + *** Mescc now supports --*p, ++*p. + *** Mescc now supports int foo[bar] = {baz,...}. + *** Mescc now supports \x00 in strings in M1. + *** Mescc now supports complex enum expressions. Thanks, rain1! + *** Mescc now supports short. + *** Mescc now supports foo.bar = foo.baz = bla. + *** Mescc now supports foo = bar < baz and comparison permutations. + *** Mescc now supports (anonymous) structs in unions. + *** Mescc now writes M1 strings if possible. + **** 2 improved mlibc functions + malloc, realloc. + **** 13 new mlibc functions + fclose, fgetc, fprintf, fwrite, memcpy, memmove, memset, snprintf, strcat, + strchr, strrchr, strtoull, vnsprintf. + **** 15 new tinycc-support tests + 76-pointer-arithmetic.c, 77-pointer-assign.c, 78-union-struct.c, + 79-int-array.c, 7a-struct-char-array.c, 7b-struct-int-array.c, + 7c-dynarray.c, 7d-cast-char.c, 7e-struct-array-access.c, + 7f-struct-pointer-arithmetic.c, 7g-struct-byte-word-field.c, + 7h-struct-assign.c, 7i-struct-struct.c, 7j-strtoull.c, + 7k-for-each-elem.c. + ** Noteworthy bug fixes + *** Mescc has many fixes for foo.bar[baz], foo[bar].baz with ./-> permutations. + *** Many bugs and limitations in mescc have been fixed and removed. + +Greetings, +janneke + +[0] https://gitlab.com/janneke/mes +[1] https://www.gnu.org/software/guix +[2] http://bootstrappable.org +[3] https://github.com/oriansj/stage0 +[4] https://github.com/schemeway/lalr-scm +[5] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html +[6] https://www.nongnu.org/nyacc/ +[7] https://www.gnu.org/software/guile/docs/master/guile.html/PEG-Parsing.html +[8] https://gitlab.com/janneke/tinycc +[9] https://lists.gnu.org/archive/html/tinycc-devel/2017-05/msg00103.html +[10] http://www.softwarepreservation.org/projects/LISP/book/LISP%25201.5%2520Programmers%2520Manual.pdf diff --git a/make.scm b/make.scm index bdc55bba..c1b0699a 100755 --- a/make.scm +++ b/make.scm @@ -636,12 +636,13 @@ exec ${GUILE-guile} --no-auto-compile -L . -L guile -C . -C guile -s "$0" ${1+"$ (compose add-target (cut install <> #:dir "share/doc/mes")) '("AUTHORS" ;;"ChangeLog" + "BOOTSTRAP" "COPYING" "HACKING" "INSTALL" "NEWS" "README" - "doc/ANNOUNCE-0.9")) + "doc/ANNOUNCE-0.10")) (add-target (install "doc/fosdem/fosdem.pdf" #:dir "share/doc/mes"))